NALU, Annex B, and Start Codes

A start code is 2-3 0x00 bytes followed by a 0x01 byte.

A NALU representation without emulation prevention bytes is called a Raw Byte Sequence Payload, or RBSP.

The four-byte sequences 0x00000000, 0x00000001, 0x00000002, and 0x00000003 are illegal within a non-RBSP NALU. In order to turn an RBSP into a NALU with start codes, byte stuffing is employed by inserting an Emulation Prevention byte 0x03 after two consecutive 0x00 bytes. To reverse this, when reading the stream, remove any 0x03 seen after two consecutive 0x00 bytes.

To determine the length of a NALU, start with a byte stream (which will have a start code), skip to the next start code, and count the bytes in between.

In order to convert between Annex B and AVCC, convert between start codes and lengths, and remove/add emulation bytes.

An AVCC stream always starts with a specific header

8 bits = 0x01 (this is illegal in Annex B)
8 bits of avc profile (from sps[0][1])
8 bits of avc compatibility (from sps[0][2])
8 bits of avc level (from sps[0][3])
6 bits = 0b111111
2 bits of NALULengthSizeMinusOne (typically 3)
3 bits of 0b111
5 bits of number of SPS NALU (usually 1)
Repeated per SPS
- 16 bits of SPS size -Variable SPS NALU data
8 bits of number of PPS NALU (usually 1)
Repeated once per PPS
- 16 bits of PPS size
- Variable PPS NALU data

FLV streams must be written as AVCC.

tbr, tbc, tbn

tbn is the time base from the container.

tbc is the time base from the codec.

tbr is video frame rate guessed from the video stream. A note from the FFMpeg source:

tbr is NOT the average frame rate, it is the smallest frame rate that can accurately represent all timestamps.

Reference

Video File Format Specification, Version 10, 2008. FLV and F4V file format spec.

Version 10.1.

OIPF, Release 2 Specification, Volume 2 – Media Formats

Overview of the H.264/AVC Video Coding Standard

ISO/IEC 14496, “Coding of audio-visual objects”

Sound

ISO/IEC JTC 1/SC 29/WG 11N2503-sec5. Information technology - Coding of audio-visual objects. Part 3: Audio. Section 5: Structured audio.

Base ISO format

ISO/IEC 14496-12. Information technology — Coding of audio-visual objects — Part 12: ISO base media file format

MP4

ISO/IEC 14496-14. Information technology — Coding of audio-visual objects — Part 14: MP4 file format.

AVC file format

ISO/IEC JTC 1/SC 29 N, ISO/IEC 14496-15:2014/PDAM 2. Information technology — Coding of audio-visual objects — Part 15: Carriage of NAL unit structured video in the ISO Base Media File Format, AMENDMENT 2: Carriage of AVC based 3D video excluding MVC.

ISO/IEC JTC 1/SC 29 N, ISO/IEC 14496-15:2010/PDAM 2. Information technology — Coding of audio-visual objects — Part 15: Advanced Video Coding (AVC) file format, AMENDMENT 2: Carriage of high efficiency video coding (HEVC).