Retrieving H.264/JPEG video using RTSP over HTTP

Retrieving H.264 video using RTSP over HTTP is one of the most protocol intensive work because it involves the following:

1. HTTP.  It is the basic layer for the stack of this case. Its GET command is used to establish the data receiving channel, and its POST command is used to send data.
2. RTSP.  It is quite similar to HTTP.  In this case, the two-way communication is a sort of hybrid of HTTP and RTSP.
3. RTP/RTCP. The pair of protocols are responsible for carrying data packets and controlling the traffic.
4. SDP. It is used in RTSP commands to describe the communication session. For this case, its media announcement (m=...) is used to specify the format (e.g. 96 or another number indicating dynamic format), then session attributes are used to provide further information about the format (e.g. a=rtpmap:96 H264/90000, a=fmtp:96 packetization-mode=1; profile-level-id=420029; sprop-parameter-sets=Z0IAKeNQFAe2AtwEBAaQeJEV,aM48gA==).
5. H.264. It is used to encode the payload.
6. RTP Payload Format for H.264 Video/RTP Payload Format for JPEG-compressed Video. They are used to package video data. The jpeg payload in RTP packets are totally different from MJPEG.  The latter is ready for displaying by most image processors.  The former needes to be demuxed, and packaged into JPEG file format which is JFIF(JPEG File Interchange Format), not JPEG interchange format.
7. RTP Payload Format for Transport of MPEG-4 Elementary Streams.  The audio stream that interleaves with H.264 video stream is most likely based on this protocol. It may also be one of the standard types.

 The steps to retrieve and display H.264 video stream using RTSP over HTTP are the following:

1. Create a network client. It may need to be a socket because a typical HTTP client may not work well with RTSP which is like a vriant of HTTP.
2. Establish a data receiving session with HTTP GET command.
3. Send video stream source RTSP commands OPTIONS (optional), DESCRIBE to retrieve the stream information.
4. Send an RTSP command SETUP  to set up the video stream and retrieve a session number from the response.
5. Send an RTSP command PLAY to start the video stream.
6. Process the stream data.

A simple scenario of processing stream data involves the following:

1. Get a RTP or RTCP data packet encapsuled with $\0x0000 or $\0x0001 respectively.
2. If it is RTCP, process its contents according to the protocol.  If it is a RTP packet, process the header, then retrieve the palyload according to the header.

A simple scenario of processing the H.264 payload involves the following:

1. Determine the payload type (Single NAL Unit Packet, Aggregation packet, Aggregation packet or Fragmentation unit) by reading the first byte.
2. Based on the information regarding the H.264 video stream retrieved via the RTSP command DESCRIBE(refer to 8.2.1.  Mapping of MIME Parameters to SDP of RTP Payload Format for H.264 Video), decode the payload.
3. Send the decoded data to a video display module

 JPEGsnoop is a good tool for examining the decoded JPEG image, and isoviewer is a good tool for examing saved mp4 files.

 Fortunately, two popular video player can be set up to streaming RTSP video over HTTP.

1. Apple QuickTime Player. ( Edit > Preferences > Quicktime Preferences > Advanced > Transport Setup > Custom... > HTTP)
2. VLC Player (Tools > Preferences > (Show settings at the bottom left corner) All > Input / Codecs > Demuxers > RTP/RTSP > Tunnel RTSP and RTP over HTTP)