WIXLIB

Multimedia NetworkingYao WangPolytechnic University, Brooklyn, NY11201yao@vision.poly.edu

These slides are adapted from the slidesmade by authors of the book (J. F. Kuroseand K. Ross), available from the publisher sitefor instructors. We would like to thank theauthors for the excellent book and the slides.Slides based on ComputerNetworking: A Top DownApproach Featuring theInternet,2nd edition.Jim Kurose, Keith RossAddison-Wesley, July 2002.Chapter 6

Multimedia, Quality of Service: What is it?Network Multimediaapplications: sending andreceiving audio and video(“continuous media”)across networksQoSnetwork providesapplication with level ofperformance needed forapplication to function.

RoadmapMultimedia Networking Applications RTP and RTCP Streaming stored audio and video RTSPInternet video phone SIPSIP vs. H.323Recovery from loss Beyond Best Effort

MM Networking ApplicationsClasses of MM applications:1) Streaming stored audioand video2) Streaming live audio andvideo3) Real-time interactiveaudio and videoJitter is the variabilityof packet delays withinthe same packet streamFundamentalcharacteristics: Typically delay sensitive end-to-end delaydelay jitterBut loss tolerant:infrequent losses causeminor glitches Antithesis of data,which are loss intolerantbut delay tolerant.

Streaming Stored MultimediaStreaming: media stored at source transmitted to client streaming: client playout beginsbefore all data has arrived timing constraint for still-to-betransmitted data: in time for playout

Cumulative dataStreaming Stored Multimedia:What is it?1. videorecorded2. videosentnetworkdelay3. video received,played out at clientstreaming: at this time, clientplaying out early part of video,while server still sending laterpart of videotime

Streaming Stored Multimedia:Interactivity VCR-like functionality: client canpause, rewind, FF, push slider bar 10 sec initial delay OK 1-2 sec until command effect OK RTSP often used (more later) timing constraint for still-to-betransmitted data: in time for playout

Streaming Live MultimediaExamples: Internet radio talk show Live sporting eventStreaming playback buffer playback can lag tens of seconds aftertransmission still have timing constraintInteractivity fast forward impossible rewind, pause possible!

Interactive, Real-Time Multimediaapplications: IP telephony,video conference, distributedinteractive worlds end-end delay requirements: audio: 150 msec good, 400 msec OK includes application-level (packetization) and networkdelays higher delays noticeable, impair interactivity session initialization how does callee advertise its IP address, portnumber, encoding algorithms?

Multimedia Over Today’s InternetTCP/UDP/IP: “best-effort service” no guarantees on delay, loss?But you said multimedia apps requires ?QoS and level of performance to be? effective!?Today’s Internet multimedia applicationsuse application-level techniques to mitigate(as best possible) effects of delay, loss

How should the Internet evolve tobetter support multimedia?Integrated services philosophy: Fundamental changes inInternet so that apps canreserve end-to-endbandwidth Requires new, complexsoftware in hosts & routersLaissez-faire no major changes more bandwidth whenneeded content distribution,application-layer multicast application layerDifferentiated servicesphilosophy: Fewer changes to Internetinfrastructure, yet provide1st and 2nd class service.What’s your opinion?

A few words about audio compression Analog signal sampledat constant rate Each sample quantized,i.e., rounded telephone: 8,000samples/secCD music: 44,100samples/sece.g., 28 256 possiblequantized valuesEach quantized valuerepresented by bits 8 bits for 256 valuesExample: 8,000samples/sec, 256quantized values -- 64,000 bps Receiver converts itback to analog signal: some quality reductionExample rates CD: 1.411 Mbps MP3: 96, 128, 160 kbps Internet telephony:5.3 - 13 kbps

A few words about video compression Video is sequence ofimages displayed atconstant rate e.g. 24 images/secDigital image is array ofpixels Each pixel representedby bits Redundancy spatialtemporalExamples: MPEG 1 (CD-ROM) 1.5Mbps MPEG2 (DVD) 3-6 Mbps MPEG4 (often used inInternet, 1 Mbps)Research: Layered (scalable) video adapt layers to availablebandwidth

RoadmapMultimedia Networking Applications RTP and RTCP Streaming stored audio and video RTSPInternet Phone SIPSIP vs. H.323Recovery from loss Beyond Best Effort

Real-Time Protocol (RTP)RTP specifies a packetstructure for packetscarrying audio andvideo data – apacketization protocol! RFC 1889. RTP packet provides payload typeidentificationpacket sequencenumberingtimestampingRTP runs in the endsystems. RTP packets areencapsulated in UDPsegments Interoperability: Iftwo Internet phoneapplications run RTP,then they may be ableto work together

RTP runs on top of UDPRTP libraries provide a transport-layer interfacethat extend UDP: port numbers, IP addresses payload type identification packet sequence numbering time-stamping

RTP ExampleConsider sending 64kbps PCM-encodedvoice over RTP. Application collectsthe encoded data inchunks, e.g., every 20msec 160 bytes in achunk. The audio chunk alongwith the RTP headerform the RTP packet,which is encapsulatedinto a UDP segment. RTP header indicatestype of audio encodingin each packet sender can changeencoding during aconference.RTP header alsocontains sequencenumbers andtimestamps.

RTP HeaderPayload Type (7 bits): Indicates type of encoding currently beingused. If sender changes encoding in middle of conference, senderinforms the receiver through this payload type field. Payload type 0: PCM mu-law, 64 kbps Payload type 3, GSM, 13 kbps Payload type 7, LPC, 2.4 kbps Payload type 26, Motion JPEG Payload type 31. H.261 Payload type 33, MPEG2 videoSequence Number (16 bits): Increments by one for each RTP packetsent, and may be used to detect packet loss and to restore packetsequence.

RTP Header (2) Timestamp field (32 bytes long). Reflects the samplinginstant of the first byte in the RTP data packet. For audio, timestamp clock typically increments by onefor each sampling period (for example, each 125 usecsfor a 8 KHz sampling clock) if application generates chunks of 160 encoded samples,then timestamp increases by 160 for each RTP packetwhen source is active. Timestamp clock continues toincrease at constant rate when source is inactive. SSRC field (32 bits long). Identifies the source of the RTPstream. Each stream in a RTP session should have a distinctSSRC.

Real-Time Control Protocol (RTCP) Works in conjunction withRTP. Each participant in RTPsession periodicallytransmits RTCP controlpackets to all otherparticipants. Each RTCP packet containssender and/or receiverreports report statistics useful toapplication Statistics include numberof packets sent, number ofpackets lost, interarrivaljitter, etc. Feedback can be used tocontrol performance Sender may modify itstransmissions based onfeedback

RTCP PacketsReceiver report packets: fraction of packetslost, last sequencenumber, averageinterarrival jitter.Sender report packets: SSRC of the RTPstream, the currenttime, the number ofpackets sent, and thenumber of bytes sent.Source descriptionpackets: e-mail address ofsender, sender's name,SSRC of associatedRTP stream. Provide mappingbetween the SSRC andthe user/host name.

Synchronization of Streams RTCP can synchronizedifferent media streamswithin a RTP session. Consider videoconferencingapp for which each sendergenerates one RTP streamfor video and one for audio. Timestamps in RTP packetstied to the video and audiosampling clocks not tied to the wallclock time Each RTCP sender-reportpacket contains (for themost recently generatedpacket in the associatedRTP stream): timestamp of the RTPpacketwall-clock time for whenpacket was created. Receivers can use thisassociation to synchronizethe playout of audio andvideo.

RTP and QoSRTP does not provide any mechanism to ensuretimely delivery of data or provide other quality ofservice guarantees. But together with RTCP, it allows monitoring ofQoS so that sender and receiver can adjust theiroperations appropriately RTP encapsulation is only seen at the end systems:it is not seen by intermediate routers. Routers providing best-effort service do not make anyspecial effort to ensure that RTP packets arrive at thedestination in a timely matter.

RoadmapMultimedia Networking Applications RTP and RTCP Streaming stored audio and video RTSPInternet Phone SIPSIP vs. H.323Recovery from loss Beyond Best Effort

Streaming Stored MultimediaApplication-level streamingtechniques for making thebest out of best effortservice: client side buffering use of RTP/UDP orUDP directly versusTCP multiple encodings ofmultimediaMedia Player jitter removal decompression error concealment graphical user interfacew/ controls forinteractivity

Internet multimedia: simplest approachaudio or video stored in file files transferred as HTTP object received in entirety at client then passed to player audio, video not streamed: no, “pipelining,” long delays until playout!

Internet multimedia: streaming approachbrowser GETs metafile browser launches player, passing metafile player contacts server server streams audio/video to player

Streaming from a streaming server This architecture allows for non-HTTP protocol betweenserver and media player Can also use UDP instead of TCP.

Streaming Multimedia: Client Bufferingvariablenetworkdelayclient videoreceptionconstant bitrate videoplayout at clientbufferedvideoCumulative dataconstant bitrate videotransmissionclient playoutdelay Client-side buffering, playout delay compensatefor network-added delay, delay jittertime

Streaming Multimedia: Client Bufferingconstantdrainrate, dvariable fillrate, x(t)bufferedvideo Client-side buffering, playout delay compensatefor network-added delay, delay jitter

Streaming Multimedia: UDP or TCP?UDP server sends at rate appropriate for client (oblivious tonetwork congestion !) often send rate encoding rate constant rate then, fill rate constant rate - packet loss short playout delay (2-5 seconds) to compensate for networkdelay jitter error recover: time permittingTCP send at maximum possible rate under TCP fill rate fluctuates due to TCP congestion control larger playout delay: smooth TCP delivery rate HTTP/TCP passes more easily through firewalls

Streaming Multimedia: client rate(s)1.5 Mbps encoding28.8 Kbps encodingQ: how to handle different client receive ratecapabilities? 28.8 Kbps dialup 100Mbps EthernetA: (1) server stores, transmits multiple copies of video, encoded atdifferent ratesA: (2) encode video in scalable mode that can be retrieved atdifferent rates! (still at research stage)

User Control of Streaming Media: RTSPHTTP Does not target multimediacontent No commands for fastforward, etc.RTSP: RFC 2326 Client-server applicationlayer protocol. For user to control display:rewind, fast forward,pause, resume,repositioning, etc What it doesn’t do: does not define howaudio/video is encapsulatedfor streaming over network does not restrict howstreamed media istransported; it can betransported over RTP/UDP,UDP or TCP does not specify how themedia player buffersaudio/video

RTSP: out of band controlFTP uses an “out-of-band”control channel: A file is transferred overone TCP connection. Control information(directory changes, filedeletion, file renaming,etc.) is sent over aseparate TCP connection. The “out-of-band” and “inband” channels usedifferent port numbers.RTSP messages are also sentout-of-band: RTSP control messagesuse different port numbersthan the media stream:out-of-band. Port 554 The media stream isconsidered “in-band”.

RTSP ExampleScenario:metafile communicated to web browser browser launches player player sets up an RTSP control connection, dataconnection to streaming server