Implementing the Media Fragments URI Specification - Talk given at the Developer's Track of the 19th World Wide Web Conference (WWW'2010), Raleigh (NC), USA, April 29th 2010

1. Implementing the Media
Fragments URI
Specification
Raphaël Troncy <raphael.troncy@eurecom.fr>,
Davy Van Deursen, Erik Mannens
Silvia Pfeiffer,
Yves Lafon

2. Use Case
 Aidem received on her Facebook
wall a status message containing a
Media Fragment URI
 Use a ‘#’ !
 Highlight a video
sequence
 Highlight a region
to pay attention to
29/04/2010 - Implementing the Media Fragments URI Specification - WWW 2010, Raleigh, USA -2

3. Media Fragments URI
Provide URI-based
mechanisms for uniquely
identifying fragments for
media objects on the Web,
such as video, audio, and
images.
Photo credit: Robert Freund
29/04/2010 - Implementing the Media Fragments URI Specification - WWW 2010, Raleigh, USA -3

4. Requirements
 r01: Temporal fragments:
 a clipping along the time dimension from a start to an end time that
are within the duration of the media resource
 r02: Spatial fragments:
 a clipping of an image region, only consider rectangular regions
 r03: Track fragments:
 a track as exposed by a container format of the media resource
 r04: Named fragments:
 a media fragment - either a track, a time section, or a spatial region -
that has been given a name through some sort of annotation
mechanism
29/04/2010 - Implementing the Media Fragments URI Specification - WWW 2010, Raleigh, USA -4

5. Side Conditions
 Restrict to what the container format (encapsulating the
compressed media content) can express (and expose),
thus no transcoding
 Protocol covered: HTTP(S), FILE, RTSP, RTMP
http://www.w3.org/TR/media-frags-reqs/
29/04/2010 - Implementing the Media Fragments URI Specification - WWW 2010, Raleigh, USA -5

6. Media Fragments processing
 General principle:
 Smart UA will strip out the fragment definition and
encode it into custom http headers ...
 (Media) Servers will handle the request, slice the media
content and serve just the fragment while old ones will
serve the whole resource
 Four recipes proposed
29/04/2010 - Implementing the Media Fragments URI Specification - WWW 2010, Raleigh, USA -6

7. Recipe 1: UA mapped byte ranges
 The User Agent knows how to map a custom unit into bytes and
sends a normal Range request expressed in bytes
29/04/2010 - Implementing the Media Fragments URI Specification - WWW 2010, Raleigh, USA -7

8. Recipe 1: UA mapped byte ranges
29/04/2010 - Implementing the Media Fragments URI Specification - WWW 2010, Raleigh, USA -8

9. Recipe 2: Server mapped byte ranges
 The UA sends a Range request expressed in a custom unit (e.g.
seconds), the server answers directly with a 206 Partial Content
and indicates the mapping between bytes and the custom unit
29/04/2010 - Implementing the Media Fragments URI Specification - WWW 2010, Raleigh, USA -9

10. Recipe 2: Server mapped byte ranges
29/04/2010 - Implementing the Media Fragments URI Specification - WWW 2010, Raleigh, USA - 10

11. Recipe 3: Proxy cacheable variant
 The UA sends a Range request expressed in a custom unit (e.g.
seconds), the server answers first with just a Head and the
mapping between the custom unit and bytes so that the UA issues
another normal
Range request
expressed this
time in bytes
making the
answer
cacheable.
29/04/2010 - Implementing the Media Fragments URI Specification - WWW 2010, Raleigh, USA - 11

12. Recipe 3: Proxy cacheable variant
29/04/2010 - Implementing the Media Fragments URI Specification - WWW 2010, Raleigh, USA - 12

13. Recipe 4: Serving playable resources
 The UA sends a Range request expressed in a custom unit (e.g.
seconds), the server provides a multipart message body reply
(multipart/byte-ranges) containing not only the bytes
corresponding to requested media fragment but also the media
header data making the resource playable.
29/04/2010 - Implementing the Media Fragments URI Specification - WWW 2010, Raleigh, USA - 13

14. Recipe 4: Serving playable resources
29/04/2010 - Implementing the Media Fragments URI Specification - WWW 2010, Raleigh, USA - 14

15. Implementation
 Media Fragment server (4 recipes supported):
 Ninsuna: http://ninsuna.elis.ugent.be/MediaFragmentsServer
 Media Fragment user agents:
 Ninsuna Flash player:
http://ninsuna.elis.ugent.be/MediaFragmentsPlayer
Supports recipe 1
 Silvia Pfeiffer's experiment with HTML5 + JS:
http://annodex.net/~silvia/itext/mediafrag.html
Supports recipe 1 (for .ogg files and time dimension)
 Firefox pluggin
development in order to
support all recipes
(HTML5 +
XMLHttpRequest)
29/04/2010 - Implementing the Media Fragments URI Specification - WWW 2010, Raleigh, USA - 15

16. Issues
 Communicating with the media player
 HTML5 browser: patch submitted for Firefox
 Flash: need to build a custom player
 Media Fragments rendering
 Spatial dimension?
 Names/Tracks discovery
 Media Multitrack (JS) API:
http://www.w3.org/WAI/PF/HTML/wiki/Media_MultitrackAPI
 Get involved?
http://www.w3.org/TR/media-frags/
http://lists.w3.org/Archives/Public/public-media-fragment/
29/04/2010 - Implementing the Media Fragments URI Specification - WWW 2010, Raleigh, USA - 16