SlideShare une entreprise Scribd logo
1  sur  25
Télécharger pour lire hors ligne
Master Degree Course in
Cinema and Media Engineering
SUPERVISOR
Giovanni Malnati
INTERNSHIP SUPERVISORS
Raphaël Troncy
José Luis Redondo Garcìa
CANDIDATE
Pasquale Lisena
Pasquale Lisena
Developing a Media Fragment Server in Node.JS
PART I What is a «Media Fragment»?
PART II State of the art
PART III Implementation
2
Pasquale Lisena
Developing a Media Fragment Server in Node.JS
PART I What is a «Media Fragment»?
PART II State of the art
PART III Implementation
3
Pasquale Lisena
Developing a Media Fragment Server in Node.JS
4
• Share
• Bookmark
• Save band
• Annotate
• Save time
• Search
• …and more
ADVANTAGES
PARTI
Pasquale Lisena
Developing a Media Fragment Server in Node.JS
5
TEMPORAL DIMENSION (T)
SPATIAL DIMENSION (XYWH)
MEDIA FRAGMENTS
URI 1.0
RECOMENDATION
PARTI
Pasquale Lisena
Developing a Media Fragment Server in Node.JS
6
MEDIA FRAGMENTS
URI 1.0
RECOMENDATION
TRACK DIMENSION
NAMED DIMENSION (ID)
PARTI
Pasquale Lisena
Developing a Media Fragment Server in Node.JS
Query format
7
Hash format
http://www.example.com/example.ogv#t=10,20
http://www.example.com/example.ogv?t=10,20
Server generates the fragment
resource
HTTP Range request (time)
User Agent is in charge of display it
correctly
PARTI
Pasquale Lisena
Developing a Media Fragment Server in Node.JS
PART I What is a «Media Fragment»?
PART II State of the art
PART III Implementation
8
Pasquale Lisena
Developing a Media Fragment Server in Node.JS
9
CLIENTS VIDEO PLATFORMS
TEMPORAL
NPT (hh:mm:ss)
SMPTE - Clock
SPATIAL
Only start
Not standard syntax
PARTII
Pasquale Lisena
Developing a Media Fragment Server in Node.JS
10
CLIENT
IMPLEMENTATIONS
• Synote Media Fragment Player
• NinSuna Media Fragment Player
SERVER
IMPLEMENTATIONS
• NinSuna Media Fragment Server
• Rafael
POLYFILL PARSER
LIBRARY
• MediaFragment.js
PARTII
Pasquale Lisena
Developing a Media Fragment Server in Node.JS
11
CLIENT IMPLEMENTATIONS
SYNOTE MEDIA FRAGMENT PLAYER
• Cross-browser (Flash fallback)
• HTML5, YouTube, Daylimotion,
Vimeo support
• HTML5-like interface
https://github.com/pasqLisena/Media-Fragment-Player
• JavaScript plugin
• Spatial and temporal support
• No Time range requests
• Highlight of fragment in timeline
• Dark mask for spatial fragment
OUR
CONTRIBUTION
PARTII
Pasquale Lisena
Developing a Media Fragment Server in Node.JS
12
SERVER IMPLEMENTATIONS
NINSUNA MEDIA FRAGMENT
SERVER RAFAEL
• Pre-processing
• Annotation-based
• Support for Time range
request
• Extraction on the fly
• Fragment stored on file
system
• Support only for query
fragments
http://ninsuna.elis.ugent.be/MediaFragmentsServer https://github.com/Noterik/Rafael
PARTII
Pasquale Lisena
Developing a Media Fragment Server in Node.JS
13
MEDIAFRAGMENT.JS
http://www.example.com/video.ogv
?t=1:00:00#t=npt:10,20
&xywh=percent:25,25,50,50
{
"query":{
"t":[
{
"value":"1:00:00",
"unit":"npt",
"start":"1:00:00",
"end":"",
"startNormalized":3600,
"endNormalized":""
}
]
},
"hash":{
"t":[
{
"value":"npt:10,20",
"unit":"npt",
https://github.com/tomayac/Media-Fragments-URI/
OUR
CONTRIBUTION
PARSING
+
UNIT NORMALIZATION
+
ERROR DETECTION
PARTII
Pasquale Lisena
Developing a Media Fragment Server in Node.JS
PART I What is a «Media Fragment»?
PART II State of the art
PART III Implementation
14
Pasquale Lisena
Developing a Media Fragment Server in Node.JS
Query Fragment
• Time (npt)
• Track (video/audio)
• Xywh
15
Hash fragment
• Range request
(npt)
PARTIII
Pasquale Lisena
Developing a Media Fragment Server in Node.JS
16
PARTIII
Pasquale Lisena
Developing a Media Fragment Server in Node.JS
17
FRAGMENT QUERY FFMPEG OPTION NOTE
t=10 -ss 10
t=,20 -to 20
t=10,20 -ss 10 -to 20
track=video -an no audio
track=audio -vn no video
xywh=10,10,50,60 -filter:v "crop=50:60:10:10" require transcoding
xywh=percent:10,10,50,60
-filter:v "crop=in_w*50/100:
in_h*60/100:in_w*10/100:
in_h*10/100"
require transcoding
PARTIII
Pasquale Lisena
Developing a Media Fragment Server in Node.JS
18
400px
220px
00:21:43
video.mp4
? t= 00:00:10, 00:00:20
& xywh=0,0,400,220
PARTIII
Pasquale Lisena
Developing a Media Fragment Server in Node.JS
ALIAS
collection
19
t=10,20 and t=11,20 are byte identical
video_10.45-19.41
video_10-20
video_11-20
GridFS
Auto-deletion of older resources.
PARTIII
Pasquale Lisena
Developing a Media Fragment Server in Node.JS
20
mediafragment.js
ffmpegmongoDB
PARTIII
Pasquale Lisena
Developing a Media Fragment Server in Node.JS
21
PARTIII
Pasquale Lisena
Developing a Media Fragment Server in Node.JS
22
HTTP/1.1 206 Partial Content
Accept-Ranges: bytes, t, id
Content-Length: 3795
Content-Type: video/ogg
Content-Range-Mapping:
{ t:npt 9.85-21.16/0.0-653.79;include-setup } =
{ bytes 0-52,19147-22880/35614993 }
Content-type: multipart/byteranges; boundary=BOUNDARY
Etag: "b7a60-21f7111-46f3219476580"
--BOUNDARY
Content-type: video/ogg
Content-Range: bytes 0-52/35614993
{binary data}
--BOUNDARY
Content-type: video/ogg
Content-Range: bytes 19147-22880/35614993
{binary data}
---BOUNDARY--
METADATA
DATA
PARTIII
Pasquale Lisena
Developing a Media Fragment Server in Node.JS
23
localhost:3000/video/video.mp4?t=10,20
localhost:3000/video/video.mp4#t=10,20
Pasquale Lisena
Developing a Media Fragment Server in Node.JS
24
• We contributed to the generic "media-fragment.js" polyfill and
prepare a Node.JS version (open source)
• We contributed to the Synote Media Player to build a more
generic media fragment player (open source)
• We discovered bugs in the W3C specification and proposed
amendments (revised text) to be endorsed by W3C
• We implemented a media fragment server in Node.JS (using
ffmeg) + a smart cache (based on MongoDB)
• Future work: finalize the Chrome extension so that the
browser understand the response from the server, extend
support to all media and test MaFFiN on a more large scale
environment.
Developing a Media Fragment Node.JS Server

Contenu connexe

Similaire à Developing a Media Fragment Node.JS Server

The future of multimedia communications and services: Kurento and it's role
The future of multimedia communications and services: Kurento and it's roleThe future of multimedia communications and services: Kurento and it's role
The future of multimedia communications and services: Kurento and it's roleLuis Lopez
 
Quality of Experience of Web-based Adaptive HTTP Streaming Clients in Real-Wo...
Quality of Experience of Web-based Adaptive HTTP Streaming Clients in Real-Wo...Quality of Experience of Web-based Adaptive HTTP Streaming Clients in Real-Wo...
Quality of Experience of Web-based Adaptive HTTP Streaming Clients in Real-Wo...Alpen-Adria-Universität
 
Kurento: a media server architecture and API for WebRTC
Kurento: a media server architecture and API for WebRTCKurento: a media server architecture and API for WebRTC
Kurento: a media server architecture and API for WebRTCLuis Lopez
 
[ElastixWorld 2016] mediasoup: Powerful WebRTC SFU for Node.js
[ElastixWorld 2016] mediasoup: Powerful WebRTC SFU for Node.js[ElastixWorld 2016] mediasoup: Powerful WebRTC SFU for Node.js
[ElastixWorld 2016] mediasoup: Powerful WebRTC SFU for Node.jsIñaki Baz Castillo
 
WebRTC Broadcasting @ TADSummit 2023
WebRTC Broadcasting @ TADSummit 2023WebRTC Broadcasting @ TADSummit 2023
WebRTC Broadcasting @ TADSummit 2023Lorenzo Miniero
 
Simplifying and accelerating converged media with Open Visual Cloud
Simplifying and accelerating converged media with Open Visual CloudSimplifying and accelerating converged media with Open Visual Cloud
Simplifying and accelerating converged media with Open Visual CloudLiz Warner
 
voip2day 2016: mediasoup, powerful WebRTC SFU for Node.js
voip2day 2016: mediasoup, powerful WebRTC SFU for Node.jsvoip2day 2016: mediasoup, powerful WebRTC SFU for Node.js
voip2day 2016: mediasoup, powerful WebRTC SFU for Node.jsIñaki Baz Castillo
 
Kurento - FI-WARE Bootcamp
Kurento - FI-WARE BootcampKurento - FI-WARE Bootcamp
Kurento - FI-WARE BootcampIvan Gracia
 
Developing rich multimedia applications with Kurento: a tutorial for JavaScri...
Developing rich multimedia applications with Kurento: a tutorial for JavaScri...Developing rich multimedia applications with Kurento: a tutorial for JavaScri...
Developing rich multimedia applications with Kurento: a tutorial for JavaScri...Luis Lopez
 
WebRTC Standards & Implementation Q&A - All about browser interoperability
WebRTC Standards & Implementation Q&A - All about browser interoperabilityWebRTC Standards & Implementation Q&A - All about browser interoperability
WebRTC Standards & Implementation Q&A - All about browser interoperabilityAmir Zmora
 
Iñaki Baz - VoIP2DAY 2016 | mediasoup: The programmable media server
Iñaki Baz - VoIP2DAY 2016 | mediasoup: The programmable media serverIñaki Baz - VoIP2DAY 2016 | mediasoup: The programmable media server
Iñaki Baz - VoIP2DAY 2016 | mediasoup: The programmable media serverVOIP2DAY
 
DevOps Training - Ho Chi Minh City
DevOps Training - Ho Chi Minh CityDevOps Training - Ho Chi Minh City
DevOps Training - Ho Chi Minh CityChristian Trabold
 
Microservices Antipatterns
Microservices AntipatternsMicroservices Antipatterns
Microservices AntipatternsC4Media
 
Janus/SIP @ OpenSIPS 2017
Janus/SIP @ OpenSIPS 2017Janus/SIP @ OpenSIPS 2017
Janus/SIP @ OpenSIPS 2017Lorenzo Miniero
 
Developing applications with Kurento
Developing applications with KurentoDeveloping applications with Kurento
Developing applications with KurentoLuis Lopez
 
WebRTC Webinar & Q&A - W3C WebRTC JS API Test Platform & Updates from W3C Lis...
WebRTC Webinar & Q&A - W3C WebRTC JS API Test Platform & Updates from W3C Lis...WebRTC Webinar & Q&A - W3C WebRTC JS API Test Platform & Updates from W3C Lis...
WebRTC Webinar & Q&A - W3C WebRTC JS API Test Platform & Updates from W3C Lis...Amir Zmora
 
Write a SocialTV app @ OpenSIPS 2021
Write a SocialTV app @ OpenSIPS 2021Write a SocialTV app @ OpenSIPS 2021
Write a SocialTV app @ OpenSIPS 2021Lorenzo Miniero
 
Developing rich multimedia applications with FI-WARE.
Developing rich multimedia applications with FI-WARE.Developing rich multimedia applications with FI-WARE.
Developing rich multimedia applications with FI-WARE.Luis Lopez
 
WebRTC Check-in (from WebRTC Boston 6)
WebRTC Check-in (from WebRTC Boston 6)WebRTC Check-in (from WebRTC Boston 6)
WebRTC Check-in (from WebRTC Boston 6)Chad Hart
 
Creativu Cebit Crs4
Creativu Cebit Crs4Creativu Cebit Crs4
Creativu Cebit Crs4mauromereu
 

Similaire à Developing a Media Fragment Node.JS Server (20)

The future of multimedia communications and services: Kurento and it's role
The future of multimedia communications and services: Kurento and it's roleThe future of multimedia communications and services: Kurento and it's role
The future of multimedia communications and services: Kurento and it's role
 
Quality of Experience of Web-based Adaptive HTTP Streaming Clients in Real-Wo...
Quality of Experience of Web-based Adaptive HTTP Streaming Clients in Real-Wo...Quality of Experience of Web-based Adaptive HTTP Streaming Clients in Real-Wo...
Quality of Experience of Web-based Adaptive HTTP Streaming Clients in Real-Wo...
 
Kurento: a media server architecture and API for WebRTC
Kurento: a media server architecture and API for WebRTCKurento: a media server architecture and API for WebRTC
Kurento: a media server architecture and API for WebRTC
 
[ElastixWorld 2016] mediasoup: Powerful WebRTC SFU for Node.js
[ElastixWorld 2016] mediasoup: Powerful WebRTC SFU for Node.js[ElastixWorld 2016] mediasoup: Powerful WebRTC SFU for Node.js
[ElastixWorld 2016] mediasoup: Powerful WebRTC SFU for Node.js
 
WebRTC Broadcasting @ TADSummit 2023
WebRTC Broadcasting @ TADSummit 2023WebRTC Broadcasting @ TADSummit 2023
WebRTC Broadcasting @ TADSummit 2023
 
Simplifying and accelerating converged media with Open Visual Cloud
Simplifying and accelerating converged media with Open Visual CloudSimplifying and accelerating converged media with Open Visual Cloud
Simplifying and accelerating converged media with Open Visual Cloud
 
voip2day 2016: mediasoup, powerful WebRTC SFU for Node.js
voip2day 2016: mediasoup, powerful WebRTC SFU for Node.jsvoip2day 2016: mediasoup, powerful WebRTC SFU for Node.js
voip2day 2016: mediasoup, powerful WebRTC SFU for Node.js
 
Kurento - FI-WARE Bootcamp
Kurento - FI-WARE BootcampKurento - FI-WARE Bootcamp
Kurento - FI-WARE Bootcamp
 
Developing rich multimedia applications with Kurento: a tutorial for JavaScri...
Developing rich multimedia applications with Kurento: a tutorial for JavaScri...Developing rich multimedia applications with Kurento: a tutorial for JavaScri...
Developing rich multimedia applications with Kurento: a tutorial for JavaScri...
 
WebRTC Standards & Implementation Q&A - All about browser interoperability
WebRTC Standards & Implementation Q&A - All about browser interoperabilityWebRTC Standards & Implementation Q&A - All about browser interoperability
WebRTC Standards & Implementation Q&A - All about browser interoperability
 
Iñaki Baz - VoIP2DAY 2016 | mediasoup: The programmable media server
Iñaki Baz - VoIP2DAY 2016 | mediasoup: The programmable media serverIñaki Baz - VoIP2DAY 2016 | mediasoup: The programmable media server
Iñaki Baz - VoIP2DAY 2016 | mediasoup: The programmable media server
 
DevOps Training - Ho Chi Minh City
DevOps Training - Ho Chi Minh CityDevOps Training - Ho Chi Minh City
DevOps Training - Ho Chi Minh City
 
Microservices Antipatterns
Microservices AntipatternsMicroservices Antipatterns
Microservices Antipatterns
 
Janus/SIP @ OpenSIPS 2017
Janus/SIP @ OpenSIPS 2017Janus/SIP @ OpenSIPS 2017
Janus/SIP @ OpenSIPS 2017
 
Developing applications with Kurento
Developing applications with KurentoDeveloping applications with Kurento
Developing applications with Kurento
 
WebRTC Webinar & Q&A - W3C WebRTC JS API Test Platform & Updates from W3C Lis...
WebRTC Webinar & Q&A - W3C WebRTC JS API Test Platform & Updates from W3C Lis...WebRTC Webinar & Q&A - W3C WebRTC JS API Test Platform & Updates from W3C Lis...
WebRTC Webinar & Q&A - W3C WebRTC JS API Test Platform & Updates from W3C Lis...
 
Write a SocialTV app @ OpenSIPS 2021
Write a SocialTV app @ OpenSIPS 2021Write a SocialTV app @ OpenSIPS 2021
Write a SocialTV app @ OpenSIPS 2021
 
Developing rich multimedia applications with FI-WARE.
Developing rich multimedia applications with FI-WARE.Developing rich multimedia applications with FI-WARE.
Developing rich multimedia applications with FI-WARE.
 
WebRTC Check-in (from WebRTC Boston 6)
WebRTC Check-in (from WebRTC Boston 6)WebRTC Check-in (from WebRTC Boston 6)
WebRTC Check-in (from WebRTC Boston 6)
 
Creativu Cebit Crs4
Creativu Cebit Crs4Creativu Cebit Crs4
Creativu Cebit Crs4
 

Dernier

Computer 10 Lesson 8: Building a Website
Computer 10 Lesson 8: Building a WebsiteComputer 10 Lesson 8: Building a Website
Computer 10 Lesson 8: Building a WebsiteMavein
 
LESSON 10/ GROUP 10/ ST. THOMAS AQUINASS
LESSON 10/ GROUP 10/ ST. THOMAS AQUINASSLESSON 10/ GROUP 10/ ST. THOMAS AQUINASS
LESSON 10/ GROUP 10/ ST. THOMAS AQUINASSlesteraporado16
 
Presentation2.pptx - JoyPress Wordpress
Presentation2.pptx -  JoyPress WordpressPresentation2.pptx -  JoyPress Wordpress
Presentation2.pptx - JoyPress Wordpressssuser166378
 
Bio Medical Waste Management Guideliness 2023 ppt.pptx
Bio Medical Waste Management Guideliness 2023 ppt.pptxBio Medical Waste Management Guideliness 2023 ppt.pptx
Bio Medical Waste Management Guideliness 2023 ppt.pptxnaveenithkrishnan
 
WordPress by the numbers - Jan Loeffler, CTO WebPros, CloudFest 2024
WordPress by the numbers - Jan Loeffler, CTO WebPros, CloudFest 2024WordPress by the numbers - Jan Loeffler, CTO WebPros, CloudFest 2024
WordPress by the numbers - Jan Loeffler, CTO WebPros, CloudFest 2024Jan Löffler
 
Check out the Free Landing Page Hosting in 2024
Check out the Free Landing Page Hosting in 2024Check out the Free Landing Page Hosting in 2024
Check out the Free Landing Page Hosting in 2024Shubham Pant
 
TYPES AND DEFINITION OF ONLINE CRIMES AND HAZARDS
TYPES AND DEFINITION OF ONLINE CRIMES AND HAZARDSTYPES AND DEFINITION OF ONLINE CRIMES AND HAZARDS
TYPES AND DEFINITION OF ONLINE CRIMES AND HAZARDSedrianrheine
 
Zero-day Vulnerabilities
Zero-day VulnerabilitiesZero-day Vulnerabilities
Zero-day Vulnerabilitiesalihassaah1994
 
LESSON 5 GROUP 10 ST. THOMAS AQUINAS.pdf
LESSON 5 GROUP 10 ST. THOMAS AQUINAS.pdfLESSON 5 GROUP 10 ST. THOMAS AQUINAS.pdf
LESSON 5 GROUP 10 ST. THOMAS AQUINAS.pdfmchristianalwyn
 
Introduction to ICANN and Fellowship program by Shreedeep Rayamajhi.pdf
Introduction to ICANN and Fellowship program  by Shreedeep Rayamajhi.pdfIntroduction to ICANN and Fellowship program  by Shreedeep Rayamajhi.pdf
Introduction to ICANN and Fellowship program by Shreedeep Rayamajhi.pdfShreedeep Rayamajhi
 
Vision Forward: Tracing Image Search SEO From Its Roots To AI-Enhanced Horizons
Vision Forward: Tracing Image Search SEO From Its Roots To AI-Enhanced HorizonsVision Forward: Tracing Image Search SEO From Its Roots To AI-Enhanced Horizons
Vision Forward: Tracing Image Search SEO From Its Roots To AI-Enhanced HorizonsRoxana Stingu
 
Benefits of doing Internet peering and running an Internet Exchange (IX) pres...
Benefits of doing Internet peering and running an Internet Exchange (IX) pres...Benefits of doing Internet peering and running an Internet Exchange (IX) pres...
Benefits of doing Internet peering and running an Internet Exchange (IX) pres...APNIC
 

Dernier (12)

Computer 10 Lesson 8: Building a Website
Computer 10 Lesson 8: Building a WebsiteComputer 10 Lesson 8: Building a Website
Computer 10 Lesson 8: Building a Website
 
LESSON 10/ GROUP 10/ ST. THOMAS AQUINASS
LESSON 10/ GROUP 10/ ST. THOMAS AQUINASSLESSON 10/ GROUP 10/ ST. THOMAS AQUINASS
LESSON 10/ GROUP 10/ ST. THOMAS AQUINASS
 
Presentation2.pptx - JoyPress Wordpress
Presentation2.pptx -  JoyPress WordpressPresentation2.pptx -  JoyPress Wordpress
Presentation2.pptx - JoyPress Wordpress
 
Bio Medical Waste Management Guideliness 2023 ppt.pptx
Bio Medical Waste Management Guideliness 2023 ppt.pptxBio Medical Waste Management Guideliness 2023 ppt.pptx
Bio Medical Waste Management Guideliness 2023 ppt.pptx
 
WordPress by the numbers - Jan Loeffler, CTO WebPros, CloudFest 2024
WordPress by the numbers - Jan Loeffler, CTO WebPros, CloudFest 2024WordPress by the numbers - Jan Loeffler, CTO WebPros, CloudFest 2024
WordPress by the numbers - Jan Loeffler, CTO WebPros, CloudFest 2024
 
Check out the Free Landing Page Hosting in 2024
Check out the Free Landing Page Hosting in 2024Check out the Free Landing Page Hosting in 2024
Check out the Free Landing Page Hosting in 2024
 
TYPES AND DEFINITION OF ONLINE CRIMES AND HAZARDS
TYPES AND DEFINITION OF ONLINE CRIMES AND HAZARDSTYPES AND DEFINITION OF ONLINE CRIMES AND HAZARDS
TYPES AND DEFINITION OF ONLINE CRIMES AND HAZARDS
 
Zero-day Vulnerabilities
Zero-day VulnerabilitiesZero-day Vulnerabilities
Zero-day Vulnerabilities
 
LESSON 5 GROUP 10 ST. THOMAS AQUINAS.pdf
LESSON 5 GROUP 10 ST. THOMAS AQUINAS.pdfLESSON 5 GROUP 10 ST. THOMAS AQUINAS.pdf
LESSON 5 GROUP 10 ST. THOMAS AQUINAS.pdf
 
Introduction to ICANN and Fellowship program by Shreedeep Rayamajhi.pdf
Introduction to ICANN and Fellowship program  by Shreedeep Rayamajhi.pdfIntroduction to ICANN and Fellowship program  by Shreedeep Rayamajhi.pdf
Introduction to ICANN and Fellowship program by Shreedeep Rayamajhi.pdf
 
Vision Forward: Tracing Image Search SEO From Its Roots To AI-Enhanced Horizons
Vision Forward: Tracing Image Search SEO From Its Roots To AI-Enhanced HorizonsVision Forward: Tracing Image Search SEO From Its Roots To AI-Enhanced Horizons
Vision Forward: Tracing Image Search SEO From Its Roots To AI-Enhanced Horizons
 
Benefits of doing Internet peering and running an Internet Exchange (IX) pres...
Benefits of doing Internet peering and running an Internet Exchange (IX) pres...Benefits of doing Internet peering and running an Internet Exchange (IX) pres...
Benefits of doing Internet peering and running an Internet Exchange (IX) pres...
 

Developing a Media Fragment Node.JS Server

  • 1. Master Degree Course in Cinema and Media Engineering SUPERVISOR Giovanni Malnati INTERNSHIP SUPERVISORS Raphaël Troncy José Luis Redondo Garcìa CANDIDATE Pasquale Lisena
  • 2. Pasquale Lisena Developing a Media Fragment Server in Node.JS PART I What is a «Media Fragment»? PART II State of the art PART III Implementation 2
  • 3. Pasquale Lisena Developing a Media Fragment Server in Node.JS PART I What is a «Media Fragment»? PART II State of the art PART III Implementation 3
  • 4. Pasquale Lisena Developing a Media Fragment Server in Node.JS 4 • Share • Bookmark • Save band • Annotate • Save time • Search • …and more ADVANTAGES PARTI
  • 5. Pasquale Lisena Developing a Media Fragment Server in Node.JS 5 TEMPORAL DIMENSION (T) SPATIAL DIMENSION (XYWH) MEDIA FRAGMENTS URI 1.0 RECOMENDATION PARTI
  • 6. Pasquale Lisena Developing a Media Fragment Server in Node.JS 6 MEDIA FRAGMENTS URI 1.0 RECOMENDATION TRACK DIMENSION NAMED DIMENSION (ID) PARTI
  • 7. Pasquale Lisena Developing a Media Fragment Server in Node.JS Query format 7 Hash format http://www.example.com/example.ogv#t=10,20 http://www.example.com/example.ogv?t=10,20 Server generates the fragment resource HTTP Range request (time) User Agent is in charge of display it correctly PARTI
  • 8. Pasquale Lisena Developing a Media Fragment Server in Node.JS PART I What is a «Media Fragment»? PART II State of the art PART III Implementation 8
  • 9. Pasquale Lisena Developing a Media Fragment Server in Node.JS 9 CLIENTS VIDEO PLATFORMS TEMPORAL NPT (hh:mm:ss) SMPTE - Clock SPATIAL Only start Not standard syntax PARTII
  • 10. Pasquale Lisena Developing a Media Fragment Server in Node.JS 10 CLIENT IMPLEMENTATIONS • Synote Media Fragment Player • NinSuna Media Fragment Player SERVER IMPLEMENTATIONS • NinSuna Media Fragment Server • Rafael POLYFILL PARSER LIBRARY • MediaFragment.js PARTII
  • 11. Pasquale Lisena Developing a Media Fragment Server in Node.JS 11 CLIENT IMPLEMENTATIONS SYNOTE MEDIA FRAGMENT PLAYER • Cross-browser (Flash fallback) • HTML5, YouTube, Daylimotion, Vimeo support • HTML5-like interface https://github.com/pasqLisena/Media-Fragment-Player • JavaScript plugin • Spatial and temporal support • No Time range requests • Highlight of fragment in timeline • Dark mask for spatial fragment OUR CONTRIBUTION PARTII
  • 12. Pasquale Lisena Developing a Media Fragment Server in Node.JS 12 SERVER IMPLEMENTATIONS NINSUNA MEDIA FRAGMENT SERVER RAFAEL • Pre-processing • Annotation-based • Support for Time range request • Extraction on the fly • Fragment stored on file system • Support only for query fragments http://ninsuna.elis.ugent.be/MediaFragmentsServer https://github.com/Noterik/Rafael PARTII
  • 13. Pasquale Lisena Developing a Media Fragment Server in Node.JS 13 MEDIAFRAGMENT.JS http://www.example.com/video.ogv ?t=1:00:00#t=npt:10,20 &xywh=percent:25,25,50,50 { "query":{ "t":[ { "value":"1:00:00", "unit":"npt", "start":"1:00:00", "end":"", "startNormalized":3600, "endNormalized":"" } ] }, "hash":{ "t":[ { "value":"npt:10,20", "unit":"npt", https://github.com/tomayac/Media-Fragments-URI/ OUR CONTRIBUTION PARSING + UNIT NORMALIZATION + ERROR DETECTION PARTII
  • 14. Pasquale Lisena Developing a Media Fragment Server in Node.JS PART I What is a «Media Fragment»? PART II State of the art PART III Implementation 14
  • 15. Pasquale Lisena Developing a Media Fragment Server in Node.JS Query Fragment • Time (npt) • Track (video/audio) • Xywh 15 Hash fragment • Range request (npt) PARTIII
  • 16. Pasquale Lisena Developing a Media Fragment Server in Node.JS 16 PARTIII
  • 17. Pasquale Lisena Developing a Media Fragment Server in Node.JS 17 FRAGMENT QUERY FFMPEG OPTION NOTE t=10 -ss 10 t=,20 -to 20 t=10,20 -ss 10 -to 20 track=video -an no audio track=audio -vn no video xywh=10,10,50,60 -filter:v "crop=50:60:10:10" require transcoding xywh=percent:10,10,50,60 -filter:v "crop=in_w*50/100: in_h*60/100:in_w*10/100: in_h*10/100" require transcoding PARTIII
  • 18. Pasquale Lisena Developing a Media Fragment Server in Node.JS 18 400px 220px 00:21:43 video.mp4 ? t= 00:00:10, 00:00:20 & xywh=0,0,400,220 PARTIII
  • 19. Pasquale Lisena Developing a Media Fragment Server in Node.JS ALIAS collection 19 t=10,20 and t=11,20 are byte identical video_10.45-19.41 video_10-20 video_11-20 GridFS Auto-deletion of older resources. PARTIII
  • 20. Pasquale Lisena Developing a Media Fragment Server in Node.JS 20 mediafragment.js ffmpegmongoDB PARTIII
  • 21. Pasquale Lisena Developing a Media Fragment Server in Node.JS 21 PARTIII
  • 22. Pasquale Lisena Developing a Media Fragment Server in Node.JS 22 HTTP/1.1 206 Partial Content Accept-Ranges: bytes, t, id Content-Length: 3795 Content-Type: video/ogg Content-Range-Mapping: { t:npt 9.85-21.16/0.0-653.79;include-setup } = { bytes 0-52,19147-22880/35614993 } Content-type: multipart/byteranges; boundary=BOUNDARY Etag: "b7a60-21f7111-46f3219476580" --BOUNDARY Content-type: video/ogg Content-Range: bytes 0-52/35614993 {binary data} --BOUNDARY Content-type: video/ogg Content-Range: bytes 19147-22880/35614993 {binary data} ---BOUNDARY-- METADATA DATA PARTIII
  • 23. Pasquale Lisena Developing a Media Fragment Server in Node.JS 23 localhost:3000/video/video.mp4?t=10,20 localhost:3000/video/video.mp4#t=10,20
  • 24. Pasquale Lisena Developing a Media Fragment Server in Node.JS 24 • We contributed to the generic "media-fragment.js" polyfill and prepare a Node.JS version (open source) • We contributed to the Synote Media Player to build a more generic media fragment player (open source) • We discovered bugs in the W3C specification and proposed amendments (revised text) to be endorsed by W3C • We implemented a media fragment server in Node.JS (using ffmeg) + a smart cache (based on MongoDB) • Future work: finalize the Chrome extension so that the browser understand the response from the server, extend support to all media and test MaFFiN on a more large scale environment.