Multimedia on the web - HTML5 video and audio
•
19 j'aime•4,329 vues
A lecture given at MIT in Boston about the benefits and technicalities of open web standards for Video and Audio. Lots of examples how to manipulate live video using CSS3 and Canvas.
Recommandé
Contenu connexe
Tendances
Tendances (20)
En vedette
En vedette (20)
Similaire à Multimedia on the web - HTML5 video and audio
Similaire à Multimedia on the web - HTML5 video and audio (20)
Plus de Christian Heilmann
Plus de Christian Heilmann (20)
Dernier
Dernier (20)
Multimedia on the web - HTML5 video and audio
- 1. Multimedia on the web. HTML5 Audio and Video Christian Heilmann, MIT Boston, January 2012
- 2. I’m Chris.
- 3. What we will cover: ★ Quick history of Multimedia on the web ★ Annoyances with Flash ★ HTML5 audio and video ★ Painful stuff – codecs and conversion ★ Embedding ★ Controlling ★ Transforming ★ Realtime changes ★ Awesome audio stuff
- 4. Quick history of Multimedia on the web
- 5. First, there were images. JPG, GIF, PNG (later), WBMP, ICO
- 6. Connections were bad. Progressive JPG, interlaced GIF, <img src=”foo.jpg” lowsrc=”foogrey.gif” alt=”a foo, what else?”>
- 7. Animation was done with animated GIFs or with JavaScript animation.
- 8. Audio was mostly Midi background music. (and if you think about using that today I will hit you!)
- 9. Other things we did were Java Applets.
- 11. And many others... ★ Quicktime / Quicktime VR ★ Microsoft Media Player ★ Shockwave ★ Acrobat had some image editing features. ★ iPix / VRML and many other forgotten ones...
- 12. They all had the same issues. ★ End users must install a plugin. ★ You need to upgrade the plugin constantly. ★ There is limited interaction with the rest of the page.
- 13. Another big issue is and was security – plugins are one of main attack vectors for malware.
- 14. In the end, one plugin prevailed over all the others: Flash.
- 15. With aggressive marketing, clever partnerships and a rich dev environment Flash became the multimedia choice for the web.
- 16. And there is the DRM stuff...
- 18. + Apple and Flash movies means a lot of times your fan will start up. =
- 19. Why would I want yet another editor to build web content?
- 20. WTF is all this? <object width="640" height="385"> <param name="movie" value="http://www.youtube.com/ v/SJixW2u4IvQ?fs=1&hl=en_US" /> <param name="allowFullScreen" value="true" /> <param name="allowscriptaccess" value="always" /> <embed src="http://www.youtube.com/v/SJixW2u4IvQ? fs=1&hl=en_US" type="application/x-shockwave- flash" allowscriptaccess="always" allowfullscreen="true" width="640" height="385" /> </object>
- 21. Flash is a black box Alien inside a document.
- 22. Therefore it is also an accessibility problem. ★ Other browsers than IE don’t allow you to access Flash with a keyboard. ★ You are at the mercy of the developers to make their movies keyboard accessible. ★ Which is bad, as audio and video can help a lot of people to understand things.
- 23. Flash can talk to JavaScript via APIs.
- 25. You are at the mercy of the API developers though – what they don’t make public, you can’t reach.
- 26. HTML5 audio and video
- 27. HTML5 audio and video make things much simpler: ★ Betteraccessibility ★ Doing one thing well ★ Much simpler API ★ Allows for styling and overlays ★ View-source “hackable”
- 28. Screencast of Oprah’s web site http://www.oprah.com/own
- 29. Painful stuff – Codecs and conversion
- 30. So, we understood that there is a need for an open high-fidelity technology if we really want the web to be a media.
- 31. What we didn’t quite consider yet is something that both accelerated and hindered innovation for a long time.
- 32. Copyright
- 33. Videos on the web are encoded to make them smaller – there is no such things as an “AVI File”...
- 34. They are containers for video in a certain compression and audio in another.
- 35. The issue was that all these compression formats had their own copyright and didn’t work with open technology like HTML5.
- 36. Therefore we needed new formats: ★ H.264 “MP4” ★ OGG/Vorbis ★ VP8 / WebM
- 37. Our job is now to convert the videos we record to these open formats – which can be annoying.
- 38. The main annoyance is that different browsers support different formats, which means we need to provide several versions :(
- 39. Audacity: http://audacity.sourceforge.net/ WebM tools: http://www.webmproject.org/tools/ Evom: http://thelittleappfactory.com/evom/ VLC: http://www.videolan.org/vlc/ Ogg convert: http://oggconvert.tristanb.net/ Firefogg: http://firefogg.org/ TinyOgg: http://tinyogg.com/ ffmpeg: http://ffmpeg.org/
- 42. Embedding
- 43. Embedding audio and video in an HTML5 document is easy: <audio src=”foo.ogg”> If your browser didn’t suck, you’d have audio here. </audio> <video src=”foo.ogv”> If your browser didn’t suck, you’d have video here. </video>
- 44. That doesn’t do anything yet <video src=”foo.ogv” controls> If your browser didn’t suck, you’d have video here. </video> ★ Controls appear on mouse hover or tabbing ★ Keyboard enabled controls ★ Video can be styled in any way.
- 45. Controls differ from browser to browser... Firefox Opera Safari Full Screen Chrome
- 46. So to give it to all browsers... <video controls> <source src="http://www.archive.org/{...}_512kb.mp4" type="video/mp4"></source> <source src="http://www.archive.org/{...}nsters.ogv" type="video/ogg"></source> <p>Your browser doesn't like HTML5 video, watch the movie on <a href="http://www.archive.org/{...}_monsters"> archive.org</a>. </p> </video>
- 47. Things to consider: ★ Use MP4 always as the first option to support iOS devices (iPads, iPhone and so on...) ★ If you omit the type attribute, the browser loads the meta data of each source!
- 48. Media queries can save bandwidth: <video controls> <source src="http://www.archive.org/{...}_512kb.mp4" type="video/mp4" media="(min-device-width:800px)"></source> <source src="http://www.archive.org/{...}_low.mp4" type="video/mp4"></source> <source src="http://www.archive.org/{...}nsters.ogv" type="video/ogg"></source> <p>watch the movie on <a href="{...}_monsters"> archive.org</a>.</p> </video>
- 49. Other attributes: ★ poster – define a picture to show before loading. ★ height/width – oh, well... ★ loop – automatically restart ★ preload (auto/none/ metadata)
- 50. But what if you don’t like the controls?
- 51. Controlling
- 52. HTML5’s Media API gives you control: ★ load() – load a new media. ★ canPlayType(type) – returns probably, maybe and “” (really!) ★ play() – play the movie ★ pause() – pause the movie. ★ addTrack(label,kind,language) -for subtitles
- 53. Video details: width / height / videoWidth / videoHeight / poster Controls: controls / volume / muted Tracks: tracks Network state: src / currentSrc / networkState / preload / buffered Ready state readyState / seeking Playback state currentTime / startTime / duration / paused / defaultPlayBackRate / playbackRate / played / seekable / ended / autoplay / loop
- 54. Writing a play button is simple: var audio = document.getElementsByTagName('audio')[0]; var play = document.getElementsByClassName('play')[0]; play.addEventListener('click',function(e){ var t = e.target; if(audio.paused){ audio.play(); t.innerHTML = 'pause'; } else { audio.pause(); t.innerHTML = 'play'; } e.preventDefault(); });
- 55. However, simply checking properties is not safe!
- 56. Any JS use gets much safer by listening to events.
- 57. HTML5 video events: loadstart / progress / suspend / abort / error / emptied / stalled / play / pause / loadedmetadata / loadeddate / waiting / playing / canplay / canplaythrough / seeking / seeked / timeupdate / ended / ratechange
- 58. video.addEventListener('play', playEvent, false); video.addEventListener('pause', pausedEvent, false); video.addEventListener('ended', function () { this.pause(); }, false); function playEvent() { play.innerHTML = 'pause'; } function pausedEvent() { play.innerHTML = 'play'; } play.onclick = function () { if (video.ended) { video.currentTime = 0; } if (video.paused){ video.play(); }else{ video.pause(); } };
- 60. You can use these events to sync animation or trigger changes with audio and video.
- 67. Transforming http://www.flickr.com/photos/jiazi/1061447777
- 68. I am a film buff and I like to find easter eggs in movies. Every pixar movie for example has A113 in it – the room in the uni a lot of pixar employees had their animation lectures in. To find things like that I sometimes zoom the screen and scan around. I thought this should be possible in HTML5.
- 70. Using SVG and other technologies you can change the look and feel of videos even more.
- 73. Realtime changes
- 74. http://introducinghtml5.com/examples/ch05/animate-with- video.html
- 76. http://people.mozilla.com/~prouget/demos/ DynamicContentInjection/play.xhtml
- 77. http://people.mozilla.com/~prouget/demos/tracker/ tracker.xhtml