HTML5 multimedia - where we are, where we're going

HTML5 multimedia
Where we are, where we're going
Bruce Lawson

Anne van Kesteren annevk at opera.com
Wed Feb 28 05:47:56 PST 2007

Hi,
Opera has some internal expiremental builds with an implementation of a <video> element. The element
exposes a simple API (for the moment) much like the Audio() object:
play()
pause()
Stop()

The idea is that it works like <object> except that it has special <video> semantics much like <img> has
image semantics. In markup you could prolly use it as follows:

<figure>
<video src=news-snippet.ogg>
...
</video>
<legend>HTML5 in BBC News</legend>
</figure>

I attached a proposal for the element and as you can see there are still some open issues. The element and
its API are of course open for debate. We're not enforcing this upon the world ;-)

Cheers,
http://lists.whatwg.org/pipermail/whatwg-whatwg.org/2007-February/009702.html

<video src=pudding.webm
controls
autoplay
poster=poster.jpg
width=320 height=240>
<a href=video.webm>Download movie</a>
</video>

<audio src=bieber.ogg
controls
autoplay>
<a href=bieber.ogg>Download horrid pap</a>
</audio>

www.w3.org/TR/html5/video.html#media-elements

controlling <video> with JavaScript
var v = document.getElementByTagName('video')
[0];

v.play();
v.pause();
v.volume = … ;
v.currentTime = … ;
…

events fired by <video>
var v = document.getElementById('player');

v.addEventListener('loadeddata', function() { … }, true)
v.addEventListener('play', function() { … }, true)
v.addEventListener('pause', function() { … }, true)
v.addEventListener('timeupdate', function() { … }, true)
v.addEventListener('ended', function() { … }, true)
…

video as native object...why is it important?

●
keyboard accessibility built-in
●
“play nice” with rest of the page
●
Simple API for controls

"In addition to giving video an HTML
element, we must also agree on a baseline
video format that will be universally
supported, just like the GIF, JPEG and PNG
image format are universally supported. It's
important that the video format we choose
can be supported by a wide range of devices
and that it's royalty-free (RF). RF is a well-
established principle for W3C standards."
http://people.opera.com/howcome/2007/video/

Ogg Theora

“free and open”, no licensing/royalties
not many tools for it, not web optimised

MP4 / H.264

ubiquitous, patent encumbered, licensing/royalties,
hardware accelerated

WebM

"open and royalty-free", web optimised,
hardware acceleration on its way

video formats

webM Ogg/ Theora mp4/ h264

Opera yes yes

Chrome yes yes Nope
(Chrome.soon)
Firefox Yes (FF4) yes

Safari yes

IE9 Yes (if installed) yes

audio formats

mp3 Ogg/ Vobis wav

Opera yes yes

Chrome yes yes yes

Firefox yes yes

Safari yes yes

IE9 yes

Giving everybody video

<video controls autoplay poster=… width=… height=…>
<source src=pudding.mp4 type=video/mp4>
<source src=pudding.webm type=video/webm>
<source src=pudding.ogv type=video/ogg>

</video>

<video controls poster="…" width="…" height="…">
<source src="movie.mp4" type="video/mp4" />
<source src="movie.webm" type="video/webm" />
<source src="movie.ogv" type="video/ogg" />

<object width="…" height="…" type="application/x-
shockwave-flash" data="player.swf">
<param name="movie" value="player.swf" />
<param name="flashvars" value=" … file=movie.mp4" />
</object>

</video>

still include fallback for old browsers
http://camendesign.com/code/video_for_everybody

Help
●
archive.org converts and hosts creative-commons
licensed media
● vid.ly has a free conversion/ hosting service (max 1GB
source file) see vid.ly/5u4h3e
● Miro video converter is a drag and drop GUI skin on
FFMPEG

video media queries

<video controls>
<source src=hi-res.ogv media="(min-device-width:800px)">
<source src=lo-res.ogv>
</video>

http://dev.w3.org/html5/spec/video.html#the-source-element

Full-screen video
●
Currently, WebkitEnterFullscreen();
●
May 11, WebKit announced it's implementing Gecko API
https://wiki.mozilla.org/Gecko:FullScreenAPI
●
Opera likes this approach, too

WebM release does not support subtitles

WHATWG / W3C RFC will release guidance on subtitles and other
overlays in HTML5 <video> in the near future. WebM intends to follow
that guidance.

http://code.google.com/p/webm/issues/detail?id=11
Egg image Kacper "Kangel" Aniołek http://commons.wikimedia.org/wiki/File:Egg.jpg

<track> element

<video controls poster=… width=… height=…>
<source src=movie.webm type=video/webm>
<track src=subtitles-en.vtt kind=subtitles srclang=en>
<track src=subtitles-de.vtt kind=subtitles srclang=de>
</video>

http://www.whatwg.org/specs/web-apps/current-work/multipage/video.html#the-track-element

webVTT

WEBVTT FILE

1
01:23:45,678 --> 01:23:46,789
Hello world!

2
01:23:48,910 --> 01:23:49,101
Mój poduszkowiec jest pełen węgorzy

WebVTT formatting

Supports positioning of text
Supports <b> and <i>
Colouring individual speakers
Support vertical text
Supports RTL
Supports ruby annotations
Validator: http://quuz.org/webvtt/

playr
http://www.delphiki.com/html5/playr/

LeanBack Player
http://leanbackplayer.com/

MediaElement.js
http://mediaelementjs.com/

Synchronising media elements
Each media element can have a MediaController. A MediaController is an object
that coordinates the playback of multiple media elements, for instance so that a
sign-language interpreter track can be overlaid on a video track, with the two
being kept in sync....
Media elements with a MediaController are said to be slaved to their controller. The
MediaController modifies the playback rate and the playback volume of each of the
media elements slaved to it, and ensures that when any of its slaved media
elements unexpectedly stall, the others are stopped at the same time.
When a media element is slaved to a MediaController, its playback rate is fixed to
that of the other tracks in the same MediaController, and any looping is disabled.
http://www.whatwg.org/specs/web-apps/current-work/multipage/video.html#synchronising-
multiple-media-elements
http://www.w3.org/WAI/PF/HTML/wiki/Media_Multitrack_Media_API

How mature is all this?
Looping latency
people.opera.com/patrickl/experiments/audio/hacky-looper/

HTML5 video issues on the iPad and how to solve them
blog.millermedeiros.com/2011/03/html5-video-issues-on-the-ipad-and-how-to-solve-them/

Unsolved HTML5 video issues on iOS
http://blog.millermedeiros.com/2011/04/unsolved-html5-video-issues-on-ios/

Audio Sprites (and fixes for iOS)
remysharp.com/2010/12/23/audio-sprites/

“...extending the language to better support Web
applications … This puts HTML in direct competition
with other technologies intended for applications deployed
over the Web, in particular Flash and Silverlight.”

Ian Hickson, Editor of HTML5
http://lists.w3.org/Archives/Public/public-html/2009Jan/0215.html

Image © Jackdrawsanything.com "I will draw anything you like to raise funds for the Sick Kids"

DISCLAIMER:

This is my personal perspective and does not reflect the
opinion of Opera.
Or my wife, children or hamster. Or anyone else who might
be embarrassed by association with me (which is mostly
everyone, to be honest).
As this is a conference, in the land of Żubrówka, I'll probably
get drunk and try to chat you up, so let's make this
disclaimer run until Saturday, OK?

DRM on HTML5 without changes

Henri Sivonen
lists.whatwg.org/htdig.cgi/whatwg-whatwg.org/2010-
July/027051.html

getUserMedia API
(previously known as "HTML5 <device>")

<video autoplay></video>
<script>
var video = document.querySelector(video);

If (navigator.getUserMedia)
{navigator.getUserMedia('video',successCallback,
errorCallback);

function successCallback(stream) {
video.src = stream;
}
</script>

Tutorial:
http://dev.opera.com/articles/view/playing-with-html5-video-and-getusermedia-support/

my.opera.com/core/blog/2011/03/23/webcam-orientation-preview

WebRTC is a free, open project that enables web browsers
with Real-Time Communications (RTC) capabilities via
simple Javascript APIs. The WebRTC components have been
optimized to best serve this purpose.

The WebRTC initiative is a project supported by Google,
Mozilla and Opera.

http://sites.google.com/site/webrtc/
http://www.whatwg.org/specs/web-apps/current-work/complete/video-conferencing-
and-peer-to-peer-communication.html

"We expect to see WebRTC support in Firefox, Opera, and
Chrome soon!"

http://www.webrtc.org/

Case study
http://www.rathergood.com/buffy

//grab all the buttons
buttons = document.querySelectorAll('button'),

//loop through all the <button> elements, and create
corresponding
//audio elements

buildAudio = (function(){
for (var i = 0, l = buttons.length; i < l; i++) {
var key = buttons[i].textContent.toLowerCase();

//using the fancy new Audio() constructor here

document.body.appendChild(new Audio(key + '.ogg'));

};
})()

document.addEventListener('ended', function(e){

if (e.target.nodeName == "AUDIO") {
// pause the video since the audio
//has stopped

vid.pause();
}

}, true)

<button data-text="<q> element">Q</button>
<button data-text="Web Workers">W</button>

text = document.getElementById('thinger')

…

text.textContent = el.dataset ? el.dataset.text :
el.getAttribute('data-text');

#thinger {position:absolute; left:40px;
top:200px;

text-align: center;

font-family: 'hixie', cursive, monospace; font-
size:3em; color:red;

transform: matrix(0.98,-0.17,-0.17,0.98,0,0);}

AEZ Segar font, made into fontface kit with
http://www.fontsquirrel.com/fontface/generator

Thanks

Mike Taylor @miketaylr for JS help, and Patrick Lauke
@patrick_h_lauke for the Hixie voice. (What does that "H"
stand for?) to me slowly and patiently

All rights reserved corner

Thanks Takara Tomy for permission to use Gigapudding
video. Buy the pudding.
Thanks Jack Henderson for permission to his scary DRM
picture. Buy his book.
HTML5 nazi-killing Bruce on unicorn picture by Marina
Lawson. Buy her Dad's book.

All others by me or (I believe) public domain.

www.opera.com/developer
bruce.lawson@opera.com
www.brucelawson.co.uk
twitter.com/brucel

HTML5 multimedia - where we are, where we're going

Recommandé

Recommandé

Contenu connexe

Tendances

Tendances (20)

Similaire à HTML5 multimedia - where we are, where we're going

Similaire à HTML5 multimedia - where we are, where we're going (20)

Plus de brucelawson

Plus de brucelawson (16)

Dernier

Dernier (20)

HTML5 multimedia - where we are, where we're going