- Nov 14, 2011
Synchronising media tracks
HTML5 will allow for alternative media tracks to be included and synchronised in a single <audio> or <video> element.
You might, for example, have several videos of a sporting event, each from different camera angles, and if the user moves to a different point in one video (or changes the playback rate for slow motion), she expects all the other videos to play in sync. Therefore, different media files need to be grouped together.
This could be a boon for accessibility, allowing for sign-language tracks, audio description tracks, dubbed audio tracks, and similar additional or alternative tracks to the main audio/video tracks.
mediagroup attribute on the <audio> or <video> element:
<div> <video src="movie.webm" autoplay controls mediagroup=movie></video> <video src="signing.webm" autoplay mediagroup=movie></video> </div>
This is very exciting, and very new, so we won’t look further: the spec is constantly changing and there are no implementations.
Video conferencing, augmented reality
Figure 4.7 Remy Sharp, with a magical HTML5 moustache. (Photo by Julia Gosling)
Norwegian developer Trygve Lie has made demos of getUserMedia that use Web Sockets (see Chapter 10) to send images from an Android phone running the experimental Opera Mobile build to a desktop computer. See https://github.com/trygve-lie/demos-html5-realtime for the source code and a video demonstrating it.
Obviously, giving websites access to your webcam could create significant privacy problems, so users will have to opt-in, much as they have to do with geolocation. But that’s a UI concern rather than a technical problem.
Taking the concept even further, there is also a Peer-to-Peer API being developed for HTML, which will allow you to hook up your camera and microphone to the <video> and <audio> elements of someone else’s browser, making it possible to do video conferencing.
In May 2011, Google announced WebRTC, an open technology for voice and video on the Web, based on the HTML5 specifications. WebRTC uses VP8 (the video codec in WebM) and two audio codecs optimised for speech with noise and echo cancellation, called iLBC, a narrowband voice codec, and iSAC, a bandwidth-adaptive wideband codec (see http://sites.google.com/site/webrtc/).
As the project website says, “We expect to see WebRTC support in Firefox, Opera, and Chrome soon!”