For many years now I have been progressing a deeper view of video on the Web than just as a binary blob. We need direct access to time offsets and sections of videos.
Such direct access can be achieved either by providing a javascript interface through which a video’s playback position can be controlled, or by using URLs that directly communicate with the Web server about controlling the playback position. I will explain the approaches that can be applied on the HTML5 <video> tag for such deep video interaction.
Controlling a video’s playback with javascript
currentTime
Right now, you can use the video element’s “currentTime” property to read and set the current playback position of a video resource. This is very useful to directly jump between different sections in the video, such as exemplified in the BBC’s recent R&D TV demo. To jump to a time offset in a video, all you have to do in javascript is:
var video = document.getElementsByTagName("video")[0];
video.currentTime = starttimeoffset;
timeupdate
Further, if you want to stop playback at a certain time point, you can use another functionality of the HTML5 <video> tag: the “timeupdate” event:
video.addEventListener("timeupdate", function() {
if (video.currentTime >= endtimeoffset) {
video.pause();
}}, false);
When the “timeupdate” event fires, which is supposed to happen at a min resolution of 250ms, you can catch the end of your desired interval fairly accurately.
setTimeout / setInterval
Alternatively to using the “timeupdate” event that is provided by the <video> tag, there is always the possibility of using the javascript “setTimeout” or “setInterval” functions:
setTimeout(video.pause(), (endtimeoffset - starttimeoffset)*1000);
The “setTimeout” function is used to call a function or evaluate an expression after a specified number of milliseconds. So, you’d have to call this straight after starting the playback at the given starttimeoffset.
If instead you wanted something to happen at a frequent rate in parallel to the video playback (such as check if you need to display a new ad or a new subtitle), you could use the javascript setInterval function:
setInterval( function() {displaySubtitle(video.currentTime);}, 100);
The “setInterval” function is used to call a function or evaluate an expression at the specified intervall. So, in the given example, every 100ms it is tested whether a new subtitle needs to be displayed for the video current playback time.
Note that for subtitles it makes a lot more sense to use the existing “timeupdate” event of the video rather than creating a frequenty setInterval interrupt, since this will continue calling the function until clearInterval() is called or the window is closed. Also, the BBC found in experiments with Firefox that “timeupdate” is more accurate than polling the “currentTime” regularly.
Controlling a video’s playback through a URL
There are some existing example implementations that control a video’s playback time through a URL.
In 2001, in the Annodex project we proposed temporal URIs and implemented the spec for Ogg content. This is now successfully in use at Metavid.org, where it is very useful since Metavid handles very long videos where direct access to subsections is critical. A URL such as http://metavid.org/wiki/Stream:Senate_proceeding_02-13-09/0:05:40/0:47:29 work well to directly view that segment.
More recently, YouTube rolled out a URI scheme to directly jump to an offset in a YouTube video, e.g. http://www.youtube.com/watch?v=PjDw3azfZWI#t=31m09s. While most YouTube content is short form, and such direct access may not make much sense for a video of less than 2 min duration, some YouTube content is long enough to make this a very useful feature.
You may have noticed that the YouTube use of URIs for jumping to offsets is slightly different to the one used by Metavid. The YouTube video will be displayed as always, but the playback position in the video player changes based on the time offset. The Metavid video in contrast will not display a transport bar for the full video, but instead only present the requested part of the video with an appropriate localised keyframe.
Having realised the need for such URLs, the W3C created a Media Fragments working group.
Proposed Time schemes
For temporal addressing, it currently proposes the following schemes:
t=10,20
t=npt:10,20
.
t=120s,121.5s
t=npt:120,0:02:01.5
.
t=smpte-30:0:02:00,0:02:01:15
t=smpte-25:0:02:00:00,0:02:01:12.1
.
t=clock:20090726T111901Z,20090726T121901Z
If there is no time scheme given, it defaults to “npt”, which stands for “normal playback time”. It is basically a time offset given in seconds, but can be provided in a few different formats.
If a “smpte” scheme is given, the time code is provided in the way in which DVRs display time codes, namely according to the SMPTE timecode standard.
Finally, a “clock” time scheme can be given. This is relevant in particular to live streaming applications, which would like to provide a URL under which a live video is provided, but also allow the user to jump back in time to previously streamed data.
Fragments and Queries
Further, the W3C Media Fragment Working Group is discussing the use of both URI addressing schemes for time offsets: fragments (“#”) and queries (“?”).
The important difference is that queries produce a new resource, while fragments provide a sub-resource.
This means that if you load a URI such as http://www.example.org/video.ogv?t=60,100 , the resulting resource is a video of duration 40s. Since relates to the full resource, it is possible to expect from the user agent (i.e. web browser) to display a timeline of 60-100 rather than 0-40 – after all, the browser could just get this out of the URL. However, it is essentially a new resource and could therefore just be regarded as a different video.
If instead you load a URI such as http://www.example.org/video.ogv#t=60,100, the user agent recognizes http://www.example.org/video.ogv as the resource and knows that it is supposed to display the 40s extract of that resource. Using no special server support, the browser could just implement this using the currentTime and timeUpdate javascript functionality.
An optimisation should, however, be made on this latter fragment delivery such that a user does not have to wait until the full beginning of the resource is downloaded before playback starts: Web servers should be expected to implement a server extension that can deal with such offsets and then deliver from the time offset rather than the beginning of the file.
How this is communicated to the server – what extra headers or http communication mechanisms should be used – is currently under discussion at the W3C Media Fragments working group.
Ginger,
Thanks for the post! I’ve been working with HTML5 for a few months and my application ties in closely with the timeupdate function. For an example, see:
http://www.ressq.com/videoTag/video.html
In the future, my application will support creating playlists of stats from multiple games. I’ll be interested to see where the fragments and queries spec goes in the future. Do you sit on the W3C Media Fragments working group?
Mike Ressler
@Micheal: yes, I do.
Hi !
I would like to be able to seek at one frame at a time.
This means that I need to have a timestamp in fraction of seconds.
Is this possible ?
Thanks in advance !
JM
What do you need that accuracy for? In theory you can address such an accuracy, but in practice, your seeking will not be able to get you that much accuracy in general. So, I wonder what the use case is.
Well, I would like to be able to do something like virtualDub or something like that. You say it is possible in theroy : how ?
And why couldn’t it be possible in practice ?
Thx !
jmh: you can address e.g. t=5.678901234 which has a very high resolution. But any encoded file, such as a mp4 or ogg theora file will not be able to deliver that kind of resolution in the compressed domain. Even when decoded it, your accuracy will be limited by the sampling frequency of the video (and to a lesser extent of the audio) channel.
The accuracy I am looking for is the framerate of the video. So something like 0.03s.
My use case for something like this would be to step through sports videos for player review. I wouldn’t need audio at all. I’d just like to be able to allow my users to step frame by frame through the video.
I’m looking forward to the variable playback rate for this very reason. Frame by frame stepping would be useful as well.
Stepping through a video is a feature of a player, so it’s up to the software to provide this. The URI addressing scheme can accommodate it. So, you’re free to implement a video player that provides that accuracy. Current browser implementations will do the best they can and with the “currenttime” attribute in HTML5 media you will be able to control it. You can write your own javascript to test this, btw. You can get inspiration from http://annodex.net/~silvia/itext/mediafrag.html . (Though: I believe there is a bug in there that ignores subseconds).
I just updated Firefox to 3.6 and now the javascript video.currentTime no longer works. It worked perfectly in 3.5.
I used the following code to start the video at a specific time that I calculated using javascript beforehand:
fallback
var video = document.getElementById(“myVideo”);
As I said, this worked fine in 3.5 but when I open the same html-file in 3.6, the video always starts playing at beginning and not at the time offset given in “TimeToJump”.
I also tried putting video.currentTime=TimeToJump into the on the bottom right under var=[…] but still, the video played from the beginning instead of the time offset.
The example given at http://annodex.net/~silvia/itext/mediafrag.html works in 3.6 so the browser seems to be capable of playing videos at a time offset, just not with the simple video.currentTime function in javascript. The code that is used for this example is way more complicated than what I used and what used to work in 3.5!
Here is the complete html-code http://pastebin.com/f40d55f69
Any help on how to get this to work as simple as possible with Firefox 3.6 (and all other browsers) would be awesome!
Thx a lot for the great information so far,
Thomas
@Thomas maybe the order in which the onload event is executed and the video load is executed has changed. I don’t actually know and it doesn’t actually matter. According to the HTML5 specification, you need to wait until the video is loaded, i.e. in loadedmetadata status before you can execute the currentTime attribute reliably. Otherwise it’s random what happens.
@Thomas, exactly to Silvia’s point, I use the loadedmetadata callback like so:
video.addEventListener(“loadedmetadata”, function () {
video.currentTime = newTime;
}, false);
And it works perfectly. I was having similar problems with Firefox 3.5 and found this solution with Silvia’s help. Good luck!
Thomas,
As of Firefox 3.6, we no longer fire the ‘load’ event for videos (bug 521359 at bugzilla.mozilla.org explains why), so the code you have in will never be executed. I suspect the other method you tried (setting currentTime after calling getElementById) tried to seek before the video had initialized (open the script error console via Tools->Error Console and look for an error to confirm this)–you need to wait until it’s safe to seek, e.g. by waiting for the loadedmetadata event.
@Thomas, you could try simply replacing the “onload” attribute with a “onloadedmetadata” – that might do the trick.
Awesome! Thanks for the great help, onloadedmetadata instead of just onload worked fine! Great and thx again!
The open media community rocks indeed 🙂 there’s help everywhere!
Silvia,
Have you heard anything about live streaming HTML5 video recently? With the industry-wide switch to WebM, I’ve been thinking about trying to get some live-streaming applications off the ground and thought you’d be a good person to ask. Is there any project working on this currently that you know of?
Mike
Flumotion have managed to get WebM streaming working. Several other efforts are being made, too. You should subscribe to the developer mailing list of WebM and discuss there.
I’m using Firefox 3.6 and javascript video.currentTime doesn’t seem to be working unless it’s just me – can anyone offer some input?
@James what does your code look like? It works fine for me, so I wonder what you may be doing wrong.
Hi Guys, Great Post!
I have a small issue, maybe I could get some help with.
I’m playing a small part of a video clip over and over again but when the clip starts again, there is a small loading pause. Is there a way to avoid this and create a smooth loop?
code:
var v = document.getElementById(‘vid’);
v.volume = 0;
v.play();
v.addEventListener(“loadedmetadata”, function () {
v.currentTime = 4;
}, false);
v.addEventListener(“timeupdate”, function() {
if (v.currentTime >= 6) {
v.currentTime = 4;
}}, false);
Thanks in advanve,
Kurt
Hi Kurt,
The better way to loop is to set the @loop attribute of the <video> element. However, the @loop attribute is not yet supported by Firefox, so you will have to do a rewind for Firefox. The loading pause should be minimal for a small file because it will still be in memory. I don’t think you can get rid of it fully.
Hi Silvia,
I am trying to request and save key frames as image files.
For example with the video http://annodex.net/~silvia/itext/elephants_dream/elephant.ogv
which is 653 seconds long, I would like to request one frame every say 10 seconds and save it as an image file. So rather than downloading the entire video I can download 65 frames. Are you able to provide any insight on how this could be done?
Hi Stephen,
You’d have to use a canvas and then a data url to do this today. E.g. context.drawImage(video, 0, 0, 160, 80); will give you a means to draw an image out of a video. Then you can do dataURL = canvas.toDataURL(“image/png”); to get that as an image back out. Let me know if it works.
Stephen,
One of the other ways to do this is with OggTools: http://dev.streamnik.de/76.html
I think the documentation is a little out of date and they prefer you use oggTranscode now. But if you wanted to present the user with a download of every 10th frame, you could use the oggTranscode to generate the vid and display it. That would cut down on bandwidth, which I think might be your intent?
if I seek the video to some time, do I have to wait for the video to downlaod till that time? I have also read somewhere, that whenever ‘seek’ is called, the video is downloaded again. Is this true?
@Poojan Web browsers try not to download everything. They use HTTP byte range requests to download only the bit of the media resource that the user needs to play back the selected interval. Only if the Web server does not support HTTP byte range requests will the browser have to revert to full file downloads.
As for the question about seeking – that depends on the buffering strategy of the browser. Browsers will always retain a certain buffer of the media resource – on mobile devices it may be a lot less (maybe a few minutes of content), while on desktops with loads of disk storage it may be the full file. Then, when seeking is required, the browser knows which parts they are buffering and will only download byte ranges that it doesn’t already have.
Hey,
I had a question. I am working on a web application, videos automatically get converted and placed in postgrese. The web application for user end will be able to view a video clip and edit multiple start-end points on same clip then save those edit clippings back in postgrese. Everything is complete but i am unsure what technology to use for this. I been looking into dojo/JS…. any suggestions?
much thanks,
anthony
Hi Anthony, what JS library you use depends on your Web development preferences. I like to develop in Ruby on Rails with jquery. Then what JS library you add for doing the video is another decision to make. There are plenty of HTML5 player libraries available now. It all depends on your needs and preferences. Just do a google search for “html5 video javascript library”.
Hello,
i can jump to a fixed position in the video with video.currentTime = 20;
but how to jump after X seconds to another position? like: if(video.currentTime == 30){video.currentTime = 20;}
but this doesn’t work…
You will never catch currentTime at an exact position. Instead you should register an event handler for ontimeupdate and check when video.currentTime > 30.
Hi Silvia,
i’m not a developer …
do you think it’s possible to create a wordpress plugin with some of the features found on Metavid (annotation or trascription of video segment) or know someone working on this idea?.
@maurizio I don’t know anyone working on it. But I know that it’s possible to integrate metavid features into wordpress.
Hello,
i need a favor, I am working woth HATML5 with Video element. I am logging users actions while playing video. I observed that in firefox for html5 video element, when user starts seeking ,its event is fired after some time when seeked happen. Is this the real case or is that only with me. ?
Please provide suggestions.
Thanks.
Seeking will fire events immediately. Maybe you are not catching them all.
Hi,
I was reading over your post, and I was trying to do some of what you describe here. Namely, I have a video and I want to be able to start the video at a certain time and then have it pause after a certain duration of time has passed. I want to make it where a user can set the start time and duration the video will play in two boxes inside a form. I’ve followed some of what you described here and in your examples, but I’m still having trouble. I’ve used the timeupdate event like you suggested and can make it work how I want if I set my start time and duration with the following code:
video.addEventListener(‘loadedmetadata’, function() {
videoStartTime = 2;
durationTime = 4;
this.currentTime = videoStartTime;
}, false);
I can’t, however, get timeupdate to work and pause the video after a certain duration has passed when I set videoStartTime and durationTime onclick through the form. I have my code here: http://jsfiddle.net/chud/Rjtxw/3/ If you have a moment, I’d appreciate any help.
@Cole I’ve updated your JS and if you open the inspector, you will see what is happening with the console.logs. I’ve added parseInt() calls in your settheVariables function – that’s where the read values are received as strings and thus confuse your logic.
@silvia okay, that makes sense. I knew there was a break in the logic there, but I’m still not conversant enough with JS to fix it. Thank you for help. I’ve learned a lot just now.
currentTime or t# doesn’t work in mobile device but works in browser. Video always starts from beginning in mobile. I want to open video in specified time location in mobile device(ie. WinPhone, IPhone). Can you please help, I am badly stuck there.
Thanks in advance
@Arin You should register bugs with the specific browser vendors. E.g. on iPhone Safari, you should register a bug on WebKit.
As per my knowledge that the video element was first proposed by Opera software.Now it advanced more. current HTML5 draft specification does not specify which video formats browsers should support. User agents are free to support any video formats they feel are appropriate, but content authors cannot assume that any video will be accessible by all complying user agents, since user agents have no minimal set of video formats to support.
@Atanu content authors can make their videos universally accessible if they provide multiple formats in elements; e.g. WebM & MP4 will make them work in all desktop browsers and if the MP4 file was baseline, it should even work in all mobile browsers
Hello,
I need only video image on video frame with video timing. when i click on video will be go on another page then play . Any one can help me .