Video Conferencing in HTML5: WebRTC via Web Sockets

code, Digital Media, open codecs, Open Source, standards, Videos, ,

A bit over a week ago I gave a presentation at Web Directions Code 2012 in Melbourne. Maxine and John asked me to speak about something related to HTML5 video, so I went for the new shiny: WebRTC – real-time communication in the browser.

Presentation slides

I only had 20 min, so I had to make it tight. I wanted to show off video conferencing without special plugins in Google Chrome in just a few lines of code, as is the promise of WebRTC. To a large extent, I achieved this. But I made some interesting discoveries along the way. Demos are in the slide deck.

UPDATE: Opera 12 has been released with WebRTC support.

Housekeeping: if you want to replicate what I have done, you need to install a Google Chrome Web Browser 19+. Then make sure you go to chrome://flags and activate the MediaStream and PeerConnection experiment(s). Restart your browser and now you can experiment with this feature. Big warning up-front: it’s not production-ready, since there are still changes happening to the spec and there is no compatible implementation by another browser yet.

Here is a brief summary of the steps involved to set up video conferencing in your browser:

  1. Set up a video element each for the local and the remote video stream.
  2. Grab the local camera and stream it to the first video element.
  3. (*) Establish a connection to another person running the same Web page.
  4. Send the local camera stream on that peer connection.
  5. Accept the remote camera stream into the second video element.

Now, the most difficult part of all of this – believe it or not – is the signalling part that is required to build the peer connection (marked with (*)). Initially I wanted to run completely without a server and just enter the remote’s IP address to establish the connection. This is, however, not a functionality that the PeerConnection object provides [might this be something to add to the spec?].

So, you need a server known to both parties that can provide for the handshake to set up the connection. All the examples that I have seen, such as https://apprtc.appspot.com/, use a channel management server on Google’s appengine. I wanted it all working with HTML5 technology, so I decided to use a Web Socket server instead.

I implemented my Web Socket server using node.js (code of websocket server). The video conferencing demo is in the slide deck in an iframe – you can also use the stand-alone html page. Works like a treat.

While it is still using Google’s STUN server to get through NAT, the messaging for setting up the connection is running completely through the Web Socket server. The messages that get exchanged are plain SDP message packets with a session ID. There are OFFER, ANSWER, and OK packets exchanged for each streaming direction. You can see some of it in the below image:

WebRTC demo

I’m not running a public WebSocket server, so you won’t be able to see this part of the presentation working. But the local loopback video should work.

At the conference, it all went without a hitch (while the wireless played along). I believe you have to host the WebSocket server on the same machine as the Web page, otherwise it won’t work for security reasons.

A whole new world of opportunities lies out there when we get the ability to set up video conferencing on every Web page – scary and exciting at the same time!

85 thoughts on “Video Conferencing in HTML5: WebRTC via Web Sockets”

  4. Yes, I’m loving WebRTC!

    I’ve being playing with image capture, and launched a really basic site: http://gotcaption.com/

    Though the WebRTC only works in Chrome Canary and Opera 12. Other browsers will fall back to flash.

    Thanks for sharing.

  6. @sime – nice one. Amazing what you can do with local video capture alone. PeerConnection is a whole different beast again.

  7. I didn’t understand how to do live demo. can any one provide steps how to start

  8. @Babu Do what I explained in the “Housekeeping” section and then open the link to the presentation. It has the demo in it.

  10. can you upload a demo?
    I cannot run your code successfully,
    how can two client connect to each other?

  11. @silvia
    I am having problem with the audio one. If I use getUserMedia to capture audio only, I will get a error “GET blob:http%3A//localhost/a5077b7e-097a-4281-b444-8c1d3e327eb4 404 (Not Found)”, have you ever try this?
    If you capture with camera, will you get the audio input?
    Also, I used a web socket server, but when I try to connect another client, I will get an error with the SDP message, ” ‘errorType’ : ‘NOMATCH’, ‘messageType’ : ‘ERROR’ “, and then there would come a send()error saying that the connection is closed, do you have this error? how can I fix it? what type of the message that a websocket can send?

  12. @Amy I’ve not tried audio alone. Make sure you fix the code to include audio if you do. I’ve also not had that SDP message, sorry.

  13. @silvia
    so when you use this code:
    navigator.webkitGetUserMedia({video:true, audio:ture},gotStream,gotError)
    you will get also the audio part right?
    I’m asking this because someone said that audio part is still not available in Chrome.
    About the websocket server, you just use the sample code you post above right?

  14. @Amy provided you use the right Chrome version and you don’t mis-type “audio:true”, you should. I’ve seen it work with some of the other demos.

    And yes, the code I posted is the one I used. You have to change the IP address of the server that you’re running it on, though.

  15. @silvia
    I use the latest beta version of chrome and I am sure I didn’t mistype. I changed the IP address, but it still does not working, I just cannot figure it out, could you show me other demos that you’ve seen?
    Thank you soooooooooo much!!

  16. @silvia
    I can’t tell whether their audio input works or not from the demos you gave.
    maybe the audio input is indeed still not available in chrome.
    thank you for being so kind and patient~~~

  19. @silvia
    Me again, lol
    Now I can successfully establish the connection, but I am wondering that whether I can choose who to connect. That is if I got a list of online clients’ IP addresses, how can I establish a specific connection? Do you know how to realize this?

  20. @Amy you need to write a Web app around that which manages your contacts etc. This is not an application – it’s a toolbox.

  21. How do I access sample data from the LocalMediaStream in Chrome? It doesn’t seem like they have the MediaStreamRecorder bits implemented the way they’re stated in the spec..

    1. @Thomas that’s possible – it’s all experimental at this stage and continues to change. You might need to inspect the Chromium source code to find out.

  22. Silvia,

    I have worked for Service Providers (Verizon, Windstream, etc.) and have worked with multiple Video Conferencing companies (Polycom, Tandberg, Cisco, Radvision, etc.) and I believe WebRTC is going to displace the hardware these companies are offering.

    I want to learn more about HTML5 and WebRTC but I am concerned your book may be too advanced for me as I do not have a programming background.

    How can I learn more and understand the basics behind HTML5 and WebRTC programming? (If I need to go back to HTML that is fine, any recommendations you have, and how I can learn how to setup a websocket server – Do I need a separate computer? – Sorry I am networking guy looking to replace the network with software)

    Thank you so much in advance,

    Eric

    1. @Eric You may want to start learning HTML, CSS and Javascript first. My book does start from scratch, so it may be helpful (well, the first half that is still accurate). But often times the best way to learn is from online websites and by just programming yourself.

  23. Hi Silvia,

    I am a beginner to use websocket. I tried to install websocket via command npm install websocket, but I met error about make problem in Mac OS X and C++ compiler in Windows. Which kind of websocket do you use? Thanks.

  24. Hi Silvia,
    In Mac it caused by not installing command line tool in Xcode.

    I have another question that I used websocket implementation in this link https://github.com/Worlize/WebSocket-Node . I used your code of websocket server. But I cannot see the remote video.
    And I tried to use None for STUN/TURN server in below line pc = new webkitPeerConnection(“NONE”, onSignal); I think this might not the cause for remote video don’t display.

    Would you please give some suggestion?

    Thanks very much.

  26. Matthew,

    You will have to change the server IP address on which you’re running the websocket server in the html file. Think through exactly what is happening when you run the websocket server – what ports it opens and listening on and where the html file’s PeerConnection call needs to connect to. Also, I don’t think “NONE” will work – you should definitely first try a STUN/TURN server before trying to use “NONE” or “”, since I’m not sure these values are supported in that implementation.

  28. Hi Sylvia,

    Thanks for the great post, I am feeling much closer to getting it now 🙂
    I think I am close, I took your server code on my own machine and got 2 PCs behind NAT on the same LAN to connect to my server (public IP@) but I have an error:

    send() error: Connection closed
    send() error: Connection closed
    Tue Jul 24 2012 16:43:58 GMT+0200 (CEST) Received Message SDP
    {
    “answererSessionId” : “JV0i1Y6mbVfwHSwCkO0p28VFW2mCPS3O”,
    “errorType” : “NOMATCH”,
    “messageType” : “ERROR”,
    “offererSessionId” : “Tl3x3arJnnOtZPHIP+l4a+Fd/2XSdLRz”,
    “seq” : 1
    }

    Any idea where it could be coming from? Feeling like an SDP mismatch, both browsers are running the same Chrome version (20.0.11) with MediaStream and PeerConnection enabled..

    Thanks a lot !

  29. HTML 5 will bring a lot of changes to the web, however, the real concern will be the web browsers, which often struggle to stay up to date with HTML 4. Video conferencing will be a big winner and over time, should help to bring costs down considerably.

  30. i am trying to use the apprtc.appspot.com on linux chrome the page turns the camera on but i am not seeing any video instead it shows a window full of green color. is any body seeing this ? i am using chrome 21 beta on linux ubuntu 11.10.
    pls advise,

  31. Hi Sylvia,

    I have a remote node.js server that is running your code verbatim (except for the WebSocket argument on the client side).
    I am connecting two different PCs, both using Chrome Canary.
    Looking at the console, it seems like PC #1 is getting useful data over the socket, but PC #2 gets:
    RECEIVED: SDP
    {
    “answererSessionId” : “oAiF7saWFAj3RIRK8w700NIbQ69vHsJ2”,
    “errorType” : “NOMATCH”,
    “messageType” : “ERROR”,
    “offererSessionId” : “41b8B0ezWOBQxX74MBcpKw/AFq6sX46E”,
    “seq” : 1
    }

    Does it matter that the PCs are on the same LAN? Any ideas?

    Thanks!
    -Brett

    1. @Brett I’ve had problems with firewalls, too. Sometimes they won’t let two computers on the same network connect to each other. Strange though that one would get data, but not the other.

  32. When i try to establish the call, I get the following error message. Any hint to fix it?

    {
    “answererSessionId” : “8q/0Zu8D0lfKJnP0WLW5dkOAo79I4ZZM”,
    “errorType” : “REFUSED”,
    “messageType” : “ERROR”,
    “offererSessionId” : “H1tz0o4s63+veRfL6fJcgjjLZ9RXO61y”,
    “seq” : 2
    }

    Thanks.

  33. Hi Silvia I am not able to use code of web socket server where to put IP address of my server server?? I can get local video but not remote video .

  34. Thankx Silvia will try But I want to add it to moodle learning environment can I do it?

  35. ON AUGUST 1ST, 2012 AT 12:53 AM – I brought up a question with SDP error.

    It happens only if I am using

    peerConn = new webkitDeprecatedPeerConnection(“STUN stun.l.google.com:19302”, onSignal);

    But with

    peerConn = new webkitPeerConnection00(“STUN stun.l.google.com:19302”, onSignal);

    The sdp error doesn’t occur, but still the call is not established.

    Did any of the followers experienced the same?

    Thanks

  36. @Ramsundar Kandasamy

    can you send me your web socket server script and the webpage and tell me how to implement it? Thankx in advance

  37. @ Anil

    You can get webSocketServer code and the webpage from the link provided by Silvia.

    After that in the webpage’s script section change the
    var socket = new WebSocket(‘ws://164.48.135.137:1337/’);
    line to point to your host where your webSocketServer is running.

    In theory then it should run.

    But I face some other problem as mentioned before. I tried running the webSocketServer and webpage on the same host, on different hosts. But the call is not established. Debugging to figure out where the problem is.

  39. With

    peerConn = new webkitPeerConnection00(“STUN stun.l.google.com:19302″, onSignal);

    the onSignal callback is not entered, which makes clear that the peerConn object is not created.

    I don’t know where is the problem.

  40. @Bret

    PC_1 receives correct SDP but PC_2 receives the following.

    Did you solve the firewall problem?

    {
    “answererSessionId” : “xxxx″,
    “errorType” : “NOMATCH”,
    “messageType” : “ERROR”,
    “offererSessionId” : “xxxx”,
    “seq” : 1
    }

  41. @ Ramsundar Kandasamy Thankx changed IP How to run webSocketServer code? Should I Use Nodejs server? Node websocket-server.js ??
    Thankx in advance

  42. @ Silvia

    Is there any sequence of steps one should follow?

    Or
    1. we just “start camera” on both clients.
    2. then click “connect” on both of them (without any order)

    Thanks

  43. Ok.

    I know what i did wrong.

    I was always “connect”ing from both end points. (in the way how one joins a conference).

    🙂

  45. @Silvia,

    Interesting!!! It works if I “connect” from one side only.

    Thanks.

  46. @ramsundar Kandasamy Please mail me the complete procedure with the code you have utilized I am unable to get it working

  49. Hello Silvia,

    You are brilliant! Thank you for sharing this. I have used your example and adapted it to my web app server and I have updated to use webkitPeerConnection00. I have found though that the connection.on method in websocket-server is not executing for some reason. I will update if I figure out the issue. If you have any ideas and want to share, it would be most appreciated. Again thank you very much!

  50. Now is not webkitDeprecatedPeerConnection support,Support only webkitPeerConnection00.
    Sent handshake information and this is not the same,Do you have about webkitPeerConnection00 API?

Comments are closed.