Video Conferencing in HTML5: WebRTC via Socket.io

code, Digital Media, open codecs, Open Source, standards, Videos, WebRTC, , ,

Six months ago I experimented with Web sockets for WebRTC and the early implementations of PeerConnection in Chrome. Last week I gave a presentation about WebRTC at Linux.conf.au, so it was time to update that codebase.

I decided to use socket.io for the signalling following the idea of Luc, which made the server code even smaller and reduced it to a mere reflector:

 var app = require('http').createServer().listen(1337);
 var io = require('socket.io').listen(app);

 io.sockets.on('connection', function(socket) {
         socket.on('message', function(message) {
         socket.broadcast.emit('message', message);
     });
 });

Then I turned to the client code. I was surprised to see the massive changes that PeerConnection has gone through. Check out my slide deck to see the different components that are now necessary to create a PeerConnection.

I was particularly surprised to see the SDP object now fully exposed to JavaScript and thus the ability to manipulate it directly rather than through some API. This allows Web developers to manipulate the type of session that they are asking the browsers to set up. I can imaging e.g. if they have support for a video codec in JavaScript that the browser does not provide built-in, they can add that codec to the set of choices to be offered to the peer. While it is flexible, I am concerned if this might create more problems than it solves. I guess we’ll have to wait and see.

I was also surprised by the need to use ICE, even though in my experiment I got away with an empty list of ICE servers – the ICE messages just got exchanged through the socket.io server. I am not sure whether this is a bug, but I was very happy about it because it meant I could run the whole demo on a completely separate network from the Internet.

The most exciting news since my talk is that Mozilla and Google have managed to get a PeerConnection working between Firefox and Chrome – this is the first cross-browser video conference call without a plugin! The code differences are minor.

Since the specification of the WebRTC API and of the MediaStream API are now official Working Drafts at the W3C, I expect other browsers will follow. I am also looking forward to the possibilities of:

The best places to learn about the latest possibilities of WebRTC are webrtc.org and the W3C WebRTC WG. code.google.com has open source code that continues to be updated to the latest released and interoperable features in browsers.

The video of my talk is in the process of being published. There is a MP4 version on the Linux Australia mirror server, but I expect it will be published properly soon. I will update the blog post when that happens.

65 thoughts on “Video Conferencing in HTML5: WebRTC via Socket.io”

  1. Hi,
    I am actually looking for video conferencing using HTML5. One to one is easy to implement using WebRTC. Is it possible to get atleast 1:N webcam video ( to simulate teaching on net ) ?
    1 –> teacher
    N –> Students who should be able to see teacher

    Cheers,
    Siddhartha

  2. I am trying to run your demo on an online node.js server. If I try to estabilish a videochat within my network it works, but when I try to videochat with a friend in another network I see only a black screen for the receiver. Did you try a similar scenario? I think that thare are some problems with the NAT.

  4. @Silvia should I only put the address of the ICE server in this?
    var pc_config = {
    “iceServers” : [“stun.l.google.com:19302”]
    };

  5. @Silvia I tried the apprtc.appspot.com with my friends using chromium and it doesn’t work; however it works in my local network.
    I tried to add an ICE server in your code but it doesn’t work for me (the ip address is from apprtc example)
    var pc_config = {
    “iceServers” : [{“url”:”stun:23.21.150.121″}]
    };

    I can see the ICE messages in the console and I can also see the address of my remote friend, but I can’t see the remote video

  6. @Luigi your friend must be using a modern version of Chrome, too (preferably a beta version). Try to get apprtc working first.

  7. I am able to successfully create webrtc video conference between two clients. But the remote videos on both the clients freeze after ~10 sec. Please suggest ways to overcome this problem.

  8. I am trying to implement one demo it is working properly on lan but when i am trying on iternet chat is working but video is not working it display black screen only plz tell me i am new one and it is argent. thank you in advance.

  9. With regards to:

    io.sockets.on(‘connection’, function(socket) {
    socket.on(‘message’, function(message) {
    socket.broadcast.emit(‘message’, message);
    });
    });

    Is it a privacy/security issue to blindly broadcast all new connection messages to all other clients? I’m not sure what data is exposed but it seems kinda dangerous to me?

    Also something minor but your indentation is quirky, prolly just a typo 🙂

    1. @John – sure is – that example was a quick and dirty hack to get the signalling working – I would not recommend using this in anger or in production!

  11. I am getting this error.

    Uncaught TypeError: Cannot call method ‘createOffer’ of null

    I believe its the sessiondescription of function setLocalAndSendMessage is causing the error.

    Please if you have any suggestions

  13. @ray you’re trying to call createOffer() with a null function – did you define setLocalAndSendMessage() after createOffer()?

  14. First i have placed the “setLocalAndSendMessage()”, then “errorCallBack()” and followed by “mediaconstraint”

    Only after that
    peerConn.createOffer(setLocalAndSendMessage,
    errorCallback,
    mediaConstraints);

    Am i missing something ?

  15. @ray s setLocalAndSendMessage(sessionDescription) setting the SDP for local and sending it to remote: peerConn.setLocalDescription(sessionDescription); ?
    Put a console.log into setLocalAndSendMessage on the sessionDescription so you can see what it contains.

  16. sessionDescription error
    console.log(“error check:” + peerConn.setLocalDescription.sessionDescription);

    Uncaught TypeError: Cannot read property ‘setLocalDescription’ of null

    And yes setLocalAndSendMessage(sessionDescription) is setting the SDP for both local and remote.

  18. @ray how much javascript programming have you done before? Your console.log should be inside the setLocalAndSendMessage function and should just print sessionDescription, not the function call. It also seems your peerConn object has not been set up yet. And yes, the first thing you need to do is opening the signalling channel.

  19. I have placed a logg once the channel is open.
    I get the createOffer error before the channel is open logg. Is there a way to first open the connection ?

    And thanks for all the replies. 😀 I know you have many other things to do. Much appreciated !

  22. hi!

    i was trying your code. i installed nodejs and i am able to run the server(reflector.js).how ever i could not figure it out how the client side can be implemented in nodejs. i want it to give html page.
    i was trying it on local host first.
    can you please tell me how to integrate the client side with the server?

  23. hi silvia
    thanks so much for your post ! This helped me a lot to set up my first working example with webRTC. It feels like this is now a good starting point to begin to implement own ideas.

    cheers

  25. HI silvia,
    Thanks a lot for your support. Your blog is very helpful.
    we successfully implemented your code and we are trying to learn by writing our own video conferencing application. I am new for java script and html. When we deploy your code(the one using socket.io) the video experienced a delay. Is it how it is expected to be or we have done something wrong in the deployment?
    Is there any drawbacks of signaling mechanism using web socket(socket.io) in comparison with the signaling mechanism using XHR and channel API?

  26. @hila Any delays in the video have nothing to do with the signaling mechanism, because the video will flow directly from browser to browser once the connection is set up. Thus, it depends on the direct connection between the two browsers and the available bandwidth and processing power of the end points. You can test that by setting up the connection and then taking down the signalling server – your video should continue streaming.

  27. hi
    i am new to webRTC. i read your post i got many things clear, but i could not get the source code. where i can get source code with proper instructions to run.
    cheers

  30. @silvia, please do i need to adjust any IP address to the address of the my local server or do i just run the code straight? Thanks very much……

    1. @ben The html page has an IP address of the signalling server. You need to adapt that to the IP address of the machine on which you run the signalling server.

  31. Hallo Silvia,

    am trying to impliment 3-party chat locally by slightly modifying your code.
    In doing so,I am able to see three videos on two tabs,whereas on the third one, only the local video; the other two being “black screen” . Seeing the console,I can only suspect that it might be a code-ordering problem if it should work this way.On the code,i simply added only a third video element, and include another event listening line for the second remote video :{
    peerConn.addEventListener(“addstream”, onRemoteStreamAdded1, false);
    peerConn.addEventListener(“addstream”, onRemoteStreamAdded2, false);
    peerConn.addEventListener(“removestream”, onRemoteStreamRemoved1, false);
    peerConn.addEventListener(“removestream”, onRemoteStreamRemoved2, false); }
    thanks a lot.

    1. @sola If you use addStream(), you can only add more video tracks and not audio tracks. So, you need to call getUserMedia with constraints that have audio set to false. Then you can multiplex the video tracks over one PeerConnection.

  32. @silvia, i adapted the IP of the signalling server to the IP address of the machine on which the server was running but I wasn’t able to get the code for the signalling to work beacuse I couldn’t locate yours. So maybe if you could please find time and take a look at this site and maybe assist me on how to correct my mistakes. Thanks very much for your time and patience. http://stackoverflow.com/questions/15361167/video-conferencing-in-html5-webrtc-via-socket-io

  33. @Silvia, or you could just send me the code for the signalling server to my mail or maybe direct me to where I can find it.(pierodizle@gmail.com). Thanks, you are the best…..

  34. @silvia, thanks . But, how is multiplexing of the video tracks possible in a peer connection ? need a media server ? or ? may be some code fragment for that ?
    Thanks a lot .

  35. @sola It’s simply an RTP connection and RTP allows multiplexing (see http://tools.ietf.org/html/rfc3550#section-5.2). The browsers take care of it. All you need to do is call addStream() for all the video streams as described above. E.g.

        peerConn.addStream(stream);
    if (videoStreams) {
      for (var i=0; i < videoStreams.length; i++) {
        console.log('Adding another local stream...' + videoStreams[i]);
        peerConn.addStream(videoStreams[i]);
      }
    }

  40. @silvia i want to make a p2p file sharing app using data channel api in webrtc is it possible?

  42. Dear Silvia,
    On your reply @sola you said “RTP allows multiplexing” does that mean when one participant in the room create offer (I creating a room) .will the offer be made to all the participants in the room automatically?
    Or I should extract the socket id of all the participants and send the offer to each of them in their respective socket id?(like in the case multi-peer video conferencing you mentioned in the document( webrtc.io) .they use web socket but I am doing using socket.io and your code.
    If it is necessary to have the socket id how can I get socket id of participants in the room?

    Thanks a lot!

  43. @silvia,

    I have been trying it editing the code as you have suggested, but need I edit in addition the signaling in some way (ref.js) ? or it should work for multi-party video conferene case too ? because the multi-party case is not working for me yet .

    Thanks a lot .

  44. @silvia and suhail, i tried running the code but when the remote computer clicks connect, i get this error message “local stream is not running yet” any idea on how to fix that please….thanks

  45. @silvia, there’s something else. When i click start video, i shows me that my webcam has been activated but it doesn’t reflect on the source video interface..such that the webcam is on but the source video interface is still blank.

  46. Hi,

    I tried to make a similar thing. My server is running, the connection is made, but the onRemoteStreamAdded function is never being triggered.
    Any idea what am I doing wrong?

    Thanks for the help,
    Eme

  49. Happy to find your blog. You might be interested in our new WebRTC real-time communications platform called Ondello. Working hard the last 6 months to get up and running. We will be at the upcoming WebRTC conference. http://www.Ondello.com
    Pamela Hadfield, Founder – Ondello

Comments are closed.