ginger's thoughts

Silvia's blog

Tag: webrtc

The Making of Coviu

Cross-posted from LinkedIn.

The Making of Coviu

The creation of a company

As I prepare to step away from Coviu, I am being asked by several startup founders to share some insights and lessons learnt. I’ll reflect on the creation story of Coviu as we grew from a research project to a VC-invested health technology startup, to a profitable SME. Like any other business, Coviu has had its ups and downs. I hope you’ll find some helpful insights, particularly if you are spinning out of a research institution.

1. The beginnings of Coviu: research

Coviu was created from a research project started in 2012 at NICTA under Dr Terry Percival. At that time, we were exploring the use of WebRTC - a new video conferencing technology in the Web browser - for healthcare and government services use cases.

I was active in the W3C - the body that writes the standards for the web - in creating the specifications for video on the Web, including WebRTC. I built a local Sydney-based community around the early use of this technology and created one of the very first implementations of a working video conferencing call in a web browser. We were at the forefront of research in WebRTC at the time, and - boy - did we push the boundaries of technology, particularly in trying to be cross device compatible.

Terry raised a research grant for the project, and I was able to hire a small research team around the technology to explore industry use cases.

One of the areas that we explored was Telehealth: more specifically, we worked with Royal Far West School on a proof of concept for the delivery of speech teletherapy into schools in the rural areas of western NSW. They had used a clunky interface built with legacy software to support their telepractice services and were excited about the modern web interface that we were able to provide. The project was called “Sounds, Words, Aboriginal Language and Yarning” (SWAY) and can be found at https://sway.org.au/.

SWAY product in action SWAY product in action

We demonstrated this interface to several NICTA visitors, including healthcare specialists, surgeons from leading hospitals, primary care physicians, and even politicians. They all saw this technology’s potential to make Telehealth a universally accessible capability in Australia. So, our team decided to build out the demonstrator into a proper web application that could be commercialised by NICTA/CSIRO.

LEARNINGS:

The early days of a startup are often messy. You have an idea, you talk with a lot of people about it, you zero in on a specific use case, talk with potential users, build a demonstrator, and build an early team. It’s still pretty non-committal. Everybody is chipping in their spare time and free advice. We were lucky to be funded by a government grant to develop the first demonstrator and to be able to execute this work within the safe confines of a NICTA/CSIRO job so we could focus on product development and use cases. We made sure at this time to set everything up for a startup: particularly that there was no joint IP ownership with other institutions, as that would have caused a lot of headaches down the road.

2. Making it real: commercial value

The next step was to consider how to commercialise our ideas and technology. We needed a market that would buy what we built, we needed a product idea that would have commercial value, we needed a business model through which we could charge customers, and we needed a path to give CSIRO a commercial return for the work.

There were several Product/Market/Business Model combinations that we explored:

  • the creation of an Open Source project that would make money from support contracts. This actually progressed and became http://rtc.io with a GitHub repository that was at one stage used by several businesses and made some small consulting income for NICTA. However, the problem that we solved was mostly the cross-browser incompatibilities. These went away as the WebRTC technology matured, so this wasn’t really a scalable business.

Open Source WebRTC library Open Source WebRTC library

  • the creation of a PaaS (Platform as a service) where we would build the backend technology required to run WebRTC applications and offer an API for customers that would build their market-specific application on top. This is the Twilio API approach. We worked a bit in this space, but several well-funded startups had a head start, including TokBox, which later was acquired by Vonage. Several successful companies are now playing in this space, incl. daily.co, agora.io and jitsi.

  • the creation of a tech consulting business where we would help others build their WebRTC applications (using our own or somebody else’s PaaS). This is what WebRTC Ventures is doing. As a business out of Panama, they have access to affordable developers, which is what you need to make such a dev-focused business scalable. We could not compete.

  • the creation of a software platform that would allow us to offer a virtual service solution for several markets, which included healthcare, finance, government services etc. The idea was to customise and white-label the software as an enterprise solution for large organisations, a bit like what Stackoverflow does for online community platforms.

All of these product ideas targeted different markets, and we explored the size of almost all of these markets at NICTA and later at CSIRO’s Data61. We had a massive spreadsheet with detailed market size estimates that identified the different market’s TAM, SAM and SOM.

We also built demonstrators for all of these products and tested market adoption by interviewing potential customers. A basis was laid for commercialisation into several different companies. But NICTA/CSIRO could not support such a diverse market entry.

It became apparent very quickly that to sell into this many different markets with this many different products, we needed specialised tech configuration, sales and support staff for each one. Each was a different business that would need a leader. NICTA/CSIRO asked me eventually what I wanted to focus on and whether I would be willing to lead a spinout. A tough question when you’re a technology leader who’s just trying to create valuable IP for a research organisation.

I eventually decided to follow my heart with a focus on the most impactful market, which was dearest to me: the healthcare market. We left some of the other opportunities behind with CSIRO, but since there was no leader behind any of the other opportunities, the rest of the technology and opportunities eventually disappeared.

LEARNINGS:

The biggest learning here is that you have to make choices. Executing on all possible commercialisation opportunities for a new technology or idea isn’t feasible. Pick one, be ultra-focused and execute on that. You don’t have the bandwidth.

We spent several years going through the motions and different platforms to test the market, but ultimately, the interest and passion of the key leader will make the decision, so don’t waste time on covering all the possible opportunities (we were only able to do this because we were at CSIRO).

Interestingly, Coviu still has some generic language in its platform from back in those days when we experimented with using the platform for several markets - the impact from the early work you do has a long-lasting effect!

3. Getting ready for independence

In parallel to exploring all these different market opportunities, I worked on a way to spin the project out of NICTA/CSIRO into an independent company. We registered “Coviu Global Pty Ltd” as an Australian business in December 2015. This allowed us to set up a bank account through which we could collect payments for the self-service platform that we had created - something that NICTA/CSIRO wasn’t able to provide for - so we collected the revenue for NICTA/CSIRO in that separate bank account. The first payment came only in mid-2016, which felt like an eternity.

I was starting to talk to potential investors, particularly VCs, as I was hopeful to get at least 50% of the 12-person large project team into a startup. The VCs gave me the feedback that this was too big of a founding team for a company selling into an emerging market with very little revenue. As it turned out, most team members weren’t interested in taking a risk on a startup anyway.

I also needed to get the IP situation with CSIRO sorted - were we going to get a license to commercialise the IP from CSIRO, or was CSIRO willing to assign the IP in return for shareholding in the company? There were internal processes to follow and committee meetings to make submissions to. It took until March 2017 for an agreement to be signed - a huge thanks goes to Shelley Copsey, who had joined Data61 as Commercialisation manager in 2016 and was helping me get through this due diligence.

In parallel, we went through the CSIRO ON Accelerate program in 2016 with 4 team members: Nathan, Jeff, Georgie and myself.

ON Accelerate Team in 2016 ON Accelerate Team in 2016

During ON Accelerate, we built a great relationship with Phil from Main Sequence Ventures. Nathan gave the final presentation on pitch night and did an amazing job (I was on holiday in Germany)! We received some funding from CSIRO to continue preparing for spin-out. After that, I was on the rollercoaster of raising capital, while Data61 started reallocating team members to other groups as the government grant for the team was running out.

It would take until 2018 to actually close our Seed Investment round. At this time, only Nathan and I were still working on Coviu, and I was only part-time. Main Sequence Ventures saw the opportunity and invested Seed funding and in May 2018, Nathan and I were the first Coviu staff and needed to find ourselves new offices. At that time, we only had a small customer base and not enough recurring revenue coming in to cover expenses, so we really had to make the investment money work for us by keeping costs low and looking to get new customers at every opportunity.

Investment Party at Data61 in 2018 Investment Party at Data61 in 2018

LEARNINGS:

This was a tough journey - don’t underestimate how difficult it is to:

  • create a product that people are willing to pay for,
  • reach an agreement with a research institute about how to create a company jointly,
  • spin a team out of a research institute, and
  • raise your first external investment round.

I can honestly say that without the help of so many people within CSIRO and outside, it would not have happened. Thanks to you all!

I also took part in another five accelerators in addition to ON Accelerate to learn more about marketing and how to run and grow a business, just to be prepared for what would come next. All those learnings were important, but I only picked accelerators that wouldn’t ask for shares.

4. The startup is alive!!

Here we were, having taken the step into independence and facing the challenge of creating a company. Some key components to take care of: hiring, company culture, mission, product, marketing & sales, operations, and a new website.

Nathan and I actually started with ground rules for the kind of people we wanted to hire and the kind of culture we wanted to build. A couple of key components were that we wanted to hire self-motivated, competent team members with a drive for constant learning and improvement and getting stuff done. We wanted to remain scrappy and retain our entrepreneurial spirit to make a difference with little resources. And we wanted people to respect each other and care for the team.

Our mission was set from the start to be making healthcare more accessible for patients through digital technology and that’s still the foundation of the company and has helped attract the right kind of staff.

With Nathan in QLD and me in NSW, we were always going to build a company with a remote working culture, which suited us both. We hired a remote marketing manager and got to work setting up a marketing strategy. At the same time, Nathan continued working on improving the product and I looked after the operational and financial side of things. Nathan hired two junior software engineers, and we set up an office in Brisbane as the engineers worked better together in the same room, especially in the early days. The more customers we got, the more requirements on the reliability and functionality of our product were uncovered. Hiring sales and marketing staff was next on the list, so we set up in a startup workspace in Sydney also.

We went to conferences to expand our reach and noticed that Facebook marketing didn’t work for us but that we would see an increase in customers after every conference - a pattern that has continued to be true throughout Coviu’s lifetime. Even with small customers, direct conversations with leads is the way to close them if you’re starting to sell in Healthcare.

We also managed to win a couple of smaller and bigger government grants that were strategically placed to push our product forward in certain areas and helped mature us as a company. We began selling to smaller practices that would deliver services into rural and remote areas, mainly in Allied Health. In parallel, we did some projects with “enterprise” customers - i.e. larger, multi-location healthcare providers - that would expand our product to their specific needs. Always with a strategic view toward growing the platform’s capabilities and aligning with future customers’ needs.

We introduced an annual “offsite” for the company, which for a distributed company like Coviu really means it’s an on-site where staff would meet and work at the same place together once a year. These were great for staff morale, annual planning, and to build relationships. Our early offsites were held in Airbnb-rented houses with enough rooms for the whole team and large sitting rooms to hold our sessions. These offsites were always a lot of fun while also getting much work done.

First Coviu Offsite in December 2018 First Coviu Offsite in December 2018

The early days were a rollercoaster with lots of great and terrible days. I remember once taking a call with a customer who screamed at me for 30 minutes that our software wasn’t working for her with her rural patients and that we were making it impossible for her to make therapeutic progress. I couldn’t get a word in edgewise to solve her technical problems. But I could understand how much she cared about her patients and just wanted the best experience for them, and I felt her pain. We knew we had to do better and always retained that drive to continuously improve. After all, we’re here to improve people’s access to healthcare, and that’s as important as it gets.

LEARNINGS:

There are so many learnings from the early days!

  • Hiring is always a challenge and you will hire different people in the early days than later. In the beginning, you need all-rounders, people who are prepared to do what it takes, no matter their job description. You can hire specialists later.
  • Building a culture in a distributed company is hard, and it needs to be done with intention and repetition. You need to create rituals and have many more meetings than you would probably do when everybody is co-located. It also takes a certain maturity with staff to be able to be self-sufficient, get through problems and know when to ask for help, as nobody will observe your struggles at home. The offsites were so important, as were our weekly team meetings. And it helped that we had a Brisbane and a Sydney co-working space at the beginning.
  • Selling wasn’t easy in the early days, and all our staff needed to know the product inside out and how to do an elevator pitch as every individual clinician counted. Everybody had an impact and contributed to growing the company. Building out templates for customer presentations, standard customer contracts, standard pricing tables, demo materials, support materials, software documentation, test suites etc, were all important to move us forward.
  • Like many SaaS companies, we started by selling to the small businesses in our target market (i.e. private healthcare practices). The amount of functionality required to land an Enterprise customer takes years to develop. It includes regulatory compliance, a security posture and reporting that is hard to do in the early days. It wasn’t possible to jump at government tenders and the like immediately. We had to make our way there.
  • Government grants can be a great source of free investment money, but can also side-track you, so be careful what you bid for, as the reporting requirements and the project management on grants can be excessive. One of the grants that we won built us an amazing partnership that continues to carry us forward even today.
  • We also expanded our team size through interns, some of whom were amazing and some not so successful. We had one intern in particular who stayed with us for a long time and was a great contributor, so it can be very successful and rewarding.
  • Key was not to run out of money - the R&D Tax rebate was important and we had to make sure that we could pay our staff from SaaS recurring revenue and custom contracts that we agreed to undertake.

The early days were really just about not giving up. We knew we were impacting people’s lives, which carried us forward. We also knew that we could only build a profitable business if we achieved scale because individual practitioners weren’t paying much money, so we just had to continue pushing the barrel up the hill until it would get momentum.

5. Unexpected silver linings: success

The momentum came with the COVID-19 pandemic in March 2020 (and yes, it is a complete coincidence that our name is one letter different from the pandemic). It took the prior year’s work to be ready for that success.

We had built ourselves a trust network in the healthcare industry. We were connected with medical industry associations that called us on the day that the Medicare MBS telehealth items were made available to ask for discount codes for their members as they were giving them recommendations on which telehealth platforms were clinically focused and to be trusted from a privacy, data encryption and data sovereignty point of view. This is how the storm of phone calls, demos and self-signups on our website was started.

We also had a partnership with Healthdirect, the government organisation that - as one of their digital health projects - supplies public hospitals in several states and all GPs with a free telehealth platform, funded by state and federal health departments. Their updated Video Call platform - which was powered by Coviu - had launched in September 2019, and it was ready to scale for the needs of the pandemic.

Once the MBS items were announced, our Website page views grew by 10,000%, and we grew the number of daily consultations by 5,000% from 400 to 20,000 calls a day. Healthcare providers of all specialties called us at all hours of the day as they were preparing to turn away from in-person consultations to virtual consultations and had no idea how to do this.

The flood of enquiries meant we had to hire and onboard support staff quickly. We had to deal with up to 1,000 enquiries a day - over 20 people were hired in the course of 2 weeks and helped onboard each other as well as new customers. We had set up an online chat support interface, a phone support number, an email support address, demo webinars, and an online knowledge base before the pandemic and these were vital to scaling up our support. We introduced early and late shifts and covered the weekend. Our internal communications revolved around Slack, daily standups, and daily video calls. All these support mechanisms had been set up during the previous years and were vital to train the nation’s healthcare providers and consumers in telehealth and onboard our staff at scale.

We were also only able to convert the storm of new customers because we had set our Web application up with a complete self-service interface - most other telehealth startups had to manually sign up every single clinician, which was not scalable during a pandemic.

Our engineers worked overtime to keep the platform online, scale the demand on our infrastructure and fix bugs exposed through the high demand. We couldn’t work on integrations with EMRs or practice management software or develop other new features until the storm subsided and we had raised a Series A investment round in December 2020. This investment round finally allowed us to develop many of the features clinicians demanded to make telehealth a seamless experience as part of digitally supported workflows in their practice or clinic.

LEARNINGS:

  • If your business model depends on scale, be prepared to scale in all aspects of your business - in sign-ups, customer acquisition, customer support, and technology.
  • Building our brand exposure in our industry through marketing partnerships, press exposure and vendor partnerships was vital to getting recognised as a solution in times of need.
  • It sometimes takes extraordinary events to create the shift in the industry that is needed for your company to succeed - a bit of “luck” and the right timing are just as important as a good product, business model and go-to-market.
  • A team built through fire creates a unique, collaborative company culture, with everybody helping everybody else.

Company Offsite November 2020 Company Offsite November 2020

6. Business as usual: a real company

Once the initial storm of the pandemic subsided, we were able to focus on many things that allowed us to become a real company. It was a real learning to mature each of the following components:

  • Organisational design: develop an org chart, deploy people into departments, and support our cultural maturity through promotion processes and engagement surveys.
  • Productivity: create a product-driven organisation with sprints and regular releases to enable engineering to become the engine of the company and be faster at addressing our bugs and the needs of healthcare providers and patients.
  • Operations: introduction of a centralised business management system covering marketing, sales and customer support from within one unified environment, and introduction of a centralised system of record to store company knowledge such as a detailed staff handbook.
  • Financials: investment through closing series A round, understanding our SaaS metrics, and deeper insights into our business metrics.
  • Expansion: start selling to enterprise customers and find a new strategic growth opportunity in the US.
  • Compliance: HIPAA, ISO27001, TGA SaaMD and the creation of compliant policies and workflows, all of which are needed for enterprise customers.
  • Ecosystem: creation of an Apps marketplace and EMR integrations and APIs to allow us to integrate with other vendors in the healthcare market and make an outstanding customer experience.

This is done to better achieve our mission of making healthcare more accessible for patients through digital technology. It has led to a more mature business that can better meet customer needs.

LEARNINGS:

  • The challenges never stop. They just change. For example: Our sign-ups and usage tracked closely with the waves of COVID making it challenging to know when we were making progress as a company versus just being subject to the whims of the pandemic. Our goal had always been to build a strong, capable and data-driven business, but all our experiments kept being disrupted by COVID, and it was very hard to attribute success to anything but new outbreaks. Only now that things have calmed down can we run experiments that give us real insights into what makes telehealth work for our customers, the clinicians. We’re fortunate that our industry has changed completely and patients now demand telehealth services, so the industry and reimbursement paths are adapting.
  • Never take your eyes off your customers. We’ve realised that what was sufficient functionality-wise for a telehealth platform before the pandemic differs from what is expected post-pandemic. We always, always have an ear for the needs of our customers and continue to address their needs with new and reworked features. This will never stop in a software business.

7. A summary: Perseverance and kindness

Finally, I want to say a word to all the founders who are trying so hard: Be kind to yourself. As a founder, you will make a lot of mistakes, you will fall down and get up again and try again and learn. That’s the only way to improve. Don’t beat yourself up over mistakes - learn from them and move on. It’s a very ungrateful position because nobody will pat you on your back and tell you how well you’ve done. People will typically just demand the next goals to be reached, and that includes your own demands on yourself.

I cherished those moments when our customers told us how much they loved our platform. I cherished the annual 1:1s I did with each one of our staff members where we could talk about their lives at home and at work and what went well and what didn’t. I cherished the Sundays that I tried to keep free of work to spend with my family and relax - it was my way to look back at the previous week’s achievements and recharge for the next week.

Find a way to stand still, look at what you have achieved already, and then gather your focus and energy for the next step, one step at a time. Best of luck to you!

WebRTC predictions for 2016

I wrote these predictions in the first week of January and meant to publish them as encouragement to think about where WebRTC still needs some work. I’d like to be able to compare the state of WebRTC in the browser a year from now. Therefore, without further ado, here are my thoughts.

WebRTC Browser support

I’m quite optimistic when it comes to browser support for WebRTC. We have seen Edge bring in initial support last year and Apple looking to hire engineers to implement WebRTC. My prediction is that we will see the following developments in 2016:

  • Edge will become interoperable with Chrome and Firefox, i.e. it will publish VP8/VP9 and H.264/H.265 support
  • Firefox of course continues to support both VP8/VP9 and H.264/H.265
  • Chrome will follow the spec and implement H.264/H.265 support (to add to their already existing VP8/VP9 support)
  • Safari will enter the WebRTC space but only with H.264/H.265 support

Codec Observations

With Edge and Safari entering the WebRTC space, there will be a larger focus on H.264/H.265. It will help with creating interoperability between the browsers.

However, since there are so many flavours of H.264/H.265, I expect that when different browsers are used at different endpoints, we will get poor quality video calls because of having to negotiate a common denominator. Certainly, baseline will work interoperably, but better encoding quality and lower bandwidth will only be achieved if all endpoints use the same browser.

Thus, we will get to the funny situation where we buy ourselves interoperability at the cost of video quality and bandwidth. I’d call that a “degree of interoperability” and not the best possible outcome.

I’m going to go out on a limb and say that at this stage, Google is going to consider strongly to improve the case of VP8/VP9 by improving its bandwidth adaptability: I think they will buy themselves some SVC capability and make VP9 the best quality codec for live video conferencing. Thus, when Safari eventually follows the standard and also implements VP8/VP9 support, the interoperability win of H.264/H.265 will become only temporary overshadowed by a vastly better video quality when using VP9.

The Enterprise Boundary

Like all video conferencing technology, WebRTC is having a hard time dealing with the corporate boundary: firewalls and proxies get in the way of setting up video connections from within an enterprise to people outside.

The telco world has come up with the concept of SBCs (session border controller). SBCs come packed with functionality to deal with security, signalling protocol translation, Quality of Service policing, regulatory requirements, statistics, billing, and even media service like transcoding.

SBCs are a total overkill for a world where a large number of Web applications simply want to add a WebRTC feature - probably mostly to provide a video or audio customer support service, but it could be a live training session with call-in, or an interest group conference all.

We cannot install a custom SBC solution for every WebRTC service provider in every enterprise. That’s like saying we need a custom Web proxy for every Web server. It doesn’t scale.

Cloud services thrive on their ability to sell directly to an individual in an organisation on their credit card without that individual having to ask their IT department to put special rules in place. WebRTC will not make progress in the corporate environment unless this is fixed.

We need a solution that allows all WebRTC services to get through an enterprise firewall and enterprise proxy. I think the WebRTC standards have done pretty well with firewalls and connecting to a TURN server on port 443 will do the trick most of the time. But enterprise proxies are the next frontier.

What it takes is some kind of media packet forwarding service that sits on the firewall or in a proxy and allows WebRTC media packets through - maybe with some configuration that is necessary in the browsers or the Web app to add this service as another type of TURN server.

I don’t have a full understanding of the problems involved, but I think such a solution is vital before WebRTC can go mainstream. I expect that this year we will see some clever people coming up with a solution for this and a new type of product will be born and rolled out to enterprises around the world.

Summary

So these are my predictions. In summary, they address the key areas where I think WebRTC still has to make progress: interoperability between browsers, video quality at low bitrates, and the enterprise boundary. I’m really curious to see where we stand with these a year from now.

It’s worth mentioning Philipp Hancke’s tweet reply to my post:

https://datatracker.ietf.org/doc/draft-ietf-rtcweb-return/ … — we saw some clever people come up with a solution already. Now it needs to be implemented :-)

Progress with rtc.io

At the end of July, I gave a presentation about WebRTC and rtc.io at the WDCNZ Web Dev Conference in beautiful Wellington, NZ.

webrtc_talk

Putting that talk together reminded me about how far we have come in the last year both with the progress of WebRTC, its standards and browser implementations, as well as with our own small team at NICTA and our rtc.io WebRTC toolbox.

WDCNZ presentation page5

One of the most exciting opportunities is still under-exploited: the data channel. When I talked about the above slide and pointed out Bananabread, PeerCDN, Copay, PubNub and also later WebTorrent, that’s where I really started to get Web Developers excited about WebRTC. They can totally see the shift in paradigm to peer-to-peer applications away from the Server-based architecture of the current Web.

Many were also excited to learn more about rtc.io, our own npm nodules based approach to a JavaScript API for WebRTC.

rtcio_modules

We believe that the World of JavaScript has reached a critical stage where we can no longer code by copy-and-paste of JavaScript snippets from all over the Web universe. We need a more structured module reuse approach to JavaScript. Node with JavaScript on the back end really only motivated this development. However, we’ve needed it for a long time on the front end, too. One big library (jquery anyone?) that does everything that anyone could ever need on the front-end isn’t going to work any longer with the amount of functionality that we now expect Web applications to support. Just look at the insane growth of npm compared to other module collections:

Packages per day across popular platforms (Shamelessly copied from: nodejitsu.com)

For those that - like myself - found it difficult to understand how to tap into the sheer power of npm modules as a font end developer, simply use browserify. npm modules are prepared following the CommonJS module definition spec. Browserify works natively with that and “compiles” all the dependencies of a npm modules into a single bundle.js file that you can use on the front end through a script tag as you would in plain HTML. You can learn more about browserify and module definitions and how to use browserify.

For those of you not quite ready to dive in with browserify we have prepared prepared the rtc module, which exposes the most commonly used packages of rtc.io through an “RTC” object from a browserified JavaScript file. You can also directly download the JavaScript file from GitHub.

Using rtc.io rtc JS library Using rtc.io rtc JS library

So, I hope you enjoy rtc.io and I hope you enjoy my slides and large collection of interesting links inside the deck, and of course: enjoy WebRTC! Thanks to Damon, JEeff, Cathy, Pete and Nathan - you’re an awesome team!

On a side note, I was really excited to meet the author of browserify, James Halliday (@substack) at WDCNZ, whose talk on “building your own tools” seemed to take me back to the times where everything was done on the command-line. I think James is using Node and the Web in a way that would appeal to a Linux Kernel developer. Fascinating!!

AppRTC : Google's WebRTC test app and its parameters

If you’ve been interested in WebRTC and haven’t lived under a rock, you will know about Google’s open source testing application for WebRTC: AppRTC.

When you go to the site, a new video conferencing room is automatically created for you and you can share the provided URL with somebody else and thus connect (make sure you’re using Google Chrome, Opera or Mozilla Firefox).

We’ve been using this application forever to check whether any issues with our own WebRTC applications are due to network connectivity issues, firewall issues, or browser bugs, in which case AppRTC breaks down, too. Otherwise we’re pretty sure to have to dig deeper into our own code.

Now, AppRTC creates a pretty poor quality video conference, because the browsers use a 640x480 resolution by default. However, there are many query parameters that can be added to the AppRTC URL through which the connection can be manipulated.

Here are my favourite parameters:

  • hd=true : turns on high definition, ie. minWidth=1280,minHeight=720
  • stereo=true : turns on stereo audio
  • debug=loopback : connect to yourself (great to check your own firewalls)
  • tt=60 : by default, the channel is closed after 30min - this gives you 60 (max 1440)

For example, here’s how a stereo, HD loopback test would look like: https://apprtc.appspot.com/?r=82313387&hd=true&stereo=true&debug=loopback .

This is not the limit of the available parameter, though. Here are some others that you may find interesting for some more in-depth geekery:

  • ss=[stunserver] : in case you want to test a different STUN server to the default Google ones
  • ts=[turnserver] : in case you want to test a different TURN server to the default Google ones
  • tp=[password] : password for the TURN server
  • audio=true&video=false : audio-only call
  • audio=false : video-only call
  • audio=googEchoCancellation=false,googAutoGainControl=true : disable echo cancellation and enable gain control
  • audio=googNoiseReduction=true : enable noise reduction (more Google-specific parameters)
  • asc=ISAC/16000 : preferred audio send codec is ISAC at 16kHz (use on Android)
  • arc=opus/48000 : preferred audio receive codec is opus at 48kHz
  • dtls=false : disable datagram transport layer security
  • dscp=true : enable DSCP
  • ipv6=true : enable IPv6

AppRTC’s source code is available here. And here is the file with the parameters (in case you want to check if they have changed).

Have fun playing with the main and always up-to-date WebRTC application: AppRTC.

UPDATE 12 May 2014

AppRTC now also supports the following bitrate controls:

  • arbr=[bitrate] : set audio receive bitrate
  • asbr=[bitrate] : set audio send bitrate
  • vsbr=[bitrate] : set video receive bitrate
  • vrbr=[bitrate] : set video send bitrate

Example usage: https://apprtc.appspot.com/?r=&asbr=128&vsbr=4096&hd=true

Use deck.js as a remote presentation tool

deck.js is one of the new HTML5-based presentation tools. It’s simple to use, in particular for your basic, every-day presentation needs. You can also create more complex slides with animations etc. if you know your HTML and CSS.

Yesterday at linux.conf.au (LCA), I gave a presentation using deck.js. But I didn’t give it from the lectern in the room in Perth where LCA is being held - instead I gave it from the comfort of my home office at the other end of the country.

I used my laptop with in-built webcam and my Chrome browser to give this presentation. Beforehand, I had uploaded the presentation to a Web server and shared the link with the organiser of my speaker track, who was on site in Perth and had set up his laptop in the same fashion as myself. His screen was projecting the Chrome tab in which my slides were loaded and he had hooked up the audio output of his laptop to the room speaker system. His camera was pointed at the audience so I could see their reaction.

I loaded a slide master URL:
http://html5videoguide.net/presentations/lca_2014_webrtc/?master
and the room loaded the URL without query string:
http://html5videoguide.net/presentations/lca_2014_webrtc/.

Then I gave my talk exactly as I would if I was in the same room. Yes, it felt exactly as though I was there, including nervousness and audience feedback.

How did we do that? WebRTC (Web Real-time Communication) to the rescue, of course!

We used one of the modules of the rtc.io project called rtc-glue to add the video conferencing functionality and the slide navigation to deck.js. It was actually really really simple!

Here are the few things we added to deck.js to make it work:

  • Code added to index.html to make the video connection work:

    <meta name="rtc-signalhost" content="http://rtc.io/switchboard/">
    <meta name="rtc-room" content="lca2014">
    ...
    <video id="localV" rtc-capture="camera" muted></video>
    <video id="peerV" rtc-peer rtc-stream="localV"></video>
    ...
    <script src="glue.js"></script>
    <script>
    glue.config.iceServers = [{ url: 'stun:stun.l.google.com:19302' }];
    </script>

    The iceServers config is required to punch through firewalls - you may also need a TURN server. Note that you need a signalling server - in our case we used http://rtc.io/switchboard/, which runs the code from rtc-switchboard.

  • Added glue.js library to deck.js:

    Downloaded from https://raw.github.com/rtc-io/rtc-glue/master/dist/glue.js into the source directory of deck.js.

  • Code added to index.html to synchronize slide navigation:

    glue.events.once('connected', function(signaller) {
      if (location.search.slice(1) !== '') {
        $(document).bind('deck.change', function(evt, from, to) {
          signaller.send('/slide', {
            idx: to,
            sender: signaller.id
          });
        });
      }
      signaller.on('slide', function(data) {
        console.log('received notification to change to slide: ', data.idx);
        $.deck('go', data.idx);
      });
    });

    This simply registers a callback on the slide master end to send a slide position message to the room end, and a callback on the room end that initiates the slide navigation.

And that’s it!

You can find my slide deck on GitHub.

Feel free to write your own slides in this manner - I would love to have more users of this approach. It should also be fairly simple to extend this to share pointer positions, so you can actually use the mouse pointer to point to things on your slides remotely. Would love to hear your experiences!

Note that the slides are actually a talk about the rtc.io project, so if you want to find out more about these modules and what other things you can do, read the slide deck or watch the talk when it has been published by LCA.

Many thanks to Damon Oehlman for his help in getting this working.

BTW: somebody should really fix that print style sheet for deck.js - I’m only ever getting the one slide that is currently showing. ;-)

WebRTC books - a brief review

I just finished reading Rob Manson’s awesome book “Getting Started with WebRTC” and I can highly recommend it for any Web developer who is interested in WebRTC.

Rob explains very clearly how to create your first video, audio or data peer-connection using WebRTC in current Google Chrome or Firefox (I think it also now applies to Opera, though that wasn’t the case when his book was published). He makes available example code, so you can replicate it in your own Web application easily, including the setup of a signalling server. He also points out that you need a ICE (STUN/TURN) server to punch through firewalls and gives recommendations for what software is available, but stops short of explaining how to set them up.

Rob’s focus is very much on the features required in a typical Web application:

  • video calls
  • audio calls
  • text chats
  • file sharing

In fact, he provides the most in-depth demo of how to set up a good file sharing interface I have come across.

Rob then also extends his introduction to WebRTC to two key application areas: education and team communication. His recommendations are spot on and required reading for anyone developing applications in these spaces.

---

Before Rob’s book, I have also read Alan Johnson and Dan Burnett’s “WebRTC” book on APIs and RTCWEB protocols of the HTML5 Real-Time Web.

Alan and Dan’s book was written more than a year ago and explains that state of standardisation at that time. It’s probably a little out-dated now, but it still gives you good foundations on why some decisions were made the way they are and what are contentious issues (some of which still remain). If you really want to understand what happens behind the scenes when you call certain functions in the WebRTC APIs of browsers, then this is for you.

Alan and Dan’s book explains in more details than Rob’s book how IP addresses of communication partners are found, how firewall holepunching works, how sessions get negotiated, and how the standards process works. It’s probably less useful to a Web developer who just wants to implement video call functionality into their Web application, though if something goes wrong you may find yourself digging into the details of SDP, SRTP, DTLS, and other cryptic abbreviations of protocols that all need to work together to get a WebRTC call working.

---

Overall, both books are worthwhile and cover different aspects of WebRTC that you will stumble across if you are directly dealing with WebRTC code.

Video Conferencing in HTML5: WebRTC via Socket.io

Six months ago I experimented with Web sockets for WebRTC and the early implementations of PeerConnection in Chrome. Last week I gave a presentation about WebRTC at Linux.conf.au, so it was time to update that codebase.

I decided to use socket.io for the signalling following the idea of Luc, which made the server code even smaller and reduced it to a mere reflector:

 var app = require('http').createServer().listen(1337);
 var io = require('socket.io').listen(app);

 io.sockets.on('connection', function(socket) {
         socket.on('message', function(message) {
         socket.broadcast.emit('message', message);
     });
 });

Then I turned to the client code. I was surprised to see the massive changes that PeerConnection has gone through. Check out my slide deck to see the different components that are now necessary to create a PeerConnection.

I was particularly surprised to see the SDP object now fully exposed to JavaScript and thus the ability to manipulate it directly rather than through some API. This allows Web developers to manipulate the type of session that they are asking the browsers to set up. I can imaging e.g. if they have support for a video codec in JavaScript that the browser does not provide built-in, they can add that codec to the set of choices to be offered to the peer. While it is flexible, I am concerned if this might create more problems than it solves. I guess we’ll have to wait and see.

I was also surprised by the need to use ICE, even though in my experiment I got away with an empty list of ICE servers - the ICE messages just got exchanged through the socket.io server. I am not sure whether this is a bug, but I was very happy about it because it meant I could run the whole demo on a completely separate network from the Internet.

The most exciting news since my talk is that Mozilla and Google have managed to get a PeerConnection working between Firefox and Chrome - this is the first cross-browser video conference call without a plugin! The code differences are minor.

Since the specification of the WebRTC API and of the MediaStream API are now official Working Drafts at the W3C, I expect other browsers will follow. I am also looking forward to the possibilities of:

The best places to learn about the latest possibilities of WebRTC are webrtc.org and the W3C WebRTC WG. code.google.com has open source code that continues to be updated to the latest released and interoperable features in browsers.

The video of my talk is in the process of being published. There is a MP4 version on the Linux Australia mirror server, but I expect it will be published properly soon. I will update the blog post when that happens.