Category Archives: women in IT

WebVTT explained

Digital Media, standards, video accessibility, Videos, women in IT, , , , , , ,

On Wednesday, I gave a talk at Google about WebVTT, the Web Video Text Track file format that is under development at the WHATWG for solving time-aligned text challenges for video.

I started by explaining all the features that WebVTT supports for captions and subtitles, mentioned how WebVTT would be used for text audio descriptions and navigation/chapters, and explained how it is included into HTML5 markup, such that the browser provides some default rendering for these purposes. I also mentioned the metadata approach that allows any timed content to be included into cues.

The talk slides include a demo of how the <track> element works in the browser. I’ve actually used the Captionator polyfill for HTML5 to make this demo, which was developed by Chris Giffard and is available as open source from GitHub.

The talk was recorded and has been made available as a Google Tech talk with captions and also a separate version with extended audio descriptions.

The slides of the talk are also available (best to choose the black theme).

I’ve also created a full transcript of the described video.

Get the WebVTT specification from the WHATWG Website.

Happy Ada Lovelace Day to Women in Multimedia R&D

women in IT, , , ,

In my field of interest, namely Multimedia, there are not many women active in research and technology development. The more reasons to expose them and point to their great achievements.

In my time as a young researcher at the University of Mannheim, I met Prof Wendy Hall of the University of Southampton (GB), who was then the University

Open Standards for Sign Languages

Digital Media, Open Source, standards, video accessibility, women in IT, , , , , ,

Looking at accessibility for video includes sign language. It is a most fascinating area to get into and an area that still leaves a lot to formalise and standardise. A lot has happened in recent years and a lot still needs to be done.

Sign languages are different languages to spoken languages: they emerged in parallel to spoken languages in communities whose boundaries may not overlap with the boundaries of spoken languages. However, most developed means to translate spoken language artifacts (i.e. letters) into sign language artifacts (i.e. signs). So, a typical signer will speak/write at least 3-4 “languages”: the spoken language of their hearing peers, lip reading of that spoken language, letter signs of the spoken language, and finally the native sign language of the community they live in.

Encoding sign language in the computer is a real challenge. Firstly, there is the problem of enumerating all available languages. Then there is the challenge to find an alphabet to represent all “characters” that can be used in sign across many (preferably all) sign languages. Then there is the need to encode these characters in a way that computers can deal with. And finally, there is the need to find a screen representation of the characters. In this blog post, I want to describe the status for all of these.

Currently, sign language can only be represented as a video track by recording sign speakers. Once a sign character list together with an encoding and representation means for them and a specification of the different sign languages is available, it is possible to encode sign sentences in computer-readable form. Further, programs can be written that can present sign sentences on screen, that translate between different sign languages, and between sign and spoken languages. Also, avatars can be programmed that actually present animated sign sentences.

Imagine a computer that instead of presenting letters in your spoken language uses sign language characters and has keys with signs on them instead of letters. To a sign speaker this would be a lot more natural, since for most sign is their mother tongue.

Listing all existing sign languages
It was a challenge to create codes for all existing spoken languages – the current list of language codes has only been finalised in 1998.

Until the 1980s, scientists assumed that it is impossible to develop as rich a language with signs as with writing and speaking. Thus, the native languages of deaf people were often regarded as inferior to spoken languages. In many countries it was even prohibited to teach the language in schools for the deaf and instead they were taught to speak an oral language and read lips. In France this prohibition was only lifted in 1991! Only in about 1985 was it proven that sign languages are indeed as rich as spoken languages and deserve the right to be called a “language” and be treated as a fully capable means of communication.

So, there hasn’t actually been much time to map out a list of all sign languages. The best list I was able to find is in Wikipedia. It lists 28 N/S American, 38 European, 34 Asia-Pacific-AU/NZ, 30 African, and 13 Middle Eastern sign languages – in summary 143 sign languages. This list contains 177 sign languages.

Interestingly, there is also a new International Sign Language in development called Gestuno which is in use in international events (Olympics, conferences etc.) but has only a limited vocabulary.

In 1999 the Irish National Body, Deaf Action Committee for SignWriting, proposed the addition of sign language codes to ISO-639-2. Instead, a single code entered the list: sgn for sign language. In 2001, this led to the development of IETF language extension codes in RFC 3066 for 22 sign languages. In September 2006, this standard was replaced by RFC 4646, which defines 135 subtags for sign languages, including one for the International Sign Language and a generic “sgn” one.

While not complete, the current IANA subtag language registry now regards sign languages as valid derivatives of a country’s languages and therefore handles them identically to spoken languages. It’s also extensible such that any sign language not yet registered can still be specified.

Characters for sign languages
The written word is very powerful for preserving and sharing information. For a very long time there has been no written representation of sign languages. This is not surprising considering that there are still indigenous spoken languages that have no written representation. Also, the written representation of the spoken language around the community of a sign language would have served the sign community sufficiently for most purposes – except for the accurate capture of their thoughts and sign communications. It would always be a foreign language.

To move sign languages into the 20th century, the invention of characters for signs was necessary.

It is relatively easy to map the alphabets of spoken languages to signs (e.g. American (ASL) manual alphabet, British, Australian and NZ (AUSLAN) manual alphabet, or German manual finger alphabet, also see fingerspelling). Interesting the AUSLAN manual alphabet is a two-handed one while the ASL one is single-handed.

Fonts are available for these alphabets, too, e.g. British Sign Font, American Sign Font, French Sign Font and more.

The real challenge lies in capturing the proper signs deaf people use to communicate amongst themselves.

This is rather challenging, since sign languages uses the hands, head and body, with constantly changing movements and orientations for communication. Thus, while spoken language only has one dimension (sound) over time, sign languages have “three dimensions” and capturing this in characters is difficult. Many sign languages to this date don’t have a widely used written form, e.g. AUSLAN. Mostly in use nowadays are sequences of photos or videos – which of course cannot be computer processed easily.

Two main writing systems have been developed: the phonemic Stokoe notation and the iconic SignWriting.

Stokoe notation was created by William Stokoe for ASL in 1960, with Latin letters and numbers used for the shapes they have in fingerspelling, and iconic glyphs to transcribe the position, movement, and orientation of the hands. Adaptations were made to other sign languages to include further phonemes not found in ASL. Stokoe notation is written left-to-right on a page and can be typed with the proper font installed. It has a Unicode/ASCII mapping, but does not easily apply to other sign languages than ASL since it does not capture all possible signs. It has no representation for facial and body expressions and is therefore a relatively poor representation for sign.

SignWriting was created by Valerie Sutton in 1974, a dancer who had two years earlier developed DanceWriting and later developed MimeWriting, SportsWriting, and ScienceWriting. SignWriting is a writing system which uses visual symbols to represent the handshapes, movements, and facial expressions of sign languages. It is a generic sign alphabet with a list of symbols that can be used to write any sign language in the world.

SignWriting can be easily learnt by signers and is more popular now than Stokoe. Signers compose the symbols together in a spatial way to represent their signs. They then write the composed symbols from top to bottom on a page, similar to other iconic character sets. SignWriting currently supports 73 different sign languages, whose dictionaries and encyclopedias are captured in SignPuddle. This will eventually allow the creation of complete corpora for all sign languages.

Unicode encoding of SignWriting and visual representation
Because of its unique challenges of having to cover the spatial combination of symbols as a new symbol rather than just the sequential combination of symbols, it took a while to get a Unicode representation of SignWriting.

About a year ago, on 19th September 2008, Valerie Sutton released the International SignWriting Alphabet (ISWA 2008).

A binary representation of SignWriting is defined in ISWA 2008. It is based on a representing 639 base symbols and their potential 6 fill and 16 rotation variants in 61,343 code points, that completely cover the subset of 35023 valid symbol codes. The spatial aspect of SignWriting are encoded in a 2-dimensional coordinate system. The dimensions go from -1919 through 1919 to place the top left corner of the symbol.

SignWriting base symbols are encoded in plane 4 of Unicode, which provides 65,536 code points, easily covering the defined 61,343 Binary SignWriting code points. Further special control and number characters are used to encode the spatial layout.

Visual Representation of SignWriting
Valerie Sutton created over 35k individual PNG images for ISWA 2008, which have been reformatted for standard color & reduced file size, and renamed to the character code. They are a font used to represent the signs. The images can be accessed on Valerie’s server.

Closing
After learning all this today, I have to say that Valerie Sutton has just turned into a new idol of mine. The achievements with SignWriting and the possibilities it will enable are massive.

Now I just have to figure out what to do when we hit on a sign language track that has been encoded in SignWriting and it represents captions. Maybe it is possible to display sign as overlay but on the left side of the video. This would be similar to some other languages that go from top to bottom rather than left to right.

FOMS 2009: video introductions available

Digital Media, FOMS, LCA, open codecs, Open Source, video accessibility, women in IT,

In January this year we had the third Foundations of Open Media software workshop for developers. The focus this year was on legal issues around codecs, Xiph and Web video (HTML5 video and video servers), authoring/editing software, and accessibility. Check out the complete set of areas of concern and community goals that we decided upon.

As every year, at the beginning of the workshop every participant provided a 5 min introduction about their field of speciality and the current challenges. These are video recorded and shared with the community.

The videos and accompanying slides have been available for about 2 months now, but I haven’t gotten around to blogging about it – apologies everyone! So, here are your star videos in reverse alphabetic order published using open source video software only:

Enjoy!

Happy Ada Lovelace Day – Pia Waugh

women in IT,

The choice of a single great woman for this day was a really difficult decision. There are so many great women to adore and point out in open source and in other fields. And since Pia has just written her Ada Lovelace Day blog post about me, I half considered changing my mind.

My favorite women of past centuries are actually Marie Curie and Joan of Arc – one for her scientific achievements and the other for her courage. In current times I could have chosen Anuradha Suraparaju, together with me one of the few female software developers for open media software (leave me a comment if I missed you!!). I could also have chosen Stormy Peters who I adore for her work in Gnome, Mitchell Baker for her work in Mozilla, or Akkana Peck who is an awesome Firefox and Gimp developer. But weeks ago I had made up my mind to write about Pia, one of my best friends and an open source advocate with endless energy.

Pia Waugh

I can’t remember the day that I met Pia first, but I remember when she made the first impression on me. She came seemingly out of nowhere and decided to stand for president of Linux Australia. At the time, Linux Australia was a pretty useless organisation, having little more than a bank account through which the annual Linux.conf.au was organised. All of a sudden, with Pia spearheading Linux Australia, the organisation received a face, a goal and a purpose. I watched in amazement as LA became more and more representative of the *nix developers and enthousiasts in Australia, while AUUG took the opposite development.

And Pia also knew when to stop standing for LA – the moment that the organisation was threatened by just becoming “Pia’s organisation”, Pia decided to stand back and leave it to others to carry on her work. All they had to do was to stand on the back of a giant (well, almost).

Yes, I regard Pia as a giant – even if she’s actually really small in bodysize. 🙂 But we all know that short people can be very energetic. And Pia is one of the most energetic people that I know. She has a talent to express the opportunities and issues in open source with powerful words and has influenced many people to change their minds about open source. She has the energy to run her consultancy together with her husband Jeff, while at the same time being president of Software Freedom International which organises Software Freedom Day internationally, being active in many other groups, and particularly being a driving and coordinating force for OLPC work in Australia the region.

While driving open source and OLPC Pia has also a huge aim in getting more girls interested in technology, since she is enjoying it so much. She does many events at schools and has inspired many a girl to try out computing. Part of this is to expose great women in computing more such that they can work as role models to the younger generation. Her “Heroes” presentation is an awesome example for her efforts. Pia is my hero for the day. 🙂

“IT’s a mad men’s world”

women in IT

Last night I took part in a panel that was organised by Rachel Slattery under the title of “IT’s a mad men’s world.”. There were a whole lot of really fascinating women there, both in the audience and the panel, but also some stray men, which was good to see. With me on the panel were Sue Klose, Corporate Development Director of News Digital Media; Juliet Potter, Founder & Director of www.autochic.com.au; and Tim Batten, Head of eChannels & Payments at Westpac. The panel was moderated by Sandra Davey, Director of kcollective.

The discussion was really awesome and the stories that each of us could tell of situations where we had to stand up for ourselves just for being a woman were shocking. But what really got me was the universal message that we all had: don’t let the morons get you down and let go of your goals. Fight the fights that are worth it. It’s OK if not everybody loves you – you have to ask yourself: do you want to be liked or respected?

I actually did a little research before the event and wasn’t able to share half of the things I learnt. So I thought I’d put some more in this blog post.

I came across this xkcd cartoon just yesterday and thought: wow, this really is the essence of the problem why we have so few women in IT.

xkcd: how it works

You may be thinking that this cartoon represents a problem with male (or indeed societal) prejudices against women. I actually think the problem is deeper.

Imagine you’re a girl and have to decide on a career. You’re pretty good at many things and could be going into a technical career. But you have little experience since you’ve had little exposure and no mentors in the field before. Would you take the chance to expose yourself to looking really dumb, possibly even failing? Not just are you taking the hard road for yourself if you do. But there’s the larger impact on the perception of women. By looking dumb or failing, you will shed a bad light on all women and thus confirm the prejudice, making it even harder for other women to go into the field. Now do you start to understand why there are so few and each year even less women in technical jobs?

You think I’m taking this too far? Don’t. Women are being taught from very early on to not just think about themselves, but to be cooperative and always consider their environment. While such thoughts might not be consciously taken, they are there and play a role.

What do I really want to say with this? It’s not just a matter of changing men and indeed societal attitudes towards women. It’s also a matter of building up women’s self-confidence, teaching women how to be competitive and independent. And you have to start at school with encouraging and introducing women into IT. Because really: “Computing is too important to be left to the men” (quote from Karen Sparck-Jones).

UPDATE: I have heard from several men that they find that quote rather offensive and read it as in “we should not trust men with computing”. That is absolutely not the way I read it. I want it to be read as an encouragement to women to go into computing – it is an important field for the future of humanity and half of humanity is not taking part in shaping it. That’s just not right.

Further reading material: