Preserving a dual screen experience for offline viewing

More and more programmes and live televised events are leveraging social media, specifically Twitter, to provide a second screen experience during the broadcast. It’s a powerful way for broadcasters to engage viewers with interaction and also to generate a sense of group participation in an event. Unfortunately this experience is lost if the viewer records the programme and watches it later. This post briefly outlines three suggested methods of recreating a form of second screen experience for offline viewing. The last of these requires no changes to viewer’s equipment and only minor changes at the broadcast end. This final method is developed in the second part of this post.

Methods to preserve a dual screen experience

Software in the player can retrieve historic tweets from Twitter on demand when playback is initiated and recreate the stream on the television or second screen device. This could be achieved independently of the broadcaster by TV and STB manufacturers. It’s also possible for a 3rd party developer to enable this functionality on devices capable of running apps, such as Google TV. By keeping the source of the tweets with Twitter this method would also avoid many of the editorial issues associated with user generated content as it is decoupled from the broadcaster.

The broadcaster can insert the Twitter stream into the MPEG metadata of the broadcast stream that a player can make available on the television or second screen device. This would require work by the broadcaster to make the streams available as well as cooperation with TV and STB manufacturers to decide on a common format.

The broadcaster can insert tweets into a subtitle track in the broadcast stream. This could be achieved using current live subtitling systems at the broadcast end and accessed on existing viewer equipment with no modification.

This third method is developed below to give an indication of the viewer experience.

Tweets embedded in the subtitle stream

Two experiments have been made using this technique with an episode of a soap opera (Eastenders) and a live sports event (the Queen’s Club tennis semi final). The Tweets were collected as each programme was being broadcast using a simple PHP script then written out to a subtitles file in .srt format using the timecodes provided from the Twitter feed. This was then used to insert a subtitle track into the original programme MPEG stream. For the purpose of this demonstration this was done offline, but in a production system this could be achieved using the same workstream as for live programme subtitling.

Observations

With the soap episode the number of tweets was low throughout the programme so there was no contention for the limited television screen real estate. However the delay between events on the screen and tweets relating to it appearing could be over a minute later. This was confusing as the expectation with subtitles is that they are immediate. This effect is exacerbated due to the short scenes lengths in Eastenders.

Tweets on rapidly changing content tend to be out of context

With the live sports event there was no shortage of material. In fact this was the biggest problem. A television doesn’t have the screen size or resolution to support such a large browsable list of content and still be viewable at a distance, so it has to be reduced to a manageable volume. This could be achieved by tighter message filtering, manual curation or simply dropping excess messages.

Tweets on live sports events are less sensitive to time delays

An example video can be seen below:

Other issues

There are a number of non-technical issues to consider:

Editorial: Does the broadcaster/programme maker want to associate themselves with the content of the messages? As with all user generated content it can be unpredictable if used raw.

Rights: Who owns the content of the messages? While it might be acceptable to associate a programme with a hashtag and encourage viewers to interact using it, blindly re-streaming it would probably be unacceptable to Twitter.

Legal: This is related to the rights issues – does inclusion of messages in the broadcast stream, by whatever means, constitute publishing and so assume the liability for the content of the messages?

These issues are not that much different from those encountered with other user generated content and so in most cases a curated stream would overcome these points.