24/09/2013

Two things I remember about the "Second Screen" and "Big Data" sessions in IBC this year and I think they are connected. The multiplicity of devices around us has changed somehow the way we watch television (I don't have a television). For some it is an improvement, it is the open door to multi-platform (TV + online presence on various networks) more than cross-media and surely a challenge to monitor the viewer attention. In other words what is doing the audience when watching television and where do I put my advertising?

England seems to be a giant laboratory where everyone can be observed, his behavior analyzed and quantified (they are statistically big twitter users). The amount a of generated data is enormous - we talk about big data - and the risk to be overloaded is real. This actually the case and I heard during the session from the panel discussion speakers that data scientists are needed (good for us).

An interesting talk from Twitter UK what to illustrate how the live audience reaction can be used to add information to a TV show. Example of the last US presidential debate was to say: six channels (not sure) were broadcasting exactly the same video stream, after days one was getting most of the viewers attention. Why? Explanation was this TV channel (it was Fox I believe) was able to analyze the tweets live during the debate (using the twitter API everyone can access all tweets) and to provide a global audience reaction to it, so nothing like "this candidate sucks" or "I like him" was appearing on screen but a simple feedback yes the candidate is answering the question or no he is not (not exactly that but not too far). You do need to have data scientists, people doing social graph analysis to retrieve such information. And there are companies offering this service to TV channels, doing interactive programming (I think they call it like that) and able to process the multiple streams of information coming from the audience, be it a tweet, sms, email, FB message... And if you know where your audience is then you are able to monetize this information.

Usability, it's nice to have many possibilities to react, send feedbacks, but they are so many options that it is difficult to drag most of the viewers attention or at least the group you have targeted without losing half of them on the way. I explain, if you are on your sofa, you don't want to follow a specific procedure, fill a form, touch your tablet screen with 3 fingers, flip the tablet in the air to be able to "interact" with the program to access something. In that sens someone from Shazam gave a great talk. He made a simple experiment to illustrate his point by asking us: who had the app on his smart-phone and what do you do to use it? You press the button to start the app and raise you phone toward the speaker. Their idea was to use this known behavior to communicate with the TV audience: you are watching a program, the shazam logo appear on screen, you raise you arm with the app on, some kind of audio qr-code are activated and you have access to new content on you tablet or smart-phone, brilliant (it was for tv program RedBull if I'm correct).

It was almost on all lips during IBC this year, you have to improve the immersive experience for the viewers. Not only the story or TV program have to be original and addictive and if possible an endless source of money generator, no, they also have to be more immersive.

But what is an immersive experience? Different people, different interpretations. The idea is to offer to a spectator - while he is at the cinema or in front of its tv - an experience the more realistic it can be, the viewer should forget he is watching a display (e.g. watching a football game on your sofa and feeling you are in the stadium). The question becomes how to improve the display to make it more immersive? That's what the conference speakers (#ibcconf hosted during the show #ibcshow early September in Amsterdam) were trying to answer in their presentation. And especially in the sessions "Cutting Edge 1 & 2" new technologies were discussed. I took part in session 2 and we are all presenting what are the things comings, which technology will be predominant in a near future and hopefully the one we propose will be.

Starting from a rectangle display with a fixed resolution (e.g. HD), a fixed frame per second rate (e.g. 25fps), a fixed color dynamic (e.g. each color channel pixel coded on 8bits) what do we do? How do we introduce new "things" without destroying completely the existing workflow (from image acquisition, encoding and compression, diffusion and display)?

Increasing the resolution to 4K was written everywhere this year, increasing the resolution almost 4 times the one corresponding to HD (which is the one for Blue-Ray, DVD is way smaller), but so far there is not real content or affordable customer product for home. Professional movie cameras (or the latest GoPro) can offer this resolution, but only movie theater could follow on that (also I don't remember what is the offcial/recommended resolution for film distribution). Comparing to some years ago, the switch to full digital is a reality in many movie theaters and even so the quality in full digital is said to be lower than with film, two things are interesting: there is limit for an average viewer to see/perceive a difference in quality above 2K (so do we need to go so high?) and depending where you sit in a theater you will perceive different resolutions. In case you don't know this already, the human eye is performing filtering on the signal he receives, details will be perceived in an image depending of the viewing distance (this is use in many image compression algorithm where you remove what is not seen anyway).

Researchers at BBC where showing their latest work on fps, which fps is achievable in order to increase the viewing comfort. They said that above 100fps it's better, for you to appreciate the difference they were showing the same sequence (athletes doing high jump) with "normal" fps and higher fps side by side.

About hdr, the display market is almost reduces to one company (Dolby). The technique for taking hdr images (high dynamic range images) is known, basically you have to combine several pictures taken at different exposures. One can see the issues when you want to record movie file and not a single image. Researchers at Fraunhofer ISS proposed a very neat solution where sensors with different sensitivities pseudo-randomly distributed where used. And taking into account sparsity they could in one single shot obtain an hdr image. A brilliant friend did suggest me the following "but why don't they read continuously the captors to get the values at different exposures and create the hdr data on the fly", hmm why? And hdr must not be misunderstood with RAW file format despite data in that format possibly available in higher that 8bits per channel.

And there is our approach which consists in surrounding physically the viewers: the screen is curved (e.g. cylinder or digital dome to spherical display). To do so we combine several projectors to project images on the whole curved surface. This year we proposed a study case where 3D stereoscopic content was streamed to a digital dome. The installation we described use two fish-eye cameras and a 2 diameters dome tilted at 90degres such that while standing you are looking straight in the middle of the curved screen (the field of view proposed is bigger than your natural field of view, you should feel immersed). 3D and immersive experience go together and research is actively done to make the 3D experience more seamless to the user, meaning no glasses to wear, auto-stereoscopy and multi-views. Here as well the colleagues at Fraunhofer HHI and ISS have a strong presence. The matter of 3D and immersive display (or surround cinema) is tricky, the whole 3D thing is tricky (which makes it very interesting), because looking with one eye in real life doesn't make you see a flat image, you still perceive depth. And coming back to immersive display/surround cinema, the way you generate 3D images has to take into account where the viewer may look at, the freedom you are offering (possibility to look in every direction) is a challenge for the generation of proper stereoscopic effect. And I probably lost 98% of my friends after these lines.

All these large, curved, multi-blended displays need to be accurately color calibrated on the top of their geometrical calibration (we also do that at Fraunhofer FOKUS in our department VISCOM). The viewers need to have the feeling to look at one single light source only. The more you forget about the technique during the show you are watching/attending, the more immersive is the experience.

Last but not least, sound is of course in the game. You can't talk about surround cinema without surround sound. That wasn't too much mentioned in the sessions, but both image and sound are working together, beautiful isn't it... That is not my specialty, mine is color and image processing and more, I let the sound to my colleague specialists.

12/09/2013

I can see a little improvement over the years coming back to Amsterdam, after almost 8 times I get less lost which is always nice. But to look like a local demands some practical organization, you don't want to not give the impression you are a tourist (by staying in front of the Van Gog museum or asking where is the red light district or example) or one the numerous IBC visitors (they are easy to spot because they often keep their badge around the neck, or they are packed in the tram line 4 and 11, men dressed in suits, some are struggling to check in and out in the public transport...) they are simple rules to follow: avoid tram line 4 and 11 during the dates of the IBC event, visit other museums than the one mentioned above or simply walk from place to place and try to get lost.

Going to IBC give you only a few hours (or to any fairs or conferences) to explore the city where the event is hosted. And after the whole day standing up, running to this or this session or talking in front of the audience of the "Cutting Edge Session 2" about your work on 3D stereoscopy camera for digital dome you want to find a place who will treat you well (I'm not talking about massage with good ending), a place with a decent coffee or a good beer who tastes different than Heineken are welcomed.

Being a spotter since this summer for Berlin, I use the spottedbylocals city guide for Amsterdam which have the great feature to be usable off-line on your smart electronic device (the other features being a list of regularly updated spots in the city you are visiting or just living). So, you are lost, turn the app on, check on the map the closest spots depending what you are looking for "et voilà".

And for my second evening in town I had the pleasure to meet the people behind the spottedbylocals city guide. Of course we met in one of the local Amsterdam spots, the stadscafe Van Mechelen which offer a large choice of beer and a delicious burger. Nice evening it was.

11/09/2013

As usual in September I'm traveling to Amsterdam for the IBC event. I'm trying to travel light but still my luggage are too heavy: one PowerPoint presentation, one large laptop, three cameras, four lenses, some cables, a suit, sun glasses, other electronic devices and a lot to do on my agenda.

I found out that a bus goes directly from the airport to my hotel, it is pretty neat from the public transport in Amsterdam to have arranged such bus line. So I jump in the bus 192, cross nice suburbs with red houses, canals, bikers, dukes, gees, more exotic birds and end up in Osdorp inWest Amsterdam. No houses anymore, it looks like a giant tetris area, square buildings everywhere, but nice ones, long bike lines, long canals, they don't like curves here, only the gouda is curved.

It is close to 10pm and even if we are 7km from Amsterdam central station it is pretty quiet here. At least my hotel is still open. Time to grab some food where it is possible, go back to my room and sleep. Tomorrow the conference starts and I give a talk in the afternoon.

08/09/2013

If you have any interests in color, food, quality light source, food, clean line in landscape, food, you should consider to go once in Scandinavia. Even a simple coffee break after your flight with a German by the new Oslo opera facing the fjord or a one time use grill between a student residence and a tiny power station in Gjovik is worth the experience when you have the sun going down nicely and slowly in your face.

Coming back from another trip to Norway, I re-edited my full time classic: color work discussions for EU meeting, color and image processing work discussions by day at conference and by night while keeping the international level in the evening.

To keep the international level high you have to dose out the ingredients with caution: mix languages, at least three of them (e.g. French, English, Norwegian or French, Italian, German or Norwegian, Hebrew, English or Norwegian, Lithuanian, English or French French and Swiss French...) and mix food specialties. For example setup a norwegian grill with norwegian orange sausages and salad prepared by an Italian or have delicious lasagna prepared by a philosopher in visit Gjovik with later iraki kuba prepared by his mother and norwegian waffle for breakfast by his father, this not to mention the indian style lamb I had in Oslo by the Lithuanian-Norwegian team being served with home made beer.