3D Video Primer, Part 2

3D Video Primer: Part 2

Why 3D, Why Now?

Three-dimensional still and video imaging has been around for a very long time, but in recent years, 3D has enjoyed something of a resurgence in popularity. Film studios are busy pumping out 3D movies, every other television set released these days seems to be capable of displaying 3D content, and it is even possible to capture images in 3D yourself, using many of the latest digital still and video cameras.

But what is driving the current upsurge of 3D? Improvements in the technology, pressure from manufacturers who are desperate to create and monitize the next 'big thing' in consumer electronics, or consumers themselves?

3D Video: Who's playing?

All of the major television manufacturers offer 3D-compatible TVs, and today's big 3D releases are available for purchase on Blu-ray, to watch in the home. After a long wait, there is now a standard for 3D video. Announced earlier this year, AVCHD 2.0 includes support for 1080 60p and 50p, and high-definition 3D movies encoded using Multiview Video Coding (MVC). As far as 3D video creation is concerned, Panasonic and Sony are currently the only two companies that offer an 'end-to-end' 3D solution in both professional and consumer spheres, but in the consumer space they are also joined by JVC.

As well as 3D display solutions, JVC currently markets two 3D camcorders, the twin-lens, twin-sensor GS-TD1B full HD 3D Everio, and the single-lens, single-sensor GZ-HM960BUS Full HD Everio, which can convert 2D footage into 3D in-camera. Like the GZ-HM960BUS, Panasonic's HDC-TM900K is a conventional single lens, single sensor 2D camcorder but when paired with an optional 3D conversion lens it can record 3D footage.

The Sony HDR-TD10 is designed in the same way, and allows videographers to manually adjust the convergence of its two lenses for fine control over the 3D effect.

Panasonic takes a different approach in the HDC-TM900K, which is a conventional single-lens single-sensor camcorder which can produce 3D footage when paired with the optional VW CLT1 stereo adapter.

Sony's 3D Bloggie pocket camcorder can produce 2D or 3D video and stills courtesy of its twin lens design. Like the Panasonic HDC-TM900K/VW CLT1 pairing, footage is captured on a single sensor, at correspondingly reduced resolution compared to 'true' twin-lens, twin-sensor designs.

Sony debuted its HD 3D Camcorder, the Handycam HDR-TD10, earlier this year. The twin-lens, twin-sensor TD10 allows the user to switch between recording in 3D and 2D, and if you don't have a 3D-compatible TV or computer, footage shot in 3D can also be played back conventionally in 2D. Sony also markets the 3D 'Bloggie' compact video camera. Dual lenses provide 1920x1080 3D HD video and like the TD10, the Bloggie offers the option to record in 2D as well.

The advantages of twin-lens, twin-sensor designs, like those shown above from JVC and Sony are primarily the ability to adjust the convergence of the two lenses (either manually or automatically), which is useful when shooting at close focussing distances, and low-light performance. With a single sensor per lens, there are many more pixels available to assemble the image, and in low light this means that downsampling (where the signals of neighboring photosites are combined to produce a cleaner, less noisy image) is more practical.

Systems that use twin lenses but a single sensor cannot be adjusted for convergence (although this isn't a problem in most shooting situations) but more seriously, they must necessarily use a smaller area of that sensor to record each of the left/right channels, limiting the options for downsampling, and thus increasing the risk of noisy footage in low light. Naturally though, twin-lens single-sensor systems are potentially less bulky and - of course - less expensive.

Still 3D Imaging

Stereography has been popular for more than a century, but until recently it was largely confined to the hobbyist sphere. Although 3D still imaging isn't being pushed as heavily as video by mainstream electronics manufacturers, an increasing number of cameras are being introduced which feature some kind of 3D functionality, either built-in or via special accessories.

Several of Sony's current range of Cyber-shot compact cameras, including the recently-reviewed DSC-HX9 and DSC-TX10, feature a 3D capture mode, and the same is true of Panasonic's DMC-ZS10, FX78 and TS3. last autumn, Panasonic also announced three new interchangeable lenses for its LUMIX G series of cameras, including the world's first interchangeable 3D lens, the H-FT012.

The only company which currently offers a dedicated 3D still image camera is Fujifilm. The FinePix REAL 3D W3 features two lenses and two sensors. Fujifilm also markets a small range of dedicated accessories, including the FinePix REAL 3D V1 viewer, an 8-inch autostereo LCD display.

Life in 3D

Manufacturers are clearly banking on the monetary potential of consumer-level 3D imaging, but stereoscopy, in many forms, has been around for over a century. Why the sudden interest in grabbing a new consumer demographic and putting 3D cameras into their hands?

According to Dave Briganti, Senior Product Manager of Imaging at Panasonic, part of the answer is that various technologies are maturing at the same time, and coming together to make it easier for manufacturers to offer complete, affordable, 3D 'solutions'. Briganti mentions 3D televisions as a specific example: 'I think bringing this phenomenon into your living room has been what's driving this current trend. Television [reflects] what happens in mainstream America. I think the fact that 3D has become available in a TV in your home is really what's lead imaging manufacturers to develop these 3D products and technology.'

If you're shopping for a new television, the chances are that you'll come across a lot of models which boast '3D compatibility'. Although availability of 3D content is still fairly low, the number of channels that offer it is growing, and 3D movie releases are available on Blu-ray disc. Manufacturers are also hoping that the emergence of affordable, consumer-level 3D video cameras onto the market will increase the popularity of home 3D, too.

The logical next step from becoming a consumer of 3D content is becoming a creator. When it comes to 3D camcorders, Andy Bubala, Sony Electronic's Camcorder Business Director echoes Briganti's sentiments 'It used to be a five-figure minimum for this type of equipment, really only available to professionals and studios. Now, for $1,500 you can buy a video camcorder that shoots really high quality 3D, and has the option to shoot and playback in 2D.'

But will consumers accept a whole new learning curve? How easy will it be for people to adapt to a new way of capturing and viewing video? Most people can't remember a time when enthusiast amateurs didn't have access to video cameras. So in 20 years' time, is it possible that we won't be able to remember a time when we didn't have 3D video cameras?

'It's easy to make 3D, but hard to make it good'. So says Buzz Hays, Senior VP of Sony's 3D Technology Center, and respected stereographer. 'As a consumer, you want to be able to turn on the camera and get baby's first steps. You don't get two chances with that. So at Sony we had to take a lot of the professional experience we have in 3D filmmaking and translate that into an experience that works in a video camera where people shooting it don't have to think at all about what they're doing.'

Sony's TD10 3D camcorder uses two lenses, and two sensors - one for each lens. As illustrated in this image (simplified for illustration only), the convergence of the lenses can be adjusted either automatically or manually by the videographer, according to the distance of the subject. This doesn't make much difference at normal viewing distances, but can help a lot when shooting subjects at close quarters.

The experience that Hays is talking about is the experience of using Sony's latest HDR-TD10 3D HD camcorder. The TD10 captures high-definition 3D video using two lenses, mated to two sensors, and its innovative '3D ready' LCD screen shows 3D footage during capture and review, without the need for special glasses.

Some 3D camcorders use a single lens/sensor combination and split a 1920 x 1080 signal into two channels, thereby reducing 'true' resolution. The TD10's twin lens/sensor design maintains full HD in both left and right channels, mimicking the human vision system. By adjusting the amount of convergence between the two lenses, advanced videographers can take precise control over the look of their 3D footage. If they want to, amateurs – of the sort that Hays is talking about – can just leave those decisions to the camera.

Adoption Worries

Despite the pace of innovation, and the raft of new 3D products on the market, manufacturers aren't getting things all their own way. Panasonic's Dave Briganti accepts that consumers aren't adopting 3D as quickly as the industry had expected. There are many possible reasons for this (a global recession certainly doesn't help) but the most likely is that consumers tend to be a little cautious about jumping onto new technological bandwagons in case they turn out to be a fad. A similar, equally cautious purchasing pattern occurred when HDTVs first debuted on the market.

In fact, Buzz Hays at Sony believes that part of the answer to the thusfar relatively slow takeup of consumer 3D lies in the timing of the new technology, so soon after HDTV – the last 'big thing' in television. As Hays points out, a significant number of consumers have only recently, within the last three or four years, adopted HD technology on their home sets.

'For those people who've just bought an expensive TV that will last them for a while, they're not ready to turn around and invest again,' he explains. Going forward, however, 'Sony is now outfitting a significant majority of its newly manufactured TV with 3D capability, so regardless of whether people buy it for 3D purposes or not, [if you buy a new Sony TV now] you have a very good chance of ending up with something that's 3D capable.'

Sony's strategy is not unique. Dave Briganti at Panasonic similarly believes that the secret to consumer adoption of 3D partly lies in the way in which the technology is promoted to consumers. Panasonic's stragegy is simple: 'When we have a customer that wants to buy a 3D television, we try to bundle that with a camcorder or another 3D device that is compatible with the set.'

Meanwhile, Nick Dager, an analyst with the Gerson Lehrman Group as well and editor and publisher of Digital Cinema Report and IndieFilm3D thinks that the reasons 3D televisions, specifically, are not selling as well as their manufacturers had hoped are threefold. 'The cost [of the hardware], confusion about [whether or not people need] special glasses – as well as the cost of the glasses - and a serious lack of quality content'.

In Dager's view, it is the third reason that holds the key to 3D's long-term viability, and that as the amount of high-quality 3D broadcast content increases, so will the willingness of consumers to watch it.

With 3D TVs filling stores, more and more 3D-capable cameras and camcorders being released and more tha 40 3D movies projected for release in 2012, 3D is growing fast. But like a teenager experiencing growing pains, it hasn't quite reached maturity yet. If Hollywood can stop making 3D just to make 3D and find out where it really fits, and if electronics manufacturers can convince consumers that their home videos reallyneed to be in 3D not 2D, then maybe, just maybe, the angry teenager will grow into a healthy adult.

Comments

I have been shooting stereo slides since 1952 with the same Stereo Realist camera.I have just about 4,000 3D images that I can look at in my Stereo Realist viewer or project on a silver screen with my TDC Stereo Projector, using polaroid glasses.I have been waiting 60 years for technology to catch up with concept.Now that is doing that, let us calm down ... let things play out ... and accept the simple fact that, unless we have only one eye, we do see things in 3D.Isn't that the way we should see things ... period?

* Size (closer looks bigger)* Focus (objects further or closer than focal point blurred)* Overlap (distant objects more likely to have objects in way.* Paralax: lines appear to converge at infinity.* Differential motion: distant objects appear to move more slowly (jet high in sky seems "slower" than at bird flying over at 5m).

Stereo vision is probably the least important, except at very short distances, as when a chimp or human grab a banana from a tree.

I cannot agree more. An image that is high resolution and large DOF (sharp) and big will look "3D". When our eyes focus on one spot, the other location automatically become blur. This is the same way we feel the environment.

Stereo vision only looks real *when you cannot move your head*. How do you explain to your brain that an object is still showing you the same side when you move your head to the side?

There are some 3D experiments that uses head tracking and render image in real time. The 3D effect is much more intense.

Maybe this is a "dumb question" but let me see if anyone better-informed can answer it.

If I have a mix of 2D and 3D video, and I mix the two in a single video file, can the 3D HDTV or Blu-ray player recognize the side-by-side 3D sequences as 3D, and display them as such, or will it show everything 2D, with two distorted side-by-side images in 2D? Does the 3D element have require a special coding? Or can the 3D HDTV take the side-by-side format as its cue?

This is a problem many might face, since they may not shoot everything in 3D, or they may want something that's playable on either a 2D or 3D system. I already know, of course, that a conventional 2D HDTV will show side-by-side images as two stretched 2D images, without any pan and scan or correction.

3D concept has been here for a long time but technology to acheive it keeps on changing. I dont think 3D equipments will become a mainstream product but will always be found in a very niche market before it disappears all together.

It is like the case when viewing the Mona Lisa painting. You don't have to do anything beyond watching a great masterpiece from the viewing public.

But with 3D viewing, you have to do something else to view the Mona Lisa. It's like the gallery guide telling you: Step here and view it here, stand there and look from there. No, it looks fuzzy there, move over here.

There is always an additional element to utilize compared to "normal" viewing which translates to special glasses, special screens, and special TV's and other gadgets. This "additional" essential accessories are what exasperates the public and prevents them from fully embracing the experience.

The camers are two of the HD hero models. The stereoscopic housing is made the same as the individual ones so the problem underwater would be the same. Some users have modified the housing to correct this.

i never managed to understand the use of "toe-in"/converging optical axis for close focusing like in the Sony TD10. it is the OA of the viewer's eyes that are supposed to converge, giving a sense of depth - not the OA of imaging device, is it?

I agree with this.What matters is that recorded infinity is displayed on the screen separated by the distance of the viewers eyes which means that the image separation in terms of percentage of the whole image is dependent on the size of the display, in the cinema a 6cm separation is negligible compared to the size of the screen while on a computer display it is usually over 10% of the image width. It makes no sense at all that this separation is fixed into the recording, it should obviously be a customizable setting at home which for a tv where the size of the screen is known by the electronics could obviously have a very simple preset.Until this becomes the standard and they drop the convergence nonsense 3D will be fundamentally flawed.

And whats with the silly 2-3 cm lens separation in many of these camera's?

In my opinion, using convergence (a.k.a. toe-in) ruins the naturalness of a 3D experience. It causes subject planes that are well in front of or behind the plane of convergence to appear doubled-up, just as they do with natural vision, but in real life, I have the CHOICE to rapidly switch to any point of convergence in my field of view, from foreground to background to any point in between. Not so with a 3D film shot with convergence - the entire audience is LIMITED (forced) to a single plane of convergence along the z-axis, and worse - this unnatural restriction is usually accompanied by selective focus or weird lighting that becomes necessary to suppress the doubled-up planes (making them darker, typically) that lie well beyond the plane of convergence.

In real life, our vision doesn't automagically lose depth of field or cast shadows on the background every time we converge our eyes on something that's close enough to cause a doubling of other objects in the subject space. In my opinion, a far more natural experience is afforded the audience when the lens axes are kept parallel at all times - in combination with extreme depth of field, except in those shots were selective focus would be applied for the exact same reasons it has been used to great effect in 2D stills and movies for decades.

When shooting with the lenses kept parallel, no doubling occurs at any plane along the z-axis, but the absence of doubled subjects goes completely unnoticed by the audience, just as the presence of doubling goes completely unnoticed in real life. And best of all, the audience has the FREEDOM to explore every scene at any point of along the z-axis they choose. Just as in real life, with two people standing side by side in a forest, one can be converging his eyes on the bark of a tree that's in the foreground while the other is simultaneously examining a distant meadow, seen through a gap in the trees. That's the natural experience that an audience should have when watching a 3D movie - with everyone looking where they choose to look - and it can only be had by shooting with parallel lens axes.

What I would have liked to see is a real analysis of the extrapolation algorithms these cameras use to make up for the tiny intra-lens distance. Human eyes are 6-8cm apart, the lenses on these cameras are 1cm apart, binoculars (particularly for ranging artillery) can have lenses over a 1m apart. So it would be great to see some hard tests of scenes shot with these cameras vs. shot with 2 cameras at the correct distance apart.

Thank you for this article, but I find it too turned to the Sony brand. It would have been necessary to mention the only real camera (a compact) that has existed for over a year now, namely the Fujifilm Finepix Real 3D W3. It would be useful to encourage camera manufacturers to produce good cameras (models for pictures, not only video cameras models) model with good sensors (Panasonic has announced a model for winter but it seems it should be not terrific in this aspect).It should be noted that among a long list of websites showing very few good photos in 3D (and anaglyph often) very few sites show to future users of 3D cameras what can be done with ... It would have be nice to present the MPO format too.May I suggest you to let some 3D experimented users to explain how make 3d photo concretelyHere are two galleries showing more than 1000 photos:Anaglyphe: www.pbase.com/christianswiss/3dMPO: www.christian-roux.ch/3d

I question the ability any 3D camera. I went to a seminar on 3D photography at my local camera club where the person giving the talk was showing his two Canon DSLRs he had mounted on a tripod. He could get the two cameras roughly 20cm apart on slides or bring them together so the cameras touched. I asked why he had to be able to get them so far apart as I thought the distance between human eyes was ideal for 3d? He explained that the further the object was from your cameras the further they have to be apart to capture 3D. To photograph clouds in 3D he uses two tripods keeping the cameras 50 meters apart! I don't know how Hollywood makes 3D films though I would imagine they too use two cameras that can be separated.

Skillful photographers have been creating the illusion of 3D for years in 2D. When I went to a 3D film 'Pina' it was pretty obvious that it would have looked just as good in 2D as in 3D. I suggest composition rather than gimmics is the answer. I could add that I was 'mad' on 3D back in the late fifties and even built my ownbeam spliotter using the prisms out of a pair of binoculars for my 16mm camera, in a balsa wood housing :-)

The basic problem with 3D is the need to wear glasses to view it and only the Russian Raster system overcame this at the expense of seating in the theatre.

Would you suggest displaying those images captured with very close convergence on a tv that is 3 meters away? If anything else is captured at a longer distance in that same shot it would require your eyes to diverge while your eyes are converging at 3 meters to see the very close object, it simply doesnt work like that. If you capture close objects with the lenses pointing straight ahead and then display those images seperated by the distance of your eyes the close object will automatically be displayed for close convergence of your eyes. The convergence distance of your eyes will then depend on the difference between the captured view-angle and the displayed view-angle. There is simply no logical reason for 3D camera's to have convergence. Displaying fixed images for 3D that are a 1 to 1 match to the display is a flawed concept. The two images need to be independent just like for HMD's which have no overlap at all while a cinema screen has almost complete overlap.

If you say so Malcolm :-) In watching Pina since 3D came to my local recently I thought that it would have looked just as good with one eye becuase to emphasise 3D the staging was arranged in depth, the way a competant cinematographer and many still photographers work when it is possible.I'm sure that when this craze dies down in a year ot two the exthusiasts will keep playing but Hollywood will bring it back in another 20 to 30 years as the 'new' thing you 'have' to watch at an inflated price. This is the third time for me.The best 3D I remember, after fifty/sixty years, are the fabulous films of Norman McLaren of the National Film Board of Canada. You may have been lucky to see his hand drawn paint-on-film in 2D but in 3D they were tremendous:-)