Monday, April 16, 2012

The Current Status and Future of 2D-to-3D Conversion

We are currently seeing a great
deal of interest from the major studios regarding the 3D conversion of select
titles from their catalogs. A newly
converted “Starwars, Episode 1 The Phantom Menace” was released over a month
ago to respectable box-office numbers. In a recent review, Richard Cosgrove writes, “From the very start, when the iconic Star
Wars logo appears on the screen to herald the famous title crawl (which really
does go off into the distance now), the 3D is both all consuming and immersive.
The detail on spaceships as they cruise by, or battle droids as they line up in
their squadrons, is awesome.” He adds, “the 3D works so well that particularly
in the more dialogue heavy scenes in the various palaces, spaceships and other
locales in the film, the feeling of actually standing in the room with the
characters is very convincing.” Similarly,
“Titanic” was converted and released on April 6th. It also pulled in very respectable
box office figures, particularly internationally and the film has since been
the focus of wide acclaim. In Lou Lumenick’s recent review, he writes “the 3-D in “Titanic’’ is more effective
and immersive than for most films I’ve seen that were originally filmed in the
process.” He goes on to write, “With
added depth, the iceberg-stricken vessel rising vertically in the water,
breaking in two and sliding under the waves is even more awe-inspiring than it
was at first sight in late 1997, when “Titanic’’ began a still-record 15 weeks
at the top of the US box office.” At
Legend3D, we are confident that "Top
Gun 3D" will do as well and may bring in even higher box office numbers because of
its potentially broader demographic appeal.

The reason catalog titles are
being tapped for conversion at this time is very simple. The catalog titles
that are going through conversion are known entities: veritable iconic cinema
from our American film heritage. They
are titles that were exceptionally successful on their first release so there
is little risk in spending a relatively modest amount of money for conversion
and re-releasing them today in 3D. These
titles are expected to reach a whole new generation of moviegoers, albeit, a
more sophisticated audience that may prefer to experience the films for the
first time in 3D. There are also those
people who were fans of these titles when they were first released and now want
to experience them again, but this time in 3D.

Real-Time
versus Non-Real-Time Conversion

Over
the past two years, just about every TV manufacturer has attempted to create
programs that can convert 2D content to 3D on the fly. Of course, the reason for this focus on real-time conversion has been the lack of available 3D content. In general, automated or real-time 2D to 3D conversion processes attempt to create the perception of 3D based on certain depth cues that we can detect with only one eye such as occlusion, saturation, brightness, texture, parallax, size, etc. The problem with these algorithms is that monocular depth cues do not remain
consistent over the course of a typical scene. Inevitably, things
become confused and objects often fall out of proper depth. If set up properly such
programs could work in certain sporting events where camera positions are
locked, the field is fully modeled ahead of time and lighting remains
consistent but by in large, the process is generally flawed and inferior in quality compared to
labor intensive, non-real-time feature film conversion. Some people have equated real-time conversion
to typical standards conversions such as the up-resolution of NTSC to HD.
However, automatically adding lines to frames does not modify the original information
or the quality of that information. Theatrical quality 3D conversion changes the
whole viewing experience in a profound manner, enhancing the storytelling via
strategically planned depth placement, volume, convergence, inter-axial
distance, etc. A Stradivarius violin can truly be appreciated only when played by a virtuoso.
Likewise, converting a feature film from 2D-to-3D requires much more than the most advanced technology, it's a very subjective
and creative process requiring a great deal of expertise and talent. There are no short cuts in the process
because attention to fine detail is an absolute necessity when attempting to
achieve the highest quality product. In
addition, as I mention in my previous Blogs, an in-depth knowledge of the
psychophysics of how we perceive stereo is essential to achieve optimal results.

The
Changing Cost of Conversion

You’ll
always be able to find a conversion company somewhere in the world that will
offer to convert footage from 2D-to-3D at $3K/minute or even less. Unfortunately, at that price range people get
precisely what they pay for and disappointment reigns supreme. On the other
hand, pricing for the highest quality conversion of new feature film releases by the major conversion
houses no longer has to be north of $100K/minute or even $90K/minute. I have recently seen conversion work by one conversion studio priced above
$110K/minute that I consider subpar in quality and I’ve seen conversion work by another conversion studio priced at
$55K/minute that was quite acceptable. Poor quality conversion at top dollar prices simply hurts the industry and the only way to counter that trend is by encouraging strong competition within the market so that quality and price prevail. Conversion of catalog titles is a fraction of the cost of converting a new feature film because the footage of a library title is completely conformed, there are no visual effects to wait for, there is no directors cut to wait for and typically the review process is much simpler and streamlined, handled by the conversion vendor with a sign off by one authority from the client studio.

Evolution
of 3D Conversion

There
are currently three major conversion studios in the U.S., some midsized international
studios and several smaller boutique operations that are all vying for new feature
film contracts or pieces of new feature film contracts. I believe that the days of multiple
conversion studios working on a single film are largely over. Today entire films are being converted by one
designated conversion vendor, though in some cases the client studios are
mitigating real or perceived risk by using one primary conversion vendor along
with a minor vendor acting as a backup resource when/if necessary. As mentioned in my opening paragraph, three iconic feature films are being re-released this year, Starwars, Titanic and Top Gun. Each was converted by one of the major conversion vendors. We've already seen from Disney that conversion of animated features is a winning proposition. I believe the quality of these three live action films will be a defining element in
the future of catalog conversion if not conversion and 3D in general. I believe that the three films, quite different in genre, will give the public an opportunity to assess what some are calling a pinnacle moment in conversion. It's my hope that an assessment of the creative and technical quality of
each of the three catalog titles will be useful in defining a new quality standard for this medium going
forward.

Followers

Barry B. Sandrew, Ph.D.

Founder - Graffiti Video, Inc.

BIOGRAPHY

This is a series of blogs by Barry Sandrew, Ph.D., an internationally recognized inventor, digital imaging expert and visual effects pioneer. Dr. Sandrew is founder of three visual effects facilities that were among the largest and most prolific production studios in Hollywood. Over the past three decades he has been and continues to be instrumental in evolving the entertainment Industry's digital standards and processes in feature filmmaking.

Sandrew earned his doctorate in neuroscience from SUNY at Stony Brook. After winning a 2 year NIH Fellowship at Columbia University, College of Physicians & Surgeons, he joined Harvard Medical School/MGH as staff Neuroscientist. In 1987, Sandrew left his academic and scientific career to found American Film Technologies (AFT) where he invented the first all digital process for colorizing black and white feature films.

At AFT he also invented a paperless animation process used to produce episodic animation for Fox Children's TV and a digital ink & paint and compositing pipeline that produced Spielberg's first digital animated feature film, "We're Back: A Dinosaur Story." Sandrew took AFT public while converting hundreds of movies for clients such as Turner, Disney, Warner Bros., Fox, Gaumont, TF1, ABC, and CBS, among many others.

In 1993 Barry Sandrew left American Film Technologies to co-found Lightspan, an animation and production edutainment studio that grew into one of the largest educational software companies in the U.S., marketing into entire school districts around the country. Lightspan ultimately went public and was later acquired by Plato Learning.

In 2000, Sandrew founded Legend Films, re-inventing colorization and a proprietary film restoration process using the latest in digital imaging technology. Over the course of the next 7 years, Legend Films converted to color approximately 145 black & white films as well as several TV series. Legend Films also produced visual effects for Scorsese's "The Aviator", HBO's "Entourage" and other high profile TV and film projects.

In 2007, with Jim Cameron's game changing "Avatar 3D" scheduled for a 2009 release, Sandrew leveraged his proprietary colorization pipeline, redirecting his company's entire R&D focus to embrace 2D-to-3D conversion and changing the company's name to Legend3D. Since 2010, Legend3D has lead the competitive field, producing 3D conversion and visual effects on over 35 of the highest performing box office tentpole films of the past 6 years.

In 2014, Dr. Barry Sandrew left Legend3D to focus on new immersive and web based media technologies as well his non-profit and for-profit board positions. He is consultant to Fortune 500 companies re: mobile 3D-sensing and motion tracking technology that he considers precursors to augmented reality. Most recently, he is Co-Founder and President/CEO of Graffiti Video, Inc. a company that is redefining the curation of web based media.