Lytro’s new light field camera lets you focus after you take a picture

A new kind of camera captures rays instead of pixels, which lets photographers …

The tiny Lytro is the most technologically advanced point-and-shoot ever made.

Image courtesy of Lytro Inc.

Imaging scientist Ren Ng's years of research into capturing "light fields" using increasingly high-resolution digital imaging sensors have finally come to fruition. Ng's company, Lytro, unveiled its first consumer product on Wednesday—a digital camera capable of capturing "living images" that can be infinitely refocused after capture. While the new camera is designed to change the way we capture and share snapshots, the technology has the potential to radically alter how all photographs are made.

The new Lytro camera is a small rectangular tube of aluminum, with an f/2 lens on one end and a small 2" touchscreen on the other. The only controls are a power button, shutter button, and a slider to control the 8x zoom range of its lens. There are no controls for aperture, shutter speed, or focus—because the Lytro doesn't need them. The Lytro is probably the closest thing to "point-and-shoot" photography that has ever existed in the digital era.

Capturing light fields

Ng first began his efforts to capture living images as a researcher at Stanford. There, his cohorts, including Lytro engineer Bennett Wilburn, developed what's know as the Stanford Multi-Camera Array. The array consists of a hundred or more CMOS sensors and matching lenses and it was used to capture, among other research oddities, what is known as a light field. Ng used a similar array in his own research into capturing and processing light field data.

A multi-camera array built at Stanford to record light fields.

Stanford University

Traditional cameras capture flat images of three-dimensional space by focusing light rays directed toward a camera's lens on to a flat piece of film or digital sensor. Effectively, a camera records a two-dimensional array of colored dots (pixels in one case, film grains in the other) corresponding to light rays directed towards the camera from objects in front of the lens. The Stanford Multi-Camera Array, however, records the color, intensity, and direction of the light rays in a scene. These light rays collectively make up a scene's light field.

If you are familiar with 3D rendering and the concept of ray tracing, that technique is essentially the generation of a light field for a synthetic scene made up of 3D models. A ray tracing renderer follows rays of light emitted from synthesized sources as they bounce from object to object, reflecting at varying angles depending on the surface qualities of an object.

Without knowing anything about the objects in a scene, however, the entire scene and the objects in it can be visually recreated if its unique light field is known. Ng wrote his PhD thesis on the computational concepts and algorithms necessary to do so, as well as methods to capture a light field using digital CMOS sensors.

Ng then set about shrinking the SMCA—its hundred sensors and lenses, four PCs, and striped RAID array wasn't very mobile—into something even a casual photographer could use. Lytro is keeping most of the details of its camera under wraps, but instead of using multiple sensors and lenses, the specially designed Lytro sensor uses a sort of micro lens array on top of a high-megapixel CMOS sensor to record 11 "megarays" of light field information.

Lytro uses the overabundance of megapixels on today's CMOS sensors to record an entire light field instead of fixed focus pixels.

Lytro Inc.

These 11 megarays are what allows a Lytro user to shoot first and focus later. "The majority of pictures are shared online casually," Ng told Ars. "10, 12, 14 megapixels doesn't help when 95 percent of them aren't ever going to be viewed. We think it's better to harness those pixels to collect richer data."

That richer data, the light field, actually makes it possible to selectively refocus an image after it's captured. No more worrying about fast-moving subjects moving out of focus, or other objects or people in a scene confusing a camera's autofocus system. In fact, the Lytro camera doesn't have a focus system at all.

"The architecture allows us to do the work of the physical hardware of a lens with computation," Ng explained. "It allows us to increase the performance of the system while simplifying the hardware."

For instance, the Lytro always shoots every image at its maximum f/2.0 aperture, allowing it to capture images in very low light. Images can be captured with very little lag, since the aperture and focus do not need to be adjusted before taking a picture. Though the Lytro includes an 8X optical zoom, it's even possible to do zooming computationally. And despite capturing an image from a single location, light field data even makes it possible to perform some limited perspective shifting, as well—enough to generate 3D images from a single image capture.

Future of photography

You can't release a consumer camera today without an array of fashionable colors like "graphite," "electric blue," and "red hot." The "red hot" model comes with 16GB of memory and is capable of recording 750 images.

Lytro Inc.

Lytro is now taking orders for its camera, which comes in an 8GB version for $399 or a 16GB version for $499. The hardware is set to ship early next year.

Using the light field data captured with a Lytro, users can view living images on the camera's tiny 2" screen, can upload and view them using a desktop utility, or can post them online to Lytro's website. The desktop app is currently Mac-only, but a Windows version is planned for next year. Images posted to the Web require Flash to view on a desktop, and can be embedded on Facebook or other sites. Mobile devices can view the images using an HTML5-based interface.

Using Lytro's web-based "living image" viewer, simply click on any point in the image to instantly change the focus point.

As far as prints are concerned, though, Lytro's living images aren't quite up to the standards that photographers have come to expect. The 11 megarays it can capture are rendered in "HD resolution," which means at best 2 megapixels. "Light field photography lets you create interactive, living pictures, unlike print, so that's our focus right now," Ng told Ars.

But that doesn't mean the technology couldn't be incorporated into higher-end cameras in the future. "Current CMOS sensor resolutions have already blown past the ability to benefit the consumer," Ng said. "But, we can go so much higher than what's in the market. For instance Canon has 120 megapixel sensor prototype—we could be using that to collect a very high resolution light field."

The tech could even conceivably be shrunk down to fit into, say, a smartphone. A simplified lens meant to capture light fields might be more rugged and less expensive to design than the higher-end lenses seen in current-generation smartphones. Images could be captured with almost no lag, and light field data could be processed into images with adjustable focus and zoom after the fact.

We were only given a short demonstration of what the Lytro can do, but the possibilities left us wanting more. We'll reserve ultimate judgement until we can get our hands on an actual camera—which won't be available for a few more months—but the technology looks extremely promising.

The technology here is very impressive, particularly for 3D applications. Although from a consumer standpoint, I'm not sure how well the "shoot first, focus later" concept will work. Some folks may find it tedious if you have to have to go in and choose the point of focus after the fact.

I've been fascinated by this technology since Lytro first got some press. Pulled the trigger and preordered today. Can't wait to start playing around with it. I'm sure it won't be replacing my 5D for awhile, but this is one of those "holy shit, the future!" moments.

Although from a consumer standpoint, I'm not sure how well the "shoot first, focus later" concept will work. Some folks may find it tedious if you have to have to go in and choose the point of focus after the fact.

I'd guess a finished product based on this would do the same automatic focusing as a normal camera, but then allow you to change it later if that's wrong, much like you can change the orientation.

A couple friends and I played around with the images on their website when they first started getting press, and it didn't seem like there were more than four focal points (at least in the samples they had then). It's still 3 more than your average point and shoot, but any photographer who can compose the picture in the first place doesn't need the other three.

Ng's doctoral thesis on the technology Lystro uses (can't believe Ars didn't include this) was voted most readable thesis or something (it actually is) and is available here: http://www.lytro.com/renng-thesis.pdf

But I wonder whether he made the right decision to go it alone with what feels like a gimmicky sidekick that you could buy as an accessory to your Canon or Nikon. I don't think it will revolutionize photography, but I wish them well.

Is the pricing mentioned correct? This looks like an effect adobe cs could perform on a high resolution image that is in focus. This looks like a neat gadget, but the effect is similar to what you get in a "narrow depth of field" image. Adobe could allow you to do this by selecting objects (which it recognizes like faces) and keeps them in focus and then fuzzes out the other objects in the field. Focusing in and out of an image with a slider is something that could probably not be done and would indeed be a nice effect and I agree with the "living image" type of description.

If the laptop you are looking at the image on could focus on your eye, it could then automatically refocus the image based on where your eye is looking, much like optometry test equipment does. That would be a very nice "hands free" type of interaction.

The tech is really cool, but that camera is a disaster from a usability standpoint. Completely non-ergonomic design, so it's hard to hold steady in anything but good light (because camera shake will still be a problem). No flash, so you'll need those longer exposure times. And no tripod mount, so you can't even bolt it to a tripod to obviate problems one and two.

The 'no flash' is a real puzzler given that this is supposedly aimed at social situations. The f/2 lens certainly helps for that, but unless the sensor chip itself has noise performance comparable to an APS DSLR sensor, a lot of indoor lighting is still going to need longer exposure times without the aid of a flash.

I don't see the point to be honest. The tech is damn cool, but the result, outside of maybe some narrow applications (security cameras? 3D video that can focus where you're looking?) the drawbacks are huge.

Their prototype took a 16.7MP MF sensor and turned out a 296x296 pixel image. They seem to have revised their tech to get a little more resolution out of an 11 million photo sites. Then again, they're suspiciously quiet about what the resolution of the actual images are (everything this show is ~550x550px). In any case, if my estimates are right and they doubled the efficiency of their prototype, that 120MP Canon sensor mentioned in the article would net you a 1.2MP image. And you'd still need 35mm sized lenses to get the focal lengths and image circle size with enough of a retro-focus design that the dual layers of micro lenses don't become problematic with vignetting.

Sure you can move the focus around, but honestly how often are you going to move the focus around? For that matter how many people can compose an image that's truly interesting on multiple levels where you'd want to fly focus though the image more than once?

Ah, the internet, where some really pretty amazing tech is dismissed as "This looks like an effect adobe cs could perform on a high resolution image that is in focus." by someone who doesn't understand what's actually happening and "++ to the disaster from a UX standpoint comment too.". Where's people's sense of wonder these days, this is incredible stuff. BTW, interesting to think about whether this is what spy satellites are already doing.

As a parent, I reckon the concept is sound. My two often don't stay still long enough to catch 'em in focus - a lot of shots we've taken of 'em in the past have been a tad off, even with our DSLR, and especially when shooting with our f/1.8.

A constant-aperture f/2 sounds pretty decent, and "8x" optical zoom sounds good, but there's no mention of what equivalent focal range that covers.

The ergonomics look fairly poor, but I reckon that might be excusable as it does result in not having to much with folded optics, etc. Still, it's basically a box. Couldn't they have put a bit more thought into it than that?

What I find inexplicable is the built-in flash. OK, so perhaps there are performance reasons that would rule out SD cards, but the faster Compactflash cards would probably do well without breaking the bank, as would the format's CFast successor.

The 8GB version can reportedly capture 350 shots, then it needs to be emptied. Go on a longish trip, and you're basically required to either buy the 16GB version (which may well not be enough) and/or bring a laptop.

That seems like a huge regression for a camera launched in 2011. Flash is cheap and plentiful, and being able to bring a few extra cards for a long trip without requiring the logistics of hauling a laptop is a godsend. Being able to use any old card reader instead of worrying about directly connecting the camera is also great, and you can't beat the time it takes to swap cards instead of having to wait for the photos to be transferred.

The same goes for the built-in battery. No flash means that it probably won't use as much power on average as a typical camera, but I've often found having a spare on hand works out well. Besides, Li-Ion batteries ultimately don't age well, with their internal impedance eventually going through the roof. I've replaced the battery on every camera I've ever owned at some point or another (Blue Nook rules!), and a built-in battery is just an embuggerance.

... it didn't seem like there were more than four focal points (at least in the samples they had then). It's still 3 more than your average point and shoot, but any photographer who can compose the picture in the first place doesn't need the other three.

I think there's a few more than that - for instance if you click around the butterfly pic you can see some subtler shifts. If they're using 16 or 9 physical pixels (each in a different direction) per rendered pixel (11 MP -> 800x800=640000 = 17 to 1). But it does seem closer to 9 than 16. I wonder how much of that is a limitation of the app trying to figure out how deep the thing you clicked on is.

The more exciting thing here to me is that you can render the whole photo sharp (even though their web app apparently doesn't let you do that). So instead of stopping down to f/22 with the speed loss, just leave it at f/2 and fire away and then render it as sharp later. And the 3D modeling. And seems like you could make a decent stereogram from one shot of this. And you can do this all later.

Seems like a fun proof of concept at least - not sure I'd shell out $500 right now.

I'd imagine (could certainly be wrong) that what they're hoping to do is, yeah sure, sell some cameras, but more importantly get some buzz and make people want this tech so they can license it to the big established camera manufacturers. Like those "ask your doctor about" ads for prescription drugs.

Isn't this something like a Shack-Hartmann sensor but with orders of magnitude more pixels?

I wonder if they could achieve the same result with an array of 2x2 or 3x3 lenses in front of their focal plane, each focusing at a different depth of field rather than using a lenslet array and significant computer processing.

Seriously? You want the first iteration of a whole new technology to be completely mature? What, they're supposed to live revenue-free for as long as it takes to get to Apple's level of polish, when they can release something up to your standards?

This isn't for you. It's a new company releasing a super-early product for early adopters who have the disposable cash. The fact that it works at all is the triumph. Early adopters and geeks will snap it up, funding further research and development, and maybe in a few years it will be the consumer-friendly product that you feel you deserve.

Yeah, same basic idea that's been around since 1910 or so. Raytrix even cites Ng for the Stanford Light Field Camera. But it's all in the implementation. It'd be nice to see a comparison because Raytrix's cameras look much better built... but probably 10x the price or more.

I don't see the point to be honest. The tech is damn cool, but the result, outside of maybe some narrow applications (security cameras? 3D video that can focus where you're looking?) the drawbacks are huge.

Their prototype took a 16.7MP MF sensor and turned out a 296x296 pixel image. They seem to have revised their tech to get a little more resolution out of an 11 million photo sites. Then again, they're suspiciously quiet about what the resolution of the actual images are (everything this show is ~550x550px). In any case, if my estimates are right and they doubled the efficiency of their prototype, that 120MP Canon sensor mentioned in the article would net you a 1.2MP image. And you'd still need 35mm sized lenses to get the focal lengths and image circle size with enough of a retro-focus design that the dual layers of micro lenses don't become problematic with vignetting.

Sure you can move the focus around, but honestly how often are you going to move the focus around? For that matter how many people can compose an image that's truly interesting on multiple levels where you'd want to fly focus though the image more than once?

++ to the disaster from a UX standpoint comment too.

You aren't thinking

*True stereoopticon & other 3D camera variants have been marketed and disappeared due to technical problems. This is an ordinary point'n'shoot that takes photos that can be post processed at any time for a ViewMaster Disk, StereoOpticon postcard, prism lensed multi-pic print and almost all of the other 3D and picture-shift methods of printing.

*One of the serious headaches of good photography is selecting the correct speed, f stop and also focusing on the subject of interest. Ordinary point'n'shoots automate this process and regularly guess wrong as to what the settings should be. A pro photographer will take some time (while that once in a life time bird photo is winging away into the sunset) to adjust all these or the same pro will use his favorite point'n'shoot and hope it guesses right. With this camera you point, click and when you get back to your photo editor you adjust the f stop, speed and focus ... In short instead of the pro hoping that his settings all work for that surprise photograph in the field...he can take the photo and worry about the fiddly stuff later.

*You take a perfectly posed photo of your family with the focus carefully adjusted to show them at their best. Then while editing the photo you realize that if you carefully crop the left side you have a perfect candid photo that you would love to have in your album except that it is completely out of focus because you were concentrating on something else at the time you took the photo...again Lytros to the rescue. Crop the photo to create the surprise print you found and correct the focus to bring this new print into perfect focus.

These are just a few thoughts off the top of my head. This camera will NOT make a photographer great, but they will allow a great photographer to capture a far greater range of images without elaborate planning & preparation. Add elaborate planning and preparation and I think we will all be amazed at what the pros can do with this kind of tool.

As for your complaint about pixel count. Remember the Kodak Brownie Box camera was not revolutionary for the quality of the photographs it could take. Pro cameras were light years ahead of the Brownie. What made it revolutionary was that it automated photography and made it accessible to everyone. This Lytros is the first consumer release of this new tech. I am looking forward to seeing what comes to market after a few more years of R&D developing high end cameras using this tech.

My first digital was a 3Mp camera and that resolution was wowing the reviewers when it was first released. Don't let the Model T of today prevent you from dreaming about the Lamborghini of the future.

Sensor resolution is given in units that are not defined. Picture output is only "HD-quality". No information is available for the lens material, the dimensions of the sensor, the focal length range of the lens, or shutter speeds. The camera can use only the built-in memory and battery. Files can be processed only with Lytro software.

To be "shared", files must be stored (by adults only) on the Lytro server. To get an account you must give a license to Lytro that allows them to use your images any way they want.

Sensor resolution is given in units that are not defined. Picture output is only "HD-quality". No information is available for the lens material, the dimensions of the sensor, the focal length range of the lens, or shutter speeds. The camera can use only the built-in memory and battery. Files can be processed only with Lytro software.

To be "shared", files must be stored (by adults only) on the Lytro server. To get an account you must give a license to Lytro that allows them to use your images any way they want.

The ability to refocus has not been demonstrated publicly.

You can look at their demos for re-focusing pics... They have many examples online that you can play with... Now what resolution the originals are I have no idea, but you can play with the ones they have..-Dan

I'll admit I may not be understanding this correctly, but saying that it has no shutter speed control because it doesn't need it sounds strange to me. What "equivalent" shutter speed does it use? Will a picture I take of a hummingbird come out as a blur because I can't pump it to 1/500 sec? Using a combination of aperture and shutter speeds for -lighting- is one thing, but there are other reasons for it as well that this doesn't seem to address at all. Maybe this camera just isn't for me, but I like being able to take long-exposure shots, also.

On second thought, this would be good at places like zoos that put animals behind chain link fences. Chain link is a bitch to focus "through" most of the time, but the effect you get with a out-of-focus fence in the foreground is interesting. Same with any sort of screening, I guess.

The technology here is very impressive, particularly for 3D applications. Although from a consumer standpoint, I'm not sure how well the "shoot first, focus later" concept will work. Some folks may find it tedious if you have to have to go in and choose the point of focus after the fact.

Also not sure about the ergonomics of the rectangular cube design.

You can choose and re-choose after the fact. Even when it gets published up to the web it can be done over and over again. It's a proprietary file that can exported flat if it needs to be printed.

Wow, buy if you have to upload and manipulate before you can share and show, I think it will have limited appeal. Sounds like it will be a clever niche product. Intrigued!

The true beauty here will be integrating this technology into a camera with a better CMOS than what Lytro is currently using. Right now it's getting a 2MP image but imagine using one of Canon's 25MP sensors with a fixed F/2? A 3D ultra HD picture that could be viewed on a television or even a computer monitor at full resolution without a compromise in quality.

I see an imaging company (Canon, Nikon, Olympus, Ricoh) buying this company out. Heck maybe even an Apple to use exclusively in a phone. This tech shrunken down will be worth millions if it isn't already.

Seriously? You want the first iteration of a whole new technology to be completely mature? What, they're supposed to live revenue-free for as long as it takes to get to Apple's level of polish, when they can release something up to your standards?

This isn't for you. It's a new company releasing a super-early product for early adopters who have the disposable cash. The fact that it works at all is the triumph. Early adopters and geeks will snap it up, funding further research and development, and maybe in a few years it will be the consumer-friendly product that you feel you deserve.

I don't feel I deserve anything, but thanks for telling me what I think.

For the record, I've been following Lytro's progress for ages now, and had considered buying one as soon as they announced any sort of availability. What arrived in my inbox this morning put me right off on a bunch of levels.

The new technologies here are their optical design and the software on the backend to make sense of what's captured by the image sensor. Does that really mean they have to say "sod it" to actually making the thing something you'd want to hold for any length of time? This thing has all the ergonomics of a brick.

I'm taking issue about the shape of the box they chose to stuff its guts into, which has nothing to do with maturing technologies. Nowt whatsoever.

When I first saw it this morning, I thought they'd ditched their initial idea and for some reason had decided instead to launch something along the lines of one of the multitude of scientific cameras that are out there that are intended to be bolted to a telescope or some other instrument. I thought it was a bit odd given they've been talking for months about a handheld camera, and I honestly did a double-take when I realised this was it.

I mean, come on - they could have thrown this in any old SLR-like frame with very little effort, or even dumped it into something similar to a micro 4/3 body plus lens. The end result would probably handle way better (plenty of space to grip it, etc), and they'd have no problems with incorporating tripod mounts, removable batteries, flash card slots, etc. At that size, they could have sourced any number of touchscreens from a large list of manufacturers instead of limiting themselves to something the size of an ipod nano.

Instead, they honestly seem to have gone with what could be the most awkward shape they could have thought of for a hand-held camera, and that's forced them into a big pile of compromises.

The only concessions to handling I see are that they've rounded off the corners and added what looks like a ribbed, rubbery bit at the back. The shutter button's in line with the camera's centre of gravity. They seem very proud of that accomplishment, but they were pretty much forced into it by adopting their stubby telescope form factor.

Oh, and colours. Don't forget that the took the time to launch their super-early product in three colours.

Telescopes work as long tube sort of affairs as their length allows them to be held fairly stably with a decent grip in both hands. This thing's no longer than an iphone, more or less requiring it to be held with fingertips. I don't see being able to grip a hand around it without obscuring the shutter button, and holding it at the other end invites fingers to occult the lens.

It's approximately 22 MB per image. Imaging recording at 24 FPS at a higher resolution to get better detail in a security camera. At the current resolution you are looking at 45.5 TB of data a day! A decent security set would keep 2 weeks worth at a minimum. I would love to develop a sec. system capable of holding ~630 TB of data at a time from only one camera. That would be quite the investment. $35k in storage alone from one camera.

The tech is really cool, but that camera is a disaster from a usability standpoint. Completely non-ergonomic design, so it's hard to hold steady in anything but good light (because camera shake will still be a problem). No flash, so you'll need those longer exposure times. And no tripod mount, so you can't even bolt it to a tripod to obviate problems one and two.

The 'no flash' is a real puzzler given that this is supposedly aimed at social situations. The f/2 lens certainly helps for that, but unless the sensor chip itself has noise performance comparable to an APS DSLR sensor, a lot of indoor lighting is still going to need longer exposure times without the aid of a flash.

1. Camera shake does not make a difference as there isn't a focus motor. It's an instant shutter.2. As for the flash, that is one point they have not discussed yet; performance in low light.3. Yes, they definitely need a mount. I see an opp for an accessory.

One thing I noticed about the same pics were they're either telephoto shots or take of close subjects, I'm guessing so that the background is out of focus. There was certainly nothing like what you can achieve with a fast lens and an SLR. If this camera a small sensor and therefore a large DoF then what good it is? It's cool tech sure, but most P&S cameras (which this is) have such a large DoF that everything is always in focus anyway.

As a parent, I reckon the concept is sound. My two often don't stay still long enough to catch 'em in focus - a lot of shots we've taken of 'em in the past have been a tad off, even with our DSLR, and especially when shooting with our f/1.8.

A constant-aperture f/2 sounds pretty decent, and "8x" optical zoom sounds good, but there's no mention of what equivalent focal range that covers.

The ergonomics look fairly poor, but I reckon that might be excusable as it does result in not having to much with folded optics, etc. Still, it's basically a box. Couldn't they have put a bit more thought into it than that?

What I find inexplicable is the built-in flash. OK, so perhaps there are performance reasons that would rule out SD cards, but the faster Compactflash cards would probably do well without breaking the bank, as would the format's CFast successor.

The 8GB version can reportedly capture 350 shots, then it needs to be emptied. Go on a longish trip, and you're basically required to either buy the 16GB version (which may well not be enough) and/or bring a laptop.

That seems like a huge regression for a camera launched in 2011. Flash is cheap and plentiful, and being able to bring a few extra cards for a long trip without requiring the logistics of hauling a laptop is a godsend. Being able to use any old card reader instead of worrying about directly connecting the camera is also great, and you can't beat the time it takes to swap cards instead of having to wait for the photos to be transferred.

The same goes for the built-in battery. No flash means that it probably won't use as much power on average as a typical camera, but I've often found having a spare on hand works out well. Besides, Li-Ion batteries ultimately don't age well, with their internal impedance eventually going through the roof. I've replaced the battery on every camera I've ever owned at some point or another (Blue Nook rules!), and a built-in battery is just an embuggerance.

P.S. They're launching only with a Mac version of the software you need to manage the camera? Really?

A lot of the criticisms you have pointed out were said of the first iPhone, and look how that turned out. It's literally, brand-new, bleeding-edge technology! Holy-shit; be happy someone has done something innovative and incredible. Imagine upping this tech by 1000x!!! Think of the possibilities.