Foveon’s Remarkable Quest for Realism

Foveon's Rudy Guttosch, left, and Shri Ramaswami with a print of a Sigma dp photo in their offices in San Jose, September 15th, 2016. Taken with a Sigma dp3 Quattro.

[Note to readers: Thanks to the recent re-design of the Barron's site, this post from September of last year was lost from the blog. I recently republished the post.]

“There are only four companies in the world who can do this, Canon, Sony, Fuji, and us."

I was talking with Shri Ramaswami, general manager for Silicon Valley chip pioneer Foveon. "Doing this," in this case, means making sensors for digital cameras.

Founded in 1997 by legendary computer scientist Carver Mead, Foveon has for years hovered in the back of my mind as something to look into.

Foveon's sign outside its headquarters in San Jose.

Earlier this year, I bought one of their cameras, a “dp 3 Quattro,” made by Foveon's parent company, family-owned Sigma Corp. of Japan.

I was at first intrigued by the stark nature of the thing — no video, no frills, weird shape.

And then as I used it, I became enamored. For anyone who’s spent years now taking pictures mostly with an Apple (AAPL) iPhone, or another smartphone, the Sigma cameras, powered by Foveon’s sensor, are a revelation.

The pictures it takes are exquisite, even for someone like myself who doesn’t know much about photography. Texture is so vivid that as one reviewer has put it, the Foveon sensor seems to capture the feel of the air itself in a scene.

Amazingly, one can achieve such results while holding the camera one-handed, without a tripod, as I did, with the shooting mode set to “shutter-priority” to run the shutter as fast as can be to accommodate for shake:

Ramaswami, looking over some of my amateur images, laughed and said, “you’re probably better than me, I tend not to go handheld; I’m a tripod guy."

Last week, I traveled to San Jose to chat with Ramaswami, and with Foveon vice president Rudy Guttosch.

Ramaswami, a 27-year semiconductor industry veteran, is like a general, strutting about the conference room, vigorously emphasizing every point of a prepared slide deck that he and Guttosch have taken on the road to Europe for various press and industry types. He speaks with rare passion.

"We have a philosophy we have been true to throughout our history as a company, and that is to make a faithful rendition of the world,” he says.

"You could say every airline has quality food, and we can each have different meanings for that when we say it, about this or that airline. What we want is one word that defines what we mean by this faithful rendition. And that is ‘realism’."

"It is all about how realistic can we possibly be. This is what drives us."

Even if “nobody is going to say I see the world exactly as it is,” concedes Ramaswami, “we have gotten closer, and no one has gotten as close as we have."

What are the elements of an image? Natural sharpness, consistent detail, smooth tonality, natural color. Consistent detail, meaning, the sharpness that already exists in the scene, rather than through software that never gets it right. Tonality, meaning, things are soft, things are sharp, things are fuzzy. The sensor has to see color the way we see color or no amount of manipulation will ever get it right. If you compare the way color spectra are seen by our eye with the Foveon sensor, they are very close. It's not just about beating you over the head with detail, but about nuance. There's this element of presence. You should feel as if you're in front of the live performance, in the same way that really clear sound reproduction gives you a feeling of being at the concert. I'm trying to mimic as closely as possible what the human eye is doing. We are trying to compress all the images we take as we scan the scene, in one shot, even though this is impossible because the optics don't exactly match the eye. We are as good as anyone can possibly get, it cannot be better than this.

The battle for Foveon has always been against “medium format,” the industry standard for high-resolution photography, made by cameras costing several thousands of dollars. Sigma’s Foveon-based "dp Quattro" series costs $700, while a new line, the “sd Quattro,” costs $800. (DPReview has a good timeline of Sigma-Foveon cameras.)

Foveon's sensors are used in Sigma's "Merrill" camera series, at left, and one of the newer "DP" cameras on the right.

Ramaswami rolled out for me on a conference table some prints of photos taken with the sd and with a medium-format camera from Hasselblad in the range of $10,000. "If you really want to see the world, you have to have a large print. A small print is like looking through a peephole."

I had to say, they seemed quite comparable, and at points at the edges, it seemed the Foveon sensor even had the advantage while the Hasselblad blurred.

5-foot by 7.5-foot prints of a Hasselblad image on the left and a Sigma sd Quattro image on the right.

The comparison on large paper was not simply to impress me, apparently. Ramaswami argues that “we are limited today by what our images can be viewed on; there isn't a device today that can reflect all of the range that we are capturing in our images."

The human eye, he explains, only really sees 6.5 bits of information, in terms of "perceivable depth in luma, or brightness” that it sees in a still photography or a video, when the image being looked at is smaller than the human field of view.

Because of that, explains Ramaswami, most of the ways we look at pictures, short of a large print, including our monitors and our smartphone screens, aren’t really showing the color a Foveon sensor can capture:

In other words, if we look at a very large print, our eyes can locally accommodate, and we can perceive more tonal depth. So the need to do large bit depth output is two-fold. If we print large, and for color tonality. The smooth variation between hues requires very large gamut (much more than current displays or printers, although they're improving) and also more bits to represent each hue.

How this quality adds up in bits and bytes is a matter of debate. Foveon’s sensor, known in industry as an “APS-C” class, is a smaller sensor than medium format's, which is itself larger than the "full frame" sensor in the standard DSLR, the equivalent of 35-millimeter film.

On paper, the Foveon’s APS-C has a 19.5-million-pixel resolution. The Hasselblad in this comparison has 39 megapixels. Sigma has often said that its cameras are the equivalent of 39-megapixel images, despite the lower count, which has led some to scratch their heads and roll their eyes.

Ramaswami explains that medium format and other cameras using “Bayer” sensors are flawed because they guess about what they’re doing. "Half the time, you're going to be wrong,” he says of such interpolation.

So, the competition's 39 million pixels are only about half right:

The one-half factor is in the luma, or brightness, information, which is missing in half the pixels in a Bayer sensor. Interpolation simply cannot create something out of nothing; it's an educated guess that works for certain cases and not for others. For example, nature is full of random textures, and capturing that is essential to the natural look of a photograph. But it's mathematically impossible for interpolation to guess random information. So, everything looks artificial to some degree. In color information, it's one fourth. So, we are conservatively saying that our pixels are equal to twice that number of Bayer sensor pixels. Hence, the 19.5-million-pixel X3 sensor is equivalent to the 39-million-pixel Bayer sensor.

"More and more,” though, "people are going to be looking at things with monitors and televisions that have a wider and wider color gamut,” observes Ramaswami, suggesting the display world will some day catch up to what he believes Foveon is achieving.

In addition to the sd, Foveon is coming out with another interchangeable-lens model, the sd-H. Sigma does not yet have a release date for the sd-H. It is possible that some data may come out at this week's Photokina trade show in Cologne.

That camera uses a larger sensor, a so-called “APS-H.” With the larger sensor, Ramaswami believes the sd-H is capable of results comparable to a 51-megapixel Pentax 645Z costing $8,500 for the body alone. It is even possible, in some instances, that the sd-H will be comparable to the Phase One, an 80-megapixel, $50,000 camera system.

Foveon's APS-C sensor, bottom, and APS-H sensor, top.

These comparisons, however, are nuanced. As Ramaswami explains:

We've put the Quattro cameras to the test against 51-megapixel medium format, and it's not possible to tell the difference. While we cannot claim to see any areas in the image where the Quattro is clearly better than the medium format camera, we also cannot honestly see any areas where the MF image is better. So our conclusion can only be that at the native 51-megapixel image size of the medium format camera, Quattro is indistinguishable. Against an 80-megapixel camera, we believe the advantage of the medium format camera starts to show at image sizes larger than 6 feet. Below that, there really isn't a perceivable difference in brightness (luma) detail, although the medium format camera suffers badly from color aliasing (moire) artifacts, showing that it's not really an 80-megapixel sensor. Which is not surprising, given it's a mix of a 40-megapixel luma sensor and a 20-megapixel color sensor.

I had come to Foveon’s offices not just for the mission statement but to ask some questions as well. As a neophyte, I wondered why the dp series was made as a fixed-focal-length lens camera.

The dp series contains four models with four different fixed focal lengths, at 14-millimeter, 28-millmeter, 45-millimeter and 75-millimeter. (Comparison of the specs of the four can be found on Sigma's Web site.) The notion, suggested by some, is that you carry around all four cameras, as Ines Mondon and Mark James Ford propose in their excellent ebook on the dp series.

"It is well know in the industry that prime lenses have much higher overall optical quality than zooms, even though zooms have gotten much better,” Ramaswami explained, using the trade term for fixed-focal-length lenses.

"Think about it,” he adds, "you are paying as much as you would for a prime lens, but you are getting the full camera."

More beguiling to me, again as a neophyte, is why this is a non-interchangeable-lens system.

The lens is built for the sensor, Ramaswami explains, and the two are tested during the camera's assembly, to avoid the photographer’s perpetual chore of re-calibrating the lens, usually done by hooking up the lens to a computer via a USB cradle:

You build a lens for a the sensor you are working with. The important thing with the dp system is that every lens element is aligned to the sensor. Now, micro-focus variation is a known issue, as is the sensor depth inside the DSLR. The higher-end cameras have in-camera focus adjustment. Sigma has done 100% MTF testing in Japan for that highest level of tolerances. I could buy a lens and calibrate that lens myself for my camera, but with dp, Sigma has done all the work for us. And not only will it be perfectly calibrated, it will stay that way. In ten years' time, it will still be perfectly calibrated, whereas with other cameras, as they age, there will be wear and tear and I will have to redo calibrations.

He points out, too, that the Sigma cameras have been designed to eliminate additional ares of shake found in DSLRs:

If you don't have perfect alignment of how the light goes through the lens elements to the sensor it will produce problems. SLRs shake the picture because there is a mirror bouncing around. SLRs lock up the mirror and let the vibration die down, but it's still not as good as a Quattro. Unlike other mirror-less cameras, we don't even have the shutter bouncing around because the lens has a leaf shutter built into it. There is no focal plane shutter.

"So everything is built toward the same goal. The dp is a beautifully engineered machine."

What, I ask, about the long processing times some have grumbled about for the dps, from the time the shutter clicks to the time an image is done?

It’s true, says Ramaswami, but things are getting better with each new Sigma machine, and each spin of the company’s “True” application processor that handles tasks after the sensor:

The True III is way faster than prior versions. There are essential engineering trade offs we are dealing with. Four principle ones are the number of bits per pixel we are pushing; the size of the device; the battery capacity; and the speed. Now, when we went from 12 bits per pixel [with Merrill] to 14 bits per pixel, this is actually quite a huge difference. Merrill took 20 seconds to process and image, now we've got it down to 7 seconds.

The dp is a finicky camera. In particular, it can be a disaster to use at night without a flash. As many have pointed out, the camera has a poor response to higher ISO ranges, basically topping out at 400 ISO. Above that, lots of digital junk starts to show up in the images.

I like to think about the dp3 as a camera that loves best taking pictures of beautiful flora in bright sunlight:

Not always, though. Some low-light, handheld images can have their magical appeal with the dp, if you're very, very patient:

But that mostly feels like getting lucky. Night-time doesn't feel like the right place for Foveon.

Ramaswami nods as I recount this, and says, "it's true, every technology has its drawbacks. It is true, it was not designed for night-time photography. A leopard can't change its spots."

But Foveon "are going to stay with our pluses,” he says, "and try to get better and better."

“We have to work on software techniques and circuit elements to reduce noise,” he says. "In our opinion, [ISO] 800 is acceptable, but we are still not happy with it."

Noise is inherent in the way that you capture the huge color range the X3 achieves. Because I am only capturing a limited amount of color in a Bayer sensor, I'm capturing a limited amount of noise. The more color we capture, the more noise we will capture. Light itself behaves like raindrops. There is a scattering, chaotic activity that will precipitate noise. So most of the noise is coming from light itself, not from the sensor. Every generation, you improve that performance. With Merrill, we improved it by two stops, and then we went another two stops with dp. We continue to push it. In low light, our sensor is sitting there trying to pick up all the colors. That is its nature, to try and pick up something. Part of it is the sensor but part of it is the amplification of the light. Software noise reduction is something we are always working on.

I note that in the race for autofocus, there are many other contenders in the mirror-less world that threaten the Sigma’s in many ways. The recently released Fujifilm “X-t2” is one that offers much-improved auto-focus, to judge by early reports. The dp series, by contrast, is not a camera I would want to bring to a soccer match.

"We have to improve autofocus,” he concedes, noting that sd is the first Foveon machine with a hybrid autofocus system, using both "phase detection" and "contrast detection,” which results, he claims, in much greater speed. I have tried an sd on loan from Sigma, and I can say it does seem a bit faster to catch the action.

"Contrast detection can tend to be finicky,” says Ramaswami. "You will see times when it is hunting for things. We know that accuracy is not enough and we want speed as well."

I also ask about the company's much-maligned "Sigma Photo Pro” software. Foveon’s RAW file format does not work in the most common photo programs, Adobe (ADBE) PhotoShop or Apple’s LightRoom, requiring one to use Photo Pro to do any post-processing. There is no end of grumbling about how slow and tedious this program is — slow to load an image, slow to click on a selection, slow to make changes, slow to save and output the result.

Ramaswami says, “We are listening and we are trying to do our best”:

We test against every other photo program there is, and for the amount of quality it produces, it is the fastest out there. Our resolution is so high, that there are demands on the software there aren't for other RAW formats in other software programs. But, this is something we are actively working to improve. It is something we are very conscious of. We cannot say that we are very satisfied with it. Look, we're users just like you!

So, he feels people's pain? "Absolutely we do!"

Perhaps the 64,000 question, toward the end of this long meeting, is what to do with the world of smartphone cameras. As wonderful as the Foveon sensor is, as great as it might be against a medium format machine, the world is increasingly enamored of its smartphone snapshots.

Two weeks ago at Apple’s unveiling of the iPhone 7, the company talked up new capabilities such as the creation of “bokeh,” the phenomenon in which the background of an image is blurred, to lend greater focus to the foreground. Apple's bokeh is just a software gimmick that will be added when the iPhone gets a forthcoming software update.

The dp series has some of the most wonderful bokeh you could want in a camera:

But as Paul Saffo, a longtime tech observer, told me last week, prior to my meeting with Foveon, the company is fighting a battle some would consider settled. It is fighting to claim the best resolution.

Ramasawmi first points out to me that you're just not going to get anywhere near the same quality from a smartphone, regardless of the nominal resolution:

It's purely an issue of physics. You're not going to get it with a phone, the sensor alone takes up too much space. Look, I have a Motorola phone. It has a 24-megapixel sensor. Fine, it looks good. But it looks good on that little screen. If you look at those images on a computer screen, they look like 2-megapixel images.

And yet, Ramaswami, who himself often carries a bag of four or more cameras, including those of competitors, also concedes that Sigma’s won’t be the camera for everyone. It will always be part of a landscape that has multiple preferences, some very high volume —in the tens of millions of units — some more of a specialist domain:

And sure, that is generally the way we shop for anything. I'm willing to accept it's not the best of everything. We totally get that. There is no device on the planet that does everything well. Car enthusiasts have multiple cars they like. I carry about four cameras with me, including other companies' cameras, one with a super zoom, one that's a giant lens. When I see something that amazes me, I pull out the dp. You can't have one tool that does everything well. We see these as adjuncts to the role that other cameras are going to play.

Unless and until the smartphone completely eliminates higher-pixel sensors, Foveon, and the other three chip makers, will battle it out for claims of ultra-realism.

Facebook shares drop almost 7% on Friday's disclosure of mishandling of user data by the consulting firm Cambridge Analytica, KLA-Tencor buys fellow chip equipment maker Orbotech, Deutsche Bank analysts are bullish on fiber optics name II-VI but negative on Arista Networks, GrubHub stock is getting pricey according to Stifel analyst John Egbert, Fitbit finds a new believer in Craig-Hallum's Alex Fuhrman, ex-chairman Paul Jacobs of Qualcomm is hoping analysts will vote in sympathy with him at the company's shareholder meeting on Friday, Dell topped server sales in Q4 and bumped aside Hewlett Packard Enterprise, Google's Diane Greene is mulling a big acquisition to boost the company's cloud services, Apple is moving forward with its own display technology called "micro LED," and Oracle is set to report results after the closing bell.

"While it's background noise for investors in the near-term, with the News Feed overhaul and other actions that Facebook has implemented over the past few months, its clear with more 'heat in
the kitchen from the Beltway' that further modest changes to their business model around advertising and news feeds/content could be in store over the next 12 to 18 months," analyst Daniel Ives wrote.

Foveon’s Remarkable Quest for Realism

[Note to readers: Thanks to the recent re-design of the Barron's site, this post from September of last year was lost from the blog.

This copy is for your personal, non-commercial use only. Distribution and use of this material are governed by our Subscriber Agreement and by copyright law. For non-personal use or to order multiple copies, please contact Dow Jones Reprints at 1-800-843-0008 or visit www.djreprints.com.