Facebook's open source Surround 360 captures 3D-360 video at up to 8K

Facebook has introduced the Surround 360, which captures 3D, 360 degree video using a total of 17 cameras and can output resolutions of up to 8K per eye. Unusually, Facebook will be making both the camera and processing software open source to give developers the opportunity to improve both.

The Surround 360 itself features 17 synchronized cameras: 14 horizontal, a fisheye on top and two more on the bottom. Each camera has a global shutter (which eliminates rolling shutter) and has been designed for long periods of operation without overheating. Raw Bayer data is captured, which is later processed in the stitching software. All 17 cameras are bolted onto an aluminum chassis so everything stays in place.

Facebook says it has used Point Grey industrial cameras in the Surround 360, which hints at the use of Sony 2nd generation Pregius CMOS sensors with global shutters. The lenses used are 7mm F2.4 lenses designed for up to 1"-type sensors, which could even mean the use of the latest Sony IMX253 or IMX255 chips. If that's the case, then these lenses are roughly equivalent to 19mm.

With incredible amounts of data coming from all of those cameras Facebook uses a Linux-based PC with a RAID 5 SSD array that shares the writing out across eight drives simultaneously. The company has made controlling the camera rig easy, via a web-based interface that allows users to adjust shutter speed, exposure, frame rate and gain.

The stitching software uses the concept of optical flow to resolve disparities between what pairs of cameras can see. The company says this method is 'mathematically trickier' than traditional systems, but yields better results. The end results are 3D/360 videos which can be output at 4K, 6K or 8K per eye. Videos can be viewed on Oculus Rift and Gear VR headsets using Facebook's Dynamic Streaming codec. Videos can also be output and shared on Facebook and other websites.

A big part of the Surround 360's story is that Facebook is opening up both the camera blueprints and processing software to developers, stating that 'we know there are ideas we haven't explored' and 'we know from experience that a broader community can move things forward faster than we can.' The company says that the design and code will be on GitHub this summer.

tiny?Not at all. They want and need content for their VR prodcts.They bought Oculus for a reason.They want to creep into our lives now with more and more VR stuff.They want us to shoot and scan everything in our environment. Get as much data as possible.

Think about it. Hypothetical: I can only move my head and I'm paralyzed from the waste down, but i want to see what it is like to be on a mountain. In VR you can get close, I know its not the same, but scoffing at new technology like an old man doesn't help anyone.

Sounds like they want their users to provide 3D scenes to kickstart their 3D VR headset sales... chicken and egg syndrome. be interested to see what the oss community can do with it when it lands on github.

Will be interested to see how Google reacts to this announcement... they already have tons of 3D data von their Streetmap data collection service... could be an interesting set of mashups.

That's actually not bad. Around the price of a Hasselblad, or a decent cinema production camera - and you get lenses with it! And with 8k *per eye* stereo, if I were a film-maker I'd be in the queue for one.

Thanks. I was really apprehensive about photos uploaded there. I'm still not sure what their policy on it is, or if it was just internet scare tactics and posts that weren't based on facts. Eh, I guess I'll take the chance. Thanks for answering my question. If you care to clarify more about their ethos and business practices, I'm curious if I've missed something. Cheers.

Huyzeraside from the fact that facebook essentially, simplified, reserves the right to do with YOUR UPLOADED CONTENT whatever THEY WANT... This is one of many, many reasons why I personally do not have facebook, want nothing to do with them or their social-sahring stuff... I don't twitter, I don't instagram, I don't facebook.... I have my own business website...

In the past few years it seems everybody has discovered that you can stick an array of cameras together for 360 capture. The Surround 360 does not seem to be a particularly clever nor cheap way to construct the multi-camera, and I wouldn't call overnight stitching fast. As far back as 1999, my research group showed two different multi-camera 360-degree rigs on autonomous platforms, with real-time stitching and video wall pan/zoom on a Linux cluster (fed captures via 802.11 wireless), in our research exhibit at IEEE/ACM SC99: http://aggregate.org/EXHIBITS/sc99_360.jpg

That said, we didn't do stereo, and the stereo video processing and encoding is far from trivial. I applaud Facebook freely releasing the design and software for others to use and help improve. I look forward to the actual open source release, which they say will happen "on GitHub this summer."

Then i read the optical flow [thanks DPR for educating us!] and stumbled after the (incomplete) equation set on:

"This is an equation in two unknowns and cannot be solved as such. This is known as the aperture problem of the optical flow algorithms. To find the optical flow another set of equations is needed, given by some additional constraint. All optical flow methods introduce additional conditions for estimating the actual flow."

I surmise that the 3-4x 'redundant' coverage may provide the additional constraint / set of equations, as used in drones and self-driving cars. I also suspect that the extra generous (over)-coverage is sufficient, but not necessary (3D may actually change this). Google has now OPEN competition :-)

Hi guys, kigathi is correct - they need enough overlap to create the 3D ability. Also the more overlap you have the better quality stitching you can get and with this many cameras the stitching will be completely seamless.

Latest in-depth reviews

Canon's EOS R, the company's first full-frame mirrorless camera, impresses us with its image quality and color rendition. But it also comes with quirky ergonomics, uninspiring video features and a number of other shortcomings. Read our full review to see how the EOS R stacks up in today's full-frame mirrorless market.

No Nikon camera we've tested to date balances stills and video capture as well as the Nikon Z7. Though autofocus is less reliable than the D850, Nikon's first full-frame mirrorless gets enough right to earn our recommendation.

Nikon's Coolpix P1000 has moved the zoom needle from 'absurd' to 'ludicrous,' with an equivalent focal length of 24-3000mm. While it's great for lunar and still wildlife photography, we found that it's not suited for much else.

The Nikon Z7 is slated as a mirrorless equivalent to the D850, but it can't subject track with the same reliability as its DSLR counterpart. AF performance is otherwise good, except in low light where hunting can lead to missed shots.

Latest buying guides

What's the best camera for under $500? These entry level cameras should be easy to use, offer good image quality and easily connect with a smartphone for sharing. In this buying guide we've rounded up all the current interchangeable lens cameras costing less than $500 and recommended the best.

Whether you've grown tired of what came with your DSLR, or want to start photographing different subjects, a new lens is probably in order. We've selected our favorite lenses for Sony mirrorlses cameras in several categories to make your decisions easier.

Whether you've grown tired of what came with your DSLR, or want to start photographing different subjects, a new lens is probably in order. We've selected our favorite lenses for Canon DSLRs in several categories to make your decisions easier.

Whether you've grown tired of what came with your DSLR, or want to start photographing different subjects, a new lens is probably in order. We've selected our favorite lenses for Nikon DSLRs in several categories to make your decisions easier.

What’s the best camera for less than $1000? The best cameras for under $1000 should have good ergonomics and controls, great image quality and be capture high-quality video. In this buying guide we’ve rounded up all the current interchangeable lens cameras costing under $1000 and recommended the best.

Canon's EOS R, the company's first full-frame mirrorless camera, impresses us with its image quality and color rendition. But it also comes with quirky ergonomics, uninspiring video features and a number of other shortcomings. Read our full review to see how the EOS R stacks up in today's full-frame mirrorless market.

We spoke to wildfire photographer Stuart Palley about his experiences shooting the recent Woolsey fire, why the Nikon Z7 isn't quite ready to take a permanent spot in his gear bag, and 'that' Tweet from Donald Trump.

The Z7 presented Nikon with a stiff challenge: how to build a mirrorless camera that measures up to its own DSLRs and can deliver a familiar experience to Nikon users. Chris and Jordan tell us whether they think Nikon succeeded.

Nikon has released firmware version 1.02 that resolves a flickering issue when scrolling through images, an ISO limitation problem, and an occasional crash that could occur when displaying certain Raw files.

The Insta360 One X is the company's latest consumer 360-degree camera, supporting 5.7K video, including excellent image stabilization, as well as 18MP photos. And, in our experience, it's a really fun camera to use.