If this is your first visit, you may have to register
before you can post: click the register link above to proceed. To start viewing messages,
select the forum that you want to visit from the selection below.

Announcement

Collapse

No announcement yet.

Tutorial "How to create a 3D panorama with a single camera and a fisheye lens"

3D images (also called stereo images) are not new, they exist for over 100 year, but 3D spherical panos are new and unknown to a lot of people.
In this tutorial I will explain how to shoot and process a 3D spherical pano with the use of a DSLR camera, a fisheye lens, a Nodal Ninja panohead with a 24 or 36 clickstop rotator, a stable tripod, PTGui Pro, the free Windows application StereoPhoto Maker (works great in OSX with Parallels or Wine) and an anaglyph viewer. The tutorial contains all steps of the workflow and at the end there are links to an example and screenshots of the processing.

But first some theory.
You already know that the brain is getting different data from the left and right eyes, the data is interpreted by the visual system of the brain and presented to the other parts of the brain as a 3D image with "depth".
We can see depth because the eyes are delivering images that are not the same and can have a lot of parallax.
It is the same parallax that we try to avoid as much as possible when we take our panos.
We will use this knowledge that parallax is needed for depth to create 3D panos.
Getting the parallax we need is very simple, all we have to do is to shift the lens forwards in an offset NPP position.

To get a better understanding about "how it works" we will follow 2 imaginary objects that are both positioned somewhere in the middle of the right half of the image, so at approx. 75% of the width of the image.
One object is placed in the foreground of the scene, close to the camera, and the other object is placed in the far distance, in the background of the scene.
Now we rotate the panohead with one clickstop (=10 degree for a 36 degree detentring).

Due to the small angle of rotation the parallax error between the objects will be small so we continue rotating with more clickstops and while doing this the objects are "moving" with each clickstop from the right part to the left part of the image and after 6 - 9 clickstops (the number is depending on the focal lenght of your lens) both objects are in the center of the left half of the image at approx. 25% of the image width.
Obviously by turning the panohead we "moved" the objects from the right to the left side of the image.

The parallax between the objects in the first shot (at 75%) and the last shot shot (at 25%) is huge and it is this huge parallax that we will use to create a 3D pano.
For this we will use StereoPhoto Maker (SPM), this app will take small strips of the left and the right half of all images (at 25% and 75% of the image width) and connect all "left" strips to make a "right eye" pano and connects the "right" strips for the "left eye" pano. Note that the left/right positions of the strips and the panos are crossed.

With SPM we can output the "left eye" and "right eye" pano in several ways, as a combined anaglyph, for crossed eye viewing, side by side, etc.

For this tutorial I will output to anaglyph to make a "full screen" pano. The other output options you can explore for yourself.
I used a Nodal Ninja 5 with a 36 stop detentring but it is also possible to use a Nodal Ninja 3 of 5 with a R-D16 rotator.
If you have a Nodal Ninja MKII you can use it with a 24 stop detentring instead but then you have to limit the forward out of NPP shift of the lens to 40 mm as described in step 2 below.
When using a 24 stop detentring you have to change "36" in "24" and "10" in "15" wherever you see "36" and "10" numbers in the steps below.

The workflow in 14 steps:

1) Set your rotator to 36 clickstops. Make sure that the free play of the rotator is less then 0.1 degree. If your rotator has a lot of free play then you can't use it with the method I describe in this tutorial.
Choose a test scene that is suited, take care that there are no objects in the scene closer then 1 m to the lens.
Only when there is very little wind outside, or better no wind at all, you can shoot outdoors, otherwise shoot indoors.
It is important that their is no variation in the light otherwise you will get a visible seem at the 360 degee border.
It is also important that you don't shoot in a dynamic scene otherwise you get lots of errors, you really need a "frozen" scene.
Mount the panohead on a sturdy tripod and level the panohead.

2) Set the upper rail in the horizontal position (zero tilt) and shift your lens forwards on the upper rail.
When shooting in a small room with objects relatively close by then shift approx. 30 mm forwards, when shooting in a larger room then shift approx 45 mm forwards and when shooting outside then shift approx. 60 mm forwards (see also the formula in the addendum of this posting).

For most cameras and lenses the maximum possible shift forwards is around 60-70mm, when turning the camera mounting adapter 180 degree you can gain some extra shift.

3) Set the camera in M mode for a fixed exposure of all images, set the white balance to a fixed setting and the focus of the lens to Manual with a fixed distance.
Shoot all 36 images around and take care that you don't move the gear by accident, you really need a stable tripod and a cable or wireless shutter release to avoid displacement of the tripod and variations in tilt and roll.

4) In PTGui Pro you load all images and apply a lens calibrated template.
In the Image Parameters tab you set a fixed increment for Yaw of 10 degree for all images. It is very important that your rotator is precise because you will not optimize the Yaw of any image.
Let PTGui automaticly place the CP's.
Make sure that you are using the Advanced mode in PTGui to get full control over the optimizing.
In the Optimizer tab you enable the linking for Roll and Pitch for all images, this is to make sure that if there are any images with wrong placed CP's that those images will be aligned properly despite their wrong placed CP's.
Enable optimizing Roll and Pitch for all images and start optimizing.
For sure you will get much larger CP errors then usual but that is normal.
You then optimize for the lens parameters D and E and optimize again.
Delete all CP's with errors larger then 10 px and optimize as many times as needed to get all CP's errors just below 10 px while keeping an eye on the Pitch and the lens Shift params.
If the values for D and E differ more then approx. 20 px. from the normal values for your lens then set them back to this normal value.
Also check the value of the Pitch, when it differ more then 1 degree compared to the normal Pitch value then set it back for all images to the normal value and optimize again.
There is a tendency that optimizing will increase the values for the lens shift D and E and the pitch parameters so you have to take care that they don't go wild.
When this happens you have to go to the Control Points tab and check if there are any CP's in the scene that are close to the setup. In an ideal situation you will only have CP's around the horizon of the scene.
When the CP max is below 10 px and the lens params D and E and the Pitch of the images are more or less normal then the optimizing is done.
Now set the Yaw of all images to an increment of 0 (zero) degree and stop optimizing.
Congratulations, when you have come to here you have finished the hardest part.

5) Open the Panorama Editor window, all images are stacked on top of each other, this is because the Yaw of all images is 0 degree.
Now reduce the width of the pano with the slider at the bottom of the panorama from 360 degree to the value that is needed to get a nice cropped image, this will be 90-180 degree, depending on the focal length of your lens and the sensor size of your camera.

6) Go to the Create Panorama tab.
Make sure that the Link width and height option is enabled.
Enter the height for a normal full screen pano.
Because the width and heigth are linked the width will be calculated by PTGui.
Make a note of the width, you need it later.
Now some values must be calculated, the corrected height and the shift (you need the value for the shift in step 10).
To show you how the math is done I will use the values of the tutorial example.

Replace the normal height of the pano by the corrected height you calculated, the width will change to a wrong value.
To correct the width first disable the Link and width option and then replace the wrong value of the width by the value you noted earlier.
It is very important that you keep the order of the entering of the values exactly as described above otherwise the height of the output is not 180 degree !
To make sure that you didn't make a mistake go to the Panorama Settings tab and check that the vertical height is indeed 180 degree. If it is not 180 degree then repeat step 6 (all except for the math).

7) Go back to the Create Panorama tab and set the output format to JPEG and Individual layers only.
Hit the Create Panorama button, ignore the message that you didn't optimize after changing CP's and after some time you will end up with 36 cropped equirectangular images.

8) The next steps are all done in StereoPhoto Maker (http://stereo.jpn.org/eng/stphmkr/)
Choose the menu option "Make slide show list..." from the File menu, check the option "For panorama", browse to the location of your cropped equirectangulars, enter a name for the list, press "Add all files" and let SPM create the list, you will get a confirmation, press "OK" and close the "Make Slde Show List" window.

9) Choose the menu option "Open slide show list" from the File menu and SPM will directly show the location of the slide show list file you just created.
Uncheck the " Auto start Slide Show" option, check the option "Color Anaglyph" and press "Open".

10) In the image window you see the first of the 2 cropped equirectangular images on top of each other.
Now they need to be shifted to a position that all objects are in line.
With the file menu "Easy adjust" you get a window with a slider on top of your images. Use the slider and the arrow buttons at the left and right side of the slider to precisely set the calculated shift of step 6 and press the OK button in the Easy adjust window.
It is very well possible that some objects are not in line with eachother when you have set the calculated shift but this is normal.
Press the keys "Shift" and "Y" simultaneously and now the shift of all images is set and the slide show list is updated and saved.

11) Choose the menu option "Mosaic images" from the File menu. Choose from the popup menu "Files of type" the option "Slide show list" and select the slide show list.
Check the options "Crop" and "Symmetrical Cropping (for Stereo pano)".
Now you enter the values needed for taking the small strips out of the images.
Enter in the first box a value equal of 25% of the width of your images, in the second box you enter 0 (zero), in the third you enter nothing (the app will fill in this value, based on the data in the slideshow list) and in the fourth box you enter the height of the images.

12) After a short time the strips are connected into a left and a right eye equirectangular and the results will be showed in the image window.
Choose from the View menu the option "Panorama Mode (360 degree)" and the option "Fit panorama height to screen".
If you see the left and right stereo images side by side then use the menu Stereo &gt; Color Anaglyph &gt; color(red/cyan) to combine them into an anaglyph image.
As you can see the stereo seperation is way to much so both images must be shifted closer to eachother.
With the Easy Adjustment menu option you align the close by objects in the foreground by shifting the images in the horizontal direction, after zooming in to 50-100% you can check and fine tune the alignment of the images with the left and right arrow keys.

Put on your anaglyph viewer and check the depth.
When you are satisfied about the result you can save the 3D pano, if not then you can output new left and right panos by setting a different value in the first Crop box (step 11).
With a smaller value in the first box the left and right strips are taken out from a position with more parallax in the images and with a larger value the parallax will be smaller.
Tip: don't put to many parallax in the 3D images, it is better to use to little then to much parallax.

13) You can save the anaglyph with the "File" menu option "Save Stereo Image ...", you will get several options how you want to save the 3D image.
The app will also save the left and right pano.

14) Convert the equirectangular anaglyph to an interactive pano just as you do with any pano.
Put it full screen and enjoy your self made interactive 3D pano.

In this tutorial I described the use of StereoPhoto Maker to make and connect the strips for the left and right equirectangulars.
It is also possible to blend strips with PTGui but then you have to cut out the strips, taken from the left and the right part of the equirectangular images, with Photoshop.
For blending the width of the strips must be much wider then when they are connected because you need an overlap for the blending. The blending of the strips is done in PTGui (without warping them).
Combining the blended equirectangulars into an anaglyph can be done with SPM.
Making blended (instead of connected) left and right equirectangulars is much more work while the quality is hardly any better.

Whatever method you use or prefer, connecting or blending, it all comes down to the same method, take parts of the left half of the image, take parts of the right half of the image, make 2 panos of the parts, combine them and you will have a 3D pano with depth.

In the first version of this tutorial (version 01-08-2009) you had to resize the anaglyph equirectangular to get the proper 2:1 ratio, in this version (19-03-2010) this is no longer neeeded
The last screenshot in the series of the processing is therefore obsolete.

Here is a formula to calculate the number of images that are needed for 3D with a one camera system, the formula is not not based on any science but as a rule of thumb it can help you to calculate how many images are needed in relation with the forward out of NPP shift of the lens.

Number of images needed: A x (B + forward shift out of NPP) x C / size of stitched pano

A - reference number of images for a shoot in NPP=4
B - reference forward shift out of NPP (cm)=1
C - reference number of pixels of stitched pano (mpx)=20

To make clear how to use the formula here is a table with the calculations for a pano of 7000x3500 px (=25 mpx)

As you can see with 36 images around it is best to limit the forward shift to approx. 6 cm and with 24 images around to approx. 4 cm.
---------------------------------------------------------------------------------

Please no copying/pasting of the complete text of this tutorial on other forums, it makes it impossible for other people to read the replies, clarifications and updates.
It is much better to post a link to the tutorial like this:

Re: Tutorial "How to create a 3D panorama with a single camera and a fisheye lens"

thanks for your tutoril

im just a bid confused! i did try to make 3d panoramas but i make 2 panoramas, one left, one rigth , so far i understand you just make 36 photos and shift your lens forwards,
i did shift the x-axis 35mm to left for one panorama and 35mm rigth for other pano.
here you end up with 36 images after stich in ptgui, than you go with this images to StereoPhoto Maker to finish the anagyph? so you just make one set of images?

sorry just a bid confused

Comment

Re: Tutorial "How to create a 3D panorama with a single camera and a fisheye len

Hello Jose,

When using the method I described you don&#039;t shift the lens sideways, so you don&#039;t move the vertical arm of the NN panohead from the position that you use for a normal pano.
All you do is shift the lens forwards a few centimeters.

In PTGui Pro you don&#039;t stitch the images ! and you don&#039;t blend them !
After the last optimizing stage you set the Yaw for each image to zero (0) and from there you can&#039;t stitch them anymore and that is OK.
By choosing the output option"Individual layers only" (so without blending !) you get a cropped equirectangular for each image.
So indeed you end in PTGui Pro with a set of 36 cropped equirectangulars.

In the next fase SPM will handle the stripping and the connection of the cropped equirectangulars .
SPM will make a left and a right pano and combine those panos into an anaglyph.

Take a look at the screenshots and I am sure that it will be clear to you.

Success,

Wim

Comment

Re: Tutorial "How to create a 3D panorama with a single camera and a fisheye lens"

Hello Wim,

Thank you for sharing this technique with us.

I have one question please; in the Tutorial you refer to using a 36 stops detentring, but the maximum I have is a 24 stops detentring with my NN3 II and I cannot find reference to a 36 stops detentring on the Nodal Ninja web site (perhaps Nick could advise). Do you think that the technique would work with 15¬? intervals instead of 10¬? with some adjustment to the arithmetic?

I am using a Nikon 10.5mm lens with an APS-C sensor so only getting 180¬? across the diagonal, but get good ‚Äúordinary‚Äù 360¬? panoramas by using the lens with two tiers at -10¬? and +50¬?, although I have kept it horizontal for my first attempt at 3D Panoramas. I can get all the way through to the anaglyph panorama, but the individual images are not blending properly although I have tried to manipulate the arithmetic.

Comment

Re: Tutorial "How to create a 3D panorama with a single camera and a fisheye lens"

Hello Wim,

I have experimented a bit more and the answer is yes, it is possible with a 24 stops detentring.

I am probably not doing it properly, but my problem was with the last line in step , ‚ÄúPress the keys "Shift" and "Y" simultaneously and now the shift of all images is set and the slide show list is updated and saved.‚Äù
The Slide Show List is not updating when I press "Shift" and "Y" simultaneously , but I edited it manually and put the Horizontal Shift in the fourth field and now all is well.

Comment

Re: Tutorial "How to create a 3D panorama with a single camera and a fisheye len

Hello Hugh,

Good to hear that you could make a 3D pano with 24 images.
Chapeau !

You are right, it is possible to use less then 36 images, sometimes 24 will do and I even made one with 18 images around but the trick is not to get to close to objects (keep a distance of at least 2 meter).
Many times there are annoying errors with fewer images so to be on the safe I advice 36 images if possible.
I changed step 1 of the tutorial with additional instructions what to do when there is no 36 but a 24 detentring available.

I am probably not doing it properly, but my problem was with the last line in step , ‚ÄúPress the keys "Shift" and "Y" simultaneously and now the shift of all images is set and the slide show list is updated and saved.‚Äù
The Slide Show List is not updating when I press "Shift" and "Y" simultaneously , but I edited it manually and put the Horizontal Shift in the fourth field and now all is well.

Thanks for bringing this important issue under my attention, I did a check of step 8 and if you don&#039;t use the OK button after setting the image shift in the Easy Adjustment window, and instead just close the window, then indeed nothing is changed in the SlideShowList file when using the Shift and Y keys.
I changed step 8 of the tutorial with an additional "Ok" instruction.

Anyway I am glad you solved both issues and that it didn&#039;t stop you from success.
BTW, any chance that we can see the result ?

Best,
Wim

Comment

Re: Tutorial "How to create a 3D panorama with a single camera and a fisheye lens"

Thanks Wim,

I think I was pressing ‚ÄúShift‚Äù and ‚ÄúY‚Äù before pressing OK. I‚Äôll have another look this evening to improve my skills with StereoPhoto Maker.
I will send you my first attempt at 3D Panoramas although the result is not suitable for public release! I was concentrating on the techniques and did not pay enough attention to the photography itself and did not take enough notice of step 3 and omitted to set the camera to manual so the exposures are not good, but I have learned the technique and produced a result that now spurs me on to improve.

Comment

Re: Tutorial "How to create a 3D panorama with a single camera and a fisheye lens"

Hello Wim,

A long time ago I tried a very similar way to do this job of creating a 3D panorama.
The result was not satisfying my needs and i let it away.

Now, i tryed your tutorial but i¬¥m usin a 24 stops detent ring.
I got some faced (cuted) strips on the panorama (curves in steps... hope you know what i¬¥m talking about). I looked your resulting pano and did not find the strips.
How did you make to have smooth transitions between them?

Comment

Re: Tutorial "How to create a 3D panorama with a single camera and a fisheye len

Hello Daniel,

Good to read that you are trying to make 3D panos again.

About the transition problem, without an example it is hard to judge what exactly is the reason for the not satisfying results.
My best guess is that the images are not properly optimized, it must be possible to get a good transition between the strips in a large area around the horizon.
Step 4 in the tutorial explains the optimizing process in detail. Normally it is not needed but in your case it is best to manually check while optimizing that there are no CP&#039;s set close to your setup.
It is best to delete all CP&#039;s that are set closer then 4-5 meter from your setup in the scene.
Another posibility for the transistion problem is that there are object(s) close by in the scene, if this is the case you must redo the shoot.
With 24 images around you can not have objects closer then 2 meter from your setup otherwise you will get nasty errors.

The rotator of your panohead must be pretty precise, a max difference of 0.1 degree in yaw is OK but any more and errors will be visible.
That said, even with the most precise rotator you will always get small errors close to zenith and nadir, only with a huge number of images you can get rid of the transition jumps in that part of the pano but even with 24 images you must can get a decent result in a large area around the horizon of the pano.

Comment

Re: Tutorial "How to create a 3D panorama with a single camera and a fisheye lens"

Hi Wim,

Nice to have your reply.

Here are some steps that let me a little confused yet:

Step 4) In PTGui Pro you load all the images, apply a calibrated lens template and set the Yaw for all images to a fixed increment of 10 degree.
Let PTGui set the CP&#039;s and then optimize for Pitch and Roll with both parameters linked. You don&#039;t optimize for YAW !

When you talk about the Calibrated lens template, I¬¥ve loaded a pano already stitched and saved the lens calibation. Is that right or is there another way to get the correct lens calibration? I use a sigma 8mm Ex DG with Canon T1i (and NN3 MK2).

In my case, i know i should set the yaw increment of 15 degrees (24 pictures).

How to let PTGui set the CP&#039;s? Using the Align Button at the main screen?

You then optimize for all lens parameters, check the values for D and E and if they differ more then approx. 20 px. from the usual value then set them back to this value.

Where can I find the D and E values? Are those found at the lens Settings tab?

Also check the value of the Pitch, when it differ more then the usual value minus 1 degree then adjust it to this value and optimize again.

Well, assuming that i inserted the correct lens database, the pitch that is set there can¬¥t vary -1 degree.... is that right?

When the CP max is below 10 px the optimizing is done.

In my case, you said that i should delete alll CP that i see after the 4m distance.
I usually set my CP&#039;s around 1.0. No problem to let them at 10px? Is it needed to delete the distant (in meters) CP&#039;s?

Well, there a lot of questions. I didn&#039;t have time to make new tests. I noticed that i put the camera back in the horizontal position of the NNP, and you put the camera ahead (in front). (that was the way i used in my tests).

Comment

Re: Tutorial &quot;How to create a 3D panorama with a single camera and a fisheye len

Hello Daniel,

I shall try to answer your questions.
Be sure you use the Pro version of PTGui (you can't link the Pitch and Roll params in the standard version and linking is needed for the optimizing).

> When you talk about the Calibrated lens template, I¬¥ve loaded a pano already stitched and saved the lens calibation. Is that right or is there another way to get the correct lens calibration? I use a sigma 8mm Ex DG with Canon T1i (and NN3 MK2).

That is right, except for the d and e params all other lens params are used.
The best way is to re-use and modify another template with the proper lens params, just add all images to it.
This way you are sure that the size and the position of the cropcircle is also set properly.
Keep in mind that the FOV of the lens is related to the size of the cropcircle .

> How to let PTGui set the CP's? Using the Align Button at the main screen?

You need to switch PTGui Pro from the Simple mode to the Advanced mode, you do this on the Project assistant tab.
In the menu Control points there is an option to let PTGui automaticly create the CP's.

> Where can I find the D and E values? Are those found at the lens Settings tab?

When you switch to the Advanced mode the lens params will be visible in the Optimizer tab, in this tab you disable all lens params except d and e.

> Also check the value of the Pitch, when it differ more then the usual value minus 1 degree then adjust it to this value and optimize again.
> Well, assuming that i inserted the correct lens database, the pitch that is set there can¬¥t vary -1 degree.... is that right?

No, when you optimize you will find out that the huge NPP offset is having a big impact on the pitch and while in fact the lens is set approx. horizontal (o degree tilt) it is possible that PTGui will calculate a pitch that is wrong.
If it happens the wrong calculation is caused by CP's that are set close to the setup in the foreground of the scene. That is why you can't optimize in one step and need to remove all CP's that have big errors and then optimize again but before you do this you have to reset the tilt to the start (zero degree) setting.

> When the CP max is below 10 px the optimizing is done.
> In my case, you said that i should delete alll CP that i see after the 4m distance.
> I usually set my CP's around 1.0. No problem to let them at 10px?

Please don't forget that you stitch with images that are shot far out of NPP, for sure there will be some CP's with very little errors but they will be set around the horizon in the background, all CP's that are set between the background and the foreground will suffer from the NPP offset and this will result in much larger then usual CP error numbers.

> Is it needed to delete the distant (in meters) CP's?

You only need to delete CP's that are placed in the foreground close to the setup, you check this in the Control Points tab, delete all other CP's with big errors in the Control Point table.

Success,
Wim

Comment

Re: Tutorial "How to create a 3D panorama with a single camera and a fisheye lens"

Hello Daniel,

I managed to get an acceptable result with the 24 detent ring as this is the largets my NN3 has.
I did have quite a lot of difficulties to overcome, but Wim&#039;s tuturial did see me through.
My comments were in a new topic:
"Success with creating a 3D panorama with a single camera and fisheye lens"
because this was suggested to me by Bill.
I think this is now the third topic down on the second page.

My first attemp had vertical banding because I did not follow Wim&#039;s instructions closly enough and let the camera chose the exposure.
Once I locked down the exposure and white balance the images blended OK.

I used PTGui Pro, but had to resort to a lot of additional work to get success.

Comment

I was wondering about the 36 pictures single row that you have taken.
what lens are you shooting with and how did you determine the 36 pictures single row?
What happens it you are shooting with a 24mm lens, single row wouldn't be an option then?