Following the poll launched by hank, let's start to study this future feature.There's 2 points here :- Improvements of SIFT cp detector : this is under heavy development and we already find a way to do that ! Now, we miss one stuff : cases studies. We have some two pictures pairs that doesn't give anything with the current system and that is doing great with the next one, but we need more such hard to stitch 2 images pairs. So if you have some, let's share them !- Manual CP editor : what do you want ? A totally manual system like in PtGui ? Something with a little helper when positionning CP ?

I don't want to manually compete with SIFT when a large number of CP per link (e.g. 50 CPs) is involved:- I would like an easier way to instruct APP and/or SIFT to place more CPs here (e.g. skyline) and to remove any CPs there (e.g. moving clouds, tree branches on a windy day, parallax errors prone locations, etc)- I would like an easier way to delete "long links" (e.g. corner to corner link on fisheye images or very large number of links when using braketing.)- I would like an easier way to move unlinked (or "miss-linked") pictures manually.

Those matters having been settled, then, if it's possible to reduce the number of CP under 10 per link or possible to implement "heavy weighted manual CP"...___________

So far I've had no problems with control points, placement or detection. But I'd like to second the request "an easier way to delete long links". Similar to the "keep only CPs with RMS higher than...". We've already had a discussion on this, where GURL was talking about angular distance, etc.

I'd rather have a threaded SmartBlend or even the new APP internal smart anti-ghosting blender (dubbed by me MagicBlend) but that's another topic altogether.

I'd like an auto way to generate CPs based on contrast or edge finding, rather than SIFT. That is, automatically add CPs based on some algorithm closer to human vision.

I like GURLs idea about keeping CPs off clouds and putting them on the horizon, one way to implement it is to show all the CPs on the main window when in the CP editor (we'd need an option "show links or show CPs".) And I'd want to be able to delete CPs on that main window so that its easier to delete CPs on clouds, water, etc. An add function could add CPs on a user-specified area (draw the box, hit the add CP button) to put CPs where you want them. -- A way to automatically remove CPs from clouds, waves and moving objects is probably more of a dream than a reality.

Another option would be for APP to give a better indication of "goodness" of the link. This can be done by taking two pictures, finding the overlap based on CPs, and doing some math on the pixels in that overlaped part. The math may be to subtract one pixel from its corresponding pixel in the other region for all the pixels, then find the average of the differences of all the pixels, and give an RMS value of all the differences from the average. APP could give the user an option to delete links where the overlap correspondence is less good. This could also shown visually by finding subtracting each pixel from its corresponding pixel to give a picture of the differences in the region, then color-coding the variation from the average difference for each pixel. This resulting map could be used as a transparent overlay on the pictures to show what items don't line up.

Last edited by hankkarl on Tue Nov 06, 2007 3:50 pm, edited 1 time in total.

I agree that a feature to manually place a CP in an exact spot would be useful sometimes.

Also what would be great would be an option to prioritise the correct alignment of 'narrow & long' areas. Let me elaborate:This would be very usefull for panos with cables, tiles, wooden beams, wall edges, girders, etc. APP would add extra priority to CPs that are placed along long but thin areas, like those examples, and do its best to align those areas between shots since misalignments between those parts would be most visible.

1. It should be improved so it can detect the low contrast patterns2. It should have some kind of automatic CP placement when adding CP manually like ptgui3.We have to manually advance to next images by manually selecting 2 images cant we have "next" button so it brings the next 2 images in the windows.4. separate Cancel geometric analysis button (as of now it takes quite long to cancel)5. Hugin kind of manual CP would be good.

Most often the parts of a panorama that are incorrectly aligned, and parts where incorrect alignment is most visible, are edges. I was wondering about using some form of edge detection to help properly align images. I see two advantages to using this:1- some edge detecting algorithm, or simply a blur/contrast combination like in photoshop/gimp, find edges. Then the SIFT algorithm places a high priority on all CPs located along these edge lines.2- the edge lines can serve as a guide as to whether the images were correctly aligned. It finds a common edge pattern in the two images and then, after stitching the two images and deciding at which exact point smartblend will chop off each image's overlap, it checks if this pattern has any sudden 'interruptions', if it has been correctly overlayed in common parts. If the edge line does suddenly stop in the stitched image but does NOT stop at this point in the source image, then a bad stitch alarm is raised.

It all is nice and easy here, and I have no experience in these kinds of things, but perhaps the idea could be used to solve some frequent problems?

Some sample screenshots of how this might work:1 - the original image. I cut it into two images with overlap.2 - an example of how APP could have detected CPs.3 - the results of these CPs gives a bad stitch, especially visible in the cables.4 - APP checks edges if there are any sudden interruptions. It finds several broken edges in places where they were not broken in the original left source photo. A bad stitch alarm is raised, 5 - APP gives priority to these places,6 - and tries to find some CPs in these broken edge areas.7 - reoptimized, and now the stitch is good.

Last edited by DrSlony on Sat Jul 05, 2008 11:10 pm, edited 1 time in total.

DrSlony wrote:I agree that a feature to manually place a CP in an exact spot would be useful sometimes.

Also what would be great would be an option to prioritise the correct alignment of 'narrow & long' areas. Let me elaborate:This would be very usefull for panos with cables, tiles, wooden beams, wall edges, girders, etc. APP would add extra priority to CPs that are placed along long but thin areas, like those examples, and do its best to align those areas between shots since misalignments between those parts would be most visible.

Perhaps APP could keep more of the points near edges with high contrast? (long, narrow features have two edges)

Or SIFT could try and put more points near the edges? Right now, the CP distribution is supposed to be random. But I think we need some CPs in each corner, some in the middle, and most CPs on high contrast edges.

Last edited by hankkarl on Sat Apr 25, 2009 5:07 am, edited 1 time in total.

One note. The internal project on this part has made good advances. We now have 4 kinds of CP and 2 type of descriptors.CP types : 2 of them are edges aware meaning they will locate themself on edges first. 2 others are blob aware like SIFT : they locate themself on uniform area.We now study how to auto switch from one type to another one. For example, is we found out that there is a lot edge aware cp and not much blob cp, then it's probably an architecture picture with a lot of lines and we'll boost the first type to be sure that edge will be continuous. On the other hand, you may not have any edge aware CP meaning you are going to stitch together clouds for example. In this case, SIFT is by far the best.Anyway. This study is really interesting and will really change the software in the future.

any news on the manual control point editing capabilities? The discussion of such a feature has been ongoing since 2007 in this forum. I would vote for similar ones like PTgui provides.

I have major issues with auto-detection of control points in my fisheye images. The stiched panoramas are totally useless. I know Autopano works well on standard lenses but fisheye seems to overburden its capabilities.

superschroeder wrote:any news on the manual control point editing capabilities? The discussion of such a feature has been ongoing since 2007 in this forum. I would vote for similar ones like PTgui provides.

I have major issues with auto-detection of control points in my fisheye images. The stiched panoramas are totally useless. I know Autopano works well on standard lenses but fisheye seems to overburden its capabilities.

Really? - the latest version of APG V2 has done a superb job with every fisheye image set I've tested, all sorts of FE lenses and a whole variety of pano scenes, daytime, nightime, interiors, exteriors...

Show us some screenshots of where it has failed for you with fisheye images.

Last edited by mediavets on Thu May 14, 2009 7:19 pm, edited 1 time in total.

Yes Please add a manual method of adding control points.. why? My work is hand held, linear panorama, using 24mm lens in a high parallax environment.. Auto pano 203 and 147 are fantastic but where there are areas that it can not recognize, I can not add points in. If the focus plane is a little off of the background, such as with a fish floating infront of the background, I need to FORCE control points in.

AlexandreJ wrote:One note. The internal project on this part has made good advances. We now have 4 kinds of CP and 2 type of descriptors.CP types : 2 of them are edges aware meaning they will locate themself on edges first. 2 others are blob aware like SIFT : they locate themself on uniform area.We now study how to auto switch from one type to another one. For example, is we found out that there is a lot edge aware cp and not much blob cp, then it's probably an architecture picture with a lot of lines and we'll boost the first type to be sure that edge will be continuous. On the other hand, you may not have any edge aware CP meaning you are going to stitch together clouds for example. In this case, SIFT is by far the best.Anyway. This study is really interesting and will really change the software in the future.

I think what you need in addition is to associate some error probability value with each CP. For example, to reflect probability that a CP in the big dark sport should not lie few pixels aside. Often, my hand held pictures have slightly different horizon and that I hardly fix by manual placement of CPs even into corners.

Currently there is no way to re-position some CPs by local fine-tuning. For example, in a cloud of "green" CPs there are few in the middle which are "brown". Those were clearly initially misplaced. Let's say I do not want to drop them but definitely APP could shift them few pixels aside within the cloud of CPs.

I am taking images on some interior shots (churches and houses) as my example as I am new to Autopano Giga which I recently registered. The stitches are almost perfect except for quite a few misalligned structural members, which sometimes are not pleasing to the eyes. The proposed manual adding of cp (similar to ptqui) will help me best.

superschroeder wrote:I have major issues with auto-detection of control points in my fisheye images. The stiched panoramas are totally useless. I know Autopano works well on standard lenses but fisheye seems to overburden its capabilities.

I often find spherical panos being much easier to optimise when what I would call "useless links" are removed (manually, alas) next to zenith and less often next to nadir.

I have my own theory about that :

when only a limited area is overlapped between two source images being near the zenith (or encompassing the zenith):1) the CP are often located next to an image top corner, a dubious location for CP2) they all are in a small area...as a result the optimizer is able to reach for this particular link a low (but wrong) RMS by making small changes in yaw, pitch, roll (*) and/or by making small changes in lens distortion correction.

About point (2): when the links disposition is: [1]---a---[2]---b---[3] \________c_______/

as I recently observed in two spherical panos , these "skip over" links (like c) tends to get a much better RMS than links like a and b. In such cases I remove the "skip over" links and as a rule this improve the stitch.

It should be noted that in a spherical pano it's often difficult or impossible to avoid groups of images overlapping each of the other images in this group. This is especially true when no less than 5 fisheyes images in the "top row" (and more than 5 when a rectilinear lens is used) are going past the nadir and reach the other side (dépassent le zénith)

As you can see in the following 24 mm equivalent non-fisheye example, I didn't removed many links but removing a few of them really helped ( there are only 2 zenith shots, though the rectilinear lens I used is not a wide-angle one, because PapySpheric designed the preset this way...)___I removed the remark I first put there

Last edited by GURL on Sun Feb 28, 2010 9:26 pm, edited 1 time in total.

AlexandreJ wrote:One note. The internal project on this part has made good advances. We now have 4 kinds of CP and 2 type of descriptors.CP types : 2 of them are edges aware meaning they will locate themself on edges first. 2 others are blob aware like SIFT : they locate themself on uniform area.We now study how to auto switch from one type to another one. For example, is we found out that there is a lot edge aware cp and not much blob cp, then it's probably an architecture picture with a lot of lines and we'll boost the first type to be sure that edge will be continuous. On the other hand, you may not have any edge aware CP meaning you are going to stitch together clouds for example. In this case, SIFT is by far the best.Anyway. This study is really interesting and will really change the software in the future.

OK, but the two types are not mutually exclusive--you may want to keep some blob CPs that are near an edge CP. etc. It may be better to prefer some CPs that are distributed evenly in the image area taken by the "sweet spot" of the lens, and also prefer some CPs near edges. This doesn't necessarily mean take half the CPs from an even distribution and half from edges.

Also, GURL made some observations:

1. automatically get rid of "skip over" links, or at least use the length of a link to weight it.2. CPs in corners shouldn't count as much. a. Looking at MFT charts, rectilinear lenses still have a circular "sweet spot". b. A mask for rectangular images like the fisheye mask would eliminate the CPs at the bad part of the lens. c. The mask may be a gradient mask, where each CP is weighted depending on its distance from the center of the lens. d. Lens correction may be accounted for, so the mask could be automatic and vary for each lens, and also vary by f/stop used. Or the lens distortion algorithm could be used to weight each CP depending on the amount of distortion at that point.

We also need a better way than RMS to determine the goodness of a pano. it could be simply subtracting one pixel from the overlaping pixel, and giving us both the average and the standard deviation of this for all the pixels in an image. APP could also show this as a picture which shows the difference (in luminance? in RGB?) between each pixel as a greyscale picture. APP could then average the pixels, use that average as black and show the difference as greyscale, so errors become more obvious..

Since most panos have regions with more than two images contributing to a pixel, the math gets harder--but it may be just taking the average values of the pixels, and summing the absolute value of the differences from average.

i hope, the totally manual cp editor like ptgui could be included in the upcoming 2.5 version. I am still getting mis allignment on interiors of houses on v2.0.7. I am using 12mm of the sigma 12-24 lens and having 8+8+nadir+zenith in a 360/180 virtual tour. This is my only problem, everything else is working fine with me. I am using mac os 10.6.3. Thank you.

I also want this, as well as hand control of blending regions. APP is great at automatically assembling many panos, and we all love when it can do this and we don't have to do any work to get a good pano. But the reality is that sometimes it just can't do it. Sometimes due to its own limitations, and sometimes due to limitations of the shots. Hand drawn CPs and hand drawn blends are a pain to do, but they are better than simply not being able to get a good pano, and learning that after coming home from somewhere thousands of miles away!

In particular, there are just times when the human eye can see the overlap clearly, but APP can't see it, no matter how you draw the boxes when telling it to add CPs. I don't know why, and of course it is good to try to make it better so this happens less and less. But as long as it can still happen we need the manual CPs. Sometimes the manual CPs will even be cheating, where the human eye can't see the CP but the brain can -- for example in sea and sky or other moving things.

AlexandreJ wrote:- Manual CP editor : what do you want ? A totally manual system like in PtGui ? Something with a little helper when positionning CP ?

Hmm, beside a better automatic recognition that is *always* desired, i would like to see a manual point editing like in ptgui. The syncing between the two tiles is simply great when manually editing and adding new points. It improves your workflow just incredible.

You can improve automatic recognition as you will, there are always some exceptions left, like regular structures, great uniformly colored areas, that will oppose a good automativ recognition.