Introduction

This article discusses about some very basic and generalized techniques for detecting image skew. Definitely there are much advanced algorithms to detect skew, but those will not be covered in this article.

So! What is image skew?

Using the above drawn figure as reference, I can say - the theta pointed out by that bluish arrow sign is the image skew .

To put it in proper words, the angle by which the image seems to be deviated from its perceived steady position is the image skew.

Some pre-requisites for detecting the skew

Before detecting the skew, the first step is to differentiate between a text image and a regular image. By text image, I mean images which are scanned documents or screenshot of a text document, or in other words, images that contains letters and texts.

And a regular image in my terms is a picture, or photo of a scenery, or something drawn by Mr. Van Gogh and Co.

In one of my articles, I have covered the topic of a basic algorithm to differentiate between regular image and text images.

However, although the skew detection of a text image is relatively easier to implement, for a regular image, it's quite difficult and next to impossible in some cases. Suppose I have a picture of only the head part of my favourite celebrity Ms. Julia Roberts, ... there is no way to know from the image whether she really has her head slanted towards one direction, or the camera man who took the beautiful picture had a problem with the camera angle.

Therefore, for regular images, we have to make some sort of assumptions. The case we are going to discuss here will assume, the pictures to be deskewed will be framed. Which means, the pictures that we are dealing with are pictures taken from another picture having frames/border around them. You can refer to the application screenshot at the very top as an example. The screenshot you see is of a camera captured image of another picture having a thick white-ish border around it.

The basic concept of detecting the skew

The main idea to detect skew for a regular image and a text image is the same. We have to first turn the pictures to Gray Scale.

Then, for regular (non text) images, we have to find the edges or feature lines of the image. There are many algorithms to detect the edges of an image, and in our case, we used the canny edge detection algorithm. We can wish to skip this part for text images.

The next step and the most important step is to cast rays from one side of the picture to the other, and by using the intersection info of the rays with the various parts of the images, we come up with a good skew angle.

As you can see from the above image, we have first converted the image (the one you see in the application screenshot) to its very basic feature lines/edges and have casted horizontal rays from one side to the other. The rays which we have casted (I mean the horizontal lines) intersect with the border/frame of the image. If I put the intersection points of the rays and the left edge of the picture in an X, Y plot (where X is horizontal and Y is vertical), I will end up with some sort of straight line curve that has a slope angle close to 2 degrees with respect to the Y axis. So, there you have it, your skew angle for the image in the application screenshot is close to 2 degrees or 0.035 radians.

This is more or less the basic concept used for detecting the skew of a regular framed image. You have to keep in mind this idea is applicable only to regular images and not text images.

By the way, the uploaded source code's skew detection algorithm does a little more than what I have explained above. What it does is - cast rays from left to right to intersect with the left edge, then top to bottom, then right to left, and then bottom to top. The algorithm considers the one with the most intersection points, and creating consistently occurring slopes as the edge to consider as the reference edge. For example, if the bottom to top rays intersects most with the bottom edge of the image and more frequently occurring slopes are observed, then that edge has the priority and the points of the bottom to top rays would be considered to be put in the X,Y plot. All these extra effort is just for the fact that, in some images, the border edges might be broken and not as beautifully framed fully around the image, like you are seeing in the application screenshot image. So, we'll have a set of rays with different slopes for such images, because the rays will go beyond the border edge wherever there is a broken edge line, and will create a different angle with another intersected point that lies somewhere inside the image bounded by the frame. So, the most occurring slopes (if two slopes have less than 0.0349 radians / 2.0 degrees difference between them - we'd consider the slopes as same valued) will be taken into consideration to detect the skew.

In order to detect the skew of text images, we don't have to worry too much about the picture having border edges or not. As mentioned before, we cast rays from left to right and find how many blackish pixels the rays intersect with. Then, we rotate the angle of the rays by a small amount and cast the rays again, and do the same process again and again.

The angle at which most white space is encountered (in other words, most black pixels are shot down, i.e., intersected), we consider that to be the skew angle. Most likely, we would start from 0 degree and go down to 90 degrees, and then again, from 0 degree go up to 90 degrees with the angle of the rays. Whenever the white ratio reaches the maximum peak and then starts to come down again, we stop immediately, and we know we have found our skew angle. This is the basic idea of detecting skew for text images.

The deskewing of an image is less of a trouble. All you need to do is rotate the image by the same amount of the skew, only to the reverse direction.

The code for the rotation is available in the ImageFunctions.h header file of the uploaded source.

Using the code

In the uploaded source code, mainly two classes are used for detecting the skew. One class, the main class, is named SkewManager, which has code to detect the skew, and the other class is ImageData, which is just a wrapper for the bitmap bits and also keeps the width and the height of the image to be used by the SkewManager to apply its algorithms to the bitmap bits.

The SkewManager expects the bitmap that will be passed to be gray scaled. The blurring of the image to its edges (if required) is done inside the ImageData class.

Following is the example of the usage of the two classes (the m_ prefixed variables as well as the skewMan variable are member variables of the CScannedDocTestDoc class).

Points of interest

In order to achieve skew detection and deskewing of images, I had to go through a lot of trouble to perform basic image operations such as creating a gray scaled image, rotating an image etc., and I have put all those functionalities inside one header, ImageFunctions.h. Most of it was improvisation on already available materials in the Internet and MSDN. The canny edge detection algorithm can be found in the the header canny_edge.h.

CSharpClient

This is a cut down version of the deskewimage code built to compile into a DLL. A small C# demo application is included in the solution showing how to call the code contained in within the DLL, pass in a file name to be analysed and the skew angle in radians is returned and displayed on the screen.

Acknowledgements

Kathey Marsden, Professor Richard J. Fateman

for the OCRchie project - the foundation upon which, the text image skew detection algorithms of this article's codes are based upon.

Also, thanks to Archie Russell, James Hopkin, and Cynthia Tian, who contributed significantly to the original design.

for providing with reading materials related to deskewing text and framed images.

History

Article uploaded: 24 August, 2010.

Article Updated 29 May 2012 CsharpClient Added

Article Updated 14 February 2013 CSharpClient.zip 1.1 AddedUpdated to compile in Visual Studio 2008 (Professional Edition)Combined into a single VS Solution containing two projects (for easier debugging)Minor Bug FixesAngle of 999 is now returned if the document could not be analysed (not a 8bpp image)Added a Text box to show the name of the file analysed

Share

About the Authors

Your Lord does not like people who waste by extravagance, and only the righteous C s, and does so in increments, and they are of those people who are mindful of what memory and time has in offer and they waste none of it.

Comments and Discussions

Hi Everyone,
Firstly this looks like a great program and very well written. Unfortunately I haven't done much C++ in many years. But I think I should just be able to call the functions I need within this DLL from My C# program without any problems. The provided demo application should give me enough clues how to do that. All I want is to detect the skew angle in documents so I can send staff back to the scanner to do the job properly .

Like others here I have run into various hard to understand error messages but after running sxstrace (sxs=side by side) I have determined that the precompiled DLL included in the zip is compiled against a debug version of the c++ runtime that was included with whatever version of Visual Studio that Mukit used to build the program. As the debug version is never redistributible and I have a different debug runtime in my version of VS 2008 I will need to recompile the DLL.

No problem, all the source is included so I should be able to rebuild in both debug and release mode in my copy of VS 2008. I found a couple of problems where the location of two or three header files were specified using a slighly different path than the actual location of the files. This was easy to correct.

The rebuild now progresses further but I get several warnings and errors. Some I think I can just ignore for now but others look more serious and there are three fatal errors.
I have tried googling these errors but due to my rusty old c++ knowledge and the fact that it was never that good in the first place I don't understand the solutions.

Can anyone here plaese assist with getting the DeskewDLL to compile on VS 2008?

Please try to make sure, the include and library paths are same in release in debug modes. I have a feeling you haven't included necessary paths in your release version.
Try to match the release version with the debug version.

The main difference between debug and release is,
In debug mode, you generate .pdb file and you have _DEBUG as a define.
In release mode, you don't have _DEBUG defined and further more, you have some optimization setting for speed and memory optimization.

I am sorry that I cannot help you too much with the compilation because of my very busy schedule at the moment. However, like I suggested in a previous reply to another question, you can put this project in odesk and ask for a proper build in release which would cost a few dollars.

Again apologies and hope somebody else from this forum can help you solve the issue.

Hi Mukit,
Thanks for your assistance, I completely understand that paid work comes well in front of free stuff. It is very generous of you to post this project and it must have taken a lot of work. Your advice has helped and I now have the test App and DLL compiling error free.

For anyone else who is reading this
I'm Using Visual Studio 2008 on W7 64bit. I compiled and run the application as 32bit.

Things that I modified in the Release Mode Configuration are as follows

3) Still received the following warning:
warning LNK4075: ignoring '/INCREMENTAL' due to '/LTCG' specification DeskewDLL DeskewDLL

It appears that the Linker –> General Optimization -> Enable Incremental Linking = YES (/INCREMENTAL) option is not compatible with Linker -> Optimization -> Link Time Code Generation (/ltcg). As the compiler preferred to ignore the /INCREMENTAL setting I turned it off to correct the warning (/INCREMENTAL:NO)

4) Still Received the warning:
warning C4996: 'fopen': This function or variable may be unsafe. Consider using fopen_s instead. To disable deprecation, use _CRT_SECURE_NO_WARNINGS. See online help for details. c:\projects\bulkdocumenthandling\bulkdocumenthandling\sdk\csharpclient\deskewimage\deskewdll\skewmanager\canny_edge.h 137 DeskewDLL

Hi Mukit,
I did try and upload my adjustments to the source to a new Code Project article but the article was rejected as it was not a new piece of work but only an update to yours.

The moderators recommended I add my work as a new version to your article but for this you would need to add me as an author. (or I can just send it to you and you can add it to the article) If you are not interested that's fine.

Additional changes I have made to the code since my last post are to combine both the C# and C++ projects into a single VS 2008 Solution and to combine the output folders so the Debug and Release folder structures make more sense. This also allows the debugger to step through the code contained in both the C# demo application project and in the C++ DLL project.

Unfortunately I'm stuck again on my way to the solution I’m looking for.
The input files for my project will always be 1 BPP Tiff files. The code now works correctly for 8 BPP Grey Scale Tiffs. Black and White tiff files always return a skew angle of 0. Is there any reason the code can't detect skew in 1 BPP files?

I have been stepping through the code for hours looking at both types of files and can’t understand what the code is doing well enough to pick what is going wrong.

I understand you are busy. Perhaps someone else will spot the reason or offer a suggestion.

Hi Mukit,
That is exactly what I needed to know.
I was thinking about converting my images to 8bpp but thought it was a silly work around if the code could deal with 1bpp. So I'll follow up on that course of action.

I'll reply to the Code Project moderator who sugested makeing me a joint author for details on how to actualy do it.

Hi Mukit,
As the C# code is a demonstration app showing how to call the dll I made a few minor changes so that it worked a bit better.

Changes to the C# CodeMainform.cs
Reset the skewangle text box to show correct status.
Added a filename text box to show the file name being analysed.
Added an error popup message box if the file could not be analysed. Previously 0 angle was displayed the same as an image with zero skew. For a start I have returned 999 to indicate an unspecified error.

All other changes in the code are minor and I commented with //DJP Date and a brief description as follows

Visual Sutdio Solution Changes
The original two solutions containing one project each were combined into a single solution containing two projects. Both projects now use the same Debug folder allowing the debuger to step through the code in both projects while debuging. It also makes sence to combine the Release folders.

I went through the compiler options comparing debugmode and release mode and fixed any inconsistancies.

If you are happy with these changes I'll zip up the project and add it to your article as a new version and not replace your existing files if anyone wants to stick with the previous version.

Thanks for the changes. I appreciate it. I hope you tested after making the changes.
But the current uploaded sources I see now doesn't have the c# version. I downloaded it and saw there were no c# files

So, if your latest changes have the c# files included, plz upload it at your convenience.

I think adding up as a new version is a better idea, since some people may have used the older versions and might like to refer to them again.

I have just gone into the article update screen now and I can see your two original files there.
When I go into the Standard public article page indeed the C# zip file is missing.
Very Strange as I haven't changed anything yet.
David

Hello Mukit,
I have just updated the article and included the new CSharpClient1.1zip file.
I added a brief description of what the CShaprClient does and a bit in this history section to show the updates and when it was added to the project.

The download link on top of the page was missing for the original zip file so I fixed it up.
The article is currently shown as "Pending" so I guess smoeone will aprove my changes shortly.

I expect you should be able to see my changes immidiately
If you arn't happy with what I have added feel free to change any and all of it