JPEG Lossless Rotation or Flip with Partial MCU

As described on the JPEG Lossless Rotation page, it is not possible to rotate
some images losslessly. Images that have dimensions that are not a multiple of
the MCU dimension (typically 8x8 pixels) will not be rotated or flipped losslessly. This page explains
the reasons why.

JPEG images are constructed with 8x8 tiles

For the background on how an image is represented by the 8x8 tiles, please
see my article on the JPEG Minimum Coded Unit (MCU).
For the purposes of the following discussion, images whose dimensions are not a multiple of the
MCU size will be termed an odd-sized image.

NOTE: For the following discussion, I am describing images that have been compressed without chroma subsampling. Many digital cameras use chroma subsampling to reduce file size at the expense of losing color information (generally 2x1 or 2x2 chrominance subsampling). For these photos, the tiles (MCUs) can be treated as being 16x8 or 16x16 pixels respectively. So, for the purposes of the discussion below, you should treat the required image dimensions as 16 pixels where it reads 8 pixels. Unfortunately, this is a problem that many users of digicams will encounter when they try to rotate their images losslessly.

Handling of odd-sized images

Let's take the following 36x28 JPEG image. Note that the horizontal dimension of
36 is 4 multiples of the MCU width (8) plus an extra
4 pixels. The vertical dimension
of 28 is 3 multiples of the MCU height (8) plus
an extra 4 pixels. Therefore, the JPEG compression and decompression algorithms
will treat this image as if it were an image of size 40x32, which is the next
largest dimension that would be an integer multiple of the MCU size in both directions.

The JPEG file format only specifies a width and height parameter, so there is an inherent
assumption that the width starts at the first horizontal pixel co-ordinate and the height starts at the
first vertical pixel co-ordinate. Therefore, any pixel dimensions that don't complete a full MCU will be simply discarded upon display (for decoding). For encoding, the partial MCU is converted into a complete MCU by replicating the last pixel value across the remaining pixels of the MCU.

Lossless Rotate for odd-sized JPEG

The following example uses an image of size 36x28.

Original odd-sized JPEG image

In the above image, the major gridlines indicate the 8 pixel x 8 pixel boundaries (ie. the MCU boundaries). The blue region marks areas that are not part of the image data, but make up the remainder of a partial MCU.

Lossless Rotation Clockwise with odd-sized JPEG

After rotating clockwise 90 deg
before cropping

After the image has been rotated 90 degrees clockwise, note that the resulting image has partial MCUs that fall on the left edge of the image. Since the JPEG format does not allow data on the left or top edges to be specified as "unused" (there isn't any offset parameter!) , the software that performed the rotation has one of three choices in what it can do:

Crop off the partial MCUs on the left edge, eliminating the 4 pixels of actual data as well.
This method allows the image to be rotated losslessly (ignoring the trimming), but necessitates the loss of some image data in the new leftmost columns of the image. The reason that the rest of the image can be rotated losslessly is that the boundaries of the MCUs don't change. Therefore, the matrix coefficients can be transposed without having to perform the any quantization of data.

Crop off the left pixels and shift the rest of the image left by 4 pixels.
This method allows the full image dimensions to be preserved after rotation, but it can cause significant recompression error throughout the entire image. As the leftmost pixels are trimmed off, the MCU boundaries now fall within different points in the image. A change in the MCU boundaries will mean that all of the decompression, rotation and recompression steps have to be redone, leading to image quality degradation.

Extend the image boundaries to the left/top to include the hidden pixels
Instead of cropping the image, it is also possible to simply treat the partial MCU as a complete, valid MCU. Note that the only reason it is "partial" is that the image dimensions stored within the image instructed the decoder to use an overall dimension that was smaller than the MCUs would imply. So, if we increase the image dimension to encompass all MCUs fully, then we can perform a lossless rotation easily. Unfortunately, this means that one will be left with a small strip on the left or top of the image that contains "unknown" data. Most encoders will simply repeat the last pixel, but it is not a requirement. While this method does in fact preserve all of the original image data losslessly through the rotation, it will leave this strip which may be undesireable for presentation purposes. (Thanks to Alex of BetterJPEG for reminding me of this method)

Many programs will crop the partial MCUs instead of cropping pixels & shifting or extending, as this avoids
the reduction of image quality. One very useful utility that provides control over many of these operations is BetterJPEG.

Windows XP Explorer and Rotation

Note that in Windows XP Explorer, there are quick-menu commands that allow one to
Rotate Clockwise and Rotate Counter Clockwise. If you use these commands on an odd-sized image, the following warning message will be displayed:

Windows XP Warning for odd-sized rotations

Windows XP warns the user with the following:

Because of the dimensions of this picture, rotating it might permanently reduce its quality.

Why? Because Windows Explorer uses the crop pixels and shift method for odd-sized rotation. As
the boundaries of the MCUs are moved, the entire image must be recompressed and will therefore suffer
recompression error.

Why didn't Windows XP simply crop the partial MCUs instead? Probably because users would be confused and
and frustrated after seeing that several columns or rows of pixels were missing from the picture after
rotation!

Digicams with Chroma Subsampling

Unfortunately, there are many digicams out there that use chroma subsampling with one image dimension (usually vertical) that is only divisible by 8, while the other may be divisible by 16. After rotation, the vertical dimension that was only divisible by 8 will now be in the horizontal.

In Windows Picture and Fax Viewer, it seems that images with these dimensions fail to rotate without prompting the warning shown above. In addition, the resulting quantization tables are NOT rotated and the chroma subsampling parameter is also changed to 2x2. It is not completely clear to me why Windows picture viewer treats images from these digicams with such a restriction as it would not seem to be a necessary limitation.

Partial MCU Cropping Results

So, getting back to real lossless rotation programs... what happens when the partial MCU crop method
is used? The following shows what the image looks like after the trim:

After rotating clockwise 90 deg
after partial MCU cropping

Note that the resulting image size is now 24x36 pixels (it was
28x36 after the rotation). The blank area to the left no longer
exists in the image, so the JPEG image is now only 3 full MCUs wide and 4 full + 1 partial
MCUs high.

Now, let's rotate the image again by 90 degrees, clockwise:

After rotating clockwise another 90 deg
before cropping

Again, note that we are left with a partial MCU on the left-hand-side after rotation.
This means that we will have to crop off this column of partial MCUs. The following
image is the result of the cropping:

... after cropping

Note that after effectively rotating the image by 180 degrees, we end up trimming
off all of the partial MCUs and are left with an image that is no longer
odd-sized. The final dimensions are 32x24 pixels.

This explains why you will see a different JPEG image size after rotating
in IrfanView and other lossless graphic rotation applications.

Repeated example with Counter Clockwise Lossless Rotation

Using the same starting example as above, but this time using counter-clockwise (CCW)
rotation instead of clockwise:

After rotating 90 deg CCWbefore cropping

Note that in this example, counter-clockwise rotation causes the partial MCUs to
appear on the top edge of the JPEG image. As these are not supported, they must
be removed:

After rotating 90 deg CCWafter cropping

Repeated example with Horizontal Lossless Flip

Flipping horizontalbefore cropping

After the lossless horizontal flip operation, the leftmost column of partial MCUs must
be trimmed off:

Flipping horizontalafter cropping

Repeated example with Vertical Lossless Flip

Flipping verticalbefore cropping

After the lossless vertical flip operation, the topmost row of partial MCUs must
be trimmed off:

Flipping verticalafter cropping

Reader's Comments:

Please leave your comments or suggestions below!

2014-09-19

John

Thanks for your answer.

I somehow managed to write 'left' when I meant 'right'. Sorry for making you answer something that one easily could understand from the article (top and left parts of a jpeg image can't have partial MCUs).

My question should have been regarding making bottom and/or right edges of an jpeg image into partial MCUs.

And it turns out jpegtran can easily do that, by for instance using this command:

Thank you for sharing your knowledge on jpeg, I have had a wonderfull time reading your articles.

I was wondering if you have stumbled upon a application which lets one create partial MCUs?

What I mean is that I have images which I want to trim (for instance) a few pixels of the left and bottom side, however I do not want to crop away a full MCU.

I tried using jpegcrop, with no luck.

Hi John - Unfortunately I don't think the basic JPEG standard allows one to specify an offset at the top-left edge. As a result, you would either need to drop the entire MCU or end up recoding these MCUs (takes a specialized program to recode only a sub-region). I believe there was some discussion of extending the JPEG standard to specify an offset, as this would also allow for lossless rotation with partial MCUs.

2008-06-09

Chris

After considerable reading, I'm losing confidence in the jeg format, and wanting to shoot in Canon RAW with my 20D.

Question: If I decide to shoot in RAW, rotate RAW losslessly w/o respect to MCU or dimensions, then convert to JPG with Irfanview's RAW to JPEG converter, would this yield a better result than using the Canon 20D's excellent jpeg encoder to simply shoot in jpeg, and rotate the jpeg with some potential loss?

Thanks for any light you can shed!

2007-11-23

Kel

Thank you. The right explanation at the right time... very clear.

2007-06-19

beni

can a jpeg image that does not consist of any partial MCU be rotated 90 degrees or cropped losslessly without having to decode the image to the pixel level?

The steps i intend to perform are

1. Huffman decoding of an MCU
2. Fetch the DC & AC coefficients for each component
3. Remove the MCUs that are outside the cropped area
4. Rearrange the position of MCUs for 90 degree rotation
5. Rearrange the AC coefficients for each component for 90 degree rotation (dont know if this helps!)
6. Perform DPCM on DC and RLE on AC coefficients
7. Perform Huffman encoding per MCU

Your steps look correct. Note that you will need to rotate the AC coefficient matrix as well. Of course, if the image has any chroma subsampling, you'll need to take care in ensuring that the JPEG markers that refer to the sampling reflect the new orientation. Good luck!

2007-05-21

AlexM

Good explanation about rotation with partial MCUs.
I just would like to point out that there seem to be an alternative way of rotation of JPEGs with partial MCUs, the one where partial MCUs are preserved instead of being discarded: http://www.betterjpeg.com/lossless-rotation.htm

Absolutely right... while it may not be the most aesthetic solution (because of the leftover strip), it certainly keeps the original image data intact nicely. I have now amended this section of the page above. Thanks Alex!

2007-03-22

Anon

Hi, it's me again.

I have treated photos from a FinePix S5000 camera in the past. I saw here that its family had 2x1 chroma subsampling, but I remembered being able to rotate the photos losslessy with the Windows XP's Image and Fax Viewer.

I grabbed a photo again to test. Surprise, it works! There's no warning and after rotation it has 1x2 chroma subsampling, the expected quantization tables (transposed since they are not symmetric) and the same huffman tables (not optimized).

There are more quirks. This camera has no JPEG quality selector, but it uses different quantization values depending on image size, so 1 megapixel photos are less quantized than 3 megapixel photos (size doesn't increase that much). It writes three (!) quantization tables: 0, 1 and 2. It uses 0 for Y, 1 for Cb and 2 for Cr. 1 and 2 are identical.

Before the lossless rotation, all 3 quantization tables are packed together in a single marker, and all 4 huffman tables are packed together in another one. After rotation, they get split onto 7 separate markers.

If you rotate the image back to its original state, the Start of Scan marker will have an higher offset by 26 (1A) bytes. This is to be expected for the extra markers. However the image is smaller because the original had several thousand restart markers and the new one has none.

These images have NOT built-in thumbnails (might have something to do with being able to losslessy rotate them...)

Interesting... When I tested out a few photos from a Fuji FinePix S5000 Z (with dimensions 2816x2120), I was still given the rotation warning by Windows Explorer.

2007-03-19

Anon

If the MCU is 16x8, wouldn't that imply that for lossless rotation you'd need the image dimensions to be multiple of 16 horizontally and just multiple of 8 vertically? After 90 degree rotation, the MCU would turn into 8x16 and it'd still work...

Yes, what you describe is exactly what I would have expected. However, it appears that Windows XP's default image viewer doesn't appear to treat images this way, and instead generates an image with 2x2 chroma subsampling (which requires a 16x16 pixel MCU for lossless rotations). In addition, the resulting quantization tables are not rotated, which will definitely lead to a lossy operation. I have no idea why this extremely popular tool has such a restriction. If anyone has any ideas, I would be very interested to hear them!

2006-10-28

Chris

I'm following up on the previous post, as I had been struggling with the same question on lossless rotation. I have images of 2848x2136 and 2272x1704 from different cameras. I've checked with JPEGsnoop and both show "Subsamp 2 x 1" in the Cb and Cr components of the SOF0 section. So I take it they are both using 2x1 chroma subsampling and the MCU tiles are 16x8. Therefore, as explained in the previous reply the images cannot be losslessy rotated. Can you please confirm that I've come to the correct conclusion?

As I understand it, any image whose height and width are BOTH multiples of 16 can ALWAYS be losslessy rotated, regardless of the chroma subsampling. Is that a correct statement?

Sorry if these are repeating anything elsewhere, but I really want to get it right before I start rotating anything!

One suggestion I'd like to make is that you have a table with some of the more standard camera image sizes down the left and columns for 1x1, 2x1 and 2x2 chroma subsampling across the top. Then the table could be filled out with yes or no depending upon whether lossless rotation is possible. I think this would be a big help to newcomers to the subject. It's then a case of confirming the subsampling with JPEGsnoop and a quick lookup in the table to decide if it can be rotated safely.

Thanks for your time and effort with these pages. They really are excellent.

Hi Chris -- Prompted by your question, I had a look at hundreds of digicams (point & shoot and dSLR) on the market today and see that there are quite a few models that will create this problem. As nearly all digital cameras use chroma subsampling, lossless JPEG image rotation will effectively require an integer multiple of 16 pixels in both horizontal and vertical image resolutions (irrespective if the subsampling was 2x1 or 2x2). Therefore, you are correct in that having both a width and height as a multiple of 16 will make it always possible (if we're talking about normal subsampling as from digicams).

As for your suggestion -- I thought it was interesting enough to create a page detailing it further. Please have a look at Digicams and Lossless Rotation for more detail. One of the interesting finds was that there are a few digital cameras out there whose horizontal and/or vertical resolution isn't even a multiple of 8 pixels! Hope this helps, Cal.

2006-10-21

Shira

Hi,
I found your article after getting the warning from windows-xp viewer. The thing is that my pictures are 2272X1704, which is divisible by 8 (yet not by 16. I read somewhere else that it should be divisible by 16).
Can I assume the rotation will be lossless?
Thanks,
shira

The rotation will likely not be lossless. It sounds like you are trying to rotate images from a point & shoot digicam that uses chroma subsampling. In such photos, the MCU can be effectively thought of as being 16x8 or 16x16 pixels instead of 8x8.

I am guessing that your camera has 2x1 chroma subsampling, which means that it has discarded half of the color resolution in the horizontal direction. Effectively, your MCU tiles are 16x8 pixels. Your original photo dimensions are divisible by 16 in the horizontal direction and divisible by 8 in the vertical direction. HOWEVER, after rotation, the new horizontal dimension will no longer be divisible by 16, meaning that lossless rotation may not be possible.

I have updated the text to make this exception more obvious. Thanks for the great question!