JPEG Lossless Rotation or Flip with Partial MCU

JPEG images are constructed with 8x8 tiles

For the background on how an image is represented by the 8x8 tiles, please see my article on the JPEG Minimum Coded Unit (MCU). For the purposes of the following discussion, images whose dimensions are not a multiple of the MCU size will be termed an odd-sized image.

NOTE: For the following discussion, I am describing images that have been compressed without chroma subsampling. Many digital cameras use chroma subsampling to reduce file size at the expense of losing color information (generally 2x1 or 2x2 chrominance subsampling). For these photos, the tiles (MCUs) can be treated as being 16x8 or 16x16 pixels respectively. So, for the purposes of the discussion below, you should treat the required image dimensions as 16 pixels where it reads 8 pixels. Unfortunately, this is a problem that many users of digicams will encounter when they try to rotate their images losslessly.

Handling of odd-sized images

Let's take the following 36x28 JPEG image. Note that the horizontal dimension of 36 is 4 multiples of the MCU width (8) plus an extra 4 pixels. The vertical dimension of 28 is 3 multiples of the MCU height (8) plus an extra 4 pixels. Therefore, the JPEG compression and decompression algorithms will treat this image as if it were an image of size 40x32, which is the next largest dimension that would be an integer multiple of the MCU size in both directions.

The JPEG file format only specifies a width and height parameter, so there is an inherent assumption that the width starts at the first horizontal pixel co-ordinate and the height starts at the first vertical pixel co-ordinate. Therefore, any pixel dimensions that don't complete a full MCU will be simply discarded upon display (for decoding). For encoding, the partial MCU is converted into a complete MCU by replicating the last pixel value across the remaining pixels of the MCU.

Lossless Rotate for odd-sized JPEG

The following example uses an image of size 36x28.

Original odd-sized JPEG image

In the above image, the major gridlines indicate the 8 pixel x 8 pixel boundaries (ie. the MCU boundaries). The blue region marks areas that are not part of the image data, but make up the remainder of a partial MCU.

Lossless Rotation Clockwise with odd-sized JPEG

After rotating clockwise 90 deg before cropping

After the image has been rotated 90 degrees clockwise, note that the resulting image has partial MCUs that fall on the left edge of the image. Since the JPEG format does not allow data on the left or top edges to be specified as "unused" (there isn't any offset parameter!) , the software that performed the rotation has one of three choices in what it can do:

• Crop off the partial MCUs on the left edge, eliminating the 4 pixels of actual data as well.
  This method allows the image to be rotated losslessly (ignoring the trimming), but necessitates the loss of some image data in the new leftmost columns of the image. The reason that the rest of the image can be rotated losslessly is that the boundaries of the MCUs don't change. Therefore, the matrix coefficients can be transposed without having to perform the any quantization of data.
• Crop off the left pixels and shift the rest of the image left by 4 pixels.
  This method allows the full image dimensions to be preserved after rotation, but it can cause significant recompression error throughout the entire image. As the leftmost pixels are trimmed off, the MCU boundaries now fall within different points in the image. A change in the MCU boundaries will mean that all of the decompression, rotation and recompression steps have to be redone, leading to image quality degradation.
• Extend the image boundaries to the left/top to include the hidden pixels
  Instead of cropping the image, it is also possible to simply treat the partial MCU as a complete, valid MCU. Note that the only reason it is "partial" is that the image dimensions stored within the image instructed the decoder to use an overall dimension that was smaller than the MCUs would imply. So, if we increase the image dimension to encompass all MCUs fully, then we can perform a lossless rotation easily. Unfortunately, this means that one will be left with a small strip on the left or top of the image that contains "unknown" data. Most encoders will simply repeat the last pixel, but it is not a requirement. While this method does in fact preserve all of the original image data losslessly through the rotation, it will leave this strip which may be undesireable for presentation purposes. (Thanks to Alex of BetterJPEG for reminding me of this method)

Many programs will crop the partial MCUs instead of cropping pixels & shifting or extending, as this avoids the reduction of image quality. One very useful utility that provides control over many of these operations is BetterJPEG.

Windows XP Explorer and Rotation

Note that in Windows XP Explorer, there are quick-menu commands that allow one to Rotate Clockwise and Rotate Counter Clockwise. If you use these commands on an odd-sized image, the following warning message will be displayed:

Windows XP Warning for odd-sized rotations

Windows XP warns the user with the following:

Because of the dimensions of this picture, rotating it might permanently reduce its quality.

Why? Because Windows Explorer uses the crop pixels and shift method for odd-sized rotation. As the boundaries of the MCUs are moved, the entire image must be recompressed and will therefore suffer recompression error.

Why didn't Windows XP simply crop the partial MCUs instead? Probably because users would be confused and and frustrated after seeing that several columns or rows of pixels were missing from the picture after rotation!

For more information about this message, please see my article on digicams and lossless rotation.

Digicams with Chroma Subsampling

Unfortunately, there are many digicams out there that use chroma subsampling with one image dimension (usually vertical) that is only divisible by 8, while the other may be divisible by 16. After rotation, the vertical dimension that was only divisible by 8 will now be in the horizontal.

In Windows Picture and Fax Viewer, it seems that images with these dimensions fail to rotate without prompting the warning shown above. In addition, the resulting quantization tables are NOT rotated and the chroma subsampling parameter is also changed to 2x2. It is not completely clear to me why Windows picture viewer treats images from these digicams with such a restriction as it would not seem to be a necessary limitation.

Partial MCU Cropping Results

So, getting back to real lossless rotation programs... what happens when the partial MCU crop method is used? The following shows what the image looks like after the trim:

After rotating clockwise 90 deg after partial MCU cropping

Note that the resulting image size is now 24x36 pixels (it was 28x36 after the rotation). The blank area to the left no longer exists in the image, so the JPEG image is now only 3 full MCUs wide and 4 full + 1 partial MCUs high.

Now, let's rotate the image again by 90 degrees, clockwise:

After rotating clockwise another 90 deg before cropping

Again, note that we are left with a partial MCU on the left-hand-side after rotation. This means that we will have to crop off this column of partial MCUs. The following image is the result of the cropping:

... after cropping

Note that after effectively rotating the image by 180 degrees, we end up trimming off all of the partial MCUs and are left with an image that is no longer odd-sized. The final dimensions are 32x24 pixels.

This explains why you will see a different JPEG image size after rotating in IrfanView and other lossless graphic rotation applications.

Repeated example with Counter Clockwise Lossless Rotation

Using the same starting example as above, but this time using counter-clockwise (CCW) rotation instead of clockwise:

After rotating 90 deg CCW before cropping

Note that in this example, counter-clockwise rotation causes the partial MCUs to appear on the top edge of the JPEG image. As these are not supported, they must be removed:

After rotating 90 deg CCW after cropping

Repeated example with Horizontal Lossless Flip

Flipping horizontal before cropping

After the lossless horizontal flip operation, the leftmost column of partial MCUs must be trimmed off:

Flipping horizontal after cropping

Repeated example with Vertical Lossless Flip

Flipping vertical before cropping

After the lossless vertical flip operation, the topmost row of partial MCUs must be trimmed off:

Flipping vertical after cropping