Return to Digital Photography Articles

JPEG Lossless Rotation, Lossless Crop

With a photographer's desire to preserve the digital negatives as much as possible, there is a strong need to be able to rotate photos without reducing the quality in the process. The rotation might be done for a variety of reasons, including: correctly orienting a vertical portrait shot when the camera doesn't have a built-in orientation sensor, rotation for stylistic reasons, true rotation to match the EXIF orientation flag, etc. Similarly, one may want to crop a photo without degrading the quality of the uncropped portions of the image.

As the "loss" (more properly: additive compression error) is completely dictated by the details of how JPEG compression works, it is important to understand some of the basic concepts. Read my summary on JPEG compression.

Standard rotate and crop error / loss

Normal Rotate Error

To introduce the idea of incremental error, let's look at an example. Let's say a user opens up IMAGE_A.JPG in an image editor. The user rotates the content clockwise by 90 degrees and then resaves it as IMAGE_B.JPG. Now, the user opens IMAGE_B.JPG, rotates the content counter-clockwise by 90 degrees and then resaves it as IMAGE_C.JPG. If these overall rotate operations were truly "lossless", then we'd expect that IMAGE_A.JPG would be identical to IMAGE_C.JPG. But we will often find this not to be the case! Not only does the image content change, but sometimes even the metadata (information from the digital camera) is lost in the process.

The image on the right shows an example of the difference between IMAGE_A.JPG and IMAGE_C.JPG when using Photoshop CS with the "Rotate Canvas" command and saving each time at "Maximum" quality setting (10). The two files were then overlayed with a "Difference" layer mode, and then a Levels adjustment was made to remap the white-point to a value of 3 or higher (to make it visible). The histogram showed that the maximum RGB difference between the original and the final version is 13 / 255, but the mean difference was only about 1.10 / 255 (standard deviation 0.82). Therefore, it would be very hard to visually observe the differences (additional error) due to the rotate and resave operation (at quality setting 10), but they obviously exist. Obviously, the selection of the quality setting has a significant impact on the degree of cumulative error that we are introducing.

How normal rotate and save operations cause incremental error

Now let's revisit the original rotation scenario. When IMAGE_A.JPG is opened in Photoshop, the software must perform all of the steps listed above (but in reverse order) to decode the JPEG image. No data is "lost" in this decompression step besides a small amount in the color space conversion (as we are converting an 8-bit value with linear equations to another 8-bit value).

A 90 degree rotate is performed, which can be done in the RGB spatial representation without problem. Now, the user decides to save the file as IMAGE_B.JPG. The user will be prompted with a Save As... dialogue box which asks for a quality setting (ie. 1-12, 1 being the poorest and 12 being the absolute maximum). Photoshop will then have to perform the DCT transform again on the image contents and then re-quantize the data. It is this re-quantizing of image data that causes the majority of the incremental error. As all of the 8x8 pixel blocks have changed content (by the rotate), the DCT result will be different, causing the re-quantization to come up with different results, again suffering the rounding loss.

Lossless rotate and lossless crop

It is first important to understand what is meant when people refer to lossless operations (lossless rotate, lossless crop, etc.). Generally, the idea is that an operation is performed on a portion of an image without incurring any additional recompression error. As others (see Doug Kerr's JPEG PDF article) have carefully pointed out, a better name for these operations would be "reversible" and not "lossless".

Programs that offer the ability to perform lossless rotation generally act on the file, rather than on the image content. This is seen in several programs (such as IrfanView) which actually place the "lossless" operations in a seperate menu tree, outside of the main editing flow. Furthermore, these operations will typically resave the file automatically, rather than ask you for any quality settings, etc. As will be described later, the reason for this is that the quantization table must be preserved exactly as it was before, and any quality setting changes would destroy this prerequisite.

For all lossless opertaions, note that the dimensions of the image must be integer multiples of the minimum coded unit size (MCU, see below). In each case, the image editor software must decompress the JPEG data to the point where the DCT coefficients become available. Then, an operation is performed on the resulting coefficients and the compression is performed again. Since no dequantization or requantization is performed, there is no additional rounding error that would contribute to any compression "loss".

Types of reversible / "lossless" operations:

  • Lossless rotate - As the DCT basis functions are symmetric across the X & Y axes, it is a simple matter to remap the coefficients in the matrix from one direction into the other. Rotation can be performed in increments of 90 degrees (ie. 90, 180, 270 degrees).
  • Lossless crop - For an image to be cropped reversibly, the program simply tosses out the entries that correspond to image blocks that are to be cropped off. As all other blocks remain untouched, they are recoded identically and reversibly.
  • Lossless flip across an axis - Just like the lossless rotate, the flip operation (which can be done across eitherr the X or Y axis) is essentially a remapping of matrix coefficients.
  • Lossless extension - Just like lossless crop, it is also possible to add additional area to the image (eg. increase "Canvas Size") by a multiple of the MCU. The original blocks will be left untouched.

Obviously, only the rotate and flip operations are "reversible". The crop and extension operations are dubbed "lossless" as the primary region of the image content has not suffered image recompression / degradation.

Test to see if a program does lossless rotation

Read my article on how to test for lossless rotation in your software.

For a starting point list of applications that support lossless rotation operations, please check out the JPEG Club Lossless Rotation Applications. However, please note that this is not a complete list -- it only includes those applications that were known to reuse the IJG code. There are quite a number of other applications that also provide lossless rotation.

For example, there has been widespread confusion about whether or not Canon's ZoomBrowser EX supports lossless rotation. I have tested Canon ZoomBrowser EX 5.7 and confirmed that it does in fact perform lossless rotation (if you set the Rotate Actual Images option, otherwise it's a virtual rotation).

Minimum Coded Unit / Image Block Size

To make the image encode and decode more manageable, the JPEG compression algorithm subdivides the original image into 8x8 pixel blocks. This is known as the Minimum Coded Unit (MCU). The alignment of these 8x8 boundaries in the image is of great importance to the resulting compression. As the original RGB image content is converted into the YCbCr color space, the algorithm is able to perform a different type of compression on each component (the components being luminance and the two chrominance components). As described on the JPEG compression page, the chrominance components can be compressed more heavily (and result in greater error) than the luminance component due to the nature of our eye's sensitivities. This is the basis for the chroma subsampling, which effectively reduces the resolution of the chrominance component.

By subsampling the chrominance component, the minimum coded unit becomes 16x8 or 16x16 pixels instead of the 8x8 pixel units. 16x8 corresponds to 2:1 chroma subsampling, while 16x16 corresponds to 2:2 chroma subsampling.

Lossless Rotation of odd-sized images or Partial MCUs

One cannot truly perform a lossless rotation on a JPEG whose dimensions are not an integer multiple of the MCU size. In other words, a photo that is 501 x 483 pixels cannot be properly rotated or flipped losslessly!

Please read my article that explains partial MCU and lossless rotate / lossless flip operations.

Photoshop CS: No lossless operations possible!

Note that it is currently impossible to perform lossless rotate or lossless crop in Photoshop, if the original was a digital camera image.

EXIF Orientation and Virtual Rotation

To add to the confusion, some products refer to modification of the EXIF Orientation flag as lossless rotation or virtual rotation. The EXIF metadata attached to a digital camera photo includes a field named "Orientation". This field describes the orientation of the camera when the shot was taken, but only if the camera contains an orientation sensor. Most digital SLRs and some advanced point and shoot cameras contain such a sensor.

By including a field in the metadata that describes the intended orientation of the photo, the camera is saved from the work of having to rotate the image immediately after the shot. Essentially, it marks the photo as "requiring a rotate" by anyone wishing to view the file. If the camera were to automatically rotate the image, a large amount of processing would be required in-camera, leading to far slower frame rates.

But, by saving the camera any of the trouble of rotating the image, the responsibility is left up to every image viewer that wants to open up the image later. Every time an image editor or browser decides to open the image, the software must rotate the original before displaying. This leads to a performance hit in any application that opens the image with the orientation flag set. A drawback to this method is that not all programs support this EXIF orientation flag. In such an application, the image will incorrectly appear un-rotated.

Some programs offer the ability to virtually rotate an image. These programs either modify the EXIF Orientation flag, or just rotate a thumbnail (in the case of a photo catalog program), without touching the original image.

For more details, please see my article on EXIF Orientation and JPEG Rotation.

 


Reader's Comments:

Please leave your comments or suggestions below!
2015-06-21Jim
 Windows Picture and Fax Viewer under Windows XP gave you a warning that the image was going to be re-compressed - but under Windows 7 there doesn't appear to be any such warning - it just goes ahead and re-compresses the image without your okay! I don't know how it decides what compression ratio to use, either, 'cause it doesn't ask, but after rotating a test image the file sizes were similar, so presumably it had some idea based on the original image. The good news (if there is any) is that it appears to retain a lot of the meta-data.
 Hi Jim -- in my testing it appears that the "Windows Photo Viewer" in Windows 7 does actually support lossless rotation (even if it might mess up the orientation flag). Please see my comment posted at 2012-07-18. Have you observed that it recompresses the file (leading to lossy rotation)?
2009-07-15Ryan
 This page (and the ones around it) was really helpful to understanding what's going on when I edit my photos... the availability of lossless cropping was a big aha moment for me.

Makes me wonder, though, what other kinds of editing are possible without introducing more error? For example, after cropping and scaling (both of which can be lossless if done carefully), by far the most common two edits I make on my photos are to fix white balance and tweak gamma.

White balance correction, especially, seems like a DC component sort of operation that could be done directly on MCU?

Thanks!
 Good question, Ryan. In some sense, a change to the DC offset of an entire image could be viewed as a reversible operation that retains all of the original image information, but I don't think it really qualifies as "lossless". That said, white balance and gamma change operations are not simple DC shifts, and so they cannot be done by a single adjustment to the DC values of the first MCU. Both operations involve scaling the ranges in non-integral ways. One somewhat useful adjustment that can be done, however, is to adjust the luminance of the entire image. This can be done by changing the DC value of the very first MCU's Y component.
2008-11-13WindsurferLA
 Appreciate the explanation of LossLess crop and rotation. I use IrfanView and Microsoft's Digital Image 10 Pro and now have better appreciation of when to use each. Thanks
 Thanks (PS> I still use my BIC Electric Rock :)
2008-06-30Napier
 Jpegsnoop is a great utility but what really seems to be lacking out there is a utility to losslessly rotate the blocks in each frame of motion jpeg avi movie! Since m-jpeg lends itself to just this type of thing, it astounds me that there ostensibly isn't any software out there to do it commercial or freeware! I have, so far, found only two programs which claim to do this and neither of them seem to do well.......ANYTHING! They either crash or just sit there at 0% and never progress! I would suggest, given the abundance of this type of file nowadays, since the rise of digital camera technology, that this might be good and very useful feature to include in a future version of you program!

Let me know if you have any plans for such future development.

Napier.
 Interesting suggestion! It should theoretically be a simple matter to write such a program. I have not written JPEGsnoop in such a way that it would perform such AVI conversions very efficiently (on longer movies), so it would require a new development.

At this point in time I don't have time to start such a development, partly because I am sure that there are other applications out there that already target such a feature. I don't want to reinvent the wheel too many times :)

That said, I'm quite surprised that there aren't more programs out there to do this -- it would seem that this is a very common need as sometimes people use their digicam in portrait orientation when shooting movie clips!
2008-01-10cansomeone help me please
 I can not rotate my 2 pictures plus there is 2 on the same page and all I see is one my hubby & dad can you help, with the others PLEASE!!
 When you try to rotate the image, do you get an error message? From what you've wrote, it sounds like you could have a corrupt photo. If you'd like me to have a look and see whether or not it is fixable, check out my fix corrupt photos page.
2007-10-15endolith
 Haha. "Reversible crop"? There's no such thing. You've thrown the information away forever. "Lossless crop" is better since it means that the area leftover has not lost any information.
2007-09-04speterson
 Do you have any experience with iPhoto ('07 preferably) regarding lossless rotation (or other operations). If the exif flag indicates a rotation is needed, it does it automatically when imported and saves a new rotated image. I'm assuming its not lossless or they wouldn't need to save it to a new file.

thanks
 I don't have any experience with iPhoto, although if you emailed me a sample image before rotation, after rotating right 90 degrees, and then this same file after rotating left 90 degrees (i.e. back to its original orientation), I should be able to give you a definitive answer.

In the meantime, it is possible that the rotation is still lossless but that iPhoto requests a new filename just in case the rotation operation corrupts some of the other JFIF / metadata fields. Even though lossless rotation doesn't damage any of the image content, it does change the size of the JPEG file. In doing so, some of the internal JPEG (JFIF) segments change their positions in the file. Some undocumented makernotes fields don't tolerate being moved very well and could end up being corrupted. Therefore, if one is very paranoid about any potential for loss (to metadata), it is safer to use a separate file for the rotated version (I generally don't bother).
2007-04-12Legion
 Thanks especially for the info about ZoomBrowser Ex, that's really useful to know. I can't imagine why Canon haven't made it explicit that rotation in ZoomBrowser is lossless. Anyway, I'm changing my default setting right now back to automatically rotate images as they're downloaded!
2006-12-19Mark Worthington
 Thanks for the reply and the new EXIF Orientation and Rotation page, good stuff indeed!

Regarding the orientation comment, I was just using that as an indication that the file was not visually changed by opening/rotating/closing-without-saving in the Windows Picture and Fax Viewer utility.

I use an image application called PolyView. I don't know if you are familiar with it, but I've been a daily user of it since 1997, and I personally rate it higher than IrfanView, but that's just personal. They do the same jobs, really. PolyView supports losses rotations/flip/mirror and deals with the Exif Orientation flag as expected. Indeed, in parallel with our discussions PolyView has now implemented a lossless auto-rotation based on the Exif info.

As a matter of interest, I use Better JPEG for lossless cropping.

Regards,
Mark

and thanks for great articles!
 I wasn't previously aware of PolyView -- thanks for pointing out an alternative. As for the orientation, after giving it some more thought I considered a scenario where the loss of the EXIF orientation flag might be problematic: performing a rotation on a portrait photo, followed by the opposite rotation. The resulting image will no longer be displayed correctly after it is loaded into an orientation-aware application such as Photoshop (as it would have prior to the double-rotation).
 
2006-12-13Mark Worthington
 I opened an image in Windows Picture and Fax Viewer then rotated. (No warning message as the file size is a multiple of 16, so the rotation would be lossless). I exited without saving. The image is still orientated the original way.

However, the Exif Orientation flag has been removed!!

I haven't seen this mentioned before..

Regards,

Mark
 Hi Mark -- The Windows Picture and Fax Viewer utility should not be used to rotate your images. Not only does it lack support for detecting the orientation flag, but it also discards much of photo's metadata (camera, shot and makernote) information in the process. Most utilities will set the EXIF Orientation flag to a new value that is appropriate for later interpretation (such as 1, which implies no further rotation necessary) -- but Windows XP's utility wipes out this field and many others.

Please see my new article on EXIF Orientation and Rotation.

You mention that your image is still oriented the original way -- which programs are you using to determine the image orientation? What was the orientation of the original image? The confusing element here is that both Windows XP's thumbnails and the Windows Picture and Fax Viewer don't support the orientation flag, and so they simply display the image content as encoded, without taking any hints as to how it should be rotated.
2006-11-26Ming Kin Lai
 Does that mean if my image dimension is a multiple of 16, I can do a lossless rotation without some special software, by using the Windows Picture and Fax Viewer for example?
 Yes. Thankfully, the Windows Picture and Fax Viewer will also give you a warning anytime that image dimensions prevent full lossless rotation.
2006-09-26Mike Lee
 Might you know of any programs which actually perform lossless extensions? I've been googling for an hour and haven't found any. Thanks for a very informative explanation of lossless functions.
2006-03-19Gabriel Harrison
 Hi,

That makes perfect sense, it is the Ixus 500 I have. I'm not sure how to work out the MCU but I found other software that was perfectly happy to rotate the image. I did the lossless on them test you describe and they were perfect.

Gabriel
 Glad to hear it!
2006-03-19Gabriel Harrison
 Brilliant Explanation!

I wonder if you can solve this mystery for me.

When I take photos on my compact digital's highest setting of 2592x1944 the Windows XP previewer says it can't process it losslessly even though it is a whole number of MCU.
 For many digicams, chroma subsampling is used to discard color information to increase compression further. Generally, a 2:1 subsampling is used, keeping half of the horizontal color component's resolution and all of the vertical resolution. This would mean an MCU of 16x8, not 8x8. There is also a possibility that your camera uses 2:2 subsampling (giving a MCU of 16x16), but this is less likely. Looking at the max vertical resolution that your camera provides (1944 pixels, typical in Canon's 5MP digicams), it is a integral multiple of 8 but not 16 pixels!

While it would seem that Window's lossless rotation algorithm (block transformation) should be able to accommodate the asymmetric MCU (with a vertical resolution to match), it doesn't appear to. I don't believe it would need to treat this as a lossless rotation with a partial MCU. However, the resulting image would end up with a 1:2 chromic subsampling that is not always supported by decoders, and hence the Windows method may err on the side of compatibility. If anyone else has any other ideas, please reply!
2005-11-17Joseph Bruno
 I am a moron.
There is no such thing as a 300x125 JPEG. There is only a 304x128 JPEG plus the (built-in) instruction "strip off the last 4 columns and 3 rows".
Such an image can easily be rotated losslessly by 180, resulting in a 304x128 JPEG that needs the instruction "strip off the first 4 columns and 3 rows" in order to produce the intended result.
But there is no way, within the JPEG format, to say that you want to lose pixels from the left or top. Hence you either have to leave the rotated image at 304x128; or truncate it to 296x120 before rotation; or, having rotated it 180 losslessly, shift it 4 pixels to the left and 3 pixels up, lossily!

Arising out of this... do you know if it is possible, looking at the encoded JPEG file, to work out the parameters that were used during compression (presumably, how the DCT coefficients were quantized) so as to be able to use the same ones when re-compressing? My guess is "Yes", before I plunge deep into the standard and IJG in order to sort out the details, it would be nice to know what your guess was.
 That is exactly right. You cannot losslessly rotate an image with partial MCUs (ie. non-integer multiple of the MCU size, 8x8, 16x8 or 16x16) because there is no way for the JPEG format to indicate where the offset for partial MCUs are.

By default, the assumption is that partial MCUs only exist on the right and bottom edge of an image (as dictated by the pixel dimensions). Padding to make a complete MCU on these two edges is done by extending the last column or row and then clipping on decode.

If the JPEG standard had an extra byte in the JFIF to indicate a starting position for the MCU boundary, one could perform a number of additional operations losslessly. Instead, we are left with the only real option: drop the columns or rows that belong to partial MCUs that end up on the top or left edge of the image after rotation. This is precisely what I had observed in the BetterJPEG tests.

I am going to update this page with a section (including examples) on the treatment of partial MCUs as I haven't seen it explained anywhere else. Thanks for your great comments.

In answer to your second question, yes, it's possible to determine the quantization matrix used in compressing an image. JPEG photos typically include a DQT marker (Define Quantization Table) that defines one or more coefficient matrices that are used in the encode / decode process. But while we might know the table, will your compression software use that exact same DQT? Unlikely, unless you are using a tool like cjpeg. So, the question might be: how can we find out what Compression Quality settings to use in an application to generate the same DQT coefficients? Well, this can actually be achieved in most cases.

Many applications generate their quantization tables from the example ones listed the JPEG Standard's Annex, but then are scaled by some quality factor. Usually, this is some form of linear scaling/multiplication. In my page on Quantization Tables, I have tried to calculate what these factors are, but I plan on doing a more complete & generic analysis later (ie. not just trying to identify a match to a digital camera's tables).
2005-11-14Joseph Bruno
 It's all excellently clear and written as if the reader had a mind. Thank you!
I do have one question, though. What is it about images that aren't an exact multiple of 8x8 pixels that prevents them from being rotated losslessly? Looking at the standard, it seems that the compression algorithm simply rounds up the image size to the next multiple of 8x8 when compressing (replicating the last rows and columns as necessary to achieve this) and discards the surplus rows and columns when decompressing... which should make lossless rotation just as easy for 300x125 (rounded up to 304x128, rotated to 128x304, relabelled as being 125x300) as it is for 640x480.
 Hi Joseph --

Great question! I have been giving this some thought (as I wasn't certain of the answer off hand). My assumption would be that at the very least one could certainly preserve the content of untouched interior blocks losslessly. What I wasn't sure, however, how accurately the right and bottom edges of the image could be preserved. At the worst case, by extending the peripheral edges to complete the MCU, you'd end up with error introduced into these regions only.

I did some tests (I'll probably add them online here as well, for interest sake), comparing how these lossless rotation utilities handle the partial MCU scenario. Windows XP simply brings up a warning, and then does a standard rotation (with no attempt to do a lossless rotation -- ie. large error everywhere). BetterJPEG (plugin within Photoshop) was able to rotate these odd-sized images without introducing any error into the main content BUT the resulting file had the partial MCU stripped off!

So, I am leaning towards the assumption that simply extending the edges will cause additional recompression error, and hence the reason why we cannot truly rotate these images losslessly in their entirety. I'll have to do further testing or investigation to prove to myself that this is the case.
2005-11-13Bill Tyrrell
 Rreellyyyyy! LOVE your photography and articles! I was a pro back in the day 40 years ago. I thought I was good back then. You guys with incredibly great equipment impress me no end. It's still the vision, "being there" and pushing the buttom at the right moment. I still miss totally manual shooting but am now "sucked under" by the magic of the instant picture and pseudo darkroom control...but it's so much harder! In the "olde" days I baredly carried a meter (electronics were in my head) and produced 98% well exposed (and seleable) pictures. Why can't we have a "looks and feels" like an OLD totally manual (maybe a built in meter) Nikon or Hassie? Could be the market needs to be rechecked?

Different Question;
Witould it be easier/simpler to to shoot in RAW rather than JPG to not have to go thru all the problems with/agravation (and losses) of the above procedures and manipulation? Is this practical?

Last observation;

WOW! You absoulely have the most beautiful site and great photographs I've ever seen on the web! You have the (so hard to consistantly find) "magic eye" and know when to "push the button". You're obviously very well traveled and incredibly tired going home. Many thanks for your insight and work. Gongrats and Kudos!

Bill Tyrrell
 Bill -- Thank you for the very kind words! I think the benefits of dSLRs over the film SLRs are really becoming clear. Even without the instant gratifaction factor, the flexibility in on-demand ISO selection (no more mid-roll rewinds!) and lower cost to experimentation are significant benefits.

As to your question: shooting in RAW should help solve the main problems, but it really depends on what you're doing with the photos and at what stage. If you're simply wanting to rotate an image 90 degrees immediately after capture, then the RAW conversion software (eg. Adobe Camera RAW or C1) will be able to rotate losslessly, as they don't need to force the images through the extra decompression / recompression steps. If the output of your RAW conversion is a lossless format, then you have sidestepped the issue entirely. The best workflows use a lossless format for editing, and then only optionally convert to a lossy format at the end for repurposing (for web, print, archiving, etc.). One shouldn't need to rotate these final products (after final conversion), so rotation error never comes into the picture. So long as one does the rotation within the RAW converter, you'll be fine. That being said, I'm sure nearly all digital cameras that provide RAW also contain an orientation sensor which should be interpreted automatically by the conversion software.

You're definitely right about arriving home from the vacations tired! Somehow I think my idea of a dream vacation would be another man's nightmare!
2005-11-04 
 

Wow - Thank you! You achieved a good balance of the technical and nontechnical...

 

Thanks!

2005-10-09 
 

great explanation

2005-04-14 
 

Fantastic explanation - thanks. I was linked to this site from a post on the TalkGraphics.com forums.

 


Leave a comment or suggestion for this page:

(Never Shown - Optional)
 

Visits!