Return to Digital Photography Articles
JPEG Color Space Conversion Error
As discussed in the JPEG Compression article, there are several sources of "error" in going from the original raw image data to the compressed JPEG photo. One of the less obvious "losses" is in the color conversion step.
|Original Image before Color Space Conversion Error|
Why change the color space? (RGB to YCbCr / YCrCb / YCC)
A single 6-megapixel digital photo should actually consume 18 MB if it were stored in its uncompressed form on a PC! Clearly, such huge file sizes would cause countless problems for the average consumer, meaning larger memory cards, longer import times and slower frame-per-second capture rates.
One of the main reasons for the popularity of the JPEG compressed file format for photographic images is its ability to encode high quality photos in a much smaller file size. 6 megapixel images can typically be compressed down to 2 to 3 MB while still retaining much of the original image quality. In order to achieve this 8:1 compression ratio, some detail is discarded in the process. The unique characteristic of JPEG image compression is the manner in which various details are selected for discard.
The JPEG format has been engineered to take advantage of the limitations of the Human Visual System (HVS), discarding "information" that the human eye has a difficult time discerning. If the average person can't differentiate particular types of image detail, then there is little benefit in retaining them in the resulting image file.
For ease of implementation, most native uncompressed image data is stored as RGB (Red-Green-Blue) tristimulus values. An image is typically composed of 3 channels, one for each of these three components (e.g. Red, Green or Blue). JPEG offers some flexibility in how each channel is compressed, both through the specification of quantization tables and chroma subsampling.
Low frequency image content is made up of the brightness or color differences that change slowly across a large area of the image. For example, a sunset might have low-frequency brightness content that starts bright at the horizon and gradually fades out to dark at the top of the photo.
High frequency information is observed in any fine detail. The highest frequency content would be a change in value (brightness or color) between every adjacent pixel in an image. From a distance, it's easy to expect that this high-frequency content can be discarded with little impact to the overall image quality.
The human eye has a frequency response that suggests a strong sensitivity to high-frequency luminance (brightness) and a weak sensitivity to high-frequency chrominance (color information). Realizing this, an optimized compression scheme would apply less compression to the luminance detail (i.e. higher quality, lower loss) than the chrominance detail.
To accomplish this, the JPEG compression scheme begins with a color space conversion from RGB (Red - Green - Blue) into YCbCr (Luminance - Blue/Yellow - Red/Green) (also called YCrCb by some). This conversion takes the three standard channels (RGB) and maps them into a different representation that is based on a luminance (brightness) channel and two opposing color channels.
|RGB color space conversion to YCbCr|
Having done this conversion step, the JPEG image compression algorithm can then apply more compression to the color information channels than the luminance information and yet still arrive at an acceptable resulting image quality.
The most significant savings in JPEG compression come from the truncation or elimination of high-frequency detail (through a process called quantization). Low-frequency information is preserved, while detail at higher frequencies is progressively discarded in greater amounts as the frequency increases.
If JPEG compression operated on RGB data instead of YCbCr data, one would not be able to discard as much of the higher frequency content without causing a noticeable loss in image quality. Some studies have compared the suitability of other color spaces (e.g. RGB, HSV, Lab, etc.) for image compression and have demonstrated that YCbCr is a very suitable choice (at least for human observation of natural photos).
Attempt to Recompress Losslessly
This article came to be as I was attempting to answer a reader's question about a way to perform a lossless extend (lossless extension of image data, not rotation) with a source JPEG image. Trying to be clever, I initially figured that I could do the following:
- Analyze the source Quantization tables
- Analyze the source chroma subsampling factor
- Decompress the source JPEG into a lossless format (such as BMP or TIFF)
- Edit the intermediate image in Photoshop and resave in a lossless format
- Recompress the intermediate image with the same quantization tables and chroma subsampling
- Compare the results
So, what happened?
|Image after Color Space Conversion Error|
|Color Space Conversion Error Differences|
The resulting images showed a nearly perfect lossless edit cycle, but there were a small number of MCU blocks (minimum coded units, 8x8, 16x8 or 16x16 pixel tiles) that exhibited some "error". While this error was very small (not more than about 1% deviation in any of the three color channels), and only affecting a fraction (0.72%) of the overall image, it still exists. More interestingly, it only seemed to be apparent in the highlight regions of my original photo.
The above image "Error Differences" is the RGB differences enhanced with a threshold cutoff of 1.
Resulting error differences:
|Total Pixel Count: 3,145,728|
|Same||3,127,957 (99.44%)||3,124,464 (99.32%)||3,118,702 (99.14%)|
|1||15,898 (0.51%)||20,608 (0.66%)||13,991 (0.44%)|
|2||1,772 (0.06%)||642 (0.02%)||11,101 (0.35%)|
|3||101 (0.00%)||14 (0.00%)||1,767 (0.06%)|
Although I had removed the quantization loss step (the primary source of the compressed JPEG file savings), there still remained the error in the color space conversion step.
Color Space Conversion Error
Color Space Conversion Error
Any time that you convert an image from one color space to another, you are almost always guaranteed to introduce some error (image degradation) in the resulting output. This is because most images are defined in 8-bit precision per channel (24 bits/pixel), and if you output to a format that is also only 8-bit precision per channel with a floating-point conversion, you are going to have to perform some degree of rounding or truncation to end up at an 8-bit integer again.
For example, let's assume that a transfer / conversion function is required that is: (R is the integer Red value 0-255 and Z is some arbitrary resulting intermediate color space component value 0-255)
Z = R * 0.29900
If the original R value is 132, then:
Z = 39.468 -> Rounding -> 39
The delta is now 0.468, which was lost in the process. You'll see that inputs of 129...132 all give a rounded integer output of 39. Therefore, we have a compression in the translation process and a loss of information. This is a many-to-one relationship, which is irreversible.
|Input R||Output Z|
If the process is now reversed, you'll see that:
|Input Z||Output R|
Therefore, in the reverse direction, there is no way that one can create a resulting R channel output of 128...129, 131...133, etc. This is known as banding or posterization and, if significant enough, it may be visible in the resulting image (blue sky gradients, etc.).
In the above, it's clear to see that round-trip conversion (into and out of the intermediate color space) results in some degree of error:
|Input R||Intermediate Z||Output R||Error|
Measuring the Error
To get an idea of the magnitude of error that one may observe in the JPEG color resampling stage, I coded up the RGB -> YCbCr conversion algorithm in Excel, then took that output and performed the inverse conversion YCbCr -> RGB, taking into account the 8-bit limits on precision (i.e. rounding / truncation).
Across a random sampling of tristimulus (RGB) inputs, I see a resulting round-trip error in the amount of:
- Red Error: -3 ... +0
- Green Error: -1 ... +1
- Blue Error: -3 ... +0
For each of these deltas, it is the measure of the difference in a given Red, Green or Blue input values to the final Red, Green or Blue output values after an intermediate conversion to YCbCr. In other words, I have observed that in both the Red and Blue channels, the round-trip process causes an overall reduction in the resulting value, while the Green channel may either experience a slight reduction or increase.
Although the graph above shows only a random sampling, these limits were shown to hold true across a significant number of input combinations.
Even with the careful application of consistent quantization tables and chroma subsampling that identically matches the source image, you cannot resave a JPEG image without some small image degradation! This slight degradation is incurred in the color space conversion step.
Therefore, the only way to resave a JPEG image is to skip the decompression stage and perform matrix manipulation on the MCUs themselves without entering the quantization stage. This process severly limits what you are capable of doing in the way of edits.
For the purposes of the JPEG color space conversion (between RGB and YCbCr or YCrCb), the formula is given below.
Note that for the following formulae, the range of each input (R,G,B) is [-128...+127]. As mentioned on the JPEG Huffman Coding page, a level shift of (+128) will be required to get the standard [0...+255] range for each value.
|Coefficients||RGB to YCbCr||YCbCr to RGB|