Return to Digital Photography Articles
Variable JPEG Compression / Auto Compression
There are now some digital cameras on the market that will automatically select the level of JPEG compression for an image based on image content, file size or other algorithms. This page explores some of these new developments.
- An overview of JPEG compression and how it is influenced by Image Quality settings in your camera.
- A recent trend in JPEG imaging in a number of Point & Shoot digicams, along with a few DSLRs, that use variable JPEG compression / quantization tables. In some cameras (e.g. Nikon D200), this feature has been called file-size priority or quality priority.
RAW vs JPEG Basics
When a digital camera takes a shot, it needs to convert the raw image data from the sensor into a standardized file format for storage on a memory card. Typically this is comprised of one of two formats: RAW or JPEG.
While RAW photos offer the highest possible image quality (because the compression is lossless), it has the disadvantage of much greater requirements in file size. The JPEG file format (more accurately called JFIF, which employs the use of JPEG compression internally) offers a very good tradeoff between file size and image quality. A reduction in file size means not only less storage required for your photos on your computer (or archival media), but also faster frame rates and more shots that can be taken in your digital camera's fastest burst sequence.
In most cases, digital photographers choose to shoot in JPEG rather than RAW, for a variety of reasons, but this always means some loss of image quality in the process. Understanding what part of the image quality suffers is complex, but an important consideration for those who want to understand the tradeoff more clearly.
Image Quality / Compression Background
For the purposes of this page's discussion, it is strongly recommended that the reader have a basic understanding of the principles behind JPEG compression. In particular, an understanding of how quantization tables affect image quality would be advised.
Digital Camera Settings for Image Quality
Nearly all digital cameras on the market provide the user with a menu option that is generally labeled Image Quality or Compression. This option often has several different possible settings, typically named: Normal, Standard, Fine, Extra Fine, Super Fine, etc.
What do these settings really mean?
In most cases, these settings are simply an arbitrary name given to an internal scale factor number. This scale factor or Q-factor value is used to scale some internal reference quantization table that is hard-coded in the digital camera. By enabling the user to change the scale factor, different levels of JPEG compression can be achieved (thereby allowing the user to decide a suitable tradeoff between file size and image quality).
To avoid confusing the average non-technical photographer, a selection of two or three quality names (e.g. normal vs fine) is provided to the user as menu options, rather than the setting of the Q-factor itself. 99% of users would have no idea what the real difference between a Q-factor of 80 and 85 really means.
Digital Camera selection of Quantization Tables
In nearly all digital cameras today, the selection of the quantization table to use for JPEG compression is done by the user, and is left constant for all digital photos until the user changes the image quality setting. Thus, for every photo taken in this period of time, the same scaled quantization table will be used.
Why is this a problem?
Image content affects compression ratio
The compression ratio is simply the original raw image data size divided by the compressed JPEG data size (without any of the file headers, etc.). The compression ratio for JPEG images varies wildly, even for the same quantization table or Q-factor.
One of the problems with JPEG compression is that one can't predict the resulting file size after compression, given the Q-factor or quality setting. The final compressed file size is heavily dependent on the image content itself. Factors that increase file size (reduce compression ratio): fine detail in the image, noise and/or high ISO setting.
Advantages with fixed compression ratios
If the compression ratio can somehow be guaranteed (irrespective of image content), then the resulting file size can also be guaranteed. There are some situations where optimizing for a particular file size may be an advantage.
- Constant bit-rate for video encoding
- Guaranteed file size per photo of a given resolution
- Guaranteed number of photos per memory card
- Guarantee of max image frame rate, writing to memory card
- Guarantee of number of photos captured at max image frame rate during frame burst
- Reducing wasted file space due to high ISO noise
- Frame count remaining predication will hold true, even with ISO changes
Digital Cameras with Variable Quantization / Auto JPEG Compression
So far, I have come across the following cameras that exhibit this feature. Note that others may exist, so if you spot one, please let me know.
Sony DSC-H9 Image Quality (also H7 / H5 / H1)
Disclaimer: I do not own a Sony H-seies, but online full-resolution samples provide a wealth of information that allows some conclusions to be drawn or hypothesized. Web forums have discussed the possibility of image quality issues with the Sony DSC-H9 camera (e.g. noise reduction smearing, etc.), but this page will be limited to discuss the objective compression characteristics only.
Summary of Findings
- The Sony DSC H1 / H5 / H7 / H9 line of digital cameras all use variable quantization tables for their JPEG Compression. This is not a common feature amongst digital cameras.
- Variable JPEG compression provides a number of potential benefits, many stemming from the possibility for some degree of control over the final output file size.
- The selection of the JPEG compression level is determined automatically by the camera (without user intervention), and is calculated by some internal algorithm (likely intended to meet a target JPEG file size)
- The algorithm used to select this compression level is ultimately responsible for the degree of JPEG artifacts that may be present in the final image.
Comparison of DSC-H9 to other H-Series
It appears that the selection process for compression quality level in the Sony DSC-H9 may be weighted differently than for the other H series cameras. This may be resulting in smaller average file sizes and more compression than from its predecessors. See below for a quantitative comparison.
Compression Levels in Sony DSC-H series
After analyzing a wide range of photos from the Sony H series, I have observed a much wider selection of Q-factors (which influence compression coefficients) than I would have expected. What this means is that the amount of compression used by this camera can vary greatly from photo to photo.
The quantization tables that I see used by the camera are not custom tables as found in most other cameras. Most digital cameras have hard-coded two or three tables that are based on the manufacturer's special definition of the most appropriate quantization table. Sometimes this is based on the JPEG Annex K suggested tables, but most often (e.g. with Canon, Nikon, etc.) these are custom tables.
Even software tools such as Photoshop CS provide a custom table for each quality setting from 1-12.
In the case of the Sony H series, the tables are instead based on the JPEG Standard proposed tables, but with an integer prescale value used to reduce the quantization coefficients as per the IJG quality scale routine. By creating the quantization tables with the IJG quality factor scaled version of the standard tables, it is possible to produce any arbitrary JPEG quality setting without having to store countless tables in the camera firmware. This method won't lead to the most optimum quantization tables, but it does simplify the process greatly.
|Q||Q Lum||Q Chr|
|Integer Quality Settings|
Note that the range in quality values (of which the above table is only a subset) is very wide, and hence it is conceivable that the camera is able to select from a very wide range in JPEG "quality" values.
My preliminary assumption is that this camera is attempting to optimize the JPEG compression to achieve a certain desired file size per photo (of a specific resolution).
High Quality versus Low Quality
With the potential to select from a range of quality values from (for example) 75 to 98, a very large range in compressed file sizes is possible for the same image content.
Selecting a quality value at the low end of the range (e.g. 75) will introduce a significant amount of JPEG compression with the accompanying loss in detail, the addition of blocking artifacts and other issues. The benefit is the reduction in file size.
On the other hand, selecting a very high quality value (e.g. 98) will allow nearly all of the original image's detail to be preserved and the resulting compressed image will be virtually indistinguishable from the original. The cost of this is clearly an increase in file size (because of lowered compression).
Not all Images are Alike
Competing with the above basic view is the fact that not all image content is alike! A photo that depicts a clear blue sky with very little else is much easier to compress than a closeup image of pebbles and sand, taken at a high ISO. There is more information content in the first example than in the second example.
Compressed with the same JPEG quality, the sky photo will have a much smaller file size than the sand photo.
|Original Image Detail||Compression Quality||... Resulting File Size|
If a camera attempts to use variable JPEG compression to keep the output file sizes constant, what happens?
A difficult tradeoff
|Original Image Detail||Target File Size||... Resulting Compression Quality|
Because we are now attempting to restrict (keep constant) the file size, we have to select a lower image quality (higher compression) on images that have a lot of image detail to begin with. Conversely, we can select a higher image quality (lower compression) for images that don't have a lot of image detail.
Therefore, we'd expect the simple sky photo to have a high JPEG Quality value and good detail reproduction where it may exist in the original. However, in the photo of the pebbles, we would probably see a loss in detail throughout the image.
Restricting compression to meet a given output file size is not good for photos that contain a lot of detail. This feature works well for mediums where data rates have a fixed maximum bandwidth (such as video tape recording and DVDs), as we can overlook the loss in detail for fast moving scenes. But we are a lot more critical of still photos, especially when the image should contain a lot of detail!
The key to how well this works is the decision about what JPEG Quality setting to use for a particular photo.
Possible Implementations of Variable JPEG Compression
Obviously there are many ways that variable JPEG compression can be implemented, and it is especially difficult to guess how a camera may be approaching the issue. That said, there are some indications about what may be happening.
The following discussion assumes that the camera is attempting to create JPEG files with the same file size.
Theoretically, it should be possible for a digital camera to select the optimum JPEG quality value that results in an image that is very close to the target file size. Several attempts at compressing the same image with different quality levels will eventually turn up a quality factor (and hence quantization table) that results in a compressed file size close to the desired size. Remember that it is not possible to predict how compressible a photo is (with normal JPEG) until one goes through the steps of doing the compression.
Towards each end of the available compression quality scale limited by the camera, we would expect that there would be no further bounding of file size. In other words, if an image were completely blank (white sheet of paper), we would probably see the highest quality rating used and a file size that is still less than the target size. Conversely, with a very noisy photo of something with significant detail, we might see the lowest quality rating used and a file size that still exceeds the target size.
Educated Guesses about Image Compressibility
Performing JPEG compression is very computation-intensive, so it is not feasible to go through countless compression attempts with various compression levels before finding the optimum one. So, the camera will make some educated guesses about the values to try The Microsoft patent described later is an example of a clever approach to this mechanism.
Instead, it is likely that this educated guess may be slightly wrong, resulting in a non-optimum quantization table selection which then leads to a compressed file size that is not really near the target file size.
So, if the camera is trying to target a resulting JPEG file size of 2.5MB, you may end up (for example) with actual file sizes anywhere from 1.5MB to 3.0MB.
File Size Targets for Sony H Series?
Working with the assumption that the Sony H9 series digital camera is using the variable JPEG compression to optimize for a particular file size, we can make some rough estimates about what these goals may be. Even though the cameras will produce compressed file sizes that deviate from the target amount, the error in the "educated guess" part described earlier can easily explain most of these differences.
As I only currently have about 40 photos per camera to work with, I have chosen not to post my comparison of average compression findings between the Sony DSC-H5 and Sony DSC-H9 cameras. These sample sets are not large enough to draw an accurate conclusion.
Sony DSC-H9 / H7 vs Sony DSC-H5
With the above caveats and disclaimer, my limited subset of photos seem to imply that the Sony DSC-H9 may be using a lower file size target (and hence higher average compression ratio) than the Sony DSC-H5, even though the resolution is higher (3264x2448 vs 3072x2304). As an aside, my subset of H9 images has a compression ratio of ~10.5:1. What's more: my Sony H9 photo set contains an abundance of high ISO images, which should have only increased the set's average file size.
NOTE: If you have encountered images that exhibit Approx Quality Factor % (as reported by JPEGsnoop) outside of the values listed in the table below, please let me know! Also, for anyone else with an H5 or H9, I would be interested in hearing your anecdotal average file size of collection (and number of photos).
|Camera||Quality @ Norm||Quality @ Fine|
|Sony DSC-H9||Nightshot: 76-95 Normal:90-98|
[FYI, the image that produced the lowest quality setting from the H9 was a Nightshot. Limited to non-Nightshot photos, I have still observed quality factors as low as 90 for High ISO photos. Other users have noted non-nightshot photos reporting quality levels in the mid 80's]
As for compression ratios, my sample set shows a definite bias towards higher compression in the H9 versus all other Sony DSC-H series cameras. NOTE: In the following, I have tossed out some of the extreme outliers, and tried to give an approximation of the median ranges.
|Sony DSC-H1||6.3:1 to 8.5:1|
|Sony DSC-H2||6.1:1 to 7.8:1|
|Sony DSC-H5||6.4:1 to 8.4:1|
|Sony DSC-H9||8.5:1 to 12.4:1|
File Size Histogram
The following histogram shows the distribution of file sizes from a Sony DSC-H7 digital camera, which presumably uses the same tuned level of variable quantization / auto compression as the DSC-H9. This data has been compiled from a reader's collection of over 1200 photos, which should provide a rough view of the expected file sizes from the camera.
From the above data set, the median file size was approximately 2.7 MB for 8 megapixel photos. While this sort of metric is not suitable to analyze for most cameras (as it is extremely variable on the basis of image content), it does have some applicability when considering cameras that may implement file-size based compression goals.
H9 Missing Fine / Standard Setting
The table above shows the range in compression quality values that I have observed in images so far from the different models. It is certainly possible that the previous models in the H-series may use lower compression quality settings for some image types, but I have not encountered them so far. Nonetheless, the trend seems to imply that the H9 might be using more compression than previous models.
As the Sony H9 does not appear to provide a user-selectable setting that allows low quality vs high quality tradeoffs to be chosen, the user is heavily dependent upon the camera's internal choice of quantization tables. Unfortunately, the trend for the non-configurable range of the H9 appears to be one that is not only wider but with a lower average image quality (higher compression). In other words, we are comparing a much tighter range in JPEG quality from the H5 (if we leave it in Fine mode) versus the automatic range selected by the H9.
How Noticeable are these differences?
The most important thing to understand is that most casual viewers will not be able to recognize the nuances associated with JPEG compression (artifacts, blocking, etc.) when high (95+) quality settings are used. Looking at an image on its own, one would be hard-pressed to find any noticeable drop in image quality with these very low compression amounts.
For reference, one can compare image qualities from different cameras to give you a more objective reference. Most recent digital cameras (point & shoot) use an equivalent JPEG quality factor of 93 or higher when set to their highest quality setting (e.g. fine, super, etc.)
So, to some degree, the concern over the variable JPEG compression's effects on the Sony H9 could be overblown somewhat. Even though the camera on average chooses a lower compression quality (with resulting smaller file sizes), it may be difficult to recognize in most prints. Some artifacts may be visible in 100% crops (known as pixel-peeping). It is possible that the camera manufacturer decided to make a tradeoff between burst buffer performance (e.g. 100 shots at high shot rate) versus image quality.
However, more of the concern is probably sourced from the instances of certain images that contain a lot of detail (which don't compress well). It is in these photos that the camera appears to select low compression qualities (e.g. 84-90), resulting in some potentially noticeable loss of detail and introduction of visible JPEG artifacts. This may be further exacerbated by any initial noise reduction algorithm that could be enabled in the camera.
Differences in Compression Quality / Objective
If this difference is true, this would be a very surprising result. Unfortunately, it is also a very difficult characteristic to prove, unless one had the ability to shoot multiple camera models under controlled conditions (same image content, etc.). Without that, the only way to increase confidence in such a hypothesis would be to look at the averages from from very large photo collections.
Does this seem likely?
While most camera manufacturers aim to increase image quality across successive models, it would be hard to imagine any reason why there would ever be a decrease. So my immediate reaction is that this is a misleading result and that it is simply due to too small a sample size.
On the other hand, if such a reduction does in fact exist, then the only plausible reason that comes to mind is the published burst photo specifications:
Sony DSC-H9 Camera Specifications
100 shots @ 2.2 frames per second @ 8 megapixels
The above specification has been provided in the H9's press releases. With this continuous shooting performance comes a hardware and processing requirement that is non-trivial. If the shot buffer is storing JPEG images (i.e. following JPEG compression), then there would normally be a variable bandwidth and storage requirement which is dependent upon image content and ISO setting.
Having some means to restrict this bandwidth requirement would facilitate a camera's ability to guarantee a certain shot buffer performance.
Reducing the file size target (or increasing the average compression ratio) would make more photos fit in the JPEG photo burst buffer. If the 100 shots characteristic were a very important marketing feature to meet, the possibility of a change in target / compression ratio might be plausible.
Microsoft holds a patent that is very much related to this technology, with an end-goal being the optimization of compression settings to target a given file size. If Sony is aiming for a similar goal, then they would need to take a different approach.
- Microsoft - US Patent 6614942:
TITLE: Constant bitrate algorithm for block based image compression
INTERPRETATION: Variable quantization scaler selected by computing a sample compression attempt on two image subsets. Based on the resulting image compression of these two attempts, the target quantization table scaler (Q-factor) can be derived by interpolating from the results of the two attempts. The hope is that the interpolated quality factor will achieve the desired compression ratio.
Soon, I will be posting a deeper summary of the above patent as it describes a fairly intuitive method by which one may implement variable JPEG compression.
At this point, I don't see a patent by Sony on this exact technology, but they do have a couple older patents in related compression quantization methodologies. These are posted here for interest purposes. I am currently looking into their more recent inventions.
- Sony Corporation - Europe Software Patent EP692881:
TITLE: Quantization apparatus, quantization method, high efficiency encoder, high efficiency encoding method, decoder, high efficiency encoder and recording media
- Sony Corporation - US Patent 5374958:
TITLE: Image compression based on pattern fineness and edge presence