Return to Digital Photography Articles

What's wrong with backing up to my Hard Drive?

One of the most important differences between film and digital photography is the ease by which years of hard work can be wiped out within seconds. Most digital photographers simply store their entire photo collection on a hard drive and expect that the drive is a safe storage medium. Wrong!

Hard drives should only be treated as temporary storage, not long-term archiving. Drives have delicate moving parts and they DO FAIL!. Personally, I have had two drives fail on me: one caused 100GB loss of video work, the other was a drive that was starting to fail -- one that contained my entire photo collection!

That said, some people have used hard drives for long term storage, but these are typically offline drives (i.e. drives that are generally disconnected & off). For the price and convenience factor, it may be a worthwhile option.

Hard drive crashes are a definite possibility, and can occur for no immediately obvious reason. But, of course there are other more obvious failures, such as vibration, water, power disruption and even viruses. Thankfully, some drives start to make odd noises (typically clunking sounds) before they completely fail. This is what occured to the data drive that contained my entire photo collection. Once my drive began to clunk every couple seconds, you can bet that I stayed up all night desperately trying to recover what I could from the failing drive. This is bad practice, and it was a wake-up call for me to establish a much more robust archiving strategy.

Everybody says that they back up their photos.... but how often? It is a nuisance. And if you are adding new photos to your collection every couple days, you can bet that you will often put off the backup or forget to do it.

Rule #1 - Backup to archival media

The media selected should not involve moving parts! From a convenience point of view, the best choice would be DVD-R, followed by CD-R. DVD-R provide you with 4.7GB of storage (enough for most serious amateurs' single-day work). CD-Rs only provide roughly 650MB, which will quickly become tiresome (as it will involve many more discs). However, it has been widely mentioned that CD-R discs provide better longetivity & durability than DVD-R. There are a lot of questions in the online community about the longetivity of these discs and how DVD-R and CD-Rs compare.

Suffice it to say that one should always use quality name-brand media, take extreme care (both in writing on the labels, storage, etc.) and assume that someday you will have to transfer them to another disc. Periodic re-verification and re-burning should be a consideration. Even still, there are many reports of discs failing well before their manufacturer-rated lifespans! Using true archival media (not just name-brand discs) will provide far better protection from these early failures.

Rule #2 - Redundancy

Starting with rule #1, archive to some external media, but make duplicate copies. It is relatively simple to make doubles of these archive discs. Keep one at home and other in another location. If possible, use another brand of disc media. This way, if one brand begins to fail over a number of years, you should have another version to work with (this assumes that you periodically check your backups). The idea behind different locations is simple: you would like to protect against things that happen at home (fire, theft, etc.), enabling you to simply recover from the collection housed at another site.

Rule #3 - Automation

Nobody enjoys the backup process... not only is it time consuming, but it needs to be done often to reduce the cost of a failure. If you can work with some sort of automated process, you are much more likely to maintain the currency of your archive. It is relatively easy to set up tasks that occur every night to perform some means of backup. Whether this is simple mirroring of your photo collection to another drive, or transfering the incremental changes to a remote FTP server, you want something that involves you as little as possible!

 


Reader's Comments:

Please leave your comments or suggestions below!
2010-06-24kmag
 1Guaraguao: nobody can tell you for sure if those discs are good for 300 years. That 300 years is just their best guess based on their knowledge of chemistry, physics, and a bit of testing. (They probably use bright UV flood lamps, elevated temperatures, and maybe some vibration for a few months to simulate a few years of aging, then extrapolating that aging out longer. A friend of mine did similar things for HP in order to estimate how long their inkjet cartridges can just sit there in your printer unused.) Until they have a big batch of CDs that has been stored for 300 years, the rest is just guesswork.

Here's an insight: things break, and things break in ways you don't expect. For a long time, the way companies built reliable systems was to buy really big really expensive machines. They paid a lot of money to computer companies that had their engineers trying to maximize the reliability of each component. I remember hearing that Altavista ran on one DEC server with a terabyte of RAM, back when a terabyte of RAM alone cost several million dollars. The way reliability was done was to design each component to never fail.

Then, along come some scrappy engineers who figure out that if you design systems to work in the face of failure, you can get the same reliability for much cheaper. RAID (Redundant Array of Inexpensive Disks) arrays and things like Google's vast farms of hundreds of thousands or millions of machines made from commodity hardware beat systems designed from the best components money can buy.

If you're trying to buy great media that will last more than 20 years, I think you're probably going at the problem the wrong way. You want to figure out a way to store your photos such that you'll be okay even if your CDs only last a few years. At that point, the question of buying the 300-year media boils down to one of expected cost over the lifetime of the data. I imagine in 20 or 30 years, you'll have all of your data on some other format anyway.

Anyway... your wedding photos, and your kids baby photos: get a GMail account and a Yahoo email account and email the photos to yourself, to both accounts. Get a Picasa account and put those photos up there. The level of storage redundancy GMail has is going to out-do anything you'll do at home. If you have a spare thumb drive sitting around, go ahead and put your most important photos on there and put it in your safe deposit box over at the bank.

I was on vacation and daily backed up my photos to my laptop as well as a thumb drive I brought with me. I was lazy about making a third backup, and a couple months later my laptop's hard drive failed. My thumb drive corrupted a few bits of this beautiful photograph taken from a mountainside looking down on this little Austrian village with snow-covered roofs covered in morning fog... which is why I came across this site... and which is why I'm now much better about making sure all of my photos are on at least 2 hard drives and one CD-R/DVD-R.

I have a Linux machine set up with 1 drive for the operating system and 2 drives set up using RAID1 (they look like one drive... anything written to one drive gets written to the other at the same time) for all of my photos. I don't think Microsoft lets you set up software RAID unless you pay for a license for a server version of Windows, but I think some of the Network Attached Storage (NAS) boxes you can buy support RAID1. Or... you can keep one copy on your desktop's internal hard drive and keep a backup copy on an external hard drive.

I've written a script to rename my photos to include a checksum of the data in the name of the file. I also wrote a script that will take two files that are supposed to be the same file and find all of the differences between the two files, and figure out which combinations of differences results in the checksum matching the original checksum embedded in the file name. This way, even if I have two corrupt copies of a photo, there's a chance I'll be able to recover the original file as long as the two corrupt photos aren't corrupt in the same place.

Maybe some day I'll get around to reading the UDF filesystem specs well enough to write a utility that will fill up the unused space on a DVD-R with Reed-Solomon error correcting data in order to maximize our abilities to recover data from age-damaged discs. (At a physical level, CDs and DVDs already use error correcting codes, but I'm talking about adding much more redundancy, and using RS codes across blocks.)
2009-04-181Guaraguao
 Is there any data available on the Gold CD's and DVD's sold by Delkin Technologies? I am buying this brand to back up my digital images as a security measure. How good are these? Do they really stand up to their 300 year longetivity claim? Thanks.
2008-03-10Arved
 Thanks for the quick response, Calvin. I thought you were on vacation! :-)

The article I cited is but one of many articles I've found extoling the virtues of DVD+R over the older DVD-R standard, so I'm not sure what additional discussion you'd like to see. If you want to see more, please contact me by e-mail, and I'll be happy to furnish some additional links.

As far as compatibility, I don't think there's a DVD drive manufactured in the last couple of years that doesn't support both DVD-R and DVD+R. Certainly, one can be picked up for under $50, so I don't consider the possibility of DVD+R incompatibility to be an issue. If you're using a drive that's DVD-R compatible, but incompatible with DVD+R, it's ripe for replacement. Compatibility issues tend to pop up with Dual Layer and DVD-RAM support.

About the only reason I can think of for manufacturers not to produce archival quality DVD+R media is that they've already provided DVD-R media, and don't see the market able to support another standard. Kind of like Beta Max.

The fact that there are no archival DVD+R media is puzzling, but it's mostly a moot point. If I was a manufacturer seriously courting the archiving community, I'd be manufacturing archival quality DVD+R meda. However, we have to choose among the products that are available. So, is an archival quality DVD-R better than a non-archival, but top grade DVD+R (such as the DVD+R disks manufactured by Taiyo Yuden)?
 Just got back (far too much to catch up on now!) As far as compatibility issues go: my problem was with trying to get maximum compatibility for clients, some of whom had much older/cheaper DVD players. In older players (even dual-format), it appeared that DVD-R was generally a safer choice, along with a slower burn rate. Your question re: DVD+R vs archival DVD-R is a good one; the lack of archival DVD+R must add to some level of confusion amongst those looking for the safest storage mechanism. Thanks for your insightful comments, Arved.
2008-03-09Arved
 There are many technical advantages to DVD+R over DVD-R, yet none of the big names (Taiyo-Yuden, Mitsui/MAM-A, Delkin) have archival quality DVD+Rs. "...The technical advantages of the DVD+R(W) format will with time turn into faster, more powerful and more reliable drives for end users." (http://www.cdfreaks.com/reviews/Why-DVDRW-is-superior-to-DVD-RW). I would think that, for an archival solution, reliability would be paramount.

So, do the archival qualities of an "archival" DVD-R surpass the greater inherent reliability of DVD+R format?

Basically, I'm torn between Taiyo Yuden DVD+R media and something like Delkin or Mitsui/MAM-A Gold Archival DVD-R media.
 Good point and great article link. I'm sure there are many out there facing the same question. The first time that I was hit with poor read compatibility at a client, I started to examine the qualtitative differences in media quality across various brands (examining PI/PO parity inner/outer failures/errors), and immediately switched over to Taiyo Yuden. At the time I had stuck with DVD-R as player compatibility was vital, not longetivity. With DVD+R offering improved error robustness, I am gradually making my shift. However, I have not yet seriously considered moving to archival media (I'd like to see some more indepth coverage of precisely the issue you're talking about).
 
2008-03-09Arved
 There are many technical advantages to DVD+R over DVD-R, yet none of the big names (Taiyo-Yuden, Mitsui/MAM-A, Delkin) have archival quality DVD+Rs. "...The technical advantages of the DVD+R(W) format will with time turn into faster, more powerful and more reliable drives for end users." (http://www.cdfreaks.com/reviews/Why-DVDRW-is-superior-to-DVD-RW). I would think that, for an archival solution, reliability would be paramount.

So, do the archival qualities of an "archival" DVD-R surpass the greater inherent reliability of DVD+R format?

Basically, I'm torn between Taiyo Yuden DVD+R media and something like Delkin or Mitsui/MAM-A Gold Archival DVD-R media.
2007-01-031Guaraguao
 In my eternal quest to preserve a lifetime of work, I have been doing great amount of research on CD longetivity and opted to have all of my digital files stored on 2 different brand names external hard drives (Iomega and CompUSA brand) plus I have 2 backup copies in CD-R's. I never buy cheap or unknown brands of cd's no matter how low they are priced.

I am now beggining to store my files on DVD+R's because my collection is getting larger as time passes and need to save space. At the moment I am using Maxell brand because I can get this brand easily and its a company that has been on the market for nearly 30 years. In my cd collection I also have other brands like Sony, Memorex, TDK, etc. that have perform well for over 2 years. I think handling and storage is an important factor because I have a cd burned in 1998 (Samsung) and I can still open and see the contents without any problems.

Since I live in the caribbean Island of Puerto Rico, humidity is high, averaging 65%, I have found it beneficial to store my CD's in dark airtight containers with Silica Gel pouches and monitor them every other month. The important thing is write the date you burned the Cd or DVD. That way it will be possible to know the average life expectancy of the particular brand media you use.
2006-12-21Janos
 Please don't forget to mention, that while DVD-R is a good medium for backup but not good for archiving purposes.

I use DVD-R for archiving my photos since years. But my branded DVD-Rs (usually verbatim, tdk) start to fail after 1-1.5 year!
I realized just yesterday, that my 1 year 3 month old archive DVD fails on one JPG. Successfully only one file and I have 2 more copies of the same DVD - yes, I regularly recopy the archive DVDs to newer ones after 1 year.
It is very annoying, lot of work but absolutely necessary :(

best regards,
hvj
 Excellent point, Janos! Having failures after only a year is a very scary prospect. Recopying (with testing) is a necessary step if the risk of early failure is a concern. I think an important point is to consider using true archival media, not just name-brand discs. The accelerated lifespan measurements (predictions) by the manufacturers certainly make a number of assumptions that may not be realistic in real-world treatment of the storage media.

However, 1-1.5 years sounds like there may be other issues at fault. To start with, I would have a careful look at DVDs you're using. Compare them to the user-submitted test results posted on CDFreaks Media Page. I always use Taiyo Yuden media as they have been highly regarded in the comparative reviews, plus they have performed far better in all of the PI/PO tests that I have performed.

Lastly, I would also look at the way these discs have been labeled (using disc-safe markers) and stored. These minor items can have considerable impact to the lifespan of your ability to recover the archived data down the road. Good luck and thanks for pointing out the potential pitfalls! I have made some updates to the page to reflect these valid concerns.
2006-06-13Chris
 I use an external hard drive and backup on cd-r's. I find imation cd-r's to be rather good quality. Is there another brand of cd-r for me to consider when backing up photos? I have approxamately 100 gig of photos backed up. I really value this question. Thanks!!
 Imation has been rated pretty well for CD-Rs (they are made by Taiyo Yuden). As for other decent brands, it seems that the following alternatives are reasonably well reviewed: Kodak, Philips Gold, TraxData Gold, Philips Silver, Richo Premium, Sony, TDK. Most of these are produced by the same factory.

While some argue about the long-term longetivity of DVD-R, I would strongly favor DVD-R for archiving over CD-R, just for the ease of handling / use, especially in light of your 100 GB comment! There are some very good DVD-R manufacturers, again with Taiyo Yuden being recognized as one of the better brands. For DVD-R media comparisons, I would suggest checking out the www.videohelp.com DVD media list. Hope that helps!

 


Leave a comment or suggestion for this page:

(Never Shown - Optional)
 

Visits!