Return to Digital Photography Articles
To provide the ultimate in protection for your data files, one should consider the benefits of online backup or remote backup. In an online backup, your data is automatically transferred to a geographically different location — one that puts your data safely out of reach of any disasters at home.
Remote / Online / Offsite Backup
You're proud of yourself for having backups of all your data... The backups might be on DVD, other hard drives or even a redundant RAID server. Now consider a scenario where your home has suffered a natural disaster (fire, flood, etc.) or theft -- are your backups truly safe? While some may dismiss these events as unlikely, the potential effects of losing your data could be devastating.
Enter the ultimate backup strategy: remote backup.
Benefits of Offsite Storage
There have been countless examples of people's homes being damaged in a fire, easily consuming any careful backups that may have been created. For many, the loss of their entire photo collection and documents is far more troubling than the process of replacing possessions through an insurance company. An even more likely scenario: theft. In a break-in, computer equipment and external drives are easy targets. If your backups are located anywhere near your computer, your risk of complete loss is high.
An offsite backup hides these backups in a completely different location, one that can easily be recovered should you have an incident at home.
Do-it-yourself Automatic Offsite Backup
Offsite backup can be accomplished in one of several ways: the manual method and the automatic method. In a manual method, you burn your backup jobs to DVD or to an external hard drive and walk these to some other location, such as your office, for storage. There's one problem: laziness! I am a firm believer that you shouldn't rely on yourself for a backup strategy -- as over time you will get lazy and fail to do it as often as you should. It's also a hassle.
Instead, this page will examine what it takes to implement an automatic offsite backup methodology.
For online backup, you have two choices:
- Pay a company lots of money each month to provide remote storage to you, or
- Provide your own!
Most people will be shocked to see just how expensive the offline backup sites are. If you are expecting to backup more than a few gigabytes, be prepared to spend a lot of money each month. For a 30 GB backup, you can easily spend $50 or more per month. Obviously, there are inherent advantages of paying someone else to host your backups (they'll take the blame if something goes wrong!)...
But if you take a few precautions, you too can create your own offsite backup for free. Obviously, you won't have a team of support staff paid to watch the blinking LEDs 24/7, or a SWAT team protecting the premises, so you'll have to weigh out the benefits carefully with your particular needs in mind.
Preferring to do things myself, I set out to create my own offsite backup setup.
Thankfully, creating an automatic offsite backup strategy is extremely easy and affordable. Most people already have all they need, with the exception of a Network Attached Storage (NAS) drive.
- Backup Software
You will need to use a suitable backup program for remote operation. There are many available, and very soon an article will be posted here describing a search for the best backup software.
- Network Attached Storage (NAS) drive
This must be a real NAS box, with FTP support. Don't get fooled by the cheaper Network Direct Attached Storage (NDAS) devices such as the Ximeta drives. You can easily find a NAS for less than $1 / gigabyte. In my case, 250GB was sufficient. I selected a Buffalo Linkstation Pro 250GB NAS, which will be the subject of another article soon.
Most routers will be suitable for this purpose. However, some models feature options that will make your setup even more secure: VPN tunnels and the ability to remap IP address ports.
- Free Internet Access!
You'll be reliant on a friend with an always-on internet connection, but unless they're nocturnal, most people don't use their connection in the dead of night.
Where should you put your NAS?
In order to house the equipment listed above, you'll need to find a willing partner... someone who has broadband internet that is always connected (e.g. Cable modem or DSL), and who you can trust. They shouldn't be the stingy type that has a problem with you using their internet connection when they're not on the computer. All this person has to do is allow you to connect your NAS box to their router and make some very slight modifications to the router settings.
Ideally, they aren't using their PC for heavy data transfers at night (if so, you'll have to configure your backup software for bandwidth throttling to be fair). Also, it will be much easier if they don't have their own FTP server that they are already running behind their firewall / router, exposed to the internet. If so, you'll need to reprogram the router to map your NAS box's FTP ports to something other than port 21 (using UPnP Forwarding or Port Forwarding).
Looking for a suitable place? Start with your friends and family. It is highly likely that you will find someone who is willing to let you place your NAS box at their site. You can also consider a trade: they provide you with a location and some of their night-time internet bandwidth, while you provide them the same in return!
All that said, you should avoid selecting your own home as the "suitable" location! Doing so will eliminate one of the benefits of offsite storage: protection against location disasters such as fire, flooding, theft, etc.
Practical Expectations for Offsite Backup
For standard residential cable and DSL internet access, upload bandwidth is often capped to data rates much lower than your download bandwidth. Remember that data rates are generally measured in kilobits per second, not kilobytes per second. Divide kbps (kilobits per second) by 8 to get KB/s (kilobytes per second).
Typical cable upload bandwidth = 128 kbps, 256 kbps or as high as 1000 kbps (if uncapped)
Typical DSL upload bandwidth = 64 kbps to 1500 kbps
In my environment, I am seeing roughly 360-600 kbps upload bandwidth from cable access. My download bandwidth is 600-1600 kbps.
Given the limited upload bandwidth, one should be realistic about what they hope to achieve with offsite / remote backup. Assuming that one automates the backup process for night time operation (outside of business hours) and that the backup should be completed by the next morning: (you'll have to adjust for your data rate and time requirements)
10 PM - 7AM = 9 hours = 32400 seconds @ 360 kbps = 1423 MB = 1.4 GB / night
This might be practical for a daily incremental backup, but it wouldn't be suitable for any full backups, which may be in the order of tens or even a hundred gigabytes. Make sure that you take into account that verification modes will probably require a download from your NAS, doubling the required bandwidth aggregate.
Full Local and Incremental Remote
In light of the upload bandwidth limitations, I run my first backup (the full backup) to the storage device (NAS) directly connected to my local LAN (or directly to my PC). Then, I move the NAS offsite and do all incremental backups remote over the internet using FTP. This way I can transfer the 90 GB of initial backup data onto the device in a matter of hours, but still have the benefit of offsite storage following this initial data transfer.
In a direct connection setup (Linkstation Pro via a Gigabit Ethernet NIC) mounting as a local network drive, I get approximately 10GB / hour with compression and AES encryption enabled. Note that to connect the Linkstation directly to my PC, I had to configure the static IP address of my local ethernet card to the same subnet as my Linkstation (e.g. 192.168.1.xx).
Digital Photos Import
Even with a incremental-only remote setup, I have to be aware of realistic daily storage limits when I am importing a day's worth of digital photos. It's not too uncommon for me to take more than a gigabyte of digital photos in a single day. Because of this, I would strongly recommend performing some of the intial filtering / discard in a non-backup folder hierarchy prior to the automated backup process.
Unfortunately, the very nature of offsite backup lends itself to security concerns. There are three areas to be concerned about:
- Offsite server location
Since your files are stored on a server in some other location, one naturally has to be aware of the fact that its possible for the remote location to also suffer the same problems your home may: theft, fire, damage, etc. If you are relying on a commercial online backup facility to host your backup server, then security is likely far in excess of what you could provide in a home environment. Safety is guaranteed by extra monitoring and redundant backups of your data. These additional services are the main value-add that a commercial online backup site will provide to you.
- Offsite server hacks
You have to expect that hackers are continually probing for open ports on the internet, attempting to find a poorly configured / protected server. If you are using a commercial online backup site, then there is an expectation that the system has been audited carefully for potential vulnerabilities.
- Data transfer
In most cases the data transfer from your home to the offsite location will be using an insecure IP protocol across a variety of ISPs (Internet Service Providers). It may be helpful to perform a traceroute to identify the typical path your transfers will take. The more hops that your route takes, the more you expose yourself to the possibility of traversing an unmanaged switch (which could potentially allow someone to sniff your backup transfer packets). As I encrypt all of my backup jobs, I have no concerns about anyone capturing all of these -- they can't decrypt them.
You must Encrypt your Data!
In light of the above, a home-brew online backup mechanism will be inherently less secure than one you pay someone else to do for you. Therefore, you should expect that hacks will happen, and that it is possible (although highly unlikely) that someone could download your backup jobs.
Since I have encrypted all of my backup data with an industry standard AES encryption, these data files will be virtually useless to anyone. Note that you will be reliant on encryption, not just password protection.
As an example, I examined the backup data from one of my password-protected (not encrypted) backup utilites, and it turned out that the data was stored in the backup file plain-text! Only a simple flag and 256-bit hash of my password was all that differentiated my password-protected job from a non-protected job! It would take me less than a minute to remove the password protection from any backup job created by this program!
When encrypting your data, you should select a strong password (mix of letters and numbers) with a length that allows the key to be more secure. For example, to get the most strength out of 128-bit encryption, you should provide a passphrase that is 16 characters or more long (8 bits per character).
Because a hacker may not be able to do anything useful with your backup data, they may look for other nefarious things to do. It is possible that a remote hacker could use your FTP server for their own purposes. Therefore, it may be worth monitoring your server logs occasionally just to ensure that other users are not taking advantage of your storage device.
Many backup utilities and NAS devices can automatically email you in the case of problems. If you are going to use such a mechanism, then it is worth enabling SSL or other protection for your SMTP (mail server) access, if possible.
FTP data transfer is not encrypted. Some NAS devices support SFTP (Secure FTP), which encrypts the data transfer between your computer's backup software and the remote storage device. This is an excellent feature if your setup provides it. While most backup software supports SFTP, many affordable NAS devices don't. Unfortunately, the processor requirements to support SFTP in the NAS box makes it a less common feature.
Setting up your remote System
Once you have performed your full backup locally (an upcoming article will show a comparison of backup software), you can transport your drive to the new location and configure it for online use:
- Connect the NAS to the Router
Simply plug in your NAS box to one of the router's ethernet LAN ports
- Assign a static IP address to the NAS
In your NAS device configuration options, give it a static local IP address such as 192.168.1.77
- Add FTP port 21
Connect to your router's administration page (e.g. http://192.168.1.1/). Open up the Port Forwarding options and set the IP port 21 to map to your NAS device's IP address. With my Linksys router, this option can be found under Applications & Gaming -> Port Range Forward . Set the start and end port to 21, protocol to TCP and the IP address to the static IP address you assigned above. Make the sure Enable checkbox is set and click on Apply Changes. If the router already has a computer with a FTP port active, then you may have to use UPnP Forwarding options to remap the port number.
Dynamic IP Addresses
One potential problem you may encounter when trying to put your FTP server in a friend's house is that their internet service provider may only grant them a dynamic IP address. In the ideal case, the router that you are connecting your NAS to would have a static IP address (one that doesn't changes). Static IP addresses are a lot more expensive to run, so most people will not have any guarantees that the IP address of their router will always stay the same.
What does this mean to you?
In order to connect to your NAS box behind your friend's router, you will need to type in their router's WAN IP address into your backup software's FTP settings section. Any time that their ISP (Internet Service Provider) decides to change the IP address, you will have to determine what the new IP address is, and then update your backup software settings.
Fortunately, it has been my experience that dynamic IP addresses rarely change (perhaps once or two per year), so this is not much of a concern when using the local ISP.
If your ISP changes the dynamic IP address more regularly, then you may want to consider using the free DynDNS (dynamic DNS) service. Otherwise, you may need to ask your friend to read out their IP address shown when they surf to whatismyip.com.
How to Workaround Dynamic IP Addresses Changing
Thankfully the setup I have been using rarely has its dynamic IP address changed. However, if did end up changing frequently, I was intending to implement one of the following automated workarounds:
- Use a free service (such as no-ip.com or dyndns.org) that gives you a "permanent" named address for your site, in addition to an Update Client (that is responsible for tracking changes to your IP address). This is the method I am currently using, with great success. This is by far the best approach to use.
- Modify NAS box to ping a page on your webserver periodically. Accessing a .php page via the GET protocol will allow the PHP script to extract the IP address using the $_SERVER["remote_addr"] variable. This PHP script then saves the address in a location that I can then use to modify the backup software/script. As I decided against hacking the LINUX install on my NAS box, I decided against this method.
- Install a "call-home" script on another computer that resides in this other location, which again accesses a web page on my web server. This type of script can be performed very easily with a typical batch job. For example, you can simply execute get <Website URL>, where <Website URL> is the location of your web server and script! Again, have the PHP script record the IP address (or email it to you) and you're done!
Of course, if you don't have your own webserver, you can instead take advantage of the scraping-friendly whatismyip.com page! Simply issue a get http://www.whatismyip.com/automation/n09230945.asp from your remote computer and you will be returned the IP address of your remote network in plain text.
Online Backup Services
If something I have said above scared you off from hosting your own do-it-yourself offline backup facility, then you may want to consider handing over your cash to an online backup service provider. These services will give you peace of mind in exchange for a monthly bill.
- Guarantees in uptime
- Redundant protection
- Some providers have varying levels of rollback capability
- Extremely expensive
- No benefit of onsite full backup (if doing a large backup)