Drobo and Amazon S3, possibly the best backup partners!

Many months ago I decided that I wanted a decent backup system for my massive photograph library. I’ve been using Apple Aperture to manage my photos, and the library has been growing and growing. Our main computer is my work station and the family computer, so the wife has been adding to the iPhoto library as well.

This year I upgraded my camera from the 6 megapixel, Nikon D40 to the 12.3 megapixel, D300. That’s over double the amount of megapixels in each photo now. This converts into much larger file sizes too!

I had the iPhoto library local on the Mac Mini back then. The Aperture library was being managed on an external LaCie harddrive. That system worked, but had no backups. If I lost a harddrive, I’d have been sunk. So I committed to reorganizing my file management system, so that I’d have an increased likelihood of not loosing data. The Drobo was my solution. With its built-in redundancy, I was able to also be free to upgrade, at minimal cost. Earlier this year, I got the basic Drobo, and two 500 Gb drives. This didn’t give me an outrageous amount of space, but it got me started. I moved my iTunes library, iPhoto library, and my complete Aperture photo library to the Drobo, freeing up space locally and a harddrive. I re-tasked the harddrive for Time Machine backups, and was generally more pleased with myself!

As time went on, I began to desire an off-site backup. What if my house burned down? In fact, what does everyone want to get, when you see the hills of California on fire every year? The people are frantic about getting into their homes, to retrieve their photo albums. That’s really all that the insurance company can’t replace.

We stopped shooting with film, and getting regular prints made years ago. So, the entirety of my 4 kids’ childhoods is contained, as photographs, on that stupid Drobo! If I lost that, I’d freak out!

It didn’t take long to convince myself that getting my important content backed up, off-site, was vital. In fact, my advice would be to get your important data off-site, before you set up a local backup system! If I lost everything, I could simply log into my account, via FTP, and one very long download later I’d have it all. This kind of off-site backup is really best described as a “cloud-based”. The regular 0ff-site backups are generally more expensive.

I looked at a wide variety of options. I host this site with Dreamhost, and thought of them first. Naw, they don’t like you using your storage for personal stuff like that. You can do it, but it’ll cost you if you’re caught. I really don’t want to do it like that. Then I looked at some of the others. None really compare to Dropbox, so I stopped looking. I signed up, and ran a test.

It’s pretty cool that you can backup as easy as just dropping your stuff on a folder. That’s really easy! Most people will get what they need with Dropbox. However, I really wanted to setup my backups through FTP, allowing me to setup an Automator action to do it regularly. Yeah there’s other ways, but that was what I wanted.

From the beginning, I had decided that I was willing to pay for this service. DropBox is cool because the first 2 Gb is free. Most will be fine with that. Me, however, I’ll be needing much, much more. The next step with DropBox is $9.99 for 50 Gb, then $19.99 for 100 Gb. Neither of those are enough either.

Poking around, I found an article that made me realize that most of these cloud-based backup solutions are using Amazon S3 for their data. They’re just writing cool apps to make it easy for regular folk. Well guess what?! I already have a sweet app for accessing Amazon S3! It’s called Transmit, by Panic Software. I’ve been using it for all my FTP needs for several years. I love it! It didn’t take a genius to find a simple article telling me how to set up Transmit to access my Amazon S3 account. Oh yeah, I had to set that up too. Being an Amazon customer, from their massive store, I already had an account, which made the whole thing much easier.

Amazon S3 is:

Amazon S3 provides a simple web services interface that can be used to store and retrieve any amount of data, at any time, from anywhere on the web.

Basically, Amazon.com has built one of the most robust websites in the world. When have you seen Amazon.com going down? It’s even older than Google! All of their data is spread all over the world, so if a server goes down in Bakersfield, and I was viewing content on that server, they just reroute my viewing experience to another server, somewhere else.

So now they’re just providing that kind of data storage to me! Here’s the catch…wait for it…IT’S CHEAP! Really really cheap! The first day, I uploaded one folder, with just a small amount of data; 2 Gbs worth of photos and some other stuff. I can’t remember what the total amount of data was, but is wasn’t much more. They charge for the upload and then the storage. My total cost for that month was $2.57. That’s it!

So, on December 1st, I decided to jump in with both feet. I set up the whole back up! Right now, I’ve backed up my entire photo library, iPhoto and Aperture. I’ve also included all my RAW files. It’s all there! The first big back up took like 3 days, non-stop! That’s mostly because my ISP is Comcast, and they suck. It would drop me occasionally, and Transmit would be forced to re-log in, and pick up where it left off.

Then on December 7th, they announced a sweet price drop, and they’re waiving the upload fee, until June of 2010! SWEET! Now is the best time to jump on that price drop! So I can upload all sorts of stuff, and I’m only paying for the storage right now. That’s 15¢ per GB! They drop the price at several tiers, the first one at 50 TB. That’s $154/mo. for 1 terabyte of storage!

Since most people won’t be backing up a whole terabyte of data, here’s a more normal look; 10 GB = $1.50/mo.

So, where am I now? I’ve set up the account. I’ve chosen the folders I want backed up. Oh, I forgot to tell you that Transmit can easily synchronize a folder, without rewriting content that’s already there. Maybe that’s normal for all FTP apps, I don’t know. It’s cool for me! This way I don’t have to worry about eating up bandwidth, when Amazon starts charging for that again.

I set up a few “Favorites” in Transmit, and then I set up a simple Automator Action. I saved the action as an .app. Lastly, I set up an event in iCal, calling my new application in the alarm.

Now I’m good to go! I have it syncing each Saturday evening, and I don’t have to think about it anymore. Below are a few screenshots of the whole process…

In Transmit, input all the important information about your new Amazon S3 account.
In Amazon S3, top most folders are called "buckets". I'm naming my main backup folder, "trevor-photos"
I chose to "Upload" my data and to have it "Update" it. This will keep my bandwidth down.
Here's the Automator Action. It's pretty self-explanatory. Make sure to save it as an Application.

Author: TREVOR

Leukemia survivor. Son of The Most High. Father. Man.

1 thought on “Drobo and Amazon S3, possibly the best backup partners!”

Comments are closed.