Having always trusted my digital archive to a random variety of techniques, I recently decided to bite the bullet and sort it out once and for all. Up to now, I’ve always gone for the very risky technique of storing my archive on a single disk until the end of the calendar year. At that point, I duplicate the files onto another drive and store that one in another location. While it’s kind of the right thing to do, it relied entirely on that first disk lasting the year without any issues. As the common I.T. mantra goes, it’s not a case of “if” a drive will fail, but “when”. As drives contain moving parts, they will at some point grind to a stuttering halt, taking all your lovely art with it. Another phrase worth remembering is that if your data is on one drive, it’s not an archive, but simply stored insecurely. Basically, it’s time to pull your socks up!

In the future, I’m sure the answer will be backing up your whole archive on cloud storage but the current prices leave this option the domain of the uber-wealthy/crazy. For now, the safest bet appears to be RAID 1. While RAID 0 offers faster access speeds by striping your data across two drives, RAID 1 is a simple direct mirror of the data on the first drive. As you back-up your images, the information is written to both drives so, if at any point a drive fails, the information is still there on the second unit. I should point out that the downside here is that whatever you do to the first disk is duplicated. While that may sound like a purely positive thing, if you accidentally break out your fat thumbs and delete a whole wodge of data or accidentally overwrite a file with a corrupt version, you’ll need to pray to the gods of data recovery as you would if you were using a single drive.

One of the stipulations I had when first looking at this kind of set up was the need to be able to take either of the drives and drop them into an external enclosure at a later point and access the information. If I move a drive to a second location and need to access it there, I want to be able to just plug it in and go, not lug the RAID enclosure with both drives to the location. Preliminary research suggested that some types of RAID encrypted the data, meaning that each drive individually was unreadable once away from it’s twin. Thankfully, I found the erotically-named Icy Box IB-RD4320StU3. Unlike using software RAID, the Icy Box uses a hardware controller, with the result being that I can pull a drive out of it at the end of the year and the information is viewable independently. Bravo!

The unit itself is small and as pretty as a metal box can ever be. The physical size of the unit is suitably compact at only 80 x 120 x 185mm so it slots in behind my screen comfortably. I opted for a pair of 1Tb Western Digital Caviar Black drives for my 2011 archive. In case, like me (heaven forbid), you wondered what the difference is between Western Digital’s Caviar Green, Blue and Black drives, here’s the low-down. Green drives are quiet and have a low power consumption but at the cost of a little bit of speed, Blue drives are the “everyday” drives and as such are mid-priced and average at everything while Black drives (being all dark and mysterious) are power-mad beasts. Okay, not quite but they have the fastest speeds of the three, are the loudest of the three but use the most power. Now all of this is relative so when it’s described as the noisiest, it’s compared to the Green drive’s totally silent operation. As for power, the street lights appear to still be working on Tabasco Avenue so I think the national grid is safe for the moment.

Installation of the drives is as easy as pie (or at least the eating of) with the two drives just slotting in with no screws or clamps needed. Once inside, you flick the switch on the back to decide what flavour of RAID you require, push a button to reset the unit and you’re ready to go. One slight complication for me is that I use an AFP PC laptop for work and an Apple iMac at home. As both use different file system formats as standard, I needed to find a common ground. Thankfully, I had to look into this a long time ago and settled on the Windows NTFS format. While Microsoft stands firm with it’s belligerent stance of only supporting it’s own file system, Apple is rumoured to be fully implementing an NTFS option in the forthcoming Lion operating system and, until then, I use a bit of software called “Paragon“. This simply gives OS X a bit of a prod and tells it to recognise NTFS drives, allowing you to both read and write to them. Once, you’ve formatted the drives, a single drive icon pops up on your desktop (in OS X) or in your “My Computer” tab (in Windows) and from now on, everything that you drag onto the disk will be copied to both drives.

The biggest pain of the whole challenge was copying the data from my previous single 2011 drive over to the RAID setup. Despite the Icy Box being USB 3.0, none of my gear is that up to date and the only connection that is shared between my laptop and my iMac is the regular USB 2.0. While that’s fine for pen drives and ipods, it becomes a major mission to copy across 450Gb+ of data. Predicting how long it would take, I started it off and headed to bed and thankfully it had finished by breakfast time. The issue then was how to verify that the information had been transferred correctly. After asking on twitter, I got many suggestions but nothing did the job. If it’s written for OS X, you’ll be amazed how many software packages only see Apple’s HFS format drives so won’t work on my NTFS drives. In the end, I found a free Windows program called WinMerge that chugged away through the drives. Four hours later and I had the thumbs up; Operation Home Raider was a success.

For other photographers’ views on archiving, check out an interesting post at Photofocus, Edmond Terakopian’s blog, Carter Hewson’s thoughts and, if you’re feeling particularly wealthy/paranoid Chase Jarvis’ page. As one final note to all those who think they’re safe, remember drives have a lifetime even if they’re not plugged in and stored in Fort Knox. The accepted belief is that it’s wise to at least plug them in and spin them up every six months or so to make sure they’re in working order. It may seem costly but it’s also worth shifting the data onto new drives every few years too. Technology moves pretty quickly with connection interfaces and plugs changing every few years. That may sound ridiculous but if you needed to access it, do you have the means to connect up that old 150Gb IDE drive you have in your cupboard? So, now that I’ve sewn the seed of doubt in your mind, take a look at your random stack of enclosures, drives and disks and consider how secure your data is. Remember, if you make your living from your photographs, can you afford to lose them?

N.B. – A colleague has pointed out that I haven’t mentioned the actual cost of the whole experiment on here. Cheers for pointing that out, Nick! The total package, including the drives, cost me less than £165. I really don’t think that’s too bad for the peace of mind.

26 Responses to “Mirrored? Single? Manoeuvre!”

  1. Timely post, as have been thinking about a RAID setup myself. Normally I use a desktop with 2 1TB drives in it and one mirroring the other. I also have an external drive which I occasionally back up onto and keep offsite. Have now moved to a laptopn and am only doing the occasional backup – so I like the look of this arrangement!

    Posted by Jonny
  2. Yeah, this was surprisingly easy to set-up with the slowest part being copying the current archive across. Well worth a look and so cheap compared to other systems.

    Posted by tabascokid
  3. heh, you use a pretty similar setup to me, and now I understand exactly what you were trying to do, I appologise if my suggestions threw you off beam, I thought you were trying to verify the RAID copy, which didn’t make sense – as RAID 1 *is* a copy…

    I’m just about to upgrade from 500Gb of RAID 1 storage to 2Tb, should be as easy as taking out 1 500Gb, inserting a new 2Tb, letting the NAS RAID box do it’s magic, then taking out the second 500Gb and putting in the second 2Tb, voila 4 times the storage for about a 100 quid.

    …nice techie writeup btw. angling for a job at pcworld?

    Posted by Jim
  4. Am delighted I’ve just found your post. A breath of fresh air and exactly what I needed, and written by a normal person too (- ‘props’ to Chase Jarvis but my God he gets on my t*ts. Ludicrous archiving system too).

    After looking into RAID storage a few months ago I was left thinking I’d have to fork out for a Drobo. Glad I didn’t. IcyBox here I come.

    Thanks so much. Great post.

    Cheers,

    Ben

    Posted by Ben
  5. ps, Quick question: where did you buy your IcyBox from?

    Posted by Ben
  6. Hi Ben, That’s great news that you found the post of use. I’m always unsure that the tech-based posts will be of limited interest but this is one of those subjects that we all have to unfortunately consider! I got mine from the dreaded evil empire (ie Amazon). Here’s a link to the actual item.

    Posted by tabascokid
  7. @Jim Glad it’s got your approval! Sorry if my phrasing wasn’t quite right in my initial cry for help in software advice. You’re actually the second person to suddenly have an “Ahhhh! THAT’S what you were after!” moment when they’ve read the article. I guess I wasn’t as clear as I’d hoped.

    Posted by tabascokid
  8. Nice one. Thanks again.

    Posted by Ben
  9. Nice article, I’m not a pro photographer, not even close, but I do work in the wonderful world of IT and managing backups is one of my roles, so both in work and at home I do lean towards the Chase Jarvis paranoia method.

    My personal photo store (total ~800Gb) is backed up to an additional drive in my house and also on an external kept offsite, sounds complicated but it’s just a USB drive i keep under my desk in work.

    I do have some photos that are important to me stored in cloud solutions but i agree the pricing just isn’t right at the minute unless you are with a larger company

    Posted by Keri
  10. It really is a “horses for courses” situation where everyone has their own way of dealing with it. If there was one definitive way, the creator would be a very rich person! :) It’s good to know that an IT pro hasn’t found any gaping holes in my technique though. Cheers for reading!

    Posted by tabascokid
  11. Damn you Leon…now I’m panicking that my cobbled together idea consisting of “hope the external HDD doesn’t break” isn’t quite as foolproof as I’d once thought.

    Though your easy and cheap suggestion does seem pretty good.

    Posted by James
  12. Your RAID-1 setup is a good one, I use something similar with Stardom drive enclosures. But I would highly suggest having a 3rd single drive that you manually back up the RAID-1 set to. As you pointed out if you manually F**k up the data on the RAID set, it will screw it up on both drives, so you need a 3rd offline drive as well to prevent against user error.
    I wrote some blog posts on this subject a while ago, and while the posts are getting very old, most of the principles remain the same:
    http://www.snappertalk.com/index.php/20080604/archiving-photos/
    http://www.snappertalk.com/index.php/archiving/
    I used to use Retrospect to do the backup from the RAID set to the third drive, but I didn’t like a lot of he problems with their latest version so switched to Chronosync. Chronosync is one of the few backup softwares capable of doing data-verification (i.e. reading what is written and comparing it bit-fo-bit with the original to ensure no errors occurred during the copy)
    If you are primarily a Mac user I would use HFS+ format, not NTFS, but that’s just because if there’s a problem with the disks I’m more comfortable with Mac data-recovery software than Windows. If you have a disk problem with NTFS disks you’ll need a Windows machine to run recovery software, and also I just think there’s less likely to be problems with odd Mac file issues involving metadata or filenames, but perhaps I’m wrong. There’s also software to allow HFS+ disks to be read/written on Windows if you really need it, but generally I would advise using the format that is native to the operating system you use most
    Regards,
    Ben
    PS-Your comments box doesn’t allow you to navigate backwards to correct spelling errors!!

    Posted by Ben
  13. @James – Ah, you’ll thank me some day…

    @Ben – Cheers for your feedback and comments. I’ll certainly check out those links and consider the 3rd drive option when I get a free moment. The software backup programs definitely look like they’re worth a bit of investigation. The reason that I opted for NTFS is due to the use of AFP-owned PC laptop that I’m not allowed to tweak too much. Any software that adjusts read/write rights would be frowned on. :(

    As for the comments box, it bugs the living hell out of me. It seems to be some glitch in Firefox. Every plug-in I have tried has exactly the same problem. If you open the page in Safari, I’ll confidently say that it won’t be an issue! Grrr…

    Posted by tabascokid
  14. Seems I overlooked the bit about your Windows laptop, in which case NTFS may not be bad. It depends whether you are likely to need to take your archive with you (i.e. away from the iMac at home). If not, you could always have it as HFS+ connected to the iMac, then share it across your home network (gigabit ethernet preferred) and it will just mount as a share on your Windows laptop and you can do what you want with it. I used to do this when I used a Windows laptop for work and Mac at home. When I needed to copy to the drive I just did it across the network. All that said there’s not all that much wrong with NTFS, I’m just a bit biased against it… I’d definitely look into the 3rd drive option, preferably in an enclosure you can hot-swap or with removable drive tray. I believe Icy-Box make some.

    Posted by Ben
  15. What are the noise levels of your IcyBox like, Leon? I hope it’s nothing like my 4-disk IcyBox hot-swappable enclosure, which would be perfect if only the bloody thing didn’t sound like Merzbow was staging an impromptu concert in it.

    Posted by Simisker
  16. Wotcha Rich! Noise levels are well within acceptable levels. When the fans kick in after hours of operation, it’s obviously a little higher but I don’t have mine running all of the time. There’s a clip on youtube that demonstrates the volume if you have a search.

    Posted by tabascokid
  17. Have been after a program that does the job WinMerge does for a while, really helpful to have one pointed out that works well – cheers!

    Posted by Tim
  18. Hi,

    A friend showed me your write-up and since I was in the market for a RAID system it cropped up just at the right time, I read your review and weighed it up against the Drobo option and went for the Icy Box.

    All good so far….but my iMac does not want to recognise the HDD as much as I try to work my way around it, I cannot get it to show. Running Snow Leopard – anyone else had this problem? Have tried looking around for answers but can’t seem to find anything.

    Great write up by the way, very clear and concise!

    Thanks

    Posted by Annabel
  19. Yup – seems I have worked it out, JOY!

    Posted by Annabel
  20. Now that’s a good example of how good I am with this stuff. I didn’t even read your comment until now and yet the problem’s been resolved. ;) Glad to hear it’s sorted! Enjoy your archive…

    Posted by tabascokid
  21. Good subject Leon.

    Looking at the Amazon link you posted, there are a range of people reporting problems with the Icy box alongside the more satisfied customers.
    Guess yours has been running smoothly & you haven’t needed tech support.

    For me, I wanted a device that could be accessed by more than one machine, so chose a NAS (network connection, rather than USB). I used to used a D-link box and am currently on a Synology unit. These are more expensive than yours, but offer other features.

    Posted by Ceildh
  22. @Ceildh – I got a NAS box initially but found that there was no simple way to easily upload or download bulk pictures over the network. At the time, I had to load a webpage and download them over my wifi network which was much slower than just throwing a cable into a box. I can appreciate the benefits of NAS but I opted for speed in the end. My only complaint with the Icy Box is purely on the fact that both copies of the data are in the same place until you’ve finished the drive. For me, this results in both versions of my data being in my house until the end of the year. While I’m hopeful nothing will happen, if something horrible did happen such as a fire or a major burglary, I’d lose it all. I haven’t found the answer yet. :(

    Posted by tabascokid
  23. Interesting. All the NAS boxes I’ve had allow you to map a drive from your work machine (in your case, your windows laptop?).

    File copies are then just as fast & easy as copying anywhere else. With many NAS boxes, you may need to set up access for external users (usually the UserID of your work machine) and there are options to set up disk quotas if you’re sharing with other people in the household. Other than that, it’s pretty straightforward. In this way, more than one device can see the same files without the need to plu/unplug between them. You can also easily replicate between devices this way.

    I’ve no experience of Macs, but would hope they’re equally easy. My iPad can’t see the NAS directly, though can connect via the web interface. Apple seems to have very strict views what directory structure IOS will let me see on itself, never mind other devices. I’ve found certain iPad apps that can read/write to the NAS, but that seems to be it.

    Posted by Ceildh
  24. > My only complaint with the Icy Box is purely on the fact that both copies of the data are in the same place

    Good point. I believe you said you were using RAID 1? That should allow you to remove one disk, replace with a blank one, where the unit should then rebuild the data across the new blank one. You can then store the removed one at a different location. Recovering the data on that disk may require you to use another Icy box, as sometimes these devices will record the data in a format specific to the unit. However these don’t seem expensive so that shouldn’t be a limiting factor.

    I suggest you read the documentation carefully before proceeding down is path, but it’s a thought.

    Posted by Ceildh
  25. @Ceildh – Thanks for sharing your information and experiences with this. It’s certainly set my mind to think about whether I should be revisiting the NAS idea. One question, with your current set-up, if you have 30Gb of pics on a laptop and needed to transfer them to the NAS, how simple is the procedure? I assume it means connecting to the network via cable as shifting that amount of data over wifi would take ages.

    Posted by tabascokid
  26. Yes, a network cable would always be quicker.
    In practice it’s not to difficult to have one from your Wireless router for the times you need to transfer in bulk.

    Personally, most of my PP is done on a desktop machine with a 24″ monitor that has a wired connection, so this isn’t an issue. Viewing my collection from other wifi devices is then easy because they all have shared access.

    Posted by Ceildh

Leave a Comment

Name*
Email*
Website
Comment*