Disk Mirroring
 

I value my catalogue of photographs and I intend to keep every RAW image shot, unfortunately compressed NEF's at 10MB on average in size use lots of HDD space. I wish to keep two copies on two separate disks but dont want the regular job of coping disk to disk etc.

I thought of disk mirroring but there are million and one products in the market to do this. Does anyone perform disk mirroring?

A second thought was taking an image of a 120GB disk to CD/DVD but I cannot work out the physics in my mind, can this be achieved also?

I recently bought an 80 gb external HD and copied all of my pictures on to it. That really didn't take all that long, now I plan to copy new pictures once a week or so. It seems to be a simple, inexpensive solution to protecting my pictures.

Maybe you could look into installing a RAID Array card together with an additional 130gig Hard Drive in your PC. The cost would be about £100 and allows for further Hard Drives to be added at a later date if you require more storage and security.
The RAID system would mean you will never loose any stored data, as everything is mirrored on the next disk, and the next, and so on. This not only mirrors data but all your programs as well.

I use a free program that routinely does a back up to my secondary HDD. I'll check up at home this evening if you wish. It seems to work, and can run in the background at a pre-set time. I plan to buy an external HDD in the next 6 months and will back up to that as it makes sense not to have all image copies stored in the same room in case of fire.

I used to use DVD's for backup but that is such a pain, and they only last a few years anyway.

Leif

Stephen, on-line backup isn't really backup at all. Sure, it's convenient a the time, but the only risk it protects you against is physical failure of a hard drive. In other words, a mirrored system is protecting you against maybe 5% or at most 10% of the risk.

Don't do it!

What you need is off-line backup - a backup system that protects you against all the possible failures that lead to data loss, not just one relatively rare type of failure.

There are many ways to achieve this. I'll list a few of them, though there are several others.

* External drives (USB, Firewire, or etc.)
* DVD-R (tedious but very effective
* Second, networked system (very convenient, not as safe as the two just mentioned though)
* Drive caddies (cheapest method of all, not too difficult to administer)
* Commercial broadband backup (probably not suitable for your volume of data)
* Tape drives. Can be worth considering, but you need to go for a higher-end unit, so hard drives often work out cheaper.
* And so on

All of the methods I just mentioned, when properly used (which isn't difficult or too tedious) protect you against all risks, not just the low-probability risk of drive failure.

I agree with what Tannin has said.

With Raid and mirror they improve your fault tolerance but with "never lose data" never say never. Sh*t happens. Raid and mirror was most useful when cheap disk drives were very unreliable and good disks were very expensive.

Rapidly changing technology causes another problem because media becomes outdated so quickly.

Your best bet is probably an external drive. Don't fiddle about with small ones, you are going to take more and more pictures as time goes by, so go with a decent-size unit: 200GB as a minimum, 250GB or 300GB is better.

The best way to do this is to buy an external case (a little thing about the size of a largish paperback novel). USB or Firewire, or better yet, a "combo case" that does both. You can buy these at any computer shop. Cost around $50AU, so maybe E20.

Then buy a drive to go into it. Samsung make the most reliable drives on the market at present, if you can't get a Samsung try Hitachi. Avoid Western Digital - they are going through a bad patch right now and their drives tend to be fast but fragile. Buy it from someone you trust not to bash it about in the warehouse - hard drives are very delicate precision devices, and you need to handle them with the same care you'd use for an expensive lens. If that means paying an extra E20, who cares?

Now you have a readily upgradable, readily repairable backup system. All you need to do is at appropriate intervals - once a week or once a month is reasonable - plug the drive in and then drag and drop your new files over onto the external drive. You should be able to start the copy inside 10 seconds - if you can't because it's too fiddly to select files scattered all over the place, then your computer method is disorganised and you need to rethink it. (Computers store so much data that you have to be organised about it - otherwise you are nostril deep in chaos every working day.) Then go away and drink a cup of tea while the data copies. Finally, disconnect the external drive and store it somewhere else!

Leaving the drive plugged in is easy and very tempting: but it's not really a backup at all if you do that. While the drive is plugged in it is vulnernable to power surges, fire, lightning strike, virus attack, user error, computer crashes, and every other risk you can think of. Once you unplug the drive and put it (gently!) somewhere else (such as at your office instead of home, or your brother's house, anywhere in a different building is good) you are very safe indeed.

A better method still is to use two external drives and swap them over alternately. You wind up with a system like this:

* Main drive: all my data
* Backup #1 all my data up to January this year
* Backup #2 all my data up to February this year

At the end of this month, I copy the last couple of month's worth of pictures over to Backup #1, then take it to the office for safekeeping. that gives me:

* Main drive: all my data
* Backup #1 all my data up to March this year
* Backup #2 all my data up to February this year

See the logic? I can lose any single drive and the worst that can happen is that I lose <30 days worth of data. I can lose two drives (big power surge while I was doing my month-end backup let's say) and I still have all my data except for the last 50-offdd days' worth. And this is on a fairly unrealistic and very lazy once-a-month backup schedule. In reality, you'd backup any time you take important pictures - every Sunday night in my case, after I get home from a field trip.

Total cost for a 2-drive system: less than half the price of a Canon 10-22mm lens. I don;t know how that translates into Nikkor terms, but you get the idea.

PS: I don't actually use that two-drive system I suggested above, I use a needlessly complicated scheme involving multiple external drives and removable IDE caddies and three different computers in two different parts of town and a great big stack of DVDs - largely because I have lots of spare computers lying around and I'm paranoid about data security. For reasons best known to my psychiatrist, I'm pondering buying a completely unnecessary industrial-size tape backup as well. But then I've worked with computers for nearly 30 years and always was fond of overkill.

It's worth pointing out that hard disk crashes are not that rare. Our group at work has had 4 or 5 over the last year or two. I've seen a few at other companies too. And I had a hard disk that required re-formatting once the IT department got hold of it. (A major cause of PC problems for me has been the IT department at work. If they can unplug something, or damage it, they will.)

Also, even if you use a backup system, a fireproof safe is essential if your data is valuable. Every company I have worked for has a policy of doing a daily backup, storing a copy (of the tape) in the fireproof safe, and storing a copy off-site.

Leif

I use the DVD-R option. It takes a couple of blanks to back-up all my personal files. I'm well organised with my directory structures, it's just a quick drag and drop then hit the burn button. Store a copy in the office at work, and a copy in the house.

Very cheap, reliable, and relatively easy.

Duncan

This depends on three main factors:

1: The way the drives are handled, both in the wharehouse/distribution channel and by the installation technician. Hard drives are very sensitive to shock, and although a nasty bump very rarely shows up with an immediate failure, it greatly increases the risk of failure later on. Many people don't fully realise this and pay only lip service to what is, in reality, the #1 rule of hard drive safety - treat them like they were made out of eggshells! This is why we only ever buy drives from a known and trusted distributor with a track record of doing it right, and even then always buy more than 10 units at once. (Hard drives come ex factory in boxes of 20. If you order 5 drives or 9, the warehouse guy takes them out of the box, maybe packs them properly, maybe doesn't, maybe bangs them down on his desk, maybe doesn't, then ships them to you. If you buy (e.g.) 15 drives, human nature being what it is, the warehouse guy takes the other five drives out of the box and your 15 stay nice and safe in the factory carton, which is proof against even couriers and truck drivers. Best, of course, is to simply buy a box of 20.) Obviously, you don't want 20 hard drives - but you can buy your drive from someone who does buy them that way, and who knows how important it is to look after them.

2: The quality of the drives used. There are major differences in reliability between different brands of drive. Seagate/Maxtor and Western Digital drives have a failure rate measured in units per hundred, typically a modest single-digit number per hundred, but higher for a bad model. Samsung drives have a failure rate measured in units per thousand, and it's a small single digit. I don't see enough Hitachi drives to put a number on their RMA rate, but it's probably closer to the Samsung figure than the Seagate/WD one. (Just guessing on this last.)

3: Clint's Rule. ("Do I feel lucky?")

The most common reason for disk failure is when the heads get mucked up by oil when PC is powered off. Especially if the drives run for very long periods ( months ) between power down cycles. The oil from the spindle bearings can leak and spray over the disk surface which is not a big problem in itself on the active area of the disk. However when you power down the heads are parked on a non active area of the disk which by now have a thicker film of oil.

I agree with Tannin some drive manufacturers are orders on magnitude better than others. I've seen IBM drives die after a short period. I was testing some software where the system was writing about 100 files a second for 2 to 3 days. The IBM drive died and refused to pass the post test. Sun Microsystems replaced with a Seagate so far it's held up.

I wouldn't dare rely on keeping all my thousands of scanned images, totalling hundreds of gigabytes, on a hard drive of any description. In the course of a year I repair and maintain many computer systems, some with large hard drives, some with raid arrays, the majority of these come in with some kind of hard disk fault! Sometimes it is quite easy to save the files stored on these disks to a CD or a DVD, slap in a new drive, re-boot the system and everything's fine. Then there are the drives that are completely trashed for one reason or another, normal file restoration and saving with these drives if not impossible can be extremely expensive, usually entailing a trip to a specialist data recovery company. In either case the average computer user doesn't want the hassle and certainly doesn't want the cost! I backup my scans and digital images on a regular basis, I scan and process 10 then save to DVD-R using Nero Burning Rom or Nero Express. Working this way the most files I can lose will be 10, it only takes a few minutes to burn 10 images to DVD, it can take hours to scan and process in Photoshop 10 images to the computer.

nirofo.

The problem with that, in my opinion anyway, is that writing DVD's is a right royal pain once you have a lot of data, and cannot be done unattended, whereas a HDD to HDD backup can be automated. Unfortunately DVD's don't last that long so you need to make further copies at regular intervals, adding to the inconvenience.

No medium is fault free, but for me an external HDD is the most convenient method at a reasonable price. High speed tapes seem to be favoured by companies though they cost a bit more at least at the outset. I think that for really large amounts of data they work out cheaper.

Which can be summarised as "Hard disk crashes are not that rare". :D

Do you have a source for your HDD failure rates? I'd assumed that HDD's are pretty much of a muchness, but if as you say there are large differences in mean time to failure rates, then obviously it's worth avoiding certain makes.

Leif

As I said earlier, I receive many computer systems for repair that have some form of hard drive fault, I would say it's probably as high as 90%. The majority of these are caused by hard drive crashes which occasionally can be repaired, but more often than not will lead to further problems in the not to distant future, the rest are catastophic hard drive failures! Usually, the best and safest remedy means a new replacement hard drive and a complete system re-boot. In some cases the data is recoverable from the old hard drive without too much effort or expense, in others the monetry cost of data recovery is outwith the majority of general PC users. Bottom line, for fast, easy storage that you could lose catastrophically at any time, go for large hard drive storage (internal or external). For long term safe storage, albeit with slightly more involved file saving methods, go for DVD-R. The longevity of DVD-R disks is supposed to be in the order of 25 years, providing you still have the PC's and drives to use them. I'm not sure what the storage longevity of say a 200Gb hard drive is drive is, (it's magnetic media), and you still need a PC to run it even after 25 years. Price of 40 x 5Gb DVD-R disksm (200Gb) approx £20, price of one 200Gb hard drive, (40x5Gb DVD-R disks) approx £70. Oh yes, and I can watch films on a DVD player-recorder.

nirofo.

Nirofo: I don't doubt your experiences with hard disks, but I do doubt the figure you give for the longevity of DVD. Many CD's will last no more than a few years, and there is a huge variation in quality. A friend found that PCLIne CD's delaminated within a year or two. Some online test reports I saw found that they did not last as long as the manufacturer's implied. There are only a few CD manufacturers, and most CD's are in fact rebadged. It is sometimes possible to determine the original manufacturer by examining the CD or using Nero to read CD meta-data. I think the best include Memorex, Fuji and Ritek, but even big names will re-badge poor quality disks. I had huge problems with Phillips disks and had to take them back to the shop.

And of course disks need to be stored in a cool, dry and dark place.

I am not too sure about DVD, but I would guess that it is worse than CD. I presume the technology is in essence the same, but with smaller pits, and higher rotation rates, and hence the tolerances are tighter, meaning more likelihood of failure.

Here's a few links I found from a quick google:

http://www.cdfreaks.com/news/12933
http://www.cdrfaq.org/faq07.html#S7-3

Sorry but I can't find the report of accelerated aging tests that I read some time ago.

Leif

I've got CD's going back to 1994, they still work perfectly, but then I store them correctly and only use them when I really need to, I catalogue all my files and CD's so that I can find the image I want without having to subject the disks to strenuous searches and handling. For most people I would think infrequent use for stored images is the norm. I have several hundred CD's with images and other data on them, I haven't lost a disk yet unless I did something stupid myself. I now use DVD-R disks, been using them for about 18 month's now, no problem so far! In the last ten years I think I've probably gone through 8 or 9 of my own hard drives, some of which cost a fortune when they first came out. I've lost many files and images from hard drive crashes and breakdowns over the years. I'll continue using DVD-R until the next generation of high definition disks become available at sensible prices. I keep a couple of old computers in my loft, fully set up and able to run any of the disks I have at present, all my images are saved as TIFF files so any of the graphics programs, past, present and future will handle them. I also have a couple of spare DVD reader drives in case of emergency (cost £11.50 each new), these are adequate for reading any of the files or images I own and will keep me going for years to come.

nirofo.

Have you been unlucky or have I been lucky ? In 18 years I've had 2 go. I have never bought a new drive they have always been pre-enjoyed.:rolleyes: I have 2 drives in my machine. From my experience of our customers systems drives typically last 5 year before you see the operating system complaining about disk problems.

Very wise but only 10 per DVD means storing a lot of DVD's.

I'ts interesting that a CD/DVD/VHS tape etc can come with a lifetime guarentee, that is up until you record something. This guarentee doesn't include any data contained on the media.
A few years ago I was employed in the digital imaging business particularly with document storage and archiving, (document includes images). Under no circumstances would CD/DVD been recommended for backing up or archiving. Slight changes in humidity, temperature or exposure to light can all effect CD/DVD's.

Having said that - I back up my RAW files onto CD (min 2 copies of each) and then to an external HD.

I think you've probably been very lucky! The majority of the computers I receive for repair or refurbishment have hard drive faults which are usually terminal, that is, requiring a new hard drive and system re-boot. I have had some computers in 2 or 3 times for replacements, generally it's the primary drive that goes, but quite often it's the slave or secondary drive that succumbs to an head crash, sometimes it just simply dies along with all it's contents. I think if a computer is just used for occasional general work, then it's quite likely a hard drive will last up to 5 years, if the computer is to be used continuosly for all sorts of duties, including graphics file manipulation, then 2 to 3 years is a more realistic life span for a hard drive. My computers are nearly always on and have a fairly heavy workout throughout the day.

The way I see it is, why subject yourself to the frailty of long term hard drive storage system when you can have a safe, robust storage system that lends itself to easy cataloguing. A system which is only operational when you need it instead of being continually on and wearing away it's components as it is with a hard drive. A system which will stand up to knocks and bangs of everyday use without destroying it's disks and the data on them permanently.

nirofo.

Hi Jon

When I said I save 10 image files to DVD I didn't mean just ten! I meant I backup to DVD every time I have 10 image files ready, I continue doing this until the DVD is full, in my case that's approx 150 x 30Mb TIFF files (can vary). Obviously with smaller files you'll get more on the disk. When the disk is full I print off a CD box insert booklet complete with CD/DVD title, (e.g. Landscapes) and a full list of the disk contents, (image titles). I have a CD storage rack system with all my CD/DVD's at hand, I have no problem finding a particular image within minutes. I also keep a database of all my image files, stating which CD or DVD they are stored on on my computer, this is also archived on CD/DVD in case of computer failure. So bottom line, the maximum number of unsaved image files I can lose at any one time from the computer is 10, the maximum database data entries loss is 10.

nirofo.

Numerous laboratory tests have proven that CD/DVD's are not safe and robust. I've also had loads of problems with DVD and player incompatibility. There are tables somewhere of which DVD and which player go together.

I've used goodness knows how many PC's over the past 13 years of working in IT, and I've never had a hard disk crash. A few colleagues have but I've shared office space with thousands of people. Many of my machines have been hand me downs. It sounds like nirofo works in PC maintenance where he/she sees failures. But if a component is going to fail then it will probably be the disk as most other bits are solid state. I've had a PSU go once. I suspect that the robustness of HDD's is improving over the years. I go running with my iPod which contains a HDD (Samsung I think) and the little marvel survives.

Even if a HDD does crash on average after 5 years (which I dispute), that's on a par with a DVD if not better. And a 200GB HDD costs ~£80, compared to maybe half that for DVD. But for the convenience it's worth it IMO.

But surely the real question is how long will a powered down device can store the information.

I'm tempted to look into tape drives as a backup method. The initial cost might be offset by the reliability compared to hard disks.

I go along with most of what you say Leif. The 5 year average I am quoting is on Servers in Regional and National Newspaper Production systems running 24/7.

Very interesting discussion regarding reliability of HDD's. If I make two copies of all photos taken in 2005 and store on two disks, how should they be stored?. Horizontally or vertically i.e. manufacturer label facing upward or IDE parallel connector facing upward. Which way would be most reliable given the bearing/oil issue discussed above?

Hi Stephen

I was always led to believe the best orientation for installing a hard drive in a computer case, (tower box) is horizontal, that is with the label facing upwards with the power connector on the left hand side at the back of the drive as you look from the front of the case. I was also led to believe that fitting a drive upside down puts unecessary wear on the bearings. I have never come across a computer fitted with the drive upside down, I have however, come across several computers where the drive/s were fitted vertically, that is with the label facing the right hand side of the computer case. I don't think orientation is as critical as it used to be, however, I still think it best to fit them in the conventional horizontal position. I also think that should anyone decide to use hard drives as their preferred long term image storage system, then they should only be installed as either separate secondary disks, (not slaves), or as external hard drives not linked to the system directly but by the use of a USB2 connection. Set up this way if the primary hard disk goes down, then it's unlikely to take your other non slaved or independant hard drives with it together with all your image files and hard work.

nirofo.