Backup Solutions

Not much blogging lately – I’m sick, and it sucks. This is the third weekend in a row when I’ve been under the influence of this stupid thing. I don’t know why, but I never get quick colds any more – they always last for weeks and weeks, and the cough always turns nastier and nastier.

Anyway, I was discussing backing up computers with some friends, and I decided to try something new. I’ve always been more than a little paranoid, mostly because I’ve had a string of really bad luck with hard drives.

Up until today, I’ve had two main methods of backing up my main Linux box.

Method 1:
I have a tape drive, a 15Gb DLT III SCSI drive, and I use AMANDA to back up. I have to be a little careful about what to back up with this method, because some of my partitions are bigger than 15Gb. I exclude my mp3s, my news spool, and the mail folders where I’ve been storing all the spam I’ve gotten in the last year or so for “training” spam filters, and a bunch of other temporary or easily recreatable files.

Method 2:
I have two disk drives in my computer. One of them has most of my stuff on it. The other is newer, and it’s main purpose is storing all my mp3s. Based on the premise that both hard drives won’t fail at the same time, I’ve been making tar files of stuff from the important onto the other drive. For additional security, I sometimes copy those tar files onto my iPod. The problem with that is that it takes up nearly 6Gb on the iPod, which I don’t like – I’d rather have the music.

So this weekend, I’ve added some new stuff. The first thing I’ve done is to start backing up my laptop. Having it gone for a week made me start to worry about it. Again, operating on the principle that two hard drives aren’t likely to fail at the same time, I figured the best solution would be just to copy files from the laptop to the Linux box. The problem with that is that Apple has these resource forks that Linux file systems know nothing about. After a bit of asking around and a bit of experimenting, I discovered that I can copy my Documents folder to a compressed cpio file using

ditto -c -rsrc Documents/ - |ssh xcski.com "cat - > /backup_2/tibook/docs.cpio"

and restore it using

ssh xcski.com "cat /backup_2/tibook/docs.cpio" | ditto -x -rsrc - .

I tried another option where I do a bzip2 on the file on the Linux box, but it takes a lot longer.

Then I got to thinking about my 80Gb primary drive and my 180Gb secondary drive. I’m only using about 70Gb of the secondary one right now, and a lot of that is the aforementioned tar files. When we were discussing backups, several friends said that what they do is buy an external drive that’s bigger than their main drive, and then make a bitwise copy of the whole main drive onto the external drive. Hey, I thought, I’ve got got more than 80Gb free on the secondary drive, why don’t I use the free space on the secondary drive to do a full image of the primary drive. If I play my cards right, I could set it up so that in the event of a failure on the first drive, I could boot from the secondary drive.

But in order to do that, I had to duplicate the partition plan on the primary drive. The secondary drive was one big 180Gb partition. So I had to shrink that first. And then I had to move it towards the end of the disk. Then I had to make new partitions at the beginning of the disk. Like I said before, I was only using 70Gb of the drive, and the image of the first drive would only be 80Gb. So the first thing I did was use resize_reiserfs to shrink the file system down to 80Gb, and use fdisk to make it into two partitions, the first exactly the same size as the primary drive, and the second to take up the whole rest of the drive. (In retrospect, it appears that I should have made the second partition an extended partition – more on that later.) Then I did a mkreiserfs on the second partition. I mounted both partitions, and used tar|tar to copy everything from the first partition to the second one.

Ok, now I had all my new stuff in a 100Gb partition at the end of the drive. So far so good. Now I went into fdisk and deleted the first partition, and tried to make partition table on the second drive the same as the first drive, except with a 100Gb drive at the end. The primary drive had all four primary partitions, with the last one being an extended partition with logical partitions going up to hda9. So I did the same thing with the first drive, making a hda10 partition that had the exact same start and end sector as the 100Gb partition did before. But bad news, that didn’t work. I think the problem is that the partition had been a primary partition, and now it’s a logical drive. Dammit.

So I had to try again in fdisk. This time, I made the third partition the extended partition, and the fourth partition a primary one the same as the 100Gb was before. Ok, now the 100Gb partition worked. And I can back up a partition just by dd-ing from the source to the target, like

dd if=/dev/hda1 of=/dev/hdc1 bs=2048K

But this has two drawbacks – the mapping from hda? partition to hdc? partition isn’t straight forward, which makes backing up a little more complicated, and also if I wanted to boot from the secondary drive, I’d have to edit the fstab first. Oh well, I guess that’s what rescue CDs are for.