Why I hate Sprint, reason #4523

I was on a conference call, and I had to switch from my cell phone to Skype because the call was breaking up too much on the cell call. Yes, it’s a pretty incredibly sad state of affairs when Skype provides a clearer, less broken up signal than your cell phone!

Only 1 year and 6 months until this contract is over and we can switch to back to AT&T or over to Verizon.

Today’s rather inconvenient discovery

If you use rsync to backup your system, and the system you’re backing up to has different uids for some userids, it converts them as it stores! I just found this out because after restoring my xen1 backup, I’ve discovered that all my postgres files belong to 114, which is the uid of postgres on my home server, not on xen1.

This is going to make restoring all the xen backups a royal pain in the ass.

More server setup crapola

I tried disabling the RAID controller, and when I go to boot it tells me that I don’t have any drives. So I re-enabled it, and it told me I didn’t have any logical drives. Also, sometimes when I boot the RAID controller BIOS tells me there are no drives, and sometimes it shows me the drives. I tried yanking the RAID daughter card entirely, but it’s got a couple of plastic offsets that it doesn’t want to come off of, and I’m reluctant to try anything that I can’t undo at this point. So I’m using the RAID controller to create 4 “Arrays” of 1 disk each. We’ll see how that goes.

Oh, that isn’t good.

I was trying to tar a bunch of stuff off a USB backup disk onto the new machine, and it suddenly started throwing all sorts of errors and couldn’t read any drives, not even the root drive to find the shutdown command.

First thing I’m going to check is moving the drives around, because I accidentally put the two new drives in the third and forth slots instead of one and two, so I’m going to fix that. If that doesn’t help, then I’m going to just turn of the Adaptec RAID controller and try a software RAID. If that doesn’t work, I don’t know what I’m going to do. Probably return the hardware and start again.

Setting up new server checklist

Don’t mind me, I’m just recording what I’ve done so far in setting up my new box.

  • Ordered new server
  • Ordered new rails for server
  • Ordered two 1Tb drives for server
  • Installed drives in server
  • Discovered rails were the wrong kind for this server
  • Grovelled around the net and found the right type of rails, ordered them.
  • Installed Debian on the server.
  • Tried just blasting the entire backup of the old server onto the new one was a disaster, went to Plan B.
  • Discovered that i386 Debian works fine, except neither the Xen nor the Bigmem kernels boot.
  • Downloaded and tried to install ia64 Debian, only to discover that’s the one for Itanic.
  • Downloaded and installed amd64 Debian. Xen kernel working fine.
  • Installed and configured munin. Discovered smartd doesn’t work because I’m using an Adaptec RAID controller. Tried to install dpt-i20-raidutils, but they don’t seem to work either. Copied some third party munin Xen nodes from old box backup.
  • Installed sshd. Copied “authorized_hosts” from old backup, configured it to only allow public key authentication.
  • Configured the dom0 to take less memory. 96M was plenty on the old box, but this one didn’t boot until I increased it to 128M.
  • Make lvm disks for the domUs.
  • Copied one of the backups. Had to change the sxp file to specify the amd64 kernel, and copy the /lib/modules/*-xen-amd64 to the disk space. It boots, but for some reason it won’t start up the network.
  • Copied another backup. This time it booted the amd64 kernel just fine, but got a lot of errors on start up. But it did connect to the internet and stuff, so I’m not sure how critical the things that didn’t start up were. May have to try installing an i686 kernel and booting the xen instances with that.
  • The box rebooted spontaneously while trying to copy a lot of files over at once. Will have to try again without the memory restrictions (and maybe with the non-xen kernel). Will also have to make sure that it doesn’t do anything bad if one of the domUs is doing heavy i/o.
  • Tried again copying everything over with the non-xen kernel with 4Gb, and it still died.
  • Tried to disable RAID controller, didn’t work. So made 4 separate 1-disk “Volumes”, and go back to install Debian amd64 again.
  • Configured with /dev/sda with 2Gb /boot, 1Gb swap, rest available. /dev/sdb with 2Gb /, 1Gb swap, and rest available. Made “available” parts of two disks into an MD0 software RAID 1, then made that into a PV for LVM.
  • Overnight untarring of backups of mp3s and xen1 didn’t crash it. Woo hoo!
  • Installed sshd, copied config from old dom0, tested sshing in with a public key.
  • Installed xen stuff, and munin-node.
  • Untarred backups of xen2-3.

Next steps:

  • Copy the backups verbatim onto those disks, and hope like hell that Xen can boot them.