Category: Rant

More problems setting up the new colo box:

• I shut down with all three domUs running, and when it came back up, xend wasn’t running so they wouldn’t start up again. Further investigation showed that I hadn’t made my root partition big enough to handle when it saved the current xen state to /var/lib/xend. So I made a new lvm partition for /var/lib and mounted that instead. So far, so good.
• While I was investigating this, I noticed my software raid was running in degraded state, because it had lost /dev/sda3. I re-added it and it started to rebuild it.
• While it was rebuilding, I noticed that xen1, my first domU wasn’t running any more. When I tried to recreate it, it told me I couldn’t recreate it because its main disk, /dev/xen-space/xen1-space, was mounted in a guest domain. Oh oh.
• Then I got a kernel panic. Double oh oh.
• I rebooted, and tried to rebuild the raid without xend running, but I got another kernel panic.

This is supposed to be easy and fun, right?

I’m currently booted with the rescue disk, and I’m trying to rebuild the raid again. If that dies, it might be time to cut my losses.

Earthlink is blocking email from my colo box (which is on a static IP, has never sent spam, and isn’t on any known block list in the world). I jumped through their hoops to report this fact, and got two emails within a few seconds of each other:

• One claiming that I’m not blocked, so why am I claiming I am?
• One claiming that they’re removed the block that they just finished claiming wasn’t there.

And of course, email is still bouncing. Of course they did caution that removing the non-existant block will take 2-24 hours.

In the last hour, I’ve been told

• Don’t use SYSDATE in inserts/updates because although it’s bog standard SQL (I’m not sure if it’s ANSI standard, but it’s pretty common), it’s not supported by all Microsoft databases (even though we only use Oracle in this project)
• Do use VARCHAR2 instead of VARCHAR, because although VARCHAR2 is only supported by Oracle, and in the current release it’s just a synonym for VARCHAR, some day it’s going to have different comparison semantics than VARCHAR and Oracle says to use it, so we have to use it.

I’ve also been told that these two tables, which I have to populate by hand using manually taking the rows and columns of a spreadsheet and writing “INSERT…VALUES(‘row’,’column’,’value’)” statements several hundred times, can’t use semantic primary keys because they want to use UUIDs. So instead of looking at the spreadsheet and seeing that in the row labelled “Insurance” and the column labelled “Security” that the value is “V” and converting that to “INSERT … VALUES(‘insurance’,’security’,’V’)”, I’ve got too look up the uuid for the Insurance row and the uuid for the Security column, and change that to “INSERT … VALUES(‘6BAC51EC-C636-4C31-9E95-367062AC23F7′,’C78BF79B-3178-4F07-ACD3-92DF2742C932′,’V’)”. And I’ve got to do that several hundred times. Yes, that seems *much* less error prone that using keys I can actually tell what they mean and easily tell if I’ve got the wrong one. Oh, and even better, the code that uses the information in this table will have to hit the database to look up these uuids so that they can find the value of the “insurance” versus “security” instead of just coding those values directly.

I was on a conference call, and I had to switch from my cell phone to Skype because the call was breaking up too much on the cell call. Yes, it’s a pretty incredibly sad state of affairs when Skype provides a clearer, less broken up signal than your cell phone!

Only 1 year and 6 months until this contract is over and we can switch to back to AT&T or over to Verizon.

If you use rsync to backup your system, and the system you’re backing up to has different uids for some userids, it converts them as it stores! I just found this out because after restoring my xen1 backup, I’ve discovered that all my postgres files belong to 114, which is the uid of postgres on my home server, not on xen1.

This is going to make restoring all the xen backups a royal pain in the ass.