Category: Geekery

Sigh

I made some major changes to the way data gets loaded into my navaid.com waypoint generator database, mostly in the processing of the “combined user data”. Mostly, I wanted to make sure that if “Bob” provides me some data on Canadian airfields that includes communications frequencies but no runway data that it doesn’t wipe out the runway data from the dataset of Ontario airfields that “Alice” provided me last year, but only updates the data that has changed in the overlapping part of those two datasets. Add in the possibility that a waypoint might have changed identifier or been resurveyed so the location has changed a bit, and you can see that there are a lot of possibilities to consider.

Unfortunately, considering all these possibilities is time consuming. I’ve been testing these new scripts with a dataset from one person that covers the entire UK and some nearby locations in varying levels of detail, and another that covers Ireland in great detail, but which is unfortunately no longer being updated because the person who provided it moved. Running both datasets would be an overnight job. But now that I’m satisfied with the results of that, I decided it was time to reload the old DAFIF data though these scripts to get the combined user data exactly the way I want it. But this has caught a couple of bugs in the scripts, one of which only manifested itself after 36 hours or so of running. That one didn’t even give me enough information as to why it failed, so I had to add some “use Carp” and “use Data::Dumper” magic to my scripts and then I re-ran it and found the actual cause after another 36 hour run. I’ve been almost continually running load scripts all week. I’m hoping this run will be it, but I’m not sure.

Since my new home box is so fast, I’m thinking one possibility might be to do the load processing on it, and then just mysqldump it and bring the dump file up to the colo.

Another reason why Exchange is not my favourite mail server

I belong to a YahooGroups mailing list for a club I joined. This morning, in quick succession, I got a message, then I got a message saying it was a correction to the previous message, and then I got 13 (so far) messages with the phrase “Message Recall Failure” followed by the subject of the first message, sometimes with some spam or anti-virus catcher stuff inserted.

A google suggests that the person who sent the first message decided to use an Exchange feature that allows you to “recall” or cancel a message you sent. Evidently the recipients who use Exchange servers got these “recall” requests, but didn’t like them for some reason, probably because the original had come through the yahoogroups server rather than the originator’s server. And Exchange had expressed its disapproval of the recall message by sending a message back to the yahoogroups list address, which went out to everybody on the list.

I’m just hoping those 13 messages are it, and we don’t get any more.

Not a good idea

This morning I was investigating a problem at a customer site where their Manager (a program I wrote most of) is hanging. Thanks to “kill -3 pid” I discovered it was building a tool tip. Thanks to a database query, I found that the tool tip it was building was trying to display 430,000+ identical date ranges. Hmmm, looks like we’re going to be a while, and probably a little short of screen real estate at the same time.

I did a few database commands to remove the 430,000+ records and put back one copy of it. (I’m sure somebody has a command that will do both in one step – I did a “delete” followed by an “insert” because I’m lame.)

The Field Engineer at the customer site wrote to ask if I could tell him the database commands in case this happens again. I politely declined. I told my boss that this would be somewhat like giving handguns and tequila to a bunch of 9 year old boys. Repeat after me:

I’m sorry, you appear to have totally hosed the database. You’re going to have to wipe the database and re-injest all your content. You’ll probably be up and running again in two days or so.

Spoke too soon

Yesterday I wrote that I’d turned off the old machine for the last time. Well, it turns out I was wrong. I had put the mp3s in the wrong location, and had the permissions wrong, and so the backup script’s rsync deleted the mp3s from the wrong place and then couldn’t write them to the correct place. Which meant that this evening I had to turn the old machine back on, copy the mp3s back over (only having to reboot the old machine twice as it froze up under heavy load as it is prone to do) and then shut down the old box once again. Now that that’s done, I have to re-import my iTunes library, which will undoubtedly continue long after I’ve gone to bed tonight.

On the plus side, though, I managed to get the UPS software NUT installed and working. My new box doesn’t have a serial port, which means I had to use the USB driver for the first time. The kernel first identifies the UPS as a X Box controller (xpad), but eventually the newhidups driver figured it out.

Cutover complete

The old box is shut down, hopefully never to be turned on again, and the new box is up and running with nearly all the old functionality. It’s very quiet in this office with that old beast off.

As well as the base install, and fixing all the Ubuntu stupidity (and a few bugs in the munin-node scripts), I had to install and configure all the following:

Apache2
Postfix
Postgresql
MovableType
Bind
user accounts
bogofilter
spamassassin
Root cronjobs
ssh tunnel
My cronjobs
trn
/etc/profile
Mutt configuration
munin
squirrelmail
iptables/ssh-black
frm/nfrm
ftp server
NFS export /mp3s

I still need to get nut set up to control the UPS, and hook up a printer and configure it. I’d also like to get Java and eclipse set up so I can do some work on it.

Right now iTunes is cranking away importing the “iTunes Music Library.xml” where I changed the old nfs mount point to the new one. It’s taking a while.