Category: Rant

Hoist by my own petard

Yesterday I thought I’d try running my loading scripts on my home machine because it’s so much faster than my colo box. And it was running faster, and had gotten further than the still running script on my colo box when I got hoist by my own petard. I like to partition my systems up with different sized partitions for different major directories, and unfortunately, on this home machine, I only gave 100Mb for /tmp, not realizing that MySQL uses a temporary directory when you’re doing a long transaction. According to munin, the script filled up /tmp and died just a few minutes before I woke up this morning. So I’ve gone into /etc/mysql/my.cnf, changed the temporary directory to /var/tmp, and restarted it. /var/tmp has 7.6Gb free, so it should be safe.

Sigh

I made some major changes to the way data gets loaded into my navaid.com waypoint generator database, mostly in the processing of the “combined user data”. Mostly, I wanted to make sure that if “Bob” provides me some data on Canadian airfields that includes communications frequencies but no runway data that it doesn’t wipe out the runway data from the dataset of Ontario airfields that “Alice” provided me last year, but only updates the data that has changed in the overlapping part of those two datasets. Add in the possibility that a waypoint might have changed identifier or been resurveyed so the location has changed a bit, and you can see that there are a lot of possibilities to consider.

Unfortunately, considering all these possibilities is time consuming. I’ve been testing these new scripts with a dataset from one person that covers the entire UK and some nearby locations in varying levels of detail, and another that covers Ireland in great detail, but which is unfortunately no longer being updated because the person who provided it moved. Running both datasets would be an overnight job. But now that I’m satisfied with the results of that, I decided it was time to reload the old DAFIF data though these scripts to get the combined user data exactly the way I want it. But this has caught a couple of bugs in the scripts, one of which only manifested itself after 36 hours or so of running. That one didn’t even give me enough information as to why it failed, so I had to add some “use Carp” and “use Data::Dumper” magic to my scripts and then I re-ran it and found the actual cause after another 36 hour run. I’ve been almost continually running load scripts all week. I’m hoping this run will be it, but I’m not sure.

Since my new home box is so fast, I’m thinking one possibility might be to do the load processing on it, and then just mysqldump it and bring the dump file up to the colo.

Another reason why Exchange is not my favourite mail server

I belong to a YahooGroups mailing list for a club I joined. This morning, in quick succession, I got a message, then I got a message saying it was a correction to the previous message, and then I got 13 (so far) messages with the phrase “Message Recall Failure” followed by the subject of the first message, sometimes with some spam or anti-virus catcher stuff inserted.

A google suggests that the person who sent the first message decided to use an Exchange feature that allows you to “recall” or cancel a message you sent. Evidently the recipients who use Exchange servers got these “recall” requests, but didn’t like them for some reason, probably because the original had come through the yahoogroups server rather than the originator’s server. And Exchange had expressed its disapproval of the recall message by sending a message back to the yahoogroups list address, which went out to everybody on the list.

I’m just hoping those 13 messages are it, and we don’t get any more.

Not a good idea

This morning I was investigating a problem at a customer site where their Manager (a program I wrote most of) is hanging. Thanks to “kill -3 pid” I discovered it was building a tool tip. Thanks to a database query, I found that the tool tip it was building was trying to display 430,000+ identical date ranges. Hmmm, looks like we’re going to be a while, and probably a little short of screen real estate at the same time.

I did a few database commands to remove the 430,000+ records and put back one copy of it. (I’m sure somebody has a command that will do both in one step – I did a “delete” followed by an “insert” because I’m lame.)

The Field Engineer at the customer site wrote to ask if I could tell him the database commands in case this happens again. I politely declined. I told my boss that this would be somewhat like giving handguns and tequila to a bunch of 9 year old boys. Repeat after me:

I’m sorry, you appear to have totally hosed the database. You’re going to have to wipe the database and re-injest all your content. You’ll probably be up and running again in two days or so.