Everything I used to bore people on newsgroups and mailing lists with, now in one inconvenient place.
Design and construction of a navigation data community to replace and supplement the DAFIF data.
Man this off-line blogging is a blessing and a curse, isn’t it?
Now that Amazon Honor System has gone away, what other options do I have for soliciting donations on my navaid.com website? I get most of my donations through PayPal, but some people preferred to stay away from PayPal for some reason. Amazon has a donation service, but only for registered charities.
We’re only talking about $5 or $10 a month, so it’s not like I need a full ecommerce solution.
Some years ago I got a set of Shape Files (much as I hated ESRI at the time, they did beat us (GeoVision) fair and square with an inferior product and superior marketing) for the provincial and territorial boundaries of Canada and wrote a short little C program to do a “point in polygon” to determine what province a particular point is in. It’s set up so that it parses the shapes, then sits there waiting for a lat/long pairs to come over a pipe and writes the province code back over the pipe. I write a special lat/long pair (-999/-999) when I want it to exit. I use it all the time when I’m loading waypoints. The program has continued to work while my waypoint generator moved from being hosted at home to a webhost (Gradwell.com) to a virtual private server (Linode) to a colocation box. Unfortunately, I just discovered that somewhere along the way I lost the source code. Right now it has a small bug in that when the program that opened the pipe to it dies, it starts consuming all the CPU instead of shutting down. I can live with that, but it’s annoying while I’m doing all this testing with buggy load scripts. That’s why I went looking for the source code.
Maybe I should write a new one just in case?
I made some major changes to the way data gets loaded into my navaid.com waypoint generator database, mostly in the processing of the “combined user data”. Mostly, I wanted to make sure that if “Bob” provides me some data on Canadian airfields that includes communications frequencies but no runway data that it doesn’t wipe out the runway data from the dataset of Ontario airfields that “Alice” provided me last year, but only updates the data that has changed in the overlapping part of those two datasets. Add in the possibility that a waypoint might have changed identifier or been resurveyed so the location has changed a bit, and you can see that there are a lot of possibilities to consider.
Unfortunately, considering all these possibilities is time consuming. I’ve been testing these new scripts with a dataset from one person that covers the entire UK and some nearby locations in varying levels of detail, and another that covers Ireland in great detail, but which is unfortunately no longer being updated because the person who provided it moved. Running both datasets would be an overnight job. But now that I’m satisfied with the results of that, I decided it was time to reload the old DAFIF data though these scripts to get the combined user data exactly the way I want it. But this has caught a couple of bugs in the scripts, one of which only manifested itself after 36 hours or so of running. That one didn’t even give me enough information as to why it failed, so I had to add some “use Carp” and “use Data::Dumper” magic to my scripts and then I re-ran it and found the actual cause after another 36 hour run. I’ve been almost continually running load scripts all week. I’m hoping this run will be it, but I’m not sure.
Since my new home box is so fast, I’m thinking one possibility might be to do the load processing on it, and then just mysqldump it and bring the dump file up to the colo.
Thanks to a post by Skud, I’ve discovered Freebase, which is, as Skud described it, “crack for information nerds”. It appears to be exactly what I was hoping to develop or find for my aviation navigation data project – a flexible structured information agregator, almost like a wiki with user definable fields.
You probably can’t see much of it, because right now it’s in alpha and by invitation only, but so far they’ve imported all of Wikipedia and added some links between then, and people have written some demo applications.
Like many AJAX applications, it has a tendency to get the dreaded “a script on this page is running slowly”, but otherwise it’s pretty nifty.
My “DAFIF Replacement” Wiki is currently shut down because of a bug that was causing my computer to hit load averages up over 10 every time Google indexed it. My log files were full of lines like:
[Thu May 31 21:09:21 2007] [error] [client 66.249.72.194] [Thu May 31 21:09:21 2007] view: Use of uninitialized value in string eq at /var/www/twiki/lib/TWiki/Plugins/TablePlugin/Core.pm line 457.
[Thu May 31 21:09:21 2007] [error] [client 66.249.72.194] [Thu May 31 21:09:21 2007] view: Use of uninitialized value in string eq at /var/www/twiki/lib/TWiki/Plugins/TablePlugin/Core.pm line 462.
I hope to have time to upgrade to a newer version of TWiki Wiki one of these days and then I’ll re-open it. Or possibly I’ll just let it die since it wasn’t getting any use anyway.