Maybe I should write a new one just in case?

Unfortunately, considering all these possibilities is time consuming. I’ve been testing these new scripts with a dataset from one person that covers the entire UK and some nearby locations in varying levels of detail, and another that covers Ireland in great detail, but which is unfortunately no longer being updated because the person who provided it moved. Running both datasets would be an overnight job. But now that I’m satisfied with the results of that, I decided it was time to reload the old DAFIF data though these scripts to get the combined user data exactly the way I want it. But this has caught a couple of bugs in the scripts, one of which only manifested itself after 36 hours or so of running. That one didn’t even give me enough information as to why it failed, so I had to add some “use Carp” and “use Data::Dumper” magic to my scripts and then I re-ran it and found the actual cause after another 36 hour run. I’ve been almost continually running load scripts all week. I’m hoping this run will be it, but I’m not sure.

Since my new home box is so fast, I’m thinking one possibility might be to do the load processing on it, and then just mysqldump it and bring the dump file up to the colo.

You probably can’t see much of it, because right now it’s in alpha and by invitation only, but so far they’ve imported all of Wikipedia and added some links between then, and people have written some demo applications.

Like many AJAX applications, it has a tendency to get the dreaded “a script on this page is running slowly”, but otherwise it’s pretty nifty.

I hope to have time to upgrade to a newer version of TWiki Wiki one of these days and then I’ll re-open it. Or possibly I’ll just let it die since it wasn’t getting any use anyway.

I was calling points the same if the types matched and they’re within 0.05 degrees latitude and 0.05 degrees longitude of each other. Unfortunately that is just about 3 nautical miles in the north/south direction, which this test has shown is too wide a net.

The problem is that I want to spot duplicates when a waypoint changes id, AND when they update the coordinates. I’ve seen places where they’ve updated the coordinates by half a degree, especially in the case of user-entered data.

I think what I’m going to have to do is trust that the coordinates aren’t going to change a whole bunch at the same time the id changes. So what I’ll do is call something a duplicate if it’s within 0.05 degrees if the ids match, but within 0.01 degrees if the ids don’t match. That’s less than a nautical mile, and it would be pretty odd to find two airports within a nautical mile of each other. (A lot less odd to find heliports or reporting points, unfortunately.)

Damn, this means another multi-day test run, unfortunately.

I have renamed the part of the Wiki that uses the trademarked word
“NavData” to “DAFIFReplacment”. However, I am going to continue to use
the “/navdata/” part of the URL as that is a generic term and
untrademarkable and changing would break people’s bookmarks. You can
have a look at http://xcski.com/navdata/ if you wish.

I hope that meets your requirements.

Evidently their lawyers work nights, or they’ve outsourced it to India or something, because I got a response at 8:47pm:

Mr. Tomblin,

We appreciate your prompt action and reply to our notice.

While we cannot agree that the navdata term is generic, we understand the
bookmark issue and are satisfied with your action regarding this matter.

John Jaugilas
Jeppesen Intellectual Property
(303) 328-4178

This morning, I get a notification from the Wiki software that somebody has edited nearly every single page in the whole damn Wiki. Needless to say, it was all spam. It took me nearly two hours of messing around in RCS to get rid of every instance of the spam. But even worse, is that it turns out that somebody had already inserted the same spam into the navigation bars of the wiki months ago, and I hadn’t noticed. So my Wiki is actually showing up quite high in Google searches for certain drugs. ARGGGH!

Also, a while ago I mentioned that this blog tends to get spam in brief spurts of a few dozen spams over the course of a weekend, and then nothing for weeks at a time. Well, that seems to have ended - I’m getting spam every day now. I knew it was too good to be true. Fortunately SpamKarma2 is doing a great job of finding it and quarantining it so I don’t have to continually check. And it’s pretty good about not having false positives either - comments that it thinks are borderline spam are given a chance to fill out a captcha and then that comment gets through, and subsequent comments from the same user are given a few positive points. So I’m actually relieved that the expected has happened, and it hasn’t inconvenienced me much.

Maybe if I’d seen Jen’s comment earlier I would have tried DokuWiki instead. It seems to compare favourably if you look on WikiMatrix.

So anyway, the NavData Wiki is now set up here. So far one person has already found it and started to contribute, much to my surprise because it hadn’t been announced yet.

One of the things I’d really like to do in this project is provide the capability for people to leverage the previous editors, but not allow them to accidentally or on purpose ruin the data. Thus I don’t want some random stranger to delete a DAFIF waypoint, or move KJFK to Timbuktu. So I was thinking that each editor would have a “dataset” of their own, but it’s based on previous datasets. There would be a heirarchy of datasets, with of course FAA and DAFIF data as the base, then an editor or end user could decide to include or exclude the other datasets based on whether that dataset intersects their area of interest, their perception of the dataset owner’s credentials, the recency of updates, or other criteria. Not sure exactly how to accomplish that. Maybe a rating system or a public comment area where another editor or an end user can say “Hey, this guy’s points are really inaccurate” or the editor himself could say “I didn’t type in the lat/longs from a publication, I just took my GPS to the airport fence”?

I don’t want an editor to be able to delete another person’s waypoint, but I also recognize that waypoints disappear or change names. So I think there needs to be a way in dataset “a” to indicate that waypoint “FOO” no longer exists. That way if you’re using dataset “a”, you won’t get waypoint “FOO”, but if you don’t trust “a” and so exclude it, you’ll still get “FOO”.