Month: September 2008

Drinking from the firehose

StackOverflow is now in open beta, so anybody can sign up and participate. And evidently, anybody has. The quality of the questions has gone way down, and the quantity has gone way up. It used to be that I’d stop back every 15-30 minutes and hit refresh, and there’d be a few new questions, but a couple I’d already seen would still be on the “Newest Questions” page. Now when I do that, not only have all the questions I’ve already seen been shoved off of page 1, sometimes they aren’t even on page 2.

And a lot of the questioners are obviously not looking as they’re typing their subject line, because one of the really nifty features of StackOverflow is that as you type your subject line, it picks out keywords and shows you other questions with the same keywords. If you pay attention, often times you’ll find your question has already been asked and answered. So seeing questions that you know were already answered before is a prime indicator that people aren’t paying attention to that feature.

In some ways, it reminds me of September in the good old days of Usenet. Hopefully it will calm down a bit after a while.

Actually, that reminds me of something – on day 1 of the open beta, somebody asked “So how is StackOverflow just not a re-implementation of Usenet groups”, which quickly got deleted as off-topic or moderated down so far that I couldn’t see it any more. (Which pretty much answers that question, doesn’t it?) I have some thoughts about that, but I should probably leave that for another post.

Nav data update

I’m testing a new update script for my navaid.com waypoint database. The old update scripts were written for when I was running on MySQL, and I’ve switched to PostGIS to support the new iPhone version of the CoPilot flight planning program. One of the salient features of the new iPhone version is that it attempts to be smart about downloading waypoints as you need them. One of the ways it does that is by asking my server for all the points in a particular area that have changed since a given date. The app keeps track of all the “areas” it has seen, and when the last time it was updated, and asks for an update of those areas at certain intervals. But that means I have to keep track of when a point was last updated. It also means that I need to keep track of what “area” a point is in. For the areas, I use a pseudo-quadtree where I allow only 500 points in an “area”, and when it gets more than that I split the cell into four sub-cells and mark the original cell as “superseded”. The new sub-cells have a “supersedes” value, so if the app asks for an area X, and area X has been split, I can say “X has been superseded, and here are the area ids A, B, C, and D that supersede it.”

But all this means that my new update scripts have to get the new data for a waypoint, figure out which old waypoint it was equivalent to (even if the waypoint has been resurveyed and is at a slightly different location and/or it has changed id), and only save the point anew if something significant has changed. Oh, and if the new data is missing information that the old data has, try to be smart about keeping the old data – for instance, George Plews’ Airports In Canada web site has data for airports in Canada that I can’t get any other way, but it’s also got data for airports that either were in the DAFIF data or are in the FAA data, and those two data sources often have much more information about runways and communications frequencies that Plews doesn’t have. So I want his latest data, but I don’t want to lose the other stuff that he doesn’t.

One of the things I do to match up the old with the new data is look with a bit of geographic “slop” – in the case where the ID matches I look within 0.05 degrees latitude and 0.05 degrees longitude (which believe me, in Alaska is way too big an area), and if the IDs don’t match, I look within 0.025 degrees longitude and 0.025 degrees longitude. These numbers were chosen extremely arbitrarily, and still causes a bit of a problem with a couple of airports that are near the US/Canada border because when I’m loading the FAA data it changes some Canadian airport to the nearby US airport, and then when I load the Plews data it changes it back.

Testing out my load scripts, I discovered two things:

  1. Sometimes the resurveyed point has moved enough that it’s in a different “area”. And that’s going to confuse the hell out of the algorithm that the app uses for getting updates, because it will ask for updates for the old area, and not get anything for that point. That’s going to require some thought to fix.
  2. In the next FAA data load, they’ve actually moved a couple of airports by 1.0 degrees of latitude or longitude. And judging by what I’m seeing on the Our Airports site maps, it appear the new values are correct, so the old ones must have been a data entry error. In this case, my “match the old” algorithm didn’t find anything to match within its radius of action, so it made a new point and marked the old one as deleted. The app should deal with that nicely.

Hmmm. Need to think how to handle this…

Stackoverflow comes through again

There is a major function in our program called the Migrator. It’s purpose is to “migrate” content from the content storage to the various feature players, either manually under user control or automatically based on what is on the schedule. Unfortunately the original requirements, put in place for a potential customer who ended up going a different way, were that the user had to be able to see and control each individual file in the migration job, and to each destination. This meant there is a tree showing each destination, and all the top level containers (playlists), all the middle level containers below that (CPLs) and then all the individual files (track files, fonts, projector control files, etc) below that, all wrapped up in a nice little collapsable tree. (Thanks to Sun providing an example implementation of JTreeTable on their web site.)
Continue reading “Stackoverflow comes through again”

Bought a paddle

Tonight’s “B Team” workout was actually a paddle tryout. We had a representative from Brača-Sport with about a dozen different paddles, and both the rep and Dan were doing a pretty good job of matching us up with paddles that are good for us. In my case, they suggested I try the Brácsa II, the VI and the VIII, but they were sure I’d like the VIII the most. And I did. The VIII has a very long and narrow blade, and it gives me a very gentle catch. With my big fat wide Big Spoons paddle, paddling hard gives me a big hard jerk when I first put the blade in the water and start pulling, but this one seems to ramp up from less drag to more gradually.

Dan, Lars (the factory rep), and several of the other paddlers remarked on how much better my stroke was with this paddle. Being a wing paddle, it comes out of the water as it passes my body almost on its own. And it’s light as a feather.

I’m really excited about this paddle – as I’ve been ramping up the amount I paddle in preparation for the Long Lake race, I’ve been getting more elbow pain, and I think the more gentle catch will go a long way towards alleviating that pain.

Funny story: the company is called Brača-Sport, but the paddles are called Brácsa. That wasn’t intentional – but after they made their first lot of paddles, they sent them out to be silk screened, and they came back with this typo on them. They had spent all their money on the carbon fibre, and had none left to fix the problem, so they shrugged and changed the name of the paddles but not the company.