Rants and Revelations

Ok, it’s been a few hours since I upgraded to Leopard. The first time it booted after it installed, I got a kernel panic. But it booted the second time. It’s been at 100% CPU usage ever since, installing the XCode stuff (which I always install but never use) and reindexing for Spotlight. That’s colouring a few impressions, because it means Cover Flow, for instance, is very slow. But in brief

• iScroll doesn’t work. It’s probably what caused the kernel panic on that first boot. I hope there is an update soon, because I like two fingered scrolling.
• Safari has a really disconcerting and distracting way it gets really dark when you switch to it, and light when you switch away. Some of the other apps do that as well, but Safari gets too dark.
• Months ago I quite distinctly remember reading that the Airport Express’s external drives would be usable with Time Machine. As a matter of fact, that was a major reason for buying the AE. But now it turns out that AE drives aren’t supported in Time Machine. And considering what a piece of shit the AE turned out to be, I can’t think of any reason not to put it on Craigs List.
• I was also looking forward to being able to use Mail again. The Tiger version of Mail insisted on indexing and scanning every file in my ~/Mail directory on the IMAP server, which was incredibly time consuming. Every other IMAP client I’ve tried, from Thunderbird to SnapperMail, has the ability to specify that you only subscribe to specific folders instead of all of them. Well, Leopard’s Mail is supposed to have that as well, but when I open up the dialog, no folders are listed there and I can’t find a way to add or remove folder subscriptions. I’ll try again when the CPU isn’t pegged, and maybe it will work right.

I was really looking forward to getting the advantages of Time Machine without having to have a hard disk on my desk, but if that’s what I have to do, that’s what I’ll do. In the mean time, I’m glad that Apple finally came out with Spaces so I don’t need a third party virtual desktop software (actually, I gave up on those ages ago because they kept getting broken by OS updates). And Vicki and I will have to experiment with some of the remote desktop stuff.

The more I read Joel on Software, the more I’m convinced that if there software jobs in heaven, they’ll look an awful lot like this. Today’s post, called Evidence Based Scheduling, is just one example of the sort of nirvana that I’m hoping awaits me after death if I’ve been sufficiently good, because I sure as hell haven’t seen anything like it in this life.

I’ve certainly seen my share of the ones he gives as bad examples. The schedules passed down from above. The “I’ll give them 30% less time than they said it would take, and then creep the feature list” managers who think it’s motivating to put you under stress, the managers who do the same and then put the blame on you when the project is late, the projects with no schedules and no clear deliverables, but which still manage to make you feel like you’re not producing enough, the “fire 30% of the team but don’t change the schedule” managers, the “if you guys were any good you wouldn’t need so much time for debugging” managers (who not coincidentally are the ones who didn’t give you any time for designing up front), the ones who are as fickle as the wind when it comes to deciding what features are absolutely 100% necessary to win this customer (who turns out to have already decided to go with your competitor), and the “compile it, run it once, deliver it to the customer and if they complain, roll them back to yesterday’s build” ones.

Yes, that’s what life is like down here on earth.

Now that Gmail supports IMAP, I got it working on my Treo. W00t! It’s pretty simple, but you have to use SnapperMail (the expensive version that supports IMAP rather than the cheap version that doesn’t). I created a new account. Obvious stuff under “Your Name” and “Email Address”. Under Server:

• Incoming Mail:
  • IMAP4 Server: imap.gmail.com
  • Username: yyyyy@gmail.com
  • Password: zzzzzzz
• Outgoing Mail:
  • SMTP Server: smtp.gmail.com
  • Username: yyyyy@gmail.com
  • Password: zzzzzz

Then click the “More…” button. Change the IMAP port to 993, and the SSL option to “Always Secure (wrapped port)”. Change the SMTP SSL option to “Always Secure (STARTTLS)”. I made a couple of other changes but they didn’t stick, and it’s still working.

The best thing is that if like me you’ve got filter options that label things and skip the inbox, IMAP treats those labels like folders so you can sync them as well. And if you delete something on your Treo, it gets archived but not deleted on GMail.

I can’t wait to see how well this work with the new Leopard Mail.app. I hated the way Mail.app worked with multiple accounts and multiple folders in Tiger, but I’m told that’s all fixed now.

I gave my GWT presentation to my fellow developers and the two project leaders today. I think it went very well.
(more…)

The new president of the flying club has been a member since Orville and Wilbur signed him off for his license. And like a lot of old-fart pilots, he seems to resent any attempt to get him to use more computer technology than just the bare minimum to get a weather briefing. (I should mention in passing that he also seems baffled by the concept of a period (full-stop for you UK readers), and seems to randomly sprinkle the gaps between his sentences with anywhere from 3 to 5 of them. It’s like reading somebody from somebody who mumbles and just trails away to nothing at the end of every sentence.)

Because of some big decisions that have to be made in the club, and the fact that with the shrinkage in pilot numbers probably nearly half the club are either on the Board of Directors or are club officers, I decided to open up the officers mailing list to BoD and asked all the officers and BoD who were not currently on the mailing list to join. When that didn’t work, I used the mailman “send an invitation” function to invite them all, including the new President.
(more…)

I’m still trying to find a way to deal with the problem of assigning Canadian waypoints to their correct province. The first thing I did was grab George Plews Airports In Canada data file, which as well as giving me hundreds of new airports that I didn’t have already, also made sure that all my airports were in the right province. But I’ve still got problems.
(more…)

Ever told yourself “oh, I don’t have to sanitize the input because I’m the only person using it” only to have it bite you in the ass?

When I’m loading waypoint data into my database, I calculate the magnetic declination of each point using a program that I got years ago and hacked the hell out of. I call it magvar, because declination is sometimes (incorrectly) called “magnetic variation” and I didn’t know any better when I did it. The program as it was written prompted for input (latitude, longitude, elevation and date) one number at a time, validated it, parsed the World Magnetic Model (WMM) data file, and told you the declination and a bunch of other stuff about the location. Well, I needed it to be faster than that, so I had it pre-parse the WMM file, then it sat there in a loop where it did a "sscanf" of the four input numbers (C programmers are now shuddering in horror), and printed the output, and then my perl script did an "open2" to open a pipe to write the four numbers on, and another pipe to read the result. And that’s worked pretty well up until today.

Today I was loading some new datasource, and I noticed that about 75% of the way through, it was hanging. And it appeared to be hanging in the write to the magvar program’s input pipe. I tried commenting out the call, and it ran fine, but of course it didn’t have any declinations. So I put the call back in and ran it again. And then attached to the executable with gdb (some old atrophied skills suddenly got refreshed in memory). And that’s where I discovered that the program seemed to be stuck in a write. And going up a few levels into the code that I’d touched and dumping the local variables, the input latitude and longitude seemed to be indicating a waypoint that was one of the first ones input. That’s when I had another look at the data I was feeding the program. And that’s where I discovered instead of writing 4 doubles that scanf could happily read using "%lf %lf %lf %lf", I hadn’t noticed that on some of the waypoints in this new datasource, the elevation was given as "apprx 123". I didn’t bother to look in detail what happened at this point, but I assume my unchecked input caused the magvar program to go into an infinite loop, spewing out the same declination value over and over onto the perl program’s input pipe until the pipe filled up.

And I haven’t learned my lesson - I have no plans to fix magvar to validate its input. I’m just going to make sure this particular data loader program does a
$elev =~ s/\s*apprx\s*//;
before calling it.

Our company has a translation group. Normally we send off our properties files, and we get back the versions for the languages we support in this application. Sometimes we get a few questions from the translators, usually just a clarification of how we’re using a term or a request for some context. I send back and answer, and a few days later I get my translation.

I don’t think their Chinese translator quite understands this process though. I got back a list of translator questions from him, and it’s all stuff like “Please confirm this translation of the string ‘Foo Bar’” followed by a string of Chinese characters. Or “Should the word ‘Digest’ be translated or left as English?” I feel like writing back and saying “HOW THE FUCK SHOULD I KNOW? YOU’RE THE ONE WHO CAN SPEAK CHINESE!” But I won’t. Instead I’m writing to his manager to explain that we can’t answer his questions because we don’t speak Chinese, and leaving out the part about how if we did speak Chinese we wouldn’t need them.

You know, if you’re going to send email to a person who runs a mailing list, don’t assume that he only runs one mailing list. Generally the skills for running one list are transferrable to other mailing lists, so the person in question might run more than one. Take me, for instance. I run probably 2 dozen or so lists. Which is why I get so annoyed when I get email like this:

To whom this may concern:

I am from [Clueless Wankertech] Inc. and we have an event coming up that we feel
the members of your mailing list will benefit from. Below is a post
which we are wondering if you could post to your mailing list or allow
us to do this. Thank you for your time and consideration.

The post is as follows:
[etc]

Every now and then, the Wordpress OpenID comment plugin I use goes mental and starts taking people’s comments and posting them to a non-existant post, and also stripping out all the text of their comments. I suspect it’s an interaction with SpamKarma2 and/or Akismet, but there is no way I’m going to do without spam protection, so I guess at times like this, my only option is to deactivate the OpenID plugin.

So sorry pir and jdev, but your comments were lost. Please feel free to submit them again.

I’ve had this iPod since April. (One thing this blog is good for is that I can use it to check on the petty details of my life.) All this time I’ve had it in an incase leather case, which puts a rubber membrane on top of the scroll wheel. That membrane kind of slows down the scrolling, so I usually use the new “Search” function because scrolling through however many hundreds of artists or thousands of albums to find the one that’s been running through my head all night is too damn slow. But today I took it out of the case for some reason, and something amazing happened. I was scrolling though the list of artists, and suddenly I got a gigantic “A” in the middle of the screen, and each tick of the scroll wheel moved one letter instead of just one album. Once I got to the first letter of the artist I wanted (Captain Tractor, btw), I paused and it went back to slow scrolling through the individual items. This is very cool - it really speeds up scrolling to where I want. Once again, Apple pleasantly surrprises me with user interface delight.

Now if only they’d make it so that going fast goes to scrolling through the first letters, then one quick pause and you’re in quick scrolling through second letters, and such.

As I wrote in Rants and Revelations » C Paul Program, C Paul Rant, I’m trying to “recreate” a C program that I wrote back in 2001. I’m using the same shapelib library for reading ESRI shape files, I’m using (what I think is) the same algorithm for doing point in polygon tests, and it’s using the same shape file, one that was published on the ESRI web site back when Nunavut first became a territory. And it seems to be up and running.

So I wrote a little perl script that goes through my database and picks out the 3400+ waypoints that are in Canada and which come from DAFIF, and run them through both the old and new program. Both the old and new programs are looking at the same shape file. So you’d expect the exact same results, right? Wrong. There are 309 points that the old program assigns provinces to that the new program says are outside of any provincial boundary. Some of those are ones that I’d already noticed were assigned to provinces even though they were way out at sea and probably shouldn’t be, but some were airports that are near the coast or on islands. That’s a problem.

My first suspicion was that the algorithm as published used floats, but I’d converted it to using doubles because face it, these days computers are so damn fast. But I switched back to floats and now there are 314 differences. Some of the original 309 are now back to what they were, but some points have jumped provinces, like COMPR which moved from Alberta to Saskatchewan (which Google Maps just barely agrees with, by the way) and KITAR which moved from British Columbia to Yukon Territory (which Google Maps says is a toss up). But most importantly, it didn’t “fix” any of the coastal or island airports that ended up with no province. I’m not convinced that’s a net positive. So I’ve gone back to using doubles.

Ok, one other difference is that the notes for the algorithm mentioned a way to compensate for shapes with holes in them by inserting (0.0,0.0) points between the rings. I don’t think I did that before. So I tried without. No dice - still 309 differences.

And then I remembered one other difference. I call SHPRewindObject on each shape as I read it. According to the docs, that’s supposed to fix any problems with shapes that go the wrong way round. But no, that didn’t change anything either.

So I’m left with a mystery about why there is a difference between the old program and the new one. Since I don’t have a GIS program that can read and manipulate shape files, I think my next step will have to be to turn these shape files into Google Maps API polygons so I can plot these wayward points and see if the problem is in the shapes or in the algorithm.

Well, only four months late, but I finally got the check I’ve been waiting for from Google.

See:
Rants and Revelations » Oh Google, you are so devoid of any semblance of clue and
Rants and Revelations » Hey, Google for previous ranting about this subject.

I won’t rant about the fact that they only allow $30 a day for meal expenses, and my breakfast in the hotel cost more than that. Nope, I won’t.

I was playing around with the Google Maps API and loading some of my nav data into Google Maps when I noticed a few waypoints with the wrong province. Oh oh. For instance, CDT5 Buctouche Airport is marked as being in Quebec instead of New Brunswick.

The DAFIF data never provided province data, so a long time ago (March 2001 according to Google Groups) I located some shapefiles with provincial boundaries and wrote a C program to do “point in polygon” to figure out what province the points are in. Obviously there is a mistake either in my algorithm or the shapefiles. Which is too bad, because a while ago I discovered that I don’t have the source code for the program any more. Not sure where it went, but I can’t find it anywhere. Funny, that program has moved without recompiling from my home system to Gradwell hosting to Linode Virtual Private Server to my rack to back on my home system. But now it’s time to recreate it.

The first step to recreating it is to find the library I used to access shapefiles, and after a bit of searching I discovered its name and web site, and also that it’s in the default Debian distribution. Of course, being Debian, in spite of the fact that it’s a positively tiny little tar file when you install from source, they had to break it up into libshp1, libshp1-dev and shapefile packages. Was that really necessary?

The next step was to find the Point In Polygon algorithm, but fortunately the newsgroup thread that I found my asking about it in 2001 also had the pointer to the comp.graphics.algorithms FAQ for that.

Now the biggest hurdle is that I’ve got to get my mind back into C programming again. Ugh. It’s hard to think that I used to do this shit every day, and it was all second nature to me. Now I’ve got to stop and think that when the function signature says “double *” and it returns X, Y, Z, and “M” (whatever that is), that I really need to declare “double maxBounds[4]” and pass it as “maxBounds”, but when it says “int *” and it returns a single value, I need to declare it as “int nVertices” and pass it as “&nVertices”. Even when I was doing C++ I didn’t have to deal with crap like this because C++ programmers were *far* more likely to declare a class and use that as the return type instead of passing in a huge list of arguments to return values in. There was a time when I thought C was the greatest language. And I helped write a gigantic Geographic Information Systems product in it. Those were the days.

My favourite airplane, our club’s Piper Lance, is up for sale on eBay Motors:

eBay Motors: 1977 Piper Lance PA-32-300R - Great Buy, Flies Often! (item 140161831797 end time Oct-14-07 17:00:00 PDT)

Personally I think the reserve is too high, but I think they’re just trying to find out what the market will really bear. It looks like the only bidders so far are dealers bidding less than half what it’s really worth, looking for a totally desperate seller, and we’re not there yet.

But the plane’s annual is due in December, and we were told last year that the engine probably won’t pass another annual, and there just aren’t enough club members willing to pay the surcharge to justify spending the money for an engine and prop overhaul. So this plane is going to get sold, somehow.

So good bye, old friend. I’ll miss your speed, your load capacity, and your outstanding interior room. But mostly I’ll miss the fact that it felt like a good solid honest plane.