Ellison Park Shuttle Again

I decided to do the Ellison Park Shuttle again, this time without Vicki. The weather was perfect, and because of that there were a lot of people out. While I was waiting for the shuttle to leave, a large group left including a woman with a dog in her kayak. The dog had on a life jacket, which is just as well because they weren’t more than 20 feet from the dock when the dog decided to jump out, and the woman had no idea how to get the dog back in. By the time we left she was coming back to the dock with the dog swimming along side. I have no idea whether they got sorted out or not.

On the shuttle with me there were two tandem kayaks, two canoes with two people in them, and one other solo kayak, none of them looking very experienced. I helped the driver get them launched, and left after them. By the time I caught up to them (at the next bend), they were all over the river and pointing in random directions. Two women in one of the canoes rammed me as they spun around and flailed at the water, bringing back some bad memories from when I hurt my wrist last year. I hope they got settled down and pointing in the right direction eventually, because there were some snares later on.

I brought my GPS along as an experiment. The manual says it’s waterproof and floats so I thought that I’d be safe. The idea was to get a better idea of the real length of the course, and also the speed I maintain. But it was kind of a waste – during the twisty part at the top of the course, it kept losing signal, often for long stretches. So distance was even less accurate than my previous attempt using the Google Maps Pedometer.

The river was pretty high after the rain, and moving pretty fast. But the recent storms also left some traps for the unwary. There was a downed tree that blocked most of the river leaving a very fast passage along one side. And then not too much later just past Browncroft Avenue there was another tree that had fallen all the way across, but it’s actually not in the water but above it. There was a group coming upstream under it when I got there, and I was able to also duck under it in one part.

There wasn’t a lot of wildlife out today unless you count hordes of inexperienced paddlers. I saw kingfishers in the twisty wooded bit at the top, and some barn swallows under the bridges, and a few ducks, but that’s about it. I didn’t see the usual swans or geese.

Back at Baycreek, there were a couple of guys paddling around with Greenland paddles and kayaks. That looks like something I’d like to try out some time. I’ve heard they’re better on your elbows and shoulders.

Well, that was fun

I noticed my backup USB drive wasn’t mounted, so my nightly backups hadn’t been happening for I don’t know how long. And when I tried to mount it, it mounted in USB “fast speed” mode, rather than “high speed”. I tried a few things, and I still couldn’t do it. So what the hell, I thought, it’s been over 6 months since my last reboot – yum has installed several new kernels, this will be a chance to use one of them.

Nice plan, but unfortunately the damn box wouldn’t boot. The first message I saw:

INIT: version 2.85 booting
mount: error while loading shared libraries: libcryptsetup.so.0:
cannot open shared object file: No such file or directory
awk: cmd. line:1: fatal: cannot open file `/proc/mounts’ for reading
(No such file or directory)

I booted with a rescue disk, and discovered that the thing it’s complaining about, libcryptsetup.so.0, is in /usr/lib. And guess what: /usr is a different partition than /, so it isn’t mounted before /etc/rc.sysinit runs. After a bit more investigation, I found that /usr/lib/libcryptsetup.so.0 was installed by a weird rpm, and that it’s used in mount and umount. I couldn’t understand why the Fedora team would be so stupid as to disobey one of the first tenants of Unix: that the system should be able to boot without /usr. mount is installed by the util-linux rpm, and I had this strange util-linux…crypt.i386.rpm. That word “crypt” in the name was evidence that it wasn’t a normal rpm. And “rpm -qi” showed that it didn’t come from Fedora. That’s when I realized that when I’d installed the yum repository for “postgrey”, I hadn’t excluded all the other packages from that repository like I should have. And this had dragged in these crappy packages.

The first thing I did was try to “work around” the problem with files in /usr/lib by copying libcryptsetup.so.0 to /lib and booting again. But that failed, complaining about another file. So I went back to the rescue disk and tried the same work around with that file, but this time it complained about yet another missing file. I realized this could be a long process.

So now I had to get rid of the bogus versions of mount and the other files that depended on this stupid crypto stuff.

Next thing I did was go to the proper Fedora upgrade site with my laptop and download the latest real version of the util-linux rpm. Then I transferred it over to the Linux box using a USB pen drive (lovely useful things, aren’t they?). The rescue boot image mounts your Fedora file system under /mnt/sysimage, so I did a “chroot /mnt/sysimage” and tried to do an “rpm -Uvh –force” on that rpm, but it didn’t work. It gave some ioctl error. At this point I really wasn’t sure what to do. I contemplated trying to combine my / and /usr partitions into one big one and making that the boot point, but I didn’t relish the work it would take to stitch everything back together.

I started downloading the Fedora Core 5 DVD image as a backup plan, but Bittorrent was telling me it would take 6 hours.

As a desparation move, I tried booting a Knoppix 4.1 boot CD I had lying around. I manually mounted the disk partitions where they belonged under /mnt/hda6, and chrooted to it, and this time “rpm -Uvh –force” actually worked. Hooray!

I booted, and had a couple of minor hickups – nfsd had taken a socket that spamd wanted, so I was getting no spam filtering until I figured out who had the socket open and restarted everything that needed to be restarted. I also had to restart a ssh tunnel on my linode to get the mail flowing correct. I think everything is working – my blog is up, mail is flowing, news is flowing, I can NFS mount my music onto my laptop, and all seems right in the world.

But I’m up, I’m running, and I’m a little less sanguine about installing new repositories into yum. Time to go to bed.