Evidently the weekend’s activities weren’t as successful as originally thought. As well as this palm-to-armpit cast thanks to being rammed by a canoe full of idiot girl scouts, I also seem to have totally fucked up the light over the garage. As of yesterday, there is no power at all in the garage, not to the new light, nor to the inside light and plug socket. And the circuit breaker is not tripped. I even tried resetting it just to see if that would help. I swear there is a secret switch in the house for the power to the garage, and we’re switching it without realizing it.
I’ve spent the last couple of months doing an all-singing, all-dancing automatic upgrader for our customers sites. This process is designed to be totally hands-off – you stick the DVD in the drive and type “upgrade”, and at the end of the theatre day it will convert the main “cms” computer (one per site) and all the “cp” computers (one per projector) from Redhat 7.3 to CentOS 3.4, upgrade from version 3.3 to 3.6 of our software, and magically preserve all your settings and configuration. You should come in the next day to find everything ready for the day’s schedule.
For the very first one at a customer site, though, they sent out a technician to babysit it. Unfortunately they send a techician who’d never done or witnessed one of our many test upgrades in-house.
You probably guessed what happened – she saw the cms come up, didn’t realize that the cps start after the cms is done, and rebooted the cms at the worst possible time – right when all 18 cps were attempting PXE (network) boots and expecting the cms to be there to send them what they needed. And the cms doesn’t start dhcpd by default, so the cps have had nothing to talk to all night. And of course everybody is screaming for me when I got in!