Reimplementing Part 3

This is part 3 of my self-imposed project to reimplement the UI for an old website. You can read part 1 and part 2.

After some playing around with the “Step 2” interface, I realized that I really needed better error checking. I had error checking on the individual fields of the Geographic Panel, but no “overarching” check that all the fields were filled or all were unfilled. Plus I wanted the user to see the errors on the Geographic Panel even if you’ve closed the panel, and I also wanted to disable the “Start Generating” button until the errors are cleared. While I was at it, I added an error message if you don’t have any waypoint types selected. I’ve made those changes and pushed them to my GitHub project as the fourth commit and I’ve pushed it as a replacement to the previous mockup here.

I haven’t worked on this all week because of other calls on my time (work, job search, kayaking), so I haven’t made a start on the backend yet. But I’m thinking I’m going to be doing it in Python with Flask or Pyramid and SQLAlchemy. The original website is written in Perl, and frankly, I think the job market for Perl programmers is pretty rapidly shrinking and nobody is doing modern websites in Perl. But speed is of the essence, so that leaves Python or Java. The decision will probably come down to whichever of those two frameworks has better support for PostGIS.

Re-implementing Part 2

As described in my previous post, I gave myself a project to reimplement an old website using new technologies in order to prove to myself and to potential employers that I can do all the modern stuff.

The “Step 1” prototype was completed 15 days ago. I only wrote about it early today because I was debating with myself whether I should write up each step or do it all at the end. But during that time I’ve managed to get the static HTML page re-written as a React+Redux dynamic front end. I included the deficiencies I mentioned in the first post – the accordion panels open and close with a click anywhere on the title bar, and I’ve added “disclosure triangles” to make it more obvious that the title bars are meant to expand.

It doesn’t talk to a back end yet (it just simulates it using setTimeout), but it does so some nice interactive features

  • The title panes on each accordion panel update based on what you’ve chosen in the panel
  • It updates the countries, states and provinces in the “Country/State/Province” panel based on the latitude and longitude range you entered in the Geographic Area panel.
  • The state checkboxes don’t show up until you check the “US” checkbox in the country list, and similarly, the province checkboxes don’t show up until you check “CA” in the country list.
  • The “Start Generating” button is disabled unless you’ve selected at least one entry in the Airport Types, Navaid Types or Fixes in Charts panels.

Even taking into account the fact that I was learning as I went, I think the code looks ok. I tried to make some common base classes, but in many or most cases, I discovered that convolutions required to make it general weren’t worth it and I just ended up with similar but not identical code – case in point: the various “*Check” classes. I also ended up putting several classes for each accordion panel together in one file – I’m not sure if that’s considered best practices, but it seemed to make sense to me.

The mockup is here. This code is now the third commit on this project on GitHub.

So for a next step, I was planning to write a back end. The back end would have no actual HTML pages returned, just a bunch of AJAX calls and responses, mostly all returning JSON. So I was thinking Django, which is what I’ve been using for the last several years, would be too heavy for that. Similarly, the Java back ends I’m familiar with are also pretty heavy. On the other hand, the existing backend is written in Perl, and frankly, I don’t want to do anything in Perl anymore. I wonder if Flask and SQLAlchemy would be a good choice?

I’ll also have to figure out how to read and store cookies so we can store a session id just like the existing site does, but that’s probably not a big deal.

I also have to think of how the file generation will work. In the old code, I spawned off (fork/exec) a program to generate the file, and then had the web server process looking at the log file the generator task to see how far it had gotten. That made sense when the back end computer was busier and the generator task took 5-10 minutes – I didn’t want to tie up an Apache thread for that long, plus I could use a very primitive hand-rolled bit of Javascript to show progress. These days generating takes less than a minute, so I’m wondering if I couldn’t just do a return “Content-type: application/octet-stream” directly in the Apache thread. I’ll have to investigate.

A much more minor change I’m thinking of is in the country list putting the US and Canada at the top of the list instead of being strictly alphabetical because most of the users of the site are from the US or Canada and it would make it easier to find them.

Cosmetically, I reused the banner from the old site. It doesn’t really match the light and clean look of the rest of the site. I should probably make something more in keeping with the look I’m going for. Similarly, the favicon looks like a Palm Pilot which was fine when most people were using my data for Palm Pilots, but kind of wrong for GPX.

Someday, I’d like to add a simple map that can use to select the geographic area. Even better would be displaying or even picking countries, states, and provinces on the map. I’ve done some quick googling and I can’t find a good map library.

Also, if I could get the backend working fast enough, it would be cool to give a running count of how many waypoints the user is going to get in his file with the current parameters.

Update: See Step 3

Re-implementing an old web site in a modern style (Part 1)

So a few weeks ago I found out that my contract was going to be ending early due to a loss of funding. This is a great shame, not least because it gave me the opportunity to learn a bunch of new technologies, and also because I’ve really enjoyed working from home with minimal supervision. It’s been really helpful to my kayaking training to be able to go out and train during the afternoon and work in the evening after it gets too dark to paddle. It’s unfortunate timing because my contract ends at the end of April, and we’d already booked our cruise for two weeks towards the end of May.

But I’ve discovered a bunch of things since I’ve started looking for work:

  • Getting remote work is hard, even if you have 7 years of remote work.
  • Lots of potential employers want to see your work and your code, or worse, they want to see your GitHub profile. The problem with that is that most of my code is written for employers who don’t allow me to share the code I’ve written for them, and the websites I’ve worked on have been internal rather than facing the internet, so I can’t show them. Even worse, the only thing I’ve got on GitHub is a WordPress plugin that I forked for a project I did for the local kayak shop in exchange for a new boat. I used to contribute more to open source projects, but that was before the days of GitHub and mostly it involved emailed patch files. That was back in the days when I didn’t have so many hobbies and obligations.
  • While I’ve been doing a lot of JavaScript over the last 7 years, I haven’t used any of the cool frameworks. We’d actually made the decision to transition our project at my soon-to-be-ex job to Angular, but we never got around to it.
  • A whole bunch of companies are embracing coding exercises as an evaluation process. I don’t blame them – I’ve given interviews and I’ve received them, and they are an utterly shit way of evaluating people’s technical ability. But it is time-consuming if you have to do this for every company you apply to.

So I thought it would be useful if I could do a bit of a project that I could show to people, and put the code on GitHub, just like all the cool kids. And it should obviously use a JavaScript framework. And you’d think that I’d choose Angular, but there is a particular job here in Rochester that is currently top of my want list, and they use React, so I thought I’d use React.

So having made a decision on the front end technology, what I needed to do was pick a project. And the obvious candidate is my website. While I keep the data updated loading FAA and Eurocontrol data every 56 days, I haven’t updated the UI in many years, and it shows. I’ve got several data generators there, but most of them are for Palm Pilot programs, so they’re probably not worth updating. But the GPX generator at is still viable. So I’ll only reimplement that one.

The first step is to make a mockup of the new site. Actually, the first step was to watch a bunch of tutorial videos for React. Which start off teaching you about WebPack and ES6, and then React, and then React-Redux. I wasn’t 100% sure if I needed Redux, but it seemed like a good way to separate the data state code from the render code. But after watching the videos, it was time to write the mockup.

Obviously, since I have been using Bootstrap for the last several years, it would be easiest to make something modern and presentable using it. I just had to find a nice theme with a CDN. I settled on the “Paper” theme from Bootswatch, which has a nice simple clean look.

The obvious thing was to replace the multiple pages of the original GPX interface with a single page. The existing interface saves your last set of parameters in the database linked to a cookie and re-uses them next time you come back, and I’ve heard from users that they often just click through to the end and hit the action button without changing any of those parameters, so I thought it would be good to be able to display what they’ve loaded so they review to make sure that they don’t need to change anything and click the “Start Generating” button. So I decided on an accordion panel where the header for each type of filter shows what’s selected for that type and if you click on the header, you can modify it. Hard to describe, hopefully, easy to see in action.

So it took a few days, but the mockup is here. The code is the first commit on this project on GitHub.

After playing with it a bit, the first thing that leaps out at me is that the accordion panels should open when you click anywhere on the title bar, not just on the text. Also, Vicki suggested that the color of the accordion panel is a little blah and I should make it a nice light blue. You’ll see that in generation 2.

Update On thing I forgot to mention: While Bootstrap has support for responsive design, I made no attempt to make it work on mobile devices because generating and downloading large files isn’t something you’re likely to do on a phone or tablet. Probably the only change I’d have to make for mobile would be to use <div class=”row”> and <div class=”col-xs-12 col-md-4″> instead of tables, and possibly some tweaking of the padding or margin around checkboxes because they appear to overlap the label text a bit in Chrome and Safari on my iPad (although they don’t overlap in Chrome on my desktop when I choose “iPad mode”).

Also, as per a comment below, I’m also going to look at increasing the text size if I can do it without messing out up the look. The default font is 13px, which I’ve always thought was plenty big, but the Roboto font is a bit “thin”. I like it because it looks elegant, but maybe it doesn’t work at 13px? Bootswatch has some other clean themes, maybe I’ll have a look at some of them instead.

Update: You can read Part 2 and Part 3.

Moving on, moving up…

I was a little bit annoyed at work this week – they had me rush rush rush to finish something, and when I did they didn’t seem to have anything more for me to do. Ok, if I’m honest with myself, this new subproject they put me on has annoyed me because nobody seemed to want to communicate, and so maybe I wasn’t making the effort I should have to go find something to work on when they didn’t give me something. But still, I was mildly annoyed.

And then out of the blue, an on-line friend popped up on Facebook and said “hey, do you know any perl programmers looking for a job?” Well, I’ve done a fair amount of perl coding for my site as well as scripts here and there at Global Crossing and Kodak, although I’m not one what you’d call a “perlmonger” or “perl monk”. As a matter of fact, I’ve been seriously telling people on StackOverflow that if I weren’t so proficient in perl already and didn’t have so much working code written in perl, I would seriously be trying to forget everything I knew about it and learning python instead. So I asked him for more details. But he quoted me an hourly rate that was more than double what I’m currently making. I suddenly remembered that I’m a perl God.

The contract involves learning all there is to know about their business process, and all there is to know about an open source project called “Request Tracker” (aka RT), and seeing how much of their business process I can shoe-horn into RT (using its built in customization hooks and APIs) and how much would require custom coding of separate apps that maybe talk to RT for some parts, and doing the customizing of RT and the custom app coding.

In some ways this job looks like enormous fun and quite a growth experience, but “growth experience” also means “opportunity to fail” so I’m also somewhat scared of this. Not only do I have to become proficient in RT and in corners of the perl world that I’ve never touched before (Mason, anybody), but I’ve got to do my own requirements gathering, project plan, and every other aspect. I’ve got nobody else to blame if things don’t go right. But looking at the sunny side, if I do this right, I’ve got a major successful project on my resume and a happy customer to vouch for me for the next thing that comes along. It’s going to be great!

Anyway, long story short, I just turned in my notice at my current job. No more 3.5 hour return drives to Ithaca every week or two. (I just spent over $1000 on flights, hotels and rental car for a week down at their site for the project kick-off one week info dump, though.)

This time I think it was the cache…

As I wrote about in 2007 in articles and , back in 2004 I wrote a cache for part of the product I was working on at Kodak. In the first release to QA, I made sure that area of the code got tested thoroughly, and they found a bug, and fortunately I got it fixed before it went out to the customers. But to my chagrin, my boss and other people on the project got it in their heads that somehow any problem anywhere near that part of the product must be the fault of my cache, even though time and time again it was proven that there were no further bugs in that code for the following 3+ years.

Now flash forward to the product I’m working on now. We have a “go live to the very important customer” happening in just a few days, and we’re supposed to be in code semi-freeze. But the “Performance Project” just put their performance cache into the product, evidently without giving the local QA much chance to test it before it went to the customer’s QA. That seems just a little bit dangerous to me. But no matter, they assure me they’ve written tons of unit tests. So what could possibly go wrong?

Today the customer called up saying that they’re setting up a new client on the admin site, but every time they go to the “branding setup” for that new client, they see some other client’s branding setup. This branding consists of things like the client logo and some “terms and conditions” text and the like. Since they’ve got literally hundreds of QA people hitting this site, I naturally wondered if they weren’t seeing some interaction between multiple people messing with the setup. But after hours of poking around on their site, one of my peers and I (neither of us members of the “Performance Product”, I might add) are convinced it’s the performance cache. Evidently if you use one browser to look at one client’s branding, and then use a different browser to look at the branding of the client who hasn’t been setup yet, you see the branding from the client that you’d looked at in the first browser. Somehow the cache is reacting to the absence of information in the database for a client by pulling up information from some other client out of the cache. That’s not good.

Hopefully that will get fixed, and hopefully somebody will set up a test plan that actually tests what the cache does not just on a cache miss, but also on a database miss as well. And hopefully the important customer won’t think we’re all a bunch of idiots for not testing this properly.