Amtrak, B-Movies, Web Development, and other nonsense

Category: Web Development (Page 4 of 4)

HEWEBNE: outbound

This coming Monday I’m giving at talk at HighEdWeb New England about collaborative development in open source, focusing on liberal arts colleges. If this were one of my movie reviews that would be the “A plot.” The “B plot” is that I’m taking the train to the conference, and that unusually for me it’ll be 100% new mileage.

The conference is Mount Holyoke but I’m taking the Vermonter from New York up to Brattleboro. There are two reasons for this. The first is that my friend who’s picking me up lives closer to Brattleboro than Amherst. The second is that this train will be rerouted to the west bank of the Connecticut River in a year or two, and I want to ride the old route before that happens. Again, I’m that guy.

The ride in from Easton to New York was uneventful. Trans-Bridge Lines does a good job. Its buses are comfortable (for 90 minutes anyway) and the free wifi gets the job done. I-78 was remarkably empty. The only hiccup was finding New York Penn overrun with mouth breathing collegians dressed in green. Sigh. Amtrak Police finally showed up with a bullhorn and cleared them out. Yay!

Today’s Vermonter has five Amfleet cars: four reserved coaches and a cafe/business class car. At the front is an EMD AEM-7, one of Amtrak’s venerable “toasters” now in its fourth decade of service. It brought the train up from Washington and will pull it to New Haven, where we’ll swap it out for a GE P42DC “Genesis” diesel locomotive. We have to do this since there’s no electrification north of New Haven. Lunch today consisted of a turkey panini (good), hummus plus pretzel bits (okay), and iced tea. I’m pleased to finally ride over the Hell Gate Bridge but it really is more impressive from the park below looking up.

Early into New Haven and the power change starts at once which means no head-end power (HEP). I think of this as a staple of American railroading, though very few passenger trains do it now. The long-distance trains which travel south from New York (Silver Star, Silver Meteor, Crescent, Palmetto) change engines in Washington. The Vermonter does here in New Haven. The Pennsylvanian does in Philadelphia. I think some of the upstate New York trains do in Albany (switching from dual-mode electro-diesel to straight up diesels). Fascinating to think about the long-haul diesels which run, essentially uninterrupted, for fifty hours on the Western trains.

I hope I don’t offend anyone (too much) when I opine that Connecticut, at least what I can see of it, is ugly. Admittedly March is an unkind month for viewing the outdoors when there’s no snow on the ground.

We hit our dwell stops (New Haven and Springfield) with plenty of time to spare. Amtrak’s issuing a faster timetable on Monday, and I’d say that’s reasonable. I’ve arrived or departed from Springfield numerous times over the years but this is the first time I’ve headed east. We go as far as Palmer, and then perform a thankfully rare “backup” maneuver. There’s no northeast/southwest connecting track at Palmer, so we have to make what amounts to a J-turn in railroading–crossing the connecting track, stopping, throwing a switch (manually, no less), then reversing direction. The re-route I mentioned earlier will eliminate this step.

Here’s some very crude text art illustrating what we’re doing:

            C
            \_ 
             \\_________________
A_____________\_________________\(E)______B
               \
                \D

A-B is the CSX main between Springfield and Boston. C-D is the New England Central Railroad. The direct intersection between the two has no switch. We pull past toward B to the switch at (E), then backup on to the second track which links up at C. It’s horribly slow and inefficient. You leave Springfield at 3:15 PM. By 3:45 PM you’re just getting down with this nonsense and heading north.

Amtrak’s obviously embarrassed about it. There’s a nice long warning about it over the PA, emphasizing that this is normal and planned. I think the only constituency who enjoys this are the railfans who gather since it’s such a great photo-op.

I’m now on the stretch that Amtrak will leave in a year or two. This New England Central RR track is rough; some stretches are worse even than CSX around Buffalo, which has always been my “gold standard” for an unpleasant ride. It makes no sense to rehabilitate it of course when Massachusetts and the Feds are fixing up the new, more direct, route. I don’t disagree with the logic at all. Still, damn. There’s some talk of new service over this route to link Amherst with New London. I don’t see it getting done without serious federal money, and there are other more pressing passenger rail projects.

As I submit this we’re fifteen minutes out from Brattleboro and dead on schedule. No better way to travel.

Draining the swamp

It’s best to imagine WordPress’s plugin ecosystem as a swamp. Swamps are terrible. You don’t want to be there. You run a constant risk of disease and/or drowning. Anything that sinks into the swamp–it’s not coming back.

Belarus-Peat Mining near Rudzensk-Swamp-2

I’ve been debugging an odd problem on our WordPress installations involving categories. On some sites, posts which are in have multiple categories don’t display more than one category. That would be strange enough, but the category permalinks are coming out in the format SITE_URL/category/foo with the title baz, where foo is one category and baz a different category:

<a href="http://my.wordpress.site/category/category1">Category2</a>

Strange, seemingly non-deterministic behavior? The usual suspects would be database corruption or a theme bug. Yet neither seemed likely in this case. Database corruption usually isn’t so…predictable…and we quickly verified that this error was occurring in both our custom themes and stock TwentyTwelve. That would leave a core bug (unlikely with something so fundamental, but still possible) or a bad plugin.

After several patient hours of tracing execution I’d narrowed the problem to the function WordPress uses when building up the category list: the_category(). The category link string was correct before going in for formatting and it came out mangled. WordPress uses filters to allow plugins to “hook in” and modify output. A search of our plugin code revealed the culprit: Remove Title Attributes.

WordPress adds title attributes to links by default, a behavior which apparently annoys the hell of many people, including at least one person at Lafayette in the past. This plugin simply removes them with a regex (I would be remiss if I didn’t link to the famous StackExchange thread about why you should never, ever, parse HTML with a regex). To accomplish this the plugin added a filter which washed the generated category code through its regex.

Unfortunately, the regex is improperly written. In jargon, it’s greedy. This is the expression evaluated:

` title='(.+)'`

If you pass a string with multiple URL fragments it’s going to match beginning on the title tag of the first URL and ending on the end tag of the last URL. A more properly focused regex would be this:

` title=\"([^"]*)\"`

That’s it. Mystery solved.

Unresolved, however, is the larger problem with the WordPress plugin ecosystem. This plugin was added to the plugin repository in August 2009. It has never been updated since. It has been broken from the very beginning. The author has disappeared. The support forums are moribund. There’s no github repository for me to fork, should I want to continue support, since WordPress in its infinite wisdom uses SVN for everything. Spend some time Googling and you’ll find people talking up this plugin, never realizing the problems inherent. It’s still being downloaded. This may be inexperience (I’m a Moodle veteran and new to WordPress) but I don’t see a good way to get the word out that this plugin has a serious bug. If WordPress allowed you to usurp a plugin then I could push out an updated version so at least you’d get notified in your Dashboard. All I can do is leave a review indicating that it’s broken in 3.5.1 (for this specific use case) and link back to this post.

Not that it matters overmuch in this case since we’re likely to deep-six it here, but the situation feels inadequate. There’s got to be a way to do better.

How to Screw Your Developers

I’m a Moodle developer by trade, with some administration duties thrown in for laughs. I see to the care and feeding of Lafayette College‘s Moodle sites. In addition, I handle some development and code management duties for the Collaborative Liberal Arts Moodle Project. All this is a long way of saying I push a lot of code around in a way that requires me to work with dozens (hundreds?) of developers whom I’ve never met. This means I have to document my work and develop against actual APIs. Further, working for an institution with thousands of students, staff, and faculty means that all our maintenances are published and (save emergencies) occur outside business hours. I can’t just shoot from the hip.

Gravatar is an online service which allows you to upload an avatar which will then be populated to other services. You don’t realize how many places use Gravatar until you upload an image and suddenly see yourself all over the web. It’s pretty cool. Gravatar provides a simple API so that you can add this functionality to your own website via web services. Moodle added support for Gravatars in the 2.2 release which dropped in December 2011. We turned it on at Lafayette since it’s easier than making thousands of people upload files.

So far, so good, right? Enter chaos.

Thursday morning we started seeing hundreds of broken user profile images on our Moodle instances. I knew we hadn’t touched anything. Stranger–we saw lots of Gravatars still appearing so it couldn’t be a recurrence of last Spring’s incident when Gravatar was unavailable. Cue an hour of tedious debugging until arriving at the conclusion that default image support had stopped working.

Gravatar’s API allows you specify a default image, to be used if the user doesn’t have a Gravatar. You may pass an image you host yourself, or use one of their stock images. Moodle uses the default headshot image, as defined by whatever theme you’re using. Moodle has a complex mechanism for determining which images to serve up, based on your current theme and context and whether caching is active. All image requests pass through theme/image.php, which acts as a clearinghouse for determining what image should be retrieved. This system is file extension agnostic for better compatibility and flexibility.

One of the benefits of collaborative development is that you’re usually not the first person to encounter a problem. I found that my issue had just been reported to the Moodle tracker by another developer who had come to the same conclusion. Here’s the issue. A Moodle request to Gravatar would look like this:

http://www.gravatar.com/avatar/6df0d771964ef4d1f10cab72149d8460?s=100&d=http://moodle.org/theme/image.php/moodleofficial/core/1348555103/u/f1

Let’s break that down. The URL contains the location of Gravatar’s image serving mechanism, an md5 hash of the user’s email address, a requested image size (s=100), and the default image url (d=<url>). If you try to use this now you get an error page which says “The type of image you are trying to process is not allowed.” Why?

I still don’t have the full picture but it appears that Gravatar made a change–unannounced and undocumented–to require file extensions in the default image. Several developers submitted support requests to Gravatar trying to confirm that this was the case. After several days(!) I got an answer back from one of Gravatar’s “Happiness Engineers” (blech). I’ve quoted the relevant part below:

We have recently implemented some changes that wouldn’t change our current documentation so much as an enforce that default images must actually be images. They should also be publicly accessible.

I noticed in the case you mentioned that you’re using a php file to serve up the image. For security reasons, that will no longer work.

This is nonsense. My own testing shows that they’re just looking at the extension, which is poor man’s security at best. Extensions can be faked. If they were serious about security they would eschew this extension malarky and just check the mimetype.

For me the greater issue is that this change happened in the dead of night, with no warning. Many hours of developer and administration time will now be consumed working around a change, leaving aside for a moment that the change is bonkers. What the support person told me perfectly explains the problem, yet as of writing the online documentation doesn’t reflect it.

Gravatar itself acknowledges on the same documentation page, when discussing displaying a Gravatar image:

If you require a file-type extension (some places do) then you may also add an (optional) .jpg extension to that URL:

I find that sentiment at odds with whatever process determined the change we’ve been discussing in this post.

So, that’s 760 words explaining the issue I spent far too much of my Thursday debugging. What does it all mean? That Gravatar is a neat, fun service which has no respect for the people who develop against its APIs. That may sound harsh but I maintain that if you have respect for your development community you don’t make changes like that without some kind of announcement or warning. That the change is technically deficient is also a cause for concern.

There’s a Moodle tracker item for providing support for alternative avatar services. It hasn’t gotten a lot of traction so far, but I intend to make a push there. Behavior like this makes me angry.

UPDATE: I’m amused, months later, to discover this blog post from Gravatar discussing the change, including the file extension requirement. It’s dated October 7, or over a week after the change and this blog post.

Newer posts »