Terrestrial Navigation

Amtrak, B-Movies, Web Development, and other nonsense

Page 3 of 7

A typical card catalog.

How I file things

March 22, 2017 / / 0 Comments

At the end of my blog post about Hammerspoon and case conversion I had this aside: ” …it works well for storing electronic articles and books.” I’m going to expand a little on that and how it fits into my scheme for organizing information. Hat-tip to nateofnine for prodding me into sharing this.

The Library of Zotero

Spreadsheet of citations

A typical Zotero collection. This one contains everything tagged for the GE E60 electric locomotive. Exciting, right?

I use Zotero to keep track of information, mostly related to railroading history. I wrote a post last year about developing a custom translator for importing articles. As of writing I have 1193 1200 items indexed in Zotero: books, chapters, journal and newspaper articles, blog posts, doctoral theses, maps, etc.

Zotero lets you attach things to an entry such as notes or tags. You can also link an external resource. This might be a website, or a link to the a local copy of the document if one exists. Zotero lets you sync content to its cloud, but the free tier is limited to 300 MB. If you’re storing article PDFs you’ll exceed that pretty quickly. Leaving aside HTML snapshots, I have 249 articles consuming ~2.5 GB of space.

Let the data flow

Electronic documents come from all over the place—interlibrary loan, online databases, websites, scans I’ve done myself of physical media that I possess but need to store. The only common factor that is that they become a PDF and I need to organize them.

I start by having a folder structure with a top-level folder named Articles. Beneath that I organize by publication, with definite articles removed to permit natural sorting (thus The New York Times becomes New York Times). Within each publication I adopt a naming convention of DATE-NAME. Date is of the format YYYY, YYYYMM, or YYYYMMDD, depending on the granularity of the publication. A monthly journal, for example, will go no further than YYYYMM. The purpose of this is to ensure chronological sorting when looking at the articles outside of Zotero.

For the NAME, I fall back on my Hammerspoon case conversion module. I’ve already created an entry for the item in Zotero, with its full title. I throw that title into the case conversion, get the slug, and append it to the date. This gives me a filename that sorts by date, is human-readable if need be, and is easy to manipulate from the command line. For example, J. David Ingles’ article in Trains magazine from the May 1979 issue entitled “How super are the Superliners?” becomes 197905-how-super-are-the-superliners.

Putting it all together

The file created, I drop it into the appropriate publication folder, then use the “Add Attachment > Attach Link to File” option in Zotero to associate it with the index. Now, when I double-click on the item in Zotero, it’ll open the file for me to read. The Articles directory tree lives on my Nextcloud which means that (a) there’s redundancy in case something happens to my laptop and (b) I can read the articles on my phone, without needing to have Zotero installed.

 

Featured image by Sue Peacock (card catalogs) [CC BY-SA 2.0], via Wikimedia Commons

Writing LDAP unit tests for a Moodle plugin

March 8, 2017 / / 0 Comments

In 2016 Lafayette College began maintaining the LDAP Syncing Scripts (local_ldap) plugin after the tragic death of the previous maintainer, Patrick Pollet.

I didn’t know Patrick but he had a strong reputation in the Moodle community. I’m pleased to say that we made few substantive changes to his code. Most of the changes were simple updates, such as migrating the command-line/cron scripts to Moodle’s task infrastructure, and various nit-picky code standards issues which didn’t affect functionality.

PHPUnit

The biggest lift was implementing PHPUnit test coverage for the plugin. I started out with the following requirements:

  • Fully-scripted setup for OpenLDAP, so that the tests can run inside a continuous integration environment
  • Test coverage for group synchronization
  • Test coverage for attribute synchronization

I started this project by building an OpenLDAP environment inside Moodle Hat, the Vagrant development profile I maintain. Implementing a configuration in Puppet is good practice for wrestling with Travis.

Starting from scratch with OpenLDAP (every time!) presents certain challenges that you don’t encounter in a mature environment. A few I encountered:

  1. When you bootstrap OpenLDAP it has a completely empty schema. PHP’s ldap libraries can’t talk to it in that state. You have to populate it with some data, even if it’s completely arbitrary.
  2. Selection of backend databases matter. LDIF is the quick and easy path, but it doesn’t support pagination and Moodle will break in obscure ways. I chose bdb because it’s available in most repositories and it worked.
  3. When you’re setting a generic testing password in your slapd.conf you can just dump in rootpw SomeArbitraryPlaintextPassword and it’ll work. Don’t run in production! Or, really, anywhere that has state.

Once I’d worked through those issues Christian Weiske’s invaluable blog post provided everything I needed for implementing on Travis.

Travis

LDAP Syncing Scripts leverages Moodlerooms’ excellent moodle-plugin-ci plugin for travis-ci integration, with a few tweaks. The full travis-ci.yml file is visible on the GitHub repository; let me walk through a few things.

We need the slapd and ldap-utils packages installed. To use Moodle’s built-in LDAP PHPUnit testing we need to define the location of the test server in the config file:

define("TEST_AUTH_LDAP_HOST_URL", "ldap://localhost:3389");
define("TEST_AUTH_LDAP_BIND_DN", "cn=admin,dc=example,dc=com");
define("TEST_AUTH_LDAP_BIND_PW", "password");
define("TEST_AUTH_LDAP_DOMAIN", "dc=example,dc=com");

We need to create an INI file to force PHP (in travis) to load the ldap extension, and a slapd.conf file to define how our OpenLDAP enviroment will function. The schema settings need to match what you added to Moodle’s config.php. We start slapd and then, as the final step, import our default data. This data isn’t used, but it gets around the problem of an empty schema. Note that while this data is stored as an ldif file for readability purpose, the backend is bdb.

Tests

The actual tests I derived from the tests for Moodle’s auth_ldap plugin. The code is long but self-documenting. There are no particular gotchas, though I found it helpful to extend auth_ldap_plugin_testcase instead of starting fresh.

Remembering the Hoosier State

March 1, 2017 / / 0 Comments

Today Amtrak resumes full operation of the Hoosier State, ending an 18-month experiment in which Iowa Pacific proved rolling stock and on-board services. I had an opportunity to take this unusual train in June 2016 while on a business trip to Indianapolis. What follows are my notes on the experience.

Aboard the Hoosier State

We’ve just wrapped up the CLAMP’s Hack/Doc Fest at Butler University in Indianapolis, Indiana. You can read Ken Newquist’s daily updates to see how the conference went, including all the gory details on the updated annotation in Moodle 3.1. Short version: it needs love. I’d like to talk about a most unusual aspect of the conference: Amtrak and Iowa Pacific’s Hoosier State, which runs between Chicago and Indianapolis.

Background

Horizon coaches on the Lincoln Service in 2009. These are a common sight on the Hoosier State. Photo by Jeramey Jannene from Milwaukee, WI, United States of America (Train Boarding) [CC BY 2.0], via Wikimedia Commons

Train service between Chicago and Indianapolis has a tortured history in the Amtrak era. Amtrak does not, except in a few cases, own its own track, and must therefore rely on access to rails owned by private freight companies. The most direct routes to Indianapolis were abandoned or downgraded in the 1970s and 1980s as freight traffic moved elsewhere. Amtrak trains which serve Indiana have repeatedly moved to less desirable routes in order to maintain service.

By the 2010s the Hoosier State operated with a couple coaches and no food service on a slow, bumpy, five-hour journey. The train ran quad-weekly; on the other days the long-distance Cardinal ran over the route.

Indiana takes control

The federal Passenger Rail Investment and Improvement Act 2008 (PRIIA 2008) required states to provide funding for short-haul trains operating within their borders beginning in 2013. Indiana cobbled together an agreement involving state and local money for 2013, but only after prolonged debate. It was clear that Indiana wasn’t satisfied with its level of service and sought alternatives.

For 2014 Indiana decided to make a change. The discussions, negotiations, false starts, and accusations would take forever to recount here and make my head hurt. The end result is that Indiana brought in a private company, Iowa Pacific, to provide rolling stock, marketing, and on-board service personnel. Amtrak personnel operate the train itself; Amtrak sells tickets and the train remains part of Amtrak’s system. The new train began running on August 2, 2015 and after some hiccups it seems to be a success. According to a recent press release, ticket revenue is up, customer satisfaction is up, and delays are down.

Hack/Doc at Butler gave me a perfect opportunity to inspect this strange new service.

First impressions

Iowa Pacific coaches on the Hoosier State. Photo by David Wilson from Oak Park, Illinois, USA (20170205 13 Hoosier State @ Rensselaer, Indiana) [CC BY 2.0], via Wikimedia Commons

I haven’t been on this route in six years. In 2010, after the Hack/Doc at DePauw, I caught the Cardinal from Indianapolis back up to Chicago. I remember an unpleasant station, a crowded cafe car, and a slow slog through rail yards south of Chicago. The brioche French toast was pretty good.

We board from the Metropolitan Lounge in Chicago, and you can’t miss the Hoosier State. Most of Amtrak’s equipment is stainless steel with red, white, and blue striping. Iowa Pacific has painted the cars for this service in classic black-and-orange Illinois Central Railroad livery. The cars themselves are unusual: an old Santa Fe Big Dome, the Summit View, and old Budd coaches with large picture windows.

Inside I’m impressed by the leg room. We’re in the Du Quoin, a 44-seat leg-rest coach. The leg room is comparable a long-distance Amfleet coach, maybe even better. There’s a proper 120V wall outlet and folding tray table. The picture windows have blinds which you can pull down. The upholstery is sort I’d expect to find on a couch or easy chair in my grandfather’s living room.

The bathroom is most unusual. It’s off to the side, about the size of a bedroom. You enter and there’s a couch, and two sinks. The toilet and urinal are in a separate room which locks. Ken likened it to a receiving room.

Crawling out of Chicago

The creature comforts on Iowa Pacific can’t do much about the route. You really do feel as though you’re sneaking out of Chicago. It’s 28 miles from Chicago to the first station stop in Dyer, Indiana. It’s timetabled for 90 minutes. A car could do it half the time, barring shenanigans on the Dan Ryan Expressway.

After a brief run down the Chicago Line and Metra track we hit the Belt Railway of Chicago at 75th Street and slow to a crawl. I took these notes at the time:

​ “Pretty good run here, but we’re doomed once we hit the Belt. There’s no escape from the BRC. Yep, we’re on the Belt and everything is slow and grinding. No one around me shares my concerns about the Belt.”

We use the Belt to reach the Union Pacific Railroad’s Villa Park Subdivision. This is a slow crawl through the south side, and it reaches its nadir at Riverdale. We have to cross the Little Calumet River, but there’s a freight train ahead of us which has to be recrewed and we need to cross a major interlocking. Having done all that, we will crawl past Dolton Yard.

We hit Dyer a little early at 6:50 PM. I can’t emphasize how frustrating the stretch on Union Pacific is.

Dinner time

The dining area on the lower level of the dome car. Photo by David Wilson from Oak Park, Illinois, USA (20170205 09 on board Hoosier State) [CC BY 2.0], via Wikimedia Commons

Business class passengers get meals as part of their booking (and sit in the dome!), but the dining car is open to everyone. We’re summoned by destination, with the Indianapolis passengers going last. As with standard Amtrak trains Iowa Pacific practices community seating. This is usually a high point of the trip. You never know who you’re going to be with.

As we hit Dyer I find myself seated in the lower level of the Summit View with my boss and a newlywed couple from Marion, Ohio. Their kids are down in Texas so they honeymooned in Chicago for the weekend. I had the pork chop, asparagus, potatoes, and a side salad. IPA to drink. Overall it was pretty good. I found the food service a little more competent than average Amtrak. I think they were a little overwhelmed—too many people summoned at once. No real complaint though.

What is to be done?

We made good time all the way to Indianapolis. The coaches rode well and we didn’t encounter many problems once we got on to CSX in Indiana. The A/C wasn’t working in our coach but it didn’t bother me much. We arrived at our destination before midnight and headed off to our hotel while the Hoosier State headed for the yard.

Fast-forward to March 2017. I appreciated the improvements in on-board service but wondered whether Iowa Pacific could really be making a go of it. The answer is that they couldn’t, although that may be more due to the parent company’s problems than the Hoosier State itself. Certainly they couldn’t have done it without Amtrak’s incremental access rights and logistical support.

What’s next? Amtrak probably puts Horizon coaches back on the Hoosier State. With everything else that’s going on a quad-weekly train from Chicago to Indianapolis isn’t high on their list of priorities. Iowa Pacific could afford to give this train individual attention; Amtrak can’t. The real question is this: what will Indiana do next? They don’t want to kill the train but they don’t like the level of service Amtrak gives them.

What needs to happen, but won’t, is finding a better route out of Chicago. That means real money; tens if not hundreds of millions of dollars.

Featured image by David Wilson from Oak Park, Illinois [CC BY 2.0], via Wikimedia Commons

The platforms at Toledo, OH. Photo by Prasenberg.

The Toledo Option

February 22, 2017 / / 0 Comments

Or, I reconsider the Lake Shore Limited yet again, and find a use for it.

A frequent complaint about Amtrak service is that “you can’t get there from here,” and it’s a fair criticism. As a Michigan expatriate living on the East Coast I’m sensitive to these limitations. Michigan itself has comparatively good service: three Wolverines to Detroit/Pontiac, the Blue Water to Lansing and Port Huron, and the Pere Marquette to Grand Rapids. Unfortunately for me all three services pivot on Chicago. Coming from the East Coast, I’m facing hours of layovers and backtracking.

As an alternative, Amtrak offers a Thruway Motorcoach connection at Toledo to various destinations in Michigan. Thruway Motorcoachs are contracted buses which you can book with trains to help get you closer to your final destination. I’d resisted this option for years because of the timings in Toledo and general uncertainty about the whole enterprise. After a positive experience with an Amtrak bus in Florida in 2014 and a growing desire to avoid driving on I-80, I decided to take the plunge.

The trains

The Lake Shore Limited at Croton-Harmon. Photo taken by Adam E. Moreira, [CC BY-SA 3.0 (http://creativecommons.org/licenses/by-sa/3.0)], via Wikimedia Commons

The Lake Shore Limited at Croton-Harmon. Photo taken by Adam E. Moreira, [CC BY-SA 3.0], via Wikimedia Commons

Amtrak has two trains serving Toledo, the Capitol Limited and the Lake Shore Limited. They are both long-distance trains running between Chicago and the East Coast; neither serves Toledo in daylight. The Capitol Limited runs southeast to Pittsburgh and Washington, D.C.; the Lake Shore Limited follows Lake Erie and serves New York City by way of Buffalo and Albany. I’ve previously written several posts about the Lake Shore Limited but I do not come here to bury it (again).

The bus

Amtrak contracts with local bus operators to provide bus connections. Trinity Transportation handles the bus from Toledo. Westbound, the bus is scheduled to depart Toledo at 6:30 AM, or thirty minutes after the scheduled arrival of the Lake Shore Limited and ninety minutes after the Capitol Limited. It will be held, barring major disruptions, for both trains. Eastbound, the bus arrives in Toledo at 10:35 PM, or an hour before the scheduled departure of the Capitol Limited and four hours before the Lake Shore Limited.

The bus itself is of the modern intercity variety, with comfortable seats, a restroom, and free Wi-Fi. When I rode the bus it was close to full between Toledo and Detroit but half the people got on (or off) in Detroit.

The station

The platforms at Toledo, OH. Photo by Prasenberg.

The platforms at Toledo, OH. Photo by Prasenberg.

Toledo is an intermodal facility, serving Amtrak, intercity buses, and local buses. It’s expanded from a New York Central Railroad station built in the 1950s. The station is open during the wee hours of the morning when Amtrak’s long-distance trains come through. It’s well-lit and there are comfortable chairs to sit on. Amenities include restrooms, Pepsi machines, a snack machine, a coffee vending machine. There’s also a small Subway sandwich shop which is open until midnight.

Where you can go

The bus serves five destinations in Michigan:

  • Detroit: the Amtrak station in the New Center neighborhood near Wayne State University.
  • Dearborn: the new Amtrak and intermodal station.
  • Ann Arbor: the Amtrak station along the river.
  • Jackson: the Amtrak station in downtown Jackson.
  • East Lansing: the new Amtrak and intermodal station off Troubridge.

I’ve done Ann Arbor and East Lansing; in both cases I needed to rent a car to complete my journey. There isn’t a car rental location near either station. In Ann Arbor there were taxis at the station; in East Lansing I needed to call one. The East Lansing station closes at 6 PM, an hour before the scheduled departure of the bus, but there’s an enclosed waiting area which stays open and which has not-entirely-uncomfortable seating.

Game planning

This is the itinerary I employed over the Christmas holidays:

Outbound

  • 3:40 PM (Day 1): Depart New York on the Lake Shore Limited
  • 5:55 AM (Day 2): Arrive in Toledo
  • 6:30 AM: Depart Toledo on the bus
  • 10:05 AM: Arrive in East Lansing

Inbound

  • 7:00 PM (Day 1): Depart East Lansing on the bus
  • 10:35 PM: Arrive in Toledo
  • 11:49 PM: Depart Toledo on the Capitol Limited
  • 1:05 PM (Day 2): Arrive in Washington, D.C.
  • 3:05 PM: Depart Washington, D.C. on the Northeast Regional
  • 6:30 PM: Arrive in New York

The choice of the Capitol Limited on the return eliminates an extra three hours layover in Toledo while not materially altering the arrival time in New York. Another option is to change to the Pennsylvanian in Pittsburgh but that makes for an early morning after a late night.

The big consideration here is meals. Unless the Lake Shore Limited is very late you’re missing breakfast westbound, and you’re definitely missing dinner eastbound. Assuming a big dinner the night before, missing breakfast is tolerable, but I suggest bringing along some granola bars or some such. Eastbound, the Subway at the Toledo station is a godsend.

Alternatives

A big theme in my travel writing and planning is resiliency: I want above all to maximize my options while controlling costs. I considered bypassing Toledo and the bus altogether in favor of South Bend, Indiana. South Bend is roughly the same driving distance from the places I would drive to in Michigan. With South Bend I had to weigh the following considerations:

  1. South Bend is 2 1/2 hours west of Toledo. Westbound that’s more time to sleep and gets you breakfast on board, but also adds that to your final ETA. Eastbound you still miss dinner and have to meet the train that much earlier.
  2. How do rental car prices at South Bend compare to the rental car prices in Michigan? What are the hours of the facility?
  3. How is taxi service in South Bend? How does Uber/Lyft availability compare to Ann Arbor or Lansing?
  4. If I encounter problems in South Bend, what are my fallback options? Is there a bus? Do I know someone who can pick me up?

These same factors applied to all the Michigan locations. I selected East Lansing because of rental car price, timing, and proximity to my final destination.

The verdict

This works as a travel option between the East Coast and mid-Michigan. Door to door it takes about the same amount of time as driving the whole way with an overnight stay, and can be price-competitive under the right conditions. It also eliminates someone sitting behind the wheel for 630 miles, and that has to be worth something.

Hammering Cases

February 15, 2017 / / 0 Comments

Last December I went down to Philadelphia for WordCamp US 2016. Met some great people, heard some great talks, overall had a good time. Holding the after party at the Academy of Natural Sciences was a genius move.

Sitting at Lisa Yoder’s talk on Version Control Your Life: Alternate Uses For Git inspired me to try taking notes in Markdown (versioned in git) instead of Evernote. I’m trying to move away from Evernote anyway and it made perfect sense. I’m always working on the command line; I always have Typora open as scratch-space.

I ran into an immediate (and silly) snag. All the WCUS sessions are titled “This Is The Name Of My Awesome Talk”. That’s a bad filename if you’re working on the command line. Ideally I want my notes on that talk to be called “this-is-the-name-of-my-awesome-talk.md”. Manually typing all that is boring, and I’m lazy. Better way? Better way.

Hammering markdown

Separately, I’d been playing around with Hammerspoon the last few days. Hammerpsoon is an automation engine for OS X. You can write Lua modules to perform various tasks, triggered by system events or manual invocation. It seemed cool and all, but I hadn’t found a concrete use case. Sitting in that talk, I had an idea—use Hammerspoon to convert arbitrary text (like a talk title) to an URL slug, which I could then use as a filename.

The actual module is pretty short; 38 lines including comments. Aside from some Wikipedia modules I don’t really have exposure to Lua, and Hammerspoon itself was terra incognita. Let’s step through this:

hs.hotkey.bind({"cmd", "alt", "ctrl"}, "T", function()

hs is the primary Hammerspoon object. I’m binding ⌃⌥⌘T; pressing that combination will activate the code within the module.

    local current = hs.application.frontmostApplication()

Brings the application forward so that we can send events to it.

    local chooser = hs.chooser.new(function(chosen)
        current:activate()
        hs.pasteboard.setContents(chosen.text)
    end)

Creates a chooser window and tells it to send the result of that window to the system clipboard. Now, here’s the actual callback:

    chooser:queryChangedCallback(function(string)
        local choices = {
            {
                ["text"] = string:lower(string),
                ["subText"] = "All lower case"
            },
            {
                ["text"] = string:upper(string),
                ["subText"] = "All upper case"
            },
            {
                ["text"] = string.gsub(" "..string, "%W%l", string.upper):sub(2),
                ["subText"] = "Capitalized case"
            },
            {
                ["text"] = string.gsub(string.gsub(string:lower(string),"[^ A-Za-z0-9]",""),"[ ]+","-"),
                ["subText"] = "Post slug"
            }
        }
        chooser:choices(choices)
    end)

This is where we actually create the chooser window and populate it with options. This is mostly string math. Hat tip to n1xx1 on stackoverflow for the capitalized case logic and Draco Blue for the post slug.

We’re almost done!

    chooser:searchSubText(false)

Tell the chooser to not search the sub text. Frankly I’m not sure what it does, but I saw it done in another module.

    chooser:show()

Finally, display the chooser window.

Putting it all together

With the module installed, I’m a keyboard combination and a copy-paste sequence away from a post-friendly slug for any bit of arbitrary text:

I finished up the module during the second round of talks. At the time I was half-worried that the project violated the XKCD rule, but I’ve found myself using it more and more over the last few months. Beyond note-taking, it works well for storing electronic articles and books. Besides, I had a good time doing it and learned something new.

What I’m reading

February 13, 2017 / / 0 Comments
  • John le Carré, The Secret Pilgrim. A semi-sequel to The Russia House, consisting of the reminisces of Ned at the end of his career. Fascinating and absorbing.
  • John R. Schindler, Fall of the Double Eagle: The Battle for Galicia and the Demise of Austria-Hungary. An account of the Battle of Galicia, the opening collision between Russia and Austria-Hungary in World War I. It’s always refreshing to read an account of Austria-Hungary which doesn’t place it in the shadow of the German Empire. I’m familiar with the subject but Schindler’s treatment of the ethnic issues in the military is fresh.
  • Tim Hadley, Military Diplomacy in the Dual Alliance: German Military Attaché Reporting from Vienna, 1879–1914. I heard Hadley give a talk on this subject at the German Studies Association conference in Pittsburgh way back in 2006. The premise is that there was a disconnect between the reporting of German military attachés in Vienna and German strategic planning. This book is the outgrowth of that talk, and I can’t wait to read it.

Nextcloud on Pi

February 11, 2017 / / 0 Comments

Following on with running Plex on a Raspberry Pi, I decided to play around with Nextcloud. This is very much a work in progress, with all my failures and blind alleys lovingly detailed. Note: some guides refer to both ownCloud and Nextcloud almost interchangeably. Nextcloud is a fork of ownCloud; technologically they’re very close. The Nextcloud desktop client is a themed fork of ownCloud’s and they’re compatible with the other’s servers.

Web application

At its core Nextcloud is a PHP application; setting those up isn’t complicated and there’s a good guide for doing so on a Raspberry Pi. I broke with it and opted for PHP 7 over for performance reasons. I used part of Andy Miller’s guide for that (ignoring the Nginx stuff) but I found I needed more PHP modules:

sudo apt-get install -t stretch php7.0 php7.0-gd php7.0-sqlite3 php7.0-curl php7.0-opcache php7.0-zip php7.0-xml php7.0-mbstring

Data storage

I went down a number of blind alleys with the backend. If you’re planning to have the bulk of your storage live external to the Pi you shoud still keep Nextcloud’s data directory local. The first thing I tried doing was simply putting it on an NFS share as I’d done with Plex. This was a bad idea and didn’t work. Nextcloud supports a concept of external storage; users can choose to add Samba shares, Google Drive folders, etc. That’s the proper way to attack the issue.

I tried Samba/CIFS first. Mounting a share from the Synology NAS worked fine but after a successful initial sync I encountered an error in which I was notified every couple minutes about remote changes and prompted to resolve them, although no such changes had taken place. I ensured that the clocks were synced between my laptop, the Pi, and the NAS, but didn’t solve it. I think the root cause is this issue in the ownCloud client; it’s solved in master but not in a compiled release. I encountered environment problems trying to compile the client and decided to try a different method.

Happily, I encountered no problems setting up WebDAV. As with NFS, you do have to enable it on the Synology NAS, but once you’ve done that you can just add it as external storage. I found that I needed to re-do the sync with the client after changing external storage methods, even though they had the same directory structure. I’m still getting some spurious notifications from the Nextcloud client but they’re infrequent, don’t require action, and don’t steal focus.

External access

I did all this as a proof-of-concept with HTTP access only and a local IP. To make this truly useful you need to be able to access your files from offsite. To make that practical and secure you need a domain name and an SSL certificate.

I registered a domain from hover; a friend recommended them and they seemed reasonably priced. I pointed an A-record at my current IP, which is somewhat static. I didn’t bother with any dynamic DNS solutions; I can accept the 24 hours it takes for the record to propagate when my IP eventually changes.

The Pi is behind two NATs: my cable modem and my wireless router. As with Plex I set 443 to forward from the outside world to the Pi. I think port 80 is blocked by my provider and I’m not offering anything on 80 anyway.

For the SSL certificate I went with Let’s Encrypt. I’ve used them with other small projects. The instructions for Certbot on Debian 8 (jessie) mostly worked; I found that I needed to import two repository keys. Once I did that I was able to run Certbot which handled all the SSL configuration on the host.

Cleaning up

With an HTTPS-fronted domain live I added that domain to the list of trusted domains on the server, then swapped out the client configuration on my laptop. Nextcloud appeared to recognize that it was dealing with the same server and didn’t need re-sync anything.

I’ve been running in this configuration for close to a week and it’s been smooth sailing.

Featured image by fir0002 | flagstaffotos.com.au [GFDL 1.2], via Wikimedia Commons

Raspberry Pi 3 B. By Sven.petersen (Own work) [CC BY-SA 4.0 (http://creativecommons.org/licenses/by-sa/4.0)], via Wikimedia Commons

Raspberry Plex

February 6, 2017 / / 0 Comments

I’ve been running Plex on a Synology NAS for the last year, with a Google Chromecast handling display. It’s worked well enough, but my DS 216 struggles at times to keep up with the movie (I’m not transcoding; that would never work). I decided to get into the Raspberry Pi game and build a Plex server on it, while keeping my media on the NAS. These are my notes on setup.

Image

I’m comfortable in Debian so I started with a stock Raspbian image on a 32 GB SD card. I used Etcher on OSX for this and didn’t encounter problems. I’d say it took 8-10 minutes.

Keyboard

Raspberry Pi’s come with the keyboard set to the UK, which works for most things but can trip you up. I changed it to Generic 105-key PC and selected a US keyboard layout.

DHCP

I created a reservation on my router for the Pi’s hardware address so that the IP would be consistent. The procedure for doing so will vary depending on your home networking environment. You’ll need to know the hardware address of the interface you’re using (wired or wireless).

I opted for the wired interface. As an additional step, I changed the configuration in /etc/network/interfaces from iface eth0 inet manual to iface eth0 inet dhcp. This is a little misleading; the interface still gets an IP address via DHCP when set to manual, but it happens later in the boot process. This causes problems with mouting an NFS share.

SSH

SSH access is disabled by default; I enabled it and setup key-based authentication as an extra security measure. You can disable password authentication by setting PasswordAuthentication to no in /etc/ssh/sshd_config and restarting ssh.

NFS

That was all preliminaries for the interesting part—making my existing media libraries available to the Pi. On my Synology NAS I enabled NFS as a service and then enabled NFS sharing for my media library.

On the Pi I installed and enabled the rpcbind service:

pi@raspberrypi:/mnt $ sudo update-rc.d rpcbind enable
pi@raspberrypi:/mnt $ sudo service rpcbind restart

I then created a mount point for the media:

sudo mkdir /mnt/media
chown pi:pi /mnt/media
Finally, I added an entry for this mount in /etc/fstab and mounted it:
{IP ADDRESS}:/volume1/Media /mnt/media nfs rw 0 0
sudo mount -a

Plex

Much of the foregoing is based on the excellent tutorial on installing Plex on a Pi from element14. At this point I’m ready to install the various packages:

sudo apt-get install apt-transport-https -y --force-yes
wget -O - https://dev2day.de/pms/dev2day-pms.gpg.key  | sudo apt-key add -
echo "deb https://dev2day.de/pms/ jessie main" | sudo tee /etc/apt/sources.list.d/pms.list
sudo apt-get update
sudo apt-get install -t jessie plexmediaserver -y

I rebooted the Pi and was good to go.

Issues

These aren’t specific to the Pi but just things I encountered.

  • I had created down-scaled “versions” of some of my media for playback purposes. The new Plex found these, but they got commingled with the actual media leading to Plex detecting the wrong audio type (e.g. AAC instead of DTS). The Chromecast refused to play said media. I resolved it by deleting the “versions”, re-matching the media in question, and re-creating the version.
  • I’d forgotten that my setup is a “Double NAT“; to allow remote access I had to pass traffic from my cable modem to my internal router and then on to the Pi.

Featured image by Sven.petersen (Own work) [CC BY-SA 4.0], via Wikimedia Commons.

A bus in blue livery on a street
A Q70 Select Bus Service laying over at 61st Street and Roosevelt Avenue under the 61st Street–Woodside subway station in Woodside, Queens. Image by Tdorante10 (Own work) [CC BY-SA 4.0 (http://creativecommons.org/licenses/by-sa/4.0)], via Wikimedia Commons

The LaGuardia-Newark Shuffle

January 24, 2017 / / 0 Comments

Due to circumstances beyond my control I found myself on a flight to LaGuardia (LGA) the other day, but needing to get back to Newark Liberty (EWR) to retrieve my car. LGA was terra incognita to me but I knew the crossing Manhattan via to reach New Jersey is a fool’s errand. I decided against the various van shuttles and used public transport. The only thing I knew about LGA going in is that it has no direct subway service, although it has been proposed. In the end, I used a combination of a bus, a subway, a train, and a people mover. On a good day it’ll take you an hour and 45 minutes and cost $15.

Bus

A bus in blue livery on a street

A Q70 Select Bus Service under the 61st Street–Woodside subway station in Woodside, Queens. Image by Tdorante10 (Own work) [CC BY-SA 4.0], via Wikimedia Commons

I chose the Q70, because it offered the quickest access to Midtown via the Roosevelt Avenue/Jackson Heights station.
The MTA makes this pretty easy on you. As you’re walking out of LGA you see various signs directing you to the “Bus to Subway.” There are several MTA express buses operating out of LGA to various destinations:

  • The M60, which runs west into Harlem for a connection with the Metro-North Railroad at 125th Street and various subway services.
  • The Q70, which runs south into Queens for a connection with the Long Island Rail Road at Woodside and various subway services.

If you have a MetroCard this is easy. There are vending machines inside LGA to buy a card or refill it. Outside at the bus pickup area there are ticketing machines for the various services. It’s proof-of-payment: you insert your card, it debits the standard fare ($2.75) and you get a receipt. There was an MTA worker aboard the bus but our receipts weren’t inspected.

You have a valid transfer with this purchase, which I discovered when I swiped through the gates at Roosevelt Avenue/Jackson Heights and went down to catch an E. I took an E because it’s familiar to me and it’s cheap. The Long Island Rail Road at Woodside is faster, but it’s more expensive and I haven’t taken it before (shame).

The bus circles through all the terminals at LGA before running nonstop to Roosevelt Avenue/Jackson Heights via the Grand Central Parkway. This does put you at the mercy of traffic; it’s unavoidable. In rush-hour traffic on a very rainy day it took 23 minutes. Roosevelt Avenue/Jackson Heights is a large station and there appeared to be plenty of shops and restaurants in the immediate vicinity.

Subway

nycs-bull-trans-e-svg

The E runs express through Queens and Manhattan before dumping you off at Pennsylvania Station. On a congested day this took 37 minutes; normally it would take 21-25. The E gives you some excellent options: right before Penn Station it stops at 42nd Street with a direct connection to the Port Authority Bus Terminal, should you wish to try your hand at one of the airport bus shuttles. If you stay on the E to the World Trade Center (another 5-10 minutes) you can take PATH to Newark Penn or the Hoboken Terminal.

Getting off the E at Penn, you follow a maze of twisty passages, all alike, to reach the New Jersey Transit (NJT) platforms. I’m convinced that this maze rearranges itself every time I visit. Numerous NJT services stop at the Airport; departure boards in Penn indicate them with an airplane symbol. During the day you can expect a train every 15-20 minutes.

Train

newarkairportmonorail

AirTrain Newark grinding over a parking lot.

NJT trains are unreserved; tickets are sold in vending machines on the NJT concourse. One-way from Penn to EWR is $13/person; this includes a surcharge for using the EWR station. Tickets are inspected aboard the train. With EWR, as with Secaucus, you will need your ticket to exit the station.

It’s a 25-minute trip from New York Penn to EWR; the train will also stop in Secaucus and Newark Penn. At Newark Penn there’s a bus that runs to the airport. I’ve never taken it, but if you feel like adding a mode of transport it’s cheaper than continuing to the airport station and it might be faster for a reason I’m about to discuss. If you chose to take PATH from the World Trade Center you’d want to take this bus in lieu of NJT.

When you get off at the EWR station you go up the stairs, swipe through a fare gate with your NJT ticket, and make your way to the Newark AirTrain. This is the weakest link in the process. It’s an airport circulator serving the station, various parking lots, and all three terminals. It’s old and prone to breakdowns. It’s vulnerable to bad weather and has to be closed when there’s high wind, in which case it’s replaced by an infrequent bus. It’s also just slow, even when it’s running.

Anyway, if the AirTrain is running, it’ll take 10-15 minutes to reach the terminals. Catch a flight. Retrieve your car. Ponder the choices that led you to this moment.

Writing a Zotero translator

May 14, 2016 / / 0 Comments

Sometimes I joke that I do web development to support my railfanning habit. It’s not entirely true, but it’s always pleasant when the two intersect.

I’m a Trains subscriber. Trains is a monthly publication which serves both those who actually work in the railroad industry and enthusiasts (railfans) like me. Beyond the monthly print publication (which I get electronically, but never mind), Trains publishes a daily news feed called News Wire. There’s lots of good information here on the various comings and goings in the industry, though US-centric.

I use Zotero to index information for research projects–mostly railroading, but other topics as well. There’s an extension for Chrome, Zotero Connector, which lets you import web content directly into Zotero, saving a lot of manual entry. Many publications like The Atlantic and The Washington Post are natively supported. When one isn’t, Zotero makes a best guess based on page structure and metadata. How well that works depends on how well-formed the page here.

This is where we have a problem. The News Wire postings don’t have proper metadata at all–you need to scrape the page to find all the relevant parts. Zotero doesn’t know how to do that. The result is that one article imported with the following values:

Title Amtrak AEM-7 arrives in Strasburg | Trains Magazine
Author 12, Wayne Laepple | June
Author 2015
Website Title TrainsMag.com
URL http://trn.trains.com/news/news-wire/2015/06/12strasburg

Not so much. This is all on page. The title comes from the page title, probably because there are multiple h1 headers defined. The date and the author are commingled in a “byline” div.

Fortunately you can roll your own definition; Zotero calls these “Translators“. There’s a primer which I found useful, though it omitted some steps. The easiest way to proceed is to use Zotero Scaffold, which is an IDE for Mozilla Firefox. I use standalone Zotero with Chrome so I didn’t have Zotero for Firefox installed. If you don’t Scaffold will install but will not work. No error messages; it just sits there. This was incredibly frustrating until I realized my error.

Zotero Scaffold will write out a completed definition to the translators directory inside your Zotero data directory. On OSX mine was in /Users/foo/Library/Application Support/Zotero/Profiles/random string/zotero/translators. I’ve posted one to github as a gist: https://gist.github.com/mackensen/981b1d5393e07e8435798eaee843e3fc. A few comments on this:

  • Scaffold takes care of all the front matter, including the GUID.
  • detectWeb and doWeb can be more complex if there are different types of data (such as a search results page). I deliberately provided a narrow page target so we’re only handling single posts.
  • All the terror is in scrape, where I used xpath queries to extract the parts of the page I needed and then format them appropriately. Don’t overlook the utility function cleanAuthor(), which takes an author string and breaks out component parts. In my first iteration I was reading the author and everything seemed fine but it didn’t import into Zotero.
  • Translators are loaded into memory by the browser. If you make a change, you’ll need to reload the browser (boo) or disable and reenable the extension (yay).

New result, same article:

Title Amtrak AEM-7 arrives in Strasburg
Author Laepple, Wayne
Blog Title Trains News Wire
Date June 12, 2015
URL http://trn.trains.com/news/news-wire/2015/06/12strasburg

Yeah, that’s much better.

« Older posts Newer posts »

© 2024 Terrestrial Navigation

Theme by Anders Noren — Up ↑