2020-11-08 Buck Mountain, Pilot Knob, NY

No progress on software development Sunday - I was out hiking. My daughter and I climbed Buck Mountain, on the east shore of Lake George, NY.

Buck is one of the Lake George 12. I know that I climbed it before, but have no idea when. Cathy, too, may have climbed it, too, on a backpacking trip from Camp Chingachgook. In any case, now we have both logged it properly with a date and photo evidence.

I'll do a proper trip report later. I've got a back catalog to get through.

Kevin on the summit of Buck Mountain

Read more...

2020-11-07 First whack at a stylesheet for osm2pgsql

I managed to do enough Lua coding today to make my first attempt at a stylesheet for the flex backend of osm2pgsql to create the tables that the renderer for pictorial highway shields in OpenStreetMap will need.

The code is still pretty nasty - I'll clean it up on Monday - but it's producing the tables. I tried it on an export for Connecticut (rather than attempting the whole map), and it's turning out tiles that look just as they did before.

True to form, I found a bug in osm2pgsql while I was working on this. Nothing I can't work around. (2020-11-09: The maintainers are amazing. It's fixed already.)
Read more...

2020-11-06 Learning Lua

I spent pretty much all of my hacking time today munching on Lua documentation and trying out random snippets of code. (Once you know a couple of dozen programming languages, you typically learn a new one by taking a few hours to read the reference documentation and try a few short programs, and you're good to go.)

This is mostly so that I'll know enough Lua to write a library for extending the style sheets of osm2pgsql to create the tables that the current incarnation of OSM shield rendering uses.

Read more...

2020-11-05 Highway shield graphics - minor updates

Not a lot of progress today. I sketched some more ideas for using the flex back end of osm2pgsql to process concurrent routs in my notebook, and I puttered about verifying that the picayune changes that I made to the shield graphics worked.

The main change there is that the highway shields are back in New Brunswick, Prince Edward Island, Saskatchewan and Alberta. (The Canadian mappers had made wholesale changes to the network tag to unify the processing across Canada, and the graphics generation needed to be updated to match.)

Map of Woodstock, New Brunswick showing rendering of highway shields. Green: primary; blue: secondary; black: tertiary; maple leaf: Trans-Canada Highway.

There are also a lot more numbered county roads in West Virginia.

Map of the area west of Womelsdorf, West Virginia, showing that state's distinctive numbering scheme for secondary and tertiary roads.

And suburban Suffolk County, New York has its numbered county highways now.

Map of the area south of MacArthur Airport, showing numbered county roads (blue pentagonal shields) rendered in a distinct style from state highways (white shields in New York's distinctive shape).

Read more...

2020-11-04 North American highway shields

As I mentioned earlier, I'm trying to resume work on the project of rendering North American highway banners on OpenStreetMap.

I had left off work on the project this spring, when I was starting the sprint toward retirement, trying to make sure that my work was handed off responsibly. I had also left with a tremendous sense of frustration.

The osm2pgsql tool, which I had been using (like most people who try to serve up map tiles) to populate a database for rendering, was not capable at the time of creating the tables I needed. I had written a detailed proposal of how what I needed might be accomplished. It was badly received; in fact, one of the core team of the project suggested that osm2pgsql was simply the wrong tool for the job.

I then withdrew the proposal, and began to contemplate alternative tools, but was quite discouraged at the time that it would take to retool. Given the pressures at work, I put the project on hold.

Fortunately, since then, the plan for alternative programs to populate the database has been overcome by events. Jochen Topf, unbeknownst to me, was working on a new flex back end for the program, which allows a significantly wider selection of database schemata than the original program did, and is, in fact, a superset of what I had proposed. While some post-processing will be required, it appears fit for purpose.

So, the next tasks will be (a) retool my existing database population to use that back end (testing against a small extract, most likely Connecticut or Rhode Island), and (b) expand that back end to produce the tables that I need to describe numbered highways. With any luck, I can make some progress on that stuff today.

Read more...

2020-11-03 The Erie Canal in Niskayuna

I've been curious for a while now about the old right-of-way of the Erie Canal along the Mohawk River in Niskayuna, and I've finally managed to explore some of it.

Most of my curiosity came from the fact that when I joined the project, someone had already drawn on OpenStreetMap a trail that followed the canal right-of-way from Aqueduct Road near the traffic circle nearly to the Schenectady city line.

Within the maintained area of Aqueduct Park, the trail pretty much follows the canal tow path, and you can see a very wet area to the southe where the canal has been filled in. There's then a rather less manicured trail that follows a natural gas pipeline (and old trolley right-of-way) up to the Mohawk-Hudson Bike/Hike Trail (Empire State Trail, Erie Canalway).

West of there, the canal right-of-way was obviously impassable. There were, however, numerous unmarked trails that headed north and west from the gas pipeline. I'd only been in there once before, and turned back because the nettles, raspberries, and poison-ivy were just too dense for my liking. Now that we've had a hard frost, I wanted to explore. Part of the area is state land, and while the tax rolls show another part as privately owned, I saw no posters, so had few reservations about bashing off down the herd paths.

Because the days are getting short, and walking there and back again from home is 10 km already, I kept the walks in that area fairly short. What I found was that there's a loop around the beaver pond near the bike path, and another, longer loop consisting of a path in the ditch and another on the south towpath. The flooded section is relatively short, and looks either to be intermittent or the result of beaver activity. Crossing it would still involve wading, so I scrambled the bank of the ditch and found a short beaten path through open woods to the gas line at that end.

There is also a well-defined unmarked trail connecting the west side of the beaver pond down to the trail on the towpath and the one in the canal bed. A very short section of that is flooded, with a well-beaten workaround.

There may be a connection with the National Grid maintenance track under the power line at the west end of the section that I explored. There's more state land to the west, so it's possible that the network goes a little farther. (There's a puzzling area on aerials - it looks as if there's a large clearing with a service building and no road access. No idea what's going on there!)

Anyway, I uploaded a rough draft of the paths that I found and some of the soggy spots that they skirted. That was the mapping activity for this couple of days.

One remaining puzzle: The pond by the bike path is full year-round. The stream under the bridge bu the power line runs continuously. But in a dry season, there's no water flowing across the trails, and no bridges or culverts that I could see. Where does the water go?

Read more...

2020-11-02 Beginning a new chapter

Today was my first day of not reporting to work at my former employer. I'm now a full-time self-funded open-source developer (that is to say, a retiree). This post is possibly the first of a series chronicling my adventures in retirement.

Many friends and colleagues have asked me, “so, what are your plans?”

I can't say that I have firm plans for anything at this point. The year 2020 has been a striking example of, “do you want to make the Almighty laugh? Tell Him your plans." My immediate plan is to try to rest up and decompress a little bit.

Frankly, I've never in my life been so without structure. I've pretty much never had a vacation longer than a week or two without an externally-imposed schedule. Even as a child, I had my parents filling, and overfilling, my “free&rdquo time with “enrichment activities”. Mary Ann and I would up doing a lot of the same with our daughter - although trying to keep her on a looser leash than the one I had - which tended to fill our time as well as hers. “Where do I go from here?” is going to taks some soul-searching. Still, I can drop some random thoughts.

I'd really like to do more hiking. I'd been putting on weight the past few years with enforced inactivity. Having the doctors work repeatedly on my eyes and feet, with activity restrictions on the rehab, has been confining. I've been doing a lot more walking in town since COVID-19 hit - it's one of the few things I can do outside the house, and working up the distance again to where I'm regularly doing 10 km a day (without a pack, of course, or with a very light day pack.) Catherine and I have also made a few trips again into the Catskills, most of which I've yet to write about. (Slovenly of me!)

In fact, if COVID-19 is ever not a blocker for crossing county and state lines, resupplying, or sharing a campsite, I'd like to try another big hike. My thru-hike of the Northville-Placid Trail turned into a series of sections, because of illnesses and injuries. Unlike a traditional 'thru hiker,' I couldn't just rest a few days after a problem and pick it up again - I'd need to go back to work! I've kind of an itch to do the Long Path of New York - the idea of hiking from Manhattan to the Adirondacks just intrigues me. (Yes, I know, the trail is built only to Altamont. There's a description of the route beyond there: who needs a maintained trail? I'm an experienced bush-whacko.) Given the current state of the world, I'm pretty sure that isn't going to happen in 2021. 2022, maybe, if all goes well.

I also have several unfinished open-source projects.

The biggest one of these, by far, is the “quadcode” system: a machine-code compiler for the Tcl programming language. Tcl is a highly dynamic language whose core paradigm is string substitution. Extracting data type information and constraining side effects to the point that safe machine code can be generated is an insanely difficult problem. Donal Fellows and I have had some limited success, but making the beast into something that's serviceable, that's deployable, and that covers much more of the language is going to be a long slog.

The next phase here will likely involve some major refactoring, and will require some really intense brain work - something that's not compatible with my goal of getting some rest after my final sprint on the job. It'll likely be a little while before I start picking up the pieces - but watch this space!

Another big unfinished project is to make a rendering pipeline for OpenStreetMap that deals in a graceful way with North American highway numbering systems. Identifying named and numbered routes with pictorial symbols on the map is a big part of it. Many North American jurisdictions have three or four different numbering schemata overlaid, so the shape of the marker is important to a driver trying to follow a route. Moreover, unlike the way most of the rest of the world does it, routes are often concurrent - it's not uncommon for a road over a bridge to have three or four different highway numbers. There's one pathological example near Indianapolis that has eight! This sort of overlaying has caused rendering gaffes, and a better approach than the current one is needed.

Again, I've had some limited success making a map that handles numbered routes in more or less the way I like, but now comes the long slog to make the implementation scalable (e.g., dealing with minutely diffs on a planetary scale; managing the expiration of tiles so that the whole map need not be pre-rendered), fortify it to be robust against future change, make it fit more closely with the rendering tools used by other OpenStreetMap servers, and find a path to widespread deployment (rather than just serving to a few friends off my home machine).

I've also been neglecting my music. I might have time in retirement to take it up again, particularly if conditions in the world improve to where people can perform again. I've been doing just a little bit - including playing recorder for a televised church service this past weekend. (The offertory , starting at 31:14, has a solo by Yours Truly.)

What else? Well, I've got a long history of going off-script. I may find myself itching to contribute to open-source machine vision, or get back into making stuff (particularly electronics), or just try something entirely new.

Or the perfect consulting gig might just fall in my lap. You never know. I'll just have to see what happens. Wish me luck!

Read more...