Trying to map a section of the Northern Excursion - not exactly successfully

Today, I went over to West Glenville to see if I could get GPS tracks of the one short section of the New York Long Path that is off-road there. I've never seen a map of that section, only the verbal description at the online softcopy guidebook. As we'll see, that description presents some issues. I'm writing this to share my experience; hopefully, the trail and guidebook maintainers will be able to enlighten me!

The trip started out all right. I parked across from the second Wolf Hollow Road turnoff at mile 4.10 of section 39. Note that the “Parking” section of the online book has a typo; it's 4.10 and not 3.10.

I found the point (mile 4.7) where the Long Path swings south into the farmer's field from West Glenville Road without trouble. The blaze at the turnoff is apparent, and there's a deeply rutted mowed line just past the hedgerow, leading right up to the road crossing at mile 4.9. The blazing there is also obvious in the northbound direction. There does not appear to be a southbound blaze at the driveway near the Cape Cod house. There is also a “For Sale” sign there - is the club aware that the property is about to change hands? Will the transaction affect the negotiated trail easement? The trailer on the north side of the road appears not to be there any more, but there is a relatively new house to the left of the trail.

The walk through the woods to mile 5.35 was pleasant. The trail is obvious and well-blazed, and the stone wall and hemlock grove appeared right on cue. At mile 5.35 the trail broke out into a hayfield as described and followed a mowed trail around the eastern edge. There is another typo in the online book here - it says to walk the northern edge of the field. It should say to walk the eastern edge.

The hedgerow and stone fence at mile 5.50 appeared on cue. At least, I presume that I have the correct hedgerow. My GPS coordinates for it are approximately 42.9240,-74.0820. This first hedgerow, though, is only about 0.1 mile from where the trail left the woods. Another hundred yards or so brought me to the corner of a second hayfield, at about 42.9247, -74.0822. This second corner is therefore about mile 5.50, which would match the guidebook description. Blazes were still visible.

At the northern end of this field, I became quite confused. The confusion between northern and eastern edges of fields was sorted out by observing the blazes, but I spent some time wandering in the woods around this corner, and saw no sign of blazing. Returning to the field - whether it was the first or second one described in the online guidebook is in some doubt - and following its northern edge to the west led me to another gap in a hedgerow, (roughly 42.9248,-72.0829). On the other side, further aqua blazes were visible, leading around the edge of a third hayfield, about 0.2 miles. The blazes are a fair distance apart, so I wound up also exploring a mown track at the southeast corner of the field. It led only to several hayricks, an overturned trailer and a considerable amount of rubbish, and no trail lay beyond.

Blazes were visible at intervals following the field edge for about another 0.2 miles to the northeast corner (about 42.9265,-74.0824). I passed several unblazed side trails coming off the east edge of the field. At least one had diagonal blazes instructing the hiker to continue following the field.

At the northeast corner, diagonal blazes instructed a turn to the west, Looking along the northern edge of the field, I saw nothing resembling a trail until a woods road turned to the northwest at about 42.9265,-74.0829. (This is perhaps mile 5.65 in the guidebook.) There were what appeared to be faint paint blazes on this road, instructing a turn on a left fork that soon appeared.

At this point, I lost the blazes entirely - neither fork had apparent markings. After exploring a short distance along the woods roads, I began to be intimidated by the frequent posters that appeared, and simply chose the road that appeared to head most directly for Touarena Road. This road led me to a driveway with a chain across it - and posters - at 42.9266,-74.0885. Not the Long Path! At no time had I come upon the stream crossing that was supposedly at mile 5.70.

The blazing in this area is clearly in need of work. I was not able to follow the guidebook directions northbound.

I then walked up Touarena Road to the correct gravel driveway (42.9291,-74.0875), which was very clearly blazed for the Long Path, and decided to see if I fared any better southbound. The gate is supposedly at mile 6.2. The “Parking” section shows it as mile 5.9 - this does not agree with the mileage table.

The trail leaves the driveway on a broad woods road at mile 6.15, with a conspicuous guidepost indicating it. It follows what is rather a maze of woods roads/ATV trails for about 0.5 miles, with all turns clearly blazed. Everything went along perfectly until I saw the last blaze at about 42.9266,-74.0811. This would be at about mile 5.7 in the guidebook if I am reckoning the mileage correctly. The blaze is near the corner of some stone walls, in a stand of hemlocks, near the poster line of Five Oaks Development.

At this point, the blazing abruptly ended. I walked a spiral pattern around the last blaze out to where I could no longer see it. No luck. I also continued some distance in roughly the same direction, and rapidly came to a property corner which was posted from all directions. Having no way to respect private property in that situation, I forged ahead a short distance farther, and saw mountain bike tracks in the dried mud at an intermittent stream. I followed the tracks across a larger stream, and up an improved but unmarked trail on its steep western bank, finding myself once again on the east edge of the hayfield, about 42.9261,-74.0823, at the head of an unmarked trail between two posters.

I spent a few minutes exploring the northern edge of the field, and the woods road that I had followed before, looking for any sign of a trail that would lead in the general direction of where I lost the blazing when moving southbound, and found nothing.

I uploaded to OpenStreetMap an edited version of the tracks that I followed, as far as I could see the blazes. The points where the blazes peter out, while not that far apart, don't even seem to be aimed for each other, as can be seeo on Waymarked Trails.

Suffice it to say that I'm puzzled.

Read more...

Making tiled maps, and importing into OpenStreetMap

A user on the OSMand group asked about my process for generating my own tiled maps, and for importing parks and preserves.

The topic, I thought, deserved a decent writeup, so I decided to work my reply into a post here.

Workflow of getting tiles into Backcountry Navigator

I've got a reasonably capable (quad Core i7, 32 GB memory, half a terabyte of SSD, 4 terabytes of RAID 0+1) Linux system at home, where I host a PostGIS database. By far the biggest part of that database is taken up with the OSM North America export from geofabrik.de, which I initially loaded with osm2pgsql, and synchronize with GeoFabrik's nightly diffs using osmosis. I retain the 'slim' tables because I have scripts that need them.

Starting from Lars Ahlzen's TopOSM, I've set up a Mapnik rendering pipeline that uses OSM, the National Elevation Dataset, the USFWS national wetlands inventory, the National Hydrography Dataset, the National Landcover Dataset, and a bunch of state and local databases to assemble an American-style topo map that shows much of the information that I want to see. I've set up a previewer for that map at

https://kbk.is-a-geek.net/catskills/test4.html

(Feel free to pan and zoom.) What I'm set up at present to render is the US east of a line from roughly Atlanta to the Mackinac Strait, and north of Atlanta. (That's because I set it up partly for some correspondents of mine who are interested in maintaining information for approach trails to the Appalachian Trail, and the area I render is roughly the bounding box of that trail.

The map is augmented with information that I got from a number of state and local GIS departments. For example, this view shows a number of trails in magenta. Those are trails that I got from NY State Department of Environmental Conservation's GIS department. I do NOT import those because

1. There are license incompatibilities
2. The data are stale, and were originally digitized from inappropriately small scale maps. In some places they're quite good indeed, but in other places they're way off.

I find them to be a useful indication of "there is a trail somewhere near here", and a "to do" list for trail mapping with GPS.

For the most part, those auxiliary data came in the form of shapefiles, and got imported into more tables in the PostGIS database. The Mapnik source for the map as you see it has dozens and dozens of layers.

Since I've served the map up, I can tell BackCountry Navigator to use it as a web map, much as it would use the US topos from ArcGIS, or OpenStreetMap tiles, or Bing aerial imagery. The URL for the map is

https://kbk.is-a-geek.net/catskills/tiles/{Z}/{X}/{Y}.jpg

(Please don't incorporate into apps to re-share, I have limited bandwidth and even more limited time to support the thing. Also, ''please'' don't try to bulk-download all the tiles! If you need large amounts of map, email me and we'll work something out.)

Since BackCountry Navigator supports downloading the tiles for an area in advance of a trip, I download from my home Wi-Fi before I go, and run happily without cell service in the woods.

Importing parks and preserves

The imports of parks and preserves have been several projects, each with its own workflow.

New York City watershed recreation

I did an import of New York City Watershed Recreation Lands. For that, the city made available PostScript maps of each of its facilities (note that these are located outside the city, protecting the watershed lands in the Catskill Mountains that provide New York with its water). It turns out that these PostScript files were already georeferenced, and that the names of layers in them were predictable, so I was able to set up a script that downloads them one at a time, scrapes out of the file just the boundary of the facility, and pushes the facility boundary into PostGIS.

The script got rather complicated, because it had to check that it wasn't overwriting data that are already in OSM, repair topology of the polygons, simplify the ways, shrink the polygons back a short distance to avoid collisions, and similar tidying operations. I also developed a mapping between the descriptive attributes in the shapefile and OSM tagging.

All the scripting was done in Tcl/Tk, for no better reason than that I'm familiar with it through having used it for about 25 years.

I proposed the import on the OSM Wiki and went through the usual storm and fury on the 'imports' mailing list.

The eventual import was done by taking the data, one parcel at a time, and using the JOSM remote control interface to push the polygons into JOSM. I did a final eyeball check for each, and committed them to OSM.

I've since revisited the import once, picking up 10 new purchases, 25 boundary changes, and six modified sets of access restrictions.

New York State Department of Environmental Conservation lands

Emboldened by this experience, I took on reworking the seven-year-old import of the NYS Department of Environmental Conservation Lands shapefile. It was a similar workflow: pour the data into PostGIS, tidy up the geometry and topology, map tags, and so on - but on a much larger scale, and starting from a single shapefile rather than several hundred PostScript maps.

Once again, since I had automatable data, I did this one as a formal (re)import proposal

This was again a parcel-by-parcel effort in JOSM, but this time, there was much more manual work, since there were existing versions of the parcels that had to be conflated. It was a pretty hellish job, completed in off-and-on evening work between May and September of 2016. The hardest areas to handle were ones where complicated shorelines formed the boundaries of reserves; Saranac Lakes Wild Forest and Lake George Islands were ones that I recall as being particularly tricky.

Conflation was also tricky if the parcels shared ways with adjacent landuse or landcover polygons. In the worst cases, I simply left the original polygons in place, but removed the tagging identifying the land as state forest, and then overlaid with the protected area.

Once again, now that I keep after it every year or so, the modifications are more straightforward. I reimported again a couple of months ago and managed to do it in a couple of evenings.

New York State Parks

I then moved on to New York's State Parks. Note that the Adirondack Park and the Catskill Park are parks owned by the state, but they are not State Parks; instead they are entities unto themselves, enshrined in the state constitution.

Each of the state parks has a georeferenced PDF trail map available from New York State Office of Parks, Recreation and Historic Preservation. Unlike the New York City PDF's, there were no vector layers for me to scrape. Moreover, the license status of the state park maps is unclear, and I live in the one Federal Circuit where government entities can claim copyright to data such as these. Instead, I treated the PDF's as a 'to do' list of parks that needed to be mapped.

For each of these, I did the following:

1. Converted the PDF to a GeoTIFF for efficiency, and loaded the PDF into Quantum GIS. (QGIS can read GeoPDF, but becomes unusably slow when it does.)

2. As a separate layer, loaded up a shapefile of tax parcels owned by New York State. This shapefile has license terms compatible with ODBL - the public has the right to use the data for any lawful purpose.

3. Selected all the tax parcels that were coterminous with the park. This could be as few as one or as many as several hundred.

4. Conflated the parcels and repaired the topology. (This was a fair amount of manual patchwork.)

5. Exported the tidied parcel from QGIS as a shapefile.

6. Opened the shapefile in JOSM and downloaded the OSM data.

7. Added tagging. For this, I wound up developing a couple of JOSM presets for 'New York State Park' and 'New York State Historic Site', and did a bunch of copy-and-paste of things like park names, web sites, and telephone numbers from parks.ny.gov.

8. Conflated with what was already in OSM. A lot of state parks were already there, with somewhat whimsical boundaries. If the boundaries were from TIGER, I had no qualms about overwriting them.

   

   If the boundaries were actually provided by a local mapper, I tried to get in touch with the mapper in question and find out how they were obtained. The mappers were very cooperative, indeed, and got back to me promptly. In virtually all cases, they had traced approximate boundaries from Bing and were happy to have the ones from the tax rolls.

   Again, there were adjacent-parcel issues, and again, I sometimes resorted to overlaying the protected area and leaving existing landcover polygons (and adjacent landuse polygons) alone.

I didn't call this one an 'import'. I was comfortable with not doing so. There was far too much manual work involved for it to fall under the definition of 'automated edits.' Everything that went in had been touched with eyeball and mouse. Nobody complained. It is more blessed to beg forgiveness than to ask permission.

Other land areas

I used the same technique, with different source datasets, to fill in a number of county and municipal parks, and some private preserves. This is a work in progress, there's always more to be done. The most recent ones that I brought in were just this past weekend (2017-07-03), with a few more parcels belonging to the nonprofit Mohawk Hudson Land Conservancy.

(I still need to get out to these and GPS the trails!)

That's also how I sorted out the unholy mess of overlapping polygons for West Point, four state parks (Bear Mountain, Harriman, Sterling Forsest, Schunnemunk, Storm King), the Federal corridor for the Appalachian Trail, a private, open-to-the-public preserve (Black Rock Forest), the villages of Harriman, Woodbury, Fort Montgomery and Stony Point, the Woodbury golf course, and the Hudson River riverbank. What a tangle that was!

TL;DR

The one-line summary: "It's never easy, is it?"

Read more...