Thursday, November 12, 2020

2020-11-11 Fixing the rivers

[OOPS - I made a typo in the date when I first posted this!]

I noticed that in my topo map, I was losing rivers. For instance, the entire east branch of the Neversink had disappeared, although several of the minor tributaries were still shown. I blew a lot of today tracking down the issue.

It turned out to be that I'd downloaded updates to NHD when I was rebuilding the database after upgrading to PostgreSQL release 12. It appears that USGS changed the file layout so that the NHDWaterbody and NHDFlowline tables now are in multiple shapefiles, all of which have to be loaded to have the full data set.

Layout of a ZIP file from NHD
Layout of a zip file from NHD

Since the first shapefile still has the same name that the old one did, the program that does the loading had no trouble, but loaded only a third of the data for New York (and even less for some other states!), so I never got an indication when loading the files that there was any sort of trouble.

Downloading and importing all the ZIPs another time took most of the day. I could have worked on other things, but the database was pretty unusable while it was thrashing over the rivers and streams.

Anyway, now it's fixed.

Before (White's Run is missing):
Map of Womelsdorf, WV, missing White's Run
After (White's Run is back):
Map of Womelsdorf, WV, with White's Run restored

No comments: