Making maps in New York - marshlands

Last time, I discussed adding surface water features to the map under construction. This time, I want to continue on the hydrography theme and add the marsh lands.

It turns that not only the marsh lands themselves, but even the process of identifying the marsh lands, leads the unwary into a swamp. The issue is that the determination of what is and is not a wetland is extremely contentious. In New York State, several agencies are involved. Among them are the NYS Department of Environmental Conservation, the US Fish and Wildlife Service, the US Army Corps of Engineers, the Environmental Protection Administration, and even the New York City Department of Environmental Protection (the protected watershed that supplies New York with its drinking water extends for well over a hundred miles away from the city itself). All of these agencies have different ideas of what is or is not a wetland, and they are tasked with the enforcement of different wetlands-protection laws, which they do with a heavy hand. In fact, just the other day a case where the owner of a suburban lot was financially ruined by an EPA wetlands determination was argued in the Supreme Court.

The end result is that there are multiple sources, often conflicting, of data on wetlands. That said, all I really care about is how likely my boots are to stay dry on a walk, not what agencies have to bless building permits or inspect properties for illegal alterations to drainage.

If I were to use only a single source, it would be the Fish and Wildlife Service National Wetlands Inventory. It lists all types of wetland (salt marsh, emergent and wooded freshwater marshes, mangrove forest [not in New York!], submerged marsh, and so on), and it's "one-stop shopping." But it's by no means complete. In places where I have "boots on the ground" knowledge, there are some awfully soggy spots that aren't in the inventory.

A more comprehensive data set for New York, which identifies more wetlands but misses some that the USFWS covers, is available from the NYS Department of Environmental Conservation. In most of the state, it comes in data sets that are one to the county. For Essex and Hamilton counties in the north of the state, the wetlands inventory is maintained by the Adirondack Park Agency. In these two counties, the data are in two sets of maps - in quarters of USGS 7.5 minute quadrangles, so be prepared for a fair number of downloads. One covers the Oswegatchie-Black, Upper Hudson and Saint Regis watersheds, and the other covers the remainder of the Adirondack Park.

The Fish and Wildlife Service data had an issue when I tried to reproject it: it's on the Albers equal-area conic projection, referenced to NAD83, for the continental US. And that's a projection that doesn't have an EPSG number. (It's also not all that well-suited to small-scale maps, although it would make a nice wall map of the Lower 48.) So I had to add it. In QGIS, this is done in the 'Settings->Custom CRS' dialog. The parameters that need to be entered are:

+proj=aea +lat_1=29.5 +lat_2=45.5 +lat_0=23 +lon_0=-96 +x_0=0 +y_0=0 +ellps=GRS80 +towgs84=0,0,0,0,0,0,0 +units=m +no_defs

Then 'ogr2ogr' can load the polygons and lines into PostGIS. We need a lot of options, because (a) we want to reproject the data onto a more sensible projection (I intend the map to use NAD83, UTM Zone 18N), (c) we have to override the layer type to MULTIPOLYGON (ogr2ogr incorrectly guesses POLYGON), and (d) we have to give the PRECISION=NO option to layer creation to work around a bug.

ogr2ogr -f "PostgreSQL" -overwrite -skipfailures -t_srs EPSG:32618 -progress \
    "PG:dbname=gis" NY_shapefile_wetlands/CONUS_wet_poly.shp \ 
    -nln fws_nys_wetlands -nlt MULTIPOLYGON -lco PRECISION=NO

Go get lunch, this one will take a long time.

The NYS DEC files are equally problematic. They're in the ARC/GIS E00 export format, which was never all that widely supported. Quantum GIS and 'ogr2ogr' both purport to be able to import them, but generate imports with corrupted polygons that will not display. I was finally able to work around this problem with another open source product called AVCE00 - which includes a program called 'avcimport' that will convert E00 files into the ARC/GIS binary coverage files, which work correctly. (Whew!)

So I did a little bit of shell scripting to run all the AVCE00 imports:

for f in `find . -name \*.e0? -print` ; do  \
    ~/avce00-2.0.0/avcimport $f `basename $f .e00`-imp ; \
done

And then for the counties of interest, I ran them into PostGIS. I should probably have scripted this part, too. You have to do arc and polygon layers separately:

ogr2ogr -f "PostgreSQL" -s_srs EPSG:26918  "PG:dbname=gis" 001fwa-imp/ \
    -nln nys_wetlands_arc arc
ogr2ogr -f "PostgreSQL" -s_srs EPSG:26918  "PG:dbname=gis" 001fwa-imp/ \
    -nln nys_wetlands_pal pal
ogr2ogr -f "PostgreSQL" -s_srs EPSG:26918 -append "PG:dbname=gis" 057fwa-imp/ \
    -nln nys_wetlands_arc arc
ogr2ogr -f "PostgreSQL" -s_srs EPSG:26918 -append "PG:dbname=gis" 057fwa-imp/ \
    -nln nys_wetlands_pal pal

With this in place, displaying polygons for the wetlands was easy. But now I wanted them to be styled prettily. I went and downloaded the National Park Service map symbols in SVG, and used Inkscape to extract the 'marsh' symbol. (SVG file attached at the end.) I changed the fill type on both the wetlands layers to 'SVG fill' and provided the file name as a custom fill pattern.


(Or you can use my QML file so you don't have to do this yourself. It's attached at the bottom of the post.)



So now we have marshlands appearing in the map, with the standard symbol. Looks nice. Next time - on to roads, railroads, trails, runways, and other linear features.

Attachments:
marsh.svg - Fill pattern for marshlands
NYS-wetlands.qml - QML to define the wetlands style

Read more...

Making maps in New York - surface water

We left off last time with a map that has the street grid (well, sort of, it still needs to be styled!) and the contour lines (so that if I'm looking at a street or trail, I can tell how steep it is).

The next key piece of information that I need for a walking map is whether I'll get my feet wet: where is the water on the surface? So, let's look at some data sources for hydrography.

The hydrologic features are, we already saw, in the digital raster quadrangles, but (I repeat yet again) we don't want to use them, because they don't scale, and because they're cut off at the margins. So let's get vector data instead.

Fortunately, for New York, surface water features are "one-stop shopping." New York State's Office of Cybersecurity has two shapefiles, one with line features and one with area features, that cover the entire state and are suitable for rendering at 1:24000.

After downloading and unpacking them, I put them in the PostGIS database and spatially indexed them:

$ /usr/lib/postgresql/9.1/bin/shp2pgsql -I -c AreaHydrography.shp nys_area_hydrography \
   | psql -d gis > /dev/null
$ /usr/lib/postgresql/9.1/bin/shp2pgsql -I -c LinearHydrography.shp nys_linear_hydrography \
   | psql -d gis > /dev/null

I wound up loading two layers for the linear features. One of them had the query condition:

name IS NOT NULL AND name != 'Stream'

while the other had:

name IS NULL OR name = 'Stream'

The reason for this was so that I could assign labels only to those streams not named 'Stream': that name appears on more than half the streams in the database! [Post hoc: I had to do the same thing for area features named 'Lake' and 'Pond', for the same reason.]

I also did some styled lines for canals, intermittent streams, and so on. I'll put the QML files at the end of this post.


So now there is open water in my map. I still want to do some more hydrography, because my feet can get just as wet in mud!

Attachments:

Styles for linear waterways: NYS-linear-waterways.qml
Styles for area waterways: NYS-areal-waterways.qml

Read more...

Making maps in New York - hydrography

We left off last time with a map that has the street grid (well, sort of, it still needs to be styled!) and the contour lines (so that if I'm looking at a street or trail, I can tell how steep it is).

The next key piece of information that I need for a walking map is whether I'll get my feet wet: where is the water on the surface? So, let's look at some data sources for hydrography.

The hydrologic features are, we already saw, in the digital raster quadrangles, but (I repeat yet again) we don't want to use them, because they don't scale, and because they're cut off at the margins. So let's get vector data instead.

Fortunately, for New York, surface water features are "one-stop shopping." New York State's Office of Cybersecurity has two shapefiles, one with line features and one with area features, that cover the entire state and are suitable for rendering at 1:24000.

After downloading and unpacking them, I put them in the PostGIS database and spatially indexed them:

$ /usr/lib/postgresql/9.1/bin/shp2pgsql -I -c AreaHydrography.shp nys_area_hydrography \
    | psql -d gis > /dev/null
$ /usr/lib/postgresql/9.1/bin/shp2pgsql -I -c LinearHydrography.shp nys_linear_hydrography \
    | psql -d gis > /dev/null

I wound up loading two layers for the linear features. One of them had the query condition:

name IS NOT NULL AND name != 'Stream'

while the other had:

name IS NULL OR name = 'Stream'

The reason for this was so that I could assign labels only to those streams not named 'Stream': that name appears on more than half the streams in the database!

I also did some styled lines for canals, intermittent streams, and so on. I'll put the QML files at the end of this post.

So now there is open water in my map. I still want to do some more hydrography, because my feet can get just as wet in mud! Swamps are important to indicate, too! Since the area that I'm immediately interested in mapping has no salt water, and since it's outside the Adirondack Park, the New York State Department of Environmental Conservation's list of protected freshwater wetlands should give me quite a reasonable starting point.

Those files are available by county. Downloading my county and several of its neighbours, I tried a quick import into QGIS and into PostGIS. For the E00 file format that was used, both imports failed silently, leaving a set of malformed polygons for the swamp areas. Fortunately, I was able to find a workaround. I used the program 'avcimport', part of the AVCE00 package, to convert the .E00 files into ARC/GIS binary coverages, and then used 'ogr2ogr' to import them:

$ for f in `find . -name \*.e0? -print` ; do  ~/avce00-2.0.0/avcimport $f `basename $f .e00`-imp ; done
$ ogr2ogr -f "PostgreSQL" -s_srs EPSG:26918  "PG:dbname=gis" 001fwa-imp/  -nln nys_wetlands_arc arc
$ ogr2ogr -f "PostgreSQL" -s_srs EPSG:26918  "PG:dbname=gis" 001fwa-imp/  -nln nys_wetlands_pal pal
$ ogr2ogr -f "PostgreSQL" -s_srs EPSG:26918 -append "PG:dbname=gis" 057fwa-imp/  -nln nys_wetlands_arc arc
$ ogr2ogr -f "PostgreSQL" -s_srs EPSG:26918 -append "PG:dbname=gis" 057fwa-imp/  -nln nys_wetlands_pal pal
... continue for several more counties ...

Yet another task that I should have scripted!

Anyway, adding the resulting 'nys_wetlands_pal' and 'nys_wetlands_arc' in QGIS and doing a bit more styling gives quite an attractive result. I extracted an SVG of a swamp fill pattern and used that as the fill pattern for the polygons.

So now there is a lot of hydrography on the basemap. Next step is to start adding some cultural and administrative layers.




Read more...