Ogr2osm presentation

1

OGR2OSM
A powerful tool for converting geodata to .osm format
SOTM-US 2012

About
2

 What ogr2osm can do for you
 How ogr2osm works
 A case study of a data conversion
 Why care about converting?

History
4

 Written in 2009 by Iván Sánchez Ortega
 Rewritten 2012 by Andrew Guertin for UVM
buildings
 I now maintain it

Features
5

 Can read any ogr supported data source
 .shp, .mdb, .gdb, sqlite, etc
 Reprojects if necessary – eliminates a step with many
sources
 Works with multiple layer sources or shapefile
directories
 Uses python translation functions that you write to
convert source field values to OSM tags
 This allows you to use complicated logic to get the tagging right
 Documentation

Installing
6

 Requires gdal with python bindings
 Simply sudo apt-get install python-gdal git on
Ubuntu
 May require compiling gdal from source and third-
party SDKs for some formats (.mdb, .gdb)
 Run git clone --recursive
https://github.com/pnorman/ogr2osm to install
 Full instructions at
https://github.com/pnorman/ogr2osm

Code flow
7

Read in data source Process each layer Merge nodes
• Uses python ogr bindings • Converts from ogr to osm • Merges duplicate nodes
to read the files tagging and objects • Adjustable threshold for
distance

preOutputTransform() Output XML
• A user-defined filtering • Write to a .osm file that
step, not commonly used can be opened in JOSM

Code flow
8

distance


Code flow
9

distance


Layer processing
10

filterLayer() Reproject filterFeature()
• Allows layers to be dropped • Projects the layer into • Allows features to be
• Allows for the creation of new EPSG:4326 removed
fields
• e.g. a field that indicates the
layer of a feature for later

Reproject Convert to OSM filterTags() filterFeaturePost()
• Projects the feature into geometries • Where all the magic occurs • A user-defined filtering step,
EPSG:4326 • Creates nodes and ways not commonly used
• Only creates multipolygons if
necessary

Layer processing
11

fields

• Projects the feature into geometries • Where all the magic occurs • A user-defined filtering
EPSG:4326 • Creates nodes and ways step, not commonly used
necessary

Layer processing
12

fields

necessary

Layer processing
13

fields

necessary

Layer processing
14

fields

necessary

Layer processing
15

fields

necessary

Layer processing
16

fields

necessary

Layer processing
17

fields

necessary

Layer processing
18

fields

necessary

Code flow
19

distance


Code flow
20

distance


Code flow
21

distance


Surrey case study
22

 Shapefile fields similar to other government GIS
sources
 Fields or values periodically change with no notice
 58 layers in 7 zip files
 Not counting orthos and LIDAR-derived contours
 153 MB compressed, 1.7 GB uncompressed
 Covers 187 km2
 Too much data to write conversions for without a
method

Reduce the amount of data
25

 ogr2osm will happily turn out a gigabyte .osm but
good luck opening it
 Use ogr2ogr -spat to trim the input files down
 Converting from some formats to shapefiles will
truncate field names
 Can use .gdb when coming from a format with long field
names and layers
 -spat wants coordinates in layer coordinate system
 Use gdaltransform to turn latitude/longitude into desired
coordinates

Drop layers
26

def filterLayer(layer):
 Use the layer layername = layer.GetName()
if layername in ('WBD_HU2', 'WBD_HU4', 'WBD_HU6'):
translation (-t layer) return

and see what layers if layername not in ('NHDArea', 'NHDAreaEventFC'):
print 'Unknown layer ' + layer.GetName()
should be dropped field = ogr.FieldDefn('__LAYER', ogr.OFTString)
field.SetWidth(len(layername))
 Most multi-layer layer.CreateField(field)

sources have layers for j in range(layer.GetFeatureCount()):
ogrfeature = layer.GetNextFeature()

that should not be ogrfeature.SetField('__LAYER', layername)
layer.SetFeature(ogrfeature)

imported layer.ResetReading()
return layer

 In the case of the
Surrey data filtering is
done in the script that
downloads the data

Writing a good filterTags(attrs)
27

 When testing you def filterTags(attrs):
if not attrs: return
tags = {}
want unknown fields if '__LAYER' in attrs and attrs['__LAYER'] ==

to be kept 'wtrHydrantsSHP':
# Delete the warranty date
if 'WARR_DATE' in attrs: del attrs['WARR_DATE']

 Delete items from if 'HYDRANT_NO' in attrs:
tags['ref'] = attrs['HYDRANT_NO'].strip()
attrs as you convert del attrs['HYDRANT_NO']
elif '__LAYER' in attrs and attrs['__LAYER'] ==

them to OSM tags 'trnRoadCentrelinesSHP':
# ... More logic ...

 Delete fields which for k,v in attrs.iteritems():
if v.strip() != '' and not k in tags:

shouldn’t be tags[k]=v

return tags
converted to an OSM
tag

What not to include
28

 Duplications of geodata
 SHAPE_AREA, SHAPE_LENGTH, latitude and
longitude
 Unnecessary meta-data
 e.g. username of the last person in the GIS
department to edit the object
 A single object ID can be useful but generally isn’t

 A good translation will often drop more than it
includes

Identify the main field
29

COMMENTS LC_COST RIGHTTO
 Convert to .osm with no CONDDATE LEFTFROM ROADCODE
CONDTN LEFTTO ROAD_NAME
translation DATECLOSED LEGACYID ROW_WIDTH
DATECONST LOCATION SNW_RTEZON
 View statistics about DESIGNTN
DISR_ROUTE
MATERIAL
MRN
SPEED
STATUS
tags FAC_ID
GCNAME
NO_LANE
OWNER
STR_ROUTE
TRK_ROUTE
 Easiest way is to open GCPREDIR
GCROADS
PAV_DATE
PROJ_NO
WAR_DATE
WTR_PRIOR
in JOSM, select GCSUFDIR RC_TYPE WTR_VEHCL
GCTYPE RC_TYPE2 YR
‐untagged, select the GIS_ES RD_CLASS YTD_COST
GREENWAY RIGHTFROM
tags, paste into a text
editor NOT INCLUDED IN ROADS TRANSLATION
NOT INCLUDED IN ANY TRANSLATION
MAIN FIELD
 Need to look at a large
area for this

The main field
30

 A numeric field and a text RC_TYPE RC_TYPE2 Count Tagging

field in this case 0 Road 11375 highway=?

 Don’t trust field 1 Frontage
Road
38 highway=residential

descriptions when writing 2 Highway highway=motorway_link
54
OSM tagging Interchange

3 Street Lane 20 highway=service
 Always verify!
4 Access Lane highway=?
 Access Lane would be 1442
highway=service from the 5 Railway 28 railway=rail

description but this would
be wrong
 Use imagery, surveys or
other sources

Looking at a value in more detail
31

 Should be carried out RD_CLAS highway= Coun
for each value, even if S t
you think you’re sure Local residential 8284
on the tagging Major tertiary 1350
Collector
 Look at all tags for primary
just those matching Arterial 1583
secondary
the field value tertiary

Provincial motorway 156
 In this case search in primary
JOSM for Highway
RC_TYPE2="Road" Translink unclassified 1

Even more detail
32

 Gets very close to MRN highway= Count
OSM tagging practice Yes secondary 504
locally No tertiary 1079
 Loss of information
with Arterial MRN=No
and Major Collector
both mapping to
tertiary
 Does this matter in
this case? No, road
classifications require
some judgment

Dropping objects
33

def filterFeature(ogrfeature, fieldNames, reproject):
 You may come across if not ogrfeature: return

objects that you index = ogrfeature.GetFieldIndex('STATUS')
if index >= 0 and ogrfeature.GetField(index) in
('History', 'For Construction', 'Proposed'):

shouldn’t add to OSM return None

return ogrfeature

 In this case there are
“paper roads” in the
data
 Use filterFeature() to
remove these

Putting it all together
34

def filterLayer(layer):
layername = layer.GetName()

field = ogr.FieldDefn('__LAYER', ogr.OFTString)
 Code presented is a
field.SetWidth(len(layername))
layer.CreateField(field)

for j in range(layer.GetFeatureCount()):
simplification and
ogrfeature = layer.GetNextFeature()
ogrfeature.SetField('__LAYER', layername)
layer.SetFeature(ogrfeature)
does not deal with
layer.ResetReading()
return layer all fields
def filterFeature(ogrfeature, fieldNames, reproject):
if not ogrfeature: return
 Filter features and
index = ogrfeature.GetFieldIndex('STATUS')
if index >= 0 and ogrfeature.GetField(index) in
('History', 'For Construction', 'Proposed'):
return None
layers
return ogrfeature

Putting it all together
35

def filterTags(attrs):
if not attrs: return elif 'RD_CLASS' in attrs and attrs['RD_CLASS'] == 'Provincial Highway':
tags = {} # Special-case motorways
if 'ROAD_NAME' in attrs and attrs['ROAD_NAME'] in
if '__LAYER' in attrs and attrs['__LAYER'] == ('No 1 Hwy', 'No 99 Hwy'):
'trnRoadCentrelinesSHP': tags['highway'] = 'motorway'
if 'COMMENTS' in attrs: del attrs['COMMENTS'] else:
if 'DATECLOSED' in attrs: del attrs['DATECLOSED'] tags['highway'] = 'primary'
# Lots more to delete del attrs['RD_CLASS']
elif 'RD_CLASS' in attrs and attrs['RD_CLASS'] == 'Translink':
if 'NO_LANE' in attrs: tags['highway'] = 'unclassified'
tags['lanes'] = attrs['NO_LANE'].strip() del attrs['RD_CLASS']
del attrs['NO_LANE'] else:
l.error('trnRoadCentrelinesSHP RC_TYPE=0 logic fell through')
if 'RC_TYPE' in attrs and attrs['RC_TYPE'].strip() == '0': # Normal roads tags['fixme'] = 'yes'
del attrs['RC_TYPE'] tags['highway'] = 'road'
if 'RC_TYPE2' in attrs: del attrs['RC_TYPE2'] elif 'RC_TYPE' in attrs and attrs['RC_TYPE'].strip() == '1':
if 'RD_CLASS' in attrs and attrs['RD_CLASS'] == 'Local': # More logic
tags['highway'] = 'residential'
del attrs['RD_CLASS'] elif '__LAYER' in attrs and attrs['__LAYER'] == 'trnTrafficSignalsSHP':
elif 'RD_CLASS' in attrs and attrs['RD_CLASS'] == 'Major Collector': # More logic
tags['highway'] = 'tertiary'
del attrs['RD_CLASS'] for k,v in attrs.iteritems():
elif 'RD_CLASS' in attrs and attrs['RD_CLASS'] == 'Arterial': if v.strip() != '' and not k in tags:
if 'ROAD_NAME' in attrs and attrs['ROAD_NAME'] in tags[k]=v
('King George Blvd', 'Fraser Hwy'):
tags['highway'] = 'primary' return tags
else:
if 'MRN' in attrs and attrs['MRN'] == 'Yes':
tags['highway'] = 'secondary'
else:
tags['highway'] = 'tertiary'
del attrs['RD_CLASS']

Ogr2osm presentation

Recommandé

Recommandé

Contenu connexe

Similaire à Ogr2osm presentation

Similaire à Ogr2osm presentation (20)

Dernier

Dernier (20)

Ogr2osm presentation

Notes de l'éditeur