When to use python in FME

1. When to use Python in FME
Dr. Jonathan Stanger
Senior Consultant – Spatial Partners Pty. Ltd.

2. FME is good for
Broad User Skill Levels
FME is good for
Maintainability
FME is good for
Portability
FME is good for
Rapid Modification

3. Why using Python is bad…
The FME transformer library is very extensive. FME does limit us, but with
good reason. It uses a simple data paradigm that aligns with most spatial
data, collections of features that each have a well defined set of attributes.
We perform operations discretely on a single feature at a time.
Sometimes this can be frustrating but remember:
Don’t reinvent the wheel! Try rephrasing the problem you are solving!

4. Where can Python be used?
● Start-up Scripts – Executes before FME transformers (even readers…)
● Shut-down Scripts – Executes after all FME transformers are finished
● Scripted Parameters – Executes on start-up and preserves the value
● PythonCreator – Has no input, Python defines all features outputted
● PythonCaller – Executes function once or calls input once per feature

5. Good examples for Python in FME

6. Smart Scripted Parameters
While the Workspace Runner is useful, in
trivial cases its not always ideal. If the
only requirement is glob style directory
search to provide file paths the workspace
runner leaves excess complexity for
deployment.
Solution: Script glob search into
parameter to return
import fme, fmeobjects, glob
# Get parameters set by user to initiate the glob
search
rootpath = fme.macroValues['rootPath']
# Glob search for file candidates
candidates = glob.glob(rootpath + '/**/data.TAB',
recursive = True)
# Return string that matches what FME expects
if candidates:
return '""' + '" "'.join(candidates) + '""'
else:
return ""
Example: Avoiding the workspace
runner

7. Front Stage File Manipulation
FME rightly treats the data schema of
some file formats quite strictly, unlike
users sometimes. This impacts MapInfo
and sometimes Excel files.
Solution: Rename/copy the file to a
standard filename
- Published Parameter (Target File)
- Private Parameter (Reader Target)
import fme, fmeobjects
from shutil import copy2
# Extract required info from macroValues
fmepath = fme.macroValues['FME_MF_DIR']
targetfile = fme.macroValues['fileName']
# Check parameter is assigned
if targetfile:
# Copy2 will automatically overwrite existing
# file. Set reader target to private parameter
# containing:
# $(FME_MF_DIR)Standard.xlsx
copy2(targetfile, fmepath + 'Standard.xlsx')
Example: File Formats playing badly

8. Inter-feature Dependant Operations
Assuming features have ID’s that are not
consecutive. If features have two
attributes, one representing the
component ahead and one the
component behind, how do we sort the
features?
Solution: Sort elements by matching
ahead ID to behind ID
Example: Paired Sorting
Sort Order 1
Behind
934AH9
Ahead
365U33
Sort Order 2
Behind
6433OE
Ahead
934AH9
Sort Order 3
Behind
18KW22
Ahead
6433OE
Sort Order 4
Behind
20554Z
Ahead
18KW22

9. Repeated(recursive) operations (N > 3 or N = ?)
FME was designed to work with tabular
not network/graph style data. As such,
recursive algorithms such as DFS or BFS
are nearly impossible to implement with
native transformers.
Solution: Gather all the data into Python
then process in memory
Think carefully about how you output!
Example: Trace network data
A
B
D E C
GFH
I

10. Non-standard Geometry Manipulation
FME offers well developed options for
direct geometry manipulation, scaling,
offsetting or rotating. Less typical
geometry manipulation can be tricky
though.
Solution: Implement custom geometry
transformation in Python
Example: Mirror around arbitrary plane
geom = feature.getGeometry()
points = line.getPoints()
x, y, z = point.getXYZ()
point.setXYZ(x, y, z)

11. Tedious Operations
For simple cases FME string manipulation
is more than sufficient. Building a string
conditionally based on 20+ attributes is
possible but very tedious and difficult to
maintain.
Solution: Python string manipulation to
efficiently build the string
import fme, fmeobjects
# Attribute to label lookup dictionary
attributeMap = {
'5_040': '5pr',
'50_040': '50pr',
'100_040': '100pr',
'200_040': '200pr',
'400_040': '400pr',
... x 25 more,
}
class FeatureProcessor(object):
def __init__(self):
pass
def input(self,feature):
# Use dictionary comprehension to conditionally build elements of label
attributes = {attributeMap[key]: feature.getAttribute(key)
for key in attributeMap.keys()
if feature.getAttribute(key) and feature.getAttribute(key) > 0}
# Join label elements to form a single string
label = ", ".join([str(int(attributes[key])) + "x" + key
for key in attributes.keys()])
feature.setAttribute('LABEL', label)
self.pyoutput(feature)
def close(self):
pass
Example: Build very complex string

12. General Rules for
Python use
● Recursion
● File Manipulation
● FME forums say its not possible

13. Hints for Python in FME
1. Logger rather than Print
2. Scripted Parameters are hard to Debug
3. Don’t import non-standard libraries
4. You can import FME libraries to external
python scripts
5. Don’t use copies of a PythonCaller

14. import fme
import fmeobjects
logger = fmeobjects.FMELogFile()
parameter = fme.macroValues['ParameterName']
logger.logMessageString("{Python Script} Parameter set: %s" % parameter)
# Template Class Interface:
# When using this class, make sure its name is set as the value
# of the 'Class or Function to Process Features' transformer parameter
class FeatureProcessor(object):
def __init__(self):
pass
def input(self, feature):
self.pyoutput(feature)
def close(self):
pass
Useful Pattern – FME Env. Dictionary

15. import fme
import fmeobjects
# Template Class Interface:
# When using this class, make sure its name is set as the value
# of the 'Class or Function to Process Features' transformer
# parameter
class FeatureProcessor(object):
def __init__(self):
self.featureList = []
def input(self, feature):
self.featureList.append(feature)
def close(self):
for feature in self.featureList:
self.pyoutput(feature)
Useful Pattern – Bulk Feature Manipulation

16. Example: Tiled Grid Viewports
PythonCaller: Consumes point and line geometry, coverts to points then divides space to provide
viewport polygons that will overlay every point in the set. Useful for creating AutoCAD viewports or
other pagination of your geometry.

17. Example: Procedural Tree Layout for Logical Networks
Start-up Python Script: Parses logical connectivity then builds GML file with generated geometry for export
via FME to other file formats such as AutoCAD to a database as a pre-calculated UI display geometry

18. THANK YOU!
Dr. Jonathan Stanger
jstanger@spatialpartners.com