Getting the best performance out of FME is as important to us as it is to you. In this webinar, you’ll get tips from three FME experts - Mark Ireland (FME Evangelist), David Eagle (FME Certified Trainer and Professional) and Dale Lutz (Safe Software Co-founder). They’ll share easy-to-apply advice on: querying databases efficiently, making the most of FME's new multiprocessing capabilities, and simple techniques to speed up your workflows.

1. Turbocharging FME:
How to Improve the Performance of Your
FME Workspaces
Host:
Ken Bragg
Panel:
Mark Ireland, David Eagle, Dale Lutz
December 12, 2012
12/12/12

2. Host
 Ken Bragg
European Services Manager
Safe Software
Questions are Encouraged!

3. Our Panel
 Mark Ireland (iMark)
Product Evangelist, Safe Software
Killarney, Manitoba, Canada
 David A. Eagle (The Eagle)
Principal Consultant, 1Spatial Group Ltd.
Cambridge, United Kingdom
 Dale Lutz (The Founder)
Co-Founder & VP of Development
Safe Software
Vancouver, BC, Canada

4. 100 exuberant employees Secret HQ: Vancouver
Thousands of happy customers, certified
professionals, trainers, system
integrators, and value added resellers;
all around the globe!

5. FME Capabilities
 Transform Data to Use and Share
 Convert spatial data between
hundreds of formats
 Transform spatial data into the
precise data model you need
 Integrate multiple different data
types into a single data model
 Share spatial data with people
where, when and how they need it

6. FME Workbench:
Graphical Data Flow Authoring

7. FME : Our Promise
Faster than a Helicopter?
Faster than ever!

8. Turbocharge your Workspaces!

9. Poll
How Long do your
Workspaces Take?

10. iMark – Mark Ireland
Workbench Performance

11. Workspace Performance
The key to workspace performance is to
reduce the amount of data being processed.

12. Feature vs Group
Some transformers work on one feature at a time; others work on a
group of features.

13. Feature Based
Measuring individual heights...
Feature-based transformers work on one feature at a time.
They are LESS resource intensive.
5’ 8”
7”
9”

14. Group Based
Measuring average heights for each gender...
Group-based transformers work on many Average = 5’ 9”
features at a time. They are MORE resource
intensive.
5’ 7” 6’ 0” 5’ 8” 5’ 9”

15. Transformers
Some group-based transformers have settings
to turn them into feature-based
• AttributeAccumulator
• Aggregator
• Clipper
• NeighborFinder

16. First Writer
Order your writers so the one receiving the
largest amount of data is first in the list.
http://evangelism.safe.com/fmeevangelist64-2/

17. Attribute Handling
• Remove unwanted attributes
• Beware of large lists
And don’t store geometry
as an attribute!

18. Poll
Where does it hurt?

19. The Eagle – David Eagle
Working with Databases

20. Turbocharging FME:
Working with Databases
David Eagle
Principal Consultant
FME Certified Professional & Trainer
1Spatial Group Ltd
December 12, 2012

21. Learn to be a ‘Speed Reader’
 Being able to read quickly is often beneficial
 When you have a database, you often have
volume
 Read data intelligently and achieve efficiency!
 A typical approach is where FME does the
heavy lifting
1 - Read_and_clip.fmw

22. Reader Parameters
 Master FME ‘Parameters’ - become an FME Hero!
 Parameters allow you to apply control
 Enable flexible workspaces – Prompt and Run
 Make the database do the work

23. Embrace the Spatial Index
 Spatial indices are used by spatial databases
(databases which store information related to
objects in space) to optimize spatial queries.
Many conventional index types do not
efficiently handle features such as how far two
points differ, or whether points fall within a
spatial area of interest.
http://en.wikipedia.org/wiki/Spatial_database

24. Read ‘within’
 All Readers since FME 2011 have allowed you
to specify a reader bounds
 Not all formats have a true spatial index
 No performance gain but less data in memory
 Spatial databases can have a spatial index
 Create index with SQL or create with FME!
2 - Read_with_spatial_index.fmw

25. Read ‘where’ and ‘what’
 Use FME to issue a SQL Where Clause
 Be specific about the features you need
 www.w3schools.com/sql beginners resource
 Leverage the power of the parameter
‘Feature Types to Read’
 Always see what’s new
3 - Read_WhereandWhat.fmw

26. Synchronous Reading/Writing
 Mid translation reading/writing may be required
 Beneficial when you want to be specific
 Allows native format query – greater efficiency
SynchronousReading.fmw

27. Ask your friendly DBA
 Prebuild the data you want as a ‘View’
 A stored query that forms a virtual table
 More efficient and dynamic for repetitive queries
 FME treats Views like normal Tables
 Allows you simplify your Workspace
 Materialized Views (Oracle, DB2, MS SQL Server, PostgreSQL)
 Results of a Query stored on disk

28. Database Joins
 There’s lots of options for carrying out Joins (not
limited to databases)
 Joiner vs FeatureMerger
 http://evangelism.safe.com/fmeevangelist79/
 FME 2012 introduced the InlineQuerier
 Cache data to disk mid-process (SQLite)
 Write SQL against non-database formats
 http://evangelism.safe.com/fmeevangelist97/
Transformation Challenges

29. Thank You!
 For more information, contact:
 fme@1spatial.com
 +44 (0)1223 420414
 @david_eagle
 @1spatial
 Web resource:
 www.1spatial.com/fme
Over to you Dale!

30. The Founder – Dale Lutz
Multiprocessing - Faster than Ever

31. Multiprocessing: Definition

32. Multiprocessing: Definition
Multiprocessing is the use of
two or more central processing units (CPUs)
within a single computer system
- Wikipedia

33. Multiprocessing: Motivation
 Multi-Core and HyperThreaded CPUs
 Very common
 Potential to increase compute-bound
throughput

34. Multi-Processing:
Sample Performance Boosts
 3D Clipping of Point Clouds
 2 X faster
 LineJoiner and Buffer
 2 X faster
 Raster Cell Coercer/Point on Raster
 4 X faster

35. Multiprocessing & FME
 Option on Many Transformers
 Multiprocesses
Groups

36. Multiprocessing & FME

37. Multiprocessing & FME
 Option on All Custom Transformers

38. Multiprocessing: Levels Of Parallelism
 None
 Minimal
 Moderate
 Aggressive
 Extreme

39. Multiprocessing: Levels Of Parallelism
Parameter Processes
No 1 License Cap
Parallelism Base 4
Minimal Cores / 2 Professional 8
Moderate Cores Other 16
Aggressive Cores * 1.5
Extreme Cores * 2

40. Multiprocessing: Summary
 New in FME 2012
 Leverages Multi-Core and HyperThreading CPUs
 Controlled by:
 Levels of Parallelism
 Group-By
 Can give significant performance boost
 http://fmeevangelist.com/fmeevangelist94
 http://fme.ly/parallel (FMEpedia Article)

41. Multiprocessing
DEMO

42. Share Today’s Webinar
 Today’s webinar was recorded
 View our archive at:
 http://fme.ly/archive

43. Special Offer!
 Win Free Training!
 View the offerings at:
 http://safe.com/training
APRIL

44. Thank You!
 For more information contact:
 Safe Partners
 http://www.safe.com/partners/find-a-sales-partner
 Safe Sales:
 sales@safe.com
 www.FMEPedia.com
 www.safe.com/support
 Dale  David
@DaleAtSafe @david_eagle
 Mark  Ken
@FMEEvangelist @KenatSafe