The document provides an overview of new transformers in FME that help optimize GIS workflows by simplifying attribute and data validation tasks. The AttributeManager allows consolidated handling of attribute tasks like creation, renaming, copying, and validation. The AttributeValidator performs validation tests on attributes and outputs validation messages. The FeatureWriter enables writing features in workflows to avoid chaining workspaces and support post-processing like notifications and automation.

1. How to Optimize
GIS Workflows
With Mark Stoakes & Steve MacCabe

2. Who are we
anyway?

3. Connect Virtually Any
Format with FME

4. FME & GIS can do
amazing things
Here is a taste of what’s
possible.
Manage your attributes
Validate your attributes
Distribute a Validation Report
Automate the process

5. Managing Attributes

6. Attribute Management
Why another attribute handling transformer?
➢ Because there are already too many
… plus we needed a couple other operations
Goal: consolidate the attribute management
transformers
Doomed transformers:
AttributeCreator
AttributeCopier
AttributeRenamer
ExpressionEvaluator
StringConcatenator
AttributeKeeper
AttributeRemover
Plus…
…many string operations:
case change
trim/pad
substring/regex
One transformer to rule all your
attributes

7. ● Create attributes
● Rename attributes
● Copy attributes
● Remove attributes
● Set attribute values
● Re-order attributes
● Concatenate attributes (Text Editor)
● Arithmetic expressions (Arithmetic Editor)
● Set conditional values (Conditional Value)
● Reference earlier attributes
AttributeManager

8. AttributeManager creates an
opportunity for more focused and
simplified attribute management.
#ThanksFME2016

9. Let’s Go - Demo!
AttributeManger:
Lets you organize
your attributes in
your workflows

10. Attribute Validation

11. Attribute Validation
– why?
If you distribute poor quality data
then you've amplified the data
issue.
If you distribute good quality data
you've amplified the benefits.
Data validation is key to many
FME workflows…

12. Attribute Validation – Standards
Bunch of ISO standards
– ISO19xxx
Mostly replaced by
- ISO 19157
Also CAD & Data Model
Standards

13. Attribute Validation – FME Tools
Attribute Validation – Key Tools
● Tester and TestFilter
● AttributeCreator (conditional tests)
● AttributeClassifier
● AttributeRangeFilter & AttributeFilter
● NullAttributeMapper
● DuplicateRemover
● StringSearcher
● StatisticsCalculator

14. AttributeValidator...
Goals:
Simplify validation workflows
(Reduce number of transformers)
Be more confident about the data you’re
loading into a destination dataset
...say goodbye to bad data
#ThanksFME2016

15. Attribute Validation
Some of the validation tests available:
➢ Attribute type (integer, float, char, xml, json, etc.)
➢ In – either a list or range – good for domain validation
➢ Regular Expressions
➢ Unique
➢ Not Null
➢ …and more…

16. Attribute Validation Output
Output is:
➢ Summary Attribute:
_fme_validation_message
➢ List of all messages:
_fme_validation_message_list{}
Attribute 'CodePrefix' with value 'ABE' fails check for Matches Regular Expression
'[ABCD]{3}'
{0} Attribute 'CodePrefix' with value 'ABE' fails check for Matches Regular Expression
'[ABCD]{3}'
{1} Attribute 'num_measures' with value '12' fails check for in Range '[0,10]'
{2} Attribute 'CodePrefix' with value 'ABE' fails check for in 'ABC,ABD,TXU,TXV'

17. Let’s Go - Demo!
AttributeValidator:
Lets you scrutinize
your attributes and
simplify your data
validation
workflows

18. Q&A
Chat your questions to us

19. Writing Features
...a different approach

20. Traditional vs. state-of-the-art...
Write data
with a writer
Write data
with a
FeatureWriter

21. Write the conclusion to your data process
Write data in the middle of
workflows to avoid chaining
workspaces.
Write the summary of your
data output.
#ThanksFME2016
Use the FeatureWriter to
allow post-processing
freedom.
#FreeYourData

22. Some Easy Wins:
• Create summary reports
– Email an attachment
• Copy data after writing
– FTP results
– Load into Dropbox

23. Automation Workflows
• Quality checking
• Database operations
• Database validation
• FME Server Notification

24. Transformation
...transform your data
A new pattern for database workflows
Transform your database workflows
SQLCreator
...drop constraints
FeatureReader
...read your data
Transformation
...transform your data
FeatureWriter
...write to database
SQLExecutor
...enable constraints
SQLExecutor
...validate/index/unique

25. A new pattern for FME Server notification workflows
FeatureReader & FeatureWriter simplies you workspace

26. Let’s Go - Demo!
FeatureWriter:
Lets you post-
process your
output data

27. Automation

28. FME Server Overview
FME Engine workhorse and runs FME Workspaces
Automate Deliver Data Download Data
Love
FME
!
ScaleReal Time Data

29. Let FME Server do the work
Examples of things we can automate:
● Directory watcher
● Run the workspace through a notification
● Data Delivery
○ Dropbox
○ email
Let’s Go - Demo!

30. Summary

31. Take advantage
of our new
transformers.

32. Free Stuff
There are too many to list so be
sure to check out www.safe.com
for more.
Knowledge Articles
● AttributeManager
● AttributeValidator
Blog Posts
● FeatureWriter
● AttributeManager
Free FME Training
● FME Desktop Basic: March 8-9
● FME Server: March 15-16
● FME Desktop Advanced: March 22-23

33. 60+ CITIES
April - June

34. Poll Question
Which of today's tasks
do you see yourself
performing on your
GIS data?

35. Q&A
Chat your questions into us

36. Thank you!
Get a free trial of FME at www.safe.com

37. Resources: Attribute Validation
Original PR: 8945
Knowledge Center: Validate your Data's Attributes with the AttributeValidator Transformer
Webinar 2015: Data Validation Victories: Tips for Better Data Quality:
Data Validation & Quality Assurance
WWTour:FME & Data Validation: http://www.slideshare.net/SafeSoftware/fme-data-validation
WWTour 2014: Validation and Quality Control with FME & FME Server and also at SlideShare
Validating your data before loading it into a database: video by Robyn
FME 2014 UC - Colonial Pipeline: CAD Data Validation using FME
Automated Quality Control: ISo 9000:2000 Quality Measures ISO 19157/19131 Metria 2014
Berk Bayer 2014 Application of FME to Maintain Quality Control (QC) over Standardized Spatial Data
FME WWTour 2013 – Introducing Geometry Validator

38. Resources: AttributeManager
Knowledge Center: Control Your Data's Attributes with the AttributeManager Transformer