Facing challenges with data management in the cloud environment? Tackling issues like inefficient data management, integration difficulties, scalability constraints, and security concerns? You're not alone. In this Part 2 of our database webinar series, we'll offer you the tools to overcome these challenges, streamline workflows, boost performance, and implement industry-leading cloud data management practices -- fueling your organization to extract maximum value from your data. In this hour-long interactive session, our experts will walk you through: The different formats and benefits of using cloud-native databases with FME Strategies for deploying FME in cloud environments to ensure seamless integration and scalability Best practices for managing and scaling data effectively in a cloud environment Exclusive tips on data processing and transformation specifically for cloud-native databases To cap it off, our experts will host a live Q&A session to answer any questions you may have. Don't let this opportunity slip by to optimize your cloud-native data workflows. Register now and secure your place at the forefront of data management innovation. Your data's untapped potential awaits.

1. Mastering Data
Management: Leveraging
FME for Cloud Native
Databases

2. Steve
MacCabe
Sr. Technical Support
Specialist
Safe Software
Trent
Kading
Technical Support
Specialist
Safe Software

3. Agenda
1 Introduction
2 FME and Cloud Native Databases
3 Optimizing Workflow Efficiency
4 Deploying FME in Cloud Environments
5 Best Practices for Managing Big Data with FME
6 Real-world example migrating 150,000,000 records
7 Q&A Session
Agenda

4. Welcome to Livestorm.
A few ways to engage with us during the webinar:
Audio issues? Click this for 4 simple
troubleshooting steps.

5. How to download slides
1. Hover over the
slide deck in the
webinar room
2. Click this button

6. Poll:
Which cloud databases are
you working with?

7. 1
Introduction

8. Effectively manage and
optimize your big data
workflows in cloud native
database with FME

9. Flash
Demo

10. Improving the Cloud
database experience

11. Differences between on-prem vs Cloud databases
Common frustrations people face with this topic are:
● Data Integration complexity
● Security
● Performance
● Lack of Best Practices

12. Poll:
What are the biggest
challenges you face with
working with cloud databases?

13. 2
Safe, FME &
Cloud Native
Databases

14. 29+
27K+
128
190
20K+
years of solving data
challenges
FME Community
members
countries with
FME customers
organizations worldwide
global partners with
FME services
29+
29K+
128
140+
25K+
years of solving data
challenges
FME Community
members
countries with
FME customers
organizations worldwide
global partners with
FME services
200K+
users worldwide
Safe & FME

15. One platform, two technologies
FME Form FME Flow
Build and run data workflows Automate data workflows
FME Flow Hosted
Safe Software managed instance
fme.safe.com/platform
FME Enterprise Integration Platform

16. Number
of
supported
data
types
in
FME
1995 2000 2005 2010 2015 2020 2023…
10
100
300
500
GIS
CAD
Database
XML
Raster
3D
BIM
Web
Point
Cloud
Cloud
Big
Data
IOT
Gaming
BI
Indoor
Mapping
AR/VR
Generative
AI
Cloud
Native
Tabular
Unrivalled Data Support

17. Supported Cloud Databases
… to name a few

18. Slide Title
Reading from
MongoDB
Goal Block Key
MongoDB Database Reading/Writing
Result
Performance
blocks
Data Medium Improved
workspace times
that avoid the
database

19. Demo

20. Importance of Cloud
Native Databases
● Purpose built
● Scalability
● Risk Management
● Cost Reduction

21. 3
Optimizing
Workflow
Efficiency

22. Techniques for maximizing
FME's performance with big
data
● Let the database do the work
● Proximity to data
● Be creative
● Let the database do the work
1 2 3
4 5 6
7 8 9
SQL

23. Slide Title
Reading from
Snowflake
Goal Block Key
Cloud Database Reading
Result
Performance Leverage the
database to
improve reading
performance
Faster you can
pull data down,
faster you can
work with it

24. Demo

25. Results
Source
Location
FME Location Bucket
Location
Method Total Time
US West US East N/A Snowflake Reader 49m 39s
US West US East US West SQL to S3 Bucket + CSV
Reader (300 files)
58m
US West US East US West SQL to S3 Bucket + CSV
Reader (8 files)
41m
Scenario
● Snowflake Reading
● 150 Million Records

26. Strategies to improve
workflow efficiency and
reduce processing time
● Can you avoid writing directly to
database?
● Does the database support other
import/export methods

27. Slide Title
Writing to
Snowflake
Goal Block Key
Cloud Database Writing
Result
Performance Leverage the
database to
improve Writing
performance
Allow Database
to process the
data most
efficiently to
reduce
downtime

28. Demo

29. Results
Source
Location
FME Location Bucket
Location
Method Total Time
US West US East N/A Snowflake Writer 4h 40m
US West US East US West CSV Writer + S3
Connector + SQL(8 files)
11m 20s
US West US East US West CSV Writer(GZ) + S3
Connector + SQL (300
files batched)
1m 22s(+8m on
database side for
processing)
Scenario
● Snowflake Writing
● 150 Million Records

30. Addressing security concerns when working with
FME and big data
● TLS/SSL Considerations
● Sensitive Data
● Where’s your Data lingering

31. 4
Deploying
FME in Cloud
Environments

32. Options for deploying FME in cloud-based setups

33. Integration considerations for
different cloud platforms
● Cloud Storage like S3 Bucket, Azure
Storage, etc
● Performance of CPU/RAM/IO bandwidth
● Performance of Disks (SSD)

34. Ensuring seamless connectivity and scalability
● Scaling FME for more throughput
● Consider network topology
● Virus scanners, file scanners,
● How close are you to the data

35. Slide Title
Maximize
Performance
Of Workflow
Goal Block Key
Maximizing Data Throughput Using Automations
Result
Don’t
Understand all
the Systems.
What can we
change in the
Workflow?
Massive
Performance
Gains

36. Demo

37. FME Flow Results
Test
Source
Location
FME
Location
Bucket
Location
Target
Location
System Count | Engine Count | CPU(RAM) Total
Time
1 US East US West US West US West 1 System | 15 Engines | 8 (32G) CPU 2h 48m
View with Group ID
● 50 Million Records
● Oracle Cloud to S3 Bucket to Snowflake
2 US East US West US West US West 2 System | 20 Engines | 8 (32G) CPU/SYS 2h 27m
View with Group ID (add more engines)
3 US East US West US West US West 1 System | 10 Engines | 8 (32G) CPU 2h 30m
Materialized View with Group ID indexed
4 US East US East US West US West 1 System | 14 Engines | 16 (64G) CPU 0h 21m
Materialized View with Group ID indexed and FME Close to DB
5 US West US West US West US West 2 System | 20 Engines | 8 (32G) CPU/SYS 1h 28m
Materialized View with Group ID indexed and FME Close to DB RDS

38. 5
Best Practices
for Managing Big
Data with FME

39. Key recommendations for effectively managing
and scaling big data workflows
● Data medium
● Breaking up the workflow
● Leverage Flow/Server and CPU engines
● Consider Network topology/architecture

40. Data processing and transformation tips
● Data delivery method
● Simplify the data
● Reduce on-the-fly transformation

41. Performance optimization techniques specific to
cloud native databases
● Leverage the database
● Reading
● FME Flow Architecture

42. Slide Title
Oracle to
Snowflake
Goal Block Key
Oracle Cloud to Snowflake
Result
Oracle Cloud to
Snowflake, 2
Cloud native
databases
Think outside the
box with FME
Drastically
improved
performance

43. Demo

44. 7
Conclusion

45. Be Creative
when moving
Big Data with
FME.

46. 8
Resources

47. Resources
● Let the Database Do the
Work tutorial
● Let Your Database Do the
Work Webinar

48. Get our Ebook
Spatial Data for the
Enterprise
fme.ly/gzc
Guided learning experiences
at your fingertips
community.safe.com
/s/academy
FME Academy

49. Check out how-to’s & demos
in the knowledge base
community.safe.com
/s/knowledge-base
Knowledge Base Webinars
Upcoming & on-demand
webinars
safe.com/webinars

50. 9
Next Steps

51. We’d love to help you get
started.
Get in touch with us at
info@safe.com
Experience the FME Accelerator
Contact Us
Unlock the power of your
data in only 90 minutes
Register for free at
fme.safe.com/accelerator

52. Our largest
FME user
conference
yet.
Sept 5-7 | 100+ sessions
co-hosted by Safe Software & con terra

53. ClaimYour Community Badge
● Get community badges for watching
webinars!
● fme.ly/WebinarBadge
● Today’s code: AGMFW
Join the Community today!

54. 10
Q&A

55. ThankYou
Recap of Next Steps
1 Join the FME Community
2 Contact us
3 Experience the FME Accelerator
Please fill out our
webinar survey