Snowflake is one of the most powerful, efficient data warehouses on the market today—and we joined forces with the Snowflake team to show you how it works!
In this webinar:
- Learn how to optimize Snowflake
- Hear insider tips and tricks on how to improve performance
- Get expert insights from Craig Collier, Technical Architect from Snowflake, and Kalyan Arangam, Solution Architect from Matillion
- Find out how leading brands like Converse, Duo Security, and Pets at Home use Snowflake and Matillion ETL to make data-driven decisions
- Discover how Matillion ETL and Snowflake work together to modernize your data world
- Learn how to utilize the impressive scalability of Snowflake and Matillion
11. Snowflake + Matillion help Unite the Data Nation
Data Integration
Business Intelligence &
Analytics
Data Warehouse
Enterprise
apps
Data Sources
Corporate
Web
Mobile
IoT
22. Customer - Upside Travel
Matt Boegner
Data Engineer, Upside Travel
Scenario
Upside ingests over 3TB of raw flight/hotel
inventory on a daily basis, in addition to website,
mobile app, and marketing data streams. A lot of
data and a lot of data processing!
Negative Consequences
Existing data warehouse couldn’t keep up
Degraded performance due to concurrent loading
Expensive to maintain
After
Matillion ETL for Snowflake made their migration to
Snowflake simple and quick, and Upside was able
to actualize the benefits of Snowflake within 2
weeks, at production workloads.
“Use Snowflake if you have a cloud data
warehouse; use Matillion if you use
Snowflake.”
23. Snowflake
Flight/ Hotel data
Website data
Mobile data
Marketing data
Looker – BI &
Visualization
Better performance, mitigation of
concurrency issues and improved
scalability
• Migration to production live in under 2 weeks
• Scalability (up and down) >> Cost Efficiency
(Transform applied directly)
24. Matillion ETL for Snowflake
• 5 minute launch process on Marketplace
• 14 day Free Trial
• From $1.37 per hr
How to engage
• SA team to support PoCs, demos and training
• Support Portal
• Tutorials & Customer references on YouTube
How to Engage
<Alex>
Introduce our speakers – pass it over to them to do brief intros.
It’s all of us
Everyone who cares about working with data
Better answers to drive the business forward
Whether you are small or big, data nation is for you
Data is critical, but it’s a challenging space. Every org struggling to get value from data.
Divide: People who are responsible for the data, and the people who are using the data
Irordinate amount of time wrangling data
Different types
It’s all over the place and difficult to bring together
Data isn’t just about whats in front of you
Connecting different applications
GDPR and other regulations
Things that used to take 24 hours, now we need under a minute or less
Effectively infinite resources -
Serve every use case with unbounded compute
Connect all of your apps with limitless connectivity
Unite your data in one place with infinite storage
Basically skip this slide
Familiar! The language of analytics.
Name other products
Not a query tool
ETL can work with Snowflake, BI works with Snowflake
-The difference is that it SCALES.
A huge difference: built as a cloud saas service
You load the data, you run the queries, and the rest we take care for you
Hard to grasp…
But it just works!
Many different types of data
Complete picture requires multiple variations of data
Different environments (SQL Hadoop)
Transformation… requires effort
Machine generated data has structure! Has a schema.
-Heirarchical, repetitive aptterns
-Just changing…
-Analytics team unaware
-Can you find that schema? We find it for you. Virtual column, access with SQL with dot notation
Anyone has the rights can gain insight
Make decisions and share
Ability to allow all of your end users when they want it
-Always a food chain – who is on top?
-Don’t touch the data warehouse!
Monday morning dashboard rush
Thousands of nodes at once
With Snowflake you only pay for what you use.
Compute is separate from storage so you pay for things separately.
Snowflake is a fully elastic cloud data warehouse—scale up and down at any time.
The cost of storage is comparable to the price of Amazon S3—with a one-year contract your cost of storage is only $23 / compressed TB
In other words, you can effectively store data in the world’s most powerful data warehouse for the same price as Amazon S3.
Leverage data as a business asset
Companies need data to gain insight into their own company
-Drive revenue
-Competitive insight
-Closer partnerships
Snowflake enables any data stored in Snowflake,
Securely share with anyone
-Your account or other Snowflake customers
Your data, you have complete ownership, pay for storage
Enable other organizations access to the data through secure view
When they run queries, they do that in their own account
We don’t participate
That’s not possible!
Multi cluster shared data
At the heart of the architecture is the data itself
-Blobs within S3 (storage engine)
Immutability is a tremendous advantage.
When you load its loaded into S3 as a set of files
-As the data is brought in, we extract metadata and put it into columns and micropartions for pruning
-Don’t declare partition keys
-Tens of millions of partitions
To do work, you need a virtual warehouse.
-Cluster of compute, EC2 nodes on AWS
-They do all the processing
-Different sizes
-Multi-cluster – more than one cluster at a time. Concurrency.
-Servcices, the layer that makes this all possible
When you login online, you authenticate and it passes through services
When you issue a query, we parse your query, perform the right prune, and efficiently scan the data for you
When a multi statement transaction starts, every other query will see the data as it was before it began… until the query is done then they see the updated version
It all works because S3 is immutable!