Leveraging Data Analytics in the Cloud to Support Data-Driven Decisions

© 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Neil Mackin, Technical Business Development Manager
Data and Analytics, AI/ML
Wednesday 18th September
Data Lifecycle –
Leveraging Data Analytics in the Cloud to Data Driven
Decisions

Driving Insights:
Deliver decision-makers the
insights to transform an
organization by identifying unmet
needs within the organization or
by optimizing operational
processes
Questions to ask:
What business question is being answered?
Does the data support answering them?
Who are the users driving the insights?
What skills do those users have?

Business needs come in various forms:
• Present actionable information and reporting to
executives and managers
• Combined heterogeneous datasets together to be
able to answer additional questions
• Query and Investigate your datasets
• Understand what’s happening in the business now
• Exploit AI/ML to predict what happens next

Analytics solutions
Data
Warehousing
Big Data
Processing
Interactive
Query
Operational
Analytics
Real time
Analytics
Analytics

Data Warehousing
Data
Warehousing
Big Data
Processing
Interactive
Query
Operational
Analytics
Real time
Analytics
Present actionable information and
reporting to executives and managers
Analytics

Amazon Redshift – data warehousing
Fast, powerful, simple, and fully managed data warehouse at 1/10th the cost
Massively parallel, scale from gigabytes to petabytes
Fast at scale
Columnar storage
technology to improve I/O
efficiency and scale query
performance
$
Inexpensive
As low as $1,000 per
terabyte per year, 1/10 the
cost of traditional data
warehouse solutions; start
at $0.25 per hour
Open file formats Secure
Audit everything; encrypt
data end-to-end;
extensive certification and
compliance
Analyze optimized data
formats on the latest SSD,
and all open data formats in
Amazon S3
Analytics

Amazon Redshift Data Warehouse
Relational data
Gigabytes to petabytes scale
Reporting and analysis
Schema defined prior to data load
AWS
Glue ETL
On Prem
Amazon
QuickSight
Existing or new
BI tool
Redshift
COPY
Analytics

Complementary to EDW (not replacement) Data lake can be source for EDW
Schema on read (no predefined schemas) Schema on write (predefined schemas)
Structured/semi-structured/Unstructured data Structured data only
Fast ingestion of new data/content Time consuming to introduce new content
Data Science + Prediction/Advanced Analytics + BI use
cases
BI use cases
Data at low level of detail/granularity Data at summary/aggregated level of detail
Loosely defined SLAs Tight SLAs (production schedules)
Flexibility in tools (open source/tools for advanced
analytics)
Limited flexibility in tools (SQL only)
Elastic storage and compute capacity – decoupled
Explicitly sized environments, compute and storage
scaled in linearly
A Data Lake is not an Enterprise Data Warehouse
Data Lake EDW
Analytics

Amazon Redshift Spectrum
E x t e n d t h e d a t a w a r e h o u s e t o e x a b y t e s o f d a t a i n A m a z o n S 3 d a t a l a k e
Amazon S3
Data Lake
Amazon
Redshift data
Amazon Redshift
Spectrum
query engine
Exabyte Redshift SQL queries against Amazon S3
Join data across Amazon Redshift and Amazon S3
Scale compute and storage separately
Stable query performance and unlimited concurrency
CSV, ORC, Grok, Avro, & Parquet data formats
Pay only for the amount of data scanned
Analytics

Data Lakes extend the
traditional data warehouse
Data warehouse
Business intelligence
OLTP ERP CRM LOB
• Relational and nonrelational data
• TBs–EBs scale
• Diverse analytical engines
• Low-cost storage & analytics
Devices Web Sensors Social
Data lake
Big data processing,
real-time, machine learning
Analytics

Visual insights for everyone
with Amazon QuickSight
Pay only for what you use
Scale to tens of thousands of users
Embedded analytics
Build end-to-end BI solutions
Visualization

Create beautiful,
interactive dashboards.
• Add rich interactivity like filters, drill downs,
zooming, and more
• Blazing fast navigation
• Accessible on any device
• Data Refresh
• Publish to everyone with a click

Insights Delivered to Your Inbox
QuickSight lets you send report snapshots directly to your users inbox.
• Schedule email reports on a daily,
weekly, or monthly basis
• Sent to individual users or groups
• Users can opt out of any report so
they can focus on what’s important.
• Uses Pay-per-Session Pricing

Introducing ML Insights
Cutting edge ML tools that automatically discover powerful insights for your users.
• ML powered Anomaly Detection
• ML Powered Forecasting
• Auto-generated natural language
narratives and summaries.

Big Data Processing
Data
Warehousing
Big Data
Processing
Interactive
Query
Operational
Analytics
Real time
Analytics
Combined heterogeneous datasets together to
be able to answer additional questions
Analytics

Amazon EMR – big data processing
Analytics and ML at scale
Run other popular distributed frameworks such as Apache Spark, HBase, Presto, and
Flink, and many others
Enterprise-grade security
$
Latest versions
Updated with the latest
open source frameworks
within 30 days of release
Low cost
Flexible billing with per-
second billing, Amazon
EC2 Spot, Reserved
Instances, and Auto
Scaling to reduce costs
50%-80%
Amazon S3 storage
Process data directly in
the Amazon S3 data lake
securely with high
performance using the
EMRFS connector
Easy
Launch fully managed
Hadoop & Spark in minutes;
no cluster setup, node
provisioning, cluster tuning
Data Lake
100110000100101011100
1010101110010101000
00111100101100101
010001100001
Analytics

Interactive Query
Data
Warehousing
Big Data
Processing
Interactive
Query
Operational
Analytics
Real time
Analytics
Query and Investigate your datasets Analytics

Amazon Athena – interactive analysis
Interactive query service to analyze data in Amazon S3 using standard SQL
No infrastructure to set up or manage and no data to load
$
SQL
Query instantly
Zero setup cost; just
point to Amazon S3
and start querying
Pay per query
Pay only for queries run;
save 30%–90% on per-
query costs through
compression
Open
ANSI SQL interface,
JDBC/ODBC drivers, multiple
formats, compression types,
and complex joins and data
types
Easy
Serverless: zero
infrastructure, zero
administration
Integrated with Amazon
QuickSight
Analytics

Operational Analytics
Data
Warehousing
Big Data
Processing
Interactive
Query
Operational
Analytics
Real time
Analytics
Present actionable information and
reporting to executives and managers
Analytics

Operational analytics for logs and search
with Amazon Elasticsearch Service
Fully managed; deploy
production-ready cluster
in minutes
Direct access to Elasticsearch
open-source APIs, Logstash,
and Kibana
Amazon VPC support; at-rest
and in-transit encryption
Easily scale up and down
Analytics

Machine Learning solutions
AI Services ML Services
ML Frameworks and
Infrastructure

22© 2019 Amazon Web Services, Inc. or its affiliates. All rights reserved |
40% of digital transformation initiatives
supported by AI in 2019
—IDC 2018
InnovationDecision
making
Customer
experience
C E N T E R P I E C E F O R D I G I T A L T R A N S F O R M A T I O N
Business
operations
Competitive
advantage

Our mission at AWS
Put machine learning in the
hands of every developer

W H Y A W S F O R M L ?
200 new features and services
launched this last year alone
Unmatched flexibility
Broadest and
deepest set of AI
and ML services
70% cost reduction
in data-labeling
10x faster performance
75% lower inference cost
Accelerate your
adoption of ML
with SageMaker
Built on the most
comprehensive cloud
platform optimized for ML
AWS holds the top spots
on Stanford’s benchmark,
for fastest training time, lowest
cost, lowest inference latency

FRAMEWORKS INTERFACES INFRASTRUCTURE
AI Services
Broadest and deepest set of capabilities
T H E A W S M L S T A C K
VISION SPEECH LANGUAGE CHATBOTS FORECASTING RECOMMENDATIONS
ML Services
ML Frameworks + Infrastructure
P O L L Y T R A N S C R I B E T R A N S L A T E C O M P R E H E N D L E X F O R E C A S TR E K O G N I T I O N
I M A G E
R E K O G N I T I O N
V I D E O
T E X T R A C T P E R S O N A L I Z E
Ground Truth Notebooks Algorithms + Marketplace Reinforcement Learning Training Optimization Deployment HostingAmazon SageMaker
F P G A SE C 2 P 3
& P 3 D N
E C 2 G 4 E C 2 C 5 I N F E R E N T I AG R E E N G R A S S E L A S T I C
I N F E R E N C E

AI Services
ML Services
I M A G E
V I D E O
F P G A SE C 2 P 3
& P 3 D N
I N F E R E N C E

Bringing machine learning to all developers
A M A Z O N S A G E M A K E R
Collect and
prepare
training data
Choose and
optimize your
ML algorithm
Set up and manage
environments
for training
Train and
tune model
(trial and error)
Deploy
model in
production
Scale and manage
the production
environment

Collect and
prepare
training data
Choose and
optimize your
ML algorithm
Set up and manage
environments
for training
Train and
tune model
(trial and error)
Deploy
model in
production
Scale and manage
the production
environment
Pre-built
notebooks for
common problems

Collect and
prepare
training data
Choose and
optimize your
ML algorithm
Pre-built
notebooks for
common problems
Built-in, high
performance
algorithms
• K-Means Clustering
• Principal Component Analysis
• Neural Topic Modelling
• Factorization Machines
• Linear Learner (Regression)
• BlazingText
• Reinforcement learning
• XGBoost
• Topic Modeling (LDA)
• Image Classification
• Seq2Seq
• Linear Learner (Classification)
• DeepAR Forecasting

Collect and
prepare
training data
Choose and
optimize your
ML algorithm
Set up and manage
environments
for training
Train and
tune model
(trial and error)
Deploy
model in
production
Scale and manage
the production
environment
Pre-built
notebooks for
common problems
Built-in, high
performance
algorithms
One-click
training

Collect and
prepare
training data
Choose and
optimize your
ML algorithm
Set up and manage
environments
for training
Train and
tune model
(trial and error)
Deploy
model in
production
Scale and manage
the production
environment
Pre-built
notebooks for
common problems
Built-in, high
performance
algorithms
One-click
training Optimization

Collect and
prepare
training data
Choose and
optimize your
ML algorithm
Set up and manage
environments
for training
Train and
tune model
(trial and error)
Deploy
model in
production
Scale and manage
the production
environment
Pre-built
notebooks for
common problems
Built-in, high
performance
algorithms
One-click
One-click
deployment

Collect and
prepare
training data
Choose and
optimize your
ML algorithm
Set up and manage
environments
for training
Train and
tune model
(trial and error)
Deploy
model in
production
Scale and manage
the production
environment
Pre-built
notebooks for
common problems
Built-in, high
performance
algorithms
One-click
One-click
deployment
Fully managed with
auto-scaling, health checks,
automatic handling
of node failures,
and security checks

34© 2019 Amazon Web Services, Inc. or its affiliates. All rights reserved | 34© 2019 Amazon Web Services, Inc. or its affiliates. All rights reserved |
Fueling product
innovation
Using Amazon SageMaker, Intuit developed ML
models that can pull a year’s worth of bank
transactions to find deductible business
expenses for customers. Using SageMaker,
Intuit reduced machine learning deployment
time by 90%, from 6 months to 1 week.

Driving better
healthcare outcomes
Using Amazon SageMaker, GE Healthcare
developed an ML model that can learn from
thousands of medical scans to detect anomalies
more accurately and efficiently, allowing
radiologists to prioritize patients needing
immediate attention.

Increasing customer
engagement
Using Amazon SageMaker, Tinder analyzes
millions of match requests a minute, billions
of swipes a day, across more than 190 countries
to make the perfect match. With Amazon
Rekognition, Tinder creates tags to highlight
photos, resulting in 20% increase in engagement.

AI Services
ML Services
I M A G E
V I D E O
F P G A SE C 2 P 3
& P 3 D N
I N F E R E N C E

Use AI services to strengthen safety and security
P U T M L T O W O R K F O R Y O U R B U S I N E S S
accurate facial analysis | identity protection | metadata extraction
REKOGNITION
IMAGE
COMPREHEND &
COMPREHEND MEDICAL
REKOGNITION
VIDEO

Real-time
identity verification
Aella Credit uses Amazon Rekognition to
analyze images to verify an individual’s identity
in real-time without human intervention,
allowing it to provide instant loans to eligible
customers through its mobile app.

Automate media workflows to reduce costs and monetize content
P U T M L T O W O R K F O R Y O U R B U S I N E S S
content moderation | contextual ad insertion | searchable media library
custom facial recognition | multi-language metadata search
REKOGNITION
IMAGE
REKOGNITION
VIDEO
COMPREHEND TRANSCRIBE TRANSLATE TEXTRACT

Scaling video
indexing
C-SPAN uses Amazon Rekognition to
automatically index video news footage
for search. With Rekognition, C-SPAN
reduced indexing time per video from 1
hour to 20 minutes and uploaded 97,000
images in under 2 hours.

AI Services
ML Services
I M A G E
V I D E O
F P G A SE C 2 P 3
& P 3 D N
I N F E R E N C E

Analytics solutions
Data
Warehousing
Big Data
Processing
Interactive
Query
Operational
Analytics
Real time
Analytics
Machine Learning solutions
AI Services ML Services
ML Frameworks and
Infrastructure

Getting Started:
• Start Small, build upon successes
• Use MVP principles incrementally building
• Build Loosely/De-coupled solutions
• Pick the right tool for the right job
• Based on business question
• Users
• Data
• Leverage Managed/Serverless solutions

Questions?

Leveraging Data Analytics in the Cloud to Support Data-Driven Decisions

Recommandé

Recommandé

Contenu connexe

Tendances

Tendances (20)

Similaire à Leveraging Data Analytics in the Cloud to Support Data-Driven Decisions

Similaire à Leveraging Data Analytics in the Cloud to Support Data-Driven Decisions (20)

Plus de Amazon Web Services

Plus de Amazon Web Services (20)

Leveraging Data Analytics in the Cloud to Support Data-Driven Decisions