Better Business from Exploring Ideas - Modern Data Architectures on AWS In this session you will learn how organisations are able to drive better business outcomes from products and deliver more personalised and real-time experiences to customers. We will walk through a common customer journey that shows how to quickly test ideas and get insights, built on top of data lakes, data pipelines, and sandboxes using the same platform advantages of Modern Data Architectures on AWS as some of our most prominent customers. See how to serve the needs of your business users, business analysts, and data scientists using AWS services for analytics, Big Data, and Machine Learning. Craig Stires, APAC Head of Analytics, Big Data, and AI, Amazon Web Services

1. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Craig Stires
Head of AI, Analytics, Big Data - Asia Pacific, Amazon Web Services
Better Business From Exploring Ideas
Modern Data Architectures On AWS

2. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Let’s Go On A Common
Customer Journey

3. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Meet Earecstasy, As They Move From B2B To B2C
* This case is representative of a common customer journey, but EarEcstasy isn’t an actual business
EarEcstasy manufacturers headsets. They ran
a traditional B2B business since 2005, selling
through distribution and retail channels.
2005
In 2018, they launched their first “Smart
Buds”. These wireless headsets have voice
enablement, GPS tracking, and heartrate
monitors built in, and the device syncs with
the users mobile phone via Bluetooth. The
mobile app also supports scene detection.
2018

4. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Earecstasy Needs To Answer New Questions And
Move Faster
Raymond, Head of ProductLim, Head of Finance
Which regions are the new earbuds selling well?
What is the demand forecast by product category?
What is the social sentiment about our products?
How do quality issues impact cost of production?
Can I look at supplier performance over time?
How can we reduce our inventory holding costs?

5. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
To Answer New Questions Quickly, We Look To A
Modern Data Architecture Design
Massive upfront costs
Overprovisioned capacity
Long implementation times
Pay as you go, for what you use
Decoupled pipelines and engines
Experimentation platform
Ingest/
Collect
Consume/
visualise
Store Process/
analyze
1 4
0 9
5

6. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Outcome 1:
Modernise And Consolidate

7. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Start With A Set Of Specific Questions To Answer,
Then Work Backwards To The Data Required
Lim, Head of Finance
How do quality issues impact cost of production?
Can I look at supplier performance over time?
How can we reduce our inventory holding costs?
Order History /
Returns (CRM)
Inventory /
Production (ERP)

8. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Ingest ServingData
sources
Modern Data Architecture
Insights to enhance business applications, new digital services
Transactions
ERP
Data analysts
DATA PIPELINES
Ingest/
Collect
Consume /
visualise
Store Process /
analyze
1 4
0 9
5

9. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Start Small And Iterate

10. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Ingest ServingData
sources
Modern Data Architecture
Insights to enhance business applications, new digital services
Transactions
ERP
DATA PIPELINES
Data
Lake
expdp
Data
Data analysts
Data Warehouse
Amazon Redshift
Direct Query
Amazon Athena
She asks for the SMALLEST amount of data to answer her questions.
If it isn’t good enough, she asks for another small slice to be loaded to the DATA LAKE

11. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon Redshift – Modern Data Warehousing
Fast, scalable, fully managed data warehouse at 1/10th the cost
Massively parallel, scales from gigabytes to exabytes
Queries data across your Redshift data warehouse and Amazon S3 data lake
Fast at scale
Columnar storage
technology to improve I/O
efficiency and scale query
performance
Open file formats
Analyze optimised data
formats on direct-attached
disks, and all open file
formats in S3
Cost-effective
Start at $0.25 per hour;
as low as $250-$333 per
uncompressed terabyte
per year
$
Secure
Audit everything; encrypt
data end-to-end; extensive
certification and compliance

12. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Characteristics Of A Data Lake
Future
Proof
Flexible
Access
Dive in
Anywhere
Collect
Anything

13. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Start With A Set Of Specific Questions To Answer,
Then Work Backwards To The Data Required
Raymond, Head of Product
Which regions are the new earbuds selling well?
What is the demand forecast by product category?
What is the social sentiment about our products?
Trending /
Mentions (Social)
Order History /
Returns (CRM)
NOW IN THE DATA LAKE

14. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Experiment, Validate, Then Scale

15. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Ingest ServingData
sources
Modern Data Architecture
Insights to enhance business applications, new digital services
DATA PIPELINES
Data
Lake
He first looks to the DATA LAKE, and imports only the category data he needs
He imports JUST ENOUGH data to see if the market is responding to products.
Business users
Transactions
ERP
Social media
Data
Stream
Capture
Amazon
Kinesis
Events
Amazon
QuickSight
Data Warehouse
Amazon Redshift
Stream Data
Amazon
ElasticSearch

16. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Common Data Pipeline Configuration
Raw Data
Amazon S3
Highly decoupled configurations scale better, are more fault tolerant, and cost optimised
ETL (Hadoop)
Amazon EMR
Triggered Code
Amazon Lambda
Staged Data
(Data Lake)
Amazon S3
ETL & Catalog Management
AWS Glue
Data Warehouse
Amazon Redshift
Triggered Code
Amazon Lambda

17. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Data security
and management
Encryption
Access Controls
Monitoring and Metrics
Audit Trails
Automation
Serverless Computing
Data Discovery and
Protection
Data Visualisation
Data movement
Physical Appliances
Hybrid Storage
Private Networks
File Data
WAN Acceleration
Third-party Applications
Streaming Data
Complete Set Of Building Blocks
FileBlock
Object Archival
Storage types

18. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Ingest ServingData
sources
Modern Data Architecture
Insights to enhance business applications, new digital services
Transactions
ERP
Data analysts
Business users
DATA PIPELINES
EVENT PIPELINES
Data
Event
Insights
Data
Lake
Social media

19. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Outcome 2:
Innovate For New Revenues

20. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Earecstasy Has Its First Direct Relationship With
Consumers
Krzysztof, Data ScientistBala, Head of Marketing
What are our customer segments, based on usage?
Can predict music preference from location and HR?
Are there additional signals in the voice commands?
Can we infer user activity, from scenes in pictures?
How are people using the Smart Buds?
How to understand what they listen to and
when?
What kinds of people are in/decreasing usage?

21. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Start With A Set Of Specific Questions To Answer,
Then Work Backwards To The Data Required
Media consumption
(Partner API)
Registration, usage
[time/place]
(Mobile app)
Bala, Head of Marketing
How are people using the Smart Buds?
How to understand what they listen to and
when?
What kinds of people are in/decreasing usage?

22. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Start With A Set Of Specific Questions To Answer,
Then Work Backwards To The Data Required
Krzysztof, Data Scientist
HR, Voice, GPS,
Images (Device data)
DATA LAKE, OR NOT?
Registration, usage
[time/place]
(Mobile app)
LOAD TO DATA LAKE
What are our customer segments, based on usage?
Can predict music preference from location and HR?
Are there additional signals in the voice commands?
Can we infer user activity, from scenes in pictures?

23. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Sandboxes - Fast, Cheap, Low Risk

24. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Ingest ServingData
sources
Transactions
Data scientists
Business users
Connected
devices
DATA PIPELINES
EVENT PIPELINES
Data
Event
Insights
Data
Lake
Sandbox
ML / Analytics / DLWeb logs /
clickstream
Modern Data Architecture
Insights to enhance business applications, new digital services

25. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Ingest ServingData
sources
Modern Data Architecture
Innovate for new revenues - personalisation and forecasting
Transactions
ERP
Data analysts
Data scientists
Business users
Connected
devices
DATA PIPELINES
EVENT PIPELINES
Data
Event
Insights
Data
Lake
ML / Analytics
Social media
Web logs /
clickstream

26. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Outcome 3:
Real-Time Engagement

27. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Earecstasy Offers A Personalised Life Soundtrack
Personalised, based on
past preferences,
people with similar behaviors,
and environments detected

28. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Use Earecstasy Voice Enablement To Play Music
I’m tired, play
me some music!
Amazon Transcribe
/ Comprehend
Action: PLAY
Category: MUSIC
Genre: <RECOMMEND>
Request content
HISTORY
Twenty One Pilots!
PEOPLE LIKE YOU
Amazon Kinesis
Streams
Connected device data
Location: <FIND GPS>
Mood: <FIND HR>

29. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Use The Mobile App To Take A Picture To Identify Activity
A QUIET OFFICE
Amazon SageMaker
Image Classification
Amazon Rekognition
Image
CHAIR
LAPTOP
LAMP
DESK
97%
95%
88%
82%
Object Identification
WORKING!
<HISTORY>

30. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Ingest ServingData
sources
Modern Data Architecture
Real-time engagement and interactive customer experiences
Transactions
ERP
Data analysts
Data scientists
Business users
Engagement platformsConnected
devices
Automation / events
DATA PIPELINES
EVENT PIPELINES
Data
Event Action
Insights
Data
Lake
ML / Analytics
Predict /
Recommend
AI Services
Social media
Web logs /
clickstream

31. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Business Outcomes on a Modern Data Architecture
Outcome 1 : Modernise and consolidate
• Insights to enhance business applications and create new
digital services
Outcome 2 : Innovate for new revenues
• Personalisation, demand forecasting, risk analysis
Outcome 3 : Real-time engagement
• Interactive customer experience, event-driven automation,
fraud detection

32. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Ready to build better business from your ideas?
Short list projects that
directly impact
customer engagement
and adoption
Build simple data
pipelines that allow
you to test new ideas,
and fill your data lake
Ask our solution
architects and
professional services
teams to help you build

33. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Thank You