Learn how Couchsurfing transformed 3 months of data loading into 3 hours.

1. Big Data Solution Showcase
Learn how Couchsurfing transformed 3 months of data
loading into 3 hours
Here is the url for the webinar:
https://connect.awswebcasts.com/p2j3kh6nmx3/

2. Presenters today
Brad Helicher
Director of Cloud Business
Attunity
Chris Keyser
Partner Solution Architect
Amazon Web Services
Charlie Killian
Founder & Owner
Bytecode IO
Thiago Bruch
Systems Engineer
Attunity

3. Agenda
• Introduction to AWS and big data
services
• Attunity Cloudbeam: moving the data
that moves your business into the cloud
• Customer case study: Couchsurfing
• Attunity Cloudbeam for Amazon Redshift:
how does it work & demo
• Q & A

4. What is AWS
AWS Global Infrastructure
Application Services
Networking
Deployment & Administration
Compute Storage Database

5. Technologies and techniques for working
productively with data, at any scale.
Big Data

6. Creating Value from Data Assets
Recommendations,
Collective Intelligence
Visualization
Measuring Functionality
and Services
Hypothesis Testing &
Predictions
Learning from Social
Media Conversations
SOCIAL
BIG DATA
Machine Learning
Dashboards
Business Intelligence
Ad Hoc Queries
A/B Testing
Statistical
Analysis
Sentiment Analysis

7. Big Data AWS Cloud
Potentially Massive Data Sets Massive, virtually unlimited capacity
Iterative, experimental style of data
manipulation and analysis
Iterative, experimental style of
infrastructure deployment/usage
Frequently not a steady-state
workload;
peaks and valleys
Efficient with highly variable
workloads
Time to results is key
Parallel compute clusters from single
data source
Hard to configure/manage
Managed services for data storage
and analysis

8. AWS Data Services
Data
Velocity
Variety
Volume
Structured, Unstructured, Text, Binary
Gigabytes, Terabytes, Petabytes
Millisecond, Second, Minute, Hour, Day
EBS EC2
Instance Storage
RDS Redshift
SQL Stores
EMR
Hadoop
DynamoDB
NoSQL
Kinesis
Stream
Storage Services
S3
Cloud
Front
Glacier
Elasticache
Caching
Data
Pipeline
Orchestrate

9. Storage Services – Object Store
Amazon S3
Designed for 99.999999999% durability
Stores anything
Lifecycle and Versioning
Fine Grained Access Control
Reduced Redundancy Storage

10. Fast, simple, petabyte-scale data warehousing for less than $1,000/TB/Year
Amazon Redshift

11. MySQL, Oracle, SQLServer, Postgres
Backup/Restore, High Availability
Push Button Scalability
Up to 3 TB and 30K IOPS
Amazon
RDS

12. Nokia: 50% Cost Savings with 2x Faster Queries
Hadoop Tools
Improving Rapidly
On-demand, Flexible,
Big Data Technologies
Cheaper and
Faster
Redshift & Hadoop Price-performance
Advantage over
RDBMS
>50% platform cost savings>2x
faster queries Minimal DBA
support
Redshift, Hadoop, S3, EMR, Data Pipeline for ETL
Cost-effective for 10s of TB data sets
AMI-based Services
Internet Speed Report Authoring
Hypothesis testing vs. waterfall

13. http://aws.amazon.com/marketplace
Big Data Case Studies
Learn from other AWS customers
aws.amazon.com/solutions/case-studies/big-data

14. AWS Marketplace
AWS Online Software Store
aws.amazon.com/marketplace

16. Moving the data that moves your business in the cloud.
Data Transfer throughout On-Premises, Amazon RDS, S3, & EC2
Brad Helicher
Director of Cloud Business
Brad.Helicher@attunity.com
+1-954-946-2274, ext. 1105
ThiagoBruch
Services Support Engineer

17. We Move the Data that Moves Your Business
 Data Load/Replication
 Change Data Capture
 Data Access/Federation
 Release Automation
 Managed File Transfer
 Enterprise File Replication
 On-Premises to Cloud
 Cloud to On-Premises
 Cloud to Cloud
 Customer Support
 Professional Services and
Consulting
17

18. Cloud & Data Transfer: Use Cases
18
»Amazon Redshift:
»Data Loading for BI / Analytics
»Amazon RDS / DBs on EC2:
»Migrations from legacy / on-prem systems to AWS
»System Cutovers
»Disaster Recovery (on-prem and across AWS regions)
»Amazon S3:
»Content Availability
»Disaster Recovery
»Hadoop EMR
»Amazon EC2 (File Replication):
»Migrations from legacy / on-prem systems to AWS
»System Cutovers
»Disaster Recovery (on-prem and across AWS regions)
www.attunity.com/cloud

19. 1.Complexity
2.Takes too long
3.Costs too much
4.Not real-time
5.Lack of Developer Resources
19
Data Replication: Common Challenges
www.attunity.com/cloud

20. CloudBeam –Value Proposition
»Key Features:
»Move any data, anytime, anywhere
»Accelerated transfers
»Quick & Easy Setup
»Intuitive Administration
»Extensive Automation
»Real-time Monitoring
»Affordable Pay-per-Use
20
»Business Benefits:
»Ensure Information Availability across the Enterprise and the Cloud
»Overcome Data Transfer Bottlenecks
»Quick Time-to-Value
www.attunity.com/cloud

21. Attunity CloudBeam
Customer Success Story: Couchsurfing
High-Performance Information Availability Solutions. Made Radically S2i1mple.

22. Who is Couchsurfing?
22
Enables a global network of travelers to exchange cultural
experiences via hosted stays and more.
10 million
Members
120,000
Cities
100,000
Event Attendees

23. Who is BytecodeIO?
23
Data and system integrators specializing in ETL, APIs, Data Warehousing and Big Data.
charlie.killian@bytecode.io• http://www.bytecode.io/

24. Problem #1
24
No data insight.
Millions of hosting requests sent.
But what does it mean?
Rely on data instead of mythology.

25. Problem #2
25
Too much data for current architecture.

26. Solution
26
Near real-time
ODS
(Operational Data Store)

27. Benefits: Short-Term
27
ODS
online
Analyst

28. Benefit: Long-Term
28
Data
Warehouse

29. Technology Selection
29
• ODS
• Data Warehouse
• Data Integration
Data
Warehouse
ODS
Data
Integration

30. Amazon Redshift
30
Data
Warehouse
ODS
• No capital outlay
• Offered as a service
• Can be implemented with limited internal resources
• Little operational overhead
• Data sources already in AWS RDS MySQL
• Combine ODS and Data Warehouse

31. Data Integration Challenges
31
•Aggregate large amounts of disjointed data
•Map and convert data types
•Build destination tables
•Cleanse data
•Perform Change Data Capture (CDC)
•Limit ongoing maintenance

32. Simplification and the AWSMarketplace
32
•Manual development estimated at 3 months
•Development cycle needed to be shortened
•Wanted the whole process to be simplified
•Searched the AWSMarketplace for solutions

33. Attunity CloudBeam for Amazon Redshift
33
ODS
(Amazon Redshift)
CloudBeam
• End-to-End database to Amazon Redshift loading
• Near real-time
• Incremental CDC
• Easy setup and configuration
• Simple job management
• Affordable

34. Attunity Solution
34
AWS Region
Amazon Redshift
Amazon S3
Amazon RDS MySQL
EC2 Machine
Windows Server
EC2 Machine -M3.Large
Windows ServerAttunity AMI (data pass through)

35. Full Stack Solution
35

36. Results
36
Relying on data instead of mythology.
3 months 3 hours
V
S
MANUAL PROCESS ATTUNITY
Data insight!

37. Attunity CloudBeam for Amazon Redshift
High-Performance Information Availability Solutions. Made Radically S3i7mple.

38. Attunity CloudBeam for Amazon RedshiftSolutions for On-Prem Sources
38
www.attunity.com/cloud
AWS Region
EC2 MachineM3.LargeAttunity AMI
Redshift
S3
On-Premises
Source DB
»How it’s Offered:
»Available on-demand via the Amazon Marketplace
»Licensing: Hourly or Bring Your Own License
»Supported Source DBs:
»Oracle
»SQL Server
»MySQL
… and many others

39. Attunity CloudBeam for Amazon RedshiftSolutions for Data Sources within AWS
39
AWS Region
Redshift
S3
Source DB
On EC2
EC2 Machine
Windows Server
Attunity Replicate Software
EC2 Machine -M3.Large
Windows ServerAttunity AMI (data pass through)
www.attunity.com/cloud

40. Attunity CloudBeam for Amazon RedshiftWhat’s Needed to Get Started?
40
www.attunity.com/cloud
AWS Region
EC2 MachineM3.LargeAttunity AMI
Redshift
S3
On-Premises
Source DB
»Windows 64-bit Server
»Located in same local network as source DB(s)
»Hosts ‘Attunity Replicate’ software
»Amazon Redshift Driver & Database Clients installed
»Ports: 5746 & 5439
»Amazon S3 & Amazon Redshift instances
»‘Attunity CloudBeam for Amazon Redshift’
»Via Amazon Marketplace
»Hourly or BYOL

41. 41
Web-based Designer andManagement Console
Target
Database
Replication Server
In Memory Processing
TransformFilter
Persistent Store
SourceDatabase
TransactionLog
Bulk Reader
CDC
Bulk Loader
Stream Loader
Data / Metadata
Data / Metadata
Attunity CloudBeam -Replicate-for-RedshiftMethod & Processes
www.attunity.com/cloud

42. Attunity CloudBeam for Amazon RedshiftHow it Works
42
www.attunity.com/cloud
AWS Region
EC2 MachineM3.LargeAttunity AMI
Redshift
S3
On-Premises
Source DB
1.Attunity Replicate extracts datafrom source database
2. Attunity Replicate applies any transformations & filters
3. Data files generated
4. Optimized Transfer from Attunity Replicate server to EC2 machine.
6.‘Instruction Channel’ used to
execute COPY command.
Data loaded into Amazon Redshift.
7. Incremental Loading via CDC, applied inside Redshift using merge commands
5. Data passes through, staging in Amazon S3.

43. Attunity CloudBeam -Performance
»Optimized transfer protocol
»Data transfer technologies:
»Leverages Amazon multi-part transfers
»Concurrent Sessions / Transfers
»Compression
»Recoverability, Guaranteed Delivery
»SSL Encryption
»Performance Gains:
»10-12x over standard copy
43
www.attunity.com/cloud

44. Thank You
High-Performance Information Availability Solutions. Made Radically S4i4mple.
On to the Demo …
Thiago BruchSenior Engineer

45. Get in touch with us…
45
www.attunity.com/cloud
Attunity Contact:
www.attunity.com/cloud
Learn.attunity.com/cloudbeam
Meet us at re:Invent!
Attunity –Booth 639
Thank you!