1.
Data Is the New Oil

2.
What is Big Data?
Quantum of data
TB to PB of data
Speed of data
Millisecond latency
Types of data
Hundreds of data sources
Quality of data
Varies greatly. Affects
accuracy of analysis
Business relevance
How does it help the business?
25+TB of data being
generated per second
globally
90+% of world’s data
created in last 2 years
90+% of data generated is
unstructured

3.
Evolution of Big Data ProcessingDescriptivePredictivePrescriptive
Batch
Real-time
Dashboards;
Traditional query &
reporting
Prediction engines;
Inventory forecasting,
cross-sell analysis
Recommendation
engines; routes, content
recos
What & Why it happened?
Probability of ‘x’ happening
What to do if ‘x’ happens
TypeofAnalytics
Speed of Analysis
It is happening!
Alerts, analysis &
detection; what is going
wrong, fraudulent use

4.
Big Data
Potentially massive datasets
Iterative, experimental style of data
manipulation & analysis
Frequently not a steady-state workload;
peaks & valleys
Variety & velocity of data
Management of tools complex
AWS Cloud
Massive, virtually unlimited capacity
On-demand infrastructure allows iterative,
experimental deployment/usage
Most efficient with highly variable
workloads
Tools & services for managing structured
& unstructured, batch & stream data
Fully managed
Big Data was built for the Cloud

5.
Ingest/
Collect
Consume/
visualize
Store Process &
Analyze
Data
1 4
0 9
5
Answers &
Insights
Broad, Tightly Integrated Capabilities
AWS provides the broadest platform for big data analytics today
Start Here
with a
business
case
Real-time
Amazon Kinesis
Firehose
Data Import
AWS
Import/Export
Snowball
Object Storage
Amazon S3
Real-time
Amazon Kinesis
Streams
Distributed
Amazon EMR
(Hadoop, Spark, etc)
BI & Data
Vizualization
Amazon Quicksoght
Real-time
AWS Lambda
Amazon Kinesis Analytics
Data Warehousing
Amazon Redshift
Machine Learning
Amazon Machine
Learning
Relational
Databases
Amazon RDS
No SQL
Databases
Amazon
DynamoDB Elasticsearch
Amazon
Elasticsearch
Data Connect
AWS Direct
Connect
Storage gateway
AWS Storage
Gateway
Database Migration
AWS Data Migration
Service
Time to
answer
(latency)
Throughput
Cost

6.
Amazon Redshift
Fast, fully managed, petabyte-scale data warehouse
•10X better performance than traditional DBs
•Less than one tenth the cost of traditional solutions
• Simple and fully managed
• Flexible & Scalable: Easily change number or type nodes
•ANSI SQL Compatible: Use familiar SQL clients/BI tools
•Secure: Encryption, network isolation, audit & compliance
•Ideal usage patterns: sales, historical, gaming, finance,
marketing, ad, social data
10 GigE
(HPC)
Ingestion
Backup
Restore
SQL Clients/BI ToolsSQL Clients/BI Tools
128GB RAM128GB RAM
16TB disk16TB disk
16 cores16 cores
Amazon S3Amazon S3
JDBC/ODBC
128GB RAM128GB RAM
16TB disk16TB disk
16 cores16 cores
Compute
Node
Compute
Node
128GB RAM128GB RAM
16TB disk16TB disk
16 cores16 cores
Compute
Node
Compute
Node
128GB RAM128GB RAM
16TB disk16TB disk
16 cores16 cores
Compute
Node
Compute
Node
Leader
Node
Leader
Node

7.
Amazon EMR
Quickly and cost-effectively process vast amounts of data
•Largest cloud operator of Hadoop infrastructure
•Open source & MapR distributions
•Most current Hadoop distribution
•Flexibility : Decoupled compute & storage, select apps, resize
•Simple : Launch a cluster in minutes, fully managed
•Scalable : Provision as much capacity as needed
•Multiple pricing option – On-demand, Reserved Instances, Spot
•Typical use cases – Clickstream analysis, log processing, genomics

8.
Amazon Kinesis
Easily work with real-time streaming data
Amazon Kinesis Streams
• Build custom apps to process or analyze streaming data
• Typical use cases – Log & event data collection, real-time analytics
Amazon Kinesis Firehose
• Easily load massive volumes of streaming data into S3, Redshift, AWS ES
• Typical use cases – Digital marketing, IoT, mobile data capture
Amazon Kinesis Analytics
• Easily analyze data streams using standard SQL queries

9.
Amazon Elasticsearch
Fully managed making it easy to set-up, operate & scale
Elasticsearch clusters in the cloud
•Easy set-up & configuration. Fully managed
•Flexible storage options
•Set-up for high availability
•Seamlessly scale
•Direct access to Elasticsearch APIs
•Support for ELK. Built-in KIbana
•Integration with AWS IAM for controlling access to your domain
•Integration with Amazon CloudTrail for auditing
Amazon
Route 53
Elastic
Load
Balancing
IAM
CloudWatch
Elasticsearch API
CloudTrailAWS

10.
Select Big Data & Analytics Customers
The vast majority of Big Data use cases deployed in the cloud today run on
AWS

11.
Now available in the Mumbai region!

12.
Real time analytics
in the cloud

13.
Data is Growing
1.7MB 58% 0.5%
of new data will be
created every second
for every human being
on the planet by 2020
http://www.whizpr.be/upload/medialab/21/c
ompany/Media_Presentation_2012_DigiUni
verseFINAL1.pdf
compound annual
growth rate of 58%
surpassing $1 billion
by 2020 forecasted for
the Hadoop market
http://www.ap-institute.com/big-data-
articles/big-data-what-is-hadoop-
%E2%80%93-an-explanation-for-
absolutely-anyone.aspx
http://www.marketanalysis.com/?p=279
of all data is ever
analyzed and used at
the moment
http://www.technologyreview.com/news/51
4346/the-data-made-me-do-it/

14.
Why AWS for Big Data
Immediately
Available
Broad and Deep
Capabilities
Trusted and
Secure
Scalable

15.
Collect, Store, Analyze, and Visualize
It’s easy to get data to AWS, store it securely, and analyze it with the engine of your
choice, without any long-term commitment or vendor lock-in
Collect
Import/Export
Snowball
Direct Connect
VM Import/Export
Store
Amazon S3
EMR
Amazon Glacier
Amazon Redshift
DynamoDB
Analyze
Amazon Kinesis
Lambda
EMR
EC2

16.
AWS – The most complete platform for Big Data
Big Data Repositories Clickstream Analysis ETL Offload
Machine Learning Online Ad Serving BI Applications

17.
Benefits
Unearth opportunities
 Save cost and improve revenue
Ease of deployment
 Technologies are powerful and flexible
yet easy to deploy and use
No upfront charges
 You can start small, scale rapidly and
pay as you go

18.
BluePi as AWS Partner
AWS Consulting Partner.
Preferred partner from last 3 years.
Have done 30+ projects and PoCs together.
PAN India collaboration to help industries like Logistics, Media, BFSI,
Startups.

19.
Why BluePi?
 Successfully executed 10+ big data projects.
 Handles more than 60TB data in total
 With 1 TB of incremental data per month
 Serves analytics for 1 billion events per month
 Solutions include
 Predictive Analysis
 Recommendation Engine
 Real Time Dashboards

20.
A cross industry need
Banking and financial services
 Authentication, validation, and fraud prevention
Logistics
 Optimum route plan, SLA tracking
Online commerce
 Recommendations, cross sell and upsell opportunities
Media and entertainment
 Context-based ads and content

21.
Trusted Partner Of..
And many more...

22.
Technologies we thrive on
REAL TIME STREAMING
VISUALIZATION
AQuickSight
ht
BIG DATA TECHNOLOGIES

23.
What is real time analytics?
The analysis of “data in motion”
 Data captured by systems for monitoring, alerts, and creating reports
 Used for high level decision making
The processing of every incoming event
 Enables organizations to inspect, analyze, and act
IT IS NOT!
 Batch processing or micro batch processing
 Analysis of static data, even if it was captured a few minutes ago

24.
Why real time analytics?

25.
Pi-stats – Analytics and Recommendation
 Real Time Dashboard
 Targeted Push Notification
 News and Tag Recommendation
 Hybrid Machine Learning Algorithm
 Trending News Prediction
 Language Agnostic

26.
 300+ million monthly events
 10000 events/sec
 6 million unique users
 6 different languages
 500 GB data/month
Pi-stats – In Numbers

27.
Business problem
 Integrated data store across multiple applications to enable complex business
analytics and reporting
 Adding ~5 GB of data daily to 5TB data warehouse increased complexity
 Daily number of dispatches: 350 – 500 k
 Number of daily package scans: 10 – 15 m
 Number of cities served: 550+
 Number of active clients: 3000+

28.
The Solution
 Implemented data warehouse and analytics solution on AWS
 Using cutting edge technologies like Redshift, Kinesis,
Redis and Storm to build solution
 Agile methodology used for delivery in 6 months
The Benefit
 Near real-time reporting to track SLAs
 Flexible architecture ensures performance at petabyte scale
 Visualization and custom BI tools available

29.
This project has been one of the most challenging and ambitious ones for
Delhivery. Given the multifaceted challenges and complexity involved, Bluepi team
took up the task enthusiastically and executed it all through with same
perseverance and rigor. Being given the kind of reassurance and ongoing support
combined with its innovative and collaborative style adds further reason for us to
appreciate them.
- Kapil Bharti, CTO Delhivery

30.
Questions and Concerns
To Know How BluePi and AWS
Can Help You Click Here
Or e-mail us at info@bluepi.in

31.
Q&A