The document discusses Cisco's Hadoop as a service offering on their Intercloud platform. Some key points: - Cisco provides managed Hadoop, including Cloudera's distribution, on optimized instances with local storage and object storage. This offers a scalable, reliable, and secure environment for Hadoop workloads. - Use cases discussed include predictive maintenance using IoT data and analyzing customer journeys across multiple channels. - A pilot test showed Cisco's platform could process over 100 million records from production data across various Hadoop jobs. - Cisco also discusses their data virtualization product CiscoDV, which can integrate data across on-premises, cloud sources on Cisco and AWS. -

2. Hadoop in the Cisco
Cloud
Kartik Kanakasbesan
kartikka@cisco.com

3. • Introduction
• What is Cisco Cloud services?
• Cisco’s Big Data as a service
• Why Hadoop in the Cloud?
• Use Cases for Hadoop in the Cloud
• Customer Experiences so far
• Going forward
Agenda

4. The Intercloud
The globally connected network of clouds
Enterprise
Private
Clouds
Public
Clouds
Intercloud
Alliance
Intercloud
Services
INTERCLOUD
Intercloud
Providers
CIMK v1.0

5. The Intercloud
Customer Value
INTERCLOUD CONTROL
Across services, in
every location
COMPLIANCE
Manage risk locally
and globally
CHOICE
Cloud the way
you need it
Public
Clouds
Enterprise
Private
Clouds
Intercloud
Alliance
Intercloud
Services
Intercloud
Providers
with…
and…
CIMK v1.0
Intercloud
Services

6.  Worldwide leader in
cloud building, cloud
services and
managed services
 Extensive global
partner network
 Global Scale
 Workload mobility
 Open Standards
Customer ValueCisco
Cisco & OpenStack – Delivering value
 430 Companies and
growing
 17000+ individual
members
 2655 Cumulative
contributions
OpenStack

7. Cisco Intercloud Services
OpenStack and
standards based
cloud
OPEN
STANDARDS
Self-service
infrastructure
enabling
application
lifecycle
GLOBAL
SCALE
PUBLIC
CLOUD
PLATFORM APIS
Workload mobility
with control and
compliance
Empowering
developers
and cloud-
scale
applications
RAPID
INNOVATION
Best-of-breed
of Cisco’s
products and
best practices

8. Cisco Intercloud Services: Target Customers
• Hybrid workloads
requiring common
network and
security policies
Enterprise
• Value-added
services with
NGN/NFV
• Federation
capabilities
Network-Based
Service Providers
Developers
• SaaS, Network-
centric workloads
• IOT/IOE, SP Video,
Collaboration, and
Mobility workloads

9. IoT World Forum Reference Model
Levels
Application
(Reporting, Analytics, Control)
Data Abstraction
(Aggregation & Access)
Data Accumulation
(Storage)
Edge Computing
(Analysis & Transformation)
Connectivity
(Communication & Processing Units)
Physical Devices & Controllers
(The “Things” in IoT)
Collaboration & Processes
(Involving People & Business Processes)
Sensors, Devices, Machines,
Intelligent Edge Nodes of all types
Center
Edge
1
2
3
4
5
6
7

10. Query
Based
Event
Based
Data at
Rest
Data in
Motion
Non-real
Time
Real
Time
Sensors, Devices, Machines,
Intelligent Edge Nodes of all types
Center
Edge
IoT World Forum Reference Model

11. Cisco Intercloud services – platform
Cloud-Centric Networking, Security, Policy
Core Cloud Services Analytics Building Blocks
Application
Enablement Tools
Enterprise/Hybrid Services
Cisco Micro Services
Collaboration SP Video Analytics
Inter-Region Virtual
Private Backbone
Automated Private
VPN Connectivity
Network-optimized
Workload Placement
Managed Public + Private Cloud
Marketplace for Cisco, Third Party ISV, Enterprise Applications and Services
Third Party Open
Source Tools
Security Network/Device ManagementIOE
Global SP Backbone
Dashboard
Basic Cloud
Resource
Monitoring
Network
Performance
Metrics
Deployment/
Management Tools,
PaaS
VNF Library Orchestration, Auto-Scaling
Federated Network
& Security Policies
Compute Storage Database Virtual Network LB VPN Hadoop
Service Chaining
Data Virtualization
Intercloud Fabric support for
heterogeneous environments
Data Ingest
App-Level Sovereignty, Privacy Policies


12. Market place
3rd party
algorithms etc.
OpenstackAPIs,SQL,REST
Data Ingestion as a
service
Hadoop as a
service
Machine Learning
as a service
Data Warehouse
services
Data Virtualization
services
Other….
Vision for Cisco Cloud Provided Data Service*
IOE/IOT Applications
Proactive
Maintenance
Manufacturing
Apps
Machine as a
service
Oil and Gas
Service Provider Analytics Apps
Network
Diagnostic
Service
Provider
Analytics
Customer
Loyalty
Analytics
Feature
Analytics
Collaboration Analytics Apps
Telepresence
Analytics
Collaboration
Analytics
Social Analytics
Sentiment
Analysis
Others Applications
Marketing Apps
Availability
Analytics
Demand
Planning
Sentiment
Analysis
Deliver an
integrated and
managed
environment of
these primitives
Deliver
analytics
applications to
customers
(Hybrid, on-
Premise or
Software as a
service
DataSources
Remove the
burden of
managing the
infrastructure
Allow Organization
and Line of
businesses to focus
on Market
Opportunities and
develop Analytics
Applications
CIS Provided Data Services
*Subject to change based on market feedback

13. Cisco’s Big Data as a service*
Hadoop as a service (aaS)
Data Ingestion -aaS
Visualization -aaS
Machine learning -aaS
Data Virtualization- aaS
Analytics -aaS• All these services
need
• Provisioning
• Monitoring
• Scaling
• Consumption model
• Integrated
• Individually
• Cisco Branded service
• Minimal Flexibility
on Vendor choice
Provisioned on
Big Optimized instances
• Local Storage
• Object and Block Storage
*Subject to change based on market feedback

14. Cisco’s Hadoop as service*
Hadoop
Reliable,
Secure &
Monitored
HaaS
Cisco’s Hadoop as a service
• Provides market leading Cloudera’s
Hadoop distribution
• Flexibility to deploy Hadoop optimized
templates for Streaming and Batch
processing
• Data ingest with Apache Kafka
• Support Apache Spark Stack
• Core, SQL, Mlib,& GraphX
• Running on YARN
• Secure access to Hadoop APIs
• Integrate with on premise Hadoop
distributions (if needed)
Openstack
*Subject to change based on market feedback

15. Does Hadoop in the Cloud make sense ?
?

16. Why Hadoop in the Cloud makes sense?
• Reducing barriers in adopting Hadoop
• Cloud and Hadoop provide the perfect
solution to “test” the Hadoop waters
• Help customers build IoE/IoT applications
faster with Cisco’s solutions
• Run your Hadoop dev/test workloads in
the cloud and provision them on premises
• Leverage Cisco’s Networking capabilities
as a differentiator for your capabilities
• Provide consistent policies on the cloud
just like on-premises
• Provide a scalable, reliable, and secure
environment

17. • $16B by 2020*
• Targeted to grow by 70.8% CAGR
• Over 20 plus players in the market
• Highly fragmented Amazon, Azure, IBM, Google, Rackspace, and many more
• North America is the leading market
• Europe is further behind
• AP markets are maturing fast
Hadoop as a service(HaaS) Market size & forecast
*Source:GigaOM

18. Use Case: Preventative Maintenance
Data ingestion Data Processing
In-memory
database
Hadoop
In-memory
querying
Real-time
query with
low latency
1000’s of robots streaming
messages (structured &
unstructured data)
Lambda architecture in the Cloud for
IoT/IoE (elastically scalability and secure)
Data
aggregation at
the plant floor
done by a
Cisco UCS
box

19. Use Case: Omni-Channel Customer Journeys
Server
Logs
Social
& Chat
Mobile
Event
Stream
s
Call
Center
S/W
Download
Open Trouble
Ticket
Assign
Engineer
Update
Trouble Ticket
Close Trouble
Ticket
Resolve
Trouble Ticket
Read Support
Documents
View Design
Documents
View Tech
Documents
New
Registration
Bug Search FAQs
Contract
Details
Product
Details
Device
Coverage
Interaction Touch points
Channels
Journey
Case Resolution
Software Upgrade
The customers’ interaction with Cisco across multiple touch points to get the desired business
outcome.

20. Pilot Test Data
• Test performed on one day’s production data
• Total no. of records processed – 110,852,667
• Total data ingest size – 32GB/day
• Total no. of M/R jobs in the data pipeline – 17
• Two test cycles
• Cycle 1: Heterogeneous CCS nodes (vCPUs,
storage, memory)
• Cycle 2: Homogeneous CCS nodes

21. AWS to CIS Migration – Success Criteria
 Successful synthesis of customer interaction data
 Successful automation of the end-end data process pipeline
 Build behavioral insight services
 Access to data and services via data discovery and visualization tools
 Meet the performance, scale and platform stability requirements
 Successful deployment of CiscoDV on CIS
 Connect HDFS and Hive DS with CiscoDV via Hive and Impala
 Build and expose insight services for consumption by limited users

22. AWS and CIS Data Node Sizing Comparison
Hadoop Cluster for Batch and Query Analytics
Node Service AWS Instance Type vCPU Mem Storage
Number of
Data Nodes
Comments
Data Nodes/
Node Master m3.2xlarge 8 30 2x80 GB 30
Each hadoop data node has 1500GB of EBS
available for HDFS storage
AWS Sizing
CCS Sizing
Node Service CCS Instance Type vCPU Mem Storage
Number of
Data Nodes
Comments
Data Nodes/
Node Master GP-2XLarge 8 32 50 35
Each hadoop data node has 1500GB of
volume storage available for HDFS storage

23. CIS Performance of Batch Analytics – Limited Test

24. Test Details by M/R job
Job Name CCS 12
nodes:
cycle1
CCS 18
nodes:
cycle1
CCS 24
nodes:
cycle1
CCS 30
nodes:
cycle1
CCS 18
nodes:
cycle2
CCS 24
nodes:
cycle2
CCS 30
nodes:
cycle2
CCS 35
nodes:
cycle2
New_cleanse 249 176 143 117 82 67 55 51
Process_private_ip 27 14 11 10 7 5 6 6
join_web_and_ip_data 142 95 76 61 49 40 34 29
combine_ip_decorated_files 26 14 11 10 9 7 8 7
filterBotEntries 34 19 15 13 10 8 7 7
sessionize 71 64 69 62 60 63 15 13
firstActivitiesFilter 26 15 13 10 9 8 6 6
allOtherActivitiesFilter 29 18 13 13 11 9 7 6
matchFirstActivities 21 13 11 13 13 11 8 8
buildActivities 27 15 12 10 7 6 9 9
filterBUG 8 5 3 2 3 3 4 4
filterSEA 8 5 3 2 3 3 4 4
filterTCO 8 5 3 2 3 3 4 4
filterTDV 8 5 3 2 3 3 4 4
filterWDV 8 5 3 2 3 3 4 4
filterMOD 8 5 3 2 3 3 4 4
filterTOOL 8 5 3 2 3 3 4 4

25.  Discover data beyond the enterprise: Virtual integration that combines
traditional enterprise data, Big Data stores on CIS and AWS, cloud data from
SaaS providers and, Cisco Customers and Partners
 Seamless interoperability offers easy access to data across distributed data
sources in the intercloud analytics platform
 Universal data governance maximizes enforcement of data security rules
 Analytics Data Hubs: Deployment flexibility to build hybrid/virtual sandboxes
that enable nimble data discovery and rapid data analytics to support multiple
LOBs
In addition to Hadoop: Cisco Data virtualization

26. CiscoDV on Intercloud Analytics Platform (CIS)
Scenario 1
CIS Cisco DV to Cisco
Enterprise Data Store
Scenario 2
CIS CiscoDV to Impala and
Hive on CIS Intercloud
Analytics Platform
Scenario 3
CIS Cisco DV to Hive on AWS
Big Data Cluster
Scenario1
Scenario 3

27. Sample result for Cisco DV

28.  Cisco’s Hadoop services is available to select customers only before
General Availability
 Part of the broader Cisco Big Data as a service play
 Let us know the kind of tools you use for
 Visualization
 Machine Learning
 How can we address your Data challenges together ?
Going forward

29. Questions ?

30. Thank you