I had the opportunity to share some of the Big Data approaches that OpenStack is offering with the Sahara project. With Sahara, you can deploy Hadoop and Spark clusters inside OpenStack infrastructure. The forum I presented this was at the Datalab Community

1. Big Data in the Cloud?
Yes, you can do it in
OpenStack

2. Hello!
I am Obed N Muñoz
I am here because I love to give presentations.
-vvvv

3. Software
Engineer
Who am I?
Musician Fast driver

4. Agenda
- Introduction: Cloud and
OpenStack
- Data-processing
- Sahara Project

5. Introduction
Cloud Computing and OpenStack
1

6. Cloud and
XaaS Era
Everything as a Service

7. “
Cloud computing term is used for a
variety of services and applications
emerging for users to access on
demand over the Internet as
opposed to being utilized via
on-premises means.

8. OpenStack

9. “
OpenStack is a cloud operating
system that controls large pools
of compute storage and networking
resources throughout a datacenter,
all managed through a dashboard,
CLI, RestFUL API ...

11. Architecture

13. Data-Processing
Data-Processing in the Cloud
2

14. What’s around
Data-Processing?
◇ Big Data
◇ Data Science
◇ Cloud
◇ Machine Learning
◇ Patterns Recognition
◇ Neural Networks
◇ Etc ...

15. Data-Processing
Technologies

16. Sahara Project
Data-Processing in OpenStack
3

17. OpenStack Sahara
The Sahara project provides a
simple means to provision
data-intensive application cluster
(Spark or Hadoop) on top of
OpenStack.
https://wiki.openstack.org/wiki/Sahara

18. Architecture
http://docs.openstack.org/developer/sahara/architecture.html

19. Getting Started
- Clusters
- Templates
- Provisioning Plugins
- Image Registry
- Data Processing Frameworks
- Elastic Data Processing (EDP)
http://docs.openstack.org/developer/sahara/userdoc/edp.html

20. More Features ...
- OpenStack Block Storage support
- Cluster Scaling
- Data locality
- Distributed Mode
- Hadoop HDFS High Availability
- Orchestration support
- …

21. Clusters (Hadoop)
http://docs.openstack.org/developer/sahara/userdoc/edp.html

22. Data-Processing
Frameworks
- Hadoop
- Spark
- Storm
http://docs.openstack.org/developer/sahara/userdoc/edp.html

23. Provisioning
Plugins
- Vanilla - Vanilla Apache Hadoop
- Ambari - Hortonworks Data
Platform
- Spark - Apache Spark with Cloudera
HDFS
- MapR Distribution - MapR plugin
with MapR File System
- Cloudera - Cloudera Hadoop
http://docs.openstack.org/developer/sahara/userdoc/edp.html

24. Elastic Data
Processing (EDP)
Allows the execution of jobs on
cluster created from Sahara. It
supports:
- Hive, Pig, MapReduce.Streaming, Java, Shell job
types on Hadoop clusters
- Spark jobs
- Shared File system service (manila), or Sahara own
database
- Access to input and output data sources in:
- HDFS
- Swift
- Manila
http://docs.openstack.org/developer/sahara/userdoc/edp.html

25. Resources
- Documentation
- http://docs.openstack.org/developer/sahara/
- https://wiki.openstack.org/wiki/Sahara
-
- Hadoop/Spark Images
- http://sahara-files.mirantis.com/images/upstream/mitak
a/
- OpenStack Auto-deployment with RDO
- https://www.rdoproject.org/install/quickstart/
- Videos
- https://www.youtube.com/watch?v=idAaLo1stbw
- https://www.youtube.com/watch?v=TgPTjrf1y0A
http://docs.openstack.org/developer/sahara/userdoc/edp.html

26. http://hackathon.openstackgdl.org/

27. Q & ACONCLUSION

28. Thanks!
Any questions?
You can find me at:
◇ @obedmr
◇ obed.n.munoz@gmail.com