Contenu connexe Similaire à [Hortonworks] Future Of Data: Madrid - HDF & Data in motion (20) [Hortonworks] Future Of Data: Madrid - HDF & Data in motion1. 1 © Hortonworks Inc. 2011 – 2017. All Rights Reserved1 © Hortonworks Inc. 2011 – 2017. All Rights Reserved
Pedro Algaba – Solutions Engineer EMEA South
Raúl Marín – Solutions Engineer EMEA South
2. 2 © Hortonworks Inc. 2011 – 2017. All Rights Reserved
Hortonworks: Enabling the Modern Data Architecture
Our durable and reliable mission continues…
Make Hadoop an enterprise viable data platform
Bring all data under management – all sources and types
Enable pre and post transaction analysis
Hortonworks consistent and continuous track record of innovation
3. 3 © Hortonworks Inc. 2011 – 2017. All Rights Reserved
Capture
streaming data
Deliver
perishable insights
Combine
new & old data
Store
data forever
Access
a multi-tenant data lake
Model
with machine learning
DATA AT REST
(Hortonworks Data Platform)
DATA IN MOTION
(Hortonworks DataFlow)
ACTIONABLE
INTELLIGENCE
Perishable Insights
Historical Insights
A Connected Data Strategy Solves for All Data
C L O U D
On Premise
4. 4 © Hortonworks Inc. 2011 – 2017 All Rights Reserved
Industry Megatrends Shifting the Data Paradigm
INTERNET
OF THINGS
Industrial Internet
Connected Business
Consumer Devices
ARTIFICIAL
INTELLIGENCE
Machine/Deep Learning
Prescriptive Analytics
Autonomy
Smart Devices
CLOUD
COMPUTING
SaaS/PaaS Applications
Ephemeral Use Cases
Operational Efficiency
Collaboration
STREAMING
DATA
Real-time Applications
Targeted Retail
Recommendations
Industrial Applications
5. 5 © Hortonworks Inc. 2011 – 2017 All Rights Reserved
HDF Data-In-Motion Platform – what’s in HDF 3.0
7. 7 © Hortonworks Inc. 2011 – 2017 All Rights Reserved
What are people talking about myself?
8. 8 © Hortonworks Inc. 2011 – 2017 All Rights Reserved
What are people talking about myself?
Let’s start with what a Real-time Processing Data Architecture looks like:
9. 9 © Hortonworks Inc. 2011 – 2017 All Rights Reserved
What are people talking about myself?
We’ll only consider Twitter as data source for demo purposes:
10. 10 © Hortonworks Inc. 2011 – 2017 All Rights Reserved
What are people talking about myself?
NiFi does a great job managing the flow of the data and guaranteeing its delivery:
11. 11 © Hortonworks Inc. 2011 – 2017 All Rights Reserved
What are people talking about myself?
SAM is where the tough work happens:
12. 12 © Hortonworks Inc. 2011 – 2017 All Rights Reserved
What are people talking about myself?
SAM has processors for most of the use causes, but can be easily extended:
13. 13 © Hortonworks Inc. 2011 – 2017 All Rights Reserved
What are people talking about myself?
SAM has processors for most of the use cases, but can be easily extended:
14. 14 © Hortonworks Inc. 2011 – 2017 All Rights Reserved
What are people talking about myself?
It’s always better if there is a contract in between; thanks Schema Registry:
16. Call for Abstracts is now open!
Call for Abstracts closes December 15,2017
DataWorks Summit Berlin
April 16–19, 2018
https://dataworkssummit.com/abstracts/
https://dataworkssummit.com/berlin-2018/#tracks
17. 17 © Hortonworks Inc. 2011 – 2017. All Rights Reserved
Do you feel like trying the demo on your own?
Get a HDF sandbox
Use the official sandbox or the one we used for the demo:
- https://hortonworks.com/downloads/#sandbox
- https://app.vagrantup.com/raulempi/boxes/hdfsandbox
Don’t be shy on giving resources to the VM (mine – Intel i7 4 vCPU 10GB RAM)
It’s just a matter of:
1. Ingest social media data (in motion) with NiFi
2. Filter this data out with NiFi and publish results in Kafka
3. Consume data in Kafka from SAM and implement the “complex processing” logic
4. Publish results whereever you need them (ie. Kafka again)
Remember that HDF is 100% OSS and you can get support from our community portal (HCC)
- https://community.hortonworks.com/index.html
Try to reproduce it
Get free support
Spread the word
We want to hear from you during the journey
- Get help from people in the Hortonworks Community
- Be part of the community and help people as well
- Share your journey as a tutorial in HCC to show off your Big Data skills
Get our respect
If you get it done…
- Beers are on us
- The stage is yours for future meetups; share your experience with us
18. 18 © Hortonworks Inc. 2011 – 2017. All Rights Reserved
Thanks!!!
See you all in the next Meetup