A talk for SF big analytics meetup. Building, testing, deploying, monitoring and maintaining big data analytics services. http://hydrosphere.io/

1. DevOps for Data Science
by Stepan Pushkarev
CTO of Hydrosphere.io

2. DevOps is a catchy buzzword to optimise things

3. © Josh Wills
http://www.slideshare.net/g33ktalk/dataengconf-sf16-bridging-the-gap-between-data-science-and-data-engineerin
g

4. Is there life after marriage data science?
Dating, Flowers,
Dreams
Marriage
Happily lived
forever?
Collect & prepare
data
Build ML Model

5. This talk is for people who are married aware of
“other 99% of data science”
Dating, Flowers,
Dreams
Marriage
Happily lived
forever?
Collect &
prepare data
Build ML Model

6. This talk is NOT about
- Setting up Apache Spark/Hadoop cluster
- Configuring CI/CD tools like Jenkins
- Configuring monitoring tools & dashboards
- Agile/DevOps brainwashing & consulting story

7. Agenda
- Challenges in deploying analytics into
production
- Deploying analytics as a service
- Feedback loops: testing, monitoring,
analytics of analytics

8. Why do companies hire data scientists?

9. Why do companies hire data scientists?
To make products smarter.

10. What is a deliverable of data scientist and data
engineer?

11. What is a deliverable of data scientist?
Academic
paper?
ML Model? R/Python
script?
Jupiter
Notebook?
BI
Dashboard?

12. What has to be a deliverable of data scientist?
Data pipelines and machine
learning models that deployed as
pluggable, testable, supportable,
monitorable analytics services.

13. Option 1: Engineer to implement academic paper

14. Option 2: Engineer to re-implement R/Python script

15. Option 3: Run notebook as it is using cron

16. Option 3: Run notebook as it is using cron

17. Option 4: Build software to eat the world Data Science

18. Eating data science
© Daniel Tunkelang - Where should you put your data scientists? -
www.slideshare.net/dtunkelang/where-should-you-put-your-data-scientists
Step 1 (management): Integrate data scientists into
cross-functional teams

19. Eating data science
Step 2 (operations): Make environments scalable
and elastic. Finally.

20. Eating data science
Step 3: Make data scientists to write less code

21. Eating data science
Step 4: Deploy analytics as services

22. Step 5: Use feedback loops: testing, monitoring,
analytics for analytics
Build ML Model
Test
Monitor,
maintain,
analyze
Deploy as a service
Collect & prepare
data

23. Agenda
- Challenges in deploying analytics into
production
- Deploying analytics as a service
- Feedback loops: testing, monitoring,
analytics of analytics

24. Deploying analytics as a service
- Defines deliverable for Data Scientist / Data Engineer.
- Plugs analytics into end-to-end products through API.
- With the right tooling allows Data Scientist to deploy it in self
serve

25. Look around - proprietary ML based APIs
- Alchemy API
- Google Prediction API
- Cloud Vision API
- Azure ML
Can we do our own on top of Apache Spark?

26. Bad Practice #1. Business logic in Spark? WTF?

27. Bad Practice #2. Database as API
Execute reporting job
Mark Job as complete &
save result
Poll for new tasks
Poll for resultSet a flag to build a report

28. Bad Practice #3. Low level HTTP API
When Data Scientists
design an API...

29. Hydrosphere Mist - a service for exposing analytics
jobs and machine learning models as web services

30. Types of analytics services
- Enterprise Analytics services
- Reactive or Streaming services
- Realtime ML services

31. Enterprise analytics services
- Could not be
pre-calculated
- On-demand
parametrized jobs
- Requires large scale
processing
- Reporting
- Simulation (pricing, bank
stress testing, taxi rides)
- Forecasting (ad
campaign, energy
savings, others)
- Ad-hoc analytics tools
for business users

32. Demo #1

33. Reactive or Streaming services

34. Reactive or Streaming Reporting services

35. Demo #2

36. Realtime Machine Learning Services
Train models in Apache Spark and deploy it for realtime
low latency serving/scoring with high throughput

37. PMML is not an option
Spark ML, TensorFlow, H2O, Vowpal Wabbit, and every new ML
library invents uses own serialisation format

38. Format is not an issue if we re-define a deliverable for
ML model
xml, json, parquet, pojo, other
Single row Serving / Scoring
layer
Large Scale,
Batch
processing
engine
Monitoring,
testing
integration
Deliverable artifact for Machine Learning Model

39. Repository
Zooming out
MLLib model TensorFlow model Other model
Unified Serving/Scoring API

40. Demo #3

41. Agenda
- Challenges in deploying analytics into
production
- Deploying analytics as a service
- Feedback loops: testing,
monitoring, analytics of analytics

42. Testing, monitoring, analytics of analytics
- Poorly discussed in community.
- We are in production, baby!
- Regression.
- State matters. Model lifetime is limited.
- Data drifts, pipelines and model fail silently.
● Saves time
● Saves money
● Saves lifes

43. TDD world

44. TDD world does not work here
Pff… easy:
- Unit tests - by platform developers
- Integration tests - often impossible
Not clear who and not clear how:
- Regression
- Data Validation
- Production testing
- Data and ML pipelines quality monitoring

45. Need either “Data QA” & “Data Ops” people
or … AI
(formula for the next 10 000 startups - take something and add AI)

46. Smart data structures and dumb code works a lot
better than the other way around

47. Can we develop DSL and Data Structure which is
smart enough to learn from data patterns, trends and
anomalies to be self-QAed?

48. QA view: a universe vs. big data analytics system
People observe and monitor signals from stars to
check that universe is not broken today

49. Monitoring! Grafana, Kibana

50. And Marijuana to make sense out of it

51. Metrics processing, monitoring, correlation insights...
...Isn’t it a big data analytics task on its own?

53. ML pipeline Kafka
Analytics jobs
for metrics
Emit Metrics
Stream it back
into Spark
Context
Use insights to
make our data
structures
smart
Solution: loop of analytics for analytics

54. Benefits
● Don’t need to talk to Ops! :)
● Already have Apache Spark and Kafka in place
● Data Scientist in the loop!
● Unlimited flexibility in analytics, correlation and
using ML for ML
● Models could feeded back into Smart self
QA-ed data structures.

55. Hydrosphere Swirl - a system that creates a swirl of
analytics for analytics

56. Original ML
pipeline
Kafka
Streaming or
Batch Swirl
jobs
Hydrosphere
Swirl
Plug, modify,
deploy, run jobs &
consume results
Metrics
definition,
Notebook
integration
Hydrosphere
Mist
(1) Emit metrics
Hydrosphere Swirl: Vision

57. Demo #4

58. Classify by
sentiment
Twitter
Prepare
Data
Serve ads to
user
Hydrosphere Swirl
Invalid records 10/sec 2k/sec0.8Ratio Clicks
Swirl Demo: Serve Ads to users with positive Tweets

59. Classify by
sentiment
Twitter
Ingest &
transform
Serve ads to
user
Hydrosphere Swirl
Invalid records 20k/sec 10/sec0.2 Clicks
Data pipeline
is broken
Ratio
Swirl Demo: Serve Ads to users with positive Tweets

60. Twitter
Ingest &
transform
Serve ads to
user
Hydrosphere Swirl
Invalid records 10/sec 10/sec0.2 Clicks
New ML model
deployment
Deployed
bug in ML
code
Ratio
Swirl Demo: Serve Ads to users with positive Tweets

61. Thank you
Looking for
- Feedback
- Advisors, mentors & partners
- Pilots and early adopters
Stay in touch
- @hydrospheredata
- https://github.com/Hydrospheredata
- http://hydrosphere.io/
- spushkarev@hydrosphere.io