The slidedeck for the presentation by my team at the Atlanta Openstack Summit 2014.

1. © 2013 Cisco and/or its affiliates. All rights reserved.
Cisco
Confidential
Cisco
Confidential© 2013 Cisco and/or its affiliates. All rights reserved.
Predictive Analytics & Visualisation
Towards Data Driven Insights for Openstack Clouds

2. © 2013 Cisco and/or its affiliates. All rights reserved.
Cisco
ConfidentialCisco ConfidentialCisco Confidential© 2013 Cisco and/or its affiliates. All rights reserved.
Cloud/Big-data Winnovation Team
Alex Holden
ajonasho@cisco.com
Damian Van Vuuren
dvanvuur@cisco.com
Ailing Zhang
ailzhang@cisco.com
Marc Solanas
msolanas@cisco.com
Matt Weeks
maweeks@cisco.com
Pengfei Zhang
penfzhan@cisco.com
Kai Zhang
kazhang2@cisco.com
Xinyuan Huang
xinyuahu@cisco.com
Debo Dutta
dedutta@cisco.com
Yathiraj Udupi
yudupi@cisco.com

3. © 2013 Cisco and/or its affiliates. All rights reserved.
Cisco
Confidential
Visibility in OpenStack today

4. © 2013 Cisco and/or its affiliates. All rights reserved.
Cisco
Confidential
Tomorrow?
Sources: 2001 A Space Oddesy, MGM. Minority Report, Twentieth Century Fox Film Corporation. Mad Men, AMC
OpenStack,
deploy me an
app

5. © 2013 Cisco and/or its affiliates. All rights reserved.
Cisco
Confidential
Our Approach:
Predictive Analytics & Awesome Visualisation
Why Real Time?
● OpenStack has lots of moving parts
● Rule/Model/Policy based system hard without insight
● Ops folks want to know right now!
● Digital exhaust to characterize system behaviour.
○ Logs
○ Metrics
Why Visualisation?
● Self Evident, easy to observe insights
● East way to consume lots of information
● Focus on the user

6. © 2013 Cisco and/or its affiliates. All rights reserved.
Cisco
Confidential
Cisco
Confidential© 2013 Cisco and/or its affiliates. All rights reserved.
Digital Exhaust = Metrics/Logs
What information can we get from our cloud? Where can we get it?
● Ceilometer
● OpenStack Syslog
● Project APIs (Nova, Neutron, Cinder etc)
What can we do with it?
● Extract Insights
● Figure out what failed and why
● Move workloads around
● Detect attacks on the cloud
● Physical Servers
○ CPU
○ Hardware Faults
○ OpenStack Logs
○ OpenStack Messaging Queue
● Tenant/Project Data
○ Quotas
● Virtual Machines
○ Config
○ CPU
○ Disk
○ Network Flows
○ Performance
○ Latency
● Application Level
○ Application Logs
○ Application Requirements
○ Application Performance

7. © 2013 Cisco and/or its affiliates. All rights reserved.
Cisco
ConfidentialCisco ConfidentialCisco Confidential© 2013 Cisco and/or its affiliates. All rights reserved.
AVOS Dashboard

8. © 2013 Cisco and/or its affiliates. All rights reserved.
Cisco
Confidential
AVOS: Summary
● Analytics and Visualization on OpenStack
● Stateless: easy deployment
● Client side architecture:
○ Polls some OpenStack APIs
○ Listens to the messaging bus
○ Minimum overhead
● Provides users and service providers full insight into cloud
○ Real time info on all instances/networks etc by project and as admin
○ Historical data on performance
○ Stack-search: Search for all components in the cloud, quickly.
○ Multiple select (See combined data)

9. © 2013 Cisco and/or its affiliates. All rights reserved.
Cisco
Confidential
Analytics and Optimization on OpenStack
AVOS

10. © 2013 Cisco and/or its affiliates. All rights reserved.
Cisco
Confidential
Bottlenecks
Use Cases: Why use AVOS?
Attacks

11. © 2013 Cisco and/or its affiliates. All rights reserved.
Cisco
Confidential
Use Case: Bottlenecks

12. © 2013 Cisco and/or its affiliates. All rights reserved.
Cisco
Confidential
Use Case: Attack Detection
● Classify Metrics
● Analyse Patterns
● E.G. is a util pattern:
○ Hadoop Traffic
○ Network Faults
○ DDoS Attack

13. © 2013 Cisco and/or its affiliates. All rights reserved.
Cisco
Confidential
Future Plans
• Advanced Search Primitives (E.G. filter by instances that are shutoff quickly)
• Reducing the number of relevant log messages
• Real-time error analytics
• Add to/edit the cluster
• Ability to preview these changes
Curvature:
https://github.com/CiscoSystems/curvature

14. © 2013 Cisco and/or its affiliates. All rights reserved.
Cisco
Confidential
OpenStack changes
In order for these changes to be possible, some modifications to OpenStack are necessary:
• Ceilometer
Better API queries, optimizing return structure
Custom meters:
VM to VM traffic flows
• Other suggestions/Ideas:
Standardization of the Python clients (several inconsistencies between method calls)
Integration into horizon
“All in view of making OpenStack easier to build on top of, without the performance
gains of applications above it lost in the overhead of running them.”

15. © 2013 Cisco and/or its affiliates. All rights reserved.
Cisco
Confidential
Conclusion
Openstack Predictive Analytics/Visibility = Reduce Ops pains
We must 1) gather data 2) extract insights 3) present to ops
folks with kickass UX

16. © 2013 Cisco and/or its affiliates. All rights reserved.
Cisco
Confidential
Thank you.
Please take our survey:
http://goo.gl/CMAZje
Coming Talk:
Storage Visibility and Optimization for OpenStack:
Wednesday, May 14 • 3:30pm - 4:10pm