1.
Watcher, a Resource Optimizer
for OpenStack: Plans for Newton
and Beyond
Austin Summit—April 28, 2016
ANTOINE CABOT (WATCHER PTL, b<>com)
JOE CROPPER (IBM)
SUSANNE BALLE (INTEL)

2.
Watcher Mission Statement
Provide a flexible and scalable resource optimization service for multi-
tenant OpenStack-based Clouds
Provide a framework to implement a wide range of optimization goals
Provide a pluggable architecture for new optimization algorithms and new
data metrics
• Energy Aware optimizations e.g. outlet temperature
• Workload consolidations and re-balancing optimizations
“Out-of-box” value, albeit extensible!
https://wiki.openstack.org/wiki/watcher
2

3.
Many Contributors
3

4.
Key Features for Watcher
Watcher provides:
• Cloud optimization using VM live migration in case of imbalance detection
• Granularity of optimization with multiple goals setting from a set of hosts to an entire
cloud
• Evolutivity via its flexible plugin structure
• « On-the-shelf » optimization strategies based on CPU, RAM and Energy
Watcher can run in:
• “ADVISE MODE” for auditing before acting
• “ACTIVE MODE” for always-on optimization (under development)
• “VERBOSE MODE” for detailed optimization decision tracking (under development)
4

5.
Watcher in the OpenStack Ecosystem
5
Watcher leverages services provided by other
OpenStack projects
• VM live migration and resize
• Metric collection
• Power cycle bare metal nodes
Monitors the infrastructure and performs
optimizations on-demand
Enables new ways for OpenStack
administrators to reduce The Cloud’s TCO
Watcher
Nova(*)
Keystone
(*)
Oslo(*)Ironic(*)
Ceilometer(*)
*Other names and brands may be claimed as the property of others.

6.
Watcher Workflow and Maturity
Profile
Apply
Plan
Monitor
Optimize
Analyse
cost model constraints
Virtual machine IOPS,
energy consumption,
resources usage
Aggregate flows of
events from the
infrastructure and
take action
Profile and predict
virtual machine
resource usage
Find trade-offs between
objectives and constraints
Schedule actions such that all
security, dependency and
performance requirements are met.
Apply the optimal state
where the infrastructure
is utilized as efficiently
as specified in goals.
6 of 8
objective
3 of 8
1 of 8
Maturity level
6

7.
Admin Control
Nova
Neutron
Cinder
Cloud Infrastructure
Ceilometer
Monasca
Horizon
Watcher
Command Line
Watcher
Dashboard
WatcherAPI
Cluster
Data Model
Cluster
History
Time series databasesMetrics Collection
OpenStackBus
Watcher decision
engine
Watcher actions
planner
Watcher actions
applier
Watcher
DB
7

8.
Workflow Description
1. Create a goal with a specific strategy
• Goal is associated with a strategy in watcher.conf
• Goal can be reused by another template
2. Create an audit template with a specific goal
• Template can be reused by another audit
3. Create an audit with a specific audit template
• This audit will be triggered immediately in one shot or in a loop at a
given interval
4. Audit will generate an action plan
5. Applier will take action manually or automatically 8

9.
Overview of Watcher Decision Engine
9

10.
Outlet Temperature Strategy Logic
Not found
Not found
Not found
Hypervisors with
enough resource
Hypervisor with
lowest outlet T
Action Plan
VM Resource
neededHypervisors
with outlet T
Hypervisors
Over Threshold
Pick one VM of
the Hypervisor
Filter
Sort
Telemetry from Ceilometer
and data of hypervisors
from Nova
Not found
Sort
10

11.
Watcher’s Current State and Next Steps
11
Project Identified &
Early Prototyping
Presented in
Vancouver
Formed Initial
Project Team
Mission
Statement & Key
Uses Cases
Identified
Build Out
Architecture
Working Code
• Rebalance on server outlet
temperature and additional
telemetry
• DevStack integration
• And much more …
• Sparked
community
interest
OpenStack
compliant and
ready for small
production
deployment
• Pluggable strategy
• Additional strategies
coming from partners
• Become part of “Big
Tent” for Newton
OpenStack
compliant and
ready for large
production
deployment

12.
Mitaka Release: Framework Improvements
Setup all the OpenStack Tooling and processes for the project
to run as an OpenStack project
Improved Watcher infrastructure
• Watcher integration with ceilometer
• Dynamic strategy load
• DevStack plugin
• Task Flow integration
• Tempest integration
• Multi-node support on the Gate
• Threshold optimization
• Watcher Dashboard
12

13.
Mitaka Release: Additional Strategies
Outlet temperature strategy based on Thermal Telemetry
Rebalance VMs on physical servers strategy based on
CPU/socket count/usage
Load consolidation strategy based on a heuristic algorithm
which focuses on measured CPU utilization and tries to
minimize hosts which have too much or too little load
Overload standard deviation strategy designed to integrate
load balancing strategy based on a standard deviation
algorithm
13

14.
Plans for the Newton Release
Watcher scoring module
• Standardize interactions between the Watcher Decision Engine and
Scoring engines, e.g., external machine learning tools or
frameworks
Active mode for always-on optimization
Verbose mode for detailed optimization decision tracking
Provide efficacy indicator to the admin before running an
optimization
Define a precise and extensible scope of resources to
optimize (compute, storage, network…)
Provide more and more “value-added” optimization strategies
14

15.
Any Questions?
Want to learn more?!?
• Wiki : https://wiki.openstack.org/wiki/Watcher
• IRC : #openstack-watcher
If you are interested, we would love for you to get involved –
come and see us!
Brown Bag session available on YouTube
15

16.
Legal Notices and Disclaimers
Intel technologies’ features and benefits depend on system configuration
and may require enabled hardware, software or service activation. Learn
more at intel.com, or from the OEM or retailer.
No computer system can be absolutely secure.
Tests document performance of components on a particular test, in
specific systems. Differences in hardware, software, or configuration will
affect actual performance. Consult other sources of information to evaluate
performance as you consider your purchase. For more complete
information about performance and benchmark results, visit
http://www.intel.com/performance.
Intel, the Intel logo and others are trademarks of Intel Corporation in the
U.S. and/or other countries. *Other names and brands may be claimed as
the property of others.
© 2016 Intel Corporation.