1. Towards a Unified Resource
Placement Service for OpenStack
Senhua Huang
senhuang@cisco.com
Office of the Cloud CTO – Lew
Tucker, Cisco Systems, Inc.

2. Motivation: A Use Case
• Application driven requirements from tenants
– Scalable, Elastic, Easily accessible
– High performance
– Pay as you go
Applications
Persistent and reliable data store
infra

3. Motivation: A Use Case
• Application driven requirements from tenants
– Scalability
– Elasticity
– Accessibility
– High performance
Applications
Persistent and reliable data store
infra

4. *-as-a-Service Provided by OpenStack
Compute Nova ✔
Glance (image)
Swift (object)
Storage
Cinder (block) ✔
Network Quantum

5. Let me take care of compute!
Nova
3
1 2
Host-1 Host-2 Host-n

6. Let me take care of storage!
Cinder
3
1
3
1 2
2
Host-1 Host-2 Host-n

7. 3
1
1
3
2
2
Host-1 Host-2 Host-n

8. Bad placement hurts!
• Network resource is wasted
• Application performance suffers
– Unpredictability
– Delay
– Throughput
• Reliability suffers
• Why not use host filters step-by-step
– Might not work
– Lose easy accessibility

9. An example of good placement
1 2 3
1
3
2
Host-1 Host-2 Host-n

10. How do we get there?

11. On the way: >15 proposals
• Network awareness (Monday 4:30 pm)
– Network proximity (Gary Kotton)
• Nova scheduler features (Tuesday 11:00 am)
– Add whole host allocation capability to Nova (Phil Day)
– The future of the scheduler (Andrew Laski)
– Coexistence of different schedulers (Alex Glikson)
– Utilization based scheduling (Revngvit Yanggratoke)
– Utilization aware scheduling (Don Dugger)
– Rack aware scheduling (Phil Day)
– Extend the Nova Scheduler view of mutable resource (Phil Day)
– Make scheduler host state extensible (David Scannell)
– List supported scheduler hints via API (Phil Day)
• Group Scheduling (Tuesday 1:50 pm)
– Placement groups in Nova (Alex Glikson)
– VM Ensembles (Group Scheduling) API (Gary Kotton and Gilad Zlotkin)
• Scheduling across nova/cinder/networking (this session)
– Scheduling across nova/cinder/networking (Alex Glikson)
– Unified resource placement module for OpenStack (Senhua Huang)
• The future of state management in nova (Joshua Harlow) (Tuesday 3:40 pm)
• Related previous proposals:
– Donabe container concept/API by Debo Dutta and et al.: Demo on Thursday April 18!

12. Two possible approaches
• An incremental improvement on the coordination
between Nova and Cinder
– Nova/Cinder decides their own resource selection
– Transfer message about scheduling between them
– Maybe a shared library for common scheduling codes
• A separate module for resource selection and
capability cache
– Having a central place to collect all capability
information
– Handle the complexity/scalability at this module

13. Unified Placement and Orchestrator
User Facing API
Resource
config Capability Orchestrator/
Selection
Cache Provision Manager
Engine
Quantu Ceilom
Nova Cinder
m eter

14. Components
• RSE
– Runs resource selection algorithms to calculate
placement results
• Capability cache
– Collects and represents the available capabilities of
infrastructure resources (“things” that are provision-
able)
• Orchestrator
– Manage the work flow of provisioning the selected
resources
– One possible solution from Y!/NTT Data

15. User request specs
• Individual instances with
different types (currently
storage and server)
• Each instance has a set of
request specs
affinity
affinity
anti-
anti- – Image/flavor/size/…
affinity affinity
affinity • Instances can be in a
grouping relationship
• Each grouping has a set of
server
grouping request specs
storage
– Affinity rules/…

16. Capability cache
• Capabilities of compute/volume hosts
– Memory, CPU, disk
– Network interface
• Topology/Inventory information
– Rack #, Cluster #, neighboring nodes
• Quantitative capabilities: updated periodically
• Qualitative capabilities
• Networking resources
• This component can be used for many other
purposes

17. RSE
• Various algorithms
– Different optimization objectives
(reliability, performance, usage efficiency, energy, etc.)
– Heuristics: trade-off between performance and
complexity
• Extensible
– Easy to plug and play any algorithm admin desires
– Easy to contribute and innovate
– As stateless as possible
– No need to worry about framework etc.

18. Interface
• User/Tenant
– Similar to other OpenStack services
– REST API for CRUD
– Keystone for authentication/authorization
• Nova/Cinder/…
– Choice-1: REST API
– Choice-2: RPC/Message queue
• Enhancement feature
– Group scheduling

19. Interface w/ Orchestrator
• RSE decides which host to place each instance
of the request
– {instance: host}
• Orchestrator calls Nova/Cinder scheduler to
create instance on the specified host
– Failure handling
– Revert
• What about update/delete operation?

20. Work Plan
• Maybe first start as a feature within Nova
– nova/rse
– nova/cap
• Optional services to user/tenant
– Exposed as API extensions in Nova
– Use existing REST APIs (available to admin) exposed by
Nova and Cinder
• Made as another incubated project when it
matures
• Prototype
– Using availability-zone hint from Nova/Cinder

21. Road Map
F1 F2 F3
• F1: Framework (API definition + design)
• F2: Compute + Storage scheduling
• F3: Grouped Compute + Storage scheduling

22. Thanks!