This is a talk from the Austin OpenStack summit. It demonstrates how a resilient, elastic and load-balanced cluster can be deployed using senlin, heat, ceilometer, lbaas v2, nova.
4. Manageability
cluster membership
Elasticity
key feature of a cloud, pay-as-you-go
Load-Balanced
distribute loads evenly on all members
Flexibility
customizable to meet the needs of real-life use cases
Extensibility
easy to add resource types and policies
4
Generic Requirements
5. Heat: The Orchestrator
AutoScalingGroup +
Scaling Policy + Alarm
Heat AutoScaling modeled after AWS
AutoScaling
It may suit some limited use cases, but not
all scenarios considering that OpenStack
aims to build a platform for both private and
public clouds
But, Heat's Mission...
Orchestrate composite cloud apps using a declarative template format
through an OpenStack-native REST API
It is not about any new service, not auto-scaling, not HA, not load-balance ...
5
What we have on OpenStack
Heat as a Compiler
YAML
Template
Heat
Nova
Cinder
Neutron
Keystone
• CREATE
• GET
• UPDATE
• DELETE
6. Cross Availability Zone Placement
Cross Region Placement
Anti-Affinity Placement
Choose a Specific Node to Delete when Scaling-In
Trigger Autoscaling Using Application-Level Metrics
Manual Scaling
Auto Recover Node when Failures Detected
Migrate Nodes from a Stand-By Cluster for Rapid Provisioning
Soft Scaling
...
6
Some Specific Requirements
8. 8
What Do We Really Need?
Scalable
Load-Balanced
Highly-Available
Manageable
......
of any (OpenStack) objects
- What is missing from
OpenStack?
A Clustering Service
- Auto-scaling?
Just one of the usage
scenario of a cluster.
- Auto-Healing (HA)?
Just another usage
scenario.
- We can address the
concerns by making
policies orthogonal
10. 10
Senlin Features
Profiles: A specification for the objects to be managed
Policies: Rules to be checked/enforced before/after actions are performed
Docker
Senlin
Nova
Kubernetes
Heat
Ironic BareMetal
VMs
Stacks
Containers
placement
deletion
scaling
health
load-balance
batching
Policies as Plugins Profiles as Plugins Cluster/Nodes Managed
11. 11
Senlin Server Architecture
openstacksdk
identity
compute
orchestration
network
...
engineengine lock
scheduler
actions
nodecluster
service
registry
receiverparser
drivers
openstack
dummy
(others)
dbapi
rpc client
policies
placement
deletion
scaling
health
load-balance
batching
receiver
webhoook
MsgQueue
extension points
for external
monitoring
services
extension points
facilitating a
smarter cluster
management
extension points to talk to different
endpoints for object CRUD operations
extension points for interfacing
with different services or clouds
profiles
os.heat.stack
(others)
os.nova.server
senlin-api
WSGI
middleware
apiv1
12. 12
Senlin Operations (Actions)
Cluster
• CREATE
• DELETE
• UPDATE
• LIST
• SHOW
• ADD_NODES
• DEL_NODES
• SCALE_OUT
• SCALE_IN
• POLICY_ATTACH
• POLICY_DETACH
• POLICY_UPDATE
Node
• CREATE
• DELETE
• UPDATE
• LIST
• SHOW
Policy
• CREATE
• UPDATE
• DELETE
• LIST
• SHOW
Profile
• CREATE
• UPDATE
• DELETE
• LIST
• SHOW
Action
• LIST
• SHOW
Event
• LIST
• SHOW
Receiver
• CREATE
• DELETE
• LIST
• SHOW
16. 16
senlin cluster-resize
usage: senlin cluster-resize [-c <CAPACITY>] [-a <ADJUSTMENT>] [-p <PERCENTAGE>]
[-t <MIN_STEP>] [-s] [-n MIN] [-m MAX] <CLUSTER>
Positional arguments:
<CLUSTER> Name or ID of cluster to operate on.
Optional arguments:
-c <CAPACITY>, --capacity <CAPACITY>
The desired number of nodes of the cluster.
-a <ADJUSTMENT>, --adjustment <ADJUSTMENT>
A positive integer meaning the number of nodes to add,
or a negative integer indicating the number of nodes
to remove.
-p <PERCENTAGE>, --percentage <PERCENTAGE>
A value that is interpreted as the percentage of size
adjustment. This value can be positive or negative.
-t <MIN_STEP>, --min-step <MIN_STEP>
An integer specifying the number of nodes for
adjustment when <PERCENTAGE> is specified.
-s, --strict A boolean specifying whether the resize should be
performed on a best-effort basis when the new capacity
may go beyond size constraints.
-n MIN, --min-size MIN New lower bound of cluster size.
-m MAX, --max-size MAX New upper bound of cluster size. A value of -1
indicates no upper limit on cluster size.
EXACT CAPACITY?
CAPACITY BY PERCENTAGE?
CHANGE IN CAPACITY?
MINIMUM STEP?
STRICT/BEST-EFFORT?
ADJUST MIN_SIZE?
ADJUST MAX_SIZE?
27. 27
Blueprints on Reworking Heat Autoscaling
BP Priori
ty
Description
autoscaling-api-
resources
high Heat resources invoking AS APIs
as-api-group-
resource
high ScalingGroup resource wrapping AS API's
group functionality
as-api-policy-
resource
high ScalingPolicy resource wrapping AS API's policy
functionality
as-api-
webhook-
resource
high Webhook resource wrapping AS API's execution
of webhooks
autoscaling-api-
client
high A python client for Heat to interact with AS API
autoscaling-api - A separate service for the implementation of
autoscaling w/ Heat
as-engine - A separate engine/service for autoscaling
support AS API
as-engine-db - A DB dedicated to autoscaling, using schema
created in as-lib-db
as-lib - A separate module to be used by the AS service
as-lib-db - A DB for autoscaling bookkeeping
28. 28
Relation To Other Projects
Senlin provides the array data type for cloud programming
Senlin
Ceilometer
Heat
Nova Cinder Neutron Swift Keystone
Horizon
Primitive Data Types
Complex Data Types
struct person {
int age;
char name[0];
}
person team[10]; // Senlin cluster of Heat stacks
// Senlin cluster of nova servers
// Heat stack containing senlin clusters
Mistral MagnumSaharaMonasca Tacker Zaqar