a quick intro to deploying and managing some complex applications in the cloud illustration of how the brooklyn.io open source project and autonomic management ideas handle cloudera hadoop, marklogic xml big data, and betfair's appcloud

1. Alex Heneveld / @ahtweetin
OpenStack Design Summit
17 April 2013
Cloud Application Management in Practice

2. http://brooklyncentral.github.com
© 2013 Cloudsoft Corporation 2

3. Uses
Strictly Confidential © 2013 Cloudsoft Corporation 03/26/13 3

4. Uses
Strictly Confidential © 2013 Cloudsoft Corporation 03/26/13 4

5. Uses
Strictly Confidential © 2013 Cloudsoft Corporation 03/26/13 5

6. Betfair
Strictly Confidential © 2013 Cloudsoft Corporation 03/26/13 6

7. Betfair
Strictly Confidential © 2013 Cloudsoft Corporation 03/26/13 7

8. Autonomic Management (M-A-P-E)
Escalate Delegate
Enrich
Sensors Effectors
Analyze Plan
Autonomic
Manager* Monitor Execute
State
Sensors Effectors
Managed
Element
*An autonomic manager can be a managed element
Strictly Confidential © 2013 Cloudsoft Corporation 03/26/13 8

9. Hierarchical Autonomic Management
Granularity can be anything from individual application to overall control plane
Strictly Confidential © 2013 Cloudsoft Corporation 03/26/13 9

10. Multi-Tier Application
Multi-Tier Application
Need to update targets
Load Balanced Cluster
NGINX to match cluster
OS
Cluster
Need to specify
autoscaling policy
JBoss7Server
OS
… JBoss7Server
OS
Need to publish MySQL
URL for cluster
MySQL
OS There has to be a better
way than scripting …
Strictly Confidential © 2013 Cloudsoft Corporation 03/26/13 10

11. MyWebCluster – Topology
MyWebCluster
MySQL ControlledDynamicWebAppCluster
MySQL and NGINX could be
swapped out for their
clustered variants
NGINX DynamicWebAppCluster
Here we explicitly model the app’s
organizational structure JBoss7Server … JBoss7Server
Abstract Entities Autonomic Managers - AMP specific constructs or building blocks
Concrete Entities Managed Elements - AMP proxies for real world components
Strictly Confidential © 2013 Cloudsoft Corporation 03/26/13 11

12. MyWebCluster – Blueprint
MyWebCluster
MySQL ControlledDynamicWebAppCluster
[targets] is a built in policy
whereas the auto scaling
[targets] Auto Scaler Policy
policy is explicitly added
NGINX DynamicWebAppCluster
An application blueprint is
completely portable JBoss7Server … JBoss7Server
Policies Attached to entities they effect but can subscribe to any sensor
Strictly Confidential © 2013 Cloudsoft Corporation 03/26/13 12

13. MyWebCluster – Customization
MyWebCluster
MySQL ControlledDynamicWebAppCluster
MySQL
[targets] Auto Scaler Policy
NGINX DynamicWebAppCluster
NGINX
The pluggable driver mechanism is an JBoss7Server … JBoss7Server
elegant way to customize a blueprint for
a particular environment JBoss7Server … JBoss7Server
Drivers Customer specific implementations (Bash, Puppet, Chef etc.)
Strictly Confidential © 2013 Cloudsoft Corporation 03/26/13 13

14. MyWebCluster – Instantiation
Location + MyWebCluster
MySQL ControlledDynamicWebAppCluster
MySQL
OS [targets] Auto Scaler Policy
NGINX DynamicWebAppCluster
NGINX
OS
JBoss7Server … JBoss7Server
JBoss7Server … JBoss7Server
OS … OS
Instances Location dependent – can be virtual machine, LXC etc.
Strictly Confidential © 2013 Cloudsoft Corporation 03/26/13 14

15. Powerful GUI and RESTful API
© 2013 Cloudsoft Corporation 15

16. Java Library
public class MyWebCluster extends AbstractApplication
implements MyWebClusterConstants {
@Override
public void init() {
MySqlNode mysql = ...;
ControlledDynamicWebAppCluster web = ...;
web.addEnricher(HttpLatencyDetector.builder().url(ROOT_URL).
rollup(10, SECONDS).build());
web.getCluster().addPolicy(AutoScalerPolicy.builder().
metric(REQUESTS_PER_SECOND_IN_WINDOW_PER_NODE).
metricRange(10, 100).sizeRange(2, 5).build());
addEnricher(SensorPropagatingEnricher.newInstanceListeningTo(web,
ROOT_URL, REQUESTS_PER_SECOND_IN_WINDOW, REQUEST_LATENCY_IN_SECONDS_IN_WINDOW));
}
}
© 2013 Cloudsoft Corporation 16

17. A DSL — Heat, CAMP, TOSCA, more ...
name: "three tier app"
components:
hello_war: # no type
content: hello.war
requires:
com.example.java:WarContainer:
initialSize: 3
fulfillment: frontend
com.example.java:Autoscaler:
metric: request.latency
max: 120ms
hello_sql:
content: hello.sql
type: com.example.database:Schema # here, type of component defined
requires: backend
frontend: # "platform component" implied by WarDeplReq above
requires:
database: # frontend type must recognise a named "database" req
mode: CDI # assume that req supports various injection modes
fulfillment: backend
© 2013 Cloudsoft Corporation 17

18. What Next?
• DSL UNCONFERENCE tomorrow 2.20pm
• Community at http://brooklyncentral.github.com
• IRC at #brooklyncentral
© 2013 Cloudsoft Corporation 18