Apache CloudStack is open source software for building public, private and hybrid Infrastructure as a Service (IaaS) clouds, it allows users to provision virtual servers, storage and networking resources through a web interface and provides APIs for management and integration with other systems, and it supports various hypervisors including KVM, Xen, VMware and Oracle VM VirtualBox as well as storage systems like iSCSI, NFS and object storage.

1. Apache CloudStack
Alex Huang
Architect, Cloud Platforms Group, Citrix Systems Inc.
 Co-founded VMOps (Cloud.Com)
 Software architect for CloudStack.
 Responsible for CloudStack’s overall architecture, performance,
and scalability.
 Currently a committer and PPMC member on Apache for
CloudStack.
 MS from Stanford and BS in Computer Science from Berkeley.

2. Apache CloudStack (Incubating)
An Introduction
Alex Huang
Apache CloudStack Committer

3. Use CloudStack to build IaaS clouds (like EC2)
• Create VMs, disks • Java based
networks, network services • Scalable
• Self service • Many vendor integrations
• Meter usage • Native and EC2 API

4. How did Amazon build EC2?
Amazon eCommerce Platform
AWS API (EC2, S3, …)
Amazon Orchestration Software
Open Source Xen Hypervisor
Commodity Commodity
Networking
Servers Storage

5. How can you build your cloud?
Amazon eCommerce Platform
Your Portal (Optional)
AWS API (EC2, S3, …)
CloudStack or AWS API
CloudStack Orchestration Software
Amazon Orchestration Software
ESXi, KVM, XenServer/XCP, OVM
Open Source Xen Hypervisor
Networking Servers Storage

6. Project history
• 2008/2009: closed-source development
• First deployments in late 2009
• May 2010: ~98% open source as GPLv3 (open core)
• August 2011: 100% open source GPLv3
• April 2012: Switch to Apache License v2
• Submit code to Apache Software Foundation

7. Project current state
• In incubation within Apache Software Foundation
• 4.0 released!
• Bugs and wiki moved to ASF infra
• Mailing list traffic moved to ASF infra
• Many non-Citrix contributors, committers, and PPMC
members

8. Yes, the ASF is great
Enter ASF

9. IaaS Cloud Concepts

10. Virtualization alone does not make a cloud
Server Virtualization Cloud
Built for traditional enterprise Designed around big data,
apps & client-server compute massive scale & next-gen apps
• Scale-up (pool-based resourcing) • Scale-out (horizontal resourcing)
• IT management-centric • Autonomic management
• 1 administrator for 100’s of servers • 1 administrator for 1,000’s of servers
• Proprietary vendor stack • Open, value-added stack

11. Clouds must reliably run all types of workloads
Traditional Workload Cloud Workload
Expect reliability Design for failure
Back-up everything Ephemeral resources
HA, Fault tolerance Multi-site redundancy
Admin control recovery Self-service recovery
Think Server Virtualization Think Amazon Web Services

12. Embrace traditional and extend to Cloud-era
Cloud-era Workloads Traditional Workloads
CloudStack Mgmt
Server Traditional Zone
vSphere
Enterprise Networking (e.g., VLAN)
Cloud-era Cloud-era Cloud-era
Availability Availability Availability
Zone Zone Zone
ESXi ESXi ESXi
Cluster Cluster Cluster
Object Storage Enterprise Storage (e.g., SAN)

13. Apache CloudStack
Management Server
Traditional Traditional Cloud-era Traditional Cloud-era
Availability Availability Availability Availability Availability
Zone Zone Zone Zone Zone

14. Deployment and Software
Architecture

15. • Single Management Server can
Data Center 1 Data Center 2 manage multiple zones
Data Center 2
Management Data Center 3
Servers
Zone 2
• Zones can be geographically
Zone 2 distributed but low latency links
Zone 3 are expected for better
Zone1
Zone 4 3
Zone
performance
• Single MS node can manage up to
Data Center 2
Data Center 2 10K hosts.
Data Center 2
Zone 2
Zone 2
• Multiple MS nodes can be
Zone 2 Zone 3 deployed as cluster for scale or
Zone 3 redundancy
Zone 3

16. Standby Mgmt
Cloud-era zone deployment Server Cluster
Admin Internet
Availability Zone 2
Primary Mgmt
Server Cluster
Primary
Router
MySQL
Backup Load Balancer
MySQL
L3 Core Switch
Top of Rack
Switch
Object Store
Servers
… … … … …
Availability Zone 1
Pod 1 Pod 2 Pod 3 Pod N

17. Internet
Traditional zone
deployment Object
Load Balancer
Store Core Switch
… Aggregation
Switch
TOR Switch
Compute
Nodes
NFS
Primary
10Gbps 1Gbps 10Gbps 1Gbps 10Gbps 1Gbps Storage
Storage Guest Storage Guest Storage Guest
& Mgmt & Mgmt & Mgmt
Pod 1 Pod 2 Pod 200

19. Management Server
XAPI HTTP
vCenter Agent Agent
XenServer
KVM OVM
XCP ESX
• XS 5.6, 5.6FP1, 5.6 SP2, • ESX 4.1, 5.0 • RHEL 6.0, 6.1, 6.2, • OVM 2.2
6.0.2, XCP 1.1 • Full Snapshots Ubuntu 12.04 • No Snapshots
• Incremental Snapshots • VMDK • Full Snapshots (not live) • RAW
• VHD • NFS, iSCSI, FC & Local disk • QCOW2 • NFS & iSCSi
• NFS, iSCSI, FC & Local disk • Storage over-provisioning: • NFS, iSCSI & FC • No storage over-
• Storage over- NFS, iSCSI • Storage over- provisioning
provisioning: NFS provisioning: NFS

20. Mgmt Server CPU Util.
Seconds to deploy
25,000 …. to …. 30,000 VMs 0 …. to …. 30,000 VMs
• Simulator developed to test massive scale
• Four Management Servers can manage 30,000 hosts
• Scale to hundreds of thousands of hosts possible with
multiple management server clusters (regions)

21. Features

22. Compute Hypervisor
XCP/XS VMware Oracle VM KVM Bare metal
Storage Block & Object
Fiber Object
Local Disk iSCSI NFS
Channel Stores
Network Network & Network Services
Network Load
Isolation Firewall VPN
Type balancer

23. Users
Change
VM Operations Console Access VM Status
Service Offering
Start
• CPU Utilized 2 CPUs 4 CPUs
Stop
1 GB RAM 4 GB RAM
Restart • Network Read 20 GB 200 GB
Destroy 20 Mbps 100 Mbps
• Network Writes

24. Add / Delete VM 1
Volumes Volume
Create Templates Volume Template
from Volumes
Schedule Now
Hourly Weekly
Snapshots Daily Monthly
View Snapshot ….
History 12/2/2012 7.30 am 2/2/2012 7.30 am

25. Specify Resource Levels Configure Properties Define Scope
Compute Disk Network
Name Name Name
CPU Cores Custom Disk Size Network Rate
CPU (MHz) Disk Size (GB) Redundant VR
Memory (MB) Storage Tag Firewall
Host Tag Storage Tag Load balancer
Enable HA Public Public
CPU Cap
Public

26. Resources
Domain VMs, IPs, Snapshots…
• Domain is a unit of isolation that
Org A represents a customer org, business
Admin unit or a reseller
Domain
• Domain can have arbitrary levels of
Reseller A
sub-domains
Admin
Resources
Sub-Domain
Org C
VMs, IPs, Snapshots… • A Domain can have one or more
Admin accounts
Account
• An Account represents one or more
Group A
users and is the basic unit of
Account
isolation
Group B
• Admin can limit resources at the
User 1
Account or Domain levels
User 2

27. • Create Networks and attach
VMs
• Acquire public IP address for
NAT & load balancing
• Control traffic to VM using
ingress and egress firewall
rules
• Set up rules to load balance
traffic between VMs

28. Network offering
• Provides cloud operator
defined service features
• Isolation
• Load Balancing
• VPN
• Firewall
• Supports Physical Devices
• NetScaler
• F5 BIG-IP Pod 1 Pod 1 Pod N
• Juniper SRX Zone 1 Zone N

29. Network Services Managed Externally Network Services Managed by CS
Public Network
65.11.0.0/16Security Security
Group 1 Public Group 1
65.11.1.2 Guest Network/Internet 65.11.1.2 Guest
VM 1 VM 1
65.11.1.3 Physical 65.11.1.3 Guest
Guest
Load VM 2
VM 2 Balancer
65.11.1.4
EIP, 65.11.1.4 Guest
Guest
VM 3 ELB VM 3
65.11.1.5 Guest
65.11.1.5 Guest
VM 4 VM 4
CS CS
Security DHCP, Virtual Security
DHCP, Virtual
Group 2 DNS Router Group 2
DNS Router

30. CS Virtual Router provides Network Services External Devices provide Network Services
Guest Virtual Network 10.0.0.0/8 Guest Virtual Network 10.0.0.0/8
Public VLAN 100 Public VLAN 100
Network/Internet Network/Internet
Guest Public IP Private IP Guest
10.1. VM 1 6.37.1.12 Juniper 10.1.1.111 10.1. VM 1
CS Gateway 1.1 SRX 1.1
6.37..1.11 Firewall
Virtual address Guest Guest
Router 10.1.1.1 10.1. VM 2 10.1. VM 2
Private IP
DHCP, DNS 1.3 Physical 10.1.1.112 1.3
NAT Guest Load Guest
Load Balancing 10.1. VM 3 Public IP Balancer 10.1. VM 3
VPN 1.4 6.37.1.11 1.4
Guest Guest
10.1. VM 4 10.1. VM 4
1.5 1.5
CS
DHCP, Virtual
DNS Router

31. Layer-2 Layer-3
Isolation VLAN/SDN Security Groups
Performance Better Better
Network setup Moderate Easy
Support broadcast Yes No
Scalability Good Best
Interoperability with Good Poor
physical servers

32. CloudStack storage
Primary Storage
• Configured at Cluster-level. Close to hosts
for better performance
• Stores all disk volumes for VMs in a cluster L3 switch
• Cluster can have one or more primary
storages Pod 1 L2 switch
• Local disk, iSCSI, FC or NFS Secondary
Cluster 1 Storage
Host 1
Secondary Storage Primary
Storage
• Configured at Zone-level Host 2
• Stores all Templates, ISOs and Snapshots
• Zone can have one or more secondary
storages
• NFS, OpenStack Swift, others coming