Considerations for building your private cloud rackspace

Considerations for
Building a Private

Ryan Richard
OpenStack Engineer
ryan.richard@rackspace.com
@rackninja

October 12, 2012

Whaaa???

source: http://ken.pepple.info/openstack/2012/09/25/openstack-folsom-
architecture/

RACKSPACE® HOSTING | WWW.RACKSPACE.COM

What is a Private Cloud?
Generally considered to be smaller than a “public”
cloud

Less than 100 physical servers (for this talk)

API endpoints may not be publicly accessible

Limited inbound connectivity. Use ﬂoating IPs to
allow for inbound connectivity

Can be customized for speciﬁc workloads (hardware/
network/etc)

Company may leverage multiple private clouds


Build with the End in
Mind
What are you building for?

A. Are you building for 10 servers? 20? 100?

B. Or are you building 500 instances? 1000?
2000?

C. Or are you building 400 CPUs? 3TB RAM?
100TB disk?


Mind
What are you building for?

A. Are you building for 10 servers? 20? 100?

B. Or are you building 500 instances? 1000?
2000?

C. Or are you building 400 CPUs? 3TB RAM?
100TB disk?

d. ALL OF THE ABOVE


Example hardware
Mind
12 Physical Cores - 24 w/ Hyperthreading - 48 vcpus w/ 2:1
overcommit ratio

128GB of RAM - 1:1 overcommit ratio

8 x 300GB drives RAID 10 - ~1.2 TB usable disk space

How many instances can I run on this physical host?


Example hardware
Mind
12 Physical Cores - 24 w/ Hyperthreading - 48 vcpus w/ 2:1
overcommit ratio

128GB of RAM - 1:1 overcommit ratio

8 x 300GB drives RAID 10 - ~1.2 TB usable disk space

How many instances can I run on this physical host?

(total VCPUs / smallest ﬂavor #VCPUs) = maximum # of
instances

Double or quadruple this to account for growth - size of
ﬁxed network range

Mind
Networking

We can build a cloud with
2 networks (3 if using
ﬂoating IPs)

Host Network
(physical machine
access, OpenStack
services)

Fixed Network
(instance network)

Floating network

Mind Networking is the important
Networking
part, get it right!

ﬂoating IPs)

Host Network
(physical machine
access, OpenStack
services)

Fixed Network
(instance network)

Floating network

Networking
part, get it right!

ﬂoating IPs)

Host Network
(physical machine Easy to add physical nodes
access, OpenStack and/or networks
services)

Fixed Network
(instance network)

Floating network

Networking
part, get it right!

ﬂoating IPs)

Host Network
services)

Don’t try to change the ﬁxed
Fixed Network network once in production
(instance network)

Floating network

Networking
part, get it right!

floating IPs)

Host Network
services)

Don’t try to change the fixed
Fixed Network network once in production
(instance network)

Easy to add additional floating
Floating network networks

Mind
Glance

Disk space on server acting as glance backend (ﬁle based) will be
a limiting factor.

Good alternatives: Swift, CloudFiles, NFS (locally mounted)

Local disk is considerably faster than the alternatives

Will you be leveraging snapshots? If so, disk space will need
to be a serious consideration

If using qcow2, set “snapshot_image_format=qcow2“ to help
limit disk usage


Mind
Glance Performance

Network throughput is a limitation

1000Mb/s = 125MB/s max (expect ~112MB/s
realistically)

Large sequential read/writes - RAID5 may be preferred

Lean towards disk bandwidth over raw IOPs

Reduce # of images to allow for more efficient local caches on
compute nodes (dramatically increasing performance of
instance creation)


Mind
Glance Performance


realistically)



instance creation)
Image Size Not Cached Cached

1.4GB 20secs 1sec

16.4GB 2min 21secs 1sec RACKSPACE® HOSTING | WWW.RACKSPACE.COM

Mind
Glance Performance


realistically)



instance creation)
Image Size Not Cached Cached *times from
“creating image” to
1.4GB 20secs 1sec “qemu-img create”

16.4GB 2min 21secs 1sec RACKSPACE® HOSTING | WWW.RACKSPACE.COM

To Swift or not to Swift?


Pros

Scalable object storage that
works great as a backend
for Glance

Can be leveraged as object
storage for other parts of
the business

Ability to quickly increase
the amount of storage
available

Extremely stable if designed
correctly


Pros
Cons

Scalable object storage that
works great as a backend Additional expertise needed to
for Glance run Swift

Can be leveraged as object Architecture (network/swift
storage for other parts of components) design is
the business important to get right

Ability to quickly increase Depending on initial usage,
the amount of storage there may be high up front
available costs to populate 5 zones

Extremely stable if designed
correctly


Architecture Examples and
Thoughts
1 - 20 physical servers 20-50 physical servers

Single controller (single API Single controller (single API
endpoint, single scheduler, etc) endpoint, single scheduler,
should suffice etc) should suffice

Single network (1Gbps) for Investigate Swift as a glance
instance connectivity and backend.
OpenStack services is sufficient
Start looking into ways to
Rackspace “Alamo” installer break apart various controller
services


Thoughts
50-100 servers

Keep an eye on the scheduler
to make sure it’s not a
bottleneck

Strongly consider swift
especially for snapshots

Consider Availability Zones/
Cells (didn’t make it into
Folsom)

Consider “frontend” and
“backend” networks for


Thoughts
50-100 servers

Keep an eye on the scheduler
to make sure it’s not a
bottleneck

Strongly consider swift
especially for snapshots

Consider Availability Zones/
Cells (didn’t make it into
Folsom)

Consider “frontend” and two or more instance networks?
“backend” networks for

Set
“use_single_default_gateway”
in nova.confRACKSPACE® HOSTING | WWW.RACKSPACE.COM

Performance Considerations and
Bottlenecks
IO

20-40 instances per
physical server causes high
random IO

Reduce IO as much as
possible - i.e. centralized
logging

Can be further mitigated
with Cinder


Performance Considerations and
Bottlenecks
Async&Random&IO&
IO rs/speed/test12"(cfq,"host"deadline,"cache=none)"

Rs/speed/test13"(noop,"cache=writeback)"

20-40 instances per
rs/speed/test13"(cfq,"cache=writeback)"

physical server causes high
Rs/speed/test12"(noop"cache=none)"

randW"(direct)"

random IO
Rs/speed/test12"(cfq"cache=none)"
randR"(direct)"

randW"
Rs/speed/test13"(cfq,"cache=none,"no"ht)"
randR"

Rs/speed/test13"(deadline"cache=none)"

Reduce IO as much as compute/host"(deadline)"

possible - i.e. centralized compute/host"(no"ht)"

logging compute/host"

0" 200" 400" 600" 800" 1000" 1200" 1400" 1600"

Host&vs.&Instance&
14000"

Can be further mitigated
with Cinder
12000"

10000"

8000"

compute/host"
6000" Rs/speed/test12"(cfq"cache=none)"

4000"

2000"

0"
randR" randW" randR" randW" seqR" seqW"RACKSPACE® HOSTING
seqR" seqw" | WWW.RACKSPACE.COM
(direct)" (direct)" (direct)" (direct)"

Final Thoughts

Lessons learned

Standardize on a design that works for your organization

Find the right questions to ask

Important to understand OpenStack as a whole

OpenStack is still changing often, keep up to date with
current state of the projects


But....

But this is a design summit also

Open to discussions/thoughts/questions


Rackspace is hiring
www.rackertalent.com

RACKSPACE® HOSTING | 5000 WALZEM ROAD | SAN ANTONIO, TX 78218
US SALES: 1-800-961-2888 | US SUPPORT: 1-800-961-4454 | WWW.RACKSPACE.COM

RACKSPACE® HOSTING | © RACKSPACE US, INC. | RACKSPACE® AND FANATICAL SUPPORT® ARE SERVICE MARKS OF RACKSPACE US, INC. REGISTERED IN THE UNITED STATES AND OTHER COUNTRIES. | WWW.RACKSPACE.COM

Considerations for building your private cloud rackspace

Recommandé

Recommandé

Contenu connexe

Dernier

Dernier (20)

En vedette

En vedette (20)

Considerations for building your private cloud rackspace

Notes de l'éditeur