Keynote by Chirag Jog, 1 November 2012, CloudStack Hyderabad Meetup, Lemon Tree, Hyderabad

1.
InnovaGon
→
ExecuGon
→
SoluGon
→
Delivered
Cloud MigraGng
ApplicaGons
to
IaaS
Clouds
Computing
-­‐
Chirag
Jog
Enterprise
DevOps
Applica2ons
Public,
Private,
PaaS,
SaaS
Datacenter
US:
+1
408
556
9645
Management
India:
+91
20
661
43
400
Web:
h8p://www.clogeny.com
Email:
contact@clogeny.com
Clogeny
ConfidenGal

2. About
me
Chirag
Jog
• Co-­‐founder
and
CTO,
Clogeny
Technologies
• Experience
in
building
some
cuYng
edge
products
in
the
cloud.
• Extensive
experGse
across
IaaS,
PaaS
&
SaaS
–
20+
cloud
pla]orms
• Deep
experGse
in
Apache
Cloudstack
Pla]orm
• Several
deployments
for
marquee
customers
©
2012
Clogeny
Technologies
h8p://www.clogeny.com

3. Agenda
" IdenGfying
the
Candidates
for
the
Cloud
" Choosing
the
right
IaaS
cloud
" MigraGon
strategy
" Case
Studies
" Conclusion
©
2012
Clogeny
Technologies
h8p://www.clogeny.com

4. IdenGfying
the
Candidates
" Does
you
applicaGon
have
varying
degree
of
demands?
• Need
more
horsepower
at
specific
Gmes/seasons
" Do
you
expect
your
compute,
storage
and
network
requirements
to
grow
over
Gme?
• Avoid
upfront
costs.
• Scale-­‐up
when
required.
" Is
your
tesGng
and
QA
acGvity
restricted
to
the
hardware
you
can
afford?
• Provision
addiGonal
hardware
only
during
QA/release
cycle
" Support,
Licensing
costs
©
2012
Clogeny
Technologies
h8p://www.clogeny.com

5. Criteria
to
choose
the
right
IaaS
Cloud
" Service
Level
Agreements
(SLAs)
" ElasGcity
for
Compute
and
Storage
resources
" Networking
and
Security
features/services
" Virtual
Machines
and
Data
Backups
and
Restores
" Availability
Regions
and
Zones
" DevOps,
AutomaGon
" Monitoring
" Image
Management
" AddiGonal
services
–
Database
as
a
service,
Object
storage,
load
balancers
©
2012
Clogeny
Technologies
h8p://www.clogeny.com

6. Public
and
Private
Cloud
Providers
" Leaders
• Amazon
Web
Services,
Terremark,
Savvis
" Challengers
• Bluelock,
Gogrid,
Joyent
" Cloudstack-­‐based
• Ninefold,
TATA
Instacompute,
Instance.com
" Private
Clouds
• Cloudstack,
Eucalyptus,
VMWare
vCloud
Director
and
Openstack
©
2012
Clogeny
Technologies
h8p://www.clogeny.com

7. MigraGon
Strategy
Investigation
&
Automation
&
Final
Setup
Replication
Phase Optimization
Phase
Phase
Deployment
Cloud
Cluster
&
Cloud
Asset
Production
Planning Architecture Performance
Setup
Monitoring
&
Analysis
Automation
Review
current
and
Arrive
at
the
right
P e r f o r m a n c e
Create
the
components
Standard
and
custom
f u t u r e
b u s i n e s s
topology
that
suits
engineering,
clustering,
and
con>igurations
for
built
reports,
metrics
r e q u i r e m e n t s
a n d
customer
needs,
plan
l o a d
b a l a n c i n g ,
y o u r
c l o u d
a s s e t s
and
parameters
on
develop
relevant
plans
s t o r a g e ,
s e c u r i t y ,
s h a r d i n g ,
c a c h i n g ,
i n c l u d i n g
i m a g e s ,
resource
utilization
scripts,
automation,
for
cloud
adoption
production
automation,
managing
networks
and
and
demand
patterns
m o n i t o r i n g
a n d
including
choice
of
user
monitoring
tools
storage
performance
plus
load
balancer
for
m a n a g e m e n t
t o o l s .
c l o u d
v e n d o r
a n d
for
analysis
of
resource
Con>igure
HA
&
auto-­‐ your
services
on
the
platform usage scaling
cloud
©
2012
Clogeny
Technologies
h8p://www.clogeny.com

8. Re-­‐architecGng
for
the
cloud
" Scalability:
Horizontal,
VerGcal,
Geographical
" Loose
Coupling
" Monitoring
and
Logging
" Automate
and
operaGonalize
as
much
as
possible
" Design
for
Failure
©
2012
Clogeny
Technologies
h8p://www.clogeny.com

9. A
case-­‐study:
Social
commerce
pla]orm
The
customer
:
US-­‐based
VC-­‐funded
startup
in
the
social
commerce
and
social
shopping
recommendaGon
space.
Challenges:
" Hosted
on
non-­‐ElasGc
infrastructure
–
shared
hosGng
" 3rd
party
plugin
in
criGcal
path
–
Highly
Available
" 30-­‐40x
increase
in
traffic
during
holiday
season
–
ability
to
scale-­‐up/down
based
on
load
" AutomaGon
and
ease-­‐of-­‐use
" Reduce
TCO
©
2012
Clogeny
Technologies
h8p://www.clogeny.com

10. A
case-­‐study:
Social
e-­‐commerce
pla]orm
FaceBook
AnalyGcs
Server
Integra;on
Redis
Server
Tornado
WebServer
Redis
Node
Feed
User
Behavior
to
•
Mobile
Apps
Analy;cs
server
• Tablet
Apps
Nginx
FrontEnd
Webserver
Schedule
workers
to
RabbitMQ
Message
Broker
Local
Redis
Cache
perform
• Desktop
Browser
tasks
Pylons
Web
framework
Celery
Worker
MongoDB
User
Interface
StaAc
Content
Javascript/CSS
Serve
Sta;c
Store
all
Content
vendor,
through
a
customer
and
Apache
Webserver
separate
server
10
behavioral
data
in
MongoDB
3rd
Party
Monitoring
(Server
Density)
Exis;ng
Infrastructure
©
2012
Clogeny
Technologies
h8p://www.clogeny.com

11. MigraGon
Steps
–
InvesGgaGon
Phase
• Understanding
the
exisGng
architecture.
• IdenGfy
horizontal
scalable
components.
• UGlize
Cloudstack’s
features
most
effecGvely.
• Right-­‐sizing
of
the
servers:
§ Web
servers
:
Mostly
request
forwarders
§ Celery
workers:
CPU
Intensive
§ Datastores
:
Memory
and
CPU
Intensive
• Storage
configuraGon
for
Volumes:
§ RAID
configuraGon
§ Filesystem
type
§ No.
of
volumes
©
2012
Clogeny
Technologies
h8p://www.clogeny.com

12. MigraGon
Steps
–
ReplicaGon
Phase
" Setup
the
servers
and
Volumes
as
per
invesGgaGon
" Setup
the
webserver
and
workers
" Database
replicaGon
• MongoDB
node
set
up
as
slaves
in
replicaGon
mode.
• Redis
node
are
set
up
as
slaves
in
master-­‐slave
mode.
• Ensures
all
the
data
–
exisGng
and
live
data
is
replicated
automaGcally.
" FuncGonal
and
IntegraGon
tesGng
to
ensure
the
applicaGon
works
as-­‐is
©
2012
Clogeny
Technologies
h8p://www.clogeny.com

13. MigraGon
Steps
–
OpGmizaGon
Phase
" Database
Layer
• Convert
MongoDB
from
single-­‐node
to
mulG-­‐node
replica-­‐
set
for
equal
load
distribuGon
• Built-­‐in
facility
for
fail-­‐over
• Setup
Redis
as
a
master-­‐slave
with
manual
failover
" ApplicaGon
Server
Layer
• Move
all
the
resources
–
message
broker
(RabbitMQ)
and
workers
(Celery)
to
separate
nodes.
• Ability
to
scale
to
the
workers
(Celery)
based
on
load
" Web
Server
• Single
point
of
failure
• Leverage
Cloudstack’s
Load
Balancer
capabiliGes
©
2012
Clogeny
Technologies
h8p://www.clogeny.com

14. MigraGon
Steps
–
OpGmizaGon
Phase
" Monitoring
and
Alerts:
• Ability
to
monitor
all
the
components
• track
a
variety
of
metrics
to
pro-­‐acGvely
idenGfy
issues
in
producGon
setup
• Automated
Email/SMS
alerts
or
Mobile
Apps
for
noGficaGons
• SoluGons
:
Nagios,
Hyperic,
Cloudkick,
Server
Density
• Metrics
to
be
tracked
:
§ process
availability
§ server
availability
§ free/used
disk
space
§ free/used
memory
§ free/used
swap
©
2012
Clogeny
Technologies
h8p://www.clogeny.com

15. MigraGon
Steps
–
AutomaGon
Phase
" Servers:
Automated
Provisioning,
ConfiguraGon
and
Backup.
" Data-­‐stores
:
Automated
backups
with
scheduling
" Self-­‐managing
Virtual
Machines
" Tools:
Chef,
Puppet,
CfEngine,
Fabric
©
2012
Clogeny
Technologies
h8p://www.clogeny.com

16. AutomaGon
scripts
" Provision
all
the
type
of
servers,
configure
appropriate
services
and
storage
" Setup
a
instance
by
pulling
the
latest
code
" For
Webservers,
the
tool
configures
the
instance
and
a8aches
it
to
an
Load
Balancer
" For
MongoDB/Redis:
automaGcally
join
the
cluster
or
create
a
new
one.
" Data
Backups:
• Automate
the
backups
of
the
data
stores
arer
quiescing
the
datastore.
• Runs
arer
a
configurable
period
of
Gme.
•
Manage
exisGng
snapshots
and
delete
really
old
snapshots
©
2012
Clogeny
Technologies
h8p://www.clogeny.com

17. Cloud-­‐based
Setup
Send
Emails
Regular
backups
of
(for
Alerts)
Volumes
using
snapshots
FaceBook
IntegraGon
Monitoring
AnalyGcs
Server
&
Alerts
Redis
Server
Store
User
Master
–Slave
feeds
Tornado
Mode
WebServer
MongoDB
Feed
User
Replica
Set
Behavior
to
Mode
Analy;cs
server
Redis
Cluster
•
Mobile
Apps
Nginx
FrontEnd
• Tablet
Apps
Webserver
RabbitMQ
Load
Local
Redis
Cache
• Desktop
Balancer
Message
Pylons
Web
Schedule
Broker
framework
workers
to
perform
tasks
MapRed
uce
User
Interface
Celery
StaAc
Content
Worker
Workers
Javascript/CSS
Serve
Sta;c
Load
Content
through
a
17
Balancer
Apache
separate
server
Webserver
Worker
Role
Cloudstack-­‐based
Infrastructure
©
2012
Clogeny
Technologies
h8p://www.clogeny.com

18. A
case-­‐study:
VMWare
to
Cloudstack
MigraGon
" The
customer
is
a
leading
managed
soluGon
provider
in
North
America.
" Challenges:
• Migrate
100+
VMware
based
VMs
to
Citrix
CloudPla]orm
3.0.5
• Design
and
develop
a
workflow
to
scan
exisGng
VMWare
vSphere
inventory,
create
images(OVAs)
and
import
into
Cloudstack
as
per
accounts.
• MulG-­‐volume
OVA
import
into
Cloudstack
• Restoring
the
associaGon
between
public
IP
address
&
Virtual
Machines
©
2012
Clogeny
Technologies
h8p://www.clogeny.com

19. A
case-­‐study:
VMWare
to
Cloudstack
MigraGon
" Key
Features:
• Developed
a
inventory
scan
engine
using
vSphere
SOAP
APIs
to
generate
a
spec
of
the
complex
hierarchies
of
datacenters,
folders,
hosts
and
clusters.
• Developed
a
mulG-­‐step
workflow
to:
§
scan
a
virtual
machine
hardware
configuraGon
and
disk
layout
§
generate
a
snapshot
(VMDK)
of
the
virtual
machine
§
extract
per-­‐volumes
OVAs
as
per
Cloudstack’s
expectaGons
from
the
snapshot.
• Leverage
Cloudstack’s
Lazy
Assignment
of
IP
Addresses
to
allocate
a
specific
IP
address
for
an
account.*
*Don’t
try
this
at
home.
We
modified
the
Cloudstack
database
directly
©
2012
Clogeny
Technologies
h8p://www.clogeny.com

20. Thanks
!
Contact:
chirag@clogeny.com
©
2012
Clogeny
Technologies
h8p://www.clogeny.com