This document provides an overview of hybrid cloud computing and outlines the steps to build a private or hybrid cloud using the open source Eucalyptus cloud platform. It describes the key components of Eucalyptus including the cloud controller, nodes, and networking configuration. The document also summarizes how to use the euca2ools to manage Eucalyptus resources such as images, instances, volumes, and security groups.
1. Hybrid Cloud Tutorial:
From Planning to Classroom
David Rilett and Charlie Wiseman
Department of Computer Science and Networking
Wentworth Institute of Technology
2. Heading into the Cloud
• Cloud computing means many different things
today
• This talk will do two things:
– Provide an overview of the cloud world
– Go through the procedure to build a private or
hybrid cloud platform using Eucalyptus, an open
source cloud system that is compatible with
Amazon Web Services (EC2/S3)
3. What was old…
• Cloud computing is the evolution of many
other computing paradigms
– Grid, Utility, Mainframes, …
• Virtualization is the key enabler
– Better utilization of physical servers and storage
with fast backup recovery
– Simpler migration of virtual instances
– Easier provisioning on demand
4. Big Picture
End Users
Application Cloud Consumer
Cloud Provider
5. Example
• Many R&D businesses rely on cloud
environments for both long -term and short-
term services
• For example:
– Deployed 50,000 cores of computing power for 3
hours
– Cost $15,000
6. Types of Cloud Models
• Cloud computing systems are characterized in
two ways: deployment model and service
model
• The deployment model describes who the
cloud consumers are
• The service model describes what types of
resources are provided to the cloud consumers
7. Deployment Models
• Public cloud
– Resources are shared by many consumers and are provided to
anyone (for a fee)
• Private cloud
– Resources are utilized by a single organization
– Can still be operated by a third party
• Hybrid cloud
– Mix of public and private, allowing the consumer to deploy
applications using either public or private resources
8. Service Models
• Infrastructure as a Service (IaaS)
– Virtual machines and storage are provided
– Consumers install their own OS from the ground up
• Platform as a Service (PaaS)
– Preconfigured computing platforms are provided
– For example, pre-built web systems
– Consumers only write the application
• Software as a Service (SaaS)
– Everything through the application is provided
– Consumers can only customize and tweak
9. IaaS or PaaS?
• Depending on the requirements, you should be
using an IaaS or PaaS model
– Give students a preconfigured image with
everything they need on it (PaaS)
– Allow them to install their own OS and do all the
configuration themselves (IaaS)
• The former is more likely in a software
development course, the later more likely in a
networking or administration course
10. Public, Private, or Hybrid?
• Benefits of public clouds like Amazon Web
Services (AWS):
– Requires no physical resources at your institution
– No limit on how many instances you can run
• Drawback of using a public cloud:
– Pay per usage (typically per server hour and per
GB of data transfer)
– For example: http://aws.amazon.com/ec2/pricing/
11. Public, Private, or Hybrid?
• Benefits of building your own private cloud:
– Full control over the resources
– No per use fees
• Drawbacks of a private cloud:
– Have to buy all the equipment up front
– Can't exceed the resources you buy (locally)
12. Public, Private, or Hybrid? (2)
• A common solution, therefore, is to use a
hybrid environment
• Build your own private cloud with a reasonable
number of resources
• Do all development and initial testing in the
private cloud
• For scale-out testing or demos, move the
applications to the public cloud
13. Enter Eucalyptus
• Eucalyptus is an open source IaaS cloud
platform
• Compatible with AWS (EC2/S3)
• Deploy your own private Eucalyptus cloud
• Move the Eucalyptus images to AWS if you
need more resources
• RightScale, euca2ools and HybridFox help with
moving and management
14. Building a Eucalyptus Cloud
• Requirements:
– At least two servers (more depending on how many instances
you need to support)
– Cooperation of your friendly local network administrator for
public IP addresses and firewall configuration
• Recommendations:
– External storage array for storing images
– Switch for building a separate private network for the cloud
15. Building a Eucalyptus Cloud
• Eucalyptus website provides a lot of information but may
lack adequate explanation
– http://open.eucalyptus.com/wiki/EucalyptusInstallation_v2.0
• There is some terminology to get out of the way
• You need one server to act as the front-end, which will run
all of the cloud-level services and store S3 snapshots in
Walrus
• Other servers are nodes for deploying instances of virtual
machine images and EBS volumes
16. Building a Eucalyptus Cloud
• The most important decision you have to make
is which networking mode you are going to use
with Eucalyptus
• There are four networking modes to choose
from
• We recommend a setup with the MANAGED-
NOVLAN mode as it provides good flexibility
without too much extra hassle
17. Networking Modes
• MANAGED-NOVLAN mode
– Require a switch to interconnect the private cloud network ( or
crossover cable to get one node up)
– The nodes will not be connected to your regular network directly
but use private network
• MANAGED mode is similar
– Requires VLAN-configured switches (not too difficult)
– The only benefit is layer 2 isolation of instances which is less
important for a course
18. Networking Modes
• The other two modes are STATIC and SYSTEM
• STATIC allows you to configure IP addressing by hand in the
cloud
– Convenient but very hard to manage in even a small environment
• SYSTEM mode should only be used if can't get a switch for
MANAGED-NOVLAN mode
– Requires more cooperation with your local network admin since
not managed by Eucalyptus
19. Network Configuration
• From now on, we will be using MANAGED-
NOVLAN mode
• One very tricky part that isn't explained well in
the documentation is that there are really
three different sets of IP addresses you must
work with
– Public, Private, and Virtual Instance IPs
21. Virtual Instances IPs
• When you configure Eucalyptus on the front-end, you have
to tell it the public and private IP addresses of the front-
end
• You also have to choose a third set of IP addresses used for
the virtual network of instances running on the nodes
• This must be separate from the other IP ranges
• Assuming you use 192.168.0.0/24 for the private ones, use
something like 10.0.0.0/16 for the virtual IPs
22. Additional Public IPs
• Lastly, you need an additional set of public IP
addresses that can be assigned to instances
that want to be accessible from outside of the
private network
• Ask your network admin for these
– For example, might get 5.5.5.10 – 5.5.5.50
– Must ensure that regular network doesn't use
these addresses already and that firewalls/routers
(DNS?) know about them
23. Example Configuration File
• Example virtual network configuration on the
cloud controller (eucalyptus.conf):
VNET_PUBINTERFACE="eth0"
VNET_PRIVINTERFACE="eth1"
VNET_DHCPDAEMON="/usr/sbin/dhcpd"
VNET_MODE="MANAGED-NOVLAN"
VNET_SUBNET="10.10.0.0"
VNET_NETMASK="255.255.0.0"
VNET_DNS="5.5.1.1"
VNET_ADDRSPERNET="32"
VNET_PUBLICIPS="5.5.5.10-5.5.5.50"
24. Other Gotchas *
• Installing the front-end (we used CentOS 5.8)
– Be sure to disable SELinux
– Enable NTP
– Install a DHCP server, but DON'T enable it or
configure it
– Configure both public and private NICs statically
* Faststart manages this for you
25. Other Gotchas*
• Installing the nodes (we used CentOS 5.8)
– They won't have Internet access in the given configuration, so you
could temporarily connect them to your regular network until the
install is done
– Disable SELinux
– Disable the local firewall (iptables)
– Enable NTP
– Set the IP address statically
– After you install xen, you might have to edit the GRUB configuration
to tell it to boot the xen kernel
– Start with a single node until things are working to simplify
debugging * Faststart manages this
26. Faststart Install CENTOS 5.7 +
EucalyptusFastInstallGuide-CentOS.pdf
• Need: CentOS 5.7 install CD #1. You can find the image on one of the mirrors here:
http://isoredirect.centos.org/centos/5/isos/x86_64/. The file is labeled CentOS-5.7-x86_64-bin-1of8.iso.
• Need: Eucalyptus Fast Start media (1GB USB) Video: http://vimeo.com/25163094
• On install, do network interface configuration, select “Edit” and manually configure IP, netmask, hostname, gateway
and DNS. You will need to check the DNS in resolv.conf, the default route in /etc/sysconfig/network and you interface
setup in /etc/sysconfig/network-scripts/ifcfg-eth0 etc.
DEVICE=eth0
BOOTPROTO=none
HWADDR=78:2B:CB:3A:A9:3A
ONBOOT=yes
IPADDR=69.43.72.190
NETMASK=255.255.255.240
GATEWAY=69.43.72.177
• http://www.eucalyptus.com/sites/all/files/EucalyptusFastInstallGuide-CentOS.pdf
27. EucalyptusConfiguration
(/etc/eucalyptus/eucalyptus.conf)
• There are two of these, one on each machine: Frontend (cc
walrus) and Node Controller (nc)
• Restart /etc/init.d/eucalyptus-cc cleanrestart for cc when edit on
frontend
• Restart /etc/init.d/eucalyptus-nc restart for nc when edit node(s)
• Be careful with /etc/init.d/eucalyptus-cloud restart on Frontend --
May lose external addresses on VM instances or may forget to
take snapshots of data
28. eucarc run-time environment
. /root/.euac/eucarc (“source” it for euca2ools)
EUCA_KEY_DIR=$(dirname $(readlink -f ${BASH_SOURCE}))
export S3_URL=http://69.43.72.190:8773/services/Walrus
export EC2_URL=http://69.43.72.190:8773/services/Eucalyptus
export EC2_PRIVATE_KEY=${EUCA_KEY_DIR}/euca2-admin-f5052272-pk.pem
export EC2_CERT=${EUCA_KEY_DIR}/euca2-admin-f5052272-cert.pem
export EC2_JVM_ARGS=-Djavax.net.ssl.trustStore=${EUCA_KEY_DIR}/jssecacerts
export EUCALYPTUS_CERT=${EUCA_KEY_DIR}/cloud-cert.pem
export EC2_ACCESS_KEY='WKy3rMzOWPouVOxK1p3Ar1C2uRBwa2FBXnCw'
export EC2_SECRET_KEY='GbxmI31soVHbDr8o6dQpOwTAmlrrrisP1HSQ'
# This is a bogus value; Eucalyptus does not need this but client tools do.
export EC2_USER_ID='339620359308903565207410324383178424330'
alias ec2-bundle-image="ec2-bundle-image --cert ${EC2_CERT} --privatekey ${EC2_PRIVATE_KEY} --user
339620359308903565207410324383178424330 --ec2cert ${EUCALYPTUS_CERT}"
alias ec2-upload-bundle="ec2-upload-bundle -a ${EC2_ACCESS_KEY} -s ${EC2_SECRET_KEY} --url ${S3_URL} --ec2cert ${EUCALYPTUS_CERT}“
* Need lines in red to register with HyperFox etc.
31. euca2ools 3
• Registering disk image:
$ euca-bundle-image -i image.img --kernel eki-XXXXXXXX --ramdisk eri-XXXXXXXX
$ euca-upload-bundle -b mybucket -m /tmp/image.img.manifest.xml
$ euca-register mybucket/image.img.manifest.xml
• Launching and manage instances:
$ euca-run-instances emi-XXXXXXXX -k mykey -t c1.medium
$ euca-get-console-output i-XXXXXXXX
• Launching an instance associated with the security group “webservers” :
$ euca-run-instances emi-XXXXXXXX -k mykey -g webservers
32. euca2ools 4
• Connecting to the instance:
$ ssh -i mykey.priv user@ip
$ euca-reboot-instances i-XXXXXXXX
$ euca-terminate-instances i-XXXXXXXX
• Create new 10GB volume:
$ euca-create-volume -s 10 -z mycloud
$ euca-create-volume -s 10 --snapshot snap-ZZZZZZZZ --zone mycloud
• Attach a volume to a runing instance:
$ euca-attach-volume -i i-XXXXXXX -d /dev/sdb vol-YYYYYYYY
33. euca2ools 5
• Detach a volume:
$ euca-detach-volume vol-YYYYYYYY
$ euca-delete-volume vol-YYYYYYYY ### for windows only
• Snapshots:
$ euca-create-snapshot vol-YYYYYYYY
$ euca-delete-snapshot snap-ZZZZZZZZ
• Allocating IP address:
• $ euca-allocate-address
ADDRESS IP
• Releasing a public ip address from a specify user:
$ euca-release-address IP
34. euca2ools 6
MORE ADDRESS IP
• Associating a public ip address to a running instance
$ euca-associate-address -i i-XXXXXXXX ip
• Disassociating a public ip address from a running instance
$ euca-disassociate-address ip
• Creating a security group named “webservers”:
$ euca-add-group -d "Web Servers" webservers
• Adding a rule to the security group “webservers” allowing icmp and tcp traffic from a.b.c.d:
$ euca-authorize -P tcp -s a.b.c.d webservers
$ euca-authorize -P icmp -s a.b.c.d webservers
35. euca2ools 7
• Removing the rule for icmp traffic from the source ip a.b.c.d from the security
• group “webservers”:
$ euca-revoke -P icmp -s a.b.c.d webservers
• Deleting the security group “webservers”:
$ euca-delete-group webservers
SOURCE:
http://cssoss.wordpress.com/2010/05/10/eucalyptus-beginner%E2%80%99s-guide-%E2%8093-uec-edition-chapter-10-
%E2%80%93-euca%C2%A0commands
50. Troubleshooting
• There are many moving parts in Eucalyptus
and it can frustrating to fix
• The most common failure case is that you try
to run a new instance and it stays "pending"
for a long time before terminating
• When this happens, there are a few places to
check
51. Troubleshooting: DHCP
• If the instance does not get an IP address (public or private) then
mostly likely the DHCP assignments are not working
• When you actually run an instance, the cloud controller on the front-
end will start a dhcpd process to assign IP addresses to the new
instance
• If you do a 'ps –ef | grep dhcpd' and don't see anything, then DHCP is
not running during setup
• This almost certainly means that the VNET configuration is wrong
(VNET overlap with private network, etc)
• You can also look in /var/run/eucalyptus/net/ for the DHCP
configuration and lease files
52. Troubleshooting: Log Files
• The best place to look for clues is in the log files
• As with most log files, it can be hard to pull the
relevant information out of the noise
• On the front-end, look in
/var/log/eucalyptus/cc.log
• On the nodes, look in /var/log/eucalyptus/nc.log
53. Troubleshooting Example: cc.log
[DAY MON DD HH:MM:SS YYYY][004513][EUCAINFO ] DescribeResources():
resource response summary (name{avail/max}): m1.small
{0/0} c1.medium{0/0} m1.large{0/0} m1.xlarge{0/0} c1.xlarge{0/0}
• Look for the above "resource response summary"
messages
• They will tell you how many resources are available for
each instance size
• In the above, you see that it says there are 0 available and
0 total for each size which means that the front-end can't
talk to the nodes, probably because of a firewall problem
54. Troubleshooting Example: cc.log
• When the nodes and front-end can talk, you
should messages like this:
[DAY MON DD HH:MM:SS YYYY][004513][EUCAINFO ] DescribeResources():
resource response summary (name{avail/max}): m1.small{16/16}
c1.medium{16/16} m1.large{8/8} m1.xlarge{8/8} c1.xlarge{4/4}
• And like this from the individual nodes:
[DAY MON DD HH:MM:SS YYYY][011297][EUCADEBUG ]
refresh_resources(): received data from node=192.168.0.2
mem=16083/16083 disk=856137/856137 cores=16/16
55. Troubleshooting Example: nc.log
[DAY MON DD HH:MM:SS YYYY][006411][EUCAERROR ] walrus_request():
couldn't connect to host (7)
[DAY MON DD HH:MM:SS YYYY][006411][EUCAERROR ]
download retry 10 of 10 will commence in 4 seconds
• In general, look for the [EUCAERROR lines to find errors
that you need to address
• In the above example, it tells you that the node can't
contact the walrus component of front end (for S3 storage)
(i.e. snapshots and images)
• Again, this is probably a firewall issue, so be sure the
firewall is off on the node
56. Troubleshooting: Images
• Another issue is broken or incompatible images
• If Eucalyptus tells you that an instance is running but you can't log in,
try pinging it
• If you can't ping it, the instance didn't boot correctly
• You can test it with xen
– Log in to the node and run "xm list"
– You can then log in directly to the instance with "xm console INSTANCE"
– If it's waiting at a maintenance prompt then something is wrong with the image
– try another image
– Note, do a CTRL-] to exit the xen console
57. How Many Servers Do I Need?
• For most purposes, you should think about 1 core, 1GB of
memory, and 10GB of storage for each student
• So, if you have a course of 30 students, you need roughly
32 cores and 32 GB of memory (HDD’s are not costly)
• You can get a single server with dual hyper-threaded quad
core processors and 16GB of memory for ~$4k
• Add another $2k for the front-end and it's about $10k total
(but don’t forget switches etc.)
58. Cloud in the Classroom
• There are many ways to bring the cloud into a course
• In software courses, every student could be assigned
their own virtual machine for development and testing
or run high performance or parallel applications
• In networking courses, students could create entire
virtual networks to study protocols and network
architectures
59. Conclusion
• If you are interested in setting up a private or hybrid cloud for a course, Xen
is reasonable option
• Eucalyptus/AWS are an infrastructure and web services platform in the
cloud
• Cloud networking is an important new tool for education, scientific
discovery and business logic services
• The benefit is that you can validate and verify locally before for-fee AWS
cloud with a larger implementation
• EBS coupled with S3 gives a fast and safe way to protect data.
• Freddy Fender “Wasted Days” is good Koala theme music