Cloud Foundry and OpenStack - World`s Top Two Open Source Projects!

Try IBM Bluemix for free, follow this link https://ibm.biz/Meetup_BayArea
http://www.meetup.com/OpenStack http://www.meetup.com/CloudFoundry
Cloud Foundry & OpenStack – World`s Top 2 Open Source Cloud Projects!
Apps
@AnimeshSingh

Cloud Foundry & OpenStack - Top Two Open Source Cloud Technologies!
http://analystpov.com/cloud-computing/top-15-open-source-cloud-computing-technologies-2014-24727

OpenStack’s Phenomenal Growth
COMPANIES
ACTIVE MEMBERS
461
18152
COMMITS
130K
Source: http://www.ohloh.net/p/openstack
2013 OCT 17
Release: Havana
1,729,137 lines of code
2011 SEP 22
Release: Diablo
478,671 lines of code
2014 APR 17
Release: Icehouse
LINES OF CODE
2M
2013 SEP 27
Release: Folsom
667,895 lines of code2012 APR 5
Release: Essex
558.368 lines of code
2013 APR 4
Release: Grizzly
2011 APR 3
Release: Cactus
2011 FEB 3
Release: Bexar
2010 OCT 21
Release: Austin
2014 OCT 16
Release: Juno

OpenStack Overview:
Collection of well integrated IaaS modules:
•  Compute (Nova)
•  Networking (Neutron)
•  Object Storage (Swift)
•  Block Storage (Cinder)
•  Identity(Keystone)
•  Image Service (Glance)
•  Dashboard (Horizon)

Meets Developer’s Needs
Focus on app development, not
provisioning VMs, databases,
messaging servers, etc
Agile development model
Deploy and scale in seconds
Open Cloud Platform
There is an increasing appetite for cloud-
based mobile, social and analytics
applications from line-of-business
executives - drives the need for a more
open cloud development platform
Compelling Community
Cloud Foundry has a compelling community
and emerging ecosystem as well as a mature
set of capabilities and robustness
Platinum Founding Sponsors
1.3k 800k
LINES OF CODETOTAL CONTRIBUTORS
Cloud Foundry Overview

Eclipse
IDE

CLI

Browser

cf
push

h4p

Cloud Foundry Developer Experience

Eclipse
IDE

CLI

Browser

cf
push

h4p

Cloud
Controller

(API)

Router

DEA
Pool

DEA
Pool

Droplet
Execu>on

Agent
(DEA)

VM
Pool

Health

Manager

NATS

(Message
bus)

Cloud Foundry Architecture

Integrate !

PaaS
Cloud
Integration
IaaS
UAA

Router

DEA
Pool

Apps

Service
Connector

Health
Manager

Messaging

Cloud
Controller

Build
Packs

Service
Nodes

BOSH

Cloud
Provider
Interface

Cloud Foundry , OpenStack Integration
•  Static / floating ips
•  Persistent disks
•  Custom VM
Configurations
•  Specialized Security
groups
•

BOSH deployment process
Deployment Manifest
•  Release name/version
•  # VMs, job params
•  Stemcells to use
Stemcell
•  Base OS
•  BOSH agent
Release
•  Name
•  Software packages
•  Config templates
•  Scripts
BOSH
Deployed
Cloud Foundry
Virtual Machine
•  Configuration
•  Software Packages
Virtual Machine
Virtual Machine
Virtual Machine
•  Software packages

Sample manifest

Open, by design
Building on open from the ground up enables IBM to maximize client investment
Hybrid CloudPrivate Cloud Off-premise cloud
IaaS
PaaS
SaaS
OAuth

IBM Contributions to OpenStack: Road To Juno
Source http://www.stackalytics.com/
Essex

54

Core
Contributors:
1

Technical
Contributors:
2

Commits:
9

Blueprints:
0

Projects:
6

Folsom

Core
Contributors:
4

Technical
Contributors:
18

Commits:
181

Blueprints:
9

Projects:
20

100

Grizzly

270

Core
Contributors:
10

Technical
Contributors:
38

Commits:
961

Blueprints:
35

Projects:
33

Havana

Core
Contributors:
13

Technical
Contributors:
85

Commits:
1595

Blueprints:
71

Projects:
48

380

Total
IBMers

Key
Contribu8ons:

Chinese
Transla>on

Nova
Hygiene

Storage
Enhancements

Total
IBMers

Key
Contribu8ons:

Integra>on
Tests

Crowd
Sourced
Transla>on

Membership
Services

Key
Contribu8ons:

API
Stability

Storage
Enhancements

21%
of
Nova
design
features

Total
IBMers

Total
IBMers

Key
Contribu8ons:

Enterprise
Security

Ceilometer

Quality
Assurance

Icehouse

380

Core
Contributors:
14

Technical
Contributors:
107

Commits:
1722

Blueprints:
85

Projects:
61

Key
Contribu8ons:

Quality
Assurance

Authen>ca>on
&
Security

15%
of
Compute
features

Total
IBMers

Juno

Core
Contributors:
15

Technical
Contributors:
109

Commits:
1669

Blueprints:
48

Projects:
78

Key
Contribu8ons:

Federated
Iden>ty

Block
Volume
Replica>on

Dashboard

Enhancements

400
Total
IBMers

IBM BlueMix (powered by Cloud Foundry)
http://bluemix.net
§  IBM initiative to develop a
Platform as a Services offering
§  IBM and partner cloud services
§  Integrated DevOps with both
Browser and Eclipse-based
tools
Services

Lifecycle

Management

(JazzHub)

Applica>on

Run>me

Run>mes
&

Frameworks

Middleware
Applica>on
Opera>onal
Mobile
External
Data

node
java
ruby
Worklight

WebSphere

Liberty

Web
IDE

(Eclipse
Orion)
Eclipse
IDE

Applica>on

Composi>on

Environment

Create
&
Manage
Services

Test/Run
Test/Run

Explore

Services

Explore

Services

IBM
Bluemix ™

Check
In
Code
Check
In
Code

Bluemix allows developers to focus on the code
Cloud Foundry provides unprecedented flexibility & power
Design Thinking
Containers
Extreme
Agile
MobileIoT
APIs
Microservices

OpenStack Swift Object Storage Service Available in Bluemix!

Docker Container Service running on OpenStack Available in Bluemix!

Automate the Lifecycle (Leverage the power of community) !

Leverage the power of Open Source Community
•  Open technologies give us a tremendous amount of power and flexibility to integrate them together
seamlessly
•  Lets see some of the examples of what we can do with the power of Open technologies to go from
bare metal hardware to a deployed Cloud Foundry ready to push apps
Open Source for Cloud Foundry and OpenStack Automation!

OpenStack Installation:
•  Leverage the open source Chef Cloud infrastructure Automation framework
•  Requires information about hardware, network environment and software repositories.
•  Automate OpenStack installation in approximately 15 minutes!
Chef for OpenStack Install Automation
Setup Compute
Setup Network
Setup Controller
Setup Storage

OpenStack Discovery:
•  Leverage the open source Fog gem to discover OpenStack artifacts in an automated manner
•  Require OpenStack credentials and discover OpenStack compute and network information.
Fog for OpenStack Discovery Automation
Discover VM
Configuration Sizes
Discover Network
Subnets
Discover Network
Security Rules
Discover DHCP , DNS
Gateway and floating IPs
Discover Security
Credentials

Fog for OpenStack Setup Automation
Cloud Foundry Pre-req setup on OpenStack:
•  Leverage the open source Fog gem to setup Cloud Foundry requirements in an automated manner
•  Setup according to best practices and guidelines – still giving users the flexibility to change if desired
Create Security
Credentials
Create VM configs for
Router, DEAs, Cloud
Controller, Service
Nodes
Create network
Security Rules
Setup tenant quota

Community stemcell image had empty fstab and this makes the root partition auto detection to fail
1.  Inject file into stemcell
Mount stemcell image using qemu-nbd - Inject /etc/fstab into mounted os
2.  Or Modify Openstack Configuration:
Change “libvirt_inject_partition” parameter in nova.conf from -1 (autodetect) to 1 (use first partition)
3. Or Install cloud-init in the image
Best solution to make it work in conjunction with metadata service
Modify Stemcells
Techniques for Stemcell Automation

BOSH and Ruby for Cloud Foundry Deployment Automation
Cloud Foundry Deployment Automation
•  Automate base OS image creation or modification
•  Automate Cloud Foundry deployment manifest file genration using Ruby ERB
•  Automate upload of Cloud Foundry core release, services and runtime frameworks, followed by Cloud
Foundry deployment
Stemcell Creation
and Upload
Generate BOSH and
Cloud Foundry
Manifest
Upload Cloud
Foundry core,
Services and runtime
Deploy Cloud Foundry
Deploy Microbosh

RUBY

BOSH

Automate end to end deployments in under an hour !
•  Previous example signify the power of Open source community
•  Leveraging tools like Chef, Fog, BOSH, Ruby ERB from the community can help create powerful
automation which can be repeatedly and consistently replicated in under an hour!
Open way is the only way!

BOSH

RUBY

Automate the Update/Upgrade!

Automate the update/upgrade lifecycle operations
•  Updates can be automated using code from the initial automated deployment (e.g. bosh deploy)
•  Follow te same workflow – do OpenStack discovery operation, and then leverage BOSH for update/
upgrade/
•  To ensure application availability throughout the update, use tools like JMeter to test application
responsiveness
Updates/Upgrades Automation!

Application Uptime during Updates/Upgrades Automation!

Scale and Optimize

Sample CF sizing for 1,000 small applications
~60 virtual machines
•  20 high memory
DEAs
•  11 CF fabric
components
•  26 service broker and
service nodes
160 CPU
~500 GB memory
~1.5 TB VM disk
~200 GB block storage

Scaling OpenStack
Optimize Internal Communication
•  Configure OpenStack for scaled concurrency
Optimize Performance
•  Configure OpenStack scheduler to evenly distribute load
Setup Highly Available Architecture for PaaS workloads
Load Balancer
Nodes
Data
Nodes
3x
Compute
Nodes
15x
Storage
Nodes
2x
Controller
Nodes
2x 6x

Scaling Cloud Foundry / BOSH
Optimize Internal Communication
•  Configure messaging bus for VM communication
Optimized routing and bandwidth allocation
•  Isolate Cloud Foundry components using multiple networks
Maintain Cloud Foundry’s Highly Available Architecture
Service
Gateways
10x
Routers
3x
Service Nodes
15x
DEAs
20x
Cloud
Controllers
2x

Deploy optimization tips
•  Increase OpenStack API rate limits (/etc/nova/api-paste.ini)
[filter:ratelimit]
paste.filter_factory = nova.api.openstack.compute.limits:RateLimitingMiddleware.factory
limits = (POST, "*", .*, 9999, MINUTE);
(POST, "*/servers", ^/servers, 9999, DAY);
(PUT, "*", .*, 9999, MINUTE);
(GET, "*changes-since*", .*changes-since.*, 9999, MINUTE);
(DELETE, "*", .*, 9999, MINUTE)
BOSH
BOSH
Administrator
NATS

PostGres

Redis

Blob
Store

Health
Monitor

Director

Registry

CloudProviderInterface
(CPI)
Cloud Foundry Environment
mongo_gw

mysql_gw

redis_gw

postgres_gw

cloud_ctrler

ccdb_ng

router

nats

debian_nfs

dea

uaa_db

uaa

loggregator

health_mgr
rabbit_gw

mongo_node

mysql_node

redis_node

postgres_node

rabbit_node

OpenStack
Environment
OpenSt
ackAPI

Deploy optimization tips
•  Increase BOSH’s NATS timeout
•  A large number of BOSH agents can overwhelm NATS
•  Place the following in your MicroBOSH manifest file:
apply_spec:
properties:
nats:
ping_interval: 30
ping_max_outstanding: 30
BOSH

NATS

mongo_gw
mysql_gw
redis_gw
postgres_gw

cloud_ctrler

ccdb_ng

router

nats

debian_nfs

dea

uaa_db

uaa

loggregator

health_mgr

rabbit_gw

mongo_node
mysql_node
redis_node
postgres_node
rabbit_node

Security optimization tips
Separate your VM network and OpenStack management network
•  One challenge with BOSH VMs deployed on OpenStack
•  Setup pin holes for those particular VMs (Director, OpenStack Registry)
Separate Cloud Foundry’s core fabric from other brokered services
•  Services connected using the service broker should be on a separate network
•  Prevents compromised services from effecting Cloud Foundry

Why Cloud Foundry and OpenStack are a great fit ?
•  100% Open PaaS and IaaS solutions – No vendor lock-ins
•  Strong and growing community of contributors and sponsors on both sides
•  Power of Open Source community can be leveraged to automate the deployment and
lifecycle management of Cloud Foundry on OpenStack
•  OpenStack meets Cloud Foundry integration requirements, and is totally configurable and
adaptable to handle the scale of a PaaS solution like Cloud Foundry
•  Bottom Line: They are a great fit!

What next? Join us at our Silicon Valley Meetups
http://www.meetup.com/BlueMix/
http://www.meetup.com/CloudFoundry/
For more technical details, refer to the slides and video here
http://www.slideshare.net/AnimeshSingh/optimizing-cloud-foundry-and-openstack-for-large-scale-deployments
https://www.openstack.org/assets/presentation-media/A-Practical-Approach-to-HA-Final.pdf
https://www.youtube.com/watch?v=jCwtV9n_ak4
https://www.youtube.com/watch?v=FsrCGkBo4Vg

Cloud Foundry and OpenStack - World`s Top Two Open Source Projects!

Recommandé

Recommandé

Contenu connexe

Dernier

Dernier (20)

En vedette

En vedette (20)

Cloud Foundry and OpenStack - World`s Top Two Open Source Projects!