Cloud Foundry and Ubuntu: A Marriage Made in Heaven
1. Cloud Foundry and Ubuntu
one year of bliss for a marriage
made in heaven!
Patrick Chanezon
Senior Director
Developer Relations
chanezonp@vmware.com
http://twitter.com/chanezon Ubuntu Cloud Summit, May 2012
Tuesday, May 8, 12
2. P@ in a nutshell
• French, based in San Francisco
• Senior Director, Developer Relations,VMware
• Software Plumber, API guy, mix of Enterprise and
Consumer
• 18 years writing software, backend guy with a
taste for javascript
• 2 y Accenture (Notes guru), 3 y Netscape/AOL
(Servers, Portals), 5 y Sun (ecommerce, blogs,
Portals, feeds, open source)
• 6 years at Google, API guy (first hired, helped start the
team)
• Adwords, Checkout, Social, HTML5, Cloud
Tuesday, May 8, 12
4. Accelerando / Singularity, in a Galaxy far far away
§ Even if we automate ourselves out of a job every 10 years
§ ...I don’t think the singularity is near!
4
Tuesday, May 8, 12
5. Moore's Law is for Hardware Only
§ Does not apply to software
§ Productivity gains not keeping up with hardware and bandwidth
§ Writing software is hard, painful, and still very much a craft
5
Tuesday, May 8, 12
6. Moore's Law’s free lunch is over
§ Herb Sutter, Welcome to the Jungle
http://herbsutter.com/welcome-to-the-jungle/
6
Tuesday, May 8, 12
7. Future
“The future is already here
— it's just not very evenly
distributed”
William Gibson
7
Tuesday, May 8, 12
8. Haiku - Skylark
“Trampling on clouds,
inhaling the mist,
the skylark soars”
—SHIKI
8
Tuesday, May 8, 12
9. Haiku - Frog
“Crouching,
studying the clouds—
a frog”
CHIYO-JO
9
Tuesday, May 8, 12
14. Back to Client Server: Groovy Baby!
14
Tuesday, May 8, 12
15. What is Cloud Computing?
Cloud According to my daughter Eliette
15
Tuesday, May 8, 12
16. Cloud Stack - Classic Pyramid
Software
As A Service
Platform As A Service
Infrastructure As A Service
16
Tuesday, May 8, 12
17. Cloud Stack - By Value
Software
As A Service
Platform As A Service
Infrastructure
As A Service
17
Tuesday, May 8, 12
18. Cloud Stack - History
§ What does cloud mean, 4 main angles
• Software 1994 Netscape
• Infrastructure 2002 Amazon AWS
• Platform 2008 Google
• Development now!
§ Industrialization of hardware and
software infrastructure
like electricity beginning of 20th century
§ But software development itself is moving towards craftmanship
18
Tuesday, May 8, 12
19. Cloud started at Consumer websites solving their needs
• Google, Amazon, Yahoo, Facebook, Twitter
• Large Data Sets
• Storage Capacity growing faster than Moore’s Law
• Fast Networks
• Vertical -> Horizontal scalability
• Open Source Software
• Virtualization
• Cloud is a productization of these infrastructures
• Public Clouds Services: Google, Amazon
• Open Source Software: Hadoop, Eucalyptus, Ubuntu, Cloud Foundry
Tuesday, May 8, 12
21. IaaS/Virtualization getting mainstream
§ AWS, Joyent, Rackspace,...
§ Open Source projects: OpenStack, DeltaCloud, Eucalyptus
§ Automation: Chef, Juju
§ Standardization? DMTF
§ Inside the Firewall, Virtualization: VMware, Microsoft, Xen, KVM
§ 50% of workloads are virtualized
§ Easy to provision, manage instance...BUT
§ Still need to manage backups, software stacks, monitor, upgrades
21
Tuesday, May 8, 12
22. With Infrastructure, you still need to build your own platform
§ Need to build a distributed platform on top of you infrastructure
§ Story of the AWS meltdown from last summer
• http://blog.reddit.com/2011/03/why-reddit-was-down-for-6-of-last-24.html
• http://www.readwriteweb.com/cloud/2010/12/chaos-monkey-how-netflix-
uses.php
• http://news.ycombinator.com/item?id=2477296
• http://stu.mp/2011/04/the-cloud-is-not-a-silver-bullet.html
§ Twilio, Smugmug, SimpleGeo survived it because they built their
own distributed platform on top of IaaS
§ Enterprise customers want to consider Infrastructure like CDNs
• Multi Cloud usage
• Based on Open Source de facto standards, or full standards whenever that
happens
22
Tuesday, May 8, 12
23. Platforms
Let the wild rumpus begin!
23
Tuesday, May 8, 12
24. Platforms
§ Raise the Unit of currency to be application & services instead of
infrastructure
§ Google App Engine, Cloud Foundry, SalesForce Heroku, Cloudbees,
Amazon Elastic Beanstalk, Microsoft ex-Azure, AppFog
§ Single or a few languages, services
§ Start of Multi language Polyglot platforms
§ Enabler for Agile Developers -> Create Business value faster
§ Lack of standards: risk, vendor lock-in
§ Enterprise needs:
• Control, customizability
• Private/Hybrid Cloud
• Avoid lock-in
24
Tuesday, May 8, 12
26. Agility as a survival skill
§ Consumer software is becoming like fashion
• Phone apps, social apps, short lifetime, fast lifecycles
• Ab testing
§ Enterprise
• Clay Shirky, rise of situational apps
§ Kent Beck, Usenix 2011 Talk, “Software G-Forces”
change in software process when frequency grows
§ Cloud Platforms enables an Agile culture, driver for innovation
• Scalability is built in the platforms
• Can iterate faster
• Focus on design
§ Cloud Platforms lets developers focus on driving business value
26
Tuesday, May 8, 12
27. Main Risk: Lock-In
Welcome to the hotel california
Such a lovely place
Such a lovely face
Plenty of room at the hotel california
Any time of year, you can find it here
Last thing I remember, I was
Running for the door
I had to find the passage back
To the place I was before
’relax,’ said the night man,
We are programmed to receive.
You can checkout any time you like,
But you can never leave!
27
Tuesday, May 8, 12
28. Cloud Foundry: The Open PaaS
• Open Source: Apache 2 Licensed
• multi language/frameworks
• multi services
• multi cloud
Ap
ce
pli
vFabric
rfa
Postgres Private
ca
nte
Data
Services tio Clouds
rI
n
e
Se
Public
vid
vFabric
RabbitMQTM rvi
Cloud
ro
Msg Services
ce
dP
Micro
ou
Other
Cloud
Cl
Services
28
Tuesday, May 8, 12
30. Open Source Advantage: moves faster
§ http://code.google.com/p/googleappengine/issues/detail?id=13
• https://github.com/cloudfoundry/vcap/pull/25
30
Tuesday, May 8, 12
31. Open Source Advantage: diverse communities
§ Juju Charms to setup a multi node Cloud Foundry in 10 minutes
§ See Brian Thomason & Juan Negron’s UDS 2011 session
http://cloud.ubuntu.com/2011/09/from-zero-to-drawbridge-via-ubuntu-
server-ensemble-and-cloudfoundry-in-less-than-10-minutes/
31
Tuesday, May 8, 12
39. What is a Micro Cloud?
Or
Entire Cloud Running inside of a single VM
39
Tuesday, May 8, 12
40. What is in Micro Cloud Foundry?
.COM
Dynamic Updating DNS
App Instances Services
Open source Platform as a Service project
10.04
40
Tuesday, May 8, 12
41. Cloud Foundry is 1 year old
§ Services
• Caldecott – tunnel into your services, explore with standard client tools
• PostgreSQL, RabbitMQ
§ Frameworks, Runtimes, and Tools
• Java and Ruby Auto-Reconfiguration
• Scala, node.JS 0.6.*, Erlang, JRuby, PHP, Python, .NET, Spring 3.1, Grails 2.0,
Play 2.0
• Multi-Node Chef based deployment tools
• Maven Plugin, Eclipse Integration
• VMC manifests, Java Debugging, Rails Console
• Standalone applications
§ Micro Cloud Foundry x 3
41
Tuesday, May 8, 12
42. Cloud Foundry is 1 year old
§ Services
• Caldecott – tunnel into your services, explore with standard client tools
• PostgreSQL, RabbitMQ
§ Frameworks, Runtimes, and Tools
• Java and Ruby Auto-Reconfiguration
• Scala, node.JS 0.6.*, Erlang, JRuby, PHP, Python, .NET, Spring 3.1, Grails 2.0,
Play 2.0
• Multi-Node Chef based deployment tools
• Maven Plugin, Eclipse Integration
• VMC manifests, Java Debugging, Rails Console
• Standalone applications
§ Micro Cloud Foundry x 3
41
Tuesday, May 8, 12
47. Production Grade Cloud Foundry Clusters
§ 500 – 5,000 VMs cloudfoundry.com
§ 40+ unique node types
§ 75+ unique software packages
§ 75+ unique environments
§ 2x/week cf.com updates
§ 24x7x365 non-stop operation
§ No-downtime deployments
§ Reliable, robust, repeatable
deployments, updates, capacity
adjustments
§ Small teams manage many
instances production, staging, stress, qa, dev
Google style problem è Google style solution
46
Tuesday, May 8, 12
48. Cloud Foundry BOSH
§ Cloud Foundry BOSH is an open source tool-chain for release
engineering, deployment, and lifecycle management of large scale
distributed services
• Prescriptive way of creating releases and managing systems and services
• It is not a collection of shell scripts, not a pile of Perl
§ Built to deploy and manage production-class, large scale clusters
• Production grade Cloud Foundry clusters: 500+ VMs, 40+ jobs, 75+ packages
• Multi-node, multi-tier, complex clusters: e.g., our Gerrit/Jenkins Cluster
§ Built for devops usage and scale by a crack team of veterans
• A project, not a product: command line interface, YAML, etc.
• Continuous improvement, iterative development, rough edges
github.com/cloudfoundry/bosh
47
Tuesday, May 8, 12
49. BOSH: under the hood
BOSH User
“BOSH is deployed by BOSH”
bosh cli director healthmon
cloudfoundry.com
db redis nats
workers blobs active jobs
agent
IaaS CPI stemcell disk
48
Tuesday, May 8, 12
50. IaaS neutral by design
vSphere: battle tested implementation, thousands of
deployments CPI: code complete
vCloud Director: “work in progress”, 2H 2012 functional status: “work in progress”
Cloud Foundry BOSH
Cloud Provider Interface (CPI)
github.com/piston/openstack-bosh-cpi
contribute: github.com/cloudfoundry/bosh
49
Tuesday, May 8, 12
51. BOSH Resources
@DrNic
http://drnicwilliams.com/2012/04/16/creating-a-bosh-from-scratch-on-aws/
@BrianMMcClain
http://www.brianmmcclain.com/2012/05/08/using-bosh-with-vsphere-part-2/
https://github.com/cloudfoundry/bosh
https://groups.google.com/a/cloudfoundry.org/group/bosh-users/topics
https://github.com/piston/openstack-bosh-cpi
50
Tuesday, May 8, 12
56. Predictions
§ Software is becoming like fashion, design rules
§ Welcome to Babel, use the best tool for the job, embrace multiple
language & heterogeneity
§ Our jobs will change, build yourself out of your current job
§ Sysadmin jobs will morph, there will be less of them
§ Many opportunities open when you embrace change
55
Tuesday, May 8, 12
57. Things to Forget
§ First normal form
§ Waterfall model
§ Single server deployment
§ Single language skill
§ Build everything from scratch
§ Build custom infrastructure
56
Tuesday, May 8, 12
58. Things to Learn
• Agile
• Take risks, fail often, fail fast and learn
• API Design: create the API first
• UI Design, Javacript, HTML5, CSS3
• A/B Testing
• Open Source, Open Standards
• Architecture, Distributed Computing (CAP theorem, 8 fallacies)
• Cloud Platforms and APIs
• Multiple types of languages (imperative, object, functional, logic)
• Ability to encapsulate domain knowledge in a DSL
• Build on the shoulders of giants: reuse, REST APIs
• Pick your battles, choose what you need to build yourself to add value
• Learn to live in a box (embrace platform limitations) to think outside the box
• Use an App Store for distribution
• Consider your career as a startup
57
Tuesday, May 8, 12
60. Such stuff as dreams are made on
§ Like a kid on a candy store: there's never been a better time to be a
software developer!
§ Welcome to the Cloud: embrace change and reinvent yourselves
§ “The future is already there, not evenly distributed” Gibson
§ We Developers, invent the future today
59
Tuesday, May 8, 12
61. Cloud Foundry Resources
Primary Site : cloudfoundry.com
Open Source Site : cloudfoundry.org
Twitter : @cloudfoundry , hash tag #cfoundry
Blog : blog.cloudfoundry.com
FB : facebook.com/cloudfoundry
Support : support.cloudfoundry.com
• Documentation
• Knowledge Base
• Q & A / Forums
We’re hiring: www.cloudfoundry.com/jobs
60
Tuesday, May 8, 12
63. Thank You!
t @CloudFoundry
t @chanezon
Website : www.cloudfoundry.com
Blog : blog.cloudfoundry.com
Forum : support.cloudfoundry.com
Tuesday, May 8, 12
64. Books / Articles
§ Nick Carr, The Big Switch
§ Eric Raymond, The Art of Unix Programming
§ Weinberg, Psychology of Computer Programming
§ Wes python book
§ Mark html5 book
§ Kent Beck XP
§ Hunt, Thomas, The Pragmatic Programmer
§ Ade Oshineye, Apprenticeship Patterns
§ Matt Cutt's Ignite Talk IO 2011, Trying different things
§ Josh Bloch talk about api design
§ Larry and Sergey, Anatomy of a Search Engine
§ Rob Pike, The Practice of Programming
63
Tuesday, May 8, 12
65. Papers / Talks
§ Simon Wardley, Oscon 09 “Cloud - Why IT Matters”
§ Tim O’Reilly article on internet os
§ Peter Deutsch’s 8 Fallacies of Distributed Computing
§ Brewer’s CAP Theorem
§ Gregor Hohpe’s Starbucks Does Not Use Two-Phase Commit
§ Stuff I tag http://www.delicious.com/chanezon/
§ More specifically http://www.delicious.com/chanezon/cloudfoundry
§ My previous Talks http://www.slideshare.net/chanezon
§ My list of favorite books
http://www.chanezon.com/pat/soft_books.html
64
Tuesday, May 8, 12
66. Acknowledgement
§ Drawings from my daughters Eliette
§ Slides from Dave McCrory, Derek Collison, Duke Leto
§ Haiku from Addiss, Stephen; Yamamoto, Fumiko; Yamamoto,
Fumiko Y.; Yamamoto, Akira Y. (2011-06-22). Haiku: An Anthology of
Japanese Poems (Shambhala Library)
§ Painting Hiroshige, Sengai
65
Tuesday, May 8, 12
