34. Accelerando / Singularity, in a Galaxy far far away
§ Even if we automate ourselves out of a job every 10 years
§ ...I don’t think the singularity is near!
34
35. Moore's Law is for Hardware Only
§ Does not apply to software
§ Productivity gains not keeping up with hardware and bandwidth
§ Writing software is hard, painful, and still very much a craft
35
36. Predictions
“The future is already here — it's just
not very evenly distributed”
William Gibson
36
42. What is Cloud Computing? - End Users
42 Cloud According to my daughter Eliette
43. What is Cloud Computing? - Analysts
Software
As A Service
Platform As A Service
Infrastructure As A Service
43
44. What is Cloud Computing? - Developers
Software
As A Service
Platform As A Service
Infrastructure
As A Service
44
45. Cloud Stack - History
§ What does cloud mean, 4 main angles
• Software 1994 Netscape
• Infrastructure 2002 Amazon AWS
• Platform 2008 Google
• Development now!
§ Industrialization of hardware and software
infrastructure
like electricity beginning of 20th century
§ But software development itself is moving towards craftmanship
45
46. Cloud started at Consumer websites solving their needs
• Google, Amazon, Yahoo, Facebook, Twitter
• Large Data Sets
• Storage Capacity growing faster than Moore’s Law
• Fast Networks
• Vertical -> Horizontal scalability
• Open Source Software
• Virtualization
• Cloud is a productization of these infrastructures
• Public Clouds Services: Google, Amazon
• Open Source Software: Hadoop, Eucalyptus, Ubuntu, Cloud Foundry
47. Google research papers: good predictor of where BigData is going
• Horizontal scalability 2004: Map/Reduce (Hadoop)
• NoSQL 2006: Bigtable (Mongo, Cassandra, HBase, Riak)
• Real Time analytics 2010: Dremel, BigQuery (Impala)
• Horizontally Scalable SQL 2012: Spanner/ Amazon Redshift
49. IaaS/Virtualization getting mainstream
§ AWS, Joyent, Rackspace,...
§ Open Source projects: OpenStack, CloudStack, Eucalyptus
§ Automation: Chef, Juju, Cloud Foundry BOSH
§ Inside the Firewall, Virtualization: VMware, Microsoft, Xen, KVM
§ 60% of workloads are virtualized in 2012
§ Easy to provision, manage instance...BUT
§ Still need to manage backups, software stacks, monitor, upgrades
49
50. With Infrastructure, you still need to build your own platform
§ Need to build a distributed platform on top of you infrastructure
§ Story of the AWS meltdown from last summer
• http://blog.reddit.com/2011/03/why-reddit-was-down-for-6-of-last-24.html
• http://www.readwriteweb.com/cloud/2010/12/chaos-monkey-how-netflix-uses.php
• http://news.ycombinator.com/item?id=2477296
• http://stu.mp/2011/04/the-cloud-is-not-a-silver-bullet.html
§ Twilio, Smugmug, SimpleGeo survived it because they built their own distributed platform on
top of IaaS
§ Enterprise customers want to consider Infrastructure like CDNs
• Multi Cloud usage
• Based on Open Source de facto standards, or full standards whenever that happens
50
52. Platforms
§ Raise the Unit of currency to be application & services instead of infrastructure
§ Google App Engine, Cloud Foundry, Heroku, CloudBees, Amazon Elastic Beanstalk, Microsoft
Azure, AppFog
§ Single or a few languages, services
§ Start of Multi language Polyglot platforms
§ Enabler for Agile Developers -> Create Business value faster
§ Lack of standards: risk, vendor lock-in
§ Enterprise needs:
•Control, customizability
•Private/Hybrid Cloud
•Avoid lock-in
52
54. Agility as a survival skill
§ Consumer software is becoming like fashion
•Phone apps, social apps, short lifetime, fast lifecycles
•A/B testing
§ Enterprise
•Clay shirky situational apps
§ Kent Beck, Usenix 2011 Talk, “Software G-Forces: the effects of acceleration”
change in software process when frequency grows
§ Cloud Platforms enables an Agile culture, driver for innovation
•Scalability is built in the platforms
•Can iterate faster
•Focus on design
§ Cloud Platforms lets developers focus on driving business value
54
55. Main Risk: Lock-In
Welcome to the hotel california
Such a lovely place
Such a lovely face
Plenty of room at the hotel california
Any time of year, you can find it here
Last thing I remember, I was
Running for the door
I had to find the passage back
To the place I was before
’relax,’ said the night man,
We are programmed to receive.
You can checkout any time you like,
But you can never leave!
55
56. Cloud Foundry: The Open PaaS
• Open Source: Apache 2 Licensed
• multi language/frameworks
• multi services
• multi cloud
Ap
pli
vFabric
Private
ca
Postgres
ce
tio
rfa
Clouds
n
e
Data Services
Se
Int
rvi
Public
er
vFabric RabbitMQTM
ce
vi d
Msg Services Clouds
Int
ro
dP
e
Micro
rfa
ou
Clouds
ce
Cl
Other Services
56
58. Micro Cloud FoundryTM – Industry First Downloadable PaaS
Micro Cloud Foundry
Frameworks
Services
Your Laptop/PC
Single VM instance of
Cloud Foundry
that runs on a developer’s
MAC or PC
Tracks CloudFoundry.com
2 releases / week
58
72. Production Grade Cloud Foundry Clusters
500 – 5,000 VMs cloudfoundry.com
40+ unique node types
75+ unique software packages
75+ unique environments
2x/week cf.com updates
24x7x365 non-stop operation
No-downtime deployments
Reliable, robust, repeatable deployments,
updates, capacity adjustments
Small teams manage many instances
production, staging, stress, qa, dev
Google style problem è Google style solution
72
73. Cloud Foundry BOSH
Cloud Foundry BOSH is an open source tool-chain for release engineering, deployment, and lifecycle
management of large scale distributed services
• Prescriptive way of creating releases and managing systems and services
• It is not a collection of shell scripts, not a pile of Perl
Built to deploy and manage production-class, large scale clusters
• Production grade Cloud Foundry clusters: 500+ VMs, 40+ jobs, 75+ packages
• Multi-node, multi-tier, complex clusters: e.g., our Gerrit/Jenkins Cluster
Built for devops usage and scale by a crack team of veterans
• A project, not a product: command line interface, YAML, etc.
• Continuous improvement, iterative development, rough edges
github.com/cloudfoundry/bosh
73
74. BOSH: under the hood
BOSH User
“BOSH is deployed by BOSH”
bosh cli director healthmon
cloudfoundry.com
db redis nats
workers blobs active jobs
agent
IaaS CPI stemcell disk
74
75. IaaS neutral by design
vSphere: battle tested implementation, thousands of
deployments
CPI: code complete
vCloud Director: “work in progress”, 2H 2012 functional status: “work in progress”
Cloud Foundry BOSH
Cloud Provider Interface (CPI)
github.com/piston/openstack-bosh-cpi
contribute: github.com/cloudfoundry/bosh
75
77. Predictions
• Software is becoming like fashion, design rules
• Welcome to Babel, use the best tool for the job, embrace multiple language &
heterogeneity
• Our jobs will change, build yourself out of your current job
• Sysadmin jobs will morph, there will be less of them
• Many opportunities open when you embrace change
77
78. Things to forget
• First normal form
• Waterfall model
• Single server deployment
• Single language skill
• Build everything from scratch
• Build custom infrastructure
78
79. Things to learn
• Agile
• Take risks, fail often, fail fast and learn
• API Design: create the API first
• UI Design, Javacript, HTML5, CSS3
• A/B Testing
• Open Source, Open Standards
• Architecture, Distributed Computing (CAP theorem, 8 fallacies)
• Cloud Platforms and APIs
• Multiple types of languages (imperative, object, functional, logic)
• Ability to encapsulate domain knowledge in a DSL
• Build on the shoulders of giants: reuse, REST APIs
• Pick your battles, choose what you need to build yourself to add value
• Learn to live in a box (embrace platform limitations) to think outside the box
• Use an App Store for distribution
79