How automated cloud infrastructure setups can help with Continuous Delivery

“How automated cloud infrastructure setups
can help with Continuous Delivery”
Edmund Haselwanter, eh@cloudbau.de, @ehaselwanter

Donnerstag, 19. September 13

Why Continuous Delivery?
For most companies, IT functions as the nervous system and provides
an increasing amount of the organizational muscle mass. Most critical
business functions are entirely automated within IT, and 95% of all
capital projects depend on IT to get done. Today, nearly every business
decision will result in at least one IT change.
„IT REVOLUTION MANIFESTO“, http://itrevolution.com

Theory of
Constraints

→

→
E.M. Goldratt, 1984

cloudbau

Gene Kim, Kevin Behr,
George Spafford, 2013

Continuous Delivery
Wikipedia: http://en.wikipedia.org/wiki/Continuous_delivery

●
●
●
●

Version Control
Continuous Integration
Automated Testing
Continuous Deployment

cloudbau

We got the software development part right!
Request

Approve

Develop

Test

WaterScrum

Deploy

- Fall

The core, chronic conflict that every IT
leader faces is the need to
simultaneously enable faster time to
market (i.e., make as many changes as
you can), while providing stable, secure
and reliable IT services (i.e., make as
few changes as you can).
Gene Kim

„Lowering risk of change through tools and culture“
John Allspaw, Paul Hammond, 2009 (Flickr)

small changes, often
reproducible


culture change Dev+Ops

standardized

cloudbau

expect failure
feedback (e.g. tests/metrics/...)

The Last Mile: Dev/Test/Prod Parity
●
●
●
●
●
●
●
●

Hardware configuration (e.g. amount of RAM, HDD vs. SSD)
Software used (e.g. sqlite in dev, mysql in prod)
Software/library versions
Deployment (e.g. automated for test, manual for prod)
System configuration (debug flags, logging)
External dependencies (dev: payment mocked out vs. prod: real gateway)
Database contents (huge prod DB with millions of entries vs. few entries in dev)
Operating system (e.g. dev: Mac OS X, prod: Linux)

See also http://12factor.net/

cloudbau

The Last Mile: Dev/Test/Prod Parity
●
●

Log level “normal” to chatty on production

●
●

Changes in external dependencies (Sun Java to Oracle Java)

●
●
●
●

SQL Query no problem in dev environment but too slow for production data

●
●

Snowflake back-port: Live Tuning of Kernel/Mysql parameters, got lost after re-install

CI build broken due to minor version change in upstream mocking framework (0.4.1 to
0.4.2)

Sessions invalid in production due to time difference on multiple servers. No problem in
dev environment (just one app and one db server)

Different hardware: 32 vs. 64 bit. libraries missing
Different hardware: optimized CPU flags on CI system. build not working in prod
Staging In-house, Prod AWS Cloud: Latency ok during staging test, problem on
production

Development on Mac OSX/Windows, deploy on Linux: File not found (case sensitive)

See also http://12factor.net/

cloudbau

Getting to the solution...

cloudbau

Infrastructure as Code

Application

Environment

Infrastructure

cloudbau

Building Blocks: Tools and Culture
Config-DB

CPUVirt.
Binary-Repo

StorageVirt.
SDN

System Description
Orchestration

Prod Env

Programmable Infrastructure (Cloud)

Continuos Delivery

Anti-Fragile Organizations

Cross-Functional Teams

Processes and Culture

cloudbau

Dev Env

Executeable System Description

Continuous Build
DevOps

Stage Env

First... automate and equalize all
environments
●
●
●
●
●

Chef/Puppet/Cfengine/.... for config
Dependency management (for programming languages, packages etc.)
Dependency management for config scripts as well (e.g. Berkshelf for chef)
Everything in version control
Infrastructure can still be different...

cloudbau

... then use a cloud to equalize
infrastructure!

●
●
●

Encode the infrastructure as well
Describe all the pieces of the infrastructure in code
Now it’s code: it can be dynamic!

cloudbau

But ... There be dragons
Infrastructure as Code => We need a process

Source
Code
Test

X

Compiler
Artifact

cloudbau

The implementation patterns using OSS

● OpenStack / Eucalyptus / ...
● Chef/Puppet/Cfengine/ ...
● Jenkins (Building and testing the SW but also
infrastructures)

● NetflixOSS (Asgard + Aminate)

cloudbau

New Question in the Cloud Era: What is my
deploy Artifact

● The Software Package itself
● The whole VM
● A prepared container
● ...

cloudbau

The Software Package is the Artifact

● The build artifact is a binary.
● Use IaaS API to create the environment
● Deployment is after the server creation
● Deploy with automation or any other tool
● The whole VM
● “from source” is just a special case

cloudbau

HP Cloud + Jenkins + Puppet

Binary Repo

HP Cloud Servers
(OpenStack)

Jenkins

Puppet Modules

cloudbau

The VM itself is the Artifact

● Build artifact is a image (optimal for stateless service)
● Only install needed, there is no upgrade/deploy
● IaaS must support Loadbalancer as a service (or something similar)
● Orchestration for deployment steps needed

cloudbau

Jenkins + AWS + Asgard + Aminate + Chef

Jenkins

Aminate

AMIs

Asgard

+ Chef

AWS EC2 (ASG + ELB)

cloudbau

A container is the Artifact

●
●
●
●
●

cloudbau

Optimization of the VM idea
Transportable lightweight OS containers (e.g. LXC/docker)
Very often back to upgrade/deploy
Needs external orchestration as well
Very new toolchain emerging (docker/maestro)

https://www.docker.io/the_whole_story/

cloudbau

Jenkins + docker + cfengine + Aminate +
Chef

Jenkins

Docker

Host VM

Maestro

+ cfengine

AWS EC2 (ASG + ELB)

cloudbau

Cloud can help with developer environments
as well

●
●
●
●

cloudbau

Have your own environment
Brings the production setup to the developer
One description for local VM and Cloud environment (Dev/Prod)
Needs external orchestration as well

OpenStack + Vagrant + Chef
Chef Repo / Server

Binary Repo
Jenkins

Vagrant

cloudbau

OpenStack Servers

One more Thing: Other Areas Cloud can
help

● Persistent Data: Copy/Clone Production Data to test migrations
● Debugging: Developer can stand up whole environment (even with data)
● Backup/Restore (snapshot whole images/servers for rollback)

cloudbau

What’s missing in the OpenSource
Ecosystem?

● Better Orchestration/ Pipeline support in Jenkins (Commercial CI-Servers
are advanced here)

● OpenSource Orchestration Engines (like OpenStack Heat)
● NetflixOSS working on OpenStack

cloudbau

Don’t forget about Operations

● Cloud/App-Monitoring Solutions (Nagios won’t cut it in dynamic
environments)

● Logfiles
● Metric driven autoscaling

cloudbau

Thank you! Questions?

cloudbau

How automated cloud infrastructure setups can help with Continuous Delivery

Recommandé

Recommandé

Contenu connexe

Tendances

Tendances (20)

Similaire à How automated cloud infrastructure setups can help with Continuous Delivery

Similaire à How automated cloud infrastructure setups can help with Continuous Delivery (20)

Dernier

Dernier (20)

How automated cloud infrastructure setups can help with Continuous Delivery