Extending Cloud Foundry with Custom Services and Buildpacks

Extending Cloud Foundry
With Custom Integration
Cornelia Davis
Scott Frederick
© 2013 SpringOne 2GX. All rights reserved. Do not distribute without permission.
Thursday, November 21, 13

Who and What...

2

Who and What...

Extending the
set of services
Customizing
application containers
2

Services


Demo

4

When you cf push
VM for Service
Instances

Cloud
Controller

Router

CLI

Browse

5


Service
Broker
(i.e. mysql)

DEA

MySQL
(process)

When you cf push
VM for Service
Instances
provision (HTTP)

Cloud
Controller

Router

CLI

Browse

5


Service
Broker
(i.e. mysql)

DEA

MySQL
(process)

When you cf push
VM for Service
Instances
provision (HTTP)

provision (HTTP)

Cloud
Controller

Router

CLI

Browse

5


Service
Broker
(i.e. mysql)

DEA

MySQL
(process)

When you cf push
VM for Service
Instances
provision (HTTP)

provision (HTTP)

Cloud
Controller

Router

CLI

Browse

5


Service
Broker
(i.e. mysql)

DEA

up to you

MySQL
(process)

When you cf push
VM for Service
Instances
provision (HTTP)

provision (HTTP)

Cloud
Controller

bind (HTTP)

Router

CLI

Browse

5


Service
Broker
(i.e. mysql)

DEA

up to you

MySQL
(process)

When you cf push
VM for Service
Instances
provision (HTTP)

provision (HTTP)

Cloud
Controller

bind (HTTP)

Router

CLI

Browse

5


bind (HTTP)

DEA

Service
Broker
(i.e. mysql)

up to you

MySQL
(process)

When you cf push
VM for Service
Instances
provision (HTTP)

provision (HTTP)

Cloud
Controller

bind (HTTP)

Router

CLI

Browse

5


bind (HTTP)

DEA

Service
Broker
(i.e. mysql)

up to you

up to you

MySQL
(process)

When you cf push
VM for Service
Instances

credentials

provision (HTTP)

provision (HTTP)

Cloud
Controller

bind (HTTP)

Router

CLI

Browse

5


bind (HTTP)

DEA

Service
Broker
(i.e. mysql)

up to you

up to you

MySQL
(process)

When you cf push
VM for Service
Instances

credentials

provision (HTTP)

provision (HTTP)

CLI

Cloud
Controller

bind (HTTP)

bind (HTTP)

go ahead and
deploy (HTTP)

Browse

5


Router

...

DEA

Service
Broker
(i.e. mysql)

up to you

up to you

MySQL
(process)

When you cf push
VM for Service
Instances

credentials

provision (HTTP)

provision (HTTP)

CLI

Cloud
Controller

bind (HTTP)

bind (HTTP)

go ahead and
deploy (HTTP)

Browse

5


Router

...

DEA
Droplet
credentials

Service
Broker
(i.e. mysql)

up to you

up to you

MySQL
(process)

When you cf push
VM for Service
Instances

credentials

provision (HTTP)

provision (HTTP)

CLI

Cloud
Controller

bind (HTTP)

bind (HTTP)

go ahead and
deploy (HTTP)

Browse

5


Router

...

Service
Broker
(i.e. mysql)

up to you

MySQL
(process)

up to you

DEA
Droplet
credentials

Note:
• Don’t call ‘em “Gateways” anymore
• Broker only used during
•(un)provision
•(un)bind

Demo

6

Agenda
§Extending services offerings within run.pivotal.io
–Partnering

§Building services
–Into your own Cloud Foundry

§User-provided service instances
–For access to pre&externally-provisioned services


run.pivotal.io Marketplace Services

run.pivotal.io

8



run.pivotal.io

Contact Nima Badiey
(nbadiey@gopivotal.com)
to play

8



Cloud
Controller

AppDirect
Service
Broker
Browser

9


DEA


Cloud
Controller

credentials

AppDirect
Service
Broker
Browser

DEA
Droplet
credentials

9


Demo

10

Cloud Foundry Services
• There is only one built in - mysql
– or many more depending on how you count

• How to build them - the evolution:
– CF v1, subclass Ruby base classes
– CF v2, a RESTful protocol (which is itself evolving)

Polyglot!
• There are three parts to the protocol...

11

1Broker Registers Offerings
On Broker Startup...

Browser
12


UAA

Router

CLI

Service
Broker
(i.e. mysql)

VM for Service
Instances
My Service
(process)

Cloud
Controller
DEA


Authenticate

Browser
12


UAA

Router

CLI

Service
Broker
(i.e. mysql)

VM for Service
Instances
My Service
(process)

Cloud
Controller
DEA


CLI

Authenticate


UAA

Router
Browser
12

Service
Broker
(i.e. mysql)

token

VM for Service
Instances
My Service
(process)

Cloud
Controller
DEA


CLI

Authenticate
token

Browser
12


UAA

Router

Services Catalog

Ge

ta
t ca

log

P)
TT
(H

Service
Broker
(i.e. mysql)

VM for Service
Instances
My Service
(process)

Cloud
Controller
DEA


CLI

Authenticate
token

Browser
12


UAA

Router

Services Catalog

Ge

Cloud
Controller

ta
t ca

da
Up

te

log

P)
TT
(H

alo
cat

g

Service
Broker
(i.e. mysql)

VM for Service
Instances

P)
TT
(H

DEA

My Service
(process)

2Broker Services (un)provision and (un)bind
Prerequisite...

13


UAA

Router

CLI

Service
Broker
(i.e. mysql)

Cloud
Controller

VM for Service
Instances
My Service
(process)

Prerequisite...

shared secret

13


UAA

Router

CLI

Service
Broker
(i.e. mysql)

Cloud
Controller

VM for Service
Instances
My Service
(process)

Prerequisite...

vid

es

ha

13


red

sec
ret

Service
Broker
(i.e. mysql)
(H

UAA

pro

Router

CLI

shared secret

VM for Service
Instances

TT
P)

Cloud
Controller

My Service
(process)


Browser
14


UAA

Router

CLI

Service
Broker
(i.e. mysql)

VM for Service
Instances
My Service
(process)

Cloud
Controller
DEA

Browser
14


UAA

Router

CLI


(un

)

vi
pro

sio

n

P)
TT
(H

Service
Broker
(i.e. mysql)

up to you

VM for Service
Instances
My Service
(process)

Cloud
Controller
DEA

CLI


Browser
14


UAA

Router

credentials

(un

Cloud
Controller

)

vi
pro

)b
(un

ind

sio

n

P)
TT
(H

P)
TT
(H

DEA
Droplet
credentials

Service
Broker
(i.e. mysql)

up to you

VM for Service
Instances
My Service
(process)

3Orphan Management

CLI

Broker is responsible for eventual consistency...

15


UAA

Router

Services Catalog

Cloud
Controller

Service
Broker
(i.e. mysql)

VM for Service
Instances
My Service
(process)

3Orphan Management

CLI

Authenticate
token

15


UAA

Router

Services Catalog

Cloud
Controller

Service
Broker
(i.e. mysql)

VM for Service
Instances
My Service
(process)

3Orphan Management

CLI

Authenticate
token

15


UAA

Router

Services Catalog

Ge

Cloud
Controller

ta
t ca

log

P)
TT
(H

Service
Broker
(i.e. mysql)

VM for Service
Instances
My Service
(process)

3Orphan Management

CLI

Authenticate
token

15


UAA

Router

Services Catalog

Ge

Cloud
Controller

ta
t ca

log

P)
TT
(H

Service
Broker
(i.e. mysql)

up to you

VM for Service
Instances
My Service
(process)

New (V2) Services API
• Simplifies things a bunch!
• Unidirectional - everything initiated from Cloud Controller
• Asynchronous provisioning
• From V1 RESTful Services API:
– Replace offerings registration with endpoint that allows the CC to
GET the offerings
– Remove orphan management

16

It’s Up To YOU Patterns
One option (CF v1 services)...
VM for Service
Instances
My Service
(process)

VM for Service
Broker
Service
Broker

VM for Service
Instances
My Service
(process)

Node

Node
VM for Service
Instances

NATS
17


My Service
(process)
Node

One option (CF v1 services)...
VM for Service
Instances
My Service
(process)

VM for Service
Broker

provision

Service
Broker

VM for Service
Instances
My Service
(process)

Node

Node
VM for Service
Instances

NATS
17


My Service
(process)
Node

Another...
VM for Service
Instances
VM for Service
Broker

VM for Service
Instances
Warden
Warden
Svc Process
Svc Process

Service
Broker

Node

Node
VM for Service
Instances
Warden

NATS
18


Svc Process
Node

Another...
VM for Service
Instances
VM for Service
Broker

provision

VM for Service
Instances
Warden
Warden
Svc Process
Svc Process

Service
Broker

Node

Node
VM for Service
Instances
Warden

NATS
18


Svc Process
Node

Another...
VM for Service
Instances
Warden

VM for Service
Broker

provision

Svc Process
Svc Process

Svc Process

Service
Broker

VM for Service
Instances
Warden
Warden

Node

Node
VM for Service
Instances
Warden

NATS
18


Svc Process
Node

And another...
VM for Service
Instances

VM for Service
Instances

VM for Service
Broker
Service
Broker

Node

Node
VM for Service
Instances

NATS
19


Node

And another...
VM for Service
Instances

VM for Service
Instances

VM for Service
Broker

provision

Service
Broker

Node

Node
VM for Service
Instances

NATS
19


Node

And another...
VM for Service
Instances
Warden

VM for Service
Broker

provision

VM for Service
Instances
Warden
Svc Process

Svc Process

Service
Broker

Node

Node
VM for Service
Instances
Warden

NATS
19


Svc Process
Node

BOSH deployment
packages
- service broker
- your impl

- service node
- services binaries
- services warden
- your node impl

Provisions
broker VMs

jobs
- service broker
- ref service broker pkg
- start web service
- service node
- ref service node pkg
- start node

Configure shared secret (for example)
20

Provisions
service VMs

But what about...
core

AppDirect
Service
Broker

Service Service
Broker Broker

MySQL

21

MySvc

But what about...
core

AppDirect
Service
Broker

Service Service
Broker Broker

?
ERP

21

Oracle

MySQL

MySvc

User-provided Service Instances

CLI

Prerequisite...

Browser
22


UAA

Router

ERP

Cloud
Controller
DEA

Oracle


vid

es

erv

Browser
22


ice

cre

de

nti

UAA

pro

Router

CLI

Prerequisite...

als

credentials

(H

ERP

TT
P)

Cloud
Controller
DEA

Oracle


pro

vid

es

erv

)bi

Browser

Router

(un

ice

22


nd

cre
(H

de

nti

TT
P)

UAA

CLI

Prerequisite...

als

credentials

(H

ERP

TT
P)

Cloud
Controller
DEA
Droplet
credentials

Oracle

Demo

23

Tell ‘em what you told ‘em...
§Extending services offerings within run.pivotal.io
–Partnering

§Building services
–Into your own Cloud Foundry

§User-provided service instances
–For access to pre&externally-provisioned services


Buildpacks


Deploying to Cloud Foundry

26

Staging and Buildpacks
Buildpacks are responsible for preparing
the machine image for an application
Application
Buildpack
DEA

Container

Libraries
Runtime

Operating System

27

}

Droplet

Compatibility
Cloud Foundry buildpacks follow the Heroku buildpack design
Cloud Foundry and Heroku buildpacks are compatible
(if you take care to make them compatible)

Other PaaS providers are adopting the buildpack design

28

Built-in or BYO
cf push

cf push --buildpack <url>

The application is tested
against a set of curated
buildpacks

The buildpack is
referenced by a Git URL

29

Built-in Buildpacks

→
→
→

30

Tested Buildpacks
https://github.com/cloudfoundry-community/cf-docs-contrib/wiki/Buildpacks

Containers

Languages
Haskell

31

Buildpack API
/bin/detect app_directory
Inspect app bits to determine buildpack applicability

/bin/compile app_directory cache_directory
Download and install runtime, container, packages, libraries;
install app bits as necessary

/bin/release app_directory
Build app start command

32

/bin/detect
Inspect the app bits to determine if the buildpack knows how
to handle the application
Gemfile exists
package.json exists
setup.py exists

On match, return exit code 0 and
write to STDOUT a string identifying the buildpack
(often just the name of the language supported)
33

/bin/detect
$ cf push

DEA iterates over built-in
buildpacks calling
/bin/detect scripts
until one of them returns
exit code 0

34

$ cf push --buildpack <url>

/bin/detect is
not called

/bin/compile
Download and install any necessary
runtime (Java VM, Ruby interpreter, JavaScript interpreter)
container (web server)
support libraries, packages, modules (Ruby gems, NPM packages)

...and then installing the app bits into the runtime or container

35

/bin/compile Caching
Runtime, container, and support packages are downloaded
from sources external to Cloud Foundry
DEA provides a location for storing downloaded artifacts to
speed subsequent staging operations

36

/bin/release
Build a YAML-formatted hash with three possible keys
addons: []
config_vars: {}
default_process_types:
web: <start command>

On Cloud Foundry, currently only the web: value
is used to get the start command for the app

37

Java Buildpack
Supports a variety of JVM languages, containers, and
frameworks with a modular, configurable, and extensible design

38

Java Buildpack Concepts

Containers

Frameworks

How an application is run

Additional application
transformations

JREs
Java Runtimes

39

Java Buildpack Concepts
Containers

Frameworks

Java main()
Tomcat
Groovy
Spring Boot CLI
Play

Spring config
Play config
Play JPA config
New Relic

JREs
OpenJDK

40

Container Detection Criteria
Java main()

META-INF/MANIFEST.MF exists with
Main-class attribute set

Tomcat

WEB-INF directory exists

Groovy

.groovy file with a main() method, or
.groovy file with no classes, or
.groovy file with a shebang (#!) declaration

Spring Boot CLI

one or more POGO .groovy files with no
main() method, and
no WEB-INF directory

Play

start and lib/play.play_*.jar exist

Choose zero or one
41

Framework Detection Criteria
Spring

spring-core*.jar exists

Play config

Play application detected

Play JPA config

play-java-jpa plugin exists in app

New Relic

New Relic service bound to app

Choose all that apply
42

/bin/compile Output Example
-----> Downloaded app package (18M)
-----> Downloading OpenJDK 1.7.0_21 JRE (17.5s)
Expanding JRE to .java (1.4s)
-----> Downloading Auto Reconfiguration 0.7.1 (1.4s)
Modifying /WEB-INF/web.xml for Auto Reconfig
-----> Downloading Tomcat 7.0.42 (3.5s)
Expanding Tomcat to .tomcat (0.2s)
Downloading Buildpack Tomcat Support 1.1.1 (0.0s)
-----> Uploading droplet (55M)

43

DEA

Buildpack

DEA

See What’s Going On
$ cf files <app-name> app
.buildpack-diagnostics/
.java/
.lib/
.tomcat/

Buildpack-installed runtime
Buildpack-installed support libraries
Buildpack-installed container

META-INF/
WEB-INF/
assets/

44

DEA-downloaded application files

See What’s Going On
$ cf files <app-name> staging_info.yml
detected_buildpack: openjdk-1.7.0_21, tomcat-7.0.42,
spring-auto-reconfiguration-0.7.1
start_command: JAVA_HOME=.java
JAVA_OPTS="-Dhttp.port=$PORT
-Djava.io.tmpdir=$TMPDIR -XX:MaxPermSize=52428K
-XX:OnOutOfMemoryError=./.buildpack-diagnostics/killjava
-Xmx384M -Xss1M" .tomcat/bin/catalina.sh run

45

Customization
Two ways to customize the Java buildpack
Configure artifacts used by standard JREs, Containers, and Frameworks
Extend the buildpack with your own JREs, Containers, and Frameworks

Customization is done by forking the buildpack

46

Customization by Configuration
Configuration files in java-buildpack/config determine the
behavior of a JRE, Container, or Framework
# http://download.pivotal.io.s3.amazonaws.com/openjdk/lucid/x86_64/index.yml
--1.6.0_27: http://download.pivotal.io.s3.amazonaws.com/openjdk/lucid/x86_64/openjdk-1.6.0_27.tar.gz
1.7.0_21: http://download.pivotal.io.s3.amazonaws.com/openjdk/lucid/x86_64/openjdk-1.7.0_21.tar.gz
1.7.0_25: http://download.pivotal.io.s3.amazonaws.com/openjdk/lucid/x86_64/openjdk-1.7.0_25.tar.gz
1.8.0_M6: http://download.pivotal.io.s3.amazonaws.com/openjdk/lucid/x86_64/openjdk-1.8.0_M6.tar.gz
1.8.0_M7: http://download.pivotal.io.s3.amazonaws.com/openjdk/lucid/x86_64/openjdk-1.8.0_M7.tar.gz

# cloudfoundry/java-buildpack/config/openjdk.yml
--version: 1.7.0_+
repository_root: "http://download.pivotal.io.s3.amazonaws.com/openjdk/{platform}/{architecture}"
memory_sizes:
memory_heuristics:
  heap: 0.75
  permgen: 0.1
  stack: 0.05
  native: 0.1

47

Example: customizing the Tomcat artifact for download
# http://files.example.com/tomcat-custom/index.yml
--7.0.40: http://files.example.com/tomcat-custom/tomcat-7.0.40.tar.gz
7.0.41: http://files.example.com/tomcat-custom/tomcat-7.0.41.tar.gz
7.0.42: http://files.example.com/tomcat-custom/tomcat-7.0.42.tar.gz

# cloudfoundry/java-buildpack/config/tomcat.yml
--version: 7.0.42
repository_root: "http://files.example.com/tomcat-custom"
support:
version: 1.1.+
repository_root: "http://files.example.com/tomcat-buildpack-support"

48

Tomcat container supports simple customization of
context.xml and server.xml.
cloudfoundry/java-buildpack/resources/tomcat/conf
context.xml
server.xml

49

Customization by Extension
Implement a JRE, Container, or Framework support class as
one Ruby file in the appropriate directory
cloudfoundry/java-buildpack/lib/java_buildpack
jre
container
framework
(with additional support classes as necessary)

50

Support class types have similar interfaces, following the
buildpack scripts naming conventions
# initialize the support class with platform information provided in context
# context includes app_dir, lib_dir, environment, java_home, java_opts, vcap_application, vcap_services
def initialize(context)
# return a String or an Array<String> that uniquely identifies the container/framework/jre, or nil
def detect
# download and unpack the container/framework/jre, and transform the application as necessary
def compile
# create and return the command to run the application with (containers) or add options to
context[:java_opts] (frameworks)
def release

51

Add new support class to config/components.yml
--containers:
  - "JavaBuildpack::Container::Groovy"
  - "JavaBuildpack::Container::Main"
  - "JavaBuildpack::Container::SpringBootCli"
  - "JavaBuildpack::Container::Tomcat"
  - "JavaBuildpack::Container::Play"
jres:
  - "JavaBuildpack::Jre::OpenJdk"
frameworks:
  - "JavaBuildpack::Framework::JavaOpts"
  - "JavaBuildpack::Framework::NewRelic"
  - "JavaBuildpack::Framework::PlayAutoReconfiguration"
  - "JavaBuildpack::Framework::PlayJpaPlugin"
  - "JavaBuildpack::Framework::SpringAutoReconfiguration"
52

Customization
Much more information and documentation included in the
GitHub repository
https://github.com/cloudfoundry/java-buildpack

53

Cornelia Davis
cdavis@gopivotal.com
@cdavisafc

Scott Frederick
sfrederick@gopivotal.com
@scottyfred

Community Engineer, Cloud Foundry at Pivotal

Extending Cloud Foundry with Custom Services and Buildpacks

Recommandé

Recommandé

Contenu connexe

Plus de cornelia davis

Plus de cornelia davis (20)

Dernier

Dernier (20)

Extending Cloud Foundry with Custom Services and Buildpacks