Solum is a native Platform-as-a-service for OpenStack providing capabilities for application lifecycle management (testing, building, deploying, scaling).

1. Solum
PaaS/ALM for OpenStack
Devdatta Kulkarni, PTL, Solum
devdatta.kulkarni@rackspace.com
(irc: devkulkarni)

2. Outline
• Project goals
• Early trials and use-cases
• Solum architecture
• Factors influencing adoption
• Upcoming features
• Question and answers
2

3. OpenStack User survey – April 2016
3

4. What is Solum?
System which provides a declarative model for application developers to
deploy and run their application’s on OpenStack starting from
application’s source code
Why should you care?
– For developers, Solum provides an easy-to-use platform for building, testing, and
deploying applications on OpenStack clouds
– For operators, Solum provides ability to make their OpenStack cloud more useful
to their application developers by enabling CI/CD capabilities for applications to
OpenStack clouds
4

5. Project goals
Developer Productivity
Ability to perform CI/CD, integration with github, integration with existing CI solutions
Application Stack Flexibility
Ability to support applications written in different languages
Add-On Services Extensibility
Ability to support different add-on services
Application Portability
Ability to deploy applications across different OpenStack clouds
5

6. Solum - System for building and deploying applications to OpenStack
Clouds
Solum in context with other
OpenStack services
Cinder
(Block)
Manila
(File)
Trove
(DB)
Neutron
(Network)
Nova
(VM)
Glance
(Image)
Magnum
(Container
orchestration
& lifecycle)
Heat
(Orchestration)
Solum
(App lifecycle)
Keystone
(Identity)
Horizon
(UI)
Murano
(App catalog)
Mistral
(Workflow)
Ironic
(Baremetal)
Swift
(Object)
Barbican
(KeyMgmt)
Ceilometer
(Monitoring)
6

7. Early trials & usecases
7

8. Solum in the wild
8

9. Solum in use
Users/use-cases:
• Wipro
• Rackspace
• Paderborn University
Interest / References:
9
https://blog.jasoncallaway.com/
http://www.tcpcloud.eu/en/blog/2015/06/27/beyond-horizon-part-1/
http://ronaldbradford.com/blog/using-your-devstack-cloud-2016-04-05/
http://egonzalez.org/nova-docker-driver/
https://www.ctl.io/developers/blog/post/openstack-solum-on-the-centurylink-
cloud/

10. How to use existing Jenkins setup with Solum for performing seamless
application CI/CD on OpenStack?
Solum features used: App image storage in Glance, API to deploy a pre-built DU
Usecase 1
Wipro: Jenkins for CI, Solum for CD
Build
Test
Release
Deploy
Continuous
Integration with
Jenkins
Continuous
Deployment with
Solum
10

11. Easily deploy applications to Carina (container cluster service from Rackspace)
Usecase 2
Rackspace: Deploying apps to Docker Swarm
User sends
command as
“Deploy
Application”
from Rackspace
UI
Solum deploys
requested
application on
Carina
Rackspace UI
shows the
deployment
progress status
Solum features used: Pluggable deployer architecture; Built a Carina specific
deployer in Solum
11

12. Easily deploy applications to Virtual Machines
Usecase 3
Rackspace: Deploying apps to VMs
Solum features used: Pluggable deployer architecture, Used Solum’s Heat
deployer
User sends
command as
“Deploy
Application”
from Rackspace
UI
Solum deploys
requested
application on
Nova using Heat
Rackspace UI
shows the
deployment
progress status
12

13. Continuous integration and testing of Chef recipes
Solum features used:
Custom languagepacks: Developed a custom languagepack containing chef testing
tools (rubocop, foodcritic, chef-spec, test-kitchen)
https://github.com/openstack/solum/tree/master/examples/language-packs/chef
Integrate with Private github repositories
Usecase 4
Rackspace: Testing Chef recipes
User commits
Chef code to
github repo
Solum runs Chef
tests – test-
kitchen,
rubocop, chef-
spec, foodcritic
Solum saves
test results in
swift and sends
notification to
github
13

14. Considering to build and deploy services from source code
with an existing Heat template
Useful Solum features: Build application DUs,; integration with
public/private git repositories
Usecase 5
Paderborn Univ: Deploying multiple services
User defines
Heat HOT
Custom Heat
plugin calls
Solum to build
images
Heat deploys
the ensemble
14

15. Solum architecture
15

16. Solum abstractions and concepts
• Languagepack (LP)
• App
• Deployment Unit (DU)
• Workflow
• Parameters
16

17. Solum abstraction - Languagepack (LP)
• A Docker image that contains libraries, packages, and tools required for testing,
building, compiling an application
– Examples:
• Python applications need: pip, tox, python-mysqldb
• Java applications need: jdk, jre, maven
• Chef recipes: chefdk, rubocop, test-kitchen, foodcritic
• Application-specific languagepacks
– Different languagepacks for different life-cycle stages of an application
• Testing libraries may be different than libraries for building application artifacts
– Multi-language applications
• Combine all libraries and tools from different languages in a single languagepack
• Operator-defined languagepacks, user-defined languagepacks
$ solum languagepack create python https://github.com/rackspace-solum-samples/solum-languagepack-python.git
17

18. Solum abstraction - Languagepack Example:
Python
FROM ubuntu:precise
MAINTAINER Murali Allada
<murali.allada@rackspace.com>
RUN apt-get -yqq update
RUN apt-get -yqq install python-pip
RUN apt-get -yqq install python-dev
COPY build.sh /solum/bin/
https://github.com/rackspace-solum-samples/solum-languagepack-python
#!/bin/bash
# Check if pip is installed
pip help
[[ $? != 0 ]] && echo python-pip is
not installed. && exit 1
# Install app dependencies
cd /app
pip install -r requirements.txt
18

19. Solum – sample languagepacks
https://github.com/rackspace-solum-samples/solum-languagepack-python
https://github.com/rackspace-solum-samples/solum-languagepack-java
https://github.com/rackspace-solum-samples/solum-languagepack-nodejs
https://github.com/rackspace-solum-samples/solum-languagepack-
nodejswithmongo
https://github.com/rackspace-solum-samples/solum-languagepack-wordpress
https://github.com/openstack/solum/tree/master/examples/language-packs/chef
19

20. Other languagepacks to support
20

21. Solum abstraction - App
Declarative application
definition
• source code repository
• run command
• port
• languagepack
version: 1
name: cherrypy
description: python web app
languagepack: python
source:
repository:
https://github.com/rackspace-solum-
samples/solum-python-sample-
app.git
revision: master
workflow_config:
run_cmd: python app.py
ports:
- 80
$ solum app register --app-file appfile.yaml
Reason to specify languagepack:
• App developer knows what libraries
are required by their app
21

22. Solum concept - Deployment Unit (DU)
• Docker image that is formed from the languagepack image with
application’s source code added to it
DU = LP + application source code
• Solum guarantees the DU contract
Application source code available at a known location on the DU image
/app
• How to build a DU?
– Construct Dockerfile with languagepack as the base image, application
source code injected, run command as the entry point
– Build the DU image from this Dockerfile
• LP and DU storage
Glance and Swift
22

23. DU Deployment: Supported options
• Nova-docker driver
Works with DU images stored in Glance
• Heat + VM image + DU location provided through user_data section
Works with DU images stored in Swift/Docker registry
1-1 mapping of DU container to VM
• Carina deployer in Rackspace’s Solum environment
Works with Rackspace Carina
23

24. Solum abstraction - Workflow
• Abstraction to represent execution of application deployment
action
• Supported application deployment actions (workflows)
–Build DU
–Run unit tests and build DU
–Run unit tests, build DU, and deploy DU
–Build DU and deploy DU
–Deploy a pre-built DU
• Workflows can be triggered from github webhooks
$ solum app deploy cherrypy
24

25. Solum concept – Parameters
• Application parameters
– Services needed by an application
• E.g.: Connection parameters for relational database such as Trove
– These are injected into the DU
• Environmental parameters
– Parameters intended for Solum to use
• E.g.: Carina cluster credentials
• Parameters specified at:
– application registration time
– application deploy time (upcoming feature)
$ solum app create --app-file appfile.yaml --param-file params.yaml 25

26. Solum – sample apps
https://github.com/openstack/solum/blob/master/examples/apps/python_app.yaml
https://github.com/openstack/solum/blob/master/examples/apps/java_app.yaml
https://github.com/rackspace-solum-samples/solum-wordpress-sample-app
https://github.com/openstack/solum/blob/master/examples/apps/nodejs_app.yaml
26

27. Summary of current Solum features
• Create languagepacks
• Register, deploy, scale an app
• Update an app keeping the application URL same
• Specify application parameters when registering an app
• Integration with public and private github repositories (via github webooks)
• Deploy app using different compute form factors
– nova-docker
– VMs
– Container cluster (such as Docker Swarm)
• Environments (dev/test/staging/production)
– Possible by creating different app for each env and deploying same DU for each
27

28. High-level Architecture
Solum
API
Solum
Worker
Solum
Deployer
Queue Queue
Queue
Solum
Conductor
Heat
Database
Glance/Swift
Builds LP; run unit
tests; Build DU
Deploys DU
- Decoupled architecture
- Asynchronous
communication
28

29. Factors influencing adoption
29

30. Usage experience
• Languagepacks
- Languagepacks built: python, java, php, nodejs, angularjs, chef testing tools
- Internal test users at Rackspace tended to use operator defined languagepacks
- Need to install appropriate OS packages in addition to the language libraries/tools
- python-msqldb not sufficient, needed OS-level mysql package
- Languagepack creation is iterative process
- Learning curve – how to lower this barrier is question?
• Apps
Need for ease of application registration led to supporting interactive prompts for getting app information
• Github webhooks
– Started with support for public repositories
– Internal use-cases required adding support for
• Private repositories
• CLI support for registering deploy keys
• CLI support for two-factor authentication 30

31. Building LPs and DUs
How to provide isolated environment for building LP and DU Docker
images and running untrusted unit test scripts?
• Isolated ‘git clone’ with resource constraints on CPU, memory, disk on
containers running unit testing scripts
• Running unit testing scripts as unprivileged user inside a container
• Timeout mechanism to constrain and limit the running time of unit testing
scripts
• Easy-to-use CLI for operator to kill long running (malicious) containers
• On Rackspace version of Solum – using Carina for building LPs, running unit
tests, and building DUs
This provides a natural isolated environment
31

32. Reliability
Experience
• DU and LP images saved in Glance and Swift using ‘docker save’
• Success rate of building and saving archives improved from 80% to
98.8%
–Retry mechanism for performing git actions (git clone) and Docker actions
(build, save, load)
–Better use of Swift client to upload/download archives
–Race condition handling in ‘docker load’ and ‘docker rmi’
Perform ‘docker rmi’ only for DU images on the worker node and not for LP images
Small risk of Worker running out of disk space
32

33. Performance
Experimental setup - 2 servers, 10 workers, 10 deployers, Deployer: Heat with deployment to
VMs
Applications used - Mix of Python and Java applications
Deployed 330 applications in 30 minutesResults
330 Queued
• 3 in ERROR_STACK_CREATE_FAILED
• 327 in READY (99.09%)
Average time from QUEUED to READY = 348 seconds (6.5 minutes)
• 36.8 seconds for unit testing
• 99 seconds to build DU
• 41 seconds to deploy DU via Heat
• 120 seconds to start the web app (wget, docker load, docker run)
• Other time spent in queuing.
We are seeing faster times when using Carina deployer (within Rackspace Solum setup)
33

34. Community involvement
http://stackalytics.com/?release=all&project_type=openstack&module=solum-group
34

35. Upcoming features
35

36. Key Features for Newton
• Multi-tier/micro-service applications
– Currently possible using the parameter feature
– We plan to build native support in Solum during Newton (app file format changes)
• Infrastructure choice for deploying applications
– Operator vs. App developer choice
• VMs for isolation
• Bare metal for performance
• Container clusters for packaging
• Application monitoring
DU monitoring is the key
Leverage Heat + Ceilometer
36

37. Other Features
Health monitoring of Solum services and
apps
Documentation, examples
37
Solum Features/Improvements

38. Conclusions
• Good news
Solum is getting interest from users.
• Rackspace
• Wipro
• Paderborn University
• Current capabilities
•CI/CD of applications starting from app source code natively on
OpenStack
•Building custom languagepacks
•Integration with Github (public and private)
•CLI
• Exciting features have been planned for Newton
38

39. References
Wiki:
https://wiki.openstack.org/wiki/Solum
Setting up development environment:
https://wiki.openstack.org/wiki/Solum/solum-development-setup
Meetings:
Tuesdays 1700 UTC in #openstack-meeting-3
IRC:
Channel: #solum on freenode
Code:
https://github.com/openstack/solum
https://github.com/openstack/python-solumclient
Demo:
https://vimeo.com/163874891
https://vimeo.com/143425822
39

40. O N E FA N AT I C A L P L A C E | S A N A N T O N I O , T X 7 8 2 1 8
U S S A L E S : 1 - 8 0 0 - 9 61 - 2 8 8 8 | U S S U P P O R T: 1 - 8 0 0 - 9 61 - 4 4 5 4 | W W W. R AC K S PAC E . C O M
© RACKSPACE LTD. | RACKSPACE® AND FANATICAL SUPPORT® ARE SERVICE MARKS OF RACKSPACE US, INC. REGISTERED IN THE UNITED S TATES AND OTHER COUNTRIES. |
WWW.RACKSPACE.COM
Thank you

41. FROM ubuntu:14.04
RUN apt-get -yqq update &&
apt-get -yqq install openjdk-7-jdk
RUN apt-get -yqq install maven
ENV JAVA_HOME /usr/lib/jvm/java-7-
openjdk-amd64
COPY bin/build.sh /solum/bin
https://github.com/rackspace-solum-samples/solum-languagepack-java
mvn -v
[[ $? != 0 ]] && echo "maven is not
installed" && exit 1
# build
cd /app
mvn -q package
Dockerfile build.sh
Solum abstractions - Languagepack Example: Java
41

42. https://github.com/rackspace-solum-samples/solum-languagepack-wordpress
Dockerfile
--------------
Install PHP, required Apache modules
build.sh
------------
Empty
App run command
--------------------------
Setting the environment variables
and starting the app
Solum abstractions - Languagepack Example:
Wordpress
42

43. Basic operational flow
• API receives a request to build and deploy an app
• API sends the request to the Worker
• Worker downloads the specified LP from configured storage (Swift or Glance)
• Worker builds the DU and stores it
• Worker informs the Deployer to deploy the DU
• Deployer deploys the DU by calling Heat
43

44. Related Projects
• Deis
• CloudFoundry
• OpenShift
44

45. Performance
Experimental setup (2 servers, 10 workers, 10 deployers)
– MemTotal: 8171504 kB
– Architecture: x86_64
– CPU(s): 8
– Vendor ID: GenuineIntel
– CPU MHz: 2593.604
– Hypervisor vendor: Xen
– L1d cache: 32K
– L1i cache: 32K
– L2 cache: 256K
– L3 cache: 20480K
45

46. OpenStack User survey – April 2016
46