Continuous delivery (CD) allows software updates to be released frequently by having each code change trigger automated builds, tests, and deployments. This document discusses best practices for implementing CD for Alfresco solutions, including using consistent project templates built with Maven or Gradle, packaging modules as AMPs, externalizing configurations, supporting multi-module deployments, using deployment frameworks like Chef, and deploying to test instances on private clouds. Common pitfalls to avoid are unrealistic time planning, lack of involvement from system admins, and developers not understanding the importance of green builds.
Build it, Test it, Ship it: Continuous Delivery at Turner Broadcasting System...
PowerPoint Presentation
1. Continuous Delivery for Alfresco Solutions
Satisfied customers and happy developers with
Continuous Delivery
2. About me
• Roeland Hofkens
• #rhofkens
• roeland.hofkens@westernacher.com
• http://opensource.westernacher.com
• CTO Westernacher Products and Services AG
• Over 15 years of ECM experience
• Over 3 years of Continuous Delivery experience
3. Introduction
• Embrace Continuous Delivery for:
• Satisfied customers
• Happy developers
•How can we use CD successfully in Alfresco Solutions?
4. Agenda
• What is Continuous Delivery?
• How can CD be used for Alfresco Solutions?
• Best practices
• Pitfalls
•Live demo
• Demonstration of Continuous Delivery in the Westernacher
Development Environment
5. What is Continuous Delivery (CD) ?
• Cornerstones of Continuous Delivery
• Each check-in creates a potential release
• Each check-in (or a batch of check-ins) triggers a build & the related tests
• Each build is automatically deployed on a production-like test system
• Automated tests are executed on the newly deployed system
• “Infrastructure is code”
• All steps in the process are fully automated. Tools are required.
• Landscape configuration is externalized and versioned
• Each step provides immediate feedback (success & failure)
• Enables the developer team to act on it immediately
• What does this mean?
•At each moment in time we have a verified build that could
potentially be released to production
6. Continuous Delivery in context
Level of Complexity
Continuous
Easy
Integration
Level of Automation
• Each check-in creates a build
• Code level analysis
• Unit testing
• Guarantees code quality and consistency across the team
7. Continuous Delivery in context
Level of Complexity
Continuous
Challenge Delivery
Continuous
Easy
Integration
Level of Automation
• Each check-in creates a release candidate
• Fully automated deployment to test production-like env.
• Push-button deployment to production
• Guarantees that code is always in a state that can be
release to production
• Guarantees availability of latest features for customer
testing at all times
8. Continuous Delivery in context
Continuous
Complex
Deployment
Level of Complexity
Continuous
Challenge Delivery
Continuous
Easy
Integration
Level of Automation
• Each check-in produces a new production release
• Requires 100% test coverage and trust in the build
• Sophisticated deployment techniques required
• Guarantees availability of latest improvements to end users
at all times
9. Continuous Delivery toolset Tests
Coverage
Ticket Build Analysis
System Server
Smoke Tests
Load Tests
IDE Deployment
Artifact
DEV ENV Framework
Repo
Source Code Virtual Manual QA
Control System Machine
10. jUnit
CD Toolset @Westernacher Tests
Sonar
Coverage
JIRA Bamboo
Selenium for
Ticket Build Analysisand UI tests
Smoke
System Server
Smoke Tests
jMeter for
Load Tests tests
load
Eclipse Artifactory Chef
IDE Deployment
Artifact
DEV ENV Framework
Repo
SVN, Git VMWare
vSphere
Source Code Virtual Manual QA
Control System Machine
13. CD & Alfresco: Best practices (1)
• Use consistent Alfresco project templates
• Every project has the same structure
• 3 levels:
• Root
• Repo
• Share
• Every project is built in the same way
• Version your templates
14. CD & Alfresco: Best practices (2)
• Build with Maven or Gradle
•Use dependency resolution
• Alfresco now offers a Maven Repo
• Can also be accessed with Gradle
•Westernacher uses Gradle
• Increased flexibility
• Groovy DSL for build tasks
• AMP Plugin for Gradle
15. CD & Alfresco: Best practices (3)
• Use AMPs to package your
deliverables
• Always create two AMPS
• Repo
• Share
• Don’t put landscape dependent config in
your AMPS
• alfresco-global.properties
• share-config-custom.xml
• Develop an AMP that shows build
information in Alfresco
16. CD & Alfresco: Best practices (4)
• Only build your binaries once
1
• One binary fits all landscapes
• Regressions not possible
• Configuration is externalized
• Stored in SCM
• Versioned and managed as code
• Fed into deployment framework
17. CD & Alfresco: Best practices (5)
• Support multi-module deployments
• Design modular solutions
• Package all your extension in AMPS
• Solutions are composed of multiple AMPS
• Use flexible deployment configurations
• Module composition in configuration file
• Linked to artifact repo to allow dependency
resolution and versioning
18. CD & Alfresco: Best practices (6)
• Use a deployment framework
• Deployment frameworks take the pain
out of deployment tasks
• Repositories with scripts available
• Handle the externalization of configuration
easily
• Westernacher uses Chef
• Open Source with commercial support
• Large community
• Ruby scripting, JSON config
19. CD & Alfresco: Best practices (7)
• Deploy two instances on the same
2
VM
• Continuous Build instance
• Redeployed on every build
• Verifies delivery
• Dynamic environment for automated testing
• Nightly Build instance
• Only runs when CB build is green
• Stable environment for manual QA testing
20. CD & Alfresco: Best practices (8)
• Support shallow and deep
2
deployments
• Deep
• Set up complete Alfresco system from scratch
• Clean DB, repo and indexes
• Shallow
• Only update web apps
• Don’t touch the repo
21. CD & Alfresco: Best practices (9)
• Create Alfresco data loaders
• Data is needed for automated testing
• Bootstrap initial settings using Alfresco
bootstrap mechanisms
• Folder templates
• Categories, users, …
• Use your REST APIs to load data
• Can double as load tests
• Excellent sample code on how to use your
API
22. CD & Alfresco: Best practices (10)
• Use the (private) cloud
• Fact: you’ll need plenty of VM’s
• Create VM templates with OS
• Include basic environment and chef client
• Public cloud e.g. Amazon EC2
• Ideal for load testing
• Ideal for short research projects
• Private cloud
• Westernacher uses vSphere 5
• Cost effective for daily use
23. CD & Alfresco: Common pitfalls (1)
• Unrealistic time planning
• Setting up a CD environment is a
complex task
• Take your time
• Allow lots of room for “refactoring” of the
pipelines
• Work Agile
• Same rules apply as for software projects
• Short cycles, deliver often
24. CD & Alfresco: Common pitfalls (2)
• No involvement of system admins &
IT stakeholders
• Involve your system administrators
• Make sure they understand the need
• Show the advantages for sys admins
• Have the sys admins create the cookbooks &
recipes
• Plan in compliance to IT policies from
the start
• Changing afterwards is a big effort
• Increase acceptance
25. CD & Alfresco: Common pitfalls (3)
• Developers don’t understand the
importance of a green build
• Evangelize CD
• Communicate the advantages
• Everybody in the team is responsible for the
build
• Make CD easy
• Show working examples
• Create end-to-end sample pipelines for copy &
paste
@CIW: main bullet points become visible one after another 15 years ecm: Documentem, Stellent (Oracle Web Center Content Document Management), Sharepoint, Tiridon ECM, Alfresco since 1.1 3 years of CD experience Started experimenting with Automated builds and deployment to Amazon EC2 in q4 2009 (with Bamboo elastic agents) SAP Alfresco platform All Westernacher Alfresco solutions since Q1 2012
@CIW: Graphic for Satisfied customers and happy developers – appear on click In this presentation I will show you how you can increase customer satisfaction if you embrace continuous delivery for your alfresco solutions. You will be able to: Increate overall quality Improve transparency Gain trust Reduce costs You are all developers and of course you’re all committed to your customers satisfaction. On the other hand, you all love coding and making great solutions. But deployment is usually a frustrating experience CD makes your life easier: Reduce deployment pain frustration: you all know the “it worked on our local machines” type of excuse. You know it was working but you have to spend ages before you can get it working @customer site. Not good. Reduce deployment time Focus on your development tasks instead of system admin
Agenda: CD Definition Components of a CD solutions 2) How to use with Alfresco Best practiices to unlock all advantages of CD Pitfalls: things to avoid when you are setting up your own CD environment 3) Live demo Make some changes im Code and check in Triggers pipelines Show pipeline progress Show changes in live system
Continuous Delivery involves the setup of a development environment that supports the cornerstone functionality of continuous delivery: A batch of checkins creates a potential release Please let this sink in for a while. This is probably the main difference in the way you’re working now. Some of you will have a build server – actually I hope most of you do Some of you might event have continuous integration but I suppose most of you simply create a nightly build Only when the 3 steps (build, deployment, test) are successful the build is green!!! Full automation is required. You can’t do manual deployment processes or manual system installation. There is still room for manual QA testing Feedback is extremely important so problems can be fixed right away
@Christian: the graph should show three levels. Each level becomes visible after clicking. For each level, the bullet points become visible after clicking Making 3 slide Context graph that compares: Continuous Integration Continuous Delivery Continuous Deployment (aka Continuous Release) CI Each check-in creates a build Code level analysis Unit Testing Guarantees code quality and consistency across the team 2) CD Each check-in creates a new release candidate Fully automated deployment to test infrastructure Push-button deployment to production Guarantees that the code is always in a state that can be released to production Guarantees availability of latest features for customer testing at all times 3) CR Each check-in deploys a new production release Highest level of automation Requires 100% test coverage and test in the build Sophisticated deployment techniques like green/blue deployment required Guarantees availability of latest improvements to end users at all times
@Christian: the graph should show three levels. Each level becomes visible after clicking. For each level, the bullet points become visible after clicking Making 3 slide Context graph that compares: Continuous Integration Continuous Delivery Continuous Deployment (aka Continuous Release) CI Each check-in creates a build Code level analysis Unit Testing Guarantees code quality and consistency across the team 2) CD Each check-in creates a new release candidate Fully automated deployment to test infrastructure Push-button deployment to production Guarantees that the code is always in a state that can be released to production Guarantees availability of latest features for customer testing at all times 3) CR Each check-in deploys a new production release Highest level of automation Requires 100% test coverage and test in the build Sophisticated deployment techniques like green/blue deployment required Guarantees availability of latest improvements to end users at all times
@Christian: the graph should show three levels. Each level becomes visible after clicking. For each level, the bullet points become visible after clicking Making 3 slide Context graph that compares: Continuous Integration Continuous Delivery Continuous Deployment (aka Continuous Release) 3) CR Each check-in deploys a new production release. This is used by some of the major players like google and salesforce.com Highest level of automation Requires 100% test coverage and test in the build Sophisticated deployment techniques like green/blue deployment required. This means you need two parallel, identical production hardware environments. New releases are deployed on each landscape in an alternating way. Access is controlled by network devices like loadbalancers and reverse proxies. Guarantees availability of latest improvements to end users at all times
@Christian: please create a nice looking graphic here. Make sure that the text is not too small and that the lines are not too thin – it needs to be clearly visible on the beamer Ticket system IDE SCM BUILD SERVER ARTIFACT REPO DEPLOYMENT FRAMEWORK VIRTUAL MACHINE TESTING TOOLS
@Christian: You can re-use the graphic of the previous slide but with an overlay that adds the names to all the tools Ticket system JIRA IDE Eclipse SCM SVN, Git BUILD SERVER Bamboo ARTIFACT REPO Artifactory DEPLOYMENT FRAMEWORK Chef VIRTUAL MACHINE VMWare vSphere TESTING TOOLS jUnit Sonar Selenium for Smoke and UI tests jMeter for load tests
@Christiab: one graphic pro bullet point, appear one after another @mouse click. Please note that I have some good graphics already from the slides at the annual meeting – see other PPT. You will be able to: Increate overall quality Improve transparency Gain trust Reduce costs
@Christiab: one graphic pro bullet point, appear one after another @mouse click You are all developers and of course you’re all committed to your customers satisfaction. On the other hand, you all love coding and making great solutions. But deployment is usually a frustrating experience CD makes your life easier: Reduce deployment pain frustration: you all know the “it worked on our local machines” type of excuse. You know it was working but you have to spend ages before you can get it working @customer site. Not good. Reduce deployment time Focus on your development tasks instead of system admin
@Christian: for each best practice tip, I would like on graphic placed at the right hand side. I will add some more text in bullet points later To make continuous delivery happen we have to make sure every step of the software development life cycle happens in a structured way. This actually starts at the very beginning of the project (requirements analysis, project and task planning) but I’ll focus on the more coding oriented stuff here. To make sure you don’t end up in chaos, first think long and hard about an alfresco project templates. Using the same templates in all projects is of paramount importance.
@Christian: Maven and gradle logo
@Christian: AMP = Alfresco Module Package. Not sure what could be a good graphic here
@Christian: graphic for “once”
@Christian: graphic that combines various parts of a puzzle
@Christian: Chef / Puppet logo
@Christian: graphic shallow vs deep
@Christian: graphic for “two” Save time with shallow deployments.
@Christian: graphic for data pump or loader
@Christian: graphic for cloud computing
@Christian: graphic for time planning with question mark or so
@Christian: graphic for time planning with question mark or so