Ce diaporama a bien été signalé.
Nous utilisons votre profil LinkedIn et vos données d’activité pour vous proposer des publicités personnalisées et pertinentes. Vous pouvez changer vos préférences de publicités à tout moment.

Delivery Pipelines as a First Class Citizen @deliverAgile2019

In this talk, we will cover important elements for successful CI and CD. We will discuss how these elements make CI and CD much simpler, and hence more attainable. We will cover some best practices / recommendations to include in your application pipelines. We will look at a sample implementation of a pipeline leveraging modern tools. Finally, we will discuss some forthcoming ideas for making it even easier to declaratively enable CI and CD for applications.

  • Identifiez-vous pour voir les commentaires

Delivery Pipelines as a First Class Citizen @deliverAgile2019

  1. 1. Delivery Pipelines as a First Class Citizen Cora Iberkleid @ciberkleid | ciberkleid@pivotal.io Advisory Platform Architect, Pivotal #deliverAgile2019 | @AgileAlliance
  2. 2. Why It Matters
  3. 3. Production, production, production awesome product in your dev env ok product in user’s hands ok > awesome ??
  4. 4. Is ok really ok? Yes, if delivery is... Fast Safe Secure So you can iterate quickly & often, because that’s how you get from ok to awesome IN PRODUCTION
  5. 5. New Measures of Success ● Release frequently ● Release independently ● Roll back independently
  6. 6. Stand on the Shoulders of Giants ● Build on a layers of modern, structured automation ● Employ modern coding & testing practices ● Don’t reinvent the wheel!
  7. 7. The Software Defined Delivery Manifesto ● Delivery is not a detail ● Time to apply our core skills to our own work
  8. 8. Continuous Delivery Foundation (CDF)
  9. 9. CI vs CD
  10. 10. CI vs……………………………. CD CI Pipeline release branch CI Pipeline master branch Artifact Repo CD Pipeline master branch test stage prod Source Code Repo
  11. 11. An Example of CI vs CD Tooling Stateless: Rather than sharing state, every task runs in its own container, controlling its own dependencies. Ephemeral: Clean and consistent builds. Minimizes test infrastructure to maintain and gives you flexibility to scale with the cloud. Continuous integration: Track versions of external artifacts used for CI (e.g., Git, S3, docker image). Build components expressed as code with simple modeling of modular components. Flexible workflow automation: Extensible for compliance or other customized tasks in a pipeline. Stateful: Maintains awareness of the topography (multiple clouds, PCF Foundations, etc.) and keeps record of all pipeline activities for compliance and audit. Application inventory: Maintains inventory of deployed applications, supporting activities like rollback, canary analysis and identifying security vulnerabilities. Sophisticated deployment scenarios: Canary deploys, red/black or blue/green deploys, progressive deployments (e.g., by time zone), multi-cloud/PCF Foundation deployments.
  12. 12. CI Best Practices ● Simple ● Consistent ● Early ● Follow cloud native / 12-factor principles ○ Boundaries to facilitate independent release cycles ○ Repository should contain everything necessary for build ○ Config management ○ Separate build, release, run stages ■ Containerization is plumbing - move it to release ● API-Driven Development ○ Use APIs to design / define component interactions ○ Limit the exposed surface area of your application ○ A good API decouples the “what” from the “how” ● Check for back compatibility ● Use circuit breakers
  13. 13. CI Best Practices ● Simple ● Consistent ● Early ● Leverage modern frameworks ● Convention over configuration ● Establish basic standards for testing organization ○ Naming conventions for test class packages ○ Naming conventions for maven/gradle profiles
  14. 14. CI Best Practices ● Simple ● Consistent ● Early ● Shift as much testing left into CI as possible ● Use contract-based stubs for API testing ● Check for breaking API changes ● Check for breaking database schema changes
  15. 15. CD Best Practices ● Simple ● Consistent ● Easy ● Leverage the highest level of abstraction possible for the workload runtime ○ Apps with sufficient cloud maturity can be containerized with an off-the-shelf process (e.g. Buildpacks) ○ Containerizing is plumbing, not value-add ● Cloud Foundry is a great example of a high abstraction level runtime target ○ OSS, multi-cloud ○ Includes Buildpacks for containerization ○ Full stack automation between IaaS and app ○ Simple API-based service provisioning, too!
  16. 16. CD Best Practices ● Simple ● Consistent ● Easy ● Leverage purpose-built, cloud-oriented CD tooling ○ Avoid using a CI tool for CD ■ CI for getting started, team-centric CI/CD ■ Expensive (CI runs jobs individually) ■ Lack org-centric views and capabilities ● Spinnaker is a great example of a modern, purpose-built CD tool for the cloud(s) ○ OSS, multi-cloud ○ Multi-platform integrations ○ Application Inventory ○ Efficient resource consumption
  17. 17. CD Best Practices ● Simple ● Consistent ● Easy ● With good CI practices and the right choice of CD tooling, sophisticated techniques can be easy ○ Blue/green, canary, and progressive deployment strategies ○ Rollbacks, canary analysis ○ Compliance, auditing, and security vulnerability identification
  18. 18. Spinnaker Integrations Cloud Providers App Engine Amazon Web Services Azure Cloud Foundry DC/OS Google Compute Engine Kubernetes Openstack Oracle CI Systems Jenkins Travis CI Wercker Concourse (coming soon) Artifact Support Docker Google Cloud Storage GitHub HTTP S3 Monitoring Datadog Prometheus Stackdriver Atlas SignalFx (10+ more on roadmap)Notifications Email HipChat Slack SMS via Twilio
  19. 19. Application-centric Control Plane
  20. 20. CI for APIs
  21. 21. Step 1: Agree on API Contract Consumer Producer API v1
  22. 22. Step 1: Codify the Contract Consumer Producer API v1 pull request Contract.make { description(""" should return a fortune string """) request { method GET() url "/" } response { status 200 body "foo fortune" } } Contract.make { description(""" should return a fortune string """) request { method GET() url "/" } response { status 200 body "foo fortune" } }
  23. 23. Tests Can* Be Auto-generated * e.g. Spring Cloud Contract
  24. 24. Why is That Beneficial? ● Producer can easily ensure it complies with the contract ● Efficient ● Repeatable ● Reliable
  25. 25. But Wait… There’s More ● Producer can easily check for API back-compatibility ○ Use an older contract to generate tests – run them against your new code ○ If the tests pass, you won’t break older clients ○ WIN: producer can release to production independently of consumer release schedule ● Even better: do this for N older contracts ○ Guarantee API back-compatibility for a range of previous contract versions ● Sounds cool, how easy is it?
  26. 26. A Stub Jar is also Auto-Generated. Just publish it. Producer uses the contract in the stub to re-generate tests and runs them against the new code base… .... for N* number of older contracts * where N is a small, manageable number What’s inside? contract (groovy) stub (json)
  27. 27. It’s Also Good for the Consumer Consumer can use the stub to mock out the producer… .... for N* number of older stubs * where N is a small, manageable number What’s inside? contract (groovy) stub (json)
  28. 28. Benefits to the Consumer ● The mock is more reliable ● The mock is available early ● Consumer can easily ensure it complies with the contract ● Consumer can validate cross-compatibility using older stubs ● Consumer will still work if producer needs to roll back ● WIN: consumer can release to production independently of producer release schedule
  29. 29. The Heart of the API Testing (local versions) ######## API CHECK: PRODUCER ./mvnw clean verify -Papicompatibility -Dproduction.version=blue -Dcontracts.mode=LOCAL ######## API CHECK: CONSUMER ./mvnw clean test -Dstubrunner.ids=com.example:producer-app:blue -Dstubrunner.stubs-mode=LOCAL
  30. 30. CI for DB Schemas
  31. 31. Database Schema Versioning & Migration Application Database v1 v1 ● Frameworks such as Flyway and Liquibase handle version schema tracking and migrations ● Write tests to validate code against the data store
  32. 32. Database Schema Versioning & Migration Application Database v1 v2 ● Test new code against new db schema ● Check out older app code and test that against the new db schema .... for N* number of older app versions
  33. 33. No More Database Rollback Nightmares ● Ensure the app can roll back in production without having to roll back the database ● Enable blue/green deployments to production as app versions share a database ● Can do this in CI using an in-memory database ● Stash away the new migration scripts ● Check out the old code ● Test the old code using the new db migration files ● WIN: breaking database schema changes can be caught earlier in the process
  34. 34. The Heart of the DB Schema Testing ######## DB CHECK cd ../app-blue; ./mvnw clean test -Dspring.flyway.locations=filesystem:../app-green/src/main/resources/db/migration
  35. 35. CD Deployment Strategies
  36. 36. Advanced Deployment Strategies ● Blue/green ● Automated canary analysis ● Rollbacks Verifying back compatibility enables you to take advantage of these advanced deployment strategies
  37. 37. Metrics-driven Canary Analysis for Prod Deployment NetFlix Tech Blog: https://medium.com/netflix-techblog/automated-canary-analysis-at-netflix-with-kayenta-3260bc7acc69 CF Summit Talk & Demo: https://www.youtube.com/watch?v=9C8m7n_sG38&feature=youtu.be Ensures your applications meet your SLAs
  38. 38. Yet Another Layer of Abstraction
  39. 39. Code Your Pipeline - Emergent Options ● Spin CLI ● Spinnaker Managed Pipeline Templates ● Canal (in development) ○ Kotlin and Java DSL for generating Spinnaker pipelines ○ Can manage both pipelines and templates ○ Potential for integration with other CI/CD tools in the future ○ Ability to validate pipelines structure programmatically ● Atomist ● Cloud Pipelines (as a reference for CI in particular)
  40. 40. In Summary… ● Keep your eye on the prize ○ Production is the place to be ○ Frequent releases ○ Independent release and rollback ● Keep it simple ○ Choose tools and frameworks that will help you ○ Avoid unnecessary complexity ● CI and CD are substantially different ○ Shift left into CI - cheaper, earlier, and under developer control ○ A purpose-built CD tool has a lot built in and is more efficient
  41. 41. Speaking of Giants A Word of Gratitude For their guidance, help, and support (in descending order of consecutive consonants in surname): Marcin Grzejszczak Jon Schneider Olga Kundzich Clayton McCoy Richard Seroter
  42. 42. THANK YOU! Demo guide: https://github.com/ciberkleid/cna-demo-setup Slides: https://www.slideshare.net/ciberkleid/delivery-pipelines-as-a-first-class-citizen-deliveragile2019