Ce diaporama a bien été signalé.
Nous utilisons votre profil LinkedIn et vos données d’activité pour vous proposer des publicités personnalisées et pertinentes. Vous pouvez changer vos préférences de publicités à tout moment.

DevOps Transformation at Dynatrace and with Dynatrace

Presentation given at CMG Boston - April 20th 2017
#1: How to explain DevOps Transformation?
#2: How Dynatrace transformed from 6months waterfall to 1h code deploy
#3: The role of Monitoring in DevOps / CI/CD
#4: Using Dynatrace for your DevOps Transformation

  • Identifiez-vous pour voir les commentaires

DevOps Transformation at Dynatrace and with Dynatrace

  1. 1. DevOps Transformation at Dynatrace and with Dynatrace CMG Boston, April 20th 2017 Andreas Grabner: @grabnerandi, andreas.grabner@dynatrace.com Podcast: https://www.spreaker.com/user/pureperformance Dynatrace Trial: http://bit.ly/dtsaastrial
  2. 2. confidential How I explain DevOps Transformation! or From Waterfall to Continuous Innovation through DevOps Automation and Culture
  3. 3. confidential 24 “Features in a Box” Ship the whole box! Photo-Bombed! Very late feedback  F r u s t r a t i o n ! Quality Control! Back to Customer
  4. 4. confidential Continuous User Driven Innovation 1 “Feature at a Time” Optimize Before DeployImmediate Customer Feedback
  5. 5. confidential Use Case: DevOps Transformation @ Dynatrace
  6. 6. confidential 2011: APM about to be disrupted!  Migrate from On-Prem to VM, Cloud, Containers and PaaS  Architectures include micro-services, on-demand scaling, self-healing  ”Cloud Natives“ demand SaaS based solutions  Digital Transformers demand Analytics for Biz, Dev, Ops & Sec  Many new players on the market
  7. 7. confidential Challenges to master!  Bridging the gap between ”New Stack“ and “Enterprise Stack“  Deploying the same way our customers do: Continuously!  Not disrupting current operations and slower moving customers  Aligning 300+ engineers across 3 different geos  Solution: Innovation through Incubation!
  8. 8. confidential % 20% organization & culture technology DevOps Transformation @ Dynatrace
  9. 9. 2 major releases/year customers deploy & operate on-prem 26 major releases/year 170 prod deployments/day self-service online sales SaaS & Managed 2011 2016 sprint releases (continuous-delivery) 1h: Code -> Prod6months major/minor release
  10. 10. NOC lessons learnt
  11. 11. 11 COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE #Perform2015 Developer will never do that! Operator’s job
  12. 12. confidential Shift-Left Quality Quality/Performance matters in Dev/Staging as well! Make Dev/CSA/PM dependent from Quality in trunk! DevOps = start thinking like an Ops before Commit Shift-Right Metrics enable DEVs defining quality metrics make DEVs to the primary consumers of their metrics
  13. 13. confidential How we increased Sprint Quality Sprint Reviews Done on “dynaSprint“ • Daily Builds get deployed on “dynaDay“. Sprint builds to “dynaSprint • If you can only show it “on your dev machine“ its NOT DONE! Deploy Sprint Builds into our internal Production Enviornment • We monitor Website, Support, Licensing, Community ... With Dynatrace • If we break our own back office software we ALL feel the pain right away
  14. 14. confidential  Which Features to Optimize? Which Features to „Phase Out“  Allows Reducing Technical and Business Debt How we Prioritized Features
  15. 15. confidential Monitoring as Pipeline & Platform Feature Dev Perf/Test Ops Biz Faster Innovation with Quality Gates Faster Acting on Feedback Unit Perf Cont. Perf New Deploy New Capability CI CD Remove/Promote Triage/Optimize Update Tests Innovate/Design $$$ Lower Costs Happy Users
  16. 16. confidential acting as Engineers Role of Dynatrace DevOps Team Dynatrace Managed/SaaS Orchestration Layer DynatracePipeline Visualization Deployment Timeline Log Overview using Dynatrace Log APIJIRA Integrations & Product Managers
  17. 17. confidential https://github.com/Dynatrace/ufo Raising Awareness of Pipeline Quality
  18. 18. confidential Learnings when scaling DevOps Pipelines Service Team A Service Team B Service Team X Improve “Efficiency” Cloud Ops Ensure “Operational Service” PM/Biz Improve“Business”
  19. 19. confidential Be proud of your feature! DevOps  NoOps
  20. 20. confidential Dynatrace Transformation by the numbers 26 170 Releases / Year Deployments / Day 31000 60h Unit & Int Tests / hour UI Tests per Build More Quality ~200 340 Code commits / day Stories per sprint More Agile 93% Production bugs found by Dev More Stability 450 99.998% Global EC2 Instances Global Availability
  21. 21. confidential Dynatrace Feedback Loop Use Cases
  22. 22. Dev: Shift-Left - Architectural Regression Decisions = Functional Result (passed/failed) + Web Performance Metrics (# of Images, # of JavaScript, Page Load Time, ...) + App Performance Metrics (# of SQL, # of Logs, # of API Calls, # of Exceptions ...) Fail the build early!
  23. 23. confidential Dynatrace Feedback Loop Use Cases
  24. 24. confidential Warm Up Phase Low Load for a couple of mins Peak Load: 2x Regular Load Simulation Twice the load requires more than twice the resources. Services start failing 1x Regular Load Validating scaling behavior. Understanding resource requirements Perf/Test Use Case: Scalability Decisions
  25. 25. confidential Service Teams: Architecture Validation
  26. 26. Service Teams: Continuous Performance Validation “Performance Signature” for Build Nov 16 “Performance Signature” for Build Nov 17
  27. 27. Service Teams: Fact-Based Actions to find Regressions GOOD BUILD BAD BUILD
  28. 28. confidential Dynatrace Feedback Loop Use Cases
  29. 29. 4x $$$ to IaaS Ops: Resource / Cost Driven Decisions
  30. 30. Ops: Resource / Cost Driven Decisions Deployment of new Release New service using most of the CPU! New service using most of the CPU!
  31. 31. confidential Ops: Deployment Rollback or Keep Decisions
  32. 32. confidential Dynatrace Feedback Loop Use Cases
  33. 33. Total Number of Users per User Experience Conversion Rate Biz: User Feedback Driven Decisions
  34. 34. New Features + Day # 1 of Mkt Push Overall increase of Users! Jump in Conversion Rate! Biz: User Feedback Driven Decisions
  35. 35. Users keep growing Increase # of “tolerating” users! Lower Conversion as Day #1 Day #2 of Marketing Campaign Biz: User Feedback Driven Decisions
  36. 36. Drop in Conversion Rate Spikes in FRUSTRATED Users! Hotfix Deployment was rolled out Biz: User Feedback Driven Decisions
  37. 37. User Experience Back to Normal Jump in Conversion Rate! Fix of the Hotfix was rolled out Biz: User Feedback Driven Decisions
  38. 38. Biz: AI-Supported Decisions
  39. 39. Biz: User Behavior Driven Decisions
  40. 40. confidential Scaling DevOps in a Cloud Native World with Dynatrace Service Team A Service Team B Service Team X Improve “Performance Signature” Continuous Performance, Shift-Left, Failure, Usage Feedback Cloud Ops Ensure “Operational Service” Monitoring as a Service, Capacity Planning, Risk/Cost Control PM/Biz Improve“BusinessSignature” Usage,Behavior,Costs,Innovate,A/BTesting,…
  41. 41. www.dynatrace.com confidential
  42. 42. confidential Additional Lessons Learned
  43. 43. #1: Going from 6 Months to 1 Month On Premise Updates • Challenge: Monolith download too big for our customers • Impact: Update Process was error prone and “All or Nothing“ • Solution: Componentize, Automate Rollout/Rollback Capability, A/B Rollout Model Increased velocity uncovered bottlenecks! @grabnerandi
  44. 44. #2: Education on Frequent Updates • Challenge: Release Education used to happen 60-90 Days after the release • Impact: Upgrade to latest version happened very late • Solution: Education Integrated into Continuous Delivery: Dev Blogs, YouTube Videos... Increased velocity uncovered bottlenecks! @grabnerandi
  45. 45. #3: Availabilty of Development / Test Environments • Challenge: Supporting many different tech stack makes it hard to maintain it • Impact: Long running support tickets and long feature development • Solution: Infrastructure as Code gives “On Demand“ access to these enviornments Increased velocity uncovered bottlenecks! @grabnerandi