Teams and monoliths - Matthew Skelton - London DevOps June 2017
•
4 j'aime•1,407 vues
Moving from a monolith to microservices can be daunting. How do we choose the right bounded contexts? How small should services be? Which teams should get which services? And how do we keep things from falling apart? By starting with the needs of the team, we can infer some useful heuristics for evolving from a monolithic architecture to a set of more loosely coupled services. Talk given at London DevOps meetup group - June 2017 - https://www.meetup.com/London-DevOps/events/238827763/
Recommandé
Contenu connexe
Similaire à Teams and monoliths - Matthew Skelton - London DevOps June 2017
Similaire à Teams and monoliths - Matthew Skelton - London DevOps June 2017 (20)
Plus de Skelton Thatcher Consulting Ltd
Plus de Skelton Thatcher Consulting Ltd (20)
Dernier
Dernier (20)
Teams and monoliths - Matthew Skelton - London DevOps June 2017
- 1. How to break apart a monolithic system safely without destroying your team Matthew Skelton, Skelton Thatcher Consulting @matthewpskelton 06 June 2017, London DevOps meetup, Facebook London - #londondevops
- 2. Today Cognitive load for teams ‘Monolith’ Code Forensics Team-first boundaries Monolith-splitting recipe
- 3. For now, let’s forget: Microservices CQRS / Event Sourcing Queues (Architectural changes)
- 5. Team-first digital transformation 30+ organisations UK, US, EU, India, China
- 6. We build modern capabilities by mentoring your teams
- 8. How to break apart a monolithic system safely without destroying your team
- 9. Safer, more rapid changes to software systems (Business Agility)
- 10. A ‘team-first’ approach to software subsystem boundaries
- 11. TEAM
- 14. (assumption) the team is stable, slowly changing, and long-lived #NoProjects
- 15. Conway’s Law
- 16. “organizations which design systems ... are constrained to produce designs which are copies of the communication structures of these organizations” – Mel Conway, 1968 http://www.melconway.com/Home/Conways_Law.html
- 18. ‘Reverse Conway’ Tobbe Gyllebring (@drunkcod)
- 20. homomorphic force (#Conway #Yawnoc) HT @allankellynet (same) (shape)
- 21. Cognitive load for teams
- 22. Cognitive load the total amount of mental effort being used in the working memory (see Sweller, 1988)
- 23. Cognitive load Intrinsic Extraneous (Irrelevant ) Germane (Relevant)
- 24. ‘Hacking Your Head’: Jo Pearce See http://www.slideshare.net/JoPearce5/hacking-your-head-managing-information-overload-45-mix @jdpearce
- 25. We have SCIENCE!
- 26. Science since 1988 (!) • Driskell et al, 1999 ‘Does Stress Lead to a Loss of Team Perspective?’ Group Dynamics: Theory, Research, and Practice 3, no. 4 (1999): 291. • Fan et al, 2010 ‘Learning HMM-Based Cognitive Load Models for Supporting Human-Agent Teamwork’. Cognitive Systems Research 11, no. 1 (2010): 108–119. • Ilgen & Hollenbeck, 1993 ‘Effective Team Performance under Stress and Normal Conditions: An Experimental Paradigm, Theory and Data for Studying Team Decision Making in Hierarchical Teams with Distributed Expertise’. DTIC Document, 1993. • Johnston et al, 2002 ‘Application of Cognitive Load Theory to Developing a Measure of Team Decision Efficiency’. DTIC Document, 2002. • Sweller, John, 1994 ‘Cognitive Load Theory, Learning Difficulty, and Instructional Design’. Learning and Instruction 4 (1994): 295–312. • Sweller, John, 1988. ‘Cognitive Load during Problem Solving: Effects on Learning’. Cognitive Science 12, no. 2 (1988): 257–285.
- 27. “stress impacts team performance … by narrowing or weakening the team-level perspective required for effective team behavior.” – Driskell et al, 1999 Group Dynamics: Theory, Research, and Practice 1999, Vol. 3, No. 4,291-302
- 28. (not just ‘pop’ science!)
- 29. ‘Monolith’
- 30. monolith μόνο λίθος “single stone” Reiner Flassig - CC BY-SA 2.0 de - Wikipedia
- 31. “Don’t start with a monolith when your goal is a microservices architecture” – Stefan Tilkov, innoQ http://martinfowler.com/articles/dont-start-monolith.html
- 32. “Start monolithic and extract” – Tammer Saleh, Pivotal https://www.infoq.com/presentations/cloud-anti-patterns
- 33. A ‘team-first’ approach to software subsystem boundaries
- 34. Types of software monoliths •Application monolith •Joined at the DB •Monolithic build (rebuild everything) •Monolithic releases (coupled) •Monolithic thinking (standardisation)
- 35. Application monolith Single block of code Deployed as a unit
- 36. Joined at the DB Difficult to change separately (but not impossible) Risk is (probably) elevated Chris Collyer, http://www.stone-circles.org.uk/stone/pentreifan.htm
- 37. Monolithic builds One gigantic CI build just to get a new version of any component
- 38. Monolithic releases Smaller components bundled together into a ‘release’
- 39. Monolithic thinking ‘One-size-fits-all’ for teams Assumption that minimising variation is A Good Thing
- 40. Dangers of splitting a monolith •Reduced domain consistency •Data duplication (unintentional) •Additional operational complexity due to distributed system and async messaging •Degraded UX across the product
- 44. Splitting a monolith Reiner Flassig - CC BY-SA 2.0 de - Wikipedia Choose the right technique for splitting Understand the nature of the monolith
- 46. ‘Fracture planes’ for code •Business domain bounded context •Regulatory compliance •Change cadence •Risk •Performance isolation •User personas •Team location
- 47. ‘Fracture planes’ for code Ask: Could we consume this thing as a service?
- 48. ‘Fracture planes’ for code Ask: Could we provide this thing as a service?
- 49. Code Forensics
- 50. Forensics Your Code as a Crime Scene Adam Tornhill
- 51. Adam Tornhill Code, Crime, Complexity: Analyzing software with forensic psychology Adam Tornhill TEDxTrondheim youtube.com/watch?v=qJ_hplxTYJw
- 52. ‘Code Maat’ tool
- 53. Adam Tornhill, http://www.adamtornhill.com/articles/crimescene/codeascrimescene.htm Code City plus Code Maat forensics
- 55. "information-poor abstract names are magnets for extra [unwanted] responsibilities" – Adam Tornhill p.185, Your Code as a Crime Scene
- 58. Team types Component team Platform / ’substrate’ team Supporting / ‘productivity’ team Product/Feature team
- 59. Team types devopstopologies.com Platform / ’substrate’ team Product/Feature team
- 60. Team types devopstopologies.com Component team Platform / ’substrate’ team Product/Feature team
- 61. Team types devopstopologies.com Component team Platform / ’substrate’ team Product/Feature team Supporting / ‘productivity’ team
- 62. Code repositories Align repositories to subsystem boundaries Avoid monolithic-y repos like TFS* * Don’t get me started on TFS, grrr…
- 63. Code repositories Repo 1 Build Test Deploy Run Repo 2 Build Test Deploy Run Repo 3 Build Test Deploy Run
- 64. “You can use a monorepo only if your organisation has published a scientific paper on Computer Science. Otherwise, use one repo per deployable runnable thing.” – Matthew Skelton LondonCD meetup group, 11 Oct 2016 ☺
- 65. Find natural or available ‘fracture planes’ for splitting a monolith (with the team in mind)
- 66. When not to split a monolith •‘Heritage’ ERP system (‘cloudified’) •No native Unit Test framework •20-30 min startup times •VMs need 56GB RAM (yes) •CI builds take 50 mins
- 67. Monolith-splitting recipe Tried and tested!
- 68. Monolith-splitting recipe 1. Instrument the monolith – logging 2. Grok data flows and fault responses 3. Align teams to available segments 4. Split off segments one-by-one
- 74. Technical Domain public enum EventID { // Badly-initialised logging data NotSet = 0, // An unrecognised event has occurred UnexpectedError = 10000, ApplicationStarted = 20000, ApplicationShutdownNoticeReceived = 20001, PageGenerationStarted = 30000, PageGenerationCompleted = 30001, MessageQueued = 40000, MessagePeeked = 40001, BasketItemAdded = 60001, BasketItemRemoved = 60002, CreditCardDetailsSubmitted = 70001, // ... }
- 75. BasketItemAdded = 60001 BasketItemRemoved = 60002
- 76. Instrument the monolith Correlation ID Logs Event ID
- 78. use logging as a channel/vector to make distributed systems more testable use logging as a channel/vector to make distributed systems more testable
- 79. Grok data flows and fault responses
- 80. Grok data flows and fault responses Correlation ID Event ID Unexpected collaborating subsystems Undetected fault condition
- 81. Grok data flows and fault responses Correlation ID Event ID Adjust subsystem boundaries Fix poor fault responses
- 82. runbooktemplate.infoRun Book dialogue sheets
- 83. Align teams to available segments
- 84. Align teams to available segments
- 85. Align teams to available segments Map to business domain
- 86. Align teams to available segments Identify likely components or ‘platform’ elements
- 87. Split off segments one-by-one
- 88. Split off segments one-by-one
- 89. Split off segments one-by-one Separate: - Builds - Infrastructure - Deployments - Versions - Lifecycle
- 90. Team needs / responsibilities / capabilities come first
- 91. use logging as a channel/vector to make distributed systems more testable Invest in Build & Release Engineering
- 93. Monolith-splitting recipe * 1. Instrument the monolith – logging 2. Grok data flows and fault responses 3. Align teams to available segments 4. Split off segments one-by-one (5. Invest in Build & Release Engineering) * The simplistic version
- 94. Monolith-splitting recipe * 1. Instrument 2. Grok behaviour 3. Align teams 4. Split off segments 5. Invest in Build & Release * The simplistic version
- 95. Determine monolith type (Apply ‘Code Forensics’) Find ‘fracture planes’ Assess cognitive load for teams Use the monolith-splitting recipe How to break apart a monolithic system safely without destroying your team
- 96. Training: 1-day workshop Splitting monolithic software safely without destroying the team training@skeltonthatcher.com
- 98. Industry experience Examples from recent work with clients Different monoliths Different ‘fracture planes’
- 99. Software for R&D teams Customers: over 80% of top 20 global pharma companies
- 100. 2014: Monolithic: builds, releases Pharma GxP: traceability Changes slow and tricky
- 101. Fracture plane: business domain
- 105. Compose with packages Logging for insights Technology splits Improved traceability
- 106. Major UK broadcaster (TV + online) Increasing digital delivery of content
- 107. 2014: Wide mix of technologies Need for visibility of changes Need for rapid turnaround
- 108. Fracture planes: read-only, location & cadence
- 109. Split off Reporting (read-only) Split on change cadence Increased change throughput
- 110. UK’s second largest credit reference agency Customers: major banks + large orgs
- 111. 2014: 6-weekly monolithic release Outage-sensitive customers
- 112. Fracture planes: risk & business domain
- 113. Split releases based on risk Align teams to value streams More rapid release cadence
- 114. Software for Internal Comms Customers: FTSE 100 & Fortune 1000
- 115. 2015: Successful system (monolithic) ISO 27001 compliance Aim to expand the org (x N)
- 116. Fracture plane: business domain
- 117. Optimise split for team engagement & personas Minimise ISO 27001 footprint Aided organisation expansion
- 118. Determine monolith type (Apply ‘Code Forensics’) Find ‘fracture planes’ Assess cognitive load for teams Use the monolith-splitting recipe How to break apart a monolithic system safely without destroying your team
- 119. Training: 1-day workshop Splitting monolithic software safely without destroying the team training@skeltonthatcher.com
- 120. Further material
- 122. Books & articles • Working Effectively with Legacy Code, by Michael Feathers • Building Microservices by Sam Newman (O’Reilly, 2015) • Managing Cognitive Load for Team Learning by Jo Pearce http://12devsofxmas.co.uk/2015/12/day-3-managing-cognitive-load- for-team-learning/ • Continuous Delivery with Windows and .NET by Matthew Skelton and Chris O’Dell (O’Reilly, 2016) http://cdwithwindows.net/ • Team Guide to Software Operability by Matthew Skelton and Rob Thatcher (Skelton Thatcher Publications, 2016) http://operabilitybook.com/ • Ye Olde Monolith by Pter Pilgrim https://dzone.com/articles/ye-olde- monolith
- 123. Training • From Monolith to Microservices (online training) – Sam Newman, author of Building Microservices http://www.oreilly.com/live-training/from-monolith-to- microservices.html • Run Book template & Run Book dialogue sheets http://runbooktemplate.info/
- 124. Talks & slides • Hacking Your Head – Managing Information Overload, by Jo Pearce - http://www.slideshare.net/JoPearce5/hacking-your- head-managing-information-overload-45-mix / http://conferences.oreilly.com/oscon/open-source- eu/public/schedule/detail/53013 • Principles of Microservices by Sam Newman @ Devoxx 2015 https://www.youtube.com/watch?v=PFQnNFe27kU
- 125. Research papers • Driskell, James E., Eduardo Salas, and Joan Johnston. ‘Does Stress Lead to a Loss of Team Perspective?’ Group Dynamics: Theory, Research, and Practice 3, no. 4 (1999): 291. • Fan, Xiaocong, Po-Chun Chen, and John Yen. ‘Learning HMM-Based Cognitive Load Models for Supporting Human-Agent Teamwork’. Cognitive Systems Research 11, no. 1 (2010): 108–119. • Ilgen, Daniel R., and John R. Hollenbeck. ‘Effective Team Performance under Stress and Normal Conditions: An Experimental Paradigm, Theory and Data for Studying Team Decision Making in Hierarchical Teams with Distributed Expertise’. DTIC Document, 1993. http://oai.dtic.mil/oai/oai?verb=getRecord&metadataPrefix=html&identifier=ADA284683. • Johnston, Joan H., Stephen M. Fiore, Carol Paris, and C. A. Smith. ‘Application of Cognitive Load Theory to Developing a Measure of Team Decision Efficiency’. DTIC Document, 2002. http://oai.dtic.mil/oai/oai?verb=getRecord&metadataPrefix=html&identifier=ADA525820. • Sweller, John. ‘Cognitive Load Theory, Learning Difficulty, and Instructional Design’. Learning and Instruction 4 (1994): 295–312. • Sweller, John. ‘Cognitive Load during Problem Solving: Effects on Learning’. Cognitive Science 12, no. 2 (1988): 257–285.
- 126. Determine monolith type (Apply ‘Code Forensics’) Find ‘fracture planes’ Assess cognitive load for teams Use the monolith-splitting recipe How to break apart a monolithic system safely without destroying your team