Recommandé
Recommandé
Contenu connexe
Similaire à Escaping 5 decades of monolithic annual releases
Similaire à Escaping 5 decades of monolithic annual releases (20)
Plus de Clare Macrae
Plus de Clare Macrae (12)
Dernier
Dernier (20)
Escaping 5 decades of monolithic annual releases
- 1. 1 @ClareMacraeUK Escaping 5 Decades of Monolithic Annual Releases Clare Macrae Cambridge Crystallographic Data Centre 20 March 2018
- 3. 3 @ClareMacraeUK@ClareMacraeUK Today • Context and motivation • Ever-growing monolithic releases • Using the pain • Starting to escape • Summary
- 4. 4 @ClareMacraeUK@ClareMacraeUK Today • Context and motivation • Ever-growing monolithic releases • Using the pain • Starting to escape • Summary
- 6. 6 @ClareMacraeUK@ClareMacraeUK Crystal Structures Flexible Plastic, Shapeways, 8cm £19 BOSQAB: doi:10.5517/cc55fnd Sandstone, Sculpteo, 13cm x 9cm x 9cm £71 LALNIN49: doi:10.5517/cctlldc Sandstone, Sculpteo, 6cm £65 SAHYIK01: doi:10.5517/cc8md6g Prices include delivery and tax (2015 prices)
- 7. 7 @ClareMacraeUK@ClareMacraeUK Unified Releases • 8 desktop applications • Database(s) • 8GB -> 18 GB • Annual licensing
- 10. 10 @ClareMacraeUK@ClareMacraeUK Why @PIPELINE? To share what we’ve learned on the journey
- 11. 11 @ClareMacraeUK@ClareMacraeUK Today • Context and motivation • Ever-growing monolithic releases • Using the pain • Starting to escape • Summary
- 12. 12 @ClareMacraeUK@ClareMacraeUK Technologies Fortran 77 & Database Fortran 90 Image: lewing@isc.tamu.edu Larry Ewing and The GIMP
- 13. 13 @ClareMacraeUK@ClareMacraeUK Technologies Fortran 77 & Database Fortran 90 Image: lewing@isc.tamu.edu Larry Ewing and The GIMP
- 15. 15 @ClareMacraeUK@ClareMacraeUK Constraints • Few million lines of C++ … • 3 scrum teams • Large, legacy database format • Decades of culture built on annual cycle • “November” release
- 17. 17 @ClareMacraeUK@ClareMacraeUK Ahead of our time? • 1988: 3 releases per year • Version control history starts in 1992 • “Kind of” CI builds in mid 1990s • But…
- 18. 18 @ClareMacraeUK@ClareMacraeUK Release Cycle Develop (trunk) • New code • Unit tests Release (branch) • More features • Test & Document • Bug Fixes
- 19. 19 @ClareMacraeUK@ClareMacraeUK Months per release 4 20031988 6 12 12 1989 2016
- 20. 20 @ClareMacraeUK@ClareMacraeUK Speeding Up? • Tried many things to improve annual releases… • … Over many years …
- 21. 21 @ClareMacraeUK@ClareMacraeUK Code & Infrastructure • Writing unit tests • Scripting (many) manual processes • TeamCity Continuous Integration builds • Overhaul ancient build and distribution scripts
- 22. 22 @ClareMacraeUK@ClareMacraeUK Culture • Trying to test features during dev • Trying to document features during dev
- 23. 23 @ClareMacraeUK@ClareMacraeUK Process • Documented manual “smoke” tests • All-staff testing day, once a year • Create release branch earlier … and earlier
- 24. 24 @ClareMacraeUK@ClareMacraeUK Manual Testing • Smoke Tests = [programs] x [OSs] x [OS versions] • Graphics Tests = [OSs] x [graphics cards] • Scientific Tests = [Tutorials + Workshops] x [OSs] 96 8 3 4 9 3 3 8 341147
- 25. 25 @ClareMacraeUK@ClareMacraeUK 2017 Annual Release • New testing hardware • Efficient tester machine setup • Started automating GUI smoke tests
- 28. 28 @ClareMacraeUK@ClareMacraeUK Are we nearly there yet? • 2016 release period noticeably more relaxed • Maybe we cracked it?
- 30. 30 @ClareMacraeUK@ClareMacraeUK Months per release 4 20031988 2017 6 12 12 1989 2016 12
- 31. 31 @ClareMacraeUK@ClareMacraeUK 2017 Annual Release • Perfect storm of Internal & External changes • Showstopper MacOS bug • Result: critical bugs and long release • Significant breaking changes during the year • Half our C++ developers needed for 4 months! • Major, visible stress
- 32. 32 @ClareMacraeUK@ClareMacraeUK 2017 Annual Release • Teams weren't able to plan for upcoming work • Agreement: "for staff well-being, can’t continue" • So painful, it was clear big change needed • But what?
- 33. 33 @ClareMacraeUK@ClareMacraeUK Today • Context and motivation • Ever-growing monolithic releases • Using the pain • Starting to escape • Summary
- 34. 34 @ClareMacraeUK@ClareMacraeUK “When something painful happens, use that opportunity to drive a long-term improvement”
- 35. 35 @ClareMacraeUK@ClareMacraeUK Plans for 2018 Do 3 Interim Releases
- 37. 37 @ClareMacraeUK@ClareMacraeUK Demo Test Automation • Personal GUI-testing demo for key stakeholders • Very valuable
- 38. 38 @ClareMacraeUK@ClareMacraeUK Internal publicity • We’re going to “develop and release on default” • Get release dates in the calendar well in advance
- 39. 39 @ClareMacraeUK@ClareMacraeUK Know the objections • Have advocated more frequent releases for years • So I knew all our “it’s impossible” reasons • Invaluable for planning change • Invaluable for reassuring those most worried
- 40. 40 @ClareMacraeUK@ClareMacraeUK Our main objections • Overheads: 4 x 4 months doesn’t fit! • Some of our users can only install once per year
- 42. 42 @ClareMacraeUK@ClareMacraeUK Meet all the teams • State the Goal: “Make Releases Boring” • Explain, explain, explain • Reassurance - it will be OK, it’s not new! • Feature toggles - only release what's ready
- 43. 43 @ClareMacraeUK@ClareMacraeUK Today • Context and motivation • Ever-growing monolithic releases • Using the pain • Starting to escape • Summary
- 44. 44 @ClareMacraeUK@ClareMacraeUK Redefine Releases • From When we fix bugs, finish features & test • To When we ship it • Only has to be better than the last release • Knowing that there's another release soon gives confidence to focus on the few critical things
- 45. 45 @ClareMacraeUK@ClareMacraeUK Change our Planning • No-one to test feature? Then defer the feature! • Critical that this first update release succeeded • Each dev team reviewed changes made in code that their products depends • So manual testing efforts focused by risk • First “extra” release after 2 months, not 3
- 46. 46 @ClareMacraeUK@ClareMacraeUK Priorities Complete feature Bug fixes Process improvements
- 47. 47 @ClareMacraeUK@ClareMacraeUK “Update 1” Release • Elapsed time: 1 month (2018-01-31 … 2018-02-28) • 90% fewer changes during release • Motivational benefits for developers • Excellent ideas from retrospective • “Not many new features”
- 48. 48 @ClareMacraeUK@ClareMacraeUK Months per release 4 20031988 2017 3 2018 6 12 12 1989 2016 12
- 50. 50 @ClareMacraeUK@ClareMacraeUK Why did it work? • All the development, testing and automation changes were necessary to improve our releases • But not sufficient • I truly believe our critical missing link was the belief & repeated expectation that we would succeed
- 52. 52 @ClareMacraeUK@ClareMacraeUK Today • Context and motivation • Ever-growing monolithic releases • Using the pain • Starting to escape • Summary
- 53. 53 @ClareMacraeUK@ClareMacraeUK Lessons learned • If you think you can achieve something, you will • Put as little as possible into first changed release • Use current tools • Discover what most needs to be improved
- 54. 54 @ClareMacraeUK@ClareMacraeUK Lessons learned • How to turn around massive monolithic processes? • Prioritise automated testing – however crude • Showcase the benefits of innovations – working things – to win support – don’t ask! • Just do it! • Or… Just get started!
- 55. 55 @ClareMacraeUK@ClareMacraeUK References • My Monolith is Melting – Meredith Williams, PIPELINE 2015 – https://vimeo.com/123620783 • On Continuous Delivery for Client Software: – http://timothyfitz.com/2009/03/09/cd-for-client-software/ • Feature Toggles: – https://martinfowler.com/articles/feature-toggles.html
- 56. 56 @ClareMacraeUK@ClareMacraeUK References • Simple 3D Printing from a Crystal Structure – https://www.ccdc.cam.ac.uk/Community/blog/2017-02-09- simple-3d-printing-from-a-crystal-structure/
- 57. 57 @ClareMacraeUK@ClareMacraeUK Contact Info • @ClareMacraeUK • macrae@ccdc.cam.ac.uk • https://www.ccdc.cam.ac.uk/ • https://www.slideshare.net/ClareMacrae/
- 61. 61 @ClareMacraeUK@ClareMacraeUK What next for us? • Eliminate time-consuming steps • More work to eliminate manual testing • Automate the data export process regularly • Make it quicker to install new builds • Suggestions welcome!
- 62. 62 @ClareMacraeUK@ClareMacraeUK That Mac challenge now? • Would not matter when the problem was discovered • Just get a small team together to work on the problem • Could release an update to users at any time • Would still need to fix it – just without the pressure
Notes de l'éditeur
- Dr Olga Kennard, founder of Cambridge Crystallographic Data Centre (CCDC) Speaking in 2015 about the organisation’s founding in 1965 The CCDC exists to curate and distribute crystal structures into a database What I learned from this celebration is that some of our remaining Fortran software dates back to the late 1960s and early 1970s
- More than 900,000 crystal structures in the database Fundamentally, this is what our database stores: The shapes of molecules (atom coordinates) Information about hydrogen bonds and other close contacts Symmetry information, to allow repeating lattices to be generated Major Applications: research, education, designing of new medicines