Ce diaporama a bien été signalé.
Le téléchargement de votre SlideShare est en cours. ×

Rescuing a-legacy-codebase

Publicité
Publicité
Publicité
Publicité
Publicité
Publicité
Publicité
Publicité
Publicité
Publicité
Publicité
Publicité

Consultez-les par la suite

1 sur 76 Publicité

Rescuing a-legacy-codebase

Télécharger pour lire hors ligne

We're often faced with the "rewrite vs. refactor" debate for legacy code bases. Here we present both business and technical considerations involved in the decision.

We're often faced with the "rewrite vs. refactor" debate for legacy code bases. Here we present both business and technical considerations involved in the decision.

Publicité
Publicité

Plus De Contenu Connexe

Similaire à Rescuing a-legacy-codebase (20)

Publicité

Plus récents (20)

Rescuing a-legacy-codebase

  1. 1. RESCUING LEGACY CODE CURTIS “OVID” POE CTO, ALL AROUND THE WORLD OVID@ALLAROUNDTHEWORLD.FR https://www.slideshare.net/Ovid/rescuing-alegacycodebase
  2. 2. ALL AROUND THE WORLD CURTIS “OVID” POE CTO, ALL AROUND THE WORLD OVID@ALLAROUNDTHEWORLD.FR
  3. 3. QUESTION POLICY CURTIS “OVID” POE CTO OVID@ALLAROUNDTHEWORLD.FR
  4. 4. SUBDUING MANAGERS mardi 3 avril 2018©2018 ovid@allaroundtheworld.fr
  5. 5. TYPICAL LARGE TEST SUITE • Hour or more to run • Tests often fail • Hard to see errors • Developers don’t run them • Or only run “some” tests • Or check Facebook while running tests mardi 3 avril 2018©2018 ovid@allaroundtheworld.fr
  6. 6. IRONY Large test suites can lead to poor quality code! mardi 3 avril 2018©2018 ovid@allaroundtheworld.fr
  7. 7. TYPICAL LARGE TEST SUITE mardi 3 avril 2018©2018 ovid@allaroundtheworld.fr With apologies to XKCD: https://xkcd.com/303/
  8. 8. WHAT THE DEVELOPER SAID “Yeah, the test suite is too slow and it’s hurting quality. I want to fix this by running most tests in a single process. It’s going to take me two to four weeks of not adding new features, but we’ll make up for it in increased developer productivity and a higher quality application.” mardi 3 avril 2018©2018 ovid@allaroundtheworld.fr
  9. 9. WHAT THE MANAGER HEARD “Yeah, the test suite is too slow and it’s hurting quality. I want to fix this by running most tests in a single process. It’s going to take me two to four weeks of not adding new features, but we’ll make up for it in increased developer productivity and a higher quality application.” mardi 3 avril 2018©2018 ovid@allaroundtheworld.fr
  10. 10. mardi 3 avril 2018©2018 ovid@allaroundtheworld.fr Public domain. Erin Whittaker, U.S. National Park Service - Grand Canyon National Park Facebook page Manager s Developers
  11. 11. WHY DON’T MANAGERS LISTEN? • Managers trust other managers • Developers are always asking to change things • Managers focus on business needs • Developers focus on technical needs • Risk profile is different • Developers like fun tasks mardi 3 avril 2018©2018 ovid@allaroundtheworld.fr
  12. 12. HOW DO WE GET PAST THIS? mardi 3 avril 2018©2018 ovid@allaroundtheworld.fr
  13. 13. THEORY • CEO and others create goals • Upper management creates a strategy • Middle management develops tactics • Developers implement those tactics mardi 3 avril 2018©2018 ovid@allaroundtheworld.fr
  14. 14. REALITY • Everybody makes shit up uses their intuition mardi 3 avril 2018©2018 ovid@allaroundtheworld.fr
  15. 15. WHAT THE MANAGER HEARD “Yeah, the test suite is too slow and it’s hurting quality. I want to fix this by running most tests in a single process. It’s going to take me two to four weeks of not adding new features, but we’ll make up for it in increased developer productivity and a higher quality application.” mardi 3 avril 2018©2018 ovid@allaroundtheworld.fr
  16. 16. THE BUSINESS CASE The “lite” version mardi 3 avril 2018©2018 ovid@allaroundtheworld.fr
  17. 17. FIRST QUESTIONS 1. What problem are you trying to solve? 2. Why are you trying to solve it? 3. What are your obstacles? 4. What is your solution?* 5. Why is it better than the alternatives? mardi 3 avril 2018©2018 ovid@allaroundtheworld.fr
  18. 18. THE SOLUTION • What are the direct costs? • What are the opportunity costs? • What are the risks? (low, medium, high) • What are the rewards? (low, medium, high) mardi 3 avril 2018©2018 ovid@allaroundtheworld.fr
  19. 19. TIPS • Be honest • Avoid loaded language • Focus on business needs • Catalyst vs. Mojolicious mardi 3 avril 2018©2018 ovid@allaroundtheworld.fr
  20. 20. CATALYST VERSUS MOJOLICIOUS RISKS Software appliances mardi 3 avril 2018©2018 ovid@allaroundtheworld.fr
  21. 21. CATALYST • No explicit guarantees in the documentation • Strong, proven track record of backwards- compatibility • Deemed “low risk” mardi 3 avril 2018©2018 ovid@allaroundtheworld.fr
  22. 22. MOJOLICIOUS « Features may only be changed in a major release, to fix a serious security issue, or after being deprecated for at least 3 months. » mardi 3 avril 2018©2018 ovid@allaroundtheworld.fr http://mojolicious.org/perldoc/Mojolicious/Guides/Contributing
  23. 23. MOJOLICIOUS « Only the two most recent stable release series of Perl are supported. » mardi 3 avril 2018©2018 ovid@allaroundtheworld.fr http://mojolicious.org/perldoc/Mojolicious/Guides/FAQ
  24. 24. MOJOLICIOUS « We will also keep the distribution installable up to a certain legacy version that we deem worthy of supporting, but not specifically optimize for it, this is currently 5.10.1. » mardi 3 avril 2018©2018 ovid@allaroundtheworld.fr http://mojolicious.org/perldoc/Mojolicious/Guides/FAQ
  25. 25. MOJO BREAKING CHANGES • May 25: 4.01 release announcing the addition of a new reserved stash value in 3 months. • June 3: 4.05 release announcing that all render methods will now return the invocant instead of true/false. • June 11: 4.09 release announcing that respond_to content negotiation will now honor other previously ignored headers. • … • August 25: 4.21 release with a breaking change. • September 3: 4.32 release with another breaking change. • September 11: 4.33 release with yet another breaking change. “We go from breaking changes once a year to breaking changes every week.” – Sebastian Riedel mardi 3 avril 2018©2018 ovid@allaroundtheworld.fr https://groups.google.com/forum/#!topic/mojolicious/WnjvVC0bYjY
  26. 26. MOJOLICOUS • Few explicit guarantees in the documentation • Documentation suggests no commitment to backcompat • History of developer complaints about breaking changes • Deemed “medium to high risk” This is not a criticism of Mojolicious! mardi 3 avril 2018©2018 ovid@allaroundtheworld.fr
  27. 27. RISK mardi 3 avril 2018©2018 ovid@allaroundtheworld.fr CC BY-SA 3.0 https://commons.wikimedia.org/wiki/User:Guma89
  28. 28. BUSINESS CASE—TEST SUITES PROS • Easier to run tests • Easier to find tests • Fewer regressions = lower maintenance costs • Increased productivity = lower development costs CONS • X? weeks of lost time • Hard to get started mardi 3 avril 2018©2018 ovid@allaroundtheworld.fr
  29. 29. LOWER DEVELOPMENT COSTS • 1 hour 20 minutes ⇒ 15 minutes • Extra hour of developer productivity per day • Equivalent of hiring having an extra developer • Pays for itself in a month mardi 3 avril 2018©2018 ovid@allaroundtheworld.fr
  30. 30. REWRITE OR RESTRUCTURE? Pros and Cons mardi 3 avril 2018©2018 ovid@allaroundtheworld.fr
  31. 31. OR THE THIRD OPTION … • Rewriting • Refactoring • Reimagining mardi 3 avril 2018©2018 ovid@allaroundtheworld.fr
  32. 32. ROBERT X. CRINGELY SAYS … “Cleaning up code” is a terrible thing. Redesigning WORKING code into different WORKING code (also known as refactoring) is terrible. The reason is that once you touch WORKING code, it becomes NON-WORKING code, and the changes you make (once you get it working again) will never be known. It is basically a programmer’s ego trip and nothing else. Cleaning up code, which generally does not occur in nature, is a prime example of amateur Open Source software. mardi 3 avril 2018©2018 ovid@allaroundtheworld.fr https://web.archive.org/web/20121113182409/http://www.pbs.org/cringely/pulpit/2003/pulpit_20030501_000770.html
  33. 33. RESTRUCTURING … Refactor [ree-fak-ter] Noun: The process of making a series of gradual code changes without behavioral changes with the goal of improving quality lowering maintenance costs. mardi 3 avril 2018©2018 ovid@allaroundtheworld.fr
  34. 34. REWRITE PROS • Cleaner architecture • Easier to maintain • More developers • Make the impossible possible CONS • Paying for two projects • Extremely high risk • Lost business knowledge • Lost technical knowledge • Larger scope of work mardi 3 avril 2018©2018 ovid@allaroundtheworld.fr
  35. 35. SPAGHETTI V. SPAGHETTI PROS • Cleaner architecture • Easier to maintain • More developers • Make the impossible possible CONS • Maintaining two projects • Extremely high risk • Lost business knowledge • Lost technical knowledge • Larger scope of work mardi 3 avril 2018©2018 ovid@allaroundtheworld.fr
  36. 36. RESTRUCTURING PROS • Business knowledge kept • Technical knowledge kept • Only maintain one system • Smaller scope of work CONS • Technology lock-in • “Impossible” needs • Getting buy-in mardi 3 avril 2018©2018 ovid@allaroundtheworld.fr
  37. 37. GETTING BUY-IN mardi 3 avril 2018©2018 ovid@allaroundtheworld.fr Business needs Technical needs Business case
  38. 38. WOULD YOU PITCH THIS? In a landmark 1995 study, the Standish Group established that only about 17% of IT projects could be considered “fully successful,” another 52% were “challenged” (they didn't meet budget, quality or time goals) and 30% were “impaired or failed.” In a recent update of that study conducted for ComputerWorld, Standish examined 3,555 IT projects between 2003 and 2012 that had labor costs of at least $10 million and found that only 6.4% of [IT projects] were successful. mardi 3 avril 2018©2018 ovid@allaroundtheworld.fr https://www.informationweek.com/strategic-cio/executive-insights-and-innovation/why-do-big-it-projects-fail-so-often/d/d-id/1112087
  39. 39. WHY REWRITES FAIL • “But we’ve spent too much money to stop now!” • Nobody wants to admit they’re wrong • No capacity planning • Unclear goals mardi 3 avril 2018©2018 ovid@allaroundtheworld.fr
  40. 40. BIGGEST REFACTORING OBJECTION? mardi 3 avril 2018©2018 ovid@allaroundtheworld.fr
  41. 41. “WE DON’T KNOW HOW.” mardi 3 avril 2018©2018 ovid@allaroundtheworld.fr
  42. 42. THIS IS “HOW”. mardi 3 avril 2018©2018 ovid@allaroundtheworld.fr
  43. 43. WHAT’S THE PROBLEM? • Too buggy? • Too slow? • Too unmaintainable? • Something else? mardi 3 avril 2018©2018 ovid@allaroundtheworld.fr
  44. 44. OODA LOOP • Observe • Orient • Decide • Act mardi 3 avril 2018©2018 ovid@allaroundtheworld.fr
  45. 45. OODA LOOP • Observe – Gather information • Orient – Process information • Decide – Take a decision • Act – Implement that decision mardi 3 avril 2018©2018 ovid@allaroundtheworld.fr
  46. 46. OBSERVE: GATHER INFORMATION • What are the functional requirements? • What is the architecture? • What documentation exists? (Danger!) • What tests exists? • What areas of the code are more fragile? • What external resources does it require? • …? mardi 3 avril 2018©2018 ovid@allaroundtheworld.fr
  47. 47. ORIENT: PROCESS INFORMATION • Identify root cause of goal obstacles • First facts, then opinions, then emotions • Focus on root causes! mardi 3 avril 2018©2018 ovid@allaroundtheworld.fr
  48. 48. DECIDE: TAKE A DECISION • Define success • 50% test coverage ☹ • 50% monthly reduction in support calls ✓ • Decide your approach • Set a timeframe mardi 3 avril 2018©2018 ovid@allaroundtheworld.fr
  49. 49. ACT: JUST DO IT! mardi 3 avril 2018©2018 ovid@allaroundtheworld.fr
  50. 50. DESIGNATED TECHNICAL EXPERT • Understands business needs • Understands system design/architecture • Expert in primary language(s) of your codebase • A strong automated testing background • Very comfortable with code coverage tools • A strong database background • Ability to admit when they're wrong mardi 3 avril 2018©2018 ovid@allaroundtheworld.fr
  51. 51. ARCHITECTURE ROADMAP • Determine the “ideal” architecture for your application • Decide how to refactor towards this ideal • Understand what must not change • Integration tests against what must not change • Pick a small initial target mardi 3 avril 2018©2018 ovid@allaroundtheworld.fr
  52. 52. PICKING YOUR TARGET sub product { my ($self, $customer_id, $product_id) = @_; … my $product = $self->dbh->selectall_arrayref( “SELECT * FROM …” ); … print <<“END_HTML”; <html><head> … mardi 3 avril 2018©2018 ovid@allaroundtheworld.fr
  53. 53. PICKING YOUR TARGET sub product ($self, $product_id) { my $product = $self->model('Products')->fetch($product_id) or $self->redirect('/status_not_found'); $self->stash( product => $product ); } mardi 3 avril 2018©2018 ovid@allaroundtheworld.fr
  54. 54. INITIAL TARGET • SQL? • HTML? • Logic? mardi 3 avril 2018©2018 ovid@allaroundtheworld.fr
  55. 55. INITIAL TARGET • Customer-facing? • Backend? • Smallest task/risk? mardi 3 avril 2018©2018 ovid@allaroundtheworld.fr
  56. 56. INTEGRATION TESTS • Messy applications are hard to unit test • Integration tests against the “view” • Don’t stress about TDD mardi 3 avril 2018©2018 ovid@allaroundtheworld.fr
  57. 57. TESTS • Restructuring should not change behavior • Tests can help (not prevent) avoid changing behavior • You probably need to rethink your testing strategy mardi 3 avril 2018©2018 ovid@allaroundtheworld.fr
  58. 58. TEST SUITE • Use a test framework! • Test database changes should be transparent • High-level integration tests are high value • Put fixtures in roles for easy sharing • Tests must be discoverable • Correctness first, then performance mardi 3 avril 2018©2018 ovid@allaroundtheworld.fr
  59. 59. CORRECTNESS EXAMPLE • Same database as production • Same version • Same configuration mardi 3 avril 2018©2018 ovid@allaroundtheworld.fr
  60. 60. TEST FRAMEWORK • Large websites use frameworks • Large test suites should use frameworks • Test::Class::Moose (TCM) mardi 3 avril 2018©2018 ovid@allaroundtheworld.fr
  61. 61. TCM BASE CLASS PSEUDO-CODE package TestsFor::My::Project { use Test::Class::Moose; sub test_startup { begin transaction } sub test_setup { begin savepoint } sub test_teardown { rollback savepoint } sub test_shutdown { rollback transaction } } mardi 3 avril 2018©2018 ovid@allaroundtheworld.fr
  62. 62. WRITE TESTS! sub test_vip_accounts_regenerate_stats_faster ( $test, $ ) { my $winston = $test->character_winston; ok !$winston->is_vip, 'Characters should not start as having a vip account'; my $yesterday = $test->now - period( days => 1 ); $winston->vip_until($yesterday); ok !$winston->is_vip, '... and vip_until in the past is not VIP'; mardi 3 avril 2018©2018 ovid@allaroundtheworld.fr
  63. 63. mardi 3 avril 2018©2018 ovid@allaroundtheworld.fr
  64. 64. PHASE TRANSITION A change in a feature of a physical system that results in a discrete transition of that system to another state. Phase transitions often involve the absorption or emission of energy from the system; ice, at 0 ° Celsius, must absorb a considerable amount of heat energy to become water. mardi 3 avril 2018©2018 ovid@allaroundtheworld.fr
  65. 65. ITERATING 1. Did the transition succeed? 2. What are our next steps towards our goal? 3. Plan next steps 4. Execute steps 5. Goto step1 mardi 3 avril 2018©2018 ovid@allaroundtheworld.fr
  66. 66. ITERATING • Separate restructuring and changing • Remember your timeframe • Remember your success criteria mardi 3 avril 2018©2018 ovid@allaroundtheworld.fr
  67. 67. PRIORITIZATION mardi 3 avril 2018©2018 ovid@allaroundtheworld.fr
  68. 68. PRIORITIZATION • You already have buy-in • But what to do next? • Build mini ‘business cases’ mardi 3 avril 2018©2018 ovid@allaroundtheworld.fr
  69. 69. A THREE-NUMBER BUSINESS CASE mardi 3 avril 2018©2018 ovid@allaroundtheworld.fr
  70. 70. A THREE-NUMBER BUSINESS CASE mardi 3 avril 2018©2018 ovid@allaroundtheworld.fr
  71. 71. THREE NUMBERS • Complexity • Monetization • Utility mardi 3 avril 2018©2018 ovid@allaroundtheworld.fr
  72. 72. SCORING score = ( ( (monetization * 40 + usability * 20) - (complexity * 40) ) + 500 )/10 mardi 3 avril 2018©2018 ovid@allaroundtheworld.fr
  73. 73. A THREE-NUMBER BUSINESS CASE mardi 3 avril 2018©2018 ovid@allaroundtheworld.fr
  74. 74. BUILDING THE NUMBERS • Automate spreadsheet creation • Devs choose the complexity value • Product owner chooses monetization and utility values • Other factors may override! mardi 3 avril 2018©2018 ovid@allaroundtheworld.fr
  75. 75. NEXT STEPS • Always focus on your measurable goal • You can stop at any time • You always have working code mardi 3 avril 2018©2018 ovid@allaroundtheworld.fr
  76. 76. QUESTIONS? mardi 3 avril 2018©2018 ovid@allaroundtheworld.fr

Notes de l'éditeur

  • The “uninitialized value” bug at the BBC
  • It’s like a paid vacation for the dev.
  • You may think your argument is solid, but managers have many other equally “solid” arguments for new features.
  • But let’s pretend they don’t
  • It’s like a paid vacation for the dev.
  • Many resources on the web for writing a business case or project proposal
    BBC: CVS vs Subversion
  • BBC: rewrote XML serialization in a month.
  • Client was building software appliances (A software appliance is a software application combined with just enough OS to run on industry-standard hardware: server or VM)
    Upgrading underlying Perl not a realistic option
    Fixing a bug in Cat often means hunting through tons of dependencies. Mojo often just needs to be updated.
  • You can’t keep piling on risks unless you like “Corporate Jenga”
  • Less than a month if you factor in the costs of catching more bugs and writing more tests
  • If you don’t mention alternatives, you look sloppy.
  • BBC: rewrote XML serialization in a month.
  • When costs encroach on revenue
    or there’s an external threat to your business model
    Needs to be addressed long before the tipping point
    Company who tried to hire us to maintain the system they were rewriting to Java
  • Don’t refactor code you don’t need to maintain.
  • Business knowledge: forgetting to assign a project to R&D and losing a tax break
    Lost technical knowledge: forgetting that some routers implement SNMP incorrectly
    Larger scope of work and more up front planning to layout what you’re going to keep
  • Buy-in: see previous “intuition” slide
    Lock-in: Companies are bad at hiring
  • Refactor: miss your goals, you still have your software.
    Rewrite: miss your goals, you may have nothing.
  • If management is keen on rewriting, call it a “in-place rewrite”. Better still, call it “reinventing.”
  • It’s not “too slow”, it’s “too slow because …”
  • “Orient” is processing the information. Here’s where we really need hard data, but it’s often where intuition (read: biases) kick in.
  • “Orient” is where we really need hard data, but it’s often where intuition (read: biases) kick in.
  • Functional requirements are high level
    Fragility: bug/issue tracker and support staff
  • 50% reduction in support calls: lose enough customers and you’ll hit that target. Hide your support number and you’ll hit that target.
  • They might lead the project or simply consult
  • For the sake of argument, we’ll assume few or no tests exist
  • EXCELLENT TIME TO GET EXPERT HELP!
  • Pull out template layer?
    Pull out SQL?
    Build a proper model?
  • But what if you have a critical demo?
    Or what if you have one task with external dependencies beyond your control?
  • But what if you have a critical demo?
    Or what if you have one task with external dependencies beyond your control?
  • Your weights and priorities will differ
  • But what if you have a critical demo?
    Or what if you have one task with external dependencies beyond your control?
  • Does one ticket block another? Do you have legal considerations?
    Is a very important customer demanding a feature?

×