Ce diaporama a bien été signalé.
Nous utilisons votre profil LinkedIn et vos données d’activité pour vous proposer des publicités personnalisées et pertinentes. Vous pouvez changer vos préférences de publicités à tout moment.

TestOps and Shift Left

Describing how shift left in Agile and Devops is creating a need for TestOps as part of ITOPs to maintain testing infrastructure and data

TestOps and Shift Left

  1. 1. 1 Prepared By: Gervais Johnson 601 Montgomery St, Suite 650, San Francisco, CA 94111 | 415-678-1350 | www.MatrixRes.com TestOps Test Driven Development and Shift Left
  2. 2. 2 Today’s Presenter Gervais (Jay) Johnson, Agile Delivery Manager  Leads, trains, coaches teams and companies in Agile development and adoption  Architect, Application and Software Engineer, and Agile Leader for cross sector and industry including NASA and DoD organizations  IBM - 16 Year Veteran and Thought Leader  30 Years developing applications and changing the world  Education and certifications  Certified Scrum Master and Scrum Professional  Scrum Master, Scrum Developer, Agile Expert Certified  PMI Project Management Professional  Scaled Agile Framework Program Consultant  IBM Certified Consultant, PM, SPSS, Agile Expert  Six Sigma Black Belt
  3. 3. 3 Outline  What is TestOps  TestOps Ecosystem  Shift Left Movement  Test Driven Development (TDD)  Behavior Driven Development (BDD)  Acceptance Test Driven Development (ATDD)  Continuous Testing (CT)  TDD Details  Open Discussion
  4. 4. 4 Introductions  What is your name?  What is your role?  What prior Testing and Agile experience do you have?  0 to 10  0 = Brand New  10 = Testing Expert
  5. 5. 5 MATRIX Confidential The Agile Manifesto- “BE AGILE” We are uncovering better ways of developing software by doing it and helping others do it. “Responding to change over following a plan” “Customer collaboration over contract negotiation” “Working software over comprehensive documentation” “Individuals and interactions over process and tools” Values “The most efficient and effective method of conveying information to and within a development team is face-to-face conversation.” “Welcome changing requirements, even late in development. Agile processes harness change for the customer's competitive advantage.” “Working software is the primary measure of progress.” “Agile processes promote sustainable development. The sponsors, developers, and users should be able to maintain a constant pace indefinitely.” “Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.” “Deliver working software frequently, from a couple of weeks to a couple of months, with a preference to the shorter timescale.” “At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly.” “Continuous attention to technical excellence and good design enhances agility.” “Business people and developers must work together daily throughout the project.” “Build projects around motivated individuals, give them the environment and support they need, and trust them to get the job done, “Simplicity -- the art of maximizing the amount of work not done – is essential.” “The best architectures, requirements, and designs emerge from self-organizing teams” Principles
  6. 6. 6 What is TestOps? Testing never stops, it is continuous and is the fabric of enterprise operations and the quality glue for Agile + DevOps
  7. 7. 7 TestOps Includes Various Testing Are we building the right product? Are we building the product right? Business Facing Technology Facing Functional, Story & Prototype Testing Unit & Component Testing Business Facing Technology Facing Exploratory & Usability Testing Performance, Load & Security Testing CritiqueProduct SupportingtheTeam
  8. 8. 8 An Agile TestOps Ecosystem Example A
  9. 9. 9 An Agile TestOps Ecosystem Example B ‘Shift Left’ – Operational Concerns Also BizDevOps and DevOpsSec is becoming an important part of the fabric or glue
  10. 10. 10 What is Shift Left Testing?  "Shift Left" is a QA practice being implemented in high-performninh Agile/DevOps teams that provides an effective means to perform testing during and in parallel of software development activities. – Software Development, test and operations work together to plan, manage and execute automated and continuous testing to accelerate feedback todevelopers – The rate of the feedback is determined by an organization’s desiredvelocity – Technology is used to automate processes and virtualizecomponents  QA/Testing is performed as a part of the development process (TDD) or as a service running (BDD or DDD) in parallel to development activities, shifting left accelerates feedback to developers and improves the quality of code delivered to QA  Shift left is an essential capability for Agile/DevOps teams and IT Operations teams
  11. 11. 11  Perspectives – Testing: Begin test executionearlier – Behavior Design Testing – A/B Testing – Feature Driven Testing – Development: Act on feedback to achieve value – PM: Resource allocation to address feedback – Infrastructure: Shift left environments available early, capacity drives availability – Deployment: Accelerated capacity to meet test periodicity  Activities: – plan, create and automate integrationtest cases in advance – support a periodic cycle of integrationtesting for the components/applications under change – prioritize, process, and resolve feedbackby development teams within the feedback period – plan, automate and deploy components, applications and virtual services – plan, create and virtualizecomponent services relevant to the development activities – execute delivery, integration, build, deploy, testing, feedback and correction during the defined periodicity. Whatever testing you can perform shifts left - Integration testing could be your first step as a means to provide a means to prepare for a formal QA and reduce bottlenecks associated with Integration testing post development. What is Left Shifting?
  12. 12. 12 Shift Left Planning Workshop – Part of Agile Adoption Roadmap Goals and Assessment •Client Capabilities: CurrentStatus •Business Goals for Improvement •Solution Capability Oriented •People, Practices, Technology,information Capability Priorities •Best Practices for SolutionCapabilities •Discussions to refine ObjectiveCapability •Prioritize Incremental Capabilities •Vision, User Value, Pain and Complexity Executive Sponsor Review •Review Outcomes •Assumptions & Risks •Gain concurrence •Identify next steps Solution Improvement Roadmap •Best Practices for AdoptingSolution •Identify Key Milestones •Roadmap activity to define actionableplan •Define Quick Win Pilot Day1Day2Day3Day4 Current State Assessment Objective & Prioritized Capabilities Agile Adoption Roadmap DraftResults Detailed 1-3 month roadmap for initial win. High level roadmap for remainder of initiative. Executable week following the workshop Theme Activities Objective
  13. 13. 13 TDD and BDD Place • Unit test (Unit) • Integration test (Collaboration) • User interface test (Frontend, ex. WatiN) • Regression test (Continuous Integration) • …, System, Performance, Stress, Usability, … Test types: • Black-box test • White-box test The only tests relevant to TDD and BDD is Black-box Unit Testing
  14. 14. 14 Feature Driven Development (FDD) ● Is a client-centric, architecture-centric, and pragmatic software process ● Feature Breakdown Structure (FBS) instead of WBS ● A feature is a small, client-valued function expressed in the form: <action> the <result> by/for/of/to a <object>
  15. 15. 15 Test Driven Development (TDD) Is a rapid cycle of testing, coding, and refactoring ● Relies on a very short development cycle ● Developer writes automated test case first ● Test case defines desired improvement or new function ● Next develops minimum amount of code to pass test ● Lastly refactors the new code to acceptable standards Code refactoring is a "disciplined technique for restructuring an existing body of code, altering its internal structure without changing its external behavior"
  16. 16. 16 Behavior Driven Development (BDD) ● Outside-in and pull-based - Wire-frame > Test Cases > Coding ● Multiple-stakeholder, multiple-scale, high-automation ● Describes a cycle of interactions with well-defined outputs ● Results in the delivery of working, tested software that matters
  17. 17. 17 BDD and TDD
  18. 18. 18 Agile Story and BDD ID: 27 STORY NAME: Save a list of potential cars for later review As a: Buyer I can: Add a car to my wish list So that: I can review my top choices at a later time. Acceptance Criteria Scenario 27.1: Add a car to my wish list Given: A potential buyer is logged in And: The car is available for sale When: The ‘Add to Favorites’ option appears And: A buyer flags the car for the wish list Then: The car details are displayed in the wish list Scenario 27.2: Review list of favorite cars Given: A potential buyer is logged in Given: A potential buyer has previously picked some favorite cars’ And: A buyer clicks on “view my favorites’ When: The buyer views the wish list And: The car is still available for sale Then: The buyer can view the car summary in the wish list Story As a <role>, I can <activity>, so that <business value> Scenarios Given <context> When <event> Then <outcome>
  19. 19. 19 Acceptance Test Driven Development
  20. 20. 20 Technical Debt and Testing TDD BDD Source: http://www.slideshare.net/nashjain/inverting-the-testing-pyramid FDD
  21. 21. 21 Automated Testing Across Platforms Loadster LogiGear TestArchitect Microsoft test tools QASymphony Telerik Test Studio Sauce Labs Selenium Appium Synopsys Xunit SDK Apache JMeter Grinder Gatling Tsung LoadRunner WebLOAD Appvance NeoLoad LoadUI WAPT LoadImpact Rational Performance Tester Testing Anywhere
  22. 22. 22 PVCS SVN Artifactory Splunk Jenkins Maven Ant Puppet chef HP ALM Policy Tester Selenium Smart Cloud Control Desk PMG ServiceNow Microsoft Team Foundation Server Serena Dimensions Example Testing Across Platforms
  23. 23. 23 TDD History • Test-driven development (TDD) is a software development technique that uses short development iterations based on pre-written test cases that define desired improvements or new functions. Each iteration produces code necessary to pass that iteration's tests. Finally, the programmer or team refactors the code to accommodate changes. A key TDD concept is that preparing tests before coding facilitates rapid feedback changes. Note that test-driven development is a software design method, not merely a method of testing.
  24. 24. 24 TDD Method and Cycle New require- ment Write new test Run tests Write new code Run tests Refactor Run tests • Write Test Code – Guarantees that every functional code is testable – Provides a specification for the functional code – Helps to think about design – Ensure the functional code is tangible • Write Functional Code – Fulfill the requirement (test code) – Write the simplest solution that works – Leave Improvements for a later step – The code written is only designed to pass the test • no further (and therefore untested code is not created). • Refactor – Clean-up the code (test and functional) – Make sure the code expresses intent – Remove code smells – Re-think the design – Delete unnecessary code
  25. 25. 25 TDD Benefits 0 1 2 3 4 5 6 7 8 9 10 0% 20% 40% 60% 80% 100% Requirements Coding Integration Testing Support Thousand$s %defectscreated % of Defects Introduced Cost to Fix a Defect The pain is here! This is too late…
  26. 26. 26 TDD Benefits 0% 20% 40% 60% 80% 100% 120% 140% 160% MS: VS MS: MSN MS: Windows IBM: Drivers Time To Code Feature Time To Code Feature using TDD Defect density of team Defect density of team using TDD Major quality improvement for minor time investment
  27. 27. 27 TDD Fundamental Mind Exercise • When you first start at doing TDD you "know" what the design should be. You "know" what code you want to write. So you write a test that will let you write that bit of code. • When you do this you aren't really doing TDD – since you are writing the code first in your mind, mind change to write test first • It takes some time and mentoring to realize that you need to focus on the test first. • Write the test for the behavior you want - then write the minimal code needed to make it pass - then let the design emerge through refactoring. • Repeat until done. • TDD and Paired Programming are a great combination to improve quality and increase delivery time
  28. 28. 28 TDD Adoption Hurdles • Why, when TDD was born a decade ago? – The one and only reason: #1: Learning curve… – Infrastructure and willingness • Why do most developers not write unit tests, still? http://weblogs.asp.net/rosherove/archive/2008/09/20/goodbye-mocks- farewell-stubs.aspx mocks, stubs, dependency injection, IoC containers, Extract & override technique, RecordReplay, AAA and more
  29. 29. 29 TDD Fundamental Infrastructure • Interfaces • Dependency Injection • Test Doubles (Mock Objects) • Mocking Framework. • The builder and fluent interface patterns. Either (the old way) “Mock” Using Interfaces, Dependency Injection and Test Doubles (Mock Objects) Or (the new way) “Isolate” and “Fake” Using Typemock Isolator and AAA - The "Arrange-act-assert" pattern • Refactoring
  30. 30. 30 Why No Unit Testing and no TDD 1. I don’t have time to unit test. 2. The client pays me to develop code, not write unit test. 3. I am supporting a legacy application without unit tests. 4. QA and User Acceptance Testing is far more effective in finding bugs. 5. I don’t know how to unit test, or I don’t know how to write good unit tests.
  31. 31. 31 TDD Maturity • Classicists – The classical TDD style is to use real objects if possible and a double if it's awkward to use the real thing. So a classical TDDer would use a real warehouse and a double for the mail service. The kind of double doesn't really matter that much. • Mockist – A mockist TDD practitioner, however, will always use a mock for any object with interesting behavior. In this case for both the warehouse and the mail service. • Isolator and Fake – An isolator TDD practitioner, will always “isolate” or “fake” any object with interesting behavior. In this case for both the warehouse and the mail service. The Classicists does White-box Integration testing First generation TDD The Mockist does White-box Unit testing Sencond generation TDD The Isolator and Fake does Black-box Unit testing Third generation TDD
  32. 32. 32 TDD Maturity, adding BDD • Behaviour-Driven Development (BDD) – By Dan North – BDD is another variation on TDD that tends to use mockist testing • Test method names should be sentences • A simple sentence template keeps test methods focused – This sentence template – The class should do something – means you can only define a test for the current class. This keeps you focused. If you find yourself writing a test whose name doesn’t fit this template, it suggests the behavior may belong elsewhere. • An expressive test name is helpful when a test fails – “Behavior” is a more useful word than “test”. – Determine the next most important behaviour. – Requirements are behavior, too. – BDD provides a “ubiquitous language” for analysis. – Acceptance criteria should be executable.
  33. 33. 33 How to start TDD/BDD • You don’t have to start big • Start new tasks with TDD • Add Tests to code that you need to change or maintain – but only to small parts • Proof of concept
  34. 34. 34 References • Books – Test-Driven Development in Microsoft® .NET (http://www.amazon.co.uk/gp/product/0735619484) – Test-Driven Development by Kent Beck (C++) (http://www.amazon.co.uk/gp/product/0321146530) – Clean Code by Martin Fowler – Refactoring by Martin Fowler – Refactoring of Patterns by Kerievsky • Blog – The Typemock Insider Blog (http://blog.typemock.com/) • Unit test study from Microsoft Research: – http://research.microsoft.com/en-us/projects/esm/nagappan_tdd.pdf – https://en.wikipedia.org/wiki/XUnit and https://en.wikipedia.org/wiki/List_of_unit_testing_frameworks • Typemock – http://www.typemock.com/ • Unit testing Silverlight: – http://www.codeplex.com/CThru • Other links: – http://en.wikipedia.org/wiki/Test-driven_development – http://www.testdriven.com/ – http://www.mockobjects.com/ - Online TDD book: http://www.mockobjects.com/book/index.html – http://www.martinfowler.com/articles/mocksArentStubs.html – http://dannorth.net/introducing-bdd – http://behaviour-driven.org/Introduction
  35. 35. 35 TesOps Future Topics for Discussion  Testing Value Chain and Tools Chain  Test Automation Tools Demo  TDD and BDD Demo  Test Environment Management  Security Testing  Cloud Testing  Mobile Testing  Performance Testing  Test Data Management
  36. 36. 36 Open Forum Discussion

    Soyez le premier à commenter

    Identifiez-vous pour voir les commentaires

  • durgasubburaman

    Jul. 10, 2017
  • tvnghiepit

    Jun. 25, 2019
  • VijayakumarSasi

    Jul. 20, 2019
  • whilpert

    Feb. 7, 2020

Describing how shift left in Agile and Devops is creating a need for TestOps as part of ITOPs to maintain testing infrastructure and data


Nombre de vues

1 406

Sur Slideshare


À partir des intégrations


Nombre d'intégrations









Mentions J'aime