Distributed Agile Testing for Enterprises
•Télécharger en tant que PPTX, PDF•
2 j'aime•653 vues
This document discusses distributed agile testing for enterprises. It covers challenges with distributed teams like reduced communication bandwidth and increased noise. It presents practices for distributed testing like using executable specifications, test automation, continuous integration (CI), and collaborating across functional teams. The presenters are Anand Bagmar and Manish Kumar from ThoughtWorks who have many years of experience in software testing.
Recommandé
Recommandé
Contenu connexe
Tendances
Tendances (9)
En vedette
Similaire à Distributed Agile Testing for Enterprises
Similaire à Distributed Agile Testing for Enterprises (20)
Plus de ThoughtWorks Studios
Plus de ThoughtWorks Studios (20)
Dernier
Dernier (20)
Distributed Agile Testing for Enterprises
- 1. Distributed Agile Testing for Enterp Anand Bagmar & Manish Kumar
- 2. Presenters Anand Bagmar Manish Kumar Lead Consultant (QA), Testing Practice Lead, ThoughtWorks India ThoughtWorks India Software testing > 11 years, Software testing > 15 years > 14 years in the industry Anand.Bagmar@thoughtworks.com Manish.Kumar@thoughtworks.com
- 3. What is your expectation from this dis
- 5. What is this session about?
- 6. This is not a …
- 9. Project Plan/Estimation Requirements Gathering Use Cases / Functional Specs Design Vision & High Specifications Level Stories Code $ Test Release 1 $ Fix / Integrate $ Release 2 $ Release 3 $ Release 1 Release 4
- 10. Emerging paradigms of testing … Building quality in Involving everyone Business optimize value The principles that matter Fast feedback Clear and consistent view of Testing Tests are an asset Faster delivery into production
- 11. Why distributed teams exist?
- 12. Reduced communication bandwidth Cultural issues Increased noise Shared understanding Visibility into progress Timely decisions Trust and rapport Working in the same way Challenges
- 13. Practices, Tips and Tricks
- 14. Mindset Cross pollination, by frequent rotation Keep an open mind Be positive KISS principle Trust your team(s) ONE PRODUCT, ONE TEAM!
- 15. Communication
- 19. ATDD – Table scenario driven Expected Actual ANYTIME DAY 5 ANYTIME DAY ‘5’
- 20. Test Automation
- 21. Toolsets Communication Overhead Testing Common & consistent Information islands Flexible Project CI Collaboration
- 22. Testing Tool DSL Communication Tagging Overhead Common & consistent Video, Screenshots Information islands Support for CI Flexible Multiple Drivers Exploratory testing support Customizations
- 23. CI Source Repository poll Compile BVT Deploy QA Run FULL Deploy to Regression staging
- 24. Collaboration What I need to do in relation to what everyone else is doing?
- 25. Environments Automation Performance Lab Production Test 1 Test Development Test 2 UAT
- 26. Distributing work Analysts Developers Testers Infrastructure Functional teams Division by role What is to be tested? From where can that be tested most effectively?
- 32. Practices, Tips and Tricks Mindset Environments IPMs, Showcases Communication Retrospectives Distributing Work NFRs Toolset Defects ATDD Reporting & Metrics Test Automation
- 33. Anand Bagmar Lead Consultant (QA), ThoughtWorks Anand.Bagmar@thoughtworks.com Manish Kumar Testing Practice Lead, ThoughtWorks India Manish.Kumar@thoughtworks.com
Notes de l'éditeur
- Anand:I am a Tester. My interest and passion for software quality and test automation has taken me on many interesting journeys in my career of over 11 years in Software testing, and overall > 14 years in the IT field. In this time period, I have had opportunities to work for organizations like WebMD, Borland, Microsoft and AmberPoint. Along with core testing activities, I have done product support, client interactions, established processes, system administration … basically whatever it needs to get the work done. In ThoughtWorkssinceover 2 years, my profile is of a QA. I am an hands-on software tester, and am involved in all aspects of (manual, automation) testing, training, consulting and practices.
- AnandThis session is about teams, which, due to various reasons that we will touch upon later, are not seated under the same roof, in the same room. These teams are almost self-sufficient in a way … each team has developers that are building functionality, and QA team members that are testing what is being built. In addition, these teams have business analysts to guide them and maybe project managers to make sure development is on track and issues are identified and mitigated at the right time, with the correct people.That said, because the teams are split in different locations, be it same building / city, or different cities / countries / continents, everyone is still working on the same product. Thus, integration of the functionality developed by each team into one product is crucial to be done correctly. That means there is more to testing than what is done by each team, for what they develop.
- Anand
- AnandThis session is about teams, which, due to various reasons that we will touch upon later, are not seated under the same roof, in the same room. These teams are almost self-sufficient in a way … each team has developers that are building functionality, and QA team members that are testing what is being built. In addition, these teams have business analysts to guide them and maybe project managers to make sure development is on track and issues are identified and mitigated at the right time, with the correct people.That said, because the teams are split in different locations, be it same building / city, or different cities / countries / continents, everyone is still working on the same product. Thus, integration of the functionality developed by each team into one product is crucial to be done correctly. That means there is more to testing than what is done by each team, for what they develop.
- AnandThe tips and strategies we are going to talk about are based on:our experiences, what we have seen, done well, learnt from what has not gone wellNO “ONE-SIZE FITS ALL” SOLUTION!!Practices should be applied within the context.EVOLUTION IS THE KEY! - cannot build a bridge by laying its surface at the same time as building its foundation - keep the big picture in mind
- Why?? - Business objectiveAnandRemain the same …..Rapid releases – to cash-in on Time to marketGood quality productWhat does it mean for IT organizations?
- Manish
- What??? Agile in Software DeliveryCan be done to accomplish the “Why”?Manish
- How? To answer the Why and the What!ManishIn our earlier webinar On "Emerging Paradigms of Testing" KristanVingrys spoke about an inside out approach to testing.Lets recap the principles that matter for implementing "The Inside Out approach".We are going to extend these principle in the context of distributed team in this webinar and talk about practices,tools and tips that allow us to use the principle on distributed teamsThe first principle is "Building the quality In".We believe testing should focus on building quality in rather than "testing it in".Remove the cost of fixing the defects don’t just reduce the cost.Find mistakes early and prevent them from becoming prevalent.Tests become executable specification and drive the development.The next principle is "Involving Everyone".Get better tests through diverse input and the team looking after the tests.Also stops over the wall syndrome that results into gap in expectation between different roles and the resulting surprises.Another Principle is about generating instant feedback. This helps in providing predictability in the software development cycle by finding and fixing issues closer to the event that caused the problem.ATDD,Running Test in a CI build pipeline,pairing on the dev box are some of the practices that help achieve this.Tests are an asset of the product allowing reuse across various projectsFast Delivery into Production - Tests should not become last hurdle to cross by testing throughout the projectClear and Consistent view of testing across projectsSimple reporting to help make decisions and navigate the projectBusiness optimize value -Not just Insurance, FIT for purpose,help drive new features and functionalityBefore start looking into how these principles apply,lets look at the reason why distributed team exist?
- ManishDistributed teams are fact of life.Some of the reasons why distributed team existGlobalization - Organizations trying to expand into emerging markets.Also there is a need for localization for the marketThere is an increasing pressure to increase the value delivered for every dollor spentFor some teams 24*7 testing is essential to help rapid releases and provide rapid feedback from test.Rapid Globalization is happening as a result of acquisitions and Mergers for companies to remain competitiveTeam size - Larger projects mean larger team.It may not be possible to locate everyone in same physical space
- ManishShared understanding of projects,goals,vision,status across teams in different locationTimely Decisions - time zone differences mean that you will have to wait unless the stakeholder is present.Defecttriaging can't be done instantaneously.Trust and rapport because of physical separationVisibility in the progress of what each team is doingCommon and consistent way of working.for example Part of the team doing ATDD and rest traditional wayWhile these challenges exist for colocated team only these become much harder for distributed teams because of Lack of communication bandwidthIncreased noise because of physical separationCultural issues and time zone difference
- AnandLets talk about some of the practices you could potentially adopt on your teams to make Distributed Testing more effective.Mindset, Toolset, ATDD, Test Automation, Environments (NFRs), Retrospectives, Defect
- AnandIndividual’s humanpsycology plays a huge part when we interact with others. We most likely would react differently with people in tough situations when they are f2f Vs when in a different location.Here are a few things that can help bridge this gap:Follow KISS (Keep It Simple Silly!) principleKeep an open mindCross-pollination by frequent rotationBe positiveTrust your team – be an optimist than a pessimist ONE PRODUCT, ONE TEAM!
- AnandPeople over process!Not talking f2f with anyone is a pain. Especially if communication is only over emails and status updates and reports. You don’t understand the thought process the other is going through and vice versa.So … what can you do to make it easier:Talk …. Don’t write -> example: GSI case: sending emails to ppl in the same roomPick up the phoneSchedule regular catch-up callsVideo conferencingAlternate daily / weekly schedule to accommodate late / early time callsThis makes it fair for all teamsIncreases trustEveryone feels valuableVideo transcripts – status, demo – with transcripts / logs / annotations to enable searchingClear way to express test for Team understanding – Create Test DSLs, pictorial when ambiguousPairing, info sharing, trainings, webex, vnc, skype, IM (share OVE example)
- AnandBusiness / domain languageGiven – When – ThenExplicitNo assumptionsATDD - BDD / tables scenario driven (FIT) / free form dsl (Twist)
- AnandBusiness / domain languageGiven – When – ThenExplicitNo assumptionsATDD - BDD / tables scenario driven (FIT) / free form dsl (Twist)
- AnandBusiness / domain languageGiven – When – ThenExplicitNo assumptionsATDD - BDD / tables scenario driven (FIT) / free form dsl (Twist)
- AnandBusiness / domain languageGiven – When – ThenExplicitNo assumptionsATDD - BDD / tables scenario driven (FIT) / free form dsl (Twist)
- Anand1. Test code should be production quality2. Automation configurable to run against different environments3. Use design patterns4. Proper abstraction layers (tools, browser agnostic, libraries)5. Extensible6. CI7. screenshots on failures8. Logging9. Video recording9. Proper logging – test * product9. Incremental refactoring, 9. Dev pairing – quality of test code, testability, system perspective, 9. Evolve framework, 9. Decouple test data from test implementation, 9. No Copy Paste, 9. Tools & Utilities
- ManishToolSetsAny toolset in a distributed team should aid toCrisp & Clear Communication∫No Extra overhead of traceability∫Common tools for consistent usage across location∫Avoid information islands∫Flexibleand should work with each other
- ManishDSL - shared understanding of test across role.helps creating executable specification.Toolscucumber,FIT,TWISTTagging - Allows tests to align to features,allows to run test only for the features that are changing, categorize them from speed of execution and in general allows test to be categorized and run in multiple conceivable waysVideo Recording and test snaphots for test failures allows clear communicationSupport for CI - support for ant,maven helps test to be hooked up in CI pipelineMultiple drivers - Tools should allow support for multiple drivers to run test through various interfaces - Web applications, webservices without changing the way you express tests - TWIST,In other cases custom abstraction layers need to be created.Exploratory testing support - session recording with annotated videos,screenshots,semi automated execution to improve efficiency - TWIST has an excellent support for thisCustomizations
- ManishCI tool helps team across locations a common way to build test and deploy applicationA common source code repository for both source code and test code and a common CI buildpipeline is essential for team working across distributed location.
- ManishProject Collaboration Tool> Requirement + Project + Test> Common dashboard> Accessible to everyoneKeep it meaningfulValue Vs “Good to have”Common dashboard – visible to ALLTrends / ChartsAutomated
- ManishDeterministic and Repeatable self contained stubbed environment are essential-Many times third party application provide interesting challenges for testing,specially when environments are centrally managed Often not available 24*7 ,some time they are under development as well, Responses from them can’t be controlled to execute specific test execution path.Third party applications are stubbed out.When communications are down helps team to continue with testing. Redundancy - Multiple teams means need for multiple showcases,demo,testing sessions these typically lock the environment and teams have to wait.Also some times you want to run your test experiment in isolation from others.Coordinating to schedule environments is difficult for teams in different time zone and locationSame images for building environment (ex: virtualization) no environment difference means overhead of understanding environmental difference is reducedSeparate automation execution environment - not affected by what others are testing or changing an environment or its configuratioTest automation sandpit to help authoring of test scriptsDedicated performance test environment
- Manish: Primary focus: test what is developed in the same location.know what testing activity needs to happen in which location?what is to be tested and from where can that be tested most effectively?ex: functional testing, regression testing, story testing, exploratory testing = done best from where the development is happeningLoad / perf testing = best from where the full production-live environment will be available.team member rotationon-site representationunderstand cultural impactskillset of people on the grounddeployment process
- ManishKeep it meaningfulValue Vs “Good to have”Common dashboard – visible to ALLTrends / ChartsAutomated
- ManishKeep it meaningfulValue Vs “Good to have”Common dashboard – visible to ALLTrends / ChartsAutomated
- ManishKeep it meaningfulValue Vs “Good to have”Common dashboard – visible to ALLTrends / ChartsAutomated
- ManishKeep it meaningfulValue Vs “Good to have”Common dashboard – visible to ALLTrends / ChartsAutomated
- AnandIPM:With all team membersPlan purpose and priorityTeams bondPer iteration – NOT big bangIdentify Features / functionality cutting across teamsShowcases:Product Understand more perceptions / other perspectivesFeatures / functionality cutting across teamsProduct meeting stakeholder’s need and expectationsRetro:Retrospectives for distributed teams play a crucial role. They are the time specifically put aside to reflect on how the team is performing and what can be done to improve. Video conferencingEveryone should be there> Check safetyBe open, not judgementalNo finger pointingEveryone is of the understanding that each team member gave 100% Create action items and assign ownersAct on action items!Followup on action items in the next retro> Tools like – ideaboardz,
- AnandAssumptionsOne location “better” / “more important” than the otherLack of trust / transparency between teams“Us Vs Them” instead of “one team”
- AnandAssumptionsOne location “better” / “more important” than the otherLack of trust / transparency between teams“Us Vs Them” instead of “one team”
- AnandAssumptionsOne location “better” / “more important” than the otherLack of trust / transparency between teams“Us Vs Them” instead of “one team”
- AnandAssumptionsOne location “better” / “more important” than the otherLack of trust / transparency between teams“Us Vs Them” instead of “one team”
- AnandAssumptionsOne location “better” / “more important” than the otherLack of trust / transparency between teams“Us Vs Them” instead of “one team”
- AnandMindset, Toolset, ATDD, Test Automation, Environments (NFRs), Retrospectives, DefectMindsetEnvironmentsCommunicationDistributing WorkToolsetTest AutomationATDDReporting & MetricsDefect reportingNFRsRetrospectives
- QuestionsHow do you manage test suites in distributed teams? - categorize tests Run in build pipeline Reduce noise from failing tests (quarantined suite, dependent tests) Consolidate overlapping tests- Not Cast in stone- Deprecate Non-applicable tests2. you mention cross-pollination and rotations. when are the best times in the project for testers to travel?3. do you have any suggestions for creating a common global testing environment to prevent confusion caused by executing tests in non-equivalent configurations?4. you mention that when you distribute the test work that you should think through what kind of testing you do in each location. what are the important factors to consider?5. Do you have any tips for NFR testing in distributed teams?6. What is bad about Testing Center of Excellence / offshore testing?