Ce diaporama a bien été signalé.
Nous utilisons votre profil LinkedIn et vos données d’activité pour vous proposer des publicités personnalisées et pertinentes. Vous pouvez changer vos préférences de publicités à tout moment.

Continuous Delivery Testing @HiQ

7 515 vues

Publié le

Talk about Continuous Delivery and Testing. Held at HiQ april 2014.

Publié dans : Technologie
  • DOWNLOAD FULL BOOKS, INTO AVAILABLE FORMAT ......................................................................................................................... ......................................................................................................................... 1.DOWNLOAD FULL. PDF EBOOK here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB Ebook here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... 1.DOWNLOAD FULL. doc Ebook here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... 1.DOWNLOAD FULL. PDF EBOOK here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB Ebook here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... 1.DOWNLOAD FULL. doc Ebook here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... ......................................................................................................................... ......................................................................................................................... .............. Browse by Genre Available eBooks ......................................................................................................................... Art, Biography, Business, Chick Lit, Children's, Christian, Classics, Comics, Contemporary, Cookbooks, Crime, Ebooks, Fantasy, Fiction, Graphic Novels, Historical Fiction, History, Horror, Humor And Comedy, Manga, Memoir, Music, Mystery, Non Fiction, Paranormal, Philosophy, Poetry, Psychology, Religion, Romance, Science, Science Fiction, Self Help, Suspense, Spirituality, Sports, Thriller, Travel, Young Adult,
       Répondre 
    Voulez-vous vraiment ?  Oui  Non
    Votre message apparaîtra ici
  • This whitepaper explains how we built a continuous testing framework for one of our high value enterprise clients and the challenges we faced along with the solutions we created to overcome those challenges. http://bit.ly/2FTSWT2
       Répondre 
    Voulez-vous vraiment ?  Oui  Non
    Votre message apparaîtra ici

Continuous Delivery Testing @HiQ

  1. 1. Continuous Delivery Continuous Testing
  2. 2. Tomas Riha Architect @ VGT/WirelessCar Passionate about creativity, change and improvement Horrible at following instructions and performing repetitive tasks MAJOR Project Liability mail: triha74@gmail.com twitter: @TomasRihaSE blog: continuous-delivery-and-more.blogspot.com
  3. 3. Agenda Why do we want to do Continuous Delivery Intro to Continuous Delivery Principles of Continuous Delivery Look at a Pipe Test Automation for Continuous Delivery Test Driven Development and Continuous Delivery Exploratory Testing and Continuous Delivery Some words on Tooling Areas not covered
  4. 4. Why do we want to do Continuous Delivery Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.
  5. 5. Why do we want to do Continuous Delivery Decreased lead time not only improves our time to market but also our ability to learn and improve through a faster feedback loop
  6. 6. Why do we want to do Continuous Delivery As soon as we have added value to the application it needs to be in the hands of the customer or else we have just added cost.
  7. 7. Why do we want to do Continuous Delivery Reduced risk through delivery of small changes.
  8. 8. Are we really delivering as soon as possible? Pre Planning Dev Sys Test Reg Test Pre Planning Dev Sys Test Reg Test Scrummerfall happens because its hard for developers to have something deployable for the testers to test. System testing on something that has not been regression tested is fundamentally flawed. Cost of regression test tends to bloat sprint content. Sprint 2-4 weeks
  9. 9. Continuous Delivery Build Release Deploy Test Prod Your application always builds, passes its unit tests, deploys and passes system tests.
  10. 10. Continuous Integration/Delivery/Deploy Continuous Integration Your application always builds and passes its unit tests. Continuous Delivery Your application always builds, passes its unit tests, deploys and passes its system tests making it always ready to deploy to production through an automated process Continuous Deployment Your application automatically deploys to production through a fully automated process
  11. 11. Continuously as in all the time? Yes! End of sprint isn't continuously. Weekly isn't continuously. Nightly isn't continuously. Every code commit is continuously!
  12. 12. What Tests? Unit? Component? System? Yes! Unit Component System Integration Rollback Load Failover
  13. 13. Principles of Continuous Delivery 1. The process MUST be repeatable, reliable and independent of key personnel
  14. 14. 2. Automate everything Principles of Continuous Delivery
  15. 15. 3. If something is difficult or painful, do it more often Principles of Continuous Delivery
  16. 16. 4. Keep everything in source control AND release it Principles of Continuous Delivery
  17. 17. 5. Done means released Principles of Continuous Delivery
  18. 18. 6. Build Quality In Principles of Continuous Delivery
  19. 19. 7. Fail Fast Principles of Continuous Delivery
  20. 20. 8. Everybody has responsibility for the release process Principles of Continuous Delivery
  21. 21. 9. Improve continuously Principles of Continuous Delivery
  22. 22. The pipe - Build once! Build Release Build and Unit test then release it!
  23. 23. The pipe - Release everything! Build Release Build Release Build Release Build Release Code DB Scripts Server Config Deploy script Feedback Feedback
  24. 24. The pipe -Bundle released artifacts Build Release Build Release Build Release Build Release Assembly Feedback
  25. 25. The pipe - Use Same Deploy Mechanism Function TestAssembly Deploy DB Server Function Test Function Test Pipe Status Feedback Feedback Feedback
  26. 26. The pipe - Use Same Deploy Mechanism Deploy DB Server Pipe Status Server DB Server Server DB Server Server Deploy Pre Prod Prod UAT FeedbackFeedback Feedback
  27. 27. Continuous Delivery Build Release Assemble Deploy Deploy PreProd/Prod Test Summary We build once Release everything Automate everything Fast feedback Automate all tests except UAT
  28. 28. Obvious benefits Continuous Regression Testing gives instant feedback. Continuously deploying to test servers tests deploy mechanism several hundred times per release. Always ready to push new release into UAT We can get customer feedback as soon as value has been added
  29. 29. Automate all tests! Wait a minute lets look at what we have been doing in the past.
  30. 30. Testing in the past How its been done SYSTEM GUI
  31. 31. Testing in the past What we have done 100s of manual tests that.. ... verify our requirements ... ensures bad input doesn't break system ... data is stored in the right place ... find bugs
  32. 32. Testing in the past We have automated it! We have tried to automate the 100s of manual tests that.. ... verify our requirements ... ensures bad input doesn't break system ... data is stored in the right place ... find bugs We do it through instrumenting the GUI and verifying Database, GUI and System resources. With very bad results....
  33. 33. Robust and Fast Test Automation Tests that execute 100s of times per day have to be... ... executable without any human interaction ... robust and give the same response every single time ... robust and have to survive refactoring ... fast and parallelizable as adding 10 tests should not slow down the feedback
  34. 34. We need to change the way we look at Test On System Level we do verification of Use Case Acceptance Criterias. Requirement Verification. On Component Level we do testing of Component Functionality. Testing.
  35. 35. We need to change the way we look at Architecture Remember “Build quality in”. We need to improve testability of our applications through... ...isolation of functionality into independent services with life cycles of their own. ...building interfaces for defined test points.
  36. 36. Building testable systems Internal Service GUI Internal Service Internal Service Service API
  37. 37. Component Tests Internal Service Internal Service Internal Service Each Service Tested as Black Box in isolation and with high detail level
  38. 38. GUI is a component GUI Service API Mock Impl GUI Test are much faster and robust if they test the GUI in isolation. High detail level.
  39. 39. Requirement Verification Internal Service GUI Internal Service Internal Service Service API Verifying Use Case acceptance criterias. Low level of detail
  40. 40. Continuous Delivery Build Release Deploy Test Build Release Deploy Test Build Release Deploy Test Build Release Deploy Test Build Release Deploy Test Assemble Release Deploy Test Component Pipe 100s of tests run in parallel testing System Pipe 10s Use Case Requirements Verified
  41. 41. Test Driven Development Pre Planning Dev Sys Test Reg Test Pre Planning Dev Sys Test Reg Test Scrummerfall happens because its hard for developers to have something deployable for the testers to test. System testing on something that has not been regression tested is fundamentally flawed. Cost of regression test tends to bloat sprint content. Sprint 2-4 weeks
  42. 42. Test Driven Development Pre Planning Dev Test Automation Reg Test Design Requirement Verification and Tests on multiple levels with the feature. Develop test automation in parallel with feature development. Cross functional pairing to improve quality of Tests, Test Architecture and Feature Testability Pre Planning Reg Test Reg Test Verification Verification
  43. 43. Test Driven Development Who should automate the tests? Test Design - Done by Testers Test Automation - Done by ???? Coding - Done by Developers
  44. 44. Test Driven Development Tests as Code Test Automation IS Code and needs to be treated as first class application. Test Architecture is super important and needs to be worked with in parallel with Application Testability Architecture.
  45. 45. Test Driven Development Tests as Code Internal Service DSL Glue Code SpecificationText or Code based Specification Implementation of Specification Domain Specific Language packaging service testability
  46. 46. Test Driven Development Internal Service DSL Glue Code Specification Give a valid user name and password when login is called then user is authorized boolean authorized = loginService.authorizeUser(“triha”,”cdrockz”); assert(authorized, true); public boolean authorizeUser(String userName, String password){ //Implementation of REST call to internal service return true; }
  47. 47. Test Driven Development Tests as Code Specification & Glue Code IS the Test DSL is testability Test Automation is Code
  48. 48. Test Driven Development Pre Planning Dev Test Automation Reg Test Specifications of Requirement Verification written during Pre-Planning. Specification of Component Tests written during Pre-Planning. Implementation of DSL and Glue Code in parallel with feature development. Implementation of Component Tests in parallel with feature development. Pre Planning Reg Test Reg Test Verification Verification
  49. 49. Test Driven Development Tomorrow Agile Team Member Developers with Testing skills Testers with Development skills WILL BE SUPER HOT AND EARN SUPER NICE $$$$ Will be the only ones who get assignments
  50. 50. Exploratory Testing Automate Everything! Yes! Everything that needs to be done in order to release a system into production. Yet we still need Exploratory testing.
  51. 51. Exploratory Testing Has been used to cover up manual regression testing or worse A way to get away with non reproducible manual regression testing
  52. 52. Exploratory Testing Test Automation ensures we build the AGREED application Exploratory Testing ensures we IMPROVE the application
  53. 53. Exploratory Testing Exploratory Testing should be... ...done without knowledge of requirements. ...in time based sessions with specific focus area. ...outside the release lifecycle of the application.
  54. 54. Benefits Continuous Regression testing gives instant feedback. Continuously deploying to test servers tests deploy mechanism several hundred times per release. Always ready to push new release into UAT We can get customer feedback as soon as value has been added Test Driven Development Everyone is responsible for Quality Sustainable Quality
  55. 55. Tools Used We love to talk about what tools are best and spend ages arguing about it. We need to understand what we must do before we pick the tools. Still which ever tool we pick it will be insufficient and crap.
  56. 56. Tools Used Don't use one tool to rule them all. Cucumber and Fitnesse are ok for system level testing RESTAssured, JUnit or pure Groovy, Ruby or Python scripts are often better for component level. For Load testing one should use load testing tools such as JMeter Continuous Delivery Process should execute tests through interface such as maven or gradle so that it can be Test Tool agnostic.
  57. 57. Areas Not Covered To do Continuous Delivery well we also have to... Reproducible Test Environments Production like Test Environments “Non functional requirements” as functional requirements A/B testing Power of Metrics and lots more...
  58. 58. Good Reading Continuous Delivery by Jez Humble & Dave Farley http://www.amazon.com/dp/0321601912?tag=contindelive-20
  59. 59. Thats it! Feedback & Any questions you forgot to ask? http://continuous-delivery-and-more.blogspot.se

×