Getting Started with Visual Testing
•
0 j'aime•343 vues
Everyone wants to make quick releases, but the look-and-feel UX validation is a manual, slow, and error-prone activity. Any UX-related issues propping up cause huge brand-value and revenue loss, may lead to social-trolling, and, even worse, dilute your user base. This is an area where AI & ML can help. In this hands-on workshop, using examples, we will explore: the importance of automated visual validation how an AI-powered tool, Applitools Visual AI, can solve this problem. Integrate Applitools Visual AI in your Selenium-Java automation and learn by practice: The different AI algorithms various Applitools capabilities and features scale your automation using the Applitools Ultrafast Grid.
Recommandé
Contenu connexe
Tendances
Tendances (20)
Similaire à Getting Started with Visual Testing
Similaire à Getting Started with Visual Testing (20)
Plus de Applitools
Plus de Applitools (20)
Dernier
Dernier (20)
Getting Started with Visual Testing
- 1. @BagmarAnand Getting started with Visual Testing Anand Bagmar Software Quality Evangelist
- 4. @BagmarAnand Bugs still escape Traditional automated testing frameworks are not built for modern apps 4
- 6. @BagmarAnand Bugs escape because our approach to testing is incorrect @BagmarAnand
- 7. @BagmarAnand 7 @BagmarAnand Spot the Difference!
- 8. @BagmarAnand 8 @BagmarAnand Spot the Difference!
- 9. @BagmarAnand 9 @BagmarAnand Spot the Difference!
- 10. @BagmarAnand 10 @BagmarAnand Spot the Difference!
- 11. @BagmarAnand @BagmarAnand Mundane Error prone Tedious Running against Time Approach to Testing is Incorrect
- 12. @BagmarAnand But, we are testing real software products @BagmarAnand
- 13. @BagmarAnand 13 @BagmarAnand Text, Images, Responsive-pages, Form-factors, User-experience Product has Context!
- 26. @BagmarAnand What does this mean?
- 27. @BagmarAnand Add Eyes to the Test Automation Pyramid 28
- 28. @BagmarAnand What is missing here?
- 29. @BagmarAnand How do you ensure what was working before, is working now as well!
- 30. @BagmarAnand UX / Visual Testing The Missing Piece
- 31. @BagmarAnand Add Visual Automation in Test Pyramid 32
- 32. @BagmarAnand Visual Testing How do you do it?
- 33. @BagmarAnand • Mostly done manually done • Functional Automation can help Visual Testing 34
- 34. @BagmarAnand • Business / Revenue loss • Brand / Credibility loss • You lose your users! If Visual Testing is NOT done … 35
- 35. @BagmarAnand • Mostly done manually done • tedious • very error prone • impossible to scale and repeat – especially in days of CD! • Functional Automation may help – a little Challenges of Visual Testing 36
- 37. @BagmarAnand • Create baselines by taking screenshots of expected UI • Compare screenshots • Whole page • Snippets • Update baselines as product evolves / changes How does it work? 38
- 38. @BagmarAnand Challenges of Automating Visual Testing
- 39. @BagmarAnand ● Creating the baseline ○ For each browser type / device / viewport size ○ Contextual (not mock / wireframe based) ● Maintaining / Updating the baseline ○ Product evolves, so do your baselines! ● Accurate comparison ○ Dynamic and contextual Data ○ Responsive Design The 3 Challenges of Visual Testing 40
- 40. @BagmarAnand AI in Visual Test Automation
- 41. @BagmarAnand A Simple One Page, Single Browser Example 42
- 42. @BagmarAnand 18 Lines of Code 21 Locators and Labels 60 Minutes Navigation Button Click Validation First, Let’s Provide Initial Coverage 43
- 43. @BagmarAnand NEW Real Bug Escapes Text Changes Caught New Feature Missed Visual Bug Escapes Broken Locators 2 Late Stage Bugs 2 Changes Caught 2 False Positives Baseline New Candidate New Feature Missed Next, We Test a New Build 44
- 44. @BagmarAnand Test Results Maintenance Action Time Functional Bug Write Another Validation 15 mins Visual Bug Try Writing New Validations, Then Give Up 15 mins Placeholder Text Differences Update Existing Validations 10 mins New Terms & Conditions Feature Write Another Validation 5 mins Changed Locators Update Locators 15 mins Total Time Spent 60 mins NEW Finally, We Have to Maintain our Tests 45
- 45. @BagmarAnand Plus Those 21 Locators and Labels That Might Change Without any Notice! Build & Maintain: 120 Minutes Bug Detection: 40% False Positive Rate: 40% Build and Maintenance Summary 46
- 46. @BagmarAnand Login Page to Test Remember Our Simple Example? Let’s Try Again. 47
- 47. @BagmarAnand 2 Minutes to Create Vs. 60 Minutes 1 Locator Vs. 21 Locators and Labels Let’s Provide Initial Coverage Using Visual AI 48
- 48. @BagmarAnand Found Functional Bug NEW Found Text Changes Found Visual Bug Found New Feature Original New Build No Broken Locators All Bugs Found Text Changes and New Feature Detected 0 Broken Locators Visual AI Handles the New Build 49
- 49. @BagmarAnand Test Results Visual AI Maintenance Action Legacy Visual AI Functional Bug None; Found Automatically 15 mins 0 mins Visual Bug None; Found Automatically 15 mins 0 mins Placeholder Text Differences “Thumbs Up” for Each Difference 10 mins 40 secs New Terms & Conditions Feature “Thumbs Up” for Difference 5 mins 20 secs Changed Locators Locators No Longer Used 15 mins 0 mins Total Time Spent 60 mins 1 min NEW Visual AI Streamlines Test Maintenance 50
- 50. @BagmarAnand Legacy Test Automation Visual AI 40x Faster 21x More Stable 2.5x Increase in Bug Detection Eliminate False Positives! Build & Maintain: 3 Minutes Locators: 1 Build & Maintain: 120 Minutes Locators and Labels: 21 Bug Detection: 100% Bug Detection: 40% False Positive Rate: 0% False Positive Rate: 40% Visual AI Delivers Results 51
- 51. @BagmarAnand A single assertion for complete functional coverage Bonus: validates the UI Does not break when the UI changes No coding skills required to maintain baselines Seamless scaling 60+ SDKs to integrate with your Automated Functional Tests Advantages of Visual Assertions 52
- 53. @BagmarAnand Help teams release perfect apps much faster at a reduced cost. Inventers of Visual AI that replicates the human eye and brain, looks at every screen and page in seconds, and spots app regressions that truly matter. Applitools Visual AI 54
- 54. @BagmarAnand Visual AI Industry’s only AI powered computer vision replicates the human eye & brain. SDKs & Integrations SDKs to augment all modern test frameworks, and integrations with popular source control, CI, and defect tracking systems. Perfection Across All Screens Ultrafast Grid ensures visual perfection across all browsers, screens, and viewports. Ultrafast Grid Architecture >> Speed of CI/CD Execute tests in seconds vs minutes for faster builds and on-time delivery. Ultimate Efficiency Create tests faster & with fewer brittle locators and labels. Increase Test Coverage Open-ended, AI powered assertions, reduce code, maintenance while increasing test coverage. Applitools Cloud Visual AI Platform Adding Applitools Visual AI to your existing test automation framework allows quality teams to release perfect apps faster and as a reduced cost. One Intelligent Platform 55
- 55. @BagmarAnand 56 Applitools - Ultrafast Grid
- 56. @BagmarAnand Applitools Native Mobile Grid 57
- 58. @BagmarAnand 1. Prequisites 2. Run test (Selenium-Java or Appium-Java) 3. Add “eyes” to your automation using Applitools Visual AI 4. Scale your test execution using Applitools Ultrafast Grid Workshop Activity 59
- 59. @BagmarAnand • Web • Clone - http://bit.ly/getting-started-with-visualtesting • Open project in IDE • Refresh dependencies • Android • Clone - http://bit.ly/getting-started-with-mobile-visualtesting • Open project in IDE • Refresh dependencies • Setup Android SDK, Emulator/Device 1. Prerequisites 60
- 60. @BagmarAnand • Web • http://bit.ly/getting-started-with-visualtesting • Run test: mvn test –Dtest=HelloWorldTest • Android • http://bit.ly/getting-started-with-mobile-visualtesting • Connect your Android device / start emulator • Run test: mvn test –Dtest=CalcTest 2. Run test (Selenium/Appium-Java) 61
- 61. @BagmarAnand • Sign up for free Applitools account - http://bit.ly/ApplitoolsSignup • Add APPLITOOLS_API_KEY to your environment variables • Web • Run test: mvn test –Dtest=EyesTest • Android • Run test: mvn test –Dtest=EyesTest 3. Add “eyes” to your tests 62
- 62. @BagmarAnand • Dashboard • Batch • Visual AI validations • Visual AI algorithms • Fluent APIs • Configuration • Branch • Saving tests • Screenshots • Custom properties Applitools configurations 63
- 63. @BagmarAnand Scaling your test execution
- 64. @BagmarAnand 65 Applitools - Ultrafast Grid
- 65. @BagmarAnand • Applitools Ultrafast Cloud • https://applitools.com/platform/ • Web using Applitools Ultrafast Grid • https://applitools.com/platform/ultrafast-grid/ • Run test: mvn test –Dtest=UFGTest • Android using Applitools Native Mobile Grid - https://applitools.com/platform/native-mobile-grid/ 4. Scale using Applitools Ultrafast Cloud 66
- 66. @BagmarAnand • Runner • Browser / Device configurations Applitools UFG configurations 67
- 67. @BagmarAnand Challenges of Automating Visual Testing
- 68. @BagmarAnand Applitools Visual AI Solution for the challenges in Visual Testing ● Creating the baseline ○ For each browser type / device / viewport size ○ Contextual (not mock / wireframe based) ● Maintaining / Updating the baseline ○ Product evolves, so do your baselines! ● Accurate comparison ○ Dynamic and contextual Data ○ Responsive Design Automatic creation of baselines Simple to update the baselines AI Algorithms 70
- 69. @BagmarAnand Visual Test Automation needs a Strategy!
- 70. @BagmarAnand Test Pyramid … includes UX & NFRs 72
- 72. @BagmarAnand • The Practical Test Pyramid - https://martinfowler.com/articles/practical-test- pyramid.html • Testing Pyramids & Ice-Cream Cones - https://watirmelon.blog/testing- pyramids/ • Applitools – www.applitools.com • Sample code: • http://bit.ly/getting-started-with-visualtesting • https://applitools.com/platform/native-mobile-grid/ References 74