Ce diaporama a bien été signalé.
Nous utilisons votre profil LinkedIn et vos données d’activité pour vous proposer des publicités personnalisées et pertinentes. Vous pouvez changer vos préférences de publicités à tout moment.
SELF-HEALING TEST
AUTOMATION 2.0
THE FUTURE
• Problem statement
• How sha works
• Demo
• TAF integration
• POS results
• F...
ANNA CHERNYSHOVA
Lead Software Test Automation Engineer
Testing Competency Center Expert
Email: anna_chernyshova@epam.com
...
PROBLEMS
Unstable automated E2E tests inhibit CI
~40% team effort spent to fix automations
issues, rather than test produc...
PROBLEMS GOALS FEATURES
Unstable automated E2E tests inhibit CI
~40% team effort spent to fix automations
issues, rather t...
PROBLEMS GOALS FEATURES
Unstable automated E2E tests inhibit CI
~40% team effort spent to fix automations
issues, rather t...
PROBLEMS GOALS FEATURES
Unstable automated E2E tests inhibit CI
~40% team effort spent to fix automations
issues, rather t...
PROBLEMS GOALS FEATURES
Unstable automated E2E tests inhibit CI
~40% team effort spent to fix automations
issues, rather t...
PROBLEMS GOALS FEATURES
Unstable automated E2E tests inhibit CI
~40% team effort spent to fix automations
issues, rather t...
What we
have on the
market
Dynamic
locators
PROS & CONS
• Dynamic DOM state updates
• Easy to write and support tests
• Tests are stable
• Cross-browser, CI, Jira int...
Epam Self-healing
Applicable at any stage of project development
Minimal dependency from the TAF
Replace static locators w...
Self-healing
library…
Gives ability to find controls (new locator) for updated
WEB pages
Use kinda ML algorithms for Web p...
HOW SHA WORKS
LCS ALGORITHM
Longest common subsequence
problem of finding the longest subsequence common to all sequences in a set of se...
LCS ALGORITHM MODIFICATION
Longest common subsequence with weight
Added extra weight for tag, Id, class, value, other attr...
Self-healing selenium (jar)
Tree-comparing(jar)
Includes Tree-comparing dependency
Implements Selenium WebDriver
Overwrite...
Self-healing selenium (jar) Tree-comparing(jar)
Reference elements path
storage
Test
Driver.findElement(PageAwareBy(…))
We...
Self-healing selenium (jar) Tree-comparing(jar)
Reference elements path
storage
Test
Driver.findElement(PageAwareBy(…))
We...
Self-healing selenium (jar) Tree-comparing(jar)
Reference elements path
storage
Test
Driver.findElement(PageAwareBy(…))
We...
Self-healing selenium (jar) Tree-comparing(jar)
Reference elements path
storage
Test
Driver.findElement(PageAwareBy(…))
We...
Self-healing selenium (jar) Tree-comparing(jar)
Reference elements path
storage
Test
Driver.findElement(PageAwareBy(…))
We...
Self-healing selenium (jar) Tree-comparing(jar)
Reference elements path
storage
Test
Driver.findElement(PageAwareBy(…))
We...
Self-healing selenium (jar) Tree-comparing(jar)
Reference elements path
storage
Test
Driver.findElement(PageAwareBy(…))
We...
SelfHealingDriver
findElement
public WebElement findElement(By by) {
if (by instanceof PageAwareBy) {
try {
Trying to find...
PageAwareBy
extends By
PageAwareBy(String pageName, By by, SelfHealingEngine
engine) {
this.pageName = pageName;
this.by =...
Functions supported
• By() -> PageAwareBy()
• @FindBy -> @PageAwareFindBy
• WebDriver -> SelfHealingDriver
• Iframe suppor...
Html report
Intellij Idea plugin to make code updates
HTML REPORT
Intellij Idea plugin to make code updates
PLUGIN
HTML REPORT
Update
request
{
filename: MainPageWithFindBy
lineNumber: 49
...
Intellij Idea plugin to make code updates
TAF CODE BASE
PLUGIN
HTML REPORT
Find locator and update
Update
request
{
filena...
TAF INTEGRATION
1. Declare custom EngineConfig with the path to store new locators. For example 'shaselenium'
EngineConfig engineConfig = ...
RESULTS
Projects for pilot
TAF based on Java +
Selenium
(WebDriver)
Continuous UI
updates
UI generated by CMS
Projects for pilot
TAF based on Java +
Selenium
(WebDriver)
Continuous UI
updates
UI generated by CMS
>20 replies
6 Web pr...
100% tests healed
Healed localization testing
Issues
Test execution time
increase
Can’t heal single element
from the list
Probability of
false/positive results
Code not...
FUTURE PLANS
Future plans
Idea plugin Mobile support
Idea Plugin
✅ @PageAwareFindBy
❌ PageAwareBy(..)
❌ Local declaring
Thanks to
Oleh Khailenko
Lead Software Engineer
Kostiantyn Morozkin
Senior Software Engineer
Nikolai Kobzev
Senior Softwar...
Email: anna_chernyshova@epam.com
FB: anna.chernyshova.79
QA Fest 2019. Анна Чернышова. Self-healing test automation 2.0. The Future
Prochain SlideShare
Chargement dans…5
×

QA Fest 2019. Анна Чернышова. Self-healing test automation 2.0. The Future

441 vues

Publié le

Мы уже разговаривали о self-healing автоматизации, как она работает, какие есть подходы, чем они хороши, плохи и о новом инструменте, который мы разрабатываем в EPAM. Наш продукт завершает стадию POC и настало время поделиться результатами и понять, насколько self-healing автоматизация поможет вашим тестам стать стабильнее? Или наоборот, навредит?... Приходи и узнаешь!

Publié dans : Formation
  • Login to see the comments

QA Fest 2019. Анна Чернышова. Self-healing test automation 2.0. The Future

  1. 1. SELF-HEALING TEST AUTOMATION 2.0 THE FUTURE • Problem statement • How sha works • Demo • TAF integration • POS results • Future plans
  2. 2. ANNA CHERNYSHOVA Lead Software Test Automation Engineer Testing Competency Center Expert Email: anna_chernyshova@epam.com FB: anna.chernyshova.79
  3. 3. PROBLEMS Unstable automated E2E tests inhibit CI ~40% team effort spent to fix automations issues, rather than test product and add new tests Web application updates constantly every sprint, which leads to locator change PROBLEM STATEMENT
  4. 4. PROBLEMS GOALS FEATURES Unstable automated E2E tests inhibit CI ~40% team effort spent to fix automations issues, rather than test product and add new tests Web application updates constantly every sprint, which leads to locator change Improve E2E automated test stability PROBLEM STATEMENT
  5. 5. PROBLEMS GOALS FEATURES Unstable automated E2E tests inhibit CI ~40% team effort spent to fix automations issues, rather than test product and add new tests Web application updates constantly every sprint, which leads to locator change Improve E2E automated test stability Reduce amount of test cases failed due to non-product issue PROBLEM STATEMENT
  6. 6. PROBLEMS GOALS FEATURES Unstable automated E2E tests inhibit CI ~40% team effort spent to fix automations issues, rather than test product and add new tests Web application updates constantly every sprint, which leads to locator change Improve E2E automated test stability Reduce amount of test cases failed due to non-product issue Reduces Tests Maintenance efforts PROBLEM STATEMENT
  7. 7. PROBLEMS GOALS FEATURES Unstable automated E2E tests inhibit CI ~40% team effort spent to fix automations issues, rather than test product and add new tests Web application updates constantly every sprint, which leads to locator change Improve E2E automated test stability Reduce amount of test cases failed due to non-product issue Reduces Tests Maintenance efforts More time to cover new functionality with tests PROBLEM STATEMENT
  8. 8. PROBLEMS GOALS FEATURES Unstable automated E2E tests inhibit CI ~40% team effort spent to fix automations issues, rather than test product and add new tests Web application updates constantly every sprint, which leads to locator change Improve E2E automated test stability Reduce amount of test cases failed due to non-product issue Reduces Tests Maintenance efforts More time to cover new functionality with tests Shipping better products faster PROBLEM STATEMENT
  9. 9. What we have on the market
  10. 10. Dynamic locators
  11. 11. PROS & CONS • Dynamic DOM state updates • Easy to write and support tests • Tests are stable • Cross-browser, CI, Jira integrations from the box • Only for new projects • Strong dependency from tool ecosystem • May be not secured • May lose tests when project finished
  12. 12. Epam Self-healing Applicable at any stage of project development Minimal dependency from the TAF Replace static locators with dynamic ones
  13. 13. Self-healing library… Gives ability to find controls (new locator) for updated WEB pages Use kinda ML algorithms for Web page changes identification Is a standalone Java jar connected to test cases code base Helps >2x times reduced test fails because of updated UI Provides Html report which handles locators analysis Provides Intellij Idea plugin to make code updates with new locator values
  14. 14. HOW SHA WORKS
  15. 15. LCS ALGORITHM Longest common subsequence problem of finding the longest subsequence common to all sequences in a set of sequences *widely used by revision control systems such as Git
  16. 16. LCS ALGORITHM MODIFICATION Longest common subsequence with weight Added extra weight for tag, Id, class, value, other attributes <button id=btn-1> <button id=btn-1>
  17. 17. Self-healing selenium (jar) Tree-comparing(jar) Includes Tree-comparing dependency Implements Selenium WebDriver Overwrites findElement() method Catch NoSuchElementException Activates LCS algorithm in Tree-comparing library Save reference element path to storage Get reference element path from storage Get current DOM state Search in current state for the best subsequence Generate new CSS locator JAR LIBRARIES
  18. 18. Self-healing selenium (jar) Tree-comparing(jar) Reference elements path storage Test Driver.findElement(PageAwareBy(…)) Web Page Target element Find Element Test Automation Framework Element Found
  19. 19. Self-healing selenium (jar) Tree-comparing(jar) Reference elements path storage Test Driver.findElement(PageAwareBy(…)) Web Page Target element Find Element Test Automation Framework Element Found Save successful path Old locator
  20. 20. Self-healing selenium (jar) Tree-comparing(jar) Reference elements path storage Test Driver.findElement(PageAwareBy(…)) Web Page Target element Find Element Element Not Found Test Automation Framework
  21. 21. Self-healing selenium (jar) Tree-comparing(jar) Reference elements path storage Test Driver.findElement(PageAwareBy(…)) Web Page Target element Find Element Element Not Found Test Automation Framework Old locator Successful path Page state
  22. 22. Self-healing selenium (jar) Tree-comparing(jar) Reference elements path storage Test Driver.findElement(PageAwareBy(…)) Web Page Target element Find Element Element Not Found Test Automation Framework Old locator New locator Successful path Page state New locator
  23. 23. Self-healing selenium (jar) Tree-comparing(jar) Reference elements path storage Test Driver.findElement(PageAwareBy(…)) Web Page Target element Find Element Element Not Found Test Automation Framework Old locator New locator Successful path Page state New locator Find New Element
  24. 24. Self-healing selenium (jar) Tree-comparing(jar) Reference elements path storage Test Driver.findElement(PageAwareBy(…)) Web Page Target element Find Element Element Not Found Test Automation Framework Element Found Save successful path Old locator New locator Successful path Page state New locator Find New Element
  25. 25. SelfHealingDriver findElement public WebElement findElement(By by) { if (by instanceof PageAwareBy) { try { Trying to find the element and save its path if success } catch (NoSuchElementException var5) { Find previous reference element path in storage Generate new best matches locator return healedLocator; } } else { return delegate.findElement(by); } }
  26. 26. PageAwareBy extends By PageAwareBy(String pageName, By by, SelfHealingEngine engine) { this.pageName = pageName; this.by = by; this.engine = engine; } PageObject Page Locator New locator processor
  27. 27. Functions supported • By() -> PageAwareBy() • @FindBy -> @PageAwareFindBy • WebDriver -> SelfHealingDriver • Iframe support • Actions support • Remote test run • Parallel test run • Works with Selenium wrappers like Selenide
  28. 28. Html report
  29. 29. Intellij Idea plugin to make code updates HTML REPORT
  30. 30. Intellij Idea plugin to make code updates PLUGIN HTML REPORT Update request { filename: MainPageWithFindBy lineNumber: 49 failedLocatorValue://input[@name='EMAIL’] healedLocatorValue: input.t186__input.js-tilda-rule.t-input }
  31. 31. Intellij Idea plugin to make code updates TAF CODE BASE PLUGIN HTML REPORT Find locator and update Update request { filename: MainPageWithFindBy lineNumber: 49 failedLocatorValue://input[@name='EMAIL’] healedLocatorValue: input.t186__input.js-tilda-rule.t-input }
  32. 32. TAF INTEGRATION
  33. 33. 1. Declare custom EngineConfig with the path to store new locators. For example 'shaselenium' EngineConfig engineConfig = EngineConfig .custom() .setStorage(new FileSystemPathStorage(Paths.get("sha", "selenium"))) .build(); 2. Init driver instance of SelfHealingDriver SelfHealingDriver driver = new SelfHealingDriver(new ChromeDriver(), engineConfig); 3. Locate elements By buttonBy = PageAwareBy.by("MainPage", By.id(testButtonId)); Or @PageAwareFindBy(page="MainPage", findBy = @FindBy(id = "markup-generation-button")) WebElement buttonBy; 4. Interact with elements as usual driver.findElement(buttonBy).click(); Important! Do not delete data from the folder where files with new locators are stored. They are used to perform self-healing in next automation runs Important! "MainPage" is the name of the page to which the WebElement belongs
  34. 34. RESULTS
  35. 35. Projects for pilot TAF based on Java + Selenium (WebDriver) Continuous UI updates UI generated by CMS
  36. 36. Projects for pilot TAF based on Java + Selenium (WebDriver) Continuous UI updates UI generated by CMS >20 replies 6 Web projects for pilot 3 projects are still in focus
  37. 37. 100% tests healed Healed localization testing
  38. 38. Issues Test execution time increase Can’t heal single element from the list Probability of false/positive results Code not updated after healing
  39. 39. FUTURE PLANS
  40. 40. Future plans Idea plugin Mobile support
  41. 41. Idea Plugin ✅ @PageAwareFindBy ❌ PageAwareBy(..) ❌ Local declaring
  42. 42. Thanks to Oleh Khailenko Lead Software Engineer Kostiantyn Morozkin Senior Software Engineer Nikolai Kobzev Senior Software Engineer Vladimir Moshkin SoftwareTest Automation Engineer
  43. 43. Email: anna_chernyshova@epam.com FB: anna.chernyshova.79

×