Ce diaporama a bien été signalé.
Nous utilisons votre profil LinkedIn et vos données d’activité pour vous proposer des publicités personnalisées et pertinentes. Vous pouvez changer vos préférences de publicités à tout moment.

Test Automation for Mobile Devices - DrupalCon Vienna 2017

301 vues

Publié le

Test Automation for Drupal 8 in Mobile Devices and Tablets by Anastasios Daskalopoulos (DrupalCon Vienna 2017)

Publié dans : Technologie
  • Soyez le premier à commenter

  • Soyez le premier à aimer ceci

Test Automation for Mobile Devices - DrupalCon Vienna 2017

  1. 1. Test Automa+on For Mobile Devices
  2. 2. Test Automa+on for Drupal 8 in Mobile Devices and Tablets Anastasios Daskalopoulos @AnDasko
  3. 3. Test Automa+on For Mobile Devices •  Me, Briefly – Originally trained as a Java Developer – SoBware Tes+ng Since 2003 •  Nokia: 2003 – 2010 –  Test Manager at Nokia.com – first experience with CMS – Test Automa+on Experience since 2006 •  QuickTest Professional - Nokia •  Robot Framework •  Nightwatchjs & Phantomjs •  Codecep+on •  Appium
  4. 4. Test Automa+on For Mobile Devices •  Brief History of Web CERN, Geneva, early to mid 1980s: physicists from around the world needed to share informa+on - but they didn't have common machines. TCP/IP protocols were installed on UNIX machines at CERN, thus making it it's own Internet site. This early infrastructure became the founda+on of the early Web.
  5. 5. Test Automa+on For Mobile Devices •  Unexpected Element To Web – Mobile Devices •  The expecta+on for the next 20 years would be that interconnected networks of devices would func+on on large computers and later on desktop computers. – Late 1990s – early 2000s •  The web starts to appear on mobile devices •  That’s when the first problems appear.
  6. 6. Test Automa+on For Mobile Devices •  History of Mobile Devices    •  Early History    –  1910s: public-switched telephone networks became common      –  1920s: radio communica+on became popular      –  1940s: portable radio-telephones became common (walkie-talkies)      –  1990s: cellular telecommunica+on became popular – developed by Mar+n Cooper
  7. 7. Test Automa+on For Mobile Devices •  -Ironic True Quotes About The Development of Technology – "I think there is a world market for maybe five computers." Thomas Watson, President of IBM, 1943 –  "There is no reason anyone would want a computer in their home." Ken Olsen, Digital Equipment Corpora+on Founder, 1977.
  8. 8. Test Automa+on For Mobile Devices •  "Cellular phones will absolutely not replace local wire systems." Mar+n Cooper, Inventor Of The Cell Phone, 1981. – Mar+n Cooper had to take a very cau+ous approach to development •  Mobile telecommunica+ons was just as much a poli+cal challenge as a engineering challenge •  Cooper had to convince on many occasions doubmul Motorola execu+ves about the usefulness of mobile phones
  9. 9. Test Automa+on For Mobile Devices •  "I predict the Internet will soon go spectacularly supernova and in 1996 catastrophically collapse." – Robert Metcalfe, founder of 3Com, Ethernet Inventor, Infoworld Columnist, 1995. •  Old paradigms die hard – the old way of thinking s+ll exists even to those at the forefront of technology
  10. 10. Test Automa+on For Mobile Devices •  "Apple is already dead." – Nathan Myhrvold, MicrosoB CTO, 1997 •  “There’s no chance that the iPhone is going to get any significant market share.” – Steve Ballmer, MicrosoB CEO, 2007. •  Business wishful thinking is also a big impediment to technology
  11. 11. Test Automa+on For Mobile Devices •  “In five years, I don’t think there will be a reason to have a tablet anymore…tablets themselves are not a good business model.” – Thorsten Heins, Blackberry CEO, 2013 – Tablet shipments fell by 12.5% last year.
  12. 12. Test Automa+on For Mobile Devices •  "Why would anyone want to look at a web site from a cell phone?" Nokia Manager, 2004 – Old paradigms die hard: even to managers at one of the biggest technology companies in the world, the paradigm of a mobile device primarily used as a telephone oral communica+on instrument never stopped
  13. 13. Test Automa+on For Mobile Devices •  The point of the previous slides: –  The point of the above quotes is that even powerful technology chiefs have for long denigrated computer, web and mobile technology. The best thing to do at this point is to take the lead in mobile development and quality assurance •  What the above-men+oned technology chiefs got wrong: –  a workable, easy to use mobile web
  14. 14. Test Automa+on For Mobile Devices •  The Cost of Buggy SoBware Is Much Greater Than We Think •  The Na+onal Ins+tute of Standards and Technology Has Calculated That SoBware Bugs Cost $59.5 Billion Annually Just in the U.S.! •  Bugs Are Inevitable In Every SoBware Development Project – Losses From Buggy SoBware Will Be Reduced By Con+nuous Tes+ng As Part of the Development Process
  15. 15. Test Automa+on For Mobile Devices
  16. 16. Test Automa+on For Mobile Devices •  The Web and the Mobile Device: When & Where Did The Above 2 Intersect? – A brief history of the mobile web •  The first widely available mobile web device: Nokia 9110
  17. 17. Test Automa+on For Mobile Devices
  18. 18. Test Automa+on For Mobile Devices •  Problems with the early mobile web: – 1) Custom code - developers had to write special code specifically for mobile devices; this limited mobile web development since most companies did not want to invest in training/hiring mobile- specific developers – 2) Low-resolu+on imagery and text-based approach to web. Also, slow loading +mes because of low compu+ng power
  19. 19. Test Automa+on For Mobile Devices •  Problems solved! – iOS & Android offered mobile browsers that could read HTML, CSS, PHP & Java. Instead of developing for a stripped-down web, iPhone & Android invested in transla+ng their web browsers to web-friendly opera+ng systems – Large screen yet light and highly portable, easy-to- use device with an and easy-to-read color screen
  20. 20. Test Automa+on For Mobile Devices •  Why Do People Web Surf On Mobile –  60% of digital media +me spent on mobile devices •  KEEP THIS IN MIND – MOST ONLINE TIME IS NOW SPENT ON MOBILE DEVICES! •  A 2012 Google survey found that: –  74% of visitors were more likely to return to mobile friendly websites –  61% were likely to leave if a site wasn’t mobile friendly –  67% were more likely to buy at a mobile friendly website
  21. 21. Test Automa+on For Mobile Devices •  Why Are People Online With Mobile Devices? – Easy and immediate web access – Discrete web surfing with small, easily-concealed and fur+ve web browsers – People are more mobile and therefore want to use mobile browsing – The habit of web mobility has started and will keep growing •  Reflexive, ins+nct ac+vity
  22. 22. Test Automa+on For Mobile Devices •  Now to actual tes+ng! –  KEEP IN MIND: test automa+on is one part of tes+ng •  Test Automa+on is oBen just a sales device –  Too oBen this simply results in the automa+on of buggy soBware •  Test Automa+on alone will not solve your Quality Assurance problems –  KEEP IN MIND EVEN MORE: mobile test automa+on is s+ll at an early stage: difficult to use with a lot that will go wrong
  23. 23. Test Automa+on For Mobile Devices •  ALSO REMEMBER THIS ALWAYS: – Plan Tes+ng/QA at the same +me you are planning Development – Stop leaving test planning, design, and even execu+on to the end – Bug finding is good! •  Celebrate finding bugs throughout your system – Learn to spot anomalies everywhere in daily life •  Is there something unexpected here? – Avoid Confirma+on Bias •  We don’t want bugs in our soBware, therefore, we don’t see them
  24. 24. Test Automa+on For Mobile Devices •  2 Different Approaches to Mobile Web Tes+ng – Decide which applies to your project – 1) Web-based mobile apps – 2) Na+ve mobile apps •  Drupal 8 can exist on both web-based and na+ve mobile apps
  25. 25. Test Automa+on For Mobile Devices •  Drupal on Mobile Devices – How does Drupal come into mobile devices? – Drupal 7 – fairly recent ac+vity with mobile devices – Drupal 8: expansion into mobile development realm and builds on the accomplishments of Drupal 7 – hyps://www.drupal.org/docs/8/mobile
  26. 26. Test Automa+on For Mobile Devices •  Drupal Development Approach Debate – Desktop development first and then “adapt” the project to mobile environment? – Should Drupal projects have a “mobile first” development approach?
  27. 27. Test Automa+on For Mobile Devices •  Mobile-First Development? •  "If I were to start Drupal from scratch today, I'd build it for mobile experiences first, and desktop experience second." – Dries Buytaert, 2011 – Remember that your site Users will look at your pages in mobile devices.
  28. 28. Test Automa+on For Mobile Devices •  So, The Big Ques+ons To Ask Before Mobile QA Begins – Is it a mobile-specific website or a na+ve mobile app? – If “mobile-specific” is the answer, then ask: •  “Is there different markup and layout than in the desktop site?” •  If the answer is yes, then test to validate the mobile markup and layout first and then the rest of the site for func+onality and content
  29. 29. Test Automa+on For Mobile Devices •  If the answer is no, then the website is basically the same that appears in desktop but viewed inside mobile devices. •  Check the site for: – Responsiveness – Layout – Feature Func+onality (do all features do what they are supposed to do) – Content (Will the User-entered informa+on appear as expected?)
  30. 30. Test Automa+on For Mobile Devices •  Na+ve Mobile App Tes+ng – According to the Drupal 8 website on na+ve mobile applica+on development, Drupal can be used as the backend for mobile applica+on development – As the website states: •  "Drupal can contain your content, business logic, user management, and search func+onality, and your app can be a front end that talks to Drupal using the Services module.” – Robot Framework has both a Drupal library and libraries to communicate and test a mobile backend
  31. 31. Test Automa+on For Mobile Devices •  Na+ve Mobile App Tes+ng – So in this instance, test the backend for storage & data valida+on – Run numerous, repeated and varied integra+on tests to verify the communica+on of the na+ve mobile app features with the Drupal 8 backend •  Test for these –  Does the na+ve mobile app communicate with the Drupal Backend? –  Are the expected files there? –  Are the files in the expected format? –  Do the files have the expected data?
  32. 32. Test Automa+on For Mobile Devices •  Tes+ng Mobile Apps •  Star+ng the mobile test process: – Emulators vs. Devices – Emulators •  Advantage –  Emulators allow you to debug flows as you test the app »  Good for early itera+ons and ini+al test rounds »  Bad - will not test realis+c client usage. •  Disadvantage –  No End User will go to your site and use your app on an emulator!
  33. 33. Test Automa+on For Mobile Devices •  Commonly Used Emulators – iOS: •  xCode – Android: •  Android SDK •  ABer the ini+al tests on emulators, tes+ng should con+nue on devices
  34. 34. Test Automa+on For Mobile Devices •  THE CHALLENGE OF SOFTWARE TESTING – What is a bug? •  Any unexpected or undesired result –  What is soMware tesOng? – The ac+ve pursuit and repor+ng of soBware errors that lead to unexpected results •  This is an ac+ve and crea+ve process – Ac+ve, formal and recorded explora+on to get informa+on about specific soBware under test
  35. 35. Test Automa+on For Mobile Devices •  Note on Tes+ng: SoBware Bugs vs. User- Expecta+on Bugs – SoBware Bug: •  Error à Fault à Failure – Error: a human ac+on that produces an incorrect result – Fault: a manifesta+on of an error in soBware, also known as Defect, Bug or Issue – Failure: a devia+on of the soBware from its expected delivery or service
  36. 36. Test Automa+on For Mobile Devices •  User-Expecta+on Bugs –  Not caused by an error in producing the soBware itself, but what appears in the web site that is contrary to what the End User would expect to see •  Examples: –  typos –  text fields that overlap text or other features –  app features that run out of the frame –  These are not minor or cosme+c problems: wrong- appearing site features adversely affect the User’s experience and the Site Owner’s credibility •  NOTE: 40% of customer complaints are for what Developers would label as “Minor” issues
  37. 37. Test Automa+on For Mobile Devices •  The 2 Jobs Of Quality Assurance – Valida+on: Is this the Right System? •  Is this what the customer wants? – Verifica+on: Does the System Work Right? •  Does the site work correctly?
  38. 38. Test Automa+on for Mobile Devices •  Bug Severity Status and Triage •  Is this a problem?
  39. 39. Test Automa+on For Mobile Devices •  This is oBen test automa+on
  40. 40. Test Automa+on for Mobile Devices •  Test Automa+on is a useful tool but oBen wrongly used – There are beyer ways to use Test Automa+on •  Keep in mind that Test Automa+on does not itself find bugs – The Tester’s crea+vity and coding skills finds the bugs and not any specific tool
  41. 41. Test Automa+on For Mobile Devices •  Is there a bug here?
  42. 42. Test Automa+on For Mobile Devices •  Test Automa+on for the Mobile Web – REALITY: Mobile Test Automa+on Is Just Beginning •  Developing mobile tests will be slow and difficult •  Mobile Test Automa+on Does Not Differ From Any Other Test Automa+on •  Test Automa+on Should Be Regarded As More A Development Project Than A Tes+ng/QA Project
  43. 43. Test Automa+on for Mobile Devices •  Test AutomaOon RealiOes 1.  The hardest and most important part of test automa+on is not the crea+on but the maintenance of the scripts 1.  Test Automa+on requires constant supervision of executed tests and their near constant maintenance and revision 1.  The Best Test AutomaOon Engineer is the one who can keep their tests execuOng and providing results and feedback over Ome 2.  It is not possible to “just automate” manual tests 3.  No one can just “show you” in an hour how to automate tests
  44. 44. Test Automa+on for Mobile Devices •  Reitera+on of Points – Test Automa+on is a soBware development project whose aim is to provide results with relevant script execu+on about the con+nuing stability of soBware features over +me •  No test automa+on tool will succeed without good principles of automa+on
  45. 45. Test Automa+on For Mobile Devices •  Test Automa+on Philosophy and Principles – Test Automa+on is not a panacea and by itself will not solve your Quality Assurance problems •  A common bad prac+ce is to “automate” tests without any plan or overall philosophy –  This results in the automa+on of buggy soBware – What you want to do is to create tests that will find bugs and inform you where they are and how they occurred! •  How do we do this?
  46. 46. Test Automa+on For Mobile Devices •  First Thing – – HAND OVER YOUR APP FOR TESTING! •  YOUR SOFTWARE HAS BUGS! –  This is not an insult – any crea+on that has features that integrate with other features will at some point have some kind of unexpected result •  Independent Tes+ng is very important in catching bugs –  Developers see in the soBware what they expect to see •  This does not mean “over-the-wall” tes+ng philosophy –  This means DevOps and Con+nuous Integra+on approaches
  47. 47. Test Automa+on For Mobile Devices – Think DevOps and Con+nuous Integra+on •  Individual features are tested in isola+on while others features are in development •  A regression test for already-created features is run directly aBer the build process – Therefore, we have a constant state of regression tests for each already-created feature being run while newer features are being built within the sprint – Contrast this with the end-of-project regression sets
  48. 48. Test Automa+on For Mobile Devices •  How Do We Begin? – Start automa+ng the most stable features – As tes+ng and bug fixing con+nue in each itera+on, the more stable areas of the app creep farther and farther up the triangle Less stable More stable
  49. 49. Test Automa+on For Mobile Devices •  Things to keep in mind always before we con+nue: – Find bugs first! •  I don’t think any tool or process will ever replace organized, structured, recorded test explora+on –  ABer developing the feature, someone, preferably an independent and experienced QA analyst, will explore it and quickly report unexpected results for fixing –  ABer this, test automa+on can begin
  50. 50. Test Automa+on For Mobile Devices •  The automaOon tool does not find the bugs! – The test automaOon tool is criOcized when the problem lies in the test automaOon approach – The person wri+ng the tests finds the bugs – No test automa+on tool can be beyer than the Tester who is crea+ng the tests – The best test automa+on tool is the one that you are most comfortable and confident in using
  51. 51. Test Automa+on for Mobile Devices •  Test the Flow of Opera+ons – Each test should be a contained flow of opera+ons that reflect user ac+vity and confirm whether each ac+on produces the expected result everywhere on the screen – Ask this: what is the flow supposed to do? – Is something preven+ng the flow? – Does the flow produce any unexpected result?
  52. 52. Test Automa+on for Mobile Devices •  Know The Expected Result! – Big Ques+on: does the actual result at the end of the flow match the pre-defined expected result? – If the answer is no, then this is a bug and should be reported for fixing – NOTE: the en+re screen is the expected result and not just one area that contains a returned value
  53. 53. Test Automa+on for Mobile Devices •  Always Keep in Mind: – FINDING BUGS IS GOOD! – A successful test run finds bugs! – There is oBen in Developer-created Test Automa+on that All Tests Passing = Good Result •  Use as many tests, asser+ons and verifica+ons as possible to find anomalies/unexpected results in the soBware •  Develop a “Spot the anomaly” approach to life -Design & Create Tests That Find Bugs!
  54. 54. Test Automa+on for Mobile Devices •  You must communicate with your tests to know that an unexpected results was found! – Know the difference between a bug in the soBware and a bug in the test script – Know what your tests are doing and what they are finding
  55. 55. Test Automa+on for Mobile Devices •  Test Automa+on Requires Monitoring – Think of Test Automa+on in the same way as the Autopilot in an airplane •  While the Autopilot is engaged, the pilots and flight officers are s+ll carefully watching and monitoring the instruments •  You need to pay ayen+on to your tests!
  56. 56. Test Automa+on for Mobile Devices •  The Applica+ons of Test Automa+on – SMOKE TESTS •  Did the build succeed? •  Is it worth con+nuing to test? – Advantages: •  Speed in development because few smoke tests are required •  Speed in execu+on – test automa+on will always be faster than manual tes+ng
  57. 57. Test Automa+on for Mobile Devices •  Regression Tests 1.  Regression tests for individual features during each sprint aBer every build 2.  End of project End-to-End regression tests that check and verify the stability of the en+re project •  With this approach, the stability of the en+re system is always checked for each new feature and how each func+on interacts as a whole
  58. 58. Test Automa+on For Mobile Devices •  How To Start Test Automa+on – Priori+ze By Importance •  What is the most important flow of the feature? •  What is the next most important, etc. – Keep The Tests Short & For A Single Purpose! •  Separate different ac+ons and varia+ons in flow into completely separate test –  This aids in script maintenance –  More importantly, it is easier to spot in which line the bug occurred in a short script
  59. 59. Test Automa+on For Mobile Devices •  Test Automa+on Scripts Should Have 3 Parts – Setup •  Start script •  Open browser •  Go to URL – Single Purpose Flow •  The flow for the ac+on to test – AsserOons and ValidaOons •  Compare the result you expect to see against what actual appears on the screen
  60. 60. Test Automa+on for Mobile Devices •  Test-Driven Development – The tests are created first and then the feature is developed so it passes the tests •  In this approach, QA/Tes+ng is baked into the process by being the first step –  This ensures communica+on between QA/Development/ Management/Client so everyone knows the expected design and func+onality of the feature –  Test runs will fail and everyone knows the reason for the failure – This can even be done at the Acceptance tes+ng level, in which End-to-End tests
  61. 61. Test Automa+on for Mobile Devices •  FINAL NOTES – Test Early, Test OBen – Use as many tests, asser+ons and verifica+ons as possible – Move fast and actually, really, break things!

×