This presentation demonstrates eBay’s exciting new world of test engineering. We will tell you all our secrets.

1. Applied Testing Heuristics
in the Context of eBay
Dominik Dary & Ilari Henrik Aegerter
eBay International AG
March 13, 2013

2. WHO ARE WE? WHAT DO WE DO?
Dominik Dary
Staff Software Engineer in Test
Mobile Test Automation Enthusiast
E-mail: ddary@ebay.com
Twitter: @dominikdary
Ilari Henrik Aegerter
Manager Quality Engineering Europe
Testing of all EU relevant features on 12 eBay sites
E-mail: iaegerter@ebay.com
Twitter: @ilarihenrik
Applied Testing Heuristics in the Context of eBay 2

3. CONTENT
•Intro
•Misconceptions
•Test Process
•Using Tours
•Regression Testing/Using Tools
•Continuous Improvement
Applied Testing Heuristics in the Context of eBay 3

4. INTRO

5. MOBILE FACTS
Mobile Facts Mobile Fun Facts
100,000,000 downloads Every 30 seconds, a woman’s handbag is
of eBay Inc’s apps purchased on eBay Mobile.
Mobile transaction volume Every 2 minutes, a tablet is bought through
has grown in 2012 to 13 billion USD eBay Mobile.
3,3 million items Every week, more than 10,000 cars and
are listed with using mobile apps every week 500,000 car parts are sold through eBay’s
mobile apps.
Source: http://www.ebayinc.com/assets/pdf/fact_sheet/eBay_Mobile_Collateral_Fact_Sheet_2_08_13.pdf
Applied Testing Heuristics in the Context of eBay 5

6. EBAY INC. FACTS
Founded in 1995
Based in San Jose, California
~28’000 Employees worldwide
Applied Testing Heuristics in the Context of eBay 6

7. WHAT DOES THIS TITLE MEAN?
Applied – What/Why?
Testing – Well, yes?
Heuristics – Huh? What?
Context – Ahem? More please?
eBay – What’s special?
Applied Testing Heuristics in the Context of eBay 7

8. WHAT DOES THIS
TITLE MEAN?
‘Applied’
Image credit: http://www.flickr.com/photos/americanartmuseum/3463141656/sizes/o/in/photostream/
Applied Testing Heuristics in the Context of eBay 8

9. WHAT DOES THIS
TITLE MEAN?
‘Testing’
Image credit: http://www.flickr.com/photos/kevlar/4198943649/sizes/o/in/photostream/
Applied Testing Heuristics in the Context of eBay 9

10. WHAT DOES THIS
TITLE MEAN?
‘Heuristics’
Applied Testing Heuristics in the Context of eBay 10

11. WHAT DOES THIS TITLE MEAN?
‘Context’
Image credit: http://www.flickr.com/photos/timdrivas/6973202868/sizes/h/in/photostream/
Applied Testing Heuristics in the Context of eBay 11

12. WHAT DOES THIS TITLE MEAN?
‘eBay’
Applied Testing Heuristics in the Context of eBay 12

13. MISCONCEPTIONS

14. MISCONCEPTIONS
Automation vs.
Manual
Applied Testing Heuristics in the Context of eBay 14

15. MISCONCEPTIONS
You need a
heavy paper trail
Image credit: http://www.flickr.com/photos/fredr/262344284/sizes/l/in/photostream/
Applied Testing Heuristics in the Context of eBay 15

16. TEST STRATEGY

17. TEST STRATEGY
Primary Information Objective:
Shall we ship?
Image credit: http://www.flickr.com/photos/liwnik/4141632990/sizes/z/in/photostream/
Applied Testing Heuristics in the Context of eBay 17

18. TEST STRATEGY
Distribution Manual/Automated
Image credit: http://www.flickr.com/photos/61091655@N00/6831352744/sizes/l/in/photostream/ Image credit: http://www.flickr.com/photos/r80o/39304743/sizes/l/in/photostream/
Applied Testing Heuristics in the Context of eBay 18

19. TEST STRATEGY
Manual
Image credit: http://www.flickr.com/photos/61091655@N00/6831352744/sizes/l/in/photostream/
Applied Testing Heuristics in the Context of eBay 19

20. TEST STRATEGY
Automated
Image credit: http://www.flickr.com/photos/r80o/39304743/sizes/l/in/photostream/
Applied Testing Heuristics in the Context of eBay 20

21. PLANNING

22. PLANNING
Dev:
San Jose, USA
Applied Testing Heuristics in the Context of eBay 22

23. PLANNING
Testing:
San Jose &
Zurich, Berlin,
Paris, London
Applied Testing Heuristics in the Context of eBay 23

24. PLANNING
1. Request
2. Self-organize
Image credit: http://www.flickr.com/photos/davidaola/317048270/sizes/z/in/photostream/
Applied Testing Heuristics in the Context of eBay 24

25. PLANNING
No written test plans
Applied Testing Heuristics in the Context of eBay 25

26. ANALYSIS/MODELING/
TEST DESIGN

27. ANALYSIS/
MODELING/
TEST DESIGN
Test Oracle:
Whatever is
available
Image credit: http://www.flickr.com/photos/amodiovalerioverde/270355045/sizes/z/in/photostream/
Applied Testing Heuristics in the Context of eBay 27

28. ANALYSIS/MODELING/TEST DESIGN
No test cases
Applied Testing Heuristics in the Context of eBay 28

29. REPORTING

30. REPORTING
Bug Reports
Applied Testing Heuristics in the Context of eBay 30

31. REPORTING
Test Reports
-General Summary
-Perceived risks
-Found issues
-What did we do
Applied Testing Heuristics in the Context of eBay 31

32. USING TOURS

33. FCC CUTS VIDS
THE MNEMONIC STANDS FOR THE FOLLOWING:
•Feature tour •Variability tour
•Complexity tour •Interopeability tour
•Claims tour •Data tour
•Structure tour
•Configuration tour
•User tour
•Testability tour
•Scenario tour
Applied Testing Heuristics in the Context of eBay 33

34. REGRESSION
TESTING

35. TEST AUTOMATION
•To get early and instant Feedback
•We automate the regression tests
•It is our safety net to save time & this is reinvested
into manual testing
Applied Testing Heuristics in the Context of eBay 35

36. PAGES AND BUSINESS CRITICAL FLOWS
e-Commerce shopping site like eBay
Select
View Order
Review Payment
Category Summary
Provider
Search View View
Update
Item Category Orderlist
View Delete
Product
View Item Shopping Cart
Page Page Checkout Page Success Page
Applied Testing Heuristics in the Context of eBay 36

37. ANDROID APP TEST CASE
Tests
Flow Objects
Objects
Page
Applied Testing Heuristics in the Context of eBay 37

38. THE IMPORTANCE OF PARALLEL TEST
EXECUTION
•Multiple Products
•12 European Sites
•Different Technologies
–The Web
–Windows
–Mobile Web
–Mobile Apps
•Android
•iOS
Picture Source: http://www.flickr.com/photos/ippei-janine/483345027/
Mobile Testing and Mobile Automation at eBay 38

39. TECHNICAL HIGH LEVEL OVERVIEW OF
THE TEST INFRASTRUCTURE
Selenium Webdriver / Grid
Automated Test
Development
via IDE
QE
Engineer Version Control System eBay web
Tests can be front end
started on Grid
Clone
Source Files
End-to-End Grid Hub
Tests +
Start the Test Suite
Build via
via Brower Maven
Test type? API Call eBay API
Run test
Agile via TestNG
Team
View the Test
Session and the Continuous Integration
Result in the System (Jenkins)
Browser DB validation Database
Our virtualized test infrastructure hosted on VMware vSphere Testing Environment
Applied Testing Heuristics in the Context of eBay 39

40. USING TOOLS

41. TOOLS FOR TESTING SUPPORT
Image credit: http://www.wenger.ch/stuff/contentmgr/files/0/0faeaa3c3c7c9e33468cf11a78cfb51e/image/evo_s54_sku_rev1.png
Applied Testing Heuristics in the Context of eBay 41

42. CONTINUOUS
IMPROVEMENT

43. CONTINUOUS LEARNING BY PRESCRIPTION?
PRESENTATION TITLE GOES HERE 43

44. Thank You!
Questions or Comments?
Dominik Dary Ilari Henrik Aegerter
E-mail: ddary@ebay.com E-mail: iaegerter@ebay.com
Twitter: @dominikdary Twitter: @ilarihenrik
Mobile Testing and Mobile Automation at eBay
44