If the future of testing is AI, I will show you how. Various automation tools are available, but due to high creation and maintenance efforts for tests, GUI-Testing is still mainly a manual task. Meanwhile overall testing effort has risen to make up 30% of a typical software budget. Is crowd-testing the answer? What if we could have automated test cases be created automatically? Monkey Testing is not a new idea. But combined with a manually trainable AI and an innovative new testing approach (dubbed ""difference testing""), we can now not only have the monkey search for technical bugs (i.e. crashes) but generate functional test cases that are optimized towards several goals and are even better than manually created ones. Visit the future of testing and see how AI can help us create better software!

1. @roesslerj
1
Applying AI to Testing
Dr. Jeremias Rößler
Kyiv 2017

2. @roesslerj
2

3. @roesslerj
3

4. @roesslerj
4

5. @roesslerj
5

6. @roesslerj
6
Oracle
Problem

7. @roesslerj
7
How do you bring AIinto testing?

8. @roesslerj
8
AI in Testing?
– so far I do not see anyone doing it
in a useful way.
Antony Edwards,
CTO of TestPlant
“

9. @roesslerj
“
9
AI in Testautomation is a hype.
Wolfgang Platz,
Founder & CPO of Tricentis

10. @roesslerj
10
I think there is a world market
for maybe five computers.“
Thomas J. Watson
CEO of IBM

11. @roesslerj
11
Goal of Testing?

12. @roesslerj
12
When is a bug a bug?
When it’s not a feature!

13. @roesslerj
13
1.if (name[0] == '.') continue;
Is it a bug?
.
..
text.c
other.files

14. @roesslerj
14
Is it a bug?

15. @roesslerj
15
Is it a bug?

16. @roesslerj
16
Without specification,
there are no bugs
— only surprises.
Brian Kernighan
“

17. @roesslerj
17
Why do we test?
After
implementation?
1 + 1 = 3
2 + 2 = 4

18. @roesslerj
18

19. @roesslerj
20

20. @roesslerj
21
Why do we test?
After
implementation?
1 + 1 = 3
2 + 2 = 4
After
change?
1 + 1 = 3
1 + 1 = 5

21. @roesslerj
regression-test
≠
test
22

22. @roesslerj
regression-test
=
version control
23

23. @roesslerj
24

24. @roesslerj
25

25. @roesslerj
26

26. @roesslerj
27

27. @roesslerj
28

28. @roesslerj
29

29. @roesslerj
30

30. @roesslerj
31

31. @roesslerj
32

32. @roesslerj
33

33. @roesslerj
34

34. @roesslerj
35

35. @roesslerj
36

36. @roesslerj
37
How do you bring AIinto testing?

37. @roesslerj
38
Infinite-Monkey-Theorem:
A monkey hitting keys at random on a typewriter
for a long enough time
will type the complete works of William Shakespeare.

38. @roesslerj
39
We replace the typewriter with a computer…

39. @roesslerj
Infinite Monkey
40
public static void main(String... args) throws Exception {
Robot robot = new Robot();
while (true) {
robot.mouseMove(random.nextInt(maxX), random.nextInt(maxY));
robot.mousePress(InputEvent.BUTTON1_DOWN_MASK);
robot.mouseRelease(InputEvent.BUTTON1_DOWN_MASK);
robot.delay(200);
for (char inputChar : randomString().toCharArray()) {
robot.keyPress((int) inputChar);
robot.keyRelease((int) inputChar);
robot.delay(10);
}
robot.keyPress(KeyEvent.VK_ENTER);
robot.keyRelease(KeyEvent.VK_ENTER);
}
}
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.
16.

40. @roesslerj
41
DEMO
Infinite Monkey

41. @roesslerj
43

42. @roesslerj
44

43. @roesslerj
Smart Monkey
45
public static void main(String... args) throws Exception {
WebDriver driver = new FirefoxDriver();
driver.get("http://www.retest.de");
while (true) {
List<WebElement> links = driver.findElements(By.tagName("a"));
links.get(random.nextInt(links.size())).click();
Thread.sleep(500);
List<WebElement> fields =
driver.findElements(By.xpath("//input[@type='text']"));
WebElement field = fields.get(random.nextInt(fields.size()));
field.sendKeys(randomString());
Thread.sleep(500);
}
}
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.

44. @roesslerj
46
Smart Monkey
DEMO

45. @roesslerj
Smart Monkey
48

46. @roesslerj
49
- gremlins.js
- NetFlix Chaos Monkey
- UI/Application Exerciser Monkey
Open Source Products

47. @roesslerj
50

48. @roesslerj
51

49. @roesslerj
52
Afraid of AI?

50. @roesslerj
53
How many blondes does it need
to screw in a light bulb?
http://www.smbc-comics.com/comic/humor
None, because automation has
eliminated the need for humans.

51. @roesslerj
“
54
AI is perfectly suited to replace testing.
Jason Arbon,
Founder & CPO of AppDiff

52. @roesslerj
55
Not so fast…

53. @roesslerj
56

54. @roesslerj
57
Be afraid if:
- test automation engineer
- manual automate

55. @roesslerj
58

56. @roesslerj
59
Intelligenter Affe
Summary
Monkey-Testing:
cheaper
faster
better
multiplies manual tests
Difference Testing
little maintenance
no wasteful testing
yet more complete

57. @roesslerj
60
November 2017

58. @roesslerj
61
@Test
public void test_login() throws Exception {
final WebDriver driver = new FirefoxDriver();
driver.get( "http://www.retest.de" );
recheck.check( driver );
driver.findElement( By.id( "username" ) ).sendKeys( username );
driver.findElement( By.id( "pwd" ) ).sendKeys( password );
driver.findElement( By.className( "button" ) ).submit();
recheck.check( driver );
}

59. @roesslerj
62
PDF
Excel
XML
HTML
Proprietary

60. @roesslerj
63
http://retest.rocks
@ReTest_en

61. @roesslerj
Questions?
https://www.facebook.com/retest.de
http://retest.rocksroessler@retest.de
The largest software testing conference in Ukraine