Я расскажу про один из проектов по тестированию производительности в котором я принимал непосредственное участие. Это история о том, как все на проекте может пойти не так; о необычных технологиях, инструментах и подходах; про паникующих клиентов и руководство. И о том, как с этим справиться и не сойти с ума.

1. Last Minute Performance Testing
t WITH PASSION TO QUALITY
Pavlo Vedilin
QA CONFERENCE #1 IN UKRAINE, KYIV 2018

2. PROJECTS
A B C
E FDCross-company
structure that serves as
a center of excellence
for delivery teams
G H
L NK
J

3. GOOD NEWS, EVERYONE!
Dear QMO
We need help with Load Testing for our project. It’s one-time activity that should be run
during the next 2 weeks. We don’t have QC there, only DevOps who don’t have experience
with Load tests.
Best Regards
Name Here

4. LOAD TESTING?
Virtual User 1
Virtual User 2
Virtual User N
…

5. SO WHAT IS THE PLAN?
The performance testing approach consists of the following activities*:
1. Identify the Test Environment.
2. Identify Performance Acceptance Criteria.
3. Plan and Design Tests.
4. Configure the Test Environment.
5. Implement the Test Design.
6. Execute the Test.
7. Analyze Results, Report, and Retest.
* According to Microsoft “Performance Testing Guidance for Web Applications”:

6. WEEK 1
PM:

7. • Client is a e-learning developer and provider
• Client`s platform consists of several servers with different roles
• Short-term DevOps contract
• Goal is to write scripts for quick client infrastructure set-up
• Contract includes an agreement to check if environment will not fail under the load
KICK-OFF MEETING
20 000 USERS
VAGRANT
CHEF
PUNJAB
OPENFIRE
JABBER
XMPP
WEBSOCKETS
BOSH PROTOCOL
DEADLINE IN TWO WEEKS!

8. WHAT?

9. WHAT?

10. WHAT?

11. WHAT NOW?

12. ANY OTHER TOOLS?

13. ANY OTHER TOOLS?

14. ANY OTHER TOOLS?

15. IT WORKS ON MY MACHINE!

16. WHAT ABOUT CLIENT ENVIRONMENT?

17. CLIENT CALL
US:
– WE FOUND A TOOL THAT WORKS
– WE DESIGNED THE TEST
– WORKING ON MONITORING OF THE APP
– WORKING ON TEST SCRIPT IMPLEMENTATION
– … BUT FOR NOW IT DOESN’T RUN ON YOUR ENVIRONMENT
CLIENT:
– THIS IS NOT WHAT I EXPECTED!
– IT WILL NOT WORK!
– DO IT WITH JAVASCRIPT!

18. WEEK 2
CLIENT:PM:

19. SO WHATS WRONG WITH
CLIENT ENVIRONMENT?

20. STICKY SESSIONS
1
2
3
Hi!
Hey 1
1 1 Tsup?

21. STICKY SESSIONS
1
2
3
Hi!
Hey 1
Tsup?
?

22. • TSUNG SUPPORTS COOKIES
• …BUT ONLY IN HTTP MODULE

23. HTTP Module
BOSH Module

24. CALL
FOR
HELP
TIME TO…

25. CLIENT CALL
US:
– IT’S DEADLINE…
– … AND WE NEED TO ADD COOKIES SUPPORT TO TSUNG
CLIENT:
– YOU ARE WASTING TIME!
– DO IT WITH JAVASCRIPT!
– … WE’LL GIVE YOU ADDITIONAL TIME FOR THAT

26. WEEK 3
CLIENT:PM:

27. THREAD 2
 CREATE NEW TOOL WITH JS
HELP NEEDED!
THREAD 1
 ADD COOKIE SUPPORT ?

28. PROVE ME THAT TSUNG WORKS!

29. WHAT ELSE CAN GO WRONG?
PROBLEM SOLVED!

30. 13 MINUTES INVISIBLE WALL

32. CLIENT CALL
1
2
3
Hi!
Hey 1
1 Tsup?
1

33. WHAT HAPPENED?

34. WEEK 4
CLIENT:PM:

35. IS IT WORKING YET?

36. SO WHAT WAS THE PLAN?
The performance testing approach consists of the following activities*:
1. Identify the Test Environment.
2. Identify Performance Acceptance Criteria.
3. Plan and Design Tests.
4. Configure the Test Environment.
5. Implement the Test Design.
6. Execute the Test.
7. Analyze Results, Report, and Retest.
* According to Microsoft “Performance Testing Guidance for Web Applications”:

37. ONE
MONTH
LATER
ME:

38. RESULTS
 GENERALLY PESSIMISTIC TEST RESULTS
 CLIENT DECIDES TO COMPLETELY CHANGE ARCHITECTURE
 … AND TO USE THIS ALTERED TOOL FOR HEALTH CHECKS
 CLIENT CONTINUED COOPERATION WITH SOFTSERVE

39. QUESTIONS?

40. LESSONS LEARNED
 DON’T LEAVE ‘SIMPLE’ TASKS FOR LAST
 …BUT IF YOU DID…
 DON’T PANIC!
 EVEN IF EVERYBODY AROUND YOU ARE IN PANIC
 COMMUNICATION IS KEY
 KEEP STAKEHOLDERS UPDATED
 IF STUCK, CALL FOR HELP
 LOG YOUR WORK
 … AND ASK FOR DETAILED ARCHITECTURE

41. THANK YOU

42. Тема доклада
Тема доклада
Тема доклада
KYIV 2018
WITH PASSION TO QUALITY
QA CONFERENCE #1 IN UKRAINE