Kyiv Quality Assurance Day 2019 АНТОН МУЖАЙЛО «Test Team Development and Management Techniques» Телеграм канал: wwww.t.me/goqameetup Фейсбук сторінці: www.fb.com/goqaevent Сайт: www.kyiv.qaday.org

1. Test Team Development
and Management
Techniques
Anton Muzhailo,
Lead Automation Engineer / Consultant, GlobalLogic
Kyiv 2019

2. Little about me
I’m basically doing:
• Test Automation / Performance Testing
• Test Management
• Consulting and assessment for a list of projects
• Teaching & Mentoring
in/muzhailo/
Anton Muzhailo
Lead Automation Engineer,
GlobalLogic
• ISTQB Certified Test Manager
• Provide ISTQB Certified Trainings in CodeSpace
• 350+ full course attendees during 4 years

3. What will not be discussed

4. Agenda
• What are the most important issues we always need to address?
• How to hire the right people to the test team?
• How to manage people skills in your team?
• How to make your team development progress visible?
• What are the formal processes that allows us to manage all above?

5. Have you seen such situations? (1/3)
What do we invest What do we receive
Ideal agile process defined with all
needed tools and activities
Unpredictable releases Complex process
Metrics doesn’t show real situation

6. Have you seen such situations? (2/3)
What do we invest What do we receive
Formal retrospectives and reviews with
everybody present.
Establishing and implementing improvements
Continue making the same mistakes
Everything seems awesome on paper only

7. Have you seen such situations? (3/3)
What do we invest What do we receive
Canonical planning with pointing poker.
Regular grooming and brainstorming, even formal
estimation techniques application.
Last minute changes
If we fail in planning
we say “We should be agile”

8. The biggest challenge
- Now in this world of cool frameworks and approaches, people remains to be
the most often bottleneck
- You can’t build effective processes before building the team
- Every process should be in engineers’ mind rather than on paper
- You must hire right people

9. Two types of team structure
Dominion model Communion model
 Clear responsibility/reporting chain
 Works with any people
 Easy to implement
X One can block others/everyone
X Always require skilled leader
 Multiple leaders in their own domains
 Resistant to bottlenecks
 More “agile”
X Doesn’t work without strong engineers
X People aren’t used to it

10. How to build the test team right?
The ideal test team has:
1. a mix of skills (hard and soft) and experience levels
2. common goals
3. willingness and ability to teach and learn
4. clear communication
5. respect & trust
You can teach monkey how to code at some point
but you will never teach human how to be human

11. Soft skills >> Hard skills
Test Manager/Lead should hire the people that are able to:
• Learn (Ask to use some new FW with the help of Google)
• Share knowledge (Ask to tell something or write some documentation)
• Explain (Ask to proof his point of view with evidences and arguments)
• Negotiate (Prepare needed circumstances and negotiate)
• Review and be reviewed (Place in appropriate situation)
• Separate work conflicts from personal (Ask the thoughts)

12. Soft skills >> Hard skills
Skill
Level
TimeA B C D E F G
Skill
Level
TimeA B C D E F G
Strong technical but lower soft skills Lower technical but strong soft skills

13. How to define the hard skills
1. List all your skills and mark those applicable for a project
2. Compare this with the list of skills provided by customer and add needed
3. Think about other skills that will be also helpful. (You may use job vacancies
web sites or linkedin  )
4. Prioritize the value of each skill you marked applicable. Assign them with a
priority level: Must have, Better to have, Will be a plus
5. Generate vacancy description based on this list

14. Skill set definition
Testing Skills
SDLC/STLC activities
Test Design
Test Management
Static Testing (Reqs, Specs)
Black-box Testing
White-box Testing
Performance Testing
Risk Based Test Strategy
Code Coverage
Test Automation
Test Planning / Estimation
Technical Skills
Win / Linux / OSX
C++ / Python / Shell
TCP/IP, networking
Patterns, O(n), SW design
Databases (Oracle, MongoDB)
Continuous Integration toolset
UI testing with Selenium
Hardware maintenance
C#-based servers
Cloud (Azure)
Unit Test frameworks
Soft skills
Mentoring
Learning
Public Speaking
Assessment / Audit
Negotiations
Customer communications
Interviewing
English
Review skills
Presenting (ppt, graphs etc)
Conflicts handling

15. Project description
Hi-load cloud-based software for smth
- FrondEnd / BackEnd: Angular 2 with Java and Oracle on AWS
- Android and iOS apps
- Fast iterations based on Agile/Scrum
- Custom automation framework on Ruby+Cucumber
- CI/CD on Jenkins/Docker
- Performance testing required from scratch
- 3 major releases during 1 year, additional 50% resources decided to hire
- Main problems are: critical defects found on RCs, metrics doesn’t show real
picture, big staff turnover, severe dependence on several people, low skill
level of a test team

16. Final skills needed
Testing Skills
SDLC/STLC activities
Test Design
Test Management
Static Testing (Reqs, Specs)
Black-box Testing
White-box Testing
Performance Testing
Risk Based Test Strategy
Code Coverage
Test Automation
Test Planning / Estimation
Technical Skills
Win / Linux / OSX
C++ / Python / Shell
TCP/IP, networking
Patterns, O(n), SW design
Databases (Oracle, MongoDB)
Continuous Integration toolset
UI testing with Selenium
Hardware maintenance
C#-based servers
Cloud (Azure)
Unit Test frameworks
Additional Skills
C# backend
Cloud (AWS)
Ruby Automation + Cucumber
Mobile manual testing
experience (Android + iOS)
Dynamic testing (DynaTrace)
Because TM is
you already
Duplicated
with Ruby
0 – no experience, 3 – able to use with some help, 5 –
able to solve anything independently

17. Skills Required/Desired Jeremy Lucy Michael
SDLC/STLC activities R3 4 3 2
Test Design R3 4 3 2
Black-box Testing R4 4 2 2
Performance Testing 0 2 0 3
Code Coverage 0 0 2 3
Win / Linux / OSX R3 3 4 4
TCP/IP, networking D3 2 3 4
Databases (Oracle) D2 0 0 3
Continuous Integration toolset D2 1 2 3
UI testing with Selenium 0 0 3 3
C# backend 0 0 0 2
Cloud (AWS) D4 4 1 4
Ruby Automation + Cucumber 0 0 0 1
Android + iOS manual testing D4 1 4 0
Dynamic testing (DynaTrace) 0 0 0 2
TOTAL POINTS 25 27 35
Manual QA

18. 0
1
2
3
4
5
SDLC/STLC activities
Test Design
Black-box Testing
Performance Testing
Code Coverage
Win / Linux / OSX
TCP/IP, networking
Databases (Oracle)
Continuous
Integration toolset
UI testing with
Selenium
C# backend
Cloud (AWS)
Ruby Automation +
Cucumber
Android + iOS manual
testing
Dynamic testing
(DynaTrace)
Manual QA
Jeremy Lucy Michael
- Lucy is needed because only she has
Android + iOS testing skills
- Michael has more automation skills and will
be proposed to apply for automation
- If we hire Jeremy and Lucy we will have
following manual qa team:
0
1
2
3
4
5
SDLC/STLC
activities
Test Design
Black-box
Testing
Win / Linux /
OSX
TCP/IP,
networking
Databases
(Oracle)
Continuous
Integration…
Cloud (AWS)
Android + iOS
manual testing
Manual QA
Ideal Minimum Team has max

19. Skills Required/Desired Susan Rick Bob
SDLC/STLC activities R3 3 3 4
Test Design R3 3 2 4
Black-box Testing 0 0 0 2
Performance Testing D2 0 3 0
Code Coverage R3 0 3 0
Win / Linux / OSX R3 3 4 4
TCP/IP, networking R3 2 4 4
Databases (Oracle) R3 4 3 2
Continuous Integration toolset R4 4 3 1
UI testing with Selenium R4 4 1 2
C# backend D3 0 4 1
Cloud (AWS) R3 2 3 3
Ruby Automation + Cucumber R4 2 4 2
Android + iOS manual testing 0 0 0 2
Dynamic testing (DynaTrace) D2 2 0 0
TOTAL POINTS 29 37 31
Automation QA

20. 0
1
2
3
4
5
SDLC/STLC activities
Test Design
Black-box Testing
Performance Testing
Code Coverage
Win / Linux / OSX
TCP/IP, networking
Databases (Oracle)
Continuous
Integration toolset
UI testing with
Selenium
C# backend
Cloud (AWS)
Ruby Automation +
Cucumber
Android + iOS manual
testing
Dynamic testing
(DynaTrace)
Automation QA
Susan Rick Josh
- Bob is totally lower than our expectations
- Susan and Rick covers the gaps of each other
like a perfect mates if we hire them, we will
have following automation qa team:
0
1
2
3
4
5
SDLC/STLC…
Test Design
Black-box…
Performance…
Code…
Win / Linux…
TCP/IP,…
Databases…Continuous…
UI testing…
C# backend
Cloud (AWS)
Ruby…
Android +…
Dynamic…
Automation QA
Ideal Minimum Team max

21. Skills Jeremy Lucy Susan Rick Average Max Total
SDLC/STLC activities 4 3 3 3 3.25 4 13
Test Design 4 3 3 2 3 4 12
Black-box Testing 4 2 0 0 1.5 4 6
Performance Testing 2 0 0 3 1.25 3 5
Code Coverage 0 2 0 3 1.25 3 5
Win / Linux / OSX 3 4 3 4 3.5 4 14
TCP/IP, networking 2 3 2 4 2.75 4 11
Databases (Oracle) 0 0 4 3 1.75 4 7
Continuous Integration toolset 1 2 4 3 2.5 4 10
UI testing with Selenium 0 3 4 1 2 4 8
C# backend 0 0 0 4 1 4 4
Cloud (AWS) 4 1 2 3 2.5 4 10
Ruby Automation + Cucumber 0 0 2 4 1.5 4 6
Android + iOS manual testing 1 4 0 0 1.25 4 5
Dynamic testing (DynaTrace) 0 0 2 0 0.5 2 2
TOTAL POINTS 25 27 29 37 118
The Team

22. 0
1
2
3
4
5
SDLC/STLC activities
Test Design
Black-box Testing
Performance Testing
Code Coverage
Win / Linux / OSX
TCP/IP, networking
Databases (Oracle)Continuous Integration toolset
UI testing with Selenium
C# backend
Cloud (AWS)
Ruby Automation + Cucumber
Android + iOS manual testing
Dynamic testing (DynaTrace)
Test Team Dynamic
Average Max

23. Q & A