What are similarities and differences in outsourced/outstaffed testing for CIS and India? What are main risks and what are the key success factors? Update: some explanation slides were added to the original presentation. They might seem dull, but they bring the main ideas which were explained during the presentation.

1. Lessons learnt in outsourced
project delivery. Test team view
from India and CIS
By Vipul Kocher & Kateryna Nesmyelova

2. Vipul Kocher
Vipul Kocher is a Co-founder of PureTesting,
a global testing services organization.
He has over 17 years of testing experience. He
built and led testing teams at reputed
companies such as Adobe Systems, Aplion
Networks and River Run Software Group.
Vipul has presented papers and tutorials at
various international testing conferences
throughout the world. He is also the inventor
of Q-Patterns, a method of capturing testing
knowledge, and writing reusable test cases
and Extension to Noun-and-Verb technique –
a test design method.
Vipul is also President of the Indian Testing
Board, Indian National Board of the ISTQB.
By Vipul Kocher & Kateryna Nesmyelova

3. Kateryna
Nesmyelova
8+ years in software testing. Kateryna
specializes in functional testing and test
management. She participated in testing of
various projects - from medical systems to
the systems of primary stock market for the
world's largest banks and innovative power
management in large data centers.
QA Club speaker and trainer, Kateryna
conducts a training to prepare for the
Foundation Level ISTQB certification and a
project for ISTQB - certification within
QAClub since 2009.
Member of USQB, Ukrainian branch of
ISTQB, since 2010;owner of the ISTQB Full
Advanced Certificate.
By Vipul Kocher & Kateryna Nesmyelova

4. What You Can’t
Outsource
By Vipul Kocher & Kateryna Nesmyelova

5. Topics to
be covered
1. Knowledge transfer.
2. Communication issues, taking into consideration time
gap and language issues.
3. Team Management (Governance).
4. Different goals for the in-house and outsourcing team.
5. Team issues (such as insufficient qualification of team
members, team member leaving the team etc.)
6. Peculiar testing issues (test environment and test bed
issues etc.)
7. Automation.
8. Cultural issues.
By Vipul Kocher & Kateryna Nesmyelova

6. Reasons to Fail
By Vipul Kocher & Kateryna Nesmyelova

7. Reasons to Fail
In any case, communication should be a top
priority in this relationship, as according to a Price
Waterhouse Coopers 2007 global outsourcing
survey, 69% of outsourcing deals fail, in whole or
part.
Actually poor communication and unclear
expectations are very frequently the main reason
outsourcing deals fail
By Vipul Kocher & Kateryna Nesmyelova

8. Communication
issues
- Wrong expectations
- Time zones difference
- Language problems (insufficient
knowledge of the communication
language)
- Slow reaction to requests
- Lack of feedback
By Vipul Kocher & Kateryna Nesmyelova

9. Communication
issues
More people mean more lines of communication, more opportunities for
miscommunication, and more misunderstandings and mistakes. Mistakes are
much harder to see and fix in outsourced projects.
First, the parties must clearly agree on the expectations from the outsourcing
team, the deliverables and the schedule.
<<Vipul – Communication is a two way street. Often the problem arises because
the client is unable to understand the MEANING of what is being said.
In India we have often found that our method of saying No is not as straight
forward as in the USA or Europe. We almost never say no to anything. This
causes unrealistic expectations. Indians are often reluctant to give bad news
immediately. This results in late surprises esp. when you can’t read the non-verbal
cues.
Some people, where their written communication is better than verbal, prefer not to
communicate in conference calls. This results in impersonal communication.
Emails are an impersonal means of communication. Phone calls are better, video
calls even better and in-person meetings are best. Once you can associate a face
to a name then emails, calls all become easier and more effective.>>
By Vipul Kocher & Kateryna Nesmyelova

10. Time Zones
If you work with multiple locations of one
client how to handle time zones?
Golden rule - Respond to those ahead of
you first and those after you later
Flexibility in working hours required to
get some overlap with the clients
By Vipul Kocher & Kateryna Nesmyelova

11. Time Zones
There is a good advice from an experienced manager that if
you want to win trust of people, you need to respond quickly
and in such a way that the matter doesn’t need any more
communication, if possible.
If more communication is required try to answer emails in
such time span that matters can be resolved on the same
day. For example, first thing in the morning, answer emails
from Japan and if you are catching tail end of work
somewhere else, like USA from India. This will give them time
to respond and you to respond back.
Changing your working hours so that you can get some
overlap with the clients is also very important esp. if you have
12 hour time zone difference.
By Vipul Kocher & Kateryna Nesmyelova

12. Knowledge
Transfer
- Fear of knowledge loss
- Insufficient business knowledge
- Too little documentations
- Afraid to ask questions
By Vipul Kocher & Kateryna Nesmyelova

13. Knowledge
Transfer
Most IT organizations experience a 20% decline in productivity during the
first year of an agreement, largely due to time spent transferring both
technical and business knowledge to the vendor. Many offshore vendors
are deploying video conferencing (avoiding travel) and classroom settings
(creating one-to-many transfer) to improve the efficacy of knowledge
transfer. In addition, employee turnover often places a burden on the IT
organization to provide additional information for new team members.
<<Vipul - One method we have tried to use is recording of knowledge
transfer sessions and then later editing those to make them short and
compact, esp. if we are dealing with complex topics and/or high turnover of
people. Wikis also help. Creating FAQ also helps. Asking clients for same
information again and again makes for a very bad experience.>>
Here I wouldn't agree with Vipul, as my own experience tells me that it's
always better to double-check if you are unsure about something. But I do
agree that the clarification shouldn't be extremely bothering.
By Vipul Kocher & Kateryna Nesmyelova

14. Different goals
- In-house team goal: make a
successful product
- Outsource/outstaff team goal:
sell as many man-hours as
possible (independently on the
product success).
- Different goals for each team
member.
By Vipul Kocher & Kateryna Nesmyelova

15. Trust
• Trust about capabilities, intentions,
security of information etc. is built over
time.
• What is your strategy for building trust?
• Some activities:
– Face-to-face time of key people from both sides
– Doing something “extra” without charging
customers
– Flexibility, willingness to adjust
– Consistent delivery
By Vipul Kocher & Kateryna Nesmyelova

16. Trust
Vipul >> Even small incidents can appear as big, relation threatening in absence of
trust and big incidents can be handled easily when trust exists.
Raising negative information as soon as possible, admitting to mistakes and putting
measures in place to avoid repeat behavior is very important for building trust.
In one case We found out that our productivity was 1/5th of what we had initially
projected. After trying hard to improve productivity we found that we could not
increase it beyond 1/4th of initial prediction. We called for a face to face meeting with
the client, explained the situation, told them about why productivity was low and
invited their teams to spend time with us to 1. understand why we were low on
productivity and where were the problems. They were upset but reluctantly agreed to
spend that time. After spending a week with us they realized that the mistake wasn’t
ours. There were a number of reasons:
1. Issues with test cases not being updated consequently our actual results seldom
matched the expected results in the test cases resulting in long cycles of clarification.
2. Environment availability
3. Slow speed of their link.
From our side we kept two extra resources (low experience, low cost) as back up
resources who could take up extra work or deliver more work. We did not charge
them anything.
These measures helped restore the client confidence in us.
By Vipul Kocher & Kateryna Nesmyelova

17. Vested Interest
Offshoring may
– put a fear of Job loss in minds of client’s
employees
– Some of them may want to see off-shoring fail
Who is your champion in the client
organization who would want you to
succeed?
Don’t get caught in the politics of your
client
By Vipul Kocher & Kateryna Nesmyelova

18. Vested Interest
On one project the outsource team found out that the person
who was responsible for outsourcing decision was battling
political battles in his organization and some people wanted him
to fail.
The outsource team made sure that they deliver reliable results
in timely manner so that the trust placed in the team by that
person was not broken. Once the team delivered good results
the political opposition died down and sailing was smooth.
In other case when the team sponsor left the company, the team
was were ill-prepared because they had not interacted at top
level with other people. Soon for various reasons the outsource
team was thrown out of that account.
One need not indulge in the politics but one needs to be aware
of it and make sure that people in client organization are aware
of good work done by you.
By Vipul Kocher & Kateryna Nesmyelova

19. Team issues
- Lack of management astuteness to
understand problems before they
happen
- Lack of skills/expertise
- Hiring wrong people
- Process Discipline
- Lack of resources
- Lack of ownership
By Vipul Kocher & Kateryna Nesmyelova

20. Team Management
(Governance).
- Turnover of key personnel
- Too many managers on both sides /
No management in Agile development
By Vipul Kocher & Kateryna Nesmyelova

21. Team Management
(Governance).
Skill training + motivation training, making sure that the
whole team shares the same goal – delivery of a
successful product.
Good feedback is vital for the team morale.
The outsourcing team, esp. the testers, should know that
they are valuable contributors to the project.
Special attention should be paid to managing people,
knowing their needs
By Vipul Kocher & Kateryna Nesmyelova

22. Governance
What is your governance strategy and plan?
– When it is a small project
– When it is a large project
What are the parameters that you measure
and analyze?
Perform Risk analysis periodically – both project risk
analysis and quality risk analysis
What is the frequency with which you
engage with client? At what levels?
Engage with the client at multiple levels,
going up as high as possible
By Vipul Kocher & Kateryna Nesmyelova

23. Governance
Weekly status for project is not sufficient. One needs to engage at multiple
levels so that in cases of escalation things can be handled.
Small projects but extensive metrics program to capture data result in too
much overhead. One needs to sit with the client to understand how do we
measure the health of the project, what are the parameters and what is the
effort in capturing the data.
There are some standard metrics – Milestones, Effort, Defect and Test
case based data. Risk is one which is often missed out. Follow top 5/10
risk every week and once a month perform complete risk analysis for long
term projects. For short term projects it may be done informally.
Customer satisfaction is another thing that needs to be measured at the
end of every project and annually covering larger cross-sections of client
management.
By Vipul Kocher & Kateryna Nesmyelova

24. Peculiar testing
issues
- High level of dependency
- Shared environment
- Equipment acquisition
By Vipul Kocher & Kateryna Nesmyelova

25. Peculiar testing
issues
Testing depends on everything, and if the scope creeps or
developers fail to deliver the product, time for testing is
reduced.
Solution: risk-based and prioritized testing.
Equipment acquisition includes not only persuading your
client to provide you with some specific equipment, but also
issues related to customs and delivery. Do not
underestimate them!
By Vipul Kocher & Kateryna Nesmyelova

26. Automation
– Access to application to be
automated
– Scripts support and
creation
– Environmental and
tool needs
– Lack of precisely
documented test STEPS
– Lack of domain knowledge in
automation teams
– Not sharing development plans
By Vipul Kocher & Kateryna Nesmyelova

27. Automation
Frequent problem areas
- Access to application to be automated
- Application keeps evolving. Customers might expect
automation done on a given build to work on more recent builds
- Number of application instances. If multiple people need to
automate, they might need multiple environments, tool licenses,
database instances etc.
- Lack of precisely documented test STEPS
- Lack of domain knowledge in automation teams and client
resources having no time to do handholding, knowledge
transfer, resolve queries
- Not sharing development plans often leaves off-shore team
automating things that need not be automated or are incorrectly
automated
By Vipul Kocher & Kateryna Nesmyelova

28. Culture Issues
Important to
– understand the culture of the client
– help them understand your culture
Most important points related to culture
– What is the meaning and method of saying
Yes/No in that culture
– What are the main festivals and holidays
– What are the religious, social sensitivities
– What is the work culture?
By Vipul Kocher & Kateryna Nesmyelova

29. Culture Issues
Work culture – some countries treat work above everything and
some treat family first.
Some people can’t say no but one has to understand the no.
For example, how to explain to a German what is the reason for
the 9th of May Holiday?
Client and service provider may have different norms in terms of
speed, style, decision making and organizational structure.
Sometimes both organizations can take extreme, inflexible
positions that serve to create tension or distrust (e.g, the client
taking a position that “I don’t care what the contract says, I’m
the customer” and the service provider equally digging in its
heels and taking a view that “we are delivering to contract
specifications”).
By Vipul Kocher & Kateryna Nesmyelova

30. Productivity
What are the client expectations of
productivity?
How are you measuring these?
What are the factors that impact your
productivity?
What are you doing to communicate
these to the client?
By Vipul Kocher & Kateryna Nesmyelova

31. Productivity
Tests executed, cycles done, tests created etc. are
common productivity metrics for testing.
For automation – scripts converted/created,
framework functions created are some others.
Outcome based testing is becoming popular where
outcomes like number of defects found, tests run, field
defects found etc. are used as parameters for testing
effectiveness and efficiency.
By Vipul Kocher & Kateryna Nesmyelova

32. Conclusions
- Set common goals
- Align expectations
- Good communication is a key to
success
- Manage carefully
- Monitor and motivate people
- Make the common process as clear as
possible
By Vipul Kocher & Kateryna Nesmyelova

33. Conclusions
- Clarify the responsibilities and role of
each party
- Determine how often reports (with what
level of detail!) are to be made and to
whom.
- Some cultures, might not feel
comfortable reporting problems, or
giving bad news. This is why you should
realistically define success, and give
clear measurements for it.
By Vipul Kocher & Kateryna Nesmyelova

34. Q&A
By Vipul Kocher & Kateryna Nesmyelova

35. Kateryna.Nesmyelova@gmail.com
kittyness - Skype
By Vipul Kocher & Kateryna Nesmyelova