IT is often considered as a technical science or engineering. Traditionally testers are techies who focus on analysing requirements and turning them into a series of test cases. Some also analyze product risks and write (master) test plans. Focus is on technical and analytical skills. But testing requires a lot more! Testing is about attitude, skills, communication, behavior, collaboration and (critical and/or systems) thinking. The seven basic principles of the Context-Driven School tell us that people, working together, are the most important part of any project's context. That good software testing is a challenging intellectual process. And that only through judgement and skill, exercised cooperatively throughout the entire project, are we able to do the right things at the right times to effectively test our products. In these principles there is a lot of non technical stuff that has a major influence in my work as a tester. This talk gives insight in why testing is a social science. It also gives some examples of what a tester can take away from social sciences. Anthropology teaches us about how people life, interact, something about culture. Education/didactic helps acquire new or modifying existing knowledge, behaviors, skills, values, or preferences. Sociology learns us empirical investigation and critical analysis and gives insight in human social activity. Psychology is the study of the mind and behavior and helps testers understand individuals and groups.
3. Conference audiences
You have a need for other people to like and admire you, and yet you
tend to be critical of yourself. While you have some personality
weaknesses you are generally able to compensate for them. You have
considerable unused capacity that you have not turned to your
advantage. Disciplined and self-controlled on the outside, you tend to be
worrisome and insecure on the inside. At times you have serious doubts
as to whether you have made the right decision or done the right thing.
You prefer a certain amount of change and variety and become
dissatisfied when hemmed in by restrictions and limitations. You also
pride yourself as an independent thinker; and do not accept others’
statements without satisfactory proof. But you have found it unwise to be
too frank in revealing yourself to others. At times you are extroverted,
affable, and sociable, while at other times you are introverted, wary, and
reserved. Some of your aspirations tend to be rather unrealistic.
codecentric nederland BV
6. Critical Thinking
Huh?
What does this mean? What is the point?
Really?
Are you absolutely certain? How can I know?
So?
Where does this lead? So what?
Source: James Bach (see video: http://youtu.be/8TX6rzz60xQ)
codecentric nederland BV
7. Critical Thinking
• Recognise argument
• Analyse argument
• Evaluate argument
• Deductive &
Inductive reasoning
• Critical vs creative
thinking?
codecentric nederland BV
Source: Critical Thinking: a user's manual – Debra Jackson
10. Effective testing
“To test effectively, our theories of error have to be theories
about the mistakes people make and when / why they make
them”
codecentric nederland BV
11. What is Social Science?
codecentric nederland BV
PHOTO: HTTP://WWW.FLICKR.COM/PHOTOS/ROBBY_VAN_MOOR/478725670/
22. About your opinion...
• What did I find out?
Not prove yourself right
Prove yourself wrong
• First collect data, then judge / have an opinion
Grounded Theory Method
codecentric nederland BV
23. Adapt to military life?
codecentric nederland BV
PHOTO: HTTP://WWW.FLICKR.COM/PHOTOS/ILUVCOCACOLA/53495590/
25. You are not so smart
• Cognitive biases
• Logical fallacies
• Heuristics
codecentric nederland BV
26. Confirmation Bias
You tend to look for information that confirms your beliefs
and ignore the information that challenges them.
• Biased search
• Biased interpretation
• Biased memory
codecentric nederland BV
35. Testing is about
Behaviour Communication
Skills
Attitude
Learning
Thinking Collaboration
codecentric nederland BV
36. Now you!
Read the links & think critical about it:
• Thinking fast & slow – Daniel Kahneman
• You're not so smart – David McRaney
• The invisible Gorilla – Christopher Chabris and Daniel Simons
• Qualitative Data Analysis: a user-friendly guide for social scientists – Ian Dey
• Critical Thinking: a user's manual – Debra Jackson
• http://steveo1967.blogspot.com/2011/01/are-testers-ethnographic-researchers.html
• http://steveo1967.blogspot.com/2011/01/what-you-believe-might-not-be-true-part.htm
• http://steveo1967.blogspot.com/2010/07/danger-confirmation-bias.html
• http://usersknow.blogspot.ca/2013/02/combining-qualitative-quantitative.html
• Research Methods: Participant observation (www.sociology.org.uk)
• Qualitative Evaluation checklist - Michael Quinn Patton (www.wmich.edu/evalctr/checklists)
• Critical & creative thinking (http://www.engin.umich.edu/~cre/probsolv/index.htm)
• Software Testing a Social Science – Cem Kaner
• Testing Through The Qualitative Lens – Michael Bolton
• Curing Our Binary Disease – Rikard Edgren
codecentric nederland BV
37. Summary
• Be aware of what you do in testing
• Learn from what people do in social sciences
• Do quantitative AND qualitative research
• Think critically
• Be aware of biases and fallacies
• Accept and deal with ambiguity, situational specific results
and partial answers
codecentric nederland BV
38. Acknowledgements
Special thanks to:
John Stevenson
http://www.steveo1967.blogspot.com/
Many of the ideas in this presentation were inspired by
colleagues including Cem Kaner, Michael Bolton, James
Bach, Rikard Edgren and Jerry Weinberg.
codecentric nederland BV
40. Contacts
Huib Schoots
huib.schoots@codecentric.nl
@huibschoots
www.huibschoots.nl/blog
codecentric nederland BV
Laan der Verenigde Naties 60
3314 DA Dordrecht
The Netherlands
+31 (0) 6 24 64 10 33
www.codecentric.nl
codecentric nederland BV
Editor's Notes
Social science: complexity of the human mind!Exploration, description and explanation are the three purposes of social science research. - Earl BabbieObservation Humans are importantStorytelling
Brain: 1.3 kilo woman, 1.4 kilo manWorking temperature37 degrees CelsiusFueled by potatoes!1015 calculations per second2 percent of your total body massNeeds 10 times more oxygen then the rest of your body20% of your blood flows to your brainUses 25% of all energySOURCE
You think you are rational, but you are not.people fail to realize the irrationality of their actions and believe they are acting perfectly rational, possibly due to flaws in their reasoning.people's actual interests differ from what they believe to be their interests.mechanisms that have evolved to give optimal behavior in normal conditions lead to irrational behavior in abnormal conditions.situations outside one's ordinary circumstances, where one may experience intense levels of fear, or may regress to a fight-or-flight mentality.apparently irrational decisions that are actually optimal, but made unconsciously on the basis of "hidden" interests that are not known to the conscious mind.an inability to comprehend the social consequences of one's own actions, possibly due in part to a lack of empathy.Some people find themselves in this condition by living "double" lives. They try to put on one "mask" for one group of people and another for a different group of people. Many will become confused as to which they really are or which they wish to become.People are predicable irrational
Systems thinking is the process of understandinghowthings, regarded as systems, influenceoneanotherwithin a whole.Systems thinking has been defined as an approach toproblemsolving, byviewing "problems" as parts of an overall system, ratherthanreactingtospecific part, outcomes or events andpotentiallycontributingtofurtherdevelopment of unintendedconsequences
Testing is questioning a product in order to evaluate it. (RST)Testing is gathering information with the intention of informing a decision (Jerry Weinberg)Providing informationInvestigate/question a product AND related products AND service: the whole solutionTesting and checking: testing is asking questions where we do not know the answers / checking: we know the answer
Programmers findand fix most of theirown bugs\r\nWhat testers find are whatprogrammersmissed\r\nTesters are lookingfor bugs thathide in programmers blind spots
Social science:SocietyHuman natureHuman interactionAnd people are irrational!Difference with exact science High tolerance for ambiguity, situational specific results and partial answers
Daniel kahnemanPsychology is the study of the mind and behaviour and helps testers understand individual behaviour.How do we decide? How do we think?Why important in testing? About people, how they think, why they do what they do, emotions, motivation, behaviour and relationships
Karl MarxPeople and their behaviour in a social environment, in groups. Moral. Ethics.It learns us empirical investigation and critical analysis and gives insight in human social activityWhy important in testing?because we work with people, in teamsand all that that brings, social class, religion, language
Claude Levi-StraussHow people life and interact (culture)Linguistic anthropologySocial, cultural or sociocultural anthropologyWhy important in testing?Different cultures, languages, habits, hierarchy, etc.
Maria MontessoriPractical application of teaching andlearning.How do peoplelearn?Why important in testing?tolearnandteachandlearninghow best the workforus[andlearninghow we can best do them
Communication studiesProcesses of human communicationWhy important in testing?Developers: not really important to know the end usersTesters: vitalBlogpost John: human impact
Communication studiesProcesses of human communicationWhy important in testing?
Research:Social distanceObjectivityAvoid intrusion of personal beliefs/values into researchUsequantitativeandqualitativere search methods
Quantitative researchQuantitative data deals with measurement and numbers: quantity, size and frequencyFind answers to closeddetermined research questions, often research methods are described upfront. Results is a data collection of numbers and statistical information.Use quantitative metrics to understand exactly what your users are doing. What features do they use? How much do they spend? Does changing something big have a big impact on real user behavior?
Qualitative researchthe description and interpretation of the problems in a specific context. It is mainly about observing and naming behaviorYou need both, story to backup numbersSocial science: “Partial answers that might be useful.” Qualitative data deals with meanings (story): when you want to understand and something: the underlying thoughts and intentionsUse qualitative research to understand why your users do what they do. What problems are they trying to solve? Why are they dropping out of a particular task flow when they do? Why do they leave and never come back. Subjective sociology: EmpathyObserveExperienceParticipate
Science is important…. They gave us critical thinking. Proving your theories.Could it be something else?Is it what I expected? What did I do differently?How can I explain what I did? We have it the wrong way around…. More emphasis on story. You miss outliers: unique random event. Grounded theory method is a research method which operates almost in a reverse fashion from traditional social science research. Rather than beginning with a hypothesis, the first step is data collection, through a variety of methods.
Please discuss!A study by the War Department on U.S. soldiers returning home after World War Two One conclusion from the study is that soldiers from rural backgrounds have an easier time adapting to military life than those from urban backgrounds. And, yes, that does make sense. Rural life is less comfortable, requires initiative, and requires more discipline in ways that match army life.The real result is that soldiers from urban backgrounds fare better. It only takes a minute of reflection to realize that this, too, makes sense. They should fare better given more experience with large organizations and dealing with strangers, for instance
http://edweb.tusd.k12.az.us/sandre/andre28.htmPotato by RikardNumbersLooking at it differently
Radiologistsskilled at searching scans fortinyanomalieswithpotentially life-threateningconsequences80%skilled
Made to add value to usersUsers are humanSoftware solves problemsProblems are not binary: programmers can come up with different ways to solve testers to find what they forgot or how they thought of it
Jerry Weinberg: quaility is value to some person…. How objective is that?
How many tests?Ask a managerSimple requirement: still infinite possibilities Coverage: N / infinite = 0
Testing is about:BehaviourCommunicationSkillsAttitudeThinkingCollaborationLearning