education

1. Should you or shouldn't you collect your data through observation? Questions to consider: Is the topic sensitive? Are people uncomfortable or unwilling to answer questions about a particular subject? For instance, many people are uncomfortable when asked about prejudice. Self-reports of prejudice often bring biased answers. Instead, a researcher may choose to observe black and white students interactions. In this case, observations are more likely to bring about more accurate data. Thus, sensitive social issues are better suited for observational research. Can you observe the Phenomena? You must be able to observe what is relevant to your study. Let's face it, you could observe and observe but if you never see what your studying your wasting your time. You can't see attitudes. Although you can observe behaviors and make inferences about attitudes. Also, you can't be everywhere. There are certain things you can't observe. For example, questions regarding sexual behavior are better left to a survey. Do you have a lot of time? Many people don't realize that observational research may be time consuming. In order to obtain reliability, behaviors must be observed several times. In addition, there is also a concern that the observer's presence may change the behaviors being observed. As time goes on, however, the subjects are more likely to grow accustomed to your presence and act normally. It is in the researchers best interest to observe for a long period of time. Are you not sure what your looking for? That's okay! Known as descriptive research, observations are a great way to start a research project. Let's say you are interested in male and female behavior in bars. You have no idea what theory to use or what behavior you are interested in looking for. So, you watch, and, wow, you see something. Like the amount of touching is related to alcohol consumption. So you run to the library, gather your research, and maybe decide to do more observations or supplement your study with surveys. Then, these observations turn into a theory once they are replicated (well, it's not quite that simple). So you see, observations are a good place to start. Types of Observations Okay, so you've decided that you think observational research is for you. Now you only have to pick which kind of observation to do. Direct (Reactive) ObservationIn direct observations, people know that you are watching them. The only danger is that they are reacting to you. As stated earlier, there is a concern that individuals will change their actions rather than showing you what they're REALLY like. This is not necessarily bad, however. For example, the contrived behavior may reveal aspects of social desirability, how they feel about sharing their feelings in front of others, or privacy in a relationship. Even the most contrived behavior is difficult to maintain over time. A long term observational study will often catch a glimpse of the natural behavior. Other problems concern the generalizability of findings. The sample of individuals may not be representative of the population or the behaviors observed are not representative of the individual (you caught the person on a bad day). Again, long-term observational studies will often overcome the problem of external validity. What about ethical problems you say? Ethically, people see you, they know you are watching them (sounds spooky, I know) and they can ask you to stop. Now here are two commonly used types of direct observations: Continuous Monitoring: Continuos monitoring (CM) involves observing a subject or subjects and recording (either manually, electronically, or both) as much of their behavior as possible. Continuos Monitoring is often used in organizational settings, such as evaluating performance. Yet this may be problematic due to the Hawthorne Effect. The Hawthorne Effect states that workers react to the attention they are getting from the researchers and in turn, productivity increases. Observers should be aware of this reaction. Other CM research is used in education, such as watching teacher-student interactions. Also in nutrition where researchers record how much an individual eats. CM is relatively easy but a time consuming endeavor. You will be sure to acquire a lot of data. Time Allocation:Time Allocation (TA) involves a researcher randomly selecting a place and time and then recording what people are doing when they are first seen and before they see you. This may sound rather bizarre but it is a useful tool when you want to find out the percent of time people are doing things (i.e. playing with their kids, working, eating, etc.). Thereare several sampling problems with this approach. First, in order to make generalizations about how people are spending their time the researcher needs a large representative sample. Sneaking up on people all over town is tough way to spend your days. In addition, questions such as when, how often, and where should you observe are often a concern. Many researchers have overcome these problems by using nonrandom locations but randomly visiting them at different times. Unobtrusive Observation:Unobtrusive measures involves any method for studying behavior where individuals do NOT know they are being observed (don't you hate to think that this could have happened to you!). Here, there is not the concern that the observer may change the subject's behavior. When conducting unobtrusive observations, issues of validity need to be considered. Numerous observations of a representative sample need to take place in order to generalize the findings. This is especially difficult when looking at a particular group. Many groups posses unique characteristics which make them interesting studies. Hence, often such findings are not strong in external validity. Also, replication is difficult when using non-conventional measures (non-conventional meaning unobtrusive observation). Observations of a very specific behaviors are difficult to replicate in studies especially if the researcher is a group participant (we'll talk more about this later). The main problem with unobtrusive measures, however, is ethical. Issues involving informed consent and invasion of privacy are paramount here. An institutional review board may frown upon your study if it is not really necessary for you not to inform your subjects. Here is a description of two types of unobtrusive research measures you may decide to undertake in the field: Behavior Trace studies:Behavior trace studies involve findings things people leave behind and interpreting what they mean. This can be anything to vandalism to garbage. The University of Arizona Garbage Project one of the most well-known trace studies. Anthropologists and students dug through household garbage to find out about such things as food preferences, waste behavior, and alcohol consumption. Again, remember, that in unobtrusive research individuals do not know they are being studied. How would you feel about someone going through your garbage? Surprisingly Tucson residents supported the research as long as their identities were kept confidential. As you might imagine, trace studies may yield enormous data. Disguised Field Observations:Okay, this gets a little sticky. In Disguised field analysis the researcher pretends to join or actually is a member of a group and records data about that group. The group does not know they are being observed for research purposes. Here, the observer may take on a number of roles. First, the observer may decide to become a complete-participant in which they are studying something they are already a member of. For instance, if you are a member of a sorority and study female conflict within sororities you would be considered a complete-participant observer. On the other hand you may decide to only participate casually in the group while collecting observations. In this case, any contact with group members is by acquaintance only. Here you would be considered an observer-participant. Finally, if you develop an identity with the group members but do not engage in important group activities consider yourself a participant-observer. An example would be joining a cult but not participating in any of their important rituals (such as sacraficing animals). You are however, considered a member of the cult and trusted by all of the members. Ethically, participant-observers have the most problems. Certainly there are degrees of deception at work. The sensitivity of the topic and the degree of confidentiality are important issues to consider. Watching classmates struggle with test-anxiety is a lot different than joining Alcoholics Anonymous. In all, disguised field experiments are likely to yield reliable data but the ethical dilemmas are a trade-off. Observational Variables Before you start on a research project make sure you how you are going to interpret your observations. Descriptive:Descriptive observational variables require no inference making on the part of the researcher. You see something and write it down. Inferential:Inferential observational variables require the researcher to make inferences about what is observed and the underlying emotion. For example, you may observe a girl banging on her keyboard. From this observation you may assume (correctly) that she is frustrated with the computer. Evaluative:Evaluative observational variables require the researcher to make an inference and a judgment from the behavior. For example, you may question whether computers and humans have a positive relationship.
Positive
is an evaluative judgment. You observe the girl banging on her keyboard and conclude that humans and computers do not have a positive relationship (you know you must replicate these findings!). When writing field notes the researcher should include descriptive as well as inferential data. It is important to describe the setting and the mood in a detailed manner. All such things that may change behavior need to be noted. Especially reflect upon your presence. Do you think that you changed the behavior noticeably? Okay, so this is a lot to remember. Go back up to the check-list of
things you should be able to...
and ask yourself some questions. Remember, observations are a great way to start and add to a research project. INTERVIEW. What Kinds of Information Can be Obtained? Rubin and Rubin (1995) identify several information types: narratives, accounts, fronts, stories, and myths (p. 24-27). To project a front is to act in an acceptable way--the way others expect, to give an impression. For example, at a meeting professors may convey confidence but not personally feel it. When person justifies their actions, they give accounts, culturally acceptable reasons for their behavior. When asked, ÒWhy were you absent from the meeting?Ó the professor responded that he felt sick. Stories, on the other hand, may communicate a broad message or set of morals. Participant Observation and Qualitative Interviewing? Crossing borders can be difficult if one is an outsider. In trying to understand a culture, a researcher needs to become a student in order to be taught, a kind of particpant observer. The process takes time, proceeds slowly, and involves times of inactivity or just Òhanging around.Ó Various approaches can be undertaken: become a novice in the desired institution (an established practice); learn the language (national and specific jargon); attend meetings, and read books about the subject. Some studies deal with a social problem in which one pursues the meaning of a perplexing problem or behavior, or a life history deals with how people understand rites of passage, and even oral histories about past values and norms. What is Triangulation? Triangulation is a process of verification [checking for truth] that increases validity by incorporating three different viewpoints and methods. Sevigny (1978) calls a combination of all three stances triangulation, a sociological process of viewing a situation from all three perspectives. You can also achieve triangulation by using different research techniques. For instance, in his study of five university drawing courses, he incorporated audiotapes, interviews, and diary writing. He also warns that accepting the full participant stance, and not relating the perspective of the observer, has its consequence of refusal of the observed to grant permission to use the study. Wolcott (1988) suggests that triangulated techniques are helpful
for cross-checking, or for ferreting out varying perspectives on complex issues and events
(p. 192). What Types of Interviews? Several types of interviews exist: topical oral history, life history, evaluation interview, focus group interview, and cultural interviews. Topical interviews are concerned with the facts and sequence of an event. The interviewer is interested in a reconstruction of the experience and what happened; for instance, what happened at the InSEA Conference in Brisbane Australia. The researcher actively directs questions in pursuit of precise facts. Life histories deal with individual experiences or rites of passage. In oral histories, one collects information about a dying lifestyle or art skills. These result in narratives and stories that interpret the past. Evaluation interviews examine new programs or school developments and suggests improvements. Since evaluation deals with incorrect behaviors as well as positive ones, justifications [accounts] of behaviors result. The result may consist of myths and unresolved tensions (Patton, 1990). In focus group interviews people meet to share their impressions and changes of thinking or behavior regarding a product or an institution. Participants may be strangers and make an effort to preserve their competency and may not admit faults. The cultural interview focuses on Òthe norms, values, understandings, and taken-for-granted rules of behavior of a group or societyÓ (p. 28). This type of interview reports on TYPICAL shared activities and their meanings. The style of interview is relaxed and questions flow naturally with no fixed agenda. People are interviewed several times so that emerging themes are pursued later. The interviewer, for example, may ask them to DESCRIBE A TYPICAL DAY. The partner then relates what is important with examples. The truth of the fact is not as important as Òhow well it illustrates the [cultural] premises and normsÓ (p.29). For example, in a Christian culture, you may be told about the significance of the value of behaving with concern for other people (p. 29). In the cultural interview, the interviewer is partner and co-constructs the interview and report. The cultural report, besides being the expertÕs story, is credible because it consists of the words of members of the culture. We assume that people are basically honest and that they share similar views. The researcher can mix types of interviews and approaches. A qualitative interview is different from everyday conversation in the following ways. First it is a research tool and a good interviewer must prepare questions in advance, and later analyze and report results. The interviewer guides the questions and focuses the study. Good interview skills require practice and reflection. Finally, beyond the acquisition of interview skills, interviewing is a philosophy of learning. The interviewer becomes a student and then tries to get people to describe their experiences in their own terms. The results are imposed obligations on both sides. The qualitative researcherÕs philosophy determines what is important, what is ethical, and the completeness and accuracy of the results (Rubin & Rubin, 1995, p.2). Several researchers have argued that structured interviews are unnatural and restrictive. Informal interviews get Òdeeper.Ó For example, if you want to find out why someone acted in a certain way, ask him/her. One must negotiate an explanation that consistent and believable. This results in an explanation of the meaning of the action for the people (Alasuutari, 1998, p. 143). The interviewer follows up an interview with more questions for clarification or understanding. The key is to establish Òrapport and trustÓ (p.145). During the interview, a person may change his/her interpretation. Contrast Between a Topical Interview and Cultural Interview? In a topical interview, you get one chance and no time to re-interview. The ideal would be a series of interviews where one can pursue a list of cultural concepts and themes. In cultural interviews, most of the selection or questions is done between interviews. One learns Òhow the people see, understand, and interpret their worldÓ (Rubin & Rubin, 1995, p.195) In contrast to the topical interview in which interviewers are concerned mainly with their own questions and agenda, the qualitative questioner is more interested in what people in the studied culture reveal and find of concern. In the beginning, keep the scope of the interview open and flexible with few interrupting questions (p.175). The researcher needs to listen to what the people are saying about their experiences. One looks at the given stories, narratives, and examples. The cultural foundations will come later. Second, one can look at a cultureÕs icons (religious or heroic) to discover admired qualities (defiance, bravery, cleverness, and persistence), or iconic statements (that reflect cultural discontent or iconic events (Woodstock). Eventually, one makes inferences about underlying norms and themes hidden in examples. In summary, similar to a fisherman, cast your net and slowly reel in the options. Interviews Interviews are among the most challenging and rewarding forms of measurement. They require a personal sensitivity and adaptability as well as the ability to stay within the bounds of the designed protocol. Here, I describe the preparation you need to do for an interview study and the process of conducting the interview itself. The Role of the Interviewer The interviewer is really the
jack-of-all-trades
in survey research. The interviewer's role is complex and multifaceted. It includes the following tasks: Locate and enlist cooperation of respondents The interviewer has to find the respondent. In door-to-door surveys, this means being able to locate specific addresses. Often, the interviewer has to work at the least desirable times (like immediately after dinner or on weekends) because that's when respondents are most readily available. Motivate respondents to do good job If the interviewer does not take the work seriously, why would the respondent? The interviewer has to be motivated and has to be able to communicate that motivation to the respondent. Often, this means that the interviewer has to be convinced of the importance of the research. Clarify any confusion/concerns Interviewers have to be able to think on their feet. Respondents may raise objections or concerns that were not anticipated. The interviewer has to be able to respond candidly and informatively. Observe quality of responses Whether the interview is personal or over the phone, the interviewer is in the best position to judge the quality of the information that is being received. Even a verbatim transcript will not adequately convey how seriously the respondent took the task, or any gestures or body language that were evident. Conduct a good interview Last, and certainly not least, the interviewer has to conduct a good interview! Every interview has a life of its own. Some respondents are motivated and attentive, others are distracted or disinterested. The interviewer also has good or bad days. Assuring a consistently high-quality interview is a challenge that requires constant effort. Training the Interviewers One of the most important aspects of any interview study is the training of the interviewers themselves. In many ways the interviewers are your measures, and the quality of the results is totally in their hands. Even in small studies involving only a single researcher-interviewer, it is important to organize in detail and rehearse the interviewing process before beginning the formal study. Here are some of the major topics that should be included in interviewer training: Describe the entire study Interviewers need to know more than simply how to conduct the interview itself. They should learn about the background for the study, previous work that has been done, and why the study is important. State who is sponsor of research Interviewers need to know who they are working for. They -- and their respondents -- have a right to know not just what agency or company is conducting the research, but also, who is paying for the research. Teach enough about survey research While you seldom have the time to teach a full course on survey research methods, the interviewers need to know enough that they respect the survey method and are motivated. Sometimes it may not be apparent why a question or set of questions was asked in a particular way. The interviewers will need to understand the rationale for how the instrument was constructed. Explain the sampling logic and process Naive interviewers may not understand why sampling is so important. They may wonder why you go through all the difficulties of selecting the sample so carefully. You will have to explain that sampling is the basis for the conclusions that will be reached and for the degree to which your study will be useful. Explain interviewer bias Interviewers need to know the many ways that they can inadvertently bias the results. And, they need to understand why it is important that they not bias the study. This is especially a problem when you are investigating political or moral issues on which people have strongly held convictions. While the interviewer may think they are doing good for society by slanting results in favor of what they believe, they need to recognize that doing so could jeopardize the entire study in the eyes of others.
Walk through
the interview When you first introduce the interview, it's a good idea to walk through the entire protocol so the interviewers can get an idea of the various parts or phases and how they interrelate. Explain respondent selection procedures, including reading maps It's astonishing how many adults don't know how to follow directions on a map. In personal interviews, the interviewer may need to locate respondents who are spread over a wide geographic area. And, they often have to navigate by night (respondents tend to be most available in evening hours) in neighborhoods they're not familiar with. Teaching basic map reading skills and confirming that the interviewers can follow maps is essential. identifying households In many studies it is impossible in advance to say whether every sample household meets the sampling requirements for the study. In your study, you may want to interview only people who live in single family homes. It may be impossible to distinguish townhouses and apartment buildings in your sampling frame. The interviewer must know how to identify the appropriate target household. identify respondents Just as with households, many studies require respondents who meet specific criteria. For instance, your study may require that you speak with a male head-of-household between the ages of 30 and 40 who has children under 18 living in the same household. It may be impossible to obtain statistics in advance to target such respondents. The interviewer may have to ask a series of filtering questions before determining whether the respondent meets the sampling needs. Rehearse interview You should probably have several rehearsal sessions with the interviewer team. You might even videotape rehearsal interviews to discuss how the trainees responded in difficult situations. The interviewers should be very familiar with the entire interview before ever facing a respondent. Explain supervision In most interview studies, the interviewers will work under the direction of a supervisor. In some contexts, the supervisor may be a faculty advisor; in others, they may be the
boss.
In order to assure the quality of the responses, the supervisor may have to observe a subsample of interviews, listen in on phone interviews, or conduct follow-up assessments of interviews with the respondents. This can be very threatening to the interviewers. You need to develop an atmosphere where everyone on the research team -- interviewers and supervisors -- feel like they're working together towards a common end. Explain scheduling The interviewers have to understand the demands being made on their schedules and why these are important to the study. In some studies it will be imperative to conduct the entire set of interviews within a certain time period. In most studies, it's important to have the interviewers available when it's convenient for the respondents, not necessarily the interviewer. The Interviewer's Kit It's important that interviewers have all of the materials they need to do a professional job. Usually, you will want to assemble an interviewer kit that can be easily carried and includes all of the important materials such as: a
professional-looking
3-ring notebook (this might even have the logo of the company or organization conducting the interviews) maps sufficient copies of the survey instrument official identification (preferable a picture ID) a cover letter from the Principal Investigator or Sponsor a phone number the respondent can call to verify the interviewer's authenticity The Interview So all the preparation is complete, the training done, the interviewers ready to proceed, their
kits
in hand. It's finally time to do an actual interview. Each interview is unique, like a small work of art (and sometimes the art may not be very good). Each interview has its own ebb and flow -- its own pace. To the outsider, an interview looks like a fairly standard, simple, prosaic effort. But to the interviewer, it can be filled with special nuances and interpretations that aren't often immediately apparent. Every interview includes some common components. There's the opening, where the interviewer gains entry and establishes the rapport and tone for what follows. There's the middle game, the heart of the process, that consists of the protocol of questions and the improvisations of the probe. And finally, there's the endgame, the wrap-up, where the interviewer and respondent establish a sense of closure. Whether it's a two-minute phone interview or a personal interview that spans hours, the interview is a bit of theater, a mini-drama that involves real lives in real time. Opening Remarks In many ways, the interviewer has the same initial problem that a salesperson has. You have to get the respondent's attention initially for a long enough period that you can sell them on the idea of participating in the study. Many of the remarks here assume an interview that is being conducted at a respondent's residence. But the analogies to other interview contexts should be straightforward. Gaining entry The first thing the interviewer must do is gain entry. Several factors can enhance the prospects. Probably the most important factor is your initial appearance. The interviewer needs to dress professionally and in a manner that will be comfortable to the respondent. In some contexts a business suit and briefcase may be appropriate. In others, it may intimidate. The way the interviewer appears initially to the respondent has to communicate some simple messages -- that you're trustworthy, honest, and non-threatening. Cultivating a manner of professional confidence, the sense that the respondent has nothing to worry about because you know what you're doing -- is a difficult skill to teach and an indispensable skill for achieving initial entry. Doorstep technique You're standing on the doorstep and someone has opened the door, even if only halfway. You need to smile. You need to be brief. State why you are there and suggest what you would like the respondent to do. Don't ask -- suggest what you want. Instead of saying
May I come in to do an interview?
, you might try a more imperative approach like
I'd like to take a few minutes of your time to interview you for a very important study.
Introduction If you've gotten this far without having the door slammed in your face, chances are you will be able to get an interview. Without waiting for the respondent to ask questions, you should move to introducing yourself. You should have this part of the process memorized so you can deliver the essential information in 20-30 seconds at most. State your name and the name of the organization you represent. Show your identification badge and the letter that introduces you. You want to have as legitimate an appearance as possible. If you have a three-ring binder or clipboard with the logo of your organization, you should have it out and visible. You should assume that the respondent will be interested in participating in your important study -- assume that you will be doing an interview here. Explaining the study At this point, you've been invited to come in (After all, you're standing there in the cold, holding an assortment of materials, clearly displaying your credentials, and offering the respondent the chance to participate in an interview -- to many respondents, it's a rare and exciting event. They hardly ever get asked their views about anything, and yet they know that important decisions are made all the time based on input from others.). Or, the respondent has continued to listen long enough that you need to move onto explaining the study. There are three rules to this critical explanation: 1) Keep it short; 2) Keep it short; and 3) Keep it short! The respondent doesn't have to or want to know all of the neat nuances of this study, how it came about, how you convinced your thesis committee to buy into it, and so on. You should have a one or two sentence description of the study memorized. No big words. No jargon. No detail. There will be more than enough time for that later (and you should bring some written materials you can leave at the end for that purpose). This is the
25 words or less
description. What you should spend some time on is assuring the respondent that you are interviewing them confidentially, and that their participation is voluntary. Asking the Questions You've gotten in. The respondent has asked you to sit down and make yourself comfortable. It may be that the respondent was in the middle of doing something when you arrived and you may need to allow them a few minutes to finish the phone call or send the kids off to do homework. Now, you're ready to begin the interview itself. Use questionnaire carefully, but informally The questionnaire is your friend. It was developed with a lot of care and thoughtfulness. While you have to be ready to adapt to the needs of the setting, your first instinct should always be to trust the instrument that was designed. But you also need to establish a rapport with the respondent. If you have your face in the instrument and you read the questions, you'll appear unprofessional and disinterested. Even though you may be nervous, you need to recognize that your respondent is most likely even more nervous. If you memorize the first few questions, you can refer to the instrument only occasionally, using eye contact and a confident manner to set the tone for the interview and help the respondent get comfortable. Ask questions exactly as written Sometimes an interviewer will think that they could improve on the tone of a question by altering a few words to make it simpler or more
friendly.
DON'T. You should ask the questions as they are on the instrument. If you had a problem with a question, the time to raise it was during the training and rehearsals, not during the actual interview. It is important that the interview be as standardized as possible across respondents (this is true except in certain types of exploratory or interpretivist research where the explicit goal is to avoid any standardizing). You may think the change you made was inconsequential when, in fact, it may change the entire meaning of the question or response. Follow the order given Once you know an interview well, you may see a respondent bring up a topic that you know will come up later in the interview. You may be tempted to jump to that section of the interview while you're on the topic. DON'T. You are more likely to lose your place. You may omit questions that build a foundation for later questions. Ask every question Sometimes you'll be tempted to omit a question because you thought you already heard what the respondent will say. Don't assume that. For example, let's say you were conducting an interview with college age women about the topic of date rape. In an earlier question, the respondent mentioned that she knew of a woman on her dormitory floor who had been raped on a date within the past year. A few questions later, you are supposed to ask
Do you know of anyone personally who was raped on a date?
You figure you already know that the answer is yes, so you decide to skip the question. Instead, you might say something like
I know you may have already mentioned this, but do you know of anyone personally who was raped on a date?
At this point, the respondent may say something like
Well, in addition to the woman who lived down the hall in my dorm, I know of a friend from high school who experienced date rape.
If you hadn't asked the question, you would never have discovered this detail. Don't finish sentences I don't know about you, but I'm one of those people who just hates to be left hanging. I like to keep a conversation moving. Once I know where a sentence seems to be heading, I'm aching to get to the next sentence. I finish people's sentences all the time. If you're like me, you should practice the art of patience (and silence) before doing any interviewing. As you'll see below, silence is one of the most effective devices for encouraging a respondent to talk. If you finish their sentence for them, you imply that what they had to say is transparent or obvious, or that you don't want to give them the time to express themselves in their own language. Obtaining Adequate Responses - The Probe OK, you've asked a question. The respondent gives a brief, cursory answer. How do you elicit a more thoughtful, thorough response? You probe. Silent probe The most effective way to encourage someone to elaborate is to do nothing at all - just pause and wait. This is referred to as the
silent
probe. It works (at least in certain cultures) because the respondent is uncomfortable with pauses or silence. It suggests to the respondent that you are waiting, listening for what they will say next. Overt encouragement At times, you can encourage the respondent directly. Try to do so in a way that does not imply approval or disapproval of what they said (that could bias their subsequent results). Overt encouragement could be as simple as saying
Uh-huh
or
OK
after the respondent completes a thought. Elaboration You can encourage more information by asking for elaboration. For instance, it is appropriate to ask questions like
Would you like to elaborate on that?
or
Is there anything else you would like to add?
Ask for clarification Sometimes, you can elicit greater detail by asking the respondent to clarify something that was said earlier. You might say,
A minute ago you were talking about the experience you had in high school. Could you tell me more about that?
Repetition This is the old psychotherapist trick. You say something without really saying anything new. For instance, the respondent just described a traumatic experience they had in childhood. You might say
What I'm hearing you say is that you found that experience very traumatic.
Then, you should pause. The respondent is likely to say something like
Well, yes, and it affected the rest of my family as well. In fact, my younger sister...
Recording the Response Although we have the capability to record a respondent in audio and/or video, most interview methodologists don't think it's a good idea. Respondents are often uncomfortable when they know their remarks will be recorded word-for-word. They may strain to only say things in a socially acceptable way. Although you would get a more detailed and accurate record, it is likely to be distorted by the very process of obtaining it. This may be more of a problem in some situations than in others. It is increasingly common to be told that your conversation may be recorded during a phone interview. And most focus group methodologies use unobtrusive recording equipment to capture what's being said. But, in general, personal interviews are still best when recorded by the interviewer using pen and paper. Here, I assume the paper-and-pencil approach. Record responses immediately The interviewer should record responses as they are being stated. This conveys the idea that you are interested enough in what the respondent is saying to write it down. You don't have to write down every single word -- you're not taking stenography. But you may want to record certain key phrases or quotes verbatim. You need to develop a system for distinguishing what the respondent says verbatim from what you are characterizing (how about quotations, for instance!). Include all probes You need to indicate every single probe that you use. Develop a shorthand for different standard probes. Use a clear form for writing them in (e.g., place probes in the left margin). Use abbreviations where possible Abbreviations will help you to capture more of the discussion. Develop a standardized system (e.g., R=respondent; DK=don't know). If you create an abbreviation on the fly, have a way of indicating its origin. For instance, if you decide to abbreviate Spouse with an 'S', you might make a notation in the right margin saying
S=Spouse.
Concluding the Interview When you've gone through the entire interview, you need to bring the interview to closure. Some important things to remember: Thank the respondent Don't forget to do this. Even if the respondent was troublesome or uninformative, it is important for you to be polite and thank them for their time. Tell them when you expect to send results I hate it when people conduct interviews and then don't send results and summaries to the people who they get the information from. You owe it to your respondent to show them what you learned. Now, they may not want your entire 300-page dissertation. It's common practice to prepare a short, readable, jargon-free summary of interviews that you can send to the respondents. Don't be brusque or hasty Allow for a few minutes of winding down conversation. The respondent may want to know a little bit about you or how much you like doing this kind of work. They may be interested in how the results will be used. Use these kinds of interests as a way to wrap up the conversation. As you're putting away your materials and packing up to go, engage the respondent. You don't want the respondent to feel as though you completed the interview and then rushed out on them -- they may wonder what they said that was wrong. On the other hand, you have to be careful here. Some respondents may want to keep on talking long after the interview is over. You have to find a way to politely cut off the conversation and make your exit. Immediately after leaving -- write down any notes about how the interview went Sometimes you will have observations about the interview that you didn't want to write down while you were with the respondent. You may have noticed them get upset at a question, or you may have detected hostility in a response. Immediately after the interview you should go over your notes and make any other comments and observations -- but be sure to distinguish these from the notes made during the interview (you might use a different color pen, for instance). 15 Methods of Data Analysis in Qualitative Research 1. Typology - a classification system, taken from patterns, themes, or other kinds of groups of data. (Patton pp. 393,398) John Lofland & Lyn Lofland Ideally, categories should be mutually exclusive and exhaustive if possible, often they aren't. Basically a list of categories. example: Lofland and Lofland's 1st edition list: acts, activities, meanings, participation, relationships, settings (in the third edition they have ten units interfaced by three aspects--see page 114--and each cell in this matrix might be related to one of seven topics--see chapter seven). 2. Taxonomy (See Domain Analysis - often used together, especially developing taxonomy from a single domain.) James Spradley A sophisticated typology with multiple levels of concepts. Higher levels are inclusive of lower levels. Superordinate and subordinate categories 3. Constant Comparison/Grounded Theory (widely used, developed in late 60's) Anselm Strauss • Look at document, such as field notes • Look for indicators of categories in events and behavior - name them and code them on document • Compare codes to find consistencies and differences • Consistencies between codes (similar meanings or pointing to a basic idea) reveals categories. So need to categorize specific events • We used to cut apart copies of field notes, now use computers. (Any good word processor can do this. Lofland says qualitative research programs aren't all that helpful and I tend to agree. Of the qualitative research programs I suspect that NUD*IST probably the best--see Sage Publishers). • Memo on the comparisons and emerging categories • Eventually category saturates when no new codes related to it are formed • Eventually certain categories become more central focus - axial categories and perhaps even core category. 4. Analytic Induction (One of oldest methods, a very good one) F. Znaniecki, Howard Becker, Jack Katz. I wrote a paper on the topic. Look at event and develop a hypothetical statement of what happened. Then look at another similar event and see if it fits the hypothesis. If it doesn't, revise hypothesis. Begin looking for exceptions to hypothesis, when find it, revise hypothesis to fit all examples encountered. Eventually will develop a hypotheses that accounts for all observed cases. 5. Logical Analysis/Matrix Analysis An outline of generalized causation, logical reasoning process, etc. Use flow charts, diagrams, etc. to pictorially represent these, as well as written descriptions. Matthew Miles and Huberman gives hundreds of varieties in their huge book Qualitative Data Analysis, 2nd ed. 6. Quasi-statistics (count the # of times something is mentioned in field notes as very rough estimate of frequency) Howard Becker Often enumeration is used to provide evidence for categories created or to determine if observations are contaminated. (from LeCompte and Preissle). 7. Event Analysis/Microanalysis (a lot like frame analysis, Erving Goffman) Frederick Erickson, Kurt Lewin, Edward Hall. Emphasis is on finding precise beginnings and endings of events by finding specific boundaries and things that mark boundaries or events. Specifically oriented toward film and video. After find boundaries, find phases in event by repeated viewing. 8. Metaphorical Analysis (usually used in later stages of analysis) Michael Patton, Nick Smith Try on various metaphors and see how well they fit what is observed. Can also ask participant for metaphors and listen for spontaneous metaphors.
Hallway as a highway.
Like highway in many ways: traffic, intersections, teachers as police, etc. Best to check validity of metaphor with participants -
member check
. 9. Domain Analysis (analysis of language of people in a cultural context) James Spradley Describe social situation and the cultural patterns within it. Semantic relationships. Emphasize the meanings of the social situation to participants. Interrelate the social situation and cultural meanings. Different kinds of domains: Folk domains (their terms for domains), mixed domains, analytic domains (researcher's terms for domains). • select semantic relationships • prepare domain analysis worksheet • select sample of field notes (statements of people studied) • look for broad and narrow terms to describe semantic relationships • formulate questions about those relationships • repeat process for different semantic relationship • list all domains discovered 10. Hermeneutical Analysis (hermeneutics = making sense of a written text) Max Van Manen Not looking for objective meaning of text, but meaning of text for people in situation. Try to bracket self out in analysis - tell their story, not yours. Use their words, less interpretive than other approaches. Different layers of interpretation of text. Knowledge is constructed – we construct meaning of text (from background and current situation - Social construction because of influence of others - symbolic interactionism) Use context - time and place of writing - to understand. What was cultural situation? Historical context. Meaning resides in author intent/purpose, context, and the encounter between author and reader - find themes and relate to dialectical context. (Some say authorial intent is impossible to ascertain.) Videotape - probably needs to be secondary level of analysis. Get with another person who is using another method and analyze their field notes. 11. Discourse analysis (linguistic analysis of ongoing flow of communication) James Gee Usually use tapes so they can be played and replayed. Several people discussing, not individual person specifically. Find patterns of questions, who dominates time and how, other patterns of interaction. 12. Semiotics (science of signs and symbols, such as body language) Peter Manning Determine how the meanings of signs and symbols is constructed. Assume meaning is not inherent in those, meaning comes from relationships with other things. Sometimes presented with a postmodernist emphasis. 13. Content Analysis (not very good with video and only qualitative in development of categories - primarily quantitative) (Might be considered a specific form of typological analysis) R. P. Weber Look at documents, text, or speech to see what themes emerge. What do people talk about the most? See how themes relate to each other. Find latent emphases, political view of newspaper writer, which is implicit or look at surface level - overt emphasis. Theory driven - theory determines what you look for. Rules are specified for data analysis. Standard rules of content analysis include: • How big a chunk of data is analyzed at a time (a line, a sentence, a phrase, a paragraph?) Must state and stay with it. • What are units of meaning?, the categories used. Categories must be: 1. Inclusive (all examples fit a category) 2. Mutually exclusive • Defined precisely: what are properties • All data fits some category (exhaustive) Also note context. Start by reading all way through, then specify rules. Could have emergent theory, but usually theory-driven. After determine categories, do the counting - how often do categories occur. Most of literature emphasizes the quantitative aspects. Originated with analyzing newspaper articles for bias - counting things in print. Very print oriented - can it be adapted for visual and verbal? 14. Phenomenology/Heuristic Analysis (phenomenological emphasis - how individuals experience the world) Clark Moustakas Emphasizes idiosyncratic meaning to individuals, not shared constructions as much. Again, try to bracket self out and enter into the other person's perspective and experience. Emphasizes the effects of research experience on the researcher-personal experience of the research. How does this affect me as researcher. Much like hermeneutical analysis, but even more focused on the researcher's experience. Some use the term
phenomenology
to describe the researcher's experience and the idea that this is all research is or can ever be (see Lofland and Lofland, p. 14). 15. Narrative Analysis (study the individual's speech) Catherine Reisman Overlaps with other approaches. (Is it distinctive?) Discourse analysis looks at interaction, narrative is more individual) The story is what a person shares about self. What you choose to tell frames how you will be perceived. Always compare ideas about self. Tend to avoid revealing negatives about self. Might study autobiographies and compare them. • context-situation • core plot in the story told about self • basic actions Narrative analysis could involve study of literature or diaries or folklore. Data Collection When evaluators have advanced to the point of planning the details of data collection, analysis must be considered again. Observations can be made and, if they are qualitative (that is, text data), converted to numbers in a variety of ways that affect the kinds of analyses that can be performed and the interpretations that can be made of the results. Therefore, decisions about how to collect data should be influenced by the analysis options in mind. Data Analysis After the data are collected, evaluators need to see whether their expectations regarding data characteristics and quality have been met. Choice among possible analyses should be based partly on the nature of the data—for example, whether many observed values are small and a few are large and whether the data are complete. If the data do not fit the assumptions of the methods they had planned to use, the evaluators have to regroup and decide what to do with the data they have.2 A different form of data analysis may be advisable, but if some observations are untrustworthy or missing altogether, additional data collection may be necessary. As the evaluators proceed with data analysis, intermediate results should be monitored to avoid pitfalls that may invalidate the conclusions. This is not just verifying the completeness of the data and the accuracy of the calculations but maintaining the logic of the analysis. Yet it is more, because the avoidance of pitfalls is both a science and an art. Balancing the analytic alternatives calls for the exercise of considerable judgment. For example, when observations take on an unusual range of values, what methods should be used to describe the results? What if there are a few very large or small values in a set of data? Should we drop data at the extreme high and low ends of the scale? On what grounds?