Did you learn everything you possibly could from your last project—not just from the final results, but by examining the process itself? Conducting a detailed review, a post mortem so to speak, can help pinpoint exactly what worked—and what needs work.
For example, let's say a recent project used in-person focus groups. Did you and your team discuss how the process went? What worked well about recruiting and what didn’t? Did the facilities and the moderator meet your expectations? Did the discussion guide support the intended goals? All great questions to reflect upon.
Good Stuff Happens in 1:1 Meetings: Why you need them and how to do them well
Post Mortems: The Anatomy of Market Research Process Improvement
1. POST MORTEMS: THE ANATOMY OF MARKET
RESEARCH PROCESS IMPROVEMENT
Did you learn everything you possibly could from your last project—not just from the final
results, but by examining the process itself? Conducting a detailed review, a post mortem so to
speak, can help pinpoint exactly what worked—and what needs work.
For example, let’s say a recent project used in-person focus groups. Did you and your team
discuss how the process went? What worked well about recruiting and what didn’t? Did the
facilities and the moderator meet your expectations? Did the discussion guide support the
intended goals? All great questions to reflect upon.
So, how do you conduct a project post mortem? By objectively evaluating the process, step-by-
step. But remember, the goal is not to place blame—it’s to find opportunities for improvement.
So hone your scalpels and focus your microscopes; come time for your next project, you’ll be
glad you did.
Let’s use the case of a quantitative project and break it down, stage by stage. For each stage, I
suggest a few key questions to get you started.
Stage 1: PROJECT DESIGN
Did the specification of project scope and objectives go smoothly? Was it challenging to get all
team members to agree on a reasonable, finite set of project goals? If an outside agency was
engaged, how did the process of writing an RFP and selecting and hiring the agency go?
Stage 2: QUESTIONNAIRE DESIGN
How many iterations were anticipated and how many were made? How did the programming
and pretest go? Were things discovered in the pretest results that needed significant
reworking? If there was a lot of reworking late in the process, could it have been avoided by
preliminary qualitative research or by clearer thinking about project hypotheses?
2. Stage 3: SAMPLE SELECTION AND SOURCING
Did the sample sources you planned on using work out? Were you satisfied with response and
incidence rates? What percent of records from a sample source did you have to toss because
they were suspicious?
Stage 4: DATA ANALYSIS
Was a clear data analysis plan in place so that everybody knew exactly what was going to be
done and how? Were there data analysis errors? Was there a lot of back and forth asking for
more cuts of data, which could have been avoided with better planning?
Stage 5: DELIVERABLES
Were deliverables created on time and error free? Were reports professional looking and well
received? Were there other types of deliverables that internal clients or colleagues would have
preferred?
And perhaps the most important measure of success: did people embrace the results, leading to
important conversations, decisions or strategies?
WRAP IT UP: THE REPORT CARD
Boil down takeaways into a single page summary, and keep it factual. Avoid sneak attacks.
Nobody should dread report card time. You’re simply identifying improvements for next time
around. This is your post mortem deliverable, an at-a-glance reference for next time.
Here are some options:
Self-assessment Report Card. In this case, the project team grades the project by phase. A
weighted scorecard approach can be used by considering grades for items such as budget
adherence, timeline adherence, usefulness of final results, and so on.
Client feedback. Collect client feedback (from internal or external clients). Ask for feedback on
overall satisfaction and on specific projects’ aspects such as communications, schedule, and
deliverables. How well did the project results and deliverables meet expectations? Did the
results sufficiently support intended decisions and strategies? If not, why not? Giving clients a
chance to grade the project can uncover some surprising areas for improvement; some clients
won’t complain along the way, but sure do have feedback to give when asked. Get it promptly,
and address it, so that they won’t turn into market research skeptics.
3. Whatever approach you take, formal or informal, self-assessed or client feedback-based, you will
certainly find opportunities for improvement. The key? Capture post mortem information before
rigor mortis sets in, within a week of the project’s conclusion or sooner. Projects over three
months long may call for a midway mini-review.
[If you haven’t done so already, click HERE to sign up for Research Rockstar’s Market
Research Newsletter.]
Author Bio:
For more expert tips on market research subscribe to Research Rockstar's free newsletter
at http://www.researchrockstar.com.
Free Membership Pass: Get unlimited access to Research Rockstar's mini-courses, newsletters, e-
books, and RFP templates: http://www.researchrockstar.com/amember/signup.php.
Have a market research question? Kathryn Korostoff is president of Research Rockstar, a
company that delivers online and in-person training to busy professionals seeking market
research excellence. Kathryn is a market research professional with a special interest in how
organizations acquire, manage, and apply market research. Over the past 20 years, she has
personally directed more than 600 primary market research projects and published over 100
bylined articles in trade magazines. http://twitter.com/ResearchRocks