Presentation by Eric Barends and Rob Briner on the outcomes of the CEBMa / CIPD REA on performance appraisal (CIPD Applied Research Conference, 2016)

1. Performance Appraisal & Workplace Performance
CIPD, Applied Research Conference
19 December 2016, London

2. Postgraduate Course

3. Postgraduate Course

5. CEBMa: what we do
 Promote (seminars, papers, blogs, tweets)
 Educate (universities & business schools)
 Train & coach (companies > projects)
 Conduct (and teach) REAs

6. EBP: what is it?
Evidence-based practice
Central Premise:
Decisions should be based on
the ‘best available evidence‘.

7. Best available =
critically evaluated evidence
from multiple sources
EBP: what is it?

8. Evidence?
information, facts or data
supporting (or contradicting)
a claim, assumption or hypothesis

9. Sources of evidence
problem solution
Practitioners
professional expertise
Organization
internal data
Stakeholders
values and concerns
Scientific literature
empirical studies
Ask
Acquire
Appraise
Aggregate
Apply
Assess

13. Claims

14. Claims

15. Claims

16. Claims

17. Claims

19. Sources of evidence
problem solution
Practitioners
professional expertise
Organization
internal data
Stakeholders
values and concerns
Scientific literature
empirical studies
Ask
Acquire
Appraise
Aggregate
Apply
Assess

20.  ABI/INFORM: more than 5,000 publications
(BTW, most of it is crap)
Problem 1: volume

21. Problem 2: incomprehensible

22. Problem 3
It depends …

23. GREAT! NOW WHAT?

24. We need evidence summaries!
Systematic Review (SR)
The agreed-upon standard approach to scientific evidence
assessment across fields as diverse as medicine, education,
criminal justice etc. and follows a transparent structured
methodology for reviewing an entire body of relevant research
evidence

25. Rapid Evidence Assessment (SR light)
 Searching: consulting a limited number of databases, and
excluding unpublished research. Sometimes an REA may be
limited to only meta-analyses or controlled studies.
 Data Extraction: only extracting a limited amount of key
data, such as year, population, sector, sample size,
moderators/mediators, main findings, and effect size.
 Critical Appraisal: limiting quality appraisal to
methodological appropriateness and methodological flaws.
We need evidence summaries!

27. Step 1: ASK
What is known in the scientific literature about the
impact of performance appraisal on workplace
performance?
1. What is performance appraisal?
2. How is it supposed to work?
3. What is the effect on workplace performance?
4. What is known about possible moderators and/or mediators
5. What is known about the reliability and validity?

28. Step 2: ACQUIRE
Search for the best available scientific evidence

29.  ABI, BSP, PsycINFO
 Scholarly journals, peer reviewed
 English, scholarly journals, peer-reviewed
 1980 to 2016 for meta-analyses; 2000 to 2016 for
primary studies
 Key words: performance appraisal, performance review,
performance evaluation, annual review, and employee
evaluation.
Step 2: ACQUIRE

31. step 3: APPRAISE

32. step 3: APPRAISE

34. Oh, and we never look at the p-value,
only at the effect sizes

36. Q1: What is it?
‘An evaluation process in which quantitative scores based
on pre-determined criteria are assigned and shared with
the employee being evaluated’
(e.g. DeNisi & Pritchard, 2006)

37. Q2: How does it work?

38. Social comparison theory: individuals tend to compare
themselves with others in order to make judgments
regarding their performance.
They are concerned not only about their performance in
an absolute sense, but also about how they measure up in
relation to relevant peers.
Q2: how is it supposed to work?
Feedback intervention theory: suggests that when
confronted with a discrepancy between what they wish to
achieve and the feedback received, individuals are strongly
motivated to attain a higher level of performance

39. Q3: Does it work?

40. Q4: Moderators and mediators?

41. Q4: Moderators and mediators?
1. Reactions to feedback, rather than the feedback itself, influence
performance (level A)
2. The perceived fairness of the performance appraisal process has a
medium to large moderating effect on future performance (level A)
3. Negative feedback adversely affects perceived fairness (level C*),
whereas feedback that focusses only on positive aspects has a medium
positive effect on both perceived fairness and overall job performance
(level A*)
4. Participation has a medium to large moderating effect on perceived
fairness (level B).
5. The quality of the relationship between manager and employee has a
substantial moderating effect on the perceived fairness of the
performance appraisal (level B)

42. Q5: Reliability & validity?

43. Q5: Rater centric errors
1. Employees’ contextual performance has a large positive effect on job
performance ratings (level A).
2. Managers’ implicit person theory regarding the malleability of personal
attributes has a large effect on how they rate their employees (level A).
3. Rater training has medium to large positive effects on rating accuracy
(level A).
4. The outcome of managers’ own performance appraisal has a large effect on
the way in which they evaluate their employees (level A).
5. Male employees who experience a conflict between family and work
receive lower performance ratings (level A)

44. Q5: Ratee centric errors
1. Employees’ tactics for influencing raters such as ingratiation or self-
promotion have a moderate effect on performance ratings (level C).
2. Employees’ organizational citizenship behaviour has a moderate to large
positive effect on performance ratings (level A).
3. Employees’ political skills have a small positive effect on performance
ratings (level C).

45. Q5: System centric errors
1. The purpose of the performance (administrative vs developmental)
moderates the performance rating (level A).
2. The reliability of individual performance measures depends on the type
of measurement, the source of measurement and the level of job
complexity (level A/B).
3. Accountability substantially affects both rating outcomes and rating
accuracy (level A).

46. Step 4: Synthesize

47. Step 5: Apply

48. Reactions
Who
knew?