Use of case pairs can potentially improve the efficiency and effectiveness of radiology residency

MHPE 2009 - 2011 Poh-Sun Goh, i665606
1
Master of Health Professions Education 2010 – 2012
Maastricht University
Unit 11
Master Thesis
Title: Use of case pairs can potentially improve the efficiency and effectiveness of radiology residency
Date
12 April 2012
Name of Student – ID number
Poh-Sun Goh – i665606
Supervisor: Ellen Kok
Co-supervisor: Anique de Bruin
Acknowledgements: Grateful thanks to Ellen Kok for much excellent advice and commitment to this project;
Prof Jan van Dalen for very helpful feedback including advice on the experimental design during Unit 7 in Maastricht,
and as part of Unit 10; and to fellow student Nynke de Jong for helpful feedback during Unit 10 assignments.

MHPE 2009 - 2011 Poh-Sun Goh, i665606
2
Table of Contents
Summary Page 3
Preface Page 4
A. Introduction Page 5
Establishing significance
Previous research and contributions
Gap/problem/questions/prediction
Research question
B. Methods Page 8
Setting
Participants
Materials
Apparatus
Procedure
Data Analysis and Statistical Analysis
C. Results Page 11
D. Discussion Page 12
E. Conclusion Page 14
F. References Page 15
G. Appendices Page 18
Figures 1 to 10 (on sequential separate pages)

MHPE 2009 - 2011 Poh-Sun Goh, i665606
3
Summary
In this Master Thesis, the topic to be investigated is a method to improve the efficiency and
effectiveness of residency training in radiology in order to potentially shorten residency training while
maintaining effectiveness, in order to increase the numbers of trained radiologists.
The research question to be investigated is whether compare and contrast case review with pairs of
cases is more time efficient and effective compared with traditional sequential case review with single
cases in radiology problem solving.
The research question was investigated by the use of a quantitative experimental design – an
independent groups (or between groups) design. 10 residents participated with a within subjects design
allowing the 10 residents to take part in both the control and experimental conditions with crossover
between the groups. Resident performance with 30 CXRs and 30 CT head scans was compared between
the experimental and control group, in a pretest, practice and posttest phase each with 10 different cases,
with all 30 cases examples from a single key clinical diagnostic category for each type of examination.
There was a consistent trend in time efficiency for the experimental group compared with the control
group, with a medium effect size comparing pretest times (d = 0.48), and a relatively smaller effect size
comparing times taken for the posttest phase (d = 0.33). A strong effect size, with statistically significant
difference in performance was also found in the pretest phase of the study favoring the experimental
group (d = 1.06). The control group comparing pretest with posttest also demonstrated a learning gain.
Though a learning effect was not demonstrated for the experimental group, this group was found to have
reached a high (good pass) level of performance in the pretest, with no significant change in performance
for the posttest.
The results of this Master Thesis project provide some support for the experimental hypothesis,
that compare and contrast case review of pairs of cases, is more time efficient than, and at least as
effective as sequential single case review, in interpreting radiology cases, and potentially in learning
radiology for residency training. There is also some support for the effect of deliberate practice.
The control group showing learning gain between the pretest and posttest phases is consistent with
the expected effect of traditional radiology residency training, with one case at a time case presentation
and review. The experimental group showing not only more efficient, but also more effective problem
solving, with moderate and strong effect respectively, in the pretest phase, raises the interesting
possibility that a much fewer number of cases might be required for training for any one diagnostic
category if paired presentation and review is systematically used.
Key words: Deliberate practice, paired and mixed practice, effectiveness, efficiency, and radiology
residency.

MHPE 2009 - 2011 Poh-Sun Goh, i665606
4
Preface
How do you learn to recognize the appearance of a dog, or a cat? A common method is to look at an
example of a dog, then an example of cat, together with a description of the key features of each animal,
followed by looking at an illustration of a dog and cat side by side. This is an example of sequential case
review, followed by contrasting case review of a pair of cases.
I would like to begin with a personal story, to explain my decision to undertake this topic of
investigation for the Master Thesis. In 2008, at the AMEE conference in Prague, I had the opportunity to
sit in the audience and listen to a plenary lecture by Professor Geoff Norman on the topic of teaching for
transfer. In his presentation, Professor Norman reviewed the evidence for deliberate practice, by quoting
the work of Anders Ericsson; and he also mentioned examples from his own work, including an early
study on the role of compare and contrast practice in teaching Chest radiograph interpretation (Anders-
Ericsson, 2004; Hatala, Brooks & Norman, 2003; Norman, 2008; Norman, Young & Brooks, 2007).
The implications of this work resonated strongly with me, and I nearly leapt out of my chair with
excitement. It was clear to me that many educators in radiology were unaware of these ideas, and
certainly have not taken these ideas further into educational practice, and I sensed a great opportunity,
not only as a budding researcher, but also as a teacher practitioner in radiology.
Using similar and contrasting case examples to highlight a point is a common practice by radiology
teachers whenever we try to explain difficult concepts to residents. This practice is also commonly used
whenever a radiologist encounters a difficult case. We immediately search for a comparable example to
confirm that a feature is a variation of normal, or the manifestation of an abnormality. We do this by
either searching other areas on the current image, retrieving a previous radiograph or scan to compare,
or search the literature for typical examples of a feature, and attempt a side-by-side comparison. It is
clear from empirical practice that compare and contrast case review facilitates diagnosis. The question
is whether this practice aids learning?
Despite the evidence in the educational literature, the systematic use of a compare and contrast
strategy, under conditions of deliberate practice, has not been implemented routinely for radiology
teaching. This has probably been due to both a lack of awareness of the potential of these ideas, and to
the limitation up till recently, of the lack of material for individual radiology teachers, encompassing
sufficient examples of the whole range or spectrum of the manifestations of each disease, to use this
teaching method as a core teaching strategy.
With the advent of digital repositories, and the increasing practice of radiology teachers archiving
teaching material in digital format, which can be shared, the limitation of availability of case material is
now potentially no longer the case. The challenge is to validate these educational ideas, and to assemble
or make accessible collections of case material. After AMEE 2008, I proceeded to first systematically
build up and organize a collection to teaching material, which demonstrates the spectrum of disease
presentation in two areas of radiology that I teach and practice in (the Chest radiograph and CT scan of
the brain). I then completed several pilot studies with small groups of medical students, and radiology
residents, in both an undergraduate, and postgraduate setting to test the hypothesis, that deliberate
practice with paired cases (allowing compare and contrast review) is more effective and efficient than
sequential practice with single cases in teaching and learning radiology (Goh, 2009, 2010 and 2011).
The positive results of these empirical qualitative studies led to the proposal, and design of this formal
quantitative study, as a Master Thesis project. I will now present the formal introduction to the Master
thesis, which will elaborate on the background to this Master Thesis project.

MHPE 2009 - 2011 Poh-Sun Goh, i665606
5
A. Introduction
Training of radiologists is a long and expensive process, taking an additional 6 to 8 years after
medical school. The bulk of this time is spent building up clinical experience and diagnostic expertise by
sequential exposure to case material with supervision and feedback.
There is a pressing need to increase numbers of radiologists due to massive growth in the demand for
radiological imaging, resulting in a worldwide shortage of trained radiologists (Royal College of
Radiologists, 2012). Measures used to address this manpower shortage have been to increase the number
of training places in radiology training programs, systematic use of innovative eLearning teaching
methods, the development of more structured shorter training programs, and by reducing the period of
specialist training to 48 months or 4 years (Specialist Accreditation Board, Singapore, 2012).
Implementation of recommendations similar to the European Time Directive furthers reduces the time
available for specialty training (Roedling, Robinson, Lough-White, & Miller, 2008).
One key requirement in the quality specifications for these programs is the requirement for the
exposure of the radiology resident to a minimum number of radiological examinations – 7,000 per year;
with the implicit idea that clinical experience by case exposure is required for developing expertise
(ACGME-I, 2010). The key assumption is that the more cases a resident is exposed to, the greater is their
experience, which should lead to greater expertise.
However, with the increasing numbers of residents, there is consequently reduced clinical case
exposure per resident. The reason is that more residents share the same case volume in each training
centre, with reduced opportunities for involvement by each resident in direct reporting or radiological
interpretation, assuming that each clinical case is reviewed by one resident. The challenge in providing
training experiences for these residents is improving the effectiveness and efficiency of the training and
learning process, to mitigate the reduction in training duration and reduction in clinical exposure.
Traditional teaching of radiology with exposure to clinical teaching material in mentored diagnostic
sessions with an experienced teacher is usually opportunistic and haphazard in nature, as it is based on
clinical case material that is encountered prospectively on a daily basis. To address this problem,
systematic thematic teaching sessions and self study with published materials, film libraries and recently
online repositories of case material have been used by radiology trainees to broaden their clinical
experience and develop a comprehensive diagnostic ability. Clinical case review, and self-study is
usually done with sequential case review, i.e. one case at a time. Due to reduced training time, any
method to improve the effectiveness and efficiency of this process will be welcome by both residents,
and teaching faculty. This is where the idea of the use of case pairs, to improve effectiveness and
efficiency of training, can potentially be applied.
The purpose of this Master Thesis project is to investigate a method of reducing training time, and
making the time used in training more effective, by the systematic use of deliberate practice of sets of
radiological cases; and use of compare and contrast practice with pairs of cases, rather than traditional
sequential case presentation and review. This idea takes advantage of a common diagnostic strategy in
day to day radiology problem solving to compare a suspected abnormal area with an equivalent normal
area on the same radiological image, or another comparable scan.

MHPE 2009 - 2011 Poh-Sun Goh, i665606
6
Radiology Training and Review of Literature
Training of radiologists has historically followed an apprentice progressive accumulation of
experience model. Skill as a radiologist requires development of the ability to recognize and differentiate
between visual representations of different diseases as displayed on plain radiographs and cross sectional
imaging (Wood, 1999). Recognition of key abnormalities on radiology images is a core skill in clinical
radiologists. Traditional teaching involves presentation of a concept followed by one or more examples,
followed by the next concept and more examples (Avrahami et al., 1997). Educational literature contains
evidence that suggests this is not the most effective or efficient way of teaching (Norman, 2005). The
literature suggests that deliberate practice with paired and mixed examples is a more effective and
efficient method for teaching and learning image recognition and discrimination (Anders-Ericsson, 2004;
Hatala, Brooks & Norman, 2003; Norman, 2008; Norman, Young & Brooks, 2007).
Deliberate practice refers to a consciously planned and executed program of repeated focused
exercise of the skills required for expert performance accompanied by appropriate feedback (Anders
Ericsson, Krampe, & Tesch-Romer, 1993). Paired and mixed practice involves the use of a series of
similar and contrasting examples of imaging abnormalities, with an attempt to describe and reflect on
similarities, and differences between the similar and contrasting examples (Norman, 2008).
There have only been a few published research studies to date on the application of compare and
contrast practice, with dermatology problems, ECG strips, and paediatric chest radiographs (Hatala,
Brooks & Norman, 2003; Norman, 2008; Norman, Young & Brooks, 2007). It has been suggested that
paired practice promotes the ability to recognize key diagnostic features, and the ability to elaborate on
these features (Markman & Gentner, 1993). Mixed practice with contrasting cases may promote schema
acquisition and analogous transfer (Gick & Patterson, 1992). Combination of paired and mixed practice,
or use of similar and contrasting examples, helps in developing and refining categorical knowledge
(Namy & Clepper, 2010).
Radiology diagnostic practice requires the radiologist to see or detect, recognize and interpret
(Krupinski, 2010; Nodine & Mello-Thomas, 2010). Two separate cognitive tasks are required as a
clinical radiologist - image or pattern recognition, which has been described as non-analytical thinking or
exemplar thinking; and the ability of list key distinguishing and discriminating features, which has been
described as analytical thinking or rule-based thinking (Norman, 2005). Professionally, a radiologist has
to state a diagnosis (image recognition), and then describe and justify the diagnosis, by listing key
features as a radiology opinion or written report.
Learning radiology can be therefore seen to involve a process of image or pattern recognition and
discrimination (Koontz & Gunderman, 2008). There is a progressive improvement in performance with
increasing experience in daily practice as a trainee and practicing radiologist involving exposure to
clinical material (radiology images) with feedback (Helberg Engel, 2008).
A combination of deliberate practice and increasing clinical experience accelerates the visual
perception and recognition process, and this decreases the amount of purposeless visual searching of an
image for diagnostic information (Kundel & Nodine, 1975). Building up schemas in long term memory
accelerates image or pattern recognition, and facilitates the global impression and holistic recognition
that is the predominant visual and thinking process of experienced radiologists (Nodine & Mello-
Thomas, 2010, pp. 144-145).
It is common practice to use a compare and contrast strategy in solving individual day-to-day clinical
problems, and this strategy facilitates diagnosis (Berbaum, Franken & Smith, 1985). Use of pairs of

MHPE 2009 - 2011 Poh-Sun Goh, i665606
7
cases would be expected to result in solving individual radiology case problems in less time, and with
equal or better accuracy when compared with single case at a time problem solving; or solving problems
with greater accuracy, in the same amount of time. The availability of pairs of cases would be expected
to therefore improve performance in radiological problem solving or diagnosis, and thereby potentially
improve the efficiency and effectiveness of radiology training.
Efficiency in the context of this study refers to the achievement of learning objectives in the least
amount of time and/or achievement of significantly increased learning compared with traditional
learning methods in the same amount of time, and effectiveness refers to the successful achievement of
learning objectives (Rumble, 1997). For the purpose of this study, this refers to the ability to recognize
an abnormality, and to discriminate between different clinical conditions on imaging and make an
argument for one or other diagnosis by describing positive and significant negative findings.
The gap in the educational and radiology literature is a systematic attempt to evaluate the potential of
deliberate practice using compare and contrast paired cases compared with traditional sequential case
review in radiology training. It is the intention of this Master Thesis project to attempt to investigate this
topic.
Research Hypothesis
The research hypothesis of this Master Thesis study is that deliberate practice using pairs of similar
and dissimilar contrasting examples of imaging (radiographic) abnormalities is a more efficient and
effective method of radiology training compared with traditional sequential single case review, with
exposure and review of concept followed by examples one case at a time.
The research question to be investigated is whether compare and contrast case review with pairs of
cases is more time efficient and effective compared with traditional sequential case review with single
cases in radiology problem solving.
Experimental Design
The research question was investigated by the use of a quantitative experimental design – an
independent groups (or between groups) design (Mulhern & Greer, 2011, pp. 109, and pp. 174 -175).
Numerical data for diagnostic performance in interpreting CXR and CT scans (the dependent variable
– effectiveness and efficiency of learning radiology measured by accuracy of diagnosis and time taken
during deliberate practice sessions) are collected for two separate groups – the sequential practice group
or control group, and the compare and contrast paired practice group, the experimental group (the
independent variable).

MHPE 2009 - 2011 Poh-Sun Goh, i665606
8
B. Methods
Setting of study
The setting of this study is the Department of Diagnostic Radiology at the National University
Hospital in Singapore. During any 6-month rotation period, approximately 30 residents in training will be
present in our department. These residents undergo a national residency rotation system to broaden their
clinical exposure amongst the three main teaching hospitals. Residents in the training program rotate every 6
months between the 3 teaching hospitals. We perform over 300,000 radiology examinations a year and
provide subspecialty training within a structured residency program characterized by guided supervised
practice by experienced radiologists.
Participants
The participants in this research project were 10 residents comprising a matched group (with equal
length of residency and experience) of 6 senior residents (who are in their 3rd
year of residency), and 4 junior
residents (in their 1st
year of residency). There were an equal number of male and female residents, with a
similar background having all attended medical school and an additional 2 years of clinical practice
experience after graduation including interpreting basic Chest radiographs and CT scans of the brain before
entering radiology residency training, with an age range between 26 and 30 years of age. Their participation
in the “experiment” took place over one month in January 2012 during their weekly scheduled academic
learning sessions (scheduled personal learning time consisting of one afternoon per week off clinical duties),
in our departmental library, using computers to present teaching material, with one computer terminal per
resident.
The involvement of the residents in the study was voluntary, and residents were free to discontinue
their participation at any stage if they felt their participation did not meet their individual learning needs,
which did not occur. The residents were informed that participation in the program provided additional
opportunity for structured practice and learning, and can be considered as part of their learning activities, but
they were not informed of the underlying structure or hypothesis of the research study, so that there would
not be any performance expectation or bias. They were reassured that the results of the study were
anonymised, and the residents were provided with individual feedback on their performance, which was
given to each resident at the end of each session. The department chairman and postgraduate training
committee approved this study.
Materials
The 30 CT and 30 CXR cases used were selected from an online repository containing over 1000
examples of Chest Radiographs (CXRs) and 1000 CT scans of the brain (CT brain) reflecting day-to-day
practice in our academic medical center, and are of moderate difficulty. A conscious decision was made to
select cases on the more difficult end of the spectrum for the posttest compared with easier cases in the
pretest phase of the study. The learning objective of the CT cases was the diagnosis of acute ischaemic
strokes and its mimics, and the CXR cases pneumothoraxes and free abdominal air beneath the diaphragm.
CXRs and CT scans are 2 of the commonest radiological examinations, and the conditions chosen are 2
categories of the most important to accurately diagnose in clinical practice.

MHPE 2009 - 2011 Poh-Sun Goh, i665606
9
How to ensure credibility, reliability and validity
To ensure credibility and replicability, the assessment items as well as all teaching and practice cases
are available for peer review online. By making the case material content as well as the methodology
available not only for local but external peer review, this allows the possibility of reproducing the research
study in other sites, promoting reliability, replicability and credibility of this study (Bryman, 2009, pp. 31-
32). To further ensure reliability of the testing process, and validity of the case items, for difficulty level and
appropriateness for radiology residents, these case items were tested on several cohorts of novice residents,
and senior residents to provide a performance benchmark, during the prestudy preparation phase on all
pretest and posttest cases in November and December 2011. As these residents rotate through the department
at 6 monthly intervals, a different group of residents was used for the study phase in January 2012.
The assessment instrument (pre-test and post-test items), and learning process (learning and practice
case selection) was validated (measurement validity) by agreement with three members of the departmental
postgraduate training committee (all senior radiology instructors), to enhance the content validity or direct
validity of the experimental design (Schuwirth, & van der Vleuten, 2011).
Apparatus
A dell PC workstation with a single widescreen 22 inch LCD monitor (Dell U2211 22 inch
Widescreen IPS Panel LCD Monitor), keyboard and infrared mouse was used for the experiment. Microsoft
Powerpoint version 2010 software was used to present the cases, and the “rehearse Powerpoint show”
function with onscreen digital timer which resets as each slide is advanced was used to display the time
taken with each case or pair of comparison / contrasting cases. The experiment took place in our department
library (Figure 1). Each resident entered answers for the cases in a hardcopy workbook with separate sheets
for each case (Figure 2).
Procedure
Experimental Design
The study followed a “classical” quantitative experimental design using 2 groups with “crossover”
between the 2 groups. 10 residents participated with a within subjects design allowing the 10 residents to
participate in both the control and experimental conditions (Bryman, A. pp.37) (Figure 3). In the first 2
weeks, a “control group” of 5 residents (group A) used sequential practice (Figure 4), and an “experimental
group” of 5 residents (group B) used case pairs (Figure 5 and 6). In second half of the month, a “cross-over”
was undertaken with a change in type of examination from CT scans of the head cases to CXR cases, with
the difference where group A now used case pairs, and group B used sequential cases (Figure 7). The
scheme for the crossover is outlined in Figure 10.
The 10 residents were randomly assigned to the experimental and control group based on the
alphabetical order of their first names, in order for each group to have 2 junior, and 3 senior residents. In
addition to the crossover, the order of case items was randomized for each resident to minimize ordering
effects. A total of 30 cases were presented, with a single radiological image for each case. Each test item was
presented without further clinical information to avoid any additional contextual learning – recall effects.
This is because the experimental hypothesis focused on visual perceptual learning – recall – and the
deliberate practice of this.

MHPE 2009 - 2011 Poh-Sun Goh, i665606
10
The case presentation sequence of pretest (10 cases), learning phase (answers for pretest), practice
phase (10 cases followed by answers) and posttest (10 cases) differed for the control and experimental
group, with the control group undergoing sequential case presentation (Figure 4), while experimental group
undergoes paired and mixed presentation with twin sets of radiological cases presented side by side, with a
total of 5 pairs of similar and 5 pairs of contrasting cases in the learning and practice phases (Figure 5 and 6,
8 and 9). Compare and contrast paired case presentation of similar and contrasting cases offers pairs of cases
(out of a total of 20 cases) similar to the total number of cases for sequential case presentation. Of the 20
cases in the learning and practice phase, there are 5 similar pairs (10 cases), and 5 contrasting pairs (10
cases).
Each resident participated individually. For example, the steps taken by each participant are
described in sequence as follows:
Each session took place in the departmental library, one resident at a time, with departmental
administrative staff in the adjoining room to maintain “privacy” of resident (there were no interruptions
during the course of each training session). Each session was self-paced by the resident advancing the
PowerPoint slides, with the time for each case recorded by the resident in the workbook. At the end of the
session, the answers for the posttest phase were given to each resident during a feedback session with the
principal investigator. For each pair of cases the time taken for the second case entered in workbook
included the time taken for first case of the paired set (as the timer resets only when the slide is advanced).
The actual time taken for every second case was obtained by subtracting the entered time from the time
taken for the first case for each pair of cases by the investigator.
In all phases, for both groups, residents were challenged for each case to write down a single
diagnosis in the workbook, followed by the reasons for the diagnosis (by listing key discriminating and
differentiating features) (Figure 2). For paired or mixed cases, a diagnosis had to be given for each one of the
cases separately. The pre-test and post-test cases follow a similar format and were scored with 0.5 for a
correct diagnosis, and 0.5 for correct key features, with maximum of 1 for each item, and a maximum score
of 10 (for each pretest or posttest phase). The principal investigator using a marking template, blinded to the
resident cohort during marking, performed the marking.
Data Analysis and Statistical Analysis
After collection of data, a comparison between the 2 groups was made by comparing averages of
scores for each resident, and times for each resident (means) (Mulhern & Greer, 2011, pp. 56-59).
For example, the difference in the mean of the time taken to complete the pretest or posttest phases,
comparing the experimental group with the control group, is a measure of efficiency. The difference between
the pretest scores, comparing the experimental with control groups, is a measure of diagnostic performance.
Statistical analysis of the data used a student’s t-test comparing the time taken, and performance
between the experimental and control groups. The t-test was initially performed using an online calculator
(http://www.physics.csbsju.edu/stats/t-test_bulk_form.html) (Kirkman, 1996) with final calculations made
with the SPSS 17 software package. The effect size was also calculated, by using an online calculator
(http://www.uccs.edu/~faculty/lbecker/) (Becker, 1999). For interpreting Cohen's d, “an effect size of 0.2 to
0.3 might be considered a "small" effect, around 0.5 a "medium" effect and above 0.8, a "large" effect”
(Cohen, 1988; Walker, 2008).

MHPE 2009 - 2011 Poh-Sun Goh, i665606
11
C. Results
A strong effect, with significant difference in performance was found in the pretest phase of the
study, with the experimental group (M = 7.5, SD = 0.85) performing better than the control group (M = 5.9,
SD = 1.60), t (9) = 2.89, p = .019. The effect size comparing pretest performance was large with d = 1.06.
The control group demonstrated a higher learning gain than the experimental group comparing pretest
scores with posttest scores (with mean gain of M = 0.90, SD = 2.16). The experimental group did not show a
learning effect with a negative gain (M = - 0.4, SD = 1.07), however the experimental group reached a
relatively high (good pass) level of performance in the pretest with no significant deterioration in
performance in the posttest. There was no significant difference in the learning gain between the 2
conditions with t (9) = 1.62, and p = 0.14.
The experimental group (M = 780.9 seconds, SD = 333) also appeared to be more time efficient, taking
less time than the control group (M = 939.5 seconds, SD = 329.8) comparing the mean of the time taken for
the pretest for the CT and CXR cases. Though this difference was not statistically significant with t (9) =
1.11, p = .30, there was a medium effect size seen comparing total time taken with d = 0.48. For the practice
phase, the experimental group also took a shorter time (M = 777.9 seconds, SD = 311.1) compared with
control group (M = 905.2 seconds, SD = 413), t (9) = 0.66, p = 0.53. This consistent trend of the
experimental group taking less time was again seen in the posttest (M = 928.9 seconds, SD = 364.5 seconds),
compared with the control group (M = 1060.7 seconds, SD = 431.1 seconds). Though this difference was not
statistically significant with t (9) = 0.79, p = .45, there was a small effect size also demonstrated with d =
0.33.

MHPE 2009 - 2011 Poh-Sun Goh, i665606
12
D. Discussion
What might explain the results of the study?
There is a consistent trend in time efficiency for the experimental group compared with the control
group in the pretest, practice and posttest phases, with the greatest difference seen in the pretest phase with
medium effect size. Having the comparison case appears to speed up decision-making, hence the more rapid
completion of cases by the experimental compared with the control group (Gick & Patterson, 1992; Hatala,
Brooks & Norman, 2003; Markman & Gentner, 1993; Namy & Clepper, 2010; Norman, 2008; Norman,
Young & Brooks, 2007). In my professional practice, having a comparison case improves my own
diagnostic certainty, and I can appreciate why this would reduce decision making and problem solving time.
A strong effect size, with statistically significant difference in performance was found in the pretest
phase of the study. The performance advantage for the experimental group, particularly in the pretest phase
of the study, comparing the performance of the experimental with the control group suggests that compare
and contrast review of pairs of cases facilitates diagnosis, and can potentially also have a learning effect
(Berbaum, Franken & Smith, 1985).
The control group comparing pretest with posttest also demonstrated a learning gain. Though a
learning gain was not demonstrated for the experimental group, this group was found to have reached a high
(good pass) level of performance in the pretest, with no significant change in performance for the posttest. It
could be argued that “a ceiling effect” in performance (potentially learning) occurs with the CXR and CT
cases relatively quickly, with the paired practice group achieving this quicker due to the availability of a
comparison similar or contrasting case (Markman & Gentner, 1993; Namy & Clepper, 2010). This again
shows the greater performance efficiency of the experimental group.
What are the limitations of the study?
There are several potential limitations to the generalizability of the results of this study, and potential
confounding factors in this experiment. While the crossover design between the CT and CXR problems
reduces the problem of potential unequal ability between the control and experimental groups, as the control
group becomes the experimental group; and vice versa between the CT and CXR problem experimental
observations; the use of CT and CXR problems which have not been matched for difficulty is a weakness in
the experimental design.
It may also be pointed out that the CT scan and CXR are different types of image, and are not
equivalent types of examination. The purpose of this study however is to train residents to pick up
abnormalities on radiological images, with a selected single relevant section of the CT scan used for each
case, and also single CXR for each case. As the task accurately reflects actual diagnostic clinical practice, it
could be argued that there is equivalency in the use of single CT brain images and single CXRs.
Another limitation is the relatively small number of residents, and residents with variable clinical
experience and clinical skill. This limitation was partly compensated for by choosing a spectrum of residents
of differing experience, from novice to relatively experienced, who were randomly and equally matched.
A further limitation to the generalizability of the results to broader radiology training is the limited
range of conditions evaluated. Though the range of problems tested was limited, it can be argued that having
both Chest radiograph and cross sectional CT cases, and showing variations and look alike examples of key
clinical diagnoses provides enough variation of visual problem examples, and helps increase the

MHPE 2009 - 2011 Poh-Sun Goh, i665606
13
generalizability of this study to diagnostic problems in radiology. Having the residents perform multiple
cases also increases the statistical power of the analysis (20 cases combining both the Pretest and Posttest
phases per resident).
While this study shows some support for the research question, and the research hypothesis grounded
in educational theory, we have not evaluated how the training effect has occurred. While we have shown that
the experimental group has a consistent trend with medium effect size being more time efficient while
performing as well as the control group, more needs to be done to discover why this effect has occurred.
Future qualitative and more detailed quantitative studies can be designed to explore this question.
What are some of the implications of this study for future research, and on teaching practice?
This study has shown some support for the use of deliberate practice in radiology training. There was
great willingness by the residents to participate in the study, and to complete both limbs of the study, which
was completely voluntary, due to the perceived learning value that the residents obtained from participation.
Half of the residents went on to perform up to 4 more sessions with additional sets of CXR and CT cases
under similar conditions. Ad hoc qualitative feedback from one resident is quite revealing about the
perceived benefits of deliberate practice in residency training – “60 minutes of deliberate practice felt like,
and has the teaching and learning value of one week of clinical experience, or even two weeks!!”
There are many implications for radiology training, and presentation of teaching materials, if the
trends in this paper are validated with larger groups of radiology residents, with a wider range of plain
radiographic, and cross sectional imaging.
The findings in this paper suggest that one method for improving learning or teaching radiology is
for a resident to engage in deliberate practice with pairs of similar and contrasting cases, after initial
presentation of exemplar case examples with descriptions of key features. This practice could reduce training
time, to an extent dependent on residency program design, and resident motivation factors, while
maintaining or improving learning and performance.
The requirements for implementing the principles of deliberate practice, with review of a wide
enough spectrum of case pairs, is a comprehensive collection of radiology cases, ideally in electronic format,
covering the whole spectrum of presentation of each radiology disease condition. This collection would
include look alike conditions. This would require at least 30 to 50 examples of each condition to allow the
full spectrum of each disease to be covered, and facilitate deliberate practice with not only exemplar cases,
but also more subtle cases at the tails of the normal distribution (Pusic, Pecaric & Boutis, 2011). This effort
would require the coordinated effort of an institutional team of radiology educators, or a multi-institutional
coordinated effort. The rewards of this initiative in reduced residency time, and potentially improved
residency learning, would be potentially significant, and could justify the time, effort and expense of this
process.
The presentation of teaching materials, in both print, and online formats, can also be improved to
take advantage of these ideas. For example, after presentation of a series of typical cases, and description of
their key diagnostic features, authors can then present a series of similar and contrasting pairs of cases, to
amplify the key diagnostic features of each disease entity. The use of online case repositories which provide
a wider range of the spectrum of diagnostic findings of each disease entity can be used to supplement print
formats, again with the material available for compare and contrast deliberate practice.

MHPE 2009 - 2011 Poh-Sun Goh, i665606
14
E. Conclusion
The results of this Master Thesis project provide some support for the experimental hypothesis, that
compare and contrast case review of pairs of cases, is more time efficient than, and at least as effective as
sequential single case review, in interpreting radiology cases, and potentially in learning radiology for
residency training. There is also some support for the effect of deliberate practice.
The control group showing learning gain between the pretest and posttest phases is consistent with
the expected effect of traditional radiology residency training, with one case at a time case presentation and
review. The experimental group showing not only more efficient, but also more effective problem solving,
with moderate and strong effect respectively, in the pretest phase, raises the interesting possibility that a
much fewer number of cases might be required for training for any one diagnostic category if paired
presentation and review is systematically used.
There are implications for structuring radiology residency programs, if the trends in this paper are
validated with larger groups of radiology residents, and a wider range of plain radiographic, and cross
sectional imaging. In addition, the application of these ideas in continuing medical education and
professional development, to maintain radiology expertise, and in teaching non-radiologists radiology
interpretation are exciting possibilities, and need to be explored further.

MHPE 2009 - 2011 Poh-Sun Goh, i665606
15
F. References
ACGME-I (2010). ACGME-International Advanced Specialty Program Requirements for Graduate Medical
Education in Diagnostic Radiology. Retrieved 30 October 2011, from Accredication Council for
Graduate Medical Education International website, http://www.acgme-
i.org/web/requirements/specialtypr.html.
Anders Ericsson, K. (2004). Deliberate practice and the acquisition and maintenance of expert performance
in medicine and related domains. Academic Medicine, 79, S70-81.
Anders Ericsson, K., Krampe, R.T., & Tesch-Romer, C. (1993). The role of deliberate practice in the
acquisition of expert performance. Psychological Review, 100, 363-406.
Avrahami, J., Kareev, Y. et al. (1997). Teaching by examples: implications for the process of category
acquisition. Quarterly Journal of Experimental Psychology, 50A, 586–606.
Becker, L.A. (1999). Effect size calculators. Retrieved 7 April 2012 from University of Colorado Colorado
Springs website, http://www.uccs.edu/~faculty/lbecker/.
Berbaum, K., Franken, A., & Smith, W.L. (1985). The effect of comparison films on resident interpretation
of paediatric chest radiographs. Investigative Radiology, 20(2), 124-128.
Cohen, J. (1988). Statistical Power Analysis for the Behavioral Sciences (second ed.). Lawrence Erlbaum
Associates.
Goh, P.S., & Tam, K.C.J. (2009). Deliberate distributed practice with paired and mixed examples is an
efficient and effective method to learn key abnormalities on the CXR. AMEE conference, Malaga.
Goh, P.S., & Tam, K.C.J. (2009). Application of Educational Theory Regarding Teaching for Skill
Transference for Chest Radiograph Interpretation in Undergraduate Education and Training. 6th
APMEC conference, Singapore.
Goh, P.S., & Tam, K.C.J. (2010). Applying deliberate practice to chest radiology teaching – further work
with undergraduates and experience with postgraduates. 7th
Goh, P.S. (2011). Deliberate Practice Using Paired and Mixed Examples of Imaging Abnormalities on the
CXR is more effective and efficient for postgraduate training compared with traditional methods. 8th
Gick, M. L., & Paterson, K. (1992). Do contrasting examples facilitate schema acquisition and analogical
transfer. Canadian Journal of Psychology-Revue Canadienne De Psychologie, 46(4), 539-550.
Hatala, R.M., Brooks, L.R., & Norman, G.R. (2003). Practice makes perfect: the critical role of mixed
practice in the acquisition of ECG interpretation skills. Adv Health Sci Educ, 8, 17–26.
Helberg Engel, P.J. (2008). Tacit knowledge and visual expertise in medical diagnostic reasoning:
Implications for medical education. Med Teach, 30(7), 184-188.
Kirkman, T.W. (1996) Statistics to Use. Retrieved 15 March 2012, from College of Saint Benedict and Saint
John's University website, http://www.physics.csbsju.edu/stats/.

MHPE 2009 - 2011 Poh-Sun Goh, i665606
16
Koontz, N.A., Gunderman, R.B. (2008). Gestalt theory: Implications for radiology education. American
Journal of Roentgenology, 190, 1156-1160.
Krupinski, E. (2010). Perceptual factors in reading medical images. In Samei, E., & Krupinski, E (Ed.), The
handbook of medical image perception and techniques. Cambridge: Cambridge University Press.
Kundel, H.L., & Nodine, C.F. (1975). Interpreting chest radiographs without visual search. Radiology,
116, 527-532.
Markman, A. B., & Gentner, D. (1993). Structural alignment during similarity comparisons. Cognitive
Psychology, 25(4), 431-467.
Mulhern, G., & Greer, B. (2011). Making Sense of Data and Statistics in Psychology. 2nd
edition. Palgrave
Macmillian, Great Britain.
Namy, L. L., & Clepper, L.E. (2010). The differing roles of comparison and contrast in children’s
categorization. Journal of Experimental Child Psychology, 107, 291–305.
Nodine, C. & Mello-Thomas, M. (2010). The role of expertise in radiologic image interpretation. In Samei,
E., & Krupinski, E (Ed.), The handbook of medical image perception and techniques. Cambridge:
Cambridge University Press.
Norman, G.R. (2005). Research in clinical reasoning: past history and current trends. Medical Education, 39,
418–427.
Norman, G.R. (2005). From Theory to Application and Back Again: Implications of Research on Medical
Expertise for Psychological Theory. Canadian Journal of Experimental Psychology/Revue canadienne
de psychologie expérimentale, 59(1), 35-40.
Norman, G.R. (2008). Making basic science relevant: Teaching for transfer. Plenary lecture. Association of
Medical Educators Europe (AMEE) Annual Scientific Meeting, Prague.
Norman, G.R., Young, M., Brooks, L. (2007). Non-analytical models of clinical reasoning: the role of
experience. Medical Education, 41, 1140-1145.
Pusic, M., Pecaric, M., & Boutis, K. (2011). How much practice is enough? Using leaning curves to assess
the deliberate practice of radiograph interpretation. Academic Medicine, 86(6), 731-736.
Roedling, S., Robinson, A., Lough-White, C., & Miller, R. (2008). Impact of a European working time
directive-compliant working pattern on delivery of medical specialty teaching for senior house officers
in a teaching hospital. Clin Med, 8, 116-7.
Royal College of Radiologists (2012). Retrieved 9 November 2011, from Royal College of Radiologists
website, http://www.rcr.ac.uk/content.aspx?PageID=164
Rumble, G. (1997). The costs and economics of open and distance learning. Cogan page Limited.
Specialist Accreditation Board, Ministry of Health, Singapore (2012). Retrieved 1 November 2011, from
Ministry of Health Singapore website,
https://www.cms.moh.gov.sg/servlet/Satellite?c=Content_C&cid=1303267836998&pagename=SAB%
2FContent_C%2FSAB_ArticleDetailPage&rendermode=preview-dawn_aio-1119926675547.

MHPE 2009 - 2011 Poh-Sun Goh, i665606
17
Walker, I. (2008). Null hypothesis testing and effect sizes. Retrieved 7 April 2012, from University of Bath
website, http://staff.bath.ac.uk/pssiw/stats2/page2/page14/page14.html.
Wood, B.P. (1999). Visual expertise. Radiology, 211, 1-3.

MHPE 2009 - 2011 Poh-Sun Goh, i665606
18
G. Appendices
Figure 1. Department Library (clockwise from top left – 2 views of computer workstations for resident use; library can
be sectioned off for privacy; answer sheet booklets laid out for the “experiment”).

MHPE 2009 - 2011 Poh-Sun Goh, i665606
19
Figure 2: Example of answer sheet. Answers are handwritten on workbooks with individual sheets for each case
divided into sections – Pretest, Learning and Practice Phase, and Posttest. The answer sheet for each case has
sections for 1) a statement of the likely diagnosis; 2) concise justification for the diagnosis offered; 3) entry of time
taken for each case.

MHPE 2009 - 2011 Poh-Sun Goh, i665606
20
Figure 3. Experimental methodology.

MHPE 2009 - 2011 Poh-Sun Goh, i665606
21
Figure 4. Example of sequential practice for CT scans of the brain.

MHPE 2009 - 2011 Poh-Sun Goh, i665606
22
Figure 5. Example of paired practice with dissimilar or contrasting cases for CT brain.

MHPE 2009 - 2011 Poh-Sun Goh, i665606
23
Figure 6. Example of paired practice with similar pairs for CT brain.

MHPE 2009 - 2011 Poh-Sun Goh, i665606
24
Figure 7. Illustration of paired practice for Chest Radiographs (CXRs).

MHPE 2009 - 2011 Poh-Sun Goh, i665606
25
Figure 8. Illustration of paired practice with dissimilar or contrasting case pairs for CXR.

MHPE 2009 - 2011 Poh-Sun Goh, i665606
26
Figure 9. Illustration of paired practice with similar case pairs for CXR.

MHPE 2009 - 2011 Poh-Sun Goh, i665606
27
Figure 10: The scheme for the experimental crossover.

Use of case pairs can potentially improve the efficiency and effectiveness of radiology residency

Recommandé

Recommandé

Contenu connexe

Tendances

Tendances (20)

Similaire à Use of case pairs can potentially improve the efficiency and effectiveness of radiology residency

Similaire à Use of case pairs can potentially improve the efficiency and effectiveness of radiology residency (20)

Plus de Poh-Sun Goh

Plus de Poh-Sun Goh (20)

Dernier

Dernier (20)

Use of case pairs can potentially improve the efficiency and effectiveness of radiology residency