March 2024 Directors Meeting, Division of Student Affairs and Academic Support
Combatting survey fatigue: uncovering the student learning experience using the
1. ID 164. Combatting survey fatigue:
uncovering the student learning
experience using the
Nominal Group Technique
Tunde Varga-Atkins, eLearning Unit
Jaye McIsaac, Educational Development
University of Liverpool
11-13 Sep 2012, ALT-C
Funded by ELESIG
2. Outcomes
•To consider the use of Nominal
Group Technique as a student
evaluation strategy
•To question the use (and concept)
of technology in relation to an
evaluation process
5. What is the
Nominal Group Technique?
• Structured group activity
• One given topic
• Facilitates group
decision-making
• Immediate results
• Reducing researcher/participant bias
- Delbecq & Van de Ven (1971)
6. Nominal Group Technique Stages
Example question: ‘What to keep doing (feedback)?’
1) Individual responses 2) Clarification and 3) Ranking responses
consolidation
‘Nominal’ Group = only in name group, most activities are individual
7. What to keep doing? (feedback)
Stage 1: individual responses
1 2 3
Annotated Model Audio
,personalised answers feedback
feedback. giving
extensive
comments
4
Giving model 5
Audio 6
answers for feedback Drafts that
essay titles for must be
everything! submitted and
marked prior
final work
8. What to keep doing? (feedback)
Stage2 : clarification and consolidation
Model answers
4
2
Giving model
Model
answers for
answers
essay titles
Audio feedback
Personalised feedbac
3 Audio 6
5 Drafts that
feedback Audio must be 1
giving feedback submitted and Annotated
extensive for marked prior ,personalised
comments final work feedback.
everything!
9. What to keep doing? (feedback)
Stage3 : ranking responses
Item no. Item description
5 points 1 Annotated, personalised feedback
4 points 2 Giving model answers
3 points 4 Audio feedback
2 points 7 etc.
1 point 10 etc.
10. The top five qualities of good
feedback
1 Timing,
quick
Detailed,
on how to
2
improve
Applicable
to future
3 work
Personal
4 Consistent
bw
5 markers
16. Our assumption:
Technology can
improve (time)
efficiency of the NGT
process.
17. revisited…
learning +
(evaluation)
technology = efficiency ?
Problem of lack of theorising technology (Oliver 2012)
18. Paper Digital
• ubiquitous • text entry
• manipulabl • legibility
e • visibility
• overlay • scoring
• multi-items • manipulable
• legible • multiitems
• visible
(Luff et al 2007)
19. Conclusion: can technology enhance
efficiency of the process?
• ubiquitous
• manipulable
• overlay
• multi-items
• text entry
• legibility
• visibility
Image by Paul Duvall
21. References
• Delbecq, A., Van de Ven, Andrew, & Gustafson, D.
(1975). Group techniques for program planning a :
guide to nominal group and Delphi processes.
Glenview Ill.: Scott Foresman.
• Luff et al (2007) Augmented Paper: Developing
relationships between digital content and paper
• Oliver, M. (2012) Learning with technology as a
coordinated sociomaterial practice: digital literacies as
a site of praxiological study
• Varga-Atkins, McIsaac et al (2011) Using the nominal
group technique with clickers to research student
experiences of e-learning: a project report
[http://slidesha.re/xQlBCg ]
• Varga-Atkins, McIsaac et al (2011) The Nominal
Group Technique – a practical guide for facilitators
[http://slidesha.re/AmYOgv]
22. Guide to NGT & Project report
See References
On slideshare.net , search for Nominal Group Technique