SlideShare une entreprise Scribd logo
1  sur  22
Télécharger pour lire hors ligne
ID 164. Combatting survey fatigue:
  uncovering the student learning
        experience using the
     Nominal Group Technique




        Tunde Varga-Atkins, eLearning Unit
        Jaye McIsaac, Educational Development
             University of Liverpool
               11-13 Sep 2012, ALT-C
                                                Funded by ELESIG
Outcomes
•To consider the use of Nominal
Group Technique as a student
evaluation strategy
•To question the use (and concept)
of technology in relation to an
evaluation process
HE context: surveys
One solution: face-to-face Nominal
   Group Technique sessions
What is the
   Nominal Group Technique?
• Structured group activity
• One given topic
• Facilitates group
  decision-making
• Immediate results
• Reducing researcher/participant bias

             - Delbecq & Van de Ven (1971)
Nominal Group Technique Stages
  Example question: ‘What to keep doing (feedback)?’




1) Individual responses       2) Clarification and             3) Ranking responses
                              consolidation

          ‘Nominal’ Group = only in name group, most activities are individual
What to keep doing? (feedback)
 Stage 1: individual responses
               1                 2                 3
               Annotated         Model                 Audio
               ,personalised     answers               feedback
               feedback.                               giving
                                                       extensive
                                                       comments
4
Giving model       5
                   Audio             6
answers for        feedback              Drafts that
essay titles       for                   must be
                   everything!           submitted and
                                         marked prior
                                         final work
What to keep doing? (feedback)
Stage2 : clarification and consolidation
                                Model answers
                                                 4
                                     2
                                                 Giving model
                                     Model
                                                 answers for
                                     answers
                                                 essay titles

      Audio feedback
                                              Personalised feedbac
  3   Audio                        6
                   5                 Drafts that
      feedback    Audio              must be        1
      giving      feedback           submitted and Annotated
      extensive   for                marked prior ,personalised
      comments                       final work      feedback.
                  everything!
What to keep doing? (feedback)
     Stage3 : ranking responses


            Item no.   Item description

 5 points   1          Annotated, personalised feedback
 4 points   2          Giving model answers
 3 points   4          Audio feedback
 2 points   7          etc.
 1 point    10         etc.
The top five qualities of good
                 feedback
1   Timing,
    quick


              Detailed,
              on how to
         2
              improve
                          Applicable
                          to future
                   3      work
                                       Personal

                              4                   Consistent
                                                  bw
                                              5   markers
Conclusion: The Nominal Group
          Technique




   is a useful student evaluation
             technique
… (minor) problems of efficiency




                   Credit by Nathan T. Baker
Can technology improve
(time & resource) efficiency
    of the NGT process?




                   Presumption!
No single technology
   could replicate
 all the 3 stages of
        NGT.

Technologies we tried: ID275, Thu 9am
and…
Our assumption:

         Technology can
          improve (time)
      efficiency of the NGT
             process.
revisited…

   learning     +
 (evaluation)
                     technology          =       efficiency          ?

                Problem of lack of theorising technology (Oliver 2012)
Paper                    Digital
• ubiquitous          •   text entry
• manipulabl          •   legibility
  e                   •   visibility
• overlay             •   scoring
• multi-items         •   manipulable
• legible             •   multiitems
• visible
  (Luff et al 2007)
Conclusion: can technology enhance
     efficiency of the process?

           •   ubiquitous
           •   manipulable
           •   overlay
           •   multi-items
           •   text entry
           •   legibility
           •   visibility
                              Image by Paul Duvall
Finally…thanks to


Contact:
  –Tünde Varga-Atkins, tva@liv.ac.uk
  –Jaye McIsaac, jayem@liv.ac.uk
References
• Delbecq, A., Van de Ven, Andrew, & Gustafson, D.
  (1975). Group techniques for program planning  a    :
  guide to nominal group and Delphi processes.
  Glenview Ill.: Scott Foresman.
• Luff et al (2007) Augmented Paper: Developing
  relationships between digital content and paper
• Oliver, M. (2012) Learning with technology as a
  coordinated sociomaterial practice: digital literacies as
  a site of praxiological study
• Varga-Atkins, McIsaac et al (2011) Using the nominal
  group technique with clickers to research student
  experiences of e-learning: a project report
  [http://slidesha.re/xQlBCg ]
• Varga-Atkins, McIsaac et al (2011) The Nominal
  Group Technique – a practical guide for facilitators
  [http://slidesha.re/AmYOgv]
Guide to NGT & Project report




                                                See References
         On slideshare.net , search for Nominal Group Technique

Contenu connexe

Tendances

Jane Schreiber
Jane SchreiberJane Schreiber
Jane SchreiberJanecs
 
AUGMENTED, ASSESSED, ADDRESSED: REFRAMING THE CURRICULA FOR FIRST YEAR STEM S...
AUGMENTED, ASSESSED, ADDRESSED: REFRAMING THE CURRICULA FOR FIRST YEAR STEM S...AUGMENTED, ASSESSED, ADDRESSED: REFRAMING THE CURRICULA FOR FIRST YEAR STEM S...
AUGMENTED, ASSESSED, ADDRESSED: REFRAMING THE CURRICULA FOR FIRST YEAR STEM S...debbieholley1
 
Just in Time Teaching - Jeff Loats @ LMU
Just in Time Teaching - Jeff Loats @ LMUJust in Time Teaching - Jeff Loats @ LMU
Just in Time Teaching - Jeff Loats @ LMUJeff Loats
 
Maximising the potential of IVLE: A showcase of good practices
Maximising the potential of IVLE: A showcase of good practicesMaximising the potential of IVLE: A showcase of good practices
Maximising the potential of IVLE: A showcase of good practicesCIT, NUS
 
Digital Activism Training of Trainers
Digital Activism Training of TrainersDigital Activism Training of Trainers
Digital Activism Training of TrainersMer Joyce
 
High School Instructional Strategies Report 2009/2010
High School Instructional Strategies Report 2009/2010High School Instructional Strategies Report 2009/2010
High School Instructional Strategies Report 2009/2010Jason Zielinski
 
Principles and Practices of Online Course Creation and Instructional Design -...
Principles and Practices of Online Course Creation and Instructional Design -...Principles and Practices of Online Course Creation and Instructional Design -...
Principles and Practices of Online Course Creation and Instructional Design -...Jeffrey Keefer
 
The Research and Evaluation of current Trend in Education-Second Life and U-L...
The Research and Evaluation of current Trend in Education-Second Life and U-L...The Research and Evaluation of current Trend in Education-Second Life and U-L...
The Research and Evaluation of current Trend in Education-Second Life and U-L...Edie Cheng
 
SFASU: Using Brightspace Tools to Increase Student Engagement and Instructor ...
SFASU: Using Brightspace Tools to Increase Student Engagement and Instructor ...SFASU: Using Brightspace Tools to Increase Student Engagement and Instructor ...
SFASU: Using Brightspace Tools to Increase Student Engagement and Instructor ...D2L Barry
 
'The role of creativity in Student Engagement - learning through Social Learn...
'The role of creativity in Student Engagement - learning through Social Learn...'The role of creativity in Student Engagement - learning through Social Learn...
'The role of creativity in Student Engagement - learning through Social Learn...mdxaltc
 
AHTR SoTL Resources: Kelly Donahue-Wallace, Prove It! Publish It! SoTL Case ...
AHTR SoTL Resources:  Kelly Donahue-Wallace, Prove It! Publish It! SoTL Case ...AHTR SoTL Resources:  Kelly Donahue-Wallace, Prove It! Publish It! SoTL Case ...
AHTR SoTL Resources: Kelly Donahue-Wallace, Prove It! Publish It! SoTL Case ...Amy Raffel
 
Starburst: A New Graphical Online Discussion Forum Interface
Starburst: A New Graphical Online Discussion Forum InterfaceStarburst: A New Graphical Online Discussion Forum Interface
Starburst: A New Graphical Online Discussion Forum Interfacealywise
 
Blogging, reflective learning and peer support
 Blogging, reflective learning and peer support Blogging, reflective learning and peer support
Blogging, reflective learning and peer supportHazel Hall
 
Engage, reflect, achieve: the blog as a learning tool in an undergraduate module
Engage, reflect, achieve: the blog as a learning tool in an undergraduate moduleEngage, reflect, achieve: the blog as a learning tool in an undergraduate module
Engage, reflect, achieve: the blog as a learning tool in an undergraduate moduleHazel Hall
 

Tendances (19)

Jane Schreiber
Jane SchreiberJane Schreiber
Jane Schreiber
 
Project def 2014
Project def 2014Project def 2014
Project def 2014
 
AUGMENTED, ASSESSED, ADDRESSED: REFRAMING THE CURRICULA FOR FIRST YEAR STEM S...
AUGMENTED, ASSESSED, ADDRESSED: REFRAMING THE CURRICULA FOR FIRST YEAR STEM S...AUGMENTED, ASSESSED, ADDRESSED: REFRAMING THE CURRICULA FOR FIRST YEAR STEM S...
AUGMENTED, ASSESSED, ADDRESSED: REFRAMING THE CURRICULA FOR FIRST YEAR STEM S...
 
Just in Time Teaching - Jeff Loats @ LMU
Just in Time Teaching - Jeff Loats @ LMUJust in Time Teaching - Jeff Loats @ LMU
Just in Time Teaching - Jeff Loats @ LMU
 
Maximising the potential of IVLE: A showcase of good practices
Maximising the potential of IVLE: A showcase of good practicesMaximising the potential of IVLE: A showcase of good practices
Maximising the potential of IVLE: A showcase of good practices
 
Podcasts for Postgrads
Podcasts for PostgradsPodcasts for Postgrads
Podcasts for Postgrads
 
Digital Activism Training of Trainers
Digital Activism Training of TrainersDigital Activism Training of Trainers
Digital Activism Training of Trainers
 
Engaging Audiences
Engaging AudiencesEngaging Audiences
Engaging Audiences
 
High School Instructional Strategies Report 2009/2010
High School Instructional Strategies Report 2009/2010High School Instructional Strategies Report 2009/2010
High School Instructional Strategies Report 2009/2010
 
Principles and Practices of Online Course Creation and Instructional Design -...
Principles and Practices of Online Course Creation and Instructional Design -...Principles and Practices of Online Course Creation and Instructional Design -...
Principles and Practices of Online Course Creation and Instructional Design -...
 
The Research and Evaluation of current Trend in Education-Second Life and U-L...
The Research and Evaluation of current Trend in Education-Second Life and U-L...The Research and Evaluation of current Trend in Education-Second Life and U-L...
The Research and Evaluation of current Trend in Education-Second Life and U-L...
 
SFASU: Using Brightspace Tools to Increase Student Engagement and Instructor ...
SFASU: Using Brightspace Tools to Increase Student Engagement and Instructor ...SFASU: Using Brightspace Tools to Increase Student Engagement and Instructor ...
SFASU: Using Brightspace Tools to Increase Student Engagement and Instructor ...
 
Esl classes
Esl classesEsl classes
Esl classes
 
'The role of creativity in Student Engagement - learning through Social Learn...
'The role of creativity in Student Engagement - learning through Social Learn...'The role of creativity in Student Engagement - learning through Social Learn...
'The role of creativity in Student Engagement - learning through Social Learn...
 
AHTR SoTL Resources: Kelly Donahue-Wallace, Prove It! Publish It! SoTL Case ...
AHTR SoTL Resources:  Kelly Donahue-Wallace, Prove It! Publish It! SoTL Case ...AHTR SoTL Resources:  Kelly Donahue-Wallace, Prove It! Publish It! SoTL Case ...
AHTR SoTL Resources: Kelly Donahue-Wallace, Prove It! Publish It! SoTL Case ...
 
Starburst: A New Graphical Online Discussion Forum Interface
Starburst: A New Graphical Online Discussion Forum InterfaceStarburst: A New Graphical Online Discussion Forum Interface
Starburst: A New Graphical Online Discussion Forum Interface
 
Atlas
AtlasAtlas
Atlas
 
Blogging, reflective learning and peer support
 Blogging, reflective learning and peer support Blogging, reflective learning and peer support
Blogging, reflective learning and peer support
 
Engage, reflect, achieve: the blog as a learning tool in an undergraduate module
Engage, reflect, achieve: the blog as a learning tool in an undergraduate moduleEngage, reflect, achieve: the blog as a learning tool in an undergraduate module
Engage, reflect, achieve: the blog as a learning tool in an undergraduate module
 

Similaire à Combatting survey fatigue: uncovering the student learning experience using the

Enhancing Nominal Group Technique using technologies: a demonstration
Enhancing  Nominal Group Technique  using technologies: a demonstrationEnhancing  Nominal Group Technique  using technologies: a demonstration
Enhancing Nominal Group Technique using technologies: a demonstrationTünde Varga-Atkins
 
Media as Levers (pdf)
Media as Levers (pdf)Media as Levers (pdf)
Media as Levers (pdf)Lawrie Hunter
 
Evaluation techniques of teaching: focus groups and Nominal Group Technique
Evaluation techniques of teaching: focus groups and Nominal Group TechniqueEvaluation techniques of teaching: focus groups and Nominal Group Technique
Evaluation techniques of teaching: focus groups and Nominal Group TechniqueTünde Varga-Atkins
 
Main session 25th copy
Main session 25th copyMain session 25th copy
Main session 25th copypetefrasers
 
Feedback - The Lost Art of Agile
Feedback -  The Lost Art of AgileFeedback -  The Lost Art of Agile
Feedback - The Lost Art of AgileAlexandru Bolboaca
 
AACTE 2013 presentation
AACTE 2013 presentationAACTE 2013 presentation
AACTE 2013 presentationlumpea
 
Task based language test
Task based language testTask based language test
Task based language testVronit82
 
Case study: evaluation of a tool for searching inside a collection of multimo...
Case study: evaluation of a tool for searching inside a collection of multimo...Case study: evaluation of a tool for searching inside a collection of multimo...
Case study: evaluation of a tool for searching inside a collection of multimo...ronchet
 
Design Doc Aldworth
Design Doc AldworthDesign Doc Aldworth
Design Doc Aldworthnatealdworth
 
Task based language test
Task based language testTask based language test
Task based language testVronit82
 
Wk7 projectrooksg.doc [autosaved]ppt
Wk7 projectrooksg.doc [autosaved]pptWk7 projectrooksg.doc [autosaved]ppt
Wk7 projectrooksg.doc [autosaved]pptgregrooks
 
Web Engineering Workshop
Web Engineering WorkshopWeb Engineering Workshop
Web Engineering WorkshopDhaval Dalal
 
Applying e-portfolios to design education
Applying e-portfolios to design educationApplying e-portfolios to design education
Applying e-portfolios to design educationRoger Chou
 
Group challenge guide
Group challenge guideGroup challenge guide
Group challenge guidepsbkebonjeruk
 
Screencast feedback
Screencast feedback Screencast feedback
Screencast feedback ktmtchl
 
Formative assessment
Formative assessmentFormative assessment
Formative assessmentAlison Hardy
 

Similaire à Combatting survey fatigue: uncovering the student learning experience using the (20)

Enhancing Nominal Group Technique using technologies: a demonstration
Enhancing  Nominal Group Technique  using technologies: a demonstrationEnhancing  Nominal Group Technique  using technologies: a demonstration
Enhancing Nominal Group Technique using technologies: a demonstration
 
Media as Levers (pdf)
Media as Levers (pdf)Media as Levers (pdf)
Media as Levers (pdf)
 
Evaluation techniques of teaching: focus groups and Nominal Group Technique
Evaluation techniques of teaching: focus groups and Nominal Group TechniqueEvaluation techniques of teaching: focus groups and Nominal Group Technique
Evaluation techniques of teaching: focus groups and Nominal Group Technique
 
Main session 25th copy
Main session 25th copyMain session 25th copy
Main session 25th copy
 
Feedback - The Lost Art of Agile
Feedback -  The Lost Art of AgileFeedback -  The Lost Art of Agile
Feedback - The Lost Art of Agile
 
Arts Achieve at Metis
Arts Achieve at MetisArts Achieve at Metis
Arts Achieve at Metis
 
Success Stories Presentation
Success Stories PresentationSuccess Stories Presentation
Success Stories Presentation
 
AACTE 2013 presentation
AACTE 2013 presentationAACTE 2013 presentation
AACTE 2013 presentation
 
2013 Update P2PU + Badges
2013 Update P2PU + Badges2013 Update P2PU + Badges
2013 Update P2PU + Badges
 
Task based language test
Task based language testTask based language test
Task based language test
 
Case study: evaluation of a tool for searching inside a collection of multimo...
Case study: evaluation of a tool for searching inside a collection of multimo...Case study: evaluation of a tool for searching inside a collection of multimo...
Case study: evaluation of a tool for searching inside a collection of multimo...
 
Design Doc Aldworth
Design Doc AldworthDesign Doc Aldworth
Design Doc Aldworth
 
Task based language test
Task based language testTask based language test
Task based language test
 
Grading Ppt
Grading PptGrading Ppt
Grading Ppt
 
Wk7 projectrooksg.doc [autosaved]ppt
Wk7 projectrooksg.doc [autosaved]pptWk7 projectrooksg.doc [autosaved]ppt
Wk7 projectrooksg.doc [autosaved]ppt
 
Web Engineering Workshop
Web Engineering WorkshopWeb Engineering Workshop
Web Engineering Workshop
 
Applying e-portfolios to design education
Applying e-portfolios to design educationApplying e-portfolios to design education
Applying e-portfolios to design education
 
Group challenge guide
Group challenge guideGroup challenge guide
Group challenge guide
 
Screencast feedback
Screencast feedback Screencast feedback
Screencast feedback
 
Formative assessment
Formative assessmentFormative assessment
Formative assessment
 

Plus de Tünde Varga-Atkins

Disciplinary digital capabilities of professionals: networked learning in eng...
Disciplinary digital capabilities of professionals: networked learning in eng...Disciplinary digital capabilities of professionals: networked learning in eng...
Disciplinary digital capabilities of professionals: networked learning in eng...Tünde Varga-Atkins
 
Networked learning Prof Chris Jones interview
Networked learning Prof Chris Jones interviewNetworked learning Prof Chris Jones interview
Networked learning Prof Chris Jones interviewTünde Varga-Atkins
 
Crafting poems for data analysis?
Crafting poems for data analysis?Crafting poems for data analysis?
Crafting poems for data analysis?Tünde Varga-Atkins
 
One stitch in time, saves nine: Illustrating the helpfulness of theory in lea...
One stitch in time, saves nine: Illustrating the helpfulness of theory in lea...One stitch in time, saves nine: Illustrating the helpfulness of theory in lea...
One stitch in time, saves nine: Illustrating the helpfulness of theory in lea...Tünde Varga-Atkins
 
How do we design for digitally fluent professionals?
How do we design for digitally fluent professionals?How do we design for digitally fluent professionals?
How do we design for digitally fluent professionals?Tünde Varga-Atkins
 
Digital capabilities now and in the world to come - Tunde Varga-Atkins
Digital capabilities now and in the world to come  - Tunde Varga-AtkinsDigital capabilities now and in the world to come  - Tunde Varga-Atkins
Digital capabilities now and in the world to come - Tunde Varga-AtkinsTünde Varga-Atkins
 
Digital capabilities in curriculum design #1771 #ALTC
Digital capabilities in curriculum design #1771 #ALTCDigital capabilities in curriculum design #1771 #ALTC
Digital capabilities in curriculum design #1771 #ALTCTünde Varga-Atkins
 
Cityscape as a metaphor for assessment practice: report on an evaluation of e...
Cityscape as a metaphor for assessment practice: report on an evaluation of e...Cityscape as a metaphor for assessment practice: report on an evaluation of e...
Cityscape as a metaphor for assessment practice: report on an evaluation of e...Tünde Varga-Atkins
 
Understanding International Students' Experiences of Learning with Technology
Understanding International Students' Experiences of Learning with TechnologyUnderstanding International Students' Experiences of Learning with Technology
Understanding International Students' Experiences of Learning with TechnologyTünde Varga-Atkins
 
Semi-structured interviews for educational research
Semi-structured interviews for educational research Semi-structured interviews for educational research
Semi-structured interviews for educational research Tünde Varga-Atkins
 
Data collection in qualitative research focus groups october 2015
Data collection in qualitative research focus groups october 2015Data collection in qualitative research focus groups october 2015
Data collection in qualitative research focus groups october 2015Tünde Varga-Atkins
 
Poster Digital Champions - University of Liverpool
Poster  Digital Champions - University of LiverpoolPoster  Digital Champions - University of Liverpool
Poster Digital Champions - University of LiverpoolTünde Varga-Atkins
 
Poster slides for tablet hea digital champions
Poster slides for tablet   hea digital championsPoster slides for tablet   hea digital champions
Poster slides for tablet hea digital championsTünde Varga-Atkins
 
Digital Champions HEA pilot: supporting Year 1 students with academic transition
Digital Champions HEA pilot: supporting Year 1 students with academic transitionDigital Champions HEA pilot: supporting Year 1 students with academic transition
Digital Champions HEA pilot: supporting Year 1 students with academic transitionTünde Varga-Atkins
 
A bit about ELESIG North West - at ALT NWE SIG meeting
A bit about ELESIG North West - at ALT NWE SIG meetingA bit about ELESIG North West - at ALT NWE SIG meeting
A bit about ELESIG North West - at ALT NWE SIG meetingTünde Varga-Atkins
 
Diagrams to discourse and discourse to diagrams: using visual representation ...
Diagrams to discourse and discourse to diagrams: using visual representation ...Diagrams to discourse and discourse to diagrams: using visual representation ...
Diagrams to discourse and discourse to diagrams: using visual representation ...Tünde Varga-Atkins
 
What’s in a name? Defining and developing information and digital literacies...
What’s in a name?  Defining and developing information and digital literacies...What’s in a name?  Defining and developing information and digital literacies...
What’s in a name? Defining and developing information and digital literacies...Tünde Varga-Atkins
 
eLearning Unit: can we help you? (ALTC-2013)
eLearning Unit: can we help you? (ALTC-2013)eLearning Unit: can we help you? (ALTC-2013)
eLearning Unit: can we help you? (ALTC-2013)Tünde Varga-Atkins
 
Using focus groups for evaluating learner experiences by Tunde Varga-Atkins a...
Using focus groups for evaluating learner experiences by Tunde Varga-Atkins a...Using focus groups for evaluating learner experiences by Tunde Varga-Atkins a...
Using focus groups for evaluating learner experiences by Tunde Varga-Atkins a...Tünde Varga-Atkins
 

Plus de Tünde Varga-Atkins (20)

Disciplinary digital capabilities of professionals: networked learning in eng...
Disciplinary digital capabilities of professionals: networked learning in eng...Disciplinary digital capabilities of professionals: networked learning in eng...
Disciplinary digital capabilities of professionals: networked learning in eng...
 
Networked learning Prof Chris Jones interview
Networked learning Prof Chris Jones interviewNetworked learning Prof Chris Jones interview
Networked learning Prof Chris Jones interview
 
Crafting poems for data analysis?
Crafting poems for data analysis?Crafting poems for data analysis?
Crafting poems for data analysis?
 
One stitch in time, saves nine: Illustrating the helpfulness of theory in lea...
One stitch in time, saves nine: Illustrating the helpfulness of theory in lea...One stitch in time, saves nine: Illustrating the helpfulness of theory in lea...
One stitch in time, saves nine: Illustrating the helpfulness of theory in lea...
 
How do we design for digitally fluent professionals?
How do we design for digitally fluent professionals?How do we design for digitally fluent professionals?
How do we design for digitally fluent professionals?
 
Digital capabilities now and in the world to come - Tunde Varga-Atkins
Digital capabilities now and in the world to come  - Tunde Varga-AtkinsDigital capabilities now and in the world to come  - Tunde Varga-Atkins
Digital capabilities now and in the world to come - Tunde Varga-Atkins
 
Digital capabilities in curriculum design #1771 #ALTC
Digital capabilities in curriculum design #1771 #ALTCDigital capabilities in curriculum design #1771 #ALTC
Digital capabilities in curriculum design #1771 #ALTC
 
Cityscape as a metaphor for assessment practice: report on an evaluation of e...
Cityscape as a metaphor for assessment practice: report on an evaluation of e...Cityscape as a metaphor for assessment practice: report on an evaluation of e...
Cityscape as a metaphor for assessment practice: report on an evaluation of e...
 
Understanding International Students' Experiences of Learning with Technology
Understanding International Students' Experiences of Learning with TechnologyUnderstanding International Students' Experiences of Learning with Technology
Understanding International Students' Experiences of Learning with Technology
 
Semi-structured interviews for educational research
Semi-structured interviews for educational research Semi-structured interviews for educational research
Semi-structured interviews for educational research
 
Data collection in qualitative research focus groups october 2015
Data collection in qualitative research focus groups october 2015Data collection in qualitative research focus groups october 2015
Data collection in qualitative research focus groups october 2015
 
Year 1 survey
Year 1 surveyYear 1 survey
Year 1 survey
 
Poster Digital Champions - University of Liverpool
Poster  Digital Champions - University of LiverpoolPoster  Digital Champions - University of Liverpool
Poster Digital Champions - University of Liverpool
 
Poster slides for tablet hea digital champions
Poster slides for tablet   hea digital championsPoster slides for tablet   hea digital champions
Poster slides for tablet hea digital champions
 
Digital Champions HEA pilot: supporting Year 1 students with academic transition
Digital Champions HEA pilot: supporting Year 1 students with academic transitionDigital Champions HEA pilot: supporting Year 1 students with academic transition
Digital Champions HEA pilot: supporting Year 1 students with academic transition
 
A bit about ELESIG North West - at ALT NWE SIG meeting
A bit about ELESIG North West - at ALT NWE SIG meetingA bit about ELESIG North West - at ALT NWE SIG meeting
A bit about ELESIG North West - at ALT NWE SIG meeting
 
Diagrams to discourse and discourse to diagrams: using visual representation ...
Diagrams to discourse and discourse to diagrams: using visual representation ...Diagrams to discourse and discourse to diagrams: using visual representation ...
Diagrams to discourse and discourse to diagrams: using visual representation ...
 
What’s in a name? Defining and developing information and digital literacies...
What’s in a name?  Defining and developing information and digital literacies...What’s in a name?  Defining and developing information and digital literacies...
What’s in a name? Defining and developing information and digital literacies...
 
eLearning Unit: can we help you? (ALTC-2013)
eLearning Unit: can we help you? (ALTC-2013)eLearning Unit: can we help you? (ALTC-2013)
eLearning Unit: can we help you? (ALTC-2013)
 
Using focus groups for evaluating learner experiences by Tunde Varga-Atkins a...
Using focus groups for evaluating learner experiences by Tunde Varga-Atkins a...Using focus groups for evaluating learner experiences by Tunde Varga-Atkins a...
Using focus groups for evaluating learner experiences by Tunde Varga-Atkins a...
 

Dernier

CapTechU Doctoral Presentation -March 2024 slides.pptx
CapTechU Doctoral Presentation -March 2024 slides.pptxCapTechU Doctoral Presentation -March 2024 slides.pptx
CapTechU Doctoral Presentation -March 2024 slides.pptxCapitolTechU
 
KARNAADA.pptx made by - saransh dwivedi ( SD ) - SHALAKYA TANTRA - ENT - 4...
KARNAADA.pptx  made by -  saransh dwivedi ( SD ) -  SHALAKYA TANTRA - ENT - 4...KARNAADA.pptx  made by -  saransh dwivedi ( SD ) -  SHALAKYA TANTRA - ENT - 4...
KARNAADA.pptx made by - saransh dwivedi ( SD ) - SHALAKYA TANTRA - ENT - 4...M56BOOKSTORE PRODUCT/SERVICE
 
Department of Health Compounder Question ‍Solution 2022.pdf
Department of Health Compounder Question ‍Solution 2022.pdfDepartment of Health Compounder Question ‍Solution 2022.pdf
Department of Health Compounder Question ‍Solution 2022.pdfMohonDas
 
What is the Future of QuickBooks DeskTop?
What is the Future of QuickBooks DeskTop?What is the Future of QuickBooks DeskTop?
What is the Future of QuickBooks DeskTop?TechSoup
 
CHUYÊN ĐỀ DẠY THÊM TIẾNG ANH LỚP 11 - GLOBAL SUCCESS - NĂM HỌC 2023-2024 - HK...
CHUYÊN ĐỀ DẠY THÊM TIẾNG ANH LỚP 11 - GLOBAL SUCCESS - NĂM HỌC 2023-2024 - HK...CHUYÊN ĐỀ DẠY THÊM TIẾNG ANH LỚP 11 - GLOBAL SUCCESS - NĂM HỌC 2023-2024 - HK...
CHUYÊN ĐỀ DẠY THÊM TIẾNG ANH LỚP 11 - GLOBAL SUCCESS - NĂM HỌC 2023-2024 - HK...Nguyen Thanh Tu Collection
 
Optical Fibre and It's Applications.pptx
Optical Fibre and It's Applications.pptxOptical Fibre and It's Applications.pptx
Optical Fibre and It's Applications.pptxPurva Nikam
 
How to Create a Toggle Button in Odoo 17
How to Create a Toggle Button in Odoo 17How to Create a Toggle Button in Odoo 17
How to Create a Toggle Button in Odoo 17Celine George
 
Vani Magazine - Quarterly Magazine of Seshadripuram Educational Trust
Vani Magazine - Quarterly Magazine of Seshadripuram Educational TrustVani Magazine - Quarterly Magazine of Seshadripuram Educational Trust
Vani Magazine - Quarterly Magazine of Seshadripuram Educational TrustSavipriya Raghavendra
 
Patient Counselling. Definition of patient counseling; steps involved in pati...
Patient Counselling. Definition of patient counseling; steps involved in pati...Patient Counselling. Definition of patient counseling; steps involved in pati...
Patient Counselling. Definition of patient counseling; steps involved in pati...raviapr7
 
How to Manage Cross-Selling in Odoo 17 Sales
How to Manage Cross-Selling in Odoo 17 SalesHow to Manage Cross-Selling in Odoo 17 Sales
How to Manage Cross-Selling in Odoo 17 SalesCeline George
 
Work Experience for psp3 portfolio sasha
Work Experience for psp3 portfolio sashaWork Experience for psp3 portfolio sasha
Work Experience for psp3 portfolio sashasashalaycock03
 
How to Add a New Field in Existing Kanban View in Odoo 17
How to Add a New Field in Existing Kanban View in Odoo 17How to Add a New Field in Existing Kanban View in Odoo 17
How to Add a New Field in Existing Kanban View in Odoo 17Celine George
 
How to Solve Singleton Error in the Odoo 17
How to Solve Singleton Error in the  Odoo 17How to Solve Singleton Error in the  Odoo 17
How to Solve Singleton Error in the Odoo 17Celine George
 
Quality Assurance_GOOD LABORATORY PRACTICE
Quality Assurance_GOOD LABORATORY PRACTICEQuality Assurance_GOOD LABORATORY PRACTICE
Quality Assurance_GOOD LABORATORY PRACTICESayali Powar
 
2024.03.23 What do successful readers do - Sandy Millin for PARK.pptx
2024.03.23 What do successful readers do - Sandy Millin for PARK.pptx2024.03.23 What do successful readers do - Sandy Millin for PARK.pptx
2024.03.23 What do successful readers do - Sandy Millin for PARK.pptxSandy Millin
 
In - Vivo and In - Vitro Correlation.pptx
In - Vivo and In - Vitro Correlation.pptxIn - Vivo and In - Vitro Correlation.pptx
In - Vivo and In - Vitro Correlation.pptxAditiChauhan701637
 
Diploma in Nursing Admission Test Question Solution 2023.pdf
Diploma in Nursing Admission Test Question Solution 2023.pdfDiploma in Nursing Admission Test Question Solution 2023.pdf
Diploma in Nursing Admission Test Question Solution 2023.pdfMohonDas
 
Drug Information Services- DIC and Sources.
Drug Information Services- DIC and Sources.Drug Information Services- DIC and Sources.
Drug Information Services- DIC and Sources.raviapr7
 

Dernier (20)

Prelims of Kant get Marx 2.0: a general politics quiz
Prelims of Kant get Marx 2.0: a general politics quizPrelims of Kant get Marx 2.0: a general politics quiz
Prelims of Kant get Marx 2.0: a general politics quiz
 
CapTechU Doctoral Presentation -March 2024 slides.pptx
CapTechU Doctoral Presentation -March 2024 slides.pptxCapTechU Doctoral Presentation -March 2024 slides.pptx
CapTechU Doctoral Presentation -March 2024 slides.pptx
 
KARNAADA.pptx made by - saransh dwivedi ( SD ) - SHALAKYA TANTRA - ENT - 4...
KARNAADA.pptx  made by -  saransh dwivedi ( SD ) -  SHALAKYA TANTRA - ENT - 4...KARNAADA.pptx  made by -  saransh dwivedi ( SD ) -  SHALAKYA TANTRA - ENT - 4...
KARNAADA.pptx made by - saransh dwivedi ( SD ) - SHALAKYA TANTRA - ENT - 4...
 
Department of Health Compounder Question ‍Solution 2022.pdf
Department of Health Compounder Question ‍Solution 2022.pdfDepartment of Health Compounder Question ‍Solution 2022.pdf
Department of Health Compounder Question ‍Solution 2022.pdf
 
What is the Future of QuickBooks DeskTop?
What is the Future of QuickBooks DeskTop?What is the Future of QuickBooks DeskTop?
What is the Future of QuickBooks DeskTop?
 
CHUYÊN ĐỀ DẠY THÊM TIẾNG ANH LỚP 11 - GLOBAL SUCCESS - NĂM HỌC 2023-2024 - HK...
CHUYÊN ĐỀ DẠY THÊM TIẾNG ANH LỚP 11 - GLOBAL SUCCESS - NĂM HỌC 2023-2024 - HK...CHUYÊN ĐỀ DẠY THÊM TIẾNG ANH LỚP 11 - GLOBAL SUCCESS - NĂM HỌC 2023-2024 - HK...
CHUYÊN ĐỀ DẠY THÊM TIẾNG ANH LỚP 11 - GLOBAL SUCCESS - NĂM HỌC 2023-2024 - HK...
 
Optical Fibre and It's Applications.pptx
Optical Fibre and It's Applications.pptxOptical Fibre and It's Applications.pptx
Optical Fibre and It's Applications.pptx
 
How to Create a Toggle Button in Odoo 17
How to Create a Toggle Button in Odoo 17How to Create a Toggle Button in Odoo 17
How to Create a Toggle Button in Odoo 17
 
Vani Magazine - Quarterly Magazine of Seshadripuram Educational Trust
Vani Magazine - Quarterly Magazine of Seshadripuram Educational TrustVani Magazine - Quarterly Magazine of Seshadripuram Educational Trust
Vani Magazine - Quarterly Magazine of Seshadripuram Educational Trust
 
Patient Counselling. Definition of patient counseling; steps involved in pati...
Patient Counselling. Definition of patient counseling; steps involved in pati...Patient Counselling. Definition of patient counseling; steps involved in pati...
Patient Counselling. Definition of patient counseling; steps involved in pati...
 
How to Manage Cross-Selling in Odoo 17 Sales
How to Manage Cross-Selling in Odoo 17 SalesHow to Manage Cross-Selling in Odoo 17 Sales
How to Manage Cross-Selling in Odoo 17 Sales
 
Work Experience for psp3 portfolio sasha
Work Experience for psp3 portfolio sashaWork Experience for psp3 portfolio sasha
Work Experience for psp3 portfolio sasha
 
How to Add a New Field in Existing Kanban View in Odoo 17
How to Add a New Field in Existing Kanban View in Odoo 17How to Add a New Field in Existing Kanban View in Odoo 17
How to Add a New Field in Existing Kanban View in Odoo 17
 
How to Solve Singleton Error in the Odoo 17
How to Solve Singleton Error in the  Odoo 17How to Solve Singleton Error in the  Odoo 17
How to Solve Singleton Error in the Odoo 17
 
Quality Assurance_GOOD LABORATORY PRACTICE
Quality Assurance_GOOD LABORATORY PRACTICEQuality Assurance_GOOD LABORATORY PRACTICE
Quality Assurance_GOOD LABORATORY PRACTICE
 
2024.03.23 What do successful readers do - Sandy Millin for PARK.pptx
2024.03.23 What do successful readers do - Sandy Millin for PARK.pptx2024.03.23 What do successful readers do - Sandy Millin for PARK.pptx
2024.03.23 What do successful readers do - Sandy Millin for PARK.pptx
 
In - Vivo and In - Vitro Correlation.pptx
In - Vivo and In - Vitro Correlation.pptxIn - Vivo and In - Vitro Correlation.pptx
In - Vivo and In - Vitro Correlation.pptx
 
Diploma in Nursing Admission Test Question Solution 2023.pdf
Diploma in Nursing Admission Test Question Solution 2023.pdfDiploma in Nursing Admission Test Question Solution 2023.pdf
Diploma in Nursing Admission Test Question Solution 2023.pdf
 
Drug Information Services- DIC and Sources.
Drug Information Services- DIC and Sources.Drug Information Services- DIC and Sources.
Drug Information Services- DIC and Sources.
 
March 2024 Directors Meeting, Division of Student Affairs and Academic Support
March 2024 Directors Meeting, Division of Student Affairs and Academic SupportMarch 2024 Directors Meeting, Division of Student Affairs and Academic Support
March 2024 Directors Meeting, Division of Student Affairs and Academic Support
 

Combatting survey fatigue: uncovering the student learning experience using the

  • 1. ID 164. Combatting survey fatigue: uncovering the student learning experience using the Nominal Group Technique Tunde Varga-Atkins, eLearning Unit Jaye McIsaac, Educational Development University of Liverpool 11-13 Sep 2012, ALT-C Funded by ELESIG
  • 2. Outcomes •To consider the use of Nominal Group Technique as a student evaluation strategy •To question the use (and concept) of technology in relation to an evaluation process
  • 4. One solution: face-to-face Nominal Group Technique sessions
  • 5. What is the Nominal Group Technique? • Structured group activity • One given topic • Facilitates group decision-making • Immediate results • Reducing researcher/participant bias - Delbecq & Van de Ven (1971)
  • 6. Nominal Group Technique Stages Example question: ‘What to keep doing (feedback)?’ 1) Individual responses 2) Clarification and 3) Ranking responses consolidation ‘Nominal’ Group = only in name group, most activities are individual
  • 7. What to keep doing? (feedback) Stage 1: individual responses 1 2 3 Annotated Model Audio ,personalised answers feedback feedback. giving extensive comments 4 Giving model 5 Audio 6 answers for feedback Drafts that essay titles for must be everything! submitted and marked prior final work
  • 8. What to keep doing? (feedback) Stage2 : clarification and consolidation Model answers 4 2 Giving model Model answers for answers essay titles Audio feedback Personalised feedbac 3 Audio 6 5 Drafts that feedback Audio must be 1 giving feedback submitted and Annotated extensive for marked prior ,personalised comments final work feedback. everything!
  • 9. What to keep doing? (feedback) Stage3 : ranking responses Item no. Item description 5 points 1 Annotated, personalised feedback 4 points 2 Giving model answers 3 points 4 Audio feedback 2 points 7 etc. 1 point 10 etc.
  • 10. The top five qualities of good feedback 1 Timing, quick Detailed, on how to 2 improve Applicable to future 3 work Personal 4 Consistent bw 5 markers
  • 11. Conclusion: The Nominal Group Technique is a useful student evaluation technique
  • 12. … (minor) problems of efficiency Credit by Nathan T. Baker
  • 13. Can technology improve (time & resource) efficiency of the NGT process? Presumption!
  • 14. No single technology could replicate all the 3 stages of NGT. Technologies we tried: ID275, Thu 9am
  • 16. Our assumption: Technology can improve (time) efficiency of the NGT process.
  • 17. revisited… learning + (evaluation) technology = efficiency ? Problem of lack of theorising technology (Oliver 2012)
  • 18. Paper Digital • ubiquitous • text entry • manipulabl • legibility e • visibility • overlay • scoring • multi-items • manipulable • legible • multiitems • visible (Luff et al 2007)
  • 19. Conclusion: can technology enhance efficiency of the process? • ubiquitous • manipulable • overlay • multi-items • text entry • legibility • visibility Image by Paul Duvall
  • 20. Finally…thanks to Contact: –Tünde Varga-Atkins, tva@liv.ac.uk –Jaye McIsaac, jayem@liv.ac.uk
  • 21. References • Delbecq, A., Van de Ven, Andrew, & Gustafson, D. (1975). Group techniques for program planning  a : guide to nominal group and Delphi processes. Glenview Ill.: Scott Foresman. • Luff et al (2007) Augmented Paper: Developing relationships between digital content and paper • Oliver, M. (2012) Learning with technology as a coordinated sociomaterial practice: digital literacies as a site of praxiological study • Varga-Atkins, McIsaac et al (2011) Using the nominal group technique with clickers to research student experiences of e-learning: a project report [http://slidesha.re/xQlBCg ] • Varga-Atkins, McIsaac et al (2011) The Nominal Group Technique – a practical guide for facilitators [http://slidesha.re/AmYOgv]
  • 22. Guide to NGT & Project report See References On slideshare.net , search for Nominal Group Technique