Y1Feedback Presentation at EdTech2015

1. Assessment Feedback Practice In
FirstYear Using Digital
Technologies – A Baseline
Review (Preliminary Findings)
Lisa O'Regan, Mark Brown, Moira Maguire, Nuala Harding,
Elaine Walsh, Gerry Gallagher, Geraldine McDermott
EdTech 2015 – May 28th

2. @y1feedback
Follow us!

3. Presentation Overview
o Why Feedback?
o Project Overview
o Planning the Current Practice Review
o Current Practice Review – Part 1 – Students
o Preliminary Findings
o Current Practice Review Part 2 – Staff
o Preliminary Findings
o Initial Conclusions & Next Steps

4. 2013
67.4%
 The Irish Survey of Student
Engagement (ISSE) 2013,
found that nationally,
22.3% of first year
undergraduate students
never, and 45.1% only
sometimes, received
timely written or oral
feedback from teachers on
academic performance.

5. 2014
68.2%
 The Irish Survey of Student
Engagement (ISSE) 2014,
found that nationally,
23.3% of first year
undergraduate students
never, and 44.9% only
sometimes, received
timely written or oral
feedback from teachers on
academic performance.

6. Worrying because …
 Feedback is one of the most powerful influences on learning
and achievement ( Hattie andTemperely 2007)
 Feedback is particularly important in supporting the
transition to Higher Education (Nicol 2009)
 Regular and frequent formative feedback in first year is
associated with student success (Tinto 2005, Nicol 2009)

7. The Project – @Y1Feedback
SupportingTransition: Enhancing
(Assessment) Feedback in FirstYear
Using DigitalTechnologies

8. Y1FeedbackTeam
 Lisa O’Regan MU (Project Manager)
 Nuala Harding & Geraldine McDermott AIT
 Moira Maguire & Gerry Gallagher DKIT
 Mark Brown & ElaineWalsh DCU

9. Y1Feedback ProjectAim
The project will develop and case
study technology enhanced
feedback approaches to facilitate
feedback, feed-forward and peer-
feedback in first year to support
student transition to higher
education.

10. An Design Based ResearchApproach
“a series of approaches, with the intent of
producing new theories, artifacts, and practices
that account for and potentially impact learning
and teaching in naturalistic settings.”
(Barab and Squire, 2004)
“research about education” rather than “research
for education”
(Alghamdi & Li, 2013)

11. Characteristics of Design Based Research
(Reeves 2006)

12. What is Assessment Feedback?
“Includes all feedback exchanges generated within
assessment design, occurring within and beyond
the immediate learning context, being overt or
covert (actively and/or passively sought and/or
received), and importantly, drawing from a range
of sources.” (Evans, 2013)

13. Facets of Feedback
Feedback – an amalgamation of a number of elements (Evans, 2013)
including:
 Information on performance or understanding (Hattie &Timperley,
2007)
 Information to influence subsequent work and improve
performance (Boud & Molloy, 2012)
 Action to alter the gap – “information used, rather than information
transmitted” (Boud & Molloy, 2012)
 Student responsibility - dialogue (Carless et al., 2011), self-
regulation (Nicol, 2009) for “sustainable feedback” (Hounsell, 2007)

14. Feedback Mediated by DigitalTechnologies
Includes:
 Written comments
 Track changes, comments, annotations
 Statement banks and checklists
 Assessment criteria and rubrics
 Audio
 Video
 Quizzes

15. Y1Feedback - Project Phases
Phase 1
• Current Practice Review (WIP)
• Literature Summary
Phase 2
• Pilot feedback, feed-forward and peer
feedback approaches and technologies
Phase 3
• Develop case studies of best practice
• Disseminate via National Symposium

16. Current Practice Review Planning
Why?
 To get a landscape snapshot of current assessment feedback
practices across the four partner institutions from a staff and
student perspective.
How?
 Staff Survey
 Student Focus Groups
 Initial EthicalApproval at MU
 Ethical approval at AIT, DKIT and DCU
 Eight Ethics Applications in total!

17. Current Practice Review Focus
 How assessment feedback is provided in first year undergraduate
programmes
 The timing and frequency of assessment feedback
 The types of assessment feedback provided to students
 The digital technologies used in the provision of assessment
feedback
 Lecturer views on assessment feedback processes and practices
 Student views on assessment feedback in first year

18. Current Practice Review – Part 1
 First year student focus groups
 MU, AIT, DKIT, DCU
 Purposive sample of first year undergraduate class
representatives
 Completed late April 2015
 Four focus groups with 36 participants in total
*Note: Preliminary findings - analysis is ongoing

19. The FirstYear Class
Representative
Response

20. How is Assessment Feedback Provided?
 ‘She [lecturer] will go
through the answers and
even after the exam she
will sit down with you to
go through it all and tell
you exactly where you
went wrong’
 ‘We do get written
feedback on the written
assignments'.
 ‘But tests, like continuous
assessment tests we
wouldn’t get feedback,
we’d just get your
percent’.
 ‘With regards to …...you
are just told that is your
grade, that is that...there is
no feedback whatsoever’.

21. When is Assessment Feedback Provided?
 ‘It probably depends on how
big the group is’.
 ‘It depends on the tutor
because the tutor I have now,
she’s really quick’.
 ‘Most of the time it is, it just
depends on like the
lecturer’.
 ‘For one of the five if it
comes, but for the other four
it doesn’t come on time’
…(for the next assignment)
 ‘It will be at least a month
and a half before you get
anything back’.
 ‘It would take a while now I’d
say for, to get your grade
back, but whenever you do
get it back, there is usually a
good amount of feedback
on it’.

22. WhatType of Assessment Feedback is Provided?
 ‘You might just get a
comment and it’d be like
“well done” and you could
have got seven out of ten and
there’s no comment like “well
you could have done this”, it’s
just “well done”.’
 'We got one aural one
before’.
 “Once on video”
 'Ours would mostly be
grades like 80% of ours
would be just our grade, no
written, no oral'.
 ‘We do get written feedback
on the written assignments,
nothing really on the online
because it is all netted and
you can get oral feedback or
a dialogue I suppose if you
engage your instructor’.

23. WhatTechnologies are Used to Provide Feedback?
 ‘It’s like a comment box that
they have underneath your
assignment'.
 ‘By email and hard copy’.
 ‘Yeah there is a feedback
online…Because you get like
an email saying you have
feedback on your
assignment, and then you can
click into it and then…’
 ‘You get the video recordings
but you also just get to see
what he is talking about and
he is able to visually show
you’.
 ‘We got one (feedback) last
semester’
 'We get nothing (feedback)
online at all’.
 'Like the facility is obviously
there for online feedback but
I don’t know whether
lecturers don’t want to do it
or they don’t know how or
how to put it across'.

24. StudentViews on Feedback in FirstYear
 “It [feedback] is the foundation of the next however many
years you are going to be here”.
 “The consensus around all of the college is that we are not
getting feedback at all. And that it would be highly
recommended if we could. For first year students coming in
after us like”.
 ”It’s [first year] one of the most important years for your
feedback because it’s completely different from anything
you had before”.

25. Suggested Changes …
 'It would be nice to have some kind of a continuance between
(modules), there's no point in us getting feedback in one
module, it'd be nice to have…Consistency'.
 'I've emailed and emailed and emailed and now it's like she's
a part-time lecturer so she's not here so I think it should be
made clear to them, you're not going to see them again at
least give them a little bit of feedback'.
 Instead of just getting grades the whole time I think we
should get more comments’.

26. Current Practice Review – Part 2
 Online survey
 Staff who teach undergraduate first year programmes
 MU,AIT, DKIT & DCU
 Anonymous
 Integrative
 Status: Ongoing closes 5th June 2015
 Current Responses 156
*Preliminary analysis –Ongoing work

27. What are Staff Saying
about Assessment
Feedback in FirstYear
Undergraduate
Programmes?

28. Respondent Profile: 156
Numbers ofYearsTeaching Experience

29. How often is assessment feedback prepared for
students on continuous assessment work?

30. How often is Assessment Feedback Provided
53% of respondents always provide feedback
32% frequently provide feedback
9% Occasionally provide feedback
3% Rarely or Never provide feedback

31. Who provides assessment feedback?

32. When is feedback provided in first year undergraduate
programmes?

33. In what format is assessment feedback
typically provided?

34. How do students access feedback?

35. Typical Feedback Provided

36. Technologies Most Highly Used to Deliver
Feedback
1. MSWord
2. Email
3. Moodle Gradebook Feedbak Comments
4. Moodle Gradebook Feedback Files
5. Moodle Quiz
6. PDF Annotation &Turnitin Grademark
7. Rubric
8. Audio, video, wiki, social media, socrative

37. Influences on Extent of Feedback
1. Own beliefs about value of feedback (52%)
2. Own workload (41%)
3. Nature of assessment (33% )
4. Level of involvement in the module (29%)

38. StaffViews On Feedback
98% of respondent Strongly agreed (80%) or
agreed that feedback is an integral part of
student learning
79% of respondents strongly agreed (32%)
or agreed (43%) that students value timely
feedback
71% of staff strongly agreed (23%) or agreed
(48%) felt that students seem to be only
interested in the grade

39. Staff Recommended FeedbackApproaches
1. Face –to- Face/ One to One (30% of responses)
2. Detailed written feedback and one-to one
discussion
3. Group/class discussions
4. Rubrics/Marking Schemes/Structured feedback
sheets
5. MCQs

40. Challenges to Providing Assessment
Feedback in FirstYear
1. Numbers 2.Time 3.Workload
4. Student Engagement
with Feedback
5.
Attendance/Absenteeism
6.Technology

41. Numbers –Time -Workload
“The sheer numbers 300…”
“Large class sizes hinder feedback”
“Workload …Too many students, too
much work, not enough time to grade
assignments quickly.Tensions between
different deadlines e.g. research vs
teaching”

42. Student Engagement with Feedback
 “Students seem to want too much detail in some cases and
think the role of feedback is to remediate ALL problems with
the essay in solution form as written feedback. In other words
– they expect to have a golden formula for guaranteed
success outlined in practical and concrete terms”
 “Lack of interest in deeper feedback based on mentality that
their grades don’t count in first year so that passing is the
only goal”
 While I provide feedback on all the assignments I set, not
many students engage with it. However, those that do
seem to benefit from it

43. Factors preventing use of technology for
feedback practices
Technology
 “Some aspects of grade administration in Moodle are clunky”
 “Highly problematic, no support or backup as local level plus
extremely time consuming process annotating in Moodle”
 “Generally poor quality of IT Service”
 “Student familiarity with technology”
 “Don’t see the value in technology”
 “One-to-one discussions can’t be replaced by technology”
Time
 “It is more time consuming”
 “Time taken to set up technology feedback would be a major factor”

44. Lecturer Suggested Solutions
 “To be able to provide effective feedback then you need to reduce the
class size significantly”
 “Unless modules are adequately resourced in terms of staffing, it is
going to be difficult to improve the situation”
 “Impact could be improved by providing feedback closer to the
student submissions and introducing feedback in the first three 3
weeks”
 “More training on and availability of new technologies”
 “Training students on feedback might help”
 “There should be some agreement between lecturers on purpose and
type of feedback provided”

45. Some Initial Conclusions
Student
 Recognise value of feedback
 Widely varied experience of
assessment feedback in first
year within and across
disciplines
 Depends on the lecturer
 Low levels of technology
enhanced feedback approaches
Staff
 Highlight importance of
feedback for student learning
 F2F No1 Recommendation
 Technology
 MoreTime Consuming
 Clunky
 Numbers/Large Classes
 Time/Workload

46. InitialTake Aways to Address
 The need for a conversation about feedback and a common
understanding of what feedback is
 A common process that be built into all feedback approaches
 Using technology for feedback is actually more time consuming?
 Myth or Reality? A question we can potentially address in the case studies
 The biggest problem with technology, is the technology!
 Can we improve the user experience of assessment/feedback interfaces in
Moodle?
 Numbers, Numbers, Numbers
 Our approaches and case studies will need a focus on large numbers, be
scalable and realistic
 Can we replicate F2F feedback?

47. Questions?

49. @y1feedback
Follow us!