1. INTED2012 (6th International Technology,
Education and Development Conference)
2012 Conference
EXPLORING THE BALANCE BETWEEN
AUTOMATION AND HUMAN
INTERVENTION IN IMPROVING FINAL
YEAR UNIVERSITY STUDENT NON-
COMPLETION
Andrea Wheeler and Melanie King
The Centre for Engineering & Design Education
http://cede.lboro.ac.uk

2. Outline
• Background: JISC ‘Pedestal for Progression’ project
• Issues related to progression in final year and failure to complete.
• Workshops with students, a whole lost of concerns – relationships
with staff and employability.
• Methods adopted: Service Design and Data Mining.
• Data mining and how we currently collect attendance data.
Enhancements to current systems / processes. How staff &
students use attendance data. Evidence that links attendance
with progression. Issues surrounding the case for wider adoption.
• Service Design, managing relationships, points of contact.
• Discussion of results.
The Centre for Engineering and Design Education

3. Pedestal for Progression Project
March 2011 – August 2012
This project is investigating the application of the Service Design
methodology within higher education: techniques usually used
within the commercial Customer Relationship Management field
(CRM). The project team will be working with students, academics
and staff from support services across the Institution using service
design and data mining techniques in order to enhance the student
experience for final year students and aid their progression to next
stage - either employment or further study.
JISC Relationship Management Programme Phase II
Strand 2 – Progression, retention & non-completion
http://progression.lboro.ac.uk
The Centre for Engineering and Design Education

4. WP1: Discovery Phase – gathering user experiences
– all sorts of sources of data and discovery methods
adopted
• National Student Survey data
• Focus groups with finalists (Programme reps, Student Union)
• Current research on campus (History HEA mini-project, BSE)
• Interviews with staff (academics, admins, technicians, support services)
• LUFBRA.net
• Staff/student committees (minutes from meetings)
• Student stories
• Identification of current data collected about students (VLE activity, library
systems, attendance data, coursework hand-ins)
http://progression.lboro.ac.uk
The Centre for Engineering and Design Education

5. Expectations of:
Issues raised: Progression in Final Year Finalists
• Increase in contact time (academic and personal tutoring)
• Additional skills support (e.g. digital and information literacy)
• Better access to library books Needs of:
ACADEMICS
• Prioritised reading lists Effective & efficient process
Expert support
• Support for progression beyond graduation Robust systems
Easy access to data
• Fair and balanced assessment (e.g. timing of coursework hand-ins)
• Consistent and timely feedback
• Access to facilities and equipment
• Ability to feedback to tutors/departments Strategic objectives:
INSTITUTION
Enhanced student experience
Innovation
Competiveness
Efficiency
Sustainability
The Centre for Engineering and Design Education

6. Recurrent Issues: Access to tutors
“I feel that the personal tutor should take a more
active role in working with the student throughout
university life.”
Sport and Exercise Science student – entering final
year in October 2011
Recurrent Issues: Performance anxiety and personal development
“I think the main emphasis for final year should be attainment, with the
student being able to achieve the most they can to fully reach their
potential.”
Chemistry student – graduated 2011
The Centre for Engineering and Design Education

7. “Monitoring” : Encouraging the active tutor
– monitoring engagement
• Improving the identification of absence at crucial lectures or tutorials
• Improving the support for personal tutors in meeting their tutees
• Help department monitors to spot students falling through the net
• Identify other engagement data that could signal non-participation
• Improve the notification of critical information to the right staff member at
the right time.
The Centre for Engineering and Design Education

8. How Attendance data is collected
Attendance at lectures
Attendance at personal tutor meetings
The Centre for Engineering and Design Education

9. ATTENDANT Creation and marking of registers for modules
The Centre for Engineering and Design Education

10. CO-TUTOR The staff and student relationship management system
Organise groups Email groups
Add comments Access course marks
Schedule meetings View attendance records
View personal information Upload related files
The Centre for Engineering and Design Education

11. The staff and student relationship management system
CO-TUTOR FLEXIBILITY & CONTINUITY:
Supporting a learner’s journey
Personal welfare Attendance monitoring:
and guidance: course tutors, programme
Personal tutors, elite monitors
athletes’ welfare
officers
Supervision of
Academic
placement activities:
performance:
Placement co-
Foundation year
ordinators, industrial
tutors, lecturers,
supervisors
key skills support
Research supervision:
cross-department supervisors,
Graduate School ‘training
needs’ advisors
Disability,
additional needs &
course support:
support officers,
Personal development departmental admin
planning: personal tutors,
skills development officers

12. Main menu for access to
all students, personal Home Page for Tutors
messages and
monitoring reports
Quick access to different cohorts of
students and personalised
bookmark groups
Personal
summaries of
tutee meetings,
Select multiple commenting and
tutees and perform attendance
group actions
Quick action links
and attendance
summaries
Useful links to
support personal
tutors and
supervisors

13. A Student’s Record
Automated and
custom flags can
be added
Actions that any
staff can perform
on a student’s
record
Data feeds of personal
info and grades from
corporate systems
Comments are
categorised to
aid viewing
permissions e.g.
only allocated
research
supervisors can
see comments in
the Research
section
A history of comments,
notices and files added
to a student’s record

14. “Monitoring” Approaches: Enhancements to current processes / systems
• Registers now marked as ‘critical’ or ‘optional’
• Checkbox for ‘include data in summary’
• Students can add a reason for absence
• At the point of marking, an automated email can be sent to absent students
• More detailed reports created on student attendance, including attendance
patterns
• Students now have access to their own attendance record and are sent
emails (by welfare officer) to view it if their attendance drops below a
certain level
• Automated emails generated to personal tutors on a regular basis
The Centre for Engineering and Design Education

15. Data Mining: Tremendous amounts of data are
being collected about student behaviour and
activities
In the 2011 Horizon report Johnson et al. predicts that in
the next two to five years, “Learning analytics promises to
harness the power of advances in data mining,
interpretation, and modelling to improve understandings
of teaching and learning, and to tailor education to
individual students more effectively” [8].
The Centre for Engineering and Design Education

16. Attendance Data Analysis
2004/5 2005/6 2006/7 2007/8 2008/9 2009/10 2010/11
Total marked 25,372 37,899 62,017 79,428 116,335 151,012 271,373
present
Total number of 38,715 56,395 89,930 117,062 166,027 215,031 386,247
records
Average 65.54 67.20 68.96 67.85 70.07 70.23 70.26
attendance (%)
Diff (pp) +1.66 +1.76 -1.11 +2.22 +0.16 +0.03
Number modules 62 121 214 229 260 383 800
The Centre for Engineering and Design Education

17. Attendance Data Analysis
Averages for 4221 students graduating between 2004 and 2009 who have had attendance
recorded on at least 10 registers per year of study.
1st 78.80% 2.i 73.26% 2.ii 61.19% 3rd 58.36%
Link between final UK degree classification and average attendance
The Centre for Engineering and Design Education

18. Service Design:
Data mining and data monitoring aims to identify patterns
of behaviour that predict behaviour…
Service Design, however, aims to manage points of contact
of user and a service and improve experience /desirability.
The Centre for Engineering and Design Education

19. Service Design: students as owners of
services…
Service Design is concerned with providing authentic
customer focused, highly desirable, and pleasurable,
services. It aims to foster a sense of ownership, and
to include often intangible customer feelings about a
service, representing them in a visual manner. Snook,
a Service Design consultancy based in Glasgow,
defines a service in the term Service Design as “…a co-
created event that delivers value to the parties
engaged in the interaction” [1].
The Centre for Engineering and Design Education

20. Service Design: students as customers,
educators as panderers…
Students-as-customers, has very different
connotations to students as customers: entitlement to
satisfaction, a duty to complain. A University
management team keen to exceed customers’
expectations.
The Centre for Engineering and Design Education

21. Service Design Workshops with students and
staff – employability and the placement
experience ….feeding back into Co-Tutor
‘Mentoring’ software…
The Centre for Engineering and Design Education

26. The staff and student relationship management system
PROVIDES IMPORTANT METRICS:
CO-TUTOR
Enhances student experience
• Provides numerous monitoring reports that make the frequency and quality of support,
provided by staff to students, completely transparent to senior colleagues and
departmental managers.
• Attendance information used to view trends.
• Provides audit trails and accountability for the quality of care provided to students.
• Reports include; • Staff online activity
• Total number of comments per student
• Total number of student/staff meetings both missed and attended
• Distribution of alert flags
• Frequency of comments, meetings and emails
• % attendance across programme of module, year group or level of study
• Reports specific to tutoring type.

27. The staff and student relationship management system
IDENTIFICATION & MONITORING:
CO-TUTOR
Loughborough University, UK
Supporting struggling students
Internal messaging to
Detailed attendance
notify relevant staff
reports highlight
when comments are
struggling students
added to records
Ability to send emails
to personal tutors,
notifying of low
attendance
Quick views of
attendance summary
and comment counts
Automated flagging of
students with < 50%
attendance

28. The staff and student relationship management system
IDENTIFICATION & MONITORING:
CO-TUTOR
Supporting struggling students

29. RECOMMENDATIONS: Challenges to Address
1. The balance of automation versus human intervention.
2. More automated methods of capturing attendance seamlessly linked to current system
3. Complex user permissions to view student data as well as staff activity.
4. Who is monitoring the monitors?
5. Identification of ‘touch-points’ that are critically linked to completion.
The Centre for Engineering and Design Education

30. 2012 Conference
Andrea Wheeler
Teaching and Learning Co-ordinator (Projects),
The Centre for Engineering & Design Education
a.s.wheeler@lboro.ac.uk
JISC Pedetal for Progression Project
http://progression.lboro.ac.uk
http://cede.lboro.ac.uk