The document discusses learning analytics and the use of student data to improve educational outcomes. It describes how various types of student data are currently collected and how that data could potentially be used to identify at-risk students, notify instructors about student performance, and improve retention. Examples are provided of learning analytics systems at other universities that monitor student activity and performance to provide targeted advising and interventions. Issues around privacy, control, and the audience for analytic insights are also raised.

1. Learning Analytics:
More Than Data-Driven Decisions
Steven Lonn
Research Fellow
USE Lab, Digital Media Commons
www.umich.edu/~uselab
1

2. Acknowledgements
• USE Lab: • John Campbell
– Stephanie D. Teasley • John Fritz
– Andrew Krumm • Tim McKay
– R. Joseph Waddington • David Wiley
USE Lab
University of Michigan 2
http://umich.edu/~uselab

3. What is Analytics?
+ +
USE Lab
University of Michigan 3
http://umich.edu/~uselab

4. Analytics in Our Lives
USE Lab
University of Michigan 4
http://umich.edu/~uselab

5. Analytics in Our Lives
USE Lab
University of Michigan 5
http://umich.edu/~uselab

6. Analytics in Our Work
USE Lab
University of Michigan 6
http://umich.edu/~uselab

7. Analytics in Our Work
USE Lab
University of Michigan 6
http://umich.edu/~uselab

8. Analytics in Our Work
at a?
hi sd
ll t
it ha
w
DO
ne
oeso
hatd
W
USE Lab
University of Michigan 6
http://umich.edu/~uselab

9. Data Collected at . .
What kind of data is
already available those
“in the know?”
USE Lab
University of Michigan 7
http://umich.edu/~uselab

10. Data Collected at . .
Admissions
• High school GPA
• SAT & ACT
• Parental education
• First generation college student?
• Socio-economic status
• Admission “rank”
• AP tests & scores
USE Lab
University of Michigan 8
http://umich.edu/~uselab

11. Data Collected at . .
Demographics
• Gender
• Ethnicity
• Age
• Michigan residency
• Country of origin & citizenship
• Athlete?
USE Lab
University of Michigan 9
http://umich.edu/~uselab

12. Data Collected at . .
Academic Record
• Cumulative GPA
• Specific course grades
• Major / minor
• Number of Michigan credits
• Number of transfer credits
• Credits / grades in subsets (e.g., math courses)
USE Lab
University of Michigan 10
http://umich.edu/~uselab

13. Data Collected at . .
Other Places Data is Gathered...
• CTools (courses, projects, etc.)
• Library (Mirlyn, website, electronic journals)
• Wolverine Access
• Other UM tools (LectureTools, SiteMaker,
UM.Lessons, MFile, Webmail, etc.)
USE Lab
University of Michigan 11
http://umich.edu/~uselab

14. Current Use of Data...
USE Lab
University of Michigan 12
http://umich.edu/~uselab

15. What if...
• Identify:
– Who needs the most help
– Most successful sequence of courses
– Most / least successful portions of a course
• Notify:
– Instructors about their students
– Students about their performance compared to peers
– Academic advisors about students “at risk”
– Staff about their resources (e.g., library use)
USE Lab
University of Michigan 13
http://umich.edu/~uselab

16. Milestones
• Stage 1: Extraction & reporting of transaction-level data
• Stage 2: Analysis and monitoring of operational performance
• Stage 3: What-if decision support (e.g., scenario building)
• Stage 4: Predictive modeling & simulation
• Stage 5: Automatic triggers of business processes (e.g., alerts)
-- Goldstein & Katz, 2005
USE Lab
University of Michigan 14
http://umich.edu/~uselab

17. !"#$%&"#'()#*+,""#'-#.//#&(0&1&02,+#"$20)($"3
USE Lab
University of Michigan
http://umich.edu/~uselab

18. Signals
• Purdue University
• System developed in 2007
• Use of analytics for:
– improving retention
– identifying students “at risk”
of academic failure
USE Lab
University of Michigan 16
http://umich.edu/~uselab

19. Signals
• NBC Nightly News Clip:
http://www.msnbc.msn.com/id/21134540/vp/32634348
• Aired August 31, 2009
USE Lab
University of Michigan 17
http://umich.edu/~uselab

20. Signals
• 6-10% improvement in retention
• 58% of students using report seeking help b/c of
Signals use
• Controlled by the instructor
• Course-by-course
• Does not show students direct comparison with
their peers
USE Lab
University of Michigan 19
http://umich.edu/~uselab

21. “Check My Activity” Tool
• University of Maryland, Baltimore County
USE Lab
University of Michigan 20
http://umich.edu/~uselab

22. “Check My Activity” Tool
• University of Maryland, Baltimore County
USE Lab
University of Michigan 20
http://umich.edu/~uselab

23. “Check My Activity” Tool
• University of Maryland, Baltimore County
USE Lab
University of Michigan 20
http://umich.edu/~uselab

24. “Check My Activity” Tool
• University of Maryland, Baltimore County
• Student-controlled
• Designed to promote student
agency & self-regulation
• Low impact for the instructor
USE Lab
University of Michigan 20
http://umich.edu/~uselab

25. Projects
• ITS UM-Data Warehouse
– One place where all data can be aggregated and reported
out.
– Currently includes:
• Student Dataset
• eResearch
• Financial
• Human Resources
• Payroll
• Physical Resources
USE Lab
University of Michigan 21
http://umich.edu/~uselab

26. Projects
• M-STEM Academy & USE Lab
– 50 Engineering students per cohort
– Use CTools data to better inform
mentor team
• When do they need mentoring /
direction to resources?
– How do mentors & students make
use of this data?
– How does behavior change?
USE Lab
University of Michigan 22
http://umich.edu/~uselab

27. Projects
• M-STEM Academy & USE Lab
– 50 Engineering students per cohort
../0)123)/*45+%"6)788)
%!!"!!#$
– Use CTools data to better inform -!"!!#$
mentor team ,!"!!#$
+!"!!#$
• When *!"!!#$ they need mentoring /
do
!"#$"%&'(")!*+%&,)
direction to resources?
)!"!!#$
4567/85$9:;35$
– How do mentors & students make (!"!!#$
'!"!!#$
<=2>>$?@/32A/$
use of this data? &!"!!#$
– How does behavior change? %!"!!#$
!"!!#$
./0$-$ ./0$%*$ 123$&$ 123$-$ 123$%*$
-'&")
USE Lab
University of Michigan 22
http://umich.edu/~uselab

28. Projects
Social Network Analysis
USE Lab
University of Michigan 23
http://umich.edu/~uselab

29. Projects
• Tim McKay
– Arthur F. Thurnau
Professor of Physics
• Taught into Physics courses for
years
• Director: LS&A Honors Program
• Used LS&A ART tool to track
student progress.
USE Lab
University of Michigan 24
http://umich.edu/~uselab

30. Projects
• Studied nearly 50,000
students over 12 years
• Can predict final grades
within 0.5 grade dispersion
• Next project: use an e-coach
programmed with analytics
data to motivate ALL students
USE Lab
University of Michigan 26
http://umich.edu/~uselab

31. Issues to Ponder
• Who is the audience?
– Students, Instructors, Advisors, Deans, Staff, Others?
• Who has the control?
– Issues of burden?
• Which views?
• Privacy concerns?
– Is their an institutional obligation?
• Is Learning Analytics just a fad?
• Others?
USE Lab
University of Michigan 26
http://umich.edu/~uselab

32. Further Reading
• Campbell, J., Deblois, P., & Oblinger, D. (2007). Academic analytics: A new tool for a new era.
EDUCAUSE Review, 42(4), 40−57.
• Fritz, J. (2011). Classroom walls that talk: Using online course activity data of successful students
to raise self-awareness of underperforming peers. The Internet and Higher Education, 14(2),
89-97. doi:10.1016/j.iheduc.2010.07.007
• Goldstein, P., & Katz, R. (2005). Academic analytics: The uses of management information and
technology in higher education — Key findings (key findings) (pp. 1–12). Educause Center for
Applied Research. http://www. educause.edu/ECAR/AcademicAnalyticsTheUsesofMana/156526
• Macfadyen, L. P., & Dawson, S. (2010). Mining LMS data to develop an “early warning system” for
educators: A proof of concept. Computers & Education, 54(2), 588−599. doi:10.1016/j.compedu.
2009.09.008.
• Morris, L.V., Finnegan, C., & Wu, S. (2005). Tracking student behavior, persistence, and achievement
in online courses. The Internet and Higher Education, 8(3), 221−231. doi:10.1016/j.iheduc.
2005.06.009.
USE Lab
University of Michigan !"#$#%&'((%)%*+'((,-./012#3- 27
http://umich.edu/~uselab