Academic Analytics is a hot topic in Higher Education. Institutions are seeking to use analytics to understand student success and academic performance, maximize retention. Increasingly, regulatory and accreditation bodies require this information to help measure effectiveness. This block session will report on a number of analytics initiatives within the Sakai Community, and higher education generally. Opportunities will be provided to interact with individual presenters, and to synthesise information available across the session.
2. Analytics:
More Than Data-Driven Decisions
Steven Lonn
Research Specialist
USE Lab, Digital Media Commons
www.umich.edu/~uselab
2
3. Acknowledgements
• USE Lab: • John Campbell
– Stephanie D. Teasley • John Fritz
– Andrew Krumm • Tim McKay
– R. Joseph Waddington • David Wiley
USE Lab Digital Media Commons
3
http://umich.edu/~uselab University of Michigan
4. What is Analytics?
+ +
USE Lab Digital Media Commons
4
http://umich.edu/~uselab University of Michigan
5. Analytics in Our Lives
USE Lab Digital Media Commons
5
http://umich.edu/~uselab University of Michigan
6. Analytics in Our Lives
USE Lab Digital Media Commons
6
http://umich.edu/~uselab University of Michigan
7. Analytics in OAE!
USE Lab Digital Media Commons
7
http://umich.edu/~uselab University of Michigan
8. Analytics in OAE!
USE Lab Digital Media Commons
7
http://umich.edu/~uselab University of Michigan
9. Analytics in OAE!
USE Lab Digital Media Commons
7
http://umich.edu/~uselab University of Michigan
10. Analytics in Our Work
USE Lab Digital Media Commons
8
http://umich.edu/~uselab University of Michigan
11. Analytics in Our Work
USE Lab Digital Media Commons
8
http://umich.edu/~uselab University of Michigan
12. Analytics in Our Work
at a?
hi sd
ll t
it ha
w
DO
ne
oeso
hatd
W
USE Lab Digital Media Commons
8
http://umich.edu/~uselab University of Michigan
13. Data Collected at . .
What kind of data is
already available those
“in the know?”
USE Lab Digital Media Commons
9
http://umich.edu/~uselab University of Michigan
14. Data Collected at . .
Admissions
USE Lab Digital Media Commons
10
http://umich.edu/~uselab University of Michigan
15. Data Collected at . .
Demographics
USE Lab Digital Media Commons
11
http://umich.edu/~uselab University of Michigan
16. Data Collected at . .
Academic Record
• Cumulative GPA
• Specific course grades
• Major / minor
• Number of Michigan credits
• Number of transfer credits
• Credits / grades in subsets (e.g., math courses)
USE Lab Digital Media Commons
12
http://umich.edu/~uselab University of Michigan
17. Data Collected at . .
Other Places Data is Gathered...
USE Lab Digital Media Commons
13
http://umich.edu/~uselab University of Michigan
18. Current Use of Data...
USE Lab Digital Media Commons
14
http://umich.edu/~uselab University of Michigan
19. What if...
• Identify:
– Who needs the most help
– Most successful sequence of courses
– Most / least successful portions of a course
• Notify:
– Instructors about their students
– Students about their performance compared to peers
–
–
Academic advisors about students “at risk”
Staff about their resources (e.g., library use) !
USE Lab Digital Media Commons
15
http://umich.edu/~uselab University of Michigan
20. Milestones
• Stage 1: Extraction & reporting of transaction-level data
• Stage 2: Analysis and monitoring of operational performance
• Stage 3: What-if decision support (e.g., scenario building)
• Stage 4: Predictive modeling & simulation
• Stage 5: Automatic triggers of business processes (e.g., alerts)
-- Goldstein & Katz, 2005
USE Lab Digital Media Commons
16
http://umich.edu/~uselab University of Michigan
22. Signals
• Purdue University
• System developed in 2007
• Use of analytics for:
– improving retention
– identifying students “at risk”
of academic failure
USE Lab Digital Media Commons
18
http://umich.edu/~uselab University of Michigan
23. Signals
567$5'8&%,9$5*:#
;383#%$<=>$?00@
!"#$%%&&&'()*+,'()*',-(%./%0102345%6#%
02307078902307078
USE Lab Digital Media Commons
19
http://umich.edu/~uselab University of Michigan
24. “Check My Activity” Tool
• University of Maryland, Baltimore County
USE Lab Digital Media Commons
20
http://umich.edu/~uselab University of Michigan
25. “Check My Activity” Tool
• University of Maryland, Baltimore County
USE Lab Digital Media Commons
20
http://umich.edu/~uselab University of Michigan
26. “Check My Activity” Tool
• University of Maryland, Baltimore County
• Student-controlled
• Designed to promote student
agency & self-regulation
• Low impact for the instructor
USE Lab Digital Media Commons
20
http://umich.edu/~uselab University of Michigan
27. Issues to Ponder
• Who is the audience?
– Students, Instructors, Advisors, Deans, Staff, Others?
• Who has the control?
– Issues of burden?
• Which views?
• Privacy concerns?
– Is their an institutional obligation?
• Is Learning Analytics just a fad?
• Others?
USE Lab Digital Media Commons
21
http://umich.edu/~uselab University of Michigan
28. Our Project
• M-STEM Academy
– 50 Engineering students
per cohort
– Use Sakai data to better
inform mentor team
USE Lab Digital Media Commons
22
http://umich.edu/~uselab University of Michigan
29. Our Project
• M-STEM Academy
– 50 Engineering students
• When do students need mentoring / direction
per cohort
toUse Sakai data to better
– resources?
• How domentor team& students make use of this
inform mentors
data?
• How does behavior change?
USE Lab Digital Media Commons
22
http://umich.edu/~uselab University of Michigan
30. USE Lab Digital Media Commons
23
http://umich.edu/~uselab University of Michigan
31. USE Lab Digital Media Commons
24
http://umich.edu/~uselab University of Michigan
32. comp
USE Lab Digital Media Commons
25
http://umich.edu/~uselab University of Michigan
33. Graphing!
USE Lab Digital Media Commons
26
http://umich.edu/~uselab University of Michigan
34. Advanced Graphing
USE Lab Digital Media Commons
27
http://umich.edu/~uselab University of Michigan
35. Project Next Steps
• What Sakai events are “meaningful” for predicting
student success?
• Presenting data displays to advisors and students in-term.
– Is a behavioral change noted? To what effect? What kinds of
outcomes are noted?
• Can this approach scale?
– Beginning with engineering college
USE Lab Digital Media Commons
28
http://umich.edu/~uselab University of Michigan
47. ATLAS Network Users
Ecosystem Public NYU
Portals Skins
Single Video
sign on sharing
OpenSSO Kaltura
Content
Identity
NetId and Digital
password
LDAP ATLAS Network eBooks
library
Grouper Echo360
SIS class Classroom
rosters capture
Pentaho Cassandra
Scalable
Analytics
storage
Data
12