Panel presentation at the DET/CHE 2012 conference on November 28, 2012 by Kathy Fernandes (Chico State), James Frazee (San Diego State), Andrew Roderick (SFSU), and Deone Zell (CSU Northridge).

1. Learning Analytics Panel Session
Panelists: Kathy Fernandes, James Frazee,
Andrew Roderick & Deone Zell
Moderator: John Whitmer
DET/CHE 2012
November 28, 2012
D/L slides @ http://goo.gl/UgBYM

2. Introductions
Kathy Fernandes
Director of Academic Technology, Chico State University
Director of CSU System-wide LMSS Project, CSU Office of the Chancellor
James Frazee
Director, Instructional Technology Services, San Diego State University
Andrew Roderick
Technology Development Manager, San Francisco State University
Deone Zell
Senior Director, Academic Technology, California State University, Northridge
John Whitmer
Associate Director System-wide LMSS Project, CSU Office of the Chancellor
Slides: http://goo.gl/UgBYM

3. Outline
1. L.A. Context: Definition & Examples
2. Panelist Discussion
3. Audience Q & A
Slides: http://goo.gl/UgBYM

4. Learning Analytics: Working Definition for
Academic Technologists
 Analysis of the relationship between the use of
academic technology and student achievement
– Uses data, frequently “big data”, disaggregated
– Builds on (and moves beyond) usage reporting
to examine impact
– Focused on making change
 Isn’t a yes/no dichotomy; better seen as a
spectrum
Slides: http://goo.gl/UgBYM

5. EXAMPLES OF LEARNING
ANALYTICS IN PRACTICE

6. Chico State Case Study: RELS 180
• Redesigned to hybrid delivery
through Academy eLearning 54 F’s
• Enrollment: 373 students
(54% increase on largest section)
• Highest LMS usage entire
campus Fall 2010 (>250k hits)
• Bimodal outcomes:
• 10% increased SLO mastery
• 7% & 11% increase in DWF
• Why? Can’t tell with aggregated
reporting data

7. Correlation: LMS Use w/Final Grade
Scatterplot of
Assessment Activity
Hits vs. Course
Grade
Statistically Significant (strong to weak) r % Variance Sign.
Total Hits 0.48 23% 0.0000
Assessment activity hits 0.47 22% 0.0000
Content activity hits 0.41 17% 0.0000
Engagement activity hits 0.40 16% 0.0000
Administrative activity hits 0.35 12% 0.0000
Mean value all significant variables 18%
Slides: http://goo.gl/UgBYM

8. Correlation: Student Char. w/Final Grade
Scatterplot of
HS GPA vs. Course
Grade
Statistically Significant (strong to weak) r % Variance Sign.
HS GPA 0.31 9% 0.0000
URM and Pell-Eligibility Interaction -0.26 7% 0.0001
Under-Represented Minority -0.21 4% 0.0001
Enrollment Status 0.19 3% 0.0003
URM and Gender Interaction -0.15 2% 0.0033
Pell Eligible -0.15 2% 0.0045
First in Family to Attend College -0.11 1% 0.0327
Mean value all significant variables 4%
Not Statistically Significant
Gender 0.10 1% 0.0557
Major-College 0.06 0% 0.2522 Slides: http://goo.gl/UgBYM

9. Most interesting finding (so far):
LMS Student
Use Characteristic
Variables
>
Variables
18% 4%
(Mean % change (Mean % change
explained by any explained by any
single LMS variable) single S.C. variable)
Slides: http://goo.gl/UgBYM

10. SIGNALS
Purdue Signals Project http://www.itap.purdue.edu/studio/signals/

11. Wordcloud of student evaluations of Course Signals (Arnold, 2010)

12. Signals Course Outcomes
 Compare same course (w/Signals v. w/o Signals)
-6.41% DWF
+10.97% A/B
(Arnold, 2010)
Slides: http://goo.gl/UgBYM

13. PANEL QUESTIONS

14. Q1: What strategic question(s) does your
campus hope to answer with Learning
Analytics?

15. Q2: What are some example(s) of
Learning Analytics that you have
conducted on your campus, no matter
how small the sample size? What’s
worked? What hasn’t? Why?

16. Q3: What should educational
technologists do to make progress
deploying Learning Analytics on their
campuses?

17. For More Information
 Learning Analytics Resources Googledoc:
http://goo.gl/Fwur6
 Society for Learning Analytics Research:
http://goo.gl/bH9ts
 Educause Learning Analytics Library:
http://goo.gl/UDRMx
Slides: http://goo.gl/UgBYM

18. AUDIENCE Q & A By WingedWolf
Damián Navas

19. Contact Information
Kathy Fernandes (kfernandes@csuchico.edu)
James Frazee (jfrazee@mail.sdsu.edu)
Andrew Roderick (roderick@sfsu.edu)
Deone Zell (deone.zell@csun.edu)
John Whitmer (jwhitmer@calstate.edu)
Slides: http://goo.gl/UgBYM