The past five years have seen a dramatic growth in interest in the emerging field of Learning Analytics (LA), and particularly in the potential the field holds to address major challenges facing education. However, much of the work in the learning analytics landscape today is closed in nature, small in scale, tool- or software-centric, and relatively disconnected from other LA initiatives. This lack of collaboration, openness, and system integration often leads to fragmentation where learning data cannot be aggregated across different sources, institutions only have the option to implement "closed" systems, and cross disciplinary research opportunities are limited. Beyond the immediate concerns this fragmentation creates for educators and learners, a closed approach dramatically limits our ability to build upon successes, learn from failures and move beyond the "pockets of excellence (and failures)? approach that typifies much of the educational technology landscape.
The potential benefits of openness as a core value within the learning analytics community are numerous. Learning initiatives could be informed by large scale research projects. Open-source software, such as dashboards and analytics engines, could be available free of licensing costs and easily enhanced by others, and OERs could become more personalized to match learners' needs. Open data sets and reproducible papers could rapidly spread understanding of analytical approaches, enabling secondary analysis and comparison across research projects. To realize this future, leaders within the learning analytics, open technologies (software, standards, etc.), open research (open data, open predictive models, etc.) and open learning (OER, MOOCs, etc.) fields have established a "network of practice" aimed at connecting subject matter experts, projects, organizations and companies working in these domains. As an initial organizing event, these leaders organized an Open Learning Analytics (OLA) Summit directly following the 2014 Learning Analytics and Knowledge (LAK) conference this past March as means to further the goal of establishing "openness' as a core value of the larger learning analytics movement. Additional details on the Summit and those involved can be found at: http://www.prweb.com/releases/2014/04/prweb11754343.htm.
This panel session will bring together several thought leaders from the Open Learning Analytics community who participated in the Summit to facilitate an interactive dialog with attendees on the intersection of learning analytics and open learning, open technologies, open data, and open research. The presenters represent a broad range of experience with institutional analytics projects, an open source development consortium, the sharing of open learner data, and academic research on open learning environments.
2. Panel Session Overview
Session Goal: Stimulate discussion around the
importance of open learning analytics to the future of
the larger open education movement
Longer-Term Objective: Form connections between
the OpenEd and Open Learning Analytics networks
Session Format:
Setting the Context – Open Learning Analytics
Examples from the Real World
Discussion/Q&A
3. What is Learning Analytics?
Academic Analytics Learning Analytics
A process for providing higher
education institutions with the data
necessary to support operational
and financial decision making*
The use of analytic techniques to help
target instructional, curricular, and
resources to support the achievement
of specific learning goals*
Focused on the business of the
institution
Focused on the student and their
learning behaviors
Management/executives are the
primary audience
Learners and instructors are the
primary audience
* - Source: Analytics in Higher Education: Establishing a Common Language
4. 2014 Open Learning Analytics Summit
Society for Learning Analytics (SoLAR) began
exploring openness in learning analytics in 2011
International OLA Summit held in March 2014
Participants identified OLA “knowledge domains”
as means to organize future work
5. OLA Knowledge Domains
Open Data and Models
Releasing data sets and models under open licenses
Open Research
Publishing research in open-access journals
Open-Source Software/Platforms
Open software, standards and APIs
Open Strategy and Policy
Open documents on strategy and policy
Open Learning Designs
Combine OER & LA to create new models of learning
6. Openness = Science
NORMAN BIER
DIRECTOR, OPEN LEARNING INI T IAT IVE
CARNEGIE MEL LON UNIVERSI TY
7. The changing value of content
Changing focus in OER community
Commoditization of content (Wiley: ‘content is
infrastructure’)
Instrumenting content is difficult and expensive
Well-instrumented content and the tools to analyze
student interactions with that content will continue to
increase in importance
9. Examples
Challenges Opportunities
CC-OLI Research
MOOC Research
UC Davis
Common measures of
outcomes and
achievement
Simon DataLab
Design Analytics
Stanford Outcomes
Analytics Service
Swappable Models
Learner Centered
10. MOOC RESEARCH AND
REPRODUCIBLE SCIENCE
ST IAN HÅKLEV
INST I TUT IONAL RESEARCHER, OPEN UTORONTO
UNIVERSI TY OF TORONTO
24. Open Data Models & OS
Learning Analytics Platform
JOSH BARON
SENIOR ACADEMIC TECHNOLOGY OF F ICER
MARIST COL LEGE
25. OAAI: Overview and Impact
EDUCAUSE Next
Generation Learning
Challenges (NGLC)
Funded by Bill and
Melinda Gates Foundations
$250,000 over a 15 month period
Goal: Leverage Big Data concepts to create an
open-source academic early alert system and
research “scaling factors”
26. Student Aptitude Data
(SATs, current GPA, etc.)
Student Demographic
Data (Age, gender, etc.)
Sakai Event Log Data
Sakai Gradebook Data
Step #1: Developed
model using historical
data
Predictive
Model
Scoring
Identifies
students
“at risk” to
not
complete
course
LMS Data SIS Data
OAAI Early Alert System Overview
Intervention Deployed
“Awareness” or Online
Academic Support
Environment (OASE)
“Creating an Open Academic
Early Alert System”
Model Developed
Using Historical Data
Academic Alert
Report (AAR)
27. Research Design
Deployed OAAI system to 2200 students across
four institutions
Two Community Colleges
Two Historically Black Colleges and Universities
Design > One instructor teaching 3 sections
One section was control, other 2 were treatment groups
Each instructor received an AAR three times during
the semester:
Intervals were 25%, 50% and 75% into the semester
28. Intervention Research Findings
Final Course Grades
Analysis showed a
statistically significant
positive impact on final
course grades
No difference between
treatment groups
Saw larger impact in
spring then fall
Similar trend amount low
income students
Mean Final Grade for "at Risk" Students
100
90
80
70
60
50
Awareness OASE Control
Final Grade (%)
29. Intervention Research Findings
Content Mastery
Student in intervention
groups were statistically
more likely to “master the
content” then those in
controls.
Content Mastery = Grade
of C or better
Similar for low income
students.
Content Mastery for "at Risk" Students
1000
800
600
400
200
0
Yes No Yes No
Control Intervention
Frequency
30. More Research Findings…
JAYAPRAKASH, S. M. , MOODY, E. W. , LAURÍA, E. J. ,
REGAN, J. R. , & BARON, J. D. (2014) . EARLY ALERT
OF ACADEMICALLY AT-RISK STUDENTS: AN OPEN
SOURCE ANALYTICS INITIATIVE. JOURNAL OF
LEARNING ANALYTICS, 1(1) , 6 -47.
31. Strategic Vision: Open
Learning Analytics Platform
Collection – Standards-based
data capture from any
potential source using
Experience API and/or IMS
Caliper/Senor API
Storage – Single repository
for all learning-related data
using Learning Record Store
(LRS) standard.
Analysis – Flexible
Learning Analytics
Processor (LAP) that can
handle data mining, data
processing (ETL), predictive
model scoring and reporting.
Communication –
Dashboard technology for
displaying LAP output.
Action – LAP output can be
fed into other systems to
trigger alerts, etc.
32. Access to Predictive Model and
related OS Software…
OAAI PREDICT IVE MODEL DOWNLOAD
HT TPS: / /CONF LUENCE.SAKAIPROJECT.ORG/X/ 8AWCB
APEREO LEARNING ANALYT ICS PROCESSOR DOWNLOAD
HT TPS: / /CONF LUENCE.SAKAIPROJECT.ORG/X/KWCVBQ
35. Discussion Questions
Do you feel LA will be important to OER and Open
Education in the future? How important?
Where do you see connections between the OLA
network and Open Education?
How might we best facilitate making connections
across different “networks”?
[insert more questions]
36. Additional Resources
European OLA Summit – December 1st (LACE)
http://www.laceproject.eu/
The Asilomar Convention for Learning Research in
Higher Education
http://asilomar-highered.info
Apereo Learning Analytics Initiative
https://confluence.sakaiproject.org/x/rIB_BQ
Society for Learning Analytics and Research
http://solaresearch.org
37. What is Student Success?
X
Z
Engagement
Learning Y
Credit: mike.sharkey@phoenix.com
Notes de l'éditeur
We should each briefly introduce ourselves…
Our goal/hope for this session is that we get folks here at OpenEd to start discussing the role that learning analytics, and specifically OPEN LA, will and should play in the future of the larger open education movement. I think many engaged in the emerging field of LA see its huge potential to impact on education in deep and significant ways yet much of the work to date in LA has been very “closed”. If this remains the case it could greatly limit the future impact of open education itself (at least that is what we might postulate). Longer term, it would be great to see connections forming between the Open Ed “network of practice” and the OLA “network”.
Before we jump in, its important that we make sure we’re all on the same page with regards to what we mean by “learning analytics” as it remains a new term that can have many definitions….
OK, so how did all of this get started? Most credit George Siemens and the Society for Learning Analytics Research, or SoLAR, with getting the conversation started around openness in learning analytics with a white paper that he and others at SoLAR drafted in 2011 which called for an “open framework and platform” for researching and deploying learning analytics at scale. More recently myself, George and Kim Arnold (who had been at Purdue working with John Campbell and is now at Wisconsin) worked to organize a two-day OLA Summit last March following the Learning Analytics and Knowledge (LAK) conference which brought together around 40 international LA leaders representing a range of institutions and organizations (only a few of which are listed here) from around the world. One of the major objectives and outcomes was the identification of key OLA “knowledge domains” around which we have started to identify current work and making connections across as means to grow and develop a large network of practice.
After two days of being locked up in a windowless hotel conference room on the outskirts of the Indianapolis airport eating bad food, the summit participants come up with these five knowledge domains…[review each one quickly…maybe give example using “knowledge maps”]….I’m going to now turn it over to my colleagues who will briefly share examples of work and issues in some of these domains….
Need for transparency into process
Broader need
Specific challenges
Interesting projects
First started thinking about this deeply from convo w/ Cable and David
Micheal Feldstein
Science vs. Alchemy
Proprietary, secret models
Cost of re-creation
Reproducability
OK, so what is the OAAI and how are we working to address this problem…with the goal of leveraging Big Data to create an open-source academic early alert system that allows us to predict which students are at risk to not complete the course (and do so early on in the semester) and then deploy an intervention to help that student succeed.
I’ll talk about our intervention strategies in a little more detail a bit later on in the presentation…