7. Learning Analytics
is an educational application of web analytics
combined with big data to gather and analyze details
of an individual learner’s interactions in an online
platform.
10. “this system was inspired
by Pandora, Amazon, or
Netflix as it employs a
prediction model that
combines hundreds of
thousands of past course
grades to make each
individualized
recommendation."
source: http://www.apsu.edu/
13. Bill Gates has advanced a
$1.1 million-plus biometric
sensor project that would
equip children with
Galvanic Skin Response
(GSR) bracelets as a
means to measure student
engagement
source: http://www.ohgizmo.com/2011/12/05/q-sensor-bracelet-can-tell-your-mood-and-tweet-about-it/
15. – Philip McRae
“Personalized learning does not build more
resilient, creative, entrepreneurial or empathetic
citizens through their individualized, linear and
mechanical software algorithms. Instead, they are
reductionist and primarily attend to those things that
can be easily digitized and tested.”
19. Paradox Learning Inc.
Dr. Stella Lee
Consultant/Researcher/Educator
stella@paradoxlearning.com
www.paradoxlearning.com
+1-403-918-5352
Editor's Notes
Thank you, thank you for the introduction.
Last summer, I was in New York City, travelling with a friend. Together, we went walking around a neighborhood that we were both unfamiliar with, just kicking around, and since we were hungry, we went looking for a place to eat.
We walked by many interesting looking restaurants but my friend insisted that she has THE place in mind based on Yelp’s recommendation and she was very proud of the fact that she has never eaten at a restaurant that has less than a four star rating on Yelp.
On the way there, we got so caught up in GPSing the location of the restaurant that we almost got hit by a cab, and we stopped paying attention to all the architecture, the shops, basically all the nuances of what make traveling so much fun.
Once we got there, there are also these “must tried” dishes according to the Yelp reviewers. If you are curious, It was the pate, and the lamb shank. Now, I am just as into trying new dishes as the next person, but this whole experience make me feel a bit contrived and even prescriptive. Never mind that I don’t even like pate but the fact that it was ravely reviewed make me feel compel to order it.
This experience makes me think about how much traveling to a new place is similar to learning, especially the kind of self-directed learning we do online. Just a little bit of background about me – I am a researcher, a consultant, and an educator in online education, I spend a lot of time thinking about e-learning, and how we can better design our learning experiences digitally.
To me, learning should be about exploring new ideas by trial-and-error, by discovery in context, and the ability to chance upon something. In other word: it is about the journey, and not just the destination.
In the e-learning space, we commonly rely on Learning Management Systems (LMS) to host, deliver, and record e-learning content and activities. There are roughly 640 LMSs on the market, most of which has some form of learning analytics built-in.
Examples of these interaction could be a click to download a course syllabus, or to watch a video lecture hosted in the system, or to post a discussion forum reply.
Learning analytics generally uses machine learning to build user model based on the continuous collection of data that describes individual users’ background, needs, preferences and interests. These analytics are then used to identify learning patterns, personalized content or some other aspect of learning, and in some instances, predict or influence learner actions.
I would like to give you some examples of how learning analytics is being used.
Austin Peay State University in Clarksville, Tennessee, has push out a course recommendation system called Degree Compass that use predictive analytics to help pair students with courses based on performances (ie. Grades) and program of study.
The system then would strongly recommends a course that is necessary for a student to graduate, that is core to the university curriculum and the student's major, and that the student is expected to succeed in academically. So the idea is that it is not based on the student’s interest in the course, or how other students enjoyed the course, but whether this student will have a high chance of getting a good grade, and therefore a higher chance of graduating, or be able to graduate faster. So the emphasis here is on student achievement based on grades, and not about the actual learning process, the application of the learning, or even the enjoyment of it.
In another example, Rio Salado College, a community college in Tempe, Arizona, runs a “predictive analytic” model that can predetermine student risk based on reduced engagement and provides intervening responses. They make these predictions based on online activity factors such as log-ins and site engagement within the LMS. And get this - this applies to every class.
The problem with intervention based on this model is that sometimes it backfire. In their case, they try to offer extra help to those students who are at risk of failing. They found that with one math class where they had repeated contact with students who were struggling, retention went down. The same intervention in a different class has a positive effect. It is also very difficult to isolate which variable has an impact on student learning, and sometimes these variable don’t really have an effect on anything.
Now this is an anonymized screenshot of my current online course aiming for working adults. You can see when they log in to the system and the frequency they log in. Please note that my student who is struggling the most and need the most support has the highest number of log-ins and has high engagement within the system. As it turned out, she has been away from school for some years, and she feels very unsure of her study skill, or how to interrupt any online instruction. And since she is currently unemployed, she tends to have a lot of screen time to post questions. But spending a lot of time and posting a lot of comments and questions in a discussion forum automatically mean that she is a star student. On the contrary, one of my best performing students in this class log in rather infrequently to the system. But when she does, she has very high quality and thoughtful comments to contribute to the discussion. And since she has already working in the field of the subject matter I teach in, she is able to follow the learning and work on the assignment independently with minimum support.
This is a wireless sensor that tracks physiological reactions of learners. This bracelet was able to distinguish between “electrodermal activity that increases during states such as excitement, attention or anxiety and decreases during states such as boredom or relaxation.”
If one were to use this data to provide an explanation - a teacher might be deemed highly effective if his students were in a statement of excitement or anxiety; and a teacher might be considered ineffective if her students were either bored or relaxed. However, this meter would be useless since a teacher might inspire anxiety by keeping students in constant fear and might look ineffective if students were silently reading a satisfying story. Imagine if you are trying to provide some sort of intervention based on the data, you might actually be harming the students and the teachers.
if you can’t appreciate the difference between an excited or an anxious learner – it is not really meaningful. Even anxiety could be a good thing sometimes. It makes us focus and it heightens our awareness. If we reduce our understanding of learning to prediction models, we are not really looking at the whole story.
Metrics alone cannot explain the complexity of human behavior.
The problem as I see it, decisions on what type of learning analytics and what data to mined is made primarily by administrators and managers and their focus is to cut cost and turn a profit.
It is not that they are wrong, but they simply have a different end goal in mind, and because of that, they are going to ask a very different set of questions. To get back to my analogy of traveling - if you were to ask a cab driver to take you some place, his motivation is to get you there in the shortest possible time so he can turn around and pick up more passengers - his goal is to make money. It is not about taking you through the most scenic route, or with the most comfortable ride. Right now, I see a huge discrepancy between measuring, modeling, predicting, and prescribing e-learning and actually engaging learners in a meaning context.
What we need to be disciplined about, and to be very mindful of, is to ask these essential questions
How can we design these so we can make our learning journey worthwhile instead of dehumanizing the whole online experience?
