Ce diaporama a bien été signalé.
Nous utilisons votre profil LinkedIn et vos données d’activité pour vous proposer des publicités personnalisées et pertinentes. Vous pouvez changer vos préférences de publicités à tout moment.
Utilizing Web Analytics in the Context of Learning Analytics
for Large-Scale Online Learning
April 9, 2019
IEEE EDUCON 201...
The HPI MOOC Platform
openHPI – Digital Literacy for Lifelong Learners
Chart 2
Tobias Rohloff
■ Since 2012
■ 600,000+
enro...
The HPI MOOC Platform
openSAP – Enterprise MOOCs
Chart 3
■ Project partner
since 2013
■ 3,300,000+
enrollments
■ Training ...
The HPI MOOC Platform
OpenWHO – Response to Health Emergencies
Chart 4
■ Project partner
since 2017
■ 100,000+
enrollments...
The HPI MOOC Platform
Learning Analytics Architecture
Chart 5
■ Improve the learning experience and outcome with Learning ...
Web Analytics
Chart 6
■ Collection and analysis of user behavior
■ Major goals
□ Improving design and UX
□ Optimizing e-co...
Web Analytics in the Context of E-Learning
Chart 7
■ Utilization for analyses of learning behavior
■ Take advantage of wel...
■ To what extend can Web Analytics methods be used in the context of Learning
Analytics to gather insights in learning beh...
The HPI MOOC Platform
Learning Analytics with Web Analytics
Chart 9
■ Send anonymized and whitelisted
events to external W...
Selected Analytics Suites for MOOC Platforms
Chart 10
Open Source Closed Source
Web
Analytics
Learning
Analytics
( )
+
… …...
■ Most popular Web Analytics service
□ Used by 56.9% of the top ten million
websites of the Alexa popularity ranking
■ Com...
■ Existing concept of processing pipelines for analytics events
□ Extract, transform and load events into a data store
Cha...
Reporting of Web Analytics Insights
Revised Platform Dashboard
Chart 13
Tobias Rohloff
Utilizing
Web Analytics
in the Cont...
Reporting of Web Analytics Insights
Revised Course Dashboard
Chart 14
Tobias Rohloff
Utilizing
Web Analytics
in the Contex...
Chart 15
Reporting of Web Analytics Insights
Web Analytics Metrics
■ Session durations
■ Session frequencies
■ Daytime act...
■ Online survey with platform owners/teaching teams of openHPI/openSAP
■ Goals
□ Measure usefulness of WA insights in cont...
Evaluation Results
Chart 17
0% 20% 40% 60% 80% 100%
Satisfaction
Ease of Use
Usefulness
Existing Dashboards Revised Dashbo...
Chart 18
Shortcomings
Interaction data is
stored aggregated
Behavior of single
user is getting lost
Strong focus on
e-comm...
Synergies of Web Analytics and Learning Analytics
Chart 19
MOOC Platform
Stakeholders
Platform
Owner
Learner
Teaching
Team...
■ How can online learning activity be mapped to Web Analytics concepts? (RQ 1.a)
□ Development of a mapping for modeling l...
■ Can Web Analytics methods improve the usefulness of Learning Analytics insights?
(RQ 1.c)
□ New metrics do not have a si...
Utilizing Web Analytics in the Context of Learning Analytics
for Large-Scale Online Learning
Tobias Rohloff, M.Sc. Learnin...
Prochain SlideShare
Chargement dans…5
×

Utilizing Web Analytics in the Context of Learning Analytics for Large-Scale Online Learning

Today, Web Analytics (WA) is commonly used to obtain key information about users and their behavior on websites. Besides, with the rise of online learning, Learning Analytics (LA) emerged as a separate research field for collecting and analyzing learners' interactions on online learning platforms. Although the foundation of both methods is similar, WA has not been profoundly used for LA purposes. However, especially large-scale online learning environments may benefit from WA as it is more sophisticated and well-established in comparison to LA. Therefore, this paper aims to examine to what extent WA can be utilized in this context, without compromising the learners' data privacy. For this purpose, Google Analytics was integrated into the Massive Open Online Course platform of the Hasso Plattner Institute as a proof of concept. It was tested with two deployments of the platform: openHPI and openSAP, where thousands of learners gain academic and industry knowledge about engineering education. Besides capturing behavioral data, the platforms' existing LA dashboards were extended by WA metrics. The evaluation of the integration showed that WA covers a large part of the relevant metrics and is particularly suitable for obtaining an overview of the platform's global activity, but reaches its limitations when it comes to learner-specific metrics.

  • Identifiez-vous pour voir les commentaires

Utilizing Web Analytics in the Context of Learning Analytics for Large-Scale Online Learning

  1. 1. Utilizing Web Analytics in the Context of Learning Analytics for Large-Scale Online Learning April 9, 2019 IEEE EDUCON 2019 Dubai, UAE Tobias Rohloff, Sören Oldag, Jan Renz, Christoph Meinel Hasso Plattner Institute Potsdam, Germany
  2. 2. The HPI MOOC Platform openHPI – Digital Literacy for Lifelong Learners Chart 2 Tobias Rohloff ■ Since 2012 ■ 600,000+ enrollments ■ Free courses by HPI professors, staff and students ■ About digital technologies, transformation and innovation https://open.hpi.de/ Utilizing Web Analytics in the Context of Learning Analytics
  3. 3. The HPI MOOC Platform openSAP – Enterprise MOOCs Chart 3 ■ Project partner since 2013 ■ 3,300,000+ enrollments ■ Training for professionals and customers https://open.sap.com/ Tobias Rohloff Utilizing Web Analytics in the Context of Learning Analytics
  4. 4. The HPI MOOC Platform OpenWHO – Response to Health Emergencies Chart 4 ■ Project partner since 2017 ■ 100,000+ enrollments ■ Contain disease outbreaks ■ Manage health emergencies https://openwho.org/ Tobias Rohloff Utilizing Web Analytics in the Context of Learning Analytics
  5. 5. The HPI MOOC Platform Learning Analytics Architecture Chart 5 ■ Improve the learning experience and outcome with Learning Analytics ■ Extensible event-driven architecture based on processing pipelines was implemented ■ But how do other large-scale applications from other domains implement analytics? Tobias Rohloff Utilizing Web Analytics in the Context of Learning Analytics
  6. 6. Web Analytics Chart 6 ■ Collection and analysis of user behavior ■ Major goals □ Improving design and UX □ Optimizing e-commerce □ Measuring success of campaigns □ Identifying website errors ■ Initially developed for e-commerce websites Measure Report Analyze Optimize Tobias Rohloff Utilizing Web Analytics in the Context of Learning Analytics
  7. 7. Web Analytics in the Context of E-Learning Chart 7 ■ Utilization for analyses of learning behavior ■ Take advantage of well-established techniques ■ Example: analogy to e-commerce process Add to cart Checkout Complete order Enroll into course Study contents Pass final exam Tobias Rohloff Utilizing Web Analytics in the Context of Learning Analytics
  8. 8. ■ To what extend can Web Analytics methods be used in the context of Learning Analytics to gather insights in learning behavior and outcome on e-learning platforms? (RQ 1) □ How can online learning activity be mapped to Web Analytics concepts? (RQ 1.a) □ How can different stakeholders of e-learning platforms be provided with Web Analytics insights? (RQ 1.b) □ Can Web Analytics methods improve the usefulness of Learning Analytics insights? (RQ 1.c) Chart 8 Research Questions Tobias Rohloff Utilizing Web Analytics in the Context of Learning Analytics
  9. 9. The HPI MOOC Platform Learning Analytics with Web Analytics Chart 9 ■ Send anonymized and whitelisted events to external Web Analytics tools ■ Query metrics about the learning behavior from external tools to gather new insights Tobias Rohloff Utilizing Web Analytics in the Context of Learning Analytics
  10. 10. Selected Analytics Suites for MOOC Platforms Chart 10 Open Source Closed Source Web Analytics Learning Analytics ( ) + … … Tobias Rohloff Utilizing Web Analytics in the Context of Learning Analytics
  11. 11. ■ Most popular Web Analytics service □ Used by 56.9% of the top ten million websites of the Alexa popularity ranking ■ Comprehensive range of features ■ Well-documented with lots of available resources ■ Support for server-side data collection via Measurement Protocol □ Integrates with existing Learning Analytics service □ Events can be sent deferred (offline-capability) □ Allows to control which data is sent to Google Analytics Chart 11 Google Analytics Tobias Rohloff Utilizing Web Analytics in the Context of Learning Analytics
  12. 12. ■ Existing concept of processing pipelines for analytics events □ Extract, transform and load events into a data store Chart 12 Web Analytics Data Collection ■ Implement Google Analytics data collection as separate pipeline Tobias Rohloff Utilizing Web Analytics in the Context of Learning Analytics
  13. 13. Reporting of Web Analytics Insights Revised Platform Dashboard Chart 13 Tobias Rohloff Utilizing Web Analytics in the Context of Learning Analytics
  14. 14. Reporting of Web Analytics Insights Revised Course Dashboard Chart 14 Tobias Rohloff Utilizing Web Analytics in the Context of Learning Analytics
  15. 15. Chart 15 Reporting of Web Analytics Insights Web Analytics Metrics ■ Session durations ■ Session frequencies ■ Daytime activity ■ Average time on item ■ Exit rate of items ■ … Tobias Rohloff Utilizing Web Analytics in the Context of Learning Analytics
  16. 16. ■ Online survey with platform owners/teaching teams of openHPI/openSAP ■ Goals □ Measure usefulness of WA insights in context of LA □ Examine whether intentions and goals of the revision have been accomplished ■ Comparison of usefulness, ease of use, satisfaction of existing and revised dashboards Chart 16 Evaluation Methodology Tobias Rohloff Utilizing Web Analytics in the Context of Learning Analytics
  17. 17. Evaluation Results Chart 17 0% 20% 40% 60% 80% 100% Satisfaction Ease of Use Usefulness Existing Dashboards Revised Dashboards n = 11 Tobias Rohloff Utilizing Web Analytics in the Context of Learning Analytics
  18. 18. Chart 18 Shortcomings Interaction data is stored aggregated Behavior of single user is getting lost Strong focus on e-commerce Sessions are the central element Tobias Rohloff Utilizing Web Analytics in the Context of Learning Analytics
  19. 19. Synergies of Web Analytics and Learning Analytics Chart 19 MOOC Platform Stakeholders Platform Owner Learner Teaching Team Web Analytics Learning Analytics Researcher Tobias Rohloff Utilizing Web Analytics in the Context of Learning Analytics
  20. 20. ■ How can online learning activity be mapped to Web Analytics concepts? (RQ 1.a) □ Development of a mapping for modeling learning activity as WA hits ■ How can different stakeholders of e-learning platforms be provided with Web Analytics insights? (RQ 1.b) □ Integration of WA metrics into dashboards intended for platform owners and teaching teams □ No difference between visualizing WA and LA insights recognized Chart 20 Conclusion Tobias Rohloff Utilizing Web Analytics in the Context of Learning Analytics
  21. 21. ■ Can Web Analytics methods improve the usefulness of Learning Analytics insights? (RQ 1.c) □ New metrics do not have a significant impact on usefulness of dashboards □ Large part of existing LA capabilities could be realized by using WA methods ■ To what extend can Web Analytics methods be used in the context of Learning Analytics to gather insights in learning behavior and outcome on e-learning platforms? (RQ 1) □ WA can be used to retrieve a large part of relevant LA metrics □ Applicability highly depends on the type of stakeholder □ Advanced features cannot be utilized due to a mismatch of concepts Chart 21 Conclusion (cont.) Tobias Rohloff Utilizing Web Analytics in the Context of Learning Analytics
  22. 22. Utilizing Web Analytics in the Context of Learning Analytics for Large-Scale Online Learning Tobias Rohloff, M.Sc. Learning Engineering Research Group Chair of Internet Technologies and Systems Hasso Plattner Institute

×