2016-05-30 Venia Legendi (CEITER): Maria Jesus Rodriguez Triana

1. learning Design &
learning Analytics
Venia Lengendi
María Jesús Rodríguez-­Triana
maria.rodrigueztriana@epfl.ch
May 30th, 2016 – Tallinn (Estonia)
Aligning

2. Outline
2

3. Background
Bachelor:
Computer Science
PhD in ICT:
Linking LD &
LA in blended
CSCL
Master:
-­ Quality Engineering
-­ Computer Science
-­ Advanced Studies
Teacher:
Computer Science
Researcher:
GSIC @ Uva (Spain)
Researcher:
React @ EPFL
(Switzerland)
2000 2004 2005 2008 2010 Present2014
LD: Learning Design;; LA: Learning Analytics;; CSCL: Computer Supported Collaborative Learning;; IBL: Inquiry Based Learning
3
PostDoc:
Applying
LA to IBL

4. 4

5. Trends shaping education
-­ Globalisation
-­ The future of the nation state
-­ Are cities the new countries?
-­ Family matters
-­ Technology
5
[OECD, 2016]

6. European Trends and Grand Challenges in TEL
22 Grand Challenge Problems focusing on:
-­ Connecting Learners
-­ Orchestrating Learning
-­ Contextualising Learning
6
[Fischer et al., 2014] [Eberle et al., 2014]

7. Strategic goals for 2020:
-­ Change in the learning approach
-­ Competent and motivated teachers
-­ A digital focus in lifelong learning
-­ Align lifelong learning opportunities & labour market needs
-­ Equal opportunities & increased participation in lifelong learning
The Estonian Lifelong Learning Strategy 2020
7
(*) The Estonian Lifelong Learning Strategy 2020 (2014): https://www.hm.ee/sites/default/files/estonian_lifelong_strategy.pdf

8. 8

9. 9
[Rodríguez-­Triana et al., 2016] [Nussbaum & Diaz, 2013] [Prieto et al.,2015]

10. LASI @ Bilbao (Spain) June 28th, 2016
Orchestrating Learning Analytics Workshop (OrLA 2016)
http://lprisan.wix.com/orla2016
10
María Jesús
Rodríguez-­Triana
EPFL
Luis Pablo
Prieto
EPFL
Roberto
Martinez-­Maldonado
UTS
Yannis
Dimitriadis
UVa

11. 11
C
E
I
T
E
R
Project Vision

12. Research question
How to provide that:
• Support new learning paradigms
• Integrate technological innovations
• Overcome the adoption challenges
12
• learning environments & tools
• LA algorithms & visualizations

13. Proposal
-­ Pedagogical
-­ Technological
-­ Methodological
13

14. Proposal
-­ Pedagogical
-­ Technological
-­ Methodological
14

15. Learning scenario lifecycle
15
Learning Design
Design
ManagementAssessment
Reflection
? ?
?
[Dillenbourg, 2011a] [Dillenbourg, 2011b] [Prieto et al., 2011]

16. 16
Learning Analytics
Design
ManagementAssessment
Reflection
Learning scenario lifecycle
?
[Ferguson, 2012] [Sutherland et al., 2012]

17. ADA – Aligning learning Design & learning Analytics
17
[Wise, 2014] [Bakharia, et al., 2016]
[Rodríguez-­Triana et al., 2015b] [Vozniuk et al., 2015]
[Manske et al., 2015]
Benefits
• Orchestration support
• Pedagogically grounded tools
• LA-­aware learning designs
• Supervised designs
• Contextualised LA
• Formative assessment
• Personalization & gamification
Challenges
• Obtain learning designs minimizing
the user workload
• Inform the teacher about the LA
features of the learning tools
• Management of Ethical & Privacy
issues

18. ECTEL @ Lyon (France) June 28th, 2016
WS on Connecting Learning Analytics and Learning Design – CLAD 2016
http://clad2016.ld-­grid.org/
SI on Connecting Learning Design and Learning Analytics the Interaction
Design & Architecture(s) journal: http://www.mifav.uniroma2.it
18
MJ. Rodríguez-­Triana
EPFL
Luis P. Prieto
EPFL
Steven Warburton
US
Paul Salvador Inventado
CMU
Peter Scupelli
CMU
Davinia Hernández-­Leo
UPF
Yishay Mor
PAU
Bart Rienties
OU

19. Proposal
-­ Pedagogical
-­ Technological
-­ Methodological
19

20. CEITER infrastructure
20
Data warehouse
Data analyses & visualization Orchestration tools
Pedagogically grounded learning tools
& other ICT tools
Design
Implementation
[Rodríguez-­Triana et al., 2011] [Rodríguez-­Triana et al., 2016c]
[Schwendimann et al., 2016] [Vozniuk et al., 2015]
Standards (activity tracking, learning objects)
Ethical & privacy issues
Data filtering
& Integration
Design decisions
Pedagogically grounded analyses

21. JTELSS @ Tallinn (Estonia) June 28th, 2016
WS on Ethics & Privacy in Learning Analytics
http://go.epfl.ch/JTELSS2016_EPLA
21
Andrii
Vozniuk
EPFL
María Jesús
Rodríguez-­Triana
EPFL
Denis
Gillet
EPFL
ORGANISERS

22. LASI @ Bilbao (Spain) June 28th, 2016
WS on Contextual Learning analytics Enforcing data Ownership (CLEO).
Building Learning Dashboards with Elasticsearch, Logstash & Kibana
http://go.epfl.ch/LASI-­CLEO2016
22
Andrii
Vozniuk
EPFL
María Jesús
Rodríguez-­Triana
EPFL
Denis
Gillet
EPFL
ORGANISERS

23. CEITER infrastructure
23
[Rodríguez-­Triana et al., 2011] [Rodríguez-­Triana et al., 2016c]
[Schwendimann et al., 2016] [Vozniuk et al., 2015]
Benefits
• Richer data -­> More accurate & deep analyses
• Interoperability between learning & analyses tools
• Ethically & privacy respectful
Challenges
• Obtain learning designs
• Degradation of privacy settings
• Maintain accuracy when students opt out of the data collection
• Align learning activities, competencies & assessment
• Integrate personal, educational & institutional ecosystems

24. Proposal
-­ Pedagogical
-­ Technological
-­ Methodological
24

25. Methodology Benefits
25
• Iterative process
• Agile prototyping
• Refinement
Development
• Socio-­Cultural Activity Theory
• Design Based Research
• Participatory design
• Integration of user needs
• Engagement (interests, motivational & gender aspects)
• ICT / Data Literacy
• Contextual constraints
Research
[Rodríguez-­Triana et al., 2015a] [Rodríguez-­Triana et al., 2015b][Rodríguez-­Triana et al., 2016a]
• Pre-­service & In-­service teachers
• School, vocational & university
settings
• Enhance ICT & data literacy
• Long term adoption
• Learning to learn
Piloting &
training
• Mixed methods
• Evidence-­based
• Authentic studies
• Teaching gains/adoption
barriers
• Learning gains/adoption barriers
Evaluation

26. Overcoming CEITER challenges
26
[Karasavvidis, 2009]
Learning
Evidence
Educational
Innovation
Adoption
Current
educational
settings
• Support new learning paradigms
• Integrate technological innovations
• Overcome the adoption challenges

27. 27

28. Building our network and planning future research
28
Estonia: Ministry of Education & Research, Archimedes
Near countries: Interreg Central Baltic, EEA & Norway grants, Nordplus
Learning Analytics
Learning Design & Evaluation (IBL)
Infraestructure
(*) Ministry of Education and Research: https://www.hm.ee/en/activities/research-­and-­development/research-­programmes
(*) Archimedes foundation: http://archimedes.ee/en/foundation/
(*) Interreg Central Baltic: http://centralbaltic.eu/programme-­pages/specific-­objective-­42
(*) EEA & Norway grants: http://eeagrants.org
(*) Nordic Council of Ministers programme in the area of lifelong learning: http://www.nordplusonline.org/

29. Building our network and planning future research
29
Learning Analytics
Learning Design (CSCL/IBL)
Infrastructure & HCI (KMS, mobile)
Communities & Networks (ATEE, EUPAS, PAU, GoLab)
Ministries of Education (EUN)
European level: H2020, Erasmus+, Bilateral programs
(*) Horizon 2020: http://ec.europa.eu/programmes/horizon2020/
(*) Erasmus+: http://ec.europa.eu/programmes/erasmus-­plus
(*) Estonian national scholarship programme for international students, researchers and academic staff: http://adm.archimedes.ee
(*) Swiss National Foundation: http://www.snf.ch/en/funding/programmes/sinergia
(*) Spanish I+D+I international program: http://www.idi.mineco.gob.es/

30. Building our network and planning future research
International level: H2020, MSCA, Bilateral programmes (Endeavour Scholarships, ARC)
30
Learning Analytics
Learning Design (CSCL/IBL)
(*) Endeavour Scholarships: http://orau.org/endeavourscholarships/
(*) Australian Research Council: http://www.arc.gov.au/

31. Coming calls
• Spain -­ Programme for Research Aimed at the Challenges of Society (June 16th, 2016)
• Australia -­ Endeavour Scholarships (June 30th, 2016)
• Switzerland -­ Sinergia (December 1st, 2016)
• H2020 (December 8th, 2016)
• ICT-­11-­2017: Collective Awareness Platforms for Sustainability and Social Innovation
• ICT-­15-­2016-­2017 Big Data PPP: Large Scale Pilot actions in sectors best benefitting from
data-­driven innovation
31
• Interreg Central Baltic (February 2nd, 2017)
• Skilled and socially inclusive region (vocational education and training)
• Horizontal objectives (Equality, ICT)

32. Concluding remarks
32
-­ Pedagogical
-­ Technological
-­ Methodological
ADA
How to provide that:
• Support new learning paradigms
• Integrate technological innovations
• Overcome the adoption challenges
learning environments & tools
LA algorithms & visualizations
C
E
I
T
E
R

33. 33
?… questions, comments

34. Thank you
for your attention
María Jesús Rodríguez-­Triana
Email: maria.rodrigueztriana@epfl.ch
Skype contact: chus_mjrt
34

35. References I
35
• [Bakharia et al., 2016] A. Bakharia, L. Corrin, P. de Barba, G. Kennedy, D. Gasevic, R. Mulder, D. Williams, S. Dawson, and L. Lockyer. A
Conceptual Framework linking Learning Design with Learning Analytics Learning Analytics. In 6th International Conference on Learning
Analytics and Knowledge, 2016.
• [Dillenbourg, 2011a] P. Dillenbourg. Trends in classroom orchestration. Second research & technology scoutingreport, 2011.
• [Dillenbourg, 2011b] P. Dillenbourg. Design for orchestration. April, 2011.
• [Eberle et al., 2014] J. Eberle, K. Lund, P. Tchounikine, and F. Fischer, Challenge Problems in Technology-­Enhanced Learning II: MOOCs and
Beyond Perspectives for Research, Practice, and Policy Making Developed at the Alpine Rendez-­Vous in Villard-­ de-­Lans. London: Springer,
2014.
• [Ferguson, 2012] R. Ferguson. Learning analytics: drivers, developments and challenges. International Journal of Technology Enhanced
Learning, 4(5/6), 2012
• [Fischer et al., 2014] F. Fischer, F. Wild, R. Sutherland, and L. Zirn, Grand Challenges in Technology Enhanced Learning: Outcomes of the 3rd
Alpine Rendez-­Vous. London: Springer, 2014.
• [Karasavvidis, 2009] I. Karasavvidis, Activity theory as a conceptual framework for understanding teacher approaches to information and
communication technologies, Computers & Education. 53 (2): 436–444, 2009.
• [Manske et a;;., 2015 ] S. Manske, I.A. Chounta, M. J. Rodríguez-­Triana, D. Gillet, H.U. Hoppe. Exploring Deviation in Inquiry Learning: Degrees
of Freedom or Source of Problems. Proceedings of the 23rd International Conference on Computers in Education, ICCE 2015. Hangzhou,
China, 2015
• [Nussbaum & Díaz, 2013] M. Nussbaum, A. Diaz. “Classroom logistics: Integrating digital and non-­digital resources”. Computers & Education.
69: 493-­495, 2013
• [OECD, 2016] OECD. Trends Shaping Education 2016, OECD Publishing, Paris. 2016
• [Prieto et al., 2011] L. P. Prieto, M. Holenko Dlab, I. Gutiérrez, M. Abdulwahed, and W. Balid. Orchestratingtechnology enhancedlearning: a
literature review and a conceptual framework. International Journal of Technology EnhancedLearning, 3(6):583-­598, 2011.
[Prieto et al., 2016 ] L. P. Prieto, K. Sharma and P. Dillenbourg. Studying teacher orchestration load in technology-­enhanced classrooms: A
mixed-­method approach and case study. 10th European Conference on Technology-­Enhanced Learning (EC-­TEL 2015), Toledo, Spain, 2015.
• [Rodríguez-­Triana et al., 2016a] M. J. Rodríguez-­Triana, A. C. Holzer, L. P., Prieto, and D. Gillet, Examining the effects of social media in co-­
located classrooms: A case study based on SpeakUp. 11th European Conference on Technology Enhanced Learning: Adaptive and adaptable
learning. ECTEL2016. Lyon, France, 2016.
• [Rodríguez-­Triana et al., 2015a] M. J. Rodríguez-­Triana, A. Holzer, A. Vozniuk, D. Gillet. “Orchestrating Inquiry-­Based Learning Spaces: an
Analysis of Teacher Needs”. Proceedings of the 14th International Conference of Web Learning. ICWL 2015. Guangzhou, China. November
2015.

36. References II
36
• [Rodríguez-­Triana, et al. 2011] M. J. Rodríguez-­Triana, A. Martínez-­Monés J. I. Asensio-­Pérez. MonitoringCollaboration in Flexible and
Personal Learning Environments. Special issue on: EvaluatingEducativeExperiences of Flexible and Personal Learning Environments. ID&A
Journal. 11(12):51-­63, November 2011.
• [Rodríguez-­Triana, et al. 2013] M. J. Rodríguez-­Triana, A. Martínez-­Monés, J. I. Asensio-­Pérez, and Y. Dimitriadis. Towards a Script-­Aware
Monitoring Process of Computer-­Supported Collaborative LearningScenarios. International Journal on Technology Enhanced Learning. Special
issue on: Learning Analytics. 5 (2):151-­167, 2013.
• [Rodríguez-­Triana et al., 2015b] M. J. Rodríguez-­Triana, A. Martínez-­Monés, J. I. Asensio-­Pérez, and Y. Dimitriadis. Scripting and monitoring
meet each other: Aligning learning analytics and learning design to support teachers in orchestrating CSCL situations. British Journal of
Educational Technology, vol. 46, no. 2, pp. 330–343, 2015.
• [Rodríguez-­Triana et al., 2015c] Rodríguez-­Triana, M. J., Martínez-­Monés, A., Asensio-­Pérez, J. I., & Dimitriadis, Y. A monitoring-­aware
learning design process: Pilot studies in authentic CSCL scenarios. Computing & Informatics, 2015
• [Rodríguez-­Triana et al., 2016b] M. J. Rodríguez-­Triana, A. Martínez-­Monés, S.Villagrá-­Sobrino. Learning Analytics in small-­scale teacher-­led
innovations: Ethical and data privacy issues. Journal of Learning Analytics. Special section on: Ethics and Privacy for Learning Analytics. 3 (1):
43-­65, 2016.
• [Rodríguez-­Triana et al., 2016c] M. J. Rodríguez-­Triana, L. P. Prieto, A. Vozniuk, M. Shirvani Boroujeni and B. A. Schwendimannet al.
Monitoring, Awareness and Reflection in Blended Technology Enhanced Learning: a Systematic Review, accepted in International Journal of
Technology Enhanced Learning.
• [Schwendimann et al., 2016] B. A. Schwendimann, M. J. Rodríguez-­Triana, A. Vozniuk, L. P. Prieto and M. Shirvani Boroujeni et al.
Understanding learning at a glance: An overview of learning dashboard studies. International Learning Analytics and Knowledge, Edinburgh,
UK, 2016.
• [Vozniuk et al., 2015] A. Vozniuk, M. J. Rodríguez-­Triana, A. C. Holzer, S. Govaerts, D. Sandoz, and D. Gillet, Contextual Learning Analytics
Apps to Create Awareness in Blended Inquiry Learning, 14th International Conference on Information Technology Based Higher Education and
Training. Lisbon, Portugal, 2015.
• [Wise, 2014] A. F. Wise. Designing pedagogical interventions to support student use of learning analytics. In Proceedings of the 4th International
Conference on Learning Analytics And Knowledge -­ LAK ’14, 2014, pp. 203–211.