Open Seminar at the University of Oulu, 4th Dec. 2018 http://www.oulu.fi/koulutusteknologia/node/56057 Learning design and data analytics: from teacher communities to computer-supported collaborative learning scripts Presenter: Davinia Hernández-Leo, Associate Professor, Information and Communication Technologies Department, University Pompeu Fabra, Barcelona Brief description: I will present an overview of the educational technologies research conducted by the TIDE research group of the Information and Communication Technologies Department at Universitat Pompeu Fabra in Barcelona (http://www.upf.edu/web/tide @TIDE_UPF). The overview will be articulated around the perspective, central to TIDE work, of supporting teachers and teacher communities (e.g a school) in the design of the best possible (technology-enhanced) learning activities considering their students and their contexts. Main research contributions that will be presented include a community platform for integrated learning design (ILDE, including multiple authoring tools e.g. edCrumble), scalable and flexible orchestration of computer-supported collaborative learning scripts (PyramidApp), and the use of data analytics at different levels (learning, design, community) to support teachers in learning (re)design. The presentation will include results of European, Spanish and Catalan projects (METIS, RESET, CoT) and our initial work in recently started projects (SmartLET, Illuminated). Hernández-Leo, D., et al. (available online) Analytics for learning design: A layered framework and tools, British Journal of Educational Technology. https://doi.org/10.1111/bjet.12645 Hernández-Leo, D., et al. (2018). An Integrated Environment for Learning Design. Frontiers in ICT, 5, 9. doi: 10.3389/fict.2018.00009 Michos, K., Hernández-Leo, D., (2018) Supporting awareness in communities of learning design practice, Computers in Human Behavior, 85, 255-270. https://doi.org/10.1016/j.chb.2018.04.008 Michos, K., & Hernández-Leo, D., Albó, L. (2018). Teacher-led inquiry in technology-supported school communities. British Journal of Educational Technology 49(6), 1077-1095. https://doi.org/10.1111/bjet.12696. Manathunga, K., Hernández-Leo, D., (2018), Authoring and enactment ofmobile pyramid-based collaborative learning activities, British Journal ofEducational Technology, 49(2),262–275,doi:10.1111/bjet.12588 Albo L, Hernández-Leo D. edCrumble: designing for learning with data analytics. Proceedings of the 13th European Conference on Technology-Enhanced Learning (EC-TEL 2018); 2018 Sep 3-6; Leeds, UK, 605-609.

1. Learning design and data analytics:
from teacher communities to CSCL scripts
Visit to University of Oulu, GenZ seminar / LET seminar 4th December 2018
Davinia Hernández-Leo
TIDE, Universitat Pompeu Fabra, Barcelona
twitter: @TIDE_UPF
website : http://www.upf.edu/web/tide

2. Public, Young, Urban
1st
Spanish university in scientific productivity,
1st
Spanish university in THE
12th
in the World (under 50 years)
~10000 undergraduate,
~4000 graduate students,
~ 600 FT academics,
~ 340 permanent professors
https://www.youtube.com/watch?v=8dgvWkICCr8
https://www.youtube.com/watch?v=FjAuggflTVE
ICT department, Engineering degrees, Interdisciplinary Master Programs,
45 permanent positions, 9 ERCs, 5 spin-off companies,
888 undergraduate students, 180 Master and PhD students,
1rst Spanish university in the THE – Engineering & Technology
UPF, ICT Department
2

3. https://www.upf.edu/web/tide/people
TIDE research group
3

4. Learning design and data analytics: from teacher communities to CSCL scripts
Support teachers in designing the “best possible”
conditions for their students to learn, documenting them,
making their design ideas explicit and sharable.
(Conole, 2012; Agostinho, 2011; Laurillard, 2013;
Mor, Craft & Hernández-Leo, 2013)
Learning Design
4

5. Learning design and data analytics: from teacher communities to CSCL scripts
Learning Design
ACAD framew ork: Design -> Runtime
(Carvahlo & Goodyear, 2015)
ACAD framework
5

6. Learning design and data analytics: from teacher communities to CSCL scripts
The case of macro-scripting
Free collaboration does not necessarily lead to fruitful
learning. Macro-scripts structure the collaborative learning
flow (activity sequence, roles, group formation, resource
allocation) to trigger potentially effective social interactions.
(Dillenbourg & Tchounikine, 2007; Kobbe et al., 2007)
Learning Design in Collaborative Learning
6

7. Collaborative learning flow patterns
7
Learning design and data analytics: from teacher communities to CSCL scripts
Learning Design in Collaborative Learning
(Hernández-Leo et al., 2007)

8. 8
Learning design and data analytics: from teacher communities to CSCL scripts
Learning Design in Collaborative Learning
(Hernández-Leo et al., 2006; Villasclaras & Hernández-Leo et al., 2013)

9.  Flexibility challenges, especially in large classrooms
— Flow creation and control mechanisms for elasticity and dynamism
— PyramidApp
9
Learning design and data analytics: from teacher communities to CSCL scripts
Learning Design in Collaborative Learning
https://www.upf.edu/web/tide/tools
Manathunga, K., & Hernández‐Leo, D. (2018). Authoring and enactment of mobile pyramid‐based
collaborative learning activities. British Journal of Educational Technology, 49(2), 262-275.

10. 10
Learning design and data analytics: from teacher communities to CSCL scripts
Learning design tools
 Tools for conceptualization, authoring, implementation
 Generic vs. specific tools wrt. pedagogical approach
 Challenges in adoption

11. 11
Learning design and data analytics: from teacher communities to CSCL scripts
Learning design tools
 Co-design with teachers
 Emphasis in design for blended learning (PBL, flipped classroom, …)

12. 12
Learning design and data analytics: from teacher communities to CSCL scripts
Learning design tools
 Co-design with teachers
 Emphasis in design for blended learning (PBL, flipped classroom, …)
Albo L, Hernández-Leo D. edCrumble:
designing for learning with data
analytics. EC-TEL 2018. Best Demo
Award.

13. 13
Learning design and data analytics: from teacher communities to CSCL scripts
Learning design tools
 Tools for conceptualization, authoring, implementation
 Generic vs. specific tools wrt. pedagogical approach
 Challenges in adoption
— Set of “disperse” methodologies, representations, tools…
— Potential for sharing not exploited

14. 14
Learning design and data analytics: from teacher communities to CSCL scripts
Integrated Learning Design Environment
 Integrated solution to support the full lifecycle of (co-)designing,
sharing, and deploying ICT-based learning activities
Hernández-Leo, D., Asensio-Pérez et al. (2018). An Integrated Environment for Learning Design. Frontiers in ICT, 5, 9

15. 15
Learning design and data analytics: from teacher communities to CSCL scripts
Integrated Learning Design Environment

16. Integrated Learning Design Environment
 Used in several different educational contexts
— Well accepted in all its functions
— Fits the needs of different types of communities
— No difference in acceptance
— Flexible design processes
 Data used to evaluate and understand processes…
16
Learning design and data analytics: from teacher communities to CSCL scripts

17. ANALYTICS LAYERS FOR LEARNING DESIGN FRAMEWORK
Understanding and supporting learning design with analytics
17
Learning design and data analytics: from teacher communities to CSCL scripts
Community of practitioners
and related stakeholders
Learning Design (LD) tools
Learners experiencing
learning designs
Community Analytics
metrics and patterns of design activity
Design Analytics
metrics of pedagogical decisions
Learning Analytics
metrics of learners’ engagement,
achievement... aligned with design intent
Hernández-Leo, D., Martinez-Maldonado, R., Pardo, A., Muñoz-Cristóbal, J. A., & Rodríguez-Triana, M. J. et al. (available online)
Analytics for learning design: A layered framework and tools, British Journal of Educational Technology.
https://doi.org/10.1111/bjet.12645

18. Understanding and supporting learning design with analytics
18
Learning design and data analytics: from teacher communities to CSCL scripts

19. Understanding and supporting learning design with analytics
 Available data
 ILDE2, extension with data analytics features
— https://www.upf.edu/web/tide/tools/ilde2
 CoT project, 2 schools
 Master program for teacher training
 MOOC for teachers
 Teacher training workshops
19
Learning design and data analytics: from teacher communities to CSCL scripts

20. Analytics for learning design
Community of practitioners
and related stakeholders
Community Analytics
metrics and patterns of design activity
 Understanding community behaviour
 Triggering orientation and inspiration
 Data includes:
— what tools are being used,
— what types of designs are created, tags,
— who are the (co-)authors, edits, views,
— versioning of designs,
— social ratings …
Michos, K., Hernández-Leo, D., (2016)
Understanding Collective Behavior of Learning
Design Communities. EC-TEL 2016.
Learning design and data analytics: from teacher communities to CSCL scripts

21. Analytics for learning design
Community of practitioners
and related stakeholders
Community Analytics
metrics and patterns of design activity
 Understanding community behaviour
 Triggering orientation and inspiration
 Data includes:
— what tools are being used,
— what types of designs are created, tags,
— who are the (co-)authors, edits, views,
— versioning of designs,
— social ratings …
Michos, K., Hernández-Leo, D., (2018) Supporting awareness in communities of
learning design practice, Computers in Human Behavior, 85, 255-270.
Learning design and data analytics: from teacher communities to CSCL scripts

22. Learning Design (LD) tools
Design Analytics
metrics of pedagogical decisions
 Categorise learning designs
 Support awareness and reflection
about design decisions
 Data includes:
— learning objectives,
— type of tasks tasks,
— the social planes (individual, collaborative),
— resources and tools associated with the task,
— expected length of time for tasks, …
Analytics for learning design
Learning design and data analytics: from teacher communities to CSCL scripts
Albo L, Hernández-Leo D. edCrumble: designing for learning
with data analytics. EC-TEL 2018. Best Demo Award.

23. Learning Design (LD) tools
Design Analytics
metrics of pedagogical decisions
 Categorise learning designs
 Support awareness and reflection
about design decisions
 Interaction with the
Community Analytics layer
Analytics for learning design
Learning design and data analytics: from teacher communities to CSCL scripts
Albo L, Hernández-Leo D. edCrumble: designing for learning
with data analytics. EC-TEL 2018. Best Demo Award.

24. 4
Learners experiencing
learning designs
Learning Analytics
metrics of learners’ engagement,
achievement... aligned with design intent
 Accumulated evidence of design’s impact
 Support awareness and reflection
about effects of designs, redesign
 Data includes:
— snapshot data capturing relevant milestones
(e.g., access to resources or completion of tasks)
— data about the process (e.g., participation in
discussions)
— performance (e.g., grades)
— students’ and teachers’ opinions…
Analytics for learning design
Learning design and data analytics: from teacher communities to CSCL scripts
Michos, K., Manathunga, K., Hernández-Leo, D. (2016). Connecting pattern-based
learning designs with analytics: The case of the PyramidApp. In the Workshop on
Connecting Learning Analytics and Learning Design, CLAD - EC-TEL 2016

25. 4
Learners experiencing
learning designs
Learning Analytics
metrics of learners’ engagement,
achievement... aligned with design intent
 Accumulated evidence of design’s impact
 Support awareness and reflection
about effects of designs, redesign
 Data includes:
— snapshot data capturing relevant milestones
(e.g., access to resources or completion of tasks)
— data about the process (e.g., participation in
discussions)
— performance (e.g., grades)
— students’ and teachers’ opinions…
Analytics for learning design
Learning design and data analytics: from teacher communities to CSCL scripts
Michos, K., Fernández, A., Hernández-Leo, D., & Calvo, R.
(2017). Ld-Feedback App: Connecting Learning Designs with
Students’ and Teachers’ Perceived Experiences. EC-TEL.

26. 4
Learners experiencing
learning designs
Learning Analytics
metrics of learners’ engagement,
achievement... aligned with design intent
 Accumulated evidence of design’s impact
 Support awareness and reflection
about effects of designs, redesign
 Interaction with Design Analytics
and Community Analytics
— Teacher inquiry, Community inquiry
Analytics for learning design
Learning design and data analytics: from teacher communities to CSCL scripts
Michos, K., & Hernández-Leo, D., Albó, L. (2018).
Teacher-led inquiry in technology-supported school
communities. British Journal of Educational
Technology 49(6), 1077-1095.

27. Conclusions
 Data-driven understanding and support for learning design
 Three analytics layers: Community, Design and Learning analytics
 And interplays!
 Teacher inquiry, learning redesign, awareness, inspiration, community inquiry
 Currently:
— Extending experimentation
— Physiological data, multimodal analytics (Milica Vujovic)
— “Across-spaces” data, adaptive scripting (Ishari Amarasinghe)
— Neuroscience and learning design (Marc Beardsley)
— Understanding ethical implications in procedures
27
Learning design and data analytics: from teacher communities to CSCL scripts

28. Thank you!
twitter: @TIDE_UPF
website : http://www.upf.edu/web/tide
METIS, RESET, CoT, MdM, SmartLET, Illuminated projects