The document discusses the design of a writing app to provide formative feedback on reflective writing. It describes why reflection is important for learning and professional development. It then outlines a conceptual model and rubric for assessing the depth of reflection in writing, from non-reflective to reflective to critically reflective. The goal is to develop students' capability for written reflection and help them progress from simply describing experiences to more deeply examining feelings, challenges, and lessons learned. The app would provide automated feedback on drafts to support developing reflective writing skills.
This document provides guidance on reflective writing for students. It discusses the key components of reflective writing such as critical analysis, description, interpretation, and outcomes. Students are encouraged to critically analyze their experiences by considering different perspectives and synthesizing literature, policy, and theories from their field. The document reviews examples of student writing and identifies where they demonstrate description, interpretation, and outcomes. It also provides tips for language use in reflective writing, such as using first person to discuss experiences and third person when referring to literature. Students practice reflective writing by discussing a session experience. The document aims to help students structure and strengthen their reflective writing skills.
This document provides guidance on reflective writing for students. It discusses the importance of critical analysis in reflection and evaluating examples of reflective writing. Students are encouraged to synthesize methods for critical thinking, reading, and writing to create pieces of reflective writing. Good reflective writing involves description, interpretation, and outcomes. It requires critically analyzing oneself and experiences using relevant theories and considering different perspectives. Models can help structure reflection, but critical questioning and analysis of the broader context are also important. The document provides examples and exercises to help students improve their reflective writing skills.
The document discusses designing reflective activities for learners. It defines reflection as a meaning-making process that involves thinking systematically and interacting with others. Reflective activities should provide opportunities for learners to think for themselves, relate ideas to experiences, and reflect on their learning. Good activities use action verbs to target higher cognitive levels and prompt critical reflection on what was learned, how, and why. The document provides guidelines for developing reflective exercises that meet these criteria.
This document provides guidance on reflective writing for postgraduate students. It explains that reflective writing involves both describing experiences and critically analyzing them using relevant theories and frameworks. Examples of student reflections are provided to demonstrate strong reflective writing. The document emphasizes that reflective writing requires interpreting experiences at both a deeper personal level and a broader contextual level. It also stresses the importance of asking critical questions to analyze situations from different perspectives rather than taking information at face value. Various reflective models are presented to help structure reflective thinking and writing, including Driscoll's "What? So what? Now what?" approach and Gibbs' reflective cycle. The document concludes by outlining linguistic features commonly used in reflective writing, such as using first person and action verbs.
The document discusses Jerome Bruner's constructivist learning theory, which posits that learning is an active process where learners construct new ideas based upon their current and past knowledge. It emphasizes that learning should involve exploring large concepts, inquiry-based learning, and making connections between ideas. According to constructivism, knowledge is temporary, culturally mediated, and developed through language. The roles of the teacher involve posing problems, structuring learning around core concepts, seeking student perspectives, and adapting instruction based on student understanding.
This document discusses reflective teaching and provides an overview of key concepts and frameworks related to teacher reflection. It defines reflective teaching as carefully considering one's practices in light of knowledge and beliefs. Reflective teachers engage in both reflection-on-action, which involves understanding past events, and reflection-in-action, which occurs while teaching. The document also outlines various approaches teachers can use to reflect, including portfolios, journaling, and using Quality Teaching frameworks as a lens. Barriers to reflection like a lack of time and perceived competence are also addressed.
This document provides guidance on reflective writing for students. It discusses the key components of reflective writing such as critical analysis, description, interpretation, and outcomes. Students are encouraged to critically analyze their experiences by considering different perspectives and synthesizing literature, policy, and theories from their field. The document reviews examples of student writing and identifies where they demonstrate description, interpretation, and outcomes. It also provides tips for language use in reflective writing, such as using first person to discuss experiences and third person when referring to literature. Students practice reflective writing by discussing a session experience. The document aims to help students structure and strengthen their reflective writing skills.
This document provides guidance on reflective writing for students. It discusses the importance of critical analysis in reflection and evaluating examples of reflective writing. Students are encouraged to synthesize methods for critical thinking, reading, and writing to create pieces of reflective writing. Good reflective writing involves description, interpretation, and outcomes. It requires critically analyzing oneself and experiences using relevant theories and considering different perspectives. Models can help structure reflection, but critical questioning and analysis of the broader context are also important. The document provides examples and exercises to help students improve their reflective writing skills.
The document discusses designing reflective activities for learners. It defines reflection as a meaning-making process that involves thinking systematically and interacting with others. Reflective activities should provide opportunities for learners to think for themselves, relate ideas to experiences, and reflect on their learning. Good activities use action verbs to target higher cognitive levels and prompt critical reflection on what was learned, how, and why. The document provides guidelines for developing reflective exercises that meet these criteria.
This document provides guidance on reflective writing for postgraduate students. It explains that reflective writing involves both describing experiences and critically analyzing them using relevant theories and frameworks. Examples of student reflections are provided to demonstrate strong reflective writing. The document emphasizes that reflective writing requires interpreting experiences at both a deeper personal level and a broader contextual level. It also stresses the importance of asking critical questions to analyze situations from different perspectives rather than taking information at face value. Various reflective models are presented to help structure reflective thinking and writing, including Driscoll's "What? So what? Now what?" approach and Gibbs' reflective cycle. The document concludes by outlining linguistic features commonly used in reflective writing, such as using first person and action verbs.
The document discusses Jerome Bruner's constructivist learning theory, which posits that learning is an active process where learners construct new ideas based upon their current and past knowledge. It emphasizes that learning should involve exploring large concepts, inquiry-based learning, and making connections between ideas. According to constructivism, knowledge is temporary, culturally mediated, and developed through language. The roles of the teacher involve posing problems, structuring learning around core concepts, seeking student perspectives, and adapting instruction based on student understanding.
This document discusses reflective teaching and provides an overview of key concepts and frameworks related to teacher reflection. It defines reflective teaching as carefully considering one's practices in light of knowledge and beliefs. Reflective teachers engage in both reflection-on-action, which involves understanding past events, and reflection-in-action, which occurs while teaching. The document also outlines various approaches teachers can use to reflect, including portfolios, journaling, and using Quality Teaching frameworks as a lens. Barriers to reflection like a lack of time and perceived competence are also addressed.
The document discusses critical reflective writing in social work. It defines critical reflection as analyzing experiences by considering various contexts and assumptions. Critical reflective writing demonstrates deeper critical thinking on experiences by incorporating experience, theory, and academic content. It differs from typical essays through its use of first person and experience as evidence. Structural models for critical reflective writing generally move from describing an experience to analyzing it to articulating implications. The document provides examples of language and structure used in academic critical reflective writing.
This document discusses reflective writing analytics (RWA), an approach to providing automated feedback on reflective writing assignments at scale. RWA was developed using a conceptual framework of reflection derived from literature. It analyzes student writing for rhetorical moves related to reflection using natural language processing. RWA was tested on pharmacy students, with most finding the feedback helpful and many revising drafts based on it. Limitations include the need for strong pedagogical framing and more discipline-specific feedback. Ongoing work aims to address these issues.
This document provides guidance on reflective writing for students. It explains that reflective writing requires both descriptive and analytical components. Students must describe events and experiences, but also critically analyze them by considering underlying reasons and implications. The document reviews examples of student reflective writing and identifies key features like using first person to describe experiences, third person for referring to literature, and hedging language. It emphasizes that reflective writing necessitates placing experiences in broader theoretical and policy contexts by synthesizing literature. Tools like reflective questions, models, and reading grids can help students structure and deepen their reflective analysis.
The document discusses the importance of teaching critical thinking skills to students. It provides examples of famous people like Einstein, Beethoven, and Edison who struggled in traditional education but excelled in creative and critical thinking. The document defines critical thinking as processing information to solve problems, make decisions, and anticipate the future. It recommends teaching critical thinking by asking open-ended questions, connecting lessons to student experience, and encouraging group work and problem solving.
Reflective learning involves reflecting on experiences to improve learning outcomes. It requires reflecting on what was learned, what gaps in knowledge remain, and how to apply lessons to new situations. Various models provide frameworks for structured reflection, including identifying key events, responding emotionally and intellectually, relating content to prior knowledge, reasoning about broader implications, and reconstructing understanding. Regular reflection improves metacognition, motivation, and learning.
This document outlines an upcoming teaching enrichment series session on critical thinking. The session will be led by Anita Gonzalez and cover definitions of critical thinking, two perspectives on teaching it, and mapping critical thinking skills. Critical thinking is defined as reasoning dispassionately and using evidence to solve problems and draw conclusions. It involves both general skills like logic as well as discipline-specific skills. Effective teaching of critical thinking engages students with specific critical thinking skills within a knowledge base.
The document discusses Kolb's experiential learning theory, which presents a four-stage learning cycle of concrete experience, reflective observation, abstract conceptualization, and active experimentation. It also describes four different learning styles - diverging, assimilating, converging, and accommodating - based on how people process information at each stage of the cycle. Kolb's model emphasizes that effective learning is an ongoing process that involves acquiring knowledge through transforming experiences.
The document discusses Bloom's Taxonomy, which is a classification of learning objectives into different levels of complexity and specificity. It was created by Benjamin Bloom in 1955 to categorize educational goals and objectives. The taxonomy contains three domains: cognitive, affective, and psychomotor. Within the cognitive domain are knowledge, comprehension, application, analysis, synthesis and evaluation. Each level builds on the previous ones as the cognitive process becomes more complex. The document provides examples of verbs to describe each level of cognitive learning. It also discusses the affective and psychomotor domains and provides guidance on how to incorporate Bloom's Taxonomy into teaching practices.
The document discusses reflective practice and its importance in learning. It provides insights from scholars on reflection and how it enables meaning making from experiences. Reflection is seen as crucial for connecting experiences and knowledge. The document also outlines John Dewey's criteria for effective reflection, including reflection as connection, systematic/disciplined practice, social pedagogy, and personal growth. Examples of reflective practices from various institutions are presented, highlighting elements like scaffolding, iteration, and facilitating social feedback. The purpose is to examine reflective practices and identify portable elements that could work across disciplines.
The document discusses teaching critical thinking skills to students. It defines critical thinking as teaching students how to think about and evaluate subject matter, not just teaching content. It provides several approaches to teaching critical thinking, such as modeling critical thinking processes, creating a culture of inquiry, and engaging students in thinking. Several teaching ideas are also presented, such as using questions to analyze engineering cases and articles. Methods for assessing critical thinking like rubrics and peer evaluations are also outlined.
This document summarizes an orientation for Medaille College interns. It discusses:
- The objectives of connecting students to resources, reviewing the internship process, and introducing experiential learning theory
- Key messages about taking ownership of one's learning and being responsible and accountable
- Definitions of internships as structured, supervised opportunities to apply classroom learning in real-world contexts
- Benefits of internships like applying knowledge to problems and exploring career choices
- Resources available to students, including guidelines and requirements
- The roles and responsibilities of students throughout the internship stages
- Kolb's experiential learning cycle of concrete experience, reflective observation, abstract conceptualization, and active experimentation as a framework
This document provides an overview of character development initiatives in the Greater Essex County District School Board. It discusses that character development is about more than academics and involves developing the whole person through engagement. Teachers play a key role in modeling behaviors and integrating character traits into lessons. Sample lessons show how to teach character traits like conflict resolution through historical examples.
The document discusses Bloom's Taxonomy and its use in developing learning objectives. It outlines Bloom's original and revised cognitive domains, providing examples of verbs and outcomes for different levels. It also provides guidance on writing measurable learning objectives, including specifying the audience, behavior, conditions, and criteria. Educators are encouraged to design objectives and assessments that promote enduring understanding and allow students to demonstrate their learning in authentic ways.
The document discusses the challenges of assessing reflective work. It notes that reflection encourages independent thinking without a set curriculum, so assessing it can be difficult. While reflection is increasingly included in education, outcomes must be attached to it if its value is to be recognized. Some propose assessing the language used in reflections to determine the depth and quality, but this favors those who recognize assessment rules over those who may reflect deeply in other ways. Overall, valid assessment of reflection is complicated and there are no simple answers, but the practice will continue growing in education.
This document discusses examination and evaluation in education. It defines key concepts like tests, measurement, and evaluation. It outlines Bloom's taxonomy of educational objectives, which categorizes learning objectives into cognitive, affective, and psychomotor domains. The document also discusses different types of tests, evaluation techniques, characteristics of good evaluation tools, and challenges in examinations like mass copying. It emphasizes using a variety of evaluation techniques to assess different domains of learning beyond just cognitive skills.
Enquiry-Based Learning: The fundamentals, the basicscilass.slideshare
Presented at the LTEA Conference 2008, University of Sheffield.
The aim of this interactive workshop is to describe and discuss the basic processes of inquiry-based learning. We will look at what we expect students to do when they make inquiries, how we can generate enthusiasm to make inquiries, what we expect them to bring back from inquiries and what we can do with all this to turn it into learning.
This document discusses impact assessment and why it is valuable. It defines impact assessment as measuring the tangible and intangible effects of an entity's actions or influence on others. Impact assessment is constructive, produces individual outcomes, and engages learners and instructors. It often uses action research methods like guided reflection and scholarly personal narratives. Impact assessment goes beyond just reflection to measure insight and the lasting effects on learners, lives, and the world. Conducting impact assessment makes education more valuable by preparing students and obtaining useful institutional data. It also more deeply engages students by meeting their needs.
This document provides an overview of a presentation on reflective writing. It defines reflective writing as focusing on personal experiences and questioning beliefs and knowledge. The document outlines different purposes for reflective writing, such as for professional development, critical thinking, and processing experiences. It also discusses challenges of reflective writing and provides a framework and examples. One example reflective journal entry describes a situation, analyzes factors, and links it to leadership theory. The document concludes with a checklist for evaluating reflective writing.
This document discusses a hybrid learning model that brings together learning and teaching. It presents different learning events such as receives, debates, experiments, and creates. For each event, it suggests describing the teacher and learner roles using verbs. Practitioners found using this model helped them better understand the learner's role and perspective. It also helped capture teaching and learning processes in a way that is understandable, generic, reusable, and shareable.
The Generative AI System Shock, and some thoughts on Collective Intelligence ...Simon Buckingham Shum
Keynote Address: Team-based Learning Collaborative Asia Pacific Community (TBLC-APC) Symposium (“Impact of emerging technologies on learning strategies”) 8-9 February 2024, Sydney https://tbl.sydney.edu.au
Could Generative AI Augment Reflection, Deliberation and Argumentation?Simon Buckingham Shum
Bing Chat analyzed an argumentative article by generating an argument map in ArgDown syntax highlighting the key claim, premises, and their relationships. However, it also introduced some fallacious elements like incorrect summaries, commentary nodes, and incorrect argument classifications. While this shows generative AI has potential for argument analysis, more evaluation is needed to address inaccuracies that could undermine critical thinking. With improvements, it may help facilitate deliberations by preparing resources or assisting engaged citizens in analyzing complex arguments.
Contenu connexe
Similaire à On the Design of a Writing App offering 24/7 Formative Feedback on Reflective Writing
The document discusses critical reflective writing in social work. It defines critical reflection as analyzing experiences by considering various contexts and assumptions. Critical reflective writing demonstrates deeper critical thinking on experiences by incorporating experience, theory, and academic content. It differs from typical essays through its use of first person and experience as evidence. Structural models for critical reflective writing generally move from describing an experience to analyzing it to articulating implications. The document provides examples of language and structure used in academic critical reflective writing.
This document discusses reflective writing analytics (RWA), an approach to providing automated feedback on reflective writing assignments at scale. RWA was developed using a conceptual framework of reflection derived from literature. It analyzes student writing for rhetorical moves related to reflection using natural language processing. RWA was tested on pharmacy students, with most finding the feedback helpful and many revising drafts based on it. Limitations include the need for strong pedagogical framing and more discipline-specific feedback. Ongoing work aims to address these issues.
This document provides guidance on reflective writing for students. It explains that reflective writing requires both descriptive and analytical components. Students must describe events and experiences, but also critically analyze them by considering underlying reasons and implications. The document reviews examples of student reflective writing and identifies key features like using first person to describe experiences, third person for referring to literature, and hedging language. It emphasizes that reflective writing necessitates placing experiences in broader theoretical and policy contexts by synthesizing literature. Tools like reflective questions, models, and reading grids can help students structure and deepen their reflective analysis.
The document discusses the importance of teaching critical thinking skills to students. It provides examples of famous people like Einstein, Beethoven, and Edison who struggled in traditional education but excelled in creative and critical thinking. The document defines critical thinking as processing information to solve problems, make decisions, and anticipate the future. It recommends teaching critical thinking by asking open-ended questions, connecting lessons to student experience, and encouraging group work and problem solving.
Reflective learning involves reflecting on experiences to improve learning outcomes. It requires reflecting on what was learned, what gaps in knowledge remain, and how to apply lessons to new situations. Various models provide frameworks for structured reflection, including identifying key events, responding emotionally and intellectually, relating content to prior knowledge, reasoning about broader implications, and reconstructing understanding. Regular reflection improves metacognition, motivation, and learning.
This document outlines an upcoming teaching enrichment series session on critical thinking. The session will be led by Anita Gonzalez and cover definitions of critical thinking, two perspectives on teaching it, and mapping critical thinking skills. Critical thinking is defined as reasoning dispassionately and using evidence to solve problems and draw conclusions. It involves both general skills like logic as well as discipline-specific skills. Effective teaching of critical thinking engages students with specific critical thinking skills within a knowledge base.
The document discusses Kolb's experiential learning theory, which presents a four-stage learning cycle of concrete experience, reflective observation, abstract conceptualization, and active experimentation. It also describes four different learning styles - diverging, assimilating, converging, and accommodating - based on how people process information at each stage of the cycle. Kolb's model emphasizes that effective learning is an ongoing process that involves acquiring knowledge through transforming experiences.
The document discusses Bloom's Taxonomy, which is a classification of learning objectives into different levels of complexity and specificity. It was created by Benjamin Bloom in 1955 to categorize educational goals and objectives. The taxonomy contains three domains: cognitive, affective, and psychomotor. Within the cognitive domain are knowledge, comprehension, application, analysis, synthesis and evaluation. Each level builds on the previous ones as the cognitive process becomes more complex. The document provides examples of verbs to describe each level of cognitive learning. It also discusses the affective and psychomotor domains and provides guidance on how to incorporate Bloom's Taxonomy into teaching practices.
The document discusses reflective practice and its importance in learning. It provides insights from scholars on reflection and how it enables meaning making from experiences. Reflection is seen as crucial for connecting experiences and knowledge. The document also outlines John Dewey's criteria for effective reflection, including reflection as connection, systematic/disciplined practice, social pedagogy, and personal growth. Examples of reflective practices from various institutions are presented, highlighting elements like scaffolding, iteration, and facilitating social feedback. The purpose is to examine reflective practices and identify portable elements that could work across disciplines.
The document discusses teaching critical thinking skills to students. It defines critical thinking as teaching students how to think about and evaluate subject matter, not just teaching content. It provides several approaches to teaching critical thinking, such as modeling critical thinking processes, creating a culture of inquiry, and engaging students in thinking. Several teaching ideas are also presented, such as using questions to analyze engineering cases and articles. Methods for assessing critical thinking like rubrics and peer evaluations are also outlined.
This document summarizes an orientation for Medaille College interns. It discusses:
- The objectives of connecting students to resources, reviewing the internship process, and introducing experiential learning theory
- Key messages about taking ownership of one's learning and being responsible and accountable
- Definitions of internships as structured, supervised opportunities to apply classroom learning in real-world contexts
- Benefits of internships like applying knowledge to problems and exploring career choices
- Resources available to students, including guidelines and requirements
- The roles and responsibilities of students throughout the internship stages
- Kolb's experiential learning cycle of concrete experience, reflective observation, abstract conceptualization, and active experimentation as a framework
This document provides an overview of character development initiatives in the Greater Essex County District School Board. It discusses that character development is about more than academics and involves developing the whole person through engagement. Teachers play a key role in modeling behaviors and integrating character traits into lessons. Sample lessons show how to teach character traits like conflict resolution through historical examples.
The document discusses Bloom's Taxonomy and its use in developing learning objectives. It outlines Bloom's original and revised cognitive domains, providing examples of verbs and outcomes for different levels. It also provides guidance on writing measurable learning objectives, including specifying the audience, behavior, conditions, and criteria. Educators are encouraged to design objectives and assessments that promote enduring understanding and allow students to demonstrate their learning in authentic ways.
The document discusses the challenges of assessing reflective work. It notes that reflection encourages independent thinking without a set curriculum, so assessing it can be difficult. While reflection is increasingly included in education, outcomes must be attached to it if its value is to be recognized. Some propose assessing the language used in reflections to determine the depth and quality, but this favors those who recognize assessment rules over those who may reflect deeply in other ways. Overall, valid assessment of reflection is complicated and there are no simple answers, but the practice will continue growing in education.
This document discusses examination and evaluation in education. It defines key concepts like tests, measurement, and evaluation. It outlines Bloom's taxonomy of educational objectives, which categorizes learning objectives into cognitive, affective, and psychomotor domains. The document also discusses different types of tests, evaluation techniques, characteristics of good evaluation tools, and challenges in examinations like mass copying. It emphasizes using a variety of evaluation techniques to assess different domains of learning beyond just cognitive skills.
Enquiry-Based Learning: The fundamentals, the basicscilass.slideshare
Presented at the LTEA Conference 2008, University of Sheffield.
The aim of this interactive workshop is to describe and discuss the basic processes of inquiry-based learning. We will look at what we expect students to do when they make inquiries, how we can generate enthusiasm to make inquiries, what we expect them to bring back from inquiries and what we can do with all this to turn it into learning.
This document discusses impact assessment and why it is valuable. It defines impact assessment as measuring the tangible and intangible effects of an entity's actions or influence on others. Impact assessment is constructive, produces individual outcomes, and engages learners and instructors. It often uses action research methods like guided reflection and scholarly personal narratives. Impact assessment goes beyond just reflection to measure insight and the lasting effects on learners, lives, and the world. Conducting impact assessment makes education more valuable by preparing students and obtaining useful institutional data. It also more deeply engages students by meeting their needs.
This document provides an overview of a presentation on reflective writing. It defines reflective writing as focusing on personal experiences and questioning beliefs and knowledge. The document outlines different purposes for reflective writing, such as for professional development, critical thinking, and processing experiences. It also discusses challenges of reflective writing and provides a framework and examples. One example reflective journal entry describes a situation, analyzes factors, and links it to leadership theory. The document concludes with a checklist for evaluating reflective writing.
This document discusses a hybrid learning model that brings together learning and teaching. It presents different learning events such as receives, debates, experiments, and creates. For each event, it suggests describing the teacher and learner roles using verbs. Practitioners found using this model helped them better understand the learner's role and perspective. It also helped capture teaching and learning processes in a way that is understandable, generic, reusable, and shareable.
Similaire à On the Design of a Writing App offering 24/7 Formative Feedback on Reflective Writing (20)
The Generative AI System Shock, and some thoughts on Collective Intelligence ...Simon Buckingham Shum
Keynote Address: Team-based Learning Collaborative Asia Pacific Community (TBLC-APC) Symposium (“Impact of emerging technologies on learning strategies”) 8-9 February 2024, Sydney https://tbl.sydney.edu.au
Could Generative AI Augment Reflection, Deliberation and Argumentation?Simon Buckingham Shum
Bing Chat analyzed an argumentative article by generating an argument map in ArgDown syntax highlighting the key claim, premises, and their relationships. However, it also introduced some fallacious elements like incorrect summaries, commentary nodes, and incorrect argument classifications. While this shows generative AI has potential for argument analysis, more evaluation is needed to address inaccuracies that could undermine critical thinking. With improvements, it may help facilitate deliberations by preparing resources or assisting engaged citizens in analyzing complex arguments.
Slides from my contribution to the panel convened by Jeremy Roschelle at the International Society for the Learning Sciences: Engaging Learning Scientists in Policy Challenges: AI and the Future of Learning
Is “The Matter With Things” also what’s the matter with Learning Analytics?Simon Buckingham Shum
The document summarizes key points from Iain McGilchrist's book "The Matter With Things" regarding how the left and right hemispheres of the brain attend to the world in different ways. Specifically:
- The left hemisphere views the world in a detached, fragmented, and mechanical way while the right hemisphere sees it as interconnected, contextualized, and animate.
- Learning analytics risks amplifying the left hemisphere tendencies by focusing too much on data, analytics, and AI without balancing it with right hemisphere qualities like intuition, imagination, and empathy.
- To address this, learning analytics should cultivate humility, critical thinking, and find ways to advance rebalancing science and reason with intuition and imagination. This could
Deliberative Democracy as a strategy for co-designing university ethics aro...Simon Buckingham Shum
Buckingham Shum, S. (2021). Deliberative Democracy as a strategy for co-designing university ethics around analytics and AI in education. AARE2021: Australian Association for Research in Education, 28 Nov. – 2 Dec. 2021
Deliberative Democracy as a Strategy for Co-designing University Ethics Around Analytics and AI in Education
Simon Buckingham Shum
Connected Intelligence Centre, University of Technology Sydney
Universities can see an increasing range of student and staff activity as it becomes digitally visible in their platform ecosystems. The fields of Learning Analytics and AI in Education have demonstrated the significant benefits that ethically responsible, pedagogically informed analysis of student activity data can bring, but such services are only possible because they are undeniably a form of “surveillance”, raising legitimate questions about how the use of such tools should be governed.
Our prior work has drawn on the rich concepts and methods developed in human-centred system design, and participatory/co-design, to design, deploy and validate practical tools that give a voice to non-technical stakeholders (e.g. educators; students) in shaping such systems. We are now expanding the depth and breadth of engagement that we seek, looking to the Deliberative Democracy movement for inspiration. This is a response to the crisis in confidence in how typical democratic systems engage citizens in decision making. A hallmark is the convening of a Deliberative Mini-Public (DMP) which may work at different scales (organisation; community; region; nation) and can take diverse forms (e.g. Citizens’ Juries; Citizens’ Assemblies; Consensus Conferences; Planning Cells; Deliberative Polls). DMP’s combination of stratified random sampling to ensure authentic representation, neutrally facilitated workshops, balanced expert briefings, and real support from organisational leaders, has been shown to cultivate high quality dialogue in sometimes highly conflicted settings, leading to a strong sense of ownership of the DMP's final outputs (e.g. policy recommendations).
This symposium contribution will describe how the DMP model is informing university-wide consultation on the ethical principles that should govern the use of analytics and AI around teaching and learning data.
ICQE20: Quantitative Ethnography Visualizations as Tools for ThinkingSimon Buckingham Shum
Slides for this keynote talk to the 2nd International Conference on Quantitative Ethnography
http://simon.buckinghamshum.net/2021/02/icqe2020-keynote-qe-viz-as-tools-for-thinking/
24/7 Instant Feedback on Writing: Integrating AcaWriter into your TeachingSimon Buckingham Shum
https://cic.uts.edu.au/events/24-7-instant-feedback-on-writing-integrating-acawriter-into-your-teaching-2-dec/
What difference could instant feedback on draft writing make to your students? Over the last 5 years the Connected Intelligence Centre has been developing and piloting an automated feedback tool for academic writing (AcaWriter), working closely with academics across several faculties. The research portal documents how educators and students engage with this kind of AI, and what we’ve learnt about integrating it into teaching and assessment.
In May, AcaWriter was launched to all students along with an information portal. Now we want to start upskilling academics, tutors and learning technologists, in a monthly session to give you the chance to learn about AcaWriter, and specifically, good practices for integrating it into your subject. CIC can support you, and we hope you may be interested in co-designing publishable research.
AcaWriter handles several different ‘genres’ of writing, including reflective writing (e.g. a Reflective Essay; Reflective Blogs/Journals on internships/work-placements) and analytical writing (e.g. Argumentative Essays; Research Abstracts & Introductions).
This briefing will demo AcaWriter, and show it can be embedded in student activities. We hope this sparks ideas for your own teaching, which we can discuss in more detail.
An introduction to argumentation for UTS:CIC PhD students (with some Learning Analytics examples, but potentially of wider interest to students/researchers)
Webinar: Learning Informatics Lab, University of Minnesota
Replay the talk: https://youtu.be/dcJZeDIMr2I
Learning Informatics
AI • Analytics • Accountability • Agency
Simon Buckingham Shum
Professor of Learning Informatics
Director, Connected Intelligence Centre
University of Technology Sydney
Abstract:
“Health Informatics”. “Urban Informatics”. “Social Informatics”. Informatics offers systemic ways of analyzing and designing the interaction of natural and artificial information processing systems. In the context of education, I will describe some Learning Informatics lenses and practices which we have developed for co-designing analytics and AI with educators and students. We have a particular focus on closing the feedback loop to equip learners with competencies to navigate a complex, uncertain future, such as critical thinking, professional reflection and teamwork. En route, we will touch on how we build educators’ trust in novel tools, our design philosophy of “embracing imperfection” in machine intelligence, and the ways that these infrastructures embody values. Speaking from the perspective of leading an institutional innovation centre in learning analytics, I hope that our experiences spark productive reflection around as the UMN Learning Informatics Lab builds its program.
Biography:
Simon Buckingham Shum is Professor of Learning Informatics at the University of Technology Sydney, where he serves as inaugural director of the Connected Intelligence Centre. CIC is a transdisciplinary innovation centre, using analytics to provide new insights for university teams, with particular expertise in educational data science. Simon’s career-long fascination with software’s ability to make thinking visible has seen him active in communities including Computer-Supported Cooperative Work, Hypertext, Design Rationale, Scholarly Publishing, Semantic Web, Computational Argumentation, Educational Technology and Learning Analytics. The challenge of visualizing contested knowledge has produced several books: Visualizing Argumentation, Knowledge Cartography, and Constructing Knowledge Art. He has been active over the last decade in shaping the field of Learning Analytics, co-founding the Society for Learning Analytics Research, and catalyzing several strands: Social Learning Analytics, Discourse Analytics, Dispositional Analytics and Writing Analytics. http://Simon.BuckinghamShum.net
Despite AI’s potential for beneficial use, it creates important risks for Australians. AI, big data, and AI-informed decision making can cause exclusion, discrimination, skill loss, and economic impact; and can affect privacy, security of critical infrastructure and social well-being. What types of technology raise particular human rights concerns? Which human rights are particularly implicated?
Abstract: The emerging configuration of educational institutions, technologies, scientific practices, ethics policies and companies can be usefully framed as the emergence of a new “knowledge infrastructure” (Paul Edwards). The idea that we may be transitioning into significantly new ways of knowing – about learning and learners, teaching and teachers – is both exciting and daunting, because new knowledge infrastructures redefine roles and redistribute power, raising many important questions. What should we see when open the black box powering analytics? How do we empower all stakeholders to engage in the design process? Since digital infrastructure fades quickly into the background, how can researchers, educators and learners engage with it mindfully? This isn’t just interesting to ponder academically: your school or university will be buying products that are being designed now. Or perhaps educational institutions should take control, building and sharing their own open source tools? How are universities accelerating the transition from analytics innovation to infrastructure? Speaking from the perspective of leading an institutional innovation centre in learning analytics, I hope that our experiences designing code, competencies and culture for learning analytics sheds helpful light on these questions.
Towards Collaboration Translucence: Giving Meaning to Multimodal Group DataSimon Buckingham Shum
Vanessa Echeverria, Roberto Martinez-Maldonado, and Simon Buck- ingham Shum.. 2019. Towards Collaboration Translucence: Giving Meaning to Multimodal Group Data. In Proceedings of ACM CHI conference (CHI’19). ACM, New York, NY, USA, Paper 39, 16 pages. https://doi.org/10.1145/3290605.3300269
Collocated, face-to-face teamwork remains a pervasive mode of working, which is hard to replicate online. Team members’ embodied, multimodal interaction with each other and artefacts has been studied by researchers, but due to its complexity, has remained opaque to automated analysis. However, the ready availability of sensors makes it increasingly affordable to instrument work spaces to study teamwork and groupwork. The possibility of visualising key aspects of a collaboration has huge potential for both academic and professional learning, but a frontline challenge is the enrichment of quantitative data streams with the qualitative insights needed to make sense of them. In response, we introduce the concept of collaboration translucence, an approach to make visible selected features of group activity. This is grounded both theoretically (in the physical, epistemic, social and affective dimensions of group activity), and contextually (using domain-specific concepts). We illustrate the approach from the automated analysis of healthcare simulations to train nurses, generating four visual proxies that fuse multimodal data into higher order patterns.
This document summarizes findings from previous research on the impact of personalized learning support and feedback. It discusses how academics found value in personalized outreach in one study. A second study found improved academic performance, especially for high users of personalized support. Students generally wanted notifications if their performance was unsatisfactory in another study. Students also reported feeling better and more able to improve after receiving personalized messages in one case, though some felt worse. Care needs to be taken in message design to avoid defensive reactions from students.
Panel held at LAK13: 3rd International Conference on Learning Analytics & Knowledge
http://simon.buckinghamshum.net/2013/03/lak13-edu-data-scientists-scarce-breed
Educational Data Scientists: A Scarce Breed
The Educational Data Scientist is currently a poorly understood, rarely sighted breed. Reports vary: some are known to be largely nocturnal, solitary creatures, while others have been reported to display highly social behaviour in broad daylight. What are their primary habits? How do they see the world? What ecological niches do they occupy now, and will predicted seismic shifts transform the landscape in their favour? What survival skills do they need when running into other breeds? Will their numbers grow, and how might they evolve? In this panel, the conference will hear and debate not only broad perspectives on the terrain, but will have been exposed to some real life specimens, and caught glimpses of the future ecosystem.
Keynote Address, International Conference of the Learning Sciences, London Festival of Learning
Transitioning Education’s Knowledge Infrastructure:
Shaping Design or Shouting from the Touchline?
Abstract: Bit by bit, a data-intensive substrate for education is being designed, plumbed in and switched on, powered by digital data from an expanding sensor array, data science and artificial intelligence. The configurations of educational institutions, technologies, scientific practices, ethics policies and companies can be usefully framed as the emergence of a new “knowledge infrastructure” (Paul Edwards).
The idea that we may be transitioning into significantly new ways of knowing – about learning and learners – is both exciting and daunting, because new knowledge infrastructures redefine roles and redistribute power, raising many important questions. For instance, assuming that we want to shape this infrastructure, how do we engage with the teams designing the platforms our schools and universities may be using next year? Who owns the data and algorithms, and in what senses can an analytics/AI-powered learning system be ‘accountable’? How do we empower all stakeholders to engage in the design process? Since digital infrastructure fades quickly into the background, how can researchers, educators and learners engage with it mindfully? If we want to work in “Pasteur’s Quadrant” (Donald Stokes), we must go beyond learning analytics that answer research questions, to deliver valued services to frontline educational users: but how are universities accelerating the analytics innovation to infrastructure transition?
Wrestling with these questions, the learning analytics community has evolved since its first international conference in 2011, at the intersection of learning and data science, and an explicit concern with those human factors, at many scales, that make or break the design and adoption of new educational tools. We are forging open source platforms, links with commercial providers, and collaborations with the diverse disciplines that feed into educational data science. In the context of ICLS, our dialogue with the learning sciences must continue to deepen to ensure that together we influence this knowledge infrastructure to advance the interests of all stakeholders, including learners, educators, researchers and leaders.
Speaking from the perspective of leading an institutional analytics innovation centre, I hope that our experiences designing code, competencies and culture for learning analytics sheds helpful light on these questions.
Kirsty Kitto, Simon Buckingham Shum, and Andrew Gibson. (2018). Embracing Imperfection in Learning Analytics. In Proceedings of LAK18: International Conference on Learning Analytics and Knowledge, March 5–9, 2018, Sydney, NSW, Australia, pp.451-460. (ACM, New York, NY, USA). https://doi.org/10.1145/3170358.3170413
Open Access: http://simon.buckinghamshum.net/2018/01/embracing-imperfection-in-learning-analytics
Abstract: Learning Analytics (LA) sits at the confluence of many contributing disciplines, which brings the risk of hidden assumptions inherited from those fields. Here, we consider a hidden assumption derived from computer science, namely, that improving computational accuracy in classification is always a worthy goal. We demonstrate that this assumption is unlikely to hold in some important educational contexts, and argue that embracing computational “imperfection” can improve outcomes for those scenarios. Specifically, we show that learner-facing approaches aimed at “learning how to learn” require more holistic validation strategies. We consider what information must be provided in order to reasonably evaluate algorithmic tools in LA, to facilitate transparency and realistic performance comparisons.
LAND USE LAND COVER AND NDVI OF MIRZAPUR DISTRICT, UPRAHUL
This Dissertation explores the particular circumstances of Mirzapur, a region located in the
core of India. Mirzapur, with its varied terrains and abundant biodiversity, offers an optimal
environment for investigating the changes in vegetation cover dynamics. Our study utilizes
advanced technologies such as GIS (Geographic Information Systems) and Remote sensing to
analyze the transformations that have taken place over the course of a decade.
The complex relationship between human activities and the environment has been the focus
of extensive research and worry. As the global community grapples with swift urbanization,
population expansion, and economic progress, the effects on natural ecosystems are becoming
more evident. A crucial element of this impact is the alteration of vegetation cover, which plays a
significant role in maintaining the ecological equilibrium of our planet.Land serves as the foundation for all human activities and provides the necessary materials for
these activities. As the most crucial natural resource, its utilization by humans results in different
'Land uses,' which are determined by both human activities and the physical characteristics of the
land.
The utilization of land is impacted by human needs and environmental factors. In countries
like India, rapid population growth and the emphasis on extensive resource exploitation can lead
to significant land degradation, adversely affecting the region's land cover.
Therefore, human intervention has significantly influenced land use patterns over many
centuries, evolving its structure over time and space. In the present era, these changes have
accelerated due to factors such as agriculture and urbanization. Information regarding land use and
cover is essential for various planning and management tasks related to the Earth's surface,
providing crucial environmental data for scientific, resource management, policy purposes, and
diverse human activities.
Accurate understanding of land use and cover is imperative for the development planning
of any area. Consequently, a wide range of professionals, including earth system scientists, land
and water managers, and urban planners, are interested in obtaining data on land use and cover
changes, conversion trends, and other related patterns. The spatial dimensions of land use and
cover support policymakers and scientists in making well-informed decisions, as alterations in
these patterns indicate shifts in economic and social conditions. Monitoring such changes with the
help of Advanced technologies like Remote Sensing and Geographic Information Systems is
crucial for coordinated efforts across different administrative levels. Advanced technologies like
Remote Sensing and Geographic Information Systems
9
Changes in vegetation cover refer to variations in the distribution, composition, and overall
structure of plant communities across different temporal and spatial scales. These changes can
occur natural.
বাংলাদেশের অর্থনৈতিক সমীক্ষা ২০২৪ [Bangladesh Economic Review 2024 Bangla.pdf] কম্পিউটার , ট্যাব ও স্মার্ট ফোন ভার্সন সহ সম্পূর্ণ বাংলা ই-বুক বা pdf বই " সুচিপত্র ...বুকমার্ক মেনু 🔖 ও হাইপার লিংক মেনু 📝👆 যুক্ত ..
আমাদের সবার জন্য খুব খুব গুরুত্বপূর্ণ একটি বই ..বিসিএস, ব্যাংক, ইউনিভার্সিটি ভর্তি ও যে কোন প্রতিযোগিতা মূলক পরীক্ষার জন্য এর খুব ইম্পরট্যান্ট একটি বিষয় ...তাছাড়া বাংলাদেশের সাম্প্রতিক যে কোন ডাটা বা তথ্য এই বইতে পাবেন ...
তাই একজন নাগরিক হিসাবে এই তথ্য গুলো আপনার জানা প্রয়োজন ...।
বিসিএস ও ব্যাংক এর লিখিত পরীক্ষা ...+এছাড়া মাধ্যমিক ও উচ্চমাধ্যমিকের স্টুডেন্টদের জন্য অনেক কাজে আসবে ...
This slide is special for master students (MIBS & MIFB) in UUM. Also useful for readers who are interested in the topic of contemporary Islamic banking.
ISO/IEC 27001, ISO/IEC 42001, and GDPR: Best Practices for Implementation and...PECB
Denis is a dynamic and results-driven Chief Information Officer (CIO) with a distinguished career spanning information systems analysis and technical project management. With a proven track record of spearheading the design and delivery of cutting-edge Information Management solutions, he has consistently elevated business operations, streamlined reporting functions, and maximized process efficiency.
Certified as an ISO/IEC 27001: Information Security Management Systems (ISMS) Lead Implementer, Data Protection Officer, and Cyber Risks Analyst, Denis brings a heightened focus on data security, privacy, and cyber resilience to every endeavor.
His expertise extends across a diverse spectrum of reporting, database, and web development applications, underpinned by an exceptional grasp of data storage and virtualization technologies. His proficiency in application testing, database administration, and data cleansing ensures seamless execution of complex projects.
What sets Denis apart is his comprehensive understanding of Business and Systems Analysis technologies, honed through involvement in all phases of the Software Development Lifecycle (SDLC). From meticulous requirements gathering to precise analysis, innovative design, rigorous development, thorough testing, and successful implementation, he has consistently delivered exceptional results.
Throughout his career, he has taken on multifaceted roles, from leading technical project management teams to owning solutions that drive operational excellence. His conscientious and proactive approach is unwavering, whether he is working independently or collaboratively within a team. His ability to connect with colleagues on a personal level underscores his commitment to fostering a harmonious and productive workplace environment.
Date: May 29, 2024
Tags: Information Security, ISO/IEC 27001, ISO/IEC 42001, Artificial Intelligence, GDPR
-------------------------------------------------------------------------------
Find out more about ISO training and certification services
Training: ISO/IEC 27001 Information Security Management System - EN | PECB
ISO/IEC 42001 Artificial Intelligence Management System - EN | PECB
General Data Protection Regulation (GDPR) - Training Courses - EN | PECB
Webinars: https://pecb.com/webinars
Article: https://pecb.com/article
-------------------------------------------------------------------------------
For more information about PECB:
Website: https://pecb.com/
LinkedIn: https://www.linkedin.com/company/pecb/
Facebook: https://www.facebook.com/PECBInternational/
Slideshare: http://www.slideshare.net/PECBCERTIFICATION
Main Java[All of the Base Concepts}.docxadhitya5119
This is part 1 of my Java Learning Journey. This Contains Custom methods, classes, constructors, packages, multithreading , try- catch block, finally block and more.
How to Manage Your Lost Opportunities in Odoo 17 CRMCeline George
Odoo 17 CRM allows us to track why we lose sales opportunities with "Lost Reasons." This helps analyze our sales process and identify areas for improvement. Here's how to configure lost reasons in Odoo 17 CRM
This presentation was provided by Steph Pollock of The American Psychological Association’s Journals Program, and Damita Snow, of The American Society of Civil Engineers (ASCE), for the initial session of NISO's 2024 Training Series "DEIA in the Scholarly Landscape." Session One: 'Setting Expectations: a DEIA Primer,' was held June 6, 2024.
This document provides an overview of wound healing, its functions, stages, mechanisms, factors affecting it, and complications.
A wound is a break in the integrity of the skin or tissues, which may be associated with disruption of the structure and function.
Healing is the body’s response to injury in an attempt to restore normal structure and functions.
Healing can occur in two ways: Regeneration and Repair
There are 4 phases of wound healing: hemostasis, inflammation, proliferation, and remodeling. This document also describes the mechanism of wound healing. Factors that affect healing include infection, uncontrolled diabetes, poor nutrition, age, anemia, the presence of foreign bodies, etc.
Complications of wound healing like infection, hyperpigmentation of scar, contractures, and keloid formation.
On the Design of a Writing App offering 24/7 Formative Feedback on Reflective Writing
1. UTS CRICOS 00099F
On the Design of a Writing App offering 24/7
Formative Feedback on Reflective Writing
Simon Buckingham Shum
Director, Connected Intelligence Centre
Professor of Learning Informatics
Simon.BuckinghamShum.net
https://cic.uts.edu.au • https://Simon.BuckinghamShum.net
twitter @sbuckshum • Simon.BuckinghamShum@uts.edu.au
University of Melbourne, Assessment Research Centre • 19th Oct 2022
3. We’re after those breakthrough moments…
“Before I came to this class I had never really thought much about gender and
what it means or that it is something that is fluid. Taking this course was
completely eye opening and really made me think about things I have never
had the chance to think about.” (Buckingham Shum, et al. 2017, p.76)
“It was a great surprise to me to realize that coordination was such an
important aspect of engineering” (Reidsema, et al., 2010, p.9)
“I had never previously given thought to this idea, as I had thought that a
patient’s medications and medical conditions are fine to discuss with other
family members.” (Lucas, et al. 2019, p.1267)
Cited in: Liu, M., Kitto, K. & Buckingham Shum, S. (2021). Combining Factor Analysis with Writing Analytics for the Formative
Assessment of Written Reflection. Computers in Human Behavior. https://doi.org/10.1016/j.chb.2021.106733
4. …IF you know how to make your
reflections visible in language
Some of the graduate attributes we want to build
Authentic Performance
Professional Disposition
Emotionally Integrated
Reflection
Theory & Practice
Reflective writing is a window
onto the mind
Sense of Calling
Epistemological Shifts
7. Reflective Writing
• Promotes self-reflection and critical thinking (Mezirow, 1990).
• Provides evidence of students’ ability to: show self-critical insight, identify
challenging issues, connect academic with experiential knowledge,
acknowledge emotions and feelings, and reflect on how they can apply
insights in the future (Gibson et al.,2017).
• Helps to reflect on practice-based work placements, teamwork and learning
(Boud, Keogh & Walker, 1985; Schön, 1987; Hatton & Smith, 1995, Tsingos et al.,2015).
8. Scholarship on reflective writing pedagogy
Challenging to teach and write
Goes against everything students have been drilled in re. performative writing
Common to get superficial or non-reflective description of experience mentioned in
reflective writing
Challenging to assess
Criteria may be unclear to the grading team
Writing is time consuming to grade: impossible to give timely, detailed feedback to a
cohort, and certainly not on draft after draft
9. Reflective Writing Analytics focus on…
• Automatic detection of reflective elements
Reflective rhetorical move detection using rule-based concept matching
(Buckingham Shum et al., 2017; Gibson et al., 2017) and machine learning (Ullmann, 2019).
• Automatic detection of reflection depth
Reflective statement classification using both rule-based and machine learning approaches
(Ullmann, 2015; Ullmann, 2011; Kovanović et al., 2018).
• Generation of formative feedback
Instant 24/7 feedback to students on their drafts (Gibson et al., 2017; Lucas et al. 2018)
10. Operationalising what we mean by
“good reflective writing” (1)
Conceptual model from the literature
11. Rhetorical moves in reflective writing
Gibson, A., Aitken, A., Sándor, Á., Buckingham Shum, S., Tsingos-Lucas, C. and Knight, S. (2017). Reflective Writing Analytics for Actionable Feedback. Proc.
LAK17: 7th Int. Conf. on Learning Analytics & Knowledge, March 13-17, 2017, Vancouver. (ACM Press). https://doi.org/10.1145/3027385.3027436 Best Paper Award
12. Rhetorical moves in reflective writing
Gibson, A., Aitken, A., Sándor, Á., Buckingham Shum, S., Tsingos-Lucas, C. and Knight, S. (2017). Reflective Writing Analytics for Actionable Feedback. Proc.
LAK17: 7th Int. Conf. on Learning Analytics & Knowledge, March 13-17, 2017, Vancouver. (ACM Press). https://doi.org/10.1145/3027385.3027436 Best Paper Award
13. Operationalising what we mean by
“good reflective writing” (2)
Mapping progression in depth of
reflective writing
14. Consider these 3 students...
Constructs Depth of Reflection
Capability for Written Reflection
(CWRef): The ability to evidence,
in writing, critical reflection on a
challenging experience
Non-Reflector
Habitual action: Engages in routine
activity with little conscious thought
(context)
Reflector
Reflective action: Reflects on what
and how s/he perceives (context),
thinks (challenges), feels (feelings),
or acts (changes)
Critical Reflector
Premise reflection: Reflects on
why s/he perceives (context), thinks
(challenges), feels (feelings), or acts
(changes) in particular ways
…is a function of…
Context: The observed learning
experience
Describes a learning event Highlights a learning event, linking it
to prior knowledge, beliefs or
assumptions
Highlights a learning event, linking it
to prior knowledge, beliefs or
assumptions, and explains the
reason for this association
Feelings: Feelings present during
the initial experience
Shows little or no evidence of
personal feelings, thoughts,
reactions
Evidences some feelings about an
experience, but does not explain
why I feel this way
Evidences personal feelings
(positive and/or negative) about an
experience, and explains the cause
for such feelings, and connects them
to challenges
Challenges: The
difficulties/problems encountered
during the experience
Shares no evidence of any problems
encountered
Evidences one or more problems
and explains why and how they
were challenging
Evidences the impact of one or more
problems on goals, and shares ideas
on how to address this
Changes: Lessons learned and
future plans
Shares no evidence of potential
solutions or learning opportunities
Evidences potential solutions or
learning opportunities
Evidences learning opportunities
from own and other perspectives,
and/or considers how change is
likely to lead to future benefits
1
15. Consider these 3 students...
Constructs Depth of Reflection
Capability for Written Reflection
(CWRef): The ability to evidence,
in writing, critical reflection on a
challenging experience
Non-Reflector
Habitual action: Engages in routine
activity with little conscious thought
(context)
Reflector
Reflective action: Reflects on what
and how s/he perceives (context),
thinks (challenges), feels (feelings),
or acts (changes)
Critical Reflector
Premise reflection: Reflects on
why s/he perceives (context), thinks
(challenges), feels (feelings), or acts
(changes) in particular ways
…is a function of…
Context: The observed learning
experience
Describes a learning event Highlights a learning event, linking it
to prior knowledge, beliefs or
assumptions
Highlights a learning event, linking it
to prior knowledge, beliefs or
assumptions, and explains the
reason for this association
Feelings: Feelings present during
the initial experience
Shows little or no evidence of
personal feelings, thoughts,
reactions
Evidences some feelings about an
experience, but does not explain
why I feel this way
Evidences personal feelings
(positive and/or negative) about an
experience, and explains the cause
for such feelings, and connects them
to challenges
Challenges: The
difficulties/problems encountered
during the experience
Shares no evidence of any problems
encountered
Evidences one or more problems
and explains why and how they
were challenging
Evidences the impact of one or more
problems on goals, and shares ideas
on how to address this
Changes: Lessons learned and
future plans
Shares no evidence of potential
solutions or learning opportunities
Evidences potential solutions or
learning opportunities
Evidences learning opportunities
from own and other perspectives,
and/or considers how change is
likely to lead to future benefits
1 2
16. Consider these 3 students...
Constructs Depth of Reflection
Capability for Written Reflection
(CWRef): The ability to evidence,
in writing, critical reflection on a
challenging experience
Non-Reflector
Habitual action: Engages in routine
activity with little conscious thought
(context)
Reflector
Reflective action: Reflects on what
and how s/he perceives (context),
thinks (challenges), feels (feelings),
or acts (changes)
Critical Reflector
Premise reflection: Reflects on
why s/he perceives (context), thinks
(challenges), feels (feelings), or acts
(changes) in particular ways
…is a function of…
Context: The observed learning
experience
Describes a learning event Highlights a learning event, linking it
to prior knowledge, beliefs or
assumptions
Highlights a learning event, linking it
to prior knowledge, beliefs or
assumptions, and explains the
reason for this association
Feelings: Feelings present during
the initial experience
Shows little or no evidence of
personal feelings, thoughts,
reactions
Evidences some feelings about an
experience, but does not explain
why I feel this way
Evidences personal feelings
(positive and/or negative) about an
experience, and explains the cause
for such feelings, and connects them
to challenges
Challenges: The
difficulties/problems encountered
during the experience
Shares no evidence of any problems
encountered
Evidences one or more problems
and explains why and how they
were challenging
Evidences the impact of one or more
problems on goals, and shares ideas
on how to address this
Changes: Lessons learned and
future plans
Shares no evidence of potential
solutions or learning opportunities
Evidences potential solutions or
learning opportunities
Evidences learning opportunities
from own and other perspectives,
and/or considers how change is
likely to lead to future benefits
1 2 3
17. Consider these 3 students...
Constructs Depth of Reflection
Capability for Written Reflection
(CWRef): The ability to evidence,
in writing, critical reflection on a
challenging experience
Non-Reflector
Habitual action: Engages in routine
activity with little conscious thought
(context)
Reflector
Reflective action: Reflects on what
and how s/he perceives (context),
thinks (challenges), feels (feelings),
or acts (changes)
Critical Reflector
Premise reflection: Reflects on
why s/he perceives (context), thinks
(challenges), feels (feelings), or acts
(changes) in particular ways
…is a function of…
Context: The observed learning
experience
Describes a learning event Highlights a learning event, linking it
to prior knowledge, beliefs or
assumptions
Highlights a learning event, linking it
to prior knowledge, beliefs or
assumptions, and explains the
reason for this association
Feelings: Feelings present during
the initial experience
Shows little or no evidence of
personal feelings, thoughts,
reactions
Evidences some feelings about an
experience, but does not explain
why I feel this way
Evidences personal feelings
(positive and/or negative) about an
experience, and explains the cause
for such feelings, and connects them
to challenges
Challenges: The
difficulties/problems encountered
during the experience
Shares no evidence of any problems
encountered
Evidences one or more problems
and explains why and how they
were challenging
Evidences the impact of one or more
problems on goals, and shares ideas
on how to address this
Changes: Lessons learned and
future plans
Shares no evidence of potential
solutions or learning opportunities
Evidences potential solutions or
learning opportunities
Evidences learning opportunities
from own and other perspectives,
and/or considers how change is
likely to lead to future benefits
1 2 3
18. Consider these 3 students...
Constructs Depth of Reflection
Capability for Written Reflection
(CWRef): The ability to evidence,
in writing, critical reflection on a
challenging experience
Non-Reflector
Habitual action: Engages in routine
activity with little conscious thought
(context)
Reflector
Reflective action: Reflects on what
and how s/he perceives (context),
thinks (challenges), feels (feelings),
or acts (changes)
Critical Reflector
Premise reflection: Reflects on
why s/he perceives (context), thinks
(challenges), feels (feelings), or acts
(changes) in particular ways
…is a function of…
Context: The observed learning
experience
Describes a learning event Highlights a learning event, linking it
to prior knowledge, beliefs or
assumptions
Highlights a learning event, linking it
to prior knowledge, beliefs or
assumptions, and explains the
reason for this association
Feelings: Feelings present during
the initial experience
Shows little or no evidence of
personal feelings, thoughts,
reactions
Evidences some feelings about an
experience, but does not explain
why I feel this way
Evidences personal feelings
(positive and/or negative) about an
experience, and explains the cause
for such feelings, and connects them
to challenges
Challenges: The
difficulties/problems encountered
during the experience
Shares no evidence of any problems
encountered
Evidences one or more problems
and explains why and how they
were challenging
Evidences the impact of one or more
problems on goals, and shares ideas
on how to address this
Changes: Lessons learned and
future plans
Shares no evidence of potential
solutions or learning opportunities
Evidences potential solutions or
learning opportunities
Evidences learning opportunities
from own and other perspectives,
and/or considers how change is
likely to lead to future benefits
1 2 3
19. Consider these 3 students...
Constructs Depth of Reflection
Capability for Written Reflection
(CWRef): The ability to evidence,
in writing, critical reflection on a
challenging experience
Non-Reflector
Habitual action: Engages in routine
activity with little conscious thought
(context)
Reflector
Reflective action: Reflects on what
and how s/he perceives (context),
thinks (challenges), feels (feelings),
or acts (changes)
Critical Reflector
Premise reflection: Reflects on
why s/he perceives (context), thinks
(challenges), feels (feelings), or acts
(changes) in particular ways
…is a function of…
Context: The observed learning
experience
Describes a learning event Highlights a learning event, linking it
to prior knowledge, beliefs or
assumptions
Highlights a learning event, linking it
to prior knowledge, beliefs or
assumptions, and explains the
reason for this association
Feelings: Feelings present during
the initial experience
Shows little or no evidence of
personal feelings, thoughts,
reactions
Evidences some feelings about an
experience, but does not explain
why I feel this way
Evidences personal feelings
(positive and/or negative) about an
experience, and explains the cause
for such feelings, and connects them
to challenges
Challenges: The
difficulties/problems encountered
during the experience
Shares no evidence of any problems
encountered
Evidences one or more problems
and explains why and how they
were challenging
Evidences the impact of one or more
problems on goals, and shares ideas
on how to address this
Changes: Lessons learned and
future plans
Shares no evidence of potential
solutions or learning opportunities
Evidences potential solutions or
learning opportunities
Evidences learning opportunities
from own and other perspectives,
and/or considers how change is
likely to lead to future benefits
1 2 3
20. Consider these 3 students...
Constructs Depth of Reflection
Capability for Written Reflection
(CWRef): The ability to evidence,
in writing, critical reflection on a
challenging experience
Non-Reflector
Habitual action: Engages in routine
activity with little conscious thought
(context)
Reflector
Reflective action: Reflects on what
and how s/he perceives (context),
thinks (challenges), feels (feelings),
or acts (changes)
Critical Reflector
Premise reflection: Reflects on
why s/he perceives (context), thinks
(challenges), feels (feelings), or acts
(changes) in particular ways
…is a function of…
Context: The observed learning
experience
Describes a learning event Highlights a learning event, linking it
to prior knowledge, beliefs or
assumptions
Highlights a learning event, linking it
to prior knowledge, beliefs or
assumptions, and explains the
reason for this association
Feelings: Feelings present during
the initial experience
Shows little or no evidence of
personal feelings, thoughts,
reactions
Evidences some feelings about an
experience, but does not explain
why I feel this way
Evidences personal feelings
(positive and/or negative) about an
experience, and explains the cause
for such feelings, and connects them
to challenges
Challenges: The
difficulties/problems encountered
during the experience
Shares no evidence of any problems
encountered
Evidences one or more problems
and explains why and how they
were challenging
Evidences the impact of one or more
problems on goals, and shares ideas
on how to address this
Changes: Lessons learned and
future plans
Shares no evidence of potential
solutions or learning opportunities
Evidences potential solutions or
learning opportunities
Evidences learning opportunities
from own and other perspectives,
and/or considers how change is
likely to lead to future benefits
1 2 3
21. Consider these 3 students...
Constructs Depth of Reflection
Capability for Written Reflection
(CWRef): The ability to evidence,
in writing, critical reflection on a
challenging experience
Non-Reflector
Habitual action: Engages in routine
activity with little conscious thought
(context)
Reflector
Reflective action: Reflects on what
and how s/he perceives (context),
thinks (challenges), feels (feelings),
or acts (changes)
Critical Reflector
Premise reflection: Reflects on
why s/he perceives (context), thinks
(challenges), feels (feelings), or acts
(changes) in particular ways
…is a function of…
Context: The observed learning
experience
Describes a learning event Highlights a learning event, linking it
to prior knowledge, beliefs or
assumptions
Highlights a learning event, linking it
to prior knowledge, beliefs or
assumptions, and explains the
reason for this association
Feelings: Feelings present during
the initial experience
Shows little or no evidence of
personal feelings, thoughts,
reactions
Evidences some feelings about an
experience, but does not explain
why I feel this way
Evidences personal feelings
(positive and/or negative) about an
experience, and explains the cause
for such feelings, and connects them
to challenges
Challenges: The
difficulties/problems encountered
during the experience
Shares no evidence of any problems
encountered
Evidences one or more problems
and explains why and how they
were challenging
Evidences the impact of one or more
problems on goals, and shares ideas
on how to address this
Changes: Lessons learned and
future plans
Shares no evidence of potential
solutions or learning opportunities
Evidences potential solutions or
learning opportunities
Evidences learning opportunities
from own and other perspectives,
and/or considers how change is
likely to lead to future benefits
1 2 3
22. Consider these 3 students...
Constructs Depth of Reflection
Capability for Written Reflection
(CWRef): The ability to evidence,
in writing, critical reflection on a
challenging experience
Non-Reflector
Habitual action: Engages in routine
activity with little conscious thought
(context)
Reflector
Reflective action: Reflects on what
and how s/he perceives (context),
thinks (challenges), feels (feelings),
or acts (changes)
Critical Reflector
Premise reflection: Reflects on
why s/he perceives (context), thinks
(challenges), feels (feelings), or acts
(changes) in particular ways
…is a function of…
Context: The observed learning
experience
Describes a learning event Highlights a learning event, linking it
to prior knowledge, beliefs or
assumptions
Highlights a learning event, linking it
to prior knowledge, beliefs or
assumptions, and explains the
reason for this association
Feelings: Feelings present during
the initial experience
Shows little or no evidence of
personal feelings, thoughts,
reactions
Evidences some feelings about an
experience, but does not explain
why I feel this way
Evidences personal feelings
(positive and/or negative) about an
experience, and explains the cause
for such feelings, and connects them
to challenges
Challenges: The
difficulties/problems encountered
during the experience
Shares no evidence of any problems
encountered
Evidences one or more problems
and explains why and how they
were challenging
Evidences the impact of one or more
problems on goals, and shares ideas
on how to address this
Changes: Lessons learned and
future plans
Shares no evidence of potential
solutions or learning opportunities
Evidences potential solutions or
learning opportunities
Evidences learning opportunities
from own and other perspectives,
and/or considers how change is
likely to lead to future benefits
1 2 3
23. Consider these 3 students...
Constructs Depth of Reflection
Capability for Written Reflection
(CWRef): The ability to evidence,
in writing, critical reflection on a
challenging experience
Non-Reflector
Habitual action: Engages in routine
activity with little conscious thought
(context)
Reflector
Reflective action: Reflects on what
and how s/he perceives (context),
thinks (challenges), feels (feelings),
or acts (changes)
Critical Reflector
Premise reflection: Reflects on
why s/he perceives (context), thinks
(challenges), feels (feelings), or acts
(changes) in particular ways
…is a function of…
Context: The observed learning
experience
Describes a learning event Highlights a learning event, linking it
to prior knowledge, beliefs or
assumptions
Highlights a learning event, linking it
to prior knowledge, beliefs or
assumptions, and explains the
reason for this association
Feelings: Feelings present during
the initial experience
Shows little or no evidence of
personal feelings, thoughts,
reactions
Evidences some feelings about an
experience, but does not explain
why I feel this way
Evidences personal feelings
(positive and/or negative) about an
experience, and explains the cause
for such feelings, and connects them
to challenges
Challenges: The
difficulties/problems encountered
during the experience
Shares no evidence of any problems
encountered
Evidences one or more problems
and explains why and how they
were challenging
Evidences the impact of one or more
problems on goals, and shares ideas
on how to address this
Changes: Lessons learned and
future plans
Shares no evidence of potential
solutions or learning opportunities
Evidences potential solutions or
learning opportunities
Evidences learning opportunities
from own and other perspectives,
and/or considers how change is
likely to lead to future benefits
1 2 3
24. Consider these 3 students...
Constructs Depth of Reflection
Capability for Written Reflection
(CWRef): The ability to evidence,
in writing, critical reflection on a
challenging experience
Non-Reflector
Habitual action: Engages in routine
activity with little conscious thought
(context)
Reflector
Reflective action: Reflects on what
and how s/he perceives (context),
thinks (challenges), feels (feelings),
or acts (changes)
Critical Reflector
Premise reflection: Reflects on
why s/he perceives (context), thinks
(challenges), feels (feelings), or acts
(changes) in particular ways
…is a function of…
Context: The observed learning
experience
Describes a learning event Highlights a learning event, linking it
to prior knowledge, beliefs or
assumptions
Highlights a learning event, linking it
to prior knowledge, beliefs or
assumptions, and explains the
reason for this association
Feelings: Feelings present during
the initial experience
Shows little or no evidence of
personal feelings, thoughts,
reactions
Evidences some feelings about an
experience, but does not explain
why I feel this way
Evidences personal feelings
(positive and/or negative) about an
experience, and explains the cause
for such feelings, and connects them
to challenges
Challenges: The
difficulties/problems encountered
during the experience
Shares no evidence of any problems
encountered
Evidences one or more problems
and explains why and how they
were challenging
Evidences the impact of one or more
problems on goals, and shares ideas
on how to address this
Changes: Lessons learned and
future plans
Shares no evidence of potential
solutions or learning opportunities
Evidences potential solutions or
learning opportunities
Evidences learning opportunities
from own and other perspectives,
and/or considers how change is
likely to lead to future benefits
1 2 3
25. Consider these 3 students...
Constructs Depth of Reflection
Capability for Written Reflection
(CWRef): The ability to evidence,
in writing, critical reflection on a
challenging experience
Non-Reflector
Habitual action: Engages in routine
activity with little conscious thought
(context)
Reflector
Reflective action: Reflects on what
and how s/he perceives (context),
thinks (challenges), feels (feelings),
or acts (changes)
Critical Reflector
Premise reflection: Reflects on
why s/he perceives (context), thinks
(challenges), feels (feelings), or acts
(changes) in particular ways
…is a function of…
Context: The observed learning
experience
Describes a learning event Highlights a learning event, linking it
to prior knowledge, beliefs or
assumptions
Highlights a learning event, linking it
to prior knowledge, beliefs or
assumptions, and explains the
reason for this association
Feelings: Feelings present during
the initial experience
Shows little or no evidence of
personal feelings, thoughts,
reactions
Evidences some feelings about an
experience, but does not explain
why I feel this way
Evidences personal feelings
(positive and/or negative) about an
experience, and explains the cause
for such feelings, and connects them
to challenges
Challenges: The
difficulties/problems encountered
during the experience
Shares no evidence of any problems
encountered
Evidences one or more problems
and explains why and how they
were challenging
Evidences the impact of one or more
problems on goals, and shares ideas
on how to address this
Changes: Lessons learned and
future plans
Shares no evidence of potential
solutions or learning opportunities
Evidences potential solutions or
learning opportunities
Evidences learning opportunities
from own and other perspectives,
and/or considers how change is
likely to lead to future benefits
1 2 3
26. Assessment rubric (Acknowledgements to Cherie Lucas, UTS Pharmacy)
Constructs Depth of Reflection
Capability for Written Reflection
(CWRef): The ability to evidence,
in writing, critical reflection on a
challenging experience
Non-Reflector
Habitual action: Engages in routine
activity with little conscious thought
(context)
Reflector
Reflective action: Reflects on what
and how s/he perceives (context),
thinks (challenges), feels (feelings),
or acts (changes)
Critical Reflector
Premise reflection: Reflects on
why s/he perceives (context), thinks
(challenges), feels (feelings), or acts
(changes) in particular ways
…is a function of…
Context: The observed learning
experience
Describes a learning event Highlights a learning event, linking it
to prior knowledge, beliefs or
assumptions
Highlights a learning event, linking it
to prior knowledge, beliefs or
assumptions, and explains the
reason for this association
Feelings: Feelings present during
the initial experience
Shows little or no evidence of
personal feelings, thoughts,
reactions
Evidences some feelings about an
experience, but does not explain
why I feel this way
Evidences personal feelings
(positive and/or negative) about an
experience, and explains the cause
for such feelings, and connects them
to challenges
Challenges: The
difficulties/problems encountered
during the experience
Shares no evidence of any problems
encountered
Evidences one or more problems
and explains why and how they
were challenging
Evidences the impact of one or more
problems on goals, and shares ideas
on how to address this
Changes: Lessons learned and
future plans
Shares no evidence of potential
solutions or learning opportunities
Evidences potential solutions or
learning opportunities
Evidences learning opportunities
from own and other perspectives,
and/or considers how change is
likely to lead to future benefits
27. AcaWriter: web app released to all
UTS students and staff
24/7 instant formative feedback (not automated grading)
to coach academic rhetorical moves in multiple genres
of writing
28. UTS-wide release: from project to product...
https://www.uts.edu.au/current-students/news/get-instant-feedback-your-academic-writing
A significant step: authentication, security, helpdesk...
Monthly training
for academics
and tutors
29. Usage of AcaWriter reflective writing
Usage by staff 2018-2022 to
teach their courses
22
Usage by students
5543
Usage by students on a course
(code) or self-initiated (no code)
30. Visual language for rhetorical moves in reflective writing
Gibson, A., Aitken, A., Sándor, Á., Buckingham Shum, S., Tsingos-Lucas, C. and Knight, S. (2017). Reflective Writing Analytics for Actionable Feedback. Proc.
LAK17: 7th Int. Conf. on Learning Analytics & Knowledge, March 13-17, 2017, Vancouver. (ACM Press). https://doi.org/10.1145/3027385.3027436 Best Paper Award
32. Lucas, C., Buckingham Shum, S., Liu, M., & Bebawy, M. (2021).
Implementing AcaWriter as a Novel Strategy to Support Pharmacy
Students’ Reflective Practice in Scientific Research. American Journal
of Pharmaceutical Education, 8320. https://doi.org/10.5688/ajpe8320
Tabs 1 and 2 of the feedback
35. How rhetorical parsing works
3 examples of
“Shift in Perception”
with the same underlying
concept pattern
Buckingham Shum, S., Sándor, Á., Goldsmith, R., Bass, R., & McWilliams, M. (2017). Towards Reflective Writing Analytics:
Rationale, Methodology and Preliminary Results. Journal of Learning Analytics, 4(58-84). https://doi.org/10.18608/jla.2017.41.5
37. How rhetorical parsing works
Uses the Athanor open source rhetorical parser developed by Ágnes Sándor & Claude Roux (Naver Labs Europe)
I do not feel as though I have sufficiently developed my
practical knowledge, or phronesis (Schwandt 2007), to a
point where I can speak up and challenge my manager.
Reflective sentence
38. How rhetorical parsing works
Uses the Athanor open source rhetorical parser developed by Ágnes Sándor & Claude Roux (Naver Labs Europe)
39. How rhetorical parsing works
Uses the Athanor open source rhetorical parser developed by Ágnes Sándor & Claude Roux (Naver Labs Europe)
40. How rhetorical parsing works
5 Athanor training webinars:
http://heta.io/online-training-in-rhetorical-parsing
42. Co-designing the feedback messages with the academics
http://heta.io/how-can-writing-analytics-researchers-rapidly-codesign-feedback-with-educators
https://www.heta.io/co-designing-automated-feedback-on-reflective-writing-with-the-teacher
43. Google Doc to draft
the feedback
contingent on
patterns:
presence/absence
sequencing
positioning
of sentence types
http://heta.io/how-can-writing-analytics-researchers-rapidly-codesign-feedback-with-educators
https://www.heta.io/co-designing-automated-feedback-on-reflective-writing-with-the-teacher
44. From co-designed draft
to implementation
http://heta.io/how-can-writing-analytics-researchers-rapidly-codesign-feedback-with-educators
https://www.heta.io/co-designing-automated-feedback-on-reflective-writing-with-the-teacher
49. Lucas, C., Buckingham Shum, S., Liu, M., & Bebawy, M. (2021). Implementing AcaWriter as a Novel Strategy to Support Pharmacy Students’
Reflective Practice in Scientific Research. American Journal of Pharmaceutical Education, 8320. https://doi.org/10.5688/ajpe8320
50. Lucas, C., Buckingham Shum, S., Liu, M., & Bebawy, M. (2021). Implementing AcaWriter as a Novel Strategy to Support Pharmacy Students’
Reflective Practice in Scientific Research. American Journal of Pharmaceutical Education, 8320. https://doi.org/10.5688/ajpe8320
51. Lucas, C., Buckingham Shum, S., Liu, M., & Bebawy, M. (2021). Implementing AcaWriter as a Novel Strategy to Support Pharmacy Students’
Reflective Practice in Scientific Research. American Journal of Pharmaceutical Education, 8320. https://doi.org/10.5688/ajpe8320
52. Lucas, C., Buckingham Shum, S., Liu, M., & Bebawy, M. (2021). Implementing AcaWriter as a Novel Strategy to Support Pharmacy Students’
Reflective Practice in Scientific Research. American Journal of Pharmaceutical Education, 8320. https://doi.org/10.5688/ajpe8320
53. Lucas, C., Buckingham Shum, S., Liu, M., & Bebawy, M. (2021). Implementing AcaWriter as a Novel Strategy to Support Pharmacy Students’
Reflective Practice in Scientific Research. American Journal of Pharmaceutical Education, 8320. https://doi.org/10.5688/ajpe8320
55. UTS CRICOS 00099F
Evaluating Machine Learning Approaches to
Classify Pharmacy Students’ Reflective Statements
Ming Liu1, Simon Buckingham Shum1,
Efi Mantzourani2 and Cherie Lucas3
1 Connected Intelligence Centre, University of Technology Sydney, Australia
2 School of Pharmacy & Pharmaceutical Sciences, Cardiff University, UK
3 School of Pharmacy, University of Technology Sydney, Australia
@MingLiuResearch
@sbuckshum
@LucasReflection
@efi_mantz
AIED 2019: International Conference on Artificial Intelligence in Education
56. Summary
• A validated, graded corpus of reflective statements from authentic work-
placements (specifically, pharmacy students) was built
• A machine learning approach for reflection depth classification at
paragraph level with features extracted from AWA + LIWC shows
promising results (F-score =.799 using random-forest classifier).
• Textual features: the LIWC emotional and cognitive features, and AWA
rhetorical move features are important indicators of the depth of the
reflection
58. Constructs Depth of Reflection
Capability for Written Reflection
(CWRef): The ability to evidence,
in writing, critical reflection on a
challenging experience
Non-Reflector
Habitual action: Engages in routine
activity with little conscious thought
(context)
Reflector
Reflective action: Reflects on what
and how s/he perceives (context),
thinks (challenges), feels (feelings),
or acts (changes)
Critical Reflector
Premise reflection: Reflects on
why s/he perceives (context), thinks
(challenges), feels (feelings), or acts
(changes) in particular ways
…is a function of…
Context: The observed learning
experience
Describes a learning event Highlights a learning event, linking it
to prior knowledge, beliefs or
assumptions
Highlights a learning event, linking it
to prior knowledge, beliefs or
assumptions, and explains the
reason for this association
Feelings: Feelings present during
the initial experience
Shows little or no evidence of
personal feelings, thoughts,
reactions
Evidences some feelings about an
experience, but does not explain
why I feel this way
Evidences personal feelings
(positive and/or negative) about an
experience, and explains the cause
for such feelings, and connects them
to challenges
Challenges: The
difficulties/problems encountered
during the experience
Shares no evidence of any problems
encountered
Evidences one or more problems
and explains why and how they
were challenging
Evidences the impact of one or more
problems on goals, and shares ideas
on how to address this
Changes: Lessons learned and
future plans
Shares no evidence of potential
solutions or learning opportunities
Evidences potential solutions or
learning opportunities
Evidences learning opportunities
from own and other perspectives,
and/or considers how change is
likely to lead to future benefits
Liu, M., Kitto, K. & Buckingham Shum, S. (2021). Combining Factor Analysis with Writing Analytics for the Formative Assessment of Written Reflection. Computers in
Human Behavior. https://doi.org/10.1016/j.chb.2021.106733. Part of a Special Issue: Towards Strengthening Links Between Learning Analytics and Assessment:
Challenges and Potentials of a Promising New Bond, (Eds.) Dragan Gašević, Samuel Greiff, David Williamson Shaffer.
59. Now formalised as a model of Capability for written reflection
Liu, M., Kitto, K. & Buckingham Shum, S. (2021). Combining Factor Analysis
with Writing Analytics for the Formative Assessment of Written Reflection.
Computers in Human Behavior. https://doi.org/10.1016/j.chb.2021.106733
60. Liu, M., Kitto, K. & Buckingham Shum, S. (2021). Combining Factor Analysis
with Writing Analytics for the Formative Assessment of Written Reflection.
Computers in Human Behavior. https://doi.org/10.1016/j.chb.2021.106733
Now formalised as a model of Capability for written reflection
61. Standardized factor loadings for the model (Pharmacy)
Liu, M., Kitto, K. & Buckingham Shum, S. (2021). Combining Factor Analysis
with Writing Analytics for the Formative Assessment of Written Reflection.
Computers in Human Behavior. https://doi.org/10.1016/j.chb.2021.106733
62. Standardized factor loadings for the model (Data Science)
Liu, M., Kitto, K. & Buckingham Shum, S. (2021). Combining Factor Analysis
with Writing Analytics for the Formative Assessment of Written Reflection.
Computers in Human Behavior. https://doi.org/10.1016/j.chb.2021.106733
63. Enabling more nuanced, helpful
automated formative feedback?
Liu, M., Kitto, K. & Buckingham Shum, S. (2021). Combining Factor Analysis with Writing Analytics for the Formative
Assessment of Written Reflection. Computers in Human Behavior. https://doi.org/10.1016/j.chb.2021.106733.
66. Progression Feedback:
<Revision 10> Well done, it appears that you have made
some progress by analyzing your experience in more
detail, and the challenges you encountered...
CWRef-based progression across 14 drafts of a reflective report (from a real student)
67. Self-regulation Feedback:
<Revision 12> Well done! Looking back, you can see
how your drafts have varied but overall, have really
improved. Does it make sense to you why the scores for
context, challenges and changes went up and down?
CWRef-based progression across 14 drafts of a reflective report (from a real student)
68. Imperfect analytics/AI for formative
feedback as a feature (not a bug)
Kirsty Kitto, Simon Buckingham Shum, and Andrew Gibson. (2018). Embracing Imperfection in Learning Analytics. In Proceedings LAK18: International Conference on
Learning Analytics and Knowledge, March 5–9, 2018, Sydney, NSW, Australia, pp.451-460. (ACM, New York, NY, USA). https://doi.org/10.1145/3170358.3170413
70. Partners in cognition: extending human intelligence with
intelligent technologies (1991)
Learners’ engagement with intelligent technology
should not be mindless.
On the contrary we should design for…
“nonautomatic, effortful and thus
metacognitively guided processes”
Gavriel Salomon, David Perkins and Tamar Globerson (1991). Partners in cognition: extending human intelligence with intelligent technologies. Educational Researcher, 20, 3 (1991), 2–9.
71. Imperfect Learning Analytics à cognitive dissonance
Kirsty Kitto, Simon Buckingham Shum, and Andrew Gibson. (2018). Embracing Imperfection in Learning Analytics. In Proceedings LAK18: International Conference on
Learning Analytics and Knowledge, March 5–9, 2018, Sydney, NSW, Australia, pp.451-460. (ACM, New York, NY, USA). https://doi.org/10.1145/3170358.3170413
“…as D’Mello and Graesser [15] demonstrate, it is when the
student experiences dissonance because the
analytics fail to match their expectations that
they are likely to reflect on why they think the
machine is wrong. We believe that this form of critical
questioning is more likely to happen if the student has been given an
underlying reason to be a little distrustful of the classifier.”
72. Summary
An example of designing “digital mirrors” to provoke
deeper human reflection
AI imperfection is not a showstopper, if the learning
design is robust, and students are empowered to
think critically
Positive responses from students and academics
Learner and educator agency are important
73. (Overview and insights from 5 years research) Knight, S., Shibani,
A., Abel, S., Gibson, A., Ryan, P., Sutton, N., Wight, R., Lucas, C.,
Sándor, Á., Kitto, K., Liu, M., Mogarkar, R. & Buckingham Shum, S.
(2020). AcaWriter: A learning analytics tool for formative feedback
on academic writing. Journal of Writing Research, 12, (1), 141-186.
Academic Writing Analytics project
https://cic.uts.edu.au/tools/awa
https://open.uts.edu.au/uts-open/faculty/connected-intelligence-centre/writing-an-abstract/
Access full version of AcaWriter via this 1 hour tutorial…