Annotation-Centric Assessment of Blogging in Higher Education
•Télécharger en tant que PPTX, PDF•
2 j'aime•3,947 vues
Open Education Conference 2015 Presentation on preliminary results from a study that aimed to explore the appropriateness of assessing student performance through student annotation use in digital learning environments
Recommandé
Recommandé
Contenu connexe
Similaire à Annotation-Centric Assessment of Blogging in Higher Education
Similaire à Annotation-Centric Assessment of Blogging in Higher Education (20)
Plus de Laura Gogia
Plus de Laura Gogia (13)
Dernier
Dernier (20)
Annotation-Centric Assessment of Blogging in Higher Education
- 1. Exploring the What, Why, & How of Social Learning Analytics OR Annotation-Centric Assessment of Blogging in Higher Education Laura Gogia - @Googleguacamole Academic Learning Transformation Lab Virginia Commonwealth University, Richmond VA
- 2. Slides are available on SlideShare.net http://www.slideshare.net/LauraGogi a/
- 3. Hello. My Name Is... Do you teach online? Blog and tweet with student- participants? In public? To what end? How do you know if you are achieving your goals?
- 4. VCU is exploring the intersection of connected learning & open education Photo Credit: http://graduate.admissions.vcu.edu/why/
- 6. Emerging-Evolving Course Experience: • Public – Open Course Website • Public and Aggregated Student Blogging • Public Discourse
- 7. By introducing students to this sort of loosely structured, flexible, open digital learning experience, we hope to promote certain learning dispositions, including connectivity.
- 8. Connectivity Creating, recognizing, understanding, and acting on connections made across content, disciplines, living spheres, people, space and time. Interdisciplinary Learning – Self-Reflection – Transferability – Social Learning – Intersection of formal and informal – Holistic
- 9. That’s great. But how do we assess connectivity?
- 10. We are thinking hard about how to build meaningful assessments in our course designs. (1) Documenting connectivity. (2) Advancing the learning. (3) Meeting 21st century goals for assessment.
- 11. Social Learning Analytics: A subset of learning analytics meant to capture, organize, and demonstrate the inherently social, open, and connective aspects of networked participatory learning. --Ferguson & Buckingham Shum (2011)
- 12. Are there ways to take advantage of the uniquely digital aspects of blogging and tweeting to assess student blogging and tweeting in these environments?
- 13. Annotations Symbols & phrases that are distinct from but included within the communication, meant to demonstrate communicative intent Hyperlinks – Embedded Images – Mentions – Hashtags
- 14. Initial argument for annotation-centric assessment: • While specific to digital, they point to the art of crafting communication. • Annotation is an act that lends itself to strategic reflection (“Why did you annotate that?”) • Documentable, extractable, quantifiable.
- 15. Strands of Inquiry 1. How are student-participants using annotations in course-related blogging and tweeting? 1. How can this information be used (organized and visualized) to inform meaningful assessment?
- 16. COURSES PARTICIPANTS DATA SOURCES DATA TYPES 4 ONLINE COURSES Undergraduate & Graduate Multidisciplinary & Gen Ed 6-26 Students 280 PARTICIPANTS Instructors & Assistants (n = 10) Students (n = 60) Open Participants (n = 12) Other Participants (n = 200) 1618 Posts (500 Sampled) 5000 Tweets POSTS Hyperlinks (n = 800) Embedded Images (n = 400) TWEETS Hyperlinks (n = 430) Mentions (n = 3000) Hashtags (n = 130) ANALYSES Descriptive Statistics Content Analysis Social Network Analysis
- 17. Two Things Happened. 1. Classification systems for describing how students annotate. 1. Dashboards for documenting and assessing student performance.
- 18. An Example CC was a graduate level elective which aimed to introduce its students to community engaged research. On average, students blogged several times a week in three formats: (1) digital makes; (2) reflective posts; (3) research proposal project. They also engaged in weekly, structured Twitter chats. Enrollment: 10 students. 12 open participants.
- 19. Classification Systems • Types and sources of hyperlinked and embedded materials • Purpose or impact of the annotation in the context of the post
- 20. But why did CC students hyperlink? Types and sources of materials that CC students used were appropriate for student level and the course content.
- 21. 1. To link to supporting documents. • Traditional citing and referencing. • Defining, describing, and providing examples in ways not supported by formal writing styles. • “This could be filled in, Mad Libs style.” • “...verbal or nonverbal communication...”
- 22. 2. To link to previous work or experience. • “As I discussed in a previous post...” • “As I reflect on my proposed research questions...”
- 23. 3. To provide course context. • “I am writing this because I am taking a course...”
- 24. But what about embedded images and videos?
- 25. The pedagogical value of embedded images and videos varied, but in a way that can be defined and organized into a spectrum.
- 26. BASIC LEVEL (IMAGES): • Serves no obvious purpose other than contributing to an aesthetic.
- 27. INTERMEDIATE LEVEL (IMAGES): • Provides additional information • Makes an otherwise unstated them explicit. • Inspires deeper questions.
- 28. ADVANCED LEVEL (IMAGES & VIDEOS): • Further the narrative (e.g. a table, chart, or infographic that the student refers to or explains in the narrative) • Demonstrate a personal connection to the subject (e.g. a photograph, graphic, or video the student made themselves and explains in the narrative)
- 29. How does can typologies like these inform student assessment? Rubrics – Peer Assessment – Self Assessment
Notes de l'éditeur
- Under the direction of Gardner Campbell, the Vice Provost of Learning Innovation and Student Success, my home university, VCU is trying to take learning through public, digital scholarship to a very large scale.
- Our goal is to build a virtual city-campus, not just for distance learners, but for everyone. Students, faculty, and staff have their personal blog spaces that can be networked at a local level for formal and informal learning. Because these are public spaces, they integrate smoothly with the global network.
- And a specific kind of course experience seems to be emerging from this experiment – Course materials, activities, and communications are found on a public course website Students complete at least some of their assignments through the act of blogging on their personal websites – these are aggregated by an RSS feed for the course website Certain level of public discourse which counts towards a participation grade, be that commenting on their colleagues’ blogs, Twitter, Discussion Forums,
- Our reasons for promoting these sorts of experiences are multifold, but we are hoping to promote certain learning dispositions, one of which is connectivity.
- One of the main learning dispositions is the concept of connectivity – the ability to recognize, understand, and act strategically on connections made across content, people, space, and time.
- Meaningful assessments are those that Document what you are hoping to achieve Advance the learning of the students by providing actionable feedback Meet goals of 21st century assessment – scalability, feasibility, peer/self assessment
- Social learning analytics may offer a solution. Ferguson and Buckingham Shum (2011) propose a shift in emphasis from the highly technical, individualized, and algorithmically motivated work of learning analytics to what they define as social learning analytics, a subset of learning analytics meant to capture, organize, and demonstrate the inherently social, open, and connective aspects of networked participatory learning. The authors identify five research methodologies that show promise in advancing digital age learning: content, disposition, context, network, and discourse analytics.
- My study focuses on the annotations of blogging and tweeting. These are the symbols and phrases that are distinct from but included within the communication, meant to demonstrate communicative intent. Specifically, I focused on hyperlinks, embedded images, mentions, and hashtags
- My argument for
- Three types of information are consistently available from clicking on hyperlinks in student works: the types and sources of the materials, and what purpose it serves. To save time I won’t review the types and sources of materials that CC students used – however they were appropriate for student level and the course content.
- Students hyperlinked to supporting documents. Oftentimes this took the form of traditional citing or referencing, or in the undergraduate courses, it was an “Additional Resources” list at the end of the post. However, sometimes, CC students used hyperlinks to define, describe, or provide examples of something in the narrative in ways that wouldn’t be possible on paper. In the two examples listed, students hyperlinked to provide explanations of what they mean by Mad Lib or nonverbal, rather than explaining it in the primary narrative. If they had stopped to explain it, it would have cluttered up the primary narrative.
- Students also used hyperlinks to link to previous work or experience. They used them to piece together their personal narrative.
- Students also hyperlinked to the course website, to provide context for what they were writing. This is interesting to me because it speaks to an awareness of audience beyond people who are in the class.
- But you can also make a dashboard, like this. 1. Students must complete the work, as demonstrated by the number of posts completed. 2. The number hyperlink-embeds per post provides information about quantity. Content type (image- vs. text-based materials) addresses the most basic characteristic of the connections being made. The presence of broken or misdirected links and aesthetic images offer measures of quality control for hyperlinks and embedded images, respectively. While the CC cohort included a high number of exemplary students, there was still some stratification in performance. At the time of sampling, students should have completed eight or nine posts; the variation occurred because the sampling frame did not precisely match the assignment completion schedule. However, students with only seven posts were falling behind. Students are not necessarily ordered with each category. Exemplary student work included hyperlinked or embedded materials frequently, averaging seven to nine hyperlinks per post. In general, these students used fewer aesthetic images and had few, if any, broken links. Average student work included just as many hyperlinks and embedded images as exemplary students, but did not have the same level of quality. Two of the three students in the final grouping had fallen behind on completing assignments. Furthermore, they had neither the frequency nor quality of hyperlinking or embedding, suggesting that intervention or, at least, further review, is required.
- This figure shows a three-post sample work from three exemplars: CC-S4 (exemplary), CC-S1 (average), and CC-S10 (needs improvement). CC-S4 tends to incorporate hyperlinks that have a variety of purposes in each post. For example, exemplary work might be a post that uses hyperlinks to support a brief description of course or personal context, followed by hyperlinks or embedded images to support their argument (description, citations, and illustration), followed by a hyperlinked list of references (references). An average post may have less links, some of which may be broken, concentrated in a list of references. Posts that need significant improvements may only include several hyperlinked references and embedded clip art that punctuate section headings but do little to further the narrative