Presentation for Queensland VDC
Topics covered include:
What is e-Assessment?
Issues and concerns around using eAssesments
Mobile evidence capture and submission
Principles for developing self paced assessment tasks
(Quizzes, Scenarios, Decision making trees)
Webinar based assessment and peer review
Using discussion forums to uncover student ?stories? and understanding
Tools and approaches for Student placement reporting
Group work projects ? measuring contributions
Other interesting approaches from the field.
Benefits and learning outcomes include:
* Understanding and application of various types e-assessments
* Application of relevant e-Assessment approaches to specific training context
3. Today’s agenda
•
•
•
•
•
•
•
What is e-Assessment?
Issues and concerns around using eAssesments
Mobile evidence capture and submission
Principles for developing self paced assessment tasks
(Quizzes, Scenarios, Decision making trees)
Webinar based assessment and peer review
Using discussion forums to uncover student “stories” and
understanding
• Tools and approaches for Student placement reporting
• Group work projects – measuring contributions
• Other interesting approaches from the field.
8. What is e-Assessment?
E-assessment is the use of information technology
for any assessment-related activity.
Assessment
Distribution
& Collection
Assessment
Response
Assessment
Management
Assessment
Verification
&
Moderation
17. Issues and Concerns
•
•
•
•
•
•
•
Identity – Personal data and questions
Cheating – Turnitin & random tutor contact
Skills – Online for Knowledge, F2F for skills
Variety of assessment Information retrieval – Problems, DMT
Just online – Blend with webinars & F2F
Content – Mix of video, case study, procedures,
problems.
• Isolation – Social network, forum, FAQ, webinars
• Teachers – Webinar, contact, forum support
• Storage –intuitive navigation, location, privacy law,
integration with SMS systems and services
31. Use the
chat
window!
Activity 4
Think of a difficult or dull process you need your
learners to understand.
Describe a scenario that might work for this
content
41. Use the
chat
window!
Activity 7
How do you use Group work assessments?
What information do you need to collect to
gauge an individuals contribution to the group’s
output?
Assessment activities include the design of assessment tools, the delivery of assessments and the reporting, storing and transferring of assessment data.
It is easier to think of Elearning as a suite of Blended delivery choices available to RTOS:
These modes can be any mix of face to face and either or a combination of online self paced, facilitated or group work sessions. Students participate in both traditional face to face classes and virtual classes such as online tutorials or discussion forum activities.
Face to Face – Trainers and learners participate in online activities or resources during face to face sessions.
Flexible, Self Paced Trainer or computer assessed – Learner works through prepared learning content that is assessed as they progress. Access to trainer generally for support or higher level assessment task feedback.
Facilitated online – learning is conducted entirely online through course material provision provided through a Learner Management System and online activities provided though either virtual classroom tutorials or discussion forums. Not such an attractive model for RTOs as it does not attract Government funding incentives for training placements.
Many RTOs commence using Elearning via the provision to learners of a Resource repository – learners access content online and refer to resources that are used in a face to face class. Not really considered elearning.
Informal networks based on knowledge sharing (also known as social learning) – “The revolution that is social media means that now everyone can have access to the Social Web and a range of services and applications to support their own as well as their team’s learning, performance and productivity. “ Jane Hart.
For each course we can determine the best mix of delivery modes according to:
Infrastructure available to deliver in each mode
Capability of our organisation and our learners to teach and learn in each mode
Strategic choices about how to package courses for blended learning
The types of content we will need to delivery
Learning activities
Assessment methods and potential for assessment submission.
For each course we can determine the best mix of delivery modes according to:
Infrastructure available to deliver in each mode
Capability of our organisation and our learners to teach and learn in each mode
Strategic choices about how to package courses for blended learning
The types of content we will need to delivery
Learning activities
Assessment methods and potential for assessment submission.
Identify is a perennial question in e-Assessment. Is the person submitting the assessment the person who is enrolled in the course? This has ramifications for both the integrity of the RTO but also the application of knowledge and skills in the workplace, not to mention potential liability. While identify of submitted written assessments is always an issue for RTO it is especially magnified in the online delivery.
Cheating – Turnitin can help with written assessments, however for shorter compliance courses tutor contact is essential to back up what the online system is telling you. If this is not possible, do a random selection of learners. Call and interview especially around the key knowledge so you have an extra observation to back up their results from the online system.
Skills – You can only use a simulated environment where the unit permits you too. Otherwise you’ll need to either do it in person or develop a strategy for remote evidence collection (apps, phone video, podcast)
Varity of Assessment – too many online courses rely soley on quizzes. While good for information retrieval and knowledge concepts, they are notoriously poor indicators of problem solving ability and information gathering to assist with new workplace processes. Decision making trees and problem based learning are useful cross checks here.
Just online – Much of corporate learning is heading towards just in time learning kiosks to support employees to solve many of their own workplace needs. The provision of a portal with video, short job aids, case studies, access to a trainer si where they are heading. Formal training is being saved for the big ticket items.
Content – A lot of online content I see from RTOs is deadly boring. Text with few or no images, little if any video, occasional use of sound to support learning concepts and assessment based purely on short answer comprehension and quizzes. Its very structure encourages learners to click through – it fails to immerse learners to think, absorb and apply information to vocational contexts. It also fails to show learners how to use the resources around them in the workplace (people, procedures, SOPS and manuals) to solve their own problems on the job. This was one of the key planks of the employability skills. Using webinars (live or recorded), PowerPoint's supported by audio, simple decision making trees (using the Lesson in Moodle or other rapid elearning software)
Isolation – Online learners generally work in isolation where the RTO has focused purely on self paced delivery mode without use of any F2F or online facilitated. Consider how learners may learn from one another by either sharing their workplace context or by running regular mandatory interactive webinars which will help you get a sense of the learners in your group and their progress toward competency.
Teachers – They are invisible in online self paced courses. The voice of the resource is generally that of the subject matter expert or instructional designer. Consider how to make the trainer active through forums, webinars and phone contact.
Identify is a perennial question in e-Assessment. Is the person submitting the assessment the person who is enrolled in the course? This has ramifications for both the integrity of the RTO but also the application of knowledge and skills in the workplace, not to mention potential liability. While identify of submitted written assessments is always an issue for RTO it is especially magnified in the online delivery.
Identity – the perennial issue raised by every dissenter to elearning. Need strategies in place to check and verify identity. The face to face savings need to be diverted to more stringent identity checks. Phone calls, support, weekly emails.
Cheating – Turnitin can help with written assessments, however for shorter compliance courses tutor contact is essential to back up what the online system is telling you. If this is not possible, do a random selection of learners. Call and interview especially around the key knowledge so you have an extra observation to back up their results from the online system.
Skills – You can only use a simulated environment where the unit permits you too. Otherwise you’ll need to either do it in person or develop a strategy for remote evidence collection (apps, phone video, podcast)
Varity of Assessment – too many online courses rely soley on quizzes. While good for information retrieval and knowledge concepts, they are notoriously poor indicators of problem solving ability and information gathering to assist with new workplace processes. Decision making trees and problem based learning are useful cross checks here.
Just online – Much of corporate learning is heading towards just in time learning kiosks to support employees to solve many of their own workplace needs. The provision of a portal with video, short job aids, case studies, access to a trainer si where they are heading. Formal training is being saved for the big ticket items.
Content – A lot of online content I see from RTOs is deadly boring. Text with few or no images, little if any video, occasional use of sound to support learning concepts and assessment based purely on short answer comprehension and quizzes. Its very structure encourages learners to click through – it fails to immerse learners to think, absorb and apply information to vocational contexts. It also fails to show learners how to use the resources around them in the workplace (people, procedures, SOPS and manuals) to solve their own problems on the job. This was one of the key planks of the employability skills. Using webinars (live or recorded), PowerPoint's supported by audio, simple decision making trees (using the Lesson in Moodle or other rapid elearning software)
Isolation – Online learners generally work in isolation where the RTO has focused purely on self paced delivery mode without use of any F2F or online facilitated. Consider how learners may learn from one another by either sharing their workplace context or by running regular mandatory interactive webinars which will help you get a sense of the learners in your group and their progress toward competency.
Teachers – They are invisible in online self paced courses. The voice of the resource is generally that of the subject matter expert or instructional designer. Consider how to make the trainer active through forums, webinars and phone contact.
Quizzes are distributed to learners via a LMS and added to mop up bits and pieces of knowledge that may not fit precisely into a skills demonstration. Quizzing is the most popular eAssessment method but they are notoriously difficult to write if you are going to go beyond simple knowledge retrieval and checking. Writing the question with a realistic set of detractors to the correct answer takes skill. Do you need to add feedback if learners are correct? Quizzes are useful for:
Replacing short answer questions that require marking
Breaking down processes into a set of stepped quiz questions
Meaning of words and phrases
Scaffold off a video or other presentation (PowerPoint)
Checking knowledge progression through a course.
Evidence capture of skills involves the use of a combination of video, audio, text.
Capturing skills evidence via mobile phones, tablets, HD portable cameras is an effective method when used with a specific evidence framework:
No hoodies!
Speaking to camera and identify self and processes working through
Reinforced by pictures and text or audio from the learner
Comments from Work supervisor
Guided by checklist provided by the learner
Before and after skills demonstration views of the work/site/ etc
Answering of supplementary questions either verbally within the mobile evidence method or later via quiz or short answer
Visual evidence (Camera Phone)
Documented Evidence & Checklists (Evernote, Writable PDFs)
Phone Casting (iPadio)
Group Work verification (Wikis)
While quiz based, DST goes beyond the linear/right/wrong nature of quizzes enabling powerful mazes of understanding to be created that test both the decision making capability of the learners as well as their resourcefulness of acquiring and apply knowledge from sources the trainer provides.
Quiz based but wraps a story around a workplace process you want to check for a learners understanding of applying problem solving and use of external resources and materials (procedures, policies etc). Scenario driven and problem based. Usually built into a LMS but many applications allow you to easily develop and distribute DMT in conjunction with content (Captivate, Articulate Storyline, iSpring etc). Articulate can also house and score the content for you in the cloud.
DMT useful for both formative and summative assessment
Decision-making trees are sometimes referred to as Action mazes. Similar in concept to a 'choose your own adventure' novel, they provide a highly effective structure within which you can lay out options. Learners can investigate the possible outcomes of choosing those options. They help the learner to form a balanced picture of the risks and outcomes associated with each possible course of action.
Students can safely explore the 'cause and effect' nature of a scenario or process. Consider:
an occupational health and safety scenario
a simulation of a practical activity that might involve dangerous chemicals or equipment
a scenario, such as a human relations dispute, that might take weeks to 'play out' in real time.
This is how the DMT differs from the quiz
An example of content integrated with activity and assessment. Starts with a problem that a learner has to solve by investing the provided content or by exploring the answers. “Ask Joe” is content in audio-visual form to assist the learner understand approaches to this particular problem. A wide variety of existing content can be used in DMT
DMT are most effective when a workplace issue is contained within a story with specific characters. Learners are usually seeing the action from a workers perspective in the role their training is preparing them to occupy. This enables learners to explore process, communication, information and the application of policy and procedure in a simulated workplace setting.
When you offer a webinar to your learners you are truly entering blended learning territory. Not only are they wonderful for building a community of learners and delivering content and offering opportunities for clarifying your expectations, webinars also provide e-assessment opportunities especially for CIV and Diploma level learners. They are very much about the students voice and if you run remote courses, provide you with good evidence on learner progress you can cross match against other submitted evidence for direct follow up or additional questioning where you have doubts on a student’s progress or identity. Methods include:
Remote skills demonstration
Assessment Task Pitch
Group problem solving, Case studies or scenarios
Peer Review of presentations and Group projects
Communication role plays
Webinars can be recorded and stored either as files or direct links to supplier cloud hosting.
These can range from simple games and decision making trees to highly developed workplace simulations or virtual worlds. Expensive to implement and usually requires industry/government support at RTO level – for example the White Card game http://www.whitecardgame.com.au/
Dialogues between trainer and student at higher level qualifications can support learners to refine their assessment response to major written assessments, project pitches, placement questions etc. Time consuming but excellent way to record learners journey to summative assessment competence as well as identify verification.
Dialogue tool in Moodle or use of targetted social media tool such as Yammer.
Course information & updates
Engaging learners in warm-up activities
Problem-based learning (debates, role-plays, problem-based activities, projects, feedback, resources)
Learners stories (industry knowledge)
Group work
Projects/journal
Feedback and evaluation on the course
Discussion activities can be
undertaken using a variety of methods:
Discussion forums
Social networks
Voice boards
Instant Messaging
Comments posted to wikis or blogs or LMS
Need to be mapped to Assessment if content driven!
Link to assessment
Base on participants sharing experiences
& prior knowledge – “learners stories”
Application to workplace or experience
Make pre readings short
Mix of problems, case studies, reflections
Commenting/Building on postings
Use to demonstrate employability skills
Track to determine contribution to group
Peer Review (Moodle post rating)
Share experiences
Wiki/LMS/Social Media
Common issues
Variety in the sector
Private V Shared
Ethics & Confidentiality practised and applied
“Case Study” style assessment creates learning for subsequent learners
Common Issues on the job
Make assessment responses visible
Third party reports
Confidentiality and non disclosure
Two ideas
Google calendar as a diary
Learner group private wiki
Group work always leaves the teacher vulnerable at audit for decisions made in determining the amount and quality of each learner’s contribution.
Peer Review tools allow teachers to survey learners anonymously on each of their team member’s contribution to group process and project outcomes as well as rate the quality of content/ideas/resources they have provided or contributed to.
Each learner as part of their assessment would rate each of their team members on
contribution to group processes
quality of work product contributed
This gives the teacher additional information for assessment.
A model that can work is:
Develop a rubric with learners
Check for mapping
Distribute back to learners as marking guide
Rehearse the marking guide
Conduct presentations (summative)
Collect peer review
Redraft and resubmit
Share and assess
Moodle has an underutilised resource called Workshop in which you can develop, distribute and mark peer review assessments. The process of participating in the peer review can be mapped to critical aspects of assessment that involve performance reflection, communication and working in groups, and industry knowledge.
Film learners/third party/supervisor demonstrations
Ensure learner speaks to camera to describe what they have done, job requirements, conditions and special aspects of job
Add to LMS as content and or as part of peer review assessment where learners contribute to assessing appropriateness of tools, equipment, OHS, sustainability, quality and finish on a job.
Learners view same job in variety of conditions and settings