Basics of Digital Testing - Online Workshop at TU Ilmenau

Basics of Digital Testing
Online Workshop at TU Ilmenau
Mathias Magdowski
Electromagnetic Compatibility
Institute of Medical Engineering
Otto von Guericke University, Magdeburg
December 6 & 7, 2021
License: cb CC BY 4.0 (Attribution-ShareAlike 4.0)

Motivation Oral Exams Written Exams Quiz-Based Exams
The Big Question
Basics of Digital Testing 3

The Big Question
+ = ?
Source left: https://pixabay.com/photos/school-exam-french-bad-wrong-fail-683553/
Source right: https://pixabay.com/vectors/germ-bacillus-angry-fight-against-158107/

The Simple Answer

The Simple Answer
Source: jeferrb via Pixabay
https://pixabay.com/de/vectors/netzwerk-iot-internet-der-dinge-782707/

The Complicated Answer

Think outside the box!
Connect the 9 points with 4 lines without putting down the pencil!

Deeper Problem
A pure digitization of existing analog exam formats
without further adaptation is not expedient.

Deeper Problem
A pure digitization of existing analog exam formats
without further adaptation is not expedient.
Thorsten Dirks, CEO of
Telefónica Germany:
“If you digitize a crap
process, you have a crap
digital process.”
Source: E-Plus Group, CC BY 2.0,
https://commons.wikimedia.org/w/index.
php?curid=40920042

Traditional Testing vs. Real-World Work
Source: https://pixabay.com/de/photos/
taschenrechner-notizblock-1687962/

Traditional Testing vs. Real-World Work
Source: https://pixabay.com/de/photos/
taschenrechner-notizblock-1687962/
Source: https://pixabay.com/de/
arbeitsplatz-team-gesch%C3%
A4ftstreffen-1245776/

Recommendation
Less than 20 examinees:
▶ online oral exams or alternative formats

Recommendation
Between 20 and 100 examinees:
▶ take home exams or open book exams with randomized,
competency-based tasks

Recommendation
More than 100 examinees:
▶ take home exams or open book exams with multiple choice
questions from large pools
▶ peer review

Recommendation
More than 100 examinees:
▶ take home exams or open book exams with multiple choice
questions from large pools
▶ peer review
No costly and non-beneficial online proctoring!

Survey
How many examinees do you have to assess (in your
largest lecture)?
Please type a number in the chat!

Why No Proctoring?
Mission statement for teaching:
“The basis of our information and communication
culture are trust and transparency.”

Why No Proctoring?
Mission statement for teaching:
“The basis of our information and communication
culture are trust and transparency.”
Open questions:
▶ safer than in the exam room?
▶ third persons in the room, in-ear phones
▶ using pre-produced video images and sounds
▶ going to the toilet
▶ required bandwidth, handling disconnections
▶ effort for the supervising person & Hotline

Why No Proctoring?
Proctoring and remote video monitoring of examinees in
online exams are either useless or disproportionate to
possible student cheating attempts in the case of
inappropriate tasks.

Why No Proctoring?
Think about your own living conditions as a student!
Source: https://twitter.com/MMagdowski/status/
1372580986882506754

Alternative Ideas
Ideas:
▶ replace one summative assessment with many small
formative assessments (process vs. result)

Alternative Ideas
Ideas:
▶ do not ignore but include the Internet in courses and
exams as an intrinsic part of students’ lives

Alternative Ideas
Ideas:
▶ freely selectable 24/7 exams with competency-based
open-book could be a competitive advantage for smaller
universities (exams for all students at the same time are no
longer necessary from an organizational point of view)

Alternative Ideas
Ideas:
▶ better accessibility of open online exams (translation)

Alternative Ideas
Ideas:
▶ allow cooperation in group exams

Alternative Ideas
Ideas:
▶ allow cooperation in group exams
▶ value giving feedback instead of giving grades

Overview
Motivation
Online Oral Exams and Alternatives
Handwritten Exams
Quiz-Based Exams

E-Learning Tools
Source:
https://iconscout.com/icon/moodle-2
(a) Moodle
Source: SuperBubble26, CC BY-SA 4.0,
https://commons.wikimedia.org/w/index.
php?curid=97319597
(b) Zoom
Source: DFN, https://www.conf.dfn.de/
(c) DFNconf
Source: https://commons.wikimedia.org/w/
index.php?curid=47104086
(d) WebEx

Online Oral Exams and Alternatives

Oral Examinations via Videoconference
Eleanor Examiner Sarah Student Owen Observer
Hello everyone to the exam
in mathematics 2! Ms. S.,
I would like to introduce
Mr. O., who will serve
as exam observer.
Hello!
Hello!
Source: “Mündliche Prüfung per Videokonferenz aus Studierendensicht” from Jörn Loviscach, translated by Mathias
Magdowski, via CC-BY-NC-SA 3.0
Slides: https://bit.ly/OralMathExams

Media Literacy Exams
Possible examination forms:
▶ Electronic portfolio
▶ Writing a blog
▶ Recording of videos or
podcasts
▶ Science communication in
social media
Source: Pixabay https://pixabay.com/vectors/internet-web-page-video-article-3738875/

Pupils or Students Create Videos Themselves
Source: vidumath – creative video for mathematics – VG-SPS-BE-15-24-013795
Website: http://vidumath.eu/

Documentation of the Solution of Personalized Tasks
https://youtu.be/8pKgmvlZG54

Improvised Tripod for Filming of Handwriting
Figure: Plastic box, spatula, book and household rubber
Mathias Magdowski, CC BY-SA 4.0, https://mathiasmagdowski.wordpress.com/2019/04/28/
improvisiertes-stativ-zum-abfilmen-einer-schreibenden-hand/

Example: Donkey Car Project
Website: https://ovgudonkey.wordpress.com/

Example: Twitter Channel for EMC Measurements
Website: https://twitter.com/EmvOvgu

Example: LEGO Programming Lab
Kevin Tom Robert Heine, CC BY-SA 4.0,
https://journals.ub.ovgu.de/index.php/LEGO/article/view/2029
Website:
https://www.eit.ovgu.de/Presse+_+Medien/
Spannende+und+innovative+Konstruktionen+beim+
LEGO_Praktikum+2021-p-2970.html

Turnitin for Originality Checks Against Plagiarism
https://twitter.com/MMagdowski/status/1395402777086599168

Survey
Which alternative forms of assessment have you tried
out?
Please raise your hand in Zoom!

Open Book Online Exam
Source: https://pixabay.com/de/photos/studium-pr%C3%BCfungen-vorbereitung-951818/

Example of an Individual Test Sheet
Source: https://mathiasmagdowski.wordpress.com/2021/05/05/
online-take-home-elektrotechnik-pruefung/

Ten Topics for the Exam
1. Charge and current (12 tasks)
2. Temperature dependent resistance, current density,
resistivity, capacitance, inductance (32 tasks)
3. Basic circuits, current & voltage divider, DC power,
equivalent resistance, power matching (26 tasks)
4. One-port and two-port network theory (33 tasks)
5. Branch current analysis, mesh current analysis or nodal
voltage analysis, superposition (33 tasks)
6. Mean and effective value, complex calculation (21 tasks)
7. Phasor & Nyquist plot, oscillating circuits (27 tasks)
8. Complex power and power factor correction (14 tasks)
9. Three-phase current (13 tasks)
10. Transient calculation (11 tasks)

Diversity Because of a Large Catalog of Tasks
Source: https://imgflip.com/i/4x1vg1

Submission Form

Overview of the Submission Assignments in Moodle

Submission Settings in Moodle

Decleration of Originality

Proctoring the Other Way Around

Text File aufgabennummern.txt
1. Task: No. 5 with 12 points
2. Task: No. 98b with 8 points
3. Task: No. 27a with 7 points
9. Task: No. 146c with 10 points

I can haz Gauzzian distributionz
≥ 0 % ≥ 10 %≥ 20 %≥ 30 %≥ 40 %≥ 50 %≥ 60 %≥ 70 %≥ 80 %≥ 90 %
0
5
10
Point share of the total score
Absolute
frequency

Timing of Submissions Before the Deadline
−60 −50 −40 −30 −20 −10 0
0
20
40
60
Time t of submission before the deadline in min
Number
of
submissions
Task 1
Task 2
Task 3
Task 4
Task 5
Task 6
Task 7
Task 8
Task 9
Task 10

The dog ate my internet connection.

Videos Do Not Help Either
Source: Eine “umgedrehte” Video-Nachbesprechung einer Leistungskontrolle im E-Technik-Grundstudium
https://dl.gi.de/handle/20.500.12116/24416

Videos Help After All

Time Expenditure
Preparation: ↑

Time Expenditure
Preparation: ↑
(Every beginning
is hard!)

Time Expenditure
Preparation: ↑
(Every beginning
is hard!)
Execution: ↓

Time Expenditure
Preparation: ↑
(Every beginning
is hard!)
Execution: ↓
Correction: ↓ or ↑

Survey
What are your experiences with open book exams and
take home exams?
https://www.menti.com/123456

Open Book Exams on Site
Source: https://twitter.com/MMagdowski/status/1422867258129461248

What Aids are Popular?

Are the Tools Used Wisely?
What I would use a laptop
for:

for:
▶ MATLAB, GNU Octave
(complex calculation,
solving systems of
equations, plotting)
▶ Wolfram Alpha
▶ LTspice, EasyEDA
(circuit simulation)
▶ Maxima (computer
algebra system,
analysis)

for:
solving systems of
▶ Wolfram Alpha
algebra system,
analysis)
What the students used
the laptop for:

for:
solving systems of
▶ Wolfram Alpha
algebra system,
analysis)
What the students used
the laptop for:
▶ Full text search in
script/book
▶ Scrolling through slides,
exercise notes & sample
solutions
▶ Research at Wikipedia
and (passive) forum use

Point Distribution
≥ 0 % ≥ 10 % ≥ 20 % ≥ 30 % ≥ 40 % ≥ 50 % ≥ 60 % ≥ 70 % ≥ 80 % ≥ 90 %
0
2
4
6
8
10
12
Punkteanteil
absolute
Häufigkeit
Punkteverteilung Gesamtpunktzahl

Paper over Paper

Let Students Create Their Own Exam Tasks

Personalized Tasks and Anonymous Peer Review
Registration
MAT-
LAB
LaTeX
Matriculation
No.
PDF
Task
Sample Solution
Folder
Form
ZIP File
MAT-
LAB ZIP File
MAT-
LAB
2 Reviews
Sample Sol.
Score
List
https://youtu.be/LDw_Ifmg2WM

PAssT (from Technical University Dresden)
Goals:
▶ Low-threshold integration of peer assessments into various
learning scenarios
▶ more transparent evaluation processes
▶ increased relevance of tasks (subjectively experienced)
▶ more speed and quality of feedback in the learning process
https://passt.mz.tu-dresden.de/

Peer Review in FeedbackFruits
https://feedbackfruits.com/peer-review

What Makes a Good Single-Choice Question?
What is the capital of
Zambia?
1. Livingstone
2. Lusaka
3. Mongu
4. Solwezi

What Makes a Good Single-Choice Question?
Zambia?
1. Livingstone
2. Lusaka
3. Mongu
4. Solwezi
Zambia?
1. London
2. Lusaka
3. Moscow
4. Stockholm

Idea

Idea
Question text:

Idea
Answer formula:

Procedure
▶ find a suitable tasks

Procedure
▶ develop a sample solution, check the effort

Procedure
▶ write down the task and solution in L
A
TEX

Procedure
A
TEX
▶ convert the task text into HTML format

Procedure
A
TEX
▶ convert the solution into an answer formula

Procedure
A
TEX
▶ generate randomized input values

Procedure
A
TEX
▶ think about a meaningful feedback

Procedure
A
TEX
▶ store the question in the catalog

Procedure
A
TEX
▶ transfer the questions into the actual Moodle test

Procedure
A
TEX
▶ transfer the questions into the actual Moodle test
▶ protect the test and forward it for testing

Feedback from colleagues
1. estimate a realistic amount of time needed for the solution
Source: https://pixabay.com/de/vectors/alarm-uhr-wecker-zeit-aufwachen-1673577/

Feedback from colleagues
1. estimate a realistic amount of time needed for the solution
2. find typos and ambiguities in the wording
Source: https://pixabay.com/de/vectors/alarm-uhr-wecker-zeit-aufwachen-1673577/

Sent about one day before the assessment
Dear students,
as discussed in the online plenary today, here is all the information about tomorrow’s
assessment. The assessment will be conducted as an online test via Moodle and will
include various numerical and multiple-choice questions.
Access link: https://elearning.ovgu.de/mod/quiz/view.php?id=xxxxx
The online assessment will be available on Wednesday, January 20, 2021, between
7:00 pm and 9:00 pm. However, the completion time only is limited to 1 hour and 30
minutes, leaving 30 minutes buffer for any technical issues. You will have one allowed
attempt for the solution to answer all nearly 30 questions. All participants get the same
questions, but in different order and with different numerical values.
Randomizing the order may separate similar subtasks from related task types.
Therefore, it is useful to note down intermediate values to avoid multiple identical
calculations, because the numerical values are always the same for these related
subtasks.
To prevent guessing the solution, the tolerance for the numerical values is set to ±1 %,
so you should not round too much.
. . .

Settings

The students are on time!
After one minute:
After ten minutes:

Problem with rounding errors

Point distribution – online assessment (2021)

Point distribution – face-to-face assessment (2019)

Main experiences and lessons learned
▶ higher preparation time (question design)

▶ drastically reduced correction effort
(worthwhile from 50 to 100 students)

▶ cannot map all competencies of handwritten assessments

▶ intensively inform students to prevent ambiguity, queries,
and frustration

and frustration
▶ trial assessment on the same platform with similar tasks

and frustration
▶ multiple, low-threshold, redundant opportunities to ask
questions (e. g., via Zoom, by phone, by email).

and frustration
▶ multiple, low-threshold, redundant opportunities to ask
questions (e. g., via Zoom, by phone, by email).
▶ students don’t take great advantage of Internet usage,
numerical calculation tools such as MATLAB/Octave, . . .

PeerWise to Collect Quizzes Created by Students
Goals:
▶ students create, share, evaluate, and discuss quiz
questions
▶ instructors can view student-created content and track
their participation
▶ easy to use
https://peerwise.cs.auckland.ac.nz/

UCAN Assess to Curate Exams Questions
Ideas:
▶ curated database of
multiple-choice exams
questions
▶ used in medicine
courses
▶ network to use common
exams resources
▶ save time
▶ manage quality
▶ requires a license
https://www.ucan-assess.org/

dailyGETquiz on Instagram
https://www.instagram.com/dailygetquiz/

Survey
Which question type of Moodle do you prefer and why?
Please raise your hand in Zoom!

Finally Something Humorous
In 1665, the University of Cambridge temporarily
closed due to the Great Plague. Isaac Newton had to
switch to a virtual learning environment, this involved
fixing course materials to cows which were sent all over
the country to students’ homes and that’s why it’s called
Moodle.
Paul Coxon auf Twitter https://twitter.com/paulcoxon/status/1238402200713465856

Thank you very much for your attention!
Are there any questions?

Basics of Digital Testing - Online Workshop at TU Ilmenau

Recommandé

Recommandé

Contenu connexe

Similaire à Basics of Digital Testing - Online Workshop at TU Ilmenau

Similaire à Basics of Digital Testing - Online Workshop at TU Ilmenau (20)

Plus de Mathias Magdowski

Plus de Mathias Magdowski (20)

Dernier

Dernier (20)

Basics of Digital Testing - Online Workshop at TU Ilmenau