SCHOLAR: eAssessment design for science and engineering - Helen Ashton

eAssessment
Design
for Science &
Engineering
Dr Helen Ashton
Head of eAssessment, SCHOLAR
Heriot-Watt University

Overview
Why bother?
What is important?
Different approaches
Assessing the right learning outcome
Supporting the learner

A programme of Heriot-Watt University  SCHOLAR  Prògram Oilthigh Heriot-Watt

Why bother?

Support students
Independent learning
Reduce marking
Immediate response
Improve feedback
To students
To staff


What is important?
Repeated practice
Constructed answers
Feedback
Appropriate assessments
Progress monitoring
How am I doing?
How are they doing? (But not ALL the time!)


Simplify
3 (7 - x + 6x2 + 4x3 - 8x4) - 4 (2 - 2x + 9x3)
The 4 options:
 -1 + 7x + 6x2 - 32x3 - 8x4 •Not using multiplier of 1st bracket

 29 - 11x + 18x2 + 48x3 – 24x4 •Not using sign for 2nd bracket

 23 - 5x + 18x2 + 21x3 – 24x4 •Not using multiplier of 2nd bracket

 13 + 5x + 18x2 - 24x3 – 24x4 •Correct


Simplify
3 (7 - x + 6x2 + 4x3 - 8x4) - 4 (2 - 2x + 9x3)
The 4 options:
 -1 + 7x + 6x2 - 32x3 - 8x4 •Not using multiplier of 1st bracket

 29 - 11x + 18x2 + 48x3 – 24x4 •Not using sign for 2nd bracket

 23 - 5x + 18x2 + 21x3 – 24x4 •Not using multiplier of 2nd bracket

 13 + 5x + 18x2 - 24x3 – 24x4 •Correct

Correct answer can be found by multiplying out the numbers only.

Need to be aware the question is different to a constructed version.

Often when MC versions are created, questions are changed dramatically.


What is the equation of the line?

Assume student knows:
• the form of a straight line equation,
• that m is the gradient & relates to how much we move up and along,
• that c is the intercept & relates to where the line crosses an axis.

So… choices are expression of the form y = m x + c
m: 2, ½, -2, -½
with possible values of
c: 4, -2


What is the equation of the line?
 y=2x+4
y=2x-2
y=½x+4
y=½x–2
 y = -2 x + 4
 y = -2 x – 2
 y = -½ x + 4
 y = -½ x - 2

Even with all these assumptions, we still have 8 choices for the answer.

How many choices are needed if we don’t make all of these assumptions?

How easy is it to find your answer in all of these choices?


Designing Assessments

Start with the learning objectives

Think about what you would LIKE to do

Technical Educational

Develop question

Review against learning objectives


Question Design
Often we are trying to create a question
online which used to exist on paper.

We need to assess the same objectives.

What are our options?

How does this effect the appropriateness of
the question?


Constructed Answers: Example
Diagrams 1 and 2 on the worksheet show part of
the graph y = f (x).
(a) On Diagram 1, draw the graph of y = - f (x).
(b) On Diagram 2, draw the graph of y = f (x + 2).


Option 1 – Multiple Choice


Option 1 – Multiple Choice
Advantages
Easy to create and mark
Easy for student to answer
Some randomisation possible
Disadvantages
Recognition and Elimination NOT
construction
High screen real estate required
Limited possible answers


Option 2 - Hotspot


Option 2 - Hotspot
Advantages
Easy to mark
Lower screen real estate
Construction of an answer - many possibilities
Partial credit possible

Disadvantages
Answer is not a graph of –f(x)
Could be fiddly for student to answer
Time consuming to create
Low reusability - no randomisation


Option 3 - Flash


Option 3 - Flash
Advantages
Lower screen real estate
Easy to mark
Construction of an answer – many possibilities
Partial credit possible
Graph shows in context
High reuse - randomisation possible

Disadvantages
Creation requires multimedia authoring skills
Some accessibility issues


Appropriate Learning Outcome
Solving simultaneous linear equations…
Partly answered question
(a) algebraically (b) graphically
Showing feedback.

Graphically, student must demonstrate
A programme of Heriot-Watt University  they can draw the appropriate graph.
SCHOLAR  Prògram Oilthigh Heriot-Watt

Appropriate marking

Follow through: allowing
students to gain credit
even if they make an initial
mistake.


Appropriate marking
Partly answered question showing follow through and feedback.

Here the best fit line is
incorrect, but the
appropriate equation of
this line has been entered.

Feedback and marks are
given as appropriate.


Be adventurous!


What do teachers think?

“Constructing questions in this format that
challenged the students was considerably
harder than I'd expected but probably took
no longer than constructing a traditional
exam. And it was nice to have no
marking!”

Computer Science Lecturer, Heriot-Watt University

What do teachers think?

“It moves the work from marking the
exam to setting the exam, and allows one
to concentrate on what is a suitable
question.”

Computer Science Lecturer, Heriot-Watt University

Any Questions?
H.S.Ashton@hw.ac.uk

Online Demonstrations
http://www.calm.hw.ac.uk/pass-it.html
A variety of questions can be found in the demos.

SCHOLAR Examples: http://scholar.hw.ac.uk
(Choose Country> Try SCHOLAR)


SCHOLAR: eAssessment design for science and engineering - Helen Ashton

Recommandé

Recommandé

Contenu connexe

En vedette

En vedette (9)

Similaire à SCHOLAR: eAssessment design for science and engineering - Helen Ashton

Similaire à SCHOLAR: eAssessment design for science and engineering - Helen Ashton (20)

Dernier

Dernier (20)

SCHOLAR: eAssessment design for science and engineering - Helen Ashton