665 Session11-data analysis-s13

Student Learning Outcomes:
Data Analysis & Interpretation
Student Learning Gains
and Input for Revisions to Instructional Design

•Dr. Diane Nahl
Spring 2013

•LIS 665 Teaching Information Technology Literacy
University of Hawaii LIS Program

Instructional Design Stages
a. Needs Assessment: What is and what is needed?

b. Define Goals & Objectives: What will they do and learn?

c. Select Formats, Methods & Materials: How is best?

d. Devise Test & Evaluation Procedures: Prototyping it.

e. Construct & Teach Prototype: Performing it.

f. Evaluate & Analyze Outcomes: What did they get?

g. Revise & Recycle Steps: What will be different next time?

LIS 665 Teaching Information Technology Literacy 2 Nahl 2013

Blended Librarians
• Embedded librarianship (2004 Barbara Dewey)

• Online librarianship

• Cybrarian

• Collaboration and partnerships with teaching
faculty, students, and institutional or community
events and initiatives.
• Often immersed within ICT environments

• Long-term involvement

Learning Fundamentals
• Learning is about actively making connections:

• Biological: establishing neural networks,
activating emotion receptors, and developing
muscle memory

• Conceptual: connecting ideas, making meaning,
and problem solving

• Experiential: interacting with self, others, and the
socio-technical information environment

Learning Fundamentals
• Learning is a developmental, cumulative process
involving the whole person.
• Learning is shared within a social process.
• Learning that is constructivist, active and
immersive is longer lasting.
• Learning requires frequent feedback.
• Learning often takes longer than teaching.
• Learning is sometimes instantaneous.

Holistic Instructional Design
for the Three-fold Self
• Teach the Heart
• [A] Affective learning

• Teach the Mind
• [C] Cognitive learning

• Teach the Body
• [S] Sensorimotor learning


Worksheet Data Analysis
• Radcliff et al. Ch 11: Which types of scoring guides will you use
on your outcomes data that measured performance?

• In your case, forms, worksheets, and discussion posts were
used to gather responses in active learning exercises.
• How will you score the responses on the worksheets? How will
you assign levels of completeness or correctness?

• To answer the questions above complete the data
analysis exercise Steps 1 through 8 on the following slides by
April 4 (2 weeks).


a. First Step: Make a copy and then examine one of the online
worksheets from an activity and read all of the responses
from every student or team/group on that worksheet.
a. Assign an ID code to each student directly on the worksheet and
use that ID code in the spreadsheet. (Work only on copies of
student worksheets)

b. Second Step: Set up your scoring criteria for correct, partial,
and incorrect or blank responses.
a. You may score multi-part questions separately (e.g., Q1a, Q1b,
etc.
b. You may use various approaches given in this chapter but you
must assign numerical scores for each level of correctness or
ideal to each worksheet response/answer.

c. Third Step: Read each worksheet again and score the
individual question item responses. Score all of the
worksheets for that exercise.

d. Fourth Step: Use the spreadsheet (shared last week) to
record the individual question scores (so you know how
everyone did on each question as in the Hillyer article Fig. 7.2).
a. This requires an ID column and showing each student’s
data in a row (e.g., Student 1, Student 2, etc.) and each
question or part of a question in columns, then enter the
student scores in the appropriate column.
b. For clarity add a meaningful content word or phrase to the
question number (e.g., Q4 Search Terms; Q5 Boolean) on
the spreadsheet to easily show what the scores relate.


e. Fifth Step: Sum the columns and compute the average score
for each question. [total of all scores on that question/total
number of students]

f. Sixth Step: Sum the rows and compute the average for each
student and convert to a percentage.
a. [(total of all scores on every question by each student/maximum
possible score) x 100]

e. Seventh Step: Sum the total scores for all students to get a
global score for the entire class on that worksheet, compute
the average and convert to a percentage.
a. [(global total of all questions for all students/maximum possible
score) x 100]


h. Eighth Step: Examine the spreadsheet and look for patterns,
e.g.,
a. What is the range of scores on each question (highest and lowest
scores)?
b. Which question was most difficult for most students? Which
question was easiest for most students?
c. If you graded each student based on the maximum possible,
what letter grade would each receive (90-100% A; 80-89% B; 70-
79% C; 60-69% D; 50-59% F)? What letter grade would the entire
class receive on this worksheet?
d. What other patterns do you see in your data? These findings will
be part of the Assessment Report write-up.


Data Analysis Exercise:
Content Analysis Method Ch 13 Radcliff et
al.
• Examine open-ended data from your session by reading all of the
responses, may use Psych GG comments posted within 2 weeks
after the session. Follow Content Analysis steps 1-7 + 8:
• Select an open-ended narrative data instrument (minute writing
reflection exercise, open-ended worksheet responses or session
evaluation, or other type) and read the entire set of responses.
• Begin with Step 1 Determine What Questions You Want to
Answer and articulate what you intend to analyze and why.
• In Step 2 determine what content you will extract and count for
the analysis.
• In Step 3 define what you will count as an occurrence
• Necessary to be very specific and concrete to avoid ambiguity


Content Analysis Method
• In Step 4 Create a Coding Sheet to organize data
extraction (Google document spreadsheet)
• In Step 5 Pre-test the coding sheet. Each Team member
codes the same section individually, then compare your
work
• If necessary, discuss the differences and standardize the
coding definitions to obtain nearly complete agreement in
coding
• Agreement among raters should be 95% or higher


• In Step 6 after achieving high agreement in coding, take
separate portions of narrative and code it individually.
• Compare coding at intervals to ensure agreement remains high
• In Step 7 Use the spreadsheet to enter and then analyze
(compute or summarize) the coded data.
• It is best if you code independently and maintain individual
spreadsheets, or separate sheets in one workbook.
• Compare your individual results and compute the percent
overlap (>94%)
• Repeat steps 1-7 to analyze the rest of your open-ended data as
well as the PSYCH GG comments posted after your session


• In Step 8 (not numbered in text) you will turn your
spreadsheet data into information
• Compute descriptive statistics of totals (frequencies),
averages (the mean), and percentages
• Clusters of terms within a concept, categories and their
members
• Compute individual student statistics and clusters
(encouraged for this project)
• Compute inferential statistics about the population (not
required for this project)
• Make a table or graphic chart to display results together.
• Give the table a meaningful title.


In Two Weeks
• Instruction Unit Parts I & II due April 4

• Guest presentation: Sean Thibadeaux, Reference and
instruction librarian, Hawaii Pacific University

• Ch 8

• CTSB report

• Mackey & Jacobson

• Bring numerical data in tables and/or graphic charts from
your session for data analysis

665 Session11-data analysis-s13

Recommandé

Recommandé

Contenu connexe

Tendances

Tendances (13)

En vedette

En vedette (20)

Similaire à 665 Session11-data analysis-s13

Similaire à 665 Session11-data analysis-s13 (20)

Plus de Diane Nahl

Plus de Diane Nahl (20)

Dernier

Dernier (20)

665 Session11-data analysis-s13

Notes de l'éditeur