1. ASSESSMENT CULTURE AND INFORMATIONAL SELF-DETERMINATION:
A COMPARATIVE STUDY OF ELECTRONIC PERFORMANCE MONITORING IN
ONLINE LEARNING ENVIRONMENTS IN THE U.S. AND GERMANY
Daniel Knox: The University at Albany, State University of New York, U.S.
Markus Deimann: Der FernUniversität in Hagen, Germany

2. Roadmap
 Conceptual frame
 Cultureof Assessment
 Informational Self-Determination
 Data & Methods
 Preliminary Results
 Implications for Research & Practice

3. Issue and Purpose

4. Culture of Assessment
 Proponents
 Institutions
should commit to assess student learning
outcomes (Michaud, 2010).
 (Typically) quantitative methods over qualitative
methods (Turgeon, 2010).
 Formative assessments over summative assessments
(Weiner, 2009).

5. Culture of Assessment
 Critics

6. Assessment as EPM
 Online assessment is electronic performance monitoring (EPM):
 The use of computer and communication technologies to collect and store
information about individual or group performance (Aiello & Douthitt,
2001)
 Two distinct aspects:
 Automated
 Totalizing
“Blackboard records every click by a user within a course and allows
instructors to generate graphical reports on course usage and actvity” (UWE
Staff Guide, 2009).

7. Informational Self-Determination
 Legal concept:
 The degree to which a subjects maintain control of their
own data (Hornung, 2009).
 Distinct from the U.S. concept of privacy.
 Legal Origins:
 Volkszählungsurteil (census verdict), December 15,
1983
 Personal data in Germany are constitutionally protected.
 Individuals have the power to decide when and to what
extent personal information is viewed.

8. Informational Self-Determination
 Implications for online learning:
 Data Protection Commissioner at every institution.
 Place strict limitations in the collection and storage of
personal information.
 Every automated act of data processing that can be directly
linked to a certain person must be regulated by a specific law.
 Otherwise data processing must be authorized in each instance
by the individual.

9. Informational Self-Determination
 In practice:
 Inonline, asynchronous courses, German faculty may
not:
 Require participation in discussions.
 Grade discussions posts.
 Creates uncertainty on the part of instructors.

10. The Culture of Assessment vs. Informational
Self-Determination
 Not universal ‘best practices,’ but culturally
embedded.
 Both encroach upon academic freedom.
 So…what do faculty do to promote engagement in
online discussions?

11. SLN Data & Methods
 Data:
 Stratified random sample of 160 asynchronous, fully-online
courses drawn from the State University of New York
Learning Network’s (SLN) archives.
 Methods:
 Course documents (syllabi) analyzed for references for
online discussion policies (1141 of 2058 documents) and
compiled.
 Discussion policy documents analyzed using grounded theory
methodology (Strauss & Corbin, 1998).
 Each sentence analyzed using the message as the unit of analysis in
Atlas.ti.

12. The Culture of Assessment Goes Online
 Absence of a critical stance.
 Universality of practices.
 National Research Council’s Commission on Behavioral and
Social Sciences and Education
 How People Learn (Bransford et. al. 2000)
 “Good learning environments are knowledge centered, learner
centered, and assessment centered.”
 “Effective teachers … do a great deal of on-line monitoring of both
group work and individual performances…” (pg. 140).
 Community of Inquiry
(Anderson, Rourke, Garrison, & Archer, 2001 )

13. FernUniversität Data & Methods
 Internet survey
 20 item questionnaire
 154 faculty from cultural and social sciences
 Delivered via LimeSurvey
 25 responses (16.2%)

14. Results SLN
Quantitative Properties (9) Quantitative Dimensions (5)
General participation Hard count
Initial posting Minimum
Lead discussion Maximum
Lines Optimum
Logins Range
Logins with post
Respond to initial posts
Respond to peers
Words
Qualitative Properties (12) Qualitative Dimensions (4)
Accuracy Undefined
Creativity General
Critical Thinking Specified
Etiquette Highly Specified
General Interactivity
Integrity
Objectivity
Reference Peers
Relevance
Subjectivity
Supporting Evidence
Writing

15. SLN Example
Monitored properties and dimensions in a graduate education course:
Property Presence Dimension Number Low Range High Range Unit
Quantitative Yes
General participation Property Not Found
Initial posting Property Not Found
Lead discussion Property Not Found
Lines Property Not Found
Logins Property Found Minimum 2 Week
Logins with post Property Not Found
Respond to initial posts Property Found Minimum 2 Module
Respond to peers Property Found Hard Count 1 Discussion
Words Property Not Found
Presence Specification
Qualitative Yes
Accuracy Property Not Found
Creativity Property Not Found
Critical Thinking Property Found General
Etiquette Property Found Specified
General Interactivity Property Not Found
Integrity Property Not Found
Objectivity Property Not Found
Reference Peers Property Not Found
Relevance Property Not Found
Subjectivity Property Found General
Supporting Evidence Property Found Specified
Writing Property Not Found

16. FernUniversität Results
 Content
 Introduce provocative topics/material
 Pose challenging questions
 Refer to previous questions
 Assignments with a practical orientation for the students
 Structure
 Small group discussion for low participating students.
 Use the LMS environment to present discussion as a ‘normal’ part of online learning.
 Modeling
 Posting by instructor to begin the course.
 Explain interactions and advantages of participation.
 Explain relevance.
 Use peer tutoring and mentoring to model online discourse.

17. Conclusion
 Implications for research
 ‘Universal’ assessment best practices should be tested for validity (and
appropriateness) across different cultures.
 Develop and test alternate models of assessment.
 Implications for practice
 Develop culturally sensitive monitoring policies and incorporate into
faculty training.
 Instructors should consider opening a dialogue with students about the
topic.

18. References
Anderson, T., Rourke, L., Garrison, D. R., & Archer, W. (2001). Assessing teaching presence in a computer
conferencing context. Journal of Asynchronous Learning Networks, 5(2) Retrieved December 10, 2004, from
http://www.aln.org/publications/jaln/v5n2/
Aiello, J. R., & Douthitt, E. A. (2001). Social Facilitation from Triplett to Electronic Performance Monitoring. Group
Dynamics: Theory, Research, and Practice, 5(3), 163-180.
Bransford, J.D., A. L. Brown, and R.R. Cocking, eds. (2000). How people learn: brain, mind, experience, and
school. Washington, D.C.: National Academy Press, 206-230.
Michaud, Olivier (2010). The effects of assessment: A reflection from within the economic worldview in education.
Analytic Teaching and Philosophical Praxis. 30 (1): 20-30.
Strauss, A., & Corbin, J. (1998). Basics of Qualitative Research: Techniques and Procedures for Developing
Grounded Theory (2nd ed.). Thousand Oaks, CA: Sage Publications, Inc.
Turgeon, Wendy C. (2010). Confessions of a department chair of assessment. Analytic Teaching and Philosophical
Praxis. 30 (1): 12-19.
University of Western England (2009). Blackboard Staff Guide: Tracking and Statistics. University of Western
England, Bristol
Weiner, Wendy F. (2009). Establishing a culture of assessment: Fifteen elements of assessment success – how many
does your campus have? Academe Online (July-August).
http://www.aaup.org/AAUP/pubsres/academe/2009/JA/Feat/wein.htm