1. A Standardized Approach to Assessment – Phase 2 New Improvements Based on Student , Consultant & Library Faculty Feedback Shannon M. Staley Education & Web Services Librarian

2. In 2008 / 2009, We Began By: Developing and testing an assessment tool that could: Facilitate the development of pre and post achievement tests Link survey questions to learning outcomes and ACRL standards Deliver real time reports and statistical analysis

3. In 2010, We Asked the Question:How can we improve the tool and the assessment process? Usability testing sessions with librarians who have not used the tool More cognitive interviews with students to improve question development Improved statistical analysis Focus group with librarians who have used the tool

4. 1. Usability Testing

5. Usability Sessions: Changes you asked for Comment: I didn’t understand where I was when I entered the tool.

6. Usability Sessions: Changes you asked for Remedy: A clearly defined menu as the starting screen

7. Usability Sessions: Changes you asked for Comment: Make it easier to toggle from my personal account to all questions and possibly select and copy some for my account.

9. It’s easier to add and remove questions.

10. The “My Questions” view and “All Questions” view are different colors for more distinction.

12. Usability Sessions: Changes you asked for Remedy: A message indicates the survey was created and the URL to the survey is provided so that it can be posted anywhere.

13. 2. More Cognitive Interviews

15. During the cognitive interview process, the interviewer uses techniques to elicit feedback on survey questions.

22. Asteriks by values in the matrix show statistically significant relationships between background factors.

25. Do numbers offer the big picture of teaching effectiveness?

26. Need more research to develop a cut score or acceptable percentage of correct responses.

27. Need to gather support from other SJSU librarians and get campus endorsement

29. Complete data collected from usability testing sessions