Recommandé
Recommandé
Contenu connexe
Tendances
Tendances (20)
Similaire à Jump Into the Game: How Libraries Can Adapt to Institutional Assessment Plans
Similaire à Jump Into the Game: How Libraries Can Adapt to Institutional Assessment Plans (20)
Dernier
Dernier (20)
Jump Into the Game: How Libraries Can Adapt to Institutional Assessment Plans
- 1. HOW LIBRARIES CAN ADAPT TO INSTITUTIONAL ASSESSMENT PLANS JUMP INTO THE GAME:
- 2. AGENDA Overview of Assessment Institutional Goals KMERI Mapping Goals Across the Institution Examples of in-class Assessment Gateway Course Collaboration Moving Forward
- 4. ASSESSMENT: THEN AND NOW Image Source: Westminster College http://www.westminster.edu/acad/oaac/cycle.cfm
- 5. THE REGIONAL ACCREDITATION BODIES AND THEIR FOCI • Resources, planning, governance, and finances (HLC, CIHE, MSCHE, SACS COC) • Teaching and Learning (HLC, CIHE, MSCHE, SACS COC) a) staffing and curricular structure b) quality teaching and learning through appropriate evaluation and improvement c) integrity in study and research
- 6. a) Knowledge of the field/disciplinary competence b) Innovation/originality c) Critical thinking/communicatio n skills d) Responsibility/ cultural competence Otterbein University formulation: Knowledgeable Multi-literate Engaged Responsible Inquisitive WHAT ARE MY INSTITUTION’S LEARNING GOALS?
- 7. KMERI Knowledgeable • Otterbein undergraduate students will acquire a deep knowledge of, and appreciation for, their chosen major(s) as well as a solid grounding in a broad range of disciplines in the liberal arts and sciences. Multi-Literate • Otterbein undergraduate students will develop and demonstrate a range of intellectual and practical skills, including written and oral communication skills; research skills; quantitative, technological, visual, and information literacies. Engaged • Otterbein undergraduate students will become intellectually, aesthetically, and civically engaged. As intellectually engaged people, they will integrate, synthesize, critically reflect upon, and evaluate what they know. As aesthetically engaged people, they will value and practice the art of imagination and creative expression. Responsible • Otterbein undergraduate students will make choices that promote their own well-being and that of others. Encouraged to grapple with diverse value systems, they will move toward ethical commitments that reflect a sense of agency, honestly, and fairness. Inquisitive • Otterbein undergraduates will know how to lean. As confident life-long learners, they will be curious and eager to discover more about themselves and the natural, cultural, and social worlds.
- 8. INSTITUTIONAL ALIGNMENT OF DEPARTMENTS AND PROGRAMS Departments and programs K M ER I
- 9. UNDERGRADUATE STUDENT LEARNING OUTCOMES (BASED ON ACRL INFORMATION LITERACY STANDARDS, BLOOM’S TAXONOMY, AND INTERNAL DOCUMENTS) Senior Junior Sophomore Freshman
- 11. ALIGNING TO THE UNIVERSITY STUDENT LEARNING GOALS (KMERI) Senior Junior Sophomore Freshman
- 12. INSTITUTIONAL ALIGNMENT OF DEPARTMENTS AND PROGRAMS Departments and programs K M ER I
- 13. TAKING OVER THE “M” IN KMERI Departments and programs K ER I
- 14. INFLUENTIAL DOCUMENTS DQP (Degree qualifications profile) from Lumina Foundation (2011) For the Lumina Foundation, the “Use of information resources” means, progressively: At Associate level: Identifies, categorizes, evaluates and cites multiple information resources necessary to engage in projects, papers or performance in his or her program. At Bachelor’s level: 1. Incorporates multiple information resources presented in different media and/or different languages, in projects, papers or performances, with citations in forms appropriate to those resources, and evaluates the reliability and comparative worth of competing information resources. 2. Explicates the ideal characteristics of current information resources for the execution of projects, papers or performances; accesses those resources with appropriate delimiting terms and syntax; and describes the strategies by which he/she identified and searched for those resources. At Master’s level: Provides adequate evidence (through papers, projects, notebooks, computer files or catalogues) of contributing to, expanding, assessing and/or refining either a broadly recognized information resource or an information base within his or her field of study
- 15. Hart Research Associates: “It takes more than a major” (2013)
- 16. “Students develop and demonstrate appropriate oral communication, reading, and writing skills” “Exhibits a mastery of statistical analysis and a facility with technological tools” “Students will synthesize information from a variety of sources to inform their course projects.” “Gather, evaluate, select, and integrate research sources in order to create relevant, dynamic, and effective writing projects and oral presentations.” “Students will be able to locate, analyze, synthesize, and critically evaluate information from a variety of sources, including print and electronic media, film and video, and internet technologies.” Although all departments have to prove that their students meet the “critical thinking/communication skills” goal, the specific ways in which this is done is different. Hence, each department writes outcomes that are meaningful to its own methods or approaches. DEPARTMENT-SPECIFIC OUTCOMES
- 17. OTTERBEIN’S GOALS DEVELOPMENT OF KMERI AND OUTCOME MAPPING
- 18. MAPPING AND DEVELOPING K M Library IL Goals E R I
- 19. REDESIGN OF LIBRARY GOALS Program 1: Information Literacy Goal or Outcome Alignment with KMERI Curriculum Year Year to be Assessed: 2012- 2013 The information literate student determines the nature and extent of the information needed. ● Recognize that information is available from a variety of resources M Freshman I X ● Formulate a research plan. K, M Freshman II ● Organize information effectively M Sophomore The information literate student accesses needed information effectively and efficiently. ● Recognize the elements of a bibliographic citation and be able to distinguish between a book citation and a journal citation. M Freshman I X ● Differentiate between Author, Title, and Keyword searches. M Freshman I X ● Search Otterbein, OPAL, and OhioLink catalogs. M Freshman I X ● Be aware that Reference Librarians are available to answer questions in person, through email, or via Chat Reference. M Freshman I X
- 20. GET STARTED INCORPORATE YOURSELF INTO THE ASSESSMENT PLAYING FIELD
- 21. TIP ONE Find the good in institutional assessment.
- 22. TIP TWO Get involved at the top.
- 23. TIP THREE Help programs/faculty design and develop assessment exercises.
- 24. TIP FOUR Sell what you can do for them.
- 25. MAKE YOUR FIRST PLAY TRY DIFFERENT TOOLS IN THE CLASSROOM
- 27. Poll Everywhere
- 29. FYS Library Hunt Go to the library website library.otterbein.edu. Find the answers to the following questions: 1. How long can a student check out a book? How long can a student check out a DVD? Copy http://www.otterbein.edu/resources/library/libpages/cirpolicy.htm 2. Find the book Communicative Understandings of Women's Leadership Development in the OPAL catalog. a. Write down the call number of the book. HQ1233.C522012 b. Click on “Suggested Terms.” Write down two of the subjects listed. Leadership in women Women school administrators 3. Find the “Ask a Librarian” button. Write down the five different ways you can reach a librarian. Reference Phone 614 823-1984 Circulation Phone 614 823-1215 ext. 4 E-mail Library@otterbein.edu Text your question to a Librarian at: 614-259-7404 Schedule a research consultation with one of the librarians
- 30. FYS Outcome Parameters Student Actions Librarian Actions Assessment Search Otterbein, OPAL, and OhioLINK catalogs • Understand which catalog represents which schools • Know how to borrow materials using each catalog • Use "Check Patron Record" to renew items and check fines • Students receive an explanation of the two catalogs used locally (OPAL and OhioLink) • Students practice using the catalogs and requesting items • Students are shown where to renew items and ask for help/practice logging in • I explained why we have two catalogs and what the main difference between them are • I showed a video about scholarly articles and had the students practice searching with the peer-review limiter • Students worked on in-class assignment brainstorming keywords for searching • Reporting: students reported their refined research question to the classroom Recognize the elements of a bibliographic citation and be able to distinguish between a book citation and a journal citation. • Students compare examples of citation for (at least) articles, books, and websites (Tutorial, other tutorials) • I explained why there are differences and showed examples of book, article and website citations in APA • Students participated in a poll answering questions about bibliographic citations and the differences between types of journals • Students showed moderate ability in answering post- class questions about what they learned on libguide- about 74% answered correctly
- 31. 0 10 20 30 40 50 60 70 80 90 Recognize the elements of a bibliographic citation Search various resources Differentiate different search terms Know how to contact librarian Student Response
- 32. WARMING UP COLLABORATION WITH A NURSING COURSE
- 33. LIBRARY EXERCISE • Their PICOT question, part of a literature synthesis • Three strong research articles supporting their query • Detailed analysis chart about researching
- 34. LIBRARY EXERCISE
- 35. LIBRARY EXERCISE
- 36. Library research grading rubric: Required Outcome Insufficient 0-40% Basic 40-70% Proficient 70-90% Advanced 90-100% Total Student uses the appropriate databases The wrong database or information source was used (Google, for example) Only one full-text database or index was used Several full-text databases and indexes have been used All or most of the appropriate full-text databases and indexes have been used Student employs the right searching techniques Imprecise keywords or Boolean searches are performed Basic keyword searches are performed and few or no Boolean searches Boolean searches are performed with considerable efficiency, and several keywords have been used Boolean searches are performed proficiently, and a number of appropriate keywords and subject headings have been used Student is able to recognize the article type The article is mistakenly identified and there is little to no understanding of the types of research The article is placed into a general category, such as original research or literature synthesis The article is correctly identified according to the types of research it contains: literature review, clinical study, evidence-based argument, analysis, etc. The article is identified by using more than one descriptor. Student evaluates and selects articles based on reliable criteria No or unreliable criteria are used in evaluating and selecting the article Only one reliable criterion is used in evaluating and selecting the article Some reliable criteria are used in evaluating and selecting the article, while others are ignored All reliable criteria are used in evaluating and selecting the article, including authority, currency, bias (if it applies), and abstract and keyword identification. Student uses the appropriate APA citation style A pattern of mistakes is identified throughout the citation The fundamental APA elements are present, but there are occasional mistakes The citation corresponds fully or almost fully to the current APA Manual of Style recommendations Total Knowledgeable Multi-literate Engaged Responsible Inquisitive
- 37. CHALLENGES • Class size: • FIVE Faculty team-taught and graded – ONE librarian….. • 60 students to over 100 students each time it was taught • Time needed to FOCUS You helped me understand I was being too specific when typing in my words for my search. – student in the 5220 course
- 38. LESSONS Don’t use “library speak” it only confuses them (boolean, Subject Headings, etc…) Stronger relationships with faculty/students Immediately ask for feedback from your faculty Record and share the raw data with the faculty and keep it for the library
- 39. MOVING FORWARD Prove and improve our services to demonstrate our value to the University Continue to develop gateway courses for departments and programs Know the regional accreditation body and their criteria for accreditation Be aware of what models your university uses in formulating its student learning goals Be privy of the conversations taking place in the institutional effectiveness committees
- 40. QUESTIONS?
Notes de l'éditeur
- Otterbein is a private liberal arts college located in central Ohio, about 20 miles north of Columbus, the capital. We have approximately 3,000 students and offer a choice of 73 majors and 44 minors, as well as individualized courses of study. We also offer five graduate programs in business administration, education and nursing, including our doctor of nursing practice degree. Our student to faculty ratio at Otterbein is 11:1 and there is a heavy focus on interdisciplinary study and general education.
- Although the library can play a significant role in the institutional assessment process by proving its importance in institutional planning (as an investment) or as a valuable resource in a physical sense (location of collections, learning spaces, etc), our emphasis is on the role of the library as a player in the formulation of learning goals and the assessment of student learning outcomes.What individual goals would best reflect b and c?The difficult language of goals: what are my institution’s learning goals?Do you know them? Can you articulate them in your own words? What are the main areas addressed?Looking across a wide array of institutions, 4 main areas come to mind:
- Which one of these do you feel is more compatible with the goals of information literacy?
- UPDATE KMERI
- SHOW IL LIBGUIDE
- Looking at the language of two basic documents from Hart Research Associates and the DQP (Degree qualifications profile) from Lumina Foundation, you should be able to single out elements of critical thinking that are highly compatible with the language of information literacy.
- According to Lumina’s DQP, one of the 5 defining “intellectual skills” that define knowledge both in strict, disciplinary, and broad, interdisciplinary sense is “Use of information resources”, right next to “analytic inquiry,” “quantitative fluency”, “communication fluency”, and “engaging different perspectives” (or cultural awareness).
- Most departments formulate student learning outcomes that address the goal of achieving advanced critical thinking/communication skills but do not include elements of information literacy among them. They need to be reminded by way of these two major documents that part and parcel of this goal are also information literacy learning outcomes.Examples of formulations ranging from traditional forms of understanding literacy to more advanced ones (that include research, technological, and information literacy) and from vaguer to more specific formulations of learning outcomes:The ideal is to move from the first ones to the last ones. We have been able to help with these formulation so that departments know what they expect their students to demonstrate and we, librarians, can provide evidence of the degree to which the students meet these outcomes that fall under the goal of “multi-literacy”
- ThanksRares, so now I am going to talk about how we used KMERI goals to establish our own library outcomes and in the process, also mapped back to university-wide outcomes. This also helped us to establish the library as a major player in the institutional assessment process.How many of you already have institutional goals incorporated into your library’s information literacy outcomes?
- So although KMERI came about around 2012, the library took its sort of first introductory look into combining our goals last spring. Rares, an intern, and I worked on re-working the library goals to easier map across to KMERI. This helped us in various ways, one being that we could now easily provide and promote institutional goal outcomes to our faculty and two, that it prepared us for institutional assessment.Each department on campus is expected to assess two of their primary goals every two years and have them mapped to KMERI. So Doing this not only put us ahead of the game as we prepare to assess two of our goals, but also set the library as a major player in the assessment process. During this process of developing library goals mapped to KMERI, we determined that many of the library goals’ aligned with KMERI’s multiliterate goal. We were also able to map to knowledgeable, responsible, and inquisitiveThe multiliterate goal was an easy match to our library’s initiatives. The goal states importance of evidence, databases, and appropriate research methods. It also includes the formulation of ideas, theories, and concepts. All major facets of the information literacy instruction. Many of the other KMERI SLO’s are also of course relevant to the library mission of course, however, we felt it would be beneficial to focus on one aspect of the SLOs put forth by the university, to clarify and make easier what our possible roles can be in the greater scheme of institutional assessment. So what does “multiliterate” mean? Otterbein specific goal, and like Rares mentioned, that critical thinking and information skills portion of many assessment goals. We determined multiliterate to mean well-versed in multiple information languages, including information literacy.
- So in the process examining our goals, we found common goals between the university and the library.Library goals were broken by academic year, so there were specific expectations for each year a student was at Otterbein. For ex. freshmen were expected to understand that information is avail. from a wide variety of sources, while juniors are expected to be able to critically evaluate and synthesize main ideas.Here is an example of some of the goals and when they are to be examined and assessed. The library would best serve its purpose of providing information literacy to students through instruction and one-on-one reference consultations. This process has helped us to identify what strengths we can provide and how to communicate that to faculty and administrators.
- So that explains a bit of the process in developing our library goals and focusing on the institutional goal of multi-literate. Now I am going to talk about some ways to reach out, get involved in assessment, and collaborate with faculty and campus programs.
- Although we all know that assessment can be a thorn in our side, much of that frustration comes from our un-involvement and confusion about moving forward. So a major step forward for many libraries regarding assessment is to find the good and search for the positive. Seek out ways that assessment can help you, your faculty, and your institution overall. When talking with other people around campus, talk up the good points and always refer to how the library can help.It is certainly a process, and we are at the level now where we are developing these steps moving forward. That brings us to step two…
- Get involved at the top. Librarians sometimes need to assert themselves at higher levels of involvement to prove our value. This is of course something that is talked much about in the profession, but what value do we offer? Getting involved at the administrative/executive level, especially with assessment, can aid in our playing a role in important areas at our institutions and show our worth. This would also allow us to trickle down effects to positively impact our library and students. A good example of this exists at Otterbein. In addition to traditional academic departments, the librarians also serve as liaisons to student groups, programs, and general education areas. One of these is our Senior Year Experience program which has worked with the librarians in the past to gather data using information literacy goals to assess their program goals. This gets us highly involved and inserts us at the top-level of the process.
- So once we are established yourself as a player in the assessment process, it is easier to provide assistance in regards to developing assignments for one-shot instruction or semester-long embedment in a course (which Amy will talk about). Use these to create new generic assignments and rubrics to share with your faculty as they move to assess their students. I am in the process now of working with our Honors program to develop SLOs for the students’ junior year. That will make it easy to create assignments that the faculty can then take on and develop themselves. Once they understand the mapping process and the role the library can play in that assessment, they will welcome the assistance.
- I mentioned value earlier, Collaborative partnerships at the liaison, program, and department level are very important Reach out to faculty to see if they need help attaining goals, developing rubrics, or creating assignments. Once you gather the data you can populate it in a usable way and really show how important assessment data can be. Use librarian’s importance in outcome mapping as a springboard for gaining attention and value at the faculty level.There is a lot of data that could become available through the process of this cross-collaboration of assessment. We are in the process now of taking some information from our Senior Year Experience program and returning assessment information back to them for their yearly assessment goals. This can be done across various departments or programs on campus and we hope to expand it. Possible graph- SYE’s KMERI outline
- So that covers a bit of the collaboration process between librarians, faculty, and the institution as a whole. There are also ways that we can assess in the classroom and map back to our institutional goals, and this can even be done retroactively.
- My first example is a libguide poll, how many have you have used this or poll everywhere in class?I am sure many of us have used various types of in-class assessment and this can be carried back towards any type of test. Here was have several question asked at the end of class. Students choose their answers and the the librarian collects the information at the end of the session from the libguide. The nice thing about this tool is that data can be easily recorded based on the response rate and the percent correct. The downside is that it has to be recorded immediately because libguide polls do reset once you clear the results. So in this example I would be able to report back to the faculty on some aspects of the multi-literate and knowledgeable goals and let them know how well the students did.Minute papers and in-class assignments can also be mapped back to the multi-literate goal or whatever goal your library maps to. It would just take
- Here is the poll everywhere example where students examine some citations and poll response to basic questions. Poll everywhere is great because it develops the data for you.
- Here you have an exact percentage you can incorporate into your assessment data. These examples are also great because they can double up as assessment for your library’s information literacy and goals AND your institutions and faculty.
- My final example in an in-class assignment. I have my students do these in the First year seminar in groups through Google drive. Here you can see they are answering very basic library centered questions using a “hunt” type style of assignment. You can easily go back and map the responses to your institutional goals and record the data.
- Finally, this is a portion of our FYS rubric. The library has rubrics similar to these for FYS and our Integrative studies programs, as well as freshman, sophomore, and junior years. This is a little snippet of one of our FYS rubrics, but it shows how you can retroactively map in-class assessments to campus-wide goals. Here in this case, using an in-class poll and quiz, the SLOs mapped to M and K.
- This graph represents a breakdown of some of the aspects of M and how we can represent that data to the faculty member. I will now pass things on to Amy, who will talk more indepth about an embedded assignment through a nursing course and how it was mapped back to KMERI.
- Background – 2010-2012 We began this formal collaboration with the nursing faculty. We were able to meet with them and design a library exercise to be included in their Graduate level class. The class was team taught 4 times with this library exercise. The class is an Evidence-based Nursing course.
- The students in this course had to complete a LITERATURE SYNTHESISPICOT= P= Patient population I= Intervention of interest C= comparison of interest O= Outcome of interest T= Timeframe EXAMPLE: A student compared different smoking cessation plans in adults to see which had better results. Another example: Adult patients scheduled for surgery, does the use of pre-operative patient education reduce post-operative pain levels compared to patients who have not received pre-operative patient education…. The Questions were complicated and challenging to search. The students needed to learn to access as many of our nursing/medical databases as they could and use specific complicated search strategies.
- Rares designed this library exercise - First they needed to choose a topic of interest and find three strong research articles to support it. I did a library orientation class demonstrating good research strategies. I showed them the nursing databases to use and demonstrated how to limit their searches. I was also embedded in their Blackboard class to answer questions and I told the students to contact me and I would be happy to sit with them and help them find their articles. I had a large number of students contact me about searching for articles.
- As they completed this assignment I received many questions/comments : The students complained to me while working on this part of their assignment: I was told this was busy work, they were frustrated that they had to take the time to fill in the boxes! They got confused with the library speak - I spent a lot of time explaining Boolean Operators and “article Type”, and “subject headings”….They complained that this was one more thing they had to get done after 8 large projects in this course. The majority of students in this course worked full time as nurses and some worked 3rd shift in the ER and would meet with me first thing in the morning after they got off from their shifts.
- I used this rubric to grade every exercise. Jessica has color coded or mapped how this rubric aligns with the university KMERI learning goals. We have done this retroactively as this course was completed BEFORE we aligned the KMERI goals more formally. I graded these and as I recall a large number of them got Proficient to Advanced on their assignments.Knowledgeable = students learned how to perform Boolean searching with keywords and possibly distinguish certain subject headings Multiliterate = Students were able to tell us that they found an authoritative quantitative article or a clinical studyResponsible = Giving a correct APA citation for each article
- CHAOS - I graded hundreds of these exercises and it took A LOT of time. Everyone is so busy I received no real guidance on what I was doing so I sort of figured it out as I went along. ( I AM A CATALOGER!!! ) Each faculty member who team-taught this course had a set of students that they were grading, and I had to keep track of which faculty member got each student’s graded exercise. They kept asking me over and over to re-send certain student’s grades and it was a real messy system.FOCUSING- I wanted to do a good job grading and it took time away from all my other duties: cataloging, preparing and teaching, and reference desk. Answering the students questions and getting the assignments graded were my top priorities when this class was in session. QUOTE – From a student in this class, this feedback helped me see that my libguide and my explanations were working for some!
- Students gave me instant feedback – confused with library speak – Boolean? Keywords? Subject Headings? LibGUIDE -- As the students would ask for help and give me constructive criticism I immediately edited the libguide to further explainPositive outcome of this - I developed stronger relationships with faculty and students Learning moments: I did not even think about asking the faculty how they felt about the assignment…and I wish I hadI learned a great deal from this experience!!! DATA - While I graded hundreds of these assignments and promptly sent them back to faculty for their grades, Why didn’t I think about keeping the grades for the library? Scattered – too many responsibilities and I didn’t think about it at the time, reference, cataloging, prepping for classes, teaching classes….. Hindsite = If I had been told that this information was important I certainly would have attempted to keep the raw data. This Course Now: Whats happening? When the course transitioned from Quarters to Semesters they redesigned it and did not include the library exercise in the syllabus. This is team-taught and they rewrote the literature synthesis assignment. I am still involved in the face to face orientation meeting at the start of the course, I keep their libguide up to date and I am embedded into their Blackboard course with a research thread. This gateway course charted a pathway to get into other courses in the future. In both the graduate and undergraduate levels.
- Six steps I got from a recent workshop on Assessment I went to Define what to assessSelect who to assess ( users? non users? faculty? )Design the process for data collectionCollect the dataAnalyze / Interpret the dataUSE the data – Tell a story They will listen to personal stories rather than statistics and percentages.