Understanding the link between eResource accesses and student success at a distance-learning digital university - Richard Nurse
•Télécharger en tant que PPTX, PDF•
1 j'aime•144 vues
Slides from sherif event on eResources data: what does it really tell you? - February 2018
Recommandé
Recommandé
Contenu connexe
Tendances
Tendances (18)
Similaire à Understanding the link between eResource accesses and student success at a distance-learning digital university - Richard Nurse
Similaire à Understanding the link between eResource accesses and student success at a distance-learning digital university - Richard Nurse (20)
Plus de sherif user group
Plus de sherif user group (20)
Dernier
Dernier (20)
Understanding the link between eResource accesses and student success at a distance-learning digital university - Richard Nurse
- 1. eResource accesses and student success at a distance-learning digital university Sherif: eResources data: what does it really tell you? Richard Nurse Open University Library 27 February 2018
- 2. 2 CONTENTS CONTEXT WHAT VALUE eResource accesses and student success at a distance-learning digital university
- 3. Some context Libraries and student success
- 4. 4 THE LIBRARY AND STUDENT SUCCESS Libraries around the world have found a relationship between library use and student success “… students who utilize academic libraries within their first year have higher cumulative grade point averages and retention, on average, than their peers who did not use library services.” University of Minnesota 2013 “There is a statistically significant relationship between library resource use and level of degree result” Library Impact Data Project, University of Huddersfield 2013
- 5. 5 The OU STUDY (LIBRARY) EXPERIENCE Part-time students, studying module-by- module Average age 28, many without previous qualifications or recent study experience Studying at a distance and probably never go to the OU campus OU students don’t use a physical OU campus library or borrow print books from the OU library OU students using library eResources Is there still the same relationship between library use and student success when the student experience of the library is a digital one?
- 6. 6 INSTITUTIONAL DRIVERS Learning Analytics is a key institutional priority but… Library data not included in institutional data stores consequently… Library data not used in institutional learning analytics activity Lack of data integration can lead to library activity not being seen as core student activity
- 7. 7 ETHICS PRIVACY BUT SHOULD WE? DATA PROTECTION
- 8. What did we do and what did we find out?
- 9. 9 OU ERESOURCE DATA EZproxy Daily log files Starting Point URL (SPU) used mainly Date/timestamp, user ID, resource/platform Processed into database and added module identifier, user category OpenAthens Monthly aggregated counts User ID, resource/platform, count Processed into database and added module identifier, user category Student results data Extracted from data warehouse Matched on user ID and module ) Preparing data is very time- consuming
- 10. 10 Analysis suggests students who pass are accessing twice as many resources as students who fail. WHAT DID WE FIND?
- 11. 11 Analysis suggests students who pass are accessing twice as many resources as students who fail. WHAT DID WE FIND?
- 12. 12 Analysis suggests similar patterns to that found in other studies Correlation study Compared number of library eResource accesses and the continuous assessment score for 199 modules starting in autumn 2015 79 had a medium positive spearman rank correlation 62 had a small positive correlation and 1 had a large correlation eResource access bands Grouped students into bands based on numbers of eResources accessed Chi-Square and Anova tests suggested a significant association between resource access bands and result. STATISTICAL ANALYSIS
- 13. 13 WHAT ELSE? eResource accesses Student results Demographic data Completion status Helpdesk contacts • 70% of students completing accessed eResources • 60% of students who did not complete had not accessed eResources • Students contacting the library helpdesk access more eResources • and a smaller percentage withdraw or fail • Are there differences between groups? • Are there barriers you need to address?
- 14. How has this been useful? What have we done with this?
- 15. 15 Library data used heavily in new annual library reports to academic schools Data broken down at a module level High-level analyses used as an advocacy tool with key stakeholders and decision makers to raise awareness of potential of library data Engagement with Strategy Office to gain support for including library data into institutional data systems HOW HAS THIS BEEN USEFUL?
- 16. 16 Similar pattern between library use and success at the OU to that found at ‘bricks-and- mortar’ universities A successful student is likely to be engaging with the library Not using the library might be a good indicator of a student who is struggling and could be a trigger for providing help Students studying with a digital library seem to demonstrate a similar relationship between student success and library use to that seen in campus-based institutions Suggesting that as more libraries move to a digital experience – they may see the same relationship maintained eResource data is valuable Analysing your own institutions data can help to target activities by identifying courses where there is less engagement KEY MESSAGES
- 17. References and contact References Stone, G. and Ramsden, B. (2013) ‘Library Impact Data Project: Looking for the Link between Library Usage and Student Attainment’, College & Research Libraries, vol. 74, no. 6, pp. 546–559 [Online]. Available at DOI: 10.5860/crl12-406 Soria, K. M., Fransen, J. and Nackerud, S. (2013) ‘Stacks, Serials, Search Engines, and Students“ Success: First-Year Undergraduate Students” Library Use, Academic Achievement, and Retention’, The Journal of Academic Librarianship, vol. 40, no. 1, pp. 84–91 [Online]. Available at DOI: 10.1016/j.acalib.2013.12.002 Nurse, R., Baker, K. and Gambles, A. (2018) "Library resources, student success and the distance-learning university", Information and Learning Science [Online]. Available at DOI: 10.1108/ILS-03-2017-0022 (pre-print) Contact richard.nurse@open.ac.uk @richardn2009 Images OU copyright
- 18. THANK YOU
Notes de l'éditeur
- I want to cover three areas around our work with library data on eResource accesses and student success I’ll provide a bit of context to the work we’ve been doing, looking at some of the motivation Then talk about what we did and what we found And then cover how we’ve been using what we found
- Let’s start with a bit of background, what’s been done already, look at some institutional drivers and think about some of the wider considerations
- I’m sure everyone will be familiar with the research that has been taking place around the world that is showing a fairly consistent pattern that there’s a relationship between using the library and doing well. There’s just a couple of examples here, but there are many more And you could say, well, that’s job done, here’s the evidence It’s helpful to see that pool of evidence and it certainly helps when making the case for the impact that the library has on the student experience But… There are some circumstances where other libraries data just isn’t enough
- If your student experience isn’t based around a physical presence and your students don’t fit the typical picture of an undergraduate student When student engagement with the library is a heavily based around their use of eResources not around a physical library building Is there still the same sort of relationship between library use and student success?
- And where there are institutional reasons As an institution there is a big push around Learning Analytics But our library data on eResource access is in separate library systems such as Ezproxy log files and not in institutional data systems And that leads to a cycle - because the library data isn’t easily available to data analysts it doesn’t get used – because it doesn’t get analysed it isn’t seen as important – if it isn’t seen as important it’s hard to convince people that it should be added to the institutional data systems – so the cycle continues Libraries are competing for resource against others in the university who have their data stories well-established and their data in core institutional data systems
- Before we look at the detail of what we did and found, I just wanted to touch on some things we need to consider in relation to libraries and the use of data The importance of complying with data protection requirements and it’s great that there is a GDPR session later because you will need to think about what data you collect and how you collect it and what data you store or retain But we also need to acknowledge that there are important ethical and privacy considerations about this area of work. We’ve been fortunate at the OU to have a well-established policy around the ethical use of data for Learning Analytics But you need to think carefully about what type of data you use and what you do with it – so for example you might count the number of eResource accesses but not look at the detail of which resources individual students accessed - what you share, who you share it with, how and when you “anonymise” the data And you need to be aware there are views within the Library sector that the type of ‘big data’ activities that are becoming increasingly common in technology circles, conflict with the ethics of providing a library service, particularly in relation to user privacy These are important things to think about
- Lets look at the detail of what we did and found out.
- We have a couple of sources of eResource data – the majority of our accesses go through ezproxy with a small number (for publishers who don’t support IP authentication) using a federated access system, OpenAthens Both our on-campus access and off-campus access is put through Ezproxy Some processing of the data was already being undertaken to produce statistics – which faculties were accessing which resource for example – so there was a SQL database that could be used to extract the library data from We ran a few pilots with small collections of data to test our approach and settled on an approach that produced an aggregated count of eResource accesses by combining data from both ezproxy and OpenAthens then matched the student ID against course results data taken from the institutional data warehouse There was a lot of time spent testing, cleaning and preparing data
- When we processed the data for all the courses starting in autumn 2015 we found that across each of the levels of undergraduate study students who were failing were accessing about half the number of library resources accessed by those who passed. Students gaining a distinction accessed still more library resources. We also were able to clearly show something we had expected but hadn’t been able to show before – that students studying at higher levels access more library resources
- When we repeated the analysis for the modules that started in autumn 2016 we found a very similar pattern and some growth in the number of eResources accessed per student. In each case these are around 90,000 students.
- We followed up the initial work with some statistical analysis, including correlation studies. These found similar correlations to that seen in other library studies Some more detailed statistical analysis was undertaken, splitting resource accesses into bands for example. These tests supported the view that there was a medium sized effect between groups. There is a forthcoming paper due for publication in Information and Learning Science in 2018 – there’s a reference at the end of the slides
- You can do more that just look at the relationship between eResource accesses and student success – we carried out a few other pieces of analysis If you can get other data you can join with your eResource data you can investigate Retention – is there a different pattern of completion between students who do or don’t access eResources Do students contacting the library Helpdesk access more eResources? Do you have demographic data that you can analyse to identify any big differences between groups – that might identify barriers you need to deal with – but you need to be aware of sensitivity of particular types of data
- What have we been able to do with what we’ve found out?
- Carrying out the analysis has been really useful for us – particularly as an advocacy tool The high-level analyses have been useful in engaging senior stakeholders The detailed course by course analyses have been helpful to have detailed conversations in the academic schools and to help to target where library resource is needed and are included in our new school reports The data has also led to library data being on the agenda in some high-level strategic reports, that now contain recommendations to make use of library data, and that work is now included in the Strategy Office’s plans Unless we had carried out the analysis we wouldn’t have got on the agenda Once we can get library data embedded into those corporate data systems then it opens up detailed statistical analysis by the university’s data experts, and the potential, for library data to be part of the predictive models that the university uses and for there to be a much more joined-up approach with library data and other institutional data sets.
- What have we learnt? Part-time, distance learning students engaging with the library through eResources show pretty much the same relationship between library use and success as seen in campus-based institutions That this is the case for what is essentially a digital library experience - suggests that as libraries move to be much more of a digital service then the link between library and student success may be maintained That a successful student is one who is engaging with library resources – that a student who isn’t using the library might be one that needs some help – they seem less likely to be successful, more likely to withdraw And that finally, eResource data can give you much more than statistics on use – vital that this is – but can help to identify impact and where you might want to target scarce resources