OpenAthens Conference 2019: “Can you do something with this data?”
•Télécharger en tant que PPTX, PDF•
1 j'aime•245 vues
Samantha Heeson, electronic and data services librarian, Leeds Beckett University. Integrated data is key to effective data analysis, but disparate data is often the reality. University library analytics is one area where this is changing. It is being driven by a desire to better understand library users, whether they be students or academics, and thus support their successful learning, researching and teaching experience, as well as the need to manage budgets and collections. This session is about enabling broader and deeper analysis of OpenAthens e-resource usage data and looks at a pilot project at Leeds Beckett University Library, which assessed the feasibility of integrating OpenAthens e-resource usage data, collected at the user ID level, with Student Record data.
Recommandé
Recommandé
Contenu connexe
Plus de OpenAthens
Plus de OpenAthens (20)
Dernier
Dernier (20)
OpenAthens Conference 2019: “Can you do something with this data?”
- 1. “Can you do something with this data?” Transforming OpenAthens e-resource usage data into student engagement insights Samantha Heeson Electronic and Data Services Librarian Libraries and Learning Innovation
- 2. Background • OpenAthens • User experience • Vendor input • Potential analysis • Active Directory Federation Services • User IDs • Approved solution • E-journals & databases • Electronic & Data Services Librarian • “Can you do something with this data?”
- 3. OpenAthens Data • Monthly CSV reports • Resource usage • E-resources • User IDs • Accesses • Basic overview • No user groups • 30,000+ students • 750,000 pieces of data • Potential…
- 4. Proof of Concept • Three month snapshot • OpenAthens • Student Record • Physical stock • Reading lists • Re-structure & re-format • User IDs • In-depth analysis • Relevant usage • Potential…
- 5. Prototype • Database • Charts • Narrative • Five month snapshot • OpenAthens • Student Record • Library impact • Focused analysis • Early insight • Appetite…
- 6. Standard Reports • Institutional & school trends • Academic year • Stable system • Static dashboard • Highlights & anomalies • Discussions with schools • Enhance academic support • Purchasing decisions • Custom reports…
- 7. Custom Reports • School by level and year • Student engagement & student satisfaction correlation • Next steps…
- 8. Longitudinal Analysis • New OpenAthens interface • Timeouts • Duplication • Structure • Student engagement vs. e-resource usage • Monthly, hourly & daily reports • “Yes, I can do something with this data!” • Student Record • Visualisations
Notes de l'éditeur
- Hello. I’m Samantha Heeson, I work in the Learning Resources team at Leeds Beckett University library. A key part of my role is to provide evidence and insight to support collections management and decision making, so that our users have appropriate and value for money learning resources. Today, I’m going to talk about a pilot project that transformed OpenAthens e-resource usage data into student engagement insights.
- Just for background… OpenAthens was reinstated at Leeds Beckett in 2017 to improve user experience, take advantage of vendor input, and for the potential to analyse the usage data. ADFS is used for identifying users by their IDs only, which is the university’s approved solution. The major e-journals and databases were put on to OpenAthens during the spring of that year. I started working in the new role of Electronic & Data Services Librarian around the same time. So, how did this pilot actually start? Well, my manager showed me an OpenAthens dataset and said “Can you do something with this data?” So, let’s find out…
- Here’s an example of that data. It came as monthly CSV reports and focused on e-resource usage. Across the top are the e-resources, down the left side user IDs, and in the middle the number of accesses for each e-resource. It provided a basic overview of e-resource usage, but the different user groups weren’t identifiable. Manually, it was too much to look at, because there are over 30,000 students and up to three quarters of a million pieces of data each month. So, I started to think about how I could demonstrate its potential to stakeholders...
- I created a proof of concept that combined three months of OpenAthens data with a snapshot of the Student Record from that time period. I also pulled in physical stock borrowing data for the same period, plus that year’s reading list data. Before I could combine it with the Student Record, I had to restructure, exclude and reformat some of the data, for example, student IDs needed trimming, so that I could tie the different datasets together. It is the Student Record that really enables the in-depth analysis; I previously worked in Student Registry as an analyst and so I was familiar with this kind of work. Anyway, this extract for an Education course showed students had accessed a range of e-resources and books during that period. In particular, they’d chosen relevant resources and accessed e-resources more than books. The full version also showed module reading lists and suggested wider reading was taking place. Colleagues from Learning Resources and Academic Support could also see the potential in this and so I was asked to do a bespoke piece of work around a particular cohort…
- So, I then worked on a prototype which refined the proof of concept by processing the data in a database, visualising it in some simple charts, and adding some narrative. It looked at e-resources accessed by a new cohort of Education students during their first five months to see what the impact of their library induction had been. This focussed analysis showed there had been an increase in the number of e-resources accessed and the overall number of e-resource accesses. Two e-resources had seen an increase, including one that had been promoted at the induction. However, it was too early to see any obvious or long lasting impact. Colleagues were enthused by this, although a little disappointed with the inconclusive findings. But it was noted that the learning from the induction would probably take place in the new year at assessment time. Anyway, they certainly had an appetite for more...
- So, I started transforming the proof of concept and prototype into a standard report that showed student engagement with e-resources at institutional and school level. This involved interim runs with six, nine and ten months of data that were tweaked to achieve the final version, which was done once twelve months of data was available at the end of the academic year. The result was a stable system for the data analysis and a static dashboard style report for sharing the visualisations and narrative. The first page of that report can be seen here. It highlights the percentages and numbers of active students, the average number of e-resource accesses by students, and the number of e-resources accessed. It also easily identifies anomalies within the expected patterns. For example, a significant peak in usage, shown here by the green spike, reflected the activity of a handful of students, on a few courses, in a couple of schools, for one e-resource. Academic Support used this to discuss student engagement and e-resource usage with academics in each school during their annual reviews and to inform in-year budget adjustments. Next steps will include refining the system, so that others can work with it and I can develop more analyses, such as pulling in physical stock usage and staff data. In the meantime, I have been working on some custom reports…
- For example, one school wanted a breakdown by level of study and year of study, i.e. undergraduate or postgraduate and first year or final year, etc. Part of that report can be seen on the top left. It shows the number of e-resources accessed by undergraduate, taught postgraduate and research students across the years of their studies, plus aggregation by level of study and school, and the total number of e-resources available that year. This led to some healthy discussions about engagement within particular groups of students and future activities for signposting and supporting students with their research. It may be rolled out to other schools in the future, but as there are 14 schools it would require quite a bit of time to do that. I’m currently working on a correlation analysis of student engagement with e-resources and student satisfaction with learning resources, which means pulling in National Student Survey data and analyzing at course level. The charts on the right are from this work. The scatter plot at the top shows the results for the learning resources question, against the percentage of active students, and the average number of e-resources accessed for each course. The column and line charts at the bottom show the same information, plus the size of the cohort and the NSS response rate for each course. At the moment, they don’t show any obvious correlation, but the next step will be to group the courses by school to see if there is any correlation at that level. Another set of next steps…
- … would be to do some longitudinal analysis. However, I haven’t been able to start work on this year’s data, because the OpenAthens reporting interface changed in November and this has led to some download time-outs and duplication in the data, which are currently being reviewed by their developers. Also, a different structure to the dataset still requires quite a bit of manipulation to get it into a suitable layout for student engagement analysis, rather than e-resource usage analysis. It is possible now though, to get hourly and daily, as well as monthly snapshots, which means there is the potential to better assess the impact of library initiatives and interventions. So the answer to the question “Can you do something with this data?” was and hopefully remains “Yes I Can!”. The key to this work is the Student Record data and the visualisations. The Student Record lets us look at student engagement with e-resources, rather than just e-resource usage, and the visualisations highlight trends, anomalies, correlation and areas for further investigation, which may be within the OpenAthens data and / or other data that we collect. So, that just leaves me to say…
- Thank you very much for listening and if you have any questions I may be able to answer them now. Or you can contact me on my work email address. And there will also be a more in-depth session on this work at the UKSG conference in April in Telford if anyone is going to that.