Providing Voyager catalog data in a custom, open source web application, "Launchpad" outlines the features of customized library catalog software application from the George Washington University.
ELUNA2013:Providing Voyager catalog data in a custom, open source web application, "Launchpad"
1. Providing Voyager catalog data in a
custom, open source web application
“Launchpad”
ELUNA 2013 Athens, GA
Michael Cummings, GWU
2. Launchpad ELUNA 2013
ELUNA 2013 Athens, GA Providing Voyager catalog data in a custom, open source web application
3. Agenda ELUNA 2013
• Making Requests (http request)
• Retrieval
• Features (http response)
• The software
ELUNA 2013 Athens, GA Providing Voyager catalog data in a custom, open source web application
4. Request: URL pattens ELUNA 2013
The application responds to http requests.
* By ‘item’ we mean the bibliographic id (bib_id)
http://findit.library.gwu.edu/item/5642529
isbn/0521800900
issn/1448-4404
oclc/ocm46792617
item/2388259.json
OpenURL requests are also accepted.
ELUNA 2013 Athens, GA Providing Voyager catalog data in a custom, open source web application
5. Request: Google ELUNA 2013
Search term(s) gwu libraries
Link to Launchpad
Voyager bibid 2388259
ELUNA 2013 Athens, GA Providing Voyager catalog data in a custom, open source web application
6. Request: schema.org ELUNA 2013
The HTML source in our pages includes markup compliant with schema.org
guidelines. This may help Google index and retrieve records from our index.
<div itemscope itemtype="http://schema.org/Book">
<span itemprop="name">The Visual display of
quantitative information / Edward R. Tufte.</span><br/>
<span itemprop="author">Tufte, Edward R., 1942-</span>
<span itemprop="publisher">Cheshire, Conn. :
Graphics Press, c1983.</span>
In <span itemprop="inLanguage"
class="language">English.</span>
ELUNA 2013 Athens, GA Providing Voyager catalog data in a custom, open source web application
7. Request: Analytics ELUNA 2013
ELUNA 2013 Athens, GA Providing Voyager catalog data in a custom, open source web application
8. Request: 360 Link ELUNA 2013
Our journal finder page has
a similar style or vice versa.
ISSN link to Launchpad
ELUNA 2013 Athens, GA Providing Voyager catalog data in a custom, open source web application
9. Request: Worldcat ELUNA 2013
Link to Launchpad
ELUNA 2013 Athens, GA Providing Voyager catalog data in a custom, open source web application
10. Request: Summon ELUNA 2013
Title and holdings are
Item links to Launchpad
ELUNA 2013 Athens, GA Providing Voyager catalog data in a custom, open source web application
11. Request: request JSON ELUNA 2013
URL ends with json
…/2388259.json
ELUNA 2013 Athens, GA Providing Voyager catalog data in a custom, open source web application
12. Retrieval ELUNA 2013
• Making Requests (http request)
• Retrieval
• Features (http response)
• The software
ELUNA 2013 Athens, GA Providing Voyager catalog data in a custom, open source web application
13. Retrieval ELUNA 2013
All of these sources may be accessed in the background to retrieve title, holding,
status, and other information using API’s, Z39.50, or SQLPlus.
SQLPlus query Z39.50 query
Innovative
Interfaces
Millenium
WRLC
Z39.50 query
Holding, item status
& bibliographic detail
Holding, item status Holding, item status
Google Book API Open Library API 360 Link API
360 LINK
Book covers,
full text online
Journal Coverage date,
provider
ILL links for titles not
held in consortium
ELUNA 2013 Athens, GA Providing Voyager catalog data in a custom, open source web application
14. Features ELUNA 2013
• Making Requests (http request)
• Retrieval
• Features (http response)
• The software
ELUNA 2013 Athens, GA Providing Voyager catalog data in a custom, open source web application
15. Features: Bib Info ELUNA 2013
We intentionally limited the amount
of information to display here. We
could show more.
A link to the classic catalog is
provided for more detail if needed.
ELUNA 2013 Athens, GA Providing Voyager catalog data in a custom, open source web application
16. Feature: Holdings Detail ELUNA 2013
Notes / Recent Issues detail
Gelman Periodicals
v.1(1986)-v.3(1988),
v.12(1997)-v.22(2007)
All volumes located off site (WRLC Center). To retrieve,
please submit a Consortium Loan Request form. Bound
volumes are Library Use Only.
ELUNA 2013 Athens, GA Providing Voyager catalog data in a custom, open source web application
17. Features: RTAC ELUNA 2013
Availability Status
via
SQLPlus -- WRLC Voyager
Z39.50 ----- GMU Voyager
Z39.50 ----- GT Millenium
ELUNA 2013 Athens, GA Providing Voyager catalog data in a custom, open source web application
18. Features: Loan ELUNA 2013
Link to Consortium Loan
only IF item is eligible
ELUNA 2013 Athens, GA Providing Voyager catalog data in a custom, open source web application
19. Features: SMS link ELUNA 2013
ELUNA 2013 Athens, GA Providing Voyager catalog data in a custom, open source web application
20. ELUNA 2013
Interlibrary Loan link
Features: ILL
ELUNA 2013 Athens, GA Providing Voyager catalog data in a custom, open source web application
21. API: Book Cover ELUNA 2013
Covers are displayed when they are available from
OpenLibrary’s covers API.
OpenLibrary cover API
See: http://openlibrary.org/dev/docs/api/covers
Example: http://covers.openlibrary.org/b/isbn/0521800900-M.jpg
ELUNA 2013 Athens, GA Providing Voyager catalog data in a custom, open source web application
22. API: Google Books ELUNA 2013
There was a search by ISBN
No match found in the
consortium catalog;
Search, found in Google books.
Provide an ILL link
Google Books API
https://developers.google.com/books/docs/v1/using
ELUNA 2013 Athens, GA Providing Voyager catalog data in a custom, open source web application
23. API: OpenLibrary fulltext ELUNA 2013
While processing the request we check the
OpenLibrary to see if a full text is available online. If
so, we generate a link to the full text.
Open Library Book API
See: http://openlibrary.org/dev/docs/api/books
Example:
http://openlibrary.org/api/books?format=json&jscmd
=data&bibkeys=LCCN:11025443
ELUNA 2013 Athens, GA Providing Voyager catalog data in a custom, open source web application
24. API: 360 Link ELUNA 2013
The 360 Link XML API provides
‘From – to’ date range and
Database provider name
From 2003 to present
Directory of Open Access Journals
ELUNA 2013 Athens, GA Providing Voyager catalog data in a custom, open source web application
25. JSON Catholic University ELUNA 2013
Consortium holdings, status
Coming from Launchpad
ELUNA 2013 Athens, GA Providing Voyager catalog data in a custom, open source web application
26. JSON George Mason ELUNA 2013
Primo
Consortium print serial holdings
select ‘More’ tab
ELUNA 2013 Athens, GA Providing Voyager catalog data in a custom, open source web application
27. The Software ELUNA 2013
• Making Requests (http request)
• Retrieval
• Features (http response)
• The software
ELUNA 2013 Athens, GA Providing Voyager catalog data in a custom, open source web application
28. Software: Environment ELUNA 2013
• Prototyped using SQLPlus, with CGI-perl
• Converted to a Django application framework
• Programming in Python on Ubuntu Linux
• Tested with Apache Bench
ELUNA 2013 Athens, GA Providing Voyager catalog data in a custom, open source web application
29. Software Open Source ELUNA 2013
• The ‘about Launchpad page’ at github is
http://gwu-libraries.github.io/launchpad/
• The code is managed with git
• Available on GW Libraries github repository
• The code is available as open source
• Updates are released on a monthly schedule
ELUNA 2013 Athens, GA Providing Voyager catalog data in a custom, open source web application
30. Software: github ELUNA 2013
https://github.com/gwu-libraries/launchpad
Select ‘Code’ tab, Readme.md for instructions
ELUNA 2013 Athens, GA Providing Voyager catalog data in a custom, open source web application
31. Software: Python code ELUNA 2013
The Python .py code, filed under / lp / ui
is probably of interest to programmers…
ELUNA 2013 Athens, GA Providing Voyager catalog data in a custom, open source web application
32. Software: Learn more ELUNA 2013
About Launchpad
http://gwu-libraries.github.io/launchpad/
Launchpad code on git hub
https://github.com/gwu-libraries/launchpad
ELUNA 2012
From Voyager to your website:
Using Linux Shell scripts and Oracle SQL*Plus to generate web pages.
https://github.com/cummingsm/VoyagerELUNA2012
Includes slides from 2012 presentation. They’re also on the ExLibris site.
Contact
Michael Cummings
Library Systems Coordinator
The George Washington University Libraries
michaelc@gwu.edu
ELUNA 2013 Athens, GA Providing Voyager catalog data in a custom, open source web application
Notes de l'éditeur
This is a screen shot of a typical book, in this case one containing information obtained from three different library catalog systems.
This presentation is only 30 minutes in total, perhaps 25 minutes after preliminaries. Subtract another 5 minutes for questions. So, figure about 5 minutes per topic on this agenda. This will be fairly high level, but the notes contained in the slides will have a lot of detail you can read later.
Using the word ‘item’ in the URL is sort of a misnomer,. It is the Voyager, or Millenium, bibid not the ‘item’ id. Each title has an item page. The other URL patterns are for look-up purposes.
If a match is found based on one of the other standard identifiers, the page displayed will show the bib id in place of the standard number in the URL.
We generate an xml site map file, and Google indexes it. Processing is performed at roughly 50k / nightly, a limit imposed by Google. As of May 2013 about a million of our titles have been indexed (out of 12 million).
To search in Google, enter search term(s) followed by ‘gwu libraries’.
We put the Google Analytics snippet on our pages. This is a screen shot from Google analytics for the first three months of 2013. Almost all requests are by the bibid.
The journal finder is at the same root URL, findit.library.gwu.edu
We made the user interfaces of the journal finder and the launchpad look pretty much the same.
This slide shows an OpenURL request to the journal finder.
Worldcat will link to GW Launchpad, if there is an ISBN or ISSN number.
Summon has been configured to link out to the GW Launchpad page when a user clicks on a title or holding of a library catalog record in the search results.
* Ooops. This Discovery system isn’t always able to provide real-time availability in search results. That’s one reason we developed the Launchpad application. With Launchpad, we are always able to get real-time availability.
Our developers prefer working with JSON instead of XML. Although there are parsers for both, XML has a lot of extra text because of the tagging it uses.
Two schools have programs that request this JSON output and then incorporate it into their discovery system using a combination of Php and Javascript.
Next, what happens when the Launchpad receives the http request.
The WRLC maintains a consortium Voyager catalog of approximately 12 million bib records including George Washington University, American, Catholic, District of Columbia, Gallaudet, George Mason (copy of bib), Georgetown (copy of bib), Howard, and Marymount. Our program retrieves bibliographic, holding, and item status data from the WRLC Voyager using SQLPlus.
Georgetown has their own ILS, III Millenium, which we query using the Z39.50 protocol in order to obtain holding and item status.
George Mason has their own which we query using the Z39.50 protocol in order to obtain holding and item status.
We use API’s to retrieve other information, which is explained in the following pages.
After retrieval of the information, we have 100% control over the display. The following slides highlight some of the features we chose to implement so far.
We found that users want the location and availability to be prominent. We decided to show only a few elements of the bibliographic description. However, we could show more.
Note: Our program is written in Python, and we can use the pymarc module to easily parse the MARC record.
Developers recommended a clean layout, one line per entry. However library staff insisted on the notes too.
This example isn’t too bad, but in some cases the notes are extensive. In an upcoming release, we convert these notes to hide/show links.
This screen shot shows a title from all three integrated library system databases which we polled for real-time item status.
Conditional logic in our application determines eligibility for consortium loans.
In this example, the item at George Mason Fenwick stacks is Charged and therefore the item is not eligible for a consortium loan request at this time.
There is a bit of Javascript loaded with the page which enables the user to click an icon and see the ‘Send to Mobile/Email window; the users may provide a cell number or email to themselves.
Launchpad opens the Interlibrary Loan form (ILLIAD), and populates some fields on the ILLIAD form. We recently updated our ILL code to handle both 0.1 and 1.0 versions of the OpenURL
Book covers are provided via OpenLibrary’s image API. This is a free service.
OpenLibrary cover API
See: http://openlibrary.org/dev/docs/api/covers
Example: http://covers.openlibrary.org/b/isbn/0521800900-M.jpg
The Google Books API is checked when Launchpad can not find a record in the consortium catalog.
If we find something via Google Books, we provide the user a link so that they can request the title through ILL.
We could provide a link to Google Books too, but so far have elected not to do so.
Google Books APIhttps://developers.google.com/books/docs/v1/using
OpenLibrary / Internet archive full text is available for thousands of books.
* Note: There are times when OpenLibrary is closed and the API call will fail. We had to make sure we coded our program to continue gracefully when requests to OpenLibrary failed.
The API accepts queries based on standard numbers LCCN, OCLC, and OpenLibrary ID.
While processing the request we check the
OpenLibrary to see if a full text is available online. If so, we generate a link to the full text.
Open Library Book API
See: http://openlibrary.org/dev/docs/api/books
Example:
http://openlibrary.org/api/books?format=json&jscmd=data&bibkeys=LCCN:11025443
360 Link API provides coverage details.
Often there are several providers with access to a different range of dates for the same publications.
A setting in the application makes a full text online button for a preferred library – in this case Gw. A text link with a lock icon is used for full text holdings from other consortium member libraries that require their own authentication for online resources.
JSON data from Launchpad is read and reformatted by an application at CUA for display within their Summon Discovery system on the page where bibliographic details are provided.
JSON data from Launchpad is read and reformatted by an application at GM for display within their Exlibris Primo Discovery system.
GM uses this for their bound periodicals because these are often held by other consortium library members, and patrons from any member library can use the consortium loan system to request them.
Next an overview of the software.
GW has been using SQLPlus in a Linux environment for a couple of years. “Using Oracle SQL*Plus to query Voyager in a Linux environment” was the topic of the presenter’s talk at ELUNA in 2012. A link to the presentation is provided later in these slides.
We were able to do a rapid prototype of our concept using SQLPlus and CGI-Perl.
We then converted the code to a more modern web application framework. Django manages cache, connections, and other aspects quite well. Normally you can model your database and then use Django to perform queries without writing SQL. However, due to the large Voyager schema and the fact that we already had the SQL from our prototype, we include those queries in our code. Python has many advantages, including modules that developers can import – such as pymarc for parsing MARC records.
We stress tested the code with Apache Bench in coordination with the Voyager system administrator.
We involved staff from all departments in the design process. We specified the scope of the project as an application that will display information about a specific title and holdings of the title across the consortium.
We have a very transparent process, with a public code repository on github.
The code is officially available as open source.
We have several projects on this site. The readme file provides installation instructions.
https://github.com/gwu-libraries/launchpad
Select ‘Code’ tab, Readme.md for instructions
We obviously don’t have time for a code review, but when you have time you can peruse the Python code here.
The Python .py code, filed under / lp / ui
is probably of interest to programmers…
About page
http://gwu-libraries.github.io/launchpad/
Code repository
https://github.com/gwu-libraries/launchpad
From Voyager to your website:
Using Linux Shell scripts and Oracle SQL*Plus to generate web pages.
https://github.com/cummingsm/VoyagerELUNA2012