Scaling API-first – The story of a global engineering organization
DPLA + WI: Building a DPLA Service Hub in Wisconsin (WAAL Conference)
1. DPLA + WI
BUILDING A DPLA SERVICE HUB IN WISCONSIN
Emily Pfotenhauer
Recollection Wisconsin Program Manager, WiLS
Wisconsin Association of Academic Librarians
April 22, 2015
2. SERVICE HUB
RESPONSIBILITIES
1. Representing their community (state, region, etc.) as the point of
contact for DPLA and obtaining community buy-in on significant
issues affecting their partners.
2. Aggregating their partners’ metadata into a single standard and
sharing it with DPLA through one harvestable data source.
3. Actively addressing metadata concerns (including copyright and
licensing labeling) and working with partners on timely remediation.
4. Providing outreach to their partners, developing local
practitioners’ capacity on topics such as open data, data quality and
standards, copyright and licensing, and other relevant subjects.
5. Maintaining technologies that allow for standardized metadata to
be shared with the DPLA on a regular, consistent basis.
6. Engaging with the broader community of data creators, providers,
and users, locally and nationally.
3. SERVICE HUB
BENEFITS AND IMPACT
1. Broaden the impact and reach of Wisconsin’s
libraries and cultural heritage institutions.
2. Enable more Wisconsin libraries and cultural
heritage institutions to share and preserve their
digital collections.
3. Inspire innovative uses of Wisconsin’s digital
content.
4. Ensure that Wisconsin is well-represented on the
national map of digital content.
4. SERVICE HUB
FOUNDATIONS
• Wisconsin’s long history of digitizing and sharing
cultural heritage resources.
• Ten years of statewide collaboration through
Wisconsin Heritage Online/Recollection Wisconsin.
• Two years of conversations among stakeholders.
5.
6. PHASE I
APRIL 2015-FEBRUARY 2016
Phase I Goals:
• Establish metadata aggregation infrastructure
• Establish workflows for metadata ingest,
remediation and sharing
• Establish initial partnerships and governance
structure
• Establish communication and outreach plans
7. PHASE II
FEBRUARY 2016-DECEMBER 2017
Phase II Goals:
Based on community needs and available resources.
Priorities:
• Evaluate and formalize Phase I governance structure
• Identify and add new Content Partners
• Investigate collaborative strategies for digital preservation
• Develop training or other opportunities to help more institutions
digitize content and share metadata
8. PHASE I
CONTENT
Approximately 400,000 metadata records representing
content from more than 140 libraries, archives, historical
societies and museums across the state.
• University of Wisconsin Digital Collections Center
• 182,000 records
• UW-Milwaukee
• 110,000 records
• Wisconsin Historical Society
• 15,000 records
• All Recollection Wisconsin content not included in above
• 94,000 records
12. PHASE I
METADATA AGGREGATION
Responsibility of Content Partners:
• Grant permission to share metadata with DPLA
through a Creative Commons Zero
declaration/public domain dedication
13. PHASE I
GOVERNANCE
• Governing Board
• One representative from each Governing Partner
• Policy and budget
• Steering Committee
• Librarians, archivists, curators and other practitioners
across the state
• Outreach, education, future directions
• Metadata Work Group
• Metadata librarians and other experts
• Recommendations for metadata compliance, updated
guidelines
14. PHASE I/PHASE II
FUNDING
• Nicholas Family Foundation grant
• 2015-2017
• LSTA grant (in development)
• 2016
• Harvesting and hosting fees from Content Partners
• In-kind contributions from Governing Partners
15. PHASE I
TIMELINE
April-June 2015
Governing Board, Steering Committee and Metadata Work Group
established.
June 2015
Service Hub application submitted to DPLA.
June-August 2015
CC0/public domain metadata dedication confirmed with Content
Partners.
August-September 2015
Data Exchange Agreement and Ingest Information Form completed.
Development of metadata aggregator begins.
September-November 2015
Iterative ingest and metadata review with DPLA staff.
January-February 2016
Data is publicly available through DPLA.
16. Q&A SESSIONS
WAAL Conference, Manitowoc
April 22
WAPL Conference, Wisconsin Rapids
May 8
Lake Superior Libraries Symposium, Duluth
June 5 (with Minnesota Digital Library)
WiLS Peer Council, Madison
June 8
I’m here to share information about the collaborative work that’s going on here in Wisconsin to create a DPLA Service Hub -- something that will benefit not only every library and culture heritage institution in our state, but will also benefit citizens, students, researchers and educators across the state and across the country. I’ll keep this relatively short because one of the things we really want to do today is to give you all the opportunity to ask questions and share your input about the Hub development process.
So Amy described DPLA’s Service Hub network model. DPLA has laid out six roles that every Service Hub is expected to fill.
A hub functions as the on-ramp for local institutions to participate in DPLA by acting as a central point of contact – essentially the liaison between DPLA and a state or region’s cultural heritage organizations.
A hub performs the technical and administrative work necessary to bring together metadata describing digital content from disparate collections and provide that data to DPLA in a single stream.
A hub works closely with its Content Partners to identify metadata concerns and ensure that data meets the requirements for inclusion in DPLA.
A hub works to build a community of practice within its state or region through education and training around best practices for copyright, shareable metadata and other digital collections issues.
A hub maintains a stable endpoint to continue to share aggregated metadata with DPLA and to regularly ingest metadata for new digital collections.
And finally, a hub connects Content Partners of all sizes and types to a national network that is actively and collaboratively shaping the landscape of open access to our nation’s digital cultural heritage.
Bringing Wisconsin into the DPLA as a Service Hub will broaden the impact and reach of Wisconsin’s libraries and cultural heritage institutions. The Minnesota Digital Library reported a 55% increase in visits to their digital collections since joining DPLA; the Mountain West Digital Library saw traffic increase by more than 100%.
Participating in DPLA will enable more Wisconsin libraries and cultural heritage institutions to bring their collections online through coordinated statewide professional development opportunities.
Inclusion in DPLA will inspire innovative uses of Wisconsin’s digital collections. As Amy mentioned, DPLA works to enhance metadata from contributors through geotagging and other tools. This enhanced metadata is made available through an open API, which developers and researchers can use to create innovative environments for learning and discovery.
A Service Hub in Wisconsin will also ensure that our state is well represented on the national map of digital content as DPLA continues its rapid growth.
A Digital Public Library of America Service Hub in Wisconsin will build on almost two decades of ongoing work by Wisconsin’s cultural heritage institutions to digitize and provide access to unique and valuable research resources online.
Wisconsin’s Service Hub will expand on the existing Recollection Wisconsin statewide collaborative initiative, formerly known as Wisconsin Heritage Online. Since 2005, WHO and Recollection Wisconsin have provided guidelines and standards for creating and sharing digital content. Recollection Wisconsin currently coordinates the aggregation of approximately 219,000 metadata records representing content from more than 140 libraries, archives, historical societies and museums across the state.
When DPLA made its public debut about two years ago, Recollection Wisconsin’s sponsoring partners began to discuss pathways to bring Wisconsin on board with this groundbreaking initiative. Those around the table for these discussions included the Milwaukee Public Library, UW-Madison, UW-Milwaukee, WiLS, the Wisconsin Department of Public Instruction and the Wisconsin Historical Society. These partners held conversations with staff at DPLA and staff at other Hubs, especially the Minnesota Digital Library, and went through numerous versions of a plan for establishing a Service Hub.
Each of these partners had already made an ongoing commitment of specific resources in support of the Recollection Wisconsin collaborative. These in-kind contributions are providing the foundation for creating a Service Hub in the state.
Milwaukee Public Library hosts content for small institutions.
UW-Madison maintains a metadata aggregator.
UW-Milwaukee will provide digitization services and consulting for Milwaukee-area cultural heritage institutions.
DPI administers LSTA funding including the Digitization of Library Historical Resources funding category.
Wisconsin Historical Society provides support for technology development and community outreach.
WiLS hosts Recollection Wisconsin program staff, provides project management, and acts as fiscal agent for the collaborative.
The work to scale up from the existing Recollection Wisconsin program to a DPLA service hub will be rolled out in phases. Phase I is relatively short and closely follows the timeline DPLA has established for adding new Service Hubs. Basically, Phase I puts procedures in place in order to share our first batch of metadata with DPLA.
Goals for Phase I are:
Build out the tools for aggregating data and sharing it with DPLA in a single stream (this work is being contributed by UW-Madison)
Creating procedures for evaluating and adding metadata from Content Partners (this is the purview of a Phase I metadata workgroup, which is made up of metadata staff at Wisconsin Historical Society, WiLS, UW-Milwaukee, UW-Madison and Marquette University)
Formalizing partner relationships and creating a governance structure (which I’ll talk about more in a moment)
Create a plan for information sharing and building community awareness and support across the state (this will be the purview of a Steering Committee which is now being established)
Phase II is slated to begin after the first batch of metadata has been provided to DPLA in early 2016. Phase II will evaluate and build on the lessons learned in Phase I. Priorities will be set based on the needs and interests of the Wisconsin cultural heritage community. The scope of Phase II will depend on available funding. Some of the priorities and interests we’re already hearing from the community are an increased focus on digital preservation issues as well as education and training around the creation of new digital content.
Phase I will provide approximately 400,000 metadata records to DPLA. That number represents content held by more than 140 libraries, archives, historical societies, museums and other institutions throughout Wisconsin. It encompasses a broad array of original materials including photographs, maps, manuscripts, books, films, oral histories, music, artwork and artifacts.
In expanding into a DPLA Service Hub, Recollection Wisconsin’s scope of content will necessarily expand as well. The data currently harvested for Recollection Wisconsin represents content describing state and local history and culture.
In Phase 1, the amount of harvested data will nearly double, and will expand beyond content about state and local history to a much broader scope of subject matter.
This expanded scope will encompass many significant research collections that are held by Wisconsin institutions but are not about Wisconsin. Just a few examples are the archives of the 1964 Freedom Summer Project documenting Civil Rights activism in Mississippi, held by the Wisconsin Historical Society; the vast holdings of the American Geographical Society Library at UW-Milwaukee; and the Digital Library for the Decorative Arts and Material Culture, digitized by UWDCC in partnership with the Chipstone Foundation.
Technology for aggregating metadata will be developed and maintained by UW-Madison using the Open Archives Initiative Protocol for Metadata Harvesting (OAI-PMH). UW-Madison will harvest data from Content Partners through OAI, then use OAI to feed that data to DPLA in a single stream. Search results made available through DPLA will always link users directly back to the original digital source material.
A Content Partner is any library, archives, historical society, museum or other institution that provides public access to digitized content and makes metadata describing that content available for OAI harvesting. In order to participate in DPLA through the Service Hub, Content Partners will need to dedicate their metadata to the public domain through a Creative Commons CC0 license. This summer, we will be getting in touch with all existing Content Providers to formalize this public domain dedication. If Recollection Wisconsin harvests your data directly or you have a collection that’s hosted by Milwaukee Public Library, you’ll be hearing from me directly about that public domain dedication for your metadata; if you have content in UWCD you’ll be hearing from the staff at UWDC.
As Amy mentioned, the CC0 declaration is essential. Opening up data in this way is what enables the work that DPLA does to enhance data and make it available through their API.
In Phase I, we’ve begun to establish a collaborative governance structure to oversee the development of our Hub.
This structure currently includes three entities:
A Governing Board made up of one representative appointed by each Governing Partner. These are the institutions making the in-kind contributions I mentioned previously. This Board will direct the initial implementation of the Hub, specifically policy and budget decisions.
A Steering Committee made up of librarians, archivists, curators, educators and other experienced practitioners will develop procedures for adding partners and will explore potential future services. The Steering Committee will also serve an advocacy and community building role.
A Metadata Work Group will evaluate metadata, make recommendations for metadata compliance, and update existing metadata guidelines.
The Steering Committee and Metadata Work Group are still being formed; if you’re interested in lending your expertise to either of these groups, please come chat with me! We’re especially looking for more representation from the far north parts of the state.
Current funding for the hub comes from multiple sources, including a grant from the Nicholas Family Foundation, an LSTA grant which is currently in development, fees contributed by Content Partners for hosting content and harvesting data, and the in-kind commitments I mentioned earlier.
We are actively pursuing additional funding for Phase II and beyond. Potential funding sources we’re looking into include federal grants, gifts from private foundations or other donors, cost recovery services and additional in-kind contributions.
Here’s our timeline for Phase I:
The creation of the board, steering committee and metadata work group are currently underway, and as I said, if you’re interested in participating in any of those, please reach out to me.
In June we will go through DPLA’s application process in order to formalize our partnership and make our Service Hub status official.
This summer we’ll be contacting Partners to secure the Creative Commons Zero determination.
Late this summer and early fall we’ll begin the metadata ingest process with DPLA staff.
In early 2016, that first batch of approximately 400,000 metadata records will be shared with the world through DPLA.