Harvard Library and Library Technology Services review the new standard in digital preservation at the Library

1. The New DRS:
Plan for Metadata Migration
Harvard Library & Library Technology Services
February 26, 2014

2. Agenda
Welcome and introduction …... Franziska Frey, Tracey Robinson
Migration challenges .…………... Randy Stern
Creating the plan ………………..... Kate Bowers
Walkthrough of plan …………….. Andrea Goethals
Imaging Services:
minimizing disruption……….… Bill Comstock
Wrap-up & next steps ………...… Kate Bowers, Andrea Goethals
Q & A ……………………………….…... All

3. WELCOME & INTRODUCTION
Franziska Frey, Tracey Robinson

4. The DRS Advisory Group…
…provides oversight and guidance during the rollout
phase of the DRS2 project and ensures that the user
community of active DRS depositors and content
owners contributes to decisions about the rollout.
Amy Benson
Kate Bowers
Bill Comstock
Franziska Frey (chair)
Andrea Goethals
Wendy Gogel
Tracey Robinson
Randy Stern

5. Why a New DRS?
• Upgrade to best-in-breed technologies
• Adopt digital preservation best practices
and standards
• Preserve metadata better
• Improve collection management
• Support preservation planning & activities
• Improve access to content & metadata
• Support more formats & genres

6. Preservation Capability Before and
After the DRS2 Project
Level One Level Two Level Three Level Four
Storage & Geographic
Location
File Fixity and Data Integrity
Information Security
Metadata
File Formats
= already compliant = will be compliant after the DRS2 project
Based on the NDSA Levels of Digital Preservation

7. Evolution of the DRS
2000 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 20122001
Current
DRS in
production
New DRS in
productionDRS enhancements
New DRS infrastructure
development
2013 2014 2015
New DRS metadata
migration
& user adoption

8. New DRS - Completed
2009 2010
2011 2012
convened
DRS
Advisory
Group
software in
production
2013 2014 2015
users trained,
phase 1
hardware in
production
migrated content
to new hardware
Infrastructure
Development
Metadata
Migration
& User
Adoption
Fedora
assessment
DuraCloud
pilot test
early release beta 1 beta 2
beta 3
first object
deposited
to the new
DRS

9. New DRS - Upcoming
2009 2010
2011 2012 2013 2014 2015
Infrastructure
Development
Metadata
Migration
& User
Adoption
metadata
migration tools
created
metadata
migrated
users moved

10. MIGRATION CHALLENGES
Randy Stern

11. Why “Metadata” Migration?
Why not
“content” migration?

12. Pre-migration
DRS Content
Current
DRS
Database

13. Post-migration
DRS Content
Current
DRS
Database New DRS Database
New DRS
Index
New DRS Object
Descriptors

14. New DRS Data Model
• Not a simple metadata conversion
• A new DRS object is a logical intellectual
entity that unifies multiple DRS files
– Still image objects - archival and production
masters, and deliverables including thumbnails
– Audio objects - archival and production
masters and deliverables
– PDS objects - page image and text files

15. Object Descriptors
• METS files generated for each object
– Standards-based internal schemas (PREMIS,
MODS, MIX, etc.)
• Metadata gathered from multiple sources
– Current DRS database
– Every content file
– HOLLIS records

16. Technical Challenges
• Many formats
– Images, audio, text, digitized books, web
sites, documents, biomedical image stacks,
opaque files
• Unique migration rules per format
– technical metadata, roles, relationships
• Large (>5000 file) PDS documents
• 45+ million DRS files

17. Technical Challenges
• At 1 sec/file, 45 million files would take
520 days!
• We are designing the migration
software tools for parallel processing
• We are configuring multiple servers to
run the migration

19. CREATING THE PLAN
Kate Bowers

20. Formulating a Migration Strategy
• Analysis of:
– DRS content
• Technical (relationships, etc. for building objects)
• Volume and type by repository
–metadata for mapping
–user activity in the DRS
–survey of highest volume, active users
–training and testing registration lists

21. Migration Strategy Factors
• Combines needs of users with technical
requirements
• User sequencing will be based on:
– Current deposit & administrative activity
– Level of preparation (training and participation
in beta testing)

22. WALKTHROUGH OF PLAN
Andrea Goethals

23. Migrating Content in 5 Stages
Migrate 1st : Tier 1 content
Migrate 2nd: Tier 2 content
Migrate 3rd: Tier 3 content
Migrate 4th: Tier 4 content
Migrate 5th: Tier 5 content

24. Migrating Content in 5 Stages
Migrate 1st : Tier 1 content
Migrate 2nd: Tier 2 content
Migrate 3rd: Tier 3 content
Migrate 4th: Tier 4 content
Migrate 5th: Tier 5 content
simpler objects
more complex objects

25. Migrating Content in 5 Stages
Migrate 1st : Tier 1 content
Migrate 2nd: Tier 2 content
Migrate 3rd: Tier 3 content
Migrate 4th: Tier 4 content
Migrate 5th: Tier 5 content
dependencies
between tiers
dependencies
within tiers

26. Migrating Content in 5 Stages
Tier Content
1 Text (Methodology, ESRI World File), Document, Color Profile, Target Image
2 PDS Document, Still Image
3 Audio, Text (SMIL)
4 Web Harvest, Opaque Container
5 Biomedical Image; Google Document Container 1, 2, 3

27. Migrating Content in 5 Stages
Tier Content
1 Text (Methodology, ESRI World File), Document, Color Profile, Target Image
2 PDS Document, Still Image
3 Audio, Text (SMIL)
4 Web Harvest, Opaque Container
5 Biomedical Image; Google Document Container 1, 2, 3

28. Migrating Content in 5 Stages
Tier Content
1 Text (Methodology, ESRI World File), Document, Color Profile, Target Image
2 PDS Document, Still Image
3 Audio, Text (SMIL)
4 Web Harvest, Opaque Container
5 Biomedical Image; Google Document Container 1, 2, 3
Tiers 1, 3, 4, 5: Migrate across all DRS owner codes at one time
Tier 2: Migrate one DRS owner code at a time

29. Tier 2: Sequence by DRS Owner Code
• Migrate just your unit’s PDS document and
still image content
• Minimize the amount of time the content
you manage the most is in 2 different
systems

30. DRS Owner Codes to be Migrated
DIV.LIBR
FHCL.HOUGH
FHCL.MUSI
GSD.LIBR
RAD.ARCH
RAD.SCHL
FHCL.JUD
FHCL.FAL
FHCL.MAPS
FMUS.MCZ
HLS.LIBR
HUL.ARCH
HUAM.MUSE
HBS.BAKR
VIT.BERE
HUL.PRES
FMUS.GRAY
HPPM.PIRC
DOAK.RESLIB
DOAK.MUS
FCOR.REISCH
FMUS.ARN
HLNC.LIBR
ARB.AAHOD
DOAK.LIBR
FCOR.FORST
FCOR.WOLBACH
FMUS.FARL
FMUS.HUH
FMUS.ORC
FMUS.PEAB
HMS.COUNT
HPRE.WARD
HUAM.SARDIS
HUL.GGL
HUL.OIS
FCOR.CARP
FCOR.HCO FCOR.URI
FHCL.CAB
FHCL.COLL
FHCL.DAVIS
FHCL.ENV
FHCL.FUNG
FHCL.GOV
FHCL.LITT
FHCL.MED
FHCL.SLV
FHCL.TOZ FHCL.YENCH
FMUS.SEM
FMUS.WARE
GSE.GUTMN
KSG.LIBR

31. Timing
• Current estimates:
–Building & testing migration tools: Now
–Begin Tier 1 content: Spring 2014
–Begin Tier 2 content: Summer 2014
• Units will be contacted about their Tier
2 migration schedule

32. After Your Tier 2 Migration
• You, and anyone depositing on your
behalf, will begin depositing only to the
new DRS
• All of your management tasks will be
done only in the new DRS

33. IMAGING SERVICES: MINIMIZING
DISRUPTION
Bill Comstock

34. Minimizing Disruption
• Testing by Imaging Services
• Uninterrupted services
• Migration sequencing
• Participating as a “pioneer”

35. Testing by Imaging Services
Alpha and beta testing:
• Depositing processes
• DRS content maintenance tools
–Searching and assembling content for
download
–Editing PDS objects

36. Uninterrupted Services
Providing services before and after your
migration
• Content needs to be deposited
• Content needs to be searched
• Content needs to be assembled
• may need to be edited
• may need to be downloaded

37. Migration Sequencing
• We will synchronize deposits with your
migration
–start depositing for you in the new DRS
after your Tier 2 content is migrated

38. Imaging Services as Pioneers
As pioneers, we:
• Learn to use the new tools
• Refine the new depositing workflows
• Identify bugs
• Suggest improvements
• Create a group of local experts that can support
those that follow
We’ll wear the scars so that you can stay pretty!

39. WRAP-UP AND NEXT STEPS
Kate Bowers and Andrea Goethals

40. Nine Pioneers
• Limited number of first depositors to new
DRS
• Factors
– New DRS-ready content from new systems
• EAS (Electronic Archiving Service), ACORN
(Weissman Preservation Center conservation
treatments), DASH (for ETD)
– Prepared and trained staff
– No content to migrate
• HUA example: opaque objects

41. First Deposit in the New DRS
大藏經 Da Zang Jing - Buddhist sutra, Qing dynasty (1644 -1911), China, Tibetan language

42. Email List
drs-migration@hulmail.harvard.edu

43. http://hul.harvard.edu/ois/systems/drs/drs2.html

44. Coming Attractions
• Open meetings
– Technical aspects brown bag (March)
– Digital preservation & DRS intro (Summer)
• Training and instruction
– Refresher training
– New training
– Onsite assistance

45. Q & A
Thanks!