For this month's community meetup, Andrew Martin from DAMsmart shared thoughts and experiences from an audiovisual LTO data migration project for a government agency. He was joined by Tom Eccles from AIATSIS. For more info, checkout the blog post here: https://www.australasiapreserves.org/2019/08/july-2019-monthly-meetup-on-lto-data.html

1. LTO DATA MIGRATION

2. WHAT IS LTO?
• Linear Tape-Open developed
in the 1990’s by the LTO
Consortium: HP, IBM, and
Quantum
• An open standard
• Magnetic tape single reel
storage format
• Single Desktop Drives, or
autoloader libraries
• Backwards compatible drives
by 2 generations

5. CURRENT ISSUES AROUND LTO 8
Changes from GMR (Giant Magneto-Resistance) heads to TMR (Tunnel Magnetoresistance
Recording) heads.
LTO 8 drives can only read LTO 8 and LTO7 Cartridges
LTO 8 cartridges currently unavailable due to manufacturer lawsuits regarding patents

6. CASE STUDY: AIATSIS ATSIC DIGITISED
VIDEO COLLECTION
• 50 LTO 4 cartridges to be migrated to LTO 6 (as delivery medium)
• 2,900 audiovisual preservation files
• 36 TB of data
• Written using the POSIX TAR format
• No tape contents information

7. • LTO drives configured and controlled with Cygwin.
Correct block size was set on the drive using –mt
Correct blocking factor for reading was selected when restoring using -tar
• Approx. 120MB/s per data stream
• 3 weeks to transfer all data onto 5 x 12 TB Western Digital Surveillance Drives
6 Gb/s SAS
6 Gb/s SAS
USB 3
USB 3

9. DATA RESTORE AND VERIFICATION
Video Files Lossless JPEG2000.mxf
Accompanying XML Metadata files that contained checksum for the video file
Data verified using custom software
Look for the same filename
Extract checksum
Create checksum of video file and compare

10. COMPARE
REPORT
• Filename compare
• SHA1 from XML
• SHA1 calculated
• Does it match!

11. DATA PREPARATION AND
LTO6 WRITING
• Before writing data, files were renamed.
ATSIC0001.mxf-0-0-1 > ATSIC0001.mxf
• Data migrated to 17 LTO6 using LTFS
• Verification of data written to tape
• Verification reports and LTO drive contents
created.

12. WHAT TO WATCH OUT
FOR WHEN PREPARING
FOR AN LTO DATA
MIGRATION PROJECT
What format / vendor has the source LTO
been written under?
Is there a database of the source data?
Does the data need specific proprietary
software to be restored?
Are there existing checksums that can be
utilised to ensure correct restore?

13. WHAT TO WATCH OUT
FOR WHEN PREPARING
FOR A DATA
MIGRATION PROJECT
• Is there any additional
processing required while files
are online in the staging
process? (e.g. metadata
creation, QC checks,
transcoding)
• Would you write tape to tape,
or tape to disk to tape?
• Is LTO the right format to be
migrating to? What are the
Preservation and business
requirements?
• Would migrating to the cloud
be appropriate?

14. THANKYOU!!
Andrew@damsmart.com.au
@audiovisualopen
www.damsmart.com.au