SlideShare une entreprise Scribd logo
1  sur  5
Télécharger pour lire hors ligne
White Paper
The New Archive Workflow:
Accessible, Online and Protected

Tom Coughlin
Coughlin Associates
www.tomcoughlin.com

March 2013
The New Archive Workflow:
                                                                        Accessible, Online and Protected


Video gives us a way to visualize the dynamic relationships of the world around us. Higher resolution video at higher
Introduction
frame rates lets us experience our world at even greater detail and with a greater potential emotional impact. Combine
higher resolution with higher frame rates as well as input from multiple cameras and the demand for digital storage for
professional media and entertainment explodes, particularly for original or master content. Today some video projects
produced with high frame rates and high resolution run a few petabytes (PB) in size. In the not too distant future the size
of higher resolution, multiple camera video projects could be 100 times that size.

As the size and complexity of master and supporting preserved original content increases (raw captured content size
could grow close to 500 X in the next few years due to higher resolution and frame rate as shown in Table 1), the digital
archival systems used to preserve this content and make it available for use must become more intelligent. They also
have to scale to meet the needs of growing content libraries and provide an easy path to continued data access as well as
data protection over the life of the archive.

Table 1. Example Resolution, Data Rates and Storage Capacity Requirements
for Professional Media Content
                            Resolution                  Frame Rate                   Data Rates                  Storage Capacity/
 Format
                            (width X height)            (fps)                        (MBps)                      Hour (GB)
 SDTV (NTSC, 4:2:2,         720 X 480                   ~30                          6.25                        22
 8-bit)
 HDTV (1080p, 4:2:2,        1920 X 1080                 24                           49.8                        179
 8-bit)
 Digital Cinema 2K          2048 X 1080                 24                           199                         716
 (4:2:4, 10-bit) YUV
 Digital Cinema 4K          4096 X 2160                 48                           1,910                       6,880
 (4:4:4, 12-bit) YUV
 Digital Cinema 8K          7680 X 4320                 120                          23,890                      86,000
 (4:4:4, 16 bit)
2012 Digital Storage in Media and Entertainment Report, Coughlin Associates, www.tomcoughlin.com
“Super Hi-Vision” Video Parameters for Next Generation Television, SMPTE Motion Imaging Journal, May/June 2012, P. 63-68


As shown above, the storage capacity needed for high-resolution imaging is monumental, and it’s growing every day.
Traditional offline archiving does not meet the fast-paced demands of modern digital workflows. Video professionals
need a real-time, online way to simultaneously access stored content. This paper will examine the role of digital active
archiving technology in professional media and entertainment workflows.

We shall show the use of LTO tape during a typical media workflow and in an active content archive. Examples of how
a content library or a content archive use LTO tape for professional content will be given and the important role that
the LTFS file system in LTO tape provides for content storage will be explained. While LTFS and LTO are important
technologies for the creation of file-based archives, on their own, they are not enough. Thus, we also need to look at
intelligent storage architectures and solutions that harness LTFS to deliver data protection, scalability and manageability
for massive data volumes. We will conclude with some thoughts on the future of content archiving and the growing value
of an intelligent preservation archive.
The New Archive Workflow:
                                                               Accessible, Online and Protected


Archiving (and digital tape) plays an important role in professional content storage from the very point of creation
Alive in the Archive: Tape’s Modern Role in Media Workflows
through distribution. In large movie projects that involve significant investor money, production houses must create
a copy of new raw content on archive media as well as production media. Digital magnetic tape, usually LTO tape, is
used in the field to create archival quality copies when the digital footage is downloaded onto a hard disk drive from the
camera content media. This is particularly true for expensive solid-state camera cartridges where the cartridges are
cycled quickly for reuse. Several media production support equipment companies make storage systems that are used in
the field for making copies of content on magnetic tape and HDDs and thus meet these bonding requirements.

Likewise, in a post-production facility for film or television, magnetic tape is often used as working archive behind a HDD
(and possibly even some flash memory) storage array that serves video workstations. These various storage media
are used to optimize a storage solution using the trade-off in performance versus storage costs of the various storage
technologies. Flash memory can be fast but the cost per GB of storage capacity is often ten times or more expensive
than HDD storage. Magnetic tape is less expensive than HDD storage, particularly in normal operating costs (e.g system
power and HAVC costs) and the storage media life of magnetic tape is longer than HDDs. In a larger facility using
network storage this archiving storage tier would likely be closely integrated with the network storage as shown in
Figure 1.

The LTFS file system on LTO tape allows the creation of file and even object-based archives using this low-cost storage
media. LTFS is useful for media and entertainment applications since file access makes LTO tape act like a storage
volume (similar to a HDD or USB drive). File information makes content on tape (metadata) viewable without special
applications. As a consequence, it is easier to use LTO tape in workflows. Increased data mobility using file formats (such
as the LTFS) makes sharing across workflows easier, and thus magnetic tape can be an important element in a M&E
“sneaker net.“
Figure 1. Video Post Production With Networked Active Archive Storage
The New Archive Workflow:
                                                                  Accessible, Online and Protected

Some media facilities use magnetic disk storage for their content archives. Magnetic tape can be much more cost
effective, particularly because the operating costs of tape sitting on a shelf in a tape library is less than spinning disks in
drive arrays.

As shown in Figure 2, over a 20 year retention time the total costs of storing content on hard disk drives is more than
twice as large as storing it on magnetic tape in a tape library.

LTFS has generated a lot of interest from content creators, even smaller facilities, for long-term tape archives. Because
LTO-5 or 6 tapes can be used across multiple platforms, they can be used in various parts of the workflow. LTO can
replace pro-video tape archives, and it is less expensive to operate than disk-based storage. LTO tape-based archives
are playing a crucial role in cost-effective archive storage because of the physical life of the storage media, the backward
compatibility of LTO tape formats and their low operating costs for long-term storage.
Figure 2. Comparison of Estimated Annual Cost to Save 1 PB for 20 Years on HDDs Arrays and a Tape Library1.




But tape, by itself, cannot deliver the rapid access to data needed by media and entertainment facilities. That’s where an
intelligent active archive management solution is needed. One such solution, the StrongBox from Crossroads Systems,
uses a disk caching layer to enable multiple, simultaneous accesses to an underlying tape library.



These factors have made LTO tapes popular with many video content libraries. For example, Fox Studios uses LTO tape
Best Practices for Protected, Accessible Tape Archives
for archiving all their high definition content. We will look at a couple of other examples of LTO tape with intelligent
storage for professional media and entertainment from boutique post-production facilities to major broadcast networks.

ProductionFor is an Austin, TX based production and post-production company that provides media acquisition, editing
and customer animation graphics. Data from a single project can start with hundreds of source files taking up to a
terabyte of storage capacity. Editing this content increased the storage capacity demand and required a scalable cost-
effective long-term storage system that could handle massive video files.

A Crossroads StrongBox tape-based NAS storage appliance using LTO tape was found to be an effective repository for
high-volume long-term post-production content. Multiple copies on several pieces of media stored at separate locations
provided protection of the archived content. StrongBox’s multi-copy, hash-coding and system monitoring helped ensure
that the video content data was securely protected and accessible. The LTFS-enabled file structure allowed content on
the tape archive to be easily available to ProductionFor’s file-based workflows. This enabled single file access availability
at the click of a mouse.
The use of LTFS file system allowed any LTFS media to be kept in the StrongBox, whether it was created by ProductionFor
or not, and independent of the operating systems used. This allowed greater collaboration with content on LTO tapes
and meant the storage content could last beyond the life of the production system.

Major League Baseball (MLB) Network is the source for premier baseball content including live games, studio shows
and feature programming. A custom developed StrongBox LTO storage system was installed in late 2011 to provide
a portable data vault for long-term file retention. The internal HDD cache enabled rapid file access, and the LTFS file
system enabled drag-and-drop file movement simultaneously for multiple users.

The MLB Network logs and categorizes HD recordings by viewing proxy video files, recorded in real-time. Most of this
HD video is stored in the LTO library and is searched using DIAMOND and the Grass Valley Aurora system. MLB Network
is looking to store the long-tail proxy content on LTO tape as well.



Archiving plays a critical role in video content workflows, from content capture, through post-production and for long-
Conclusions
term preservation and library access. LTO tapes offering the LTFS file system provide file-based storage that allows tape
to be partitioned and used for project based storage of high resolution original content as well as transcoded files for
distribution networks or proxy files.

Archiving on magnetic tape results in significant cost reductions over the life of a digital archive. The operating cost
advantages of LTO tape storage and the file access of the LTFS file system make intelligent archiving solutions, like
StrongBox, an important element in a video facility’s media asset management, real-time content access, disaster
recovery strategy, and in a preservation repository.

As the size and complexity of professional video projects increases in coming years, file-based and even object-based
access to well-organized content will be even more critical to successful video production and broadcast. Future
magnetic tape technology will continue to develop both in storage capacity as well as data accessibility.



Tom Coughlin, President, Coughlin Associates is a widely respected storage analyst and consultant. He has over 30 years
About the Author, Tom Coughlin
in the data storage industry with multiple engineering and management positions at high profile companies.

Dr. Coughlin has many publications and six patents to his credit. Tom is also the author of Digital Storage in Consumer
Electronics: The Essential Guide, which was published by Newnes Press. Coughlin Associates provides market and
technology analysis (including reports on several digital storage technologies and applications and a newsletter) as well
as Data Storage Technical Consulting services. Tom publishes the Digital Storage Technology Newsletter, the Media and
Entertainment Storage Report, and the Capital Equipment and Technology for the Hard Disk Drive Industry Report. Visit
www.tomcoughlin.com.



Crossroads Systems, Inc. is a global provider of solutions and services that ensure stored data is proactively protected
About Crossroads Systems and StrongBox
and reliably recovered. Crossroads offers organization powerful data protection, proactive data security, intelligent
storage connectivity, unmatched performance, and significant cost savings. Crossroads was founded in 1996 and is
based in Austin, TX and holds more than 100 patents granted and pending.

StrongBox is a shared storage solution purpose-built for data archival and preservation. Using Linear Tape File System
(LTFS) technology and intelligent storage architecture with standard file systems (CIFS/NFS), StrongBox empowers
online, all the time, file availability with uncompromised data protection, full data mobility, non-proprietary file storage
and significant cost-savings. StrongBox unifies your archive - past and present - while providing seamless scalability
and reliability for the future.

Visit Crossroads at: www.crossroads.com




© Coughlin Associates 2013 All trademarks are the property of their respective owners.

Contenu connexe

Similaire à Ltfs new-archive-workflow-whitepaper

High Efficiency of Media Processing Amos K.
High Efficiency of Media Processing Amos K.High Efficiency of Media Processing Amos K.
High Efficiency of Media Processing Amos K.
Amos Kohn
 
Download presentation source
Download presentation sourceDownload presentation source
Download presentation source
Videoguy
 
Powering the Creation of Great Work Solution Profile
Powering the Creation of Great Work Solution ProfilePowering the Creation of Great Work Solution Profile
Powering the Creation of Great Work Solution Profile
Hitachi Vantara
 
Genetic Engineering - Teamworking Infrastructure For Post And DI
Genetic Engineering - Teamworking Infrastructure For Post And DIGenetic Engineering - Teamworking Infrastructure For Post And DI
Genetic Engineering - Teamworking Infrastructure For Post And DI
Quantel
 
Digitization of Physical Assets
Digitization of Physical AssetsDigitization of Physical Assets
Digitization of Physical Assets
Daniel Novak
 

Similaire à Ltfs new-archive-workflow-whitepaper (20)

Implementing an Intelligent Storage Policy.pdf
Implementing an Intelligent Storage Policy.pdfImplementing an Intelligent Storage Policy.pdf
Implementing an Intelligent Storage Policy.pdf
 
Video Compression
Video CompressionVideo Compression
Video Compression
 
Digital video
Digital videoDigital video
Digital video
 
A pragmatic archiving strategy
A pragmatic archiving strategyA pragmatic archiving strategy
A pragmatic archiving strategy
 
Issue in handling multimedia online
Issue in handling multimedia onlineIssue in handling multimedia online
Issue in handling multimedia online
 
NHK Challenges for Preserving UHD Materials (HIRAKAZU)
NHK Challenges for Preserving UHD Materials (HIRAKAZU)NHK Challenges for Preserving UHD Materials (HIRAKAZU)
NHK Challenges for Preserving UHD Materials (HIRAKAZU)
 
High Efficiency of Media Processing Amos K.
High Efficiency of Media Processing Amos K.High Efficiency of Media Processing Amos K.
High Efficiency of Media Processing Amos K.
 
Download presentation source
Download presentation sourceDownload presentation source
Download presentation source
 
IBM Tape Update Dezember18 - TS1160
IBM Tape Update Dezember18 - TS1160IBM Tape Update Dezember18 - TS1160
IBM Tape Update Dezember18 - TS1160
 
Powering the Creation of Great Work Solution Profile
Powering the Creation of Great Work Solution ProfilePowering the Creation of Great Work Solution Profile
Powering the Creation of Great Work Solution Profile
 
Genetic Engineering - Teamworking Infrastructure For Post And DI
Genetic Engineering - Teamworking Infrastructure For Post And DIGenetic Engineering - Teamworking Infrastructure For Post And DI
Genetic Engineering - Teamworking Infrastructure For Post And DI
 
A New Approach to Digital News Archiving
A New Approach to Digital News ArchivingA New Approach to Digital News Archiving
A New Approach to Digital News Archiving
 
proposal
proposalproposal
proposal
 
Future-Proofing Your Digital Media Storage
Future-Proofing Your Digital Media StorageFuture-Proofing Your Digital Media Storage
Future-Proofing Your Digital Media Storage
 
Future-Proofing Your Digital Media Storage
Future-Proofing Your Digital Media StorageFuture-Proofing Your Digital Media Storage
Future-Proofing Your Digital Media Storage
 
OSBConf 2015 | Contemporary and cost efficient backups to to tape by josef we...
OSBConf 2015 | Contemporary and cost efficient backups to to tape by josef we...OSBConf 2015 | Contemporary and cost efficient backups to to tape by josef we...
OSBConf 2015 | Contemporary and cost efficient backups to to tape by josef we...
 
The future of tape april 16
The future of tape april 16The future of tape april 16
The future of tape april 16
 
Blu Ray Disc Technology
Blu Ray Disc TechnologyBlu Ray Disc Technology
Blu Ray Disc Technology
 
Digitization of Physical Assets
Digitization of Physical AssetsDigitization of Physical Assets
Digitization of Physical Assets
 
Blue Ray Disk
Blue Ray DiskBlue Ray Disk
Blue Ray Disk
 

Plus de Marc Bourhis

Présentation 4 mars au cnc CST-rt30 -sécurisation des données numériques tour...
Présentation 4 mars au cnc CST-rt30 -sécurisation des données numériques tour...Présentation 4 mars au cnc CST-rt30 -sécurisation des données numériques tour...
Présentation 4 mars au cnc CST-rt30 -sécurisation des données numériques tour...
Marc Bourhis
 
Riam bilan 2011 2013 - projets retenus
Riam   bilan 2011 2013 - projets retenusRiam   bilan 2011 2013 - projets retenus
Riam bilan 2011 2013 - projets retenus
Marc Bourhis
 
Présentation Muriel Lebellac Videomenthe sur son offre Cloud Computing lors d...
Présentation Muriel Lebellac Videomenthe sur son offre Cloud Computing lors d...Présentation Muriel Lebellac Videomenthe sur son offre Cloud Computing lors d...
Présentation Muriel Lebellac Videomenthe sur son offre Cloud Computing lors d...
Marc Bourhis
 

Plus de Marc Bourhis (20)

Enjeux de la Réalité Virtuelle 15 avril 2017
Enjeux de la Réalité Virtuelle 15 avril 2017Enjeux de la Réalité Virtuelle 15 avril 2017
Enjeux de la Réalité Virtuelle 15 avril 2017
 
Formation à la Réalité Virtuelle - La Plateforme 1er décembre 2016
Formation à la Réalité Virtuelle - La Plateforme 1er décembre 2016Formation à la Réalité Virtuelle - La Plateforme 1er décembre 2016
Formation à la Réalité Virtuelle - La Plateforme 1er décembre 2016
 
Sotrycode Grand Ouest 1er juillet - La Réalité Virtuelle, un média global
Sotrycode Grand Ouest 1er juillet - La Réalité Virtuelle, un média globalSotrycode Grand Ouest 1er juillet - La Réalité Virtuelle, un média global
Sotrycode Grand Ouest 1er juillet - La Réalité Virtuelle, un média global
 
Screen4AllForum UltraHDay - Analyse du marché de l’UHD - Marc Bourhis Diginov
Screen4AllForum UltraHDay - Analyse du marché de l’UHD - Marc Bourhis DiginovScreen4AllForum UltraHDay - Analyse du marché de l’UHD - Marc Bourhis Diginov
Screen4AllForum UltraHDay - Analyse du marché de l’UHD - Marc Bourhis Diginov
 
Management de l'information et veille stratégique - IPAG - Manelle Guechtouli
Management de l'information et veille stratégique - IPAG - Manelle GuechtouliManagement de l'information et veille stratégique - IPAG - Manelle Guechtouli
Management de l'information et veille stratégique - IPAG - Manelle Guechtouli
 
Reallocation bande 700_mhz_livre_blanc
Reallocation bande 700_mhz_livre_blancReallocation bande 700_mhz_livre_blanc
Reallocation bande 700_mhz_livre_blanc
 
Dossier de l'écran n°1 Ultra HD et 4k à la télévision et au cinéma
Dossier de l'écran n°1 Ultra HD et 4k à la télévision et au cinémaDossier de l'écran n°1 Ultra HD et 4k à la télévision et au cinéma
Dossier de l'écran n°1 Ultra HD et 4k à la télévision et au cinéma
 
Présentation 4 mars au cnc CST-rt30 -sécurisation des données numériques tour...
Présentation 4 mars au cnc CST-rt30 -sécurisation des données numériques tour...Présentation 4 mars au cnc CST-rt30 -sécurisation des données numériques tour...
Présentation 4 mars au cnc CST-rt30 -sécurisation des données numériques tour...
 
Riam bilan 2011 2013 - projets retenus
Riam   bilan 2011 2013 - projets retenusRiam   bilan 2011 2013 - projets retenus
Riam bilan 2011 2013 - projets retenus
 
Livre blanc la+securisation+des+donnees
Livre blanc la+securisation+des+donneesLivre blanc la+securisation+des+donnees
Livre blanc la+securisation+des+donnees
 
Présentation Ficam de la filière production 4K/UHD devant le HD Forum le 18 d...
Présentation Ficam de la filière production 4K/UHD devant le HD Forum le 18 d...Présentation Ficam de la filière production 4K/UHD devant le HD Forum le 18 d...
Présentation Ficam de la filière production 4K/UHD devant le HD Forum le 18 d...
 
Spivet workflow33
Spivet workflow33Spivet workflow33
Spivet workflow33
 
Spivet workflow23
Spivet workflow23Spivet workflow23
Spivet workflow23
 
Spivet workflow13
Spivet workflow13Spivet workflow13
Spivet workflow13
 
Vue synthétique du workflow Cinema par Franck Montagné
Vue synthétique du workflow Cinema par Franck MontagnéVue synthétique du workflow Cinema par Franck Montagné
Vue synthétique du workflow Cinema par Franck Montagné
 
Présentation Muriel Lebellac Videomenthe sur son offre Cloud Computing lors d...
Présentation Muriel Lebellac Videomenthe sur son offre Cloud Computing lors d...Présentation Muriel Lebellac Videomenthe sur son offre Cloud Computing lors d...
Présentation Muriel Lebellac Videomenthe sur son offre Cloud Computing lors d...
 
Présentation Cloud Computing Philippe Recouppé Président du Forum Atena lors ...
Présentation Cloud Computing Philippe Recouppé Président du Forum Atena lors ...Présentation Cloud Computing Philippe Recouppé Président du Forum Atena lors ...
Présentation Cloud Computing Philippe Recouppé Président du Forum Atena lors ...
 
Présentation colorimétrie & préservation François Helt (hts) Atelier focus In...
Présentation colorimétrie & préservation François Helt (hts) Atelier focus In...Présentation colorimétrie & préservation François Helt (hts) Atelier focus In...
Présentation colorimétrie & préservation François Helt (hts) Atelier focus In...
 
Présentation du SOA et BPM par Rs2i_AtelierFocusInnovation_06022014
Présentation du SOA et BPM par Rs2i_AtelierFocusInnovation_06022014Présentation du SOA et BPM par Rs2i_AtelierFocusInnovation_06022014
Présentation du SOA et BPM par Rs2i_AtelierFocusInnovation_06022014
 
Présentation des différents standards de fichiers Mezzanine_AtelierFocusInnov...
Présentation des différents standards de fichiers Mezzanine_AtelierFocusInnov...Présentation des différents standards de fichiers Mezzanine_AtelierFocusInnov...
Présentation des différents standards de fichiers Mezzanine_AtelierFocusInnov...
 

Dernier

Artificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and MythsArtificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and Myths
Joaquim Jorge
 

Dernier (20)

Powerful Google developer tools for immediate impact! (2023-24 C)
Powerful Google developer tools for immediate impact! (2023-24 C)Powerful Google developer tools for immediate impact! (2023-24 C)
Powerful Google developer tools for immediate impact! (2023-24 C)
 
Developing An App To Navigate The Roads of Brazil
Developing An App To Navigate The Roads of BrazilDeveloping An App To Navigate The Roads of Brazil
Developing An App To Navigate The Roads of Brazil
 
HTML Injection Attacks: Impact and Mitigation Strategies
HTML Injection Attacks: Impact and Mitigation StrategiesHTML Injection Attacks: Impact and Mitigation Strategies
HTML Injection Attacks: Impact and Mitigation Strategies
 
🐬 The future of MySQL is Postgres 🐘
🐬  The future of MySQL is Postgres   🐘🐬  The future of MySQL is Postgres   🐘
🐬 The future of MySQL is Postgres 🐘
 
The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024
 
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
 
A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?
 
Data Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt RobisonData Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt Robison
 
presentation ICT roal in 21st century education
presentation ICT roal in 21st century educationpresentation ICT roal in 21st century education
presentation ICT roal in 21st century education
 
Boost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdfBoost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdf
 
Real Time Object Detection Using Open CV
Real Time Object Detection Using Open CVReal Time Object Detection Using Open CV
Real Time Object Detection Using Open CV
 
Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024
 
Artificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and MythsArtificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and Myths
 
[2024]Digital Global Overview Report 2024 Meltwater.pdf
[2024]Digital Global Overview Report 2024 Meltwater.pdf[2024]Digital Global Overview Report 2024 Meltwater.pdf
[2024]Digital Global Overview Report 2024 Meltwater.pdf
 
2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...
 
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
 
GenAI Risks & Security Meetup 01052024.pdf
GenAI Risks & Security Meetup 01052024.pdfGenAI Risks & Security Meetup 01052024.pdf
GenAI Risks & Security Meetup 01052024.pdf
 
AWS Community Day CPH - Three problems of Terraform
AWS Community Day CPH - Three problems of TerraformAWS Community Day CPH - Three problems of Terraform
AWS Community Day CPH - Three problems of Terraform
 
Handwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed textsHandwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed texts
 
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
 

Ltfs new-archive-workflow-whitepaper

  • 1. White Paper The New Archive Workflow: Accessible, Online and Protected Tom Coughlin Coughlin Associates www.tomcoughlin.com March 2013
  • 2. The New Archive Workflow: Accessible, Online and Protected Video gives us a way to visualize the dynamic relationships of the world around us. Higher resolution video at higher Introduction frame rates lets us experience our world at even greater detail and with a greater potential emotional impact. Combine higher resolution with higher frame rates as well as input from multiple cameras and the demand for digital storage for professional media and entertainment explodes, particularly for original or master content. Today some video projects produced with high frame rates and high resolution run a few petabytes (PB) in size. In the not too distant future the size of higher resolution, multiple camera video projects could be 100 times that size. As the size and complexity of master and supporting preserved original content increases (raw captured content size could grow close to 500 X in the next few years due to higher resolution and frame rate as shown in Table 1), the digital archival systems used to preserve this content and make it available for use must become more intelligent. They also have to scale to meet the needs of growing content libraries and provide an easy path to continued data access as well as data protection over the life of the archive. Table 1. Example Resolution, Data Rates and Storage Capacity Requirements for Professional Media Content Resolution Frame Rate Data Rates Storage Capacity/ Format (width X height) (fps) (MBps) Hour (GB) SDTV (NTSC, 4:2:2, 720 X 480 ~30 6.25 22 8-bit) HDTV (1080p, 4:2:2, 1920 X 1080 24 49.8 179 8-bit) Digital Cinema 2K 2048 X 1080 24 199 716 (4:2:4, 10-bit) YUV Digital Cinema 4K 4096 X 2160 48 1,910 6,880 (4:4:4, 12-bit) YUV Digital Cinema 8K 7680 X 4320 120 23,890 86,000 (4:4:4, 16 bit) 2012 Digital Storage in Media and Entertainment Report, Coughlin Associates, www.tomcoughlin.com “Super Hi-Vision” Video Parameters for Next Generation Television, SMPTE Motion Imaging Journal, May/June 2012, P. 63-68 As shown above, the storage capacity needed for high-resolution imaging is monumental, and it’s growing every day. Traditional offline archiving does not meet the fast-paced demands of modern digital workflows. Video professionals need a real-time, online way to simultaneously access stored content. This paper will examine the role of digital active archiving technology in professional media and entertainment workflows. We shall show the use of LTO tape during a typical media workflow and in an active content archive. Examples of how a content library or a content archive use LTO tape for professional content will be given and the important role that the LTFS file system in LTO tape provides for content storage will be explained. While LTFS and LTO are important technologies for the creation of file-based archives, on their own, they are not enough. Thus, we also need to look at intelligent storage architectures and solutions that harness LTFS to deliver data protection, scalability and manageability for massive data volumes. We will conclude with some thoughts on the future of content archiving and the growing value of an intelligent preservation archive.
  • 3. The New Archive Workflow: Accessible, Online and Protected Archiving (and digital tape) plays an important role in professional content storage from the very point of creation Alive in the Archive: Tape’s Modern Role in Media Workflows through distribution. In large movie projects that involve significant investor money, production houses must create a copy of new raw content on archive media as well as production media. Digital magnetic tape, usually LTO tape, is used in the field to create archival quality copies when the digital footage is downloaded onto a hard disk drive from the camera content media. This is particularly true for expensive solid-state camera cartridges where the cartridges are cycled quickly for reuse. Several media production support equipment companies make storage systems that are used in the field for making copies of content on magnetic tape and HDDs and thus meet these bonding requirements. Likewise, in a post-production facility for film or television, magnetic tape is often used as working archive behind a HDD (and possibly even some flash memory) storage array that serves video workstations. These various storage media are used to optimize a storage solution using the trade-off in performance versus storage costs of the various storage technologies. Flash memory can be fast but the cost per GB of storage capacity is often ten times or more expensive than HDD storage. Magnetic tape is less expensive than HDD storage, particularly in normal operating costs (e.g system power and HAVC costs) and the storage media life of magnetic tape is longer than HDDs. In a larger facility using network storage this archiving storage tier would likely be closely integrated with the network storage as shown in Figure 1. The LTFS file system on LTO tape allows the creation of file and even object-based archives using this low-cost storage media. LTFS is useful for media and entertainment applications since file access makes LTO tape act like a storage volume (similar to a HDD or USB drive). File information makes content on tape (metadata) viewable without special applications. As a consequence, it is easier to use LTO tape in workflows. Increased data mobility using file formats (such as the LTFS) makes sharing across workflows easier, and thus magnetic tape can be an important element in a M&E “sneaker net.“ Figure 1. Video Post Production With Networked Active Archive Storage
  • 4. The New Archive Workflow: Accessible, Online and Protected Some media facilities use magnetic disk storage for their content archives. Magnetic tape can be much more cost effective, particularly because the operating costs of tape sitting on a shelf in a tape library is less than spinning disks in drive arrays. As shown in Figure 2, over a 20 year retention time the total costs of storing content on hard disk drives is more than twice as large as storing it on magnetic tape in a tape library. LTFS has generated a lot of interest from content creators, even smaller facilities, for long-term tape archives. Because LTO-5 or 6 tapes can be used across multiple platforms, they can be used in various parts of the workflow. LTO can replace pro-video tape archives, and it is less expensive to operate than disk-based storage. LTO tape-based archives are playing a crucial role in cost-effective archive storage because of the physical life of the storage media, the backward compatibility of LTO tape formats and their low operating costs for long-term storage. Figure 2. Comparison of Estimated Annual Cost to Save 1 PB for 20 Years on HDDs Arrays and a Tape Library1. But tape, by itself, cannot deliver the rapid access to data needed by media and entertainment facilities. That’s where an intelligent active archive management solution is needed. One such solution, the StrongBox from Crossroads Systems, uses a disk caching layer to enable multiple, simultaneous accesses to an underlying tape library. These factors have made LTO tapes popular with many video content libraries. For example, Fox Studios uses LTO tape Best Practices for Protected, Accessible Tape Archives for archiving all their high definition content. We will look at a couple of other examples of LTO tape with intelligent storage for professional media and entertainment from boutique post-production facilities to major broadcast networks. ProductionFor is an Austin, TX based production and post-production company that provides media acquisition, editing and customer animation graphics. Data from a single project can start with hundreds of source files taking up to a terabyte of storage capacity. Editing this content increased the storage capacity demand and required a scalable cost- effective long-term storage system that could handle massive video files. A Crossroads StrongBox tape-based NAS storage appliance using LTO tape was found to be an effective repository for high-volume long-term post-production content. Multiple copies on several pieces of media stored at separate locations provided protection of the archived content. StrongBox’s multi-copy, hash-coding and system monitoring helped ensure that the video content data was securely protected and accessible. The LTFS-enabled file structure allowed content on the tape archive to be easily available to ProductionFor’s file-based workflows. This enabled single file access availability at the click of a mouse.
  • 5. The use of LTFS file system allowed any LTFS media to be kept in the StrongBox, whether it was created by ProductionFor or not, and independent of the operating systems used. This allowed greater collaboration with content on LTO tapes and meant the storage content could last beyond the life of the production system. Major League Baseball (MLB) Network is the source for premier baseball content including live games, studio shows and feature programming. A custom developed StrongBox LTO storage system was installed in late 2011 to provide a portable data vault for long-term file retention. The internal HDD cache enabled rapid file access, and the LTFS file system enabled drag-and-drop file movement simultaneously for multiple users. The MLB Network logs and categorizes HD recordings by viewing proxy video files, recorded in real-time. Most of this HD video is stored in the LTO library and is searched using DIAMOND and the Grass Valley Aurora system. MLB Network is looking to store the long-tail proxy content on LTO tape as well. Archiving plays a critical role in video content workflows, from content capture, through post-production and for long- Conclusions term preservation and library access. LTO tapes offering the LTFS file system provide file-based storage that allows tape to be partitioned and used for project based storage of high resolution original content as well as transcoded files for distribution networks or proxy files. Archiving on magnetic tape results in significant cost reductions over the life of a digital archive. The operating cost advantages of LTO tape storage and the file access of the LTFS file system make intelligent archiving solutions, like StrongBox, an important element in a video facility’s media asset management, real-time content access, disaster recovery strategy, and in a preservation repository. As the size and complexity of professional video projects increases in coming years, file-based and even object-based access to well-organized content will be even more critical to successful video production and broadcast. Future magnetic tape technology will continue to develop both in storage capacity as well as data accessibility. Tom Coughlin, President, Coughlin Associates is a widely respected storage analyst and consultant. He has over 30 years About the Author, Tom Coughlin in the data storage industry with multiple engineering and management positions at high profile companies. Dr. Coughlin has many publications and six patents to his credit. Tom is also the author of Digital Storage in Consumer Electronics: The Essential Guide, which was published by Newnes Press. Coughlin Associates provides market and technology analysis (including reports on several digital storage technologies and applications and a newsletter) as well as Data Storage Technical Consulting services. Tom publishes the Digital Storage Technology Newsletter, the Media and Entertainment Storage Report, and the Capital Equipment and Technology for the Hard Disk Drive Industry Report. Visit www.tomcoughlin.com. Crossroads Systems, Inc. is a global provider of solutions and services that ensure stored data is proactively protected About Crossroads Systems and StrongBox and reliably recovered. Crossroads offers organization powerful data protection, proactive data security, intelligent storage connectivity, unmatched performance, and significant cost savings. Crossroads was founded in 1996 and is based in Austin, TX and holds more than 100 patents granted and pending. StrongBox is a shared storage solution purpose-built for data archival and preservation. Using Linear Tape File System (LTFS) technology and intelligent storage architecture with standard file systems (CIFS/NFS), StrongBox empowers online, all the time, file availability with uncompromised data protection, full data mobility, non-proprietary file storage and significant cost-savings. StrongBox unifies your archive - past and present - while providing seamless scalability and reliability for the future. Visit Crossroads at: www.crossroads.com © Coughlin Associates 2013 All trademarks are the property of their respective owners.