AWS Media and Entertainment Cloud Symposium - Managing digital assets in a serverless architecture

1. Managing Digital Assets in a
Serverless Architecture
Chris Elleman – Manager, Solutions Architect (Media Practice)
Erik Åhlin – Founder and CEO, Vidispine
March 2016

2. We see lots of customers at different points on the spectrum
File Based Management Asset Management
Challenges of Managing Content

3. Integrated services for ingestion, annotation, cataloguing, storage, retrieval and
distribution of digital assets (photos, video, audio, documents)
CMS DAM MAM
Documents
Web Content
Photos
Creatives
Models
Video
Audio
Wordpress
Drupal
Censhare
Escenic
Adobe CQ
OpenText CHP
T3 Media/Wazee
Vidispine
Dalet Examples
Creation
Distribution
Edit
Repurpose
Review
Approve
Asset Management Definition

4. • Gap: Reconciling Legacy vs. Born-In-The-
Cloud
– Large enterprises are challenged with
keeping legacy systems and processes in-
house
– Need to share/collaborate is strong Legacy
DAM
S3 Glacier
DAM DAM DAM
Asset Management Gaps (1)
• Solution: Federate DAM solutions
Edge DAMs + Core DAMs
Deep Storage on AWS

5. • Gap: Reconciling Content Gravity
– For heavy-duty DAM/MAM
requirements the volume and mass of
the generated content makes file-
based workflows very painful
• Solution: co-locating the
creative/edit/render/review process
with the data
– Amazon AppStream
– GPU Enabled VDI (AWS Workspaces,
Teradici)
– Others: browser based tooling (edits,
modeling, correction)
S3(ex: Avere,SoftNAS)
DAM
Virtual
Creative
Apps
Asset Management Gaps (2)

6. Online
Storage
Catalog
Management
Proxy &
Transcode
Processing
Search | Collaboration
Services
Process Management | Workflow Services
Nearline
Storage
Offline
Storage
Import / Export
Services
Tools
Adapter
Ingest
Services
Security | Rights Management Services
CDN
File Transfer
WAFS
Architecture Concepts – Digital Asset Management

7. Online
Storage
Catalog
Management
Proxy &
Transcode
Processing
Search | Collaboration
Services
Process Management | Workflow Services
Nearline
Storage
Offline
Storage
Import / Export
Services
Tools
Adapter
Ingest
Services
Security | Rights Management Services
CDN
File Transfer
WAFS
Standard S3-IA Glacier
S3 Glacier
Architecture Concepts – with AWS Services

8. Online
Storage
Catalog
Management
Proxy &
Transcode
Processing
Search | Collaboration
Services
Process Management | Workflow Services
Nearline
Storage
Offline
Storage
Import / Export
Services
Tools
Adapter
Ingest
Services
Security | Rights Management Services
CDN
File Transfer
WAFS
Standard S3-IA Glacier
S3 Glacier
Cognito Identity & Access
Management
Architecture Concepts – with AWS Services

9. Online
Storage
Catalog
Management
Proxy &
Transcode
Processing
Search | Collaboration
Services
Process Management | Workflow Services
Nearline
Storage
Offline
Storage
Import / Export
Services
Tools
Adapter
Ingest
Services
Security | Rights Management Services
CDN
File Transfer
WAFS
Standard S3-IA Glacier
S3 Glacier
Cognito Identity & Access
Management
DynamoDB
Architecture Concepts – with AWS Services

10. Online
Storage
Catalog
Management
Proxy &
Transcode
Processing
Search | Collaboration
Services
Process Management | Workflow Services
Nearline
Storage
Offline
Storage
Import / Export
Services
Tools
Adapter
Ingest
Services
Security | Rights Management Services
CDN
File Transfer
WAFS
Standard S3-IA Glacier
S3 Glacier
Cognito Identity & Access
Management
DynamoDB
Elastic Transcoder & Elemental
Architecture Concepts – with AWS Services

11. Online
Storage
Catalog
Management
Proxy &
Transcode
Processing
Search | Collaboration
Services
Process Management | Workflow Services
Nearline
Storage
Offline
Storage
Import / Export
Services
Tools
Adapter
Ingest
Services
Security | Rights Management Services
CDN
File Transfer
WAFS
Standard S3-IA Glacier
S3 Glacier
Cognito Identity & Access
Management
DynamoDB
Elastic Transcoder & Elemental
API
GatewayLambda
Architecture Concepts – with AWS Services

12. Online
Storage
Catalog
Management
Proxy &
Transcode
Processing
Search | Collaboration
Services
Process Management | Workflow Services
Nearline
Storage
Offline
Storage
Import / Export
Services
Tools
Adapter
Ingest
Services
Security | Rights Management Services
CDN
File Transfer
WAFS
Standard S3-IA Glacier
S3 Glacier
Cognito Identity & Access
Management
DynamoDB
Elastic Transcoder & Elemental
API
GatewayLambda
ElasticSearch
Architecture Concepts – with AWS Services

13. Online
Storage
Catalog
Management
Proxy &
Transcode
Processing
Search | Collaboration
Services
Process Management | Workflow Services
Nearline
Storage
Offline
Storage
Import / Export
Services
Tools
Adapter
Ingest
Services
Security | Rights Management Services
CDN
File Transfer
WAFS
Standard S3-IA Glacier
S3 Glacier
Cognito Identity & Access
Management
DynamoDB
Elastic Transcoder & Elemental
API
GatewayLambda
ElasticSearch
AWS Import/
Export Snowball
Architecture Concepts – with AWS Services

14. Ingest
Bucket
CloudFormation
Template
DynamoDB
Cognito
IAM
role
Lambda
MetaData
Extract
Asset Bucket
multimedia
User
Index
HTML5
Static Site
Elastic
Transcoder
IAM
ElasticSearch
API
Gateway
Lambda
App Logic
SimpleDAM – Serverless Architecture
Glacier
Archive
JS SDK
Meta Data
Searches

15. Ingest
Bucket
Multimedia
file
1. S3 Upload
2. S3 Event
Lambda
Function
4. Transcode Job
Lambda
Sub-Function
(metaDataExtract)
Elastic
Transcoder
3. Extract MetaData 5. Build source XML
Lambda
Sub-Function
(metaTransform)
+
DynamoDB
6. ETL Job
format data for DynamoDB
7. Create
DynamoDB record
Asset
Bucket
9. Copy Asset Into Asset Bucket
ElasticSearch
8. Index Content
SNS
Topic
10. On Success or Failure,
Remove Ingest Object
Lambda
Function
SimpleDAM – Video Ingest Process

16. Key Decision for a DAM – Buy or Build?
Buy
• Lack of free development
resource or skills
• Exiting on premise commercial
relationship
• DAM capabilities aren’t a market
differentiator
• Broad set of requirements
• Fast to set-up and provision
Build
• Development resource and skills
available in-house
• Lack of cap’ex budget for
licensing
• DAM capabilities will be a
market differentiator
• Narrow or specialist
requirements
• Start with an MVP and iterate
with the needs of the business

17. ©Amazon.com, Inc. and its affiliates. All rights reserved.
Thank you

18. Vidispine -
Cloud Native Content Management for
Developers and Media Professionals
Erik Åhlin, CEO, co-founder

19. Vidispine in 15s – the problem we solve
Broadcasters and media companies are looking for
• Incorporate cloud and cloud services in a pace the budget
allows. An evolution rather than a revolution.
• Complete, managed and real elasticity for video content to
control cost but still being agile
• Repurposing content to a much higher degree
• Fast turn-around story-telling with video and images on all
platforms
• Flexibility in business models and technology choices

20. Vidispine API-based Content Management
PaaS
Fully featured back-end for any media application
Key areas
• Metadata
• Multi-format management
• Performance & Scalability
• Cloud native architecture (componentized and distributed)
• Support for Multi-application

21. VidiXplore – The Gravity Point for your content
”Easy to Like” user experience
Support for core media management tasks
Extensible
Component in a ”bigger picture”
SaaS (private or public)
Leverage Vidispine Content Management PaaS

22. Combine on-prem & cloud - start your
migration
1. Place Vidispine Server Agent (VSA) next to existing on-prem
storage
2. Create proxy, analyze metadata, connect to MAM/DAM locally
3. Keep link between ’cloud-side’ asset and ’on-prem’ asset
4. Start execute your cloud strategy by doing what cloud does best
5. Migrate infrastructure in your pace means you can focus on
business

23. VidiXplore – The Gravity Point for your content
VIDISPINE PaaS with APIs
Transcoder
& QC
VDA
Storage
On-Prem
Infrastructure
Vidispine
Server
Agent
Cloud Services
VSAs
and/or Biz
Destinations

24. Real World Example #1
Global Premium Content Delivery
Spread out workforce and clients
Cost efficient and adaptive workflows

25. VidiXplore at Premium Content Delivery Leader
VidiXplore with Custom Panels
for managing audio tracks,
mezzanine files, delivery points,
etc
Amazon
EC2
Amazon
CloudFront
Amazon
S3 w SSE
hosting
8 TB proxies
65 GB
thumbnails
Amazon
VPC
Amazon
Route 53
On-Prem
VSAs
connected
to local
storage
Vidispine
APIs for
integration
to workflow
engine and
more
Data & Metadata to Vidispine PaaS
Proxies direct to S3 for performance gain
Amazon
RDS

26. Migrate and Scale
By knowing your content you are ready to scale
Spend wisely - let cloud do what cloud do best
• Workflows, common user interface for spread-out workforce
• Distribute and share ready content
• Store/archive and ”second copy”
• Burst-out transcoding and compute
Integrate and leverage popular cloud services
• Dropbox, Slack, Box.com, WeVideo, CloudConvert, Metadata
harvesting
Migrate, Scale, Adapt also Business Processes, Functions,

27. Real World Example #2
Partner Company

28. Alfred – the Dev Ops Butler
Partner Company: DSB
Using Vidispine dockerized
Scaling, Logging, Action
Packs
AWS Services used
• EC2 Container Service
• Docker
• Node.Js
• MongoDB

29. Real World Example #3
Leading US-based Media Giant
Quickly prototyped a complete DAM workflow (Clip Library)
Validated the technology stack on very low budget

30. Prototyping on AWS Marketplace
Leading US-based Media Giant
10+ MAMs and DAMs already
Some workflows are fit and ready for cloud
Overview
• Total POC went 6 weeks – total of 4 developers.
• Total assets added (5 thousand).
• Used the bare minimum box that Vidispine AMI Trial would run
on (m3.medium)

31. Prototyping on AWS Marketplace
Next steps
• Run Vidispine as HA, where SOLR runs in another EC2 and Postgres
runs in RDS
• Performance testing with hundreds of thousands of assets
Resources used
• AWS EC2 (website PoC ran here)
• AWS RDS (Postgres DB)
• AWS S3 (hot folder in/out)
• Vidispine Developer Edition AMI
• Bitbucket (PoC code)
• Jenkins (for deploying to EC2)
• MEAN stack(code stack)

32. What’s next?
Editing
• Some very promising prototypes using AWS WorkSpaces
Expert systems based on machine learning
• Systems adapting to the task at hand rather than how it was
originally designed
Large-scale metadata harvesting
True ubiquitous processing and storing
• What is a file, what is a computer?
• What to we really want computers to do for us and how?
Amazon
WorkSpaces
Amazon Machine
Learning
AWS
Lambda