Revolutionizing the Digital Transformation Office - Leveraging OnePlan’s AI a...
Project update - João Fernandes
1. ARCHIVER – Project Update
Open Market Consultation
5th of June - CERN
João Fernandes
Project Coordinator
CERN
2. Pre-Commercial Procurement
07.05.2019 ARCHIVER Project Overview 2
Focus: Archiving and Data Preservation Services using commercial cloud services
Procurement Budget: 3.4M euro
Starting Date: 1st of January 2019
Duration: 36 Months
Coordinating Partner: CERN
3. Consortium
Includes Buyers and Experts in the preparation, execution and promotion of the Procurement of R&D
07.05.2019 ARCHIVER Project Overview 3
Procurers - Public organisations committing funds to contribute to a joint-procurement
Experts – Partner organisations bringing expertise in requirement assessment and promotion activities, not part of the Buyers Group
4. Challenge
Data volumes and complexity growing; Communities Growing; Needs of advanced functionality increase
Objective: perform R&D to demonstrate long-term preservation and archiving services for scientific data in the PB range under
F.A.I.R. principles, using commons solutions, while research groups keep total stewardship of their data sets
Scalability: PB data volume of multi domain scientific data; High sustained ingest rates (1-10 Gbps)
OAIS, FAIR and European Regulation (GDPR)
European Open Science Cloud (EOSC) integration
Reduce fragmentation expanding existing solutions to several scientific domains
Access through GÉANT network; Federated AAI support
Cost-effective and transparent business models: range of purchasing and deployment options, stimulating uptake by the wider
research community
07.05.2019 ARCHIVER Project Overview 4
Basic Scientific Data
Archiving & Bit
Preservation
5. Cost-effective
Business Model
Taking into account:
- Scale
- Ingest rates
- Archive lifetime
- # of copies
- Exit strategies
- Portability
- SLAs
Regulation &
Legislation
- Auditing
- Self-assessment
- Data Retention
- GDPR
R&D Challenge
07.05.2019 ARCHIVER Project Overview 5
Data integrity/security; cloud/hybrid deployment; data
volume in the PB range; high, sustained ingest data rates;
ISO certification: 27000, 27040, 19086 and related
Archives connected to the GEANT network.
OAIS conformant services: data readability formats, normalization,
obsolesce monitoring, files fixity, authenticity checks, etc.;
ISO 14721/16393, 26324 and related standards
User services: search, discover, share, indexing, data removal, etc.;
Access under Federated AAI
Layer 1
Storage/Basic Archiving/Secure
backup
Layer 2
Preservation
Layer 3
Baseline user services
Layer 4
Advanced services
High level services: visual representation of data (domain specific),
reproducibility of scientific analyses, etc.;
Minimum
R&D
Objectives
Added
value
R&D
Fundamental aspect
of the challenge
6. Market Potential
07.05.2019 ARCHIVER Project Overview 6
ESFRI Landmarks
RIs already implemented or in an advanced Implementation Phase.
ESFRI Projects
RIs in a preparation phase selected for scientific case excellence and maturity.
http://roadmap2018.esfri.eu/media/1049/roadmap18-part3.pdf
7. ARCHIVER & European Open Science Cloud (EOSC)
07.05.2019 ARCHIVER Project Overview 7
ARCHIVER Services can be procured
2019 2020 2021 2022 2023
EOSC vision:
To give the European Union a global lead in research data management and ensure that European
scientists reap the full benefits of data-driven science
ARCHIVER Objective: to make resulting services available in the EOSC catalogue
EOSC Rules of Engagement evolving: will be presented during the the project
8. ARCHIVER timeline
07.05.2019 ARCHIVER Project Overview 8
Suppliers to sign a framework agreement in order to participate in the call-offs
Goal: Services to be commercialized after the end of the PCP
We are here
10. Open Market Consultation Kick-Off event – 8th of April
07.05.2019 ARCHIVER Project Overview 10
70 participants from 26 companies physically
present
Digital Preservation Experts, Cloud Providers, Software
integrators
42 participants connected remotely (via webcast)
Initial list of deployments and challenges presented
Initial planning poker to assess Value vs. Risk in a
direct dialogue with companies
All material available at:
https://www.archiver-project.eu/open-market-consultation-kick-event
11. Demand side activity
07.05.2019 ARCHIVER Project Overview 11
EOSC-hub week demand side event
Overview of current EOSC activities
Session on role of Procurement in the EOSC context
State-of-the art and landscaping discussions
Number of organisations expressing interest in ARCHIVER
Early Adopters
OAIS Webinar – 9th of May
All material available at:
https://www.archiver-project.eu/all-events
12. Open Market Consultation event – 7th of May Barcelona
07.05.2019 ARCHIVER Project Overview 12
40 participants from 17 companies physically
present
Digital Preservation Experts, Cloud Providers,
Software integrators
Focused planning poker in a subset of atomic use
cases
All material available at:
https://www.archiver-project.eu/open-market-consultation-kick-event
13. Value vs Risk (Planning Poker)
04/06/2019 ARCHIVER Project Overview 13
• Management of scientific datasets
• Portability of Archives / Exit Strategies
• On-prem deployments
• Replication of archives on multiple sites
• Very high sustained ingests
• Personal data management
15. Open Market Consultation event – 23rd of May Stansted (UK)
07.05.2019 ARCHIVER Project Overview 15
40 participants from 15 companies physically
present
Digital Preservation Experts, Cloud Providers,
Software integrators
Focused on technical details of deployment scenarios
All material available at:
https://www.archiver-project.eu/open-market-consultation-kick-event
16. Network connectivity
Overview of the GEANT network connectivity options
Current List of peers
04/06/2019 16
Source: Enzo Capone, vincenzo.capone@geant.org (GÉANT)
Peer Name Location Capacity
AWS Vienna, Frankfurt 10G + 10G
T-Systems Frankfurt, Amsterdam 10G + 10G
Dimension Data Amsterdam 20G
Exoscale Geneva, Frankfurt 10G+10G
Microsoft Vienna, Dublin, London
10G+10G+
10G
All material available at:
https://www.archiver-project.eu/open-market-consultation-kick-event
17. Federated AAI
04/06/2019 17
All material available at:
https://www.archiver-project.eu/open-market-consultation-kick-event
Source: Hannah Short, hannah.short@cern.ch (CERN on behalf of the GÉANT Project)
18. Stansted Feedback
Webinars from Suppliers
The Buyers Group encourages potential bidders to show their current state-of-the-art via
recorded webinars
04/06/2019 18
19. Draft Tender Documents Published
Draft versions of the tender documents are available during the OMC process,
including:
RfT, Framework Agreement and Functional Specification;
Annex 2 (Selection Criteria); Annex 4 (Award Criteria);
Initial list of Milestones and Deliverables;
Q&A from the OMCs also publicly available
The Buyers Group encourages you to make comments
07.05.2019 ARCHIVER Project Overview 19
https://www.archiver-project.eu/draft-tender-documents
20. Preparation of a technical summary for each deployment
Problem to solve
Lifecycle and workflow characteristics
Including basic and “bonus” mapping on the 4-Layer high level architecture
Authentication and Management
Data and Metadata description
Interface
Compliance requirements
Cost-effectiveness requirements
Some including minimum cost thresholds
04/06/2019 ARCHIVER Project Overview 20
21. Timeline for what’s left before publication
5th June: OMC Consolidation event
6th of June: Update of draft tender documents available online
10th of June: Publication of technical summaries of each use case/deployment
14th of June: Deadline to receive comments on the tender documents
21st of June: Deadline to receive recorded webinars of the current state of the art
21st June - 05th July: Draft documents internal Buyers Group reviews. All FAQs to
be published on the ARCHIVER website.
Early October: PCP Contract Notice published
04/06/2019 21
22. Agenda for Today
DESY/XFEL Deployment Scenario, Martin Gasthuber/Sergey Yakubov (DESY)
Deployment Scenarios Summary, Vaggelis Motesnitsalis, Technical Coordinator (CERN)
Q&A on Deployment Scenarios and Functional Requirements
15.00 - 15.30 – Coffee Break
Tender Process and Requirements, Joshua Luke Davison, Procurement Officer (CERN)
Overview of the EOSC, Bob Jones (CERN)
Wrapping up and Next Steps, Marion Devouassoux (CERN)
Event ends at 17:00
07.05.2019 ARCHIVER Project Overview 22
23. Summarizing, what’s in it for you?
07.05.2019 ARCHIVER Project Overview 23
Demonstrate archiving & preservation services for petabyte data volumes, high
ingest sustained data rates (1-10 Gbps per day) for multi-disciplinary scientific data
Diversify your offer and commercialize resulting services by the end of the project
Support the open data movement
Offer services via the European Open Science Cloud (EOSC) Catalogue
Access the public funded European Research Infrastructures (RIs) as a new market
segment