SlideShare une entreprise Scribd logo
1  sur  22
Télécharger pour lire hors ligne
NRP & the Path forward
Frank Würthwein
Director, San Diego Supercomputer Center
February 9th 2023
Vision
NATIONAL RESEARCH
3
https://mindsweneed.org
Democratize Access
NATIONAL RESEARCH
NATIONAL RESEARCH
Long Term Vision
• Create an Open National Cyberinfrastructure that
allows the federation of CI at all ~4,000 accredited,
degree granting higher education institutions, non-
profit research institutions, and national laboratories.
§ Open Science
§ Open Data
§ Open Source
§ Open Infrastructure
4
Openness for an Open Society
Open Compute
Open Storage & CDN
Open devices/instruments/IoT, …?
NATIONAL RESEARCH
NATIONAL RESEARCH
Community vs Funded Projects
5
Community with
Shared Vision
Lot’s of funded projects that
contribute to this shared
vision in different ways.
We want you to …
… grow NRP.
… build on NRP.
NRP is “owned” and “built” by the community for the community
How is NRP different from OSG?
We’ve been doing federated cyberinfrastructure since 2005.
Why is NRP even needed given that OSG exists?
NATIONAL RESEARCH
HTCondor/OSG
Cyberinfrastructure Stack
7
Hardware
IPMI, Firmware, BIOS
Kubernetes
Admiralty
SLURM
NRP operates at all layers of the stack, from IPMI up
• IPMI reduces TCO and lower threshold to entry
• Kubernetes allows service deployments
• Also the natural layer for application container deployment
• Admiralty allows K8S federation with folks who want control
• Including cloud integration to access TPUs & other cloud only architectures
• HTCondor allows NRP to show up as a “site” in OSG
The layer you integrate at depends on
• Control you want
• Effort you can afford
Complementarity in Implementation of
“Bring Your Own Resource” model
OSG/PATh focused on campus cluster integration.
NRP focused on individual node integration instead of clusters.
NATIONAL RESEARCH
HTCondor/OSG
Cyberinfrastructure Stack
9
Hardware
IPMI, Firmware, BIOS
Kubernetes
Admiralty
SLURM
NRP operates at all layers of the stack, from IPMI up
• IPMI reduces TCO and lowers threshold to entry
• Kubernetes allows service deployments
• Also the natural layer for application container deployment
• Admiralty allows K8S federation with folks who want control
• Including cloud integration to access TPUs & other cloud only architectures
• HTCondor allows NRP to show up as site in OSG
• Under-resourced institutions
• Network providers and their POPs
• CS & ECE faculty specialized on:
• AI/ML => gaming GPUs
• systems R&D
All of these find it difficult to
justify staff to support all layers
NATIONAL RESEARCH
Hardware on NRP is Global
10
NRP integrates hardware in USA, EU, and Asia
NATIONAL RESEARCH
Cyberinfrastructure Stack
11
Hardware
IPMI, Firmware, BIOS
Kubernetes
NRP operates at all layers of the stack, from IPMI up
• IPMI reduces TCO and lowers threshold to entry
• Kubernetes allows service deployments
• Also the natural layer for application container deployment
• Admiralty allows K8S federation with folks who want control
• Including cloud integration to access TPUs & other cloud only architectures
• HTCondor allows NRP to show up as site in OSG
• Open Science Data Federation
• Origins & Caches in US, EU, Asia
• Protein Data Bank
• (Future) Replicas in EU & Asia
NRP is unique in its support of
global service deployments
OSDF ops by PATh PDB ops by PDB ???
Supporting Nautilus
for the next decade
Nautilus = K8S infrastructure of PRP for the last 5+ years
Nautilus = K8S of NRP for the next 10 years
NATIONAL RESEARCH
NATIONAL RESEARCH
The NSF Cat-II Program
• NSF supports via the Cat-II program novel systems ideas.
- 3 year “testbed” phase
§ The PI owns the resource, and has (some) freedom regarding who uses it.
Ÿ No requirements for making it available via any specific allocation mechanism.
§ It is expected that not all features work on day 1.
Ÿ 3 years of experimentation & development of features
- 2 year “allocation” phase
§ The resource is made available via an NSF supported allocation mechanism.
- The solicitation mentions the possibility of an additional 5 year
renewal without re-competition if system is successful.
• We decided that this is an ideal program to try and secure
NRP core operations funding for the next 10 years
- And thus provide the stability necessary for growth of NRP.
13
NATIONAL RESEARCH
NATIONAL RESEARCH
Cat-II: Prototype National
Research Platform (PNRP)
14
Funded as NSF 2112167
80GB A100
A10
64
288
5 year project with $5M hardware & $6.45M people
Supports Nautilus, and thus the core NRP infrastructure
Promises to build on “PRP” functionality, and go beyond
NSF Acceptance Review of System scheduled for March 8 & 9th 2023
PI = Wuerthwein; Co-PIs: DeFanti, Rosing, Tatineni, Weitzel
NATIONAL RESEARCH
NATIONAL RESEARCH
Innovations
• I1: Innovative network fabric that allows “rack” of hardware to
behave like a single “node” connected via PCIe.
• I2: Innovative application libraries to expose FPGAs hardware
to science apps at language constructs scientists understand
(C, C++ rather than firmware)
• I3: A “Bring Your Own Resource” model that allows campuses
nationwide to join their resources to the system.
• I4: Innovative scheduling to support urgent computing, including
interactive via Jupyter.
• I5: Innovative Data Infrastructure, including national scale
Content Delivery System like YouTube for science.
15
I3 & I4 & I5 turn ”PRP” into “NRP” and sustains it into the future.
I1 & I2 are totally new.
NATIONAL RESEARCH
NATIONAL RESEARCH
Data Infrastructure Model of NRP
• Support regional Ceph storage systems across the USA.
- Campuses can join individual storage hosts to the Ceph system in their region.
- All regional storage systems are Origins in OSG Data Federation (OSDF)
- Deploy replication system such that researchers can decide what part of their
namespace should be in which regional storage.
• Deploy caches in Internet2 backbone such that no campus nationwide is
more than 500 miles from a cache.
16
NRP data infrastructure model combines best of PRP & OSG
From PRP we take the regional Ceph storage concept
From OSG/PATh we take the data origin & caching concepts
And then we add as a totally new feature:
User controlled replication of partial namespaces across regions.
(We will develop this during 3 year “testbed” phase)
Want Others to build higher level data services on top
NATIONAL RESEARCH
NATIONAL RESEARCH
Matrix of Science x Innovations
17
NRP. The networks of researchers we create will use the NRP systems innovations to achieve broader im-
pact, with the help from our collaborators, users, staff, Co-PIs, and PI.
Table 3.1 Representative Science and Engineering Use Cases
Application domain
Lead researcher &
Institution
Science Driver
Themes NRP Innovations
LIGO
Peter Couvares, LIGO Lab;
Erik Katsavounidis, MIT
BGS, UC, AI I2, I3, I4, I5
IceCube Benedikt Riedel, UW Madison BGS, UC, AI I3, I4
Astronomy (DKIST &
Sky Surveys)
Curt Dodds, U. Hawai’i BGS, AI I3, I5,
Campus Scale Instru-
ment Facilities
Mark Ellisman, NCMIR; Sa-
mara Reck-Peterson, Nicon
Imaging Center; Johannes
Schoeneberg, Adaptive Op-
tics Lightsheet Microscopy;
Kristen Jepsen, Institute for
Genomic Medicine; Tami
Brown-Brandl, Precision Ani-
mal Management
SD, UC, H I1, I2, I3, I4, I5
Molecular Dynamics
Rommie Amaro, UCSD; An-
dreas Goetz, SDSC; Jona-
than Allen, LLNL
MD, AI, H I1, I2, I3
Human microbiome Rob Knight, UCSD G, AI, H I1, I2, I3
Genomics & Bioinfor-
matics
Alex Feltus, Clemson G, AI, H I3, I4, I5
Fluid Dynamics Rose Yu, UCSD AI I1, I2, I3
Experimental Particle
Physics, IAIFI Phil Harris, MIT AI, BGS, SD I1, I2
Computer Vision Nuno Vasconcelos, UCSD AI, CV I3
Computer Graphics Robert Twomey, UNL CV, AI I3
Programmable Storage Carlos Malzahn, UCSC SD I1, I2, I5
AI systems software
stack for FPGAs Hadi Esmaeilzadeh , UCSD SD I1, I2
WildFire Analysis &
Prediction
Ilkay Altintas, UCSD UC, AI, CV I3, I4
Key: The NRP innovations column lists those innovations among I1 through I5 listed in Section 2.1 that a
given science driver most benefits from.
NSF MREFCs
Incl. 4 campus scale
instrument facilities
Incl. a very diverse
set of sciences and
engineering
Lot’s of AI …
but so much more …
NATIONAL RESEARCH
NATIONAL RESEARCH
FKW’s Wishlist for the Future
• Growth of NRP infrastructure
- 1,000++ GPUs end of 2022
- 50 PB storage end of 2024
- Growth in diversity of community
§ # and types of campuses and their researchers
• Introduce new capabilities to NRP
- Machine learning at 100TB scale
- Support Domain Specific Architecture R&D on NRP
- Expand NRP into Wireless, Edge, IoT
- Towards “FAIR” on OSDF
• New Directions initiated by the Community
18
NATIONAL RESEARCH
NATIONAL RESEARCH
“Domain Specific Architectures”
• I1: Innovative network fabric allowing “composable hardware”.
• I2: Innovative application libraries allowing “domain optimized
architectures” on FPGAs
19
“end of Moore’s law” motivates new architectures
Mark Papermaster, CTO of AMD
PRISM, a Jump 2.0 project
funded by SRC
is early user of FPGAs@PNRP
John Shalf (2020)
https://doi.org/10.1098/rsta.2019.0061
PI, Tajana Rosing
NATIONAL RESEARCH
NATIONAL RESEARCH
New Data Origins
• The NSF CC* 2022 program awarded 9
campuses with $500k storage awards each.
- We guess this pays for 5PB of storage each.
• Some of these campuses may decide to
integrate their CC* storage into the OSDF.
• Some of these campuses have storage from
other projects that they may integrate with the
OSDF in addition.
• NSF 23-523 includes $500k storage
solicitation again, Spring & Fall 2023.
20
NATIONAL RESEARCH
NATIONAL RESEARCH
Summary & Conclusions
• PRP ended, and was replaced by NRP
- Significant new capabilities via Cat-II system “PNRP”
§ PNRP provides ops effort for Nautilus for the future
- # of GPUs available double in 2022.
§ new GPUs (A10, 3080, 3090, A100) much more powerful than older GPUs
- # of FPGAs increase from a few to a few dozen in 2022.
- # of caches grow by 50% in 22/23
=> more consistent coverage across USA
- Data volume served expected to grow substantially in 23/24/25.
§ How much? As yet too hard to predict.
• Hoping to recruit new partners to build FAIR capabilities on
top of OSDF within the next 5 years.
• Hoping to expand NRP into sensor networks using 5G &
6G in the next 10 years.
21
NATIONAL RESEARCH
NATIONAL RESEARCH
Acknowledgements
• This work was partially supported by the NSF
grants OAC-1541349, OAC-1826967, OAC-
2030508, OAC-1841530, OAC-2005369,
OAC-21121167, CISE-1713149, CISE-
2100237, CISE-2120019, OAC-2112167
22

Contenu connexe

Similaire à Frank Würthwein - NRP and the Path forward

Utilising Cloud Computing for Research through Infrastructure, Software and D...
Utilising Cloud Computing for Research through Infrastructure, Software and D...Utilising Cloud Computing for Research through Infrastructure, Software and D...
Utilising Cloud Computing for Research through Infrastructure, Software and D...David Wallom
 
Enabling Insight to Support World-Class Supercomputing (Stefan Ceballos, Oak ...
Enabling Insight to Support World-Class Supercomputing (Stefan Ceballos, Oak ...Enabling Insight to Support World-Class Supercomputing (Stefan Ceballos, Oak ...
Enabling Insight to Support World-Class Supercomputing (Stefan Ceballos, Oak ...confluent
 
Azure Brain: 4th paradigm, scientific discovery & (really) big data
Azure Brain: 4th paradigm, scientific discovery & (really) big dataAzure Brain: 4th paradigm, scientific discovery & (really) big data
Azure Brain: 4th paradigm, scientific discovery & (really) big dataMicrosoft Technet France
 
NSF Software @ ApacheConNA
NSF Software @ ApacheConNANSF Software @ ApacheConNA
NSF Software @ ApacheConNADaniel S. Katz
 
Cloud Standards in the Real World: Cloud Standards Testing for Developers
Cloud Standards in the Real World: Cloud Standards Testing for DevelopersCloud Standards in the Real World: Cloud Standards Testing for Developers
Cloud Standards in the Real World: Cloud Standards Testing for DevelopersAlan Sill
 
Looking Back, Looking Forward NSF CI Funding 1985-2025
Looking Back, Looking Forward NSF CI Funding 1985-2025Looking Back, Looking Forward NSF CI Funding 1985-2025
Looking Back, Looking Forward NSF CI Funding 1985-2025Larry Smarr
 
Network research
Network researchNetwork research
Network researchJisc
 
IRJET-Open Curltm Cloud Computing Test Structure:Confederate Data Centers for...
IRJET-Open Curltm Cloud Computing Test Structure:Confederate Data Centers for...IRJET-Open Curltm Cloud Computing Test Structure:Confederate Data Centers for...
IRJET-Open Curltm Cloud Computing Test Structure:Confederate Data Centers for...IRJET Journal
 
SKA NZ R&D BeSTGRID Infrastructure
SKA NZ R&D BeSTGRID InfrastructureSKA NZ R&D BeSTGRID Infrastructure
SKA NZ R&D BeSTGRID InfrastructureNick Jones
 
Software, Licences etc
Software, Licences etcSoftware, Licences etc
Software, Licences etcDART Project
 
The Pacific Research Platform: a Science-Driven Big-Data Freeway System
The Pacific Research Platform: a Science-Driven Big-Data Freeway SystemThe Pacific Research Platform: a Science-Driven Big-Data Freeway System
The Pacific Research Platform: a Science-Driven Big-Data Freeway SystemLarry Smarr
 
Larry Smarr - NRP Application Drivers
Larry Smarr - NRP Application DriversLarry Smarr - NRP Application Drivers
Larry Smarr - NRP Application DriversLarry Smarr
 
Federated Cloud Computing
Federated Cloud ComputingFederated Cloud Computing
Federated Cloud ComputingDavid Wallom
 
Spark at NASA/JPL-(Chris Mattmann, NASA/JPL)
Spark at NASA/JPL-(Chris Mattmann, NASA/JPL)Spark at NASA/JPL-(Chris Mattmann, NASA/JPL)
Spark at NASA/JPL-(Chris Mattmann, NASA/JPL)Spark Summit
 
OpenStack Toronto Q3 MeetUp - September 28th 2017
OpenStack Toronto Q3 MeetUp - September 28th 2017OpenStack Toronto Q3 MeetUp - September 28th 2017
OpenStack Toronto Q3 MeetUp - September 28th 2017Stacy Véronneau
 
CloudLab Overview
CloudLab OverviewCloudLab Overview
CloudLab OverviewEd Dodds
 
Ceph, Open Source, and the Path to Ubiquity in Storage - AACS Meetup 2014
Ceph, Open Source, and the Path to Ubiquity in Storage - AACS Meetup 2014Ceph, Open Source, and the Path to Ubiquity in Storage - AACS Meetup 2014
Ceph, Open Source, and the Path to Ubiquity in Storage - AACS Meetup 2014Patrick McGarry
 

Similaire à Frank Würthwein - NRP and the Path forward (20)

Utilising Cloud Computing for Research through Infrastructure, Software and D...
Utilising Cloud Computing for Research through Infrastructure, Software and D...Utilising Cloud Computing for Research through Infrastructure, Software and D...
Utilising Cloud Computing for Research through Infrastructure, Software and D...
 
Enabling Insight to Support World-Class Supercomputing (Stefan Ceballos, Oak ...
Enabling Insight to Support World-Class Supercomputing (Stefan Ceballos, Oak ...Enabling Insight to Support World-Class Supercomputing (Stefan Ceballos, Oak ...
Enabling Insight to Support World-Class Supercomputing (Stefan Ceballos, Oak ...
 
Azure Brain: 4th paradigm, scientific discovery & (really) big data
Azure Brain: 4th paradigm, scientific discovery & (really) big dataAzure Brain: 4th paradigm, scientific discovery & (really) big data
Azure Brain: 4th paradigm, scientific discovery & (really) big data
 
NSF Software @ ApacheConNA
NSF Software @ ApacheConNANSF Software @ ApacheConNA
NSF Software @ ApacheConNA
 
Cloud Standards in the Real World: Cloud Standards Testing for Developers
Cloud Standards in the Real World: Cloud Standards Testing for DevelopersCloud Standards in the Real World: Cloud Standards Testing for Developers
Cloud Standards in the Real World: Cloud Standards Testing for Developers
 
Looking Back, Looking Forward NSF CI Funding 1985-2025
Looking Back, Looking Forward NSF CI Funding 1985-2025Looking Back, Looking Forward NSF CI Funding 1985-2025
Looking Back, Looking Forward NSF CI Funding 1985-2025
 
Network research
Network researchNetwork research
Network research
 
IRJET-Open Curltm Cloud Computing Test Structure:Confederate Data Centers for...
IRJET-Open Curltm Cloud Computing Test Structure:Confederate Data Centers for...IRJET-Open Curltm Cloud Computing Test Structure:Confederate Data Centers for...
IRJET-Open Curltm Cloud Computing Test Structure:Confederate Data Centers for...
 
ApacheCon NA 2013
ApacheCon NA 2013ApacheCon NA 2013
ApacheCon NA 2013
 
SKA NZ R&D BeSTGRID Infrastructure
SKA NZ R&D BeSTGRID InfrastructureSKA NZ R&D BeSTGRID Infrastructure
SKA NZ R&D BeSTGRID Infrastructure
 
Software, Licences etc
Software, Licences etcSoftware, Licences etc
Software, Licences etc
 
The Pacific Research Platform: a Science-Driven Big-Data Freeway System
The Pacific Research Platform: a Science-Driven Big-Data Freeway SystemThe Pacific Research Platform: a Science-Driven Big-Data Freeway System
The Pacific Research Platform: a Science-Driven Big-Data Freeway System
 
Larry Smarr - NRP Application Drivers
Larry Smarr - NRP Application DriversLarry Smarr - NRP Application Drivers
Larry Smarr - NRP Application Drivers
 
Federated Cloud Computing
Federated Cloud ComputingFederated Cloud Computing
Federated Cloud Computing
 
Spark at NASA/JPL-(Chris Mattmann, NASA/JPL)
Spark at NASA/JPL-(Chris Mattmann, NASA/JPL)Spark at NASA/JPL-(Chris Mattmann, NASA/JPL)
Spark at NASA/JPL-(Chris Mattmann, NASA/JPL)
 
OpenStack Toronto Q3 MeetUp - September 28th 2017
OpenStack Toronto Q3 MeetUp - September 28th 2017OpenStack Toronto Q3 MeetUp - September 28th 2017
OpenStack Toronto Q3 MeetUp - September 28th 2017
 
Grid computing
Grid computingGrid computing
Grid computing
 
CloudLab Overview
CloudLab OverviewCloudLab Overview
CloudLab Overview
 
Ceph, Open Source, and the Path to Ubiquity in Storage - AACS Meetup 2014
Ceph, Open Source, and the Path to Ubiquity in Storage - AACS Meetup 2014Ceph, Open Source, and the Path to Ubiquity in Storage - AACS Meetup 2014
Ceph, Open Source, and the Path to Ubiquity in Storage - AACS Meetup 2014
 
Digitisation and institutional repositories 2
Digitisation and institutional repositories 2Digitisation and institutional repositories 2
Digitisation and institutional repositories 2
 

Plus de Larry Smarr

My Remembrances of Mike Norman Over The Last 45 Years
My Remembrances of Mike Norman Over The Last 45 YearsMy Remembrances of Mike Norman Over The Last 45 Years
My Remembrances of Mike Norman Over The Last 45 YearsLarry Smarr
 
Metagenics How Do I Quantify My Body and Try to Improve its Health? June 18 2019
Metagenics How Do I Quantify My Body and Try to Improve its Health? June 18 2019Metagenics How Do I Quantify My Body and Try to Improve its Health? June 18 2019
Metagenics How Do I Quantify My Body and Try to Improve its Health? June 18 2019Larry Smarr
 
Panel: Reaching More Minority Serving Institutions
Panel: Reaching More Minority Serving InstitutionsPanel: Reaching More Minority Serving Institutions
Panel: Reaching More Minority Serving InstitutionsLarry Smarr
 
Global Network Advancement Group - Next Generation Network-Integrated Systems
Global Network Advancement Group - Next Generation Network-Integrated SystemsGlobal Network Advancement Group - Next Generation Network-Integrated Systems
Global Network Advancement Group - Next Generation Network-Integrated SystemsLarry Smarr
 
Wireless FasterData and Distributed Open Compute Opportunities and (some) Us...
 Wireless FasterData and Distributed Open Compute Opportunities and (some) Us... Wireless FasterData and Distributed Open Compute Opportunities and (some) Us...
Wireless FasterData and Distributed Open Compute Opportunities and (some) Us...Larry Smarr
 
Panel Discussion: Engaging underrepresented technologists, researchers, and e...
Panel Discussion: Engaging underrepresented technologists, researchers, and e...Panel Discussion: Engaging underrepresented technologists, researchers, and e...
Panel Discussion: Engaging underrepresented technologists, researchers, and e...Larry Smarr
 
The Asia Pacific and Korea Research Platforms: An Overview Jeonghoon Moon
The Asia Pacific and Korea Research Platforms: An Overview Jeonghoon MoonThe Asia Pacific and Korea Research Platforms: An Overview Jeonghoon Moon
The Asia Pacific and Korea Research Platforms: An Overview Jeonghoon MoonLarry Smarr
 
Panel: Reaching More Minority Serving Institutions
Panel: Reaching More Minority Serving InstitutionsPanel: Reaching More Minority Serving Institutions
Panel: Reaching More Minority Serving InstitutionsLarry Smarr
 
Panel: The Global Research Platform: An Overview
Panel: The Global Research Platform: An OverviewPanel: The Global Research Platform: An Overview
Panel: The Global Research Platform: An OverviewLarry Smarr
 
Panel: Future Wireless Extensions of Regional Optical Networks
Panel: Future Wireless Extensions of Regional Optical NetworksPanel: Future Wireless Extensions of Regional Optical Networks
Panel: Future Wireless Extensions of Regional Optical NetworksLarry Smarr
 
Global Research Platform Workshops - Maxine Brown
Global Research Platform Workshops - Maxine BrownGlobal Research Platform Workshops - Maxine Brown
Global Research Platform Workshops - Maxine BrownLarry Smarr
 
Built around answering questions
Built around answering questionsBuilt around answering questions
Built around answering questionsLarry Smarr
 
Panel: NRP Science Impacts​
Panel: NRP Science Impacts​Panel: NRP Science Impacts​
Panel: NRP Science Impacts​Larry Smarr
 
Democratizing Science through Cyberinfrastructure - Manish Parashar
Democratizing Science through Cyberinfrastructure - Manish ParasharDemocratizing Science through Cyberinfrastructure - Manish Parashar
Democratizing Science through Cyberinfrastructure - Manish ParasharLarry Smarr
 
Panel: Building the NRP Ecosystem with the Regional Networks on their Campuses;
Panel: Building the NRP Ecosystem with the Regional Networks on their Campuses;Panel: Building the NRP Ecosystem with the Regional Networks on their Campuses;
Panel: Building the NRP Ecosystem with the Regional Networks on their Campuses;Larry Smarr
 
Open Force Field: Scavenging pre-emptible CPU hours* in the age of COVID - Je...
Open Force Field: Scavenging pre-emptible CPU hours* in the age of COVID - Je...Open Force Field: Scavenging pre-emptible CPU hours* in the age of COVID - Je...
Open Force Field: Scavenging pre-emptible CPU hours* in the age of COVID - Je...Larry Smarr
 
Panel: Open Infrastructure for an Open Society: OSG, Commercial Clouds, and B...
Panel: Open Infrastructure for an Open Society: OSG, Commercial Clouds, and B...Panel: Open Infrastructure for an Open Society: OSG, Commercial Clouds, and B...
Panel: Open Infrastructure for an Open Society: OSG, Commercial Clouds, and B...Larry Smarr
 
Panel: Open Infrastructure for an Open Society: OSG, Commercial Clouds, and B...
Panel: Open Infrastructure for an Open Society: OSG, Commercial Clouds, and B...Panel: Open Infrastructure for an Open Society: OSG, Commercial Clouds, and B...
Panel: Open Infrastructure for an Open Society: OSG, Commercial Clouds, and B...Larry Smarr
 
Panel: Open Infrastructure for an Open Society: OSG, Commercial Clouds, and B...
Panel: Open Infrastructure for an Open Society: OSG, Commercial Clouds, and B...Panel: Open Infrastructure for an Open Society: OSG, Commercial Clouds, and B...
Panel: Open Infrastructure for an Open Society: OSG, Commercial Clouds, and B...Larry Smarr
 
Global Network Advancement Group Next Generation Network-Integrated Sys...
      Global Network Advancement GroupNext Generation Network-Integrated Sys...      Global Network Advancement GroupNext Generation Network-Integrated Sys...
Global Network Advancement Group Next Generation Network-Integrated Sys...Larry Smarr
 

Plus de Larry Smarr (20)

My Remembrances of Mike Norman Over The Last 45 Years
My Remembrances of Mike Norman Over The Last 45 YearsMy Remembrances of Mike Norman Over The Last 45 Years
My Remembrances of Mike Norman Over The Last 45 Years
 
Metagenics How Do I Quantify My Body and Try to Improve its Health? June 18 2019
Metagenics How Do I Quantify My Body and Try to Improve its Health? June 18 2019Metagenics How Do I Quantify My Body and Try to Improve its Health? June 18 2019
Metagenics How Do I Quantify My Body and Try to Improve its Health? June 18 2019
 
Panel: Reaching More Minority Serving Institutions
Panel: Reaching More Minority Serving InstitutionsPanel: Reaching More Minority Serving Institutions
Panel: Reaching More Minority Serving Institutions
 
Global Network Advancement Group - Next Generation Network-Integrated Systems
Global Network Advancement Group - Next Generation Network-Integrated SystemsGlobal Network Advancement Group - Next Generation Network-Integrated Systems
Global Network Advancement Group - Next Generation Network-Integrated Systems
 
Wireless FasterData and Distributed Open Compute Opportunities and (some) Us...
 Wireless FasterData and Distributed Open Compute Opportunities and (some) Us... Wireless FasterData and Distributed Open Compute Opportunities and (some) Us...
Wireless FasterData and Distributed Open Compute Opportunities and (some) Us...
 
Panel Discussion: Engaging underrepresented technologists, researchers, and e...
Panel Discussion: Engaging underrepresented technologists, researchers, and e...Panel Discussion: Engaging underrepresented technologists, researchers, and e...
Panel Discussion: Engaging underrepresented technologists, researchers, and e...
 
The Asia Pacific and Korea Research Platforms: An Overview Jeonghoon Moon
The Asia Pacific and Korea Research Platforms: An Overview Jeonghoon MoonThe Asia Pacific and Korea Research Platforms: An Overview Jeonghoon Moon
The Asia Pacific and Korea Research Platforms: An Overview Jeonghoon Moon
 
Panel: Reaching More Minority Serving Institutions
Panel: Reaching More Minority Serving InstitutionsPanel: Reaching More Minority Serving Institutions
Panel: Reaching More Minority Serving Institutions
 
Panel: The Global Research Platform: An Overview
Panel: The Global Research Platform: An OverviewPanel: The Global Research Platform: An Overview
Panel: The Global Research Platform: An Overview
 
Panel: Future Wireless Extensions of Regional Optical Networks
Panel: Future Wireless Extensions of Regional Optical NetworksPanel: Future Wireless Extensions of Regional Optical Networks
Panel: Future Wireless Extensions of Regional Optical Networks
 
Global Research Platform Workshops - Maxine Brown
Global Research Platform Workshops - Maxine BrownGlobal Research Platform Workshops - Maxine Brown
Global Research Platform Workshops - Maxine Brown
 
Built around answering questions
Built around answering questionsBuilt around answering questions
Built around answering questions
 
Panel: NRP Science Impacts​
Panel: NRP Science Impacts​Panel: NRP Science Impacts​
Panel: NRP Science Impacts​
 
Democratizing Science through Cyberinfrastructure - Manish Parashar
Democratizing Science through Cyberinfrastructure - Manish ParasharDemocratizing Science through Cyberinfrastructure - Manish Parashar
Democratizing Science through Cyberinfrastructure - Manish Parashar
 
Panel: Building the NRP Ecosystem with the Regional Networks on their Campuses;
Panel: Building the NRP Ecosystem with the Regional Networks on their Campuses;Panel: Building the NRP Ecosystem with the Regional Networks on their Campuses;
Panel: Building the NRP Ecosystem with the Regional Networks on their Campuses;
 
Open Force Field: Scavenging pre-emptible CPU hours* in the age of COVID - Je...
Open Force Field: Scavenging pre-emptible CPU hours* in the age of COVID - Je...Open Force Field: Scavenging pre-emptible CPU hours* in the age of COVID - Je...
Open Force Field: Scavenging pre-emptible CPU hours* in the age of COVID - Je...
 
Panel: Open Infrastructure for an Open Society: OSG, Commercial Clouds, and B...
Panel: Open Infrastructure for an Open Society: OSG, Commercial Clouds, and B...Panel: Open Infrastructure for an Open Society: OSG, Commercial Clouds, and B...
Panel: Open Infrastructure for an Open Society: OSG, Commercial Clouds, and B...
 
Panel: Open Infrastructure for an Open Society: OSG, Commercial Clouds, and B...
Panel: Open Infrastructure for an Open Society: OSG, Commercial Clouds, and B...Panel: Open Infrastructure for an Open Society: OSG, Commercial Clouds, and B...
Panel: Open Infrastructure for an Open Society: OSG, Commercial Clouds, and B...
 
Panel: Open Infrastructure for an Open Society: OSG, Commercial Clouds, and B...
Panel: Open Infrastructure for an Open Society: OSG, Commercial Clouds, and B...Panel: Open Infrastructure for an Open Society: OSG, Commercial Clouds, and B...
Panel: Open Infrastructure for an Open Society: OSG, Commercial Clouds, and B...
 
Global Network Advancement Group Next Generation Network-Integrated Sys...
      Global Network Advancement GroupNext Generation Network-Integrated Sys...      Global Network Advancement GroupNext Generation Network-Integrated Sys...
Global Network Advancement Group Next Generation Network-Integrated Sys...
 

Dernier

Cybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptxCybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptxGDSC PJATK
 
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdfUiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdfDianaGray10
 
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve DecarbonizationUsing IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve DecarbonizationIES VE
 
Comparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioComparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioChristian Posta
 
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAAnypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAshyamraj55
 
UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7DianaGray10
 
9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding Team9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding TeamAdam Moalla
 
Bird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystemBird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystemAsko Soukka
 
Designing A Time bound resource download URL
Designing A Time bound resource download URLDesigning A Time bound resource download URL
Designing A Time bound resource download URLRuncy Oommen
 
COMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a WebsiteCOMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a Websitedgelyza
 
UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8DianaGray10
 
Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Commit University
 
Videogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdfVideogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdfinfogdgmi
 
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019IES VE
 
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online CollaborationCOMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online Collaborationbruanjhuli
 
Introduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptxIntroduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptxMatsuo Lab
 
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsIgniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsSafe Software
 
Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024SkyPlanner
 
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesAI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesMd Hossain Ali
 

Dernier (20)

Cybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptxCybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptx
 
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdfUiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
 
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve DecarbonizationUsing IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
 
Comparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioComparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and Istio
 
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAAnypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
 
UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7
 
9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding Team9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding Team
 
Bird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystemBird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystem
 
Designing A Time bound resource download URL
Designing A Time bound resource download URLDesigning A Time bound resource download URL
Designing A Time bound resource download URL
 
COMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a WebsiteCOMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a Website
 
UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8
 
Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)
 
201610817 - edge part1
201610817 - edge part1201610817 - edge part1
201610817 - edge part1
 
Videogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdfVideogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdf
 
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
 
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online CollaborationCOMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
 
Introduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptxIntroduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptx
 
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsIgniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
 
Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024
 
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesAI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
 

Frank Würthwein - NRP and the Path forward

  • 1. NRP & the Path forward Frank Würthwein Director, San Diego Supercomputer Center February 9th 2023
  • 4. NATIONAL RESEARCH NATIONAL RESEARCH Long Term Vision • Create an Open National Cyberinfrastructure that allows the federation of CI at all ~4,000 accredited, degree granting higher education institutions, non- profit research institutions, and national laboratories. § Open Science § Open Data § Open Source § Open Infrastructure 4 Openness for an Open Society Open Compute Open Storage & CDN Open devices/instruments/IoT, …?
  • 5. NATIONAL RESEARCH NATIONAL RESEARCH Community vs Funded Projects 5 Community with Shared Vision Lot’s of funded projects that contribute to this shared vision in different ways. We want you to … … grow NRP. … build on NRP. NRP is “owned” and “built” by the community for the community
  • 6. How is NRP different from OSG? We’ve been doing federated cyberinfrastructure since 2005. Why is NRP even needed given that OSG exists?
  • 7. NATIONAL RESEARCH HTCondor/OSG Cyberinfrastructure Stack 7 Hardware IPMI, Firmware, BIOS Kubernetes Admiralty SLURM NRP operates at all layers of the stack, from IPMI up • IPMI reduces TCO and lower threshold to entry • Kubernetes allows service deployments • Also the natural layer for application container deployment • Admiralty allows K8S federation with folks who want control • Including cloud integration to access TPUs & other cloud only architectures • HTCondor allows NRP to show up as a “site” in OSG The layer you integrate at depends on • Control you want • Effort you can afford
  • 8. Complementarity in Implementation of “Bring Your Own Resource” model OSG/PATh focused on campus cluster integration. NRP focused on individual node integration instead of clusters.
  • 9. NATIONAL RESEARCH HTCondor/OSG Cyberinfrastructure Stack 9 Hardware IPMI, Firmware, BIOS Kubernetes Admiralty SLURM NRP operates at all layers of the stack, from IPMI up • IPMI reduces TCO and lowers threshold to entry • Kubernetes allows service deployments • Also the natural layer for application container deployment • Admiralty allows K8S federation with folks who want control • Including cloud integration to access TPUs & other cloud only architectures • HTCondor allows NRP to show up as site in OSG • Under-resourced institutions • Network providers and their POPs • CS & ECE faculty specialized on: • AI/ML => gaming GPUs • systems R&D All of these find it difficult to justify staff to support all layers
  • 10. NATIONAL RESEARCH Hardware on NRP is Global 10 NRP integrates hardware in USA, EU, and Asia
  • 11. NATIONAL RESEARCH Cyberinfrastructure Stack 11 Hardware IPMI, Firmware, BIOS Kubernetes NRP operates at all layers of the stack, from IPMI up • IPMI reduces TCO and lowers threshold to entry • Kubernetes allows service deployments • Also the natural layer for application container deployment • Admiralty allows K8S federation with folks who want control • Including cloud integration to access TPUs & other cloud only architectures • HTCondor allows NRP to show up as site in OSG • Open Science Data Federation • Origins & Caches in US, EU, Asia • Protein Data Bank • (Future) Replicas in EU & Asia NRP is unique in its support of global service deployments OSDF ops by PATh PDB ops by PDB ???
  • 12. Supporting Nautilus for the next decade Nautilus = K8S infrastructure of PRP for the last 5+ years Nautilus = K8S of NRP for the next 10 years
  • 13. NATIONAL RESEARCH NATIONAL RESEARCH The NSF Cat-II Program • NSF supports via the Cat-II program novel systems ideas. - 3 year “testbed” phase § The PI owns the resource, and has (some) freedom regarding who uses it. Ÿ No requirements for making it available via any specific allocation mechanism. § It is expected that not all features work on day 1. Ÿ 3 years of experimentation & development of features - 2 year “allocation” phase § The resource is made available via an NSF supported allocation mechanism. - The solicitation mentions the possibility of an additional 5 year renewal without re-competition if system is successful. • We decided that this is an ideal program to try and secure NRP core operations funding for the next 10 years - And thus provide the stability necessary for growth of NRP. 13
  • 14. NATIONAL RESEARCH NATIONAL RESEARCH Cat-II: Prototype National Research Platform (PNRP) 14 Funded as NSF 2112167 80GB A100 A10 64 288 5 year project with $5M hardware & $6.45M people Supports Nautilus, and thus the core NRP infrastructure Promises to build on “PRP” functionality, and go beyond NSF Acceptance Review of System scheduled for March 8 & 9th 2023 PI = Wuerthwein; Co-PIs: DeFanti, Rosing, Tatineni, Weitzel
  • 15. NATIONAL RESEARCH NATIONAL RESEARCH Innovations • I1: Innovative network fabric that allows “rack” of hardware to behave like a single “node” connected via PCIe. • I2: Innovative application libraries to expose FPGAs hardware to science apps at language constructs scientists understand (C, C++ rather than firmware) • I3: A “Bring Your Own Resource” model that allows campuses nationwide to join their resources to the system. • I4: Innovative scheduling to support urgent computing, including interactive via Jupyter. • I5: Innovative Data Infrastructure, including national scale Content Delivery System like YouTube for science. 15 I3 & I4 & I5 turn ”PRP” into “NRP” and sustains it into the future. I1 & I2 are totally new.
  • 16. NATIONAL RESEARCH NATIONAL RESEARCH Data Infrastructure Model of NRP • Support regional Ceph storage systems across the USA. - Campuses can join individual storage hosts to the Ceph system in their region. - All regional storage systems are Origins in OSG Data Federation (OSDF) - Deploy replication system such that researchers can decide what part of their namespace should be in which regional storage. • Deploy caches in Internet2 backbone such that no campus nationwide is more than 500 miles from a cache. 16 NRP data infrastructure model combines best of PRP & OSG From PRP we take the regional Ceph storage concept From OSG/PATh we take the data origin & caching concepts And then we add as a totally new feature: User controlled replication of partial namespaces across regions. (We will develop this during 3 year “testbed” phase) Want Others to build higher level data services on top
  • 17. NATIONAL RESEARCH NATIONAL RESEARCH Matrix of Science x Innovations 17 NRP. The networks of researchers we create will use the NRP systems innovations to achieve broader im- pact, with the help from our collaborators, users, staff, Co-PIs, and PI. Table 3.1 Representative Science and Engineering Use Cases Application domain Lead researcher & Institution Science Driver Themes NRP Innovations LIGO Peter Couvares, LIGO Lab; Erik Katsavounidis, MIT BGS, UC, AI I2, I3, I4, I5 IceCube Benedikt Riedel, UW Madison BGS, UC, AI I3, I4 Astronomy (DKIST & Sky Surveys) Curt Dodds, U. Hawai’i BGS, AI I3, I5, Campus Scale Instru- ment Facilities Mark Ellisman, NCMIR; Sa- mara Reck-Peterson, Nicon Imaging Center; Johannes Schoeneberg, Adaptive Op- tics Lightsheet Microscopy; Kristen Jepsen, Institute for Genomic Medicine; Tami Brown-Brandl, Precision Ani- mal Management SD, UC, H I1, I2, I3, I4, I5 Molecular Dynamics Rommie Amaro, UCSD; An- dreas Goetz, SDSC; Jona- than Allen, LLNL MD, AI, H I1, I2, I3 Human microbiome Rob Knight, UCSD G, AI, H I1, I2, I3 Genomics & Bioinfor- matics Alex Feltus, Clemson G, AI, H I3, I4, I5 Fluid Dynamics Rose Yu, UCSD AI I1, I2, I3 Experimental Particle Physics, IAIFI Phil Harris, MIT AI, BGS, SD I1, I2 Computer Vision Nuno Vasconcelos, UCSD AI, CV I3 Computer Graphics Robert Twomey, UNL CV, AI I3 Programmable Storage Carlos Malzahn, UCSC SD I1, I2, I5 AI systems software stack for FPGAs Hadi Esmaeilzadeh , UCSD SD I1, I2 WildFire Analysis & Prediction Ilkay Altintas, UCSD UC, AI, CV I3, I4 Key: The NRP innovations column lists those innovations among I1 through I5 listed in Section 2.1 that a given science driver most benefits from. NSF MREFCs Incl. 4 campus scale instrument facilities Incl. a very diverse set of sciences and engineering Lot’s of AI … but so much more …
  • 18. NATIONAL RESEARCH NATIONAL RESEARCH FKW’s Wishlist for the Future • Growth of NRP infrastructure - 1,000++ GPUs end of 2022 - 50 PB storage end of 2024 - Growth in diversity of community § # and types of campuses and their researchers • Introduce new capabilities to NRP - Machine learning at 100TB scale - Support Domain Specific Architecture R&D on NRP - Expand NRP into Wireless, Edge, IoT - Towards “FAIR” on OSDF • New Directions initiated by the Community 18
  • 19. NATIONAL RESEARCH NATIONAL RESEARCH “Domain Specific Architectures” • I1: Innovative network fabric allowing “composable hardware”. • I2: Innovative application libraries allowing “domain optimized architectures” on FPGAs 19 “end of Moore’s law” motivates new architectures Mark Papermaster, CTO of AMD PRISM, a Jump 2.0 project funded by SRC is early user of FPGAs@PNRP John Shalf (2020) https://doi.org/10.1098/rsta.2019.0061 PI, Tajana Rosing
  • 20. NATIONAL RESEARCH NATIONAL RESEARCH New Data Origins • The NSF CC* 2022 program awarded 9 campuses with $500k storage awards each. - We guess this pays for 5PB of storage each. • Some of these campuses may decide to integrate their CC* storage into the OSDF. • Some of these campuses have storage from other projects that they may integrate with the OSDF in addition. • NSF 23-523 includes $500k storage solicitation again, Spring & Fall 2023. 20
  • 21. NATIONAL RESEARCH NATIONAL RESEARCH Summary & Conclusions • PRP ended, and was replaced by NRP - Significant new capabilities via Cat-II system “PNRP” § PNRP provides ops effort for Nautilus for the future - # of GPUs available double in 2022. § new GPUs (A10, 3080, 3090, A100) much more powerful than older GPUs - # of FPGAs increase from a few to a few dozen in 2022. - # of caches grow by 50% in 22/23 => more consistent coverage across USA - Data volume served expected to grow substantially in 23/24/25. § How much? As yet too hard to predict. • Hoping to recruit new partners to build FAIR capabilities on top of OSDF within the next 5 years. • Hoping to expand NRP into sensor networks using 5G & 6G in the next 10 years. 21
  • 22. NATIONAL RESEARCH NATIONAL RESEARCH Acknowledgements • This work was partially supported by the NSF grants OAC-1541349, OAC-1826967, OAC- 2030508, OAC-1841530, OAC-2005369, OAC-21121167, CISE-1713149, CISE- 2100237, CISE-2120019, OAC-2112167 22