This document discusses the role of XSEDE (Extreme Science and Engineering Discovery Environment) in supporting scientific research through advanced cyberinfrastructure and its relationship to the National Strategic Computing Initiative (NSCI). It provides an overview of XSEDE's resources, services, and support for researchers. Key points include: - XSEDE coordinates over $100 million in advanced computing resources from the NSF to support over 15,000 researchers annually. - Resources include supercomputers, data storage, visualization systems, and software from various partner institutions. - Services include training, allocations management, user support, and collaborations to optimize applications. - XSEDE aims to define an integrated national cyber

1. May 23, 2017
XSEDE National Cyberinfrastructure, NIST,
and Supporting NCSI Objectives
John Towns
PI and Project Director, XSEDE
Executive Director, Science & Technology, NCSA
jtowns@ncsa.illinois.edu

2. License terms
• Please cite as: Towns, John. XSEDE National Cyberinfrastructure, NIST, and
Supporting NCSI Objectives, May 2017,
[https://www.slideshare.net/jtownsil/xsede-national-cyberinfrastructure-
nist-and-supporting-ncsi-objectives]
• ORCID ID: http://orcid.org/0000-0001-7961-2277
• Except where otherwise noted, by inclusion of a source URL or some other
note, the contents of this presentation are © by the Board of Trustees of
University of Illinois. This content is released under the Creative Commons
Attribution 3.0 Unported license
(http://creativecommons.org/licenses/by/3.0/). This license includes the
following terms: You are free to share – copy and redistribute the material
in any medium or format; and to adapt – remix, transform, and build upon
the material for any purpose, even commercially.
• This can be done under the following conditions: attribution – You must
give appropriate credit, provide a link to the license, and indicate if
changes were made. You may do so in any reasonable manner, but not in
any way that suggests the licensor endorses you or your use.

3. Motivation for XSEDE:
• Scientific advancement across multiple disciplines
requires a variety of resources and services
• XSEDE is about increased productivity of the
community and providing expanded capabilities
– leads to more science
– is sometimes the difference between a feasible project
and an impractical one
– lowers barriers to adoption
• XSEDE provides a comprehensive eScience
infrastructure composed of expertly managed and
evolving advanced heterogeneous digital resources and
services integrated into a general-purpose
infrastructure
3

4. XSEDE – accelerating scientific discovery
XSEDE’s Vision:
a world of digitally enabled scholars, researchers, and
engineers participating in multidisciplinary collaborations
while seamlessly accessing advanced computing
resources and sharing data to tackle society’s grand
challenges.
XSEDE’s Mission:
to enhance the productivity of a growing community
of scholars, researchers, and engineers through access to
advanced digital services that support open research by
coordinating and adding value to the leading
cyberinfrastructure resources funded by the NSF and
other agencies.
4

5. Vision/Mission: Enable Realizing Best Science
5

6. Total Research Funding Supported
by XSEDE to Date
6
$2.84 billion in research
supported by XSEDE
July 2011-March 2017
Research funding only. XSEDE leverages
and integrates additional infrastructure,
some funded by NSF (e.g. Track 2 systems)
and some not (e.g. Internet2).
NSF, $978.5M,
34%
NIH, $607.2M,
21%
DOE, $558.2M,
20%
DOD, $217.5M,
8%
NASA, $113.1M,
4%
DOC, $53.7M,
2%
All Others, $311.7M,
11%

7. XSEDE Factoids: high order bits
• 5 year, US$110M project
– pursuing additional funding via independent proposals
– initial 5 year award: $121M project + ~$4.6M in supplements
• plus $9M, 5 year Technology Investigation Service
– separate award from NSF
• No funding for major hardware
– coordination, support and creating a national/international
cyberinfrastructure
– coordinate allocations, support, training and documentation for
>$100M of concurrent project awards from NSF
• ~90 FTE /~240 individuals funded across 19 partner
institutions
– this requires solid partnering!
7

8. – partnership led by NCSA, PSC, TACC, SDSC
and NICS
– CI centers with deep experience
– partners who strongly complement these CI
centers with expertise in science, engineering,
technology and education
XSEDE’s Distinguishing Characteristics:
World-class Leadership
8

9. 9
Accelerating
Scientific
Discovery
Darter
SuperMIC
Stampede
Maverick
XStreme
WranglerMason
Ranch
Data Oasis
And many more …
XD Service Providers Forum
9 Level 1 SPs
7 Level 2 SPs
19 Level 3 SPs
People, organizations,
& communities
Software
Data
Scientific
Instruments
Computational
Resources
Networking &
Cybersecurity
HYPOTHESIZE
EXPERIMENTANALYZE
THEORIZE
OBSERVE
CYBERINFRASTRUCTURE
ECOSYSTEM
Courtesy of NSF

10. Significant Contributions to Community by XSEDE:
Supporting the Community
• Annually supporting 15,000 researchers and students via allocated access
to resources
– 6,000 through direct allocations + 9,000 through use by science gateways
– science gateways allowing many more to leverage XSEDE-allocated resources
– many more leveraging unallocated services via the XSEDE User Portal
• More than 21,000 publications supported to date since July 2011
– analysis shows significantly higher citation rate than other publications in the
same journals
• Over 15,000 active users of XSEDE User Portal in the past 12 months
• Over 900 proposals for major compute allocations reviewed annually
– a comparable number of requests for startup and educational allocations
• More than 77,000 user requests addressed to date since July 2011
– answering questions and resolving technical issues
• To date, completed more than 375 projects that assisted computational
research teams
– users reported an average productivity gain of 14.75 months
– average three months investment of XSEDE staff time
10

11. XSEDE offers efficient and effective
integrated access to a variety of resources
• Leading-edge distributed memory systems
• Very large shared memory systems
• High throughput systems, including Open Science
Grid (OSG)
• Support for VM’s and containers and HPC Cloud
• Visualization engines
• Accelerators like GPUs and Xeon PHIs
• Extensive library of research applications
Many scientific problems have components that call
for use of more than one platform.
11

12. XSEDE User Portal: THE User Site
portal.xsede.org
• XSEDE User Portal (XUP) is designed to be the only site
a user needs to use XSEDE
• XUP presents information relevant to users
– user info is easier to find
– XUP also provides dynamic data about XSEDE systems
– capabilities to manage usage, files, data
• As a user you can
– request an allocation, and manage allocations
– sign up for training
– request help
– manage files and data, and much more!
– Portal provides single sign-on to all XSEDE resources

13. Current XSEDE Compute Resources
• Stampede @ TACC
– 9.5 PFLOPS (PF) Dell
Cluster w/ GPUs and Xeon
PHIs
• Comet @ SDSC
– 2.1 PF cluster w/GPUs
• Bridges @ PSC
– 1.3 PF w/ large memory
(274 TB)
• XStream @ Stanford
– 1.0 PF GPU Cray CS-Storm
cluster
• SuperMIC @ LSU
– 925 TF Dell Cluster w/
GPUs and Xeon PHIs
• Jetstream @ Indiana
– 516 TF HPC Cloud
• Wrangler @ TACC
– 62 TF data analytics system
• Open Science Grid
– 60,000 CPU cores
13
https://www.xsede.org/web/xup/resource-monitor

14. Current XSEDE Visualization, Data, and
Software Resources
• Storage
– Ranch @ TACC
• 61 PB tape
– Pylon @ PSC
• 10 PB disk
– Wrangler @ TACC
• 10 PB disk
– Data Oasis @ SDSC
• 4 PB tape
• Visualization
– Maverick @ TACC
• 59 TF HP/NVIDIA cluster
• 20 PB disk
• Software: 100s of titles
– domain software
• chemistry, CFD,
bioinformatics, physics,
astronomy, biology,
engineering, statistics,…
– tools
• middleware, visualization,
scripting, performance
analysis, data storage and
management, …
– compilers and libraries
• most languages supported,
math libraries, machine
learning, …
14
https://www.xsede.org/web/xup/resource-monitor
https://portal.xsede.org/software#

15. Extended Support Available - ECSS
• The Extended Collaborative Support Service:
– improves the productivity of the XSEDE user
community through successful, meaningful
collaborations to
• optimize their applications,
• improve their work and data flows, and
• increase their effective use of the XSEDE digital
infrastructure and
– broadly expands the XSEDE user base by engaging
members of underrepresented communities
and domain areas
15

16. What is ECSS?
• Extended Collaborative Support
Service
• Expert staff can be requested for
collaborations lasting months to
a year
– requests made through the XSEDE
allocation system when requesting
compute/data/viz resources
• Typical collaborations require 20-
25% staff time for one year
• Critical mass engenders success
– ~28 FTEs (~70 individuals), 10 sites
• 24.5 w/out management
– advanced degrees in a variety of
science and technology fields
16

17. A Few NIST Examples
17
Title:
Sequence specific binding of single-stranded DNA (ssDNA) with single-walled
carbon nanotubes (SWCNTs): An investigation on the effect of SWCNT chirality
via replica exchange molecular dynamics simulations
PI: Kevin Hinkle
Institution: National Institute of Standards and Technology
Field of Science: Materials Research
Resource Allocation
TACC-RANCH 1,000
TACC-STAMPEDE 1,020,092
Title:
Multi-Scale Synthesis and Functionalization of 2D Materials for Nano-Electronic
Devices and Energy Conversion Applications
PI: Arunima Singh
Institution: National Institute of Standards and Technology
Field of Science: Materials Research
Resource Allocation
TACC-RANCH 1,000
TACC-STAMPEDE 511,593
TACC-RANCH 1,000
TACC-STAMPEDE 797,665

18. 18
Title:
Structure and function of membrane-associated proteins: computational
approaches to understanding experimental data
PI: David Hoogerheide
Institution: National Institute of Standards and Technology
Field of Science: Biophysics
Resource Allocation
NICS-DARTER 100,000
PSC-BRIDGES 453,035
PSC-BRIDGES-LARGE 10,000
PSC-BRIDGES-PYLON 500
Title: Simulating 3D Reaction Kinetics and Microstructure Development in Plaster
PI: Judith Terrill
Institution: National Institute of Standards and Technology
Field of Science: Materials Research
Resource Allocation
TACC-RANCH 500
TACC-STAMPEDE 566,448
TACC-RANCH 15,900
TACC-STAMPEDE 743,117

19. Startups Allocations Also Awarded
• Joseph Curtis (biophysics)
– Towards a SASSIE-web Scientific Gateway
• Nathan Mahynski (materials research)
– Designing Open Colloidal Crystals with Structure
Directing Agents
• Benjamin Neely (biological sciences)
– Developing novel tools for analyzing *omic data
sets and for biomarker discovery
19

20. XSEDE in a Broader Context:
Positioning vis-à-vis NSCI
• NSF called on to provide leadership
– scientific discovery advances
– broader HPC ecosystem for scientific discovery
– workforce development
• OAC playing key roles
– NSF lead for NSCI: Irene Qualters (OAC AD)
– NSF-wide working group: co-chaired by Rudi
Eigenmann (OAC) and Eduardo Misawa (MPS)
• Eigenmann is XSEDE Cognizant Program Officer
20

21. Scientific Discovery Advances
• Anticipate this is largely funding of basic
research
• XSEDE potential contribution
– provision of integrated resources, services, and
support enabling scientific discovery
21

22. Broader HPC Ecosystem for Scientific
Discovery
• XSEDE potential contribution
– defining and instantiating integrating
infrastructure bringing order to the ecosystem
– connector of services
• supporting integration of wide range of capabilities into
the ecosystem enabling coordinated use of same
– introduction to new technologies and facilitated
use without investment
– support services enabling use of ecosystem
22

23. Workforce Development
• XSEDE potential contribution
– significant success already in training the
workforce
– supporting the emergence of a new profession
• CI Professional/CI Engineer/etc.
– complementing campuses in developing
computational science and engineering certificate
and degree programs
• will need to treat this more broadly than CDS&E
23

24. Questions?
24