2. 6,902
active shared
endpoints
70+
petabyte movers
675 PB
moved
23,450
active personal
endpoints
93 billion
files processed
1,868
active server
endpoints
110+
subscribers
2.9 PB
largest transfer
to date
99.9%
availability
710
identity providers
1,923
most shared
endpoints
at a single
institution 111,000
registered users
Globus by the numbers
3. Manage Protected Data
3
Higher assurance levels for HIPAA and other regulated data
• Transfer and share…
– PHI (Protected Health Information)
– PII (Personally identifiable
information)
– Controlled Unclassified Information
• Security controls comply with…
– NIST 800-53 Low
– Superset of NIST 800-171 Low
• Optional BAA with UChicago
4. Product enhancements for high assurance
• Additional authentication assurance
– Authenticate with specific identity…
– …within specific time
– …within specific session
• Application instance isolation
– Per application
– Per session (~browser session)
• Encryption of user data in transit and Globus data at rest
• Detailed audit logs: Globus service + your DTNs
5. Product enhancements for high assurance
• Additional security requirements enforced on
management of all high assurance resources
– Data access, and any interaction that can lead to data access
– Examples: Groups, Management Console
• Enhanced user interfaces (web app and CLI) for
seamless management of protected data
6. Services enabled
• Globus Services: Auth, Transfer & Sharing, Groups
• Globus Connect Server v5.2 and above
• Globus Connect Personal v3.x
• Web app (app.globus.org)
• Globus Command Line Interface (CLI)
• Connectors: POSIX, Google Drive, AWS S3, CEPH
7. Operational enhancements for high assurance
• Intrusion detection and prevention
• Encryption
• Enhanced logging
• Secure remote access, access control, and secure
practices for laptops
• Uniform configuration management and change control
• AWS best practices for secure environment: VPCs,
security groups, IAM best practices
8. New subscription levels
• High Assurance
– 33% uplift on Standard
subscription and on premium
connectors
• BAA
– 50% uplift on Standard
subscription and on premium
connectors
10. Web app enhancements
• Accessibility
– Target WCAG 2.0 AA compliance
• Responsiveness and touch
• Works with new connectors
collections.globus.org
10
11. Web app enhancements
• Customizable interface
– Single vs. dual panel
– Compact file listing display
– Columns displayed
• Continue incorporating
user feedback
12. CLI enhancements
• Support for use with high assurance collections
• '--format UNIX': output suitable for line-oriented
processing with typical Unix tools
• Added 'globus rm' command
• 'globus whoami --linked-identities': shows all
linked identities
• '--timeout-exit-code': overrides the default exit code
for commands which wait on tasks
• Enhancements to SDK as needed
12
13. Globus for Box
• Extends the value of your Box deployment
• Unifies access to cloud and on-premise storage
• Transitions protected data (HIPAA-regulated, CUI)
seamlessly between Box and other storage systems
13
16. Make Box part of your
research storage ecosystem
globus.org/connectors/box
docs.globus.org/premium-storage-connectors/box
17. Connector updates
• Enhanced user experience for credential handling for
several connectors (GCSv5)
• AWS S3
– Automated multi-region support
• Google Drive
– Enhancement to retry handling for large transfers
• HPSS
– Support added for HPSS 7.5 (7.3 to 7.5 supported)
– Improved asynchronous staging from tape
17
18. S3 compatible systems
• Initial customer deployments
• Validation, testing and vendor
engagement planned
• Additional systems driven by
customer demand
PLEASE CONTACT US BEFORE
DEPLOYING ANY OF THESE!
18
21. Globus Connect Server v5.3
• Subsumes GCS version 5.0, 5.1, 5.2 (upgrade now)
• Standard and high assurance guest collections (sharing)
• High assurance mapped collections
• Connectors: POSIX, AWS S3, CEPH, Google Drive, Box
• High assurance, standard gateways on same endpoint
• Data access protocols: GridFTP and HTTPS
22. HTTPS access to Globus endpoints
• Browser based
up/download
• Put your (research)
storage “on the web”
• Enforce same
security policies
22
23. Globus Connect Server v5 Milestones
v5.0: Google
Drive
v5.1: POSIX guest
collections, HTTPS
v5.x: v4 feature parity+
v5.3
• Multi DTN support
• Additional storage
systems
• Endpoint specific
identity providers
• …
Other
features
v5.2: High
assurance
v5.4: …
24. Recent Transfer enhancements
• Verify transfer using client provided checksums
– User provided checksum used rather than source checksum for
verification
• Improvements for scaling transfer service
– Multiple nodes for transfer service for higher availability and
reliability
– Allows for code updates with no downtime
24
25. SSH with OAuth
• Securely access resource using SSH with federated identity
– Facilitates automation, eliminates SSH key management
– Replacement for deprecated GSI OpenSSH
• First version released
– Server side PAM module with Globus Auth support
– Command line client
• Open source, community support
– Not part of the standard subscription
– OAuth SSH Client: https://pypi.org/project/oauth-ssh/
– OAuth SSH Server PAM module: https://github.com/xsede/oauth-ssh
28. Globus Transfer: A complete solution
☑ Bulk transfer and sync
☑ Good end-to-end performance in myriad of real world settings
☑ End-to-end reliability
☑ Robust security, with federated identities
☑ Layers onto diverse storage systems
☑ Web-compatible client/server remote access
☑ Easy to use interfaces
☑ Easy installation and administration
☑ Sharing data with guest users
☑ Dedicated professional support
28
29. Rethinking data publication
• Limited adoption
– Not easily customizable
• Maintenance Challenges
– Costly to maintain
– JRE licensing concerns
• Going forward
– Code will be open source
– Leverage platform
• Invest in higher priorities
30. JLSEUChicago
ALCFAPS
Publication7
Kasthuri Lab: Building the connectome
Imaging1
Lab Server 1
Acquisition2
Lab Server 2
Pre-processing3 Preview/Center4
Reconstruction6Visualization8
User validation5
Science!9
Neuroanatomy
reconstruction
pipeline
32. Our (ambitious) goals for the Globus platform
• Transform how research applications, services, and
workflows are created, delivered, used, and sustained
– Scientific instrument data processing
– Repositories: Make data more FAIR
– Science gateways
• Facilitate creation of interoperable app ecosystem
32
33. Globus platform services
• Identity and Access Management (IAM)
– Federated identity login, Groups, Attributes, Access Control
– Globus Auth: Oauth authorization provider
• Connect
• Transfer
– Building a family of services
• Execution
• Search, Identifiers
• Automation
– Queues, Events
– Triggers, Actions, Flows
33
34. Platform status
• Generally Available in a few years
• Separate product with separate sustainability model
• Early engagements help shape product direction
– Argonne Leadership Computing Facility, Materials Data Facility,
– NCAR Research Data Archive, NSO, …
– Use in Globus products
• Multiple integrations facilitate more complete solution
– e.g. Django, JupyterHub
– Follow progress: globus-integration-examples.readthedocs.io
• Currently accessible via professional services team
35. Thank you to our sponsors...
U . S . D E P A R T M E N T O F
ENERGY
Use cases – HIPAA/protected data enclave, multi-institutional trials,
Access Control
Identities provided and managed by institution
Globus acts as identity broker only, does not access or store any institutional user credentials
Institution controls all access policies (at multiple levels)
who can access what data and with what permissions
who can share what data and with what permissions
all access policies can be changed or revoked at any time
Protected Environment using either AWS KMS encryption or AWS service-specific encryption options.
Data access
CLI access
Private window
Groups page
we are working towards being compliant with W3 established accessibility standards –increased visual contrast, ability to resize all of the GUI text using standard browser controls, code to support screen readers, the GUI adapts to a users’ screen from mobile devices all the way up to large high resolution desktop displays
Greg’s talk
Backup, data management plans, archive…use Globus for all those use cases
Layers on Box
Our current set of premium connectors; care and feed
Box collection creation
Data transfer
Sharing
Pricing – same as Google Drive connector
Care and feed
Customer demand + sustainble model for maintaining the connectors. Ask for any input.
And what we once called endpoints are now called Collections
Mapped = host endpoint
Guest = shared endpoint
Don’t forget we now also offer HTTPS as well!
Hand to Steve who takes it from here…
BE POSITIVE!
IT’S ALL GOOD!
WE’RE DOING IT FOR YOUR OWN GOOD!
WE CAN INVEST ON COOL NEW STUFF
A good example of the science that Globus facilitates is the work being done by Bobby Kasthuri Lab at Argonne Lab
HIs group have set out to map the brain, or build the connectome, as it’s known
It’s an ambitious undertaking that involves massive mounts of data
They start with samples that are imaged using a beamline at the Advanced Photon Source
They get time from APS every 2 to 3 days, and efficiency and automation allows them to make the most of the time at APS.
The initial set of raw images undergo some prep-processing and are sent to the Argonne Leadership Computing Facility for analysis
A scientist then previews the images and makes any needed adjustments to the experiment
Once everything is properly configured the sample is fully imaged and reconstructed at the ALCF
The datasets are then moved to a petascale storage system called Petrel where they are annotated with metadata and published with a persistent identifier
Researchers at Uchicago and elsewhere can then search and extract relevant subsets of the data to analyze further
…and then Science happens!
No timeframe for general availability
Last year we talked about services such as Automate and Search. These are fundamentally platform services, for developers to leverage in their own applications, services, and solutions.
We continue to make good progress on these
However, realistically, due to funding constraints, these platforms services will not be generally available for several more year
In the mean time we are partnering with select groups to use prototype and limited production platform services
To learn what is really needed before going GA
If you are interested in experimenting with them, please talk with us. We will be selective.