The DFC project aims to federate data grids to enable collaboration. It uses iRODS to build a federated data grid that supports reproducible science with workflows as first class objects and provenance. The project focuses on interoperability by allowing iRODS grids to interface with other systems like DataONE. It also develops tools for data discovery, access, manipulation, transformation, subsetting, and visualization from workflows. Current work involves client side tools for ingestion, access control, and integrated analysis. The project also works on standards, policies, and repository management tools to support trustworthy and sustainable data curation practices.

1. National Science Foundation Cooperative Agreement: OCI-0940841
Update March 2014
Mary Whitton, Project Manager
Reagan Moore, PI

2. Who is DFC ?

3. What does DFC do?
• Federate to enable collaboration
– Federation of iRODS-based data grids
– Interoperability for federation with other systems
• Enable reproducible science
– Workflows as first class data objects; provenance
• Build on policy-based data system (iRODS)
– Best practices in curation, archiving
– Automated data grid administration functions

4. Production
Ready

5. Production
Ready
Data
Producers
Data
Users
Curators,
Archivists
Data
Center
Managers

6. Production
Ready
Data
Producers
Data
Users
Curators,
Archivists
Data
Center
ManagersSustain-
ability
Interoper-
ability

7. SW Quality & User Community
Version 4.0 release March 31
Sustainability

8. SW Quality & User Community
Sustainability
iRODS User Meeting
June 18-19, 2014 Boston

9. Federating across systems
Interoperability
DataONE member node looks like a
another iRODS grid to iRODS user
Capability has been demonstrated
DataONE
Member
Node
iRODS
Federated
Grid
iRODS Data Grids
Interface
(via APIs)
to DataONE
Cloud
Storage

10. Federating across systems
Interoperability
DataONE
Member
Node iRODS Grid
DataONE
Coordinating
Node
Interface
(via APIs)
to DataONE
iRODS grid looks like a DataONE
member node to a DataONE user
Work is underway

11. Federating across systems
Interoperability
iRODS Grid
DataVerse
Network
DataVerseDataVerse
Work is underway

12. What our users want
• Data discovery
• Data access from a workflow
• Data manipulation (parsing of a data format)
• Data transformation
– converting to a new coordinate system)
– creating new physical variables by combining other variables
– converting to new physical units
• Data subsetting (extracting a sub-region)
• Data registration (GIS co-registration)
• Data visualization
• Creation of derived data products
Data
Users

13. Current work: client side tools
• Ingest-MediaWiki, iDropWeb
– Metadata templating, bulk uploads
– Database and indexing: plug-in in V. 4.0
• Access control
– Access for user defined “group” (my team)
• Integrated access to analysis tools
• Interfaces: Jargon, message-passing IF
framework
Data
Producers
& Users

14. Standards and Policies
Curators,
Archivists
• Community practices
and policies
– Unwritten, non-existent
• Developing
international standards
• Implementation in iRODS server
• Future: tools to make writing
rules easier

15. Repository management tools
• Best practices embodied
in iRODS rules and
policies
– Trustworthy repository
• Automatic execution
– Copy, backup,
checksum
– Triggers: time, event
Data
Center
Managers
• Tools for grid administrators

16. Production
Ready
Data
Producers
Data
Users
Curators,
Archivists
Data
Center
ManagersSustain-
ability
Interoper-
ability

17. National Science Foundation Cooperative Agreement: OCI-0940841
www.datafed.org
www.irods.org