SlideShare une entreprise Scribd logo
1  sur  16
The Federated Data System DataFed: Experiences in Data Homogenization and Networking ,[object Object],[object Object],Spring AGU, May 29, 2008, Ft. Lauderdale
DataFed in a Nutshell: A Federation of  autonomous, distributed  data providers Performs non-intrusive  wrapping   of data into web services Provides service-based  analysis services  and tools General Experience with DataFed: It is an  agile  virtual data system can deliver  info products to diverse users Third-party  mediation can homogenize  distributed data on the fly Since 2005,  DataFed is used  by EPA and in research   DataFed Motivated by GEOSS DataFed development is guided by the meme of GEOSS
Five practices for agile, seamless data federation: ,[object Object],[object Object],[object Object],[object Object],[object Object]
Parameter-Space-Time Query  Using OGC WCS Data Access Protocol ,[object Object],[object Object],[object Object],Coverage=THEEDDS.T&  BBOX=-126,24,-65,52,0,0 &TIME=2002-07-07/2002-07-07&FORMAT=NetCDF Coverage=SEAW.Refl&  BBOX=-126,24,-65,52,0,0 &TIME=2002-07-07/2002-07-07&FORMAT=GeoTIFF Coverage=SURF.Bext&  BBOX=-126,24,-65,52,0,0 &TIME=2002-07-07/2002-07-07&FORMAT=NetCDF-table Grid Image Station Data Parameter Bounding Box Time Range Out Format
[object Object],Third Party Data Wrappers   Heterogeneous input data  >>> Homogeneous (WCS) Query
Mediated User-Data Interface Mediator turns data into Views Mediated Integration  is a flexible design pattern for System of Systems   Client-Server  design is demanding: User carries the burden of integration Query Data Views
SOAP RDF Mashup Workflow Mashups: Loose Coupling of Autonomous Applications DataFed – Wiki -- GoogleEarth
DataSpaces for Datasets GEOSS Comp. Registry Community AQ Portal extracts Service Offeror registers GEOSS Clearinghouse Catalog list Searches,  harvests invokes references publishes provides Standards; SIF Registry Adopted from  Percivall , Feb 2008  by R. Husar, March 2008 Community AQ Catalog Catalog User Service Workflow composes Data Analyst visualizes Reports to Decision Maker Policy Analyst Informs find links to GEOSS Core Service Offerors and  Users Shared Metadata by the Users, for the Users   Services Community DataSpaces
GEOSS Comp. Registry Community AQ Portal extracts Service Offeror registers GEOSS Clearinghouse Catalog list Searches,  harvests invokes references publishes provides Standards; SIF Registry Adopted from  Percivall , Feb 2008  by R. Husar, March 2008 Community AQ Catalog Service Workflow composes Data Analyst visualizes Reports to Decision Maker Policy Analyst Informs find links to GEOSS Core Service Offerors and  Users Shared Metadata by the Users, for the Users   views report Services Community DataSpaces
-+ GEOSS Comp. Registry Community AQ Portal extracts Service Offeror registers GEOSS Clearinghouse Catalog list Searches,  harvests invokes references publishes provides Standards; SIF Registry Adopted from  Percivall , Feb 2008  by R. Husar, March 2008 Community AQ Catalog Service Workflow composes Data Analyst visualizes Decision Maker Policy Analyst Informs find links to GEOSS Core Service Offerors and  Users Shared Metadata by the Users, for the Users   views report Services Community DataSpaces
Wiki ‘DataSpaces’ Creating and Sharing Metadata Community Catalog - Find Dataset Describe Dataset Discuss Dataset ESIP Communal Wiki ,[object Object],[object Object],[object Object]
Sharing Best Practices: GEO Best Practice Wiki
Developments and Challenges: ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
ESIP Coordination Application
 
 

Contenu connexe

Plus de Rudolf Husar

111018 geo sif_aq_interop
111018 geo sif_aq_interop111018 geo sif_aq_interop
111018 geo sif_aq_interop
Rudolf Husar
 
110823 solta11 intro
110823 solta11 intro110823 solta11 intro
110823 solta11 intro
Rudolf Husar
 
110823 data fed_solta11
110823 data fed_solta11110823 data fed_solta11
110823 data fed_solta11
Rudolf Husar
 
110510 aq co_p_network
110510 aq co_p_network110510 aq co_p_network
110510 aq co_p_network
Rudolf Husar
 
110509 aq co_p_solta
110509 aq co_p_solta110509 aq co_p_solta
110509 aq co_p_solta
Rudolf Husar
 
110421 exploration of_pm_networks_and_data_over_the_us-_aqs_and_views
110421 exploration of_pm_networks_and_data_over_the_us-_aqs_and_views110421 exploration of_pm_networks_and_data_over_the_us-_aqs_and_views
110421 exploration of_pm_networks_and_data_over_the_us-_aqs_and_views
Rudolf Husar
 
110410 aq user_req_methodology_sydney_subm
110410 aq user_req_methodology_sydney_subm110410 aq user_req_methodology_sydney_subm
110410 aq user_req_methodology_sydney_subm
Rudolf Husar
 
110408 aq co_p_uic_sydney_husar
110408 aq co_p_uic_sydney_husar110408 aq co_p_uic_sydney_husar
110408 aq co_p_uic_sydney_husar
Rudolf Husar
 
110105 htap pilot_aqco_p_esip_dc
110105 htap pilot_aqco_p_esip_dc110105 htap pilot_aqco_p_esip_dc
110105 htap pilot_aqco_p_esip_dc
Rudolf Husar
 
100615 htap network_brussels
100615 htap network_brussels100615 htap network_brussels
100615 htap network_brussels
Rudolf Husar
 
121117 eedss briefing_nasa_epa
121117 eedss briefing_nasa_epa121117 eedss briefing_nasa_epa
121117 eedss briefing_nasa_epa
Rudolf Husar
 
120910 nasa satellite_outline
120910 nasa satellite_outline120910 nasa satellite_outline
120910 nasa satellite_outline
Rudolf Husar
 
120612 geia closure_ofeo_ms_soa_subm
120612 geia closure_ofeo_ms_soa_subm120612 geia closure_ofeo_ms_soa_subm
120612 geia closure_ofeo_ms_soa_subm
Rudolf Husar
 
110414 extreme dustsmokesulfate
110414 extreme dustsmokesulfate110414 extreme dustsmokesulfate
110414 extreme dustsmokesulfate
Rudolf Husar
 
Aq Gci Infrastructure
Aq Gci InfrastructureAq Gci Infrastructure
Aq Gci Infrastructure
Rudolf Husar
 
AQ GCI Infrastructure
AQ GCI InfrastructureAQ GCI Infrastructure
AQ GCI Infrastructure
Rudolf Husar
 
2004-06-20 Fast Aerosol Sensing Tools for Natural Event Tracking FASTNET
2004-06-20 Fast Aerosol Sensing Tools for Natural Event Tracking FASTNET2004-06-20 Fast Aerosol Sensing Tools for Natural Event Tracking FASTNET
2004-06-20 Fast Aerosol Sensing Tools for Natural Event Tracking FASTNET
Rudolf Husar
 
2004-06-23 Retrieval of smoke aerosol loading from remote sensing data
2004-06-23 Retrieval of smoke aerosol loading from remote sensing data2004-06-23 Retrieval of smoke aerosol loading from remote sensing data
2004-06-23 Retrieval of smoke aerosol loading from remote sensing data
Rudolf Husar
 
2004-06-24 Co-retrieval of Aerosol and Surface Reflectance: Analysis of Daily...
2004-06-24 Co-retrieval of Aerosol and Surface Reflectance: Analysis of Daily...2004-06-24 Co-retrieval of Aerosol and Surface Reflectance: Analysis of Daily...
2004-06-24 Co-retrieval of Aerosol and Surface Reflectance: Analysis of Daily...
Rudolf Husar
 
2004-06-24 Fast Aerosol Sensing Tools for Natural Event Tracking FASTNET Proj...
2004-06-24 Fast Aerosol Sensing Tools for Natural Event Tracking FASTNET Proj...2004-06-24 Fast Aerosol Sensing Tools for Natural Event Tracking FASTNET Proj...
2004-06-24 Fast Aerosol Sensing Tools for Natural Event Tracking FASTNET Proj...
Rudolf Husar
 

Plus de Rudolf Husar (20)

111018 geo sif_aq_interop
111018 geo sif_aq_interop111018 geo sif_aq_interop
111018 geo sif_aq_interop
 
110823 solta11 intro
110823 solta11 intro110823 solta11 intro
110823 solta11 intro
 
110823 data fed_solta11
110823 data fed_solta11110823 data fed_solta11
110823 data fed_solta11
 
110510 aq co_p_network
110510 aq co_p_network110510 aq co_p_network
110510 aq co_p_network
 
110509 aq co_p_solta
110509 aq co_p_solta110509 aq co_p_solta
110509 aq co_p_solta
 
110421 exploration of_pm_networks_and_data_over_the_us-_aqs_and_views
110421 exploration of_pm_networks_and_data_over_the_us-_aqs_and_views110421 exploration of_pm_networks_and_data_over_the_us-_aqs_and_views
110421 exploration of_pm_networks_and_data_over_the_us-_aqs_and_views
 
110410 aq user_req_methodology_sydney_subm
110410 aq user_req_methodology_sydney_subm110410 aq user_req_methodology_sydney_subm
110410 aq user_req_methodology_sydney_subm
 
110408 aq co_p_uic_sydney_husar
110408 aq co_p_uic_sydney_husar110408 aq co_p_uic_sydney_husar
110408 aq co_p_uic_sydney_husar
 
110105 htap pilot_aqco_p_esip_dc
110105 htap pilot_aqco_p_esip_dc110105 htap pilot_aqco_p_esip_dc
110105 htap pilot_aqco_p_esip_dc
 
100615 htap network_brussels
100615 htap network_brussels100615 htap network_brussels
100615 htap network_brussels
 
121117 eedss briefing_nasa_epa
121117 eedss briefing_nasa_epa121117 eedss briefing_nasa_epa
121117 eedss briefing_nasa_epa
 
120910 nasa satellite_outline
120910 nasa satellite_outline120910 nasa satellite_outline
120910 nasa satellite_outline
 
120612 geia closure_ofeo_ms_soa_subm
120612 geia closure_ofeo_ms_soa_subm120612 geia closure_ofeo_ms_soa_subm
120612 geia closure_ofeo_ms_soa_subm
 
110414 extreme dustsmokesulfate
110414 extreme dustsmokesulfate110414 extreme dustsmokesulfate
110414 extreme dustsmokesulfate
 
Aq Gci Infrastructure
Aq Gci InfrastructureAq Gci Infrastructure
Aq Gci Infrastructure
 
AQ GCI Infrastructure
AQ GCI InfrastructureAQ GCI Infrastructure
AQ GCI Infrastructure
 
2004-06-20 Fast Aerosol Sensing Tools for Natural Event Tracking FASTNET
2004-06-20 Fast Aerosol Sensing Tools for Natural Event Tracking FASTNET2004-06-20 Fast Aerosol Sensing Tools for Natural Event Tracking FASTNET
2004-06-20 Fast Aerosol Sensing Tools for Natural Event Tracking FASTNET
 
2004-06-23 Retrieval of smoke aerosol loading from remote sensing data
2004-06-23 Retrieval of smoke aerosol loading from remote sensing data2004-06-23 Retrieval of smoke aerosol loading from remote sensing data
2004-06-23 Retrieval of smoke aerosol loading from remote sensing data
 
2004-06-24 Co-retrieval of Aerosol and Surface Reflectance: Analysis of Daily...
2004-06-24 Co-retrieval of Aerosol and Surface Reflectance: Analysis of Daily...2004-06-24 Co-retrieval of Aerosol and Surface Reflectance: Analysis of Daily...
2004-06-24 Co-retrieval of Aerosol and Surface Reflectance: Analysis of Daily...
 
2004-06-24 Fast Aerosol Sensing Tools for Natural Event Tracking FASTNET Proj...
2004-06-24 Fast Aerosol Sensing Tools for Natural Event Tracking FASTNET Proj...2004-06-24 Fast Aerosol Sensing Tools for Natural Event Tracking FASTNET Proj...
2004-06-24 Fast Aerosol Sensing Tools for Natural Event Tracking FASTNET Proj...
 

Dernier

Histor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slideHistor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slide
vu2urc
 
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
?#DUbAI#??##{{(☎️+971_581248768%)**%*]'#abortion pills for sale in dubai@
 

Dernier (20)

How to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerHow to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected Worker
 
Histor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slideHistor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slide
 
HTML Injection Attacks: Impact and Mitigation Strategies
HTML Injection Attacks: Impact and Mitigation StrategiesHTML Injection Attacks: Impact and Mitigation Strategies
HTML Injection Attacks: Impact and Mitigation Strategies
 
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, AdobeApidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
 
Understanding Discord NSFW Servers A Guide for Responsible Users.pdf
Understanding Discord NSFW Servers A Guide for Responsible Users.pdfUnderstanding Discord NSFW Servers A Guide for Responsible Users.pdf
Understanding Discord NSFW Servers A Guide for Responsible Users.pdf
 
Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024
 
Exploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone ProcessorsExploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone Processors
 
Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024
 
Scaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organizationScaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organization
 
Tech Trends Report 2024 Future Today Institute.pdf
Tech Trends Report 2024 Future Today Institute.pdfTech Trends Report 2024 Future Today Institute.pdf
Tech Trends Report 2024 Future Today Institute.pdf
 
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
 
TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data DiscoveryTrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
 
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
 
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law DevelopmentsTrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
 
GenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day PresentationGenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day Presentation
 
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
 
Boost PC performance: How more available memory can improve productivity
Boost PC performance: How more available memory can improve productivityBoost PC performance: How more available memory can improve productivity
Boost PC performance: How more available memory can improve productivity
 
Strategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a FresherStrategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a Fresher
 
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
 
Handwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed textsHandwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed texts
 

2008-05-27: Spring AGU, DataFed Best Practices

  • 1.
  • 2. DataFed in a Nutshell: A Federation of autonomous, distributed data providers Performs non-intrusive wrapping of data into web services Provides service-based analysis services and tools General Experience with DataFed: It is an agile virtual data system can deliver info products to diverse users Third-party mediation can homogenize distributed data on the fly Since 2005, DataFed is used by EPA and in research DataFed Motivated by GEOSS DataFed development is guided by the meme of GEOSS
  • 3.
  • 4.
  • 5.
  • 6. Mediated User-Data Interface Mediator turns data into Views Mediated Integration is a flexible design pattern for System of Systems Client-Server design is demanding: User carries the burden of integration Query Data Views
  • 7. SOAP RDF Mashup Workflow Mashups: Loose Coupling of Autonomous Applications DataFed – Wiki -- GoogleEarth
  • 8. DataSpaces for Datasets GEOSS Comp. Registry Community AQ Portal extracts Service Offeror registers GEOSS Clearinghouse Catalog list Searches, harvests invokes references publishes provides Standards; SIF Registry Adopted from Percivall , Feb 2008 by R. Husar, March 2008 Community AQ Catalog Catalog User Service Workflow composes Data Analyst visualizes Reports to Decision Maker Policy Analyst Informs find links to GEOSS Core Service Offerors and Users Shared Metadata by the Users, for the Users Services Community DataSpaces
  • 9. GEOSS Comp. Registry Community AQ Portal extracts Service Offeror registers GEOSS Clearinghouse Catalog list Searches, harvests invokes references publishes provides Standards; SIF Registry Adopted from Percivall , Feb 2008 by R. Husar, March 2008 Community AQ Catalog Service Workflow composes Data Analyst visualizes Reports to Decision Maker Policy Analyst Informs find links to GEOSS Core Service Offerors and Users Shared Metadata by the Users, for the Users views report Services Community DataSpaces
  • 10. -+ GEOSS Comp. Registry Community AQ Portal extracts Service Offeror registers GEOSS Clearinghouse Catalog list Searches, harvests invokes references publishes provides Standards; SIF Registry Adopted from Percivall , Feb 2008 by R. Husar, March 2008 Community AQ Catalog Service Workflow composes Data Analyst visualizes Decision Maker Policy Analyst Informs find links to GEOSS Core Service Offerors and Users Shared Metadata by the Users, for the Users views report Services Community DataSpaces
  • 11.
  • 12. Sharing Best Practices: GEO Best Practice Wiki
  • 13.
  • 15.  
  • 16.  

Notes de l'éditeur

  1. This talk is about the federated.. In particular our experineces with datafed over past 4 years of data homog and networking