SlideShare une entreprise Scribd logo
1  sur  7
Télécharger pour lire hors ligne
HDF-EOS Datablade:
Efficiently Serving Earth Science Data

http://heineken.gsfc.nasa.gov/hdfeos/
What is a Datablade?
Image
HDF-EOS

Spatial

DataBlade API

Text

Web

Extending the server by
adding DataBlades: new Types
and new functions

Extensible
OR Engine

Scaleable
Data
Manager

INFORMIX-Universal Server

http://heineken.gsfc.nasa.gov/hdfeos/
What does the HDF-EOS Datablade
provide?
• Inventory management of large volume data and
metadata
• Flexible access to the data as a file or BinaryLarge-Object inside the database.
• Cross-granule searching and subsetting
• Re-use other datablades such as spatial search
tools from Informix
• Implements Grid subsetting operations in the
Database
http://heineken.gsfc.nasa.gov/hdfeos/
Plans for the Second Year
•
•
•
•

Implement Swath, Point, HDF datatypes
General subsetting tool: stored queries
Metadata search and query tools
Implement MISR Quality-Assessment database

http://heineken.gsfc.nasa.gov/hdfeos/
HDF-EOS Datablade Software Architecture
Informix DB

SQL
SQL
InterInterface
face
and
API

HDF-EOS
C library
(Unchanged)

Valuable
Data
Exposed
in Searchable
Tables

File
I/O
Emulator

http://heineken.gsfc.nasa.gov/hdfeos/

Granules
in
File
System

Granules
in
Informix
Overlapping Granule Scenario

Latitude

90
80
70
60
50
40
30
20
10

Subset: Lat: 65 to 75
Lon: 15 to 25

0
1
2
3
4
10

30

50

70

90

Longitude

http://heineken.gsfc.nasa.gov/hdfeos/
Overlapping Granule Scenario

Latitude

90
80
70
60
50
40
30
20
10

Subset: Lat: 65 to 75
Lon: 15 to 25

0
1
2
3
4
10

30

50

70

90

Longitude

http://heineken.gsfc.nasa.gov/hdfeos/

Contenu connexe

Tendances

MongoDB for Spatio-Behavioral Data Analysis and Visualization
MongoDB for Spatio-Behavioral Data Analysis and VisualizationMongoDB for Spatio-Behavioral Data Analysis and Visualization
MongoDB for Spatio-Behavioral Data Analysis and VisualizationMongoDB
 
DataTalks #4: Построение хранилища данных на основе платформы hadoop / Игорь ...
DataTalks #4: Построение хранилища данных на основе платформы hadoop / Игорь ...DataTalks #4: Построение хранилища данных на основе платформы hadoop / Игорь ...
DataTalks #4: Построение хранилища данных на основе платформы hadoop / Игорь ...WG_ Events
 
041018 Esds Poster
041018 Esds Poster041018 Esds Poster
041018 Esds PosterRudolf Husar
 
Data Management on Hadoop at Yahoo!
Data Management on Hadoop at Yahoo!Data Management on Hadoop at Yahoo!
Data Management on Hadoop at Yahoo!Seetharam Venkatesh
 
Hadoop 2 cluster architecture
Hadoop 2 cluster architectureHadoop 2 cluster architecture
Hadoop 2 cluster architectureSandeep Patil
 
Media iQ fifth elephant teaser
Media iQ   fifth elephant teaser Media iQ   fifth elephant teaser
Media iQ fifth elephant teaser prabhuprakash
 
Data Center Operating System
Data Center Operating SystemData Center Operating System
Data Center Operating SystemKeshav Yadav
 
Cloudgene - A MapReduce based Workflow Management System
Cloudgene - A MapReduce based Workflow Management SystemCloudgene - A MapReduce based Workflow Management System
Cloudgene - A MapReduce based Workflow Management SystemLukas Forer
 
Observability for Data Pipelines With OpenLineage
Observability for Data Pipelines With OpenLineageObservability for Data Pipelines With OpenLineage
Observability for Data Pipelines With OpenLineageDatabricks
 

Tendances (20)

view_hdf
view_hdfview_hdf
view_hdf
 
PoolParty Search Server
PoolParty Search ServerPoolParty Search Server
PoolParty Search Server
 
MongoDB for Spatio-Behavioral Data Analysis and Visualization
MongoDB for Spatio-Behavioral Data Analysis and VisualizationMongoDB for Spatio-Behavioral Data Analysis and Visualization
MongoDB for Spatio-Behavioral Data Analysis and Visualization
 
HDFCloud Workshop: HDF5 in the Cloud
HDFCloud Workshop: HDF5 in the CloudHDFCloud Workshop: HDF5 in the Cloud
HDFCloud Workshop: HDF5 in the Cloud
 
DataTalks #4: Построение хранилища данных на основе платформы hadoop / Игорь ...
DataTalks #4: Построение хранилища данных на основе платформы hadoop / Игорь ...DataTalks #4: Построение хранилища данных на основе платформы hadoop / Игорь ...
DataTalks #4: Построение хранилища данных на основе платформы hadoop / Игорь ...
 
Improved Methods for Accessing Scientific Data for the Masses
Improved Methods for Accessing Scientific Data for the MassesImproved Methods for Accessing Scientific Data for the Masses
Improved Methods for Accessing Scientific Data for the Masses
 
Scientific Computing and Visualization using HDF
Scientific Computing and Visualization using HDFScientific Computing and Visualization using HDF
Scientific Computing and Visualization using HDF
 
041018 Esds Poster
041018 Esds Poster041018 Esds Poster
041018 Esds Poster
 
Data Management on Hadoop at Yahoo!
Data Management on Hadoop at Yahoo!Data Management on Hadoop at Yahoo!
Data Management on Hadoop at Yahoo!
 
Hadoop 2 cluster architecture
Hadoop 2 cluster architectureHadoop 2 cluster architecture
Hadoop 2 cluster architecture
 
Media iQ fifth elephant teaser
Media iQ   fifth elephant teaser Media iQ   fifth elephant teaser
Media iQ fifth elephant teaser
 
Mapreduce Tutorial
Mapreduce TutorialMapreduce Tutorial
Mapreduce Tutorial
 
NASA Terra Data Fusion
NASA Terra Data FusionNASA Terra Data Fusion
NASA Terra Data Fusion
 
Hadoop 1 vs hadoop2
Hadoop 1 vs hadoop2Hadoop 1 vs hadoop2
Hadoop 1 vs hadoop2
 
VINEYARD Overview - ARC 2016
VINEYARD Overview - ARC 2016VINEYARD Overview - ARC 2016
VINEYARD Overview - ARC 2016
 
Data Center Operating System
Data Center Operating SystemData Center Operating System
Data Center Operating System
 
SPD and KEA: HDF5 based file formats for Earth Observation
SPD and KEA: HDF5 based file formats for Earth ObservationSPD and KEA: HDF5 based file formats for Earth Observation
SPD and KEA: HDF5 based file formats for Earth Observation
 
Practical Use of a NoSQL Database
Practical Use of a NoSQL DatabasePractical Use of a NoSQL Database
Practical Use of a NoSQL Database
 
Cloudgene - A MapReduce based Workflow Management System
Cloudgene - A MapReduce based Workflow Management SystemCloudgene - A MapReduce based Workflow Management System
Cloudgene - A MapReduce based Workflow Management System
 
Observability for Data Pipelines With OpenLineage
Observability for Data Pipelines With OpenLineageObservability for Data Pipelines With OpenLineage
Observability for Data Pipelines With OpenLineage
 

En vedette

Beer Facts
Beer FactsBeer Facts
Beer Factsclapp3k
 
Marketing Research - Heineken
Marketing Research - HeinekenMarketing Research - Heineken
Marketing Research - HeinekenKhoa Fanda
 
Heineken summary
Heineken summaryHeineken summary
Heineken summaryJCDecauxUK
 
People's Insights Quarterly Magazine Issue 2
People's Insights Quarterly Magazine Issue 2People's Insights Quarterly Magazine Issue 2
People's Insights Quarterly Magazine Issue 2Olivier Fleurot
 
Carlsberg: Student Trends
Carlsberg: Student TrendsCarlsberg: Student Trends
Carlsberg: Student TrendsNice Curve
 
IESEG (Lille) - 'Contemporary Market Research Management & Techniques'
IESEG (Lille) - 'Contemporary Market Research Management & Techniques'IESEG (Lille) - 'Contemporary Market Research Management & Techniques'
IESEG (Lille) - 'Contemporary Market Research Management & Techniques'Tom De Ruyck
 
Heineken Experience
Heineken ExperienceHeineken Experience
Heineken ExperienceKlaus Hoven
 
Marketing 360 presentation by lawrence villegas - Connections Media examples...
Marketing 360 presentation by lawrence villegas -  Connections Media examples...Marketing 360 presentation by lawrence villegas -  Connections Media examples...
Marketing 360 presentation by lawrence villegas - Connections Media examples...Lawrence Villegas
 
David Spacey, Director of Global Procurement at Heineken - Incorporating sus...
David Spacey, Director of Global Procurement at  Heineken - Incorporating sus...David Spacey, Director of Global Procurement at  Heineken - Incorporating sus...
David Spacey, Director of Global Procurement at Heineken - Incorporating sus...Global Business Events
 
David Paterson from Heineken PLC - Profiting from Sustainability Feedback Ses...
David Paterson from Heineken PLC - Profiting from Sustainability Feedback Ses...David Paterson from Heineken PLC - Profiting from Sustainability Feedback Ses...
David Paterson from Heineken PLC - Profiting from Sustainability Feedback Ses...Stevencann1
 
Heineken measuring YouTube roi vs tv
Heineken measuring YouTube roi vs tvHeineken measuring YouTube roi vs tv
Heineken measuring YouTube roi vs tvMitya Voskresensky
 
Brands & social networks flash survey: New Findings on how consumers connect ...
Brands & social networks flash survey: New Findings on how consumers connect ...Brands & social networks flash survey: New Findings on how consumers connect ...
Brands & social networks flash survey: New Findings on how consumers connect ...VIVALDI
 
QR Codes: A Point of View
QR Codes: A Point of ViewQR Codes: A Point of View
QR Codes: A Point of ViewBBDO
 
Heineken Case Study: Perceptual Mapping of Consumers Analysis
Heineken Case Study: Perceptual Mapping of Consumers AnalysisHeineken Case Study: Perceptual Mapping of Consumers Analysis
Heineken Case Study: Perceptual Mapping of Consumers AnalysisLara Zaccaria
 
Heineken presentation bingül&eda
Heineken   presentation bingül&edaHeineken   presentation bingül&eda
Heineken presentation bingül&edaYalcin Bingul
 
Social Media Marketing Case Study - Heineken International
Social Media Marketing Case Study - Heineken International Social Media Marketing Case Study - Heineken International
Social Media Marketing Case Study - Heineken International SocialMedia8
 

En vedette (20)

Beer Facts
Beer FactsBeer Facts
Beer Facts
 
Marketing Research - Heineken
Marketing Research - HeinekenMarketing Research - Heineken
Marketing Research - Heineken
 
Heineken summary
Heineken summaryHeineken summary
Heineken summary
 
People's Insights Quarterly Magazine Issue 2
People's Insights Quarterly Magazine Issue 2People's Insights Quarterly Magazine Issue 2
People's Insights Quarterly Magazine Issue 2
 
Carlsberg: Student Trends
Carlsberg: Student TrendsCarlsberg: Student Trends
Carlsberg: Student Trends
 
Best cases time
Best cases timeBest cases time
Best cases time
 
IESEG (Lille) - 'Contemporary Market Research Management & Techniques'
IESEG (Lille) - 'Contemporary Market Research Management & Techniques'IESEG (Lille) - 'Contemporary Market Research Management & Techniques'
IESEG (Lille) - 'Contemporary Market Research Management & Techniques'
 
WFA Annual Report 2013
WFA Annual Report 2013WFA Annual Report 2013
WFA Annual Report 2013
 
Ethnic Marketing
Ethnic MarketingEthnic Marketing
Ethnic Marketing
 
The Best Intranets in Europe
The Best Intranets in EuropeThe Best Intranets in Europe
The Best Intranets in Europe
 
Heineken Experience
Heineken ExperienceHeineken Experience
Heineken Experience
 
Marketing 360 presentation by lawrence villegas - Connections Media examples...
Marketing 360 presentation by lawrence villegas -  Connections Media examples...Marketing 360 presentation by lawrence villegas -  Connections Media examples...
Marketing 360 presentation by lawrence villegas - Connections Media examples...
 
David Spacey, Director of Global Procurement at Heineken - Incorporating sus...
David Spacey, Director of Global Procurement at  Heineken - Incorporating sus...David Spacey, Director of Global Procurement at  Heineken - Incorporating sus...
David Spacey, Director of Global Procurement at Heineken - Incorporating sus...
 
David Paterson from Heineken PLC - Profiting from Sustainability Feedback Ses...
David Paterson from Heineken PLC - Profiting from Sustainability Feedback Ses...David Paterson from Heineken PLC - Profiting from Sustainability Feedback Ses...
David Paterson from Heineken PLC - Profiting from Sustainability Feedback Ses...
 
Heineken measuring YouTube roi vs tv
Heineken measuring YouTube roi vs tvHeineken measuring YouTube roi vs tv
Heineken measuring YouTube roi vs tv
 
Brands & social networks flash survey: New Findings on how consumers connect ...
Brands & social networks flash survey: New Findings on how consumers connect ...Brands & social networks flash survey: New Findings on how consumers connect ...
Brands & social networks flash survey: New Findings on how consumers connect ...
 
QR Codes: A Point of View
QR Codes: A Point of ViewQR Codes: A Point of View
QR Codes: A Point of View
 
Heineken Case Study: Perceptual Mapping of Consumers Analysis
Heineken Case Study: Perceptual Mapping of Consumers AnalysisHeineken Case Study: Perceptual Mapping of Consumers Analysis
Heineken Case Study: Perceptual Mapping of Consumers Analysis
 
Heineken presentation bingül&eda
Heineken   presentation bingül&edaHeineken   presentation bingül&eda
Heineken presentation bingül&eda
 
Social Media Marketing Case Study - Heineken International
Social Media Marketing Case Study - Heineken International Social Media Marketing Case Study - Heineken International
Social Media Marketing Case Study - Heineken International
 

Similaire à HDF-EOS Datablade: Efficiently Serving Earth Science Data

Hw09 Rethinking The Data Warehouse With Hadoop And Hive
Hw09   Rethinking The Data Warehouse With Hadoop And HiveHw09   Rethinking The Data Warehouse With Hadoop And Hive
Hw09 Rethinking The Data Warehouse With Hadoop And HiveCloudera, Inc.
 
Modernizing Your Data Warehouse using APS
Modernizing Your Data Warehouse using APSModernizing Your Data Warehouse using APS
Modernizing Your Data Warehouse using APSStéphane Fréchette
 
Testing Big Data: Automated ETL Testing of Hadoop
Testing Big Data: Automated ETL Testing of HadoopTesting Big Data: Automated ETL Testing of Hadoop
Testing Big Data: Automated ETL Testing of HadoopRTTS
 
Lessons Learned Migrating from IBM BigInsights to Hortonworks Data Platform
Lessons Learned Migrating from IBM BigInsights to Hortonworks Data PlatformLessons Learned Migrating from IBM BigInsights to Hortonworks Data Platform
Lessons Learned Migrating from IBM BigInsights to Hortonworks Data PlatformDataWorks Summit
 
Hadoop Administration pdf
Hadoop Administration pdfHadoop Administration pdf
Hadoop Administration pdfEdureka!
 
Hadoop Adminstration with Latest Release (2.0)
Hadoop Adminstration with Latest Release (2.0)Hadoop Adminstration with Latest Release (2.0)
Hadoop Adminstration with Latest Release (2.0)Edureka!
 
M. Florence Dayana - Hadoop Foundation for Analytics.pptx
M. Florence Dayana - Hadoop Foundation for Analytics.pptxM. Florence Dayana - Hadoop Foundation for Analytics.pptx
M. Florence Dayana - Hadoop Foundation for Analytics.pptxDr.Florence Dayana
 
Research on vector spatial data storage scheme based
Research on vector spatial data storage scheme basedResearch on vector spatial data storage scheme based
Research on vector spatial data storage scheme basedAnant Kumar
 
Hadoop add-on API for Advanced Content Based Search & Retrieval
Hadoop add-on API for Advanced Content Based Search & RetrievalHadoop add-on API for Advanced Content Based Search & Retrieval
Hadoop add-on API for Advanced Content Based Search & Retrievaliosrjce
 
Modernize Your Existing EDW with IBM Big SQL & Hortonworks Data Platform
Modernize Your Existing EDW with IBM Big SQL & Hortonworks Data PlatformModernize Your Existing EDW with IBM Big SQL & Hortonworks Data Platform
Modernize Your Existing EDW with IBM Big SQL & Hortonworks Data PlatformHortonworks
 
Virtualized Big Data Platform at VMware Corp IT @ VMWorld 2015
Virtualized Big Data Platform at VMware Corp IT @ VMWorld 2015Virtualized Big Data Platform at VMware Corp IT @ VMWorld 2015
Virtualized Big Data Platform at VMware Corp IT @ VMWorld 2015Rajit Saha
 

Similaire à HDF-EOS Datablade: Efficiently Serving Earth Science Data (20)

Hw09 Rethinking The Data Warehouse With Hadoop And Hive
Hw09   Rethinking The Data Warehouse With Hadoop And HiveHw09   Rethinking The Data Warehouse With Hadoop And Hive
Hw09 Rethinking The Data Warehouse With Hadoop And Hive
 
Modernizing Your Data Warehouse using APS
Modernizing Your Data Warehouse using APSModernizing Your Data Warehouse using APS
Modernizing Your Data Warehouse using APS
 
What is hadoop
What is hadoopWhat is hadoop
What is hadoop
 
Hadoop.pptx
Hadoop.pptxHadoop.pptx
Hadoop.pptx
 
HDF Update
HDF UpdateHDF Update
HDF Update
 
Testing Big Data: Automated ETL Testing of Hadoop
Testing Big Data: Automated ETL Testing of HadoopTesting Big Data: Automated ETL Testing of Hadoop
Testing Big Data: Automated ETL Testing of Hadoop
 
hadoop resume
hadoop resumehadoop resume
hadoop resume
 
Lessons Learned Migrating from IBM BigInsights to Hortonworks Data Platform
Lessons Learned Migrating from IBM BigInsights to Hortonworks Data PlatformLessons Learned Migrating from IBM BigInsights to Hortonworks Data Platform
Lessons Learned Migrating from IBM BigInsights to Hortonworks Data Platform
 
Hadoop Administration pdf
Hadoop Administration pdfHadoop Administration pdf
Hadoop Administration pdf
 
Hadoop Adminstration with Latest Release (2.0)
Hadoop Adminstration with Latest Release (2.0)Hadoop Adminstration with Latest Release (2.0)
Hadoop Adminstration with Latest Release (2.0)
 
hadoop exp
hadoop exphadoop exp
hadoop exp
 
M. Florence Dayana - Hadoop Foundation for Analytics.pptx
M. Florence Dayana - Hadoop Foundation for Analytics.pptxM. Florence Dayana - Hadoop Foundation for Analytics.pptx
M. Florence Dayana - Hadoop Foundation for Analytics.pptx
 
Research on vector spatial data storage scheme based
Research on vector spatial data storage scheme basedResearch on vector spatial data storage scheme based
Research on vector spatial data storage scheme based
 
Big data applications
Big data applicationsBig data applications
Big data applications
 
Hadoop jon
Hadoop jonHadoop jon
Hadoop jon
 
M017619194
M017619194M017619194
M017619194
 
Hadoop add-on API for Advanced Content Based Search & Retrieval
Hadoop add-on API for Advanced Content Based Search & RetrievalHadoop add-on API for Advanced Content Based Search & Retrieval
Hadoop add-on API for Advanced Content Based Search & Retrieval
 
Robin_Hadoop
Robin_HadoopRobin_Hadoop
Robin_Hadoop
 
Modernize Your Existing EDW with IBM Big SQL & Hortonworks Data Platform
Modernize Your Existing EDW with IBM Big SQL & Hortonworks Data PlatformModernize Your Existing EDW with IBM Big SQL & Hortonworks Data Platform
Modernize Your Existing EDW with IBM Big SQL & Hortonworks Data Platform
 
Virtualized Big Data Platform at VMware Corp IT @ VMWorld 2015
Virtualized Big Data Platform at VMware Corp IT @ VMWorld 2015Virtualized Big Data Platform at VMware Corp IT @ VMWorld 2015
Virtualized Big Data Platform at VMware Corp IT @ VMWorld 2015
 

Plus de The HDF-EOS Tools and Information Center

STARE-PODS: A Versatile Data Store Leveraging the HDF Virtual Object Layer fo...
STARE-PODS: A Versatile Data Store Leveraging the HDF Virtual Object Layer fo...STARE-PODS: A Versatile Data Store Leveraging the HDF Virtual Object Layer fo...
STARE-PODS: A Versatile Data Store Leveraging the HDF Virtual Object Layer fo...The HDF-EOS Tools and Information Center
 

Plus de The HDF-EOS Tools and Information Center (20)

Cloud-Optimized HDF5 Files
Cloud-Optimized HDF5 FilesCloud-Optimized HDF5 Files
Cloud-Optimized HDF5 Files
 
Accessing HDF5 data in the cloud with HSDS
Accessing HDF5 data in the cloud with HSDSAccessing HDF5 data in the cloud with HSDS
Accessing HDF5 data in the cloud with HSDS
 
The State of HDF
The State of HDFThe State of HDF
The State of HDF
 
Highly Scalable Data Service (HSDS) Performance Features
Highly Scalable Data Service (HSDS) Performance FeaturesHighly Scalable Data Service (HSDS) Performance Features
Highly Scalable Data Service (HSDS) Performance Features
 
Creating Cloud-Optimized HDF5 Files
Creating Cloud-Optimized HDF5 FilesCreating Cloud-Optimized HDF5 Files
Creating Cloud-Optimized HDF5 Files
 
HDF5 OPeNDAP Handler Updates, and Performance Discussion
HDF5 OPeNDAP Handler Updates, and Performance DiscussionHDF5 OPeNDAP Handler Updates, and Performance Discussion
HDF5 OPeNDAP Handler Updates, and Performance Discussion
 
Hyrax: Serving Data from S3
Hyrax: Serving Data from S3Hyrax: Serving Data from S3
Hyrax: Serving Data from S3
 
Accessing Cloud Data and Services Using EDL, Pydap, MATLAB
Accessing Cloud Data and Services Using EDL, Pydap, MATLABAccessing Cloud Data and Services Using EDL, Pydap, MATLAB
Accessing Cloud Data and Services Using EDL, Pydap, MATLAB
 
HDF - Current status and Future Directions
HDF - Current status and Future DirectionsHDF - Current status and Future Directions
HDF - Current status and Future Directions
 
HDFEOS.org User Analsys, Updates, and Future
HDFEOS.org User Analsys, Updates, and FutureHDFEOS.org User Analsys, Updates, and Future
HDFEOS.org User Analsys, Updates, and Future
 
HDF - Current status and Future Directions
HDF - Current status and Future Directions HDF - Current status and Future Directions
HDF - Current status and Future Directions
 
H5Coro: The Cloud-Optimized Read-Only Library
H5Coro: The Cloud-Optimized Read-Only LibraryH5Coro: The Cloud-Optimized Read-Only Library
H5Coro: The Cloud-Optimized Read-Only Library
 
MATLAB Modernization on HDF5 1.10
MATLAB Modernization on HDF5 1.10MATLAB Modernization on HDF5 1.10
MATLAB Modernization on HDF5 1.10
 
HDF for the Cloud - Serverless HDF
HDF for the Cloud - Serverless HDFHDF for the Cloud - Serverless HDF
HDF for the Cloud - Serverless HDF
 
HDF5 <-> Zarr
HDF5 <-> ZarrHDF5 <-> Zarr
HDF5 <-> Zarr
 
HDF for the Cloud - New HDF Server Features
HDF for the Cloud - New HDF Server FeaturesHDF for the Cloud - New HDF Server Features
HDF for the Cloud - New HDF Server Features
 
Apache Drill and Unidata THREDDS Data Server for NASA HDF-EOS on S3
Apache Drill and Unidata THREDDS Data Server for NASA HDF-EOS on S3Apache Drill and Unidata THREDDS Data Server for NASA HDF-EOS on S3
Apache Drill and Unidata THREDDS Data Server for NASA HDF-EOS on S3
 
STARE-PODS: A Versatile Data Store Leveraging the HDF Virtual Object Layer fo...
STARE-PODS: A Versatile Data Store Leveraging the HDF Virtual Object Layer fo...STARE-PODS: A Versatile Data Store Leveraging the HDF Virtual Object Layer fo...
STARE-PODS: A Versatile Data Store Leveraging the HDF Virtual Object Layer fo...
 
HDF5 and Ecosystem: What Is New?
HDF5 and Ecosystem: What Is New?HDF5 and Ecosystem: What Is New?
HDF5 and Ecosystem: What Is New?
 
HDF5 Roadmap 2019-2020
HDF5 Roadmap 2019-2020HDF5 Roadmap 2019-2020
HDF5 Roadmap 2019-2020
 

Dernier

A Framework for Development in the AI Age
A Framework for Development in the AI AgeA Framework for Development in the AI Age
A Framework for Development in the AI AgeCprime
 
Glenn Lazarus- Why Your Observability Strategy Needs Security Observability
Glenn Lazarus- Why Your Observability Strategy Needs Security ObservabilityGlenn Lazarus- Why Your Observability Strategy Needs Security Observability
Glenn Lazarus- Why Your Observability Strategy Needs Security Observabilityitnewsafrica
 
Scale your database traffic with Read & Write split using MySQL Router
Scale your database traffic with Read & Write split using MySQL RouterScale your database traffic with Read & Write split using MySQL Router
Scale your database traffic with Read & Write split using MySQL RouterMydbops
 
Design pattern talk by Kaya Weers - 2024 (v2)
Design pattern talk by Kaya Weers - 2024 (v2)Design pattern talk by Kaya Weers - 2024 (v2)
Design pattern talk by Kaya Weers - 2024 (v2)Kaya Weers
 
Potential of AI (Generative AI) in Business: Learnings and Insights
Potential of AI (Generative AI) in Business: Learnings and InsightsPotential of AI (Generative AI) in Business: Learnings and Insights
Potential of AI (Generative AI) in Business: Learnings and InsightsRavi Sanghani
 
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptx
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptxPasskey Providers and Enabling Portability: FIDO Paris Seminar.pptx
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptxLoriGlavin3
 
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptxThe Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptxLoriGlavin3
 
MuleSoft Online Meetup Group - B2B Crash Course: Release SparkNotes
MuleSoft Online Meetup Group - B2B Crash Course: Release SparkNotesMuleSoft Online Meetup Group - B2B Crash Course: Release SparkNotes
MuleSoft Online Meetup Group - B2B Crash Course: Release SparkNotesManik S Magar
 
UiPath Community: Communication Mining from Zero to Hero
UiPath Community: Communication Mining from Zero to HeroUiPath Community: Communication Mining from Zero to Hero
UiPath Community: Communication Mining from Zero to HeroUiPathCommunity
 
Connecting the Dots for Information Discovery.pdf
Connecting the Dots for Information Discovery.pdfConnecting the Dots for Information Discovery.pdf
Connecting the Dots for Information Discovery.pdfNeo4j
 
Tampa BSides - The No BS SOC (slides from April 6, 2024 talk)
Tampa BSides - The No BS SOC (slides from April 6, 2024 talk)Tampa BSides - The No BS SOC (slides from April 6, 2024 talk)
Tampa BSides - The No BS SOC (slides from April 6, 2024 talk)Mark Simos
 
Emixa Mendix Meetup 11 April 2024 about Mendix Native development
Emixa Mendix Meetup 11 April 2024 about Mendix Native developmentEmixa Mendix Meetup 11 April 2024 about Mendix Native development
Emixa Mendix Meetup 11 April 2024 about Mendix Native developmentPim van der Noll
 
Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...
Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...
Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...Alkin Tezuysal
 
Microsoft 365 Copilot: How to boost your productivity with AI – Part one: Ado...
Microsoft 365 Copilot: How to boost your productivity with AI – Part one: Ado...Microsoft 365 Copilot: How to boost your productivity with AI – Part one: Ado...
Microsoft 365 Copilot: How to boost your productivity with AI – Part one: Ado...Nikki Chapple
 
Infrared simulation and processing on Nvidia platforms
Infrared simulation and processing on Nvidia platformsInfrared simulation and processing on Nvidia platforms
Infrared simulation and processing on Nvidia platformsYoss Cohen
 
Testing tools and AI - ideas what to try with some tool examples
Testing tools and AI - ideas what to try with some tool examplesTesting tools and AI - ideas what to try with some tool examples
Testing tools and AI - ideas what to try with some tool examplesKari Kakkonen
 
The State of Passkeys with FIDO Alliance.pptx
The State of Passkeys with FIDO Alliance.pptxThe State of Passkeys with FIDO Alliance.pptx
The State of Passkeys with FIDO Alliance.pptxLoriGlavin3
 
2024 April Patch Tuesday
2024 April Patch Tuesday2024 April Patch Tuesday
2024 April Patch TuesdayIvanti
 
The Future Roadmap for the Composable Data Stack - Wes McKinney - Data Counci...
The Future Roadmap for the Composable Data Stack - Wes McKinney - Data Counci...The Future Roadmap for the Composable Data Stack - Wes McKinney - Data Counci...
The Future Roadmap for the Composable Data Stack - Wes McKinney - Data Counci...Wes McKinney
 
Email Marketing Automation for Bonterra Impact Management (fka Social Solutio...
Email Marketing Automation for Bonterra Impact Management (fka Social Solutio...Email Marketing Automation for Bonterra Impact Management (fka Social Solutio...
Email Marketing Automation for Bonterra Impact Management (fka Social Solutio...Jeffrey Haguewood
 

Dernier (20)

A Framework for Development in the AI Age
A Framework for Development in the AI AgeA Framework for Development in the AI Age
A Framework for Development in the AI Age
 
Glenn Lazarus- Why Your Observability Strategy Needs Security Observability
Glenn Lazarus- Why Your Observability Strategy Needs Security ObservabilityGlenn Lazarus- Why Your Observability Strategy Needs Security Observability
Glenn Lazarus- Why Your Observability Strategy Needs Security Observability
 
Scale your database traffic with Read & Write split using MySQL Router
Scale your database traffic with Read & Write split using MySQL RouterScale your database traffic with Read & Write split using MySQL Router
Scale your database traffic with Read & Write split using MySQL Router
 
Design pattern talk by Kaya Weers - 2024 (v2)
Design pattern talk by Kaya Weers - 2024 (v2)Design pattern talk by Kaya Weers - 2024 (v2)
Design pattern talk by Kaya Weers - 2024 (v2)
 
Potential of AI (Generative AI) in Business: Learnings and Insights
Potential of AI (Generative AI) in Business: Learnings and InsightsPotential of AI (Generative AI) in Business: Learnings and Insights
Potential of AI (Generative AI) in Business: Learnings and Insights
 
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptx
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptxPasskey Providers and Enabling Portability: FIDO Paris Seminar.pptx
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptx
 
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptxThe Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
 
MuleSoft Online Meetup Group - B2B Crash Course: Release SparkNotes
MuleSoft Online Meetup Group - B2B Crash Course: Release SparkNotesMuleSoft Online Meetup Group - B2B Crash Course: Release SparkNotes
MuleSoft Online Meetup Group - B2B Crash Course: Release SparkNotes
 
UiPath Community: Communication Mining from Zero to Hero
UiPath Community: Communication Mining from Zero to HeroUiPath Community: Communication Mining from Zero to Hero
UiPath Community: Communication Mining from Zero to Hero
 
Connecting the Dots for Information Discovery.pdf
Connecting the Dots for Information Discovery.pdfConnecting the Dots for Information Discovery.pdf
Connecting the Dots for Information Discovery.pdf
 
Tampa BSides - The No BS SOC (slides from April 6, 2024 talk)
Tampa BSides - The No BS SOC (slides from April 6, 2024 talk)Tampa BSides - The No BS SOC (slides from April 6, 2024 talk)
Tampa BSides - The No BS SOC (slides from April 6, 2024 talk)
 
Emixa Mendix Meetup 11 April 2024 about Mendix Native development
Emixa Mendix Meetup 11 April 2024 about Mendix Native developmentEmixa Mendix Meetup 11 April 2024 about Mendix Native development
Emixa Mendix Meetup 11 April 2024 about Mendix Native development
 
Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...
Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...
Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...
 
Microsoft 365 Copilot: How to boost your productivity with AI – Part one: Ado...
Microsoft 365 Copilot: How to boost your productivity with AI – Part one: Ado...Microsoft 365 Copilot: How to boost your productivity with AI – Part one: Ado...
Microsoft 365 Copilot: How to boost your productivity with AI – Part one: Ado...
 
Infrared simulation and processing on Nvidia platforms
Infrared simulation and processing on Nvidia platformsInfrared simulation and processing on Nvidia platforms
Infrared simulation and processing on Nvidia platforms
 
Testing tools and AI - ideas what to try with some tool examples
Testing tools and AI - ideas what to try with some tool examplesTesting tools and AI - ideas what to try with some tool examples
Testing tools and AI - ideas what to try with some tool examples
 
The State of Passkeys with FIDO Alliance.pptx
The State of Passkeys with FIDO Alliance.pptxThe State of Passkeys with FIDO Alliance.pptx
The State of Passkeys with FIDO Alliance.pptx
 
2024 April Patch Tuesday
2024 April Patch Tuesday2024 April Patch Tuesday
2024 April Patch Tuesday
 
The Future Roadmap for the Composable Data Stack - Wes McKinney - Data Counci...
The Future Roadmap for the Composable Data Stack - Wes McKinney - Data Counci...The Future Roadmap for the Composable Data Stack - Wes McKinney - Data Counci...
The Future Roadmap for the Composable Data Stack - Wes McKinney - Data Counci...
 
Email Marketing Automation for Bonterra Impact Management (fka Social Solutio...
Email Marketing Automation for Bonterra Impact Management (fka Social Solutio...Email Marketing Automation for Bonterra Impact Management (fka Social Solutio...
Email Marketing Automation for Bonterra Impact Management (fka Social Solutio...
 

HDF-EOS Datablade: Efficiently Serving Earth Science Data

  • 1. HDF-EOS Datablade: Efficiently Serving Earth Science Data http://heineken.gsfc.nasa.gov/hdfeos/
  • 2. What is a Datablade? Image HDF-EOS Spatial DataBlade API Text Web Extending the server by adding DataBlades: new Types and new functions Extensible OR Engine Scaleable Data Manager INFORMIX-Universal Server http://heineken.gsfc.nasa.gov/hdfeos/
  • 3. What does the HDF-EOS Datablade provide? • Inventory management of large volume data and metadata • Flexible access to the data as a file or BinaryLarge-Object inside the database. • Cross-granule searching and subsetting • Re-use other datablades such as spatial search tools from Informix • Implements Grid subsetting operations in the Database http://heineken.gsfc.nasa.gov/hdfeos/
  • 4. Plans for the Second Year • • • • Implement Swath, Point, HDF datatypes General subsetting tool: stored queries Metadata search and query tools Implement MISR Quality-Assessment database http://heineken.gsfc.nasa.gov/hdfeos/
  • 5. HDF-EOS Datablade Software Architecture Informix DB SQL SQL InterInterface face and API HDF-EOS C library (Unchanged) Valuable Data Exposed in Searchable Tables File I/O Emulator http://heineken.gsfc.nasa.gov/hdfeos/ Granules in File System Granules in Informix
  • 6. Overlapping Granule Scenario Latitude 90 80 70 60 50 40 30 20 10 Subset: Lat: 65 to 75 Lon: 15 to 25 0 1 2 3 4 10 30 50 70 90 Longitude http://heineken.gsfc.nasa.gov/hdfeos/
  • 7. Overlapping Granule Scenario Latitude 90 80 70 60 50 40 30 20 10 Subset: Lat: 65 to 75 Lon: 15 to 25 0 1 2 3 4 10 30 50 70 90 Longitude http://heineken.gsfc.nasa.gov/hdfeos/

Notes de l'éditeur

  1. &amp;lt;number&amp;gt;
  2. &amp;lt;number&amp;gt;
  3. Features of the HDFEOS Datablade Software Architecture The HDF-EOS library, although non-reentrant, still runs unchanged in the database, with no code modifications. The File-I/O-Emulator (FIOE) catches HDF-EOS I/O calls, and is optimized to handle subsetting I/O patterns. Potential to optimize for other patterns as well. The FIOE allows us to manage the data inside the database, or allow visibility to the data outside. User is not forced into one world or the other. Advantage of putting it inside is that you get extra safety features such as transaction management and recovery. Legacy science software can continue to run on the data if left outside. Can expose valuable indexable data inside the database as necessary. (Vdata example) So how does this architecture perform?….
  4. This picture goes with demo #2.