Since 1972 and the launch of Landsat 1– the first Earth Observation civilian satellite - millions of images have been acquired all over the Earth by a constantly growing fleet of more and more sophisticated satellites. Generally, searching within this huge amount of Earth Observation (EO) images is limited by the description of the acquisition conditions stored in the related metadata files, i.e. Where (footprint), When (time of acquisition) and How (viewing angles, instrument, etc.). Thus the larger community of end users misses the What filter - i.e. a way to filter search in term of image content. RESTo [1] uses the iTag [2] footprint-based tagging system to enhance image metadata and hopefully provides a way to express semantic queries on images content in term of land use. We investigated the performance of RESTo against a 12 millions simulated Sentinel-2 granules database representative of the forthcoming French national mirror site of Sentinel products (PEPS).

1. SEMANTIC SEARCH WITHIN EARTH OBSERVATION PRODUCTS DATABASES
BASED ON AUTOMATIC TAGGING OF IMAGE CONTENT
Jérôme Gasperi
2014 Conference on Big Data from Space
Frascati - Italy - November 12th, 2014

2. Big Data ?
The data deluge
The search paradigm
iTag
An EO tagging library
resto
An EO product search engine
What’s next ?
Conclusion and perspectives

3. The data deluge
Brett Ryder - http://www.economist.com/node/15579717

4. Earth Observation products search paradigm is to use
the acquisition parameters stored in the metadata

5. When Where How

6. What ?
i.e. image content

7. Sven Sachsalber | http://www.palaisdetokyo.com/fr/events/sven-sachsalber

8. iTag
Automatic tagging of Earth Observation products

9. Orthorectified image Characterized image
This is urban
This is water
This is forest
What we got What we need

10. iTag provides semantic enhancement of Earth
Observation data

11. It uses metadata footprint to enrich metadata
from exogenous data
i.e. no image processing !

12. Out of the box tagging sources
Continents,
Countries,
Regions,
States,
Cities,
Land cover,
Rivers,
Population count

13. # Polygon around Moscow
$moscow = ‘POLYGON((37.1351 55.9655,38.1006 55.9640,38.0525
55.4969,37.0926 55.5171,37.1351 55.9655))’;
# Initialize iTag
$iTag = new iTag();
# Tag polygon for land cover
$result = $iTag->tag($moscow, array(
‘landcover’ => true
));

14. Tag footprint around Moscow
http://goo.gl/6AkU4y

15. github.com/jjrom/itag

16. resto
Toward an Earth Observation products search engine

17. Search, visualize and download
Earth Observation data

18. Architecture

19. Gazetteer Query Analyzer
Administration
REST Webservices
Abstract Database Access
Layer
PostgreSQL
Driver
iTag 2.0
resto 2.0
Search
Visualize
Download
Users
POST
DELETE
Admin
Data

20. Abstract Database Access Layer
PostgreSQL Driver
database
resto
schema
_collection1
schema
_collection2
…etc…
schema
resto
schema
usersmanagement
PostGIS
hstore
Table inheritance

21. Rresto
Search Ingest
GET POST

22. Ingest

23. Gazetteer Query Analyzer
Administration
REST Webservices
Abstract Database Access
Layer
PostgreSQL
Driver
iTag 2.0
resto 2.0
Search
Visualize
Download
Users
POST
DELETE
Data

24. During ingestion process, resources are automatically
tagged thanks to iTag library

25. Why to tag image first ?

26. Search images over Russia
Bounding box !!

27. Search

28. resto provides semantic search capabilities
It uses a Query Analyzer to translate natural language query into
a set of EO OpenSearch parameters

29. <with> "keyword"
<without> "keyword"
"quantity" <lesser> (than) "numeric" "unit"
"quantity" <greater> (than) "numeric" "unit"
"quantity" <equal> (to) "numeric" "unit"
<lesser> (than) "numeric" "unit" (of) "quantity"
<greater> (than) "numeric" "unit" (of) "quantity"
<equal> (to) "numeric" "unit" (of) "quantity"
"quantity" <between> "numeric" <and> "numeric" ("unit")
<between> "numeric" <and> "numeric" "unit" (of) "quantity"
<today>
<yesterday>
<before> "date"
<after> "date"
<between> "date" <and> "date"
"numeric" "(year|day|month)" <ago>
<last> "(year|day|month)"
<last> "numeric" "(year|day|month)"
"numeric" <last> "(year|day|month)"
"(year|day|month)" <last>
<since> "numeric" "(year|day|month)"
<since> "month" "year"
<since> "date"
<since> "numeric" <last> "(year|day|month)"
<since> <last> "numeric" "(year|day|month)"
<since> <last> "(year|day|month)"
<since> "(year|day|month)" <last>
Query string analysis algorithm
is based on simple recognition
of words and patterns

30. Example
« Images of urban area in Russia acquired in last year with less than 5 % of cloud cover »

31. Example
« Images of urban area in Russia acquired in last year with less than 5 % of cloud cover »
keyword location date acquisition parameter

32. 2. Each search result has an « human readable url » that can
be indexed by web crawler (i.e. google robots)
1. Search parameters are derived from
Natural Language query
3. Keywords on resources are links to search requests :
they can be indexed by web crawler…and so on

33. 2. Each search result has an « human readable url » that can
be indexed by web crawler (i.e. google robots)
http://goo.gl/BCZ3z4
1. Search parameters are derived from
Natural Language query
3. Keywords on resources are links to search requests :
they can be indexed by web crawler…and so on

34. As of version 2.0, resto supports faceted search

35. http://dinosaurs.wikia.com/wiki/Coelurosauria
Facets

36. Performances
iTag / resto

37. 1 000 000
SPOT DATABASE
New products retrieved every 3 hours from ADS catalog
0.2s
SEARCH
0.5s
Time period of 1 month within a 10x10 km2 box
INGEST
Per product for a ~5000 products ingestion
Order of magnitude compute on a Dual Core 2.6 GHz | 4 Go RAM | HDD 500 To

38. What’s next ?
Conclusion and perspectives

39. Need for « fresh » tagging reference databases
(e.g. GLC2000 replacement)

40. Enhance metadata with twitter trends hashtags
Add tags #mh370,#plane,#malaysianairline
to resources acquired between 2014, march 8th and 2014, april 14th
in the south of the Indian Ocean

41. « Linked data is the right way to do Semantic Web »
Tim Berners-Lee

43. Update iTag JSON model to follow JSON-LD format
{
"@context": "http://json-ld.org/contexts/person.jsonld",
"@id": "http://dbpedia.org/resource/John_Lennon",
"name": "John Lennon",
"born": "1940-10-09",
"spouse": "http://dbpedia.org/resource/Cynthia_Lennon"
}