Inspire-hands_on-data_transformation

INSPIRE "Hands on"
Data transformation & download services
European Commission
Joint Research Centre
Institute for Environment
and Sustainability
Digital Earth and Reference
Data Unit
www.jrc.ec.europa.eu
Serving society
Stimulating innovation
Supporting legislation
Chris Schubert et al.

2Training: INSPIRE the INSPIRE "Hands on" - Data transformation & download services
EC JRC
Objectives
• to raise awareness about the technical application / transformation of
data sets in order to ensure compliance with the INSPIRE Directive.
• you will get a feeling and basics what a transformation process for
data sets into data services means, how to apply datasets INSPIRE
compliant.
• Learning outcomes:
• how to understand the INSPIRE DS, to use xml schemas, to deal
with controlled vocabularies ...
• how to transform datasets, to create gml files, ...
• how to create INSPIRE-compliant web services with GeoServer.
• …

EC JRC
What’s going on today
• Intro to INSPIRE data specs (retrospection)
• Relationship between source data and INSPIRE
themes & features
• Exercise mapping data into INSPIRE requirements
• Transformation / Configure mappings & validation
• Continue transformation & exercises
• Publish Data GeoServer
• Discussion

INSPIRE "Hands on"
Introduction - Just as a quick reminder
European Commission
and Sustainability
Data Unit
Serving society
Chris Schubert

EC JRC
INSPIRE
what it is about …

EC JRC
INSPIRE
• Comprehensive data inventory (Monitoring & Reporting IR)
• Facilitate data discovery through standardised discovery
services & metadata (IR on Network Services & Metadata)
• Data sharing (IR on Data and Service Sharing; Article 17)
• Facilitate data access by allowing standardised view, download
and transformation (IR on Network Services)
• Facilitate data use and interoperability by adopting common
cross-domain models to exchange data (IR on Data
Interoperability)

EC JRC
INSPIRE components

EC JRC
INSPIRE Directive
General rules to establish an Infrastructure
for Spatial Information in Europe for
 Community environmental policies
 Policies or activities which impact on the
environment
INSPIRE is built on the SDIs established and
operated by the Member States
Spatial data held by/on behalf of public
authorities
Does not require collection of new data
INSPIRE is a Framework Directive
 Detailed technical provisions in Implementing
Rules
JRC is/was the technical coordinator

EC JRC
INSPIRE Thematic Scope
Annex I
Coordinate reference
systems
Geographical grid
systems
Geographical names
Administrative units
Addresses
Cadastral parcels
Transport networks
Hydrography
Protected sites
Annex II
Elevation
Land cover
Ortho-imagery
Geology
Annex III
Statistical units Area management/restriction/
regulation zones & reporting units
Buildings Natural risk zones
Soil Atmospheric conditions &
Meteorological
geographical features
Land use Oceanographic geographical
features
Human health and safety Sea regions
Utility and governmental
services
Bio-geographical regions
Environmental monitoring
facilities
Habitats and biotopes
Production and industrial
facilities
Species distribution
Agricultural and aquaculture
facilities
Energy resources
Population distribution –
demography
Mineral resources

EC JRC
Legally binding documents

EC JRC
INSPIRE Implementing Rules (IRs)
+
No 1253/2013
21 Oct 2013
No 102/2011
(code values Annex I)
No 1089/2010
(Annex I)

EC JRC
INSPIRE Implementing Rules (IRs) - ISDSS
defines the requirements of
•spatial object types,
•their attributes,
•association roles
•code lists,
•code list values,
•layers for the spatial data theme
for all EU Member States (MS) public
sector (PS) organisations

EC JRC
Implementing Rules vs. Technical
Guidelines

EC JRC
Technical Guidelines (TG-Data Specification)
Framework Documents
TG Annex I, Annex II & III
Metadata
& services
…
…
Interoperability of spatial data sets & services

EC JRC
Conceptual
data models
Registers
• objects types,
properties &
relationships
• cross-domain
harmonization
• based on a
common
modelling
framework
• managed in a
common UML
repository
Harmonised
vocabularies
• to overcome
interoperabilit
y issues
caused by
free-text
and/or multi-
lingual
content
• allow
additional
terms from
local
vocabularies
Encoding
• conceptual
models
independent
of concrete
encodings
• standard
encoding:
GML, but also
possible to
derive other
encodings
(e.g. based
on RDF)
• provide
unique and
persistent
identifiers
for reference
to resources
• allow their
consistent
management
and
versioning
Key pillars of data interoperability

EC JRC
Conceptual data modelsclass EnvironmentalMonitoringFacilities
GCM Base Types 2 Additional classes from GCM Observations
DataType
ISO FDIS 19156:2011 Observations and Measurements
«featureType»
AbstractMonitoringFeature
«voidable»
+ reportedTo :ReportToLegalAct [0..*]
constraints
{Observation and ObservingCapability}
«featureType»
EnvironmentalMonitoringProgramme
«featureType»
EnvironmentalMonitoringNetwork
«voidable»
+ organisationLevel :LegislationLevelValue
«dataType»
ReportToLegalAct
+ legalAct :LegislationCitation
«voidable»
+ reportDate :DateTime
+ reportedEnvelope :URI [0..1]
+ observationRequired :Boolean
+ observingCapabilityRequired :Boolean
+ description :CharacterString [0..1]
«featureType»
ObservingCapability
«voidable»
+ observingTime :TM_Object
+ processType :ProcessTypeValue
+ resultNature :ResultNatureValue
+ onlineResource :URL [0..1]
«featureType»
AbstractMonitoringObject
+ inspireId :Identifier
+ mediaMonitored :MediaValue [1..*]
+ geometry :GM_Object [0..1]
«voidable»
+ name :CharacterString [0..*]
+ additionalDescription :CharacterString [0..1]
+ legalBackground :LegislationCitation [0..*]
+ responsibleParty :RelatedParty [0..*]
+ onlineResource :URL [0..*]
+ purpose :PurposeOfCollectionValue [0..*]
«featureType»
EnvironmentalMonitoringFacility
«voidable»
+ representativePoint :GM_Point [0..1]
+ measurementRegime :MeasurementRegimeValue
+ mobile :Boolean
+ resultAcquisitionSource :ResultAcquisitionSourceValue [0..*]
+ specialisedEMFType :SpecialisedEMFTypeValue [0..1]
constraints
{GeometryRequired}
Observation and ObservingCapability
/* If Observation(s) are attached to an AbstractMonitoringFeature this must have
an ObservingCapability attached to it. The ObservingCapability must reference
the same Domain, Phenomenon and ProcessUsed as the Observation. */
inv: hasObservation->notEmpty() implies observingCapability->notEmpty() and
hasObservation.OM_Observation.featureOfInterest =
observingCapability.featureOfInterest and
hasObservation.OM_Observation.observedProperty =
observingCapability.observedProperty and
hasObservation.OM_Observation.procedure = observingCapability.procedure
NetworkFacility
«voidable»
+ linkingTime :TM_Object
AnyDomainLink
«voidable»
+ comment :CharacterString
GeometryRequired
/* Geometry and
representativePoint can't be
empty at the same time.*/
inv: geometry ->notEmpty() or
representativePoint ->notEmpty()
«featureType»
EnvironmentalMonitoringActivity
«voidable»
+ activityTime :TM_Object
+ activityConditions :CharacterString
+ boundingBox :GM_Boundary [0..1]
+ responsibleParty :RelatedParty
+ onlineResource :URL [0..*]
Hierarchy
«voidable»
+ linkingTime :TM_Object
EF-Level
Base Types 2::LegislationCitation
+ identificationNumber :CharacterString [0..1]
+ officialDocumentNumber :CharacterString [0..1]
+ dateEnteredIntoForce :TM_Position [0..1]
+ dateRepealed :TM_Position [0..1]
+ level :LegislationLevelValue
+ journalCitation :OfficialJournalInformation [0..1]
Base Types 2::DocumentCitation
+ name :CharacterString
«voidable»
+ shortName :CharacterString [0..1]
+ date :CI_Date
+ link :URL [1..*]
+ specificReference :CharacterString [0..*]
«FeatureType»
observation::OM_Observation
+ phenomenonTime :TM_Object
+ resultTime :TM_Instant
+ validTime :TM_Period [0..1]
+ resultQuality :DQ_Element [0..*]
+ parameter :NamedValue [0..*]
constraints
{observedProperty shall be a phenomenon associated with the
feature of interest}
{procedure shall be suitable for observedProperty}
{result type shall be suitable for observedProperty}
{a parameter.name shall not appear more than once}
«FeatureType»
observation::OM_Process
«FeatureType»
General Feature Instance::
GFI_Feature
observation::
ObservationContext
+ role :GenericName
«metaclass»
General Feature Model::
GF_PropertyType
{root}
+ memberName :LocalName
+ definition :CharacterString
«type»
Records and Class Metadata::Any
{root}
«featureType»
Processes::Process
«voidable»
+ name :CharacterString [0..1]
+ type :CharacterString
+ documentation :DocumentCitation [0..*]
+ processParameter :ProcessParameter [0..*]
+ responsibleParty :RelatedParty [1..*]
«Type»
Observable Properties::
AbstractObservableProperty
+ label :CharacterString [0..*]
«dataType»
Processes::ProcessParameter
+ name :ProcessParameterNameValue
+ description :CharacterString [0..1]
«featureType»
OperationalActivityPeriod
+ activityTime :TM_Object
+operationalActivityPeriod
«voidable»
1..*
realises
Phenomenon
+observedProperty1
+propertyValueProvider
0..* Domain
+featureOfInterest
1
Domain
+featureOfInterest
«voidable»0..1
+generatedObservation
0..*
ProcessUsed +procedure
1
ProcessUsed
+procedure1
+hasObservation
«voidable»0..*
+result
Range 0..* +relatedObservation 0..*
Phenomenon +observedProperty
1
+uses
«voidable»
0..*
+involvedIn
«voidable»
0..*
+relatedTo
«voidable»
0..*
+belongsTo
«voidable»
0..*
+contains
«voidable»
0..*
+supersedes
«voidable» 0..*
genealogy
+supersededBy
«voidable» 0..*
+observingCapability
«voidable» 0..*
+broader
«voidable»
0..1
hierarchy
+narrower
«voidable»
0..*
+triggers
«voidable»
0..*
+setUpFor
«voidable»
0..*
class Sampled Land Use
«featureType»
ExistingLandUseSample
+ location :GM_Point
+ hilucsLandUse :HILUCSValue [1..*]
«lifeCycleInfo, voidable»
+ beginLifespanVersion :DateTime
+ endLifespanVersion :DateTime [0..1]
«voidable»
+ hilucsPresence :HILUCSPresence
+ specificLandUse :LandUseClassificationValue [1..*]
+ observationDate :Date
+ specificPresence :SpecificPresence
+ validFrom :Date [0..1]
+ validTo :Date [0..1]
«featureType»
SampledExistingLandUseDataSet
+ extent :GM_MultiSurface
+ name :CharacterString
+ beginLifespanVersion :DateTime
+ endLifespanVersion :DateTime [0..1]
«voidable»
+ validFrom :Date [0..1]
+ validTo :Date [0..1]
+dataset
1
+member
0..*

EC JRC
Encoding
Encoding guidelines (D2.7)
updated during Annex II+III development
Minor changes in the encoding rule
• references to code list values
• association classes
GML application schema - XML schema
kind of templates that is used to express a set of
conformance rules for an GML file
• defines also relations to other (base) schemas by
imported namespace
These are our Target Schemas

EC JRC
Code lists
4 types of INSPIRE code lists
according to extensibility
a) not extensible – only values included in IRs are allowed
b) narrower extensible – values included in IRs and narrower
values are allowed
c) freely extensible – values included in IRs and any other values
are allowed
d) empty – any values are allowed
For code lists of types (b), (c) and (d), additional values have to be
published in a register
TG-DS may include additional proposed values that will be published in
the INSPIRE code list register

EC JRC
Registers
EC provides central registries
for INSPIRE resources
published on
http://inspire.ec.europa.eu/registry
• registers: code lists, themes, application schemas, fcd
• browsing and accessing register content
• Formats: HTML, XML, Atom, JSON and RDF/SKOS
• Multilingual content (based on IR content)
Open to external contributions

EC JRC
Conceptual
data models
Registers
• objects
types,
properties &
relationships
• cross-
domain
harmonizatio
n
• based on a
common
modelling
framework
• managed in
a common
UML
Harmonised
vocabularies
• to overcome
interoperabili
ty issues
caused by
free-text
and/or multi-
lingual
content
• allow
additional
terms from
local
vocabularies
Encoding
• conceptual
models
independent
of concrete
encodings
• standard
encoding:
GML, but
also
possible to
derive other
encodings
(e.g. based
on RDF)
• provide
unique and
persistent
identifiers
for
reference to
resources
• allow their
consistent
managemen
t and
versioning
Key pillars of data interoperability
described in INSPIRE Conceptual Framework documents
D2.6:Methodology
for Specification
Development
D2.10.3: Common
data models
D2.9: O&M
Guidelines
D2.5: Generic
Conceptual Model
D2.7: Guidelines
for Encoding

EC JRC
UML models, xml schemas, registers,
that what JRC delivers & maintains
http://inspire.ec.europa.eu/index.cfm/pageid/2/list/datamodels

INSPIRE "Hands on"
How to read Technical Guidelines –
basic notations
European Commission
and Sustainability
Data Unit
Serving society
Chris Schubert

EC JRC
Technical Guidelines (TG-Data Specification)
1 Scope
2 Overview and Description
3 Specification scopes
4 Identification information
5 Data content and structure
Application schemas, Feature catalogue,
Notations, Voidable characteristics,
Enumerations, Code lists
Identifier management
Geometry representation,
Temporality representation
6 Reference systems, units of measure and grids
Theme-specific requirements & recommendations
7 Data quality
8 Dataset-level metadata
9 Delivery incl. Encoding
10 Data Capture
11 Portrayal
12 Bibliography
Annex A (normative) Abstract Test Suite
Annex B (informative) Use cases
Annex C (normative) Code list values
Annex D (informative) Examples

EC JRC
Basic notations in TG-DS
D2.5-GenericConceptualModel

EC JRC
INSPIRE base types and basic notions
Stereotypes
Fixed set of values
Closed or extendable set of
values

EC JRC
INSPIRE identifier
• Used to reference the object
A namespace to identify the data
source. The namespace is owned by
the data provider
A local identifier, assigned by the data
provider. The local identifier is
unique within the namespace, i.e.
no other spatial object carries the
same unique identifier.
namespace will consist of two parts: The first part will identify the data
provider within the member state and the second part will be used to
distinguish between different data sources maintained and provided by the
data provider. <<―NL.TOP10NL>> may be the namespace for spatial objects
in the TOP10 NL product in the Netherlands.

EC JRC
Object life-cycle
4 cross-thematic attributes:
• beginLifespanVersion: DateTime
Date and time at which this version of the spatial object was inserted
or changed in the spatial data set.
• endLifespanVersion: DateTime [0..1]
Date and time at which this version of the spatial object was
superseded or retired in the spatial data set.
• validFrom: DateTime [0..1]
The time when the phenomenon started to exist in the real world.
• validTo: DateTime [0..1]
The time from which the phenomenon no longer exists in the real world

EC JRC
<<Voidable>> elements
• Related to INSPIRE obligation
• Data providers are authorized not to provide voidable elements:
• EITHER if no corresponding data is captured,
• OR if no corresponding data can be derived from other existing data at
reasonable costs.
When a voidable element is not provided, the
reason should be given:
Unpopulated: Not part of the dataset
maintained by the data provider.
Unknown: Not known to, and not
computable by the data provider
Withheld: Confidential and not divulged
by the data provider.

EC JRC
Delivery
the use of GML as the default encoding is proposed
Rules for default encodings for application schemas are defined in
GML application schema, also called XML schema
Eg. http://inspire.ec.europa.eu/schemas/gelu/3.0/GriddedExistingLandUse.xsd
No (legal) obligation to use GML in INSPIRE.
For INSPIRE download services - Media types used for spatial data sets are
listed: http://inspire.ec.europa.eu/media-types/
• x-shapefile
• x-filegdb
• image/tiff
• google-earth.kml+xml
• …

EC JRC
Land Use – INSPIRE data theme
• http://inspire.ec.europa.eu/index.cfm

EC JRC
Existing & Planned Land Use
Land Use - description of land in terms or its socio-economic and
ecological purpose - territory characterized according to its current and
future functional dimension or socio-economic purpose , and is split in
two different types:
The Existing Land Use
Geographical data-sets that provide Land Use information, at the time
of observation
The Planned Land Use
Which corresponds to spatial plans, defined by spatial planning
authorities, depicting the possible utilization of the land in the future.

EC JRC
Application schemas included in the IRs
The types to be used for the exchange and classification of spatial
objects from data sets related to the spatial data theme Land Use are
defined in the following application schemas:
The Existing Land Use
Existing Land Use application schema (organized as a partition of a
given area)
Gridded Land Use application schema (organized as a set of pixels)
Sampled Land Use application schema (organized as a set of
discrete observation points )
Planned Land Use application schema

EC JRC
Sampled Land Use – UML model
Application schema corresponds to a dataset that depicts the reality of the land surface at
discrete location on the earth. Often these datasets are collected for statistical purposes to
provide estimates of land use over wider areas.

EC JRC
Feature catalog VS UML model
Name: inspireId
Definition: External object identifier of the land use sample.
Description:
An external object identifier is a unique object identifier
published by the responsible body, which may be used
by external applications to reference the spatial object.
The identifier is an identifier of the spatial object, not an
identifier of the real-world phenomenon.
Voidable: false
Multiplicity: 1
Value type: Identifier (data type)
Name: location
Definition: Location where the land use sample is taken.
Voidable: false
Multiplicity: 1
Value type: GM_Point
Name: beginLifespanVersion
Definition:
Date and time at which this version of the spatial
object was inserted or changed in the spatial data
set.
Voidable: true
Multiplicity: 1
Value type: DateTime
Name: endLifespanVersion
Definition:
Date and time at which this version of the spatial
object was superseded or retired in the spatial data
set.
Voidable: true
Multiplicity: 0..1
Value type: DateTime
Name: hilucsLandUse
Definition: Land use HILUCS classes that are present in this existing land use sample.
Description:
NOTE The Sampled Existing Land Use model enables the provision of
information on land uses inside one land use object. The
ExistingLandUseObject may be associated with 1 to many HILUCSLandUse that
represents the Land Uses for the polygon from the economical point of view. It
makes possible the assignment of more than one HILUCSLandUse existences
when they cannot be managed by HILUCSPresences.
Voidable: false
Multiplicity: 1..*
Value type: HILUCSValue (code list)
Name: hilucsPresence
Title: land use presence
Definition:
Actual presence of a land use category according to
HILUCS within the object.
Voidable: true
Multiplicity: 1
Value type: HILUCSPresence (union data type)
Name: observationDate
Title: Observation Date.
Definition: The observation date associated to a description.
Description:
Defines the observation date of the description. It could be the
date of an aerial/satellital acquisition or a field survey. The
observation date allows the user to have accurate date of when
the description was made in the real word. In a database, not
all object informations are necessarily captured at the same
time.
Voidable: true
Multiplicity: 1
Value type: Date
Name: specificLandUse
Definition:
Land Use Category according to the nomenclature
specific to this data set.
Description:
Reference to an entry in the classfication that is part
of the SpecificLandUseClassification provided by the
data producer.
Voidable: true
Multiplicity: 1..*
Value type: LandUseClassificationValue (code list)
Name: specificPresence
Definition:
Actual presence of a land use category within the
object.
Voidable: true
Multiplicity: 1
Value type: SpecificPresence (union data type)
Name: validFrom
Definition:
The time when the phenomenon started to exist in
the real world.
Voidable: true
Multiplicity: 0..1
Value type: Date
Name: validTo
Definition:
The time from which the phenomenon no longer
exists in the real world.
Voidable: true
Multiplicity: 0..1
Value type: Date
«featureType»
+ hilucsLandUse: HILUCSValue [1..*]
+ inspireId: Identifier
+ location: GM_Point
+ beginLifespanVersion: DateTime
+ endLifespanVersion: DateTime [0..1]
«voidable»
+ hilucsPresence: HILUCSPresence
+ observationDate: Date
+ specificLandUse: LandUseClassificationValue [1..*]
+ specificPresence: SpecificPresence
+ validFrom: Date [0..1]
+ validTo: Date [0..1]
«featureType»
+ extent: GM_MultiSurface
+ name: CharacterString
«voidable»
+dataset
1
+member
0..*
Association role:
Name: dataset
Definition: Data set to which this sample belongs.
Voidable: false
Multiplicity: 1
Value type: SampledExistingLandUseDataSet (spatial object
type)

EC JRC
Feature catalog VS UML model
Name: extent
Definition: The convex hull of all the instances of the spatial object type ExistingLandUseSample.
Voidable: false
Multiplicity: 1
Value type: GM_MultiSurface
Name: name
Definition: Human readable name of the data set.
Voidable: false
Multiplicity: 1
Value type: CharacterString
Name: member
Definition: Reference to the members of the sampled existing land use data set.
Voidable: false
Multiplicity: 0..*
Value type: ExistingLandUseSample (spatial object type)
«featureType»
+ hilucsLandUse: HILUCSValue [1..*]
+ location: GM_Point
«voidable»
+ hilucsPresence: HILUCSPresence
+ observationDate: Date
+ specificLandUse: LandUseClassificationValue [1..*]
+ specificPresence: SpecificPresence
«featureType»
+ extent: GM_MultiSurface
+ name: CharacterString
«voidable»
+dataset
1
+member
0..*

EC JRC
Hierarchical INSPIRE Land Use Classification
System (HILUCS)
The Land Use data specification supports two systems of
classification:
the obligatory –HILUCS multi-level, classification system - all
application schemas (existing and planned)
the (optional) specific classification system in use in a member
state (LUCAS, NACE, SEEA…)

European Commission
Institute for Environment and
Sustainability
Digital Earth and Reference Data
Unit
Serving society
LUCAS
Land use/cover statistics
Understanding the Source Schema and Data Set

Index
1. What is LUCAS?
2. Difference between Land Cover and Land Use
3. Data provided by LUCAS
4. What does LUCAS look like?
4.1. LUCAS attributes
4.2. LUCAS codelists
5. Our Sample Data Set
6. Links
40

What is LUCAS?
 Is the EU´s harmonized land use and land cover survey
 + module providing Soil information (TOPSOIL)
 Implemented by Eurostat with the member states collaboration
 Survey from 2006. Updated every 3 years.
 Surveys are made in-situ
after a design phase for selecting the sample locations
1 point every 2 square kilometers
41
2006
11 countries
2009
23 countries
2012
27 countries

42
Difference between Land Cover and Land Use

Data provided by LUCAS
43
LUCAS
Statistical
Tables
Primary
Data
Photos

44
http://ec.europa.eu/eurostat/statistical-atlas/gis/viewer/?myConfig=LUCAS-2012.xml
What does LUCAS look like?

45
LUCAS attributes
Point_ID X_LAEA Y_LAEA STRATA NUTS0 NUTS1 NUTS2
area2 WH PESO_F2 SURV_DATE OBSERVED OBS_TYPE GPS_PROJ
GPS_PREC GPS_Y_LAT GPS_EW
GPS_X_LON
G
Y_LAT EW X_LONG
OBS_DIST OBS_DIRECT LC1 LC2 OBS_RADIUS LU1 LU2
LC1_SPECIES
LC1_PERCEN
T
LC2_SPECIES
LC2_PERCEN
T
AREA_SIZE
TREES_HEIG
HT
FEATURES_W
IDTH
LAND_MNGT
WM_WATER
_MNGT
WM_SRC_IR
RIGATION
WM_TYP_IR
RIGATION
WM_DELIVE
RY_SYST
SOIL_SURVE
Y
SOIL_PLOUG
H
SOIL_CROP SOIL_STONES SOI_LABEL
PHOTO_POI
NT
PHOTO_N PHOTO_E PHOTO_S
PHOTO_W TR1 …
More than 50
Identifier
Location
Time
Land Use
Land
Cover
Soil
Water
Others
Point_ID X_LAEA Y_LAEA STRATA NUTS0 NUTS1 NUTS2
area2 WH PESO_F2 SURV_DATE OBSERVED OBS_TYPE GPS_PROJ
GPS_PREC GPS_Y_LAT GPS_EW
GPS_X_LON
G
Y_LAT EW X_LONG
OBS_DIST OBS_DIRECT LC1 LC2 OBS_RADIUS LU1 LU2
LC1_SPECIES
LC1_PERCEN
T
LC2_SPECIES
LC2_PERCEN
T
AREA_SIZE
TREES_HEIG
HT
FEATURES_W
IDTH
LAND_MNGT
WM_WATER
_MNGT
WM_SRC_IR
RIGATION
WM_TYP_IR
RIGATION
WM_DELIVE
RY_SYST
SOIL_SURVE
Y
SOIL_PLOUG
H
SOIL_CROP SOIL_STONES SOI_LABEL
PHOTO_POI
NT
PHOTO_N PHOTO_E PHOTO_S
PHOTO_W TR1 …

LUCAS code lists
Classifications are hierarchical
 different levels of information
 structured broad-level classes
 allow subdivision into more
detailed sub-classes.
 At each level the defined classes are
mutually exclusive.
46
Classification
U100 –
Agriculture,
Forestry and
Fishing
U200 –
Manufacturing
and energy
…
U110 -
Agriculture
U120 –
Forestry
U130 –
Fishing
…
U111 –
Agriculure
U112 –
Fallow and
abandoned land
U113 –
Kitchen gardens

Our sample Data Set: SampleLUCAS.shp
47
Download
France and
Spain 2009
data
Merge in a
single CSV
Create
shapefile from
coordinates
fields
Create a
crossborder
selection
Export to
shapefile

LUCAS viewer http://ec.europa.eu/eurostat/statistical-atlas/gis/viewer/?myConfig=LUCAS-2012.xml
About LUCAS http://epp.eurostat.ec.europa.eu/portal/page/portal/lucas/introduction
http://epp.eurostat.ec.europa.eu/portal/page/portal/lucas/data
The EU’s land use and land
cover survey
http://epp.eurostat.ec.europa.eu/cache/ITY_OFFPUB/KS-03-13-587/EN/KS-03-13-
587-EN.PDF
Example of indicator in
Eurostat Database
http://appsso.eurostat.ec.europa.eu/nui/show.do?dataset=lan_lcv_ovw&lang=en
LUCAS TOPSOIL data http://eusoils.jrc.ec.europa.eu/projects/Lucas/Data.html
LUCAS microdata used Spain:
http://epp.eurostat.ec.europa.eu/portal/page/portal/lucas/documents/ES_2009.csv
France:
http://epp.eurostat.ec.europa.eu/portal/page/portal/lucas/documents/FR_2009.csv
LUCAS Metadata http://epp.eurostat.ec.europa.eu/cache/ITY_SDDS/en/lan_esms.htm#relatedmd1392
278553624
48
Links

Transformation, ETL never heard …
ETL is a process to integrate different information from multiple
applications. (data base management)
ETL is short for:
Extract is the process of reading & understand information
Transform is the process of converting the extracted information by using
rules, or by combining the data with other information.
Load is the process of writing the information according a target
schema
That’s why it is important (we showing again)
understand the specific scope TG-DS, getting close to the domain
expertise
understanding the data models helps
have an idea about the content of frame work documents
basic knowledge about gml (technicians will help you)

Transformation
simple transformation - renaming, assign new properties
complex transformation - reclassification, geometry calulation
origin conformant
transformation

Transformation
... is an ETL repeatable procces

Transformation needs domain expertise
load target SCHEMA
define INSPIRE mapping
create transformation rules
analytical
task
run transformation

architectural approaches, an Overview
One-off transformation + external web based services | Atom/WFS/…
One-the-fly transformation | Atom/WFS/…
One-off/the-fly transformation + integrated web based services |
Atom/WFS/…
but what
happened with
changes (update)?
need to maintain,
don’t forget
Consider for choosing an
approach (operational process)
will the data set in future
• +/- static,
e.g. geology
• under frequently change,
e.g. land use
• under permanent change
e.g. air quality reporting

One-off transformation + external web based services
positive aspects are:
transformed once for all users
• better performance when delivering the
data
• no transformation during delivery
• Free choice of software components
negative aspects are:
Requires storage and management of
transformed data
in addition to original data
• high processing effort
• the entire database is transformed
• to be maintained also transformed data
This approach is usefull if data quite stabil

On the fly transformation+ integrated web services
only the original data has to be
maintained
only the requested data has to be
transformed
• limited on the geometric & semantic
request
Performance issues
• high processing required before delivery
especially for large volumes of data and
complex transformations
• the same data is potentially transformed
multiple times
• Unless caching or pre-processing
mechanisms has to be used
This approach is useful if data is continuously or frequently updated
eg. Snowflake/GoLoader/-PublisherWFS, GeoServer (?)

One-off transformation + integrated web based services
Data transformed offline can be
managed in same system as original
data e.g. in the same database management
system, not as GML files
• ‘On-the-fly’ get more performance
because of predefined data structure
Data provider still has to store
transformed data
no really FOSS-GIS solutions (current)
This approach is useful if data is continuously or frequently updated
eg. ArcGIS for INSPIRE

European Commission
Sustainability
Unit
Serving society
HALE
Setting up and Understanding the workbench

Index
1) What’ s HALE?
2) Starting HALE
3) Configuring the workbench
4) Schema Explorer tab in detail
5) HALE GUI
6) Functions for transformation
7) Links
80

What’ s HALE?
HALE: HUMBOLDT Alignment Editor
- Part of HUMBOLDT Harmonisation Toolkit
- Maintained by the Data Harmonisation Panel.
Tool useful to:
- Create mappings (alignments)
- Validate
- Transform data
- Styling (SLD generation)
Current version 2.8.0 (from 2013-11-13)
- Working on the next version 2.9.0
- We can see the features added or fixed in the issue tracker
website
81

Starting HALE
1. Download
2. Unzip
3. Double click on HALE.exe
82

Configuring the workbench
83
1. Import a source Schema
FILEIMPORTSOURCE SCHEMA SampleLUCAS.shp
2. Import a target Schema
FILEIMPORTTARGET SCHEMA SampledExistingLandUse.xsd
3. Import Source Data
FILEIMPORTSOURCE DATA SampleLUCAS.shp
4. Remove the FeatureType don´t needed
EDITEDIT TARGET MAPPEABLE
TYPES
REMOVE THE
SampleLandUseDataset
!
Remember to save the
project
Source
Schema &
Data
Target
Schema

Schema Explorer tab in detail:
Elements
84
Feature Type
Attributes
Data type
Cardinality
Number of features

Asterisk Mandatory field
Arrow XML attribute
Question mark Choice
Schema Explorer in detail:
Simbology
85
Icons: Data Types
Icons: Attribute information
Text
Number
Geometry
Complex

Schema Explorer in detail:
Simbology
86
No color Not mapped
Green Mapped explicitly with a relation
Yellow
Mapped implicitly due to the mapping of a sub-
property
Purple
Value assignment independent of the source
schema
Background colors: State of the mapping

HALE GUI
Usable
Easy to use and understand
Flexible
GUI completely customizable
Complete
Provides different tabs and
predefined views of the same
project
87
This is very helpful and useful to
look easily if our transformation
is correctly done

88
Schema
Explorer
Alignment
Error log Report list
Default View

89
Data View
Source Data
Transformed Data
Alignment
Properties

90
Map View
Source Data map Transformed Data map
Source Data attributes
Transformed Data
attributes

Functions for transformation
General
Retype
Merge
Join
Create
Date extraction
Regex Analysis
Rename
Assign
Generate Unique Id
Classification
Formatted string
Geometric
Ordinates to Point
Network Expansion
Calculate Length
Calculate Area
Centroid
Compute Extent
Groovy
Groove Retype
Groovy Create
Groovy Merge
Groovy script
Inspire
Inspire Identifier
Geographical Name
Numeric
Mathematical Expression
Generate sequential I
91
You can arrive to the same
output by using different tools

Some functions
Functions Description
Type
relation
Retype For each instance of the source type, an instance of
the target type is created.
Property
relation
Rename Copy a source property to a target property
Assign Assigns a value to a target property
GenerateUniqueID Assigns a generated unique id to a target property
Classification Map classifications
Formatted String Creates a formatted string based on a pattern and
the input variables
92

Links
HALE website http://www.dhpanel.eu/humboldt-framework/hale.html
HALE Wiki http://www.esdi-community.eu/projects/hale/wiki
HALE Tutorial http://www.dhpanel.eu/humboldt-framework/hale-
tutorial.html
HALE download http://www.esdi-community.eu/projects/hale/files
HALE User Guide http://hale.igd.fraunhofer.de/2.8.0/help/index.jsp
93

European Commission
Sustainability
Unit
Serving society
INSPIRE network services

INSPIRE principles
Data should be collected once and maintained at the level
where this can be done most effectively
Combine seamlessly spatial data from different sources and
share it between many users and applications (the concept
of interoperability)
Spatial data should be collected at one level of government
and shared between all levels
Spatial data needed for good governance should be available
on conditions that are not restricting its extensive
use
It should be easy to discover which spatial data is
available, to evaluate its fitness for purpose and to know
which conditions apply for its use

Relevant components
INSPIRE Network services
In INSPIRE data should be made available where best
managed
1. Discover
 Expose metadata through INSPIRE compliant discovery service
2. View
 Interactive view of data through an INSPIRE compliant data service
 Unified portrayal through OGC Styled Layer Descriptor (SLD)
3. Download
 Web Feature Service
 Atom feeds
4. Transform
 ETL

Discovery services
„discovery services making it possible to search for spatial data sets and
services on the basis of the content of the corresponding metadata and
to display the content of the metadata“
http://inspire-geoportal.ec.europa.eu/discovery/

INSPIRE Metadata
Distinguish between
• Spatial object metadata
• Dataset-level metadata
Tools available at JRC site
• INSPIRE Metadata editor
• INSPIRE metadata validator

View services
„view services making it possible, as a minimum, to display, navigate,
zoom in/out, pan, or overlay viewable spatial data sets and to display
legend information and any relevant content of metadata“

View and discovery services at INSPIRE
Geoportal

Download services
„download services, enabling copies of spatial data sets, or parts of such
sets, to be downloaded and, where practicable, accessed directly“

INSPIRE Download services
Available options (now)
• Atom feeds
• WFS (Web Feature Service)
Work ongoing for
• INSPIRE compliant download service based on OGC Sensor
Observation Service (SOS)

Download services
Predefined
Direct access
Service
implementation
Predefined dataset
download service
Direct access download
service
SOS X X
WFS Х Х
Atom Х

Relevant components
INSPIRE Geoportal
• Central access point to the INSPIRE
infrastructure and resources (250
000+)
 “The face” of INSPIRE
• Connection to all MS
network services
 cross-border data
discovery and
visualisation
 support to
 policy making

Introduction to OSGEO Live
What we use:
• Apache
• GDAL (ogr2ogr)
• PostgreSQL/PostGIS
• Geoserver
• Geonetwork - open source
• Text editor 

European Commission
Sustainability
Unit
Serving society
INSPIRE "Hands on"
Summary Data transformation
Chris Schubert et al.

INSPIRE “status”
Leaving the conceptual level
Final TG-DS II+III (3.0)
published end of 2013
Update TG-DS I published
2014
Final XML schema published
2014
Starting with the Implementation
There are still open issues, eg.
cross-cutting coherence,
validation - ATS abstract test
suite
(compliance vs. conformance),
etc…
INSPIRE Maintenance and Implementation Framework
• Support implementation
• Corrective maintenance
• Evolutive maintenance
… for exchange of experience and good practice
Designed by NINJAinfographics

Discussion and Conclusion
• Learning outcomes:
• how to understand the INSPIRE DS,
• to use xml schemas,
• to deal with controlled vocabularies ...
• how to transform datasets,
• to create gml files, ...
• how to create INSPIRE-compliant web services with GeoServer.
Did we met expectations

Inspire-hands_on-data_transformation

Recommandé

Recommandé

Contenu connexe

Tendances

Tendances (19)

En vedette

En vedette (20)

Similaire à Inspire-hands_on-data_transformation

Similaire à Inspire-hands_on-data_transformation (20)

Dernier

Dernier (20)

Inspire-hands_on-data_transformation