SlideShare une entreprise Scribd logo
1  sur  61
UKGovLD Registry
A register. Image by http://www.flickr.com/photos/tekniskamuseet/
UKGovLD Registry
Outline
• Background and motivation
• Details
– information model
– core functionality
– advanced topics
– status
• Applying it
• Discussion and next steps
UKGovLD Registry
Motivation
Support coordinated publication of reference
information as Linked Data
We want people to use URIs to identify things
– codes and code lists
– ontologies, thesauri and concepts
– units of measure
– organizations
– spatial objects
– licenses
– coordinate reference systems
– ...
UKGovLD Registry
but it’s not trivial ...
Each organization use its own namespace?
– need infrastructure and expertise to serve the data
– fragmented namespace
– where do consumers get the complete authoritative list?
Shared namespace?
– how to allocate new identifiers in that shared space?
– how to collaboratively maintain identifiers and lists?
– how to scale?
Either way:
– how to manage the life-cycle of introducing and
deprecating terms?
UKGovLD Registry
Linked Data Registry
Infrastructure to enable organizations to create
and manage identifiers and authoritative lists
– allocation and registration of approved codes,
vocabularies and reference URIs
– management of shared namespaces, including
clash-free allocation and delegation
– registration of published datasets supporting
discovery and cross-reference
UKGovLD Registry
Three functions
• identifier & list management
– create, update, reserve, lifecycle, versioning ...
• repository
– managed identifiers (information served from registry) as well
as referenced identifiers
• namespace management
– delegate a list or part of namespace to others
– federate to other repositories
=> very general platform
– can register anything you can give a URI and RDF description
– use only the aspects you need for a given situation
– “registry does X” does not mean it answers every X-shaped need
you might have
UKGovLD Registry
Conceptual architecture
router
renderer
request
processor
user
credentials
roles and
bindings
auth
registry
core
logic
Registry
RDF store
text index
style and
templates
external
UI
admin
UI
log
audit
trail
storeAPI
nginx
proxy
conf
API
UKGovLD Registry
Running examples
• Code list of colours
– simple, neutral
• UK Local authorities
– centrally created identifiers v. local creation?
– local control and maintenance?
– offloading management
UKGovLD Registry
Outline
• Background and motivation
• Details
– information model
– core functionality
– advanced topics
– status
• Applying it
• Discussion and next steps
UKGovLD Registry
Information model
• hierarchy of Registers
• things in registers are Register Items – metadata
• thing described by a RegisterItem called an entity
/
/reg1 /def /system
/def/colours /def/weather-types
/def/colours/_red /def/colours/_green
(describes /def/colours/red) (describes /def/colours/green)
Naming convention:
metadata item has “_”
UKGovLD Registry
Information model
complicated by:
– item v. entity
– versioning
reg:Register entity
reg:register
reg:definition
reg:entity
reg:RegisterItem reg:EntityReferencereg:RegisterItem reg:EntityReference
UKGovLD Registry
Information model
• default linked data view of Register is simplified
• configurable
– alternative membership property or inverse property
– so can make a register look like a skos:Collection,
skos:ConceptScheme, owl:Ontology ...
– also acts as a LDP container
• but can request full view (?_view=withMetadata)
reg:Register entity
reg:register
reg:definition
reg:entity
reg:RegisterItem reg:EntityReferencereg:RegisterItem reg:EntityReference
induced membership
relation default is
rdfs:member
container view
full view
UKGovLD Registry
Full information model
• Derives from ISO 19135
Geographic information - procedures for item registration
• Also considered OASIS ebRIM
– not so suited to flexible RDF model
– RegRep API did provide API checklist
• Container model compatible with Linked Data
Platform (1st draft!)
• Versioning model based on approached used
in Environment Agency linked data
UKGovLD Registry
reg:Register
rdfs:label [1..*]
dct:description [1..*]
reg:owner [1] (foaf:Agent)
reg:manager [1] (foaf:Agent)
dct:license [0..*]
reg:containedItemClass [0..*]
reg:operatingLanguage [0..*]
reg:governancePolicy [0..*] (rdfs:Resource)
reg:validationQuery [0..*]
dct:modified [0..1] (inferred)
void:uriLookupEndpoint [0..*]
void:uriSpace [0..1]
void:exampleResource [0..*]
void:openSearchDescription [0..*]
reg:subregister
reg:RegisterItem
rdfs:label [1..*]
dct:description [0..*]
dct:dateSubmitted [1] (automatic)
dct:dateAccepted [0..1]
dct:modified [0..1] (inferred)
reg:itemClass [1..*]
reg:submitter [1] (foaf:Agent)
dct:license [0..*]
reg:status [1..*]
reg:category [0..*] (skos:Concept)
reg:notation [0..1]
reg:alias [0..*]
reg:hasView [0..*]
reg:representationOf [0..*]
reg:register
reg:predecessor
version:Version
owl:versionInfo [1]
time:Inteval
version:interval
dct:replaces
dct:replacedBy
version:VersionedThing
version:currentVersion
reg:Status
reg:statusNotAccepted
reg:statusSubmitted
reg:statusInvalid
reg:statusAccepted
reg:statusValid
reg:statusExperimental
reg:statusStable
reg:statusDeprecated
reg:statusSuperseded
reg:statusRetired
dct:isVersionOf
reg:status
reg:EntityReference
reg:entity [1]
reg:sourceGraph [0..1]
reg:definition
Void:Dataset
ldp:Container
ldp:membershipPredicate
reg:inverseMembershipPredicate
Full information model
UKGovLD Registry
Status lifecycle
UKGovLD Registry
Status lifecycle
• controls visibility
– not accepted => doesn’t appear (register view, validation)
• guidance to consumers
– experimental/stable => no effect on behaviour
• workflow is external
– registry provides machinery to
• store status
• control who is authorized to change the status
• some validation of submissions
• list submissions in a given state
– how/whether submissions are scrutinized, how decision is
made up to the register manager
UKGovLD Registry
Outline
• Background and motivation
• Details
– information model
– core functionality
– advanced topics
– status
• Applying it
• Discussion and next steps
UKGovLD Registry
Functions: basic access
• read (GET)
– entity, item (its metadata), specific versions
– register (metadata, list of contents, page-able, prior states)
– content negotiation – Turtle, RDF/XML, JSON/LD, HTML
• entity lookup
– return what registry knows about any URI
• validate
– test a set of URIs to see they are known and Accepted
(within a register sub-tree)
• search
– free text search (within a register sub-tree)
UKGovLD Registry
Demo
• Browse of colours register
• RDF of colours register and a colour
UKGovLD Registry
Functions: Registration
• single entity
– POST simple entry to register
– basic metadata created automatically
– UI allows for extensible forms to create entries
• entity with metadata
– include explicit RegisterItem to annotate
– allows you to register an external URI
– can register multiple entries at once this way
• batch registration
– register a complete collection – register + contents
– e.g. upload a complete skos:ConceptScheme or skos:Collection
– can configure bulk types
UKGovLD Registry
Status and validation
• some automated checks at registration
– minimal properties of entity (label, type)
– type matches one of (optional) allowed types for this register
– register can specify SPARQL ASK queries for validation
• item status
– default is “Submitted”
– can set initial status in API or explicit metadata, if you have the
authority
– can change status through API or UI following the lifecycle
– administrators can force a status change outside lifecycle
• approval process
– is outside the registry itself
– just have property to state the policy and API/UI to do the updates
– some operations may have their own complex workflow
UKGovLD Registry
Versioning and history
• changes to an item generates new version
– metadata record is versioned _red:1 _red:2 ....
– separate copy of entity (separate named graph)
• Register is versioned if items added
– or their status is changed
• API supports
– list versions
– version at point in time (including for Register)
– get version
UKGovLD Registry
Demo
• Add a new colour via form
• Change its status
• Update it
• View history
• Upload a complete concept scheme
• Show registration of referenced entities
UKGovLD Registry
Outline
• Background and motivation
• Details
– information model
– core functionality
– advanced topics
– status
• Applying it
• Discussion and next steps
UKGovLD Registry Federation, delegation and namespaces
reg:Delegated
reg:delegationTarget
reg:NamespaceForward
reg:forwardingCode [0..1]
reg:DelegatedRegister
reg:enumerationSubject [0..1]
reg:enumerationPredicate [0..1]
reg:enumerationObject [0..1]
reg:Register
reg:FederatedRegister
reg:forwardingCode [0..1]
UKGovLD Registry Federation, delegation and namespaces
Case 1: External entities
– identifier published in different namespace
– want to include it in authoritative list
Solution:
– just register as a referenced entity
– already seen this
– authoritative because it’s on the list
– can record properties of the entity, and maintain
history
– no namespace management involved
UKGovLD Registry
Referenced entities
/local
/id
/local-authority
Registry External service
e.g. opencommunities.org
Hosted by LA directly
UKGovLD Registry
Case 2: Namespace allocation
– want someone else to serve part of the registry
namespace
– might be a single item or a complete register sub tree
– e.g. allocating namespace in location.data.gov.uk for
serving INSPIRE spatial object identifiers
Solution:
– reg:NamespaceForward
– can be a redirect (30X) or proxy (200)
– no constraints on whether target acts like a Registry
– target ought to serve linked data with URIs in the right
namespace, but not required
Federation, delegation and namespaces
UKGovLD Registry
Namespace forward
/local
/id
/local-authority
Registry External web site
could be anything
UKGovLD Registry Federation, delegation and namespaces
Case 3: Federated register
– want someone else to run part of the registry
infrastructure but act like one big registry
– integrated search, validation etc
Solution:
– reg:FederatedRegister
– can be a redirect (30X) or proxy (200)
– target endpoint must comply with Registry API at
least for search, validation and entity lookup
UKGovLD Registry
Federated register
/local
/id
/local-authority
Registry
Federated registry
/local-authority
/id
UKGovLD Registry Federation, delegation and namespaces
Case 4: Delegating a register
– some one else to serve the list of contents of the
register
– but they only have triple store, not full registry
implementation
Solution:
– solution eg:DelegatedRegister
– specify SPARQL endpoint and triple
pattern to enumerate members
reg:DelegatedRegister
reg:delegationTarget [1]
reg:enumerationSubject [0..1]
reg:enumerationPredicate [0..1]
reg:enumerationObject [0..1]
UKGovLD Registry
Delegated register
/local
/id
/local-authority
Registry
External SPARQL service
UKGovLD Registry
Demo
• Proxy namespace
• Delegated register (if time)
UKGovLD Registry
Security model
• authentication
– OpenID (e.g. Google, Google profile)
• authorization
– permissions
• Register, Update, StatusUpdate, Force, Grant, GrantAdmin
• inherit down the tree
• e.g.: Register,Update:/example/local
– can grant to known user or anyone authenticated
– bundled into roles
• Maintainer – Update, Grant
• Manager – Register, StatusUpdate, Update, Grant
• Authorized – Register, Update, StatusUpdate - for experimental areas
• Administrator - anything
UKGovLD Registry message queue infrastructure (tbd)
• publish/subscribe on notification of changes
– plug in audit trail recording
– replicate to duplicate stores for read-only load
balance
– hook for workflow
• scheduling analysis of registrations
– e.g. analyse registered datasets to map and
visualize identifier usage
UKGovLD Registry
Status
• Proof of concept
– API implementation build on top of Jena and Lucene
– everything except federated search
– UI based on simple templating
– proxy mode handled by dynamic reconfig of external nginx
– open source (license details to be agreed)
• Remaining planned work
– notification
– registration of datasets
– packaging and deployment instructions
– optional: some GML rendering support?
• Next steps
– pilot applying it
UKGovLD Registry
Conceptual architecture
router
renderer
request
processor
user
credentials
roles and
bindings
auth
registry
core
logic
Registry
RDF store
text index
style and
templates
external
UI
admin
UI
log
audit
trail
storeAPI
nginx
proxy
conf
API
UKGovLD Registry
Outline
• Background and motivation
• Details
– information model
– core functionality
– advanced topics
– status
• Applying it
• Discussion and next steps
UKGovLD Registry
Local authorities example
• illustrated various options
– maintain list in register
• some might be managed in registry, LA can edit
• some might be served by LA
• some might be served by DCLG
• single list supporting search, validation
– DCLG run a registry service and federated (or just
forward) part of namespace
– DCLG maintain master list in triple store, use
delegated register
UKGovLD Registry
Dataset registration
• would like to know how identifiers are being used
and how they connect data sets
• support registration of RDF data sets
– queue separate analysis suite to extract usage
information
– create metadata
– add these back into the registry record
– support visual browse
• POC does not include analysis and visualization
but provides hooks to third party modules
• e.g. Epimorphics InfoMap visualizer
UKGovLD Registry
Dataset analysis
UKGovLD Registry
WMO application of Registry
publishing existing authoritative
code-tables as web-accessible
resources
43
UKGovLD Registry WMO data exchange governance
WMO No. 306 Manual on Codes: the ‘crown-jewels’ of WMO … decades
of expert effort in developing and maintaining a ‘shared language’ for
describing meteorological phenomena.
44
UKGovLD Registry Model-driven data exchange standards
WMO data product specifications are being updated in line with
community best practice ; a model driven approach based on the ISO
19100-series of International Standards enables a semantic model to be
encoded in a variety of formats – including GML.
This is very similar to the approach taken within INSPIRE.
45
GML/XML
data products
«conformsto»
XML Schema 1.0
Schematron .
Validation schema
and rules
Validatable
data products
********
****
*******
*********
*********
********
**
*******
**********
**
«derivedfrom»
Application Schema
(aka ‘conceptual model’)
Technology independent description of
content and structure of information to
be exchanged for a given application
FullMoon
UKGovLD Registry Binding data model to WMO code-tables
46
Excerpt from BUFR edition 4 Code- and Flag-tables
IWXXM METAR/SPECI
«CodeList»
RunwayDeposits
URI: http://codes.wmo.int/bufr4/codeflag/0-20-086
tagged values:
vocabulary = “http://codes.wmo.int/bufr4/codeflag/0-20-086”
extensibility = “none”
«DataType»
AerodromeRunwayState
+ runway : Runway [0..1]
+ cleared : Boolean [0..1]
+ contamination : RunwayContamination [0..1]
+ snowClosure : Boolean [0..1]
+ depositType : RunwayDeposits [0..1]
+ depthOfDeposit : DepthOfDeposit [0..1]
+ brakingAction : BrakingAction [0..1]
+ frictionCoefficient : FrictionCoefficient [0..1]
UKGovLD Registry Code-table as web-accessible register
47
UKGovLD Registry Referencing terms from data products
48
UKGovLD Registry
NAMESPACE FORWARD:
publishing a simple ontology within
the namespace of the registry
49
UKGovLD Registry Pre-determined properties
WMO code-tables (e.g. Common code-table
C-6: List of units) have a number of specific
properties defined …
50
UKGovLD Registry Published definitions required
To represent the information from the code-tables in RDF we need to
reference definitions of those properties …
51
UKGovLD Registry A simple ontology
As the properties support the definition of the code-tables themselves,
the definitions are published within the namespace of the registry …
52
UKGovLD Registry A simple ontology (with #frag IDs)
Following de facto practice, the definitions are published using the
#fragment identifier pattern …
53
UKGovLD Registry Namespace forward (proxy)
The #fragment identifier pattern means that the registry’s “/ID”
‘containment’ pattern is not applicable – therefore we register the
entire ontology as a single DELEGATED resource using a PROXY pattern
54
UKGovLD Registry Namespace forward: configuration
The NAMESPACE FORWARD is configured using the web UI by an
authenticated user with appropriate permissions …
(but can also be done programmatically via the RESTful API)
55
UKGovLD Registry Externally managed HTTP server
54.246.66.105 is an anonymous HTTP server (hosted on Amazon EC2)
with content negotiation configured for HTML, RDF/XML and TTL – all of
which are provided as static resources …
56
UKGovLD Registry
ukgovld-registry.dnsalias.net
Apache2 +
mod_negotiation
54.246.66.105
NGINX
UKGovLD Registry Tracking changes; version & history
This approach enables one to track changes as new versions of the
ontology are published; with each new version published to a new
directory one can amend the delegation target to update the registry.
57
UKGovLD Registry
Discussion
• use cases for where to apply this?
• missing features needed for these?
• what would it take to pilot one?
UKGovLD Registry
Links
• Design notes and API details
https://github.com/der/ukl-registry-poc/wiki
• Proof of concept deployment
http://ukgovld-registry.dnsalias.net/
No service guarantees
UKGovLD Registry
UKGovLD Registry
Convenient views
• full RegisterItem/Register structure complex
• versioning makes that a lot worse
//registry
Register
VersionedThing
//registry:1
Register
Version
//registry/_reg
RegisterItem
VersionedThing
//registry/_reg:1
RegisterItem
Version
//registry/reg
Register
VersionedThing
//registry/reg/_foo
RegisterItem
VersionedThing
//registry/reg/foo
(entity)//registry/reg:1
Register
Version
//registry/reg/_foo:1
RegisterItem
Version
dct:versionOf
dct:versionOf dct:versionOf
dct:versionOf
reg:register
reg:register
reg:definition
reg:definition
//registry/_reg:2
RegisterItem
Version
//registry:2
Register
Version
//registry/reg:2
Register
Version
//registry/reg/_foo:2
RegisterItem
Version

Contenu connexe

Tendances

Shrinking the silo boundary: data and schema in the Semantic Web
Shrinking the silo boundary: data and schema in the Semantic WebShrinking the silo boundary: data and schema in the Semantic Web
Shrinking the silo boundary: data and schema in the Semantic WebGordon Dunsire
 
Resource description framework
Resource description frameworkResource description framework
Resource description frameworkhozifa1010
 
20080917 Rev
20080917 Rev20080917 Rev
20080917 Revcharper
 
Owl web ontology language
Owl  web ontology languageOwl  web ontology language
Owl web ontology languagehassco2011
 
Site Interoperability Projects at DERI Galway's SW Cluster
Site Interoperability Projects at DERI Galway's SW ClusterSite Interoperability Projects at DERI Galway's SW Cluster
Site Interoperability Projects at DERI Galway's SW ClusterJohn Breslin
 
Open Data and CKAN Data Catalogues
Open Data and CKAN Data CataloguesOpen Data and CKAN Data Catalogues
Open Data and CKAN Data Cataloguesdavid-read
 
The Semantic Web #9 - Web Ontology Language (OWL)
The Semantic Web #9 - Web Ontology Language (OWL)The Semantic Web #9 - Web Ontology Language (OWL)
The Semantic Web #9 - Web Ontology Language (OWL)Myungjin Lee
 
Virtuoso Sponger - RDFizer Middleware for creating RDF from non RDF Data Sources
Virtuoso Sponger - RDFizer Middleware for creating RDF from non RDF Data SourcesVirtuoso Sponger - RDFizer Middleware for creating RDF from non RDF Data Sources
Virtuoso Sponger - RDFizer Middleware for creating RDF from non RDF Data Sourcesrumito
 
First Steps in Semantic Data Modelling and Search & Analytics in the Cloud
First Steps in Semantic Data Modelling and Search & Analytics in the CloudFirst Steps in Semantic Data Modelling and Search & Analytics in the Cloud
First Steps in Semantic Data Modelling and Search & Analytics in the CloudOntotext
 
Customizing CKAN
Customizing CKANCustomizing CKAN
Customizing CKANOKCon2013
 
Metadata for your Digital Collections
Metadata for your Digital CollectionsMetadata for your Digital Collections
Metadata for your Digital CollectionsJenn Riley
 
AAT LOD Microthesauri
AAT LOD MicrothesauriAAT LOD Microthesauri
AAT LOD MicrothesauriMarcia Zeng
 
Ckan tutorial odw2013 131109
Ckan tutorial odw2013 131109Ckan tutorial odw2013 131109
Ckan tutorial odw2013 131109Chengjen Lee
 
Linked Data for Czech Legislation
Linked Data for Czech LegislationLinked Data for Czech Legislation
Linked Data for Czech LegislationMartin Necasky
 

Tendances (20)

Shrinking the silo boundary: data and schema in the Semantic Web
Shrinking the silo boundary: data and schema in the Semantic WebShrinking the silo boundary: data and schema in the Semantic Web
Shrinking the silo boundary: data and schema in the Semantic Web
 
Globe seminar
Globe seminarGlobe seminar
Globe seminar
 
NISO/DCMI Webinar: Metadata for Managing Scientific Research Data
NISO/DCMI Webinar: Metadata for Managing Scientific Research DataNISO/DCMI Webinar: Metadata for Managing Scientific Research Data
NISO/DCMI Webinar: Metadata for Managing Scientific Research Data
 
Resource description framework
Resource description frameworkResource description framework
Resource description framework
 
20080917 Rev
20080917 Rev20080917 Rev
20080917 Rev
 
Introduction to RDF
Introduction to RDFIntroduction to RDF
Introduction to RDF
 
Owl web ontology language
Owl  web ontology languageOwl  web ontology language
Owl web ontology language
 
Site Interoperability Projects at DERI Galway's SW Cluster
Site Interoperability Projects at DERI Galway's SW ClusterSite Interoperability Projects at DERI Galway's SW Cluster
Site Interoperability Projects at DERI Galway's SW Cluster
 
Open Data and CKAN Data Catalogues
Open Data and CKAN Data CataloguesOpen Data and CKAN Data Catalogues
Open Data and CKAN Data Catalogues
 
The Semantic Web #9 - Web Ontology Language (OWL)
The Semantic Web #9 - Web Ontology Language (OWL)The Semantic Web #9 - Web Ontology Language (OWL)
The Semantic Web #9 - Web Ontology Language (OWL)
 
Introduction to HDF5 Data Model, Programming Model and Library APIs
Introduction to HDF5 Data Model, Programming Model and Library APIsIntroduction to HDF5 Data Model, Programming Model and Library APIs
Introduction to HDF5 Data Model, Programming Model and Library APIs
 
Virtuoso Sponger - RDFizer Middleware for creating RDF from non RDF Data Sources
Virtuoso Sponger - RDFizer Middleware for creating RDF from non RDF Data SourcesVirtuoso Sponger - RDFizer Middleware for creating RDF from non RDF Data Sources
Virtuoso Sponger - RDFizer Middleware for creating RDF from non RDF Data Sources
 
First Steps in Semantic Data Modelling and Search & Analytics in the Cloud
First Steps in Semantic Data Modelling and Search & Analytics in the CloudFirst Steps in Semantic Data Modelling and Search & Analytics in the Cloud
First Steps in Semantic Data Modelling and Search & Analytics in the Cloud
 
Customizing CKAN
Customizing CKANCustomizing CKAN
Customizing CKAN
 
Metadata for your Digital Collections
Metadata for your Digital CollectionsMetadata for your Digital Collections
Metadata for your Digital Collections
 
AAT LOD Microthesauri
AAT LOD MicrothesauriAAT LOD Microthesauri
AAT LOD Microthesauri
 
Ckan tutorial odw2013 131109
Ckan tutorial odw2013 131109Ckan tutorial odw2013 131109
Ckan tutorial odw2013 131109
 
CKAN as an open-source data management solution for open data
CKAN as an open-source data management solution for open data CKAN as an open-source data management solution for open data
CKAN as an open-source data management solution for open data
 
Linked Data for Czech Legislation
Linked Data for Czech LegislationLinked Data for Czech Legislation
Linked Data for Czech Legislation
 
Semantic web Technology
Semantic web TechnologySemantic web Technology
Semantic web Technology
 

En vedette

Maximizing benefit of Open Data through Linked Data Services
Maximizing benefit of Open Data through Linked Data ServicesMaximizing benefit of Open Data through Linked Data Services
Maximizing benefit of Open Data through Linked Data ServicesDave Reynolds
 
Using linked data for dataset publication
Using linked data for dataset publicationUsing linked data for dataset publication
Using linked data for dataset publicationDave Reynolds
 
Linked Data Hypercubes - Semtech London
Linked Data Hypercubes - Semtech LondonLinked Data Hypercubes - Semtech London
Linked Data Hypercubes - Semtech LondonDave Reynolds
 
Validation: Requirements and approaches
Validation: Requirements and approachesValidation: Requirements and approaches
Validation: Requirements and approachesDave Reynolds
 
Resilient Linked Data
Resilient Linked DataResilient Linked Data
Resilient Linked DataDave Reynolds
 
Industrialized Linked Data
Industrialized Linked DataIndustrialized Linked Data
Industrialized Linked DataDave Reynolds
 
Ukgovld registry-intro
Ukgovld registry-introUkgovld registry-intro
Ukgovld registry-introDave Reynolds
 

En vedette (7)

Maximizing benefit of Open Data through Linked Data Services
Maximizing benefit of Open Data through Linked Data ServicesMaximizing benefit of Open Data through Linked Data Services
Maximizing benefit of Open Data through Linked Data Services
 
Using linked data for dataset publication
Using linked data for dataset publicationUsing linked data for dataset publication
Using linked data for dataset publication
 
Linked Data Hypercubes - Semtech London
Linked Data Hypercubes - Semtech LondonLinked Data Hypercubes - Semtech London
Linked Data Hypercubes - Semtech London
 
Validation: Requirements and approaches
Validation: Requirements and approachesValidation: Requirements and approaches
Validation: Requirements and approaches
 
Resilient Linked Data
Resilient Linked DataResilient Linked Data
Resilient Linked Data
 
Industrialized Linked Data
Industrialized Linked DataIndustrialized Linked Data
Industrialized Linked Data
 
Ukgovld registry-intro
Ukgovld registry-introUkgovld registry-intro
Ukgovld registry-intro
 

Similaire à Manage identifiers and authoritative lists with a Linked Data Registry

Linked Data from a Digital Object Management System
Linked Data from a Digital Object Management SystemLinked Data from a Digital Object Management System
Linked Data from a Digital Object Management SystemUldis Bojars
 
DSpace-CRIS Workshop OR2015: Slides
DSpace-CRIS Workshop OR2015: SlidesDSpace-CRIS Workshop OR2015: Slides
DSpace-CRIS Workshop OR2015: SlidesAndrea Bollini
 
Why do you consider to adopt Koha Open Source Integrated Library System for y...
Why do you consider to adopt Koha Open Source Integrated Library System for y...Why do you consider to adopt Koha Open Source Integrated Library System for y...
Why do you consider to adopt Koha Open Source Integrated Library System for y...Md. Zahid Hossain Shoeb
 
S. Bartoli & F. Pompermaier – A Semantic Big Data Companion
S. Bartoli & F. Pompermaier – A Semantic Big Data CompanionS. Bartoli & F. Pompermaier – A Semantic Big Data Companion
S. Bartoli & F. Pompermaier – A Semantic Big Data CompanionFlink Forward
 
WIPS Global Brochure, New
WIPS Global Brochure, NewWIPS Global Brochure, New
WIPS Global Brochure, Newshikha gupta
 
FAIR Workflows and Research Objects get a Workout
FAIR Workflows and Research Objects get a Workout FAIR Workflows and Research Objects get a Workout
FAIR Workflows and Research Objects get a Workout Carole Goble
 
EAD Revision, EAC-CPF introduction
EAD Revision, EAC-CPF introductionEAD Revision, EAC-CPF introduction
EAD Revision, EAC-CPF introductiontimothyryan50
 
spatial data infrastructure : data modelling and web services for data access
spatial data infrastructure : data modelling and web services for data accessspatial data infrastructure : data modelling and web services for data access
spatial data infrastructure : data modelling and web services for data accessDesconnets Jean-Christophe
 
(ATS6-PLAT02) Accelrys Catalog and Protocol Validation
(ATS6-PLAT02) Accelrys Catalog and Protocol Validation(ATS6-PLAT02) Accelrys Catalog and Protocol Validation
(ATS6-PLAT02) Accelrys Catalog and Protocol ValidationBIOVIA
 
Hypermedia for Machine APIs
Hypermedia for Machine APIsHypermedia for Machine APIs
Hypermedia for Machine APIsMichael Koster
 
Alfresco Business Reporting - Tech Talk Live 20130501
Alfresco Business Reporting - Tech Talk Live 20130501Alfresco Business Reporting - Tech Talk Live 20130501
Alfresco Business Reporting - Tech Talk Live 20130501Tjarda Peelen
 
hibernateormfeatures-140223193044-phpapp02.pdf
hibernateormfeatures-140223193044-phpapp02.pdfhibernateormfeatures-140223193044-phpapp02.pdf
hibernateormfeatures-140223193044-phpapp02.pdfPatiento Del Mar
 
Roaring with elastic search sangam2018
Roaring with elastic search sangam2018Roaring with elastic search sangam2018
Roaring with elastic search sangam2018Vinay Kumar
 

Similaire à Manage identifiers and authoritative lists with a Linked Data Registry (20)

Linked Data from a Digital Object Management System
Linked Data from a Digital Object Management SystemLinked Data from a Digital Object Management System
Linked Data from a Digital Object Management System
 
Elasticsearch Introduction at BigData meetup
Elasticsearch Introduction at BigData meetupElasticsearch Introduction at BigData meetup
Elasticsearch Introduction at BigData meetup
 
DSpace-CRIS Workshop OR2015: Slides
DSpace-CRIS Workshop OR2015: SlidesDSpace-CRIS Workshop OR2015: Slides
DSpace-CRIS Workshop OR2015: Slides
 
Tthornton code4lib
Tthornton code4libTthornton code4lib
Tthornton code4lib
 
Why do you consider to adopt Koha Open Source Integrated Library System for y...
Why do you consider to adopt Koha Open Source Integrated Library System for y...Why do you consider to adopt Koha Open Source Integrated Library System for y...
Why do you consider to adopt Koha Open Source Integrated Library System for y...
 
Sql Server2008
Sql Server2008Sql Server2008
Sql Server2008
 
S. Bartoli & F. Pompermaier – A Semantic Big Data Companion
S. Bartoli & F. Pompermaier – A Semantic Big Data CompanionS. Bartoli & F. Pompermaier – A Semantic Big Data Companion
S. Bartoli & F. Pompermaier – A Semantic Big Data Companion
 
Legislation.gov.uk
Legislation.gov.ukLegislation.gov.uk
Legislation.gov.uk
 
WIPS Global Brochure, New
WIPS Global Brochure, NewWIPS Global Brochure, New
WIPS Global Brochure, New
 
FAIR Workflows and Research Objects get a Workout
FAIR Workflows and Research Objects get a Workout FAIR Workflows and Research Objects get a Workout
FAIR Workflows and Research Objects get a Workout
 
EAD Revision, EAC-CPF introduction
EAD Revision, EAC-CPF introductionEAD Revision, EAC-CPF introduction
EAD Revision, EAC-CPF introduction
 
spatial data infrastructure : data modelling and web services for data access
spatial data infrastructure : data modelling and web services for data accessspatial data infrastructure : data modelling and web services for data access
spatial data infrastructure : data modelling and web services for data access
 
(ATS6-PLAT02) Accelrys Catalog and Protocol Validation
(ATS6-PLAT02) Accelrys Catalog and Protocol Validation(ATS6-PLAT02) Accelrys Catalog and Protocol Validation
(ATS6-PLAT02) Accelrys Catalog and Protocol Validation
 
Hypermedia for Machine APIs
Hypermedia for Machine APIsHypermedia for Machine APIs
Hypermedia for Machine APIs
 
JOSA TechTalk: Metadata Management
in Big Data
JOSA TechTalk: Metadata Management
in Big DataJOSA TechTalk: Metadata Management
in Big Data
JOSA TechTalk: Metadata Management
in Big Data
 
Alfresco Business Reporting - Tech Talk Live 20130501
Alfresco Business Reporting - Tech Talk Live 20130501Alfresco Business Reporting - Tech Talk Live 20130501
Alfresco Business Reporting - Tech Talk Live 20130501
 
hibernateormfeatures-140223193044-phpapp02.pdf
hibernateormfeatures-140223193044-phpapp02.pdfhibernateormfeatures-140223193044-phpapp02.pdf
hibernateormfeatures-140223193044-phpapp02.pdf
 
Design patternsforiot
Design patternsforiotDesign patternsforiot
Design patternsforiot
 
Roaring with elastic search sangam2018
Roaring with elastic search sangam2018Roaring with elastic search sangam2018
Roaring with elastic search sangam2018
 
DL-architecture.ppt
DL-architecture.pptDL-architecture.ppt
DL-architecture.ppt
 

Manage identifiers and authoritative lists with a Linked Data Registry

  • 1. UKGovLD Registry A register. Image by http://www.flickr.com/photos/tekniskamuseet/
  • 2. UKGovLD Registry Outline • Background and motivation • Details – information model – core functionality – advanced topics – status • Applying it • Discussion and next steps
  • 3. UKGovLD Registry Motivation Support coordinated publication of reference information as Linked Data We want people to use URIs to identify things – codes and code lists – ontologies, thesauri and concepts – units of measure – organizations – spatial objects – licenses – coordinate reference systems – ...
  • 4. UKGovLD Registry but it’s not trivial ... Each organization use its own namespace? – need infrastructure and expertise to serve the data – fragmented namespace – where do consumers get the complete authoritative list? Shared namespace? – how to allocate new identifiers in that shared space? – how to collaboratively maintain identifiers and lists? – how to scale? Either way: – how to manage the life-cycle of introducing and deprecating terms?
  • 5. UKGovLD Registry Linked Data Registry Infrastructure to enable organizations to create and manage identifiers and authoritative lists – allocation and registration of approved codes, vocabularies and reference URIs – management of shared namespaces, including clash-free allocation and delegation – registration of published datasets supporting discovery and cross-reference
  • 6. UKGovLD Registry Three functions • identifier & list management – create, update, reserve, lifecycle, versioning ... • repository – managed identifiers (information served from registry) as well as referenced identifiers • namespace management – delegate a list or part of namespace to others – federate to other repositories => very general platform – can register anything you can give a URI and RDF description – use only the aspects you need for a given situation – “registry does X” does not mean it answers every X-shaped need you might have
  • 7. UKGovLD Registry Conceptual architecture router renderer request processor user credentials roles and bindings auth registry core logic Registry RDF store text index style and templates external UI admin UI log audit trail storeAPI nginx proxy conf API
  • 8. UKGovLD Registry Running examples • Code list of colours – simple, neutral • UK Local authorities – centrally created identifiers v. local creation? – local control and maintenance? – offloading management
  • 9. UKGovLD Registry Outline • Background and motivation • Details – information model – core functionality – advanced topics – status • Applying it • Discussion and next steps
  • 10. UKGovLD Registry Information model • hierarchy of Registers • things in registers are Register Items – metadata • thing described by a RegisterItem called an entity / /reg1 /def /system /def/colours /def/weather-types /def/colours/_red /def/colours/_green (describes /def/colours/red) (describes /def/colours/green) Naming convention: metadata item has “_”
  • 11. UKGovLD Registry Information model complicated by: – item v. entity – versioning reg:Register entity reg:register reg:definition reg:entity reg:RegisterItem reg:EntityReferencereg:RegisterItem reg:EntityReference
  • 12. UKGovLD Registry Information model • default linked data view of Register is simplified • configurable – alternative membership property or inverse property – so can make a register look like a skos:Collection, skos:ConceptScheme, owl:Ontology ... – also acts as a LDP container • but can request full view (?_view=withMetadata) reg:Register entity reg:register reg:definition reg:entity reg:RegisterItem reg:EntityReferencereg:RegisterItem reg:EntityReference induced membership relation default is rdfs:member container view full view
  • 13. UKGovLD Registry Full information model • Derives from ISO 19135 Geographic information - procedures for item registration • Also considered OASIS ebRIM – not so suited to flexible RDF model – RegRep API did provide API checklist • Container model compatible with Linked Data Platform (1st draft!) • Versioning model based on approached used in Environment Agency linked data
  • 14. UKGovLD Registry reg:Register rdfs:label [1..*] dct:description [1..*] reg:owner [1] (foaf:Agent) reg:manager [1] (foaf:Agent) dct:license [0..*] reg:containedItemClass [0..*] reg:operatingLanguage [0..*] reg:governancePolicy [0..*] (rdfs:Resource) reg:validationQuery [0..*] dct:modified [0..1] (inferred) void:uriLookupEndpoint [0..*] void:uriSpace [0..1] void:exampleResource [0..*] void:openSearchDescription [0..*] reg:subregister reg:RegisterItem rdfs:label [1..*] dct:description [0..*] dct:dateSubmitted [1] (automatic) dct:dateAccepted [0..1] dct:modified [0..1] (inferred) reg:itemClass [1..*] reg:submitter [1] (foaf:Agent) dct:license [0..*] reg:status [1..*] reg:category [0..*] (skos:Concept) reg:notation [0..1] reg:alias [0..*] reg:hasView [0..*] reg:representationOf [0..*] reg:register reg:predecessor version:Version owl:versionInfo [1] time:Inteval version:interval dct:replaces dct:replacedBy version:VersionedThing version:currentVersion reg:Status reg:statusNotAccepted reg:statusSubmitted reg:statusInvalid reg:statusAccepted reg:statusValid reg:statusExperimental reg:statusStable reg:statusDeprecated reg:statusSuperseded reg:statusRetired dct:isVersionOf reg:status reg:EntityReference reg:entity [1] reg:sourceGraph [0..1] reg:definition Void:Dataset ldp:Container ldp:membershipPredicate reg:inverseMembershipPredicate Full information model
  • 16. UKGovLD Registry Status lifecycle • controls visibility – not accepted => doesn’t appear (register view, validation) • guidance to consumers – experimental/stable => no effect on behaviour • workflow is external – registry provides machinery to • store status • control who is authorized to change the status • some validation of submissions • list submissions in a given state – how/whether submissions are scrutinized, how decision is made up to the register manager
  • 17. UKGovLD Registry Outline • Background and motivation • Details – information model – core functionality – advanced topics – status • Applying it • Discussion and next steps
  • 18. UKGovLD Registry Functions: basic access • read (GET) – entity, item (its metadata), specific versions – register (metadata, list of contents, page-able, prior states) – content negotiation – Turtle, RDF/XML, JSON/LD, HTML • entity lookup – return what registry knows about any URI • validate – test a set of URIs to see they are known and Accepted (within a register sub-tree) • search – free text search (within a register sub-tree)
  • 19. UKGovLD Registry Demo • Browse of colours register • RDF of colours register and a colour
  • 20. UKGovLD Registry Functions: Registration • single entity – POST simple entry to register – basic metadata created automatically – UI allows for extensible forms to create entries • entity with metadata – include explicit RegisterItem to annotate – allows you to register an external URI – can register multiple entries at once this way • batch registration – register a complete collection – register + contents – e.g. upload a complete skos:ConceptScheme or skos:Collection – can configure bulk types
  • 21. UKGovLD Registry Status and validation • some automated checks at registration – minimal properties of entity (label, type) – type matches one of (optional) allowed types for this register – register can specify SPARQL ASK queries for validation • item status – default is “Submitted” – can set initial status in API or explicit metadata, if you have the authority – can change status through API or UI following the lifecycle – administrators can force a status change outside lifecycle • approval process – is outside the registry itself – just have property to state the policy and API/UI to do the updates – some operations may have their own complex workflow
  • 22. UKGovLD Registry Versioning and history • changes to an item generates new version – metadata record is versioned _red:1 _red:2 .... – separate copy of entity (separate named graph) • Register is versioned if items added – or their status is changed • API supports – list versions – version at point in time (including for Register) – get version
  • 23. UKGovLD Registry Demo • Add a new colour via form • Change its status • Update it • View history • Upload a complete concept scheme • Show registration of referenced entities
  • 24. UKGovLD Registry Outline • Background and motivation • Details – information model – core functionality – advanced topics – status • Applying it • Discussion and next steps
  • 25. UKGovLD Registry Federation, delegation and namespaces reg:Delegated reg:delegationTarget reg:NamespaceForward reg:forwardingCode [0..1] reg:DelegatedRegister reg:enumerationSubject [0..1] reg:enumerationPredicate [0..1] reg:enumerationObject [0..1] reg:Register reg:FederatedRegister reg:forwardingCode [0..1]
  • 26. UKGovLD Registry Federation, delegation and namespaces Case 1: External entities – identifier published in different namespace – want to include it in authoritative list Solution: – just register as a referenced entity – already seen this – authoritative because it’s on the list – can record properties of the entity, and maintain history – no namespace management involved
  • 27. UKGovLD Registry Referenced entities /local /id /local-authority Registry External service e.g. opencommunities.org Hosted by LA directly
  • 28. UKGovLD Registry Case 2: Namespace allocation – want someone else to serve part of the registry namespace – might be a single item or a complete register sub tree – e.g. allocating namespace in location.data.gov.uk for serving INSPIRE spatial object identifiers Solution: – reg:NamespaceForward – can be a redirect (30X) or proxy (200) – no constraints on whether target acts like a Registry – target ought to serve linked data with URIs in the right namespace, but not required Federation, delegation and namespaces
  • 30. UKGovLD Registry Federation, delegation and namespaces Case 3: Federated register – want someone else to run part of the registry infrastructure but act like one big registry – integrated search, validation etc Solution: – reg:FederatedRegister – can be a redirect (30X) or proxy (200) – target endpoint must comply with Registry API at least for search, validation and entity lookup
  • 32. UKGovLD Registry Federation, delegation and namespaces Case 4: Delegating a register – some one else to serve the list of contents of the register – but they only have triple store, not full registry implementation Solution: – solution eg:DelegatedRegister – specify SPARQL endpoint and triple pattern to enumerate members reg:DelegatedRegister reg:delegationTarget [1] reg:enumerationSubject [0..1] reg:enumerationPredicate [0..1] reg:enumerationObject [0..1]
  • 34. UKGovLD Registry Demo • Proxy namespace • Delegated register (if time)
  • 35. UKGovLD Registry Security model • authentication – OpenID (e.g. Google, Google profile) • authorization – permissions • Register, Update, StatusUpdate, Force, Grant, GrantAdmin • inherit down the tree • e.g.: Register,Update:/example/local – can grant to known user or anyone authenticated – bundled into roles • Maintainer – Update, Grant • Manager – Register, StatusUpdate, Update, Grant • Authorized – Register, Update, StatusUpdate - for experimental areas • Administrator - anything
  • 36. UKGovLD Registry message queue infrastructure (tbd) • publish/subscribe on notification of changes – plug in audit trail recording – replicate to duplicate stores for read-only load balance – hook for workflow • scheduling analysis of registrations – e.g. analyse registered datasets to map and visualize identifier usage
  • 37. UKGovLD Registry Status • Proof of concept – API implementation build on top of Jena and Lucene – everything except federated search – UI based on simple templating – proxy mode handled by dynamic reconfig of external nginx – open source (license details to be agreed) • Remaining planned work – notification – registration of datasets – packaging and deployment instructions – optional: some GML rendering support? • Next steps – pilot applying it
  • 38. UKGovLD Registry Conceptual architecture router renderer request processor user credentials roles and bindings auth registry core logic Registry RDF store text index style and templates external UI admin UI log audit trail storeAPI nginx proxy conf API
  • 39. UKGovLD Registry Outline • Background and motivation • Details – information model – core functionality – advanced topics – status • Applying it • Discussion and next steps
  • 40. UKGovLD Registry Local authorities example • illustrated various options – maintain list in register • some might be managed in registry, LA can edit • some might be served by LA • some might be served by DCLG • single list supporting search, validation – DCLG run a registry service and federated (or just forward) part of namespace – DCLG maintain master list in triple store, use delegated register
  • 41. UKGovLD Registry Dataset registration • would like to know how identifiers are being used and how they connect data sets • support registration of RDF data sets – queue separate analysis suite to extract usage information – create metadata – add these back into the registry record – support visual browse • POC does not include analysis and visualization but provides hooks to third party modules • e.g. Epimorphics InfoMap visualizer
  • 43. UKGovLD Registry WMO application of Registry publishing existing authoritative code-tables as web-accessible resources 43
  • 44. UKGovLD Registry WMO data exchange governance WMO No. 306 Manual on Codes: the ‘crown-jewels’ of WMO … decades of expert effort in developing and maintaining a ‘shared language’ for describing meteorological phenomena. 44
  • 45. UKGovLD Registry Model-driven data exchange standards WMO data product specifications are being updated in line with community best practice ; a model driven approach based on the ISO 19100-series of International Standards enables a semantic model to be encoded in a variety of formats – including GML. This is very similar to the approach taken within INSPIRE. 45 GML/XML data products «conformsto» XML Schema 1.0 Schematron . Validation schema and rules Validatable data products ******** **** ******* ********* ********* ******** ** ******* ********** ** «derivedfrom» Application Schema (aka ‘conceptual model’) Technology independent description of content and structure of information to be exchanged for a given application FullMoon
  • 46. UKGovLD Registry Binding data model to WMO code-tables 46 Excerpt from BUFR edition 4 Code- and Flag-tables IWXXM METAR/SPECI «CodeList» RunwayDeposits URI: http://codes.wmo.int/bufr4/codeflag/0-20-086 tagged values: vocabulary = “http://codes.wmo.int/bufr4/codeflag/0-20-086” extensibility = “none” «DataType» AerodromeRunwayState + runway : Runway [0..1] + cleared : Boolean [0..1] + contamination : RunwayContamination [0..1] + snowClosure : Boolean [0..1] + depositType : RunwayDeposits [0..1] + depthOfDeposit : DepthOfDeposit [0..1] + brakingAction : BrakingAction [0..1] + frictionCoefficient : FrictionCoefficient [0..1]
  • 47. UKGovLD Registry Code-table as web-accessible register 47
  • 48. UKGovLD Registry Referencing terms from data products 48
  • 49. UKGovLD Registry NAMESPACE FORWARD: publishing a simple ontology within the namespace of the registry 49
  • 50. UKGovLD Registry Pre-determined properties WMO code-tables (e.g. Common code-table C-6: List of units) have a number of specific properties defined … 50
  • 51. UKGovLD Registry Published definitions required To represent the information from the code-tables in RDF we need to reference definitions of those properties … 51
  • 52. UKGovLD Registry A simple ontology As the properties support the definition of the code-tables themselves, the definitions are published within the namespace of the registry … 52
  • 53. UKGovLD Registry A simple ontology (with #frag IDs) Following de facto practice, the definitions are published using the #fragment identifier pattern … 53
  • 54. UKGovLD Registry Namespace forward (proxy) The #fragment identifier pattern means that the registry’s “/ID” ‘containment’ pattern is not applicable – therefore we register the entire ontology as a single DELEGATED resource using a PROXY pattern 54
  • 55. UKGovLD Registry Namespace forward: configuration The NAMESPACE FORWARD is configured using the web UI by an authenticated user with appropriate permissions … (but can also be done programmatically via the RESTful API) 55
  • 56. UKGovLD Registry Externally managed HTTP server 54.246.66.105 is an anonymous HTTP server (hosted on Amazon EC2) with content negotiation configured for HTML, RDF/XML and TTL – all of which are provided as static resources … 56 UKGovLD Registry ukgovld-registry.dnsalias.net Apache2 + mod_negotiation 54.246.66.105 NGINX
  • 57. UKGovLD Registry Tracking changes; version & history This approach enables one to track changes as new versions of the ontology are published; with each new version published to a new directory one can amend the delegation target to update the registry. 57
  • 58. UKGovLD Registry Discussion • use cases for where to apply this? • missing features needed for these? • what would it take to pilot one?
  • 59. UKGovLD Registry Links • Design notes and API details https://github.com/der/ukl-registry-poc/wiki • Proof of concept deployment http://ukgovld-registry.dnsalias.net/ No service guarantees
  • 61. UKGovLD Registry Convenient views • full RegisterItem/Register structure complex • versioning makes that a lot worse //registry Register VersionedThing //registry:1 Register Version //registry/_reg RegisterItem VersionedThing //registry/_reg:1 RegisterItem Version //registry/reg Register VersionedThing //registry/reg/_foo RegisterItem VersionedThing //registry/reg/foo (entity)//registry/reg:1 Register Version //registry/reg/_foo:1 RegisterItem Version dct:versionOf dct:versionOf dct:versionOf dct:versionOf reg:register reg:register reg:definition reg:definition //registry/_reg:2 RegisterItem Version //registry:2 Register Version //registry/reg:2 Register Version //registry/reg/_foo:2 RegisterItem Version