ANNOTOPIA ARCHITECTURE" Annotopia: A Universal Annotation Hub Architecture

Annotopia Architecture by Paolo Ciccarese
Paolo Ciccarese, PhD
Assistant Professor of Neurology at Harvard Medical School
Co-chair of the W3C Open Annotation Community Group
Architect of the Annotopia Open Annotation Server
ANNOTOPIA
UNIVERSAL ANNOTATION HUB
Overview (Draft 11 – 2014-11-19)
This work is licensed under the Creative Commons Attribution-ShareAlike 4.0 International License. To
view a copy of this license, visit http://creativecommons.org/licenses/by-sa/4.0/

What is it Annotopia?
• It is a Universal Annotation Hub
• It provides you with back-end technology so
that you can focus on the user interface and
the knowledge creation process
• It consists of a collections of modules that
provide useful services for creating, persisting,
searching, sharing and enriching annotation
• It is Open Annotation* compliant
*Open Annotation Data Model (http://www.openannotation.org/spec/core/)

Imagine annotating…
• Documents (HTML and PDF) with multiple tools
(Domeo, Utopia, annotator.js)
• (Bibliographic) records with bibliographic
management platforms
• (Biological) Entities in databases and knowledge
bases
• Hi-res imaging (Brain imaging) and 3D models
(proteins, brain anatomy…)
• Data in various shapes and domains (genomic
data, clinical data…)

Annotopia: De-Siloing Annotation
Custom Annotators
Annotopia
Instance 1
Applications
Text Mining
Aggregated
querying and
browsing
Mashups
APIs
APIs
Annotopia
Instance 2
JSON-LD
JSON-LD

Current Architecture (see github*)
Spring security supporting OpenIDs and OAuth
Admin Dashboard
Annotopia
Storage APIs Data Cruncher
Agents/Users/Groups
Smart
Storage
Virtuoso Triple Store
MySQL Relational DBMS
Connectors
Text Mining
Terms Search
Entity Recognition
Vocabulary search
Linked Data Mashups
Bibliographic
Metadata Manager
Ext 1
Ext 2
Ext n
…
https://github.com/Annotopia

Connectors (Annotation services)
• They are extension points, new connectors
can be added for integrating new resources
• Provide one or more of the following features
– Terms search
– Text mining
– Vocabulary listing
• They normalize all the content so that clients
get always the same format, no matter what
service they asked for

Connectors (2)
• They are dynamically loaded by Annotopia
• They can be access to a façade that provides
for an API
• They can be ran standalone without having to
run the full Annotopia infrastructure
• They normalize the text mining results to the
Open Annotation Model

Implemented Connectors
• BioPortal: allows for term searching,
vocabulary listing and text mining through the
NCBO Annotator
– https://github.com/Annotopia/CnBioPortalConnector
• NIF: allows for term searching and text mining
through the NIF Annotator
– https://github.com/Annotopia/CnNifConnector

Connectors in progress
• EBI (European Bioinformatics Institute)
• DBpedia

Current Architecture (see github*)
Annotopia
Spring security supporting OpenIDs and OAuth
Admin Dashboard
Storage APIs Data Cruncher
* https://github.com/Annotopia
Connectors
NCBO Annotator
NIF Annotator
NCBO Terms Search
NIF Resources Search
….
Agents/Users/Groups
Smart
Storage
MySQL Relational DBMS
Bibliographic
Metadata Manager
Ext 1
Ext 2
Ext n
…

Annotopia: Smart Storage
• It is the module in charge of persisting
annotation
• It uses Virtuoso triple store through a Jena API
• Uses JSON-LD as serialization method
https://github.com/Annotopia/AtSmartStorage/

Currently Available APIs
• Open Annotation: for storing/retrieving Open
Annotation content
• Open Annotation Sets: for storing/retrieving
Open Annotation content organized in
annotation sets
• Open Annotation Validation: provides services
for validating Open Annotation content
• Reporting: provides some basic counters of
the storage status

Annotopia Storage APIs
openAnnotationSet openAnnotation
openAnnotationSetStorage
openAnnotationVirtuoso
jenaVirtuosoStore
openAnnotationStorage
jena
APIs
Services
openAnnotationValidation
Elastic Search

Open Annotation API

Posting Open Annotation Content
• The Open Annotation API is a RESTful API that
supports GET, POST, PUT and DELETE
• Payload is represented in JSON-LD, which
through appropriate context/framing can be
stored/retrieved from the triple store
• POST, PUT and DELETE now support a single
item at a time

Open Annotation API: POST (1)
• Creates a new Annotation.
Param Description Default
apiKey The application assigned apikey. The API calls are available
only for registered and authorized applications.
POST list of parameters
REQUIRED
item The Annotation content REQUIRED
outCmd It determines the kind of JSON-LD post processing. Possible
values are 'none', 'context', 'frame'. In the case of the POST
the framing is applied to the response which will send the
updated annotation back to the client.
none
incGph If true, this will include the metadata (provenance and
more) of the graph that is wrapping the Annotation
false
validate If 'ON' the software will go through the appropriate
validation algorithm before being stored
OFF

Open Annotation API: POST (2)
• The Current JSON-LD specification do not
support framing for Named Graphs. Therefore
the option ”outCmd":”frame” cannot be used
when:
– The annotation consists of multiple graphs
– The option ”incGph":”true” as the metadata graph
will be returned, as a separate Named Graph,
together with the annotation. This result in a
result of at least two graphs.

An example of POST (with OAuth)
curl -i -X POST http://serverip:port/s/annotation
-H "Content-Type: application/json"
-H "Authorization: Bearer 521b611a-efc6-4487-bb09-f8c8556b2cb0"
-d'{"apiKey":"testkey”, ”outCmd":”frame", "item":{ OA CONTENT/PAYLOAD }}
• The content or payload is formatted according to the Open
Annotation model (Eds. Sanderson, Ciccarese, Van de Sompel;
03 Feb 2013): http://www.openannotation.org/spec/core/

Ex of Open Annotation content - POST
{
"apiKey": "testkey", ”outCmd":”frame”,
"item": {
"@context": "https://raw2.github.com/Annotopia/AtSmartStorage/master/web-app/data/OAContext.json",
"@id": "urn:temp:001",
"@type": "http://www.w3.org/ns/oa#Annotation",
"motivatedBy": "oa:commenting",
"annotatedBy": {
"@id": "http://orcid.org/0000-0002-5156-2703",
"@type": "foaf:Person",
"foaf:name": "Paolo Ciccarese"
},
"annotatedAt": "2014-02-17T09:46:11EST",
"serializedBy": "urn:application:domeo",
"serializedAt": "2014-02-17T09:46:51EST",
"hasBody": {
"@type": [
"cnt:ContentAsText",
"dctypes:Text"
],
"cnt:chars": "What does this mean for...",
"dc:format": "text/plain"
},
"hasTarget": {
"@id": "urn:temp:2",
"@type": "oa:SpecificResource",
"hasSelector": {
"@type": "oa:TextQuoteSelector",
"exact": "a sequence of progressive...",
"prefix": "The data suggest that ",
"suffix": ""
},
"hasSource": {
"@id": "http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3292759/",
"@type": "dctypes:Text"
}
}
}
}
body
target

Smart Storage behavior
• Smart storage performs some basic checks on the
payload
• It mints stable URIs for the components of the
payload (Annotation, Body, Target)
• It stores each Open Annotation in a Named Graph
and it also generates additional metadata that
are stored in a separate global Provenance Graph
• It returns the payload with the above updates
and a short summary wrapper

Open Annotation POST response (a)
{"status":"saved", "result": {"duration": "89ms","graphs":"1","item":[{
"@graph" : [ {
"@id" : "http://localhost:8090/s/annotation/C59DBC51-D81E-4B4D-B71B-822BB8FA2C43",
"@type" : "oa:Annotation",
"http://purl.org/pav/previousVersion" : "urn:temp:001",
"annotatedAt" : "2014-02-17T09:46:11EST",
"annotatedBy" : {
"@id" : "http://orcid.org/0000-0002-5156-2703",
"@type" : "foaf:Person",
"name" : "Paolo Ciccarese"
},
"hasBody" : {
"@id" : "http://localhost:8090/s/content/EC3E3D9C-18E9-4820-9A35-5B0E979A8F6E",
"@type" : [ "dctypes:Text", "cnt:ContentAsText" ],
"format" : "text/plain",
"http://purl.org/pav/previousVersion" : "blank",
"chars" : "What does this mean for..."
},
"hasTarget" : {
"@id" : "http://localhost:8090/s/resource/701FDC2F-59AE-4AC1-BF5E-20B57B5146ED",
"@type" : "oa:SpecificResource",
"hasSelector" : {
"@id" : "_:b0",
"@type" : "oa:TextQuoteSelector",
"exact" : "a sequence of progressive...",
"prefix" : "The data suggest that ",
"suffix" : ""
},
"hasSource" : {
"@id" : "http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3292759/",
"@type" : "dctypes:Text"
}
},
"motivatedBy" : "oa:commenting",
"serializedAt" : "2014-02-17T09:46:51EST",
"serializedBy" : "urn:application:domeo"
} ]
}]}}
Summary
body
target

Open Annotation POST response (b)
{"status":"saved", "result": {"duration": "89ms","graphs":"1","item":[{
"@graph" : [ {
"annotatedBy" : {
"@id" : "http://orcid.org/0000-0002-5156-2703",
},
"hasBody" : {
},
"hasTarget" : {
"hasSelector" : {
"@id" : "_:b0",
"suffix" : ""
},
"hasSource" : {
}
},
} ]
}]}}
Minted URIs
old URI
Minted URIs
Minted URIs
old URI

Open Annotation API: PUT (1)
• Updates an existing Annotation.
PUT list of parameters
REQUIRED
item The Annotation content REQUIRED
none
false
OFF

An example of PUT (with OAuth)
curl -i -X PUT http://serverip:port/s/annotation
-d'{"apiKey":"testkey”, ”outCmd":”frame", "item":{ CONTENT/PAYLOAD }}
• The content or payload is formatted according to the Open
Annotation model (Eds. Sanderson, Ciccarese, Van de Sompel;
03 Feb 2013): http://www.openannotation.org/spec/core/

Ex of Open Annotation content - PUT
{
"apiKey": "testkey", ”outCmd":”frame”,
"item": {
"@context": "https://raw2.github.com/Annotopia/AtSmartStorage/master/web-app/data/OAContext.json",
"@id" : "http://localhost:8090/s/annotation/830ED7EE-BF7B-4A18-8AE1-A9AF96AC135B",
"annotatedBy" : {
"@id" : "http://orcid.org/0000-0002-5156-2703",
},
"hasBody" : {
"@id" : "http://localhost:8090/s/content/79317F9E-57FC-4938-AF52-C66962371451",
"chars" : ”New updated body"
},
"hasTarget" : {
"@id" : "http://localhost:8090/s/resource/73AC02CA-B719-478E-B6A3-EE8D18D57ED2",
"hasSelector" : {
"@id" : "_:b0",
"suffix" : ""
},
"hasSource" : {
}
},
}}
body
target

REQUIRED
max The number of records to display per page 10
offset The page to be returned 0
tgtUrl If specified, this will limit the results to the Annotations
that are targeting the specified URL
none
tgtFgt If true, this will include in the results the Annotations that
are targeting a fragment of the specified URL
true
values are 'none', 'context', 'frame'.
none
false
Open Annotation API: GET (1)
• Get existing Annotation(s). GET list of parameters

Some examples of GET (with OAuth)
curl -i -X GET http://localhost:8080/s/annotation/
--data '{"apiKey":"testkey", “max”:”5”,”offset”:”0”,”outCmd":”frame"}'
curl -i -X GET http://localhost:8080/s/annotation/
--data '{"apiKey":"testkey", “max”:”5”,”offset”:”0”,”outCmd":”frame”,”tgtUrl”:”
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3292759”}'

Ex of Open Annotation content - GET
{
{"status":"results", "result": {"total":”1", "pages":"0", "duration": ”46ms", "offset": "0", "max": ”5", "items":[{
"@graph" : [ {
"annotatedBy" : {
"@id" : "http://orcid.org/0000-0002-5156-2703",
},
"hasBody" : {
},
"hasTarget" : {
"hasSelector" : {
"@id" : "_:b0",
"suffix" : ""
},
"hasSource" : {
}
},
} ]
}]}}
Summary
body
target

Annotation Sets API

Annotation Sets
• They are not part of the current specification
• They allow to manage one or more Open
Annotations as a bundle
• Annotation Sets are used for instance by the
Domeo Annotation Tool*
• The current draft of Open Annotation in EPUB
is including the notion of set**
*http://annotationframework.org
** http://www.idpf.org/epub/oa/

Annotation Sets API: POST (1)
• Creates a new set of Annotations.
POST list of parameters
REQUIRED
set The set of Annotation items REQUIRED
none
false
OFF

In progress…

Annotation Faceted Search

ANNOTOPIA ARCHITECTURE" Annotopia: A Universal Annotation Hub Architecture

Recommandé

Recommandé

Contenu connexe

Tendances

Tendances (20)

En vedette

En vedette (20)

Similaire à ANNOTOPIA ARCHITECTURE" Annotopia: A Universal Annotation Hub Architecture

Similaire à ANNOTOPIA ARCHITECTURE" Annotopia: A Universal Annotation Hub Architecture (20)

Plus de Paolo Ciccarese

Plus de Paolo Ciccarese (6)

Dernier

Dernier (20)

ANNOTOPIA ARCHITECTURE" Annotopia: A Universal Annotation Hub Architecture