LoCloud: Micro Services and the Digitisation Workflow Walter Koch, AIT

1. XIIth Annual International Conference for
Professionals in Cultural Heritage
LoCloud Micro Services and
the Digitisation Workflow
Walter Koch (kochw@ait.co.at)
AIT‐Angewandte Informationstechnik
Forschungs GmbH, Graz ‐ Austria
LoCloud is funded by the
European Commission's ICT Policy Support Programme

2. 2015‐11‐08 Walter Koch (AIT) 2
Content
• LoCloud Project (relevant activities)
• Vocabulary Microservice
• Microservice used in the OpenUp! Project
• Digitisation Workflow (Books)
• Digital Curation (Long Term Preservation)

3. LoCloud Project
(relevant activities)
2015‐11‐08 Walter Koch (AIT) 3

4. 2015‐11‐08 Walter Koch (AIT) 4
locoud.eu
About:
– LoCloud is a Best Practice Network co‐funded under the
CIP ICT‐PSP programme of the European Commission.
– LoCloud is supporting small and medium‐sized
institutions in making their content and
metadata available to Europeana.
– A cloud‐based technology infrastructure will enable the
aggregation of local content, and
– a number of micro‐services will help to reduce technical,
semantic and skills barriers and to render the content
more discoverable and interoperable.

5. 2015‐11‐08 Walter Koch (AIT) 5
Microservices
Six development teams:
AIT, Athena RC, AVINET, IPCHS, UPV/EHU, VUKFC
All the services are implemented on virtual machines in a
cloud testlab (using the OpenNebula cloud computing
platform)
API documentation is available for each service
End‐user interfaces are provided for three microservices
Each service has been integrated in MORe

6. 2015‐11‐08 Walter Koch (AIT) 6
LoCloud Microservices
http://support.locloud.eu/
Cloud‐based micro‐services for metadata enrichment and
capture:
• Geo‐location API
• Vocabulary service
• Historic Place Names service
• Geo‐coding application
• Vocabulary matching service
• Background linking service
• Wikimedia application

7. 2015‐11‐08 Walter Koch (AIT) 7
LoGeo API
• Uses NER-Name Entity Recognition
• Recognizes and returns one or more place names
candidates accompanied by geographical coordinates.
GPL
IPCHS

8. 2015‐11‐08 Walter Koch (AIT) 8
Vocabulary service
• Based on “TemaTres”
• Incorporates 29 standard
thesauri
• allows users to add and
create vocabularies, and
add terms or translations
• add thesauri terms to all
items of a metadata
package
• Integrated in MORE
• Exploited by the LoCloud
vocabulary matching
microservice via its API
GPLAIT

9. 2015‐11‐08 Walter Koch (AIT) 9
Historic Place Names service
• A semi‐automatic historical geo‐
information management function
and web service.
• Enables local cultural institutions
to collaborate in developing the
Thesaurus of HPN and enriching
ingested metadata.
• Enables export of selected
LoCloud HPN Thesaurus data sets.
GPLVUKF

10. 2015‐11‐08 Walter Koch (AIT) 10
Geo‐coding application
• End-user oriented application
• allows institutions to add geographical locations to existing content in
a controlled crowd-sourcing environment and then
• load the augmented data back into their original production
databases.
GPLAVINET

11. 2015‐11‐08 Walter Koch (AIT) 11
Vocabulary matching service
• Builds on work in the PATHS project (FP7)
• Automatically assigns relevant concepts and terms to item records
• Retrieves terms from the Vocabulary service
UPV/EHU Apache License 2.0

12. 2015‐11‐08 Walter Koch (AIT) 12
Background linking service
• Based on work in PATHS
• Follows the Named Entity
Disambiguation approach
• Enriches items with links to
external contextual
information in DBpedia and
Wikipedia
Apache License 2.0UPV/EHU

13. 2015‐11‐08 Walter Koch (AIT) 13
Wikimedia application
• A web service that uses a REST interface to communicate with
Wikimedia commons
• The main functionalities are harvesting content form
Wikimedia, parsing the harvested content, and identifying
useful entities that can be mapped to the ESE or EDM
metadata schemas.
• The mapped ESE / EDM records are then sent to MORe to be
delivered to Europeana.
Wikimedia Commons LicensingAthena RC

14. 2015‐11‐08 Walter Koch (AIT) 14
The LoCloud
Aggregation Structure
Enrichment
Services
EuropeanaEuropeana
Cloud
Testlab

15. 2015‐11‐08 Walter Koch (AIT) 15
About: Microservice
Wikipedia: https://en.wikipedia.org/wiki/Microservices (2015‐09‐15)
[...] microservices is a
• software architecture style in which
• complex applications are composed
of small, independent processes
• communicating with each other using language‐
agnostic APIs.
• These services are small, highly decoupled and
focus on doing a small task,facilitating a modular
approach to system‐building.

16. 2015‐11‐08 Walter Koch (AIT) 16
Evolution of Architectures
pwc: Technology Forecast: Rethinking Integration, Issue 1, 2014, page 3

17. Vocabulary Microservice
2015‐11‐08 Walter Koch (AIT) 17

18. 2015‐11‐08 Walter Koch (AIT) 18
http://vocabulary.locloud.eu/
• Vocabulary Microservices
• This is the entry page for the
LoCloud vocabulary application.
• LoCloud uses a variety of standard vocabularies
during the metadata enrichment process.
These vocabularies have been established with the
TemaTres tool and are available in the LoCloud cloud
testlab.
• In addition, it is possible to feed the tool with your
own vocabularies, establish new vocabularies or
add languages to existing vocabularies.

19. 2015‐11‐08 Walter Koch (AIT) 19
Vocabulary WebService
20+ Functions, API
Parameters
Example

20. 2015‐11‐08 Walter Koch (AIT) 20
Example: Search
UNESCO‐Thesaurus
http://test113.ait.co.at/tematres/unesco/services.php?task=search&arg=peace

21. 2015‐11‐08 Walter Koch (AIT) 21
SPARQL‐Example
DISMARC‐Genres
• Query: "Find all Broader Term for 'Acid Jazz‘“
• SPARQL End Point: http://test113.ait.co.at/tematres/vocab/sparql.php
Query
Result
Thesaurus
Entry

22. 2015‐11‐08 Walter Koch (AIT) 22
Service Call („fetchTopTerms“)
embedded in a test application
Micro-
Service
Vocabulary Web
Service

23. 2015‐11‐08 Walter Koch (AIT) 23
Test Application Development
Designer View
Development of a
Drop Down List
Development of a
Drop Down List

24. 2015‐11‐08 Walter Koch (AIT) 24
The Test Application:
Part of Data Entry
Path to the
Thesaurus Item
„Candidate Term“ 
Crowdsourcing activity 
Next Slide
(Vocabulary Management)

25. Microservice used in the
OpenUp! Project (open‐up.eu)
„Common Name Service“
2015‐11‐08 Walter Koch (AIT) 25

26. 2015‐11‐08 Walter Koch (AIT) 26
OpenUp! ‐ Natural History
Aggregator for Europeana
Micro Service:
Common Name
Service
Micro Service:
Common Name
Service

27. 2015‐11‐08 Walter Koch (AIT) 27
OpenUp!
Transformation Workflow
Invoke Micro
Service:
Common Name
Service
Next Slide

28. 2015‐11‐08 Walter Koch (AIT) 28
OpenUp!
Transformation Parameters
Micro Service:
Common Name Service
Service End Point (Cache)

29. 2015‐11‐08 Walter Koch (AIT) 29
Result: OpenUp!
Enriched Meta Data Set
Latin Name
Common
Names

30. Digitisation Workflow
(Books)
2015‐11‐08 Walter Koch (AIT) 30

31. 2015‐11‐08 Walter Koch (AIT) 31
Bibliotheca Alexandrina
http://www.bibalex.org
Points for MicroService Integration

32. 2015‐11‐08 Walter Koch (AIT) 32
Digital Curation
(Long Term Preservation)

33. 2015‐11‐08 Walter Koch (AIT) 33
Digital Curation
State‐of‐the‐Art Examples (2010)
• Archivematica
– Linux VM based toolset allows producers and archivists
to implement a small scale workflow
• UC3 / CDL Merritt
– Micro‐Services specifications
– implement building blocks for the Digital Curation
Lifecycle

34. 2015‐11‐08 Walter Koch (AIT) 34
Archivmatica
Micro Services

35. 2015‐11‐08 Walter Koch (AIT) 35
California Digital Library
UC3 Merritt
• California Digital Library UC Curation Center
• Independant, Interoperable Micro‐Services
• Complex curation function through combination
of atomistic services
Knowledge
Enhancement
Annotation

36. 2015‐11‐08 Walter Koch (AIT) 36
Micro‐Services
(Merritt vs. Archivematica)
CDL/UC3-MERRITT ARCHIVEMATICA

37. 2015‐11‐08 Walter Koch (AIT) 37
High‐Level Workflow
• OAIS activities provide
a sound basis
• Archivematica
Use Cases
• Merritt specifications
• Micro‐Services are the
building blocks

38. 2015‐11‐08 Walter Koch (AIT) 38
CDL
Curation Micro Services
1) „An Emergent Micro‐Services Approach to Digital Curation Infrastructure“, IPRES 2009
https://escholarship.org/uc/item/5313h6k9
2) UC Curation Center / California Digital Library, merritt: An Emergent Micro‐services Approach
to Digital Curation Infrastructure; Rev. 0.6, 2010‐03‐25
1) Annotation service 2)
The Annotation service provides
a mechanism by which the
consumers of managed digital
content can enrich that content
through additional description,
association, or related content.
The service supports the
following methods:
• Annotate
• Submit.

39. 2015‐11‐08 Walter Koch (AIT) 39
Contact
Walter Koch, kochw@ait.co.at
http://www.ait.co.at