Call Girls Bommasandra Just Call 👗 7737669865 👗 Top Class Call Girl Service B...
Building Enterprise-Ready Knowledge Graph Applications in the Cloud
1. Building Enterprise-Ready
Knowledge Graph Applications in the Cloud
Peter Haase
CEO, metaphacts GmbH
Michael Schmidt
Senior Software Development Engineer, Amazon Neptune
4. 4
COMPANY FACTS
• Founded in Q4 2014
• Headquartered in Walldorf, Germany
• Currently ~10 people
• Platform for knowledge graphs and knowledge graph applications
metaphacts at a Glance
5. 5
1. Welcome
2. Introduction to Knowledge Graphs
1. Basic Concepts and Technologies
2. Storing, Querying, and Maintaining Knowledge Graphs
3. Applications of Knowledge Graphs
3. Managing Knowledge Graphs
1. Creating Knowledge Graphs
2. Querying Knowledge Graphs
4. Building End-user Facing Applications on top of Knowledge Graphs
1. Visualization and Templating
2. Semantic Search
5. Building Developer APIs on top of Knowledge Graphs
1. Exposing Queries via REST
2. Scaling up with Neptune
Workshop Agenda
Hands on!
Demo
Hands on!
Hands on!
Hands on!
Hands on!
Hands on!
Demo
6. 6
• Build a portal to make current human proteomic
knowledge available
• Answer detailed questions about the human proteome
• I am researcher working researching heart diseases.
I want to know proteins associated with heart diseases.
Which of these proteins are located in the cell membrane?
• Enable querying via
• End user-oriented search
• APIs for easy consumption by applications
Use Case Scenario
7. 7
• Wikidata
• Community curated knowledge graph
• Broad coverage
• Integration hub for the Life Sciences
• Nextprot
• A very detailed knowledge graph
about human proteomics, aggregated
and curated knowledge
from many sources
• Federation across data sources
Knowledge Graphs Used for the Tutorial
8. 8
• Understanding of Knowledge Graphs
• Technology Stack
• Benefits of Knowledge Graphs
• Application Areas of Knowledge Graphs
• Setup of Enterprise-ready Knowledge Graph Architecture
• Scalable Storage and Querying with Enterprise Features (Amazon Neptune)
• Rapid Prototype & Application Building (metaphactory)
• Concepts for …
• … maintaining, querying, and searching knowledge graphs
• … building end-user facing applications on top of knowledge graphs
• … building and exposing scalable APIs on top of knowledge graphs
• End-to-end experience: throughout the tutorial, you’ll have the chance to set up a small
knowledge graph application on your own!
Learning Goals
16. 16
• We need a structured and formal representation of
knowledge
• We are surrounded by entities, which are connected by
relations
• Graphs are a natural way to represent entities and their
relationships
• Graphs can capture a broad spectrum of data from
structured to unstructured
• Graphs can be managed efficiently
Why (Knowledge) Graphs?
17. 17
A (very small) Knowledge Graph
http://www.w3.org/TR/2014/NOTE-rdf11-primer-20140225/example-graph.jpg
18. 18
• Semantic descriptions of
entities and their relationships
• Uses a knowledge
representation formalism
(RDF, RDF-Schema, OWL)
• Entities: real world objects (things, places, people) and
abstract concepts (genres, religions, professions)
• Relationships: graph-based data model where relationships
are first-class
• Semantic descriptions: types and properties with a well-
defined meaning (e.g. through an ontology)
• Possibly axiomatic knowledge (e.g. rules) to support
automated reasoning
What are Knowledge Graphs?
19. 19
Knowledge Graphs Built on the Semantic Web Layer Cake
Unicode URIs
RDF (Resource Description Framework)
RDF-Schema
OWLSKOS
SPARQL
Query language
Entities
Relationships
Vocabularies
Ontologies
Expressive Ontology
Language
Thesauri,
classification
schemes
Graph data
model
Simple vocabulary
language
20. 20
• Linked Data
• Set of standards, principles for publishing, sharing
and interrelating structured knowledge
• From data silos to a Web of Data
• Linked Data principles
1. Use URIs as names for things
2. Use HTTP URIs so that people can look up those names.
3. When someone looks up a URI, provide useful information, using the
standards: RDF, SPARQL
4. Include links to other URIs, so that they can discover more things.
• Linked Data as abstraction layer for virtualized data access across data spaces
• Platform independent data access
• Enables data portability and interoperability across current data silos
Linked Data
22. 22
How do we integrate these resources?
Michel Dumontier, Chicago Semantic Web Meetup
23. 23
Uniform Resource Identifier (URI) can be used as entity names
http://bio2rdf.org/uniprot:P05067
is a name for Amyloid precursor protein
http://bio2rdf.org/omim:104300
is a name for Alzheimer disease
uniprot:P05067
omim:104300
Allows one to talk about anything
Resource Description Framework (RDF)
Michel Dumontier, Chicago Semantic Web Meetup
24. 24
Allows one to express statements
An RDF statement consists of:
• Subject: resource identified by a URI
• Predicate: resource identified by a URI
• Object: resource or literal
Resource Description Framework (RDF)
25. 25
Protein
rdf:type
A RDF statement consists of:
– Subject: resource identified by a URI
– Predicate: resource identified by a URI
– Object: resource or literal
uniprot:P05067
Allows one to express statements
Resource Description Framework (RDF)
Michel Dumontier, Chicago Semantic Web Meetup
26. 26
uniprot:P05067 Membrane
Protein
is a
located in
uniprot:P05067
uniprot:P05067 uniprot:P05067
interacts with
UniProt
Gene Ontology
uniprot:P05067
is a
located in
interacts with
Unified view
+
+
iRefIndex
depends on consistent naming
Multi-Source Data Integration
Membrane
Protein
uniprot:P05067
Michel Dumontier, Chicago Semantic Web Meetup
68. 68
Current Uptake of Knowledge Graphs
See the panel on Wednesday, October 10. 17:00-18:00
PANEL: ENTERPRISE-SCALE KNOWLEDGE GRAPHS
See the presentation on Friday, October 12, 11:45 in the Industry Track:
AMAZON NEPTUNE: GRAPH DATA MANAGEMENT IN THE CLOUD
78. 82
metaphactory Login
Hands on!
Open the MetaphactoryURL noted down earlier in your browser (recommended: Firefox or Chrome) and log
in using user “admin”, using the MetaphactoryPassword noted down earlier.
80. 84
Data Loading with metaphactory: Upload Files
Load the Nextprot mini-ontology linked from
http://nextprot-neptune.metaphacts.cloud/
1. In the menu bar, select Quick Links / Data Import & Export
Drag and Drop your file
Hands on!
81. 85
Data Loading with metaphactory: Load from S3
Hands on!
See output of the CloudFormation Stack
Use one of
• s3://nextprot-mini/1
• s3://nextprot-mini/2
• s3://nextprot-mini/3
• s3://nextprot-mini/4
Turtle format
1. In the menu bar, select Quick Links / Data Import & Export
2. Select tab “Neptune Load”, set parameters as shown below
3. Select “Start Load”
85. 89
Knowledge Graphs Built on the Semantic Web Layer Cake
Unicode URIs
RDF (Resource Description Framework)
RDF-Schema
OWLSKOS
SPARQL
Query language
Entities
Relationships
Vocabularies
Ontologies
Expressive Ontology
Language
Thesauri,
classification
schemes
Graph data
model
Simple vocabulary
language
87. 91
Main idea: Pattern matching
• Queries describe sub-graphs of the queried graph
• Graph patterns are RDF graphs specified in Turtle syntax,
which contain variables (prefixed by “?”)
• Sub-graphs that match the graph patterns yield a result
SPARQL Query
?isoformentry:
NX_P20592
:isoform
90. 94
SPARQL Query: Components
Query form:
• ASK, SELECT, DESCRIBE or CONSTRUCT
• SELECT retrieves variables and their bindings as a table
PREFIX : <http://nextprot.org/rdf#>
PREFIX entry: <http://nextprot.org/rdf/entry>
SELECT ?isoform ?canonical
WHERE {
entry:NX_P20592 :isoform ?isoform .
?isoform :canonicalIsoform ?canonical
}
ORDER BY ?isoform
91. 95
SPARQL Query: Components
Query pattern:
• Defines patterns to match against the data
• Generalises Turtle with variables and keywords – N.B. final
period optional
PREFIX : <http://nextprot.org/rdf#>
PREFIX entry: <http://nextprot.org/rdf/entry>
SELECT ?isoform ?canonical
WHERE {
entry:NX_P20592 :isoform ?isoform .
?isoform :canonicalIsoform ?canonical
}
ORDER BY ?isoform
92. 96
SPARQL Query: Components
Solution modifier:
• Modify the result set
• ORDER BY, LIMIT or OFFSET re-organise rows;
• GROUP BY combines them
PREFIX : <http://nextprot.org/rdf#>
PREFIX entry: <http://nextprot.org/rdf/entry/>
SELECT ?isoform ?canonical
WHERE {
entry:NX_P20592 :isoform ?isoform .
?isoform :canonicalIsoform ?canonical
}
ORDER BY ?isoform
93. 97
SPARQL supports different query forms:
• ASK tests whether or not a query pattern has a solution.
Returns yes/no
• SELECT returns variables and their bindings directly
• CONSTRUCT returns a single RDF graph specified by a
graph template
• DESCRIBE returns a single RDF graph containing RDF
data about resource
Query Forms
95. 99
• The solution modifier projection nominates which
components of the matches should be returned
• “*” means all components should be returned
Query Form: SELECT
Query: Proteins that have a Transit peptide?
PREFIX : <http://nextprot.org/rdf#>
PREFIX cv: <http://nextprot.org/rdf/terminology/>
SELECT ?entry where {
?entry :isoform ?isoform .
?isoform :keyword ?keyword .
?keyword :term cv:KW-0809
}
96. 100
• A property path is a possible route through a graph
between two graph nodes
• Property paths allow for more concise expression of some
SPARQL basic graph patterns and also add the ability to
match arbitrary length paths
Query Form: SELECT
PREFIX : <http://nextprot.org/rdf#>
PREFIX cv: <http://nextprot.org/rdf/terminology/>
SELECT ?entry WHERE{
?entry :isoform / :keyword / :term cv:KW-0809
}
Query: Proteins that have a Transit peptide?
97. 102
Create and Execute your own Queries
• See sample queries from
http://nextprot-neptune.metaphacts.cloud/resource/Queries
98. 103
Query Catalog with Sample Queries
• Use the query catalog from
http://nextprot-neptune.metaphacts.cloud/sparql
Hands on!
99. 104
• Write a SPARQL query that
Returns proteins
• that are associated with heart diseases
• along with their location in the cell
• Save the queries to the query catalog
Exercise: SPARQL Query
Hands on!
100. 105
• Write a SPARQL query that ...
1. Returns isoforms of Hyaluronan synthase 3
2. Finds proteins phosphorylated and located in the cell
membrane
3. Returns proteins with more than 2 intramembrane
regions
4. Returns proteins located in mitochondrion and are
associated with a disease
Optional: Exercises
Hands on!
102. 107
Table
Transform your queries into
durable, interactive tables
Many customization
possibilities, e.g. pagination,
filters and cell templates
Graph
Visualize and explore connections in a graph view
Custom styling of the graph
Variety of graph layouts
Carousel
Animated browsing through a list of
result items
Chart
Visualize trends and
relationships between
numbers, ratios, or
proportions
Tree Table
Tree-based
visualization,
navigation and
browsing through sub-
tree structures
Map
Displaying spatial
data on a
geographic map
Visualization using metaphactory
104. 109
HTML Pages with Web Components
See example at http://nextprot-neptune.metaphacts.cloud/resource/Graph
105. 110
1. Open a new page. e.g.
http://ec2-52-71-247-162.compute-1.amazonaws.com/resource/MyPage
2. Click ”Edit page”
3. Paste the example snippet from
http://nextprot-neptune.metaphacts.cloud/resource/Graph
4. Save the page
5. Optional: Extend the query to show also children of child concepts (2 hops)
• Further documentation:
http://help.metaphacts.com/resource/Help:SemanticGraph
Exercise: Create Own Page with Visualization
Hands on!
106. 111
• Associate a class in the knowledge graph with a template
• The template is applied to instances of the class
HTML5 Template Pages
entry:NX_P20592
:Entry
rdf:type
108. 113
End-user Oriented Search Processes
Image based on: Tran, T., Herzig, D., Ladwig, G. SemSearchPro- Using semantics through the search process
Knowledge graph Query
Result visualization /
presentation
User query
(e.g. keywords, NL)
Query visualization
(Optional) User
System
Refinement
Presentation
Analysis
Presentation /
Ranking
Graph matching
Entity Extraction /
Semantic query analysis
109. 114
• Search functionalities
• Graphical query construction
• End user friendly search
interfaces for building and
sharing complex queries
• Semantic auto suggestion
• Interactive result visualization
• Faceted search and exploration
of item collections
• Ability to invoke external full text
search indices such as Solr including
the possibility to score, rank and limit
the results for responsive
autosuggestion
Example: Structured Search Generation
113. 118
Example: Simple Search Component
See example at http://nextprot-neptune.metaphacts.cloud/resource/Search
114. 119
1. Open a new page. e.g.
http://ec2-52-71-247-162.compute-1.amazonaws.com/resource/Search
2. Click “Edit Page”
3. Paste the example snippet from
http://nextprot-neptune.metaphacts.cloud/resource/Search
4. Save the page
• Further documentation:
https://help.metaphacts.com/resource/Help:SemanticSimpleSearch
Exercise: Create Own Page with Search Component
Hands on!
127. 132
• Tutorial material
• http://knowledgegraph.info
• Tutorial repeatable in your own AWS account
• Amazon Neptune
https://aws.amazon.com/neptune/
• metaphactory available on Amazon Marketplace
https://metaphacts.com/amazon-neptune
Links and References