3. About Ontotext
• Provides products and services for creating,
managing and exploiting semantic data
– Founded in 2000
– Offices in Bulgaria, USA and UK
• Major clients and industries
– Media & Publishing (BBC, Press Association)
– HCLS (AstraZeneca, UCB)
– Cultural Heritage (The British Museum, The National
Archives, Polish National Museum, Dutch Public Library)
– Defense and Homeland Security
Semantic Technologies for Big Data Sep 2012 #3
4. Outline
• Semantic Technologies for the Enterprise
• Semantic Technologies for Big Data
• Success stories
Semantic Technologies for Big Data Sep 2012 #4
6. The need for a smarter Web
• "The Semantic Web is an extension of the current web in
which information is given well-defined meaning, better
enabling computers and people to work in cooperation.“ (Tim
Berners-Lee, 2001)
• “PricewaterhouseCoopers believes a Web of data will develop
that fully augments the document Web of today. You’ll be
able to find pieces of data sets from different places,
aggregate them without warehousing, and analyze them in a
more straightforward, powerful way than you can now.”
(PWC, May 2009)
Semantic Technologies for Big Data Sep 2012 #6
7. Linked Data
• Linked Data is a set of principles that allows
publishing, querying and consumption of RDF data,
distributed across different servers
• Design principles
– Use unambiguous identifiers for resources (URIs)
– Use HTTP URIs (dereference-able)
– Provide useful information for URI lookups
– Interlink resources
Semantic Technologies for Big Data Sep 2012 #7
8. The Semantic Web timeline
RDF RDF 2
DAML+OIL OWL OWL 2
SPARQL SPARQL 1.1
RIF
RDFa
SAWSDL
LOD
SKOS
HCLS
SSN
RDB2RDF
PIL
GLD
LDP
1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012
Semantic Technologies for Big Data Sep 2012 #8
9. Enterprise Information Management Challenges
• Many disparate data sources and data silos
• Many point-to-point interfaces
• Data sources with similar/inconsistent information
• Complex data integration processes inadequate for
changing business requirements
• Most of the knowledge is hidden in texts
• Difficult to integrate & analyse structured data and
text
Semantic Technologies for Big Data Sep 2012 #9
10. Semantic Web and Linked Data Opportunities for the
Enterprise
• Simplify the information integration processes
– Flexible, easy to evolve data model
– Bottom-up / incremental integration
– Efficiently integrate structured and unstructured data
• Provide an enterprise metadata layer
– Unified metadata vocabulary for the enterprise
– Align the legacy data silos
– Improve the information sharing and reuse
Semantic Technologies for Big Data Sep 2012 #10
11. Semantic Web and Linked Data Opportunities for the
Enterprise (2)
• Discovery and enrichment of information
– Interlink people, organisations, events, etc.
– Enrich enterprise content with structured annotations
– Discover implicit links and relationships
• Unified access to information within the enterprise
– Simplified infrastructure based on open web standards
• Information interchange across a value chain
– Easy publishing and consumption of Linked Data
• Augments existing IT assets and technologies
– No need for disruptive replacement
Semantic Technologies for Big Data Sep 2012 #11
12. XML and RDF: friends or foes
• Complement each other
– XML best for content, structure and interchange format
– RDF for metadata layer and semantics
• Typical use case
– Many XML content data sources
• Content stored in an XML store (XQuery and XSLT)
– Structured data sources & external Linked Data
• RDF-ized and stored in an RDF store (SPARQL)
– Metadata extracted from content
• stored in an RDF store (SPARQL)
• semantic search and metadata driven content delivery
Semantic Technologies for Big Data Sep 2012 #12
13. BBC Sports
(c) BBC
Semantic Technologies for Big Data Sep 2012 #13
14. Added value of RDF
• Explicit semantics
– Intended meaning of entities and relations
• Global identifiers (URIs)
• Simple and flexible graph-based data model
• Easier data mapping & integration
– Bottom-up / incremental data integration with owl:sameAs
• Inference of implicit information
• Working with distributed information
– Linked Data, federated SPARQL
Semantic Technologies for Big Data Sep 2012 #14
15. Added value of RDF
• Descriptive / agile schema
– Open World Assumption, don’t restrict predicates
– Generated dynamically from data
• Queries based on meaning
– Not depending on structure / order of statements
• Data and queries may use different vocabularies
• Exploratory queries
• Choice of OWL2 profiles
– Tradeoff features vs performance
– New profiles may emerge in the future
Semantic Technologies for Big Data Sep 2012 #15
17. The three V’s of Big Data
• Velocity
– Streaming, sensor, real-time data
– Solution: distributed processing & storage
– Semantic challenge: stream reasoning
• Volume
– Petabytes of data
– Solution: distributed processing & storage
– Semantic challenge: distributed reasoning & querying
• Variety
– Structured, semi-structured and unstructured data
– Semantic Technologies (RDF) are a good fit
Semantic Technologies for Big Data Sep 2012 #17
18. Types of Big Data (NIST)
• Type 1
– Velocity (-), Volume (-), Variety (+)
– Perfect fit for Semantic Technologies
• Type 2
– Velocity and/or Volume, Variety (-)
– Only horizontal scalability required, traditional approaches
are a good enough fit
• Type 3
– All V’s
– Semantic Technologies not a good fit yet, but moving in
that direction
Semantic Technologies for Big Data Sep 2012 #18
19. Semantic Technologies for Volume and Velocity
• Promising ongoing research
• Distributed inference with Hadoop/Storm
• Stream reasoning
– Continuous queries
– Continuous (dynamic) semantics
• SPARQL to Pig translation
• Distributed RDF stores on top of NoSQL
• C-SPARQL, EP-SPARQL, CQELS
Semantic Technologies for Big Data Sep 2012 #19
20. Linked Open Data Cloud (Sep 2011)
(c) Cyganiak & Jentzsch
Semantic Technologies for Big Data Sep 2012 #20
21. From Big Linked Data to Linked Big Data
• Big Linked Data
– Big Data approach adopted by the Linked Data community
• In particular handling Volume and Velocity
– Exponential growth of Linked Data in the last 5 years
• Linked Big Data
– Linked Data approach adopted by the Big Data community
– RDF data model for Variety
– Enrich Big Data with metadata and semantics – more
powerful analytics on top of it
– Interlink Big Data sets
– Simplify data access and data integration
Semantic Technologies for Big Data Sep 2012 #21
22. SUCCESS STORIES
Semantic Technologies for Big Data Sep 2012 #22
23. Typical Use Cases for Linked Data and Semantic
Technologies
• Publish / consume Linked Data across enterprises
– Linked Data is not necessarily free data
– Facilitate data interchange within the value chain
• Information integration within the enterprise
– Integrated asset management / align data silos
– Master Data Management
• Knowledge discovery and semantic search
– Integrate structured and unstructured data
– Enrich and interlink information
– Semantic search and exploration of information
Semantic Technologies for Big Data Sep 2012 #23
25. The National Archives (Ontotext)
• Challenge
– Large archive of various UK Government websites since
1997
– Lots of duplicated information & documents
– Inefficient search & navigation
• Semantic Knowledge Base project goals
– Integrate multiple data sources
– Extract information & metadata from archived documents
– Interlink the web archive with data.gov.uk and LOD data
– Advanced search & navigation of the archive
Semantic Technologies for Big Data Sep 2012 #25
26. The National Archives (Ontotext)
Front Ends:
Semantic
Search
O1 SPARQL A
3rd party C
O2 Ontology graph B
D
Editors exploration
O3 Data
Trans-
formation
and
Semantic Repository Integration
Semantic
Annotation
SKB Ontologies
Factual Knowledge
(TNA data, LOD,
data.gov.uk)
Identity
Semantic annotations Resolution
Annotation Process
(GATE Teamware)
Semantic Index
Semantic Technologies for Big Data Sep 2012 #26
27. The National Archives (Ontotext)
• The numbers
– 2.5 billion input files
– 40TB compressed archive data
– 10 billion RDF triples stored in OWLIM
– 33,000 EC2 hours used on AWS
– Dynamic EC2 cluster (180 instances average, 500 max)
• Major challenges
– Complex pre-processing of documents
– De-duplication of information & documents
– EC2/RRS performance & reliability
Semantic Technologies for Big Data Sep 2012 #27
28. Dutch Public Library (Ontotext + Dayon)
• Challenge
– Many disparate data sources, inefficient search
• Goals
– Data integration
– Automated metadata generation
– Open search platform
• Numbers
– 500 heterogeneous data sources
– 40 million cultural heritage artifacts to be describes
– 6-8 billion triples to be stored into the knowledge base
Semantic Technologies for Big Data Sep 2012 #28
29. Linked Life Data (Ontotext)
• Challenge
– Disparate, heterogeneous and unaligned data silos lock
valuable biomedical information
• Goals
– Semantic warehouse integrating and interlinking public
biomedical data sources
– Interactive discovery and exploration
• Numbers
– 25+ heterogeneous biomedical data sources integrated
– 1 billion entities described
– 5.5 billion RDF triples
Semantic Technologies for Big Data Sep 2012 #29
30. Linked Life Data (Ontotext)
Semantic Technologies for Big Data Sep 2012 #30
31. Linked Life Data-as-a-Service (Ontotext)
• More data sources
• Large scale text mining over the LOD cloud
• Adapted for specific use cases
• UCB use case
– 2 billion entities described
– 11 billion RDF triples
Semantic Technologies for Big Data Sep 2012 #31
32. Dynamic Semantic Publishing (Ontotext)
• Challenge
– Difficult & slow to aggregate content from various sources
• Goals
– Metadata generation for news (semantic annotation)
– Interlink & categorize content
– Metadata driven web pages
• Numbers
– Nearly real-time processing & annotation required
– Tens of millions (SPARQL) queries to the knowledge base
per day
Semantic Technologies for Big Data Sep 2012 #32
33. Trillion RDF triples (Franz Inc.)
• Use case
– Use RDF for the customer management database of a
telecom
• Challenge
– 4,000 triples per customer, more than a trillion for the
whole customer base
• Numbers
– 1 trillion triples stored in AllegroGraph by Franz Inc
• Hardware requirements undisclosed
• The 310 billion triple result used 8-CPU system with 2TB RAM
Semantic Technologies for Big Data Sep 2012 #33
34. uRiKA (Cray/YarcData)
• Big Data appliance for graph analytics
– Based on the Threadstormtm architecture
– Up to 8K processors, 512TB RAM, 350TB/hr IO throughput
• In-memory RDF database
• SPARQL 1.0 engine
Semantic Technologies for Big Data Sep 2012 #34
(c) YarcData
35. TAKEAWAYS
Semantic Technologies for Big Data Sep 2012 #35
36. Semantic Technologies for Big Data
• Rich ecosystem of Semantic Technologies since 1999
• Strong Enterprise focus in the last 5 years
• Semantic Technologies provide opportunity for
reducing the cost and complexity of data integration
• Common metadata layer for the enterprise
• More powerful ways to find and explore information
• RDF complements XML within the enterprise
• Semantic Technologies are a good fit for Big Data’s
Variety
Semantic Technologies for Big Data Sep 2012 #36
37. Semantic Technologies for Big Data
• Velocity and Volume still challenging for Semantic
Technologies, but lots of progress in that direction
• Linked Data will grow into Big Linked Data, but Big
Data will also benefit from evolving into Linked Big
Data
• Interesting success stories for Semantic Technologies
in Big Data scenarios
Semantic Technologies for Big Data Sep 2012 #37
38. THANK YOU!
Semantic Technologies for Big Data Sep 2012 #38