Keynote at Industry Event: Technology Landscape 2013, Dayton, OH, USA. May 26, 2010.

1. 48th ACM Southeast Conference. ACMSE 2010.
Oxford, Mississippi. April 15-17, 2010.
Semantic Web powering Intelligent Enterprise and
Web Applications
Amit P. Sheth
LexisNexis Ohio Eminent Scholar
Ohio Center of Excellence in Knowledge enabled Computing (Kno.e.sis)
Wright State University, Dayton, OH
Technology Landscape 2013, Dayton OH. May 26, 2010

2. Ohio Center of Excellence on
Knowledge-Enabled Computing (Kno.e.sis)

3. 2D-3D & Immersive
Visualization, Human Impacting
affects
Computer Interfaces bottom line
Migraine
Domain Magnesium
Models/ Stress inhibit isa
Knowledge
Patient Calcium Channel
Blockers
Knowledge
discovery
Biomedical
SEMANTICS, MEANING PROCESSING Knowledge
Discovery,
Patterns / Inference / Reasoning
Meta data / Knowledge
Semantic Management &
Annotations Visualization
Search and
Metadata Extraction/Semantic Annotations browsing
Massive amounts of data
Structured text
(Scientific
Experimental Public domain
publications / Clinical Trial Data knowledge
white papers) Results
(PubMed)
3

4. Kno.e.sis Vision
Kno.e.sis’ leadership in semantic processing will
contribute to basic theory about computation and
cognitive systems, and address pressing practical
problems associated with productive thinking in the
face of an explosion of data.
Kno.e.sis intends to lead a march from information age
to meaning age.
4

5. Globally Competitive Careers
and Economic Development
WPAFB Directorates
Dayton Region Companies
Tech^Edge
Human
Sensor Woolpert REI Tech, Aptima
Effectiveness SAIC LexisNexis
Knowledge Workers, Products, Services and Applications
Defense/Aerospace Advanced Data Human Sciences
R&D Management & Health Care
Application to Regional Industry Cluster
Kno.e.sis+Faculty Strengths daytaOhio – a WCI
• Cognitive Science & Human Factors
• Data Analysis/Mining/Visualization • Visualization and Data Mgt
• Info. & Knowledge Mgmt Infrastructure
• Web 3.0 (Semantics, Services, Sensors) • Consulting and Technology
• Virtual Worlds, Social Computing
Transfer
• High Performance/Cloud Computing
• Bioinformatics/Biomedicine, Healthcare
Academic Research and Infrastructure
5

6. 6

7. Significant
Infrastructure
VERITAS Whole-Body Laser
Range Scanner
stereoscopic 3D
visualization
NMR AVL
7

8. Exceptional
Regional Collaboration
• At least 6 active projects with AFRL/WPAFB
• Human Effectiveness Directorate
• Sensors Directorate
8

9. Exceptional
National Collaboration
• Univ. of Georgia, Stanford, Purdue, OSU, Ohio U., Indiana U.
UC-Irvine, Michigan State U., Army, W3C
• Microsoft, IBM, HP, Google
9

10. Exceptional
International Collaboration
• U. Manchester, TU-Copenhagen, TU-Delft, DERI (Ireland),
Max-Planck Institute, U. Melbourne, U Queensland, NICTA-
Australia,CSIRO, DA-IICT (India)
10

11. 48th ACM Southeast Conference. ACMSE 2010.
Oxford, Mississippi. April 15-17, 2010.
Semantic Web powering Intelligent Enterprise and
Web Applications
Amit P. Sheth
LexisNexis Ohio Eminent Scholar
Ohio Center of Excellence in Knowledge enabled Computing (Kno.e.sis)
Wright State University, Dayton, OH
Technology Landscape 2013, Dayton OH. May 26, 2010

12. Evolution of the Web
Web as an oracle / assistant / partner
- “ask the Web”: using semantics to
leverage text + data + services
- Powerset
2007
Web of people
- social networks, user-created casual
content
- Twine, GeneRIF, Connotea
Web of resources
- data = service = data, mashups
- ubiquitous computing
Web of databases
- dynamically generated pages
1997 - web query interfaces
Web of pages
- text, manually created links
- extensive navigation
12

13. OUTLINE
• Semantic Web –key capabilities and
technlologies
• Real-world Applications demonstrating benefit
of semantic web technologies
• Exciting on-going research
13

14. Introduction
1
2
3
of
Semantic Web
14

15. Introduction [1]
• Ontology: Agreement with a common
vocabulary/nomenclature, conceptual models
and domain Knowledge
• Schema + Knowledge base
• Agreement is what enables interoperability
• Formal description - Machine processability is
what leads to automation
15

16. Introduction [2]
• Semantic Annotation (Metadata Extraction):
Associating meaning with data, or labeling
data so it is more meaningful to the system
and people.
• Can be manual, semi-automatic (automatic
with human verification), automatic.
16

17. From Syntax to Semantics
Deep semantics
Shallow semantics
17

18. Introduction [3]
• Reasoning/Computation: semantics enabled
search, integration, answering complex
queries, connections and analyses (paths, sub
graphs), pattern finding, mining, hypothesis
validation, discovery, visualization
18

19. Characteristics of Semantic Web
Self Easy to
Describing Understand
The Semantic Web:Machine &
Issued by
XML, RDF & Ontology
a Trusted Human
Authority Readable
Can be
Convertible Secured
Adapted from William Ruh (CISCO)
19

20. SW Stack: Architecture, Standards
20

21. a little bit about ontologies

22. Many Ontologies Available
e.g. Open Biomedical Ontologies
Open Biomedical Ontologies, http://obo.sourceforge.net/
22

23. From simple ontologies

24. Drug Ontology Hierarchy
(showing is-a relationships)
formulary_
non_drug_ interaction_ property formulary
reactant property
indication
indication_ property
owl:thing
monograph property
_ix_class prescription interaction_
_drug_ with_non_
brandname_ prescription
brand_name drug_reactant
prescription individual _drug interaction
_drug_
property brandname_
brandname_ composite prescription interaction_
undeclared _drug_ with_mono
interaction_
generic graph_ix_cl
with_prescri
cpnum_ generic_ ass
ption_drug
group composite
generic_
individual
24

25. to complex ontologies

26. N-Glycosylation metabolic pathway
GNT-I
attaches GlcNAc at position 2
N-glycan_beta_GlcNAc_9 N-acetyl-glucosaminyl_transferase_V
N-glycan_alpha_man_4
GNT-V
attaches GlcNAc at position 6
UDP-N-acetyl-D-glucosamine + alpha-D-Mannosyl-1,3-(R1)-beta-D-mannosyl-R2
<=>
UDP + N-Acetyl-$beta-D-glucosaminyl-1,2-alpha-D-mannosyl-1,3-(R1)-beta-D-mannosyl-$R2
UDP-N-acetyl-D-glucosamine + G00020 <=> UDP + G00021
26

27. A little bit about semantic metadata extractions and
annotations

28. Metadata Creation
Nexis Digital Videos
UPI
AP ... ...
Feeds/ Data Stores
Documents
WWW, Enterprise Digital Maps
Repositories
...
Digital Images Digital Audios
Create/extract as much (semantics)
metadata automatically as possible;
Use ontlogies to improve and enhance EXTRACTORS
extraction
METADATA
28

29. Automatic Semantic Metadata
Extraction/Annotation
29

30. Significant presence
• Life Science (biomedical)
• Health Care (clinical)
• Defense & Intelligence
• Web

31. 48th ACM Southeast Conference. ACMSE 2010.
Oxford, Mississippi. April 15-17, 2010.
Semantic Web in Action
Financial Services
Risk Management

32. Semagix Freedom Architecture
(a platform for building ontology-driven information system)
Knowledge Knowledge
Semantic Enhancement Server Agents Sources
KS
Automatic Entity Extraction,
Classification Enhanced KA
Metadata, Ontology KS
KA
KS
Content Content
Sources Agents
Metabase KA KS
Databases
CA
XML/Feeds
Websites CA
Metadata Metadata Semantic Query Server
Email adapter adapter
Ontology and Metabase
Existing Applications Main Memory Index
CA
Reports
Documents
ECM CRM EIP
© Semagix, Inc.

33. Global Bank 6/3/201
33
0
• Aim
• Legislation (PATRIOT ACT) requires banks to identify ‘who’ they
are doing business with
• Problem
• Volume of internal and external data needed to be accessed
• Complex name matching and disambiguation criteria
• Requirement to ‘risk score’ certain attributes of this data
• Approach
• Creation of a ‘risk ontology’ populated from trusted sources
(OFAC etc);
Sophisticated entity disambiguation
• Semantic querying, Rules specification & processing
• Solution
• Rapid and accurate KYC checks
• Risk scoring of relationships allowing for prioritisation of results
• Full visibility of sources and trustworthiness
2004 SEMAGIX All rights
reserved.

34. The Process
Ahmed Yaseer:
• Appears on Watchlist
‘FBI’
Watch list Organization
• Works for Company
‘WorldCom’
Hamas
FBI Watchlist • Member of
member of organization organization ‘Hamas’
appears on Watchlist
Ahmed Yaseer
works for Company
WorldCom
Company
2004 SEMAGIX All rights
reserved.

35. Global Investment Bank
Law Public World Wide BLOGS,
Watch Lists Enforcement Regulators Records Web content RSS
Semi-structured Government Data Un-structure text, Semi-structured Data
Establishing
New Account
User will be able to navigate
the ontology using a number
of different interfaces
Scores the entity
based on the
content and entity
relationships
Example of
Fraud Prevention
application used in
financial services
2004 SEMAGIX All rights

36. Equity Research Dashboard
Equity Research Dashboard with Blended Semantic Querying and Browsing
Automatic
3rd party Focused
content relevant
integration content
organized
by topic
(semantic
categorization)
Related relevant
content not
explicitly asked for
(semantic
associations)
Automatic Content
Aggregation
from multiple
Competitive content providers
research and feeds
inferred
automatically

37. 48th ACM Southeast Conference. ACMSE 2010.
Oxford, Mississippi. April 15-17, 2010.
Semantic Web in Action
Defense & Intelligence

38. An Ontological Approach to
Assessing IC Need to Know
Sponsored by ARDA
Work performed at LSDIS Lab, Univ. of Georgia
March2005

39. Security and Terrorism Part of SWETO Ontology
6/21/2004

40. Schematic of Ontological Approach to the Legitimate Access Problem
Semagix Freedom
Semagix Freedom
6/21/2004

41. Graph-based creation:
A Context of Investigation
26,489 entities
34,513 (explicit) relationships
Add relationship
to context
6/21/2004

42. Show me the stuff …
See demonstration at:
http://knoesis.org/library/demos
6/21/2004

43. 48th ACM Southeast Conference. ACMSE 2010.
Oxford, Mississippi. April 15-17, 2010.
Semantic Web in Action
Supporting Clinical Decision Making

44. Clinical Decision Making
• Status: In use today
• Where: Athens Heart Center
• What: Use of Semantic Web technologies for
clinical decision support

45. Operational Since January 2006

46. Active Semantic Electronic Medical
Records (ASEMR)
Goals:
• Increase efficiency with decision support
•formulary, billing, reimbursement
• real time chart completion
• automated linking with billing
• Reduce Errors, Improve Patient Satisfaction & Reporting
•drug interactions, allergy, insurance
• Improve Profitability
Technologies:
• Ontologies, semantic annotations & rules
• Service Oriented Architecture
Thanks -- Dr. Agrawal, Dr. Wingeth, and others. ISWC2006 paper

47. ASEMR - Demonstration
See demonstration at:
http://knoesis.org/library/demos

48. ASMER Efficiency
Chart Completion before the preliminary deployment
600
500
400
Charts
Same Day
300
Back Log
200
100
0 Chart Completion after the preliminary deployment
Se 4
5
04
05
04
05
04
05
04
04
l0
l0
n
n
ay
ay
pt
ar
ar
ov
Ju
Ju
700
Ja
Ja
M
M
M
M
N
600
500 Month/Year
Charts
400 Same Day
300 Back Log
200
100
0
Sept Nov 05 Jan 06 Mar 06
05
Month/Year

49. 48th ACM Southeast Conference. ACMSE 2010.
Oxford, Mississippi. April 15-17, 2010.
Scooner: Semantic Browser
A tool for knowledge discovery with
examples from Scientific Literature

50. OVERVIEW
1. Novel Information Exploration Paradigm
 Text Exploration on the context of relationships
 Not hyperlinks
2. Demonstrate use of background knowledge
 Named Entities, Relationships
3. Prototype Implementation
 Semantic annotations for navigation
4. Aggregation Utilities
 Saving, bookmarking, publishing etc
50

51. WHY SCOONER?
 Query Reformulations
 Impatient users
 Recognition over Recall
 Constrained navigation
 Hyperlink dependent - apriori
Fuzzy User Interests
 Haiti Earthquake – Recovery, Relief, Political Climate, Crime
Current approaches are not as effective for
Exploratory Search (Search-and-Sift)
Amit P. Sheth, Cartic Ramakrishnan: Relationship Web: Blazing Semantic Trails between Web Resources. IEEE Internet
Computing 11(4): 77-81 (2007)

52. MOTIVATION
 Users are information seekers
Information is embedded in documents
 A priori hyperlink dependent
 Semantic Web Standards
 Entity Identification (Semantic Annotations)
 Relationshipand Triple Identification
 Explore documents/information via relationships
52

53. Use Case Scenario
Search Phrase: Magnesium
53

54. Use Case Scenario
54

55. Use Case Scenario
55

56. SUMMARY
 Novel Information Exploration Paradigm
 Semantic Browser support Contextual Navigation
 Identify Named Entities and Relationships
 Provide Semantic Annotations
 Utilities for Aggregation
 Semantic Trails to Knowledge Discovery
See demonstration at:
http://knoesis.org/library/demos
56

57. 48th ACM Southeast Conference. ACMSE 2010.
Oxford, Mississippi. April 15-17, 2010.
Semantic Sensor Web
Kno.e.sis Center
Wright State University
http://knoesis.org/projects/sensorweb

58. Semantic Sensor Web
Sensors are now ubiquitous,
and constantly generating observations about our world

59. Semantic Sensor Web
However, these systems are often stovepiped,
with strong tie between sensor network and application

60. Semantic Sensor Web
We want to set this data free

61. Semantic Sensor Web
With freedom comes new responsibilities ….

62. Semantic Sensor Web
(1) How to discover, access and search the data?
Web Services
- OGC Sensor Web Enablement (SWE)

63. Semantic Sensor Web
(2) How to integrate this data together,
when it comes from many different sources?
Shared knowledge models, or Ontologies
- syntactic models – XML (SWE)
- semantic models – OWL/RDF (W3C SSN-XG)

64. Semantic Sensor Web
Sensor Observation Ontology

66. Semantic Sensor Web
The SSN-XG Deliverables
• Ontology for semantically describing sensors
• Illustrate the relationship to OGC Sensor Web Enablement standards
• Semantic annotation of OGC Sensor Web Enablement standards

67. Semantic Sensor Web
Linked Open Data: a community-led effort to create openly accessible, and interlinked,
semantic (RDF) data on the Web.

68. Semantic Sensor Web
Sensors Dataset
• RDF descriptions of ~20,000 weather stations in the United States.
• Observation dataset linked to sensors descriptions.
• Sensors link to locations in Geonames (in LOD) that are nearby.
near
weather station

69. Observations Dataset
• RDF descriptions of hurricane and blizzard observations in the United States.
• The data originated at MesoWest (University of Utah)
• Observation types: temperature, visibility, precipitation, pressure, wind speed,
humidity, etc.
69

70. Linked Datasets
procedure location
Observation Location KB
Sensor KB
KB (Geonames)
Example
procedure location
720F Thermometer Dayton Airport
• ~2 billion triples • 20,000+ systems • 230,000+ locations
• MesoWest • MesoWest • Geonames
• Dynamic • ~Static • ~Static
70

71. Semantic Sensor Web
(3) How to make numerical sensor data meaningful
to web applications and naïve users?
Symbols more meaningful than numbers
- active perception

72. Active Perception:
• is an iterative, bi-directional feedback loop for collecting and explaining
sensor data
Explanation
Observation Expectation
Attention
72

73. Overall Architecture
73

74. DEMOS
Semantic Sensor Web
Demos at
http://wiki.knoesis.org/index.php/SSW
•Sensor Discovery On Linked Data
•Semantic Sensor Observation Service (MesoWest)
•Video on the Semantic Sensor Web
74

75. Ohio Center of Excellence
Knowledge-Enabled Computing
(Kno.e.sis)
SEMANTIC SOCIAL WEB

76. Everyone Wants to talk
…and be heard!
Hundreds and thousands of tweets, facebook posts, blogs
about a single event, multiple narratives, strong opinions,
breaking news..
76

77. TWITRIS : Twitter+Tetris
• Our attempt to help you keep up with citizen
observations on Twitter
– WHAT are people saying, WHEN, from WHERE
• Puts citizen reports in context for you by
overlaying it with news, wikipedia articles!
77

78. See demo and live system at
http://twitris.knoesis.org
78

79. How we work with industry
Interns, Training
SBIR/STTR
Joint contracts
Tech Transfer/licensing
79

80. More of Web 3.0
Semantics enhanced
Web, Social, Sensor and Services
Computing, and their
applications to
health care, life sciences, DoD,
IT/Data management, … at
http://knoesis.org