Slides discussing how semantics empowers community participation. Presented at STI Innsbruck Summit at lake Garda, June 27, 2012. Credits to my present and past employers: STI Innsbruck, FTW, University of Surrey.

1. Empowering user participation with
converged semantic services
Dr. Anna Fensel
27 June, 2012; STI Innsbruck Summit
©www.sti-innsbruck.at INNSBRUCK www.sti-innsbruck.at
Copyright 2012 STI

2. Outline
• Motivation for the Research Agenda
• About Participation
• About Convergent Services
• How Semantics and Convergence Foster Participation - Examples
– User Generated Mobile Services
– Smart Buildings and Grid
• Conclusions
Disclaimer:
The contents of this presentation are not
necessarily reflecting opinions of any of my
current, past or future employers.
www.sti-innsbruck.at 2

3. Outline
• Motivation for the Research Agenda
• About Participation
• About Convergent Services
• How Semantics and Convergence Foster Participation - Examples
– User Generated Mobile Services
– Smart Buildings and Grid
• Conclusions
Disclaimer:
The contents of this presentation are not
necessarily reflecting opinions of any of my
current, past or future employers.
www.sti-innsbruck.at 3

4. Motivation
Converged Semantic Services
For Empowering Participation
Aims:
• Enabling efficient participation vs. current social network silos and groups
– More possible roles for an individual
– More roles at a time for an individual
– More matching and satisfying roles for an individual
=> Motivation, added value and revenue increase
Technologically that means:
• Benefiting from data and services reuse at the maximum
• Enabling participators to establish added value new and converged
services on top of the data
– commercially re-applying them across platforms
=>There is a need to „understand“ and interlink content and objects coming
from heterogeneous numerous sources
www.sti-innsbruck.at

5. Motivation: From Heterogeneity to Convergence
“Service Science is just ___<name your discipline>____”
Service
Operations
Marketing
Management
Quality
Supply Chain
General Human Factors
Systems A Service Design
Theory System is Innovation
Complex Engineering
Systems
Computing
OR/IE Economics
MS Arts
Science
Information
Science
(i-schools)
CS/AI Economics & Law MIS Anthropology Organization
Multiagent Systems
www.sti-innsbruck.at
Game Theory & Psychology Theory 5

6. Positive Example from the Web: Open Graph Protocol
• Open Graph Protocol - enables any web page
to become a rich object in a social graph.
• Developed and used by Facebook
– e.g. external “Like button”
• Keywords: semantics (RDFa) and simplicity
• Can be referred as a “converged service”
www.sti-innsbruck.at

7. Outline
• Motivation for the Research Agenda
• About Participation
• About Convergent Services
• How Semantics and Convergence Foster Participation - Examples
– User Generated Mobile Services
– Smart Buildings and Grid
• Conclusions
Disclaimer:
The contents of this presentation are not
necessarily reflecting opinions of any of my
current, past or future employers.
www.sti-innsbruck.at 7

8. Young People„s Participation
• Psychology perspective:
„Child-Adult“
www.sti-innsbruck.at 8

9. Participation in Terms of Social Media
www.sti-innsbruck.at 9

10. 90-9-1 Rule for Participation Inequality
• Web use follows a Zipf distribution
• Also applicable to social media
• Also to working groups?
• Is that wrong?
– In some cases (e.g. inappropriate
match), yes.
– In many cases (e.g.
dissemination effect), no.
Jakob Nielsen,
http://www.useit.com/alertbox/participation_inequality.html
www.sti-innsbruck.at 10

11. Participation is Linked to Value
• Participation level relates to the value one gets from participation
• Participation also has a value in itself
Lurkers‘
Perspective
www.sti-innsbruck.at 11

12. Participation is Linked to a Role
Äns - 1 person: gatherer or hunter
Zwo - 2 persons: gatherer and hunter?
– Problem with the role choice starts from
the moment where there is a choice.
Having more persons implies:
• fine-grained devision of labor and
service economy,
• community as a regulator on which
roles are appropriate and which not,
as well as their values.
www.sti-innsbruck.at 12

13. Impact of Roles/Relations and their Weights on
Ontology Evolution Dynamics
• People and relations are inherently associated with / connected to / can
be decomposed into concepts and properties.
– See also: Peter Mika, „Ontologies are Us: A Unified Model of Social Networks and
Semantics”. International Semantic Web Conference 2005: 522-536.
• Changing the roles drive social, ontology and market evolution.
• One of the important drive factors are the quantity of concepts/people
relating to another concept/person via a specific property (hub vs.
stub), e.g. a property spouse is stronger than friend. Thus, the networks
are self-restructuring depending on the roles and weights put on them.
– See also: Zhdanova, A.V., Predoiu, L., Pellegrini, T., Fensel, D. "A Social Networking
Model of a Web Community". In Proceedings of the 10th International Symposium on
Social Communication, 22-26 January 2007, Santiago de Cuba, Cuba, ISBN: 959-
7174-08-1, pp. 537-541 (2007).
www.sti-innsbruck.at 13

14. Participation is Linked to a Role
• (Semantic) content
creation, and thus,
participation, is
driven by a role
– „Role“ is a steadier
form of insentives
(as e.g. reputation vs.
yield management)
• Hence, in
participation, people/
companies are optimi-
zing their roles by
taking ones and drop- PICTURE FROM: Zhdanova, A.V., Shvaiko, P. "Community-Driven Ontology Matching".
ping others In Proceedings of the 3rd European Semantic Web Conference (ESWC'2006),
11-14 June 2006, Budva, Montenegro, Springer-Verlag, LNCS 4011, pp. 34-49 (2006).
– Limited time and money
• Converged semantic services are to enable users performing in roles
unavailable to them before & changing the roles faster when needed.
www.sti-innsbruck.at 14

15. Outline
• Motivation for the Research Agenda
• About Participation
• About Convergent Services
• How Semantics and Convergence Foster Participation - Examples
– User Generated Mobile Services
– Smart Buildings and Grid
• Conclusions
Disclaimer:
The contents of this presentation are not
necessarily reflecting opinions of any of my
current, past or future employers.
www.sti-innsbruck.at 15

16. Communication Media Development
• Technology Development List
• The printed newspaper (1436)
• The 'Silent Pictures (1888)
• Radio (1896)
• Telephone (1876)
Communication technollgies are a
• Silicon Chip (1896) medium to participation.
• Cellphone (1973)
• Digital Camera (1981) To communicate though does not
automatically imply to particiapte.
• PDA (1981)
• The Internet (1983)
• Email (1965)
• Wikis (1995)
• Facebook (2004)
• Twitter (2006)
– BG Creative, A Brief History of Media Convergence: 4000 BC to 2009 AD, August 20, 2009
www.sti-innsbruck.at 16

17. Convergence
• “Telecommunications convergence, network convergence or
simply convergence are broad terms used to describe emerging
telecommunications technologies, and network architecture used to
migrate multiple communications services into a single network.[1]
Specifically this involves the converging of previously distinct media
such as telephony and data communications into common interfaces on
single devices.”
– Wikipedia
• Convergent technologies/services include:
– IP Multimedia Subsystem
– Session Initiation Protocol
– IPTV
– Voice over IP
– Voice call continuity
– Digital video broadcasting - handheld
www.sti-innsbruck.at 17

18. Link to Value - Mobile Operators‘ Use Case -
Business Potential of Openness and Collaboration
www.sti-innsbruck.at 18

19. Increasing Participation –
From Static Social Network Silos to Pervasive Social Spaces
...where everyone
benefits.
Semantic
technologies
take you there.
www.sti-innsbruck.at

20. Semantics in One Slide
• Larger markets are to come
2010 • Linked Open Data cloud
counts 25 billion triples
• Open government initiatives
• BBC, Facebook, Google,
Yahoo, etc. use semantics
2008 • SPARQL becomes W3C
recommendation
• Life science and other Source: Open Knowledge Foundation
scientific communities use ontologies
2004 • RDF, OWL become W3C
recommedations
• Research field on ontologies
and semantics appears
2001 • Term „Semantic Web“ has been „seeded“, Scientific American article,
Tim Berners-Lee et al.
www.sti-innsbruck.at

21. From Semantic Web to Semantic World:
Data Challenges
• Large volumes of raw data to smaller volumes of
„processed“ data
– Streaming, new data acquisition infrastructures
– Data modeling, mining, analysis, processing, distribution
– Complex event processing (e.g. in-house behaviour identification)
• Data which is neither „free“ nor „open“
– How to store, discover and link it
– How to sell it
– How to define and communicate its quality / provenance
– How to get the stekeholders in the game, create marketplaces
• Establishment of radically new B2B and B2C services
– „Tomorrow, your carton of milk will be on the Internet“ – J. da Silva,
referring to Internet of Things
– But how would the services look like?
www.sti-innsbruck.at

22. Outline
• Motivation for the Research Agenda
• About Participation
• About Convergent Services
• How Semantics and Convergence Foster Participation - Examples
– User Generated Mobile Services
– Smart Buildings and Grid
• Conclusions
Disclaimer:
The contents of this presentation are not
necessarily reflecting opinions of any of my
current, past or future employers.
www.sti-innsbruck.at 22

23. Mobile Ontology
Villalonga, C., Strohbach, M., Snoeck, N., Sutterer, M., Belaunde, M., Kovacs, E., Zhdanova,
A.V., Goix, L.W., Droegehorn, O. "Mobile Ontology: Towards a Standardized Semantic Model for the Mobile
Domain". In Proceedings of the 1st International Workshop on Telecom Service Oriented Architectures
(TSOA 2007) at the 5th International Conference on Service-Oriented Computing, 17 September 2007,
Vienna, Austria (2007).
www.sti-innsbruck.at 23

24. Participation in Different Roles in Ontology Construction
Actual Split – SPICE Integrated Project Example
Zhdanova, A.V., Li, N., Moessner, K. “Semantic Web in Ubiquitous
Mobile Communications”. The Semantic Web for Knowledge and Data Management
(Ed.: Ma, Z.), IGI Global (August 2008).
www.sti-innsbruck.at 24

25. Participation in Collaborative Ontology
Construction for „Newbies“ - Challenges
• Educational: people with no/little knowledge on ontologies require at
least an introduction to the field;
• Methodology: yet no widely accepted or best practice solutions on how
to acquire ontologies from people in such a setting;
• Basic technology: current ontology language standards (such as
OWL) cause confusion and awkward modelling solutions;
• Tool support: better tools for ontology construction process
coordination, documentation would help to avoid ad-hoc solutions and
manual work.
Zhdanova, A.V., Li, N., Moessner, K. “Semantic Web in Ubiquitous
Mobile Communications”. The Semantic Web for Knowledge and Data Management
(Ed.: Ma, Z.), IGI Global (August 2008).
www.sti-innsbruck.at 25

26. Microservices Scenario: Traffic Jam Killer
Motivation:
Share knowledge
about the fluidity of
the traffic and
presence of mobile
radars with friends.
26
www.sti-innsbruck.at

27. m:Ciudad – Underlying Magic
Service User
NET warehouse management
Knowledge
WORK warehouse
Execution Environment
Operating System
Services
TER Service
MI Capabilities
NAL
Capabilities
Management
27
www.sti-innsbruck.at

28. Architecture - detailed building blocks
mCiudad GUI / launcher m:Ciudad
Framework
SSEs
Authoring / SCK Authorization, Profile mgr
Accounting
access control
User & Group
Hosted
mgmt
SEE backend communication
Notif. mgmt
view capabilities
Capabilities Mgr Service publisher
Service Exec Service provider enablers
Embedded capab. Remote caps Env Metadata creation
Service storage
(templates, KW
service Serv Metadata ontology
instances lifecycle SW SSEs)
My Service & State Search engine Provider/service Ontology parsing
Registry Mgr client Matching table
persistent
DB
Rule & Policy Context &
controller profile Search engine
manager Service
availability
sensors Recommender tracker
gateways
Overlay network
www.sti-innsbruck.at 28

29. Microservice description language
Made of
SSE‟s Semantic Service
Semantic Description
Content
Description
Semantic Operational
Description Service Profile Description
Rendering
(UDL-SP) Description
SSE‟s
Capability Service Local Operational Searchable Service Service
Profile Content Meta- Metadata Metadata Logic Rendering
(UDL-CP) (UDL-CD) data (UDL-SL) (UDL-SR)
Service Backus-Naur
SSE Content Service & XML-Schema
Domain profile XML-Schema
Semantic Characterization Operational Characterization
of Service and Capabilities of Service
Legend
Ont. Instances Ontology XML doc
www.sti-innsbruck.at 29

30. Architecture of Knowledge Warehouse
www.sti-innsbruck.at 30

31. Service Creation Kit – First Mock-up and
Approaches: “Block-based” and “Question
Answering”
Version 1
• Visual C++, Windows
Mobile
• Goal: Study on Block
approach usability
Version 2
• Flash Lite, Windows
Mobile
• Goal: Wizzard approach,
Carroussel UI
31
31
www.sti-innsbruck.at

32. User Survey – Study Set Up
• Goal: improve understanding of users' needs, experiences, and
expectations on user-generated mobile services
– From a knowledge management point of view
• URI: http://survey.ftw.at/microservices
• 38 questions, incl. video demonstrations
• Distributed via professional and interest mailing lists, social
networks
• Answers being collected since June 2009
• Participants: 138 persons (52 fully completed)
• Plus several face-to-face usability tests with persons (to confirm
the findings)
Danado, J., Davies, M., Ricca, P., Fensel, A. "An Authoring Tool for User Generated Mobile Services".
In Proceedings of the 3rd Future Internet Symposium (FIS'10), 20-22 September 2010, Berlin, Germany;
Springer Verlag, LNCS 6369, pp. 118-127 (2010).
www.sti-innsbruck.at

33. User Survey – Need for Our Technology
• Ca. 2/3 of users feel the need to adapt services or apps they
use
• Ca. 1/3 of users feel the need to create their own services and
apps
User profile:
– Almost all between 20 and 50 years old, Europeans
– ca. 70% male, 30% female
– Majority is a researcher or engineer with a Master degree,
also large shares with a Bachelor or a PhD
– Daily average internet usage is 5 hours
– Half of the respondents access the internet via mobile
www.sti-innsbruck.at

34. Evaluations – Mobile Service Creation
• Customisation – drag&drop (matching blocks) – end-user programming
• Davies, M., Carrez, F., Heinilä, J., Fensel,
A., Narganes, M., Danado, J.
"m:Ciudad -- Enabling End-User Mobile
Service Creation", International Journal of
Pervasive Computing and Communications
Emerald Group Publishing,
Vol. 7 Iss: 4, pp. 384-414 (2011).
• Davies, M., Carrez, F., Urdiales, D.,
Fensel, A., Narganes, M., Danado, J.
"Defining User-Generated Services in a
Semantically-Enabled Mobile Platform".
In Proceedings of 12th International
Conference on Information Integration
and Web-based Applications &
Services (iiWAS2010), 8-10 November
2010, Paris, France, ACM (2010).
www.sti-innsbruck.at 34

35. Outline
• Motivation for the Research Agenda
• About Participation
• About Convergent Services
• How Semantics and Convergence Foster Participation - Examples
– User Generated Mobile Services
– Smart Buildings and Grid
• Conclusions
Disclaimer:
The contents of this presentation are not
necessarily reflecting opinions of any of my
current, past or future employers.
www.sti-innsbruck.at 35

36. Smart Grids - Technology Radar
empowering renewable
energy „prosumers“ consumer „manipulation“
raising consumer
demand-response management
Web-Grid awareness
convergence data-intensive
services
automatisation
Internet of Things
M2M services
energy control & monotoring
large-scale &
stream data processing
CIM, OPC & other models EU 2050 nearly-zero goal
(semantic) service smart metering
description, discovery, On market
composiion
Product concept
Applied Research
Relevance
Basic Research high
medium
low
www.sti-innsbruck.at 36

37. Market Situation
• Currently: fragmented offers, closed systems, low interoperability,
information difficult to find or combine
– Information Services
• E-Control, energy companies, energy consultancies
– Smart Metering & Home Automation
• Still waiting to come, trials, closed systems, services offered by energy
companies
• Closed sniffer systems (optical sensors, clams) , no closed loop – no impact
on energy management
• Specialized systems (security, heating), some open, most closed, some portal
based , not part of energy management
www.sti-innsbruck.at 37

38. Green Marketplace
• Information portal to disseminate in a personalized
manner the information about available products and
services
• RECOIL = Recommender Optimized via Identified Links for Renewable
Energy
• E‐commerce platform combining an advertisement
platform and an online shop, through which customers
can purchase
– Partner hardware Context
/ services
– Mobile apps and games Energy
Data
– Coupons for appliances Products
Portal
– Add-value info services
Consumer
Services
www.sti-innsbruck.at 38

39. Potential Customers / Partners &
Their Benefits
Customer Group Benefit
 Citizens and private households  Energy awareness and control -> cost
 Facility managers (private and public) and energy savings
 Construction companies and investors  Personalized recommendation of
products and services
 Large manufacturers of energy efficient  Targeted promotion of products,
appliances , home automation devices and winning of new customers
renewable energy equipment,  “Green” PR
 Small manufacturers of appliances and  Better benchmarking through
specialized devices such as for smart home consumption data and information
automation, retailers, especially those without about energy efficiency of business
strong Web presence, market holders processes
 Municipalities and their utility companies who  Advertisement of services, programs,
offer energy optimization services and energy solutions
consultancy agencies, ministries (energy,  Better customer management
environment, spatial planning)  Better in-sector awareness
 Energy supplier companies
 Tourism companies: hotels, tourism settlements
 Energy efficiency bodies
www.sti-innsbruck.at 39

40. Background
2 FFG COIN Projects (sesame-s.ftw.at)
• SESAME – Semantic Smart Metering,
Enablers for Energy Efficiency (9’09-11’10,
800k Euro)
– Prototype, proof of concepts, feasibility
study
• SESAME-S – Services for Energy
Efficiency (4’11-9’11, 770k Euro)
– setting up usable smart home
hardware, a portal and repository
– organizing a test installation in real
buildings: in a school (Kirchdorf,
Austria) and a factory (Chernogolovka,
Russia)
– developing specialized UIs and
designing mobile apps for the school
use case
• Consortium partner network of 6
organizations
www.sti-innsbruck.at 40

41. Data Acquisition
© FTW 2011
www.sti-innsbruck.at 41

42. Data Acquisition – Extended,
SESAME-S
© FTW 2011
www.sti-innsbruck.at 42

43. Data Acquisition
www.sti-innsbruck.at 43

44. Extension to More Buildings
• Research challenge: moving logics components, such as building
automation settings, user preferences.
www.sti-innsbruck.at 44

45. Many Stakeholders - Same Data
• Ministries (Ministry of Infrastructure and Energetics, Ministry of
Environment…)
• Provincial councils and centers
• Energy efficiency bodies
• Energy companies
• Municipalities
• Construction companies and Investors
• Home-automation market holders
• Home-appliance market holders
• Tourism companies: hotels, tourism settlements
• Telecommunication companies
• Cloud service providers
• …
© FTW 2011
www.sti-innsbruck.at 45

46. Smart Home End
User Services
© FTW 2011
www.sti-innsbruck.at 46

47. Energy Efficient Buildings –
User Trials
• Over 50 users were interviewed f2f plus over a 100 online
• Some outcomes
– „Saving costs“ is the strongest motivator, “reputation“ is the weakest
– Main system cost expectation is 200 Euro per installation, plus up to 5
Euro as a monthly fee, with energy savings of 20%
– Preference to delegate unobtrusive tasks (e.g. stand by device
management vs. lights control)
– Every 4th user will choose the „fanciest“ and not the „easiest to use“
interface
– 2/3rds of users are „absolutely sure“ or „sure“ they„d use such or a
similar system in the future
– 2/3rds of users would also share their home settings with „friends“
• Fensel, A., Tomic, S., Kumar, V., Stefanovic, M., Aleshin, S., Novikov, D. "SESAME-S: Semantic Smart Home System for
Energy Efficiency". In Proceedings of D-A-CH Energieinformatik 2012, 5-6 July 2012, Oldenburg, Germany.
• Schwanzer, M., Fensel, A. "Energy Consumption Information Services for Smart Home Inhabitants". In Proceedings of the
3rd Future Internet Symposium (FIS'10), 20-22 September 2010, Berlin, Germany; Springer Verlag, LNCS 6369, pp. 78-87.
www.sti-innsbruck.at

48. End User Attitudes
© FTW 2011
www.sti-innsbruck.at 48

49. End User Expectations
© FTW 2010
www.sti-innsbruck.at 49

50. Smart Home Installation
School, Kirchdorf - AT
• Several Smart Meters
• Sensors (e.g. light, temperature,
humidity)
• Smart plugs, for individual sockets
• Shutdown services for PCs
• User interfaces and apps: Web,
tablet, smartphone (Android)
Factory, Chernogolovka - RU
• Heating system regulation and
monitoring extension
www.sti-innsbruck.at 50

51. Services Addressing Users
@ School
• Energy awareness,
monitoring
• Remote control - manual and
programmed - e.g. scheduled
activities and triggering rules
• How do we get the users?
– By having workshops with pupils:
introduction to energy efficiency,
building analysis, explaining the
system and services
www.sti-innsbruck.at 51

52. Demand Management
@ Smart Building
Millions of triples collected
in the semantic repository
www.sti-innsbruck.at 52

53. How Green I Am
@ www.alphaverda.com
www.sti-innsbruck.at 53

54. Outline
• Motivation for the Research Agenda
• About Participation
• About Convergent Services
• How Semantics and Convergence Foster Participation - Examples
– User Generated Mobile Services
– Smart Buildings and Grid
• Conclusions
Disclaimer:
The contents of this presentation are not
necessarily reflecting opinions of any of my
current, past or future employers.
www.sti-innsbruck.at 54

55. Conclusions
• Semantic technology as an enabler for the individuals and
organisations to participate productively
– By getting new roles.
– By changing existing roles easier.
• Examples have been shown:
– Mobile prosumers creating mobile services
– Energy prosumer in smart buildings
Possible future research aspects include data analytics e.g. for:
• Scenarios involving heterogeneous multiple stakeholders.
• Changing/steering behavior, engagement of users/customers.
• Enabling participation vs. yield management / resilience.
– “Resilience is the ability to provide and maintain an acceptable level of service in the
face of faults and challenges to normal operation.”, “A superset of survivability.” -
Wikipedia
www.sti-innsbruck.at 55

56. Questions?
www.sti-innsbruck.at 56