"LiquidPub: Services at Service of Science". Invited talks of Fabio Casati at the European Conference on Web Services 2009 and in the Politechnico di Milano

1. LiquidPub: Services at the service of
science
Marcos Baez, Fabio Casati , and the LiquidPub
team
Department of Information Engineering and Computer Science
University of Trento
ECOWS 2009 – Eindhoven, The Netherlands – 10 Nov 2009

2. In a nutshell: the goal
Capture the lessons learned and opportunities provided by the Web
and open source, agile development to develop
concepts, models, metrics, and tools for an efficient (for
people), effective (for science), and sustainable (for publishers and
the community) way of creating, disseminating, evaluating, and
consuming scientific knowledge. Understand what’s good for
science, and make it happen
project.liquidpub.org
•Publish and perish: why the current publication and review model is killing
research and wasting your money (ACM Ubiquity 8(3), Feb 2007), and
•Liquid Publications: Scientific Publications meet the Web
With Fausto Giunchiglia and Maurizio Marchese.

3. Objective
Understand Improve
 Dissemination models and  Better ways to do the
overhead same things
 Evaluation processes, and  Different things
quality/impact of research
(people, papers, projects)
 Principles
 Peer review and
 Models
innovation
 IT services
 Scientific communities
 Funding models and
processes
project.liquidpub.org 3

4. Understanding
project.liquidpub.org 4

5. PR: Initial Goals
 Understand how well peer
review works
 Understand how to
improve the process
 Metrics + Analysis
 (refer to liquid doc)
 Focus only on
“Not everything that can be counted counts,
gatekeeping aspect and not everything that counts can
be counted.” -- Albert Einstein

6. Comparing rankings
28 33
17 89
2 2
45 17
67 67
.. ..
.. 28
89 ..
33 ..
.. 45

7. Comparing rankings
T=3
N=10
Divergence: Div (t,N)
Kendall τ

8. 8

9. Results:
peer review ranking vs. citation
count
Divergence
Div
Normalized t
9

11. Quality-related Metrics: Statistics
Distribution of marks (integer marks)
0.18
0.16
0.14
0.12
Probability
0.1
0.08
0.06
0.04
0.02
0
0 1 2 3 4 5 6 7 8 9 10
Marks
Also: (-3,3) very different from (1,7) 11

12. Disagreement vs number of
reviews
2.5
2
1.5
1
R1 R2 R3+

13. Fairness
 Definition: A review process is fair if and only of
the acceptance of a contribution does not
depend on the particular set of PC members
that reviews it
 The key is in the assignment of a paper to
reviewers: A paper assignment is unfair if the
specific assignment influences (makes more
predictable) the fate of the paper.

14. The road ahead

15. About citations…
project.liquidpub.org

16. Mikalai Krapivin, Maurizio Marchese, Fabio Casati: Exploring and
Understanding Scientific Metrics in Citation Networks. Complex 2009
16

17. Positional effect on citation and readership
17
in arXiv, by Haque and Ginsparg

18. project.liquidpub.org
Fabio Casati - ECOWS 2009
Positional effect on citation and readership
18
in arXiv, by Haque and Ginsparg

19. Improving
Principles
Models
IT Services
project.liquidpub.org Fabio Casati - ECOWS 2009 19

20. Principles and objectives
 Everything counts!! (not just papers, not just ―innovations‖)
 Minimal dissemination overhead
 Early sharing
 Early feedback/interaction
 Find diversity
 Interestingness and sharing as measures of reputation
 No gatekeeping. Use the filtering power of the community
 We are not necessarily right!! Let the community select the
principles and models

21. Model:
Liquid Journals

22. Liquid Journals
project.liquidpub.org Fabio Casati - ECOWS 2009 22

23. project.liquidpub.org Fabio Casati - ECOWS 2009 23

24. Internet
Data
project.liquidpub.org Fabio Casati - ECOWS 2009 24

25. Data
project.liquidpub.org Fabio Casati - ECOWS 2009 25

26. project.liquidpub.org Fabio Casati - ECOWS 2009 26

27. 3. What you can do with it
project.liquidpub.org Fabio Casati - ECOWS 2009 27

28. 2. How it gets filled…
 Can be edited, but can be (semi-)automated
 Topics and tags (+ tags by specific people)
 Subscriptions/preferences
 Journals—editing power of the community!!
 Authors
 Wisdom of people you trust (as editor)
 Wisdom of the crowd (if any)
 Diversity!!!

29. Results
 Everybody becomes a journal editor (selfishly)
 No gatekeeping, quality enforced by the community
(gatekeeping is the noise)
 You get things you want to read (or you want your group
to read)
 You get diversity
 You leverage the selection work of your friends
 Measure interestingness and quality by sharing
 Reduce (optimize) dissemination overhead and
encourage early sharing
project.liquidpub.org Fabio Casati - ECOWS 2009 29

30. IT Services: LiquidPub
infrastructure
project.liquidpub.org Fabio Casati - ECOWS 2009 30

31. Engineer a system that
 Accesses heterogeneous src w/ no API
 Is modular so that functions can be reused
 In a world where modularity had little success
 Facilitates the creation of arbitrary dissemination and
evaluation models (by non-programmers)
 Provides commonly needed research services
 Implements liquid journals (agile – our main macro-
story)
 Simple and lightweight
project.liquidpub.org Fabio Casati - ECOWS 2009 31

32. Liquid Journals dashboard
Liquid journals API
Search Personalized
Sharing
computing tagging
Diversity Caching
Reputation
Cached metadata
Search by
topic Crawling
Access
Disambigua
tion

33. UI
A
Research services P
I Crawling
Karaku client-code Cache mgmt
Subscriptions/stream
Disambiguation
Personalized tagging
REST API Liquidity
Karaku Rendering
REST API Cached SKOs
ResMan
Adapters
project.liquidpub.org Fabio Casati - ECOWS 2009 REST API
REST API 33

34. Karaku: Uniform access
Repositories Documents (CRUD)
Resource 1..N 1..N 1..N 1..N
Resource type Action type
Manager type
Abstraction
Resource 1..1 1..N 1..1 1..N
Resource Actions
Manager
Google Docs Documents in Google Docs GDocs-specific Operations
 Handling semantically-equivalent resources with the same set
of actions and interface
 Interoperability at the type level
 The semantics are defined by the type

35. Vertical services
Liquid journals Liquid books …
programming
Collaborative
Gelee
Search computing
Mashart
Community
TopicSearch Reseval
discovery
project.liquidpub.org Fabio Casati - ECOWS 2009 35

36. [Insert header here] [Insert footer]

37. Gelee
Paper at ICDE 09. Video avail online and demo at icsoc
[Insert header here] [Insert footer]

38. Mashart
Paper at ER‘09, demo at icsoc ‗09
project.liquidpub.org 38

39. RESEVAL http://demo.liquidpub.org/reseval/ complex’09
project.liquidpub.org 39

40. Group Management

41. Group Management
 Compare the metrics
of people in
group.
 Total citations
 Total publications
 H-index
 G-index
 Noise ratio

42. comparison
 ―Search comparison‖ shows all the comparison made.
We can see and recompute them (for updated data).

43. Communities
project.liquidpub.org 43

44. Overview of the Community Network
SOFTWARE ENGINEERING(kbse,icse)
DIST. SYSTEM/COMPILER(ipps,iccS) APPLIED COMPUTING/CRYPTO(sac,compsac)
TELETEACHING/HUM_INT (chi,hicss)
TELECOM (icc,globecom)
HUMMAN –COMP INTER(icchp,hci)
GENETIC AND EVO ALG(cec,gecco)
AI/DB (icai,aaai)
ROBOTIC/M.MEDIA (icra,icpr)

45. Community Comparison and
normalization
50
45
40
35
30
[icc- globecom]
h-index
[ijcai- aaai]
25
[icra- icpr]
20 [iscas- date]
[kbse- icse]
15
10
5
0
1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 41 43 45 47 49 51 53 55 57 59 61 Author

46. Hi Alex!
Create Journal
U You haven’t created any LJ yet.
Start sharing scientific content!

47. Hi Alex!
Untitled
1 General Info
Journal Name Peer-review journal
Sources to
2 overlay
Description Journal that groups the relevant
Information about peer-review
3 Process
Editorial board Joe
Alex Joe
Cancel Next

48. Hi Alex!
Peer-review journal To be included in your
1 General Info journal
I WANT
Sources to
peer review Test • papers,
2 • blogs,
overlay
• datasets
TALKING ABOUT
3 Process Contributions Sources peer-review
papers Springer FROM
blogs Arxiv Springer
dataset Scopus Arxiv
wikis Scholar UPDATE RATE
weekly
Edit relations | Advanced filters
Rank by citations
Peer review, an overview [none] Springer
Peter, Pablo, Pedro (0) LJ references
The world of scientif ic publications has been largely… (4) citations
Peer review, scientific publications
Rethinking peer review [Carl’s LJ] Arxiv
Joe (1) LJ references
Exploring new ideas f or peer review in physics (3) citations
Peer review, open access
Anonymized review data [Liquidpub LJ] Arxiv
Juan, Alejandro (1) LJ references
Review data f rom 10 conf erences in biology (1) citations
Peer review,
Cancel Next

49. Hi Alex!
Peer-review journal
1 General Info
Sources to Traditional journal process
2 overlay Model f or traditional journals
3 Process Publication via voting
“Democratic model” f or agreement
Total agreement
Full agreement model
Cancel Done

50. Hi Alex!
Peer-review journal
Create Issue Update Now Latests activities
Journal Created
Home Springer
11 Apr 9.00am
Peer review, an overview [none]
Content
Issues Peter, Pablo, Pedro (0) LJ references
The world of scientif ic publications has … (4) citations
Stats Latests subscribers
Settings Peer review, scientific publications,
[no subscribers yet]
Rethinking peer review [Carl’s LJ] Arxiv
Joe (1) LJ references
General Info (3) citations
Exploring new ideas f or peer review in …
Created on 11 Apr 9.00am
Owner Alex Peer review, open access
Subscribers 0
Anonymized review data [Liquidpub LJ] Arxiv
Editors Juan, Alejandro (1) LJ references
Review data f rom 10 conf erences in …
(1) citations
Alex
Peer review,
Joe
Settings
Issues
[no issues yet]
U We are still learning from you.
Soon you’ll get our suggestions :)
Disable
Settings
U No suggestions from your friends Disable

51. Hi Alex!
Peer-review journal
Create Issue Add Contribution Citations Latests activities
Joe joined the LJ
Home Springer
11 Apr 9.02am
Peer review, an overview [none]
Content Journal Created by Alex
Issues Peter, Pablo, Pedro (0) LJ references 11 Apr 9.00am
The world of scientif ic publications has … (4) citations Latests subscribers
Stats
Settings Peer review, scientific publications,
[no subscribers yet]
Rethinking peer review [Carl’s LJ] Arxiv
Editors Joe (1) LJ references
Exploring new ideas f or peer review in … (3) citations Popular Tags
Alex
Peer review, open access Peer review
Scientific publications
Joe Anonymized review data [Liquidpub LJ] Arxiv Open access
Juan, Alejandro (1) LJ references
Review data f rom 10 conf erences in …
(1) citations
Issues
Peer review,
[no issues yet]

52. Hi Alex!
Peer-review journal » Issue on new review models
Create Issue Published Latests activities
“Issue on new review models” moved to
Home Springer
the Publication phase, by Joe
Peer review, an overview [none] 11 Apr 9.09am
Content
Issues Peter, Pablo, Pedro (0) LJ references (1) Contributions added to the Issue on
The world of scientif ic publications has … (4) citations new review models” by Alex
Stats 11 Apr 9.08am
Settings Peer review, scientific publications, by Alex “Issue on new review models” moved to
the Selection phase, by Alex
Rethinking peer review [Carl’s LJ] Arxiv 11 Apr 9.07am
Editors Joe (1) LJ references Latests subscribers
Exploring new ideas f or peer review in … (3) citations
Alex
Peer review, open access by Alex [no subscribers yet]
Joe Benefits of open peer review [none] Arxiv
Charles Popular Tags
This paper illustrates the benef its of …
(0) LJ references
(2) citations Peer review
Issues
Peer review, open access Scientific publications
Issue on new by Joe Open access
review models
(4)
Peer review, past and future [none]
Ronald, James Scopus (0) LJ references
This paper illustrates the benef its of … (1) citations
Peer review by Alex

53. Take-Home Message
 Flaws of current practices (or, lack of evidence
that they work as expected)
 Research services for novel dissemination model
 Principles, models, composable IT services in a
restricted domain
 Liquid journals: Interestingness, reward
innovation, sharing, everything contributes, use
the filtering power of the community
project.liquidpub.org Fabio Casati - ECOWS 2009 53

54.  Google liquidpub-announce and subscribe if
you are interested (VERY low traffic)
 First release at the end of the month
project.liquidpub.org Fabio Casati - ECOWS 2009 54

55. How to collaborate
 Liquid journals server
 http://demo.liquidpub.org/liquidjournals/
 Liquid journals dashboard
 http://demo.liquidpub.org/liquidjournals/web/work
space/
 Liquid journals SVN
 https://dev.liquidpub.org/svn/liquidpub/prototype
/liquidjournals/

56. How to collaborate
 http://project.liquidpub.org/karaku/
 http://project.liquidpub.org/resman/
project.liquidpub.org Fabio Casati - ECOWS 2009 56

57. 57