Presentation given at WISE 2014, 15th International Conference on Web Information System Engineering, Thessaloniki, Greece, October, 2014 ABSTRACT: Digital documents often do not exist in isolation but are implicitly or explicitly linked to parts of other documents. Nevertheless,most existing document formats only support links to web resources but not to parts of third-party documents. An open cross-document link service should address the multitude of existing document formats and be extensible to support emerging document formats and models. We present an architecture and prototype of an open cross-document link service and browser that is based on the RSL hypermedia metamodel. A main contribution is the specification and development of a visual plug-in solution that enables the integration of new document formats without requiring changes to the cross-document browser’s main user interface component. The presented visual plug-in mechanism makes use of the Open Service Gateway initiative (OSGi) specification for modularisation and plug-in extensibility and has been validated by developing data as well as visual plug-ins for a number of existing document formats Paper: https://www.academia.edu/7757836/Open_Cross-Document_Linking_and_Browsing_based_on_a_Visual_Plug-in_Architecture

1. 2 December 2005
Open Cross-Document Linking and Browsing
based on a Visual Plug-in Architecture
Ahmed A.O.Tayeh and Beat Signer
Web & Information Systems Engineering Lab (WISE)
Department of Computer Science
Vrije Universiteit Brussel
WEB & INFORMATION
SYSTEMS ENGINEERING

2. Ahmed Tayeh -Department of Computer Science -atayeh@vub.ac.be
October14, 2014
Isolated Digital Documents
×
1
HTML document
PDF document

3. Ahmed Tayeh -Department of Computer Science -atayeh@vub.ac.be
October14, 2014
Isolated Digital Documents …
2
×
×
×
Word document
PDF document

4. Ahmed Tayeh -Department of Computer Science -atayeh@vub.ac.be
October14, 2014
Isolated Digital Documents …
Limited possibilities to create hyperlinks between snippets of different document formats
document formats mainly address links to web resources (HTML)
simple embedded unidirectional links
linked documents are not aware of any links that have been defined from other source documents
3

5. Ahmed Tayeh -Department of Computer Science -atayeh@vub.ac.be
October14, 2014
Existing Support for Hyperlinks
4

6. Ahmed Tayeh -Department of Computer Science -atayeh@vub.ac.be
October14, 2014
Cross-Document Linking
Anchor (pageindex + ec(x,y,w,h))
Anchor(XPointer-like expression)
Anchor(start and end indices)
Anchor(XPointerexpression)
5
PDF (linear-like model)
HTML (tree model)
Text (linear model)
XML (tree model)

7. Ahmed Tayeh -Department of Computer Science -atayeh@vub.ac.be
October14, 2014
Open Hypermedia
No support for cross-document linking
Limited extensibility
on the dataas well ason the visuallevel
6
MADCOW: A Multimedia Digital Annotation System,
Paolo Bottoniet al., Proceedings of AVI 2004

8. Ahmed Tayeh -Department of Computer Science -atayeh@vub.ac.be
October14, 2014
Requirements for a Linking Service
No changes requiredto existing document formats and standards
No assumptions about the document models
e.g. WYSIWYG, tree models, constrained tree models, …
Support existing as well as emerging document formats
Support advanced linkingfeatures
7

9. October 14, 2014 Ahmed Tayeh - Department of Computer Science - atayeh@vub.ac.be
Open Cross-Media Architecture
9
Annotation/Link
Service
Client Application
Annotation/Link
Browser & Editor
Visual Plug-ins
Resource Plug-in Repository
Data Plug-ins
Visual Plug-ins Visual Plug-ins Data Plug-ins
An Architecture for Open Cross-Media Annotation Services,
Beat Signer and Moria C. Norrie, Proceeding of WISE 2009

10. Ahmed Tayeh -Department of Computer Science -atayeh@vub.ac.be
October14, 2014
Data Level Extensibility (RSL Approach)
As We May Link: A General Metamodelfor Hypermedia Systems,
Beat Signer and MoriaC. Norrie, Proceeding of ER 2007
…
…
11
file://
URL
file://
URL
…
shape
XPointer
(s,e)
XPointer
…

11. Ahmed Tayeh -Department of Computer Science -atayeh@vub.ac.be
October14, 2014
Visualisation Layer Extensibility
12
local visual plug-ins
external visual plug-ins
communicate
Link Browser

12. Ahmed Tayeh -Department of Computer Science -atayeh@vub.ac.be
October14, 2014
VisualisationLayer Extensibility …
Local visual plug-ins vs. external visual plug-ins
Link Browser
Link visual plug-in
Link visual plug-in
13
Acrobat Reader
Google Chrome

13. Ahmed Tayeh -Department of Computer Science -atayeh@vub.ac.be
October14, 2014
VisualisationLayer Extensibility …
DefaultDocumentclass
necessary methods to visualiseany document format
-getSelector()
-openDocument()
-…
event listeners and handlers
Local visual plug-ins
extend DefaultDocumentclass
might use existing visualisationlibraries
Link browser
instantiates DefaultDocument
14
DocFormat1DocFormat3DocFormat2Visual Plug-insVisualisationGateway DocFormat3DocFormat3 ApplicationThird-Party Applications VisualisationDocFormat1VisualisationDocFormat2 Browser

14. Ahmed Tayeh -Department of Computer Science -atayeh@vub.ac.be
October14, 2014
VisualisationLayer Extensibility …
External visual plug-ins
provide methods to handle selections
in third-party applications
communicate with the link browservia a specific gateway plug-in
Gateway plug-ins
launch third-party application
offer different communicationprotocols
-TCP sockets
-WebSockets
-REST API as a fallback
DocFormat1DocFormat3DocFormat2Visual Plug-insVisualisationGateway DocFormat3DocFormat3 ApplicationThird-Party Applications VisualisationDocFormat1VisualisationDocFormat2 Browser
15

15. Ahmed Tayeh -Department of Computer Science -atayeh@vub.ac.be
October14, 2014
Open Cross-Document Link Service Architecture
DocFormat3DatabaseDocFormat2DocFormat1DocFormat1DocFormat3DocFormat2Visual Plug-insVisualisationGateway DocFormat3RSLDatabase ManagerDocFormat3 ApplicationThird-Party ApplicationsData Plug-insCoreData Layer VisualisationDocFormat1VisualisationDocFormat2 Browser
16

16. Ahmed Tayeh -Department of Computer Science -atayeh@vub.ac.be
October14, 2014
Open Service Gateway initiative (OSGi)
Enhances modularisationof our linking service
Use OSGidynamic extensibility for dynamic extensibility of the linking service
OSGiversioning and dependency resolution reduces problems in dealing with multiple libraries (“JAR hell”)
linking service might provide different visual plug-ins (versions) for asingle document format
17

17. Ahmed Tayeh -Department of Computer Science -atayeh@vub.ac.be
October14, 2014
Open Service Gateway initiative (OSGi) …
User Interface Manifest
Manifest-Version: 1.0
Bundle-ManifestVersion: 2
Bundle-Name: UserInterface
Bundle-SymbolicName: com.rsl.userInterface
Bundle-Version: 1.0.0.qualifier
Bundle-RequiredExecutionEnvironment: JavaSE-1.7
Export-Package: org.rsl.userInterface
Import-Package: org.rsl.core, org.rsl.service,
org.rsl.databasemanager,
org.associations.collections,
org.osgi.framework;version="1.7.0"
PDF Local Visual Plug-in Manifest
Manifest-Version: 1.0
Bundle-ManifestVersion: 2
Bundle-Name: Visual
Bundle-SymbolicName: org.rsl.pdf.visual
Bundle-Version: 1.0.0.qualifier
Bundle-RequiredExecutionEnvironment: JavaSE-1.7
Import-Package: org.rsl.core, org.rsl.userInterface,
org.rsl.pdf.data
Extension-Class: org.rsl.pdf.visual.Pdf
Extension-Type: visual
Extension-Format: application/pdf
18

18. Ahmed Tayeh -Department of Computer Science -atayeh@vub.ac.be
October14, 2014
Link Navigation Scenario
BrowserRSLRegistryDocFormat1Gateway DocFormat3DocFormat2DocFormat3getSelection() selectoraltgetPluginType(formatName) instantiateClass() openDocument(resource, selectors, selector) instantiateClass() openDocument(resource, selectors, selector) openDocument(resource, selectors, selector) launchApp() [visualPlugin==local] [else] getLinkTarget(selector) getDocument(resource)
19

19. Ahmed Tayeh -Department of Computer Science -atayeh@vub.ac.be
October14, 2014
Evaluation: PDF, XML, Text and HTML Plug-ins
20

20. Ahmed Tayeh -Department of Computer Science -atayeh@vub.ac.be
October14, 2014
Conclusion and Future Work
Cross-document link service prototype
Extensibilityon the data as well as the visualisationlayer
OSGiframework for extensible architecture
Evaluation with PDF, XML, Text and HTML plug-ins
Dynamic plug-in extensibility
Investigate extensibility in a study with developers
Usability evaluation
21

21. Ahmed Tayeh -Department of Computer Science -atayeh@vub.ac.be
October14, 2014
References
A.A.OTayehand Beat Signer, Open Cross- Document Linking and Browsing based on a VisualPlug-in Architecture, Proceedings of WISE 2014,15th International Conference on Web Information System Engineering, Thessaloniki, Greece, October, 2014
http://wise.vub.ac.be/sites/default/files/publications/WISE2014.pdf