Presentation given at ER 2010, 29th International Conference on Conceptual Modeling, Vancouver, Canada, November 2010
ABSTRACT: Many of today's digital document formats are strongly based on a digital emulation of printed media. While such a paper simulation might be appropriate for the visualisation of certain digital content, it is generally not the most effective solution for digitally managing and storing information. The oversimplistic modelling of digital documents as monolithic blocks of linear content, with a lack of structural semantics, does not pay attention to some of the superior features that digital media offers in comparison to traditional paper documents. For example, existing digital document formats adopt the limitations of paper documents by unnecessarily replicating content via copy and paste operations, instead of digitally embedding and reusing parts of digital documents via structural references. We introduce a conceptual model for structural cross-media content composition and highlight how the proposed solution not only enables the reuse of content via structural relationships, but also supports dynamic and context-dependent document adaptation, structural content annotations as well as the integration of arbitrary non-textual media types. We further discuss solutions for the fluid navigation and cross-media content publishing based on the proposed structural cross-media content model.
What is Wrong with Digital Documents? A Conceptual Model for Structural Cross-Media Content Composition and Reuse
1. What is Wrong with Digital Documents?
A Conceptual Model for Structural Cross-Media
Content Composition and Reuse
Beat Signer, http://www.beatsigner.com
Web & Information Systems Engineering Lab (WISE)
Department of Computer Science
Vrije Universiteit Brussel
ER 2010, November 4, 2010
2. PARC User Interface (Mid 70s)
Desktop metaphor
WIMP interaction
"filing cabinets" with
hierarchical folders
Paper paradigm
"What You See Is What
You Get" (WYSIWYG)
what about richer
document formats that
existed at that time?
Xerox Star 8010 [http://www.digibarn.com/collections/systems/xerox-8010/]
ER 2010, November 4, 2010 Beat Signer - Vrije Universiteit Brussel - bsigner@vub.ac.be
3. Digital Documents as a Paper Simulator?
Most people don't understand the logic of the
concept: "What You See Is What You Get" is
based on printing the document out ("get"
means "get WHEN YOU PRINT IT OUT"). And that
means a metaphysical shift: a document can
only consist of what can be printed! [...] No
overlays [...] – PAPER UNDER GLASS.
Geeks Bearing Gifts: How the Computer World Got This Way, Mindful Press 2009 Ted Nelson
When data of any sort are placed in storage,
they are filed alphabetically or numerically,
and information is found (when it is) by
tracing it down from subclass to subclass. It
can be in only one place, unless duplicates
are used [...] The human mind does not work
that way. It operates by association.
As We May Think, Atlantic Monthly, July 1945 Vannevar Bush
ER 2010, November 4, 2010 Beat Signer - Vrije Universiteit Brussel - bsigner@vub.ac.be
4. Paper-based Work Practices
Marginal
annotations
#
Cut and paste
versioning
Marcel Proust, À la recherche du temps perdu
Du côté de chez Swann [http://expositions.bnf.fr/brouillons/grand/45.htm]
ER 2010, November 4, 2010 Beat Signer - Vrije Universiteit Brussel - bsigner@vub.ac.be
5. oN-Line System (NLS)
Collaborative System
by D. Engelbart et al.
demonstrated in 1968
addressing and linking
parts of files
flexible file views
overlay file structures
cross-file editing
...
ER 2010, November 4, 2010 Beat Signer - Vrije Universiteit Brussel - bsigner@vub.ac.be
7. Challenges
Representation of cross-media relationships
navigational relationships
structural relationships
- within document structure
- external document structure
Content reuse across media boundaries
cross-media transclusion
Distribution of data and metadata across
multiple devices and media formats
ER 2010, November 4, 2010 Beat Signer - Vrije Universiteit Brussel - bsigner@vub.ac.be
8. Structure Metamodel (RSL)
structure link link
(0,*) (1,1) Structural (1,*) Navigational
Structures HasElements |HasChild|
Links Links
partition
link parameter
(1,*) (1,*) Has (0,*)
HasSource Links HasTarget Preferences Preferences
(0,*) (0,*)
parameter (0,*)
(0,*) (0,*)
Properties HasProperties (0,*)
entity (0,*) AccessibleTo (0,*) user
(0,*)
Entities (1,1) Users HasMembers
contextResolver (0,*)
(0,*)
Context HasResolver CreatedBy
partition
Resolvers (0,*) partition (0,*)
selector resource individual group
(1,1) (0,*)
Selectors RefersTo Resources Individuals Groups
(1,1) (0,*)
layer
(0,*) (0,*) OM notation
OnLayer Layers |HasLayers| (developed at ETH Zurich)
B. Signer and M.C. Norrie, As We May Link: A General Metamodel for Hypermedia Systems. In Proc. of ER 2007, Nov 2007
ER 2010, November 4, 2010 Beat Signer - Vrije Universiteit Brussel - bsigner@vub.ac.be
9. Associative File System (RBAF)
structural link
navigational link
signer
My Pictures My Videos My Presentations
Vancouver ER 2010 Vancouver ER 2010 ER 2010 WISE 2009
Aquarium Keynote
single slide
cross-media transclusion
Granville Vancouver Public Keynote Olga and Dinner transclusion (via OOXML
Island Harbour Market J. Thorp Lamia ER 2010 RSL selector)
multiple classification cross-media association
ER 2010, November 4, 2010 Beat Signer - Vrije Universiteit Brussel - bsigner@vub.ac.be
10. Associative File System (RBAF) ...
ER 2010, November 4, 2010 Beat Signer - Vrije Universiteit Brussel - bsigner@vub.ac.be
11. Structure Metamodel Features
Multiple overlay structures can be defined on top
of a set of entities (views)
Structural relationships between parts of
resources (fragments) via selector concept
Structural relationships as first class entities
tagging via properties (key/value pairs)
navigational links to other resources
context-dependent adaptive structures via context
resolvers
ER 2010, November 4, 2010 Beat Signer - Vrije Universiteit Brussel - bsigner@vub.ac.be
12. Cross-media Information Browser & Editor
How to build a cross-media structure browser & editor?
ER 2010, November 4, 2010 Beat Signer - Vrije Universiteit Brussel - bsigner@vub.ac.be
13. Conclusions
Existing document formats are very much
based on the simulation of paper affordances
on static desktop computers
Structure metamodel for the representation of
open and fluid cross-media information spaces
associative file system example
cross-media information browser & editor
Remediation of the "paper simulation" approach
WYSIWYG is only one out of many options!
ER 2010, November 4, 2010 Beat Signer - Vrije Universiteit Brussel - bsigner@vub.ac.be
14. Acknowledgments
Colleagues at ETH Zurich for collaborating on
the RSL link metamodel; in particular
Prof. Dr. Moira C. Norrie
Gregory Cardone for implementing the initial
prototype of the RSL-based associative file
system (RBAF)
ER 2010, November 4, 2010 Beat Signer - Vrije Universiteit Brussel - bsigner@vub.ac.be
15. References
B. Signer and M.C. Norrie, As We May Link: A General
Metamodel for Hypermedia Systems, Proc. of ER 2007,
Auckland, New Zealand, November 2007, 359–374
V. Bush, As We May Think, Atlantic Monthly 176(1), 1945
T. Nelson, Geeks Bearing Gifts: How the Computer World
Got This Way, Mindful Press, 2009
B. Signer, Fundamental Concepts for Interactive
Paper and Cross-Media Information Spaces,
ISBN 978-3-8370-2713-6, May 2008
ER 2010, November 4, 2010 Beat Signer - Vrije Universiteit Brussel - bsigner@vub.ac.be
16. ER 2010, November 4, 2010 Beat Signer - Vrije Universiteit Brussel - bsigner@vub.ac.be
17. ER 2010, November 4, 2010 Beat Signer - Vrije Universiteit Brussel - bsigner@vub.ac.be
18. ER 2010, November 4, 2010 Beat Signer - Vrije Universiteit Brussel - bsigner@vub.ac.be