Computational Fields meet Augmented Reality: Perspectives and Challenges

Computational Fields meet Augmented Reality:
Perspectives and Challenges
Danilo Pianini, Angelo Croatti, Mirko Viroli, Alessandro Ricci
{danilo.pianini, a.croatti, mirko.viroli, a.ricci}@unibo.it
Alma Mater Studiorum—Universit`a di Bologna a Cesena
Spatial and COllective PErvasive Computing Systems (SCOPES)
September 21, 2015 - Cambridge, USA
Pianini et. al (UniBo) Computational Fields and Augmented Reality 2015-09-21 SCOPES 1 / 25

Outline
1 Introduction
Motivation
2 Augmented Reality
Basics
3 Computational fields
Introduction to aggregate programming and computational fields
Languages for aggregate computing
4 Augmented Fields
Augmented reality as visual interface for fields
Augmented reality-based input for CF program
Computational fields as enabling technology for AR applications
Example
5 Conclusion
Conclusion and future work

Introduction Motivation
Outline
1 Introduction
Motivation
2 Augmented Reality
Basics
4 Augmented Fields
Example
5 Conclusion

Introduction Motivation
Why Augmented Reality and Computational Fields?
Differences
They are definitely different entities
CF is a programming abstraction. AR is a mean of interaction.
CF focuses on collectivity, AR (mostly) on the single user.
Commonalities
They share a common context of application
Both are devoted at environments pervaded with computational
devices
Both are bound to the physical world

Augmented Reality Basics
Outline
1 Introduction
Motivation
2 Augmented Reality
Basics
4 Augmented Fields
Example
5 Conclusion

Deﬁnition
Augmented Reality
is a technology that allows the user to see the real world, with virtual
objects superimposed upon or mixed with the real world [Azu97].
The ultimate goal is a seamless integration of reality and virtuality
Mostly via see-through devices
Images from Microsoft Hololens presentation video

Degrees of augmentation
GHOST
AGENT
GHOST
BODY STREET
LAMP
PLAYER
BODY
PLAYER
ASSISTANT
AGENT
MIRROR WORLD
PHYSICAL WORLD
8
Augmentation can happen at diﬀerent levels:
1 Unrelated to the elements in the user’s FOV
E.g. displaying a map on a visual overlay
2 Dynamically associated to elements in the user’s FOV
E.g. displaying historical information next to a monument in FOV
3 Entirely virtual and interactive elements on real world
E.g. the two images of the previous slide, the ghost game [RPTC15]
Image on left: DashWare. Image on right from [RPTC15]

Computational ﬁelds Introduction to aggregate programming and computational ﬁelds
Outline
1 Introduction
Motivation
2 Augmented Reality
Basics
4 Augmented Fields
Example
5 Conclusion

Manifesto of aggregate computing
Main observation
In a world with ever increasing number of deployed devices, one
conveniently views a dense aggregation of interacting devices as a discrete
approximation of the space of computational environment through which
they are distributed.
neighborhood
device

Manifesto of aggregate computing
Main observation
1 the reference “machine” for pervasive computing processes is
abstracted to be the continuum of computational devices;
2 the reference “elaboration process” is the manipulation of a physically
distributed data structure;
3 how computation is carried on by the cooperation of devices in that
region is hidden “under-the-hood” of the model/platform
neighborhood
device

Pervasive continuum and computational ﬁelds
Aggregate computing
From “what value the single device computes”...
...to “what distributed data structure the pervasive fabric computes”
The notion of computational ﬁeld arises [MZ09, BB06]
⇒ a map from the space to (structured) values
typically evolving over time, possibly self-stabilising
better “understood” on continuous domains
neighborhood
device

Computational ﬁelds Languages for aggregate computing
Outline
1 Introduction
Motivation
2 Augmented Reality
Basics
4 Augmented Fields
Example
5 Conclusion

Existing languages
MIT Proto [BB06] is the most known and successful
Developed at MIT and maintained at BBN Technologies
Functional language, LISP-like syntax (I know you hate it too)
All devices run the same program
Computation happens in rounds:
Every device sleeps for some time
Processes the messages received from the neighbours
Executes its program
Sends all the neighbours its result
Complex operational semantics
Diﬃcult to maintain and extend

Field Calculus
A “distillate” of Proto
Provides a lightweight operational semantics [VDB13]
((((LISP-like syntax))))
Still a functional language
Simple enough to formally prove properties, powerful enough to be
universal (proved!)
Theoretical object, no runtime nor simulation tool provided

Protelis [PVB15]
Ordinary language features
Functional language
Same operational semantics of the ﬁeld calculus
C-family syntax with inﬁx operators
Java interoperability: static methods imports and calls, method
invocation with dynamic binding
Higher order functions (functions as arguments, lambdas)
Dynamic code
Suggested read [BPV15]
Jacob Beal, Danilo Pianini, and Mirko Viroli. Aggregate programming for
the internet of things.
IEEE Computer, 48(9), 2015

Code example
def count() {
rep(x<-0) { x + 1 }
}
def maxh(field) { maxHood(nbr(field)) }
def distanceTo(source) {
rep (d <- Infinity) {
mux (source) { 0 } else { minHood(nbr(d) + nbrRange) }
}
}
def distanceToWithObstacle(source, obstacle) {
if (obstacle) { Infinity } else { distanceTo(source) }
}

Architecture
Implementation, distribution
Based on Xtext
Eclipse plugin
Now distributed through Maven Central a
Integrated with Alchemist [PMV13]
Easy to port to new platforms
a
artifact: org.protelis:protelis

Augmented Fields Augmented reality as visual interface for ﬁelds
Outline
1 Introduction
Motivation
2 Augmented Reality
Basics
4 Augmented Fields
Example
5 Conclusion

See the fields
Inject the ability to use AR to see the ongoing CF computation
How
Read the program output, display it properly
Requires basically no intervention on the CF program
Why
The mapped field may be directly mapped to the user (e.g. a crowd
warning system)
Useful for seeing the shape of a field while developing
Limitations
Some fields cannot be trivially mapped to a graphical output
How would you map a field of anonymous functions?
No interaction

User view
Crowd detection scenario: change the perceived hue in such a way that
dangerously dense areas are properly advertised.

Developer view
Crowd detection program development: see the ﬁeld.

Augmented Fields Augmented reality-based input for CF program
Outline
1 Introduction
Motivation
2 Augmented Reality
Basics
4 Augmented Fields
Example
5 Conclusion

Weak interaction
Provide input to a CF program through AR interaction
How
Connect the (processed) AR input to the “sensors” of a CF program
Requires basically no intervention on the CF program
New information must be added to an existing device participating
the aggregate
Why
Some system settings could be very naturally set by means of a
gesture
Limitations
The new information is constrained in a device (a single point in
space)

Strong interaction
GHOST
AGENT
GHOST
BODY STREET
LAMP
PLAYER
BODY
PLAYER
ASSISTANT
AGENT
MIRROR WORLD
PHYSICAL WORLD
8

Strong interaction
Inject new information that lives in a “Mirror World”
How
Modify the CF program to “sustain” the existence of entities in areas
where no device is located
e.g. with a ﬁeld of mappings between virtual entities and their position
Why
Support for highest degree of augmented reality
Mixed real-virtual collective applications
Limitations
The program must be conceived with this kind of interaction in mind
Harder to realize

Augmented Fields Computational ﬁelds as enabling technology for AR applications
Outline
1 Introduction
Motivation
2 Augmented Reality
Basics
4 Augmented Fields
Example
5 Conclusion

A look the other way around
We have discussed increasing degrees of interaction between augmented
reality and computational fields, up to a “strong” interaction, but...
...are computational fields a valid abstraction for creating such programs?
Progression
“Weak” interaction is almost a free lunch
“Strong” interaction seems powerful, but requires CF programs to be
designed with AR in mind
AR is no longer confined to be an I/O component of the system, it
impacts the “business logic” of the application instead

Mirror Worlds on Augmented Fields
Basic strategy:
Switch from a simple ﬁeld of mirrored entities to a ﬁeld of entities with
location
Exploit CF programming to keep the entities alive and their status
aligned on multiple devices
If entities have their own behavior, mobile code support is required
[DVPB15]
Reusable building blocks can be exploited in order to devise a
“standard” solution [BPV15]

Augmented Fields Example
Outline
1 Introduction
Motivation
2 Augmented Reality
Basics
4 Augmented Fields
Example
5 Conclusion

Augmented Fields Example
Aggregate triage

Conclusion Conclusion and future work
Outline
1 Introduction
Motivation
2 Augmented Reality
Basics
4 Augmented Fields
Example
5 Conclusion

Conclusion Conclusion and future work
Conclusion
We noted that both computational fields and augmented reality work
best in environments densely populated with computational devices
We have explored increasing degrees of integration
AR as UI ⇒ not hard, no change in aggregate programs
Local input from AR devices ⇒ still not hard, still no change
Injection of (possibly proactive) augmented entities, ⇒ requires CF
programs to be properly designed
Modern aggregate programming languages support code mobility and
reusability
We believe that CF are a promising programming abstraction for
supporting AR applications, and languages and technologies are maturing
Future work
Build a working demo is the most effective way of experimentally confirm
our hypothesis.

References
References I
Ronald Azuma.
A survey of augmented reality.
6(4):355–385, 1997.
Jacob Beal and Jonathan Bachrach.
Infrastructure for engineered emergence on sensor/actuator networks.
IEEE Intelligent Systems, 21(2), 2006.
Jacob Beal, Danilo Pianini, and Mirko Viroli.
Aggregate programming for the internet of things.
IEEE Computer, 48(9), 2015.
Ferruccio Damiani, Mirko Viroli, Danilo Pianini, and Jacob Beal.
Code mobility meets self-organisation: A higher-order calculus of computational ﬁelds.
In Susanne Graf and Mahesh Viswanathan, editors, Formal Techniques for Distributed
Objects, Components, and Systems, volume 9039 of Lecture Notes in Computer Science,
pages 113–128. Springer International Publishing, 2015.
Marco Mamei and Franco Zambonelli.
Programming pervasive and mobile computing applications: The tota approach.
ACM Transactions on Software Engineering and Methodologies, 18(4), 2009.

References
References II
Danilo Pianini, Sara Montagna, and Mirko Viroli.
Chemical-oriented simulation of computational systems with Alchemist.
Journal of Simulation, 2013.
Danilo Pianini, Mirko Viroli, and Jacob Beal.
Protelis: Practical aggregate programming.
In Roger L. Wainwright, Juan Manuel Corchado, Alessio Bechini, and Jiman Hong, editors,
Proceedings of the 30th Annual ACM Symposium on Applied Computing, Salamanca,
Spain, April 13-17, 2015, pages 1846–1853, Salamanca, Spain, 2015. ACM.
Alessandro Ricci, Michele Piunti, Luca Tummolini, and Cristiano Castelfranchi.
The mirror world: Preparing for mixed-reality living.
IEEE Pervasive Computing, 14(2):60–63, 2015.
Mirko Viroli, Ferruccio Damiani, and Jacob Beal.
A calculus of computational ﬁelds.
In Carlos Canal and Massimo Villari, editors, Advances in Service-Oriented and Cloud
Computing, volume 393 of Communications in Computer and Information Sci., pages
114–128. Springer Berlin Heidelberg, 2013.

Computational Fields meet Augmented Reality: Perspectives and Challenges

Recommandé

Recommandé

Contenu connexe

Similaire à Computational Fields meet Augmented Reality: Perspectives and Challenges

Similaire à Computational Fields meet Augmented Reality: Perspectives and Challenges (20)

Plus de Danilo Pianini

Plus de Danilo Pianini (20)

Dernier

Dernier (20)

Computational Fields meet Augmented Reality: Perspectives and Challenges