This document reviews the Pervasive Experience project. In this project the driving assumption is that increasingly pervasive, networked technologies are impacting our lives. The research question is: How is Pervasive Computing changing you?
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
The Pervasive Experience - project review July 2010
1. The Pervasive Experience
will a second Cognitive Revolution turn us into d-zombies...
Project Review : July 2010
roBman@mob-labs.com / @nambor
photo by club silencio
3. what is the Pervasive Experience?
Ubiquitous Computing was pioneered by Weiser (1991) at Xerox Parc.
“Ubiquitous computing names the third wave in computing,
just now beginning. First were mainframes, each shared by
lots of people. Now we are in the personal computing era,
person and machine staring uneasily at each other across
the desktop. Next comes ubiquitous computing, or the age of
calm technology, when technology recedes into the
background of our lives.”
Mark Weiser
That was obviously from a 90's perspective. This project proposes a
structural and ethical argument that Pervasive is a more appropriate
and Humanistic term than Ubiquitous.
So the Pervasive Experience is a new, truly modern Human Condition
that is now possible in today's anytime, anywhere internet world.
4. what is a Cognitive Revolution?
From Descartes (1637) and the Enlightenment of the 1600's through to
the 1950's and 60's, Positivism was the dominant epistemology in
scientific thought and Western Philosophy.
Built upon this world view, Pavlov (1904) and Skinner (1938) defined
Behaviourism within the field of experimental psychology.
Yet starting in the 1950's the dominant epistemology began to turn
towards Realism. A reaction against Behaviourism became a
movement known as the Cognitive Revolution.
This movement recognised our internal phenomenological experiences
as a valid object of research.
5. what was that about Zombies?
Chalmers (1995) is an interesting Philosopher that has contributed to
the field of Consciousness.
In order to explore and discuss the concepts surrounding the
phenomenological aspect of Consciousness he has proposed what has
become known as the Philosophical Zombie thought experiment.
In its simplest form it poses the question “if people existed that
behaved exactly like a normal human, but didn't experience the
phenomenon of internal representation – are they still human?”.
These meaty automata like creatures are called Philosophical Zombies
or p-zombies for short.
It can also be adapted to explore similar questions about the concept
of the Soul. Hypothetical humans without Souls are called s-zombies.
6. so what's the big idea then?
Pervasive Computing could be creating a second Cognitive Revolution.
It is changing the geometry of our sensory input and our presence. It is
changing the structure of our sphere of control. Our very Body Image.
I believe Pervasive Computing is adding an interesting new dimension
to Chalmers' Philosophical Zombie thought experiment...lets give the
theoretical zombies in a Pervasive Computing Experience a name.
Lets call them Digital Zombies or d-zombies
These are people whose experience of the phenomenon of internal
representation is more than a certain percentage digital, in some way.
People whose sensory experience is digitally mediated beyond a
defined d-zombie threshold.
7. this raises some interesting questions
Does this d-zombie threshold exists at all.
When we reach a point where our sensory input is more digitally
mediated than solely biologically mediated, can our conscious
subjective experience somehow become less Human and more
like a d-zombie?
Can it lead us to become d-zombies in ANY way at ANY point at all?
What if our attention itself became incrementally digital?
(See the Pervasive Reality Engine defined below)
8. contents
The Project : An introduction and brief history
Expanding my view : The project's new focus
Reality Engines : The key model proposed
Philosophical context : The related body of knowledge
Ubiquitous vs. Pervasive : Pinning down key terms
Reading the Web : Seeing the web as a digital prosthetic
The Pervasive Experience : Reflections on this project
What's next? : Where to from here
Experimental Technology : Rubber hands and AR helicopters
10. The Pervasive Experience Project
Hold your hand up and away from you. Imagine your fingertips are dipped in vomit! Feel how dirty and contaminated your fingers are.
Can you sense changes in how you hold your body image? Take a moment to feel how this part of your body holds your attention.
Even if you look away it seems as if your hand is tagged with meaning and still to be attended to.
This positive, lead hallucination provides you with your own subjective experience of a traditional "real world" Body Image.
Biological, grounded, open to symbolic manipulation...yet essentially "real".
By contrast, this document reviews the Pervasive Experience project. In this project the driving assumption is that
increasingly pervasive, networked technologies are impacting our lives. The research question is:
How is Pervasive Computing changing you?
This applied research explores how Pervasive Computing may impact our Philosophy of Mind. It looks at our
perception of space and self, and their underlying Neurobiology. It aims to integrate theories from these four
broad areas in a high-level, structural way. Its aim is to explore through practice and reflection how these theories
may be able to be reshaped by the pervasive diffusion of digital networked technologies.
Based on this analysis, an integrated, pragmatic, working model of the end-to-end process is synthesised - from
the environment, body, nervous system and brain to sensory input, representation, consciousness and cognition.
This model represents a Reality Engine. This document aims to frame this model within the existing body of
knowledge, then extend it to support the rapidly growing, digital environment of Pervasive Computing.
I believe this model opens a new perspective on network enabled devices where the web itself can be seen as a
form of digital prosthetic that is increasingly pervading our experience and becoming more organ-ised.
So after completing this review I now believe the most likely answer to this research project's question may be:
By driving rapid Body Image modification.
This could well be the defining structural process of Pervasive Computing...
11. Methodologies
Over the past two years I have been exploring the impact of Pervasive Computing upon my sense of space using
Action Research and reflecting through the lenses of Diffusion of Innovations and Distributed Cognition.
Action Research:
As defined by Dick (1999), Action Research is a "cyclic or spiral process" that "alternates between action and
critical reflection". This review document represents the output of one of these critical reflection cycles.
Diffusion of Innovations:
Initially defined by Tarde (1890) the theory of Diffusion of Innovations was crystalised in the seminal work by
Rogers (1962). It aims to describe and analyse why and how ideas and technologies diffuse through cultures,
especially the rate of diffusion.
Distributed Cognition:
As defined by Hutchins (1995), Distributed Cognition is a framework involving individuals, artefacts and the
environment. It is strongly based upon Vygotsky's (1920's) work that led to the development of Activity Theory.
Distributed Cognition that takes as its unit of analysis a culturally constituted functional group
rather than an individual mind. This theory is concerned with how information is propagated
through a system in the form of representational states of mediating structures. These
structures include internal as well as external knowledge representations, (knowledge, skills,
tools, etc.). This approach permits us to describe cognitive processes by tracing the movement
of information through a system and characterize the mechanisms of the system which carry out
the performance, both on the individual and the group level.
Hutchins (1995)
13. The bigger picture
This research has now been extended to include the lenses of Situational Awareness, Neuroplasticity and
Sensorimotor Theory. This change has expanded my research focus to include “consciousness” and "self" and
their underlying "Neurobiology". However this creates such a broad and technically deep domain that the goal is
simply to “integrate current theories from these areas at a purely structural level in relation to the Reality Engine”.
I have also been aware of Gibson's (1992) work on Visual Perception for a long time now but have struggled to
integrate it fully into this work. Recently I came across O'Regan's (2010) work on Sensorimotor Theory, which
seems to develop a situated aspect of Gibson's work within the modern context. To me this work provides a
strong foundation for a wide range of discussions from Action Research through to Philosophy of Mind.
I have also been researching Bach-y-Rita's (1969) work on Sensory Substitution that lead to the development of
the field of Neuroplasticity. This work leads to some amazing predictions that can be tested through direct
experience as shown by Bach-y-Rita, Ramachandran (2007) and Ehrsson (2010).
As you will see below I have found that what I am exploring appears to be built upon a form of
Neo-Cartesian Dualism. Decartes' statement in Principals of Philosophy, “I think, therefore I am” (Cogito ergo
sum) are a compelling fit for the traditional Body Image experience of our reality. Yet, in the Pervasive,
Neruoplastic, digitally networked Body Image experience of our proximate future we may be better supported by
the philosophical statement “I am what I think” (EGO sum quis EGO reputo?) or perhaps more explicity “I am the
sum of my thoughts”. Not in a frivilous “positive thinking” kind of way, but a “my thinking alters digital hardware
and software” kind of way. This is not contradicting Descartes work, but reflecting on it from this new angle.
If convergence/divergence continues to develop so our sensory input is increasingly digitally mediated then the
relevance of this statement should become even more significant over time. Distributed, interactive software and
hardware literally becomes part of our Extended Mind.
This is where the d-zombie questions come in.
14. definition: Situational Awareness
Situational Awareness is the awareness of what's going on around you. How you make sense of information,
events and actions. How well you understand how they'll effect your goals and objectives. Related concepts such
as Sensemaking take a more historical perspective and Situational Awareness tends to have a now and near
future perspective.
Inadequate Situational Awareness is seen as one of the key factors in accidents through human error. Especially
in roles where there are serious consequences (e.g. drivers, soldiers, pilots, doctors, etc.).
diagram credit
15. definition: Neuroplasticity
The concept was initially proposed by James (1907),
yet the term Neuroplasticity wasn't coined until (1948)
by Konorski.
Neuroplasticity was first experimentally demonstrated
by Bach-y-Rita (1969). His work in Sensory Substitution
suggested that the brain can rewire input from one
sensory mode into internal representations in another
mode. Not just in a form of synaesthesia, but by
completely re-wiring the cortical maps.
So at a time when the Cartesian homunculus appears
to be identifiable within our physical neurobiology in the
form of the cortical maps, we also appear to be finding
that this homunculus is also plastic, malleable and
changeable based on action and even thought alone.
Neuroplasticity is also revolutionary because in some
ways it brings cutting edge neurological research within
the reach of anyone with a working brain. Adding in the
Augmented Reality technology I work with everyday
and you have a powerful set of research tools.
photo credit
16. definition: Sensorimotor Theory
"The sensorimotor view conceives of feel ("subjective experience") as a way of interacting with
the environment. The quality of feel is simply an objective quality of this way of interacting."
O'Regan (2010)
I believe this provides support for the view that the self is the mental model that integrates input from other models
and senses. It is a by-product of the creation of attention. By it's very definition, the sensorimotor view “requires”
attention. I believe the emergent property of this is the self. A situated, attentive feedback loop.
This perspective is expanded further below.
diagram credit
18. Descartes' Reality Engine
To provide a starting point lets look at Descartes
proposal for a dualistic reality.
“I have a clear and distinct idea of myself as a thinking,
non-extended thing, and a clear and distinct idea of
body as an extended and non-thinking thing.”
So, Descartes argues, “The mind, a thinking thing, can
exist apart from its extended body. And therefore, the
mind is a substance distinct from the body, a substance
whose essence is thought.”
As we will see, this initial proposition has fragmented
into a wide range of Dualisms with different structures
and theories.
This project takes an Interactionist Dualism as it's
starting point.
Descartes' Dualism Diagram
diagram credit
19. The Reality Engine
elements of the diagram
a/b = concepts
self = center of attention
environment = mental model of the world
Body Image = mental model of your body
X <- you are here
ROM = rest of mind
brain = the biological organ
ROB = rest of body
ROW = rest of world
ROB <-> brain = nervous system
brain <-> Body Image/a/b = perception
Body Image <-> self = attention
sensory perception is dualistic
● sensory input is one world
● perception is the other
This diagram describes the single integrated Reality Engine model proposed by this project. A working system
whose feedback loop creates a cognitive conscious self that interacts with the perceived Real World.
Our brains seem to crave sustainable models that turn sensory input into actionable behaviour. Feedback loops
that integrate all the aspects of sensory input (internal and external!?) and physical action into an attentive self.
To me they appear from the outside to be a rich ecosystem of self-reflexive Hidden Markov Models (ironic?).
20. The Reality Engine extended
additions to the diagram
a = subconscious interactive concept
a1 = subconscious mental concept
a2 = subconscious body concept
X <- you are here
b = conscious interactive concept
b1 = conscious mental concept
b2 = conscious body concept
so what is the “self”?
● reality is rendered through attention
● attention is the integration of sensing,
reacting, remembering, thinking, doing
● the by-product of this is the self...
because attention needs a point of origin
A concepts are subconscious and can't be accessed by the self.
B concepts are conscious and may be accessed by the attentive self.
1 concepts (A1 & B1) are mental only and have no direct relationship with the body.
2 concepts (A2 & B2) are concepts that are pre-formed within the body itself before the sensory data is sent
through the nervous system to the the brain. This includes things like reflexes and muscle memory.
21. Mixing Realities
Milgram's Virtuality Continuum
If the input for this Reality Engine is substituted with completely digitally produced vision and sound, etc. then this
becomes a Virtual Reality Engine (VR).
And if we only overlay digital elements onto our perception of the Real World then this becomes an Augmented
Reality Engine (AR). While VR and Artificial Intelligence were an aspirational focus of the original Cognitive
Revolution, it seems likely from todays perspective that AR will continue to pervade our experience...but in folded
and bent ways.
In this case, pixels (or other sense based equivalents e.g. sound) can provide a good measure of how much
digital sensory mediation has pervaded a person's Body Image experience. That is, “What percentage of our total
sensory input is digitally mediated?”. In Augmented Reality the pixels that make up the overlaid information can
be counted to calculate this ratio based on our total field of visual sensory input.
Within this data is an important sub-set that is not just how much of our environmental sensory input is digitally
augmented, but how much of our attention is digitally mediated. Through the lenses of Distributed Cognition, and
Situational Awareness it's natural to ask “how much of our attention, and therefore self, can be Digital without
turning us into d-zombies?”.
Is there any limit to this digitally Extended Mind?
22. Multiplayer Realities
Multiple Reality Engines
A useful set of terms breaks the world into the Geosphere (physical earth/planets), the Biosphere (living things)
and the Noosphere (symbolic or thought related things). The Pervasive Experience is transforming the
Noosphere in terms of how interconnected it is. The relationship between space and even the point of origin of
thoughts and symbols is being completely reformed. While in the traditional world each “self” has been cacooned
within their own Cartesian isolation chamber, the Pervasive Experience may be opening the door to change this.
Shannon's (1948) work on a Theory of Communication has some interesting application here and is an application
of Hidden Markov Models. Symbolic communication between two Reality Engines seems to involve building a
Theory of Mind of the “other” and creates a pragmatic use of encoding/encryption in a Shannon Information sense.
23. Body Image
Based upon the full Reality Engine model proposed above, the term Body Image can be used to define the
specific mental model used to represent our body. It can also more flexibly be seen to represent all the mental
models that go in to making up our entire “self” and the broader context we exist within. Our entire Self Image and
the contexts that it implies.
Often, philosophical dualistic thought experiments and discussions isolate “special” dis-embodied beings to
explore or prove a point. From the perspective of this project this is a non-sense. A “dis-embodied experience” is
an altered form of consciousness outside the Sensorimotor world view. Even hallucination or meditation upon
“mental only” (e.g. B1) concepts are spatially related to the Body Image. The word “internal” is often used to
describe these types of phenomenon.
In this way attention is a situated experience (or feedback loop) that requires (or perhaps generates) a point of
origin.
Interestingly, early research into Attention during the first Cognitive Revolution was focused upon “divided
attention” (e.g. dichotic listening).
By contrast, the experiments of this research project are focused on in some senses the opposite. The goal here
is to integrate objects from two (or more) places/times into a single coherent attention, instead of attempting to
split attention into two (or more) separate streams.
I believe this is the complete reformation of the concept of Body Image that the Pervasive Experience is enabling.
25. Dualisms
In order to relate my reflections from this research review to the existing body of knowledge I have developed the
following map of what appears to be the related philosophical terrain. I am also open to new perspectives in this
area and reviewing new concepts and frameworks drives a lot of my reflection. Here's the map so far.
Cartesian Dualism:
This philosophy of mind separates reality into a dual system of mind and body. Descartes' work was an extension
of thought that dates back to Plato and Aristotle. This separation into physical and non-physical is also often
bound up with a spiritual discussion of the soul as well.
Interactionist Dualism:
Interactionism is the form of Dualism that proposes that the mind and body can interact bi-directionally in a causal
way. While it is often stated in the literature that it “is difficult” to establish how this may occur, I find it almost self
evident. fMRI's, etc. clearly show that brain activity is tied to symbolic phenomenal experience. Neuroplasticity
takes this one step further stating that the phenomenal experience alone can persistently change neruobiology.
Interactive Dualism:
There is also an area of Interaction Design/CHI where Theory of Mind is related to interactions with any dynamic
system. For example Norman (1988). diagram credit
26. Neurobiology
Neurobiology is an amazingly deep and technically complex area. This literally includes “brain surgery” and is
dealing with the most complex structure in the known universe.
This project focuses on the perspective of Cognitive Neuroscience and is primarily focused upon the theoretical
frameworks available that show how cognitive processes can impact the biological structures within the brain.
Neuroplasticity is the key lense used for this reflection and literature review.
I believe it is important to include this from at least a high-level in order to fully explore the Interactionist Dualism
foundations of the Reality Engine. I think it is also very relevant for the phrasing of the research question “How is
Pervasive Computing changing you?”.
There is also likely to become more closely integrated into the overall Pervasive Computing domain with the rise
of Computer/Mind interfaces. This is also closely related to the Digital Prosthetics content discussed below.
Interestingly there are some mechanical devices/technologies (e.g. TMS) that are widely available and are actively
being used to map and alter specific neruobiologies. Personally I find TMS quite frightening and think that this
project specifically raises some serious ethical issues that relate directly to it's use. (e.g. can mechanically
changing the operation of certain Neurobiological structures change your consciousness to the point of creating a
d-zombie? Could it be used as a literally de-humanising experience?). Not to mention the unknown long term
impacts of high-intensity EMF on delicate and critical cell structures.
27. The Mind
Philosophy of Mind:
This is a very broad and rich area. Dennett's (1991) work breaking the philosophy of mind into two problems
“consciousness” and “content” is interesting. His Neural Darwinism perspective is also useful and relevant for this
project.
Chalmers' work on consciousness, especially his p-zombie thought experiment is obviously relevant.
Theory of Mind:
The ability to form models that attempt to predict the behaviour of objects, animals and other behaviours is a
critical conscious activity. Interestingly, the fact that people form Theories of Mind implies the belief that a
representation system inside another's mind can causally interact to create actions.
Cognitive Revolution:
Essentially this was a rebellion against an automata style view of behaviourism and a recognition of the
phenomenological aspects of the human experience. Interestingly, in 1950 Turing published "Computing
machinery and intelligence" in the journal “Mind”. His proposal was that you could use questions and answers to
test machines for intelligence is often included as part of the Cognitive Revolution. His view here appears to be
that we can use our own innate Theory of Mind models (B1 concepts) to interrogate an “other” via a text based
interface to determine if it is Human or synthetic.
This highlights a consciousness continuum (or perhaps even triangle) between biological entities and
synthetic/mechanical entities.
28. Functionalist Hidden Markov Models
Functionalism:
Philosophical Functionalism provides a post-Cartesian perspective that is highly relevant to the Pervasive
Experience and it's digital-isation. However, the Reality Engine provides a perspective where Functionalism and
Interactionist Dualism need not be mutually exclusive. I believe it is possible for Functionalism to exist at the
physical/material Neurobiological level to create a subjective experience that is Dualistic and Interactionist in
nature. In fact I believe that the two are perfectly fitting sides of the same coin and that the nature of attention/self
requires both.
Hidden Markov Models:
The Cartesian cacoon that the Reality Engine suggests we all live within also suggests that the
Hidden Markov Model is a useful metaphor. This creates a nice, ironic, self-recursive Solipsistic loop
Shannon's Theory of Communication:
Shannon's (1948) work on a “Mathematical Theory of Communication” is also highly relevant to this discussion.
For example his “Fig 1.” is almost identical to the Multiplayer Reality Engine diagram above.
diagram credit
30. Weiser's Ubiquity
My starting point for the whole project was a diagram created by Mark Weiser in (1996) - see above.
My interpretation of this diagram is that the left and right elements on the first row contrast non-Interactionist
Cartesian Dualism (left) with Situated Cognition (right). The labels beside clearly state that Weiser believed a
non-Interactionist Dualism was "wrong".
The second row of left and right elements contrast Current Computing Technology with Ubiquitous Computing
Technology. My interpretation here is that Weiser was showing how Computing Technology will pervasively
spread throughout more and more aspects of our Situated reality, as opposed to the unipresent Desktop
Computing model that dominated the 1980's and 90's.
31. Spatial Maps
Review Edward T. Hall et al's work on Personal Space. Weiser's diagram is obviously focused upon the spatial
Relate this to the reality engine. relationship between you and the world. All Cartesian
like dualisms appear to have this spatial aspect.
The spatial models we hold can also be broken down
into finer grained regions. At the highest level of the
physical world is Inside and Outside our body.
This implies a Body Image and surrounding that, and
inter-meshed within that, is our overall sense of space.
Beyond proprioception are our exteroceptive senses
that collect stimulus from the outside world. From this
we build models about these spaces and relationships.
Hall's (1966) work on Personal Space provides a useful
framework for a User Centred world view.
These concepts also have a pragmatic relationship to
Situational Awareness too.
The Reality Engine aims to ensure Hall's work
integrates with Cartesian Dualism, Extended Mind,
Distributed Cognition and Situational Awareness.
32. Divergence
Applying this type of spatial analysis to the concept of Convergence has also led me to some interesting
reflection. I believe Divergence is the by-product of Object:Network Convergence.
As the distance between an Object and the Network passes a point of Convergence the distance between the
“Situated Point of Origin” of that Object and “Control” of that Object Diverges. In other words, once you connect
an Object to the Network you're free to remove its “Physical Interface” and simply “Control” it via the Network.
This is exemplified by the marketing term “Anywhere, Anytime”.
33. Putting YOU at the centre
I think that the framework above, especially the Pervasive Web integrated into the Reality Engine, highlights why
Pervasive is a more appropriate term than Ubiquitous.
Ubiquitous suggests that the technology is “everywhere”. While it may at first seem like that's the case, in reality I
believe we are discussing how this technology is soaking into our attention.
If you look at mobile network coverage maps they don't cover the whole world, just the parts where the majority of
the population are likely to be. This is a natural evolutionary pattern that allows for the conservation of energy.
Pervasive suggests the technology is soaking into “your” subjective experience - with you at it's heart.
Ubiquitous on the other hand suggests technology is soaking into the universe - whether you are there or not.
To me this appears to be an important structural and ethical distinction.
The next section aims to share some insights that should clarify why this may become even more important as
convergence/divergence continues to develop. Where distributed, interactive software and hardware literally
becomes part of our extended self. If the impact of Moore's law and the Network Effect (Metcalfe's law) keep
driving the Pervasive Network then I believe we need to make these technologies an extension of our “self”, or risk
becoming diluted and de-Humanised.
Again, this is where the d-zombie questions come in.
35. Prosthetics
Prosthetics are traditionally defined as an addition to the body (coming from the greek word for addition) that
replaces a missing body part. If it's related to mobility or a directly sense-able part of our Body Image then it is
likely to be a limb.
If it is to do with sub-conscious processes then it is likely to be an organ and may include real or artificial organs.
This analysis is blurred even further when the prosthetic is a sensing organ that impacts your Body Image.
photo credit
36. Digital Prosthetics
Now, there also exists the real opportunity to utilise Digital Prosthetics that enhance a person's sensory input and
even cognitive capabilities like memory. This obviously heads us into the well populated cyborg territory.
While the traditional focus of Prosthetics was upon repair and replacement, the Digital Prosthetic view of the
Pervasive Reality Engine is squarely focused upon “beyond repair” and towards “enhancement”.
Jamais Cascio presents a perspective that many people will be facing over the next few decades as their options
turn from repair to enhancement. This is also an interesting convergence of Digital Technologies, Medical
Miniaturisation and an Aging Population.
37. The Pervasive Web
It's amazing to see how the web has expanded from a single point on the planet (CERN) out to becoming a
pervasive cultural concept. In fact it could be seen to have initially existed as just a model within
the Reality Engine of Tim Berners-Lee.
Now it seems to be a model that exists in almost everyone's Reality Engine and it has distorted how they use and
think of space, place and time.
Pervasive Web devices can then be seen as a form of Digital Prothstetic and the Web itself is pervading
throughout our planet and our experience. Literally soaking into our consciousness.
However, it's in a unique class of mental models that not only cement themselves into Reality Engines, they
provide networked access and become the point of exchange for other models too.
39. First Reflections
The heart of my recent reflection has been upon Neuroplasticity and Situational Awareness and how they relate to
the concept of Personal Space and Theory of Mind in light of the Reality Engine. Extend this to interacting with an
increasing number of web applications and web enabled devices and you have a Pervasive Reality Engine.
The addition of Pervasive Computing to the Reality Engine creates a structural change to the models that defined
the Cognitive Revolution. While the original Cognitive Revolution was focused upon what happened within the
brain and mind, the Pervasive Reality Engine is focused upon what happens to us as our self moves
out of our brain and possibly even out of our mind if that is achievable (imagine extending the A2 & B2 concepts in
the Reality Engine using remote sensors). To use buzz terminology, this is Cognitive Revolution 2.0.
So it appears we have a relatively mature conceptual framework for the individual mind. At the other extreme it
also appears that the impact of interactive networks have been explored broadly from the macro, national scale.
Now seems like the perfect time to integrate these two perspectives into an exploration of the impact of pervasive
networks on the mind. This exploration leads to some interesting questions.
If consciousness could be a property achieved by any system that can create the right type of attentive self
awareness feedback loop, then we as humans could “safely” extend our Human experience using digital
technology. This would benefit us by allowing us to tag along with rapidly evolving digital technology. A chance
to keep up with any possible proximate Technological Singularity.
If not, then this has obvious implications for the ethics of technology and the discussion of the d-zombie threshold
and the possibly de-Humanising nature of technology.
However, if it were true then this has implications for the ethics of machine intelligence and even biology. If
consciousness were an achievable property of the right type of attentive loop then any “thing” that displays that
type of loop is essentially “equivalent to a self conscious human being” - at least at a category level.
Interestingly, either way there is an impact.
40. Content Reflection
Pervasive Computing is a movement driven by the structural reformation of our Reality Engines through
convergence/divergence. I believe it can be shown to enable the following measurable changes:
– rapid Body Image modification
– flexible Body Image design and re-engineering/sculpting
– dynamic multi-POV Body Image experiences
I am confident now that the content outlined above provides a robust theoretical and practical framework for
imagining, designing and building rich, interactive and "world changing" Pervasive Experiences.
This model obviously needs to be reviewed, tested and revised on a regular basis, however it provides a workable
starting point for some truly innovative experimentation.
41. Context Reflection
My personal experience of this Action Research project has been an interesting, stimulating and very challenging
one. The breadth of content and reading has been quite challenging, as has been keeping up with the constant
pace of change on the technology side. Luckily, my day-to-day work keeps me focused upon this type of change.
There are a number of limitations of my work that have been pointed out by the people that have been kind
enough to review it. The first is that it is almost exclusively from a western thought perspective. The world is
obviously full of a broad range of interesting and insightful perspectives that don't fit into that bucket. I hope to at
least start to address this in the next phase. I have also received feedback stating that this work appears strongly
philosophically monistic. Personally, I don't believe that the output is quite that simple. However the density of
the content here masks some of the subtleties. I hope I can communicate this aspect of the project more clearly
in the next phase and aim to make my stance here more transparent.
I have also receive feedback on how overwhelming and sometimes un-approachable this document can be. I've
made a number of revisions that have hopefully addressed some of this, but this is a continual challenge and I am
continuing to attempt to distill this down to a simpler set of key messages without diluting the depth of the work.
While this work is obviously strongly academic in nature, I am completing it independently from any academic
institution as an ongoing attempt to develop my practice. I have learned and grown a lot through the process, but
obviously a more formal context may have helped me stay more focused. Luckily I've been able to seek
constructive feedback and support from some academic friends that I'll thank in due course.
On the whole I feel I've achieved something of real value and I'm comfortable that I now have a good framework
with which to begin my experiments. This is where the project gets really interesting.
42. Ethical Reflection
There is an obvious commercial bias I have in my world view. I work as a Technologist for a Research Lab that
helps it's clients commercialise new technologies. While my company (MOB) are happy to support this research
project, both the company and I obviously benefit from the output of this project. However naive this may be, my
underlying perspective is not so much that “I do this research because my job is closely related”, but more “I am
good at my job because I am naturally drawn to a research project like this”. While this may be a subtle
difference, I believe it has significant implications to the overall work.
The questions raised by this research project are essentially Humanist in nature and are exploring the continuum
between today's common Human Condition and the other extreme of Artificial Intelligence. While traditional AI
starts at a synthetic point and heads towards consciousness, this project starts with the User and pushes the
boundaries in the other direction. The Ethical focus of Humanism sets out the framework for this reflective
process.
The logical conclusion of this reflection is the structural proposition for a new terminology with Pervasive being
more appropriate and Humanistic than Ubiquitous.
Further and deeper reflection upon the ethical aspects of this project as it relates to the following topics is also
planned. These obviously covers a very broad set of domains which will only be touched upon to gather insights.
- Power
- Digital Divide and Abandonment/Neo-luddites
- Environmental
- Cultural/social
- Economic
44. Dance of the bees...
When bees return to the hive they communicate the location of pollen rich plants to other bees through a
structured dance that apparently transmits a usable mental map. This is perhaps the only known symbolic
language outside mankind.
The Reality Engine model proposed above sets out a dualism between the physical and the symbolic worlds and
suggests that our conscious self is a product of our attention through our neurobiology and sensory input.
Pervasive Computing is literally giving us the ability to sculpt and dynamically re-design our sensory input,
attention and through that neurobiology. To reshape our consciousness/self through Pervasive Experiences.
But the content and answers outlined in this review just open deeper questions. e.g.
Does the increasingly Digital nature of our new attention/self make us any less human at some definable point?
At that point, do we turn into d-zombies? Or could it possibly make our experience even more human and
humane? NOTE: Multi-POV is a structural change to our existing empathy model and may have significant
implications for our Theory of Mind.
The goal now is to continue to explore this model and the d-zombie thought experiment through applied
experimentation. The focus is upon the subjective experience of using Pervasive Computing technologies and the
methodology is based upon Action Research and reflection.
The method involves creating and using Pervasive Experiences and measuring how, if at all, they change the
user's Body Image. The specifics of this study are the human equivalent of the “dance of the bees” and the aim is
to model the user's internal mental maps by observing and manipulating external behaviours and technologies.
46. The rubber hand
The starting point for the planned experimentation is the rubber hand illusion which uses a mirror to mechanically/
physically bend light to change your perception. This altered sensory input is used to fool our Reality Engine into
associating a foreign object into our Body Image. This literally changes our sense of who and where we are.
The Pervasive Experience also allows us to flexibly distort our sensory input to other places and points in time
through networked devices. Using networked cameras, microphones and user interfaces we can digitally bend
sound, light and actions. Using digital sensors we can bend other forms of electromechanical energy to give us
an expanded range of sensory input beyond the normal human range.
The experiment this project is pursuing attempts to merge these two aspects. To use digitally altered sensory
input to fool our Reality Engine into associating foreign & digital objects into our Body Image. One interesting tool
that we are starting to explore for this experimentation is the AR Drone.
By exploring this in different ways it is hoped that the limits of this new form of Extended Mind (as Chalmer's put it
“panprotopsychism”) can be mapped. To contribute to establishing where the limits to this extension may lay.