SlideShare une entreprise Scribd logo
1  sur  46
The Pervasive Experience
 will a second Cognitive Revolution turn us into d-zombies...




                                                          Project Review : July 2010
                                                          roBman@mob-labs.com / @nambor

                                                                       photo by club silencio
what the hell
does that mean?!
what is the Pervasive Experience?
Ubiquitous Computing was pioneered by Weiser (1991) at Xerox Parc.

          “Ubiquitous computing names the third wave in computing,
          just now beginning. First were mainframes, each shared by
          lots of people. Now we are in the personal computing era,
          person and machine staring uneasily at each other across
          the desktop. Next comes ubiquitous computing, or the age of
          calm technology, when technology recedes into the
          background of our lives.”
                                                                      Mark Weiser


That was obviously from a 90's perspective. This project proposes a
structural and ethical argument that Pervasive is a more appropriate
and Humanistic term than Ubiquitous.


So the Pervasive Experience is a new, truly modern Human Condition
that is now possible in today's anytime, anywhere internet world.
what is a Cognitive Revolution?
From Descartes (1637) and the Enlightenment of the 1600's through to
the 1950's and 60's, Positivism was the dominant epistemology in
scientific thought and Western Philosophy.


Built upon this world view, Pavlov (1904) and Skinner (1938) defined
Behaviourism within the field of experimental psychology.


Yet starting in the 1950's the dominant epistemology began to turn
towards Realism. A reaction against Behaviourism became a
movement known as the Cognitive Revolution.


This movement recognised our internal phenomenological experiences
as a valid object of research.
what was that about Zombies?
Chalmers (1995) is an interesting Philosopher that has contributed to
the field of Consciousness.


In order to explore and discuss the concepts surrounding the
phenomenological aspect of Consciousness he has proposed what has
become known as the Philosophical Zombie thought experiment.


In its simplest form it poses the question “if people existed that
behaved exactly like a normal human, but didn't experience the
phenomenon of internal representation – are they still human?”.
These meaty automata like creatures are called Philosophical Zombies
or p-zombies for short.


It can also be adapted to explore similar questions about the concept
of the Soul. Hypothetical humans without Souls are called s-zombies.
so what's the big idea then?
Pervasive Computing could be creating a second Cognitive Revolution.


It is changing the geometry of our sensory input and our presence. It is
changing the structure of our sphere of control. Our very Body Image.


I believe Pervasive Computing is adding an interesting new dimension
to Chalmers' Philosophical Zombie thought experiment...lets give the
theoretical zombies in a Pervasive Computing Experience a name.


Lets call them Digital Zombies or d-zombies


These are people whose experience of the phenomenon of internal
representation is more than a certain percentage digital, in some way.
People whose sensory experience is digitally mediated beyond a
defined d-zombie threshold.
this raises some interesting questions
  Does this d-zombie threshold exists at all.


  When we reach a point where our sensory input is more digitally
  mediated than solely biologically mediated, can our conscious
  subjective experience somehow become less Human and more
  like a d-zombie?


  Can it lead us to become d-zombies in ANY way at ANY point at all?


  What if our attention itself became incrementally digital?
  (See the Pervasive Reality Engine defined below)
contents

The Project : An introduction and brief history

Expanding my view : The project's new focus

Reality Engines : The key model proposed

Philosophical context : The related body of knowledge

Ubiquitous vs. Pervasive : Pinning down key terms

Reading the Web : Seeing the web as a digital prosthetic

The Pervasive Experience : Reflections on this project

What's next? : Where to from here

Experimental Technology : Rubber hands and AR helicopters
The Project




              photo credit
The Pervasive Experience Project
Hold your hand up and away from you. Imagine your fingertips are dipped in vomit! Feel how dirty and contaminated your fingers are.
Can you sense changes in how you hold your body image? Take a moment to feel how this part of your body holds your attention.
Even if you look away it seems as if your hand is tagged with meaning and still to be attended to.

This positive, lead hallucination provides you with your own subjective experience of a traditional "real world" Body Image.
Biological, grounded, open to symbolic manipulation...yet essentially "real".

By contrast, this document reviews the Pervasive Experience project. In this project the driving assumption is that
increasingly pervasive, networked technologies are impacting our lives. The research question is:

 How is Pervasive Computing changing you?

This applied research explores how Pervasive Computing may impact our Philosophy of Mind. It looks at our
perception of space and self, and their underlying Neurobiology. It aims to integrate theories from these four
broad areas in a high-level, structural way. Its aim is to explore through practice and reflection how these theories
may be able to be reshaped by the pervasive diffusion of digital networked technologies.

Based on this analysis, an integrated, pragmatic, working model of the end-to-end process is synthesised - from
the environment, body, nervous system and brain to sensory input, representation, consciousness and cognition.
This model represents a Reality Engine. This document aims to frame this model within the existing body of
knowledge, then extend it to support the rapidly growing, digital environment of Pervasive Computing.

I believe this model opens a new perspective on network enabled devices where the web itself can be seen as a
form of digital prosthetic that is increasingly pervading our experience and becoming more organ-ised.

So after completing this review I now believe the most likely answer to this research project's question may be:

 By driving rapid Body Image modification.

This could well be the defining structural process of Pervasive Computing...
Methodologies
Over the past two years I have been exploring the impact of Pervasive Computing upon my sense of space using
Action Research and reflecting through the lenses of Diffusion of Innovations and Distributed Cognition.


Action Research:
As defined by Dick (1999), Action Research is a "cyclic or spiral process" that "alternates between action and
critical reflection". This review document represents the output of one of these critical reflection cycles.


Diffusion of Innovations:
Initially defined by Tarde (1890) the theory of Diffusion of Innovations was crystalised in the seminal work by
Rogers (1962). It aims to describe and analyse why and how ideas and technologies diffuse through cultures,
especially the rate of diffusion.


Distributed Cognition:
As defined by Hutchins (1995), Distributed Cognition is a framework involving individuals, artefacts and the
environment. It is strongly based upon Vygotsky's (1920's) work that led to the development of Activity Theory.

       Distributed Cognition that takes as its unit of analysis a culturally constituted functional group
       rather than an individual mind. This theory is concerned with how information is propagated
       through a system in the form of representational states of mediating structures. These
       structures include internal as well as external knowledge representations, (knowledge, skills,
       tools, etc.). This approach permits us to describe cognitive processes by tracing the movement
       of information through a system and characterize the mechanisms of the system which carry out
       the performance, both on the individual and the group level.
                                                                                            Hutchins (1995)
Expanding my view




                    photo credit
The bigger picture
This research has now been extended to include the lenses of Situational Awareness, Neuroplasticity and
Sensorimotor Theory. This change has expanded my research focus to include “consciousness” and "self" and
their underlying "Neurobiology". However this creates such a broad and technically deep domain that the goal is
simply to “integrate current theories from these areas at a purely structural level in relation to the Reality Engine”.

I have also been aware of Gibson's (1992) work on Visual Perception for a long time now but have struggled to
integrate it fully into this work. Recently I came across O'Regan's (2010) work on Sensorimotor Theory, which
seems to develop a situated aspect of Gibson's work within the modern context. To me this work provides a
strong foundation for a wide range of discussions from Action Research through to Philosophy of Mind.

I have also been researching Bach-y-Rita's (1969) work on Sensory Substitution that lead to the development of
the field of Neuroplasticity. This work leads to some amazing predictions that can be tested through direct
experience as shown by Bach-y-Rita, Ramachandran (2007) and Ehrsson (2010).

As you will see below I have found that what I am exploring appears to be built upon a form of
Neo-Cartesian Dualism. Decartes' statement in Principals of Philosophy, “I think, therefore I am” (Cogito ergo
sum) are a compelling fit for the traditional Body Image experience of our reality. Yet, in the Pervasive,
Neruoplastic, digitally networked Body Image experience of our proximate future we may be better supported by
the philosophical statement “I am what I think” (EGO sum quis EGO reputo?) or perhaps more explicity “I am the
sum of my thoughts”. Not in a frivilous “positive thinking” kind of way, but a “my thinking alters digital hardware
and software” kind of way. This is not contradicting Descartes work, but reflecting on it from this new angle.

If convergence/divergence continues to develop so our sensory input is increasingly digitally mediated then the
relevance of this statement should become even more significant over time. Distributed, interactive software and
hardware literally becomes part of our Extended Mind.

This is where the d-zombie questions come in.
definition: Situational Awareness




Situational Awareness is the awareness of what's going on around you. How you make sense of information,
events and actions. How well you understand how they'll effect your goals and objectives. Related concepts such
as Sensemaking take a more historical perspective and Situational Awareness tends to have a now and near
future perspective.

Inadequate Situational Awareness is seen as one of the key factors in accidents through human error. Especially
in roles where there are serious consequences (e.g. drivers, soldiers, pilots, doctors, etc.).
                                                                                                     diagram credit
definition: Neuroplasticity

              The concept was initially proposed by James (1907),
              yet the term Neuroplasticity wasn't coined until (1948)
              by Konorski.

              Neuroplasticity was first experimentally demonstrated
              by Bach-y-Rita (1969). His work in Sensory Substitution
              suggested that the brain can rewire input from one
              sensory mode into internal representations in another
              mode. Not just in a form of synaesthesia, but by
              completely re-wiring the cortical maps.

              So at a time when the Cartesian homunculus appears
              to be identifiable within our physical neurobiology in the
              form of the cortical maps, we also appear to be finding
              that this homunculus is also plastic, malleable and
              changeable based on action and even thought alone.

              Neuroplasticity is also revolutionary because in some
              ways it brings cutting edge neurological research within
              the reach of anyone with a working brain. Adding in the
              Augmented Reality technology I work with everyday
              and you have a powerful set of research tools.




                                                            photo credit
definition: Sensorimotor Theory




         "The sensorimotor view conceives of feel ("subjective experience") as a way of interacting with
         the environment. The quality of feel is simply an objective quality of this way of interacting."
                                                                                           O'Regan (2010)


I believe this provides support for the view that the self is the mental model that integrates input from other models
and senses. It is a by-product of the creation of attention. By it's very definition, the sensorimotor view “requires”
attention. I believe the emergent property of this is the self. A situated, attentive feedback loop.

This perspective is expanded further below.
                                                                                                          diagram credit
Reality Engines




                  photo credit
Descartes' Reality Engine
                             To provide a starting point lets look at Descartes
                             proposal for a dualistic reality.

                             “I have a clear and distinct idea of myself as a thinking,
                             non-extended thing, and a clear and distinct idea of
                             body as an extended and non-thinking thing.”

                             So, Descartes argues, “The mind, a thinking thing, can
                             exist apart from its extended body. And therefore, the
                             mind is a substance distinct from the body, a substance
                             whose essence is thought.”

                             As we will see, this initial proposition has fragmented
                             into a wide range of Dualisms with different structures
                             and theories.

                             This project takes an Interactionist Dualism as it's
                             starting point.




Descartes' Dualism Diagram
                                                                            diagram credit
The Reality Engine
                                                                                 elements of the diagram
                                                                                 a/b = concepts
                                                                                 self = center of attention
                                                                                 environment = mental model of the world
                                                                                 Body Image = mental model of your body
                                                       X <- you are here
                                                                                 ROM = rest of mind
                                                                                 brain = the biological organ
                                                                                 ROB = rest of body
                                                                                 ROW = rest of world

                                                                                 ROB <-> brain = nervous system
                                                                                 brain <-> Body Image/a/b = perception
                                                                                 Body Image <-> self = attention


                                                                                 sensory perception is dualistic
                                                                                 ● sensory input is one world

                                                                                 ● perception is the other




This diagram describes the single integrated Reality Engine model proposed by this project. A working system
whose feedback loop creates a cognitive conscious self that interacts with the perceived Real World.

Our brains seem to crave sustainable models that turn sensory input into actionable behaviour. Feedback loops
that integrate all the aspects of sensory input (internal and external!?) and physical action into an attentive self.

To me they appear from the outside to be a rich ecosystem of self-reflexive Hidden Markov Models (ironic?).
The Reality Engine extended
                                                                              additions to the diagram
                                                                              a = subconscious interactive concept
                                                                              a1 = subconscious mental concept
                                                                              a2 = subconscious body concept
                                                     X <- you are here
                                                                              b = conscious interactive concept
                                                                              b1 = conscious mental concept
                                                                              b2 = conscious body concept


                                                                              so what is the “self”?
                                                                              ● reality is rendered through attention

                                                                              ● attention is the integration of sensing,

                                                                                 reacting, remembering, thinking, doing
                                                                              ● the by-product of this is the self...

                                                                                because attention needs a point of origin




A concepts are subconscious and can't be accessed by the self.
B concepts are conscious and may be accessed by the attentive self.

1 concepts (A1 & B1) are mental only and have no direct relationship with the body.

2 concepts (A2 & B2) are concepts that are pre-formed within the body itself before the sensory data is sent
through the nervous system to the the brain. This includes things like reflexes and muscle memory.
Mixing Realities



                                          Milgram's Virtuality Continuum
If the input for this Reality Engine is substituted with completely digitally produced vision and sound, etc. then this
becomes a Virtual Reality Engine (VR).

And if we only overlay digital elements onto our perception of the Real World then this becomes an Augmented
Reality Engine (AR). While VR and Artificial Intelligence were an aspirational focus of the original Cognitive
Revolution, it seems likely from todays perspective that AR will continue to pervade our experience...but in folded
and bent ways.

In this case, pixels (or other sense based equivalents e.g. sound) can provide a good measure of how much
digital sensory mediation has pervaded a person's Body Image experience. That is, “What percentage of our total
sensory input is digitally mediated?”. In Augmented Reality the pixels that make up the overlaid information can
be counted to calculate this ratio based on our total field of visual sensory input.

Within this data is an important sub-set that is not just how much of our environmental sensory input is digitally
augmented, but how much of our attention is digitally mediated. Through the lenses of Distributed Cognition, and
Situational Awareness it's natural to ask “how much of our attention, and therefore self, can be Digital without
turning us into d-zombies?”.

Is there any limit to this digitally Extended Mind?
Multiplayer Realities




                                            Multiple Reality Engines

A useful set of terms breaks the world into the Geosphere (physical earth/planets), the Biosphere (living things)
and the Noosphere (symbolic or thought related things). The Pervasive Experience is transforming the
Noosphere in terms of how interconnected it is. The relationship between space and even the point of origin of
thoughts and symbols is being completely reformed. While in the traditional world each “self” has been cacooned
within their own Cartesian isolation chamber, the Pervasive Experience may be opening the door to change this.

Shannon's (1948) work on a Theory of Communication has some interesting application here and is an application
of Hidden Markov Models. Symbolic communication between two Reality Engines seems to involve building a
Theory of Mind of the “other” and creates a pragmatic use of encoding/encryption in a Shannon Information sense.
Body Image
Based upon the full Reality Engine model proposed above, the term Body Image can be used to define the
specific mental model used to represent our body. It can also more flexibly be seen to represent all the mental
models that go in to making up our entire “self” and the broader context we exist within. Our entire Self Image and
the contexts that it implies.

Often, philosophical dualistic thought experiments and discussions isolate “special” dis-embodied beings to
explore or prove a point. From the perspective of this project this is a non-sense. A “dis-embodied experience” is
an altered form of consciousness outside the Sensorimotor world view. Even hallucination or meditation upon
“mental only” (e.g. B1) concepts are spatially related to the Body Image. The word “internal” is often used to
describe these types of phenomenon.

In this way attention is a situated experience (or feedback loop) that requires (or perhaps generates) a point of
origin.

Interestingly, early research into Attention during the first Cognitive Revolution was focused upon “divided
attention” (e.g. dichotic listening).

By contrast, the experiments of this research project are focused on in some senses the opposite. The goal here
is to integrate objects from two (or more) places/times into a single coherent attention, instead of attempting to
split attention into two (or more) separate streams.

I believe this is the complete reformation of the concept of Body Image that the Pervasive Experience is enabling.
Philosophical context




                    photo credit
Dualisms




In order to relate my reflections from this research review to the existing body of knowledge I have developed the
following map of what appears to be the related philosophical terrain. I am also open to new perspectives in this
area and reviewing new concepts and frameworks drives a lot of my reflection. Here's the map so far.

Cartesian Dualism:
This philosophy of mind separates reality into a dual system of mind and body. Descartes' work was an extension
of thought that dates back to Plato and Aristotle. This separation into physical and non-physical is also often
bound up with a spiritual discussion of the soul as well.

Interactionist Dualism:
Interactionism is the form of Dualism that proposes that the mind and body can interact bi-directionally in a causal
way. While it is often stated in the literature that it “is difficult” to establish how this may occur, I find it almost self
evident. fMRI's, etc. clearly show that brain activity is tied to symbolic phenomenal experience. Neuroplasticity
takes this one step further stating that the phenomenal experience alone can persistently change neruobiology.

Interactive Dualism:
There is also an area of Interaction Design/CHI where Theory of Mind is related to interactions with any dynamic
system. For example Norman (1988).                                                                      diagram credit
Neurobiology
Neurobiology is an amazingly deep and technically complex area. This literally includes “brain surgery” and is
dealing with the most complex structure in the known universe.

This project focuses on the perspective of Cognitive Neuroscience and is primarily focused upon the theoretical
frameworks available that show how cognitive processes can impact the biological structures within the brain.
Neuroplasticity is the key lense used for this reflection and literature review.

I believe it is important to include this from at least a high-level in order to fully explore the Interactionist Dualism
foundations of the Reality Engine. I think it is also very relevant for the phrasing of the research question “How is
Pervasive Computing changing you?”.

There is also likely to become more closely integrated into the overall Pervasive Computing domain with the rise
of Computer/Mind interfaces. This is also closely related to the Digital Prosthetics content discussed below.

Interestingly there are some mechanical devices/technologies (e.g. TMS) that are widely available and are actively
being used to map and alter specific neruobiologies. Personally I find TMS quite frightening and think that this
project specifically raises some serious ethical issues that relate directly to it's use. (e.g. can mechanically
changing the operation of certain Neurobiological structures change your consciousness to the point of creating a
d-zombie? Could it be used as a literally de-humanising experience?). Not to mention the unknown long term
impacts of high-intensity EMF on delicate and critical cell structures.
The Mind
Philosophy of Mind:
This is a very broad and rich area. Dennett's (1991) work breaking the philosophy of mind into two problems
“consciousness” and “content” is interesting. His Neural Darwinism perspective is also useful and relevant for this
project.

Chalmers' work on consciousness, especially his p-zombie thought experiment is obviously relevant.

Theory of Mind:
The ability to form models that attempt to predict the behaviour of objects, animals and other behaviours is a
critical conscious activity. Interestingly, the fact that people form Theories of Mind implies the belief that a
representation system inside another's mind can causally interact to create actions.

Cognitive Revolution:
Essentially this was a rebellion against an automata style view of behaviourism and a recognition of the
phenomenological aspects of the human experience. Interestingly, in 1950 Turing published "Computing
machinery and intelligence" in the journal “Mind”. His proposal was that you could use questions and answers to
test machines for intelligence is often included as part of the Cognitive Revolution. His view here appears to be
that we can use our own innate Theory of Mind models (B1 concepts) to interrogate an “other” via a text based
interface to determine if it is Human or synthetic.

This highlights a consciousness continuum (or perhaps even triangle) between biological entities and
synthetic/mechanical entities.
Functionalist Hidden Markov Models




Functionalism:
Philosophical Functionalism provides a post-Cartesian perspective that is highly relevant to the Pervasive
Experience and it's digital-isation. However, the Reality Engine provides a perspective where Functionalism and
Interactionist Dualism need not be mutually exclusive. I believe it is possible for Functionalism to exist at the
physical/material Neurobiological level to create a subjective experience that is Dualistic and Interactionist in
nature. In fact I believe that the two are perfectly fitting sides of the same coin and that the nature of attention/self
requires both.

Hidden Markov Models:
The Cartesian cacoon that the Reality Engine suggests we all live within also suggests that the
Hidden Markov Model is a useful metaphor. This creates a nice, ironic, self-recursive Solipsistic loop

Shannon's Theory of Communication:
Shannon's (1948) work on a “Mathematical Theory of Communication” is also highly relevant to this discussion.
For example his “Fig 1.” is almost identical to the Multiplayer Reality Engine diagram above.
                                                                                                             diagram credit
Ubiquitous vs. Pervasive




                           photo credit
Weiser's Ubiquity




My starting point for the whole project was a diagram created by Mark Weiser in (1996) - see above.

My interpretation of this diagram is that the left and right elements on the first row contrast non-Interactionist
Cartesian Dualism (left) with Situated Cognition (right). The labels beside clearly state that Weiser believed a
non-Interactionist Dualism was "wrong".

The second row of left and right elements contrast Current Computing Technology with Ubiquitous Computing
Technology. My interpretation here is that Weiser was showing how Computing Technology will pervasively
spread throughout more and more aspects of our Situated reality, as opposed to the unipresent Desktop
Computing model that dominated the 1980's and 90's.
Spatial Maps
Review Edward T. Hall et al's work on Personal Space.   Weiser's diagram is obviously focused upon the spatial
Relate this to the reality engine.                      relationship between you and the world. All Cartesian
                                                        like dualisms appear to have this spatial aspect.

                                                        The spatial models we hold can also be broken down
                                                        into finer grained regions. At the highest level of the
                                                        physical world is Inside and Outside our body.

                                                        This implies a Body Image and surrounding that, and
                                                        inter-meshed within that, is our overall sense of space.
                                                        Beyond proprioception are our exteroceptive senses
                                                        that collect stimulus from the outside world. From this
                                                        we build models about these spaces and relationships.

                                                        Hall's (1966) work on Personal Space provides a useful
                                                        framework for a User Centred world view.

                                                        These concepts also have a pragmatic relationship to
                                                        Situational Awareness too.

                                                        The Reality Engine aims to ensure Hall's work
                                                        integrates with Cartesian Dualism, Extended Mind,
                                                        Distributed Cognition and Situational Awareness.
Divergence




Applying this type of spatial analysis to the concept of Convergence has also led me to some interesting
reflection. I believe Divergence is the by-product of Object:Network Convergence.

As the distance between an Object and the Network passes a point of Convergence the distance between the
“Situated Point of Origin” of that Object and “Control” of that Object Diverges. In other words, once you connect
an Object to the Network you're free to remove its “Physical Interface” and simply “Control” it via the Network.

This is exemplified by the marketing term “Anywhere, Anytime”.
Putting YOU at the centre
I think that the framework above, especially the Pervasive Web integrated into the Reality Engine, highlights why
Pervasive is a more appropriate term than Ubiquitous.

Ubiquitous suggests that the technology is “everywhere”. While it may at first seem like that's the case, in reality I
believe we are discussing how this technology is soaking into our attention.

If you look at mobile network coverage maps they don't cover the whole world, just the parts where the majority of
the population are likely to be. This is a natural evolutionary pattern that allows for the conservation of energy.

Pervasive suggests the technology is soaking into “your” subjective experience - with you at it's heart.

Ubiquitous on the other hand suggests technology is soaking into the universe - whether you are there or not.

To me this appears to be an important structural and ethical distinction.

The next section aims to share some insights that should clarify why this may become even more important as
convergence/divergence continues to develop. Where distributed, interactive software and hardware literally
becomes part of our extended self. If the impact of Moore's law and the Network Effect (Metcalfe's law) keep
driving the Pervasive Network then I believe we need to make these technologies an extension of our “self”, or risk
becoming diluted and de-Humanised.

Again, this is where the d-zombie questions come in.
Reading the Web




                  photo credit
Prosthetics




Prosthetics are traditionally defined as an addition to the body (coming from the greek word for addition) that
replaces a missing body part. If it's related to mobility or a directly sense-able part of our Body Image then it is
likely to be a limb.

If it is to do with sub-conscious processes then it is likely to be an organ and may include real or artificial organs.
This analysis is blurred even further when the prosthetic is a sensing organ that impacts your Body Image.



                                                                                                             photo credit
Digital Prosthetics




Now, there also exists the real opportunity to utilise Digital Prosthetics that enhance a person's sensory input and
even cognitive capabilities like memory. This obviously heads us into the well populated cyborg territory.

While the traditional focus of Prosthetics was upon repair and replacement, the Digital Prosthetic view of the
Pervasive Reality Engine is squarely focused upon “beyond repair” and towards “enhancement”.

Jamais Cascio presents a perspective that many people will be facing over the next few decades as their options
turn from repair to enhancement. This is also an interesting convergence of Digital Technologies, Medical
Miniaturisation and an Aging Population.
The Pervasive Web




It's amazing to see how the web has expanded from a single point on the planet (CERN) out to becoming a
pervasive cultural concept. In fact it could be seen to have initially existed as just a model within
the Reality Engine of Tim Berners-Lee.

Now it seems to be a model that exists in almost everyone's Reality Engine and it has distorted how they use and
think of space, place and time.

Pervasive Web devices can then be seen as a form of Digital Prothstetic and the Web itself is pervading
throughout our planet and our experience. Literally soaking into our consciousness.

However, it's in a unique class of mental models that not only cement themselves into Reality Engines, they
provide networked access and become the point of exchange for other models too.
The Pervasive Experience




                     photo credit
First Reflections
The heart of my recent reflection has been upon Neuroplasticity and Situational Awareness and how they relate to
the concept of Personal Space and Theory of Mind in light of the Reality Engine. Extend this to interacting with an
increasing number of web applications and web enabled devices and you have a Pervasive Reality Engine.

The addition of Pervasive Computing to the Reality Engine creates a structural change to the models that defined
the Cognitive Revolution. While the original Cognitive Revolution was focused upon what happened within the
brain and mind, the Pervasive Reality Engine is focused upon what happens to us as our self moves
out of our brain and possibly even out of our mind if that is achievable (imagine extending the A2 & B2 concepts in
the Reality Engine using remote sensors). To use buzz terminology, this is Cognitive Revolution 2.0.

So it appears we have a relatively mature conceptual framework for the individual mind. At the other extreme it
also appears that the impact of interactive networks have been explored broadly from the macro, national scale.
Now seems like the perfect time to integrate these two perspectives into an exploration of the impact of pervasive
networks on the mind. This exploration leads to some interesting questions.

If consciousness could be a property achieved by any system that can create the right type of attentive self
awareness feedback loop, then we as humans could “safely” extend our Human experience using digital
technology. This would benefit us by allowing us to tag along with rapidly evolving digital technology. A chance
to keep up with any possible proximate Technological Singularity.

If not, then this has obvious implications for the ethics of technology and the discussion of the d-zombie threshold
and the possibly de-Humanising nature of technology.

However, if it were true then this has implications for the ethics of machine intelligence and even biology. If
consciousness were an achievable property of the right type of attentive loop then any “thing” that displays that
type of loop is essentially “equivalent to a self conscious human being” - at least at a category level.

Interestingly, either way there is an impact.
Content Reflection
Pervasive Computing is a movement driven by the structural reformation of our Reality Engines through
convergence/divergence. I believe it can be shown to enable the following measurable changes:

       –   rapid Body Image modification
       –   flexible Body Image design and re-engineering/sculpting
       –   dynamic multi-POV Body Image experiences
I am confident now that the content outlined above provides a robust theoretical and practical framework for
imagining, designing and building rich, interactive and "world changing" Pervasive Experiences.

This model obviously needs to be reviewed, tested and revised on a regular basis, however it provides a workable
starting point for some truly innovative experimentation.
Context Reflection
My personal experience of this Action Research project has been an interesting, stimulating and very challenging
one. The breadth of content and reading has been quite challenging, as has been keeping up with the constant
pace of change on the technology side. Luckily, my day-to-day work keeps me focused upon this type of change.

There are a number of limitations of my work that have been pointed out by the people that have been kind
enough to review it. The first is that it is almost exclusively from a western thought perspective. The world is
obviously full of a broad range of interesting and insightful perspectives that don't fit into that bucket. I hope to at
least start to address this in the next phase. I have also received feedback stating that this work appears strongly
philosophically monistic. Personally, I don't believe that the output is quite that simple. However the density of
the content here masks some of the subtleties. I hope I can communicate this aspect of the project more clearly
in the next phase and aim to make my stance here more transparent.

I have also receive feedback on how overwhelming and sometimes un-approachable this document can be. I've
made a number of revisions that have hopefully addressed some of this, but this is a continual challenge and I am
continuing to attempt to distill this down to a simpler set of key messages without diluting the depth of the work.

While this work is obviously strongly academic in nature, I am completing it independently from any academic
institution as an ongoing attempt to develop my practice. I have learned and grown a lot through the process, but
obviously a more formal context may have helped me stay more focused. Luckily I've been able to seek
constructive feedback and support from some academic friends that I'll thank in due course.

On the whole I feel I've achieved something of real value and I'm comfortable that I now have a good framework
with which to begin my experiments. This is where the project gets really interesting.
Ethical Reflection
There is an obvious commercial bias I have in my world view. I work as a Technologist for a Research Lab that
helps it's clients commercialise new technologies. While my company (MOB) are happy to support this research
project, both the company and I obviously benefit from the output of this project. However naive this may be, my
underlying perspective is not so much that “I do this research because my job is closely related”, but more “I am
good at my job because I am naturally drawn to a research project like this”. While this may be a subtle
difference, I believe it has significant implications to the overall work.

The questions raised by this research project are essentially Humanist in nature and are exploring the continuum
between today's common Human Condition and the other extreme of Artificial Intelligence. While traditional AI
starts at a synthetic point and heads towards consciousness, this project starts with the User and pushes the
boundaries in the other direction. The Ethical focus of Humanism sets out the framework for this reflective
process.

The logical conclusion of this reflection is the structural proposition for a new terminology with Pervasive being
more appropriate and Humanistic than Ubiquitous.

Further and deeper reflection upon the ethical aspects of this project as it relates to the following topics is also
planned. These obviously covers a very broad set of domains which will only be touched upon to gather insights.

- Power
- Digital Divide and Abandonment/Neo-luddites
- Environmental
- Cultural/social
- Economic
What's next?




               photo credit
Dance of the bees...
When bees return to the hive they communicate the location of pollen rich plants to other bees through a
structured dance that apparently transmits a usable mental map. This is perhaps the only known symbolic
language outside mankind.

The Reality Engine model proposed above sets out a dualism between the physical and the symbolic worlds and
suggests that our conscious self is a product of our attention through our neurobiology and sensory input.

Pervasive Computing is literally giving us the ability to sculpt and dynamically re-design our sensory input,
attention and through that neurobiology. To reshape our consciousness/self through Pervasive Experiences.

But the content and answers outlined in this review just open deeper questions. e.g.
Does the increasingly Digital nature of our new attention/self make us any less human at some definable point?

At that point, do we turn into d-zombies? Or could it possibly make our experience even more human and
humane? NOTE: Multi-POV is a structural change to our existing empathy model and may have significant
implications for our Theory of Mind.



The goal now is to continue to explore this model and the d-zombie thought experiment through applied
experimentation. The focus is upon the subjective experience of using Pervasive Computing technologies and the
methodology is based upon Action Research and reflection.

The method involves creating and using Pervasive Experiences and measuring how, if at all, they change the
user's Body Image. The specifics of this study are the human equivalent of the “dance of the bees” and the aim is
to model the user's internal mental maps by observing and manipulating external behaviours and technologies.
Experimental Technology
                          photo credit
The rubber hand
The starting point for the planned experimentation is the rubber hand illusion which uses a mirror to mechanically/
physically bend light to change your perception. This altered sensory input is used to fool our Reality Engine into
associating a foreign object into our Body Image. This literally changes our sense of who and where we are.

The Pervasive Experience also allows us to flexibly distort our sensory input to other places and points in time
through networked devices. Using networked cameras, microphones and user interfaces we can digitally bend
sound, light and actions. Using digital sensors we can bend other forms of electromechanical energy to give us
an expanded range of sensory input beyond the normal human range.

The experiment this project is pursuing attempts to merge these two aspects. To use digitally altered sensory
input to fool our Reality Engine into associating foreign & digital objects into our Body Image. One interesting tool
that we are starting to explore for this experimentation is the AR Drone.

By exploring this in different ways it is hoped that the limits of this new form of Extended Mind (as Chalmer's put it
“panprotopsychism”) can be mapped. To contribute to establishing where the limits to this extension may lay.

Contenu connexe

Tendances

Mind computer societies
Mind computer societiesMind computer societies
Mind computer societiesFranco Bagnoli
 
Transhumanism and Religion
Transhumanism and ReligionTranshumanism and Religion
Transhumanism and ReligionGiulio Prisco
 
Education, technologies, cognition: triangulations possibles
Education, technologies, cognition: triangulations possiblesEducation, technologies, cognition: triangulations possibles
Education, technologies, cognition: triangulations possibleselena.pasquinelli
 
The hard problem of consciousness
The hard problem of consciousnessThe hard problem of consciousness
The hard problem of consciousnessMasatoshi Yoshida
 
Why the "hard" problem of consciousness is easy and the "easy" problem hard....
 Why the "hard" problem of consciousness is easy and the "easy" problem hard.... Why the "hard" problem of consciousness is easy and the "easy" problem hard....
Why the "hard" problem of consciousness is easy and the "easy" problem hard....Aaron Sloman
 
Simulation theory says more about human nature than it does the universe.
Simulation theory says more about human nature than it does the universe.Simulation theory says more about human nature than it does the universe.
Simulation theory says more about human nature than it does the universe.Eric Schmidt
 
Preparing for post-human audience: a digital artwork survival toolkit
Preparing for post-human  audience:  a digital artwork  survival toolkitPreparing for post-human  audience:  a digital artwork  survival toolkit
Preparing for post-human audience: a digital artwork survival toolkitKovács Balázs
 
3 a cognitive heuristic model of community recognition final
3 a cognitive heuristic model of community recognition final3 a cognitive heuristic model of community recognition final
3 a cognitive heuristic model of community recognition finalAle Cignetti
 
Mind in a designed world
Mind in a designed world Mind in a designed world
Mind in a designed world Karlos Svoboda
 
Futures of a Complex World - Futures of Education
Futures of a Complex World - Futures of EducationFutures of a Complex World - Futures of Education
Futures of a Complex World - Futures of EducationHeiner Benking
 
Minds and Spaces by Prof. Shalini R. Urs
Minds and Spaces by Prof. Shalini R. UrsMinds and Spaces by Prof. Shalini R. Urs
Minds and Spaces by Prof. Shalini R. UrsNIFT
 
The Dawn of the Suprasystem
The Dawn of the SuprasystemThe Dawn of the Suprasystem
The Dawn of the SuprasystemAna Soric
 
Networked Creativity: How optimal experience shapes creative networks
Networked Creativity: How optimal experience shapes creative networksNetworked Creativity: How optimal experience shapes creative networks
Networked Creativity: How optimal experience shapes creative networksUniversità Cattolica del Sacro Cuore
 
Neuroeconomics Critique Part 2
Neuroeconomics Critique Part 2Neuroeconomics Critique Part 2
Neuroeconomics Critique Part 2sderbysh1
 
Contextual Landscaping Project Zero
Contextual Landscaping Project ZeroContextual Landscaping Project Zero
Contextual Landscaping Project ZeroAna Soric
 

Tendances (20)

Transformative experience design
Transformative experience designTransformative experience design
Transformative experience design
 
Mind computer societies
Mind computer societiesMind computer societies
Mind computer societies
 
Pn1
Pn1Pn1
Pn1
 
Mind compass.b copy
Mind compass.b copyMind compass.b copy
Mind compass.b copy
 
Transhumanism and Religion
Transhumanism and ReligionTranshumanism and Religion
Transhumanism and Religion
 
Prosthetic lecture.a copy 5
Prosthetic lecture.a copy 5Prosthetic lecture.a copy 5
Prosthetic lecture.a copy 5
 
Education, technologies, cognition: triangulations possibles
Education, technologies, cognition: triangulations possiblesEducation, technologies, cognition: triangulations possibles
Education, technologies, cognition: triangulations possibles
 
The hard problem of consciousness
The hard problem of consciousnessThe hard problem of consciousness
The hard problem of consciousness
 
Why the "hard" problem of consciousness is easy and the "easy" problem hard....
 Why the "hard" problem of consciousness is easy and the "easy" problem hard.... Why the "hard" problem of consciousness is easy and the "easy" problem hard....
Why the "hard" problem of consciousness is easy and the "easy" problem hard....
 
Simulation theory says more about human nature than it does the universe.
Simulation theory says more about human nature than it does the universe.Simulation theory says more about human nature than it does the universe.
Simulation theory says more about human nature than it does the universe.
 
Time
TimeTime
Time
 
Preparing for post-human audience: a digital artwork survival toolkit
Preparing for post-human  audience:  a digital artwork  survival toolkitPreparing for post-human  audience:  a digital artwork  survival toolkit
Preparing for post-human audience: a digital artwork survival toolkit
 
3 a cognitive heuristic model of community recognition final
3 a cognitive heuristic model of community recognition final3 a cognitive heuristic model of community recognition final
3 a cognitive heuristic model of community recognition final
 
Mind in a designed world
Mind in a designed world Mind in a designed world
Mind in a designed world
 
Futures of a Complex World - Futures of Education
Futures of a Complex World - Futures of EducationFutures of a Complex World - Futures of Education
Futures of a Complex World - Futures of Education
 
Minds and Spaces by Prof. Shalini R. Urs
Minds and Spaces by Prof. Shalini R. UrsMinds and Spaces by Prof. Shalini R. Urs
Minds and Spaces by Prof. Shalini R. Urs
 
The Dawn of the Suprasystem
The Dawn of the SuprasystemThe Dawn of the Suprasystem
The Dawn of the Suprasystem
 
Networked Creativity: How optimal experience shapes creative networks
Networked Creativity: How optimal experience shapes creative networksNetworked Creativity: How optimal experience shapes creative networks
Networked Creativity: How optimal experience shapes creative networks
 
Neuroeconomics Critique Part 2
Neuroeconomics Critique Part 2Neuroeconomics Critique Part 2
Neuroeconomics Critique Part 2
 
Contextual Landscaping Project Zero
Contextual Landscaping Project ZeroContextual Landscaping Project Zero
Contextual Landscaping Project Zero
 

Similaire à The Pervasive Experience - project review July 2010

The Methodological Base Of Cybernetic Reasoning
The Methodological Base Of Cybernetic ReasoningThe Methodological Base Of Cybernetic Reasoning
The Methodological Base Of Cybernetic ReasoningKrystal Ellison
 
PhD panel: Uses and abuses
PhD panel: Uses and abusesPhD panel: Uses and abuses
PhD panel: Uses and abusesJennifer Jones
 
Body and Embodiment: Media Extension, Disembodiment, and the Cyborg
Body and Embodiment: Media Extension, Disembodiment, and the CyborgBody and Embodiment: Media Extension, Disembodiment, and the Cyborg
Body and Embodiment: Media Extension, Disembodiment, and the CyborgElizabeth Gartley
 
Marshall Mc Luhan - The extensions of man
Marshall Mc Luhan - The extensions of manMarshall Mc Luhan - The extensions of man
Marshall Mc Luhan - The extensions of manFrancesco Parisi
 
Overbeeke2007
Overbeeke2007Overbeeke2007
Overbeeke2007swaipnew
 
A Review Of Artificial Intelligence
A Review Of Artificial IntelligenceA Review Of Artificial Intelligence
A Review Of Artificial IntelligenceMandy Brown
 
Understanding Computers and Cognition
Understanding Computers and CognitionUnderstanding Computers and Cognition
Understanding Computers and CognitionR. Sosa
 
Analytical-frameworks - Methods in user-technology studies
Analytical-frameworks - Methods in user-technology studiesAnalytical-frameworks - Methods in user-technology studies
Analytical-frameworks - Methods in user-technology studiesAntti Salovaara
 
Philosophy of Technology
Philosophy of TechnologyPhilosophy of Technology
Philosophy of TechnologyMelanie Swan
 
Do Intelligent Machines, Natural or Artificial, Really Need Emotions?
Do Intelligent Machines, Natural or Artificial, Really Need Emotions?Do Intelligent Machines, Natural or Artificial, Really Need Emotions?
Do Intelligent Machines, Natural or Artificial, Really Need Emotions?Aaron Sloman
 
Just How Free Is The Question Of Free Will For Many Millennia
Just How Free Is The Question Of Free Will For Many MillenniaJust How Free Is The Question Of Free Will For Many Millennia
Just How Free Is The Question Of Free Will For Many MillenniaRebecca Buono
 
Flickr, Photosynth, And Strange Loops
Flickr, Photosynth, And Strange LoopsFlickr, Photosynth, And Strange Loops
Flickr, Photosynth, And Strange LoopsBrian McNely
 
Can abstraction lead to intelligence?
Can abstraction lead to intelligence?Can abstraction lead to intelligence?
Can abstraction lead to intelligence?Dr Janet Bastiman
 
Educational Robotics Between Narration and Simulation
Educational Robotics Between Narration and SimulationEducational Robotics Between Narration and Simulation
Educational Robotics Between Narration and SimulationAlessandri Giuseppe
 
Software update for human brain, at a large scale
Software update for human brain, at a large scaleSoftware update for human brain, at a large scale
Software update for human brain, at a large scale2co
 

Similaire à The Pervasive Experience - project review July 2010 (20)

Are robots present?
Are robots present?Are robots present?
Are robots present?
 
Systems Theory Paper
Systems Theory PaperSystems Theory Paper
Systems Theory Paper
 
The Methodological Base Of Cybernetic Reasoning
The Methodological Base Of Cybernetic ReasoningThe Methodological Base Of Cybernetic Reasoning
The Methodological Base Of Cybernetic Reasoning
 
PhD panel: Uses and abuses
PhD panel: Uses and abusesPhD panel: Uses and abuses
PhD panel: Uses and abuses
 
Body and Embodiment: Media Extension, Disembodiment, and the Cyborg
Body and Embodiment: Media Extension, Disembodiment, and the CyborgBody and Embodiment: Media Extension, Disembodiment, and the Cyborg
Body and Embodiment: Media Extension, Disembodiment, and the Cyborg
 
Smart landscapes concept
Smart landscapes conceptSmart landscapes concept
Smart landscapes concept
 
Marshall Mc Luhan - The extensions of man
Marshall Mc Luhan - The extensions of manMarshall Mc Luhan - The extensions of man
Marshall Mc Luhan - The extensions of man
 
Overbeeke2007
Overbeeke2007Overbeeke2007
Overbeeke2007
 
A Review Of Artificial Intelligence
A Review Of Artificial IntelligenceA Review Of Artificial Intelligence
A Review Of Artificial Intelligence
 
Understanding Computers and Cognition
Understanding Computers and CognitionUnderstanding Computers and Cognition
Understanding Computers and Cognition
 
Analytical-frameworks - Methods in user-technology studies
Analytical-frameworks - Methods in user-technology studiesAnalytical-frameworks - Methods in user-technology studies
Analytical-frameworks - Methods in user-technology studies
 
Machines in the ghost
Machines in the ghostMachines in the ghost
Machines in the ghost
 
Philosophy of Technology
Philosophy of TechnologyPhilosophy of Technology
Philosophy of Technology
 
Do Intelligent Machines, Natural or Artificial, Really Need Emotions?
Do Intelligent Machines, Natural or Artificial, Really Need Emotions?Do Intelligent Machines, Natural or Artificial, Really Need Emotions?
Do Intelligent Machines, Natural or Artificial, Really Need Emotions?
 
Just How Free Is The Question Of Free Will For Many Millennia
Just How Free Is The Question Of Free Will For Many MillenniaJust How Free Is The Question Of Free Will For Many Millennia
Just How Free Is The Question Of Free Will For Many Millennia
 
Road Not Taken
Road Not TakenRoad Not Taken
Road Not Taken
 
Flickr, Photosynth, And Strange Loops
Flickr, Photosynth, And Strange LoopsFlickr, Photosynth, And Strange Loops
Flickr, Photosynth, And Strange Loops
 
Can abstraction lead to intelligence?
Can abstraction lead to intelligence?Can abstraction lead to intelligence?
Can abstraction lead to intelligence?
 
Educational Robotics Between Narration and Simulation
Educational Robotics Between Narration and SimulationEducational Robotics Between Narration and Simulation
Educational Robotics Between Narration and Simulation
 
Software update for human brain, at a large scale
Software update for human brain, at a large scaleSoftware update for human brain, at a large scale
Software update for human brain, at a large scale
 

Plus de Rob Manson

The Power of AR & VR in Learning
The Power of AR & VR in LearningThe Power of AR & VR in Learning
The Power of AR & VR in LearningRob Manson
 
Immersive Sydney - #WebXRWeek
Immersive Sydney - #WebXRWeekImmersive Sydney - #WebXRWeek
Immersive Sydney - #WebXRWeekRob Manson
 
The Extended Reality Landscape #Reality17
The Extended Reality Landscape #Reality17The Extended Reality Landscape #Reality17
The Extended Reality Landscape #Reality17Rob Manson
 
ISMAR17 - Augmenting, Mixing & Extending Reality on the web
ISMAR17 - Augmenting, Mixing & Extending Reality on the webISMAR17 - Augmenting, Mixing & Extending Reality on the web
ISMAR17 - Augmenting, Mixing & Extending Reality on the webRob Manson
 
Immersive Computing @ #YOWConnected 2017
Immersive Computing @ #YOWConnected 2017Immersive Computing @ #YOWConnected 2017
Immersive Computing @ #YOWConnected 2017Rob Manson
 
Immersive - Sydney July 2017
Immersive - Sydney July 2017Immersive - Sydney July 2017
Immersive - Sydney July 2017Rob Manson
 
Computer Vision - now working
 in over 2 Billion Web Browsers!
Computer Vision - now working
 in over 2 Billion Web Browsers!Computer Vision - now working
 in over 2 Billion Web Browsers!
Computer Vision - now working
 in over 2 Billion Web Browsers!Rob Manson
 
Build your own reality
Build your own realityBuild your own reality
Build your own realityRob Manson
 
Web Standards for AR workshop at ISMAR13
Web Standards for AR workshop at ISMAR13Web Standards for AR workshop at ISMAR13
Web Standards for AR workshop at ISMAR13Rob Manson
 
The future is un-Real
The future is un-RealThe future is un-Real
The future is un-RealRob Manson
 
The Augmented Web delivers AR to the masses
The Augmented Web delivers AR to the massesThe Augmented Web delivers AR to the masses
The Augmented Web delivers AR to the massesRob Manson
 
Augmented Reality is just a feature!
Augmented Reality is just a feature!Augmented Reality is just a feature!
Augmented Reality is just a feature!Rob Manson
 
Introduction to AR
Introduction to ARIntroduction to AR
Introduction to ARRob Manson
 
Web Standards adoption in the AR market
Web Standards adoption in the AR marketWeb Standards adoption in the AR market
Web Standards adoption in the AR marketRob Manson
 
Now here is everywhere
Now here is everywhereNow here is everywhere
Now here is everywhereRob Manson
 
AR is the UI for Pervasive Computing
AR is the UI for Pervasive ComputingAR is the UI for Pervasive Computing
AR is the UI for Pervasive ComputingRob Manson
 
Your browser can see and hear and ...
Your browser can see and hear and ...Your browser can see and hear and ...
Your browser can see and hear and ...Rob Manson
 
Web Standards based Augmented Reality
Web Standards based Augmented RealityWeb Standards based Augmented Reality
Web Standards based Augmented RealityRob Manson
 
Is that sense-able?
Is that sense-able?Is that sense-able?
Is that sense-able?Rob Manson
 

Plus de Rob Manson (20)

The Power of AR & VR in Learning
The Power of AR & VR in LearningThe Power of AR & VR in Learning
The Power of AR & VR in Learning
 
Immersive Sydney - #WebXRWeek
Immersive Sydney - #WebXRWeekImmersive Sydney - #WebXRWeek
Immersive Sydney - #WebXRWeek
 
The Extended Reality Landscape #Reality17
The Extended Reality Landscape #Reality17The Extended Reality Landscape #Reality17
The Extended Reality Landscape #Reality17
 
ISMAR17 - Augmenting, Mixing & Extending Reality on the web
ISMAR17 - Augmenting, Mixing & Extending Reality on the webISMAR17 - Augmenting, Mixing & Extending Reality on the web
ISMAR17 - Augmenting, Mixing & Extending Reality on the web
 
Immersive Computing @ #YOWConnected 2017
Immersive Computing @ #YOWConnected 2017Immersive Computing @ #YOWConnected 2017
Immersive Computing @ #YOWConnected 2017
 
Immersive - Sydney July 2017
Immersive - Sydney July 2017Immersive - Sydney July 2017
Immersive - Sydney July 2017
 
Computer Vision - now working
 in over 2 Billion Web Browsers!
Computer Vision - now working
 in over 2 Billion Web Browsers!Computer Vision - now working
 in over 2 Billion Web Browsers!
Computer Vision - now working
 in over 2 Billion Web Browsers!
 
Build your own reality
Build your own realityBuild your own reality
Build your own reality
 
Web Standards for AR workshop at ISMAR13
Web Standards for AR workshop at ISMAR13Web Standards for AR workshop at ISMAR13
Web Standards for AR workshop at ISMAR13
 
The future is un-Real
The future is un-RealThe future is un-Real
The future is un-Real
 
The Augmented Web delivers AR to the masses
The Augmented Web delivers AR to the massesThe Augmented Web delivers AR to the masses
The Augmented Web delivers AR to the masses
 
Augmented Reality is just a feature!
Augmented Reality is just a feature!Augmented Reality is just a feature!
Augmented Reality is just a feature!
 
Introduction to AR
Introduction to ARIntroduction to AR
Introduction to AR
 
Web Standards adoption in the AR market
Web Standards adoption in the AR marketWeb Standards adoption in the AR market
Web Standards adoption in the AR market
 
Now here is everywhere
Now here is everywhereNow here is everywhere
Now here is everywhere
 
AR is the UI for Pervasive Computing
AR is the UI for Pervasive ComputingAR is the UI for Pervasive Computing
AR is the UI for Pervasive Computing
 
Your browser can see and hear and ...
Your browser can see and hear and ...Your browser can see and hear and ...
Your browser can see and hear and ...
 
Web Standards based Augmented Reality
Web Standards based Augmented RealityWeb Standards based Augmented Reality
Web Standards based Augmented Reality
 
Is that sense-able?
Is that sense-able?Is that sense-able?
Is that sense-able?
 
What is web3?
What is web3?What is web3?
What is web3?
 

Dernier

The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...Aggregage
 
20230202 - Introduction to tis-py
20230202 - Introduction to tis-py20230202 - Introduction to tis-py
20230202 - Introduction to tis-pyJamie (Taka) Wang
 
Building AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptxBuilding AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptxUdaiappa Ramachandran
 
9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding Team9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding TeamAdam Moalla
 
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online CollaborationCOMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online Collaborationbruanjhuli
 
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDEADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDELiveplex
 
VoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBXVoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBXTarek Kalaji
 
Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )Brian Pichman
 
How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?IES VE
 
Nanopower In Semiconductor Industry.pdf
Nanopower  In Semiconductor Industry.pdfNanopower  In Semiconductor Industry.pdf
Nanopower In Semiconductor Industry.pdfPedro Manuel
 
Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Commit University
 
Introduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptxIntroduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptxMatsuo Lab
 
Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024D Cloud Solutions
 
Computer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and HazardsComputer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and HazardsSeth Reyes
 
Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1DianaGray10
 
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...DianaGray10
 
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdfIaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdfDaniel Santiago Silva Capera
 
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAAnypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAshyamraj55
 

Dernier (20)

The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
 
20230202 - Introduction to tis-py
20230202 - Introduction to tis-py20230202 - Introduction to tis-py
20230202 - Introduction to tis-py
 
Building AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptxBuilding AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptx
 
9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding Team9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding Team
 
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online CollaborationCOMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
 
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDEADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
 
VoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBXVoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBX
 
Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )
 
How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?
 
Nanopower In Semiconductor Industry.pdf
Nanopower  In Semiconductor Industry.pdfNanopower  In Semiconductor Industry.pdf
Nanopower In Semiconductor Industry.pdf
 
Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)
 
20230104 - machine vision
20230104 - machine vision20230104 - machine vision
20230104 - machine vision
 
Introduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptxIntroduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptx
 
Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024
 
20150722 - AGV
20150722 - AGV20150722 - AGV
20150722 - AGV
 
Computer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and HazardsComputer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and Hazards
 
Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1
 
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
 
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdfIaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
 
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAAnypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
 

The Pervasive Experience - project review July 2010

  • 1. The Pervasive Experience will a second Cognitive Revolution turn us into d-zombies... Project Review : July 2010 roBman@mob-labs.com / @nambor photo by club silencio
  • 2. what the hell does that mean?!
  • 3. what is the Pervasive Experience? Ubiquitous Computing was pioneered by Weiser (1991) at Xerox Parc. “Ubiquitous computing names the third wave in computing, just now beginning. First were mainframes, each shared by lots of people. Now we are in the personal computing era, person and machine staring uneasily at each other across the desktop. Next comes ubiquitous computing, or the age of calm technology, when technology recedes into the background of our lives.” Mark Weiser That was obviously from a 90's perspective. This project proposes a structural and ethical argument that Pervasive is a more appropriate and Humanistic term than Ubiquitous. So the Pervasive Experience is a new, truly modern Human Condition that is now possible in today's anytime, anywhere internet world.
  • 4. what is a Cognitive Revolution? From Descartes (1637) and the Enlightenment of the 1600's through to the 1950's and 60's, Positivism was the dominant epistemology in scientific thought and Western Philosophy. Built upon this world view, Pavlov (1904) and Skinner (1938) defined Behaviourism within the field of experimental psychology. Yet starting in the 1950's the dominant epistemology began to turn towards Realism. A reaction against Behaviourism became a movement known as the Cognitive Revolution. This movement recognised our internal phenomenological experiences as a valid object of research.
  • 5. what was that about Zombies? Chalmers (1995) is an interesting Philosopher that has contributed to the field of Consciousness. In order to explore and discuss the concepts surrounding the phenomenological aspect of Consciousness he has proposed what has become known as the Philosophical Zombie thought experiment. In its simplest form it poses the question “if people existed that behaved exactly like a normal human, but didn't experience the phenomenon of internal representation – are they still human?”. These meaty automata like creatures are called Philosophical Zombies or p-zombies for short. It can also be adapted to explore similar questions about the concept of the Soul. Hypothetical humans without Souls are called s-zombies.
  • 6. so what's the big idea then? Pervasive Computing could be creating a second Cognitive Revolution. It is changing the geometry of our sensory input and our presence. It is changing the structure of our sphere of control. Our very Body Image. I believe Pervasive Computing is adding an interesting new dimension to Chalmers' Philosophical Zombie thought experiment...lets give the theoretical zombies in a Pervasive Computing Experience a name. Lets call them Digital Zombies or d-zombies These are people whose experience of the phenomenon of internal representation is more than a certain percentage digital, in some way. People whose sensory experience is digitally mediated beyond a defined d-zombie threshold.
  • 7. this raises some interesting questions Does this d-zombie threshold exists at all. When we reach a point where our sensory input is more digitally mediated than solely biologically mediated, can our conscious subjective experience somehow become less Human and more like a d-zombie? Can it lead us to become d-zombies in ANY way at ANY point at all? What if our attention itself became incrementally digital? (See the Pervasive Reality Engine defined below)
  • 8. contents The Project : An introduction and brief history Expanding my view : The project's new focus Reality Engines : The key model proposed Philosophical context : The related body of knowledge Ubiquitous vs. Pervasive : Pinning down key terms Reading the Web : Seeing the web as a digital prosthetic The Pervasive Experience : Reflections on this project What's next? : Where to from here Experimental Technology : Rubber hands and AR helicopters
  • 9. The Project photo credit
  • 10. The Pervasive Experience Project Hold your hand up and away from you. Imagine your fingertips are dipped in vomit! Feel how dirty and contaminated your fingers are. Can you sense changes in how you hold your body image? Take a moment to feel how this part of your body holds your attention. Even if you look away it seems as if your hand is tagged with meaning and still to be attended to. This positive, lead hallucination provides you with your own subjective experience of a traditional "real world" Body Image. Biological, grounded, open to symbolic manipulation...yet essentially "real". By contrast, this document reviews the Pervasive Experience project. In this project the driving assumption is that increasingly pervasive, networked technologies are impacting our lives. The research question is: How is Pervasive Computing changing you? This applied research explores how Pervasive Computing may impact our Philosophy of Mind. It looks at our perception of space and self, and their underlying Neurobiology. It aims to integrate theories from these four broad areas in a high-level, structural way. Its aim is to explore through practice and reflection how these theories may be able to be reshaped by the pervasive diffusion of digital networked technologies. Based on this analysis, an integrated, pragmatic, working model of the end-to-end process is synthesised - from the environment, body, nervous system and brain to sensory input, representation, consciousness and cognition. This model represents a Reality Engine. This document aims to frame this model within the existing body of knowledge, then extend it to support the rapidly growing, digital environment of Pervasive Computing. I believe this model opens a new perspective on network enabled devices where the web itself can be seen as a form of digital prosthetic that is increasingly pervading our experience and becoming more organ-ised. So after completing this review I now believe the most likely answer to this research project's question may be: By driving rapid Body Image modification. This could well be the defining structural process of Pervasive Computing...
  • 11. Methodologies Over the past two years I have been exploring the impact of Pervasive Computing upon my sense of space using Action Research and reflecting through the lenses of Diffusion of Innovations and Distributed Cognition. Action Research: As defined by Dick (1999), Action Research is a "cyclic or spiral process" that "alternates between action and critical reflection". This review document represents the output of one of these critical reflection cycles. Diffusion of Innovations: Initially defined by Tarde (1890) the theory of Diffusion of Innovations was crystalised in the seminal work by Rogers (1962). It aims to describe and analyse why and how ideas and technologies diffuse through cultures, especially the rate of diffusion. Distributed Cognition: As defined by Hutchins (1995), Distributed Cognition is a framework involving individuals, artefacts and the environment. It is strongly based upon Vygotsky's (1920's) work that led to the development of Activity Theory. Distributed Cognition that takes as its unit of analysis a culturally constituted functional group rather than an individual mind. This theory is concerned with how information is propagated through a system in the form of representational states of mediating structures. These structures include internal as well as external knowledge representations, (knowledge, skills, tools, etc.). This approach permits us to describe cognitive processes by tracing the movement of information through a system and characterize the mechanisms of the system which carry out the performance, both on the individual and the group level. Hutchins (1995)
  • 12. Expanding my view photo credit
  • 13. The bigger picture This research has now been extended to include the lenses of Situational Awareness, Neuroplasticity and Sensorimotor Theory. This change has expanded my research focus to include “consciousness” and "self" and their underlying "Neurobiology". However this creates such a broad and technically deep domain that the goal is simply to “integrate current theories from these areas at a purely structural level in relation to the Reality Engine”. I have also been aware of Gibson's (1992) work on Visual Perception for a long time now but have struggled to integrate it fully into this work. Recently I came across O'Regan's (2010) work on Sensorimotor Theory, which seems to develop a situated aspect of Gibson's work within the modern context. To me this work provides a strong foundation for a wide range of discussions from Action Research through to Philosophy of Mind. I have also been researching Bach-y-Rita's (1969) work on Sensory Substitution that lead to the development of the field of Neuroplasticity. This work leads to some amazing predictions that can be tested through direct experience as shown by Bach-y-Rita, Ramachandran (2007) and Ehrsson (2010). As you will see below I have found that what I am exploring appears to be built upon a form of Neo-Cartesian Dualism. Decartes' statement in Principals of Philosophy, “I think, therefore I am” (Cogito ergo sum) are a compelling fit for the traditional Body Image experience of our reality. Yet, in the Pervasive, Neruoplastic, digitally networked Body Image experience of our proximate future we may be better supported by the philosophical statement “I am what I think” (EGO sum quis EGO reputo?) or perhaps more explicity “I am the sum of my thoughts”. Not in a frivilous “positive thinking” kind of way, but a “my thinking alters digital hardware and software” kind of way. This is not contradicting Descartes work, but reflecting on it from this new angle. If convergence/divergence continues to develop so our sensory input is increasingly digitally mediated then the relevance of this statement should become even more significant over time. Distributed, interactive software and hardware literally becomes part of our Extended Mind. This is where the d-zombie questions come in.
  • 14. definition: Situational Awareness Situational Awareness is the awareness of what's going on around you. How you make sense of information, events and actions. How well you understand how they'll effect your goals and objectives. Related concepts such as Sensemaking take a more historical perspective and Situational Awareness tends to have a now and near future perspective. Inadequate Situational Awareness is seen as one of the key factors in accidents through human error. Especially in roles where there are serious consequences (e.g. drivers, soldiers, pilots, doctors, etc.). diagram credit
  • 15. definition: Neuroplasticity The concept was initially proposed by James (1907), yet the term Neuroplasticity wasn't coined until (1948) by Konorski. Neuroplasticity was first experimentally demonstrated by Bach-y-Rita (1969). His work in Sensory Substitution suggested that the brain can rewire input from one sensory mode into internal representations in another mode. Not just in a form of synaesthesia, but by completely re-wiring the cortical maps. So at a time when the Cartesian homunculus appears to be identifiable within our physical neurobiology in the form of the cortical maps, we also appear to be finding that this homunculus is also plastic, malleable and changeable based on action and even thought alone. Neuroplasticity is also revolutionary because in some ways it brings cutting edge neurological research within the reach of anyone with a working brain. Adding in the Augmented Reality technology I work with everyday and you have a powerful set of research tools. photo credit
  • 16. definition: Sensorimotor Theory "The sensorimotor view conceives of feel ("subjective experience") as a way of interacting with the environment. The quality of feel is simply an objective quality of this way of interacting." O'Regan (2010) I believe this provides support for the view that the self is the mental model that integrates input from other models and senses. It is a by-product of the creation of attention. By it's very definition, the sensorimotor view “requires” attention. I believe the emergent property of this is the self. A situated, attentive feedback loop. This perspective is expanded further below. diagram credit
  • 17. Reality Engines photo credit
  • 18. Descartes' Reality Engine To provide a starting point lets look at Descartes proposal for a dualistic reality. “I have a clear and distinct idea of myself as a thinking, non-extended thing, and a clear and distinct idea of body as an extended and non-thinking thing.” So, Descartes argues, “The mind, a thinking thing, can exist apart from its extended body. And therefore, the mind is a substance distinct from the body, a substance whose essence is thought.” As we will see, this initial proposition has fragmented into a wide range of Dualisms with different structures and theories. This project takes an Interactionist Dualism as it's starting point. Descartes' Dualism Diagram diagram credit
  • 19. The Reality Engine elements of the diagram a/b = concepts self = center of attention environment = mental model of the world Body Image = mental model of your body X <- you are here ROM = rest of mind brain = the biological organ ROB = rest of body ROW = rest of world ROB <-> brain = nervous system brain <-> Body Image/a/b = perception Body Image <-> self = attention sensory perception is dualistic ● sensory input is one world ● perception is the other This diagram describes the single integrated Reality Engine model proposed by this project. A working system whose feedback loop creates a cognitive conscious self that interacts with the perceived Real World. Our brains seem to crave sustainable models that turn sensory input into actionable behaviour. Feedback loops that integrate all the aspects of sensory input (internal and external!?) and physical action into an attentive self. To me they appear from the outside to be a rich ecosystem of self-reflexive Hidden Markov Models (ironic?).
  • 20. The Reality Engine extended additions to the diagram a = subconscious interactive concept a1 = subconscious mental concept a2 = subconscious body concept X <- you are here b = conscious interactive concept b1 = conscious mental concept b2 = conscious body concept so what is the “self”? ● reality is rendered through attention ● attention is the integration of sensing, reacting, remembering, thinking, doing ● the by-product of this is the self... because attention needs a point of origin A concepts are subconscious and can't be accessed by the self. B concepts are conscious and may be accessed by the attentive self. 1 concepts (A1 & B1) are mental only and have no direct relationship with the body. 2 concepts (A2 & B2) are concepts that are pre-formed within the body itself before the sensory data is sent through the nervous system to the the brain. This includes things like reflexes and muscle memory.
  • 21. Mixing Realities Milgram's Virtuality Continuum If the input for this Reality Engine is substituted with completely digitally produced vision and sound, etc. then this becomes a Virtual Reality Engine (VR). And if we only overlay digital elements onto our perception of the Real World then this becomes an Augmented Reality Engine (AR). While VR and Artificial Intelligence were an aspirational focus of the original Cognitive Revolution, it seems likely from todays perspective that AR will continue to pervade our experience...but in folded and bent ways. In this case, pixels (or other sense based equivalents e.g. sound) can provide a good measure of how much digital sensory mediation has pervaded a person's Body Image experience. That is, “What percentage of our total sensory input is digitally mediated?”. In Augmented Reality the pixels that make up the overlaid information can be counted to calculate this ratio based on our total field of visual sensory input. Within this data is an important sub-set that is not just how much of our environmental sensory input is digitally augmented, but how much of our attention is digitally mediated. Through the lenses of Distributed Cognition, and Situational Awareness it's natural to ask “how much of our attention, and therefore self, can be Digital without turning us into d-zombies?”. Is there any limit to this digitally Extended Mind?
  • 22. Multiplayer Realities Multiple Reality Engines A useful set of terms breaks the world into the Geosphere (physical earth/planets), the Biosphere (living things) and the Noosphere (symbolic or thought related things). The Pervasive Experience is transforming the Noosphere in terms of how interconnected it is. The relationship between space and even the point of origin of thoughts and symbols is being completely reformed. While in the traditional world each “self” has been cacooned within their own Cartesian isolation chamber, the Pervasive Experience may be opening the door to change this. Shannon's (1948) work on a Theory of Communication has some interesting application here and is an application of Hidden Markov Models. Symbolic communication between two Reality Engines seems to involve building a Theory of Mind of the “other” and creates a pragmatic use of encoding/encryption in a Shannon Information sense.
  • 23. Body Image Based upon the full Reality Engine model proposed above, the term Body Image can be used to define the specific mental model used to represent our body. It can also more flexibly be seen to represent all the mental models that go in to making up our entire “self” and the broader context we exist within. Our entire Self Image and the contexts that it implies. Often, philosophical dualistic thought experiments and discussions isolate “special” dis-embodied beings to explore or prove a point. From the perspective of this project this is a non-sense. A “dis-embodied experience” is an altered form of consciousness outside the Sensorimotor world view. Even hallucination or meditation upon “mental only” (e.g. B1) concepts are spatially related to the Body Image. The word “internal” is often used to describe these types of phenomenon. In this way attention is a situated experience (or feedback loop) that requires (or perhaps generates) a point of origin. Interestingly, early research into Attention during the first Cognitive Revolution was focused upon “divided attention” (e.g. dichotic listening). By contrast, the experiments of this research project are focused on in some senses the opposite. The goal here is to integrate objects from two (or more) places/times into a single coherent attention, instead of attempting to split attention into two (or more) separate streams. I believe this is the complete reformation of the concept of Body Image that the Pervasive Experience is enabling.
  • 24. Philosophical context photo credit
  • 25. Dualisms In order to relate my reflections from this research review to the existing body of knowledge I have developed the following map of what appears to be the related philosophical terrain. I am also open to new perspectives in this area and reviewing new concepts and frameworks drives a lot of my reflection. Here's the map so far. Cartesian Dualism: This philosophy of mind separates reality into a dual system of mind and body. Descartes' work was an extension of thought that dates back to Plato and Aristotle. This separation into physical and non-physical is also often bound up with a spiritual discussion of the soul as well. Interactionist Dualism: Interactionism is the form of Dualism that proposes that the mind and body can interact bi-directionally in a causal way. While it is often stated in the literature that it “is difficult” to establish how this may occur, I find it almost self evident. fMRI's, etc. clearly show that brain activity is tied to symbolic phenomenal experience. Neuroplasticity takes this one step further stating that the phenomenal experience alone can persistently change neruobiology. Interactive Dualism: There is also an area of Interaction Design/CHI where Theory of Mind is related to interactions with any dynamic system. For example Norman (1988). diagram credit
  • 26. Neurobiology Neurobiology is an amazingly deep and technically complex area. This literally includes “brain surgery” and is dealing with the most complex structure in the known universe. This project focuses on the perspective of Cognitive Neuroscience and is primarily focused upon the theoretical frameworks available that show how cognitive processes can impact the biological structures within the brain. Neuroplasticity is the key lense used for this reflection and literature review. I believe it is important to include this from at least a high-level in order to fully explore the Interactionist Dualism foundations of the Reality Engine. I think it is also very relevant for the phrasing of the research question “How is Pervasive Computing changing you?”. There is also likely to become more closely integrated into the overall Pervasive Computing domain with the rise of Computer/Mind interfaces. This is also closely related to the Digital Prosthetics content discussed below. Interestingly there are some mechanical devices/technologies (e.g. TMS) that are widely available and are actively being used to map and alter specific neruobiologies. Personally I find TMS quite frightening and think that this project specifically raises some serious ethical issues that relate directly to it's use. (e.g. can mechanically changing the operation of certain Neurobiological structures change your consciousness to the point of creating a d-zombie? Could it be used as a literally de-humanising experience?). Not to mention the unknown long term impacts of high-intensity EMF on delicate and critical cell structures.
  • 27. The Mind Philosophy of Mind: This is a very broad and rich area. Dennett's (1991) work breaking the philosophy of mind into two problems “consciousness” and “content” is interesting. His Neural Darwinism perspective is also useful and relevant for this project. Chalmers' work on consciousness, especially his p-zombie thought experiment is obviously relevant. Theory of Mind: The ability to form models that attempt to predict the behaviour of objects, animals and other behaviours is a critical conscious activity. Interestingly, the fact that people form Theories of Mind implies the belief that a representation system inside another's mind can causally interact to create actions. Cognitive Revolution: Essentially this was a rebellion against an automata style view of behaviourism and a recognition of the phenomenological aspects of the human experience. Interestingly, in 1950 Turing published "Computing machinery and intelligence" in the journal “Mind”. His proposal was that you could use questions and answers to test machines for intelligence is often included as part of the Cognitive Revolution. His view here appears to be that we can use our own innate Theory of Mind models (B1 concepts) to interrogate an “other” via a text based interface to determine if it is Human or synthetic. This highlights a consciousness continuum (or perhaps even triangle) between biological entities and synthetic/mechanical entities.
  • 28. Functionalist Hidden Markov Models Functionalism: Philosophical Functionalism provides a post-Cartesian perspective that is highly relevant to the Pervasive Experience and it's digital-isation. However, the Reality Engine provides a perspective where Functionalism and Interactionist Dualism need not be mutually exclusive. I believe it is possible for Functionalism to exist at the physical/material Neurobiological level to create a subjective experience that is Dualistic and Interactionist in nature. In fact I believe that the two are perfectly fitting sides of the same coin and that the nature of attention/self requires both. Hidden Markov Models: The Cartesian cacoon that the Reality Engine suggests we all live within also suggests that the Hidden Markov Model is a useful metaphor. This creates a nice, ironic, self-recursive Solipsistic loop Shannon's Theory of Communication: Shannon's (1948) work on a “Mathematical Theory of Communication” is also highly relevant to this discussion. For example his “Fig 1.” is almost identical to the Multiplayer Reality Engine diagram above. diagram credit
  • 29. Ubiquitous vs. Pervasive photo credit
  • 30. Weiser's Ubiquity My starting point for the whole project was a diagram created by Mark Weiser in (1996) - see above. My interpretation of this diagram is that the left and right elements on the first row contrast non-Interactionist Cartesian Dualism (left) with Situated Cognition (right). The labels beside clearly state that Weiser believed a non-Interactionist Dualism was "wrong". The second row of left and right elements contrast Current Computing Technology with Ubiquitous Computing Technology. My interpretation here is that Weiser was showing how Computing Technology will pervasively spread throughout more and more aspects of our Situated reality, as opposed to the unipresent Desktop Computing model that dominated the 1980's and 90's.
  • 31. Spatial Maps Review Edward T. Hall et al's work on Personal Space. Weiser's diagram is obviously focused upon the spatial Relate this to the reality engine. relationship between you and the world. All Cartesian like dualisms appear to have this spatial aspect. The spatial models we hold can also be broken down into finer grained regions. At the highest level of the physical world is Inside and Outside our body. This implies a Body Image and surrounding that, and inter-meshed within that, is our overall sense of space. Beyond proprioception are our exteroceptive senses that collect stimulus from the outside world. From this we build models about these spaces and relationships. Hall's (1966) work on Personal Space provides a useful framework for a User Centred world view. These concepts also have a pragmatic relationship to Situational Awareness too. The Reality Engine aims to ensure Hall's work integrates with Cartesian Dualism, Extended Mind, Distributed Cognition and Situational Awareness.
  • 32. Divergence Applying this type of spatial analysis to the concept of Convergence has also led me to some interesting reflection. I believe Divergence is the by-product of Object:Network Convergence. As the distance between an Object and the Network passes a point of Convergence the distance between the “Situated Point of Origin” of that Object and “Control” of that Object Diverges. In other words, once you connect an Object to the Network you're free to remove its “Physical Interface” and simply “Control” it via the Network. This is exemplified by the marketing term “Anywhere, Anytime”.
  • 33. Putting YOU at the centre I think that the framework above, especially the Pervasive Web integrated into the Reality Engine, highlights why Pervasive is a more appropriate term than Ubiquitous. Ubiquitous suggests that the technology is “everywhere”. While it may at first seem like that's the case, in reality I believe we are discussing how this technology is soaking into our attention. If you look at mobile network coverage maps they don't cover the whole world, just the parts where the majority of the population are likely to be. This is a natural evolutionary pattern that allows for the conservation of energy. Pervasive suggests the technology is soaking into “your” subjective experience - with you at it's heart. Ubiquitous on the other hand suggests technology is soaking into the universe - whether you are there or not. To me this appears to be an important structural and ethical distinction. The next section aims to share some insights that should clarify why this may become even more important as convergence/divergence continues to develop. Where distributed, interactive software and hardware literally becomes part of our extended self. If the impact of Moore's law and the Network Effect (Metcalfe's law) keep driving the Pervasive Network then I believe we need to make these technologies an extension of our “self”, or risk becoming diluted and de-Humanised. Again, this is where the d-zombie questions come in.
  • 34. Reading the Web photo credit
  • 35. Prosthetics Prosthetics are traditionally defined as an addition to the body (coming from the greek word for addition) that replaces a missing body part. If it's related to mobility or a directly sense-able part of our Body Image then it is likely to be a limb. If it is to do with sub-conscious processes then it is likely to be an organ and may include real or artificial organs. This analysis is blurred even further when the prosthetic is a sensing organ that impacts your Body Image. photo credit
  • 36. Digital Prosthetics Now, there also exists the real opportunity to utilise Digital Prosthetics that enhance a person's sensory input and even cognitive capabilities like memory. This obviously heads us into the well populated cyborg territory. While the traditional focus of Prosthetics was upon repair and replacement, the Digital Prosthetic view of the Pervasive Reality Engine is squarely focused upon “beyond repair” and towards “enhancement”. Jamais Cascio presents a perspective that many people will be facing over the next few decades as their options turn from repair to enhancement. This is also an interesting convergence of Digital Technologies, Medical Miniaturisation and an Aging Population.
  • 37. The Pervasive Web It's amazing to see how the web has expanded from a single point on the planet (CERN) out to becoming a pervasive cultural concept. In fact it could be seen to have initially existed as just a model within the Reality Engine of Tim Berners-Lee. Now it seems to be a model that exists in almost everyone's Reality Engine and it has distorted how they use and think of space, place and time. Pervasive Web devices can then be seen as a form of Digital Prothstetic and the Web itself is pervading throughout our planet and our experience. Literally soaking into our consciousness. However, it's in a unique class of mental models that not only cement themselves into Reality Engines, they provide networked access and become the point of exchange for other models too.
  • 38. The Pervasive Experience photo credit
  • 39. First Reflections The heart of my recent reflection has been upon Neuroplasticity and Situational Awareness and how they relate to the concept of Personal Space and Theory of Mind in light of the Reality Engine. Extend this to interacting with an increasing number of web applications and web enabled devices and you have a Pervasive Reality Engine. The addition of Pervasive Computing to the Reality Engine creates a structural change to the models that defined the Cognitive Revolution. While the original Cognitive Revolution was focused upon what happened within the brain and mind, the Pervasive Reality Engine is focused upon what happens to us as our self moves out of our brain and possibly even out of our mind if that is achievable (imagine extending the A2 & B2 concepts in the Reality Engine using remote sensors). To use buzz terminology, this is Cognitive Revolution 2.0. So it appears we have a relatively mature conceptual framework for the individual mind. At the other extreme it also appears that the impact of interactive networks have been explored broadly from the macro, national scale. Now seems like the perfect time to integrate these two perspectives into an exploration of the impact of pervasive networks on the mind. This exploration leads to some interesting questions. If consciousness could be a property achieved by any system that can create the right type of attentive self awareness feedback loop, then we as humans could “safely” extend our Human experience using digital technology. This would benefit us by allowing us to tag along with rapidly evolving digital technology. A chance to keep up with any possible proximate Technological Singularity. If not, then this has obvious implications for the ethics of technology and the discussion of the d-zombie threshold and the possibly de-Humanising nature of technology. However, if it were true then this has implications for the ethics of machine intelligence and even biology. If consciousness were an achievable property of the right type of attentive loop then any “thing” that displays that type of loop is essentially “equivalent to a self conscious human being” - at least at a category level. Interestingly, either way there is an impact.
  • 40. Content Reflection Pervasive Computing is a movement driven by the structural reformation of our Reality Engines through convergence/divergence. I believe it can be shown to enable the following measurable changes: – rapid Body Image modification – flexible Body Image design and re-engineering/sculpting – dynamic multi-POV Body Image experiences I am confident now that the content outlined above provides a robust theoretical and practical framework for imagining, designing and building rich, interactive and "world changing" Pervasive Experiences. This model obviously needs to be reviewed, tested and revised on a regular basis, however it provides a workable starting point for some truly innovative experimentation.
  • 41. Context Reflection My personal experience of this Action Research project has been an interesting, stimulating and very challenging one. The breadth of content and reading has been quite challenging, as has been keeping up with the constant pace of change on the technology side. Luckily, my day-to-day work keeps me focused upon this type of change. There are a number of limitations of my work that have been pointed out by the people that have been kind enough to review it. The first is that it is almost exclusively from a western thought perspective. The world is obviously full of a broad range of interesting and insightful perspectives that don't fit into that bucket. I hope to at least start to address this in the next phase. I have also received feedback stating that this work appears strongly philosophically monistic. Personally, I don't believe that the output is quite that simple. However the density of the content here masks some of the subtleties. I hope I can communicate this aspect of the project more clearly in the next phase and aim to make my stance here more transparent. I have also receive feedback on how overwhelming and sometimes un-approachable this document can be. I've made a number of revisions that have hopefully addressed some of this, but this is a continual challenge and I am continuing to attempt to distill this down to a simpler set of key messages without diluting the depth of the work. While this work is obviously strongly academic in nature, I am completing it independently from any academic institution as an ongoing attempt to develop my practice. I have learned and grown a lot through the process, but obviously a more formal context may have helped me stay more focused. Luckily I've been able to seek constructive feedback and support from some academic friends that I'll thank in due course. On the whole I feel I've achieved something of real value and I'm comfortable that I now have a good framework with which to begin my experiments. This is where the project gets really interesting.
  • 42. Ethical Reflection There is an obvious commercial bias I have in my world view. I work as a Technologist for a Research Lab that helps it's clients commercialise new technologies. While my company (MOB) are happy to support this research project, both the company and I obviously benefit from the output of this project. However naive this may be, my underlying perspective is not so much that “I do this research because my job is closely related”, but more “I am good at my job because I am naturally drawn to a research project like this”. While this may be a subtle difference, I believe it has significant implications to the overall work. The questions raised by this research project are essentially Humanist in nature and are exploring the continuum between today's common Human Condition and the other extreme of Artificial Intelligence. While traditional AI starts at a synthetic point and heads towards consciousness, this project starts with the User and pushes the boundaries in the other direction. The Ethical focus of Humanism sets out the framework for this reflective process. The logical conclusion of this reflection is the structural proposition for a new terminology with Pervasive being more appropriate and Humanistic than Ubiquitous. Further and deeper reflection upon the ethical aspects of this project as it relates to the following topics is also planned. These obviously covers a very broad set of domains which will only be touched upon to gather insights. - Power - Digital Divide and Abandonment/Neo-luddites - Environmental - Cultural/social - Economic
  • 43. What's next? photo credit
  • 44. Dance of the bees... When bees return to the hive they communicate the location of pollen rich plants to other bees through a structured dance that apparently transmits a usable mental map. This is perhaps the only known symbolic language outside mankind. The Reality Engine model proposed above sets out a dualism between the physical and the symbolic worlds and suggests that our conscious self is a product of our attention through our neurobiology and sensory input. Pervasive Computing is literally giving us the ability to sculpt and dynamically re-design our sensory input, attention and through that neurobiology. To reshape our consciousness/self through Pervasive Experiences. But the content and answers outlined in this review just open deeper questions. e.g. Does the increasingly Digital nature of our new attention/self make us any less human at some definable point? At that point, do we turn into d-zombies? Or could it possibly make our experience even more human and humane? NOTE: Multi-POV is a structural change to our existing empathy model and may have significant implications for our Theory of Mind. The goal now is to continue to explore this model and the d-zombie thought experiment through applied experimentation. The focus is upon the subjective experience of using Pervasive Computing technologies and the methodology is based upon Action Research and reflection. The method involves creating and using Pervasive Experiences and measuring how, if at all, they change the user's Body Image. The specifics of this study are the human equivalent of the “dance of the bees” and the aim is to model the user's internal mental maps by observing and manipulating external behaviours and technologies.
  • 45. Experimental Technology photo credit
  • 46. The rubber hand The starting point for the planned experimentation is the rubber hand illusion which uses a mirror to mechanically/ physically bend light to change your perception. This altered sensory input is used to fool our Reality Engine into associating a foreign object into our Body Image. This literally changes our sense of who and where we are. The Pervasive Experience also allows us to flexibly distort our sensory input to other places and points in time through networked devices. Using networked cameras, microphones and user interfaces we can digitally bend sound, light and actions. Using digital sensors we can bend other forms of electromechanical energy to give us an expanded range of sensory input beyond the normal human range. The experiment this project is pursuing attempts to merge these two aspects. To use digitally altered sensory input to fool our Reality Engine into associating foreign & digital objects into our Body Image. One interesting tool that we are starting to explore for this experimentation is the AR Drone. By exploring this in different ways it is hoped that the limits of this new form of Extended Mind (as Chalmer's put it “panprotopsychism”) can be mapped. To contribute to establishing where the limits to this extension may lay.