Talk and workshop on the MINDtouch research project and progress

1. Camille Baker, MASc Interactive Arts
SMARTlab PhD Candidate
SLSA 2010
Riga, Latvia
MindTouch:
embodied ephemeral transference:
mobile participatory performance research

2. SMARTlab PhD research: performance media
–has been to uncover any new understandings of the
sensations of ‘liveness’ and ‘presence’ that may emerge
when using mobile technologies and wearable devices in
participatory performance media contexts–

3. liveness in mobile video participatory performance
 liveness (mediated)/ virtual (real)
 presence (absence)
 experience (perception)
 embodiment of technology
 locative mobile media / video
 visceral / sensation
 intimacy / immediacy /mobility
 sensor technology and the body
 audience as performer

4. mobile devices as non-verbal expression
first video collection workshop in Vancouver June
2007

5. presence

6. liveness

7. embodiment

8. mini-doc on the project

10. mobile embodied performance media

11. Image © 2007 C. Baker workshop still Dublin July 2007
video collection workshops : two - Dublin

12. video collection workshops : three - Vancouver
Images © 2007 C. Baker - stills from participants’ videos from Vancouver workshop August 2007

13. video collection workshops : fourth - Dublin
Images© 2007 C. Baker - stills from participants’ videos from Dublin workshop October 2007

14. mobile video – examples
videos © 2007-2009 C. Baker - participants’ videos from Vancouver and Dublin workshops 2007

15. sensors + software
Images © 2008 C. Baker – Michael Markert working on biosensing electronics for custom software

16. mobile video software – development
Images © 2008 C. Baker – custom software by Michael Markert

17. - placing the biosensors

18. smart garments
Images © 2008 C. Baker – sketches of garments with embedded electronics by Michael Markert and Tara Mooney

19. - making the 1st prototype
Images © 2008 C. Baker – sketches of garments with embedded electronics by Tara Mooney with Rachel Lesebikan

20. the 1st prototype
Images © 2008 C. Baker – design stages of garments with embedded electronics by Tara Mooney with Rachel Lesebikan

21. finished prototype 1: garment and embedded sensors
(completed November 2008)
Images © 2008 C. Baker – alpha garment with embedded electronics by Tara Mooney with Rachel Lesebikan

22. wearable devices + embedded sensors exploration
From Leah Buchley’s site : http://www.cs.colorado.edu/~buechley/projects/performance/performance.html

23. wearable devices + embedded sensors exploration

24. wearable devices + embedded sensors exploration

25. prototype 2: garment design
– design process

26. prototype 2: garment design
– design process
Images © 2009 C. Baker – beta garment with embedded electronics by Tara Mooney with Rachel Lesebikan

27. prototype 2: garment design
(completed August 2009)
Images © 2009 C. Baker – beta garment with embedded electronics by Tara Mooney with Rachel Lesebikan

28. figure © 2009 C. Baker

29. mobile video software – vision
Images © 2007 C. Baker

30. mobile
video
software
development

31. mobile video software
– development

32. mobile video software – available options
Image from http://r3nder.net/mt/mt-search.cgi?IncludeBlogs=7&search=mobileImage from http://mobile.processing.org/learning/example.php?name=photoslider

33. video software – beta (not ready for phones)

35. performance experiment – London July 2009

36. mobile video from performance for mixing

37. still of mixes for broadcast

38. streamed output videos

39. performance 1 – video documentation

40. streaming server

42. network
chain /
interaction

43. remote
mobile
streaming
interaction

44. remote
mobile
streaming
interaction

45. tech challenges – overall
• sensors: getting the right ones, moving to cheaper versions that needed
support to construct; advised to use ‘off the shelf’, which were cheaper but not
as robust and still have wiring / mobility issues – can’t change this at this point 
not the main thrust of project;
• phones and mobile software: getting the phones was not a problem; finding
a programmer to program on them was much trickier – 2 started, but Michael did
only sensor aspect  other intended mobile software shelved;
• garment construction took longer and money than intended, due to available
time of designers; still not ‘correct’ i.e. easy to work with in terms of easy access
to electronics or embedding wiring within for easy dressing;
• server and networking: got a shared server last summer, but had little
access to consistent support to set it up properly and quickly, lost easy access in
the fall, when it was most needed and then was made to wrap up the research
due to contractual end to funding.

46. • the whisper project a wearable devices and biofeedback
research project for Prof. Thecla Schiphorst and Dr. Susan
Kozel 2003-2006;
Siggraph 2005
related work – video + biosensing
http://whisper.iat.sfu.ca/
“…technology and
communications metaphors that
enable networked wearable
devices to communicate affective
states in a continuous manner”

47. SKIN, which examines the future integration of
sensitive materials in the area of emotional
sensing – the shift from ‘ intelligent’ to
‘sensitive’ products and technologies.
As part of SKIN, we have developed two ‘Soft
Technology’ outfits to identify the future for
high tech materials and Electronic Textile
Development in the area’s of skin and
emotional sensing.
The dresses show emotive technology and
how the body and the near environment can
use pattern and color change to interact and
predict the emotional state.
Philips Skin Probes
related work – biosensing + wearables
http://www.design.philips.com/probes/projects/dresses/index.page

48. related work – video + biosensing
Tina Gonsalves http://www.tinagonsalves.com/
Tina Gonsalves is one of those artists, whose work is most similar in intention, nature and
outcomes to mine (although I just discovered her).
FEEL:TRACE : responsive biofeedback installation (Australia 2006) :
“…is a psychophysiologically responsive video installation synthesizing art, neuroscience
and technology. The project explores new, more embodied languages of interactive and
emotional communication, investigating the inter-relationship of the internal body and the
external world. Using biosensors, the participant’s heart rate responses are monitored.

49. Blast Theory (2003) – Uncle Roy All Around You a participatory
performance as well as a mobile and internet game. It engaged users in
unique, theatrical activities using the participants / gamers in the city
using portable devices and phones rather than only avatars only on
computers.
related work – participatory performance

50. related work – mobile video
Mobile Video Art : mobile video paintings by Dean Terry
painting from a cameraphone & unedited. www.100lies.com
http://www.deanterry.com/blog/index.php/2005/10/
Speech Marks 2004 Steve Hawley
http://www.artdes.mmu.ac.uk/profile/shawley/image/10349

51. questions?