Looking ahead – field trials in iterative and exploratory design of ubicomp systems
•
2 j'aime•707 vues
We investigate in which forms field trials are a workable model as part of an exploratory design process for sporadic, mobile, non-work settings. A major concern of evaluating ubicomp systems is to study how practices and context of use emerge and develop over time when new technology is introduced. To introduce a sophisticated version of our own prototype in the course of an iterative design process, we conducted a public field trial of the system—a new platform for mobile democratic discussions in municipal planning—that we distributed via the Android Market. However, it turned out to be surprisingly difficult to evaluate our design in a setting that stretches over time, place, and without a preselected set of users. Analyzing our difficulties, we develop a general model for methods studying ubicomp systems. On the basis of this model, we characterize an openly interactive approach to field trials in order to look ahead rather than back.
Recommandé
Recommandé
Contenu connexe
Similaire à Looking ahead – field trials in iterative and exploratory design of ubicomp systems
Similaire à Looking ahead – field trials in iterative and exploratory design of ubicomp systems (20)
Dernier
Dernier (20)
Looking ahead – field trials in iterative and exploratory design of ubicomp systems
- 1. AARHUS UNIVERSITY UBICOMP 2012 SEPTEMBER 5-8 LOOKING AHEAD – FIELD TRIALS IN ITERATIVE AND EXPLORATORY DESIGN OF UBICOMP SYSTEMS MATTHIAS KORN AND SUSANNE BØDKER CENTER FOR PARTICIPATORY IT AND DEPARTMENT OF COMPUTER SCIENCE AARHUS UNIVERSITY MKORN@CS.AU.DK HTTP://CS.AU.DK/~MKORN/ @MATSCH_O0 Comp Ubi 1
- 2. AARHUS UNIVERSITY LOOKING AHEAD – FIELD TRIALS AND DESIGN UBICOMP 2012 MATTHIAS KORN AND SUSANNE BØDKER SEPTEMBER 5-8 MOTIVATION › A major concern is to study how use practices and context of use emerge and develop over time when we introduce new technology. Problem: › A mismatch between how field trials are understood and practiced, and the very nature of ubicomp systems. › Use situations that are not well understood and in the making. › Field trials are frequently conducted in a ‘design then study’ mentality. Thinking of field trials as a part of an iterative design process in order to look ahead rather than back. 2
- 3. AARHUS UNIVERSITY LOOKING AHEAD – FIELD TRIALS AND DESIGN UBICOMP 2012 MATTHIAS KORN AND SUSANNE BØDKER SEPTEMBER 5-8 AGENDA › Motivation › A general model for ubicomp trial methods › Our openly interactive approach to field trials › Looking ahead rather than back 3
- 4. AARHUS UNIVERSITY LOOKING AHEAD – FIELD TRIALS AND DESIGN UBICOMP 2012 MATTHIAS KORN AND SUSANNE BØDKER SEPTEMBER 5-8 CASE: NATIONALPARK MOLS BJERGE Photo: Niels Jepsen, http://da.wikipedia.org/wiki/Fil:Trehoje_i_Mols_Bjerge.jpg 4
- 5. AARHUS UNIVERSITY LOOKING AHEAD – FIELD TRIALS AND DESIGN UBICOMP 2012 MATTHIAS KORN AND SUSANNE BØDKER SEPTEMBER 5-8 SYSTEM: MENING@PARK 5
- 6. AARHUS UNIVERSITY LOOKING AHEAD – FIELD TRIALS AND DESIGN UBICOMP 2012 MATTHIAS KORN AND SUSANNE BØDKER SEPTEMBER 5-8 SYSTEM: MENING@PARK Hvad er din mening om stedet? Hvad mener de andre? Nationalparkbesøgscenter i Kalø — !"#$!%&%'()*+,#-&.,/..&0%.!1%! ? Scan denne QR kode med din smartphone til at deltage i diskussionen. Brug f.eks. Google Goggles, Barcoder Scanner eller QR Droid fra Android Market. Kræver “Mening@Park” fra Android Market Et forskningsprojekt ved For yderligere information kontakt Matthias Korn (mkorn@cs.au.dk) 6
- 7. AARHUS UNIVERSITY LOOKING AHEAD – FIELD TRIALS AND DESIGN UBICOMP 2012 MATTHIAS KORN AND SUSANNE BØDKER SEPTEMBER 5-8 SETTING: EBELFESTIVAL Photos (right): http://ebelfestival.dk/ 7
- 8. AARHUS UNIVERSITY LOOKING AHEAD – FIELD TRIALS AND DESIGN UBICOMP 2012 MATTHIAS KORN AND SUSANNE BØDKER SEPTEMBER 5-8 GENERAL MODEL FOR UBICOMP TRIAL METHODS impromptu passive walkshops interviews observations '0&%!2%'+ !"#$"%&!'&%() 0#'+%"&%2 %)*(+*#,#)&$(-$ .%/. +(4 &.#$%)*#"&%/'&(0 0#20!% '&&0'2 1'0&%2%1')&" 0#10#"#)&'&%*# '2&!'+ 2(,10#""#5 &%,# 0#'+6&%,# %)2(,1+#&# "(1.%"&%2'&%()$(-$ 2(,1+#&# ,(27#5$!1 &.#$10(&(&31# -!)2&%()'+ 8
- 9. AARHUS UNIVERSITY LOOKING AHEAD – FIELD TRIALS AND DESIGN UBICOMP 2012 MATTHIAS KORN AND SUSANNE BØDKER SEPTEMBER 5-8 OUR APPROACH: OPENLY INTERACTIVE › We attract, rather than recruit, actual future users to an open system. › We interact with (some of) them directly. › We take their experiences of using the system as a springboard for in-depth explorations of their perspectives and insights about › the concrete system, › the broader concept, › and potentials and tensions for future approaches in general. 9
- 10. AARHUS UNIVERSITY LOOKING AHEAD – FIELD TRIALS AND DESIGN UBICOMP 2012 MATTHIAS KORN AND SUSANNE BØDKER SEPTEMBER 5-8 LOOKING AHEAD › Studies based on trials that ‘leave use alone’ will always look back. › We need ways of working with fully functional prototypes to look ahead towards use practices that are still in the making. › Thinking of field trials as a part of an iterative design process in order to look ahead rather than back. 10
- 11. AARHUS UNIVERSITY UBICOMP 2012 SEPTEMBER 5-8 LOOKING AHEAD – FIELD TRIALS IN ITERATIVE AND EXPLORATORY DESIGN OF UBICOMP SYSTEMS MATTHIAS KORN AND SUSANNE BØDKER CENTER FOR PARTICIPATORY IT AND DEPARTMENT OF COMPUTER SCIENCE AARHUS UNIVERSITY MKORN@CS.AU.DK HTTP://CS.AU.DK/~MKORN/ @MATSCH_O0 Thank You!
Notes de l'éditeur
- Hi, I am from the new center for Participatory IT and the Department of CS at Aarhus Uni.This talk is not a success story.Rather, I am reflecting on our experiences with conducting field trials in a participatory design context – so a process that is meant to be iterative and exploratory.We think field trials should look ahead.We think field trials should be future-directed.We think field trials should be used to explore potential challenges and tensions in the introduction of new technologies in new contexts and settings.I wrote the paper together with Susanne Bødker, who is my PhD supervisor.We both come from a participatory design background where we explore ubicomp technologies with those ‘affected’ to empower them in their life and their technology use.
- -(1) A major concern is to study how use practices and context of use emerge and develop over time when we introduce new technology.(2) However, there is a mismatch b/w how FTs are understood and practiced today and the very nature of ubicomp systems being brought into sporadic and mobile settings. On the one hand, ubiquitous technologies are often designed for use situations that are not well understood and where use practices are really only in the making.Grudin, here, speaks of the ‘everywhere and forever’ challenge. That is, ubicomp systems are no longer about the ‘here and now’, but use situations stretch over time, place, and don’t have a predefined set of users. On the other hand, FTs in ubicomp are frequently conducted in a ‘design then study’ mentality, looking only at use how it is here and now and maybe how it has developed to become what it is.So, this mentality of ‘evaluating’ or ‘testing’ the prototype at the very end of a project is contradictory to a not well understood and evolving use setting and emergent use practices.->(3) We, therefore, don’t want to think of FTs as an assessment of an iterative design process, but rather we want to think of them as a part of it in order to look ahead rather than back.===Our paper adds to the renewed interest in method and the value of field trials, and how such experiments should (or should not) be conducted in more naturalistic settings.Brown et al. CHI ‘11, Kjeldskov et al. MobileHCI ‘04, McMillan et al. Pervasive ’10, Rogers et al. Ubicomp ‘07
- Throughout my talk I will make three major points, illustrated by a case example.First: A general model for ubicomp trial methods,which we used to analyze and reflect on a field study of our own prototype that hasn’t been quite as successful as we had hoped for.Second: Our openly interactive approach to field trials that we realized we should have made use more of.And third, I will argue for a need for methods to look ahead rather than back.But first a little bit about the case and the prototype system with which we sought to explore the field…
- The research project, within which this study took place, explores ways to support citizen participation with mobile, situated technology. The particular case is about a civic involvement process in the forming of a national park in DK, which is called MolsBjerge or Mols Mountains – well, the Danish version of mountains. This is a viable domain because the park is not a strict nature reserve, but many people actually live and work there. Thus, they have a strong interest in its development. As further evidence for that, we have seen lively participation in the parallel, offline citizen participation process.What we did here then was that we conducted a public field trial of an Android app. We developed this app in an earlier project through an extensive PD process and now transferred it into this new domain in order to probe the use of such technology here. The app was eventually used to solicit comments and feedback from citizens on a draft of the first 6-year-plan for the NP.
- The app is called Mening@Park -- ‘mening’ being the Danish word for opinion or point of view. After one has registered,Mening@Park allows its users, while out and about, to explore topics around the park through, for example, the usual maps and lists (a, b). It also allows users to add topics of their own (this is a topic’s detail view); to favorite a topic, express agreement or disagreement, to comment, and upload photos (c, d).In the later stages of the participatory design process, we paid special attention to how topics could be anchored in the physical world and to how they can be tied to users’ experiences in the park. One way we explored that is, first, [STEP] throughlocation-specific QR codes that lead to a specific topic and that we deployed at popular locations around the park. And, second, through location-based notifications that would trigger when a user is in the vicinity of a topic she has previously shown some interest in or otherwise matches her interest profile.
- The app is called Mening@Park -- ‘mening’ being the Danish word for opinion or point of view. After one has registered,Mening@Park allows its users, while out and about, to explore topics around the park through, for example, the usual maps and lists (a, b). It also allows users to add topics of their own (this is a topic’s detail view); to favorite a topic, express agreement or disagreement, to comment, and upload photos (c, d).In the later stages of the participatory design process, we paid special attention to how topics could be anchored in the physical world and to how they can be tied to users’ experiences in the park. One way we explored that is, first, [STEP] throughlocation-specific QR codes that lead to a specific topic and that we deployed at popular locations around the park. And, second, through location-based notifications that would trigger when a user is in the vicinity of a topic she has previously shown some interest in or otherwise matches her interest profile.
- We introduced this app during the Ebelfestival. The Ebelfestival is a week-long local folk and harvest fair in the Danish autumn holiday week. We wanted to use the buzz of the festival and the activity around it to deploy the app and get in contact with citizens. The app was made publicly available on the, then called, Android Market and advertized via fliers and posters at the festival and other means beyond the festival. As I said, we also hung up these QR codes around the park. On the left is one at a very popular location that tourists visit a lot and where locals take a walk or walk their dogs.During the festival, we mainly stayed at the official tent of the NP and roamed around the festival site. We engaged with visitors in informal impromptu interviews where we also let them try out the prototype. We asked about their use of mobile phones and other gadgets in the park and generally about their perceptions of the citizen participation process going on. We also observed how visitors interacted with the various forms of information provided at the tent and with NP staff. We also interviewed the staff on-site and, in a more structured format, after the festival. Lastly, we also conducted an outdoor workshop to which I will come back in a minute.So, [.] we ourselves fell to the fallacy of the design-then-study mentality in ubicomp, what we now reflected on more thoroughly with this paper. Large parts of the setup were, without us being much aware of it, implicitly geared towards ‘evaluating’ or ‘testing’ the app in the field.-Our main criticism to our own approach is that we were hoping for people to actually use the system on their own will in a sufficient fashion, that is, sufficient people and sufficient use.-And we, therefore, were in part relying on later analyzing both the created content and usage logs in addition to our other interventions (workshops, interviews, and observations).-But sufficient use was not the case – even in this lively and meaningful context it was situated in.
- In order to understand what went wrong, we went about analyzing and reflecting on our method on a number of dimensions, using this general model for methods studying ubicomp systems. It describes the 5 dimensions: use situation, involvement of the investigator, participants, time, and sophistication of the prototype.For an example, we could see classical field trials and PD workshops at opposite extremes:In FTs, the finished prototype is used by voluntary users in real, un-tampered use situations at a safe distance from designers and researchers. Whereas in WSs, we typically find mock-up prototypes used in workshop settings in close interaction with the investigators.However, there are of course many other ways to combine these dimensions [a methodological choice]. For the talk, I tried to directly map some of our interventions onto the model. (Disclaimer: Be aware, though, that this is just for a descriptive, visual account of the methods we applied. There are endless aspects to each of these dimensions.)According to the true ‘design then study’ mentality having a “complete” and functional prototype was kind of a given for us and our starting point in this study. This is how we came into the study in the first place – that is, by taking an existing prototype into a new domain (as I have mentioned).From there, we carried out several different interventions. For example, on the far right, you can see the passive observations that we carried out at a popular tourist location out there in the park, closer to the envisioned use situation [realistic]. So, passive observations also means no involvement, coincidental participants, and we didn’t do anything to compress time, to make things happen faster. This intervention was of little use to us, because we could not learn much with the low level of interaction with the participants and the limited use of the app.In contrast, on the far left, we tried to recruit citizens for an outdoor workshop walking around the festival site where we together (co-)explored topics in the area through Mening@Park. Here, we for example compressed time. Instead of letting use of the system progress in real-time (ie. on its own pace), we rather ‘made use happen’ by bringing QR codes that we installed to the participants’ attention and asking them to react to the connected topics using Mening@Park. For this activity, we tried to recruit participants among interested stakeholders and we were, obviously, highly involved. This intervention, while not being the most critical part, provided the most interesting insights for our ongoing exploration of the field.In the middle I have also mapped our informal interviews at the festival site.But as I said, there are many other ways to work on (and between) these dimensions.What we have learned from this mix of methods is not so much about the suitability of the concrete prototype, but more about the general tensions and challenges of introducing some form of mobile technology into the citizen participation process in this specific setting in the NP.Two examples!:First: While users of the system actually used the agree/disagree functionality (that we saw) a lot, we found out that stories and other personal narratives is how people [lay citizens] express themselves eg. to the NP staff at the tent.Second: We became aware of that individual topics are (even spatially) much more complex than we could possibly express with one single location. They are not only about a specific place itself, but more about the area, the infrastructure, and how that place is connected to other places among other aspects. (The concept of location does not suffice in our context.)
- So, reflecting on the analysis of our method, we propose this openly interactive approach.While our starting point was a sophisticated version of the prototype for people to try out and use on their own, we found that we actually fell short of interacting with participants sufficiently and in a way that would help us to really explore the future application of mobile deliberation technologies in the park.We think our shortcomings lay more in how we approached the study (with what mentality), rather than what methods we actually used. The way we want to think of FTsin the future is codified in this approach.… (rather than only representative participants)… and more closely in whatsoever method one chooses for the purpose.…So, basically an intentional retreat from purely ‘naturalistic’ (or even ‘realistic’) field trials, which they really never are anyway. We take exploration of the field serious by investigating what implications future technologies pose on people’s practices, on the field, and so on. We use prototypes as probes to better understand the future use situation.
- In conclusion, we found it difficult to both study use as un-tampered by the designer, and look ahead at the same time. We argue that studies based on trials that ‘leave use alone’ will always look back, and hence cause problems as part of an exploratory design process.So, whileclassical field trials fundamentally look back on the past, we need ways of working with fully functional prototypes to look ahead towards use practices that are still in the making.In our case, people do not, already, walk around in nature and comment on a plan, even if they are concerned citizens that otherwise take part in local democracy.Our main point is that we don’t want to think of FTs as an assessment of an iterative design process, but rather we want to think of them as a part of it – i.e., a counterpoint to the prevalent ‘design then study’ mentality in ubicomp.
- But we do have learned something about the (possible) use of mobile technology for citizen involvement in the park’s development. So, it was a success story after all.