Metanomics Transcript May 27 2009, Measuring Value in Virtual Worlds

METANOMICS:
DO VIRTUAL WORLDS PROVIDE MEASURABLE VALUE?
INSIGHT FOR ENTERPRISE WITH DR. MITZI MONTOYA
MAY 27, 2009

ROBERT BLOOMFIELD: Hi. I’m Robert Bloomfield, professor at Cornell University’s
Johnson Graduate School of management. Each week I have the honor of hosting a
discussion with the most insightful and the most influential people who are taking Virtual
Worlds seriously. We talk with the developers who are creating these fascinating new
platforms, the executives, entrepreneurs, educators, artists, government officials who are
putting these platforms to use. We talk with the researchers who are watching the whole
process unfold. And we talk with the government officials and policymakers who are taking a
very close look on how what happens in the Virtual World can affect our Real World society.

Now naturally, we hold our discussions about Virtual Worlds in Virtual Worlds. How else
could we find a very real place where our global community can convene, collaborate and
connect with one another. So our discussion is about to start. You can join us in any of our
live Virtual World studio audiences. You can join us live on the web. Welcome, because this
is Metanomics.
ANNOUNCER: Metanomics is filmed today in front of a live audience at our studios in
Second Life.

ROBERT BLOOMFIELD: Hi, and welcome again to Metanomics. Today we’re going to get
certainly very different looks at the future of work and Virtual Worlds. And their focus is
going to be on concrete data. Professor Mitzi Montoya, North Carolina State University, will
tell us about her experimental research on how virtual teams work and how that work is
affected by a range of technologies, not just these newfangled Virtual Worlds, but going all
the way back to Lotus Notes at the “turn of the century.” Mitzi’s segment will be sandwiched
by other great data. Tony O’Driscoll, of Duke University, kicks us off with a look at some
research on the future of virtual organizations. And Erica Driver, of ThinkBalm, will close by
telling us about her hot off-the-presses survey on how businesses are using Virtual Worlds,
how they evaluate their efforts and, of great interest to all of us, their investment plans for

the coming year.

Thanks to all of you who are attending Metanomics today, including those who are viewing
live on the web. Please do join in with you comments and your questions.

ANNOUNCER: We are pleased to broadcast weekly to our event partners and to welcome
discussion. We use ChatBridge technology to allow viewers to comment during the show.
Metanomics is sponsored by the Johnson Graduate School of Management at Cornell
University and Immersive Workspaces. Welcome. This is Metanomics.

ROBERT BLOOMFIELD: Before we get to our main guest, we’re going to take a few
minutes to talk with our corporate learning correspondent, Tony O’Driscoll, who will help us
put the future of the virtual workplace in the spotlight. Tony, welcome back to Metanomics.

TONY O’DRISCOLL: Thanks so much, Rob. Good to be back.

ROBERT BLOOMFIELD: Yeah. Glad to have you here. Our focus today is on virtual teams
and, indeed, entire virtual organizations, and, of course, where Virtual World technologies
might fit in. Since a key focus of our talk is going to be on research, I thought we might start
by noting that yesterday was the deadline for the National Science Foundation solicitation
for research on VOSS, Virtual Organizations and Sociotechnical Systems. Now what types
of research do you expect to see coming out of that?

TONY O’DRISCOLL: I think we’ll see a lot coming out of this, Rob. VOSS, V O S S, is part
of OCI, the NSF’s office of cyber infrastructure, and it’s actually only one of 43 areas of
research. And OCI has really focused on the development and provision of state of the art
cyber infrastructure resources for 21st century science, engineering, research and
education. That’s a mouthful, but that’s right off their website. I guess OCI’s big claim to
fame, Rob, is their $3 million investment in this little thing called NCSA Mosaic, which has
led to over a trillion dollars they claim of economic benefit. So VOSS specifically is focused
on what are effective virtual organizations, what constitutes effective virtual organizations
and under what conditions can they enable scientific engineering and education production.

So they’re really investing in academics and researchers, like myself and like Mitzi, to take
on the notion of virtual organizations and really understand, from an empirical perspective,
whether or not they add value.

ROBERT BLOOMFIELD: How much do you think we’ll see that is not strictly technical or
computer-oriented, but just thinking about things like corporate governance, management
strategies, things that would more--be school outside of computer science?

TONY O’DRISCOLL: There’s a couple of other programs under OCI that probably address
that. One is a cyber infrastructure team, which talks about training, education, advancement
and mentoring. And that one’s really focused on the collaborative endeavor. I think there’s
definitely going to be spillover effects there. The other one is CDI, Cyber Enabled Discovery
and Innovation, which would ask the questions: Can you actually do generative
learning-type work inside of these environments? In general, the research--and Mitzi will
be talking more about this--is: Is there a theory of instrumentation for us to do this kind of
research? But, fundamentally, the question is: Is it better? Is it better than X? Is it better
than WebEx?

And then the next question would be: Why is it better? What are the reasons that these
environments feel better to participants? And then the third question is: Where does it add
the most value? So that’s the kind of streams of research I’m seeing is instrument
development, kind of like what Mitzi’s going to talk about, the application of that instrument
to do comparative analysis. And then outcome-based variables to say it really does make a
difference, and it’s worth investing in.

ROBERT BLOOMFIELD: Okay. Now Virtual Worlds play potentially an obvious role in
virtual organizations by giving people a rich way to communicate across distances, and a lot
of us, I think, do look at Virtual Worlds as being primarily a place to eliminate geographical
constraints. But your own interest in Virtual Worlds, for the virtual organization, came
through their use as gaming platforms. So what role do you see gaming platforms having in
virtual organizations and the future of work?

TONY O’DRISCOLL: This work came out of some of the work done at IBM and the Global
Innovation Outlook, and the big question was: What’s the future of enterprise? And the old
saw “the future’s already here, just not evenly distributed” is applied here, where we said,
“Hmm. Can we find a place where we think the future work already exists?” And the
characteristics there was that it was open, it was global, it was virtual, it was volunteer
workforce, and it was knowledge-based. As we looked around to try and find some kind of
environment with that signature massively multiplayer online role-playing games emerged.
And what we started to do was observe how leadership and decision-making and
innovation occurred within these massively multiplayer online role-playing games in an
environment that is essentially polar opposite of command and control, but still, things get
done. That’s kind of how I ended up really becoming enamored with this whole environment.

ROBERT BLOOMFIELD: And about when was that?

TONY O’DRISCOLL: About three and a half years ago was when we started looking at
MMOs. And then, once we got through looking at MMOs and their applications to
leadership, the Second Life awareness bubble kind of came into light, and IBM’s Virtual
Universe community was established, so it was a natural extension then to start looking at
more of the virtual social worlds rather than the MMORPGs.

ROBERT BLOOMFIELD: That inspires me to give a plug for another Metanomics show.
Those of you who are interested in what IBM has been doing, we had Ian Hughes, former
IBM Metaverse evangelist, on the show a while back, and you’ll definitely want to take a
look at what he had to say. He was instrumental in creating a lot of IBM’s web-based and
then Virtual World-based community and the blog eightbar.

Moving on, Tony, everywhere I turn this month, I see someone else foretelling the death of
the traditional company. For example, Margaret Regan, who was at your conference in
Washington, D.C. last month, wrote for CNN this month and says, and this is a quote, “By
2042, there will be no workplace as we know it. Goodbye, wired desks. Hello, wireless

personal wearable networks.” And she goes on to say, another quote, “And work will feel
like a Hollywood movie, in which managers, directors and team members come together
around a specific project, collaborate intensely and then disband.” So she clearly sees
virtual companies in virtual teams as the way of the future. Would you sign on to that?

TONY O’DRISCOLL: Yeah, I think, to a certain extent, looking out to 2042 is a long, long
way, and one thing we know about technology is, it’s iterative and evolves on top of itself so
it tends to go at an exponential rate. The work we did with the Global Innovation Outlook
future of enterprise, we went all around the world and interviewed visionaries, like Margaret,
as to where things were going. And one of the most provocative quotes that came out of
that, not necessarily saying it’s where things are going, but it made us think was the future
could consist of one billion one-person enterprises, people who move frequently from
project to project as their skills and focus shift.

Now underlying that, Rob, is Kose’s law. I’m getting into your territory here so I better watch
out, but the notion that firms will grow or shrink to the size of a transaction cost. So if the
transaction cost inside the firm is lower, then we’ll do it. At one point in time, Ford imported
sand to make windshields. They don’t do that anymore. They can find someone else who
can make the windshields. They could have FedEx ship it, and they can manage the whole
through NIP Network. So if we buy that concept that there’s going to be shrinkage around
core competence and then you can create ecosystems to wrap capability around endeavor,
it does seem that moving from hierarchy to matrix to ecosystem does seem like a logical
progression. How that unfolds within the enterprise, however, I don’t think it’s going to be a
“flip the switch” and you have more of a network ecosystem type of model, as opposed to a
matrix. But I do see it moving in that direction.

ROBERT BLOOMFIELD: Do you see certain industries leading the way?

TONY O’DRISCOLL: Yeah. As we talked at the 3D TLC Conference, one of the reasons
3D TLC was pulled away from Engage by Chris Sherman was entertainment. It’s clearly
hitting entertainment pretty hard right now. As I observe it, industries that have to do it from
a survival perspective where their business model is being threatened are obviously more

motivated to move. So as the advertising revenue is drying up in the entertainment industry
and as the bigger amount of the demographic is moving into Virtual Worlds, and couch
potatoes are becoming mouse potatoes, the entertainment industry is essentially being
dragged into trying to figure out how to leverage these new interactive platforms to generate
revenue. I think there’s a lot of issues for them there. One advertising dollar in broadcast
media is not equal to one advertising dollar in web-based or 3D-avatar-mediated media,
but they’re certainly dealing with that.

The second one is business opportunity, so clearly the Suns and the Cisco’s and the IBM’s,
they’re looking to generate the next generation collaborative infrastructure for the enterprise,
which will be a “mashup” of 2D and 3D internet technologies that allow you to be there
virtually and leverage that at a distance. And I think there’s a race on there to come up with
the quote/unquote “killer” enterprise collaborative infrastructure app.

Then the next one which you and I have spoken about before, very prominent in 3D TLC,
were the oils and pharma. I’m not a hundred percent sure why. At first blush, they would
seem like they’re quite risk averse. But, if you think about oils for instance, a company like
BP, it’s very global in nature, and they need to distribute their expertise into regions of the
world that are perilous, you know, either the North Sea or certain regions in Sub-Saharan
Africa where you need expertise, and you need access to expertise, and perhaps
sometimes those experts are not as willing to go to those areas because of danger. So that
might be one reason where they are taking it up.

And then, finally, we’re seeing a lot of industries pick up the application of these
technologies for efficiencies. I’m sure Erica will talk about this from her report, but the
application of these technologies for training, for collaborative events and meetings, those
type things. And that tends to be broad-brush across many industries. It’s just basically a
cost play.

ROBERT BLOOMFIELD: Very interesting stuff. I guess I have one remark on the Kosian
view of transaction costs because actually what we’re seeing right now--you wouldn’t think

that health-care policy would be a natural topic for a show on virtual teams and virtual
workplace in Virtual Worlds, but, in fact, right now U.S. health-care policy really forces
formal employment relationships because of the way that we tie health care and tax breaks
into formal employment, as opposed to independent contractor types of arrangements. I’ve
been following the health-care debate a little bit, and people mostly on the right, mostly the
conservatives have been very sensitive to this issue and have been talking about changing
to more portable health care, to foster--in their words, they’re talking about fostering
entrepreneurship. But I think it fits very well if you talk about virtualizing the organization and
having, as you said, a billion businesses run by a billion entrepreneurs. I think that’s what
you would need to do. Oh, and I see someone has a link for that. Wonderful.

TONY O’DRISCOLL: The one-liner there, Rob, is kind of, the traditional enterprise wants
command and control of employees on projects, and the idea here is perhaps this virtual
organization infrastructure--and there is a lot written about it in medicine--is the
orchestration and coordination of capability around endeavor. So it’s more fluid, and it’s
about being able to more dynamically bring together various and sundry sets of capability
around an endeavor, which is an activity that may be presaged or may not be. And the
question becomes: Is the enterprise the most effective mechanism for that or not? And it’s
popping up in health care, but also globally integrated enterprises, like IBM, want to figure
that out. As well as the smaller companies who are, you know, they’re the niche players,
and they want to create ecosystems of value that they could collaborate around. All of those
point very much towards the notion of virtual organizations.
ROBERT BLOOMFIELD: Well, thanks so much for joining us again, Tony.

TONY O’DRISCOLL: It was a pleasure, as always, Rob. Thank you.

ROBERT BLOOMFIELD: I’m sure we didn’t solve all the issues so I’m sure we’ll see you
back.

TONY O’DRISCOLL: I look forward to it.

ROBERT BLOOMFIELD: Now let’s turn to the main event. Our main guest today is
Mitzi Montoya. Dr. Montoya, a profession and assistant dean at North Carolina State
University College of Business, has been measuring the effectiveness of Virtual Worlds with
her colleague Dr. Anne Massey and has developed a measurement scale called perceived
Virtual Presence or Collaborative Virtual Presence, to help assess the value of interaction in
Virtual Worlds.

When Dr. Montoya and her research team announced the scale late last year, she did say
the more present that users feel in Virtual Worlds the greater the effectiveness of training,
collaboration, education or presentation. So those types of results I know are of great
interest to those of us who are spending a lot of time studying these Worlds, and I, for one,
am really looking forward to hearing about it. So, Mitzi, welcome to Metanomics.

MITZI MONTOYA: Thank you, Rob. Good to be here.

ROBERT BLOOMFIELD: Glad to have you. I’d like to start off with some research you
published way, way back in 2001 and 2002, in which you had 35 virtual teams, with
members in the U.S. and Japan, working on a marketing case. And they all worked together
using Lotus Notes. Before we talk about the study, let’s just talk about teams for a minute.
You argue that teams engage in four different types of behavior: conveyance, which is
sharing thoughts and opinions; convergence, which is critically examining others’
contributions; social relational behaviors, which are generally off task, but kind of fun; and
process management, where they’re trying to direct the team. So could you walk us through
some examples of the difference between conveyance and convergence?

MITZI MONTOYA: Sure. So as an example, conveyance would be if we’re working on a
project together and we need to just exchange information. So I might share documents or
papers or data with you, and it’s primarily for the purpose of giving you my opinion or just
information and facts I’ve found out. Whereas, convergence, we’re trying to make a
decision. We’re trying to come together. And convergence is where you primarily have in
teamwork, that’s where you see the conflict and the negotiation and different decision
processes at play. But, ultimately, that’s where the rubber hits the road, and the team needs

to decide what it’s going to do as a next step or complete the project and make a single
unified recommendation.

ROBERT BLOOMFIELD: So you had a paper called Because Time Matters: Temporal
Coordination in Global Virtual Project Teams and a companion paper Synchronizing Pace in
Asynchronous Teams. It seems like teams that do well spend a lot of their time on
convergence. So why is that?

MITZI MONTOYA: In these particular studies that you mentioned here, we were looking at
time-limited teams. So that means these teams had something to do and a deadline in
which to do it. So think new product development teams or new service development teams.
You have work to complete. You have a launch date and a schedule to keep. And so in
these kind of contexts where you have a deadline, coordinating the work of specially
distributed teams is the most significant challenge. And, therefore, it really does require
working through these typical team behaviors in a systematic and coordinated way. So all
teams do these things. All teams convey information and data.

They ultimately have to converge and make decisions, but, to get there, there also are
social relational aspects of teamwork, of trust, all the things we normally think of and even
just interpersonal relationships. And there’s also process. So when we have teams that
have project deadlines, they have to work through these things in a coordinated fashion,
and that’s where virtual teams often have breakdown is in the lack of synchronized
coordination of their effort.

ROBERT BLOOMFIELD: Now you had a controlled manipulation in that study that helped
half of the teams with that challenge. You called it a temporal coordination mechanism, and
it was actually a combination, as I understand it, of instructions and some things you
actually implemented into the technology Lotus Notes. So can you tell us what that
manipulation was and why it helped?

MITZI MONTOYA: Yeah, basically what we did was, in this case, the teams had a work
project to complete, and, for half of the teams, we simply turned them loose and let them

work however they saw fit, and, for the other half of the teams, we gave them a process
structure that facilitated those four behaviors that characterize all teamwork. So that is, we
gave them a process structure for how and when they should convey information, what
would be some techniques for converging. We kicked off the teams with a little bit of a social
exercise, and we gave them a process by which to manage their entire activity.

So not entirely surprisingly, although when we drilled down into the details of why, we
discover that teams that have a better process structure that, in fact, mimics face-to-face
teamwork which allows for efficiencies and more effective process and decision-making,
they perform better. And then when you look at the aspects about that, that they do, why is it
they perform better. What we were able to do is back into studying exactly their
communications and behaviors and understand why were some teams more or less
effective, and that really is where the research has taken us as we try to understand virtual
team behaviors as it relates to performance.

ROBERT BLOOMFIELD: I know one of the things that you looked at, you wrote another
paper, I understand, using the same data, but looking at a different question. The paper I’m
thinking of is Getting It Together: Temporal Coordination and Conflict Management in Global
Virtual Teams. And so there you showed that different teams would deal with conflict in
different ways, and the lists, or the five, just to sort of get them out there, the five conflict
management behaviors that you looked at were: avoidance, basically failing to confront the
problem; accommodation, which is showing a concern for others and giving in to their
concerns; and then on the flip side, competition, which is pursuit of self-interest. Then you
have collaboration, which is integrating the interests of everyone involved. And compromise,
which is pretty much what it sounds like.

So just to elaborate a little bit, here’s how you describe competition, and here I’m quoting
from your paper, “Competition behavior is characterized by each party’s pursuing his or her
own interest without regard for others. This behavior involves concealment of information,
competitiveness and negative attitudes toward alternative solutions. Competitive
interactions typically involve the use of power and domination as one party tries to force its

views on the other.” This doesn’t sound like teamwork to me, but you find in your data that
competition aids performance. Why is that? And do you think there is something about the
context of Lotus Notes as opposed to face-to-face interaction that makes the difference?

MITZI MONTOYA: Yeah. This is a really counterintuitive finding, and it’s entirely a function
not of Lotus Notes but of the characteristics of Lotus Notes. So that is the way in which
these teams had to communicate was very lean communication media. So that is, it was
asynchronous communication. They were communicating via discussion forums. No
interactive feedback. Obviously, no verbal cues whatsoever. So that’s the notion of a lean
communication context. And, as it turns out, what would normally be a negative conflict
management behavior, which is, competition is ordinarily considered to be a negative
behavior, it had a positive effect on performance because of the way it played out.

So this study was a classic example of illustrating that there are differences from
face-to-face, in terms of how you need to communicate to be effective in these different
communication environments. So the reason competition and competitive behavior
exhibited positive effects on performance was because the way it played out and Lotus
Notes was over-communication. And what’s interesting is that’s something that a lot of
research has tended to show fairly consistently now. For most virtual teams communicating
by all other forms, let’s leave off the new ones, the exciting new Virtual Worlds and virtual
environments, all prior forms of communication, which would include email, postings on
various discussion forums, exchanging of documents, even telephone, that almost always a
common approach to managing virtual teams effectively is to over-communicate--so
increase the volume of communication because there is something lost in translation. And
what we argue what’s lost in translation is the interactivity, the richness of communication
that we have when we’re face to face, and some of which is a function of this sense of
presence. Right? And a very recent study that we just completed, where we put teams
together to complete work in a Virtual World context, in this case Second Life, we see the
normal conflict management behavior rules apply. So things that--

ROBERT BLOOMFIELD: Normal meaning as if they’re face to face.

MITZI MONTOYA: As if face to face. So all old research on conflict management, in terms
of what’s good or bad as a conflict management style, originated out of small-group
research looking at face-to-face teams. And so what we’ve studied over the last 15 years,
when you have mediated communications, how do those things change? In effect, we’ve
parsed through five or six studies here, and the short answer is: a lot of things change. And
you have to pay attention to many little things that you don’t have to pay attention to in face
to face because it comes natural and naturally to the participants. In a collaborative virtual
environment, where you have much richer environment and a greater sense of presence,
we find a much closer parallel, the closest parallel that we’ve seen to date in studying these
different technologies to face-to-face communications.

ROBERT BLOOMFIELD: Before we get into your study in Second Life, I’d like to talk a little
bit about culture and context because, when we move into these virtual spaces, they do
provide more context. And I understand from reading one of your papers, when culture and
style aren’t about clothes, that you actually see some differences in how you need to
provide technologies across cultures. In that paper, you spell out different communication
styles, in particular I guess the strongest distinction I was seeing was between the U.S. and
UK people and Asian, Chinese in particular, I believe you mentioned.

MITZI MONTOYA: Right.

ROBERT BLOOMFIELD: And I see this a lot in the literature--the story goes, I guess, that,
in the U.S., we are more individualistic, which I knew, compared to the more collectivist
perspective that a lot of people in Asia are taking. But I was surprised that you then translate
that into different styles of communication, that in the U.S. more direct and independent of
context, and in Asia communication tends to be more indirect and context dependent. Can
you just elaborate on that a little bit, to give us a sense of what that means?

MITZI MONTOYA: Yes. And actually there’s been extensive research, which was an
interesting area to learn about as we moved into this space, that very tightly links

communication style to national culture. Now these are generalizations, and it’s a
continuum, and you find differences even within any single country, but these difference in
communication style can create challenges for virtual teams. One of the most common
problems you hear in virtual teams, the two things almost always mentioned will be, they
have trouble coordinating, and they have communication problems. And the communication
problems are almost always communication style differences. So whether you are more
direct or indirect. Are you more [alert?] or more succinct? And are you more personal in your
communications, or is it more about the context that you’re studying? And is it more about
the task or how you feel about what you’re doing? Right? There’s not a right or a wrong
about those different dichotomies; they are simply differences. And when you have
opposites meeting and working together, that can cause conflict that leads to breakdown in
team process.
So not only do we have individual differences along those lines, a lot of research shows that
different countries have general tendencies. And so, as one example that comes up in a
virtual team is when you have members from different countries, who speak different
languages, you will find that not only is language a problem, but their communication style
can come into play. And so you can use the technology to facilitate or, I should say, support
multiple styles of communication, if you know and are aware of the fact that your different
team members have different communication styles. And that’s one of the advantages,
actually, that a technology-supported team can have over a face-to-face team.

ROBERT BLOOMFIELD: I had a really interesting discussion on Metanomics back in
December, with Victoria Coleman, vice president of a research group for Samsung
Electronics, a Korean company. She was running a group that was actually headed in
San Jose, and they were trying to use conference calls with their Korean counterparts. One
of the big problems they ran into was that they had to do it in English, of course, because
the English speakers didn’t know any Korean at all. But the Koreans didn’t have sufficient
command of English, and so they didn’t feel comfortable using a phone conference call. So
Victoria took them into Second Life, where they could talk there, and there they had a
surprise. Here I’ll just quote from what she said to me on the show. She said, “The same
Korean people that were really reluctant to get on the phone and were very shy and
wouldn’t say anything would show up in the Virtual World environment, decked out in

completely fantastic outfits. They would be very sociable, very talkative. It was really like
talking to a completely different set of people.” So her conclusion is, she says the fact that
Second Life created this medium that let them connect with us, but in a way that amplified
their skills versus making the lack of English into a central point, all of a sudden became a
truly empowering experience for them. I mean I guess if I interpret that through the lens of
the cultural differences you just emphasized, it would be that Second Life gave the Koreans,
who prefer more context for their communications, a benefit that maybe Americans wouldn’t
have gotten. Am I on the right track?

MITZI MONTOYA: Yeah, that’s a great example in fact. And another example, another way
that you could see this would be something we saw illustrated in our studies where we had
Chinese participants, and the language was English in this case, although it could have
been reversed. We could have had Americans working, and they speak Chinese. It’s
obviously not their native language, but, if you make the medium then text-based, one thing
that a lot of research shows is that people can write and read generally better in a foreign
language than they speak or hear. So if you reduce the speaking requirement and you move
toward a text-based communication, that actually benefits the non-native speaker so that
they can express themselves more clearly. So those are both examples of how a different
medium for communication can reduce some of the communication challenges that the
team might have been facing.

And, to answer a question that was posed, which is a good one, is that, yes, a team leader
or team facilitator should, in fact, help raise awareness of the different communication styles
and then ideally be aware of the technology affordances that could be utilized to address
those differences and reduce the barriers to communication that might exist due to those
differences.

ROBERT BLOOMFIELD: Okay. And that takes us fairly naturally then to this notion of
virtual presence. Regular viewers of Metanomics have heard a lot of people define what a
Virtual World is, and does it need this capability or that capability. But you’ve taken a more
empirical approach where you have been trying to assess how present users feel in Virtual
Worlds. Can you talk a little bit, I guess, first just about the scale, what are the dimension of

the scale that you are trying to capture?
MITZI MONTOYA: So yes. And part of what started us down this path, as my colleague
Anne Massey and I started looking at Virtual Worlds, is, we started looking at the research,
much of which has been done in the gaming and military simulation context. One thing you
see is this idea of presence seems to be a desirable attribute as some kind of self-evident
goal and a pervasive belief that more sense of presence is good. But we could not find any
consistent way to assess that, like there were many different measures and studies from all
sorts of angles, but no validation using fairly standard measurement development
techniques.

So part of what we started doing is, we looked at the literature which suggests that presence
is a very complex notion, and we started talking about it as presence as metadata. It really
has everything to do with your sense of the context around you and the information around
you and others around you and how those things are all interrelated. Presence includes
input from multiple sources so it’s not just sound or text or visual; all of these things can
create a sense of presence. And so this is a very individual factor. The more we studied
what others were looking at and what has been said, we’ve broken it down into three major
dimensions of factors that we think sit underneath this concept of collaborative virtual
presence. And our focus is on collaborative so not just my sense of awareness in a virtual
space, that I’m here to work with someone else. So my ability to collaborate in the
collaborative virtual presence that might be a part of that.

So we’ve broken it down into three pieces and the idea that there are three relationships
that are essential to collaborative work and how presence might contribute to that. And that
is the relationship between self and the environment, and that’s described as immersion,
and it’s the degree to which I am immersed in the environment and feel myself to be there.
And second is the relationship between myself and the task, and we call that absorption,
and that’s the degree to which I get lost in what I’m doing and what I’m working on with you
in this virtual environment. And then the third dimension has to do with the relationship
between self and others, so that’s my awareness of others in this space with me. And so
we’re defining collaborative virtual presence along these three dimensions: immersion,
awareness and absorption, which are a function of my relationship to other things: people,

the task and the environment.

And our study thus far relates back to something Tony mentioned at the outset. The NSF
VOSS program, as well as the NSF CDI program, are both programs that have provided
preliminary support for our work, and we have additional proposals underway. What we are
doing is, we have developed the scale. We started with some 50-odd measures. We’ve
reduced it down to 29. This is involved data collection with, to date, 190 people participating
in exercises in Second Life. We have another 145-ish participating in exercises in
Wonderland. We are collecting data with participants in ProtoSphere because an important
part of having a valid measurement scale is that you validate across platforms and with
different affordances provided by the environment, to understand what drives perceptions of
collaborative virtual presence.

And we’re also collecting additional sensory data, including eye-tracking data, as well as
physiological response data, to understand the relationship to what you feel what you say,
which is what we typically capture as perceptual measures, but also how you are
responding physically to your perception in a space.

ROBERT BLOOMFIELD: So I guess to summarize, you have these three dimensions of
this collaborative presence: It’s to what extent are you in the space, to what extent are you
in the task, and to what extent do you feel like you’re really with other people. Am I close
enough summarizing it that way?
MITZI MONTOYA: That’s exactly right. Yes, that’s right.

ROBERT BLOOMFIELD: Have you gotten enough data to figure out whether these are
positively or negatively correlated? You talked about one of the things groups do is engage
in social interaction, which is off task. I think a lot of people have worried that you come into
a place, like Second Life, it’s not just that the place itself is game-like, but that you may be
tempted to interact with people off task. Are you going to be able to get a sense of whether
that’s the case?

MITZI MONTOYA: Right. So a couple of things that we’ve done in our first preliminary
studies, we do see a positive relationship with performance. That’s an overarching
statement. But then, when you break it down and look at what people are doing and what
the benefits are to performance, one of those studies we’ve run is, we’ve allowed people to
choose which tools they want to use for their team work. So providing a whole suite of
Web 2.0 tools plus Virtual World access. You provide a whole suite of tools, allow teams to
choose and make sure they all have sufficient training and experience with these various
technologies. And, interestingly enough, they all gravitate toward, or I should say the best
performing teams all gravitate toward Virtual World technologies for the social/relational
piece of teamwork, and that is important for their performance. So for teams that don’t do
that and don’t, in fact, establish a relationship among team members, they have lower
performance, and the Virtual World tools appear to be highly effective for that.

So a big part of what we’re doing now is breaking down then which of the dimensions of
collaborative virtual presence contribute most significantly to performance. And I can tell you
my advanced guess, my hypothesis is that it’s going to depend on the task. Right? There
are lots of different types of tasks, as we know from past research so, depending on what
the team is trying to do, should have a significant impact on what matters in terms of my
sense of being there in a Virtual World.

ROBERT BLOOMFIELD: Let me ask, there’s already in chat people are asking for the
paper. I understand you’re revising the first paper that really lays out the scale right now.
When do you think we’ll be able to see it in a working-paper form?

MITZI MONTOYA: Well, we hope that, if any reviewers are listening, very soon. We’re in
second-round revisions here. Well, we just completed second round so we’re in third review
on the paper. We are very hopeful that we will be able to release the paper shortly, which
should mean in the next month or two. By end of summer, we expect to be able to post the
working paper because we should be through the worst part of running the gauntlet of
reviews.

ROBERT BLOOMFIELD: Okay. I know what that is like, and best of luck with that process.

MITZI MONTOYA: Thank you.

ROBERT BLOOMFIELD: I am not your reviewer, just in case you were wondering. I guess
there’s another question. You were talking about giving people the options of tools, and
Valiant Westland actually had a question early on, which this is a good time to get to. He
asks, “You acknowledged using the intranet tool, Lotus Notes, to help work together as a
team. Do you see the integration of this type of toolset in the Virtual World client as an
essential element for a Virtual World collaboration success?” I mean right now it seems to
me you really can’t, you know, there’s nothing like Lotus Notes built into really any of these
Worlds that you’ve mentioned, in an effective way.

MITZI MONTOYA: And my answer is, I absolutely do see the integration of the tools to be
important, particularly in the context that I’m interested in, which is collaborative work in an
enterprise setting, although that concept applies to education as well. If you want people to
work collaboratively on project teams, then this is less about forcing people into one tool as
if it is the hammer that will solve all their problems, but rather the technologies, the tools,
should support teamwork. And we know a lot about teamwork, and right now what teams
have to do is, we have to kluge together multiple different tools, to get the right set and the
right combination.

Yeah, and IBM Sametime is another example. Right? It’s a great example of a certain set of
tools, but it certainly does not have a virtual environment, Virtual World component to it, not
that it couldn’t. But, again, yes, integration of tools. So it’s not about the tool; it’s about
matching the capability of the technology to the purpose of use as required in any given
context or situation by a team.

ROBERT BLOOMFIELD: Are you willing to tell us what you see is the future of these Virtual
World platforms? You mentioned you’ve been working with Sun’s Wonderland and
ProtonMedia’s ProtoSphere, as well as Linden Lab’s Second Life. I guess I’m just
wondering, first, without getting specific on any platforms, are you bullish on the industry?
And then, if you have any thoughts on particular platforms, I think people would love to hear

them.

MITZI MONTOYA: Definitely bullish on the industry, although I think, in current form, it’s
better suited for some purposes than others. And I think right now, from a training and
education standpoint, some are easier to use than others because, if you think about the
types of content we have to present, to do training and/or education, whether that’s K-12 or
university or corporate, some of these tools make it more or less difficult to port in content
that you might want to share. So for example, to get ready to be on your show today, I had
to have help putting on hair. Right? So it’s a great example of not necessarily being
user-friendly and makes it difficult for us to have an opportunity to interact. And, if I had
wanted to bring PowerPoints, in the event I wanted to do death by PowerPoint, for example,
that would not have been simple for me to do, although we have people who could
[CROSSTALK]

ROBERT BLOOMFIELD: Actually that’s one of the advantages of Second Life. I totally
agree. Second Life is definitely not user-friendly enough. I’m using it for some enterprise
work, but that is the big hurdle that we have to get over. Do you find ProtoSphere a big step
up on that dimension?

MITZI MONTOYA: It’s a step up on the dimension of easy to use in terms of integration with
current applications that are commonly used. On the other hand, when you’re using those
applications, why do you need to be in a Virtual World? Right? I don’t really need to feel
your avatar next to me if we’re looking at a shared document. So those are the things that--
I’m not bullish on the technology and that I think it will be the future of work necessarily. I
think, before we go that far, we need to understand the nature of work and then use the
appropriate tools to support the work. And I do think Virtual Worlds and Virtual World
technologies do serve a purpose that has previously not been tapped by any other
technologies. So I think that’s the key is that we have to match technologies and tasks, and,
when you do that, then I believe this is going to be an important tool, to give that sense of
presence that no other tool does right now.

ROBERT BLOOMFIELD: Okay. We are pretty much out of time, but one last question I’d
like to ask, which is: What’s next for you? What’s on the top of your agenda?

MITZI MONTOYA: Well, that’s a great question. Next for me are application areas, and I
am particularly interested these days in health care and the application of virtual
environments in health-care space. So that is a significant area of research for me right
now, and particularly sort of the reinvention and rebirth of telemedicine. And I believe this
environment provides an interesting new opportunity to address that space in any number of
ways.

ROBERT BLOOMFIELD: Wait. The rebirth? Has telemedicine come and gone, and I
missed it?

MITZI MONTOYA: Well, the telemedicine is your new levels that we can achieve in
telemedicine beyond what has traditionally been thought of as telemedicine. And, with the
increased opportunities that we have for interactions between patients and doctors, which is
not something really that we’ve seen previously, this becomes an opportunity with new
virtual environment technologies to really change the nature of the patient/provider
interaction. And that could be an important opportunity and change when you think about
global health, rural health, people who previously have not had access and in a way that
does still provide a sense of a doctor/patient relationship. It’s a whole new opportunity.

ROBERT BLOOMFIELD: Okay. It’s pretty easy for me to think of your three dimensions,
how present are people in the space, in the task. I’d like the doctor to be very present,
thinking about my illnesses. And then how present is the other person? How close of a
connection do you have with them? And it’s pretty easy to see the application of that to
telemedicine. Good luck. On behalf of all of us who may someday need a doctor, but not
want to drive to an office.

MITZI MONTOYA: Hopefully, you’ll see a real one. That will be good.

ROBERT BLOOMFIELD: Thank you so much for joining us, Dr. Mitzi Montoya, of North

Carolina State University. And I hope we will see you back on Metanomics sometime soon.

MITZI MONTOYA: Well, thank you for having me on, Rob.

ROBERT BLOOMFIELD: Now it is time for our regular closing comment, Connecting The
Dots. With our focus on virtual teams and collaboration, it made sense to bring in
Erica Driver, of ThinkBalm, who just yesterday published a new report The Immersive
Internet Business Value Study. The study has lessons for all of us interested in enterprise
uses of Virtual Worlds, and, rather than have me talk about someone else’s work, I am
delighted to have Erica here herself to tell us what we can learn. Erica, thanks for coming on
to Metanomics. And, take it away.

ERICA DRIVER: Thank you, Rob, and thank you for having me on the show, at such a
last-minute notice. I’m chagrined to tell you that I have PowerPoints. I’m hoping that you’ll
stick with me for the six minutes that I have, and I’ll try to redeem myself by letting you know
that one of the projects that I’m working on is building a brand new ThinkBalm island out
that will be an immersive data experience, where we’ll be able to bring you, to come and
give you an experience of the data that I’ll be talking with you about now.

So the study that Rob just mentioned was published yesterday, and the core question that
we set out to answer was: What is the business value of using immersive technologies in
the workplace? I’m an industry analyst. I focus exclusively on this area of work-related use
of immersive technology. So to do this research, we conducted an online anonymous survey
of 66 highly qualified practitioners who are using this technology, and we conducted 15
in-depth interviews. So what I’m going to do is to first share with you some of the positive
findings. First, the sentiment that people have, the feeling they have toward the projects that
they worked on last year and the beginning of this year is highly positive. Ninety-four
percent of the survey respondents reported some level of project success.

So about a third of them said that the actual project data showed success. Another
two-thirds, almost, said that the project is ongoing, and there’s no real data, but that it feels

like a success. And we didn’t ask them to define what they mean by success, but we did
gain insight into that through open-ended questions, as well as through the interviews. And
just a few of the metrics that came up quite a lot were: improved employee productivity,
increased revenues and increased employee retention rate.

We then asked about the economic benefit and asked people to try to quantify that for us. I
mean this is the million dollar question, right? It comes up all the time, and we found that
more than 40 percent of respondents saw positive economic benefit from the investments
they made in this technology last year and early this year. And you’ll see that, as you look
across this chart, there’s quite a spread. The quantification of this value ranged from less
than $10,000 USD on the low end up to more than a million dollars on the high end, and
there are many reasons for this, one of them being that the business value depends on the
use case. And the use cases varied quite widely. Also, business value depends on the
maturity and the breadth of the rollout, and many of the projects people talked about are still
ongoing, and many of them are experiments and pilots, so very early-stage market that
we’re talking about here.

If we shift focus a little bit to what people are doing with this technology, we heard a lot in
the earlier discussion about training and learning and meetings, and you’ll see again yet
another point of evidence here that people are picking this low-hanging fruit, particularly
learning and training, followed by meetings. And they’re doing exactly what Mitzi was talking
about. They’re trying to find other ways to have face-to-face time--we call it face
time--without being together in person. And you’ll see on a coming slide that the number of
people who value this is very, very high. So learning and training, meetings. A couple of
interesting things here, one of them was that we asked an open-ended question about
usage, and, through that question, we found that about twice as many respondents used
this technology for internal meetings versus external meetings, and that’s probably due to
security issues, training issues which I’ll talk about in a minute.

We also, at the last minute, before putting our survey up, we separated out conferences
from meetings, and we’re glad we did because, as you can see from this chart, conferences

is the third bar down and quite a few fewer participants actually selected that as a use case
compared to meetings. And one of the reasons for this is that there’s some scalability issues
with the technology, with much of the technology today that really gets in the way of having
200, 500, 5,000 people together in the same place at the same time.

So let’s talk a little bit about the market as a whole, when we look at the big picture. When
ThinkBalm published our first analyst report in--and I understand from someone on the web
that they can’t see the charts; it’s too bad. But we published a report in November, showing
this technology adoption life cycle, and we positioned the immersive internet in the very
seedling or innovator stage at that time, and, in the short six months since then, we now see
us being in the early adopter phase. And not only are we in the early adopter phase, but we
are looking across this grand gaping chasm.

Now why do I say that we’ve moved into the early adopter phase? Well, one reason is that
it’s no longer just Virtual World geeks, like myself, or technology people who are
evangelizing and promoting and using this technology, but it’s moved into the business
realms. So for example, at the 3D TLC Conference that Tony O’Driscoll chaired last month
in D.C., a lot of the panelists were actually businesspeople. They were in HR. They were in
marketing, sales, and they had become evangelists for immersive technology because the
technology delivered business value. It helped them solve real business problems. So we’re
starting to see this shift into people who are recognizing benefit, who aren’t necessarily the
people who have created this technology.

So let’s talk about the chasm. If you would like to see some detail about these barriers--this
is huge. This is huge. While the payoff can be great, we face a lot of issues and challenges
ahead of us. The main one or the most common one according to our survey is that the
target users tend to have inadequate hardware. I have experienced it. Many of you probably
have as well. The typical corporate laptop does not have an adequate graphics card. Many
people at work don’t have headsets yet so they can’t really use voice. So there are a lot of
hardware issues that prevent people from fully experiencing this.

Second on the list was corporate security restrictions, and this comes up all the time. In fact,

one of the people we interviewed for this research was Eric Hackathorn, works for NOAA,
and he made a comment that 95 percent of U.S. government employees can’t get to
YouTube, never mind Second Life. And so that’s partly because of security restrictions, and
there are other issues too, so this is just enormous. Any time you have to look at opening up
a port in the firewall to let this immersive technology through, you’ll face serious resistance.

And then you’ll see a couple of other bars here. So getting people interested in this
technology and the effort required to train people, both of these are big issues. We heard
about some perception problems earlier, and those persist. Training: I do a lot of this myself,
with the ThinkBalm innovation community, and it’s very difficult when someone comes into
an environment, and they can hear a voice, but they say, “Well, what do you mean turn
around? Why can’t I see you? What do you mean?” So it’s a very, very different way of
working, of communicating.

But here’s the ending note I’d like to leave you with, which is that despite these issues,
despite the early-stage nature of this market, despite all the challenges before us, nearly
three-quarters of the people we surveyed said that their organization either might or will
increase their investment in this area in 2009 and ’10. So if you look at the data, over a
third--36 percent--said they definitely will increase their investment, compared to 2008 in
the first quarter of ’09. And 38 percent said that they might. So again, here’s a wonderful
thing, which is that the people who are wrestling with these issues are finding enough value
out of their experiments and pilots and investments, that they’ll continue to spend in this
area.

And the reasons for it are the benefits. So if you take a look at this last chart, what we’ve
done is ranked the benefits that people report from their investments in their projects last
year, in order, so at the very top of the list is face time. Right? It’s enabling people, who are
in disparate locations, to spend time together. And second on the list is increased
innovation. This one we did not expect to see. It was a really nice surprise because what I
thought would happen is that the only people who would choose this answer option would
be the people who were on formal innovation teams or part of R&D, and that wasn’t the

case at all. Our survey respondents were from all kinds of functional units and all sorts of job
roles.

And then the third at the top of this list here is costs savings or avoidance, and this was not
a surprise. We found that a large chunk of our respondents said that the immersive
technology was less expensive than the alternatives. They, on average, spent less than 25
or 35,000, not on average, but a good chunk of them spent less than 25 or $35,000 U.S.,
and a big portion spent less than 160 person hours on their projects, so it’s not a huge
investment you’re looking at, to get your feet wet with this technology and to derive some
serious business value.

So I encourage you to visit our website, thinkbalm.com. The full 36-page report is there,
and you’re welcome to download it. And I’m happy to answer any questions. I’ll post my
email address here in the text chat. So back to you, Rob.

ROBERT BLOOMFIELD: Okay. Thank you, Erica Driver, of ThinkBalm, and I do encourage
all of you to take a look at that report. This has been a week devoted to data and largely
quantitative research. We’re going to be changing gears next week, and we’re going to be
talking with anthropologist Thomas Malaby about his new book Making Virtual Worlds:
Linden Lab and Second Life.

So please do join us next week for that show. We’ll kick it off with an opening segment from
Metanomic cultural correspondent Bettina Tizzy, and we will be getting an inside look at
Linden Lab, as well as an anthropologist’s view of how the making of Second Life and the
making of Linden Lab have lessons for actually democracy and political engagement,
among other topics in the Real World. So please do join us. I think you’ll find it very
interesting and maybe learn some things about Linden Lab you didn’t know.

This is Robert Bloomfield signing off. Take care. And I’ll see next Wednesday.
Transcribed by: http://www.hiredhand.com
Second Life Avatar: Transcriptionist Writer

Metanomics Transcript May 27 2009, Measuring Value in Virtual Worlds

Recommandé

Recommandé

Contenu connexe

Tendances

Tendances (20)

En vedette

En vedette (7)

Similaire à Metanomics Transcript May 27 2009, Measuring Value in Virtual Worlds

Similaire à Metanomics Transcript May 27 2009, Measuring Value in Virtual Worlds (20)

Plus de Doug Thompson

Plus de Doug Thompson (20)

Dernier

Dernier (20)

Metanomics Transcript May 27 2009, Measuring Value in Virtual Worlds