1. OU Learning Design Initiative
Indicators of Community: a framework for evaluating relational and transient communities on Cloudworks
Rebecca Galley 19th March 2010
In our evaluation of relational and transient communities on Cloudworks, our interest is in the process of
evolution from loosely tied webs or networks to the more cohesive productive groups that can be seen
to emerge from repeated and iterative collaborative activity that happens within, across and between
groups from more established Communities of Practice. The indicators identified are those we believe
promote this evolution i.e. the factors which support the development of emerging Communities of
Practice.
Community definition: “ a persistent, sustained [socio-technical] network of individuals who share and
develop an overlapping knowledge base, set of beliefs, values, history and experiences focused on a
common practice and/ or mutual enterprise” Barab, A., Kling, B., & Gray, J.H., (2003) p. 23 (Italics in
original)
Community Evaluation question Evaluation methodologies Success looks like...
Indicators
Participation Did participants take on any special roles or duties Analysis of user activity over time. Sustained activity and core People learn through
(e.g. leader, conflict resolution, social facilitator)? groups identified on the basis of frequency of posting and rate of participation (Tu and Corry,
• Sustained over time What was the hierarchical structure? Were these response received to messages posted, or via text-based social 2001; 2002)
• Commitment from a effective in promoting and supporting collaborative net-work analysis.(Herring, 2004, p. 356)
core group of activity? Participation is sustained
participants Was there a core group of participants, who Roles and hierarchy can be adduced through participation without encouragement from
• Emerging roles and contributed regularly? How far did a core group of patterns and speech analysis (e.g., Herring & Nix, 1997, which developers.
hierarchy participants encourage the engagement and activity of considers the different acts performed by group leaders and non-
others? leaders). Discussion and debate are
How far did participants make repeated vibrant. ‘Buzz’ (Gratton, 2007)
contributions? Did they continue to contribute into
the wider Cloudworks space? Commitment demonstrated
through repeated and
sustained interaction (Erickson,
1997)

2. OU Learning Design Initiative
Indicators of Community: a framework for evaluating relational and transient communities on Cloudworks
Rebecca Galley 19th March 2010
Cohesion Were people polite and friendly to others? Was there Observed verbal humour (Baym, 1995), jokes, banter and People trust each other
evidence of a willingness to listen and learn from playfulness. Sociality characterized by combination of work and (Clifton, 1999) and have fun.
• Support and others? play (Wittel, 2001)
tolerance Were less confident participants encouraged to Support(Herring, 1994) and tolerance(Walzer 1997) through
• Turn taking and participate further? Can this kind of behaviour be seen speech act analysis focusing , for example, on acts of positive
response to impact on engagement? politeness openness, curiosity, and respect - a willingness to listen
• Humour and Did participants take turns in discussions and respond and learn
playfulness to each others’ comments? Reciprocity through analysis of turn initiation and response
Did participants ask or answer questions of others? (Rafaeli & Sudweeks, 1997)
Identity Did participants use similar vocabulary and Group self-awareness can be demonstrated in its members’ People feel a sense of shared
phraseology? references to the group as a group i.e. “us verses them” language, ownership for the community
• Group self- Was a similar tone and style used? particularly in statements such as, “We do things this way here” and connection with others.
awareness Was the style and tone used inclusive or exclusive of (implying an awareness that they might be done differently
• Shared language other groups? elsewhere)
and vocabulary When asked, did participants feel like they were part Shared language Baym identifies 4 types of ‘consistent and
• Sense of of a community? What factors made them feel this distinctive language practices’ that indicate the emergence of a
Community way? coherent online community: group specific vocabulary; forms of
non-verbal communication; genres; and humour (Baym 2003,
p1016)
A ‘sense of community’ can be captured in user surveys and
interviews. The concept is personal and based on feelings and
personal values and is likely to be influenced by a range of factors
which should also be captured.
Creative capability Did visitors to the site understand the purpose of what Igniting purpose - Areas of significantly higher activity indicating Innovation is developed
they were doing? flashpoints of interest and engagement (Gratton, 2001, through new combinations of
• Igniting purpose Did they feel drawn to participate and get involved? Engestrom, 2007) ideas, knowledge and insights.
• Multiple points of Were multiple points of view expressed? Contradictions in terms of experience and knowledge.
view expressed and Did people from different types of roles and Multiple points of view expressed and contradicted or challenged. New meanings and
contradicted or workplaces contribute? Evidence of networks of relationships cross teams, disciplines, understandings are constructed
challenged Did people find participating engaging, interesting and function and organisations. collaboratively
• Creation of relevant to them?
knowledge links and Were links made between concepts and ideas? People find participation
patterns Did participants attempt to connect their knowledge exciting, interesting, fulfilling
and experience to that of others? and relevant to them.
Did participants challenge existing knowledge and
practices and work with others to conceive
alternatives?

3. OU Learning Design Initiative
Indicators of Community: a framework for evaluating relational and transient communities on Cloudworks
Rebecca Galley 19th March 2010
References
Barab, A., Kling, B., & Gray, J.H., (2003) Designing for Virtual Communities in the Service of Learning. Cambridge, UK: Cambridge University Press
Baym, N (1995). The performance of humour in computer-mediated communication. Journal of Computer-Mediated Communication, 1(2). Online
http://jcmc.indiana.edu/vol1/issue2/baym.html
Engestrom, Y. (2001). Expansive Learning at Work: toward an activity theoretical reconceptualization. Journal of Education and Work, 14 (1), 133-156.
Engeström, Y (2007), From Communities of Practice to Wildfire Activities and Mycorrhizae, Transcript of lecture given at the ’Talking Practice’ event, Practice-based
Professional Learning Centre for Excellence in Teaching and Learning, Paper 11
Erickson, T (1997). Social Interaction on the Net: Virtual Community as Participatory Genre Online http://www.pliant.org/personal/Tom_Erickson/VC_as_Genre.html
Gratton, L. (2007). Hot Spots: Why some Companies Buzz With Energy and Innovation – and Others Don’t. Financial Times Prentice Hall. Chapter 1 available online
http://www.lyndagratton.com/downloads/hot-spots-chapter-1.pdf
Herring, S., (2004). Computer-Mediated Discourse Analysis: An Approach to Researching Online Behaviour in Barab et al Designing for Virtual Communities in the
Service of Learning p356-357
Jenkins, H., (2009). Confronting the challenges of participatory culture: Media education for the 21st Century, Mit Pr.
Tu, C.-H., & Corry, M. (2001). A paradigm shift for online community research. Distance Education Journal, 22 (2), 245-263.
Tu, C.-H., & Corry, M. (2002). Research in online learning community, Online
http://www.usq.edu.au/electpub/e-jist/docs/html2002/chtu.html
Walzer, M. (1997). On Toleration. Yale University Press: New Haven
Wittel, A. (2001). Toward a Network Sociality Theory, Culture & Society (SAGE) Vol.18(6):51-76
Also:

4. OU Learning Design Initiative
Indicators of Community: a framework for evaluating relational and transient communities on Cloudworks
Rebecca Galley 19th March 2010
McInnerney, J. M., & Roberts, T. S. (2004). Online Learning: Social Interaction and the Creation of a Sense of Community. Educational Technology & Society, 7 (3),
73-81. Online http://www.ifets.info/journals/7_3/8.pdf