State-of-cultural-indicators-research-Nick-Howlett-2016

© Copyright 2016 Nick Howlett, School of Humanities, Language and Social Science, Griffith University 1
The handshake between practice and policy? Remarks on the state of cultural
indicators research
Cultural indicators have become a fixture in the cultural policy landscape, and are making inroads
into broader public policy-making as well. Despite some resistance from arts practitioners to
measurement, and an early overemphasis on the instrumental value of cultural activity, the
development of cultural indicators has now progressed to the point where they are seen as a practical
means of communicating the value of cultural activities. Communities, organisations and
governments across the world use them to provide the “conceptual handshake between practice and
policy” (Badham 2009, p. 2) that interrogates public involvement with arts and culture and an ongoing
conversation about their costs and benefits to society. Efforts to implement cultural indicators in
different countries have in common the principle that indicators need to be specific to local contexts,
and that each context, be it a city, a state, a nation or a subregion, is singular and dynamic, and
therefore not directly comparable. There is also a common understanding that the instrumental
benefits of arts and culture are not fully divisible from the intrinsic benefits, and that while intrinsic
benefits are not a means to an economic or social end, they are nevertheless beneficial themselves,
from the personal through to the societal level (McCarthy, Ondaatje, Zakaras & Brooks 2004 p. xv).
However, many issues—the efficacy of various methodologies of cultural measurement, and the
boundaries of what can and can’t be measured—remain contentious.
The interest in cultural indicators is a relatively recent phenomenon. It has arisen as economic
liberalism (under the rubric of ‘neoliberalism’) has become the dominant cultural mode globally.
Associated with this ideology is an emphasis on quantifiable economic mechanisms (Synexe
Consulting Limited 2010, p. 25). In their earliest iterations, cultural indicators conformed to the logic
of the market, of “micromanagement… and other forms of accountability” (Keaney 2006, p. 3).
Resistance to this ‘micromanagement’ has been felt, especially from within the arts and cultural
community itself (Badham 2010, p. 5). Some within these groups see the application of metrics to
artistic expression as a reduction of the complexity and diversity of art and cultural activities to mere
“weights and measures”: reductionism that causes harm to the very things being measured (Goldbard
2008, in Badham 2010, p. 5). Eleonora Belfiore (2004), writing about the British cultural policy
context, uses the term “instrumental turn” (Belfiore 2004, p. 183) to show how Britain’s culture sector
responded to the ascendancy of the post-1980s “audit society” (Belfiore 2004, p. 190): by seeking to
produce evidence that it was contributing “to other policy spheres” with “a heftier political weight”
(Belfiore 2004, p. 200). Belfiore cites policy designed to address social exclusion and local economic
development as two areas that British arts and culture were expected to advance during New Public
Management’s period of pre-eminence (Belfiore 2004, p. 183). She is concerned that the tendency

toward instrumental cultural measurement “might in fact turn out to be ‘politics of extinction’” for
some arts organisations (Belfiore 2004, p. 200) if the justification for ongoing public support of
cultural activities is sought solely on the basis of their value to economic and social policy agendas.
Researcher John Holden draws similar conclusions, though without quite the same degree of
pessimism. Also drawing on the British experience, Holden (2004), like Belfiore, agrees that “the
identifiable measures and ‘ancillary benefits’ that flow from culture” … “have become more
important than the cultural activity itself: the tail is wagging the dog” (Holden 2004, p. 21).
Concentrating attention on the instrumental value of culture is problematic, Holden suggests. To
begin with, the evidence base is flawed. One national museums organisation he quotes from concedes
that across their membership base, “‘there is no ready-made and reliable methodology in place for
calculating the economic impacts of cultural institutions’” (Holden 2004, p. 17). With such a weak
evidence base, claims that culture produces instrumental outcomes are dubious. At the same time, the
focus on instrumental impacts distorts arts bodies, affecting the very nature of cultural activities: “the
long-term, cumulative impact is to separate both what gets funded and who gets funded from the most
innovative, creative, dynamic or publicly engaging work” (Holden 2004, p. 21).
With the instrumental value of arts and culture being questioned for its efficacy and found wanting,
the attention of cultural indicator researchers has turned to intrinsic values as a possible ‘missing link’
between cultural activities and their social utility. As McCarthy et al. point out, “People are drawn to
the arts not for their instrumental effects, but because the arts can provide them with meaning and
with a distinctive type of pleasure and emotional stimulation” (McCarthy et al, 2004 p. xv)—in other
words, the intrinsic effects of the artistic or cultural experience in question. Their impacts on
individuals form the basis for wider community and societal-level impacts (Brown & Novak-Leonard
2013, p. 223).
McCarthy et al (2004) and other researchers (including Badham 2010; Brown & Novak 2007; Brown
& Novak-Leonard 2013; Dunphy 2015) recommend that research into intrinsic value should be at the
centre of cultural indicator development, mainly because “quantitative evidence of non-economic
impact is scarce, although anecdotal evidence is abundant” (Brown & Novak 2007, p. 5). Instrumental
measures of cultural activities—for example, ticket sales for a theatre performance—may give some
indication of the economic worth of the performance, but they explain little about the inherent value
of the experience of a cultural activity for the audience. In the parlance of the market, ‘intrinsic’ is
equated with ‘non-economic’, and is therefore of less interest to those seeking to justify expenditure
on arts and culture via external socio-economic impacts (Badham 2010, p. 6).

Research into how intrinsic value might be measured and utilised in cultural indicators is emergent,
but many researchers — including Holden (2004), Dunphy (2015), and Badham (2010) – have
concluded that the ‘intrinsic-versus-instrumental’ debate is based on the flawed premise that they are
dichotomous,. These researchers also point out that overemphasising intrinsic value is also
problematic, albeit in a different way to an excessive reliance on instrumental value. One concern
with placing too much stress on intrinsic value is that the language used to describe it is insufficient
to the task: “postmodern questioning of concepts such as beauty, truth, delight, transcendence and the
like, coupled with the insight that these ideas are temporally and geographically specific, have made
using them in debate an embarrassment at best, contemptible at worst” (Holden 2004, p. 23).
Holden (2004, p. 24) also says some arts organisations insist that intrinsic values should trump other
values, without articulating what these intrinsic values are. A possible counter to this is David
Throsby’s (2001) categorisation of intrinsic value, which is intended to widen the descriptive
vocabulary. In it, intrinsic value is viewed through aesthetic, spiritual, social, historical, symbolic or
authenticity lenses (Throsby 2001, in Dunphy 2015, p. 245).
A multi-dimensional approach like Throsby’s above allows more nuance for cultural indicators to be
based upon. By allowing a broader sense of the value of a cultural activity, the circular logic
accompanying some accounts of intrinsic value is replaced with “a shared set of ideas of what is
valuable” (Dunphy 2015, p. 248), rather than leaving this to the artist or curator. A salient example is
Throsby et al’s 1982 study into public interest in the arts for the Australia Council for the Arts
(Throsby & Withers 1985, in Bakhshi, Freeman & Hitchen 2009, pp. 5-6), which found that just under
half of those surveyed regularly attended arts events. More surprisingly, those who didn’t actively
participate in cultural activities still perceived some benefits to be gained by the wider community
from arts and culture, to the degree that around 75% of respondents favoured increasing government
financial support to the cultural sector (Throsby & Withers 1985 , in Bakhshi, Freeman & Hitchen
2009, pp. 5-6). This survey didn’t recourse to any external, instrumental socio-economic impacts—it
asked the public for their own valuation of culture, and received the response that culture was valued
for itself. Which is to say, the intrinsic value of culture is indivisible from its instrumental value:
“culture is about both intrinsic and instrumental values”, writes Holden (2004, p. 25): “arts transform
people and places, giving spiritual and personal sustenance” while also being “instrumental in
meeting public policy objectives” (Holden 2004, p. 25).
As a more balanced approach that balances intrinsic and instrumental values has evolved, cultural
indicators have begun to be included in integrated models framed around ‘quality of life’ (Duxbury
2003, p. 4) and ‘community vitality’ (Badham 2009; Bianchini & Landry 1994). For example, in
Australia, the Australia Council for the Arts includes social and artistic outcomes when measuring

the level of engagement of activities, while in the USA, the National Endowment for the Arts includes
measures of community “strengthening” via artistic activities as a priority within its funding
programmes (Dunphy 2015, p. 251).
Problems of measurement are still being resolved. The different contexts to which cultural indicators
are being applied are incredibly diverse. Leaving aside questions about different global contexts, even
at the local level, there is a miscellany of contexts and concerns. For example, metrics developed for
an organisation specialising in preserving architectural heritage are unlikely to be applicable to an
avant-garde dance company, and vice versa. Many cultural activities are also non-linear, and different
forms of each discipline can represent a spectrum of values for audiences (Brown & Novak 2007,
p. 2). There is also the audience question: how cultural indicators are to articulate the often
idiosyncratic impacts on an individual audience member as well as larger, more amorphous ‘publics’,
while also measuring the benefits of cultural activities that accrue cumulatively (Brown 2006, p. 19).
Given the nature of artistic and cultural works—the diversity and complexity of their audiences and
of the circumstances audiences encounter them in—it is unsurprising that the formation of a
consensus on the methodology of cultural measurement (MacDowall 2015, p. 2) is a vexed issue.
Research on cultural indicators has made the connection that local communities across the world need
cultural indicators that are specific to them, that adequately describe their cultural forms, and local
patterns of the production and consumption of arts and culture. Localised definitions of how cultural
indicators fit into broader frameworks of meaning are needed as well (Duxbury 2003, p. 4), to guard
against the homogenising effects of globalisation. These are also required at the regional and national
level to ensure that cultural perspectives are included in measures of development, and so that the
role that indigenous cultures play in the overall wellbeing of some societies is recognised (MacDowall
2015, p. 3, in MacDowall, Badham, Blomkamp & Dunphy 2015).
The emergence of a more integrated, holistic approach to cultural indicators has reinforced the need
for methodologies that produce more meaningful evaluations of arts and culture. Principally, this has
involved a move away from quantitative methods to qualitative and mixed methods data gathering.
The “technocratic world view” of quantitative research that has been preoccupied with “tracking
outcomes against predetermined expectations” (Holden 2004, p. 59), has been replaced with a
preference for more open-ended research methods that recognise the “affective elements of cultural
experience, practice and identity, as well as the full range of quantifiable economic and numerical
data” (Holden 2004, p. 59).
Qualitative measures can provide a meticulous level of detail; Brown & Novak-Leonard 2013 cite
examples of various research approaches for assessing the “emotional, intellectual, social and

aesthetic” impact of artistic experiences on audiences (Brown & Novak-Leonard 2013, p. 223). One
such example used audience surveys containing questions grouped around abstract “impact
constructs”, such as captivation, emotional resonance, and social bonding (Brown & Novak-Leonard
2013, p. 227) to gather data from theatre patrons. This particular study was able to capture information
on the correlations between different variables, finding for instance that anticipation of an arts event
is “the single best predictor of impact” (Brown & Novak-Leonard 2013, p. 229). Surveys were given
pre– and post-performance, enabling longitudinal analysis (Brown & Novak-Leonard 2013, p. 224).
This allowed for a determination of the net benefit (or otherwise) to audience members over time,
thereby avoiding what Dunphy (2015, p. 247) identifies as a drawback of qualitative data for arts
evaluation, accounting for change over time.
The continued search for improved research methods notwithstanding, there is an acknowledgement
that cultural indicators are important because they help arts organisations, governments and
communities to articulate the value of cultural activities and art within wider contexts. Their evolution
from an ancillary measure of socio-economic ‘progress’ to a metric of the overall wellbeing of arts
organisations, communities and even whole nations, is a welcome development, as is the move to
include more qualitative and mixed methods data gathering. The use of qualitative research
methodologies in studies of intrinsic value adds an important multi-dimensional aspect to the extant
quantitative data, and allows for much broader insights from cultural indicators.

Reference List
Badham, M 2009, ‘Cultural indicators: tools for community engagement?’, The International
Journal of the Arts in Society, vol. 3, no. 5, pp. 67–75.
Badham, M 2010, The Menace of Measurement: A discussion about arts indicators, Saskatchewan
Arts Alliance, Regina.
Bakhshi, H, Freeman, A & Hitchen, G 2009, ‘Measuring intrinsic value – how to stop worrying and
love economics’, missionmodelsmoney.org.uk, vol. 9, no. 2, pp. 1–34, viewed April 17, 2016,
http://www.jstor.org/stable/41810355.
Belfiore, E 2004, ‘Auditing Culture: The subsidised cultural sector in the New Public
Management’, International journal of cultural policy, vol. 10, no. 2, pp. 183–202.
Bianchini, F & Landry, C 1994, ‘The Creative City Working Paper 3: Indicators of a Creative City’,
Comedia, Bournes Green.
Brown, A 2006, ‘An Architecture of Value’, Grantmakers in the Arts Reader, vol. 17, no. 1,
pp. 18–25.
Brown, AS & Novak, JL 2007, Assessing the intrinsic impacts of a live performance, WolfBrown,
San Francisco.
Brown, AS & Novak-Leonard, JL 2013, ‘Measuring the intrinsic impacts of arts attendance’,
Cultural Trends, vol. 22, no. 3-4, pp. 223–233.
Dunphy, K 2015, ‘A Holistic Framework of Evaluation for Arts Engagement’, in Making Culture
Count: The Politics of Cultural Measurement, Palgrave Macmillan, London, pp. 243–263.
Duxbury, N 2003, ‘Cultural Indicators and Benchmarks in Community Indicator Projects:
Performance Measures for Cultural Investment?’, Proceedings of the Accounting for Culture:
Examining the Building Blocks of Cultural Citizenship Conference, Gatineau, Quebec, 13-15
November 2003, pp. 1–24.
Goldbard, A 2008, ‘The Metrics Syndrome’, arlenegoldbard.com, viewed April 15, 2016, from
http://arlenegoldbard.com/wp-content/uploads/2005/12/Metrics-Syndrome-10-13-08.pdf.
Holden, J 2004, Capturing Cultural Value: How culture has become a tool of government policy,
Demos, London.
Keaney, E 2006, Public value and the arts: Literature review, Arts Council England, London.
MacDowall, L 2015, ‘Introduction: Making Culture Count’, in Making Culture Count: The Politics
of Cultural Measurement, Palgrave Macmillan, London, pp. 1–5.
MacDowall, L, Badham, M, Blomkamp, E & Dunphy, K 2015, Making Culture Count: The Politics
of Cultural Measurement, Palgrave Macmillan, London.
McCarthy, KF, Ondaatje, EH, Zakaras, L & Brooks, A 2004, Gifts of the Muse: Reframing the
Debate about the Benefits of the Arts, Rand Corporation, Santa Monica.

Synexe Consulting Limited 2010, Valuing Culture in Oceania: Methodology and indicators for
valuing culture, including traditional knowledge, in Oceania, Secretariat of the Pacific
Community, Nouméa.
Throsby, D & Withers, G 1985, ‘What Price Culture?’, Journal of Cultural Economics, vol. 9, no.
2, pp. 1–34, viewed April 12, 2016, http://www.jstor.org/stable/41810355.
Throsby, D 2001, Economics and Culture, Cambridge University Press, Cambridge.

State-of-cultural-indicators-research-Nick-Howlett-2016

Recommandé

Recommandé

Contenu connexe

Tendances

Tendances (20)

En vedette

En vedette (9)

Similaire à State-of-cultural-indicators-research-Nick-Howlett-2016

Similaire à State-of-cultural-indicators-research-Nick-Howlett-2016 (20)

State-of-cultural-indicators-research-Nick-Howlett-2016