1. The current issue and full text archive of this journal is available at
www.emeraldinsight.com/0144-3577.htm
The effectiveness
Factors influencing the of PMSs
effectiveness of performance
measurement systems
1287
Amy Tung, Kevin Baird and Herbert P. Schoch
Department of Accounting and Corporate Governance, Received June 2010
Macquarie University, Sydney, Australia Revised November 2010
Accepted April 2011
Abstract
Purpose – The purpose of this paper is to examine the association between the use of
multidimensional performance measures and four organizational factors with the effectiveness of
performance measurement systems (PMSs).
Design/methodology/approach – Data were collected by mail survey questionnaire from a
random sample of 455 senior financial officers in Australian manufacturing organizations.
Findings – The results reveal that the use of multidimensional performance measures is associated
with two dimensions of the effectiveness of PMSs (performance and staff related outcomes). The
results also reveal that organizational factors were associated with the effectiveness of PMSs.
Specifically, top management support was found to be associated with the effectiveness of PMSs in
respect to the performance related outcomes, and training was associated with the staff related
outcomes.
Practical implications – The findings provide managers with an insight into the desirable PMS
characteristics and the specific organizational factors that they can focus on in order to enhance the
effectiveness of their performance measurement system.
Originality/value – This study contributes to the limited empirical research examining the
effectiveness of PMSs regarding the extent to which organizational processes are achieved. In addition,
the study provides an empirical analysis of the association between the five perspective (financial,
customer, internal business process, learning and growth, and sustainability) BSC model and four
organizational factors with the effectiveness of PMSs.
Keywords Australia, Manufacturing industries, Performance measures,
Performance measurement system, Multidimensional performance measures, Top management support,
Training, Employee participation, Link of performance to rewards
Paper type Research paper
1. Introduction
To survive in today’s rapidly changing environment, organizations must identify their
existing positions, clarify their goals, and operate more effectively and efficiently.
Performance measurement systems (PMSs) assist organizations in achieving such
objectives. Neely et al. (1995, p. 81) defines a PMS as “a set of metrics used to
quantify both the efficiency and effectiveness of actions”. An effective PMS enables
an organization to assess whether goals are being achieved, and facilitates the
improvement of the organization as a whole (Lebas, 1995) by identifying their position,
clarifying goals, highlighting areas requiring improvement, and facilitating reliable International Journal of Operations
forecasts (Neely et al., 1996). Hence, an effective PMS enables an organization to & Production Management
Vol. 31 No. 12, 2011
measure and control its performance in line with the defined strategy. pp. 1287-1310
q Emerald Group Publishing Limited
While the recent PMS literature has focused on the shift from traditional PMSs, which 0144-3577
focus on financial measures, to multidimensional PMSs such as the performance DOI 10.1108/01443571111187457

2. IJOPM pyramid (Lynch and Cross, 1991), the balanced scorecard (BSC) (Kaplan and Norton,
31,12 1992), and the performance prism system (Neely and Adams, 2000), there is limited
empirical evidence examining the effectiveness of such PMSs. Furthermore, the majority
of these studies assess PMS effectiveness in relation to overall organizational
performance (Crabtree and DeBusk, 2008; Braam and Nijssen, 2004; Davis and Albright,
2004; Ittner et al., 2003; Hoque and James, 2000), thereby assuming a direct association
1288 between the PMS and performance. This approach is inconsistent with Hamilton and
Chervany’s (1981) claim that the impact of the PMS on performance is indirectly
influenced by the effect on improvements in organizational processes. In other words,
organizational objectives such as sales revenue, profit contribution and customer
satisfaction will not be realized unless specific organizational objectives (e.g. motivating
performance, developing individual’s skills and knowledge, providing useful feedback
to employees, and providing an accurate assessment of business unit performance) are
achieved. Accordingly, the first objective of this study is to contribute to the limited
empirical research (Malina and Selto, 2001; Whorter, 2003) examining the effectiveness
of PMSs based on the extent to which organizational processes are achieved.
The measurement of performance is an on-going task, hence, in order to achieve
system effectiveness, organizations need to devote time and effort to managing the
system (Neely et al., 2000). Hence, in an attempt to provide practitioners with an insight
into how to achieve and maintain effectiveness, the second objective of the study is to
contribute to the contingency literature by examining the factors associated with
the effectiveness of PMSs. The first factor examined, the use of multidimensional
performance measures, has been advocated by both academics and practitioners in
order to complement the limitations of traditional financial PMSs and to increase the
effectiveness of PMSs (Van der Stede et al., 2006; Kaplan and Norton, 2001, 1996, 1992).
While many multidimensional frameworks have been advocated, and the benefits of
using multidimensional performance measures have received wide publicity in the
literature (Van der Stede et al., 2006; Bryant et al., 2004), there is considerable variation in
the adoption rates reported for the most common multidimensional approach, the BSC
(Rigby and Bilodeau, 2009 (53 percent); Chung et al., 2006 (31 percent); Ittner et al., 2003
(20 percent); Speckbacher et al., 2003 (26 percent)). The variation in the adoption of
multidimensional performance measures raises concerns regarding the contribution
of such measures towards the effectiveness of PMSs. Accordingly, this study aims
to contribute to the literature by examining the association between the use of
multidimensional performance measures and the effectiveness of PMSs.
The study also aims to provide an empirical analysis of the association between
specific organizational factors (top management support, training, employee
participation and the link of performance to rewards) with the effectiveness of PMSs.
While these organizational factors do not represent a comprehensive list of all
relevant factors, they were chosen for two reasons. First, they have been widely cited
as factors contributing to the success of various management accounting practices such
as activity-based costing (ABC) (Baird et al., 2007; Shields, 1995), enterprise resource
planning (Motwani et al., 2002; Rao, 2000), and management information system (MIS)
(Raghunathan et al., 1999; Doll, 1985; Schultz and Ginzberg, 1984). Second, while they
have been identified in previous studies as the main contingency factors associated with
the effectiveness of PMSs (Burney et al., 2009; Hoque and Adams, 2008; Cheng et al.,
2007; Kleingeld et al., 2004; Chan, 2004), this was in isolation, and no study has analysed

3. all four factors together. Hence, this study is motivated to fill this gap in the literature by The effectiveness
examining the link between all four organizational factors and the effectiveness of PMSs of PMSs
within Australian manufacturing organizations.
In addition, given the majority of previous studies examining the influence of
organizational factors on PMS effectiveness have used the case study approach
(Kleingeld et al., 2004; Bourne et al., 2002; Emerson, 2002; Kennerley and Neely, 2002;
Kaplan, 2001), there is a gap in the literature empirically examining this association. 1289
Hence, the current study is motivated to fill this gap by using the survey method in an
attempt to enhance the generalizability of the findings.
The remainder of this paper is structured as follows. Section 2 provides the literature
review and develops the relevant hypotheses. Sections 3 and 4 then discuss the method
and results. Finally, Section 5 provides the conclusion, limitations, and future directions
for research.
2. Literature review
2.1 Performance measurement systems
PMSs have become a field of interest over the last two decades with many studies
discussing various aspects of performance measurement such as: the purpose and
usage (Marchand and Raymond, 2008; Horngren et al., 2005; Simons, 2000), design
(Bhasin, 2008; Kennerley and Neely, 2002; Neely and Adams, 2000; Kaplan and Norton,
1996, 1992; Lynch and Cross, 1991), and implementation (Ratnasingam, 2009; Othman,
2008; Speckbacher et al., 2003; Kaplan, 2001).
An effective PMS, which is defined as the achievement of the objectives set for
a task (Clinquini and Mitchell, 2005), is important for a number of reasons. First,
it can encourage goal congruence. For example, an appropriate PMS can be used to
communicate the strategy and goals of an organization and align employees’ goals with
organizational goals. Second, an effective PMS can provide accurate information to
enable managers to track their own performance and evaluate employees’ performance
in an effective and efficient manner. Finally, an effective PMS can provide organizations
with an indication of their current market position and assist them in developing future
strategies and operations (Langfield-Smith et al., 2009). This study operationalises an
effective PMS as the extent to which 16 desired PMS outcomes are achieved.
Traditionally, PMSs have focused mainly on financial measures such as profit, cash
flow and return on investment to evaluate the performance of employees (Chan, 2004).
This focus has a number of shortcomings. First, these outcome-oriented measures do
not allow managers to assess how well employees perform across the full range of
strategically important areas, such as quality and service delivery. Second, traditional
financial measures describe consequences rather than causes, hence they are not
actionable. Such measures provide limited guidance for future actions since they do not
tell managers what needs to be fixed (Langfield-Smith et al., 2009). Third, the focus on
aggregate financial outcomes may encourage managers to engage in “gaming” behavior
to maximize short-term results at the expense of long-term effectiveness (Chow and
Van der Stede, 2006). Finally, traditional financial measures can conflict with strategy and
they are not externally focused (Chow and Van der Stede, 2006; Kaplan and Norton, 1996).
The limitations of traditional PMSs, together with intense competitive pressures
and changing external demands, have led to the increased advocacy of non-financial
measures (Neely, 1999). Such contemporary PMSs have been espoused by both

4. IJOPM academics and practitioners in order to address the limitations of traditional financial
31,12 performance measures and to assist organizations to build competitive advantage
under changing economic conditions (Kaplan and Norton, 2006, 2004, 2001, 1996, 1992).
The common characteristics of contemporary systems include the linking of strategies,
objectives and measures, and the incorporation of both financial and non-financial
measures that cover a range of perspectives (Langfield-Smith et al., 2009). Since the
1290 BSC is the most recognized and utilized contemporary PMS (Rigby and Bilodeau, 2009;
Chang et al., 2008; Jusoh et al., 2008; Bedford et al., 2006; Pike and Roos, 2004;
Atkinson et al., 1997), it is used in this study to exemplify the use of multidimensional
performance measures.
2.2 The BSC
The first-generation BSC was mainly a PMS which proposed a specific structure to
measure tangibles and intangibles (Speckbacher et al., 2003; Kaplan and Norton, 1992).
The framework complemented the financial perspective measures with non-financial
operational measures emphasizing three other perspectives: customer satisfaction,
internal processes and learning and growth. It provided a more balanced view of
organizational performance by capturing both leading (e.g. customer satisfaction,
on-time delivery, employee training, etc.) and lagging (e.g. sales revenue, ROI, cash
flows, etc.) performance measures (Kaplan and Norton, 1996, 1992).
In 1996, Kaplan and Norton advocated the causal links between the perspectives
included within the BSC. The refined model communicated the organization’s desired
outcomes and hypothesized the means by which the desired outcomes could be achieved.
For instance, if organizations trained their employees well, then the quality of service
would be improved as well as customer satisfaction; if customer satisfaction improved,
then customers would purchase more, thereby improving the overall profitability of the
organization. Hence, the second-generation BSC was proposed as a multidimensional
PMS which describes strategy through cause and effect relationships (Speckbacher et al.,
2003; Kaplan and Norton, 1996). It enabled organizational units and employees to
understand the strategy and identify how they can contribute to its achievement by
becoming aligned with the strategy. Consequently, today’s BSC has become a strategic
management system that implements strategy through communication, action plans
and incentives (Speckbacher et al., 2003; Kaplan and Norton, 2001).
As a further development, the BSC included additional perspectives (Kaplan and
Wisner, 2009; Kaplan and Norton, 2006, 2004, 2001). With sustainability becoming a
major concern for various stakeholders (e.g. customers, investors, and the government)
and affecting the organizational “bottom line”, a sustainability BSC was subsequently
advocated (Langfield-Smith et al., 2009; Epstein, 2008; Figge et al., 2002). Epstein (2008)
suggested that the inclusion of the sustainability perspective is appropriate where
sustainability is considered a part of the business core strategy and important to
creating competitive advantage. To provide a more comprehensive account of the use of
multidimensional performance measures, this study adopts the five perspective
(financial, customer, internal business process, learning and growth, and sustainability)
BSC model.
2.2.1 Adoption and use of the BSC. Silk (1998) estimated that 60 percent of the
Fortune 1000 companies in the USA have had experience with a BSC. In the UK,
57 percent of businesses were reported to use a BSC and 53 percent of non-users

5. were discussing possible implementation. In contrast, Speckbacher et al. (2003) reported The effectiveness
that more than 60 percent of the companies in their study had not considered the BSC. of PMSs
Similarly, Ittner et al. (2003) indicated that only 20 percent of the firms in their study used
a BSC, while 50 percent of the firms had not even considered implementing it.
Use of the BSC however does not guarantee satisfaction with De Geuser et al. (2009)
referring to the literature highlighting the gap between the use of the BSC and evidence
of its effectiveness (Davis and Albright, 2004; Norreklit, 2003; Speckbacher et al., 2003; 1291
Otley, 1999). Thus, while the Management Tools and Trends Survey (Rigby and
Bilodeau, 2009) showed that in 2008, 53 percent of organizations globally used the BSC
and by the end of 2009, the usage rate was expected to reach 69 percent, it was found that
51 percent of user organizations were not satisfied with their BSC. Similarly, Ittner et al.
(2003) revealed that organizations were only moderately satisfied with the measurement
system with 37.2 percent of respondents rating it as not meeting expectations.
Bedford et al. (2006) also concluded that while respondents agreed that the BSC had
helped in achieving some objectives, the extent to which the proclaimed benefits of the
BSC were achieved was still fairly low. Given the mixed findings with respect to the
success of the BSC, this study investigates the association between the use of
multidimensional performance measures and the effectiveness of PMSs.
2.3 The association between the use of multidimensional performance measures and the
effectiveness of PMSs
Multidimensional PMSs assist organizations by enhancing the likelihood that all
relevant performance dimensions are considered (Ittner et al., 2003). Furthermore, such
systems allow managers to focus on the “means to the end”, while also enabling them
to demonstrate strong performance in a variety of areas (Baird, 2010). Hoque and
Adams (2008) suggest that multidimensional PMSs are capable of providing signals
and motivating improvement in crucial activities. Similarly, Van der Stede et al. (2006)
found that regardless of strategy, organizations with more extensive PMSs, especially
those that included objective and subjective non-financial measures, have better
overall performance. Van der Stede et al. (2006) also demonstrated that non-financial
performance measures are better than financial measures in helping organizations
implement and manage new initiatives.
A growing stream of literature provides evidence that the use of multidimensional
performance measures contributes to the effectiveness of PMSs (Crabtree and DeBusk,
2008; Braam and Nijssen, 2004; Davis and Albright, 2004; Ittner et al., 2003; Whorter,
2003; Malina and Selto, 2001; Hoque and James, 2000). Most of these studies examined
the effectiveness of PMSs from the perspective of their contribution to the company’s
financial performance. For example, Davis and Albright (2004) applied a
quasi-experimental study in a US banking organization to investigate the relationship
between BSC implementation and the financial performance of bank branches.
The study supports the theory that the BSC can be used to improve financial
performance, with bank branches that implemented the BSC outperforming other
branches on key financial measures. Similarly, Braam and Nijssen (2004) suggest that
BSC usage, which is aligned to company strategy, positively influences overall company
performance.
Ittner et al. (2003) found that while BSC usage was associated with higher
measurement system satisfaction, there was no evidence that BSC usage was related

6. IJOPM to stock returns. However, Crabtree and DeBusk (2008) extended this study to
31,12 investigate the contribution of the BSC to shareholder returns in different public sector
companies, and found that BSC usage was associated with higher stock returns.
Malina and Selto (2001) and Whorter (2003) assessed the effectiveness of PMSs based
on organizational processes (e.g. communicating strategic objectives, creating strategic
alignment, motivating employees and serving as a management control device)
1292 as opposed to financial performance. Malina and Selto (2001) found that the BSC was an
effective device for evaluating corporate strategy. Their results also show evidence of
casual relations between motivation, strategic alignment and effective management
control with the BSC. Similarly, Whorter (2003) showed that BSC users consistently
reported higher agreement about having the information needed for making the best
work-related decisions. Whorter (2003) also concluded that the BSC not only provides
useful performance feedback to employees but is also an aid in the accurate assessment
of employee performance:
H1. The extent of use of multidimensional performance measures is associated
with the effectiveness of the PMS.
2.4 The association between organizational factors and the effectiveness of PMSs
Prior studies have identified top management support (Hoque and Adams, 2008;
Johanson et al., 2006; Bourne, 2005; Chan, 2004; Bourne et al., 2002; Kennerley and Neely,
2002; Kaplan, 2001), training (Chan, 2004; Emerson, 2002), employee participation
(Hoque and Adams, 2008; Kleingeld et al., 2004), and the link of performance to rewards
(Burney et al., 2009; Chan, 2004) as key organizational factors associated with the
effectiveness of PMSs.
2.4.1 Top management support. Top management support has been highlighted as
an important contingency factor in supporting various management accounting practices
such as ABC (Baird et al., 2007; Shields, 1995) and MISs (Doll, 1985). The impact of top
management support on PMS effectiveness has been referred to in a number of studies
(Bourne, 2005; Chan, 2004; Bourne et al., 2002; Emerson, 2002; Kennerley and Neely, 2002).
For example, Bourne et al. (2002) investigated the success of the redesign of PMSs. They
found that top management support was influential in the successful implementation
and on-going usage of the new PMS. This study also indicated that the continuous
involvement by top management was invaluable in resolving problems when crises and
conflicts arose. Chan (2004) and Emerson (2002) also reported that top management
commitment and leadership buy in are key factors in enhancing PMS effectiveness.
Similarly, Kennerley and Neely (2002) found that top-level management support was
critical for PMS design and implementation, while the availability of management time to
reflect on measures was a major contributor to the effectiveness of PMSs:
H2. The extent of top management support is associated with the effectiveness of
the PMS.
2.4.2 Training. Training is defined as “a planned effort by an organization to facilitate
the learning of job-related behavior” (Wexley, 1984, p. 13). The importance of training
in relation to the development and implementation of a successful PMS is highlighted
in a number of studies. Cavaluzzo and Ittner (2004, p. 249), for example, found that
performance measurement development and outcomes are positively associated with
the extent of related training provided to the manager. The provision of training

7. resources indicates that an organization is willing to provide sufficient resources to The effectiveness
support the development and implementation of PMSs. of PMSs
Chan (2004) cites training as a crucial factor for PMSs to be effective. All performance
measures need to have a clearly communicated purpose and be perceived as both
relevant and reliable so that managers can access useful information for decision
making. Without training, managers may perceive the PMS measures as less useful and
ignore them when making decisions. Similarly, Emerson (2002) concluded that training 1293
is the key to maintaining the usefulness and the effectiveness of PMSs. Training not only
allows users to understand performance measurement concepts and principles, but also
provides both employees and managers with an opportunity to operate the system.
Hence, the better that users understand the purposes of the system and how to
operationalise it, the more likely they will commit to it, thereby enhancing the likelihood
that the desired results will be achieved:
H3. The extent of PMS-related training provided is associated with the
effectiveness of the PMS.
2.4.3 Employee participation. Many studies have referred to the benefits of employee
empowerment (Morrell and Wilkinson, 2002; Koberg et al., 1999; Chiles and Zorn,
1995) and employee involvement and participation (Cox et al., 2007, 2006; Pun et al.,
2001; Wimalasiri and Kouzmin, 2000). These studies tend to operationalise these
concepts in terms of employees’ involvement in decision making. Similarly, employee
participation refers to the “involvement of managers and their subordinates in
information processing, decision making, or problem solving endeavors” (Wagner,
1994, p. 312). This study operationalises employee participation in terms of the extent
to which lower level employees participate in designing the PMS.
The association between employee participation and the effectiveness of PMSs has
support from prior studies (Chan, 2004; Kleingeld et al., 2004; Kaplan and Norton,
2001). These studies report that a higher level of employee participation contributed to
the effectiveness of PMSs. For instance, Kleingeld et al. (2004) found that on average
the improvement in performance was significantly greater for those employees in a
high participation situation as opposed to those in a low participation situation. This
performance improvement was attributed to both cognitive mechanisms (including
increased communication, better utilization of knowledge, increased understanding of
the job) and motivational mechanisms (less resistance to change, commitment to the
system, acceptance of feedback and goals).
Similarly, Kaplan and Norton (2001) maintained that in order to achieve an effective
BSC, employees at lower levels in the organizational hierarchy should be involved in
the establishment of performance measures. This bottom-up participation approach
allows employees to take the initiative in defining their responsibilities as well as the
associated performance indicators. Therefore, employees will commit to the system
and desired outcomes can be achieved to a greater extent:
H4. The extent of employee participation in designing the PMS is associated with
the effectiveness of the PMS.
2.4.4 The link of performance to rewards. The link of performance to rewards is a
vital contingency factor in motivating employees (Rynes et al., 2005; McShane and
Travaglione, 2003; Bonner and Sprinkle, 2002; PA Consulting Group, 1998).

8. IJOPM A survey of 500 companies reported that companies that link performance to pay
31,12 showed twice the shareholder returns as those who did not (PA Consulting Group, 1998).
McShane and Travaglione (2003) suggested that companies need to align rewards with
performance that is within the employee’s control. Hence, the more employees see a
“line of sight” between their daily actions and the reward, the more motivated they will
be to improve performance.
1294 Linking performance to rewards has also been identified as a crucial factor
influencing the effectiveness of PMSs (Burney et al., 2009; Johanson et al., 2006; Chan,
2004). For instance, in Chan’s (2004) study of municipal governments in the USA and
Canada, it was found that the linkage of the PMS to compensation was uncommon,
and “the lack of linkage of the BSC to rewards” was considered to be a barrier to the
systems’ effectiveness.
While there is a lack of empirical evidence examining the link of performance to
rewards on the effectiveness of PMSs, given the importance of the link of performance to
rewards and the increasing number of large businesses rewarding both employees and
managers based on BSC performance (Epstein and Manzoni, 1998), H5 is stated as follows:
H5. The extent of the link of performance to rewards is associated with the
effectiveness of the PMS.
3. Method
A survey questionnaire was mailed to the senior financial officer of a random sample of
445[1] Australian manufacturing business units identified from the Kompass Australia
(2009) directory[2]. The manufacturing industry was selected as a number of prior
studies on PMSs suggest that manufacturing organizations are more likely to have a
mature and comprehensive PMS in place (Malina and Selto, 2001; Simons, 2000; Kaplan
and Norton, 1996, 1992). Business units were chosen as the unit of analysis because PMS
characteristics may differ across business units within an organization. Senior financial
officers were chosen as they were expected to have a sound understanding of their
business unit’s PMS. The Dillman (2007) tailored design method was employed to
administer the survey[3]. In total, 141 responses were received for a response rate of
30.9 percent. In total, 23 of the questionnaires were incomplete, hence 118 questionnaires
were used for the data analysis. As was the case in Robert (1999), non-response bias was
assessed by comparing the independent and dependent variable values across early and
late respondents. No significant differences were detected.
3.1 Variable measurement
3.1.1 The effectiveness of the PMS. The effectiveness of PMSs is measured by
assessing the extent to which 16 desired outcomes of PMSs have been achieved. The
16 measures (the Appendix) were developed based on a review of the literature relating
to the effectiveness of PMS (Lawler, 2003) with minor modifications made to fit the
context of the study. Respondents were required to indicate the extent to which their
PMS had achieved each of the 16 perceived outcomes using a five-point Likert scale
with anchors of 1 “not at all” and 5 “to a great extent”.
Factor analysis (principal components with varimax rotation) using a cutoff point of
0.60 revealed that the 16 outcomes loaded onto two dimensions, with the factor structure
consistent with Baird (2010). The first dimension included nine items which all refer
to the achievement of organizational goals and objectives, hence, this dimension

9. was labeled “performance-related outcomes”. The second dimension included seven The effectiveness
items which are more concerned with employees, hence this dimension was labeled of PMSs
“staff-related outcomes”. These two dimensions were subsequently scored as the
average score of the items loading on to each dimension with higher (lower) scores
representing a more (less) effective PMS.
3.1.2 The usage of multidimensional performance measures. The extent to which
respondents were using multidimensional performance measures was measured using 1295
two approaches. The first approach required respondents to simply indicate if they were
using a BSC (“yes” or “no”). Since this approach is reliant on respondents understanding
of the nature of a BSC, a more comprehensive approach which focuses on the performance
measures employed within organizations, was also adopted. This approach required
respondents to indicate the extent to which they were using 26 different performance
measures (the Appendix) to assess their business units’ performance, on a five-point
Likert scale with anchors of 1 “not at all” to 5 “to a great extent”. These measures were
derived primarily from the BSC literature and were mainly designed for manufacturing
organizations (Epstein, 2008; Jusoh et al., 2008; Van der Stede et al., 2006; Bryant et al.,
2004; Ittner et al., 2003; Kaplan and Norton, 2001, 1996).
Factor analysis (principal components with varimax rotation) using a cutoff point of
0.6 revealed that the 26 items loaded onto six specific dimensions covering the following
perspectives: financial, customer, internal business, learning, growth and sustainability.
These findings are in line with Figge et al. (2002), except that the learning and growth
perspectives were separated. These two perspectives were subsequently combined in
accordance with the five perspectives BSC model.
Each of the five perspectives were scored as the sum of the items loading onto each
perspective with higher (lower) scores indicating the PMS focused on each perspective
to a greater (lesser) extent. Since a different number of items loaded onto each of
the perspectives, average scores were calculated with the use of multidimensional
performance measure scored as the sum of the averages across the five perspectives
with higher (lower) scores indicating that multidimensional performance measures
were used to a greater (less) extent.
3.1.3 Organizational factors. Each of the four organizational factors was measured
using a summated five-point Likert scale with anchors of 1 “strongly disagree” and
5 “strongly agree”.
Top management support was measured using a three-item summated
scale (the Appendix) with respondents required to indicate the extent to which top
management provided adequate resources (Krumwiede, 1998), communicated effectively
(Grover, 1993) and exercised its authority in support of the PMS. Top management
support was measured as the average score for the three items, with higher (lower) scores
indicating a higher (lower) level of top management support.
The level of related training was measured using three items (the Appendix) drawn
from Baird et al. (2007), with minor adjustments made to fit the context of the current
study. Specifically, respondents were required to indicate if adequate training had been
provided to develop, to implement and to ensure employees understood the PMS.
Training was measured as the average score for the three items, with higher (lower)
scores indicating a higher (lower) level of related training provided by the organization.
In the absence of specific measures in the literature on employee participation in a
PMS context, two self-developed items (the Appendix) were adopted following a review

10. IJOPM of the employee participation/involvement literature (Sinclair et al., 2005; Harel and
31,12 Tzafrir, 1999; Huselid, 1995; Wagner, 1994). Specifically, respondents were required to
indicate the extent to which lower level employees participated in designing the PMS
and were involved in selecting performance measures. The perceived level of employee
participation was subsequently scored as the average score for the two items with
higher (lower) scores indicating a higher (lower) level of employee participation.
1296 The link of performance to rewards was assessed using two items (the Appendix)
based on the literature on performance and rewards (Rynes et al., 2005; Lawler, 2003;
Huselid, 1995). Respondents were required to indicate the extent to which performance
is linked to financial rewards such as pay or bonus, and non-financial rewards such as
recognition or service awards in their organization. The analysis revealed that the two
questions were measuring different factors: the extent to which performance is linked
to financial rewards and to non-financial rewards. These measures are analyzed as
separate independent variables, with higher (lower) scores indicating a stronger
(weaker) link of performance to rewards.
4. Results
Table I shows summary statistics for the dependent and independent variables. For the
multi-item scales, the actual range was comparable with the theoretical range, and the
Cronbach’s a coefficients met or exceeded the 0.70 threshold generally considered
acceptable in regard to scale reliability (Nunnally, 1978, p. 245).
The mean scores of the effectiveness of PMSs for both the performance-related
outcomes (3.50) and the staff-related outcomes (3.26) are slightly higher than the
mid-point of the range, indicating that on average the respondents assessed their
PMS to be moderately effective. The performance-related outcomes were achieved
to a greater extent, with the mean scores of all nine items equal to or greater than the
seven staff-related outcomes. The performance-related outcomes that were achieved
Minimum Maximum
Variables n a Mean SD (theoretical) (theoretical) Cronbach’s a
Independent variables
Use of multidimensional
performance measures 118 2.94 0.70 1.17 (1) 4.67 (5)
Top management support 117 3.51 1.02 1 (1) 5 (5) 0.915
Training 117 3.11 1.07 1 (1) 5 (5) 0.963
Employee participation 117 2.41 1.02 1 (1) 5 (5) 0.761
Link of performance to financial
rewards 117 3.50 1.16 1.00 (1) 5.00 (5)
Link of performance to non-
financial rewards 117 2.93 1.13 1.00 (1) 5.00 (5)
Dependent variables
Effectiveness of PMS
(performance-related outcomes) 117 3.50 0.81 1 (1) 5 (5) 0.932
Effectiveness of PMS (staff-related
outcomes) 117 3.26 0.93 1 (1) 5 (5) 0.924
a
Table I. Note: The number of responses (n) varies due to the fact that not all survey items were completed by
Descriptive statistics respondents

11. to the greatest extent included: assisting in achieving the goals (mean score of 3.68); The effectiveness
providing useful performance feedback to employees (mean score of 3.64); developing of PMSs
a performance-oriented culture (mean score of 3.59); and providing an accurate
assessment of business unit performance (mean score of 3.59). The staff-related
outcomes that were achieved to the greatest extent included: developing individual’s
skill and knowledge (mean score of 3.38), identifying talented employees (mean score
of 3.36), and rewarding talented employees (mean score of 3.31). 1297
In regard to the four organizational factors, while the mean score of most of the
factors lie on the higher end of the scale, the mean value of the link of performance to
non-financial rewards (2.93) was slightly below the mid-point of the range indicating a
relative weak link between performance and non-financial rewards.
As discussed in the method section, two approaches were used to assess
the use of multidimensional performance measures. Table II reveals that 39 respondents
(33.1 percent) indicated that they were using a BSC in their business unit.
The more comprehensive approach to measuring the use of multidimensional
performance measures focused on the extent to which business units were employing
26 performance measures covering the five perspectives of the BSC. Table I reveals that
the mean score for the use of multidimensional performance measures (2.94) was slightly
lower than the mid-point of the range, indicating a moderate use of multidimensional
performance measures in Australian manufacturing organizations.
Table III provides a more detailed analysis of the extent to which measures relating
to each of the five perspectives were employed. The greatest emphasis was placed on
the financial perspective (3.59) followed by the customer (3.43), learning and growth
(3.11), and internal business process (3.06) perspectives. The mean score of the
sustainability perspective (2.19) was below the mid-point of the range indicating a
relatively low level of usage of this perspective.
4.1 Analysis of the association between the use of multidimensional performance
measures and organizational factors with the effectiveness of PMSs
Table IV presents the results of the one-way analysis of variance (ANOVA) used to
examine the difference in the level of PMS effectiveness based on whether respondents
were using a BSC. Respondents using a BSC reported a significantly higher level of
PMS effectiveness with respect to both performance- and staff-related outcomes.
BSC usage Frequency Adjusted percentage
Yes 39 33.1 Table II.
No 79 66.9 BSC usage
BSC perspectives n Minimum Maximum Mean Rank
Financial 118 1.00 (1) 5.00 (5) 3.59 1
Customer 118 1.00 (1) 5.00 (5) 3.43 2
Internal business process 118 1.00 (1) 5.00 (5) 3.06 4 Table III.
Learning and growth 118 1.17 (1) 5.00 (5) 3.11 3 Use of multidimensional
Sustainability 118 1.00 (1) 5.00 (5) 2.19 5 performance measures

12. IJOPM These results provide preliminary evidence that the use of multidimensional
31,12 performance measures is associated with the effectiveness of PMSs, thereby providing
support for H1.
The association between the use of multidimensional performance measures and
PMS effectiveness was also analyzed using a more comprehensive approach based on
the extent of use of multidimensional performance measures. Stepwise regression was
1298 used to examine the association between both the use of multidimensional performance
measures and organizational factors with PMS effectiveness, with the results presented
in Table V. For the effect on performance-related outcomes, the model was statistically
significant (F ¼ 63.812, p ¼ 0.000) with an R 2 of 0.530 indicating that 53 percent of
the variance in the achievement of performance-related outcomes can be explained by the
explanatory factors. The model reveals that the use of multidimensional performance
measures ( p ¼ 0.000) was significantly associated with the effectiveness of PMSs.
In addition, top management support ( p ¼ 0.000) was significantly associated with the
performance-related outcomes.
Table V also provides the findings for staff-related outcomes, with the model found
to be statistically significant (F ¼ 38.535, p ¼ 0.000) with an R 2 of 0.405 indicating
that 40.5 percent of the variance in the achievement of staff-related outcomes can be
explained by the explanatory factors. The model reveals that the use of
multidimensional performance measures was found to be significantly associated
with the achievement of staff-related outcomes ( p ¼ 0.000). The level of training
( p ¼ 0.000) was also significantly associated with PMS effectiveness.
The findings provide further support for H1 and partially support H2 and H3.
The importance of the use of multidimensional performance measures in explaining
the level of PMS effectiveness prompted further exploratory analysis to investigate the
association between each of the five perspectives of the BSC with the effectiveness of
PMSs. These findings are presented in Section 4.2.
Table IV. Performance-related outcomes Staff-related outcomes
Results of the one-way BSC usage n Mean F-statistic Significance Mean F-statistic Significance
ANOVA comparing the
level of PMS effectiveness BSC user 39 3.88 14.297 0.000 3.71 15.869 0.000
based on BSC usage Non-BSC user 78 3.31 3.03
Performance-related outcomes Staff-related outcomes
Variables Coefficient t-statistics Significance Coefficient t-statistics Significance
Table V.
Results of stepwise Multidimensional PMS 0.343 4.512 0.000 0.374 4.465 0.000
regression analysis Top management
of the association support 0.487 6.411 0.000
between the use Training 0.362 4.325 0.000
of the multidimensional F-value 63.812 38.535
performance measures p-value 0.000 0.000
and organizational R2 0.530 0.405
factors with the Adjusted R 2 0.522 0.395
effectiveness of PMSs n 115 115

13. 4.2 Analysis of the association between the five perspectives of the BSC with the The effectiveness
effectiveness of PMSs of PMSs
Table VI reveals the stepwise regression analysis findings. The performance-related
outcomes model was statistically significant (F ¼ 63.847, p ¼ 0.000) with an R 2 of
0.528 indicating that 52.8 percent of the variance in the achievement of the
performance-related outcomes can be explained by the two perspectives of the BSC
found to be significantly associated with the performance-related outcomes: the internal 1299
business process ( p ¼ 0.000) and learning and growth ( p ¼ 0.000) perspectives.
The staff-related outcomes model was also statistically significant (F ¼ 56.768,
p ¼ 0.000) with an R 2 value of 0.499 indicating that 49.9 percent of the variance in the
achievement of the staff-related outcomes can be explained by the two perspectives of
the BSC found to be significantly associated with the staff-related outcomes: the
learning and growth ( p ¼ 0.000) and sustainability ( p ¼ 0.000) perspectives.
5. Conclusion
5.1 Discussion
The first objective of the study was to examine the effectiveness of PMSs in respect to
their impact on organizational processes. The study evaluated the effectiveness of PMSs
based on the extent to which 16 desired outcomes were achieved. By focusing on the
outcomes achieved, the study contributes to the empirical body of knowledge on PMSs
since the majority of previous studies have only assessed PMS effectiveness based on
overall organizational performance. This approach provides managers with a more
detailed insight into the ability of the PMS to assist their organization in achieving
specified desired outcomes. Factor analysis revealed that these items reflected two
dimensions of PMS effectiveness: performance- and staff-related outcomes. The results
revealed that the mean score for the effectiveness of PMSs for both dimensions was
slightly above the mid-point of the range, indicating that the PMSs of Australian
manufacturing organizations were only moderately effective. This finding highlights
the significance of the study’s investigation of the contingency factors associated with
the effectiveness of PMSs.
The results also showed that organizations were more successful in achieving
the performance-related outcomes than the staff-related outcomes. This suggests
that PMSs have mainly been used as a managerial tool to assist the organization in
motivating performance, implementing the organizational strategy and achieving goals.
Performance-related outcomes Staff-related outcomes
Variables Coefficient t-statistics Significance Coefficient t-statistics Significance
Internal business
process 0.277 3.830 0.000 Table VI.
Learning and growth 0.558 7.730 0.000 0.539 7.445 0.000 Results of stepwise
Sustainability 0.289 4.001 0.000 regression analysis
F-value 63.847 56.768 of the association
p-value 0.000 0.000 between each of the five
R2 0.528 0.499 perspectives of the BSC
Adjusted R 2 0.520 0.490 with the effectiveness
n 116 116 of PMSs

14. IJOPM Less emphasis is being placed on achieving staff-related outcomes such as addressing
31,12 the concerns of staff, ensuring staff time is used efficiently, and managing poorly
performing staff.
The latter finding is of concern given that survival in today’s rapidly changing world
is dependent on the achievement of both staff- and performance-related outcomes.
Harel and Tzafrir (1999, p. 185) highlighted the importance of focusing on employees,
1300 suggesting that an organization’s staff are its strategic assets which “form a system of
resources and rare abilities that cannot easily be copied, and provide the company with
its competitive edge”. Hence, organizations which view staff as potential partners and
important assets enhance the likelihood of achieving better organizational performance.
There is also evidence that the achievement of staff-related outcomes can assist in the
achievement of performance-related outcomes. If organizations adequately address
the concerns of their employees, they are more likely to be emotionally attached
to a particular organization, and hence more willing to assist in the achievement of
organizational goals (Myer and Allen, 1991). Accordingly, we suggest that managers
place greater emphasis on the achievement of staff-related outcomes. This should be
embodied in the design of the PMS so as to incorporate both contributions from
employees as well as reflecting their personal needs.
The second objective of the study was to examine the association between the use of
multidimensional performance measures and four organizational factors with the
effectiveness of the PMS. The initial analysis focused on ascertaining the extent to
which organizations were using multidimensional performance measures. Results
revealed that only 33.1 percent of organizations were using the BSC, which is
consistent with previous findings (Crabtree and DeBusk, 2008 (35 percent); Chung et al.,
2006 (31 percent); Speckbacher et al., 2003 (26 percent); Whorter, 2003 (35 percent)).
A more comprehensive analysis of the use of multidimensional performance
measures revealed that Australian manufacturing organizations placed the greatest
emphasis on measures relating to the financial perspective of the BSC, followed by
the customer, learning and growth internal business process, and sustainability
perspectives. This finding is consistent with the majority of the BSC literature which
suggests that financial measures are still used to the greatest extent (Crabtree and
DeBusk, 2008; Hoque and Adams, 2008; Davis and Albright, 2004; Braam and Nijssen,
2004; Ittner et al., 2003; Hoque and James, 2000; Lipe and Salterio, 2000; Ittner and
Larcker, 1998). The findings indicate that while organizations may be enticed to use a
BSC, and even claim to use the BSC, the reality is that the greatest emphasis is still placed
on the traditional financial-based perspective. Therefore, if organizations are to reap the
benefits of using multidimensional PMSs such as the BSC, it is crucial that they do not
just pay lip service to the inclusion of measures covering the other perspectives. Rather
they need to acknowledge the importance of the other perspectives of the BSC and place
increasing emphasis on using measures relating to each of the perspectives.
Analysis of the association between the use of multidimensional performance
measures and organizational factors with the effectiveness of PMSs revealed that the
use of multidimensional performance measures, as operationalized by the BSC, and two
organizational factors (top management support, and training) exhibited a significant
association with the effectiveness of PMSs.
The use of multidimensional performance measures was positively associated with
both the performance- and staff-related outcomes. This finding is in line with previous

15. studies (Chow and Van der Stede, 2006; Van der Stede et al., 2006; Bryant et al., 2004) The effectiveness
which have advocated that organizations should incorporate both financial and of PMSs
non-financial measures in the PMS. Similarly, the findings reinforce the literature
advocating the benefits of the BSC (Langfield-Smith et al., 2009; Epstein, 2008; Kaplan
and Norton, 2006; Speckbacher et al., 2003). The findings highlight the need for
managers to evaluate the inherent characteristics of their PMS and their impact on the
achievement of such outcomes. In particular managers need to focus on the extent to 1301
which diversified performance measures reflecting the five perspectives of the BSC are
incorporated in their PMS.
Additional exploratory analysis revealed that the internal business and learning and
growth perspectives were associated with the effectiveness of PMSs regarding the
performance-related outcomes, while the learning and growth and sustainability
perspectives were significantly associated with the staff-related outcomes. While this
finding highlights the importance of adopting a BSC, it also provides managers with
insight into the specific BSC dimensions which warrant their attention in order to
enhance PMS effectiveness. Managers are therefore encouraged to ensure that their PMS
emphasises the use of performance measures in relation to the internal business process
(e.g. productivity, usage of resources, cycle time and number of product returns),
learning and growth (e.g. hours of training provided, improvements made to employee
facility, number of new product produced and percentage of revenue from new
applications) and sustainability (e.g. investment in environmental management,
promotion of environmental causes and investment in community services) perspectives
in order to enhance the effectiveness of their PMS.
Analysis of the association between the organizational factors and the effectiveness
of the PMS provides an insight into the prevailing organizational conditions that
could enhance/prohibit PMS effectiveness. Top management support was found to be
associated with the performance-related outcomes, and the level of training was
associated with the staff-related outcomes. While top management support has been
found to be a critical success factor for PMS implementation (Bourne, 2005; Chan, 2004;
Bourne et al., 2002; Emerson, 2002; Kennerley and Neely, 2002), the findings highlight the
importance of the continued involvement and support from top management. Hence, in
order to achieve the desired performance-related outcomes, a concentrated effort by top
management aimed at continuous improvement, open communication and consistent
support is required (Kaynak, 2003). Top management is therefore encouraged to
personally commit to the PMS and ensure that enough time and resources are dedicated
on an on-going basis to properly develop and manage the existing PMS. In addition,
organizations which provide more related training to their staff are able to achieve the
desired staff-related outcomes. This supports Harel and Tzafrir’s (1999) suggestion that
moving knowledge information and power to lower levels of the organization is a way to
sustain competitive advantage. Organizations could therefore employ appropriate
training with respect to the use of PMSs across different business levels to enhance the
knowledge and skill of employees in developing and implementing the systems.
The study contributes to the literature by examining PMS effectiveness in terms of
the effect on organizational processes. The two dimensions of PMS effectiveness,
performance- and staff-related outcomes, serve to make management more aware of the
need to focus on different aspects of PMS effectiveness as well as providing researchers
with a new measure which can be used to evaluate its effectiveness. In addition,

16. IJOPM the association between the use of multidimensional performance measures
31,12 and organizational factors with the effectiveness of the PMS provides managers of
organizations with an insight into the desirable characteristics of an effective PMS and
the prevailing organizational conditions which can support the PMS. Hence, managers
need to focus on using multidimensional performance measures, and increase the level of
top management support and related training in relation to their PMS.
1302
5.2 Limitations and future research
The study is subject to the usual limitations of the survey method. While the survey
method is useful in ascertaining associations rather than causal relationships between
variables (Singleton and Straits, 2005), this approach generates potential threats as
respondents may answer questions in accordance with social desirability bias. Future
studies may incorporate face-to-face interviews in order to provide richer descriptions
into the hypothesised associations. Future studies could also collect data from multiple
respondents across different management levels. This may assist in overcoming the
common method bias associated with the single respondent approach.
The study also used a number of self-developed measures. For instance, the measures
of PMS effectiveness, the usage of multidimensional performance measures, and two
organisational factors, employee participation and the link of performance to rewards,
were self-developed. The face validity of these measures was enhanced through a pilot
study of ten academics with relevant expertise, and their content validity was enhanced
by developing the measures based on an extensive review of the relevant literature.
However, while factor analysis provides evidence of the construct validity of the first
three measures, the validity of these measures still needs to be confirmed in future
studies, especially given the sample size is considered small for performing factor
analysis. In addition while the Cronbach’s a scores confirm the reliability of the first
three of these measures, the two items used to measure link of performance to rewards
were found to be measuring separate constructs. Hence, there is concern as to the
reliability of this measure and future studies may explore alternative ways of measuring
this factor.
In addition, the current study only provides empirical evidence in relation to the
association between four organizational factors (top management support, training,
employee participation and link of performance to rewards) and the effectiveness of
PMS. Future studies may consider the association between other organizational factors
such as organizational structure, and management style, with PMS effectiveness.
To enhance the generalizability of the findings, future studies could be conducted
using similar parameters in other industries such as service and the non-profit sector.
Notes
1. Cohen’s (1988) formulae which considers the number of independent variables, statistical
significance and power, and effect size, was used to determine the required number of valid
responses (91). Assuming a conservative response rate of 20 percent, a sample size of 445 was
determined.
2. The Kompass Australia business directory provides details of manufacturing businesses in
Australia. It is assumed that a random sample taken from this directory is representative of
the Australian manufacturing industry.

17. 3. The Dillman (2007) Tailored Design Method provides guidelines in respect to the format and The effectiveness
style of questions, personalisation, and distribution procedures. There is evidence that this
approach leads to improved response rates. of PMSs
References
Atkinson, A., Balakrishnan, A.R., Booth, P., Cote, J.M., Groot, T., Malmi, T., Roberts, H., Uliana, E.
and Wu, A. (1997), “New directions in management accounting research”, Journal of 1303
Management Accounting Research, Vol. 9, pp. 79-108.
Baird, K. (2010), “The effectiveness of strategic performance measurement systems”,
unpublished working paper, Macquarie University, Sydney.
Baird, K., Harrison, G. and Reeve, R. (2007), “Success of activity management practices: the influence
of organizational and cultural factors”, Accounting and Finance, Vol. 47 No. 1, pp. 47-67.
Bedford, D.S., Brown, D.A. and Malmi, T. (2006), “Balanced scorecard content, use,
and performance impacts: some Australian evidence”, paper presented at the European
Accounting Association, 29th Annual Conference, Dublin.
Bhasin, S. (2008), “Lean and performance measurement”, Journal of Manufacturing Technology
Management, Vol. 19 No. 5, pp. 670-84.
Bonner, S.E. and Sprinkle, G.B. (2002), “The effects of monetary incentives on effort and task
performance: theories, evidence, and a framework for research”, Accounting,
Organizations and Society, Vol. 27 Nos 4/5, pp. 303-45.
Bourne, M.C. (2005), “Researching performance measurement system implementation: the
dynamics of success and failure”, Production Planning and Control, Vol. 16 No. 2, pp. 101-13.
Bourne, M.C., Neely, A.D., Platts, K.W. and Mills, J.F. (2002), “The success and failure of
performance measurement initiatives: the perceptions of participating managers”,
International Journal of Operations and Production Management, Vol. 22 No. 11,
pp. 1288-310.
Braam, G.J.M. and Nijssen, E.J. (2004), “Performance effects of using the balanced scorecard:
a note on the Dutch experience”, Long Range Planning, Vol. 37 No. 4, pp. 335-49.
Bryant, L., Jones, D.A. and Widener, S.K. (2004), “Managing value creation within the firm:
an examination of multiple performance measures”, Journal of Management Accounting
Research, Vol. 16 No. 1, pp. 107-31.
Burney, L.L., Henle, C.A. and Widener, S.K. (2009), “A path model examining the relations among
strategic performance measurement system characteristics, organizational justice,
and extra- and in-role performance”, Accounting, Organizations and Society, Vol. 34
Nos 3/4, pp. 305-21.
Cavaluzzo, S.K. and Ittner, C.D. (2004), “Implementing performance measurement innovations:
evidence from government”, Accounting, Organisations and Society, Vol. 29 No. 1,
pp. 243-67.
Chan, Y.C.L. (2004), “Performance measurement and adoption of balanced scorecards: a survey of
municipal governments in the USA and Canada”, The International Journal of Public
Sector Management, Vol. 17 No. 3, pp. 204-21.
Chang, C.J., Lee, H.I. and Chen, W.C. (2008), “A fuzzy AHP and BSC approach for evaluating
performance of IT department in the manufacturing industry in Taiwan”, Expert Systems
with Applications, Vol. 34 No. 1, pp. 96-107.
Cheng, M.I., Dainty, A. and Moore, D. (2007), “Implementing a new performance management
system within a project-based organisation: a case study”, International Journal of
Productivity and Performance Management, Vol. 56 No. 1, pp. 60-75.

18. IJOPM Chiles, A.M. and Zorn, T.E. (1995), “Empowerment in organizations: employees’ perceptions of
the influences on empowerment”, Journal of Applied Communication Research, Vol. 23
31,12 No. 1, pp. 1-25.
Chow, C.W. and Van der Stede, W.A. (2006), “The use and usefulness of nonfinancial
performance measures”, Management Accounting Quarterly, Vol. 7 No. 3, pp. 1-8.
Chung, L.H., Gibbons, P.T. and Schoch, H.P. (2006), “The management of information and
1304 managers in subsidiaries of multinational corporations”, British Journal of Management,
Vol. 17 No. 2, pp. 153-65.
Clinquini, L. and Mitchell, F. (2005), “Success in management accounting: lessons from the
activity-based costing/management experience”, Journal of Accounting and Organizational
Change, Vol. 1 No. 1, pp. 63-78.
Cohen, J. (1988), Statistical Power Analysis for the Behavioral Sciences, 2nd ed.,
Lawrence-Erlbaum, Mahwah, NJ.
Cox, A., Marchington, M. and Sutter, J. (2007), Embedding the Provision of Information and
Consultation in the Workplace: A Longitudinal Analysis of Employee Outcomes in 1998
and 2004, DTI Employee Relations Research Series, No. 72, Department of Trade and
Industry, London.
Cox, A., Zagelmeyer, S. and Marchington, M. (2006), “Embedding employee involvement and
participation at work”, Human Resource Management Journal, Vol. 16 No. 3, pp. 250-67.
Crabtree, A.D. and DeBusk, G.K. (2008), “The effects of adopting the balanced scorecard on
shareholder returns”, Advances in Accounting, Vol. 24 No. 1, pp. 8-15.
Davis, S. and Albright, T. (2004), “An investigation of the effect of balanced scorecard
implementation on financial performance”, Management Accounting Research, Vol. 15
No. 2, pp. 135-53.
De Geuser, F., Mooraj, S. and Oyon, D. (2009), “Does the balanced scorecard add value? Empirical
evidence on its effect on performance”, European Accounting Review, Vol. 18 No. 1, pp. 93-122.
Dillman, D.A. (2007), Mail and Internet Surveys: The Tailored Design Method, 2nd ed., Wiley,
New York, NY.
Doll, W.J. (1985), “Avenues for top management involvement in successful MIS development”,
MIS Quarterly, Vol. 9 No. 1, pp. 17-35.
Emerson, B. (2002), “Training for performance measurement success: an effective training
program can help get performance measurement off the ground and sustain the system
as it matures into a catalyst for government accountability and improvement”,
Government Finance Review, April, available at: www.thefreelibrary.com/Trainingþ
forþperformanceþmeasurementþsuccess%3aþAnþeffectiveþtraining-a085048611
(accessed 1 May 2009).
Epstein, M. (2008), Making Sustainability Work: Best Practices in Managing and Measuring
Social and Environmental Impacts, Greenleaf, Sheffield.
Epstein, M. and Manzoni, J.F. (1998), “Implementing corporate strategy: from tableaux de bord to
balanced scorecard”, European Management Journal, Vol. 16 No. 2, pp. 190-203.
Figge, F., Hahn, T., Schaltegger, S. and Wagner, M. (2002), “The sustainability balanced
scorecard – linking sustainability management to business strategy”, Business Strategy
and the Environment, Vol. 11 No. 5, pp. 269-84.
Grover, V. (1993), “An empirical derived model for the adoption of customer-based
inter-organizational systems”, Decision Sciences, Vol. 24 No. 3, pp. 603-39.
Hamilton, S. and Chervany, N.L. (1981), “Evaluating information system effectiveness – part I:
comparing evaluation approaches”, MIS Quarterly, Vol. 5 No. 3, pp. 55-69.

19. Harel, H.G. and Tzafrir, S.S. (1999), “The effect of human resource management practices on the The effectiveness
perceptions of organizational and market performance of the firm”, Human Resource
Management, Vol. 38 No. 3, pp. 185-200. of PMSs
Hoque, Z. and Adams, C. (2008), Measuring Public Sector Performance: A Study of Government
Departments in Australia, CPA Australia, Melbourne.
Hoque, Z. and James, W. (2000), “Linking balanced scorecard measures to size and market factors:
impact on organizational performance”, Journal of Management Accounting Research, 1305
Vol. 12 No. 1, pp. 1-17.
Horngren, C.T., Bhimani, A., Datar, S.M. and Foster, G. (2005), Management and Cost
Accounting, 3rd ed., Pearson Education, Harlow.
Huselid, M.A. (1995), “The impact of human resource management practices on turnover,
productivity, and corporate financial performance”, Academy of Management Journal,
Vol. 38 No. 3, pp. 535-672.
Ittner, C.D. and Larcker, D.F. (1998), “Innovations in performance measurement: trends and
research implications”, Journal of Management Accounting Research, Vol. 10, pp. 205-38.
Ittner, C.D., Larcker, D.F. and Randall, T. (2003), “Performance implications of strategic
performance measurement in financial services firms”, Accounting, Organizations and
Society, Vol. 28 Nos 7/8, pp. 715-41.
Johanson, U., Skoog, M., Backlund, A. and Almqvist, R. (2006), “Balancing dilemmas of the balanced
scorecard”, Accounting, Auditing & Accountability Journal, Vol. 19 No. 6, pp. 842-57.
Jusoh, R., Ibrahim, D.N. and Zainuddin, Y. (2008), “The performance consequence of multiple
performance measures usage: evidence from the Malaysian manufacturers”, International
Journal of Productivity and Performance Management, Vol. 57 No. 2, pp. 119-36.
Kaplan, R.S. (2001), “Strategic performance measurement and management in nonprofit
organizations”, Nonprofit Management and Leadership, Vol. 11 No. 3, pp. 353-70.
Kaplan, R.S. and Norton, D.P. (1992), “The balanced scorecard-measures that drive performance”,
Harvard Business Review, Vol. 70, January/February, pp. 71-9.
Kaplan, R.S. and Norton, D.P. (1996), “Linking the balanced scorecard to strategy”, California
Management Review, Vol. 39 No. 1, pp. 53-79.
Kaplan, R.S. and Norton, D.P. (2001), The Strategy-focused Organization: How Balanced
Scorecard Companies Thrive in the New Business Environment, Harvard Business School
Press, Boston, MA.
Kaplan, R.S. and Norton, D.P. (2004), “Measuring the strategic readiness of intangible assets”,
Harvard Business Review, February, pp. 52-63.
Kaplan, R.S. and Norton, D.P. (2006), Alignment: Using the Balanced Scorecard to Create
Corporate Synergies, Harvard Business School Press, Boston, MA.
Kaplan, S.E. and Wisner, P.S. (2009), “The judgmental effects of management communications
and fifth balanced scorecard category on performance evaluation”, Behavioral Research in
Accounting, Vol. 21 No. 2, pp. 37-56.
Kaynak, H. (2003), “The relationship between total quality management practices and their
effects on firm performance”, Journal of Operations Management, Vol. 21 No. 4, pp. 405-35.
Kennerley, M. and Neely, A. (2002), “A framework of the factors affecting the evolution of
performance measurement systems”, International Journal of Operations and Production
Management, Vol. 22 No. 11, pp. 1222-45.
Kleingeld, A.D., Tuijl, H.V. and Algera, J. (2004), “A participation in the design of performance
management systems: a quasi-experimental field study”, Journal of Organizational
Behavior, Vol. 25 No. 7, pp. 831-51.

20. IJOPM Koberg, C.S., Boss, R.W., Senjem, J.C. and Goodman, E.A. (1999), “Antecedents and outcomes of
empowerment: empirical evidence from the health care industry”, Group and Organsiation
31,12 Management, Vol. 24 No. 1, pp. 71-91.
Kompass Australia (2009), Kompass Australia, Peter Isaacson Publications, Elsternwick.
Krumwiede, K.R. (1998), “The implementation stages of activity-based costing and the impact of
contextual and organizational factors”, Journal of Management Accounting Research,
1306 Vol. 10, pp. 239-77.
Langfield-Smith, K., Thorne, H. and Hilton, R. (2009), Management Accounting: An Australian
Perspective, 6th ed., McGraw-Hill, Sydney.
Lawler, E.E. (2003), “Reward practices and performance management system effectiveness”,
Organizational Dynamics, Vol. 32 No. 4, pp. 396-404.
Lebas, M.J. (1995), “Performance measurement and performance management”, International
Journal of Production Economics, Vol. 41 Nos 1-3, pp. 23-35.
Lipe, M.G. and Salterio, S.E. (2000), “The balanced scorecard: judgmental effects of common and
unique performance measures”, Accounting Review, Vol. 75 No. 3, pp. 283-9.
Lynch, R.L. and Cross, K.F. (1991), Measure Up! Yardsticks for Continuous Improvement,
Blackwell, Cambridge.
McShane, S. and Travaglione, T. (2003), Organizational Behaviour on the Pacific Rim,
McGraw-Hill, Sydney.
Malina, M.A. and Selto, F.H. (2001), “Communicating and controlling strategy: an empirical
study of the effectiveness of the balanced scorecard”, Journal of Management Accounting
Research, Vol. 13, pp. 47-90.
Marchand, M. and Raymond, L. (2008), “Researching performance measurement systems:
a information systems perspective”, International Journal of Operations and Production
Management, Vol. 28 No. 7, pp. 663-86.
Morrell, K. and Wilkinson, A. (2002), “Empowerment: though the smoke and past the mirrors?”,
Human Resource Development International, Vol. 5 No. 1, pp. 119-30.
Motwani, J., Mirchandani, D., Madan, M. and Gunasekaran, A. (2002), “Successful
implementation of ERP projects: evidence from two case studies”, International Journal
of Production Economics, Vol. 75 Nos 1/2, pp. 83-96.
Myer, J.P. and Allen, N.J. (1991), “A three-component conceptualization of organizational
commitment”, Human Resource Management Review, Vol. 1, pp. 61-89.
Neely, A. (1999), “The performance measurement revolution: why now and what next?”,
International Journal of Operations and Production Management, Vol. 19 No. 2, pp. 205-28.
Neely, A. and Adams, C. (2000), Perspectives on Performance: The Performance Prism,
Gee Publishing, London.
Neely, A., Gregory, M. and Platts, K. (1995), “Performance measurement system design:
a literature review and research agenda”, International Journal of Operations and
Production Management, Vol. 25 No. 12, pp. 1228-63.
Neely, A., Mills, J., Platts, K., Gregory, M. and Richards, H. (1996), “Performance measurement
system design: should process based approaches be adopted?”, International Journal of
Production Economics, Vol. 46, pp. 423-31.
Neely, A., Mills, J., Platts, K., Richards, H., Gregory, M., Bourne, M. and Kennerley, M. (2000),
“Performance measurement system design: developing and testing a process-based
approach”, International Journal of Operations and Production Management, Vol. 20
No. 10, pp. 1119-45.

21. Norreklit, H. (2003), “The balanced scorecard: what is the score? A rhetorical analysis of the The effectiveness
balanced scorecard”, Accounting, Organizations and Society, Vol. 28 No. 6, pp. 591-619.
of PMSs
Nunnally, J.C. (1978), Psychometric Theory, McGraw-Hill, New York, NY.
Othman, R. (2008), “Reflective practice: enhancing the effectiveness of the balanced scorecard
with scenario planning”, International Journal of Productivity and Performance
Management, Vol. 57 No. 3, pp. 259-66.
Otley, D. (1999), “Performance management: a framework for management control systems
1307
research”, Management Accounting Research, Vol. 10 No. 4, pp. 282-362.
PA Consulting Group (1998), “An incentive scheme that works: pay purview, how one
knowledge-intensive company has overhauled its incentives”, The Economist, August,
available at: www.paconsulting.com/introducing-pas-media-site/archive/an-incentive-
scheme-that-works-pay-purview-how-one-knowledge-intensive-company-has-overhauled-
its-incentives/ (accessed 10 April 2009).
Pike, S. and Roos, G. (2004), “Mathematics and modern business management”, Journal of
Intellectual Capital, Vol. 5 No. 2, pp. 243-56.
Pun, K., Chin, K. and Gill, R. (2001), “Determinants of employee involvement practices in
manufacturing enterprises”, Total Quality Management, Vol. 12 No. 1, pp. 95-109.
Raghunathan, B., Raghunathan, T. and Tu, Q. (1999), “Dimensionality of the strategic grid
framework: the construct and its measurement”, Information Systems Research, Vol. 10
No. 4, pp. 343-55.
Rao, S.S. (2000), “Enterprise resource planning: business needs and technologies”, Industrial
Management and Data Systems, Vol. 100 No. 1, pp. 81-8.
Ratnasingam, P. (2009), “Service quality management applying the balanced scorecard:
and exploratory study”, International Journal of Commerce and Management, Vol. 19
No. 2, pp. 127-36.
Rigby, D. and Bilodeau, B. (2009), Management Tools and Trends 2009, Bain & Company,
Boston, MA.
Robert, E.S. (1999), “In defence of the survey method: an illustration from a study of user
information satisfaction”, Accounting and Finance, Vol. 39 No. 1, pp. 53-77.
Rynes, S.L., Gerhart, B. and Parks, L. (2005), “Personnel psychology: performance evaluation and
pay for performance”, Annual Review of Psychology, Vol. 56, pp. 571-600.
Schultz, R. and Ginzberg, M.J. (1984), “Implementation research-third generation”, Applications of
Management Science, Vol. 4, pp. 1-83.
Shields, M.D. (1995), “An empirical analysis of firms’ implementation experiences with
activity-based costing”, Journal of Management Accounting Research, Vol. 7, pp. 148-66.
Silk, S. (1998), “Automating the balanced scorecard”, Management Accounting, Vol. 79 No. 11,
pp. 38-44.
Simons, R. (2000), Performance Measurement and Control Systems for Implementing Strategy,
Prentice-Hall, Englewood Cliffs, NJ.
Sinclair, R.R., Leo, M.C. and Wright, C. (2005), “Benefit system effects on employees’ benefit
knowledge, use, and organizational commitment”, Journal of Business and Psychology,
Vol. 20 No. 1, pp. 3-29.
Singleton, R.A. and Straits, B.C. (2005), Approaches to Social Research, Oxford University Press,
New York, NY.

22. IJOPM Speckbacher, G., Bischof, J. and Pfeiffer, T. (2003), “A descriptive analysis on the implementation
of balanced scorecard in German speaking countries”, Management Accounting Research,
31,12 Vol. 14 No. 4, pp. 361-88.
Van der Stede, W.A., Chow, C.W. and Lin, T.W. (2006), “Strategy, choice of performance
measures, and performance”, Behavioral Research in Accounting, Vol. 18, pp. 185-205.
Wagner, A.J. (1994), “Participation’s effects on performance and satisfaction: a reconsideration of
1308 research evidence”, Academy of Management Review, Vol. 19, pp. 312-30.
Wexley, K.N. (1984), “Personnel training”, Annual Reviews, Vol. 35, pp. 519-51.
Whorter, L.B.M. (2003), “Does the balanced scorecard reduce information overload?”,
Management Accounting Quarterly, Vol. 4 No. 4, pp. 23-7.
Wimalasiri, J. and Kouzmin, A. (2000), “A comparative study of employee involvement
initiatives in Hong Kong and the USA”, International Journal of Manpower, Vol. 21 No. 8,
pp. 614-34.
Appendix. Variable measurement
Effectiveness of PMS (adapted from Lawler (2003), Ittner et al. (2003) and Kaplan and Norton
(2001, 1996))
Please indicate the extent to which your business unit’s PMS assists your business unit in
achieving each of these.
Performance-related outcomes:
Motivating performance.
Assisting in the achievement of goals.
Developing a performance-oriented culture.
Supporting change efforts.
Providing useful performance feedback to employees.
Implementing the organizational strategy.
Providing an accurate assessment of business.
Ensuring staff commitment to organizational objectives.
Linking individual performance to business unit performance.
Staff-related outcomes:
Developing individual’s skill and knowledge.
Addressing the concerns of staff.
Ensuring the concerns of staff.
Identifying talented employees.
Rewarding talented employees.
Identifying poor performing staff.
Managing poor performing staff.
The use of multidimensional performance measures
Financial perspective:
Sales revenue.
Return on investment.
Improvement in net assets/liabilities.

23. Customer perspective: The effectiveness
On-time product delivery. of PMSs
Number of new customers.
Quality of products.
Number of product returns.
Internal business process perspective: 1309
Usage/wastage of resources.
Productivity.
Cycle time.
Expenditure on warranty claims.
Learning and growth perspective:
Hours of training provided.
Improvements made to employee facilities.
Number of employee suggestions implemented.
Number of new products produced.
Time to market for new products.
Percentage of revenue from new products/new applications.
Sustainability:
Investment in environmental management.
Promotion of environmental causes.
Investment in community services.
Community connectedness services.
Promotion of community causes.
Organizational factors
Top management support:
Top management has provided adequate resources to support the PMS.
Top management has effectively communicated its support for the PMS.
Top management exercises its authority in support of the PMS.
Training:
Adequate training has been provided to ensure employees understand the PMS.
Adequate training has been provided to develop the PMS.
Adequate training has been provided to implement the PMS.
Employee participation:
Lower level employees participated in designing the PMS.
Lower level employees were involved in selecting performance measures.
Link of performance to rewards:
Performance is linked to financial rewards (pay, bonuses, etc.) in your business unit.
Performance is linked to non-financial rewards (recognition, service awards, etc.) in your
business unit.

24. IJOPM About the authors
Amy Tung has taught both undergraduate and postgraduate subjects in the management
31,12 accounting area. Her research interests include performance measurement systems,
environmental management and employee organizational commitment. She is undertaking her
PhD in sustainability, with focus on environmental management systems and environmental
performance. Amy Tung is the corresponding author and can be contacted at: manamy.
tung@mq.edu.au
1310 Kevin Baird has taught both undergraduate and postgraduate subjects in the management
accounting area for 16 years. He has also supervised Honours and PhD students across many
different topic areas within the management accounting discipline including activity-based
management practices, total quality management, performance measurement systems,
management control systems, outsourcing, employee organizational commitment and
employee empowerment.
Herbert P. Schoch has taught both undergraduate and post-graduate courses, primarily in
Management Accounting and he has supervised PhD and Honours students. He has also taught
Financial Accounting, Business Strategy and Entrepreneurship and Entrepreneurial
Management. He has taught in Australia, Singapore, Hong Kong, Canada and the USA. His
research interests include management control systems, management accounting, outsourcing,
accounting education and entrepreneurship. He has published numerous journal articles, book
chapters and monographs. He also has experience in working in manufacturing, public
accounting and has managed and operated his own business.
To purchase reprints of this article please e-mail: reprints@emeraldinsight.com
Or visit our web site for further details: www.emeraldinsight.com/reprints