Cecilia Tsui 2008 Higher Education Development: Quality, Relevance and Impact

Report - OECD, IMHE General Conference 2008
Outcomes of Higher Education – Quality, Relevance and Impact
8-10 September 2008 - Paris, France
Cecilia C B S Tsui, PhD candidate, University College, The University of London, United Kingdom

Please note I submitted this report to the OECD, IMHE in October 2008.
I am translating this report into a paper. Please do not quote.

1 Introduction

The OECD, IMHE Conference 2008, entitled “Outcomes of Higher Education: Quality, Relevance
and Impact”, was well attended with participants from over 65 countries. In addition to the plenary
speeches and panels, there were sessions elaborating on the following:

Parallel Group Sessions Sub-themes Parallel Group Sessions on OECD Projects
 Institutional measures to assess and  Thematic review of tertiary education policy
improve quality and policy determinants of investment in
 Reputation and ranking – the impact of tertiary education
institutional strategy and behaviour of  University futures and open educational
international ranking tables resources
 Assessing learning and employment outcomes  Facilities for higher education /
 Balancing the needs and expectations of higher education and regions
society with the autonomy of institutions  Supporting quality teaching and the mobility
 Value for money and efficiency in higher of researchers
education
 Quality and relevance: some policy responses
 Institutional diversity and transparency

The conference was a great success: as reflected in the number of participants from various
countries, and the impressive quality of papers / presentations. The conference theme was timely,
with documentation clearly setting out the objectives. The plenary speeches projected proactive
approaches stimulating participants to reflect critically on key issues, as well as the emerging
combination of outcome measures, including global rankings, and institutional classifications,
which are constituent technologies of a global knowledge-status system. The speakers contributed
a wealth of information on diverse initiatives. In sum, the conference achieved the aims to
facilitate information and experiences sharing, to identify best practices, to develop
recommendations, as well as to facilitate international co-operation and networking within the
higher education community.

Report - OECD, IMHE General Conference 2008 (Cecilia Tsui) 1

Furthermore, the conference provided an intellectually and a culturally rich platform for me to
reflect on related issues in higher education. I also treasure the chance of conversing with experts,
and policy-makers. Some of them have driven the development of quality assurance in different
places, such as, Australia, Europe, the United States and Hong Kong. I particularly appreciate
listening to presentations demonstrating analytical reasoning with socio-economic and theoretical
backgrounds by leading scholars. I find the conference presentations highly relevant to my PhD
study, titled “Quality Assurance in Higher Education: A Hong Kong Perspective”, focusing on
conceptual issues of quality, accountability and managing change.

This paper reports my observation of the quality assurance movement in relation to the conference,
in capacity as a research student in higher education. While I have gained my experience
primarily from Hong Kong and England, I will also draw on development of other countries, where
appropriate. This paper commences with the changing socio-economic contexts, leading to a
sketch of the evolution of the quality assurance movement. It is followed by a highlight of key
messages I perceived at the conference. This paper concludes with a discussion of main issues, as
well as achievements and challenges of the quality assurance movement in higher education.

2 Changing Contexts…
Changing Needs, Expectations and Roles of Universities

In giving his opening address, Professor Berdahl alerted that the environment in which all of our
universities operate is remarkably different from what it was two or three decades ago. There have
been significant global economic and social changes.

2.1 Globalization - Global Knowledge Economy and Higher Education

In today’s global knowledge economies, HEIs are mediums for a wide range of cross-border
relationships and continuous global flows of people, information, knowledge, technologies,
products and financial capital (Marginson and Wende van der 2007a). As objects and, at the same
time, agents (Scott 1998) of globalization, universities will become both less and more important: (a)
less important – for universities will no longer be on their own or out in front in future society; as
there is a proliferation of “knowledge institutions” (which may have little in common with
traditional universities even if they adopt “university” brands); and (b) more important – for the
society will be increasingly structured in terms of knowledgeability, and the production of
‘technical’ will become even more important as an engine of economic development (Scott 2000).
In face of socio-economic challenges, Professor Berdahl emphasized:


… The process of globalization has increased demand for greater accountability and comparability of
educational outcomes. Accountability and transparency are essential ingredients of a globalized
economy. Outcome measures or assurances of the comparability of degrees are a natural consequence
of globalization.

In today’s globalized world coupled with knowledge- and conceptual-economy, universities are no
longer the only contributors to the production of knowledge (Bleiklie and Powell 2005, Scott 2004).
I believe new approaches to accountability, and outcome measures will boost higher education to
develop new capacity, to reinvent, re-enchant itself, to compromise its own integrity in order to
allow a new configuration of “knowledge institutions” to develop (Scott 2000). At the same time,
HEIs will go through waves of learning curves, and will be challenged by new forms of “knowledge
institutions”, such as consultants and think-tanks (Scott 2004).

2.2 Transformation of Technology, K-economy and Open Source Ecology

Professor Berdahl pointed out that the transformation of technology had facilitated the development
of distance learning, as well as virtual universities and virtual classrooms. This is, in part, the
promise of the for-profit institutions. He added that if learning outcomes of such institutions can
be shown to be comparable to traditional universities, such efficiencies are thought to be able to
relieve governments and parents bearing the high costs of education.

In his speech, Professor Marginson alerted the importance of the ‘global knowledge economy’ or
‘k-economy’ transformation ushered in by the Internet. He explained different interpretations of
‘k-economy’ and conceptual issues relating to knowledge as a global public good. Professor
Marginson delineated that the production and dissemination of knowledge goods, the creation of
communicative networks and the emergence of markets are all convergent processes. In the
knowledge economy there are two heterogenous sources of dynamism: (a) economic commerce and
(b) free cultural creations. Freely circulating knowledge goods blend by osmosis into the
communicative open source ecology, which is their medium of production, and multiply. The
benefits of being in the network grow exponentially, because of the ever-expanding number of
connections. The cost of each new unit addition to the network is constant. Total cost grows in
linear pattern and the benefit/cost ratio continually increases. The rate of expansion of the network
increases over time until all potential nodes are included. Hence, the open source ecology is
dynamic - characterized by its scale, fertility, flexibility and disorder.

I consider that the technological advancement and the Internet have brought changes to processes of
teaching and learning, such as (a) change in the learning behaviour and competencies of students, (b)
possible shift of the role of teaching staff in higher education towards an advisor, feedback provider
and controller, (c) rapid spread and turnover of knowledge and so on (Teichler 2001).


In addition, the rapid development of information and computing technologies has facilitated the
open source flows of information and ideas. As delineated in Professor Marginson’s presentation,
the open source ecology has great implications on economic and educational systems.

2.3 Massification … Concerns for Standards… Demands for Outcome Measures

The development of mass higher education after the Second World War exhibited a rapid growth in
the number of students seeking a university-type education (Gibbons 1998, Trow 2000). Since
then, the history of higher education in the United States and Europe has been a history of the
expansion of access and its consequences (Trow 2000) (Note 1). As for Asia and the Pacific, there
has been rapid growth in student enrolments and expansion in higher education provision since
1998 (UNESCO 2003). Nowadays, there are many developed countries near to achieving
‘universal’ higher education (Brennan 2007, Guri-Rosenblit et al 2007); following Trow’s (1973,
1974) celebrated distinction among elite, mass and universal (Note 1) forms of higher education.
The massification of higher education has created substantial pressures and changes in higher
education.

There is tremendous demographic transformation of tertiary education. Professor Berdahl pointed
out that the number of students enrolled in higher education worldwide has doubled since 1990,
reaching an estimated 135 million today (Richard Freeman, Harvard and NBER cited in Berdahl
2008) [conference presentation]. In some countries, such as, Australia, Denmark, Finland, Iceland,
Italy, the Netherlands, New Zealand, Norway, and Poland, the tertiary education enrollment rate is
more than 40% (OECD, Education at a Glance cited in Berdahl 2008) [conference presentation].
With the increasing demand for tertiary education, Professor Berdahl concerned that the numbers
merely seeking credential rather than an education have increased. As a result, there have been
calls for more rigorous standards on HEIs to ensure that the learning of students meets minimum
requirements.

Professor Berdahl pointed out that with the process of ‘massification’ and the recognition of the
economic utility of higher education; it is increasingly viewed as a commodity. The rapid growth
of for-profit institutions, with their emphasis on preparation for employment, has spurred the
demands for outcome measures, especially for those outcomes, like job placement, which can be
easily measured.


2.4 Quality Needs Diversity

Amidst the changing contexts and anxiety for falling standards and reputation in today’s higher
education sector, Professor Brink presented alternative, but impressive, views on widening
participation (increasing the ratio of participation of societal groups under-represented in higher
education) and equality issues; based on Michael Young’s (1973) concept of meritocracy. On the
basis of his experience in South Africa, Australia, and the United Kingdom, he discussed some
common fears and negative opinions, including (a) ‘standards will drop’, (b) ‘our reputation will
suffer’, (c) ‘it’s not our problem’, (d) ‘it’s unfair’, and ‘it’s a waste of time’. To argue that
diversity is a necessary ingredient of quality, Professor Brink illustrated cases of successful
widening participation programmes, such as ‘Flexible on Access, Firm on Success’, which turned
weak starters into strong finishers with extra support and guidance. He alerted that quality needs
diversity.

Based on Professor Brink’s viewpoints, I suggest that we need to embrace the changing phenomena
of massification and diversification as part of social development for knowledge-based society. As
Bruno Carapinha, a student representative, responded that higher education must build upon two
pillars of equal relevance: (a) the pursuit of quality of education and (b) equity of our societies.
Indeed, by the advent of mass education beginning in North America in the late 1960s and in full
swing in Europe by the beginning of the 1990s, the notion of quality has evolved into ‘fitness for
purpose’, allowing HEIs to demonstrate their achievement of objectives according to purposes and
missions (Lenn 2004). Quality is a matter of interpretation; as Professor Brink stated “… different
interpretations are possible, because you put words to numbers, meanings to words, and social
parameters to meanings”. In a sense, performance is relative to context, as the concept of ‘quality’
is relative.

On the other hand, Teichler (2008) analyzes that expansion of student enrollment is desirable, and
expansion is interwined with diversification of HEIs. The majority of actors and experts predict a
growing diversification (Teichler 2003) - through the establishment of new providers and the
creation of various consortia and partnerships between universities for research and / or teaching
purposes (Guri-Rosenblit 2007). Teichler (2008) suggests that increasing diversity of HEIs is
beneficial, in terms of quantity, quality, relevance and efficiency of higher education. I consider
Teichler’s (2008) analysis relates to Trow’s (1974) views on the functions of higher education in its
three forms as being:


 Elite: shaping the mind and character of the ruling class; preparation for elite roles
 Mass: transmission of skills; preparation for a broader range of technical and economic elite
roles
 Universal: adaptation of the ‘whole population’ to rapid social and technological change

I observe that, nowadays, to meet changing socio-economic challenges, these three forms – elite,
mass and universal – of higher education co-exist in different stratification of societies. The
massified and diversified higher education systems also call for multi-dimensional quality assurance
approaches (see Appendix A) (Brennan 2007:39), as advocated by stakeholders. In sum, I support
Teichler (2001) that it is the co-existence of interesting interpretations and confused claims of key
actors that drive the higher education sector to find new ways of mastering the changing conditions.

2.5 Tertiary Education and Society – Relevance and Impact

Education relates to every part of a person’s life, and to every aspect of our society. We are in a
changing society, with lots of unannounced changes (Clark 1996). Hence, the education sector
and the society need to have closer collaboration to map out education, and research for future
development. Regarding social relevance and impact issues, at the conference, there were three
institutional cases demonstrating how HEIs collaborate with communities for (a) curriculum
development, (b) learning enhancement and (c) research with entrepreneurial spirits. These cases
demonstrated that nowadays HEIs integrate with the society at large, but not just ivory towers. I
summarize these three cases as follows:

Knowledge as a public property: the societal relevance of scientific research

At the beginning of his presentation, Professor Bouter raised four questions – (a) Why did you start?
(b) What did you do? (c) What answer did you get? and (d) What does it mean anyway? His
presentation then focused on the last question – What does it mean anyway? - so as to alert the
academic community that scientific research should have social-cultural value and economic value.
He cited two examples, in areas of health and life sciences, illustrating how scientific research
makes societal relevance and impacts at the VU University of Amsterdam, the Netherlands. While
there have been lots of efforts to identify indicators of societal relevance, we must note that the
nature of research and time frame vary enormously. Sometimes the societal impact of a study is
readily apparent, but it often takes many years to make it felt. Professor Bouter finally invited the
academic community to take up the challenge of developing concrete performance indicators, in a
creative and constructive manner.


I consider Professor Bouter’s proposed performance indicators would help researchers to better
formulate research questions / frameworks, as well as to articulate and apply research results to the
society with close community connectivity. Nevertheless, how well Professor Bouter’s proposal
would be received by the academia is another matter of interest.

Applying knowledge for the public good: societal relevance of
university social service programmes for sustainable community development

Dr Landeros illustrated how social service is integrated into the substantive functions of the
university. The integration can be an effective strategy to address societal relevance and attend to
issues of inequality and sustainability, especially in developing countries. Universities in Latin
America face the challenge of being a direct agent of change and a force of social integration.
Through community engagement, universities can better fulfill their mission and justifiably
compete on par with first world institutions in their relevance to society. So, teaching, research
and service should work as complementary parts of a whole, which mutually enhance and fortify
each other. In her presentation, Dr Landeros also shared the tradition of university-based
community social service in Mexico, and illustrated the case of Universidad La Salle, Mexico City,
Community Service in Residence Programme aiming to poverty reduction. This presentation
demonstrated that social service could be a linchpin among research, teaching and extension to
achieve a greater positive impact on society.

Balancing the needs and expectations of society
with the autonomy of higher education institutions

Professor Purcell raised a few key questions about needs, expectations, roles, and autonomy of
universities in our contemporary society. She then addressed the following questions:

 What are the HE needs and expectations of society?

The role of a university is to further the knowledge of the community and to encourage and develop
scholarship and learning. Based on contexts of the United Kingdom, and the University of
Plymouth, Professor Purcell delineated that universities should (a) contribute to growing the
knowledge economy, (b) develop and maintain a broad, advanced knowledge base, (C) provide
quality education, training along the ideals of lifelong learning, (d) create centers of excellence in
research, consultancy and advice, (e) embed international, multi-cultural education in the
curriculum, (f) promote equal opportunities and access to higher education.


 What does a university bring to a region?

Universities impact greatly on their community - not only contributing to the regional skills base,
local employment and educational opportunities, but also in the provision of cultural and sporting
facilities. Professor Purcell cited how the University of Plymouth and universities in the United
Kingdom contribute to the economy in the region, and in the country respectively. In addition, she
pointed out that universities bring an array of opportunities to the community, such as (a)
education – lifelong learning, (b) employment and providers of skilled graduates, (c) cultural and
sporting facilities, (d) business support – personal, professional and business development, (e)
supporting the public sector and local services, (f) enterprise, innovation and knowledge transfer
opportunities, (g) significant contribution to the regional economy.

 What is meant by the autonomy of universities?

Professor Purcell reiterated the important role of a university in our society, and cited the following
quotation to illustrate the importance of autonomy.

In order to compete effectively at home and abroad, universities must be able to control their policies
and resources. Without such autonomy for universities we cannot play our full part in realizing the
Government’s and the country’s objectives (Rick Trainor, Universities UK Annual Meeting 2007).

Professor Purcell further stressed that a university should have autonomy in a number of areas, for
example (a) staff and students – admissions, progress and discipline, (b) curriculum and teaching –
methods, examinations, content, text books, (c) academic quality – academic standards, quality
audits, accreditation working with QAA, (d) research and publication – postgraduate teaching,
priorities, freedom to publish, (e) governance – councils, academic boards, student associations, (f)
administration and finance-funding of institutions, operating grants, capital, (g) global competition
and so on.

 How can we balance society’s expectations and a university’s autonomy? Is there a conflict?

Universities must ensure society and students are getting the best deal without compromising
academic freedom and standards. Universities do not exist in isolation but as part of a wider
community – often with global reach and impact. If HEIs are truly to meet the needs of
stakeholders in business, the professions, the public sector and the community at large – we need to
become more demand-led and better attuned to the priorities of our community. We can influence
the development of society through our academic values, quality standards and stakeholder
engagement. To conclude the presentation, Professor Purcell’s advice was:


 To ensure there is a balance between society’s expectations and individual institution autonomy:
 Higher Education Institutions need to be transparent and accountable, particularly in the areas
of academic quality and income, building sustainable partnerships both in the UK and overseas.
 Organizations need to regularly assess risk and diversify their funding streams to ensure
financial security, running the institution in a professional and business-like approach.
 Universities need to work with stakeholders to gain market intelligence and have robust
governance structures and procedures.
 We can neither afford to be ‘an ivory tower’ nor do we want to be one!

As illustrated in presentations of this conference, the higher education community has been trying
to figure out how to assess learning outcomes in this new environment. In the past, quality, impact
and relevance pointed to student access issues, to academic governance issues and to relevance for
society. In recent years, they have changed their meanings and have become familiar in different
contexts, that of levels of professional quality, international culture and global competition.

Professor Noorda pointed out that quality, relevance, and impact are key criteria for measuring
university performance. The three criteria are powerful drivers of strategic developments in higher
education. However, with the changing environment, HEIs must embrace and promote transparent
and robust diversity. For example, responding to staff / student mobility, existing systems
(arrangements) need to make room for more joint programmes and degrees, as well as the
development of inter-national textbooks, curricula and a multi-national teaching faculty.

In addition, Professor Berdahl remarked that HEIs have been struggling to deal with the impact that
demographic, consumer-oriented, technological, political and global imperatives have brought to
institutions. I agree with Professor Berdahl that a discussion of the quality, relevance, and impact
of higher education outcomes must begin with these changing realities. We must ensure that HEIs
are relevant to and have an impact on society’s needs of the twenty-first century.


3 An Overview of Evolution of Quality Assurance Mechanisms …
Meeting Changing Socio-economic Needs

The story of quality assurance in higher education is both old and new. Responding to changing
societal development and needs, quality assurance mechanisms have evolved, and have become
more sophisticated emphasizing on evidence-based learning outcomes. At the conference, Dr
Ewell gave a brief review of learning assessment in the United States. Professor Amaral and
colleagues delineated the developments in Europe and the United States. Observing presentations
of this conference, I see that there are changing conceptions of quality assurance resulting from the
expansion of higher education and social changes. I, therefore, present an overview of the
evolution of quality assurance since the 1980s, hoping it helps analyzing the contemporary context
in this paper.

3.1 Quality is Always a Concern

Quality has always been a concern in higher education. van Vught (1993) states that the debate on
the most appropriate form of quality assurance dates back to academic medieval Europe. Neave
(2004a) (at a personal interview in 2004) boldly claims:

… quality has always been an issue… except the vocabulary, and the new procedures, which we have
to be sensitive because the world is moving so fast … what we are making sure that the certificates
issued by universities are up to a minimal standard… that function has always been there… it has
formed in different modes… in different institutional and political rotations…

The higher education community has had different mechanisms in place to uphold academic
standards of courses and degrees (Harman 2000). He illustrates that for countries and places with
strong connections with Britain (e.g. Hong Kong, Singapore); there are well-developed systems of
external examiners. Some other countries (e.g. United States, the Philippines) have had long
experience with accreditation mechanisms (Harman 2000). Following the Second World War, the
tertiary education sector has been expanding, and responding to diverse social changes.

3.2 Since 1980s – Public Concerns for Explicit Quality Assurance Mechanisms

Since the early 1980s, there have been explicit concerns for quality and change in higher education
(Dill et al 1996, Dill 2007, Green 1994, Mayhew et al 1990, OECD 2008a, van Vught and
Westerheijden 1994). Systematic and explicit mechanisms of quality assurance started from a
majority of states in the United States (Dill 2007). Dr Ewell, at the conference, made a remark
on the changing phenomenon:


The beginning of assessment as an explicit topic of policy discussion in the U.S. is usually marked at
1985, the year that the first national conference on this topic was held… Despite… voluntary efforts,
state mandates constituted the most important stimulus for assessment in its early years… more than
90% of institutions reported that they were in some way engaged with “assessment” by 1990.

With occasional exceptions, the federal government was not much concerned about the quality of
student learning outcomes until about three years ago. This changed markedly in the summer of 2005
when Secretary of Education Margaret Spellings – stimulated by evidence of eroding U.S.
competitiveness in tertiary degree production when compared to leading OECD countries – convened
the Secretary’s Commission on the Future of Higher Education.

Following the United States, new national quality assurance policies were introduced to the France
in 1984, the United Kingdom in 1985 and the Netherlands in 1985 (van Vught and Westerheijden
1993, Vroeijenstijn 2007). According to Dill (2007), the developments [in the United States] were
diffused to other countries in Europe, Asia and eventually around the globe (Dill 2007). At the
conference, Professor Amaral and colleagues pointed out:

In Europe the development of quality assurance activities has started much later than in the U.S. The
emergence of the “Evaluative State” (Neave 1998:7) was observed in the late 1980s, with increasing
public relevance given to quality… The development of quality assurance in Europe has been fast.
Schwarz and Westerheijden (2004) report that in the early 1990s less than 50% of the European
countries had initiated quality assessment activities at supra-institutional level, while in 2003 all
countries except Greece entered into some form of supra-institutional assessment.

The European quality assurance systems share important procedural elements – internal self-evaluation,
visit by an external expert review panel, external evaluation and public reporting (Thune 2002).
However, there are important differences in political discourses (Neave 1998, 2004b) that range from a
mainly European and political discourse, with universities assumed as a public service (e.g. France and
Sweden) to a mainly economic discourse, market-based and inspired in the U.S. (e.g. UK and the
Netherlands) with the role of the state seen as excessive (Neave 2004b). There are also differences in
the ownership of the system and in the consequences of quality assessment – with or without direct
influence on funding.

Regarding development in Asia and the Pacific, Harman (1996) delineates that in most cases the
initiative has come from governments and government agencies. However, in some cases, such as
Korea, New Zealand, Taiwan and the Philippines, the HEIs themselves have played an important
role in moves to establish new quality assurance mechanisms. The following highlights the
chronology of early development in a few countries in Asia and the Pacific Region:


 In Australia, quality assurance emerged on the political agenda in the early 1990s. The
Committee for Quality Assurance in Higher Education (CQAHE) was established in 1993, and
the Committee was discontinued after its 1995 review. By the end of the 1990s there was
renewed interest in establishing formal quality assurance mechanisms. The Australian
Universities Quality Agency (AUQA) was set up in 2000 (Meek 2005:27-30).
 China introduced national quality evaluation mechanisms in 1985. By 1993, eight disciplines
encompassing 55 degrees had been reviewed (Sensicle 1993 cited in Harman 1996).
 Korea introduced a comprehensive system of university accreditation in 1991 (Harman 1996).
 In Taiwan, since 1991 efforts have been under way to improve the system of specialized
accreditation of programmes and institutions operated for some years by the Ministry of
Education (Su 1993 cited in Harman 1996).

As a gauge for the growth of quality assurance in higher education globally, the International
Network of Quality Assurance Agencies in Higher Education was founded in 1991, in Hong Kong,
with 20 member countries (Lenn 2004). The growth of [quality assurance] is mirrored in East
Asia and the Pacific region – 12 of the agencies were founded since 1991, and 11 of those since
1994 (Lenn 2004). Despite differences in ideology, size and stage of development of tertiary
education sectors (Amaral et al 2008 [conference presentation], Scott 2005), governments have
decided that traditional academic controls are inadequate to contemporary challenges (see Appendix
B for rationale for the rapid growth of quality assurance in higher education). In sum, since the
early 1980s, there have been growing interests in establishing policy mechanisms to ensure quality
and accountability in tertiary education (El-Khawas 1998).

3.3 Quality Assurance Mechanisms – Old Wine in New Bottles

Based on principles of (a) accountability, (b) compliance with standards and (c) quality
improvement, there have been various scopes of responsibility, approaches and methods of quality
assurance at different levels to meet respective national and international needs. Comparing with
traditional methods, I observe that the quality assurance movement has induced a few explicit and
systematic procedures - aiming to strengthen core values, incorporate new forms of management,
and project new roles of higher education. I summarize my observation below:

 Strengthening emphasis on teaching / learning
Governments create mechanisms to investigate teaching / learning conditions or encourage HEIs to
set up internal mechanisms for assuring that established standards are met (Martin and Stella 2007).
This has strengthened that learning and teaching are key missions of HEIs. However, Scott
(2004:13) alerts: “… we are moving towards a ‘new economy’ in which segregated domains labeled
research or scholarship (…learning and teaching) are becoming redundant...”.


I suggest we should take heed of the fact that the boundary between ‘teaching’ and ‘research’ is
getting fuzzier.

 Demonstrating regular and systematic documentation and reporting to stakeholders
There are more formal and deliberate mechanisms of review and reporting, as well as new ways of
demonstrating to a wider group of stakeholders the standing and relevance of degrees. The more
systematic monitoring of performance helps to provide evidence to substantiate claims about quality,
and ensuring course relevance to community needs (Harman 2000). I believe this development is
built on the Total Quality Management spirit of keeping records systematically, and reporting
regularly with reflective learning, and client-focused culture.

 Emphasis on Outcomes
The outcome-based approach has been practised in the United States for about twenty-five years
(Ewell n.d.). In recent years, there is a global trend of adopting this approach to ensure academic
standards. It is becoming more concerned with outcomes than with inputs and throughputs (Martin
and Stella 2007). I consider this is in parallel with the ideals of New Public Management stressing
the need for hands-on management of organizational units, and performance need to be evaluated in
terms of clear, measurable output indicators. It is also due to the rise of global knowledge
economies, calling for new modes of knowledge production (Scott 2004) and mobility of
knowledge workers – hence, evidenced-based learning outcomes.

 From National to International Arenas - Diplomatic Relations
In addition, the recent initiatives of quality assurance, such as, development of international quality
assurance and worldwide quality, are forming part of the education acumen of the diplomatic
relations between great nations (Neave 2004a). Moreover, with increasing globalization and
internationalization of higher education, the outcome-based approach quality assurance will help
facilitating mobility of staff and students.

My view is that from public management perspective, these changes for explicit, regular measures
are good. Nevertheless, there are cultural conflicts against the traditional academic freedom. But,
with quality assurance development evolving in the last two decades, the professional academic
values, and culture should have changed in one way or another.


3.4 Quality Assurance Movement – Journey of Change

The quality assurance movement is a journey of change. As from the early 1980s, there have been
four to five phases of quality assurance schemes (see Appendix C) focusing on different problems
and with different roles of quality assurance (Westerheijden 2007:11). As a matter of fact, quality
assurance mechanisms are designed with a built-in facility for learning and change; while
maintaining the delicate balance between the functions of improvement and accountability, even if
all other conditions (e.g. external aspects) remain equal. The changing nature of quality assurance
mechanisms also avoids bureaucratization and window dressing so as to ensure improvement and
accountability (Jeliazkova & Westerheijden 2001). On the other hand, Brennan (2007:39)
summarizes five types of quality assurance methods (see Appendix A), reflecting a continuing
balance of power within and outside higher education in the United Kingdom. In my view, the
movement of quality assurance has facilitated the development of higher education, which was, at
one time, in crisis (OECD 1987, World Bank 1994); following rapid expansion of higher education
after the Second World War, and against the backdrop of global economic and social changes.
Nowadays, higher education is a huge and diversified sector; operating in changing contexts with
new forms of management and educational modes so as to meet stakeholders’ needs, which carry
consumer-oriented, political and global imperatives.

I observe that quality assurance is a policy instrument, as an umbrella, embracing a wide range of
interests and issues, such as, policy, governance, teaching, learning, assessment, student support and
performance management. The wide spectrum of issues is also due to the fact that changing one
component in a system will inevitably affect the others. The quality assurance movement has been
a change agent, under the ideals of New Public Management (Note 2). Despite tensions, I believe
quality assurance will continue to play significant functions to ensure academic standards,
accountability, cost-effectiveness, and to promote transparency, improvement, and change. To
sustain efforts and to build capacity, quality culture in HEIs, as well as to provide better information
to stakeholders, I consider it timely to discuss outcomes, relevance, and impact issues, in
contemporary sense.


4 Key Messages

This section delineates the key messages that I perceived at the conference. There are three
messages; namely: (a) public demand for outcomes – learning and research, (b) rankings, league
tables and classifications, and (c) the growing accountability agenda.

4.1 Public Demand for Outcomes – Learning and Research

At his opening address, Professor Berdahl remarked that responding to a call from the OECD
Education Ministers in June 2006, the discussion of ‘student learning outcomes’ has been the major
topic in quality assurance. In April 2007, the experts concluded that such assessment was a
‘reachable’ goal and that ‘reliable information on learning outcomes would only rise in importance’
in the future. Professor Berdahl stated:

Let us begin with a discussion of “learning outcomes” which has gained such traction over the last
decade. The question of “learning outcomes” has its origins from outside the academy in the demands
for greater efficiency and accountability that arose in the 1980s, largely from the governments funding
higher education.

We must note that the public demand for greater transparency in the learning outcomes for students
has its parallel in the public’s demand for outcomes for its investment in research. Professor
Berdahl also analyzed the university-industry partnership, and its consequences to socio-economic
development, as well as research in various disciplines in HEIs.

In the wake of the above external, deeply-held, and widespread concerns, there are various studies
to figure out how to assess learning outcomes, or how much students are actually learning in HEIs,
in this new environment. Among the various assessments, I would like to highlight the following
three, as Professor Berdahl spelled out:

 Collegiate Learning Assessment (CLA) in the United States

CLA is test designed to assess how students’ critical thinking, analytical reasoning, problem
solving and written communication competencies have developed during the course of their
college years.


 Roadmap for the OECD Assessment of Higher Education Learning Outcomes (AHELO)
Feasibility Study (launched in July 2008)

AHELO seeks to examine achievement in both generic skills and disciplinary skills, with a
value-added and contextual component of both.

 Qualifications Framework

It is a framework of competencies, which students at each level of degree attainment must
demonstrate in some manner. There are three levels; namely:

Transnational level – the ‘Framework for Qualifications of the European Higher Education Area’
provides the broadest, most generic levels of attainment, with a progression from more concrete,
occupation-oriented competencies at the lowest level, to more abstract, independent and complex
competencies at higher levels.

National level – the development of ‘National Qualification Frameworks’ (consistent with the
European-wide qualifications framework) are specific to the institutional structure and needs of
each of the participating nations. The purpose is not to create identical degrees in each country, or
identical paths for attaining degrees, but to create analogous degrees that have common
competencies that can be understood everywhere, creating a ‘zone of trust’.

Institutional level – ‘Institutional Qualification Frameworks’ are specific to the curriculum of each
institution and, beyond that, in what is known as the ‘Tuning Project’ qualifications frameworks
specific to each discipline or field, with benchmarks for subject specific knowledge and generic
skills and competencies to be attained at each level of study.

In general, there are two views of assessment:

 Assessment for institutional and disciplinary improvement
 Assessment for public information, policy formation and resource allocation

Furthermore, Professor Marginson alerted that comparative measures of student learning in higher
education are an obvious gap in the apparatus of policy instruments and a frontier issue for the
sector. In his words:


… If established on a credible and comprehensive basis, and communicated well, such measures could
be expected to become important to prospective domestic and foreign students, employers and investors
in higher education, not to mention governments focused on national capacity and the competitiveness
of national k-economies…

… AHELO is likely both to emphasize the claims of those institutions that position themselves as
doyens of synergistic Humboltian leadership in research and teaching simultaneously; and also to
encourage some bifurcation between the two missions, as alternative routes to institutional primacy.
In the longer term such developments are potentially transformative of the sector and its relations with
society and economy, in the national, meta-national regional and global dimensions of action.

He then commented on the present assessment of learning outcomes in the OECD countries, and
made a few remarks about assessment for consideration as follows:

 Unit of measurement – institutional vs national basis
 Breath of inclusion of nations – AHELO assessment of student learning outcomes would be
viable if most but not all countries be included
 Capacity of measurement – measurement should encompass the diversity of learning
objectives, modes and contents and in different languages, as well as cultural variations in
academic disciplines and the manner in which the learned professions are practised
 Components of measurement – mix of generic and discipline-based skills and knowledge
deemed appropriate for comparative purpose
 Form of assessment – summative or value-added assessments
 Employability – employability vs proxy measures, e.g. graduate preparedness

Measuring outcomes of education is not an easy task, but essential for students. As reflected in
some conference presentations, such as Campbell (2008), and Chalmers (2008), studies are
underway to identify and implement teaching / learning quality indicators in terms of input, process,
output and outcome. I consider that in today’s knowledge-based society with massified and
diversified higher education sector, the above-mentioned assessments are proactive measures.
HEIs need to provide students with clear indications of what their paths should look like, what
levels of knowledge and skills will qualify them for degree awards, and what their degrees mean.
These road signs will be highly beneficial for future growth and development of students.


4.2 Rankings, League Tables and Classifications

Rankings – High on Agendas of Stakeholders

“… In recent years, as many countries have introduced tuition fees, and as tuition prices have
escalated in the United States and elsewhere, rankings have been the focus of increased scrutiny…”,
remarked by Alisa Cunningham at the conference. In her presentation, Professor Hazelkorn
claimed that rankings started out as an innocuous consumer product (Note 3) for undergraduate
domestic students, and it has become both a manifestation and driver of the global battle for
excellence.

In some countries, such as, Argentina, Nigeria, and India, the ranking exercise is undertaken as part
of the accreditation process (Salmi and Saroyan 2007). On the other hand, concerning about the
academia, Professor Marginson remarked that the new system for assigning ordered values to
knowledge (‘k-status’) includes: (a) league tables and university rankings in research, (b)
publication metrics and citation metrics, (c) journal hierarchies, and (d) other comparative measures
such as learning outcomes.

Professor Hazelkorn delineated findings of an OECD comparative study conducted in these few
years. Rankings and league tables are high on agendas of various stakeholders - politicians, HE
leaders, staff, students and parents. In her words:

… Politicians regularly refer to them as a measure of their nation’s economic strengths and aspirations,
universities use them to help set or define targets mapping their performance against the various metrics,
while academics use rankings to bolster their own professional reputation and status…

According to Professor Hazelkorn, mounting evidence indicates that rankings are perceived as
playing critical roles as follows:

 Enabling and facilitating universities to maintain and build reputation
 High-achieving students, and international / postgraduate students use rankings to ‘short list’
institutions
 External stakeholders use rankings to influence decisions about funding, sponsorship and
recruitment / employment

Rankings-consciousness is rising rapidly because benefits are perceived to flow from high ranking.
Conversely, ‘fear of falling’ and the negative publicity associated with it can be as great for highly
ranked or ambitious universities as non-appearance can be for others.


I agree with Professor Hazelkorn that rankings are encouraging and influencing a number of matters,
including:

 Modernization and rationalization of institutions
 Professionalization of services
 Marketization of higher education
 Research mission and fields of investigation
 Curriculum and disciplines
 Faculty recruitment and new career / contractual arrangements
 Student choice and employment opportunities

As illustrated in conference presentations, rankings and similar benchmarking assessments do
influence institutional behaviour and performance. At the same time, we should beware of
shortcomings of rankings.

Furthermore, Professor Marginson pronounced, rankings, metrics, classifications, learning
outcomes are common terms for HEIs nowadays. I particularly appreciate Professor Marginson’s
warning about the culture of comparison, and his call for clean comparisons - which are (a)
inclusive on a global scale, (b) permit a broad range of missions and institutional cultures, (c)
sustain self-determination, (d) support genuine creativity, (e) foster universal improvement, as well
as (f) competition for measured performance open to merit and innovation. The most important
thing is that global rankings as the benchmark only makes sense if the indicators are appropriate –
otherwise, governments and institutions risk transforming their system as institutions to conform to
metrics designed by others for unrelated purposes.

Rankings – Pros and Cons

Rankings and league tables have been around for a long time, but there were dramatic growth in the
past few decades (Altbach 2006). In the era of massification, rankings are also inevitable. It is
natural that those who finance higher education and the public want to know which institutions are
the best. Governments and funding authorities have to decide how best to invest resources. Mass
higher education requires differentiation since institutions serve diverse purposes and students
attend HEIs for many reasons. Rankings can help to define differentiated academic systems if
they can be devised to capture a variety of metrics, and thus help stakeholders to make
well-informed decisions (Altbach 2006).


At the conference, Alisa Cunningham remarked that we have to beware of problems and
weaknesses, such as, inconsistent results, data accuracy, and methodological issues. She then
illustrated a few examples:

 Rankings may skew student, institutional or governmental decision-making
 Changes in methodology and weighting may result in inconsistent results
 Single number, overall rankings do not reflect nuances within institutions
 Available data may not be ideal to achieve purposes

Harvey (2008) criticizes that the link of rankings to quality is naïve. The construction of indices
by which institutions / departments are ranked is arbitrary, inconsistent and based on convenience
measures. The operationalisation of the concept of quality is cursory at best, and nearly all focus
on just one notion of quality - excellence. He further remarks that critiques of rankings come from
an array of sources, and are based on methodological, pragmatic, moral and philosophical concerns;
including: (a) selection of indicators, (b) weighting of indicators, (c) reliability, (d) statistical
insignificance, (e) institutional, political and cultural environment, (f) focus of attention, and (g)
competition (Harvey 2008).

On the other hand, Altbach (2006) comments the problems of rankings as follows:

 Many rankings resemble ‘popularity contests’ – asking groups in the academic community
their opinions about peer institutions
 Rankings count factors such as (a) external funding, (b) numbers of articles and books written
by faculty members, (c) library resources, (d) proportion of faculty members with advanced
degrees, and (e) quality of students (measured by scores on admissions or other tests). These
numbers are assumed to be a proxy for quality, which they are to a significant extent.
However, the number of articles published does not necessarily relate to the quality or impact
of the articles.
 Rankings generally do not include teaching quality. There are no widely accepted methods
for measuring teaching quality, and assessing the impact of education on students is so far an
unexplored area.
 Rankings do not accommodate universities have different missions and goals. Rankings tend
to emphasize the norms of the top research universities. The assumption is that ‘one size fits
all’ and that the norms of the research universities are the gold standard.

As illustrated, ranking is a tool, and an outcome from stakeholders / market perspective. There are
pros and cons of rankings (see Appendix D for a summarized list). Its recent rise also reflects a
widespread phenomenon in an increasingly competitive world of higher education.


There is strong realization that some form of national and international comparators is useful,
inevitable and here to stay. Higher education has to learn to live with those commercialized
rankings. There are also huge policy implications, and a role for educating public opinion.
Various international initiatives, such as the OECD, the EU, are responding to these challenges
(Hazelkorn 2007). Amidst existing rankings, there are various research studies being conducted to
prepare for the growing mobility of students / staff and to provide more channels for information.
I consider that the CHE ranking study and the classification study, reported in the following
sub-sections, are proactive approaches to counterbalance the global competitive scenarios.

Searching for Best Practice - The CHE Ranking

The landscape of rankings of HEIs has become diverse at international and national levels. Dr
Westerheijden and colleagues remarked:

Increasing public interest in university rankings is reflected also in the amount of academic literature
that has been devoted to the issue in recent years. University rankings have been examined from
methodological, technical and conceptual aspects. While the literature points to many serious
problems in university rankings, there seems to be a general consensus that “rankings are here to stay”
(Merisotis 2002) and that energy should go into improving rankings rather than fighting them (e.g.
Marginson and Wende 2007b; Dill and Soo 2005; van Dyke 2005). Moreover, well-designed rankings
can provide students with valuable information and encourage accountability in universities. Recent
research consistently draws attention on potential dangers of poorly constructed rankings and suggests
some principles that would make a ranking sound and beneficial.

They also shared findings of their pilot project – the CHE Ranking at the conference. The aim is
to pilot the CHE Ranking further, beyond the German-language area, to HEIs in the Netherlands
and in the Flemish community of Belgium. Dr Westerheijden and colleagues pointed out various
issues in the design of university rankings. I sum up the main points below:

 Critical issues in the design of university rankings
 Aggregate vs multi-dimensional rankings
 University vs discipline rankings
 Ranking vs clustering
 Choice of indicators and measurement
 The effect of rankings on HE systems
 International rankings


As shown in various conference presentations, the international experience with university rankings
provides many lessons about the implications of different ranking systems. I support views of Dr
Westerheijden and colleagues that while rankings have been heavily criticized from conceptual and
methodological standpoints, and for their potentially dysfunctional effects, the criticisms should be
considered as a constructive input in the process of improving the quality and effectiveness of
rankings.

Institutional Diversity and Transparency

Classifying European Institutions of Higher Education

Professor van Vught’s presentation addressed the importance of diversity in European higher
education, and proposed developing different ranking instruments. Bartelse and van Vught (2007)
argue that a better understanding of the various types of HEIs, their missions and provisions, will
support the European aim of (a) increasing student mobility, (b) inter-institutional and
university-industry cooperation, (c) the recognition of degrees and hence (d) the international
competitiveness. In addition, Professor van Vught emphasized classifications help to enhance the
benefits of rankings and minimize their shortcomings:

If we wish to maintain and even increase the diversity of higher education systems, we will have to
develop different ranking instruments in which different forms of institutional performance can be
compared. We should design multiple ranking instruments that enable us to make inter-institutional
comparisons per category or type of institution. In order to create higher levels of diversity in higher
education systems, we therefore need to develop typologies of higher education (or classifications) the
diversity of institutional missions or profiles should be made transparent, offering the different
stakeholders a better understanding of the specific ambitions and performances of the various types of
higher education institutions.
van Vught (2008:21)

At the conference, Professor van Vught pointed out the need to create more transparency about
institutional diversity. He explored the concept of diversity, and discussed the history of European
higher education from the perspective of this concept. Then, he took stock of the outcomes
regarding diversity in the Bologna process, and remarked that the institutional diversity of European
higher education largely remains ‘hidden’. Professor van Vught also delineated the classification
instrument as a tool to create transparency about institutional diversity. He stated that the
classification instrument should be highly useful for institutional strategies, inter-institutional
partnerships, benchmarking and networking. I observe that this classification instrument should
provide a reliable and transparent source of information, and profiles of HEIs, from educational and
professional perspectives, rather than the present market-dominated ones.


Convergence and / or Diversity? The Argument of Transparency

Professor van Damme stated that higher education systems are engaged in processes of convergence
driven by an array of factors, including (a) globalization, (b) the integrated system of scientific
research, (c) the transnational mobility of students, teaching staff, researchers and institutional
leaders, (d) the technology-driven expansion of new teaching and learning modes, and (e) the
international labour market of graduates. He then illustrated the importance of regional processes
of integration, and cited the Bologna Process in Europe as an example. In addition, he stressed the
importance of transparency, and shared a number of models of convergence, such as the Tuning
Project, Qualification System, AHELO and Global Rankings.

Professor van Damme then raised two questions: (a) Does convergence on systems level mean that
institutions become more and more similar to one another? (b) Is system convergence a favourable
condition for institutional diversification? to further illustrate that transparency becomes crucial.
As a result, new instruments of enhancing system transparency are needed to complement national
regulation, based on systematically collected and comparable data. He concluded that
evidence-based transparency creates a more favourable environment for both the development of
integrated higher education systems as for the safeguarding of institutional autonomy than ‘wild’
competition based on unproven institutional reputations.

4.3 The Growing Accountability Agenda

There has been growing accountability agenda, as illustrated in a few conference presentations. In
his opening address, Professor Berdahl started with a brief discussion of ‘learning outcomes’, and
‘rankings’. He then elaborated that in fulfilling the test of accountability, there are four
fundamental principles:

 The essential element in the learning process is the role of the faculty
 Peer review is important in the assessment of learning outcomes and expectations
 The crux of public accountability is to assure the degrees awarded attain commonly understood
levels of attainment
 Accountability is not just to satisfy public authorities and choice in a comparative market
model, but also to inform students of the expectations that the pursuit of a particular degree
will impose upon them.


Dr Salmi pointed out two significant examples of increasing calls for accountability, and raised the
question whether this is a favourable development for higher education:

… In Europe, an important part of the ongoing Bologna process consists in designing a
qualifications framework that will provide common performance criteria in the form of
learning outcomes and competencies for each degree awarded. In the US, one of the key
recommendations made by the Commission on the Future of Higher Education set up by the
Secretary of Education Spellings in 2004 was to call for measures of student learning to assess
the actual added value achieved by tertiary education institutions.

He further alerted that the growing accountability agenda is also reflected in the multiplicity of
stakeholders, the multiplicity of themes under scrutiny, and the multiplicity of instruments and
channels of accountability. I agree with Dr Salmi that these three dimensions make for a situation
of unprecedented complexity. As a result, in a number of countries, there are complaints from
universities about performance indicators overload, stressing that too much energy and time is spent
on mining and reporting the data monitored by the government.

Moreover, the rankings published by the press have given extra burden and frustration to the
academia. There is a risk of pushing HEIs to ‘doctor’ their statistics to improve their standing in
the rankings. Another concern is that institutions may pay more attention to factors, such as SAT
scores and donations from alumni that receive prominence in the rankings at the expense of
improving teaching and learning, which should be more important from an education viewpoint. I
agree with Dr Salmi that the universal push for accountability has made the role of university
leaders much more demanding.

Despite the above-mentioned negative impacts, there are positive benefits of accountability. Dr
Salmi revealed that employers, students and HEIs all benefit from increased information about
existing programmes and labour market outcomes. For example, in countries where surveys of
student engagement are conducted regularly, e.g. Australia, Canada, United Kingdom, United States,
higher school graduates are better equipped to choose which HEIs they would like to attend. Also,
accountability can be useful when HEIs use their reporting obligations as a management tool to
monitor ability to meet strategic targets.

To alleviate the pressures, Dr Salmi put forward two principles of good accountability, which I
perceive as positive and proactive:


(a) Accountability works better when it is experienced in a constructive way. Hence, accountability
should be more about making strategic choices to improve results; rather than justifying a poor
performance.
(b) The most effective accountability mechanisms are those that are mutually agreed or are
voluntarily embraced by HEIs. This ensures a greater sense of responsibility with respect to the
feedback process and fuller ownership of the agreed instruments.

Apart from Dr Salmi’s recommendations, Professor El-Khawas, at the conference, suggested to
define the role of academics in accountability. She pointed out that the policy debate on
accountability had given too little attention to systems of assessing learning outcomes (too much
focused on inputs). To improve the situation, academics must be involved in (a) developing
assessments, (b) testing and refining them, and (c) responding with changes in programmes or
delivery. I consider this is a very good suggestion because it provides top-down, and bottom-up
approaches in policy design and implementation. It also helps academic staff to reflect on their
long held values, as well as assumptions, and to adopt a student outcomes approach to
accountability.

In sum, I support that academic staff must be accountable to those whom we serve – our students
and our societies; as Professor Berdahl put forward in his speech. On the other hand, academic
staff must be accountable to the academia itself. Hence, academic freedom in research and
teaching is also important so as to achieve a balanced approach to accountability in higher
education. I believe that stakeholders and the academia should work hand-in-hand to draw on the
positive side of accountability.

5 Discussions and Conclusions

As illustrated in sections two and three of this paper, quality assurance mechanisms have been
developed within the political, socio-economic and higher education policy contexts in which the
HEIs situate. Despite various tensions (relating to accountability, improvement, changing HEI
structure, culture and roles of academic staff), this paper delineates that the quality assurance
movement has been playing significant roles to transform higher education to meet challenges of
today’s knowledge society. It is generally agreed that quality assurance mechanisms serve as
policy instruments to achieve a variety of matters. This section discusses main issues of quality
assurance, including (a) maintaining the essence of education, (b) enhancing human capital, (c)
serving as a change agent, as well as (d) achievements and challenges ahead.


5.1 Quality Assurance: Quality vs Diversity
– Maintaining the Essence of Education, Preventing Diploma Disease

In face of expanding higher education systems all over the world coupled with globalization and
rise of new providers, I observe that quality assurance mechanisms play significant roles to ensure
academic standards; as well as to facilitate the development of higher education, which was in crisis
(OECD 1987, World Bank 1994) associated with diverse socio-economic factors and decreasing
government funding support per student unit after the Second World War. On reflection, I consider
that quality assurance mechanisms serve as a gatekeeper to prevent problems of Diploma Disease
(Dore 1976) projected by Ronald Dore (1976:8).

Dore’s (1976) major claim is that the Diploma Disease is a serious problem – with an expanded
education system (a) pupils will become more concerned about being certified, or ‘credentialling’;
but not with the mastery of subject matters, or being educated; and (b) educational institutions, at all
levels, will be at risk of becoming mere certificate-issuing factories. These concerns do happen in
contemporary society. But, I consider that the quality assurance mechanisms have, in some ways,
given certain degrees of protection to safeguard academic standards. In fact, since the expansion
of higher education during the 1960s-1990s, there have been critiques about falling academic
standards, and poor attitudes towards learning and a number of other problems. Harvey (2008)
maintains that massification has not broadened access. There has been an overall deterioration in
higher education. In recent months, Alderman (2008) warns that academic standards are in decline
- students who would once have been failed their degrees pass, and students who would once have
been awarded respectable lower seconds are now awarded upper seconds and even firsts.

Williams (Chief Executive of QAA, UK) (2008) responds to comments about falling academic
standards. He argues that standards are robust, and that the QAA procedures regularly and
repeatedly demonstrate the strength of academic quality and standards throughout a very diverse
system. Furthermore, degree classification grew up when the higher education system was small
and homogeneous. These days, the significance of classification for the future lives and careers of
hundreds of thousands of graduates each year is huge. Also, Williams (2008) alerts that in such a
complex world as higher education, the old is replaced by the new in every-decreasing timeframes.
New challenges need new solutions, and HEIs are facing up to this with an impressive willingness.
Quality and standards are easy words to say, but they represent very complex ideas. I consider
debates about academic standards result from different interpretations on notions of quality. Let’s
contemplate - which notion of quality, excellence, or fitness for purpose, one should hold in today’s
massified, or even universalized higher education systems?


Regarding falling academic standards, Harvey (2008) delineates that when exploring impact on
‘quality of degree’ is whether this refers to some abstract ‘gold standard’ notion of what an
ideal-type degree should be, or whether it relates to the student learning experience. There are
those who would argue that, despite quality assurance, a degree is not what it used to be. Indeed,
when a tiny proportion of (privileged) people attended university, the degree was exclusive and
quite different from today. Whether the exclusivity made it any better (other than more marketable)
is a moot point. The suggestion by advocates of a golden age is that when it was exclusive, the
quality of an awarded degree was ‘higher’, implying that students did more in order to successfully
achieve an award. These comments are highly contentious. Harvey (2008:4) quotes the
Vice-Chancellor of the University of Surrey as follows:

I believe it can be argued that the quality of the corresponding top 13% (that is 13% of 43% [of the
cohort going into higher education in England]) of graduates in the HE sector is at least as strong as it
was for the full cohort in the 1970s.

Many senior managers and government leaders experienced a more elite education system and may not
fully appreciate the balance between quality and diversity we see today. The vast majority of the
additional 30% who gain a HE experience will benefit, raising the overall educational attainment of the
nation’s workforce significantly (Snowden 2008).

We should note that since massification of higher education during 1960s-1990s, the notion of
quality has evolved into ‘fitness for purpose’. At the conference, Professor Brink alerted that
quality is a matter of interpretation, and quality needs diversity. It is clear that debates about
quality and standards are influenced by perception and experiences of individuals. I agree with
Snowden (2008) that we cannot compare the present massified and diversified systems with the
elite one. I appreciate Teichler’s views (2001) that it is the co-existence of interpretations and
confused claims of key actors that drive the higher education sector to find new ways of mastering
the changing conditions. Also, I support Williams (2008) that quality and standards are easy
words to say, but they represent very complex ideas. I suggest we need to ask ‘what if’ questions.
If there were no quality assurance mechanisms in massified education systems, what would happen?

5.2 Outcome-Based Quality Assurance: Building Academic Culture of Evidence
– Enhancing Human Capital in Knowledge Society

Nowadays, there has been increasing demand and attention for higher education, which is both a
wealth generating body, and a wealth distributing body (Neave 2004a). Investment of human
capital, as a key growth input (Fuente 2003), is at the heart of strategies in many countries to
promote economic prosperity, fuller employment and social cohesion (OECD 1998).


Human capital is considered as one of the four essential ‘pillars’ – being (a) innovation, (b)
technology, (c) human capital, and (d) enterprise dynamics - in societal development
(Asgeirsdottir 2005) (See Appendix E). Furthermore, OECD (2007) reiterates that education is a
major key factor in forming human capital.

Owing to the centrality of the knowledge economy to 21st century development, higher education
has assumed unprecedented and significant roles in educating people for the new economy and in
creating new knowledge (Altbach 1998). Hence, the higher education sector pays extensive and
visible attention to ‘student learning outcomes’. Ewell (n.d.) delineates that the conversation about
outcomes and assessment in higher education began in earnest about twenty-five years ago in the
United States, about fifteen years ago in Australasia, and somewhat more recently in Europe with
the emergence of the Bologna process. As well, Hong Kong is presently undergoing outcome-based
audit with government funded HEIs.

The rise of outcome-based approach has led to two important trends. First, evidence of the
attainment of important learning outcomes – increasingly aligned to global or cross-national
standards – is becoming a central element of national accreditation or quality assurance mechanisms.
Second, equally important, an institution’s capacity to harness evidence about student learning
outcomes that go beyond individual classrooms is critical to self-improvement and curricular
alignment. HEIs worldwide are reaching the conclusion that this kind of intentionality about
student learning is increasingly necessary in today’s world (Ewell n.d.).

I believe the development of assessments, such as, the Collegiate Learning Assessment (CLA) and
the Roadmap for the OECD Assessment of Higher Education Learning Outcomes (AHELO)
Feasibility Study, and Qualification Frameworks presented at the conference will facilitate learners
to take charge of their portfolios, and enable HEIs to prepare for increasing collaboration with
stakeholders. I observe that outcome-based quality assurance will take the higher education sector
to another step forward. It will facilitate unlocking the black box of the
teaching-learning-assessment processes. Also, as Dr Ewell suggested at the conference that this
outcome-based approach should enable HEIs to adopt a proactive stance in order to harness external
mandates for internal improvement, and to communicate actively with stakeholders.

According to Ewell (n.d.), outcome-based approaches provide a way to establish aligned standards
of achievement consistent with an increasingly globalized economy and civil society. At the same
time, through their incorporation in authentic academic cultures of evidence in individual
institutional settings, they provide a way of continuously improvement to the process of teaching
and learning. My view is that this approach will boost the education sector to work closely with
stakeholders in a proactive manner. Again, it will raise the par for HEIs.


In particular, this outcome-based approach quality assurance serves as an agent to meet changing
needs of learners as well as workplace in knowledge society, where traditional bureaucracies in
industrial era are collapsing, and there are now new ways of organizing work. As educators, we
need to ask the fundamental question – are we preparing youngsters for societal changes? (Cheng
2004)

5.3 Quality Assurance - A Change Agent

Quality assurance is a change agent: responding to social changes and under the influence of New
Public Management (NPM). The quality assurance movement is a result of direct governmental
initiative (e.g. France and the UK), or indirect government pressure (e.g. the Netherlands); aiming
to change the steering of and, or renew the higher education sector (Kogan et al 2000, Neave 1988,
Stensaker 2002, Westerheijden, Brennan and Maassen 1994). As Professor Berdahl’s opening
address emphasized:

…But the movement to assess and measure the outcomes of higher education stems, I believe, from
much more fundamental changes in the societies of the developed world

There are studies showing that quality assurance changes institutional policies and practices in both
positive and negative ways (Brennan and Shah 2000, Gosling 2004, Stensaker 2002). On the
positive side, studies show that there is raising profile of teaching and learning in HEIs (Brennan
and Shah 2000), greater centralization (Stensaker 1996, 1999a, 1999b cited in Stensaker 2002) in
procedures and in organizational decision-making, as well as more autonomous role for the
institutional management, HEIs are more open and transparent - with quantifiable measures, more
information than ever before are published about higher education and its outcomes (Stensaker
2002). On the other hand, studies show that there are negative changes, such as HEIs becoming
more ‘bureaucratic’. As well, there are studies reporting on tensions among quality, culture and
change (Strydom et al 2004).

In my opinion, quality assurance has facilitated higher education to meet changing societal needs,
assuring academic standards and fulfilling accountability and enhancement. Bearing in mind that
(a) HEIs originate from the oldest type of organization since medieval ages, and (b) with
massification and globalization of higher education after the Second World War, the purpose, scope,
scale, level, direction, and frequency of change have been huge and diverse; not to mention
organizational issues, such as, changing student profiles, culture and resistance to change. I
believe the quality efforts, research findings, and experiences gained in the last quarter century
should form a good foundation for further studies and adaptations for organizational learning.


Moreover, an array of factors, including (a) the missions and purposes of higher education, (b) the
expectation to contribute to social goals, (c) the recent emphasis on economic utility, (d) the
established professional culture of HEIs, and (e) the tensions resulted from contextual changes
(Gibbons 2002:7) make changing HEIs profound, complex and challenging. I consider these are
reasons why the quality assurance movement itself is a journey of change – with quality assurance
mechanisms changing to meet new demands and to stimulate organizational learning. I estimate
that the outcome-based quality assurance approach will further raise the par.

5.4 Outcomes of Higher Education: Quality, Relevance and Impact
- Achievements and Challenges Ahead

Achievements

Since 1980s, the quality assurance movement has been evolving and contributing to the
development of higher education. Professor Berdahl alerted the importance of ‘learning outcomes’
at his opening speech, and remarked:

The question of “learning outcomes” has its origins from outside the academy in the demands for
greater efficiency and accountability that arose in the 1980s, largely from the governments funding
higher education. From within academia, it began with a subtle, but significant shift from the
emphasis on improving the quality of faculty teaching to an emphasis on improving the quality of
student learning.

Comparing the above remark by Professor Berdahl with other conference presentations -
elaborating on various themes, such as, (a) applying the Balance Scorecard to raise institutional
profile, (b) researching quality indicators through ‘input-process-output-outcome’ model, (c)
researching new forms of rankings and classifications to counter-balance the market dominant
scenarios and so on - I see that significant progress with quality assurance mechanisms has been
made. Since the 1980s there have been tensions and controversies on various issues, including:
language, concepts, power, and change, but quality assurance mechanisms have evolved, and have
become more sophisticated emphasizing on evidence-based learning outcomes.

Referring to presentations of this conference and the body of literature on the subject of ‘quality’, I
agree with other observers that both national and institutional quality assurance have played and
will continue to play important roles in (a) maintaining academic standards, (b) achieving
improvements specially in teaching and learning, and (c) giving employers and the wider public
increased confidence in higher education awards, as well as the knowledge and skills gained by
students (Harman 2000).


As delineated in section 3.4 of this paper, there are four to five phases of quality assurance
mechanisms focusing on different problems and with different roles of quality assurance
(Westerheijden 2007:11). The question of whether quality assurance has had any impacts remains
to be answered (Harvey 2008). Yet, Stensaker (2007:60-61 cited in Harvey 2008) identifies four
areas of impact; namely, (a) power, (b) professionalism, (c) permeability and (d) public relations. I
perceive that quality assurance, through hard and soft managerialism, has brought the higher
education sector forward from elite to mass, and to universal systems meeting challenges in today’s
global knowledge economies.

Challenges Ahead

Higher education has been facing strong external and internal forces since the 1980s. With trends
of outcome-based quality assurance, proliferation of commercialized rankings, and growing
accountability agenda, I consider that there will be great challenges ahead. As Dr Salmi, at the
conference, remarked, higher education is facing a situation of unprecedented complexity. The
following section delineates a few major issues.

 Concepts – Excellence vs Fitness for Purpose
As shown in section 5.1, the debates about quality and academic standards result from different
interpretations. Many observers and stakeholders had their higher education experience in an elite
system. Their comments and views are based on the notion of ‘excellence’, and, in many cases
conflict with the current mass, or universal contexts (with different types and sectors of higher
education), where the notion of ‘fitness for purpose’ is more appropriate. I would suggest that
publicity work should be done to educate stakeholders in this aspect.

 Learning Outcomes
From perspectives of employers and students, measuring outcomes seems imperative in our
knowledge societies with high staff and student mobility. However, assessing the learning-
teaching-assessment processes means turning tacit knowledge into explicit, measurable formats. It
is not an easy task. It may take some time to research performance indicators, and to train up
academic staff to make good use of new processes / procedures so as to achieve the desired
outcomes. Governments and HEIs have to put in great efforts and resources for staff development.

 Rankings
The current situation is market dominated. Harvey (2008:13) warns that rankings provide a real
threat to quality processes. In his words:


… The simplistic processes and idiosyncratic construction of league tables appear to have more popular
appeal and even credibility amongst a range of stakeholders, including political decision makers, than
the meticulous hard work of quality agencies.

… the fundamental problem is that they do not appear to ‘reward’ teaching. In particular, contrary to
some recent developments in quality assurance that focus on enhancing the student experience of
learning, rankings place a potential brake on the development of critical transformed learners.
Developing a critical education is not a way to move up league tables.

Many stakeholders, including quality agencies, researchers and academic staff hold strong views
against rankings. The OECD (2008b cited in Harvey 2008) also points out that the bias in the
information base of existing rankings towards research outcomes could detract from efforts to
improve educational performance. As shown in section 4 of this paper, I am glad the CHE ranking
and Classification studies are under-way so as to counter-balance the current market-dominated
situation.

 Diversity… Marketization… Commodification… Diploma Disease…?
Higher education is massified, marketized, diversified and commodified. There are different types
of higher education at different levels, including: transnational higher education, on-line education
and virtual universities and academic-business collaboration and so on. The diversified systems
pose challenges for quality assurance agencies. There is a growing phenomenon of
commodification of higher education. The implications are that HEIs are drawn into the market,
producing and selling knowledge as a commodity. There is a risk of putting consumers at the
center of institutional strategy. Worst of all, there are increasing degree mills selling low-quality
or fake qualifications. Whenever I see advertisements of degree mills, I ask two questions – Why
higher education is commodified? Are there any organizations to stop the danger of falling into
diploma disease?

 Sustaining Quality
– Establishing Institutional and Professional Cultures amidst Changes
Change is the only constant. Higher education has been subjecting to increased accountability and
scrutiny since the 1980s. There have been waves of changes in various aspects. These system
level and institutional changes have had fundamental implications for staff workloads, as well as
institutional and professional cultures. To sustain quality efforts in the past two decades or so,
HEIs have to pay attention to build quality culture, and to shape HEIs as an ideal workplace for
talented people to join the force, as some participants pointed out at the end of the conference. I
consider that the enhancement of institutional and professional cultures is essential.


 Changing Academic Values
Under the influence of New Public Management, HEIs have changed in different ways. The
relationships between universities and stakeholders have changed as well. Henkel (1997) reflects
that institutional leaders have to respond to the insinuation of new values - performance, efficiency,
and competition in the emerging markets. Many academics, if not all, have to struggle holding on
to the traditional values and conceptions of professional practice that belonged to an elite system.
I foresee that the rise of outcome-based quality assurance and the growing accountability agenda
will drive HEIs and academic staff to go through another wave of changes. Hence, governments
and HEIs need to provide support to facilitate change management, and to establish academic
values in contemporary sense.

All in all, the roles of HEIs are immense, complex and vital in today’s globalized economy. There
are many opportunities and challenges emerging, with political, economic and social implications.
As a result, governments, supranational organizations, HEIs and stakeholders have been re-defining
various issues in higher education from time to time. In sum, with (a) the rise of knowledge
institutions, (b) new modes of knowledge production, (c) diversification of higher education, and (d)
public demand for outcomes in learning and research, the management of higher education calls for
a mastery of the art and science of applying the right dose of managerialism and academic
autonomy.

Cecilia C B S Tsui
30 October 2008


Notes:

1 Expansion of Higher Education
 According to Martin Trow (1973) The Transition from Elite to Mass to Universal Access, the fundamental
character of a nation’s higher education system changes – its purposes, academic standards, curricula,
internal governance etc., as the portion of a society’s young people attending increase from below 15%
(elite), to below 50% (mass), to greater than 50% (universal access).
 Neave (2004a) points out that UK became a mass higher education system in the course of the late 1980s.

2 New public management
 Shattock, M. (2008) “The change from private to public governance of British higher education: its
consequences for higher education policy making 1980-2006”, Higher Education Quarterly, 162 (3)
181-203

 According to Pollitt and Boukaert (2000), New Public Management (NPM) is a “bundle of disparate
elements”. There are three main themes:
 Modernization – bringing in faster and more flexible ways of budgeting, managing and accounting for
the delivery of services.
 Marketization – introducing market type mechanisms, separating purchasers from producers,
encouraging user responsiveness, turning clients into customers, making public sector organizations
compete with one another.
 Minimization – outsourcing or “hollowing out”, pushing decision-making downwards to smaller units,
under the semblance of giving them greater autonomy.

 Marginson and Wende (2007a:9) states” “The new public management tends towards universality in the
United Kingdom, Australia and New Zealand, in much of Eastern Europe and Asia, and in parts of the
developing world where reforms in higher education are often generated in World Bank loans-financed
programmes. In developed nations and the relatively robust policy systems of emerging nations such as
China, Singapore and Malaysia, the reforms are often motivated by desires for global competitiveness but
generated from within the nation. The new public management has been applied less completely in Western
Europe and North America. But it has influence everywhere...”

3 League Tables as Policy Instruments
 Salmi and Saroyan (2007:1) states “In 1963, at the University of California at Berkeley, faculty and the
administration objected when the radical student newspaper on campus, Cal Reporter, took the initiative to
publish student evaluations of courses and professors. Despite this initial resistance, student evaluations
have become part of many universities’ internal accountability mechanisms”.


Appendix A

The Evolution of Quality Assurance Methods in UK Higher Education
Brennan (2007: 39)

Type 1 Subject focus – knowledge and curricula
‘Academic’ Professional authority
Quality values vary across institution

Type 2 Institutional focus – policies and procedures
‘Managerial’ Managerial authority
Quality values in variant across institution

Type 3 People focus – skills and competencies
‘Pedagogic’ Staff developers / educationalist influence
Quality values in variant across institution

Type 4 Output focus – graduate standards / learning outcomes
‘Employment Focus’ Employment / professional authority
Quality values both variant and in variant
across institution

Type 5 Student focus – student satisfaction
‘Consumerist’ Authority of the marketplace
Quality values vary across institution


Appendix B

Rationale for Rapid Growth of Quality Assurance in Higher Education

The heighten interests for quality assurance is attributed to an array of pressures, influences and
interrelated trends (Altbach et al 1999, Brennan and Shah 2000, Craft 1992, El-Khawas 1998,
Harman 1994, 2000, Kristoffersen and Woodhouse 2005, Martin and Stella 2007, van Damme 2002,
Westerheijden et al 1994, World Bank 1994); including:

 Massification of higher education with increased numbers of students and demographic
changes – leading to a change in the nature of higher education, and a feeling that it is
necessary to check explicitly those institutional QA procedures are keeping pace with the
change. More money is being spent on higher education – leading to an increased desire to
ensure that the money is being well-spent
 Diversification and privatization of higher education – leading to the need for stringent
external checks on them
 Deteriorating infrastructure and lack of resources for non-salary expenditures
 Decline of academic standards, and concerns on the quality of teaching / research activities
 Growing demands from employers and the professions
 Increased pressures for increased public accountability – with increased public funding for
higher education, and increased national needs for educated manpower, governments want to
hold HEIs explicitly accountable for the nature of the graduates
 Changing social contexts with fiscal constraints calling for performance and cost-effectiveness
 In some countries, decreasing micro-management by governments of HEIs in return for the
introduction of an external QA process
 Globalization and internationalization of tertiary education, and trade agreements – leading to
transnational mobility of students, and a need to have national QA process that is visible to and
credible in other countries
 Mobility of professionals requiring greater standardization among qualifications
 GATs and borderless markets for tertiary education
 Changing public management ideologies, i.e. the New Public Management concept
 International market for quality assurance services
 Gradual development of a more competitive education market where quality becomes an asset
and labeling device
 Rise of New Public Management and related concepts, i.e. new managerialism and reinventing
government (Osborne and Gaebler 1992)
 Decline of trust in public institutions and in professionals (Amaral et al 2008)


Cecilia Tsui 2008 Higher Education Development: Quality, Relevance and Impact

Cecilia Tsui 2008 Higher Education Development: Quality, Relevance and Impact

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Cecilia Tsui 2008 Higher Education Development: Quality, Relevance and Impact

Similar to Cecilia Tsui 2008 Higher Education Development: Quality, Relevance and Impact (20)

Recently uploaded

Recently uploaded (20)

Cecilia Tsui 2008 Higher Education Development: Quality, Relevance and Impact