Workshop by Pooja Takhar (Senior Manager: HEIs, Vitae) and Emma Gillaspy (Vitae NW Hub Manager) at the Vitae event 'Preparing for the Research Excellence Framework: Researcher development, the environment and future impact' on 11 July 2012 in Manchester www.vitae.ac.uk/preparingfortheref
5. Key changes: REF vs. RAE
Inclusion of assessment of impact
Fewer UoAs/panels, operating more consistently
Strengthened equality and diversity measures
Revised eligibility criteria for staff
Addition of (limited) use of citation data in some UOAs
Removal of ‘esteem’ as a distinct element
Revised approach to ‘environment’ and data collection
Publication of overall quality profiles in 1% steps
7. REF impact criteria
The criteria for assessing impacts are reach and significance
In assessing the impact template (REF3a) the panel will consider the
extent to which the unit’s approach described in the template is conducive
to achieving impacts of ‘reach and significance’
Four star: Outstanding impacts in terms of their reach and significance
Three star: Very considerable impacts in terms of their reach and
significance
Two star: Considerable impacts in terms of their reach and significance
One star: Recognised but modest impacts in terms of their reach and
significance
Unclassified: The impact is of little or no reach and significance; or the impact
was not eligible; or the impact was not underpinned by excellent research
produced by the submitted unit
9. Examples
Panel criteria and working methods (
www.ref.ac.uk/pubs/2012-01/)
A social enterprise
Research has enabled initiative has been
Changes to the Production costs
stakeholders to challenge created
design or delivery of have reduced
conventional wisdom
the school curriculum Improved forensic
Enhanced preservation, Improved access to methods or expert
Policy debate or decisions conservation or presentation justice, employment systems
have been influenced or of cultural heritage or education
shaped by research Improved management or
Jobs have been Research has informed conservation of natural
Organisations have
created or protected public understanding, values, resources
adapted to changing
cultural values attitudes or behaviours
Enhanced corporate The policies or activities of Changes to
Levels of waste have
social responsibility NGOs or charities have been legislation or
reduced
policies informed by research regulations
New forms of artistic Changes in Enhanced technical
A new product has
expression or changes to professional practice standards or
been commercialised
creative practice protocols
10. Researcher Development
Framework (RDF)
RDF is an operational framework for planning,
promoting and supporting the personal, professional
and career development of researchers
How was it created?
Core of the framework consists of data drawn from
over 100 interviews
Phenomenographic method* – identified over 1,000
characteristics and their variants
Input from experts, specialists and stakeholders
Clustered into the 4 main areas or Domains
* See http://www.vitae.ac.uk/policy-practice/167281/Background-documents.html for
further details on method.
14. Strengths of the RDF
Designed by researchers for researchers
Use of a common language
National consistency
International competitiveness
Independent resource
Highlights value of developmental opportunities
Supports researchers to take control
15. Using the RDF (researchers)
Personal and professional development
Understand strengths
Identify areas for development
Set goals
Self-assessment and review
Job applications
Next step on the career ladder
Transferable skills
Broad view of available career options
16. RDF Lenses
Focus on key knowledge, behaviour and
attributes of researchers acquired through or
used in various contexts or environments
Lenses derived from the RDF
employability, leadership, enterprise, intrapreneurship
Lenses mapped from other frameworks
public engagement, teaching, information literacy
Next steps
supervisor, impact, knowledge exchange
17.
18. Impact Lens
To highlight the skills required by researchers to conduct
research which has an impact within and outside academia
19. What makes an impactful
researcher?
What:
Knowledge
Skills
Qualities
Behaviours
Etc
would they exhibit?
21. Benefits of developing
impactful researchers
Enhancing research quality and its impact
Increasing awareness of the value of research to UK
society
Skills development
New research perspectives / career enhancement
Higher personal and institutional profile
Forming new collaborations and partnerships
Enjoyment and personal reward
Additional funding
Inspiring the next generational of researchers
23. Links and resources
RDF: www.vitae.ac.uk/rdf
RDS: www.vitae.ac.uk/rds
RDF lenses: www.vitae.ac.uk/rdflenses
Contact: rdf@vitae.ac.uk
REF impact pilot exercise: www.ref.ac.uk/background/pilot/
RCUK Pathways to Impact www.rcuk.ac.uk/kei/impacts
NCCPE REF summary:
www.publicengagement.ac.uk/sites/default/files/NCCPE%20REF%20update%20
Top research departments fail to shine in impact pilot:
www.timeshighereducation.co.uk/story.asp?storycode=414898
Notes de l'éditeur
13:00 Each of us to introduce ourselves and relevance to this workshop
EG Outline workshop objectives
EG What is impact? Well, RCUK defines impact as... Diverse ways that research-related skills benefit individuals, organisations and nations. These include: - global economic performance, and specifically the economic competitiveness of the UK - increasing the effectiveness of public services and policy - enhancing quality of life, health and creative output Above all, research must be of the highest quality: you can't have impact without excellence.
Here is the RCUK Pathways to Impact diagram. Wide range of possible routes to impact. Academic impact is the demonstrable contribution that excellent social and economic research makes to scientific advances, across and within disciplines, including significant advances in understanding, method, theory and application. Economic and societal impact is the demonstrable contribution that excellent social and economic research makes to society and the economy, of benefit to individuals, organisations and nations. The impact of social science research can be categorised as: Instrumental : influencing the development of policy, practice or service provision, shaping legislation, altering behaviour Conceptual : contributing to the understanding of policy issues, reframing debates Capacity building : through technical and personal skill development.
PT 13:10 These are some of the key changes being implemented in this new system of assessment The main new element being introduced is the assessment of impact: inclusion of impact is seen to give encouragement towards achieving the widest benefits beyond the academy, while providing public accountability for investment Impact assessment into the REF will explicitly reward research that has sustainability benefits
PT
PT Impact is broadly defined for the purpose of the REF as an effect on, change or benefit to the economy, society, culture, public policy or services, health, the environment or quality of life, beyond academia the two central criteria for assessing impact REACH: how widely felt it was SIGNIFICANCE: how much difference it made to the beneficiaries how to measure and quantify impact, both in terms of the numbers benefiting and the quantum of benefit?
PT To count in REF2014, an impact needs to have manifested at some point during, or indeed throughout, the given time window (Jan 2008 – Nov 2013) it must relate unambiguously and decisively to one or other aspect of a university department’s, institute’s or centre’s research the research that has generated the impact must have produced a formal research output at some point in the period since 1993 Case studies: A ‘scene-setting’ inventory of all impacts generated by the research group as a whole Must explain the position of the case studies within the group’s overall research activity Designed to prevent the prioritisation of certain subgroups and their portrayal as representative of the group as a whole
PT 13:15 Give out handout featuring examples, evidence and indicators. This information is more for them to take away but get them to have a flick through and think about how they can help support the development of impact in their institution. Ask for a few people to share their ideas in plenary
EG 13:20 What makes a successful researcher? We asked that question in semi-structured interviews with over 100 experienced researchers (mostly profs and PIs). The RDF was the result of that research. The RDF has been designed to help you plan, promote and support your personal, professional and career development. It was developed by you for you. The project began with an initiative at the 2008 Roberts Policy Forum. Vitae pulled together a working group in March 2009 and the project got underway. How was the RDF created? Empirical data from analysis of audio-recorded,. Everyone was asked to identify what they thought was important for a research career. Representative sample: range of experiences, institution types, geographical context, disciplines and demographics. Results: > 1000 characteristics and variants, clustered into common groups’ This provided the core of the framework – then consulted all the stakeholders who have a view on researchers Cross-referred results with other competency type frameworks, expert and specialist input – Research councils, careers, RIN, UCU Built a larger picture of what is means to be a researcher. What does the end result look like?
EG Highlight RDF A5 leaflet 4 domains which form the strategic RDS 12 sub-domains 63 descriptors Up to 5 phases of development for each descriptor
EG Point out 5 phases of development
EG Only 3 phases for development here
EG Designed primarily as a flexible framework for researchers but useful for supervisors, researcher developments, policy makers, employers etc too
EG
PT 13:25
PT The leadership lens on the Vitae Researcher Development Framework has been developed to focus on the knowledge, behaviours and attributes that one develops as a researcher and how these can be acquired through or used in leadership opportunities
PT Working together with our networks and other stakeholders, we want to develop a lens that will encourage researchers to be actively involved in thinking about how they can perform research with impact and to explore the knowledge, behaviour and attributes required for realising the impact RDF enables assessment of strengths and areas for further development - impact lens can put a spotlight to show how to improve your skills to perform research which has a wider impact Definition An effect on, change or benefit to: Economy Society Culture Public policy or services Health Environment Quality of life
EG 13:30 Map out on posters what skills they think are needed to develop impactful researchers (5min) Get them to prioritise their top 5 (5min). Pull together common themes and reach consensus if possible (5-10min)
PT 13:50 What should we call the lens? Possibly a new name: Is "impact" is too generic? Misleading? has too many perceptions around it?
PT
EG 13:55 Prep a flipchart with some questions How are we going to develop these skills that are vital in impact? What training, development, strategy, policy etc do we need? Is some of this in place already? What else do we need to do – researchers, institutions, national/Vitae