Update on the Research Excellence Framework

To:

Heads of HEFCE-funded higher education institutions

Dear Vice-Chancellor or Principal

1.   I am writing to update you on our progress with developing the Research Excellence Framework (REF), and our plans for further work and consultations with the sector.

2.   In May this year, following our earlier consultation, we announced some key changes to the proposals for the REF (HEFCE Circular letter 13/2008). We committed to developing the REF as a single unified framework for assessing and funding research across all disciplines, which will combine expert input and indicators as appropriate to each subject group. We revised the timetable so there would be sufficient time for further development and consultation.

3.   We are now in the process of developing detailed proposals for this more flexible framework. We aim to design a framework that:

• produces robust and rounded indicators of research quality across all disciplines
• provides a basis for distributing funding primarily by reference to research excellence
• recognises and rewards excellent research of all types
• avoids undesirable behavioural incentives
• avoids unnecessary administrative burden
• promotes equality and diversity.

4.   We are working closely with the other UK higher education funding bodies and in consultation with the Department for Innovation, Universities and Skills, and the Research Councils, to ensure that the REF fully complements other parts of the research funding system.

Progress so far

5.   Within the REF, research will be assessed through a combination of indicators and expert review processes, including the reviewing of outputs, the use of bibliometrics and other quantitative indicators, and the use of supplementary qualitative information, as appropriate to each subject group. A key challenge is to design a 'common family' of such indicators and review processes for the REF as a whole, upon which each subject group can draw as appropriate, to inform robust and rounded overall quality profiles that take full account of all types of research.

6.   We have begun to explore, initially through a high-level workshop, how such a framework should take account of research that is valued by users. The workshop identified several key issues that we will address as we develop the 'family' of indicators and assessment processes. These include:

• how to ensure that the criteria for excellence give due weight to the rigour and originality of all types of research, while recognising different audiences in terms of the significance of different types of research
• how to build on efforts made in the 2008 Research Assessment Exercise (RAE) to further encourage institutions to put forward a wide range of work for assessment in the REF
• how to refine the existing business research element of quality-related (QR) research funding, introduced as an initial proxy for user value, and include an appropriate range of evidence to take account of user value and engagement with research users
• how best to seek inputs from a wide range of research users into the assessment process.

7.   We have made good progress with the pilot exercise to test and develop a bibliometric indicator of research quality, so that this new way of measuring research can be a useful addition to the 'family' of indicators. The data collection stage of the pilot is nearing completion and I extend my thanks to all the institutions that have worked at relatively short notice to process data for the pilot. We will now work with Evidence Ltd to analyse the data and test a range of models, and will publish the outcomes of this work in mid-2009. We have also commissioned work to capture the experience of the pilot institutions and disseminate this to the wider sector. Further details of the pilot are provided at Annex A.

8.   As we continue to develop the REF we want to draw lessons from and build on the experience of the 2008 RAE, so that the REF can retain those elements that worked well while making changes where the framework can be improved.

9.   We are establishing a series of expert advisory groups for the REF, drawing largely on chairs and members of RAE panels, as well as nominations from the Research Councils, research users and other key stakeholders. These groups will meet throughout the first half of 2009 and advise us as we develop detailed proposals for the REF. There are over 100 members, providing a broad coverage across the full range of disciplines and types of research and research users. We will seek their advice on all key features of the REF, including the 'family' of indicators and peer review processes to be included in the REF, how to combine them into overall quality profiles and the subject structure for assessment. As part of this we will ask these groups to review the outcomes of the bibliometrics pilot and advise on the interpretation and use of such data in the REF. We will encourage members of the groups to consult with their wider communities to inform their advice. Further details of the advisory groups, including their membership, will be available on our web-site shortly.

10. We are working with partner organisations to review elements of the RAE and gather evidence about the potential impact on the sector of the transition from the RAE to the REF.

1. We have appointed PA Consulting, as part of a wider review of the accountability burden on higher education institutions, to identify the accountability burden associated with the 2008 RAE and assist in identifying opportunities for the REF to reduce burden.
2. The Equality Challenge Unit has recently commissioned the Centre for Diversity Policy Research and Practice at Oxford Brookes University to review the equality and diversity measures taken in the RAE, to help identify institutional good practice and inform our approach for the REF. We will also undertake a quantitative analysis of staff selected for the 2008 RAE.
3. The Research Information Network is commissioning work to investigate the influence of research assessment systems on researchers' publication behaviour.

Next steps

11.   Throughout the first half of 2009 we will continue to develop proposals by analysing the bibliometrics pilot data, gathering evidence through studies, taking advice from the expert advisory groups and holding discussions with a wide range of stakeholders.

12.   We will publish proposals on all the key features of the REF for consultation with the sector in autumn 2009, and expect to announce the outcomes in early 2010.

13.   We remain committed to full implementation of the REF in 2013, to drive QR allocations from 2014. The timetable for the transitional stages leading up to full implementation are yet to be determined in detail; the REF proposals that we publish in autumn 2009 will include a proposed timetable for phasing in the REF leading to full implementation in 2013. At this stage we envisage that during calendar year 2010 we will initiate a sector-wide bibliometrics process in an appropriate range of subjects. We recognise that not all institutions will, at that stage, have systems in place that will have systematically recorded all relevant research outputs. It is therefore likely that this first sector-wide bibliometrics process starting in 2010 will be developmental to some extent, and we will use it to inform a small element of funding only if found to be sufficiently robust.

14.   A range of further information about the development of the REF is available.

15.   We look forward to further discussions with institutions over the coming months and to continuing to work in partnership to develop a system that will command the confidence of the sector.

Yours sincerely

David Sweeney
Director - Research, Innovation and Skills

Share this:

• 
• 
• 
• 
• 
• 
• 

---