This report, commissioned by HEFCE, analyses all research publications submitted for research assessment between 1988 and 2014.
The analysis covers almost 1 million submitted research outputs and more than 35,000 impact case study references across 25 years and five assessment exercises. The report considers the relationship between submitted outputs (as evidence of academic research quality) and impact case study references (as evidence of societal and economic impact). It explores what the data says about the selection of research outputs for assessment, and about changes in selection behaviour as a response to assessment.
Key findings include:
- Over time, journal articles have become a more commonly submitted output type (when compared with books, conference proceedings and other media). This is despite the fact that HEFCE analysis comparing quality profile by output type found that monographs were awarded the highest proportion of four-star scores.
- In early research assessment cycles, the research outputs submitted tended to be those published towards the end of the publication period. In Science and Engineering recent exercises saw submissions from earlier in the cycle in all relevant disciplines for all institutions.
- Of the impact case study references with digital object identifiers, 42 per cent were submitted to Research Assessment Exercises or the 2014 Research Excellence Framework (REF2014) as research outputs; these are evenly spread between publication years from 1996 to 2013.
- Earliest references are skewed towards recent publications for social science and humanities. In science and engineering, the date of the earliest reference in each impact case study is uniformly spread through the eligible publication period.
- The overlap between case study references and submitted outputs for assessment appears to be higher in more applied areas, and lower in the core humanities and pure sciences.
- For science and engineering, the case study references have highest overlap with the research outputs submitted in early cycles. The overlap between case study references and assessment outputs submitted to REF2014 is higher in social sciences.