Information about the TEF subject pilot
The Department for Education (DfE) published the TEF subject pilot specification on 20 July 2017. HEFCE is implementing this pilot on behalf of the DfE during the 2017-18 academic year.
The pilot will inform the future development of the TEF by testing how ratings could be assigned at subject level as well as institutional level. This has the potential to provide students with more detailed information to inform their choices of where to study.
The first subject pilot will run between autumn 2017 and summer 2018, and will evaluate how two models generate subject-level TEF ratings. It will not interact with TEF Year Three. No subject level ratings will be published.
In mid-December 2017 HEFCE will make available the following resources:
- Subject-level metrics, which will be shared with all providers via the extranet
- Guidance on interpreting subject-level metrics
- The pilot teaching intensity information.
All providers will be able to engage with and draw on this information to inform their responses to DfE’s technical consultation, should they wish to submit a response.
Please note that guidance documents relating to the subject pilot are intended only to support providers who are participating in the pilot, and do not reflect final processes or policy decisions.
Timetable of the pilot
The providers participating in the Year Three subject pilot have now been selected. Due to the strong interest in the subject pilot, it was decided to increase the size of the pilot from 40 to 50 providers. Therefore the make-up of the pilot will be:
- Model A – 19 providers
- Model B – 19 providers
- Both model A and model B – 12 providers
The full list of selected participants in each model can be found below:
|Model A participants||Model B participants||Participants in models A & B|
Bath Spa University
Hertford Regional College
Imperial College London
Liverpool John Moores University
London Metropolitan University
London School of Management Education Limited
University of Bedfordshire
University of Cambridge
University of Essex
University of Hull
University of Nottingham
University of St Andrews
Warwickshire College Group
Abingdon and Witney College
Arts University Bournemouth
Glasgow School of Art
Kaplan Open Learning (Essex) Limited
Leeds Beckett University
Leeds Trinity University
London School of Economics and Political Science
London South Bank University
New College Durham
School of Oriental and African Studies
South and City College Birmingham
Tyne Coast College
University of Exeter
University of Hertfordshire
University of Leicester
University of Portsmouth
University of Westminster
Bishop Grosseteste University
Bromley College of Further and Higher Education
De Montfort University
Nelson College London Limited
Pearson College Limited
St George’s, University of London
University of Birmingham
University of Bradford
University of East Anglia
The University of Law
University of Northumbria at Newcastle
University of Wolverhampton
Key features of the pilot's models
|Model A||Model B|
Metrics will be released for the 35 subjects and are the same for both models. The calculation and presentation of metrics and benchmarks will mirror TEF Year Three.
For single subject providers, subject-level metrics are identical to their provider-level metric. See paragraphs 6.2.6 and 6.3.5 of the pilot specification for full details.
Assessment at subject level is 'by exception'. Subjects are awarded the same rating as the provider, where the initial hypothesis is the same. There is fuller assessment (and potentially different ratings) for subjects where their initial hypothesis differs from the provider.
Assessment is 'bottom-up': each subject is fully assessed to give subject-level ratings. These feed into the provider-level assessment and rating.
Number of providers to participate
19 providers solely participating in model A
19 providers solely participating in model B
12 providers participating in both models
A provider-level assessment is retained in both models (please note below the refinement to submissions where at least 35 per cent of students are part-time)
A provider-level submission of up to 15 pages (covering all 10 TEF criteria)
A provider-level submission of up to 10 pages, covering TEF criteria:
Subject identification and submissions
The 35 CAH2 subjects are used for subject-level metrics, submissions (where they are identified as exceptions), and subject-level ratings.
The 35 CAH2 subjects are used for subject-level metrics and ratings.
Model B maps the 35 subjects into 7 subject groups, which are used for submissions.
Subject submissions can include up to 5 pages per subject (where identified as exceptions).
There are 7 subject group submissions. These will have a variable maximum page length (between 5 and 13 pages) depending on the number of subjects the provider delivers within the subject group.
Selecting pilot participants
HEFCE’s aim in selecting the Year Three pilot participants was to create a sample to effectively test the features of the TEF subject pilot that represents, as far as possible, the overall profile of providers in the higher education sector.
Our final selection was based on whether providers had suitable data, the model(s) in which providers wished to participate and the key characteristics set out in the Department for Education’s TEF: Subject-level pilot specification. Our selection includes a spread of providers in terms of:
- provider size and subject breadth
- operational type (i.e. higher education institutions, further education colleges and alternative providers)
- the type of student being taught
- TEF Year Two ratings (including a spread of providers with gold, silver, bronze and provisional awards, and some providers who did not participate in TEF Year Two)
- the nation in which providers are based.
The sample of selected providers was finalised with advice from the TEF Project Board.
Download the TEF Project Board's terms of reference
As part of the subject pilot, HEFCE will also pilot two teaching intensity measures, exploring how to collect data on issues such as class size and contact time, and their potential use to inform a subject TEF assessment. The teaching intensity pilot will be limited to full-time undergraduates.
The DfE’s subject pilot specification indicated that the subjects provisionally selected for the teaching intensity pilot would be reviewed. Having examined the breadth of provision at the selected providers, and in order to ensure all provider types are included, the following changes have been made: Business and management will be in scope in place of Law; Engineering will be in scope in place of Physics and astronomy. Therefore the subjects in scope for the teaching intensity pilot are:
- Creative arts and design
- History and archaeology
- Business and management.
In order to capture information on the cost of implementing subject-level TEF, HEFCE will provide a costing tool for providers to complete.
The DfE has published a summary of refinements to be made to the provider-level TEF following the lessons learned from TEF Year Two. These refinements will, where appropriate, be carried forward into the pilot at both provider level and subject level.
- The weighting of NSS metrics will be halved
- Very high and very low absolute values will be explicitly marked and taken account of by panellists
- Supplementary metrics derived from Longitudinal Education Outcomes (LEO) data will be included
- Any alterations to the benchmarking factors set out in the forthcoming TEF specification will apply to both provider- and subject-level metrics.
- Where at least 35 per cent of students are part-time, the provider will have the opportunity to submit an additional page of quantitative information in their provider submission. This additional page will not apply to subject submissions
- The grade inflation metric will be implemented in the pilot. However, the approach to generating the metric will be different to TEF Year Three
- The processes involving the Director for Fair Access and Power of referral will not be included in the pilot.
Subject pilot resources and guidance for providers
HEFCE delivered a number of subject pilot briefings for providers during September 2017. The briefing slides and further information for providers can be accessed below.