Dear Vice-Chancellor or Principal
2016-17 HESA data checking tool
1. This circular letter notifies institutions of the upcoming release of HEFCE’s data checking tool for Higher Education Statistics Agency (HESA) data. This data checking tool is provided to institutions so that they can verify, and where appropriate correct, their 2016-17 HESA student data before signing off this data with HESA. The first outputs for the data checking tool will be released in late August. We will notify contacts at institutions when these outputs, and the tool, are available.
2. There has been a change in terminology for the uses of individualised student data. This includes the renaming of the ‘web facility’ to the ‘data checking tool’ and changes to the categories of outputs. Further information can be found in ‘Change to terminology for uses of individualised student data’ (HEFCE Circular letter 22/2017).
3. Higher education institutions (HEIs) will be able to access the 2016-17 data checking tool from the Information Reporting Interface Service (IRIS) report which is available on HESA’s data collection site. HEIs will no longer be able to access the data checking tool via the HEFCE extranet.
4. HESA and HEFCE strongly encourage use of the data checking tool, which we regard as an essential element of all institutions’ data quality processes. It will help institutions:
- to return accurate data to HESA
- to verify the accuracy of the derived fields that may inform and monitor funding allocations
- to respond to data verification queries (DVQs) raised by HEFCE staff
- to reduce the likelihood of selection for the reconciliation exercise for 2016-17 (selected institutions are typically subject to considerable additional work and potential funding adjustments)
- to identify discrepancies between within-year HEFCE data returns, such as the Higher Education Students Early Statistics survey (HESES), and the outturn position for 2016-17
- to identify errors in HEFCE data returns
- to verify that the 2016-17 HEFCE-fundable full-time equivalent student numbers used in the transparent approach to costing for teaching (TRAC(T)) are suitable to inform the periodic review of the relative costs of different subjects.
5. The data checking tool will not provide outputs that explicitly derive institutional performance metrics, such as those used in the Teaching Excellence Framework (TEF). HESA and HEFCE encourage institutions to pay particular attention to the credibility checks and frequency counts that HESA’s 2016-17 student data collection provides in relation to key data items. These include credibility checks on those data items which act as benchmarking factors, or that define ‘splits’ and populations of the TEF metrics.
6. Data submitted to us and signed off as correct by the head of institution, or finally submitted to HESA, is the final data that we will use; this is also consistent with published data. We will only accept amendments after this point in exceptional circumstances, where any identified data errors are widespread and significant and make a material difference to HEFCE’s use of the data. If an institution identifies errors and wishes to submit amendments, it will be required to complete an amendments action plan. Once an action plan is received, amendments will be required to pass a HEFCE assessment process. In assessing amendments we will consider the multiple purposes for which we use the data, in the categories below:
- regulation and assurance
- quality and institutional performance, including the TEF and the Annual Provider Review
- public information, including Unistats and the National Student Survey
- statistical and policy analysis.
For further information regarding data amendments, please see the ‘HEI technical guidance 2017-18’, which was published as part of the 2017-18 spring grant announcement.
7. We strongly encourage institutions to analyse the data checking tool outputs as part of their HESA student data quality processes. Past reviews have confirmed that this is an essential element of most higher education institutions’ HESA data quality processes. HEFCE staff may query (via HESA’s Minerva Data Quality System) potential errors in HESA data that could affect our future use of the data. In particular, institutions that do not scrutinise the funding-related outputs are more likely to be selected for the reconciliation exercise described in paragraphs 20 to 23, which may have funding consequences.
8. Where discrepancies occur between within-year HEFCE data returns (such as HESES) and the outturn position for 2016-17, we expect institutions to take full account of the outputs from the data checking tool when preparing future data returns. We therefore strongly encourage institutions to analyse the data checking tool outputs as part of their data quality, planning and audit processes.
9. We would be grateful for feedback on the algorithms used to create the data checking tool outputs, in particular where these algorithms have changed from previous years. The algorithms used for each output are available on our website. Feedback should be emailed to firstname.lastname@example.org.
Data checking tool outputs
10. The 2016-17 data checking tool generates the following outputs:
- HESES16 comparison
- student premium allocations data summary
- Quality-related Research (QR) Research Degree Programme (RDP) supervision funds data summary
- HEFCE-fundable full-time equivalences for the TRAC(T) data summary.
These outputs will not all be available initially. We will release outputs in phases and will notify contacts at institutions when new outputs become available.
11. The HESES16 comparison output compares 2016-17 HESA data with the institution’s HESES16 data returned at the start of the academic year. There is also a comparison between the institution’s latest 2017-18 grant and grant calculated using the 2016-17 HESA data.
12. The algorithms used to generate the HESES16 comparison output are intended to be those used for the reconciliation exercise next spring. We may, however, make changes where we believe these will improve the algorithms.
Student premium data summary
13. The student premium data summary applies the 2017-18 student premium funding methods to the 2016-17 HESA data submitted to the data checking tool, to illustrate the effect that changes to HESA data may have on student premium funding. It also compares student characteristics derived from the 2015-16 and 2016-17 HESA data to allow institutions to check the quality of the data we are likely to use to calculate the 2018-19 full-time student premium and disabled students’ premium allocations.
QR RDP supervision funds data summary
14. The QR RDP supervision funds data summary applies the 2017-18 QR RDP supervision funding methods to the 2016-17 HESA data submitted to the data checking tool, to illustrate the effect that changes to HESA data may have on QR RDP supervision funding.
Full-time equivalences for the TRAC(T) data summary
15. The TRAC(T) data summary provides an output on the HEFCE-fundable full-time equivalent student numbers by cost centre and price group using the 2016-17 HESA data. It allows institutions to check the data that will be used in calculating the subject-related full average costs of teaching a HEFCE-fundable student in their TRAC(T) return.
Data verification queries
16. HEFCE staff may raise DVQs (via HESA’s Minerva Data Quality System) during the data collection window where we assess the impact that potential errors in data could have on the areas listed in paragraph 10.
17. Further details of this process are described in the ‘Data verification’ section of our website. We expect all institutions to respond to DVQs before the HESA sign-off deadline.
18. In the funding-related outputs we generally assume the funding rates used in the spring 2017 grant announcement.
19. The data checking tool calculations will incorporate any HESA amendments (for 2015-16 and earlier years) that were approved by HEFCE’s data amendment panel on 9 June 2017 and were signed off by institutions by Friday 28 July 2017. We do not intend to update the data checking tool outputs to reflect any amendments or overrides that may be signed off subsequently.
Relationship with HESA reconciliation exercise
20. We use the annual HESA reconciliation exercise to monitor institutions’ HESES returns using HESA student data. This reconciliation exercise occurs after we have received a final copy of all institutions’ data from HESA, typically in the December following the data checking tool launch.
21. Our funding allocations are informed by the data provided by institutions. If we find, either through reconciliations with HESA or other organisations’ data, or during the course of any data audit, that erroneous data has resulted in institutions receiving incorrect funding allocations, then we may adjust their funding accordingly (subject to any appeals process that may apply and the availability of our funds).
22. Any funding adjustments arising from the reconciliation of a comparison between HESA student data and HESES16 are likely to affect the funding previously announced for 2017-18.
23. Institutions selected to respond to the reconciliation exercise must typically undertake a substantial amount of work, which may take several months to complete. The data checking tool is provided to complement the reconciliation exercise but it does not replace it.
24. The ‘Data reconciliations’ area on the HEFCE website provides further detail regarding the reconciliation exercise.
25. We will only access data processed via the data checking tool to support institutions in returning high-quality data to HESA. We do not intend to use data processed by the data checking tool to make decisions about individual providers, but may use responses to DVQs in our assessments of data quality. Further information on how we handle personal data can be provided on request. See our website for further details.
26. HEFCE staff may examine outputs from the data checking tool in any of the following circumstances:
- where an institution explicitly gives permission
- to produce early modelling of funding allocations
- to verify HESES16 data
- to study general data quality and algorithmic issues as part of the DVQ exercise described in paragraphs 16 and 17.
27. A number of help guides are available in the ‘How we use individualised student data’ area of our website. Some specific areas may be of interest:
- Guidance for using the data checking tool can be found in the ‘How to use the HEFCE data checking tool’ web page.
- Extranet locations, deadlines and documentation can be found under ‘2016-17 uses of individualised student data overview’.
- Technical information (such as algorithms and problems of fit) is listed under ‘2016-17 uses of individualised student data overview’.
- Information on how to obtain data from the HEFCE extranet can be found in the ‘How to access outputs’ guide.
- Answers to frequently asked questions regarding the data checking tool are on the HEFCE website.
28. For any further information or guidance, contact Sarah Cronk (tel 0117 931 7401, email email@example.com).
Director (Analytical Services)