Information for higher education institutions selected for the HESA funding and monitoring data 2010-11: funding data reconciliation exercise

The comparisons

In December 2010 institutions submitted HESES10 and RAS10, both of which were used to inform the 2011-12 grant allocations. HESES10 was also used to monitor the 2010‑11 teaching funding allocation. In May or June 2011 institutions submitted CFEE10. By November 2011 the 2010-11 HESA student return was submitted directly to HESA. In December 2011 the HESA supplied us with this data.

We have used the 2010-11 HESA student data to re-create HESES10, RAS10 and CFEE10 for each institution. (Institutions would have also been able to use HEFCE’s HESA funding and monitoring data 2010-11 web facility to view these re‑creations during the HESA student return submission process, to improve or correct any incorrect data in the return.) The HESES10, RAS10 and CFEE10 re‑creations are built using algorithms which turn the HESA student data into an aggregate return that is comparable with the original HESES, RAS and CFEE submissions, respectively.

 The 4 re-creations that we produce using the HESA data are:

• HESES10 re‑creation
• HESES10 re‑creation based on cost centre sector norms
• RAS10 re‑creation
• CFEE10 re-creation.

We compare these re‑creations with the original funding returns. If we find, either through reconciliations with HESA or other organisations’ data, or any data audit, that erroneous data have resulted in institutions receiving incorrect funding allocations, we will adjust their funding accordingly (subject to any appeals process that may apply and the availability of our funds).

The way the re-creation tables are made

The procedure behind the re-creations is as follows:

1. We start with the latest version of each institution’s 2010-11 HESA student record.
2. Algorithms produce a number of derived fields for the re-creation.
3. We produce an individualised file that has all of the derived fields and the necessary HESA fields that are listed alongside each record. Each record will either be excluded or included in the re‑creation depending on whether it appears to meet the relevant population definition.
4. Each record that is included, is then assembled into the re-creation tables based on the values in its derived fields. Each record will ultimately appear in the appropriate cells in the re‑created tables based upon how it has been returned to HESA (for example, a student reported on the HESA return as full-time will appear in Table 1a of the HESES re‑creation).
5. The re-creations are produced, and grant information calculated and presented in Excel format together with the original return for comparison.

In the same way that we build the re‑creation tables, an institution, using their individualised file, can reproduce the re-creations. This is an important part of the analysis that a responding institution will undertake. See the guide to working with individualised files for further detail.

Selecting institutions to make a formal response

In order to select institutions to make a formal response, as a risk assessment, we compare all institutions’ re-creations with their original returns and set criteria based on the differences between the two. Any institution that exceeds a criteria threshold is selected to make a formal response.

For HESES, these thresholds are based on the grant adjustment report (see the worksheet named Coversheet in the main workbook in the re-creation showing the comparison, and the other funding worksheets). Further details on how HESES is used to inform the calculation of the grant that an institution receives can be found in the technical guidance which accompanies the March grant tables.

For RAS, these thresholds are based on the QR RDP supervision fund report (see the worksheet named Coversheet in the main workbook in the re-creation showing the comparison). Further details on how RAS is used to inform the calculation of the QR RDP supervision fund are available.

For CFEE, these thresholds are based on the funding adjustment for under recruitment. Further details on how CFEE is used to monitor the co-funded employer engagement funding are available.

Institutions selected for the exercise

When an institution is selected for the exercise they may be required to:

• gather all the sources of information about the exercise results
• identify all causes of discrepancy
• explain all material discrepancies through an action plan
• produce an amended HESA return where appropriate and/or overrides
• ultimately sign off the final re-creation as reasonable *
• calculate funding adjustments. 

* Institutions often need to submit an action plan and amendments more than once when the reproduced re-creation contains further unexplained errors.

Gathering all the sources of information about the exercise results

Institutions responding to the exercise will need to obtain some or all of the following outputs from the extranet. We sent the necessary group keys to institutions by post.

For example, for the HESES10 re‑creation:

• HESR10XXXX.xls. This is an Excel workbook containing the HESES10 re-creation tables
• HESR10XXXX.ind. This is a comma‑separated file containing the derived fields that generate the HESES10 re‑creation
• APHESR10XXXX.xls. This is the HESES10 and HESES10 re‑creation comparison action plan template. 

In each case, XXXX is the institutional identifier.

Institutions responding are also likely to need the following information:

• Original data: access to the original data and records used to compile the HESES, RAS or CFEE return is needed. It is often wise to check the methods used to extract HESA data from the student record system to ensure no cohorts/students have been incorrectly excluded from the data submitted to HESA. 

• Technical documentation: The technical documentation is available in the 2010-11 funding and monitoring data overview section and consists of the following information
  • Algorithms: The algorithms are the logic which we apply to the HESA fields to produce the re‑creation.
  • Troubleshooting: In order to find out the areas where incorrect HESA data or algorithmic problems of fit have led to discrepancies between the re-creation and the original return we expect institutions to use the relevant troubleshooting documentation, which points the reader to the corresponding algorithm(s) for the discrepancy they see between the returns. 

• HESA manual: the 2010-11 HESA student manual is essential to the exercise, particularly the list of fields and their contents. This manual can be accessed through the 'Student stream' section of the HESA web-site. 

• HESES10 or RAS10 publications: for the correct guidance and definitions for completing the returns. 

Identifying all causes of discrepancy

There may be a number of discrepancies between the re-creation and the original return. These discrepancies can be identified by comparing the re-creation with the original return. By following the logic below it should be possible to uncover the underlying causes for each area of discrepancy:

Step 1: For the appropriate cell in the original return, identify the population on the original return from data/records on the student record system (for example if there is a discrepancy with column 1 HEFCE‑funded full-time student numbers, it is useful to identify the students that were included in the cell on HESES).

Step 2: For the appropriate cell in the re-creation, identify the population on the re-creation using the individualised file and the troubleshooting guide (to produce the filter criteria). It is often wise to ensure that cells are in the correct format so that (for example) leading zeros are not lost at this stage.

Step 3: Using information held about the students and all the other information gained from the sources listed in paragraph above, compare the results of step 1 and 2 to identify the cause of discrepancy. 

The cause of discrepancy can be as follows:

1. HESA data error (i.e. the re-creation is incorrect because the data submitted to HESA is incorrect).
2. HESES/RAS/CFEE error (i.e. the original return is incorrect).
3. Problem of fit with the re‑creation algorithms (that is, the way we have assigned a student to a cell is incorrect).

If, after taking the above steps and using the additional guidance provided, the discrepancy is unexplained, HEFCE staff should be able to provide further assistance.

Explaining the discrepancies on an action plan

The action plan is the main source we use to assess if an institution is in a position to fully respond to the exercise. We will be looking for completeness: that all constituent material discrepancies have been explained and that the discrepancies are explained in a way that is understandable and credible. The plan will contain information about the cause, effect and remedial action that will be taken for each item. More information on the action plan is available.

Where an action plan is insufficient in any way, we may visit institutions to gather the information we need.

Producing an amended HESA return and/or overrides

For all areas of discrepancy which are caused by HESA data error, the institution will need to re-submit the HESA return to HESA in order to correct the error(s). More information on submitting HESA data amendments is available.

Where the approximation we have used in the algorithm is not valid we require that an override file which will ‘correct’ the way the algorithm works is submitted. More information on submitting overrides is available.

HEFCE visits

We may decide to visit an institution that is having a difficulty responding to the exercise. The purpose of our visits is for us to gain assurance concerning one or more of the following:

• the reliability of data returns
• the methods used to compile data returns
• the ability to respond in a full and timely manner to this exercise. 

Signing off the final re-creation as reasonable

We will only ask institutions to sign off a re-creation as reasonable after any necessary amendments to HESA data or any overrides have been incorporated and all remaining discrepancies (which will be HESES/RAS/CFEE errors) have been explained. Once we are in this position the re-creation will be signed off using a confirmation form provided by us.

The calculation of funding adjustments

For all institutions selected for the exercise the re-creation, once signed off, will supersede the original funding return. Therefore once all HESA data errors have been corrected, all overrides inserted, and all remaining discrepancies have been explained, we will use the signed off re-creation to re‑calculate the funding position. As a result grant adjustments are likely to occur for the majority of institutions selected. If a holdback of teaching grant is the outcome, the institution will be given an opportunity to appeal, and this appeal will be considered before the grant adjustments are enacted.