Information for further education colleges selected for the ILR funding and monitoring data 2010-11: funding data reconciliation exercise

The comparisons

In December 2010 colleges submitted HEIFES10, which was used to inform the 2011-12 grant allocations and to monitor the 2010‑11 teaching funding allocation. In May or June 2011 colleges submitted CFEE10. In November 2011 the 2010-11 ILR LR05 student return was submitted directly to the Data Service. In December 2011 the Data Service supplied us with this data.

We have used the 2010-11 ILR LR05 student data to re-create HEIFES10 and CFEE10 for each college. (Colleges would have also been able to use HEFCE’s ILR funding and monitoring data 2010-11 web facility to view these re‑creations during the ILR LR05 student return submission process, to improve or correct any erroneous data in the return.) The HEIFES10 and CFEE10 re‑creations are built using algorithms which turn the ILR LR05 student data into an aggregate return that is comparable with the original HEIFES and CFEE submissions, respectively.

The two re-creations that we produce using the ILR LR05 data are:

• HEIFES10 re‑creation
• CFEE10 re-creation.

We compare these re‑creations with the original funding returns. If we find, either through reconciliations with Data Service or other organisations’ data, or any data audit, that incorrect data have resulted in colleges receiving incorrect funding allocations, we will adjust their funding accordingly (subject to any appeals process that may apply and the availability of our funds).

The way the re-creation tables are made

 The procedure behind the re-creations is as follows:

1. We start with the latest version of each college’s 2010-11 ILR LR05 student record.
2. Algorithms produce a number of derived fields for the re-creation.
3. We produce an individualised file that has all of the derived fields and the necessary ILR LR05 fields that are listed alongside each record. Each record will either be excluded or included in the re‑creation depending on whether it appears to meet the relevant population definition.
4. Each record that is included is then assembled into the re-creation tables based on the values in its derived fields. Each record will ultimately appear in the appropriate cells in the re‑created tables based upon how it has been returned to the Data Service (for example, a student reported on the ILR LR05 return as full-time will appear in Table 1 of the HEIFES re‑creation).
5. The re-creations are produced, and grant information calculated and presented in Excel format together with the original return for comparison.

In the same way that we build the re‑creation tables, a college, using their individualised file, can reproduce the re-creations. This is an important part of the analysis that a responding college will undertake. See the guide to working with individualised files for further detail.

Selecting colleges to make a formal response

In order to select colleges to make a formal response, as a risk assessment, we compare all colleges’ re-creations with their original returns and set criteria based on the differences between the two. Any college that exceeds a criteria threshold is selected to make a formal response.

For HEIFES, these thresholds are based on the grant adjustment report (see the worksheet named Coversheet in the main workbook in the re-creation showing the comparison, and the other funding worksheets). Further details on how HEIFES is used to inform the calculation of the grant that an institution receives can be found in the technical guidance which accompanies the March grant tables.

For CFEE, these thresholds are based on the funding adjustment for under recruitment. Further details on how CFEE is used to monitor the co-funded employer engagement funding can be found in 'Recurrent grants for 2011-12'. 

Colleges selected for the exercise

When a college is selected for the exercise they may be required to:

• gather all the sources of information about the exercise results
• identify all causes of discrepancy
• explain all material discrepancies through an action plan
• produce amendments to the ILR LR05 return where appropriate and/or overrides
• ultimately sign off the final re-creation as reasonable *
• calculate funding adjustments. 

* Colleges often need to submit an action plan and amendments more than once when the reproduced re-creation contains further unexplained errors.

Gathering all the sources of information about the exercise results

Colleges responding to the exercise will need to obtain some or all of the following outputs from the extranet. We sent the necessary group keys to colleges by post.

For example, for the HEIFES10 re‑creation:

• HEIFER10YYYYYY.xls. This is an Excel workbook containing the HEIFES10 re-creation tables
• HEIFER10 YYYYYY.ind. This is a comma‑separated file containing the derived fields that generate the HEIFES10 re‑creation.
• APHEIR10 YYYYYY.xls. This is the HEIFES10 and HEIFES10 re‑creation comparison action plan template.

In each case, YYYYYY is the provider number (L01).

Colleges responding are also likely to need the following information:

• Original data: access to the original data and records used to compile the HEIFES or CFEE return is needed. It is often wise to check the methods used to extract the ILR LR05 data from the student record system to ensure no cohorts/students have been incorrectly excluded from the data submitted to the Data Service. 

• Algorithm documentation and troubleshooting guidance. The algorithms are the logic which we apply to the ILR LR05 fields to produce the re‑creation. Details of the algorithms used to re-create the return from ILR LR05 data using derived fields are available in the 2010-11 funding and monitoring data overview. In order to find out the areas where incorrect ILR LR05 data or algorithmic problems of fit have led to discrepancies between the re-creation and the original return we expect colleges to use the relevant troubleshooting documentation, which points the reader to the corresponding algorithm(s) for the discrepancy they see between the returns. 

• ILR manual: the 2010-11 ILR manual is essential to the exercise, particularly the list of fields and their contents. This manual can be accessed through the 'ILR' section of the Information Authority web-site. 

• HEIFES10 publication: for the correct guidance and definitions for completing the return.

Identifying all causes of discrepancy

There may be a number of discrepancies between the re-creation and the original return. These discrepancies can be identified by comparing the re-creation with the original return. By following the logic below it should be possible to uncover the underlying causes for each area of discrepancy:

Step 1: For the appropriate cell in the original return, identify the population on the original return from data/records on the student record systems (for example, if there is a discrepancy with column 1 HEFCE‑funded full-time student numbers, it is useful to identify the students that were included in the cell on HEIFES).

Step 2: For the appropriate cell in the re-creation, identify the population on the re-creation using the individualised file and the troubleshooting guide (to produce the filter criteria). It is often wise to ensure that cells are in the correct format so that (for example) leading zeros are not lost at this stage.

Step 3: Using information held about the students and all the other information gained from the sources listed in paragraph above, compare the results of step 1 and 2 to identify the cause of discrepancy.

The cause of discrepancy can be as follows: 

1. ILR LR05 data error (the re-creation is incorrect because the data submitted to the Data Service is incorrect).
2. HEIFES/CFEE error ( the original return is incorrect).
3. Learning Aims Reference Application (LARA) error (information on the LARA is inappropriate for the aim).
4. Problem of fit with the re‑creation algorithms (the way we have assigned a student to a cell is incorrect). 

If, after taking the above steps and using the additional guidance provided, the discrepancy is unexplained, HEFCE staff should be able to provide further assistance.

Explaining the discrepancies on an action plan

The action plan is the main source we use to assess if a college is in a position to fully respond to the exercise. We will be looking for completeness: that all constituent material discrepancies have been explained and that the discrepancies are explained in a way that is understandable and credible. The plan will contain information about the cause, effect and remedial action that will be taken for each item. More information on the action plan is available.

Where an action plan is insufficient in any way, we may visit colleges to gather the information we need.

Producing amendments to the ILR return and/or overrides

For all areas of discrepancy which are caused by ILR data error, the college will need to submit amendments to the ILR data in order to correct the error(s). More information on submitting ILR LR05 data amendments  is available.

Where the approximation we have used in the algorithm is not valid we require that an override file which will ‘correct’ the way the algorithm works is submitted. More information on submitting overrides is available.

HEFCE visits

We may decide to visit a college that is having a difficulty responding to the exercise. The purpose of our visits is for us to gain assurance concerning one or more of the following:

• the reliability of data returns
• the methods used to compile data returns
• the ability to respond in a full and timely manner to this exercise.

Signing off the final re-creation as reasonable

We will only ask colleges to sign off a re-creation as reasonable after any necessary amendments to ILR data or any overrides have been incorporated and all remaining discrepancies (which will be HEIFES/CFEE errors) have been explained. Once we are in this position the re-creation will be signed off using a confirmation form provided by us.

The calculation of funding adjustments

For all colleges selected for the exercise the re-creation, once signed off, will supersede the original funding return. Therefore once all ILR data errors have been corrected, all overrides inserted, and all remaining discrepancies have been explained, we will use the signed off re-creation to re‑calculate the funding position. As a result, grant adjustments are likely to occur for the majority of colleges selected. If a holdback of teaching grant is the outcome, the college will be given an opportunity to appeal, and this appeal will be considered before the grant adjustments are enacted.