How to submit historical data error (HDE) files

Process for submitting HDE corrections

Action plan

Institutions that wish to submit an HDE file are required to submit an action plan before we will accept the files.

Authorisation

On approval of the action plan we will, where appropriate, e-mail the institution to confirm that we are content for it to submit the HDE file. The e-mail will contain:

• a summary of the nature of the corrections to be made extracted directly from the approved action plan (for example, the derived fields that we expect to be amended along with an approximation of the number of records that will be affected)
• a date by which we expect the HDE file to be submitted.

Format of the HDE file

HDE files must be given a name in the form hdeXXXXn.amd, where:

For example, the first HDE file submitted by institution 9999 would be called hde99991.amd, and the second would be called hde99992.amd.

The HDE header

The HDE must contain a header in the following form:

• line 1 – HDE file reference in the form described above, with the '.amd' file extension removed.
• line 2 – creation date of the HDE file in the form ddmmyyyy.
• line 3 – a brief description of the change.
• line 4 – this line must contain the following: HDE.
• line 5 – the HESA fields used to identify records on the HDE file, comma separated. For example, COURSEID could be used to identify records at the course level or HUSID and NUMHUS could be used to identify records at the instance level.
• line 6 – the primary derived fields being changed, comma separated.
• line 7 – number of records in the file, excluding the first 12 lines of header information.
• line 8 – the field used to calculate the file check sum.
• line 9 – the value of the check sum.
• lines 10 and 11 – these lines can contain any notes the institution wishes to include.
• line 12 – fields included in the HDE file. These fields must appear in the same order as each row of the data and must be comma separated. For example: COURSEID, RDPEXCL.
• line 13 – correction data must begin on this line.

Check sum

To ensure HDE files have been received in their entirety, or have not been otherwise corrupted during transmission, we use a check sum. The check sum is calculated by summing the values of the field specified on line 8 over all records in the file. The calculated value should be returned on line 9 of the file. The field used to compute the check sum must be numeric, and must not contain any values greater than 20,000. If no suitable field is available, then a sequential field called RECNO may be created. For example RECNO may contain 1, 2, 3, 4, 5 and so on.

Identifying records

To enable us to link HDE files to our derived HESA dataset, we must be able to identify the records on the HESA return where the correction should be applied. The field, or combination of fields, enabling us to achieve this must be listed, comma separated, on line 5 of the HDE file. We do not prescribe which fields should be used for this purpose, only that they enable us to identify the records on the HESA return.

Saving files

Saving HDE files in Microsoft Excel usually results in the loss of leading zeros and the corruption of very large values into exponential form (for example, 9.91E+12). To avoid this, we recommend that HDE files are produced, viewed and saved using a text editor, for example Notepad.

Submitting the HDE file

HDE files must be uploaded as a comma-separated file via the HEFCE extranet to the ‘FAMD General upload area (GUA)’ area. The group key is as follows:

P699 8C22EF79

Once logged in, navigate to the 'HEFCE resources' page, then under ‘Applications’click on the ‘FAMD General upload area (GUA)’ link. This will take you to the upload page where HDE files may be submitted.

HEFCE processing

When we receive a valid HDE file in the structure and format detailed above, we will produce updated funding and monitoring data outputs which will be made available to the institution via the HEFCE extranet. If we are content with the outputs, where appropriate, we will ask the institution to confirm that the amendments have had the impact expected.

To allow institutions to check that an HDE file has had the desired effect, a field called 'HDE', is included in the individualised file. This takes the value 1 if a historical data error correction has been applied to the record; otherwise its value is 0. This field also highlights where an institution may have records with inconsistent secondary derived fields, as the HDE file will only alter primary derived fields.

Further corrections

If, during the above process, we or the institution identify that further data amendments are required, we will ask the institution to submit a revised action plan and the above process will be repeated.

We will identify and contact institutions that have not submitted valid files by the expected date or where there are other significant delays.

Page last updated 17 May 2012