HEFCE Consultation 00/18

Consultation on performance indicators

Respond by 31 May 2000

April 2000

Note: Annexes to this document are available separately and also in Word (4.2Mb).

Executive summary

Purpose

1. Last year, we published a set of performance indicators for higher education (HEFCE publication 99/66). This year we will publish the second set of indicators in this series.
2. The consultation exercise carried out before publication last year allowed us to take on board some of the concerns institutions had about the proposed indicators and their presentation. The consultation had to take place before institutions could see the complete set of indicators, so one purpose of the present exercise is to allow institutions to make further comments now they have had the opportunity to see the indicators as a whole.
3. The second purpose is to allow each institution to check its own indicators. We have supplied information with this document which will allow institutions to look at the data for their students, derived from the HESA records.
4. Thirdly, we pose a number of specific questions relating to the performance indicators and the data used in their construction, to which we would like answers.
5. Finally, a number of institutions have asked for further information about the indicators, or for further work to be carried out into the application of the indicators. This exercise allows institutions to comment on these aspects.

   Action required

6. For any further information about technical aspects of these indicators, or the data files and documentation that have been provided, please contact either Judy Akinbolu or Ali Emrouznejad at HEFCE, David Bailey at HEFCW, or Reg DeMellow at SHEFC.
7. Responses to the consultation should be sent to the relevant funding council by 31 May 2000.

   Background

8. In 1999, we published a set of performance indicators relating to higher education institutions (HEFCE 99/66, ‘Performance indicators in higher education’). We intend to make this an annual publication, and to extend the areas covered.
9. In future years, for HEFCE-funded institutions we intend to include the draft performance indicators with other derived statistics, so that all data checking can be done together.

   Information included

10. In paragraphs 26 to 43 there are a number of issues on which we would welcome comments, and questions that we would like institutions to answer. These cover the figures already published, as well as those to be produced this year.
11. The tables containing each institution’s data are provided as an attachment to this document. They are now in the format which was used for publication last year. The attachment does not include all the introductions to tables, but they will be similar to those in the publication. Where a new table is proposed, we have included a draft of its introduction.
12. This year, disks are included with this document which contain data on each institution’s individual students, and various other files. The student data files contain the codes we have used for different indicators, and details of the transition matrix used for projecting the outcomes for each institution. Institutions may wish to use the files provided to check their own results, to see how we have made certain calculations, or to check to which categories we have allocated their students in producing the indicators. For full details of the data disks see Annex A.
13. Annex B includes definitions used in this document, where these have changed from last year. Other definitions, as well as details of how to carry out the computations for the adjusted sector benchmarks and for the projected outcomes and efficiency, can be found in HEFCE 99/66.
14. Annex C includes graphs showing the spread of the institutional indicators across the sector. These provide information about the performance of institutions as a whole.
15. Although we have repeated some of the information from HEFCE 99/66, it is assumed that readers have a copy of it available.

    Data from UCAS

16. Last year, we explained that we needed to use data from UCAS because the HESA record did not hold details of Social Class. The new HESA record now includes this information, but our analysts have found that the data on this field, as well as on some other fields, are not of sufficiently high quality. We have therefore used the UCAS database again this year to enhance the quality of data used in calculating performance indicators. We have only used the UCAS database where the information on the HESA database is missing, or has a code that shows it is unknown.
17. The funding councils are sponsoring a study on ways of improving the process of incorporating data from UCAS into HESA records. The study is expected to report in the summer. It is hoped that the results of this study, when implemented, will mean we no longer need to use the UCAS database.

    Data amendments

18. Quality assurance procedures for HESA data currently take place at the time of submission to HESA, and culminate in HEIs signing off their data to HESA. This sign-off process means that the institutions agree that their data are accurate. However, last year a number of institutions found that some of their data needed amending if they were to be used for PIs. Since this may happen again this year, we will exceptionally allow institutions to update their individual student data for incorporation into the PI calculations. For details of how this can be done see Annex A.

    HEFCE-funded institutions

19. From next year, we expect to include these checks with others concerning HESA-derived statistics. Therefore, it should not be assumed that there will then be any further opportunity to amend data submitted to HESA.
20. Because of the timing of the exercise this year, any amendments already sent to us following circulation of HEFCE 00/06 (‘HESA 1998-99 derived statistics for funding allocations and monitoring’) will not have been included in the draft indicators presented here. These amendments will be taken into account before the indicators are published.

    All institutions

21. In future years, it is hoped that the checks on data for the performance indicators will take place slightly earlier in the year. The need for data amendments should diminish as institutions see what is required.

    Follow-up procedures

22. Following the consultation last year, we placed information on the HEFCE web-site to keep institutions informed. This year, we hope there will be fewer problems, and that we can work to a shorter timescale. We will again keep institutions informed through the HEFCE web-site at www.hefce.ac.uk, under ‘Good practice’.
23. Also available at this site are last year’s indicators, certain extra statistics and further explanatory notes, and the full publication.

    Timetable

24. Please send responses to this document to the HEFCE, HEFCW or SHEFC no later than 31 May 2000. Earlier responses would be appreciated. Note that paragraphs 26 to 35 list questions to which we would like responses from all institutions, while questions in paragraphs 36 to 43 are only relevant to institutions that make a modular return for part-time students to HESA.
25. This year we intend to publish the indicators in early summer.

    Issues for consultation

    General comments

26. In producing HEFCE 99/66, we tried to take comments from last year’s consultation exercise into account. These comments ranged from presentational aspects to points about the content. It is only since the document has been published that institutions have been able to see the indicators in context, and how they reflect aspects of performance across the sector. We welcome any further general comment.

    Ongoing developments

27. We aim to produce a number of reports over the next 12 months on topics connected with performance indicators. Topics include:
    • monitoring participation rates by type of geodemographic area
    • factors important in determining progression rates.

    We welcome comments on the usefulness of such reports.

28. We have also been investigating what further information relating to PIs can be made available, and what improvements can be made to existing indicators. We have included the following in this consultation:
    • which institutions are chosen by students who transfer. (This information will not be for publication, it will be available to individual institutions only)
    • benchmarks for the projected outcomes and the ‘All students’ non-continuation rate which take age into account.

    Is this extra information helpful?

29. We are still working on a number of other areas. We hope to be able to introduce the following improvement shortly:
    • a facility to benchmark against user-defined groups (such as neighbouring institutions, or institutions deemed similar).

    Would this be useful? What could be added?

30. A number of people have commented that there are too many access indicators, and that these could be reduced in number. (See paragraphs 52 and 53.) Institutions are asked to comment.

    Existing indicators

31. The tables attached show, for each institution, the figures for this year. Institutions are asked to check them, to compare them with last year’s published figures, and to comment on any major changes that have taken place since then. (But see paragraph 51.)
32. Last year, institutions asked for an indication of whether the difference between an indicator and its benchmark was significant. We therefore provided standard deviations, and marked large differences with an asterisk. Is this helpful?
33. We produced indicators last year for a number of different student groupings. Please comment on the groupings used, in particular:
    • the omission of separate sections in Tables T1 and T2 for ‘other undergraduates’ (see paragraph 58)
    • the inclusion of young entrants by neighbourhood type in Table T3 (see paragraph 59)
    • the inclusion of mature entrants by previous HE qualification in Table T3 (see paragraph 59).
34. The HESA record this year uses new codes to identify students with A-levels, Scottish Highers, or GNVQ level 3 qualifications (see paragraph 49). Has the institution ensured that we can distinguish between, for example, students with a mixture of Scottish Highers, and those with a mixture of GNVQs, using these codes? If not, will it be able to do so in future returns?

    New indicators

    Fee split

35. We have explained at paragraph 101 why we are not including a new access indicator this year detailing the liability of students to pay tuition fees. Would this indicator be useful if the data quality could be improved? Please comment on the way the field used, MSTUFEE, is coded.

    Module completion rates – Table T6

    (This section is only relevant to institutions that make modular returns to HESA.)

36. Overall, do the results in this table appear realistic for the institution?
37. If the institution does not provide information on module levels, please indicate why not. Does the institution intend to provide this information in future?
38. Does the institution’s average FTE per student accurately reflect the size of module? (This is particularly important if the figure is either very small, say less than 5 per cent, or very large, say over 75 per cent.) If it is not, please give an indication of what it should be, and why it appears to be inaccurate.
39. We have counted as modules with no results those where the result is given a code of 8 (not tested) and 9 (not known) as well as those where the result has been left blank. Does the institution use these codes to distinguish between different reasons for leaving, for example? Please indicate how many student modules with these codes are due to students having left the course before the end; how many are because the student opted not to be assessed; how many modules cover two academic years, and so do not have results available; and any other important reasons for using these codes. (See paragraph 95.)
40. We have shown results separately for modules with a zero FTE and those with a non-zero FTE. (See paragraphs 82 to 85.) Is our assumption that zero FTE modules contain results from students returned last year correct? Is this division useful?
41. Given the results for this table, and the proposed introduction to it (given at Annex D), is the institution willing to be included in the table this year? (Note that HEFCW expects all the institutions which it funds to be included.)
42. Even if the institution considers its data to be unfit for publication this year, does it expect its data quality in this area to improve in the next year or two, to a point where it would accept publication? What help would be needed in order to reach this stage?
43. Any general comments about this table would be welcomed.

    Notes on existing indicators

44. This section summarises information already available, both from last year’s consultation exercise and HEFCE 99/66, as well as providing details of changes that we have made, and information about this year’s indicators. Paragraphs covering information available elsewhere fall under the heading ‘Summary information’. Other paragraphs may contain new information.

    Changes to the indicators

45. All indicators that were produced last year are included this year in the same format. We have had to make a few changes to the codes used this year because of changes to the HESA record for 1998-99, but these have been kept to a minimum.

    Mode and level of study

46. The definitions of mode and level of study are based, for the 1998-99 data, on the coding frames used in the new HESA record. As far as possible these mirror the definitions used for the 1997-98 data. For the actual definitions see Annex B and HEFCE 99/66.
47. A few institutions appear to have coded some students as MODE=02 (other full-time) or MODE=25 (other sandwich course), when the students satisfy all the criteria for full-time or sandwich modes according to the funding council conditions. For publication, and in future years, we intend to classify students with MODE=02 or MODE=25 as part-time students. However, for this consultation we have included those with either of these codes, and an expected course length of at least 24 weeks, as full-time.
48. A number of institutions in England and Northern Ireland have already responded to HEFCE 00/06 about this coding. These responses will be taken into account before publication.

    Entry qualifications

49. The codes for the ‘highest qualification on entry’ field, QUALENT2, have changed this year so that this field on its own does not distinguish between students with A-levels, students with Scottish Highers, and students with GNVQ at level 3. Three new fields have been provided to make this distinction. To calculate the adjusted sector benchmark last year, we used a separate category for students recorded as entering with level 3 GNVQs, together with 14 categories for students entering with A-levels or Scottish Highers, by number of points achieved. We have tried to re-create this breakdown for the 1998-99 data, but the new fields have not always provided the necessary distinction. How has the institution coded these fields?

    School type

50. The definition of school type on the UCAS database has changed this year. In particular, some specialist colleges which were defined as ‘independent other’ last year are now defined with respect to their specialisation.
51. As last year, we have taken as ‘state school or college’ all those which are not coded as independent. However, because of the change noted above, the proportion of students from state schools appears to have increased. This is apparent from the individual figures, as well as from Chart 1 in Annex C.

    Access indicators

52. Last year, we produced three indicators of access relating to young entrants to HE, for a number of reasons. In an area like access, where the measurement of socio-economic characteristics is involved, it is not easy to agree on one definition. It can be useful to have several indicators to cover different aspects, and to allow a certain amount of cross-checking between them. In addition, institutions may measure the success of their access policies in different ways, and these indicators allow for such variety.
53. If we were to provide just one indicator, it would have to be the proportion of students from low participation neighbourhoods, as this has the most comprehensive coverage. We are looking at other measures, as explained in paragraphs 99 to 102, and in addition we are considering whether the existing measures can be combined in some way.

    Adjusted sector benchmarks

54. As last year, we have included adjusted sector benchmarks in most of the tables. These provide comparators for the indicators that are based on sector figures, but take into account the profile of each institution in terms of the entry qualifications of its students, and the subjects they are studying.
55. At consultation last year, a number of institutions said it would be helpful to know if the differences between the indicators and their benchmarks were significant. In the publication we therefore included standard deviations, and marked with an asterisk those considered to be significant. We used the same method this year to produce standard deviations, which again are included with each institution’s figures.
56. Last year, we did not take age into account in producing the adjusted sector benchmarks. This year, we have included age in the benchmarks for table T5 (projected outcomes) and also for the benchmark in Table T3 (non-continuation) for all students.

    Small numbers

57. Last year we had to decide how to treat small numbers. We supplied the actual numbers in all cases, as well as the percentages. We omitted some groupings from some tables where there would have been large numbers of percentages based on very small figures. In addition, for institutions where the number of students for whom there were known data was very small, we omitted values completely. We intend to do this again this year, but we have included these values based on small numbers in the attached tables.
58. For the access indicators in Tables T1 and T2, we did not provide separate figures for ‘other undergraduates’. The tables covered full-time first degree entrants (Table T1a for young entrants, T2a for mature entrants); full-time, all undergraduate entrants (Table T1b for young, T2a for mature); and part-time, all undergraduate entrants (Table T2b). ‘All undergraduates’ included entrants to first degree courses, plus entrants to other undergraduate courses such as HND and DipHE. At many institutions, the number of ‘other undergraduate entrants’ (that is, excluding entrants to first degree courses) was too small to provide robust percentages, which is why this was not included as a separate category. Indeed, about half of all institutions had fewer than 50 students in at least one of the young/mature ‘other undergraduate’ entrants categories.
59. The indicators in Table T3 were provided for young entrants and mature entrants separately. Young entrants were split between those from low participation neighbourhoods and those from other types of neighbourhood (in Table T3b); mature entrants were split between those with a previous HE qualification and those with no previous HE qualification (in Table T3c). Some of the numbers in Tables T3b and T3c were small, but the indicators produced were felt to be informative. We did not include adjusted sector benchmarks in Tables T3b and T3c because of these small numbers.
60. The numbers of students in Table T4 were small for many institutions, so the percentages obtained were not used as performance indicators. However, again it was felt that the information was useful. Because this table is provided only as context, we intend to include it in an annex this year rather than as one of the main tables.

    Summary information

    (This information is also available elsewhere.)

    Access indicators – Tables T1 and T2

61. The access indicators are provided for young and mature entrants separately, and for full-time and part-time entrants separately. For young entrants to full-time undergraduate courses, the indicators are:
    • percentage from state schools or colleges
    • percentage from Social Classes IIIm to V
    • percentage from low participation neighbourhoods.
62. For mature entrants to full-time undergraduate courses, and all part-time entrants, only the percentage from low participation neighbourhoods is used. For these groups, we have excluded students who already have an HE qualification, so that the access indicator is the proportion of students who have no previous HE qualification, and come from low participation neighbourhoods.

    Progression indicators – Tables T3 and T4

63. Tables T3 and T4 provide information about the progression of full-time first degree entrants beyond their first year at the institution. The indicator in Table T3 was obtained by tracking students who entered a full-time degree course at an institution in 1997-98 through to 1998-99. The percentage of entrants who were not found at any HEI in 1998-99, and had not qualified with a degree in 1997-98, has been taken as the performance indicator. In addition, we show the percentage who are still at the same HEI in 1998-99 or obtained a degree in 1997-98, and the percentage who have been found at another HEI, in other words, who have transferred.
64. The indicator is provided separately for young and for mature entrants, as well as for all entrants. The total entrants may not be the sum of young and mature entrants, as there are a few students for whom birth dates are not available. The indicator is also provided for young entrants from low participation neighbourhoods, and for mature entrants with no previous HE qualification.
65. Table T4 provides a context for Table T3. Students who leave an institution in one year may return to it, or to another institution, a year later. The scale to which this occurs is shown in Table T4. Here, students who entered a full-time first degree course in 1996-97, and were not found in any HEI in 1997-98, were tracked through to 1998-99. The percentages of these students who were still not at any HEI, were back at their original institution, or were at some other institution, are all shown in this table. (Students who obtained a sub-degree qualification in 1996-97 were included with those returning to the HEI or qualifying.)
66. Figures are provided separately in Table T4 for young and mature entrants. They should tie in with the figures published last year in Table T3 of students not continuing in HE.

    Learning outcomes and efficiency – Table T5

67. This table provides the projected outcomes of a cohort of students starting on first degree courses at each institution in 1997-98. The projection is based on the progression patterns of all first degree students at the institution between 1997-98 and 1998-99.
68. It is not possible, using the current method, to split this table between young and mature starters. Last year, we therefore provided the proportion of starters who were mature as a context statistic. This year, we have been able to include age as a factor in calculating the adjusted sector benchmark, so have omitted the proportion of mature students. Would it still be helpful to include this information?

    Research indicators – Table R1

69. As last year, we have provided four measures of research output per unit input. The inputs used are academic staff costs and funding for research from the funding councils – quality-related (QR) funding only. The outputs are PhDs awarded, taken from the HESA student records, and income from research grants or contracts, taken from the finance record. All four indicators are standardised by cost centre. Values close to 1 show that the output per input is close to the sector average.

    Values of the indicators in 1998-99

70. Graphs showing the spread of these indicators are given at Annex C. The graphs are similar to those in HEFCE 99/66, and will allow institutions to see where they stand in relation to the other HEIs in the sector, although without making any allowance for entry qualifications or subject of study. By comparing these graphs with those of last year’s indicators, it is possible to see how the indicators have changed across the sector.
71. For most access indicators, the spread of values is similar to the spread last year, and most institutions have not seen any substantial change. The changes occur mainly for institutions that only take in relatively small numbers of entrants.
72. The main change to the access indicators has already been mentioned. The proportion of students from state schools or colleges is apparently greater than last year, but this is wholly due to the change in the definition used. The UK figure has increased from 82 per cent in 1997 to 85 per cent in 1998.
73. As last year, there is a marked difference between rates of non-continuation beyond the first year for young entrants and those for mature entrants. The UK figures are similar to those for last year.
74. The projected outcomes and efficiencies for most institutions also show a similar pattern to last year. Many changes are probably due to poor quality data, either last year or this year. Please check the data provided, and contact us if there are problems.
75. The research indicators show similar patterns to those of last year. Most institutions will find that their indicators have not changed significantly.

    Notes on the proposed new indicators

    Module completion rates – Table T6

    Background

76. The progression and completion rates published last year were only calculated for full-time first degree students. The assumptions made to obtain these figures cannot be used with part-time students, so we are looking at other ways of measuring progression for such students.
77. We have looked at completion rates for parts of courses, specifically where a course is returned as modules for which results can be obtained. We can only obtain such a measure where institutions make a modular return to HESA. The indicator is defined as the proportion of modules passed among those modules for which results are known.
78. Last year we did not publish these rates because of the problems raised by institutions. However, this table would give important information relating to part-time students so we are proposing to include it this year, with certain safeguards.
79. Where an institution has stated its intention to provide flexible part-time learning, we feel this information will be of legitimate public interest. Those concerned with ensuring the achievement of that aim, inside and outside the institution, as well as potential and existing students, will be able to see how part-time students progress. This is likely to be the only source of such information.
80. A number of institutions last year felt that they would be penalised if this table was published, because not all institutions make modular returns. This is not the intention of publication. There is no obligation on institutions (except those funded by HEFCW) to provide data in this format. We will make clear that institutions have agreed to be included in the interests of providing better information in this area. The draft introduction to the table, which is included at Annex D, covers this point.
81. Institutions funded by the Higher Education Funding Council for Wales already have to make modular returns, and the HEFCW has indicated that all such institutions should be included in this table. It is likely that shortly, although not this year, institutions funded by the Scottish Funding Councils will be expected to make modular returns, and will then be included in this table. At present, institutions in England and Northern Ireland can choose whether to make modular returns. We propose that, this year, institutions in England, Scotland, and Northern Ireland whose returns are in the required format can choose to be excluded from this table. We strongly encourage all such institutions to be included.

    Problems with returning module results

82. One concern was the difficulty in returning results for modules in certain circumstances. In particular, for modules that straddle the academic year where an institution has elected to use what has been called the ‘100:0’ method of returning student load, there was no way of returning module results. HESA has issued further guidance (Student Circular 99/01, July 1999) providing the opportunity to return such results using a module with an FTE of zero.
83. This seems to have been only a partial solution. For example, some institutions appear to have returned such late module results using modules with non-zero FTE. Also, there can be difficulties where a module is completed, but the results are not known, at the time of data submission to HESA. We are working with HESA to address these difficulties.
84. For each institution, we need to assess the scale of the difficulties in order to decide whether the resulting indicators would be misleading. Please tell us the number of modules so affected, and the policy adopted for returning module results.
85. In the meantime, we have provided the pass rate in the table in two parts, one covering modules with non-zero FTEs, and one for modules whose FTE is zero. Please comment on this.

    Criteria for returning a module result as passed

86. A question was raised last year about the treatment of module results. It was suggested that, where students do not have to pass all modules, for example if it is sufficient to pass six out of eight modules to progress, then some institutions are returning all eight modules as passed, whatever the actual marks imply. We will ask HESA to issue specific guidance in future, making it clear that module results returned should relate to the specific module, not to overall student progress. Meanwhile, we ask institutions who have adopted a policy of returning all individual modules in the way described above to let us know.

    Data problems

87. There are a number of problems with these data, even for institutions that provide modular returns to HESA. There are two main difficulties. First, some institutions use modular returns for only some students, and therefore the numbers provided do not give a picture of the whole student body. Second, parts of the modular return are not completed fully by all institutions, so that even where all students are covered, only partial information for each student may be available.
88. Of the 66 institutions in the UK that provided modular returns for part-time students in 1998-99, 42 made such returns for all part-time students, and 24 made such returns for only some of them. We propose to include an indication of how many part-time students there are in total at an institution plus how many we have modular information about, so that the results can be put into context.
89. As explained above, we have tried to include ‘late results’ from the previous year by separating out modules whose FTE is zero from the others. Unfortunately, there is no way at present in which we can pick up module results for students whose own FTE is zero. We are having talks with HESA to resolve this problem.
90. As the information on module level is not provided by some institutions, we have decided to leave it out this year. If the institution does not provide this information, please indicate why not.

    Module size

91. We have included two columns in the table which give the average number of modules per student at each institution, and the average size of modules as a percentage FTE. For some institutions, these values appear to contradict the fact that only part-time students are included in this table. For example, some institutions have an average FTE per module of 100 per cent, while others have an FTE per module of 50 per cent and an average of three modules per student. Institutions should check their returns if these values appear inconsistent.
92. It was suggested last year that module size might affect the pass rate, with larger modules having higher pass rates, although we have no evidence that this so. Would it be helpful to split up the pass rates by module size?
93. It has been suggested that we should include the number of credit points per module, and per student. However, the range of values obtained suggests that there are substantial differences in the way credit points are allocated to modules, which would make these figures of no practical use.

    Pass rates

94. Where module results have been provided, three codes imply a result and two codes imply no result. The three codes implying a result are:
    • 1 - pass
    • 2 - fail
    • 3 - referred.

    The number of modules with one of these codes is given as the number of modules with results, and those with code 1 are taken as passed. The pass rate is therefore the number coded 1 divided by the number coded 1 or 2 or 3.

95. At present, we do not know in detail why modules are coded 8 (not tested) or 9 (not known) or left blank. Some students so coded may have left before the end of the course, or may not be interested in being assessed. Others may be on modules that cross HESA reporting periods, or for which results are not available for other reasons. Please explain why some modules from the institution do not include valid results. (We have included details with this document – which would not be for publication – showing how many and what proportion of modules are coded 8 and 9 at each institution, and what proportion are left blank. This should give institutions some idea of how relevant this question is to them.)

    Adjusted sector benchmark

96. We have looked at the feasibility of including adjusted sector benchmarks with this table, and have decided not to do so at present. The benchmarks for existing indicators take account of differences in progression rates between students taking different subjects, and those with different entry qualifications, and provide comparators which take these factors out of the equation. However, as entry qualification data for part-time students are not as good as for full-time students, and as many part-time modular students are registered on ‘combined studies’ courses, the benchmarking procedures do not work satisfactorily in this case.

    Overall picture

97. Although the proportion of modules with results varies greatly between institutions, the actual pass rate is not as variable. (See Chart 7, Annex C.) About half of all modules have results provided, and 85 per cent of these are recorded as passed. Only five institutions have pass rates below 70 per cent, out of the 57 for whom some results are available.
98. The average number of modules per student varies between one and five, while the module size ranges from less than 1 per cent FTE to 100 per cent FTE.

    Tuition fee payment

99. At present, there is only one access indicator for mature students. Last year a number of institutions asked why we did not include social class for mature students, as well as for young. The main reasons are:
    • defining and interpreting Social Class for mature students is not straightforward.
    • information on Social Class is not as complete for mature students as for young students – only about half of mature students in the UK have information available about their social class.
100. Therefore this year we decided to look at the proportion of students who get some help with paying tuition fees at each institution. We looked at the HESA field MSTUFEE, and allocated students to one of three groups on the basis of this field, as follows:
     • students with MSTUFEE = 01 pay the full tuition fee
     • students with MSTUFEE = 02, 03, or 04 have some help from the local authority or equivalent with paying the tuition fee, the amount of help being based on a means test of income
     • students with other codes for MSTUFEE have not been assessed for their fee liability.

     The indicator was defined as the percentage of students assessed for fee liability who do not pay the whole fee, that is, group 2 as a percentage of groups 1 and 2.

101. The indicators produced in this way did not appear sensible. There were major inconsistencies between institutions, suggesting that institutions use different criteria in coding their students. Therefore this indicator has been dropped for the present.
102. Institutions in England and Northern Ireland have already been asked about the field MSTUFEE in HEFCE publication 00/06, ‘HESA 1998-99 derived statistics for funding allocations and monitoring’ (paragraph 51 of that document). Institutions in Wales and Scotland may like to add comments about this field. We shall discuss with HESA how to ensure that this field is returned correctly in future, with a view to looking again at the fee liability split as a PI.

     Employability indicator

103. The Government is anxious to see an indicator of ‘employability’ produced this year. At present, the only information available from which we could derive such an indicator comes from the HESA First Destination Survey (FDS). An audit report of this survey concluded that, while the survey is fit for its current uses, it does not provide sufficiently robust data from which to construct a performance indicator.
104. Following discussions with interested parties, it has been agreed that procedures for collecting data for FDS 2000, concerning 1999-2000 graduates, will be tightened, and HESA has already sent out new guidelines to institutions. The auditors expect that these will improve the data sufficiently to allow them to be used for PIs.
105. The intention is to produce the first employability indicator in March 2001. The basic indicator will look at all home full-time students graduating with a first degree who are in the job market, and specifically at what percentage of them are in employment at 31 December of the academic year following their graduation. We are developing a benchmark for the indicator; it will be more complex than the one used for current indicators because of the need to take many more factors into account. Mathematical modelling is being used to determine the format of such a benchmark.
106. Further details are not available at present, but once the benchmarking procedure has been more fully developed we will consult institutions about the format it is likely to take, and the sorts of values we expect to get, based on previous years’ data.
107. A further indicator, looking at ‘graduate jobs’, has been proposed. This will not be available initially, but we are looking at the proposal, which follows on from work done by the Institute for Employment Research (IER) at Warwick University. (‘Moving On: graduate careers three years after graduation’ published by the Higher Education Careers Service Unit (CSU), November 1999.)

Annexes

Note: Annexes to this document are also available in Word (4.2Mb).