Guide 99/67

December 1999

Performance indicators in higher education

Overview

Introduction

Performance indicators have been introduced for the first time for all 175 publicly-funded universities and colleges in the United Kingdom. This initial set covers: access to higher education, non-completion rates for students, outcomes and efficiencies for learning and teaching in universities and colleges, and research output.

This overview explains the purpose of and background to the exercise, and the national outcomes. It also summarises the content of the six tables of indicators, which encompass a wide range of information about individual institutions.

Further information

More detailed information and a complete set of tables are published in 'Performance indicators in higher education' (reference HEFCE 99/66) which is available from the Higher Education Funding Council for England, and on the web.

Why produce performance indicators?

The purpose of performance indicators is to:

• provide better and more reliable information on the nature and performance of the UK higher education sector
• allow comparison between individual institutions
• enable institutions to benchmark their own performance
• inform policy developments
• contribute to the public accountability of higher education.

Once indicators have been available for a number of years, it will be possible to track changes over time in the sector and in individual universities and colleges. The trends revealed as a result will enable higher education institutions to benchmark their performance; funding bodies to assess the effectiveness of policies; and international comparisons to be made.

The initial set of indicators covers teaching and learning and research. In future the range will be extended further.

Who are they for?

These performance indicators have been produced primarily to meet the information needs of government, the universities and colleges themselves, and the four UK higher education funding bodies - the Higher Education Funding Council for England (HEFCE), the Higher Education Funding Council for Wales (HEFCW), the Scottish Higher Education Funding Council (SHEFC) and the Department of Education Northern Ireland (DENI).

Although these organisations are expected to be the main users, the indicators are also of interest to wider audiences including schools, prospective students and employers.

How did they come about?

The need to establish a common system for indicating aspects of the performance of higher education institutions has been recognised for some time. Most recently it was highlighted by the National Committee of Inquiry into Higher Education (the Dearing Committee), which reported in July 1997.

In November 1997 the Secretary of State for Education and Employment, and the Secretaries of State for Scotland and Wales, asked their respective funding councils to investigate the feasibility of introducing such measurements.

The process has been led by the Performance Indicators Steering Group (PISG). Its members are drawn from government departments, the three UK funding councils, the Higher Education Statistics Agency (HESA), and universities and colleges through the Committee of Vice-Chancellors and Principals (CVCP), and the Standing Conference of Principals (SCOP).

How have the indicators been developed?

For this first set of indicators, existing data have been used, which are known to be reliable, robust and verifiable. As a result the indicators concentrate primarily on full-time undergraduates, who form 70 per cent of the full-time student population.

Universities and colleges have been consulted on the compilation and preparation of the indicators, and given the opportunity to verify the data for their own institution.

More work is required to gather sufficient data on part-time students and postgraduates.

What are the international comparisons?

The thorough process of compiling these indicators from individual student records has produced for the first time an accurate and reliable figure for full-time undergraduates who complete their course of higher education.

Using the 1997-98 data, the completion figure for the UK is 82 per cent, which is slightly better than previous estimates and compares very favourably with other countries.

In the UK the proportion of young people moving into higher education has doubled from 15 per cent to 30 per cent in the last 10 years. Yet in the same period the non-completion (or ‘drop-out’) rate has only risen from 14 per cent to 18 per cent, demonstrating that a movement to a more inclusive system of higher education in the UK has not resulted in a comparable increase in non-completion.

International comparisons of non-completion rates are complicated by the variation in the length of degree courses in different countries. For instance, in the UK first degree programmes typically last three years, whereas in Germany they last an average of six years.

However, figures provided by the Organisation for Economic Co-operation and Development (OECD) in 1998 show that among industrialised countries only Japan (at 11 per cent) has lower non-completion rates than the UK. Equivalent rates are much higher in France (45 per cent), the United States (37 per cent) and Germany (28 per cent).

How can performance indicators inform policies?

Performance indicators will provide benchmarks enabling institutions to assess their performance in increasing access, improving equal opportunities, supporting lifelong learning, and maximising achievement for all who can benefit from higher education.

The results will help the funding bodies develop policies to encourage institutions to reach out to disadvantaged and under-represented groups, particularly students from poorer backgrounds. They will also provide vital information on how effective institutions are in supporting such students and enabling them to succeed in higher education.

From these indicators, higher education institutions and funding councils will be able to monitor the effects of policies over time, identify good practice and help disseminate it throughout the sector.

How current is the information?

Most of these indicators have been prepared from data supplied by HESA for 1997-98, which is the most recent information available for the topics covered.

Factors such as the effects of changes in student funding, or the impact of funding council initiatives to promote widening participation, will not be reflected in these figures.

Are the tables comparing like with like?

None of these performance indicators attempts to demonstrate who or what is ‘best’ overall. Higher education is too diverse for that.

Furthermore, it is not sensible to compare the outcomes for an institution where most students have very good A-levels, with one taking students from a wide range of educational backgrounds. The subjects studied at an institution also affect performance: when looking at non-completion, for example, medical students have very low rates of non-completion, and engineering and technology students relatively high rates – wherever they study.

However, the tables are intended to allow meaningful comparisons to be made between institutions. They therefore include benchmarks which take account of these two major factors: the subjects studied at the institution, and the qualifications on entry of its students.

In each case, this 'adjusted sector benchmark' is the value that the whole UK sector would have if it had the same subject and entry qualification profile as the institution.

Although there will be situations where it is useful to compare an indicator with the actual figures for the rest of the UK, in most cases it is more appropriate to compare it with the benchmark.

When is the difference significant?

Most people would agree that small differences between institutions, or between an indicator and the benchmark, should be disregarded. However, the problem is how to define 'small'. To do this the tables include a ‘standard deviation’, which is marked with an asterisk where there is a significant difference between indicator and benchmark.

A significant difference suggests that the institution is doing better or worse than expected in a given aspect. This information focuses attention – either on the need to address weak performance or on the need to provide pointers to good practice.

How is the information presented?

The indicators are set out in six separate tables. All publicly funded higher education institutions in the UK are included, but not all will feature in every table. Most data relate to the 1997-98 academic year. Of the 290,000 entrants to first degree courses in that year, about 75 per cent were young (under 21) and 25 per cent mature (21 or over).

Access indicators

Two of the tables (T1 and T2) give information about the participation of groups that are under-represented in HE, relative to the population as a whole. Results are shown separately for young and mature students, and for full-timers and part-timers, because they have different characteristics.

For young full-time students, for example, it is possible to see for each institution:

• the percentage who attended a school or college in the state sector
• the percentage whose parents' occupation is 'skilled manual', 'semi-skilled' or 'unskilled'
• the percentage who come from a neighbourhood (as denoted by its postcode) which is known to have a low proportion of 18 and 19-year-olds in higher education.

For part-time students and mature students, the information on access is the percentage of entrants who have no previous HE qualification and come from a low-participation neighbourhood.

Nationally, over 95 per cent of 17-year-olds in full-time education attend state schools, but only 82 per cent of young entrants to higher education come from such schools. The indicators show that most institutions take over 80 per cent of their young students from state schools. However, nearly one in six institutions take less than 70 per cent from state schools, and a few take less than half their students from such schools.

Skilled manual, semi-skilled or unskilled people form about 50 per cent of the UK population, whereas only about 25 per cent of young entrants to HE are from these groups. Most institutions take between 20 and 40 per cent of young entrants from these social groups.

About one-third of young people live in ‘low participation’ areas. Nationally, 12 per cent of young entrants and 14 per cent of mature entrants to first degree courses are from these areas. Most institutions take between 5 and 20 per cent of their young full-time entrants from low participation areas, with similar figures for mature full-time students. However, most institutions take less than 10 per cent of their mature part-time students from low-participation areas.

Non-completion rates

Non-completion rates for students at an institution are presented in two ways. The first considers students who start in a particular year, and looks at whether they are still in higher education one year later. The second method looks at projected outcomes over a longer period (see below).

Using the first method, the tables (T3) show what percentage of students at each institution who started in 1996-97 have continued at the same institution, transferred to another institution, or left higher education completely by the following year.

Figures are shown separately for young and mature entrants, for young students from low participation areas and from other areas, and for mature entrants with and without previous higher education qualifications.

Nationally, more mature students (15 per cent) than young students (8 per cent) do not continue in higher education after the first year. In one in eight institutions, more than 20 per cent of mature students do not continue after the first year.

Of course some of those who leave will return, so another table (T4) gives additional information on these. This shows that, nationally, about 27 per cent of young students and 17 per cent of mature students who did not continue beyond the first year then return after a year out – either to the original institution or to another one.

Projected outcomes

Another way to look at non-continuation rates is to use information on current movements of students to project what would happen over a period. Thus the indicators (Table T5) project what proportion of students will eventually gain a degree, what proportion will leave their current university or college but transfer into higher education elsewhere, and what proportion will leave the institution without any qualification and without moving into higher education elsewhere.

Nationally, 80 per cent of students who start on a first degree course are projected to get a degree eventually, although some may transfer to another institution along the way. Only 18 per cent of students are projected not to gain any qualification; most of the remaining 2 per cent are expected to achieve a qualification below degree level.

For the majority of universities and colleges, between 70 and 90 per cent of entrants are projected to graduate from the institution where they started. At a small number of institutions this figure is less than 60 per cent.

Efficiency

Projected outcomes can also be used to give a measure of efficiency. If all its students qualify within the expected time (in three years for a three-year course, for example) the institution would be 100 per cent efficient. However, some students may repeat a year – because of exam failure, changing course, or illness – and will take longer to complete. Others will drop out.

The efficiency of an institution is defined as the ratio of the time students should ideally take to obtain a qualification, to the time they are projected to take on average (taking account of repeat years and drop-outs). Nearly all institutions show an efficiency of over 80 per cent.

Research outputs

The main indicators of research in UK higher education are the ratings from the Research Assessment Exercise (RAE), held every four or five years. The research indicators produced as part of the performance indicators initiative (Table R1) provide additional information on the quantity of research outputs relative to the resources consumed, and will show how this changes from one year to the next.

Research outputs are measured by looking at the number of PhDs produced, and the amount of research grants and contracts obtained, for different subject areas. These outputs are related to the inputs of resources: academic staff costs and research funding provided by funding councils.

These research indicators, unlike those for learning and teaching, are standardised to a value of 1. If an institution has an indicator of 1, this shows that it is producing the same as the rest of the sector, relative to its input. A value above 1 means that the institution is producing more than the sector, relative to its input.