Report by Segal Quince Wicksteed to DENI, HEFCE, HEFCW, QAA, SHEFC

Providing public information on the quality and standards of higher education courses

HEFCE ref 99/61

October 1999

Segal Quince Wicksteed Limited
economic development consultants
The Grange
Market Street
Swavesey
CAMBRIDGE
CB4 5QG
Tel: (44) (0)1954 231931
Fax: (44) (0)1954 231767
E-mail: Mailbox@sqw.co.uk

October 1999

This document contains the Contents and Summary only. The full report is available in Word or RTF formats.

Contents

Glossary

Summary

Introduction
Findings: applicants
Findings: recruiters
Recommendations

1. Introduction

Acknowledgements
Study context and objectives
Methods and approach
Interview groups
Structure of the report

2. Findings: selection process

Applicants Selection modes
Factors which influence the selection process Number of HEIs considered
Type of HEI applied to
Choice of HEI: key influences in deciding on preferred institution

Recruiters

3. Findings: current information sources

Applicants Sources of information which helped in the choice of HEI

Recruiters

4. Findings: quality and standards information preferences

Applicants

Awareness and potential use of quality and standards information
Type of quality and standards information
Format of quality and standards information

Recruiters

Awareness
Adequacy of information

5. Conclusions and recommendations

Conclusions
Applicants
Recruiters

Context

Recommendations

Raising awareness and understanding
General awareness raising
Targeting
Content and format
Delivery

Annexes

A. Terms of reference for the study
B. Sample characteristics
C. List of main consultees
D. Questionnaires used in the postal survey
E. Schools, colleges and universities involved in the study
F. Social groups

Summary

Introduction

1. This is a report on providing public information on the quality and standards of higher education courses (the full terms of reference are to be found in Annex A.) The main objectives of the study were to:
   • review stakeholders’ information needs regarding quality and standards of courses (principally applicants to higher education (HE) and graduate recruiters)
   • identify current information usage and sources
   • recommend how needs can be met within the framework being developed by the Quality Assurance Agency for Higher Education (QAA).
2. The study was undertaken between January and June 1999. Fifteen discussion groups were held involving school and sixth form/further education college students, as well as undergraduates and parents. Additionally, a large-scale postal survey was carried out of applicants to HE, parents of applicants, first year undergraduates, and recruiters, which in total received nearly 1,000 responses. (Annex B provides a breakdown of the sample.) Focus groups and the surveys involved organisations and individuals from across England, Wales, Scotland and Northern Ireland. Face-to-face interviews were also held with careers advisors and admissions tutors for higher education institutions (HEIs) and a range of interested parties such as SCOP, CVCP, NUS and professional bodies (Annex C provides a list of consultees).

   Findings: applicants

3. School leavers tend to select HEIs by initially narrowing down a long list of options based on advice from peers, parents and school careers advisors. They turn to published sources of information when options have been narrowed. Mature students are generally more constrained in their choice of options than school leavers, due to different motivations for learning, entry requirements, and constraints such as family commitments.
4. The most important factor influencing choice is the course(s) on offer. This was specified by over one-third of survey respondents, but a fifth also chose location, which ranks equally with reputation. Quality of teaching as the single most important factor influencing choice is low on the ranking - selected by only 6% of the postal survey sample. However, when asked to identify any factor which influenced choice, quality of teaching and facilities are rated by two-thirds of applicants as important. In other words, quality of teaching is important to applicants but not decisive.
5. By far the most important sources of information informing student choice are prospectuses and open days. Parents, friends, advisors and publications such as the UCAS Big Book are collectively important too. But different types of applicants use different sorts of general information and they are influenced by peers in different ways. In particular, mature students rely far more on friends and guidance services and less on publications than school leavers.
6. In relation to awareness and use of quality and standards information:
   • approximately half the current and recent applicants in the postal survey were aware of information about teaching and quality (including, but not exclusively, QAA reports)
   • around two-thirds of those aware of the information claim to have used it
   • nearly all current or recent applicants stated that, had they known about the various sources of information, they would have used them
   • the most influential sources of information in relation to quality and standards are the HEIs themselves, league tables and schools careers advisors. Only 12% of respondents considered QAA reports to be the single most important source of information about quality.
7. Further requirements specified by applicants included:
   • quality and standards information relating to courses; there was relatively little interest in information at the HEI level
   • curriculum information, and levels of academic attainment, are of major interest, but other factors such as what the HEI is seeking to equip its graduates for, and how well a course is taught, are also important
   • how gradings should be presented. Respondents were given options of a scale of 1 (unsatisfactory) to 5 (excellent), marks out of 100 and a simple satisfactory/unsatisfactory threshold, as well as the opportunity to specify 'other'. As a minimum, a rating of 1 to 5 for quality and standards was sought, but more interest was expressed in more detailed scores that rate quality and standards out of 100. Respondents also sought different grades for different aspects of a course, as currently prepared by QAA
   • an explanation of how the assessments are carried out
   • printed formats are marginally preferred to web-based delivery.

   Findings: recruiters

8. Recruiters are interested most of all in the personal qualities of the individual graduate. Information on teaching quality and standards of attainment is therefore secondary, and there is considerable reliance on a range of other sources of information and on more general perceptions.
9. Employers, especially larger organisations and those recruiting for specialist skills, tend to target particular HEIs for recruitment purposes. The key factors determining which are targeted are the track record of graduates within the company from the HEI, the reputation of the HEI amongst colleagues, and the relevance of the course to the employer's needs. Specific information about the quality of teaching on a course is a criterion that only around 5% of employers currently use in their targeting, according to the postal survey findings. In terms of sources of information on quality and standards, colleagues are a major source, and league tables in newspapers are also important. Only a small minority used the reports prepared by QAA or its predecessors.
10. There are clearly issues of awareness of quality and standards information amongst employers. The majority are unaware that information on teaching quality is available. But importantly, most indicated that if they had been aware, they would have consulted it. Smaller firms in particular tend to be less aware, but they too would consult such information, if it were made available to them. This information would be most useful for firms recruiting for specialist skills.
11. Recruiters’ requirements are for:
    • information on course content, followed by benchmarking and comparison of courses. The types of general skills that graduates would be equipped with are also of interest; not surprisingly, employers of generalist graduates ranked this most highly
    • information at the course level rather than the more detailed module level or less defined institutional level. Within the course level, a relatively simple grading system is preferred: one which scales a course from 1 to 5, ranging from unsatisfactory to excellent, was the preferred option, when offered alongside simple thresholds or more detailed ratings using a scale out of 100
    • a breakdown by different aspects of a course, as with the current QAA approach. They are also interested in how the scores are arrived at
    • there is a perceived 'shelf life' to the information, and most felt an assessment would be needed every two to three years
    • both printed and web-based media are desired, with a marginal preference for web materials. Summarised information rather than the whole report is required.

    Recommendations

12. These findings suggest that there are a number of ways in which quality and standards information could be prepared and delivered in the future to public users. Our recommendations are targeted at three areas. First awareness raising, where the aim is to raise awareness of the existence, and understanding, of the value of QAA outputs. We recommend:
    • general awareness raising through a variety of means designed to provide QAA outputs with a higher profile in the media
    • targeting awareness campaigns on careers advisors as key intermediaries between the HEIs and applicants
    • promoting QAA information at events which applicants attend, for example UCAS conventions.
13. The second area for consideration relates to content and format. The findings suggest preferences that:
    • different aspects of a course are assessed and reported separately, as at present
    • the assessment is made on a scale with a limited number of points ranging from unsatisfactory to excellent. Whether the scale is numerical or narrative is of secondary importance but it should enable clear comparisons to be made
    • many applicants would find a single overall assessment (in addition to assessments of separate aspects) convenient but it would have limited influence on their decisions.
14. Most aspects of the current and envisaged QAA outputs are of interest but there is a need to provide the information in a more concise format and in lay language.
15. The final set of recommendations concerns delivery. Information should be freely available to all intermediaries as at present. We also recommend that:
    • summaries of QAA outputs should be carried in prospectuses in an agreed and consistent format
    • the proposed UK HE Mall, currently being developed as a gateway to UK higher education information on the World Wide Web, incorporates initiatives which focus on quality issues.