Invitation to tender for a contract to investigate ways of improving the public information on the quality and standards of higher education courses

Purpose

1. The purpose of this project is to:

Review from first principles the needs of different stakeholders (including actual and potential students, careers advisers, graduate employers, the HE Funding Councils, Government, and the wider public, as well as higher education institutions (HEIs) themselves) for published information about the quality and standards of higher education provision.
Identify what information about quality and standards the stakeholders currently use.
Make recommendations for how stakeholders’ information needs could best be met within the new HE quality assurance framework which the Quality Assurance Agency is currently developing.

2. This project is a joint study for the Higher Education Funding Council for England (HEFCE) in co-operation with the Scottish Higher Education Funding Council (SHEFC), Higher Education Funding Council for Wales (HEFCW), the Department of Education for Northern Ireland (DENI) and the Quality Assurance Agency (QAA).

Background and Context

3. The HE Funding Councils have a statutory responsibility to secure assessment of the quality of learning and teaching at the institutions they fund.

4. It has from the outset been one of the purposes of the quality assessment procedures in higher education that they should generate reports and information which would be published for the benefit of stakeholders (including the Funding Councils themselves). The method to date has involved teams of outside reviewers visiting HEIs to review each subject programme (known as Teaching Quality Assessment or TQA) and the institution overall.

5. Since 1993 the HEFCE has published 1500 individual subject assessments on 150 HEIs in 25 subject areas. Since the process began the subject assessment method has been refined to provide more quantitative information. In 1997 the assessment operation was transferred to the Quality Assurance Agency (QAA) based in Gloucester. The QAA has several stakeholders and operates under service level agreements with the HEFCE, HEFCW and DENI.

6. In Scotland and Wales, SHEFC and HEFCW have managed their own assessment processes and reporting.

7. The current subject level reports include, in addition to narrative descriptions of the strengths and weaknesses of aspects of provision, summative ratings of quality. In England these ratings are on a scale of 1 (unsatisfactory) to 4 (excellent) and a rating is given to each of 6 standard aspects of the provision. These aspects are Curriculum Design, Content and Organisation; Teaching, Learning and Assessment; Student Progression and Achievement; Student Support and Guidance; Learning Resources; and Quality Management and Enhancement.

8. The Funding Councils have distributed copies of subject overview reports (summaries of the individual HEI subject assessment reports) to schools, careers services, FE colleges and public libraries.

9. Reports are also available on the Funding Council websites. The HEFCE website receives over 600,000 `hits’ for quality assessment reports per year - although it is not clear what can be deduced from this as to the size and nature of the audience for these documents.

10. The QAA (and prior to its formation, the HE Quality Council) also undertakes whole-institution reviews of HEIs, focussed on the institution’s procedures and mechanisms for maintaining academic standards. The QAA prepares and publishes reports on these reviews. Every HEI in the UK has had an initial review completed.

11. Following the Dearing report on the future of higher education, the QAA has been developing proposals for a new quality assurance framework which would focus more on outcomes and standards, and which would integrate the subject-level and institutional-level reviews. The latest statement of the proposals is contained in a QAA bulletin published in October. The Funding Councils have also published a commentary setting out their requirements for the new QA framework, including their requirements relating to information on quality and standards.

12. The discussions surrounding these proposals have identified as an issue needing more exploration, how best the new framework can meet the public need for information on quality and standards. Hence this study.

Audiences for public information on the quality and standards of higher education courses

13. There is a concern that the style and language of the reports currently produced are not immediately accessible to:

potential students and their advisers, who are looking for simple straightforward information and comparisons on which to select their courses, or to
employers, who are similarly looking for simple straightforward information and comparisons on which to recruit graduates.

14. Partly in consequence, some third parties have produced guides to higher education and have constructed league tables of institutions based on their own criteria, often including QAA assessments and drawing particularly on the summative ratings contained in reports.

15. HE institutions for their part have reservations about the rating process, and are particularly concerned to ensure that any summative ratings in the new framework are a valid reflection of their quality and standards. They have a wider concern to ensure that students and employers have access to the range of information which will enable them to make informed judgements about whether a particular HEI’s provision will suit their individual needs and interests, without disproportionate focus on ratings.

16. In addition to students and employers, there are other audiences with an interest in information about the quality and standards of higher education courses. They include professional and statutory bodies involved in HE; careers advisers and others who advise potential students; overseas student sponsors; the Funding Councils (whose requirements are set out in the joint Funding Council commentary on the QAA proposals); and the Government.

17. The DFEE have established the `Student Information Needs Working Group (SINWG)’ with representatives from many organisations, to identify the needs of potential students. That group overlaps with, but is not co-terminous with, the present study. The SINWG is about student needs for all types of information, not just quality and standards. The present study is limited to quality and standards, but across the range of stakeholders, not just students.

Objectives of the project

18. Against this background, the objective is to undertake a broad-based investigation of what information stakeholders require, at both at the subject level and the institutional level, on the quality and standards of the learning experience at different universities and colleges. The focus of the study is identified below.

19. The tenderer is required to:

identify and take account of what information already exists about stakeholders’ information needs relating to HE quality/standards, and relevant work currently in progress (including the DFEE SINWG).
distinguish the needs of the various stakeholders (including potential students; actual students; graduate employers; professional and statutory bodies; student sponsors; careers guidance staff; and the Funding Councils).
establish:
- the main sources of information currently available to stakeholders;
- how far stakeholders are aware of those existing sources of information (including the TQA reports, and the narrative descriptions and summative ratings they contain; and the institutional audit reports)
- how far stakeholders understand how those sources of information are derived and their limitations (for example, that the TQA procedures seek to assess the institution’s effectiveness in achieving its own objectives for the programme);
- stakeholders’ views on their relative usefulness.
establish stakeholders’ preferences for the types of information about quality and standards they would like to have - including content; format; medium (eg the scope for electronic distribution); and currency (how long different forms of information are felt to remain valid).
make recommendations on how stakeholders’ preferences can best be delivered in the QAA’s proposed new QA framework, including specifically through the use of the new programme specification templates being developed by QAA.

20. In formulating recommendations under paragraph 19v, the study must take account of the costs of collecting information, value for money, and likely future resource constraints, both for QAA and for HE institutions, in responding to the wide array of information which stakeholders might ideally wish to have. The study should therefore take into account the forms of information on quality and standards which the QAA’s proposed framework is (and is not) capable of generating. It should also prioritise the information needs identified, distinguishing the essential from the desirable, and that which would be useful to many audiences from that which is of more limited interest.

21. As stated in paragraph 19v. above, the focus of the study is to identify how the proposed new QA framework in HE can best meet information needs. But it may well be that the study will identify needs which can be better met through mechanisms other than the QAA procedures. If so, the report should draw attention to those, and it will be for the body or bodies concerned to consider whether and how to pursue those conclusions.

Process

22. The successful tenderer is free to suggest their own working methods, but should include the following elements:

a review of current work in the area, and existing evidence relating to user preferences and the information they actually use at present (including consideration of what conclusions might be derived from analysis of the pattern of user “hits” on the HEFCE website)
interviews with stakeholder and user organisations
data collection from end users of public information on the quality and standards of higher education, in sufficient detail to draw reliable conclusions about differences of information needs between different stakeholders.

Outputs

23. The main output from the consultancy will be a report which:

identifies for each main group of users the value of their current sources of information on the quality and standard of higher education
provides feedback from users on the format and ratings used in QAA assessment reports
provides feedback from users on their preferred method of obtaining information on HEI quality and standards in the future
recommends (with examples) how users needs can be met within the new QA framework.

24. The tenderer would also be expected to present the report to the sponsors and a group of key stakeholders and users.

Costs

25. The tenderer is asked to provide a fully costed proposal for all work described in this invitation to tender. The proposal should show the time to be spent, the methods to be used, and the people who will be involved in each stage of the project.

Personnel and reference projects

26. The tenderer should provide biographies of staff who will be responsible for the project together with summaries of any previous work relevant to this project. The Council would expect the tenderers to have the following skills/experience:

excellent communications skills
research, opinion survey, marketing and analysis skills
knowledge and understanding of higher education
experience of design, print, electronic communications and the World Wide Web
knowledge of good practice in the field

Confidentiality

27. The views expressed during the fieldwork will be treated in confidence by the tenderer and the Council so that individuals are not identified. A decision on whether to publish a report of the project will be made by the HEFCE.

Conflicts of interest

28. Tenderers must advise how they would deal with any possible conflict of interest between the Council and any other client they may have. Tenderers must state any connections between their firm and members and officers of the Council.

Submissions

29. Tenders should be submitted in the enclosed envelope no later than Friday 11 December 1998 addressed to:

Susan Clark
Office Services,
HEFCE,
Northavon House,
Coldharbour Lane,
Bristol
BS6 5RR.

30. In evaluating the tenders, the Council will pay particular attention to the following criteria (not listed in any particular order):

The credibility and creativity of the proposed methods for undertaking the study.
The track record of the tenderer in past assignments of a similar nature.
The experience and qualifications of the consultants proposed for the contract.
The quality assurance and control procedures adopted by the tenderer.
The cost of the proposal.

Timetable

31. Interview and selection will take place before Christmas. A four month programme with delivery of a final report in April is the target. A small steering group will advise the consultants at key stages.

Payment

32. The successful contractor will be paid on the following schedule:

one-third of fee on appointment
one-third of fee on delivery of initial findings
one-third of fee on presentation of final report.

Further information

33. In order to help you prepare the tender, we enclose the following documents:

Quality Assurance: a new approach
The Funding Councils joint response to the QAA consultation
QAA Subject Review handbook
QAA notes for Editors of Assessment Reports

34. You can access further information about the HEFCE from this web site.

35. Please contact Greg Kerchhoff on (0117) 931 7317 if you have any queries regarding this tender. However, no pre-tender negotiations are permissible.

17 November 1998