Joint Planning Group for Quality Assurance in Higher Education
Final Report - Draft Version
September 1996(Note: The final version of this report is available electronically at http://www.cvcp.ac.uk/pubs/jpg.html.)
HEFCE Reference JPG 2
Contents
Background
The purposes of quality assurance
The quality assurance process
Collaboration with other funding bodies and with professional and statutory bodies
A dynamic process
The new agency
Cost implications
Transition arrangements
Future work
Annex A: Joint Planning Group Terms of Reference
Annex B: Membership of the Joint Planning Group
Annex C: Joint Planning Group Working Party
Annex D: Academic standards
Annex E: Higher education and professional and statutory bodies
Annex F: The constitution and membership of the new agency
AAU Academic Audit Unit
CCETSW Central Council for Education and Training in Social Work
COSHEP Committee of Scottish Higher Education Principals
CVCP Committee of Vice Chancellors and Principals of the Universities of the United Kingdom
DfEE Department for Education and Employment
DENI Department of Education for Northern Ireland
FHE Further and Higher Education
GSP Graduate Standards Programme
HEFCE Higher Education Funding Council for England
HEFCW Higher Education Funding Council for Wales
HEI Higher Education Institution
HEQC Higher Education Quality Council
NIHEC Northern Ireland Higher Education Council
OFSTED Office for Standards in Education
PSB Professional and Statutory Body
QAD Quality Assessment Division
RIBA Royal Institute of British Architects
SCOP Standing Conference of Principals
SHEFC Scottish Higher Education Funding Council
SLA Service Level Agreement
SOEID Scottish Office Education and Industry Department
TTA Teacher Training Agency
WOED Welsh Office Education Department
Executive summary
1. This is the Joint Planning Group's second and final report. We believe that our proposals bring significant gains that will both reduce, and assist higher education institutions in managing, the number and level of external demands placed on them. They are intended to help institutions improve the quality and standards of their educational provision; provide public information on the quality and standards of that provision; and satisfy the responsibilities of the funding councils to account for the public money invested in higher education. [paragraph 6 of the report] Our first report generated considerable interest and evoked more than a hundred responses. We have given detailed consideration to these responses for which we were very grateful. [4]
2. We recommend the establishment of a single agency as soon as possible, and the creation of an integrated process of quality assurance. The new agency will take over all the functions of the HEQC, and the main assessment functions of those funding councils choosing to contract with the agency for the discharge of those functions; the HEQC will be wound up. Our report indicates:
- a. the services that the new agency should provide;
- b. what its composition should be; and
- c. the shape of the transitional arrangements which will be necessary. [7]
3. As required by our terms of reference, our report is circulated for comment. We will take account of any comments received when we meet finally in November before sending the complete report to the commissioning bodies and Ministers. Our proposals are a blueprint for the future rather than an operational manual. Specifying the details of the new arrangements will be the task of the new quality assurance agency following consultation, development and piloting of the arrangements proposed in this report. [3 and 32]
An integrated process of quality assurance
4. The new agency will operate an integrated process of quality assurance covering the totality of each institution's provision, wherever and however delivered, and however funded, with significant benefits for institutions. Our proposals can, and should, eliminate areas of overlap and duplication in current quality assurance arrangements; are sufficiently flexible to accommodate a wide range of academic structures; involve discussion between the agency and each institution concerning the number, scope and timing of reviews; enable the agency and each institution to harmonise internal and external review arrangements; and involve the head of each institution in the selection and composition of review teams. [8]
5. The quality assurance framework outlined in this report assumes that institutions will monitor and review their provision; the agency will manage a series of external reviews of that provision. [8]
6. The main elements of the Group's proposals for an integrated process of quality assurance are:
- a. an eight-year, national review timetable, to be published by the agency, to assist institutions in planning their internal monitoring and review arrangements, reflecting their own priorities, educational provision, and academic structures; [9]
- b. a maximum number of units of assessment, perhaps 61, as suggested by current HEFCE practice; [21]
- c. the 'brigading' of cognate subject/programme areas in the agency timetable to facilitate combined reviews and help to ensure that a two-year subject/programme review cycle does not inhibit flexibility; [21 & 23]
- d. an institutional quality assurance plan agreed with the agency setting out the educational provision to be reviewed by an institution over the eight-year cycle; [9]
- e. flexibility for each institution to agree with the agency the number, scope and timing of reviews to enable an institutional perspective to be brought to bear on the process from the outset, and ensure that the agency and the institution can harmonise internal and external review arrangements; [20]
- f. the opportunity for each institution to combine subject/programme reviews, thereby reducing the number of 'engagements' with the agency; [21 and 46]
- g. the possibility of extending the subject/programme review cycle for subjects where there is a case for combination of units, for instance, or where a PSB timetable could be more satisfactorily accommodated. [23]
- h. the timing of reviews to take account of other external timetables, for example those of the professional and statutory bodies, thus allowing scope for greater co-ordination and collaboration with such bodies. [20 & 23]
- i. confirmation of the extent to which an institution is discharging its educational responsibilities, and has procedures securely in place which will enable it to continue to do so. [18]
7. Taken as a whole, the process will assess the extent to which an institution is discharging its responsibilities for teaching and learning effectively at subject/programme level; indicate areas of strength and weakness, including aspects of provision requiring attention; involve an evaluation of the validity and reliability of an institution's internal review procedures, and provide reassurance that each institution has in place effective arrangements for assuring academic standards in the institution. [12]
8. The review process will involve visits to institutions and result in reports published by the agency. Each review team will consist of academic, and where appropriate, professional peers, selected by the head of the institution involved and agreed with the agency, against published criteria for the composition of review teams. The chairman of each review team would be appointed by the agency, and be responsible for the team's published report. [26]
9. Institutions will be able to nominate an observer to each review team. [27]
10. The new quality assurance framework will take into account, as far as possible, the requirements of other funding bodies, such as the NHS, accrediting, and professional and statutory bodies. [28]
The establishment of a new quality agency.
11. We propose that:
- a. the new quality agency be a company limited by guarantee and registered as a charity; [F 1 a]
- b. the company's members be the representative bodies of the heads of higher education institutions; [F 2]
- c. there be 14 directors on the board of the company including nominees from these bodies and the funding bodies (the funding councils and DENI), and representatives of the wider community with an interest in quality and standards in higher education. [F 3] Wide consultation will ensure an appropriate range of experience and expertise on the board. It is expected that at least seven members of the board will be staff drawn from the HE institutions. [F 10]
12. We propose two types of service level agreement:
- a. between the agency and the institutions; and
- b. between the agency and the funding bodies. [37]
13. The link between both types of agreement will be provided by a common foundation document, which would be a single statement of the integrated quality assurance process. The two types of SLA would differ in their scope and in the range of the required outputs, including their handling of individual national requirements and approaches across the different parts of the UK. [37]
14. These agreements will enable the agency to:
- a. confirm that institutions' internal quality assurance procedures are working effectively;
- b. identify and disseminate information about innovation and best practice in teaching, learning and student assessment;
- c. provide information and reports about quality and standards in individual institutions, subjects, programmes and aspects of provision for both domestic and overseas audiences; [34 a to c]
- d. undertake quality enhancement activities; [38 c] and
- e. focus on those issues necessary for the funding bodies to secure their responsibilities for public accountability and public information. [39]
15. We expect that the funding councils and DENI will wish to make it a condition of their grants to institutions that they draw on the agency's services to the extent that it is necessary to allow the agency to meet the terms of its service level agreements with those bodies. [35]
16. The agency may also wish to contract, as appropriate, with Government Departments and other bodies (for example, PSBs), and with institutions not in membership of the representative bodies and/or not in receipt of public funds. [36]
Transition arrangements
17. The agency's immediate tasks on establishment will be to:
- a. ensure that the momentum of current programmes for assessment and audit is maintained; [48]
- b. take over responsibility for running the current teaching quality assessment and quality audit activities, and all HEQC's other functions, from 1 April 1997. [52]
- c. pilot, develop and consult upon the new process [13, 19, 21, 27, 32, 48 and 53]
Cost implications
18. We envisage savings from eliminating the current duplication between audit and assessment. Our proposals for an eight-year review timetable will reduce the average number and cost of engagements with the agency, the number of reviews conducted each year [46], and the central administrative load. [45, 46 and 47]
Background
1. The CVCP's Academic Audit Unit was established in October 1990. On the other side of the binary line the former Council for National Academic Awards was responsible for validating and awarding the qualifications of polytechnics and colleges of higher education. In 1992 the Government set up the Higher Education Funding Councils with quality assessment responsibilities. The machinery set up by the Councils for this purpose operates alongside the quality audit machinery which is now part of the HEQC. Within the universities and colleges there is a widespread feeling that these arrangements lead to overlap and duplication, thus creating unnecessary work.
2. Following an initiative by the HEFCE, CVCP and SCOP in the summer of 1995 Ministers welcomed the establishment of a Joint Planning Group to produce proposals for developing quality assurance arrangements, including proposals for a single agency. The Secretary of State for Education and Employment agreed terms of reference (Annex A) and membership (Annex B) proposed by an informal group consisting of representatives of CVCP, COSHEP, SCOP, HEFCE and HEFCW. She also agreed to the appointment of Sir William Fraser as Chairman of the Group.
3. This is the Group's second and final report. As required by our terms of reference it is circulated for comment. We will take account of any comments received when we meet finally in November before sending the complete report to the commissioning bodies and Ministers.
4. The Group has met seven times to date. Our first report generated considerable interest and evoked more than a hundred responses. We have given detailed consideration to these responses for which we were very grateful. We were also helped by the report of a working party chaired by one of our members and including members from institutions. The working party was asked to develop the quality assurance framework set out in our first report, and it took into account responses to that report as they were received. The terms of reference and membership of the working party are at Annex C. In addition, a consultation meeting with professional and statutory bodies (PSBs) was attended by about fifty representatives.
The purposes of quality assurance
5. In developing the proposals contained in this report, we have given careful consideration to the question: what is the purpose of quality assurance in higher education? Who needs to be assured about what, by whom, and how should this be done? We believe that the prime responsibility of an external quality assurance agency should be to support higher education institutions in discharging their responsibility for the maintenance and enhancement of the quality and standards of their educational provision. Students, both within and beyond the UK, need to know about the quality and standards of educational provision in an increasingly diverse higher education sector. So do employers, who recruit the graduates of higher education institutions, the taxpayer and Parliament, which has placed on the funding bodies a statutory responsibility to ensure proper accountability for the use of public funds.
6. Against this background, we believe that the purposes of quality assurance in higher education can be summarised as follows:
- a. to facilitate continuous quality improvement through the sharing of good practice and innovation;
- b. to enable the funding bodies and institutions to discharge their statutory responsibilities;
- c. to provide timely and accessible public information, on a consistent and, where possible, comparable basis, on the quality and standards of the educational provision for which each institution is responsible;
- d. to ensure that any unacceptable provision is speedily addressed.
7. We recommend the establishment of a single agency as soon as possible. The relationship between it, the institutions, and the funding councils should be one of partnership and co-operation. It should replace the HEQC, all of whose functions should be transferred to the new agency, as should the main quality assessment functions of those funding councils choosing to contract with the agency for the discharge of these functions. We indicate in this report the services it should provide to the funding councils, the institutions and, where appropriate, Ministers. We suggest what its composition should be and the shape of the transitional arrangements which will be necessary.
8. We are sensitive to the criticism that what we propose might be represented as simply a continuation of audit and assessment but under a single body. We do not believe this to be the case: the new agency will operate an integrated process of quality assurance covering the totality of each institution's provision, wherever and however delivered, and however funded, with significant benefits for institutions. Our proposals can, and should eliminate, areas of overlap and duplication in current quality assurance arrangements; are sufficiently flexible to accommodate a wide range of academic structures; involve discussion between the agency and each institution concerning the number, scope and timing of reviews; enable the agency and each institution to harmonise internal and external review arrangements; and involve the head of each institution in the selection and composition of review teams. The quality assurance processes outlined in this report assume that institutions will monitor and review their provision, whilst the agency will manage an integrated programme of reviews of that provision.
The quality assurance process
Agency review timetable and institutional quality assurance plan
The review process
10. The new agency will consider the quality and standards of educational provision at subject/programme level in each institution, however delivered and however funded, by means of subject/programme area review. In addition to the need to meet the obligations laid on the funding councils, this approach acknowledges that it is at subject/programme level that staff and students engage most regularly, and that it is subject/programme information which prospective students particularly seek.
11. The new agency will also review the arrangements made by each higher education institution for assuring academic standards in the institution as a whole. This acknowledges that responsibility for quality and standards, and the means by which they are assured, resides ultimately with an institution corporately, and not with its constituent elements. Many activities crucial to academic standards and quality are normally provided, resourced and managed on an institution-wide basis. These may include library, laboratory and IT services; student support and guidance services, including provision for international students; staff induction, appraisal and development; and the dissemination of good practice within an institution. In addition, many matters covered by the Higher Education Charters, including student complaints, institutional publicity and recruitment material, are normally an institution-wide responsibility.
12. Taken as a whole, the process will assess the extent to which an institution is discharging its responsibilities for teaching and learning effectively at subject/programme level; indicate areas of strength and weakness, including aspects of provision requiring attention; involve an evaluation of the validity and reliability of an institution's internal review procedures, and provide reassurance that each institution has in place effective arrangements for assuring academic standards in the institution.
Form of reviews
13. We envisage that aspects of provision to be covered by subject/programme area review will be based broadly on those currently covered by teaching quality assessment. In the light of our working party's report, we believe that the processes to be prescribed by the agency can and should aim to eliminate overlap and duplication; but this will require detailed consultation between the new agency and institutions prior to October 1998. The agency will also be expected to develop proposals to ensure that the processes are able to cover provision not currently addressed by teaching quality assessment, namely research degrees/diplomas, and provision not funded by the funding councils.
14. It will be the responsibility of those conducting subject/programme area review to consider
- (a) the way in which the academic standards which an institution expects students to attain in the area under review are identified by the provider unit involved;
- (b) the way in which the attainment of those standards is monitored; and
- (c) the form in which their attainment is described.
(See also Annex D)
15. Subject/programme area review reports will be based on evidence provided by an institution of the quality and standards of educational provision in that area, and on evidence gathered by a review team during a review visit. The review process will need to be sufficiently flexible to accommodate a wide range of academic structures, both in subject and organisational terms, and a variety of sources of evidence. The latter will include evidence generated by each institution's internal monitoring and review processes, which normally include an external contribution. It might also include evidence generated to meet the requirements of external bodies such as professional and statutory bodies and OFSTED.
16. The agency will report separately on an institution's overall quality strategy, and the arrangements by which it assures its academic standards, its learning infrastructure, and its internal and external communications. In addition, it will cover the quality assurance arrangements for any collaborative work with other institutions or organisations, in this country and overseas. Institution-wide review will consider an institution's own account of its quality management arrangements; the experience and record of subject/programme area reviews; and will involve a visit to the institution by a review team. The precise emphasis of each institution-wide review will be subject to discussion between the agency and the individual institution, and may involve a review of areas of activities identified by the institution itself and not referred to above.
17. Against the background set out in paragraphs 11 and 16, a substantial majority of us consider that all institutions should be subject to an institution-wide review once in each eight-year cycle. The alternative approach which we do not favour would be to require such a review only where there is some concern about an institution as a result of evidence from subject/programme area review or some external cause.
Outcomes
18. Each review will result in a report published by the agency. At the subject/programme level these reports will involve consideration of the quality and standards of educational provision, however delivered and funded, with the strengths and weaknesses of such provision summarised. At the institutional level they will involve consideration of an institution's policies and arrangements for securing its educational objectives, and the management by it of its quality and quality assurance processes. They will describe the extent to which an institution as a whole is discharging its educational responsibilities and has procedures securely in place which will enable it to continue to do so.
19. Within the higher education sector there is great diversity of missions. When the polytechnics obtained university title in 1992, Ministers laid great emphasis on the maintenance of diversity of mission. But the greater the diversity, the more difficult to construct a system which provides comparable information about institutions and subjects across the sector. The need to provide such information wherever possible must be taken into account in settling the detailed criteria to be used by review teams in reaching their judgements, and the way in which they are described, including the use of a graded profile. The way in which these activities and the quality assurance systems associated with them are reviewed by the agency will be for determination by it following consultation with institutions prior to the implementation of the new arrangements from October 1998. The form of various service level agreements which we propose to be concluded with the funding councils and DENI and with institutions, and/or their representative bodies (see paragraphs 36 to 39) will obviously be relevant in this context.
Arranging subject/programme area reviews
20. We have proposed (paragraph 8) that each institution be invited to discuss with the agency the timing of subject/programme area reviews, and their focus. This could include the possibility of combining subject/programme areas across the eight-year cycle. An institution's quality assurance plan will enable an institutional perspective to be brought to bear on the process from the outset, and ensure that the agency and the institution can harmoniseinternal and external review arrangements. Such discussion should also seek to accommodate the accreditation arrangements of professional and statutory bodies where relevant.
21. We propose that the agency publish a national framework with a maximum number of units of assessment (perhaps 61, as suggested by current HEFCE practice), thus allowing institutions who so wish to have all subject/programme area reviews undertaken singly. Each institution would be invited to map all of its provision on to the agency review timetable, agreeing with the agency the number, scope and timing of reviews. Those institutions wishing to combine units, thereby reducing the number of 'engagements', would be free to do so within limits determined by the new agency following consultation with institutions prior to the implementation of the new arrangements.
Review cycles for individual subject/programme areas
22. In letters to the Chairman of the CVCP and the Chief Executive of the HEFCE in the summer of 1995, the Secretary of State for Education and Employment indicated her concern at the possibility that the assessment of individual subject areas would take place over a too lengthy cycle. She proposed that the review cycle for individual subjects should not exceed two years. We have considered the implications of this requirement in the light of the diversity of the educational provision and academic structures of higher education institutions, and in the light of representations from professional and statutory bodies, most of whose reviews take place between three and five years.
23. The extent to which reviewing subject/programmes within a two-year cycle raises difficulties will become clearer as institutional quality assurance plans are agreed with the agency. Appropriate 'brigading' of cognate subject/programme areas in the agency timetable should help to ensure that a two-year subject/programme review cycle does not inhibit flexibility. However, in some circumstances the possibility of greater harmonisation with the internal arrangements of institutions, of collaboration with professional and statutory bodies, and of a consequent reduction in what is currently a significant area of duplication and overlap, would be enhanced if the review period for individual subject/programme areas was extended. We propose that subject/programmes be 'brigaded' into two-year blocks, each comprising, as far as possible, a number of cognate subject/programme areas. Reviews of single units would normally take place within the stated two-year period, but if, within the agreed institutional quality assurance plan, there is a case for combination of units, for instance, or where a PSB timetable could be more satisfactorily accommodated, then the review period could be extended.
24. We recognise that eight years is a long period between reviews; a good deal can happen in that time. We have, therefore, considered the case for mid-cycle (four-yearly) reviews. We believe that, if these were to be mandatory, the result would be to reduce the eight-year cycle to a four-year cycle. This would greatly add to costs. In any case, providing up-to-date, factual information on provision is an institutional responsibility. Where relevant, and in the light of the procedures of individual professional and statutory bodies, accreditation reports could be placed in the public domain by an institution.
25. Where particular subject/programme area provision is considered to be 'deficient' as a result of a subject/programme area review there would be a requirement for a further review visit in accordance with procedures to be laid down by the agency.
Review teams
26. Each review team would consist of academic, and where appropriate, professional peers, selected by the head of the institution involved and agreed with the agency, against published criteria for the composition of review teams. Each team would be chosen from a list of trained reviewers. Training should be an agency responsibility. The chairman of each review team would be appointed by the agency, and be responsible for the team's published report.
Institutional observers
27. Institutions should be able, if they so wish, to nominate an observer to each review team, but such observers would not be members of a team, nor would they be identified with its conclusions. Their precise role would be agreed following consultation between the agency and institutions prior to the implementation of the new quality assurance arrangements from October 1998.
Collaboration with other funding bodies and with professional and statutory bodies
28. At various points in this report, reference has been made to the need to ensure that, as far as possible, the new quality assurance process takes into account the requirements of other funding bodies, such as the NHS, accrediting, and PSBs. Working collaboratively, groups of cognate PSBs have developed:
- a. common documentation - using the same documentation for two (or more) separate accreditation events;
- b. joint visits - with two (or more) visits taking place at the same time in the higher education institution or unit concerned;
- c. joint processes - leading to a single, integrated report and judgement.
The above components may be present in various combinations in particular circumstances.
29. Some examples of collaboration between PSBs and higher education institutions are given in Annex E. We would expect the agency to give collaboration with PSBs strong encouragement.
A dynamic process
30. The quality assurance process for which the new agency will be responsible, should be sufficiently flexible to take account of the dynamic and diverse nature of higher education institutions, and the changing environment within which they operate. We would expect the new agency, once established, to keep the quality assurance process itself under review and to ensure its continuing appropriateness to the achievement of the purposes of quality assurance set out above. It should operate as cost-effectively as possible in order keep to a minimum the level of external demands on institutions.
31. By the time the new quality assurance agency is fully operational from October 1998, the quality of most (in some cases all) of the education funded by the funding councils will have been assessed through their teaching quality assessment processes. In addition, the internal quality assurance arrangements made by all higher education institutions will have been subject to an HEQC quality audit visit. The quality assurance process for which the new agency will be responsible builds, therefore, on considerable experience within institutions themselves, and within the funding councils and the HEQC. It will have available evidence from quality audit that most institutions have in place internal quality control and assurance procedures. Our proposals for the quality assurance process assume consultation and partnership between the agency and individual institutions concerning its implementation. They assume that significant responsibility for the management of the external process will be shared between higher education institutions and the agency.
32. Our proposals are a blueprint for the future rather than an operational manual. Specifying the details of the new arrangements will be the task of the new quality assurance agency following consultation, development and piloting of the arrangements proposed in this report.
The new agency
33. The constitution which we propose for the new agency follows closely the ideas which we put forward in our first report. They are set out in Annex F.
The functions of the agency
34. We propose that, under service level agreements, the agency will provide reports to funding bodies and accrediting bodies about quality and standards in individual institutions, subjects, programmes and aspects of provision, and about quality and standards in UK higher education in general for both domestic and overseas audiences. The agency will also agree with institutions, individually or collectively through their representative bodies, to:
- a. confirm that their internal quality assurance procedures are working effectively;
- b. identify and disseminate information about innovation and best practice in teaching, learning and student assessment;
- c. provide specified services.
Condition of funding
35. We believe that quality assurance processes in a form that meets the requirements of the funding councils and DENI, and that commends itself to the institutions must necessarily embrace the maintenance and enhancement both of quality and standards. We expect, therefore, that the funding councils and DENI will wish to make it a condition of their grants to institutions that they draw on the agency's services to the extent that it is necessary to allow the agency to meet the terms of its service levels agreements with those bodies.
Funding the agency
Service level agreements
36. We propose that service level agreements (SLA) be embodied in contracts, with associated financial arrangements, between the agency on the one hand and individual institutions (or their representative bodies), and the funding bodies on the other. The agency may also wish to contract, as appropriate, with Government Departments and other bodies (for example, PSBs), and with institutions not in membership of the representative bodies and/or not in receipt of public funds.
37. There would be two types of SLA:
- a. between the agency and the institutions;
- b. between the agency and the funding bodies.
The link between both types of SLA would be provided by a common foundation document, which would be a single statement of the integrated quality assurance process. The two types of SLA would differ in their scope and in the range of the required outputs, including their handling of individual national requirements and approaches across the different parts of the UK. The SLA with the funding bodies would be sensitive to the institutions' responsibilities for academic standards. Both types of SLA would involve payment by the customer for services received.
The parties to the agreements
38. We propose that the SLA between the institutions, individually or collectively through their representative bodies, and the agency would cover:
- a. the integrated process for quality assurance (foundation document);
- b. the outcomes from that process - public reports and information on quality and standards in individual institutions, subjects, programmes and aspects of provision, for a range of domestic (including funding and accrediting bodies and government) audiences and overseas audiences - and including investigative work and summative reports;
- c. quality enhancement activities, including summative and investigative work that builds on the agency's published reports, and addresses sector-wide and subject-wide issues;
- d. Access Courses Recognition.
39. We propose that the second type of SLA, between the funding bodies and the agency, would focus on those issues necessary for the funding bodies to secure their responsibilities for public accountability and public information, namely:
- a. the integrated process for quality assurance (foundation document).
- b. the specific outcomes required by each funding body, in terms of published reports and information, for example:
- i. content, structure and length of the agency's review reports;
- ii. a graded profile at the subject/programme area level; elements of the profile; and criteria for judgements;
- iii. independence and comparability of judgements;
- iv. publication targets and distribution.
Other Services
40. Other services would require specific contracts with individual institutions, organisations, professional and statutory bodies, or Government Departments, as appropriate. For example:
- a. with an institution applying for degree-awarding powers or for university title;
- b. with institutions not in receipt of public funds, for their inclusion in the integrated quality assurance process, and for any other specified services;
- c. with professional and statutory bodies for services related to accreditation;
- d. with Government Departments or other sponsors, in respect of advice or research.
Cost implications
41. We are required by our terms of reference to determine the costs and perceived benefits of the new arrangements by comparison with present arrangements. Such a comparison would, ideally, involve consideration of those costs likely to be incurred centrally by the new agency, and those incurred by institutions. Estimating both is difficult for different reasons. We have not, as a result, felt able to produce, on the basis of the material before us, detailed estimates of relative costs and benefits of present and proposed assurance processes, including possible alternative approaches considered by the Group.
42. Our proposals for quality assurance imply that significant development and piloting should be carried out by the new agency prior to the implementation of the new arrangements in October 1998. Detailed evaluation of the potential costs to the agency of the new arrangements must necessarily follow such development and piloting. Estimating the costs to institutions of external quality assurance is difficult, not only for the above reasons, but also because different factors are taken into account by each institution in calculating such costs. As internal procedures have developed to take account of external requirements, whether of quality audit, quality assessment or of accreditation by professional and statutory bodies, it has become increasingly difficult to distinguish in the total costs of institutional quality assurance those solely attributable to external requirements.
43. We have, nevertheless, considered the likely impact on central costs of our proposals based on the current level of the HEFCE's teaching quality assessment activities, and of the quality audit activities of the HEQC. We have assumed an eight-year review cycle, and that institutions will be given greater scope than at present for agreeing with the agency arrangements for reviews.
44. The HEFCE's first quality assessment visits took place in early 1993 and the first cycle is timetabled to be completed by the end of the year 2001. This will have involved an average of 250 assessment visits each year over the eight-year period. (Initially, assessment in England did not involve universal visiting. This practice was adopted by the HEFCE with effect from April 1995). The first quality audit visits took place in early 1991 under the auspices of the former Academic Audit Unit (AAU) of the CVCP. By the end of April 1997, the first full audit cycle, a total of 235 audits will have been carried out, involving on average 40 audit visits each year. In addition, by mid-1997 a total of seven overseas collaborative provision audits will have been undertaken, four in 1996 and three in 1997 respectively.
45. We have proposed (see paragraph ) that the agency publish an eight-year review timetable. This is similar to the current timetable for teaching quality assessment, but longer than the six-year timetable for quality audit. This factor alone will reduce the annual average number, and therefore cost, of institution-wide reviews by comparison with the current arrangements for quality audit. In addition, our proposals envisage savings from elimination of the current duplication between audit and assessment.
46. Our proposals also assume that, within agreed guidelines, and in the context of an institutional quality assurance plan, institutions will be given much greater scope for agreeing with the agency reviews which cover a wider range of subjects. This could reduce the number of engagements with the agency and the number of reviews conducted each year, depending on the extent to which institutions choose to aggregate subjects for review.
47. There may be some consequential reductions in the central administrative load arising from carrying out fewer engagements. At the same time, the administrative load is likely to increase because of increasing flexibility and negotiation with institutions. Finally, the agency will have development costs, particularly in the financial year 1997-98, as it develops, pilots and consults on the new process.
Transition arrangements
48. Our terms of reference require the Group to provide for a transitional period which, while the momentum of current programmes for assessment and audit is maintained, allows for a staged development of the new arrangements. The new agency will, therefore, have the task of implementing the new arrangements, after piloting, evaluation and consultation.
49. In order to take account of the different timetables and life cycles of the current processes of audit and assessment, implementation should be in stages as indicated below.
50. The continuation of the HEQC audit arrangements will make maximum use of existing internal and external assessment and review reports.
51. The new subject/programme area reviews will start, following piloting and evaluation, from October 1998. This will enable completion of the 1996-98 round of subject assessments in England under currently published arrangements. For English institutions, it is envisaged that units of assessment currently timetabled for 1998-2001 will be carried out within the published timetable.
52. The new agency will take over responsibility for running the current teaching quality assessment activities of those funding councils choosing to contract with the agency for the discharge of these functions, and all HEQC's functions, from 1 April 1997.
53. The new agency will pilot, develop and consult on the new process while continuing to run the existing processes of audit and assessment.
Future work
54. The company's Memorandum and Articles of Association will be finalised.
55. The company's first board of directors will be established along the lines set out in Annex F.
56. The first board will appoint the first chief executive after advertisement of the position.
57. The agency will have as one if its first tasks the negotiation of service level agreements.
58. Staff affected by the establishment of the new agency will be consulted, and arrangements put in place for their transfer to the new agency.
Annex A
Joint Planning Group for Quality Assurance in Higher Education Institutions:
Terms of Reference
Introduction
1. Members of the group will include CVCP, SCOP and HEFCE, COSHEP, the Northern Ireland HEC and HEFCW. Assessors from DENI, DfEE, SOEID and WOED will be invited. SHEFC will be invited to appoint an observer on the basis of the Secretary of State for Scotland's request that they should be appropriately involved in further planning but without commitment at this stage.
2. The independent Chairman, confirmed by the Secretary of State, will be Sir William Fraser.
Aims
3. The aim of the group will be to develop in detail proposals for a new Agency submitted to the Secretary of State for Education and Employment on 20 July, and to produce an agreed implementation plan. The primary function of the new Agency will be to provide a service for assuring the quality of higher education and the standards of programmes and awards for HE institutions in England, Northern Ireland and Wales, and if appropriate, HE institutions in Scotland. The aim is to start the new Agency by January 1997.
4. To achieve this the group will need to:
- draw up the specifications for an integrated quality assurance process which will supersede the current audit and assessment processes carried out by HEFCE's and HEFCW's QAD and HEQC, address issues concerning educational standards and the enhancement of teaching and learning, and ensure the availability of appropriate public reports;
- determine in respect of the new Agency:
- (i) its powers and responsibilities;
- (ii) the composition of its board of Management;
- (iii) its relationship to the HE sectors of England, Wales and Northern Ireland and if appropriate to the HE sector in Scotland, if the outcome of the current review concludes that Scottish HE institutions wish to participate;
- (iv) its relationship and responsibilities to the Funding Councils of England and Wales, to the Northern Ireland office and to other funding bodies and professional and statutory bodies;
- identify a strategy and the timetable to ensure a suitable transition from the existing processes of audit and assessment to the integrated processes of the new Agency;
- determine the method of funding of the services provided by the new Agency;
- determine the supplementary functions which the Agency will provide.
With respect to these aims, the group will need to ensure that appropriate consultation takes place with the HE sectors in England, Northern Ireland and Wales, and of HE institutions in Scotland, that its proposals enable the Funding Councils to fulfil their statutory duties, and that, with respect to academic standards, the arrangements will obviate any need for the Secretary of State to exercise her reserve powers under the FHE Act 1992.
Context
5. In pursuit of its aims the Group will have regard to:
- the conditions set out in the letters to the Chairman of CVCP and the Chief Executive of HEFCE from the Secretary of State for Education and Employment dated 21 September 1995 headed "Developing Quality Assurance in Partnership with Institutions of Higher Education";
- the letter to the Secretary of State for Scotland form the Chairman of the Committee of Scottish Higher Education Principals (COSHEP) dated 21 September 1995 and the reply from the Secretary of State for Scotland dated 22 September 1995 headed "Further Arrangements for Quality Assurance of Higher Education";
- the CVCP proposals submitted to the Secretary of State for Education and Employment on 20 July 1995 documented in "Developing Quality Assurance in Partnership with Institutions of Higher Education";
- the HEFCW submission to the Secretary of State for Wales dated June 1995 - "The Development of Quality Assurance in Wales";
- the SCOP submission to the Secretary of State for Education and Employment dated July 1995.
Consultation
6. Before concluding its final report, the Group will ensure that it has:
- consulted with HE institutions in England, Northern Ireland, Scotland, and Wales and with the representative bodies of the Heads of those institutions, COSHEP, CVCP, SCOP and the meeting of Heads of HEIs in Wales;
- fully consulted with the Funding Councils and that its proposals ensure that the Funding Councils are able to fulfil their statutory responsibilities;
- further consulted with such other bodies, agencies and groups with a direct or indirect interest in higher education as seems appropriate, including HEQC, the professional statutory bodies and the bodies representing academic staff and students.
Reports
7. The Group will report to the Secretaries of State and others on a three-monthly basis and, following the consultation referred to in paragraph 6, will provide the with a final report.
Scotland
8. In the event of SHEFC, COSHEP and the Scottish Office so determining, these terms of reference shall be appropriately amended to ensure participant representation of SHEFC on the group and, all cases, references to England, Northern Ireland and Wales shall be amended to read England, Northern Ireland, Wales and Scotland.
Terms of Reference
Detailed Requirements
In order to meet the secretary of State's requirements, set out in her letter of 21 September 1995, and to achieve the aims set out above, the group will need in particular to:
- a. determine a structure for the governance and operation of the Agency which respects the academic autonomy of institutions and their individual responsibility for standards, but also meets the needs of the Funding Councils in respect of their duties to secure arrangements for assessing the quality of the education for which they provide funds;
- b. establish arrangements for assurance which balance the need for independence and consistency in the process with the need to have regard for institutions' own self-evaluation and review;
- c. determine the costs and perceived benefits of the new arrangements and compare these with present costs and benefits;
- d. provide for a transitional period which, while the momentum of current programmes for assessment and audit is maintained, allows for staged development towards the new arrangements to commence;
- e. continue the dialogue with all relevant institutions in Scotland with a view to the possible emergence of a UK wide scheme;
- f.
- i. determine the appropriate point for the external evaluation of the educational effectiveness of institutions;
- ii. ensure that, in promoting a greater degree of synchronisation between institutions' own departmental reviews and the assessment of subject and programme areas by the Agency, there is sufficient currency in the outcomes of students and employers for up-to-date, comparative information on the quality of cognate areas across the sectors;
- iii. determine an appropriate cycle of reviews of cognate areas;
- g. determine the arrangements for the recruitment, training and deployment of assessors to provide for equity and integrity in the new arrangements;
- h. ensure continuity in many of the functions now provided by HEQC and, in particular, establish how the Agency can offer advice to the Secretary of State on applicants for degree awarding powers and the university title;
- i. ensure that the new arrangements can earn the confidence of the international community;
- j. take steps to allow for the continuity of employment of staff currently engaged in work on standards in HEQC, and on quality in HEQC and the QADs of the Funding Councils;
- k. take into account and provide a steer for the work on standards which will continue parallel to the work of the Joint Planning Group;
- l. allow for a research and development function for the new Agency which might include innovation in teaching methods and the development and dissemination of good practice.
- m. make recommendations on the nature and level of service agreements which the new Agency will need to enter into with the parent bodies and appropriate agreements with its various partners, including the professional bodies throughout the UK and the special needs of teacher education and relations with the TTA;
- n. evaluate the need for an external system of dealing with student complaints.
Annex B
Membership of the Joint Planning Group
Chairman
- Sir William Fraser
Members
- Nominated by HEFCE:
- Professor Brian Fender, Chief Executive, HEFCE
- Professor David Watson, Director, University of Brighton
(formerly Chairman of HEFCE Quality Assessment Committee) - Nominated by the Chief Executive of HEFCW:
- Mr Mike Laugharne, Head of Quality Assurance Division, HEFCW
- Nominated by SHEFC:
- Professor John Sizer, Chief Executive, SHEFC
- Nominated by CVCP:
- Professor Gareth Roberts, Vice-Chancellor, University of Sheffield
(currently Chairman of CVCP) - Mr John Stoddart, Principal, Sheffield Hallam University
(currently Chairman of HEQC) - Nominated by SCOP:
- Dr Martin Gaskell, Director, Nene College, Northampton
(currently Chairman of SCOP) - Nominated by COSHEP:
- Professor John Arbuthnott, Principal, University of Strathclyde
(formerly Convenor of COSHEP) until June 1996. - Professor Richard Shaw, Principal, University of Paisley
(currently Convenor of COSHEP) from July 1996. - Nominated by NIHEC:
- Professor Peter Bush, Vice-Principal, Glasgow Caledonian University
Assessors
- Nominated by DfEE:
- Ms Caroline Macready
- Nominated by SOEID:
- Mr Tom Kelly until April 1996
- Mr Ian Walford from May 1996
- Nominated by WOED:
- Mr Bob Evans
Joint Secretariat
- Nominated by HEFCE:
- CVCP and HEFCE, with support from HEQC
Annex C
Joint Planning Group Working Party
Terms of Reference
1. To develop a framework for a single process of quality assurance for higher education guaranteeing commonality of approach to institutions, and integrating issues of subject-level activity, institutional effectiveness and standards. In respect of each institution, the process should contain three elements as follows:
- a. Rigorous, self-critical internal review with appropriate external participation. Any such review, which could be subject/programme related or an institutional overview, would be commissioned by and report to the head of the institution.
- b. Agency-managed, external subject/programme area review which would:
- i. take into account the reports of subject/programme reviews at a. above;
- ii. guarantee openness, independence of judgement, consistency of process and comparability of outcomes across the sector;
- iii. lead to published reports;
- iv. address issues of academic standards and quality enhancement
- v. involve external assessors appointed by, and reporting to, the agency.
- c. Agency-managed, external institution-wide review, examining overall provision, at set intervals, and in a form which might cover central services and aspects of standards, and have regard to the accumulated results of the reviews described in a. and b. above.
2. The framework should build on the proposals outlined in the interim report of the JPG, including reference (at paragraph 27 to 29) to existing and planned co-operation between the funding councils and HEQC on the one hand, and professional and statutory bodies on the other.
3. In developing the framework, the Working Party should consider the following questions:
- a. what criteria should the new agency use to satisfy itself that an institution's internal quality assurance processes are effective, valid and reliable; and how should those criteria be applied?
- b. in what way(s) could the new process reduce demands on institutions?
- c. what steps should be taken by October 1998 to address the matter of academic standards within each element of the framework?
4. The Working Party should complete its work in time for the JPG Meeting on 14 June 1996.
Membership
5. The Working Party will be chaired by Professor David Watson, a member of the JPG.
6. It will include members of staff with responsibility for quality assurance at a senior level in higher education institutions in the United Kingdom as follows:
- a. Mrs Jennifer Bone
Pro Vice-Chancellor
The University of the West of England - b. Dr Ron Emanuel
Quality Assurance Office
Glasgow University - c. Dr David Timms
Head of Quality Support
Bath College of Higher Education
7. Dr Paul Clark, Director of Quality Assessment, the Higher Education Funding Council.
8. Mr Peter Williams, Director of the Quality Assurance Group, the Higher Education Quality Council.
9. The Working Party will be serviced by David W Parry of the JPG Secretariat's support group.
Annex D
Academic standards
1. Throughout our discussions we have found ourselves attaching different meanings to the term 'standards', and we have been conscious of the request in our terms of reference that we should give a 'steer' on this issue. The term 'standards' is sometimes used for comparison over time, as in the question 'is the standard required for a first class honours degree in engineering from the university of X higher or lower than it was twenty years ago?' It is sometimes used to compare degrees of different institutions; and it may also be used to indicate what an institution is seeking to achieve in terms of the quality of its degrees, the educational experience which it provides, what it expects of students, and the level of student attainment within the institution. We have kept in touch with the work of the HEQC's Graduate Standards Programme (GSP), noting that the main focus of attention of the GSP has been on output standards, defined as '... the nature and levels of student attainment associated with the award of an undergraduate degree ...' Because of this variety of meanings, whenever the word 'standards' appears in this report, it is used in this last sense, unless otherwise indicated.
2. Our proposals invite each institution to indicate the way in which the academic standards students are expected to attain in each subject/programme area under review are identified by the provider unit involved; the way in which the attainment of those standards is monitored; and the form in which their attainment is described. In addition, the new agency will, by means of institution-wide review, consider the arrangements made by each higher education institution for maintaining academic standards in the institution as a whole.
3. The matter of the 'appropriateness' of particular standards was raised in our first report (paragraph 25 of that report). Our approach to this assumes that the standards which are deemed appropriate for the award of a particular qualification are a shared responsibility involving those who design and validate programmes, those who teach and assess students, and those who award the degrees or diplomas. In our view, it would be neither appropriate nor desirable for the new agency to define what the standards of degrees and other awards should be in terms of curricular content and assessment criteria.
4. Higher learning is a complex activity and the output standards associated with it are necessarily rooted in, and realised through, the expert practice of academics and other professionals and the learning process engaged in by students. Academic standards must always remain somewhat tacit, implicit and open-ended. There will always be a need for opportunities to exchange and confirm peer judgement.
Annex E
Higher education and professional and statutory bodies
1. Work has already been undertaken, by the HEQC, by the HEFCE and by the Welsh Funding Councils, on links between the higher education sector and professional and statutory bodies. The HEQC has, for example, carried out a mapping exercise of the accreditation criteria prescribed in respect of the assurance of quality and standards across some 25 professional and statutory bodies, suggesting that a good deal of collaboration already exists between PSBs in their interaction with higher education institutions. The Quality Assessment Division of the HEFCE has developed models for closer liaison with a wide range of PSBs, and has put arrangements in place with a number of them.
2. One example of what might be achieved is described in the report of the Non-Medical Health Care Education and Training project, which was undertaken by the HEQC, and involved the NHS Executive, relevant PSBs, and higher education institutions providing such education and training. Amongst the project's main recommendations are that these stakeholders should:
- a. adopt a single core quality specification covering those areas where there is a shared need for evidence;
- b. agree that the specification is incorporated into the quality documents which are already widely used by higher education providers in developing and monitoring course provision;
- c. use these quality documents, modified where necessary, as the primary evidence base in their interactions over quality.
3. Since its establishment, the Quality Assessment Division of the HEFCE has considered the possibilities for joint assessment/accreditation visits, or at the very least sharing documentation with PSBs. This has involved reciprocal observation of assessment and accreditation (1993-95); a paper summarising background, processes and a tentative model for joint assessment/accreditation which was sent to PSBsin June 1995; and a joint accreditation/assessment visit involving the Institution of Chemical Engineers in February 1996. This last involved assessment and accreditation teams undertaking separate, but concurrent, processes using shared documentation and information; one assessor also acted as an accreditor. More joint accreditation and assessment visits are planned for the 1996-98 assessment period, involving collaboration with a number of PSBs. The form of collaboration varies, but one professional body has suspended its accreditation programme for the duration of the 1996-98 quality assessment programme period, and will use the reports of assessment visits as a means of ascertaining whether an accreditation visit is necessary.
4. The Welsh Funding Councils have undertaken a number of initiatives including, inter alia, the RIBA, CCETSW (Wales), and the Welsh National Board for Nursing, Midwifery and Health Visitors. These initiatives have variously involved, or are planned to involve, observation of each others processes; contemporaneous Funding Council and PSB visits; overlap in review team membership; and sharing of documentation.
5. Finally under this heading of collaboration involving professional and statutory bodies, the HEQC is providing professional and financial support for a pilot project being conducted under the auspices of the Engineering Professors' Council, on the development of a Developmental Integrated Quality Management Framework. The project is supported by all the main engineering institutions, by the HEQC and by the HEFCE, and involves the design and testing - in six engineering departments in England and Scotland - of a draft specification for the Framework, and its alignment with professional accreditation and quality assessment requirements.
6. All these initiatives provide evidence of what is possible,although we would not wish to minimise the difficulties involved in achieving agreement between the various stakeholders. These difficulties include, for example, the constraints placed on bodies with statutory responsibilities, and the possible implications of introducing new programmes requiring professional body accreditation outside a scheduled subject/programme area review cycle.
Annex F
The constitution and membership of the new agency
The constitution of the new agency
1. We propose that the framework for the constitution of a new independent agency be as follows.
- a. It would be a company limited by guarantee, and registered as a charity.
- b. The objects of the company would include:
- i. the promotion and maintenance of quality and standards in higher education provided by UK universities and colleges;
- ii. the enhancement of teaching and learning, and the identification and promotion of innovation and good practice in teaching and learning;
- iii. the provision of information and the publication of reports on quality and standards in higher education provided by UK universities and colleges; and
- iv. the provision of advice to government as requested.
Membership of the company
2. We propose that the members of the company, to whom the company's board of directors will be accountable, be the representative bodies of the heads of higher education institutions.
The board of directors
3. We propose that there be 14 directors on the board of the company: four directors nominated by the HEIs' representative bodies; four directors nominated by the funding bodies (the funding councils and DENI); and six 'independent' directors representative of the wider community with an interest in quality and standards in higher education.
4. No director should be a salaried staff member of the HEIs' representative bodies or of any of the funding bodies (the funding councils and DENI) nominating directors to the board.
6. Directors would not be representatives of nominating or other bodies but would be appointed as individuals in their own right.
7. Directors' terms of office would be limited, as would the number of times they might be re-appointed.
8. After the establishment of the first board, subsequent appointments to the board, including that of the Chairman, would be made by the board itself, retaining the same overall balance of membership as the first board.
Constituting the first board of directors
9. We propose that the process for constituting the first board of directors be as indicated below.
- a. The first Chairman of the board would be identified and invited to accept appointment by the Chairman of the Joint Planning Group, after consultation with the members of the Group.
- b. The Chairman of the Joint Planning Group and the Chairman-designate of the company's board would receive nominations from the HEIs' representative bodies and the funding bodies (the funding councils and DENI) for filling the eight places available to them. There would be consultation between the representative bodies and the funding bodies before names are put forward in order to ensure, as far as possible, a spread of experience.
- c. The Chairman-designate and the board members-designate would then invite suggestions for filling the remaining 'independent' places in the board's membership, and the approval of the members of the company to appointments would be sought.
- d. The first board thus constituted would take steps to appoint the first Chief Executive after advertisement of the position.
- e. The first board would be established on a 'shadow' basis as soon as possible.
Balance of membership of the board
10. In respect of both initial and subsequent appointments to the board of directors, we believe that it will be important to ensure that a balance is achieved, and maintained, among the various interested parties. With this in mind, we propose the appointment by the board of a representative sub-committee, chaired by a member of the board, to be responsible for ensuring wide consultation concerning the appointment of independent members to the board. In respect of nominees from the HEIs' representative bodies and the funding bodies, those appointed would include practising academics, and should reflect, as far as possible, the diversity of the UK higher education sector.