Dear Vice-Chancellor or Principal
Invitation to submit expressions of interest in piloting and evaluating measures of learning gain
1. This letter invites expressions of interest for funding for projects that will pilot and evaluate measures of learning gain in higher education in England. The deadline for expressions of interest is noon on 5 May 2015. Expressions of interest should be submitted using the template at Annex A.
2. Learning gain can be defined as the ‘distance travelled’ by students during their studies, demonstrable by an improvement in knowledge, skills, work-readiness and personal development between two points in time.
3. Over the next three years HEFCE intends to make available a minimum of £1 million to support projects to pilot and evaluate measures of learning gain.
4. The main aims of this pilot exercise are:
- To test and evaluate different methodologies currently used to measure and assess learning in higher education, and the potential suitability and scalability of these methods in England.
- To promote partnerships between institutions, departments and disciplines to facilitate the transfer of expertise and new ways of measuring learning gain more widely across the sector.
- To build knowledge about, and capacity for, measuring learning gain within the higher education sector in England.
- To facilitate dialogue across the sector and with experts, and thereby to develop a shared understanding to underpin future developments (such as the development of national measures).
Purpose of the funding
5. As part of our commitment to supporting excellence and innovation in teaching and learning, as outlined in the HEFCE Business Plan, we are working with the sector to develop better ways of capturing excellent educational outcomes, including new approaches to measuring students’ learning. Developing our understanding of student learning is integral to ongoing debates about the quality and impact of higher education, and how we evidence the value of investment in it.
6. The 2014-15 grant letter from the Department for Business, Innovation and Skills (BIS) asked HEFCE to ‘consider whether there are better indicators, such as measures of student engagement, to provide information on what a high quality student experience looks like’. An expectation of progress towards developing and testing new measures of learning gain is also set out in the 2015-16 BIS grant letter to HEFCE.
7. Learning gain has been defined and conceptualised in a number of ways. For our purposes, we define learning gain as the ‘distance travelled’: the improvement in knowledge, skills, work-readiness and personal development demonstrated by students at two points in time. There has been increasing interest, both nationally and internationally, in this concept in higher education, influenced by a number of key publications and programmes of work.
8. Our work on learning gain is part of a broader programme of work that is intended to improve students’ educational outcomes, and to provide insight to improve teaching practice and enhance capability (at an institutional strategic, department and individual academic level) across the sector. The development of a better range of indicators of the outcomes of higher education also has the potential to enhance student decision-making and to demonstrate more clearly, to Government and to students, the value of their investments in higher education.
Methods of evaluating learning gain
9. As the first phase of the project,we commissioned RAND Europe to undertake a critical evaluation of the range of assessment methods and tools for the measurement of learning gain, to inform future work in the context of English higher education. That evaluation provides a resource to help institutions who are interested in undertaking pilot projects, and will shortly be published on the HEFCE website.
10. The next stage of the project will comprise piloting and testing these groups of methods (as outlined below) within the English context, alongside an ongoing evaluation. We are seeking to support both:
- cross-sectional approaches (looking at different cohorts at the same point in time)
- longitudinal approaches (following the cohort over the duration of their studies).
11. We aim to pilot and evaluate the five groups of methodologies that have been identified as currently in use to assess learning gain:
- Standardised tests – measuring the acquisition of certain skills, which may be generic or specialised. They could be administered to students either as part of their formative or summative assessment for their degree or as an additional exercise alongside the course. These tests could be discipline-specific, or focus on generic skills.
- Grades – measuring the progress in students’ achievement by comparing the difference between grades at two points in time. This could include: comparing the difference between actual grades at two points in time, using a standardised measure of these grades (such as the grade point average) as a comparison, or using a set of grades (standardised or not) to make predictions on future grades.
- Self-reporting surveys – asking students to self-report the extent to which they consider themselves to have gained knowledge and developed skills through a survey that is administered at a number of points throughout their degree programme.
- Mixed methods – using a range of tools and indicators to track improvement in performance, for example through a combination of grades, student learning data and student surveys, or a combination of grades and a test to predict students’ performance, which could be presented as akin to measuring learning gain.
- Other qualitative methods – including encouraging students to reflect on their learning, acquired skills and skills gaps as well as to stimulate a formative exchange between students and their tutors.
12. We expect to provide additional support for the pilot projects from experts in the field. More details will be available at the first Capacity Building and Evaluation Network meeting in September 2015. The final stage of the project will be a concluding comparative evaluation.
Eligibility and assessment criteria
13. We plan to provide a minimum budget of £1 million, over three years, to support projects that pilot and evaluate one or more of the approaches outlined above. We expect a wide range of proposals in terms of focus, scale and scope, but would particularly welcome, and will give priority to, proposals that include more than one institution, department or discipline.
14. The funding available to individual projects is intended to contribute to the costs of developing and implementing the proposed pilot projects. However, we would highlight that funding is not generally available for capital projects, capital elements within projects, or significant staff costs.
15. Funding is available to higher education institutions (HEIs) and further education colleges (FECs) that are directly funded by HEFCE. Individual institutions can lead only one project, but can be a partner in any number of other proposals. This flexibility is to provide opportunity for institutions to address different approaches, to allow for collaborative projects, and to encourage innovative, high impact activity.
16. A panel will be convened to consider the proposals, consisting of staff from HEFCE, BIS and the Higher Education Academy, academic experts working in the field of learning gain, and student representatives. This panel will use the assessment criteria, as outlined below, to assess all applications and select the most appropriate for funding, and will take steps to ensure that pilot projects covering all five groups of methodologies are included. Applications will be assessed using the following criteria:
- the extent to which the proposal has the potential to demonstrate that it will deliver a valid measure or measures of learning gain
- the extent to which the proposal has demonstrated innovation, and the potential viability, suitability and scalability of the method or methods in England
- value for money, including likely sustainability
- whether the pilot project will promote partnerships between institutions, departments and disciplines, and where relevant with employers, in order to facilitate the transfer of expertise.
17. Where appropriate, we will encourage and broker collaborations, between institutions that have made separate applications but who wish to undertake projects using the same methods, and to ensure that there is a balanced portfolio of methods across all selected projects.
18. We also welcome expressions of interest from institutions which are not putting forward a detailed proposal but would like to be considered to be involved in a pilot in any one (or more) of the groups of methodologies noted in paragraph 11. Institutions should indicate which group of methodologies they would be interested to take part in. As part of the assessment process, we will contact institutions that have indicated an interest in order to broker collaborations and involvement.
19. To support the capacity-building aims of the project and to ensure that we maximise the evaluative insight gained from the pilot activity, we will seek to encourage communications between the award holders by convening a network, to serve as a vehicle for the sharing of progress and insights from the pilot projects. This network, which we hope will meet twice a year, will help to develop connections between award holders and UK and international experts in the field of learning gain, and with the project steering group. Applicants must confirm in their application that they are willing to participate in the activities of the network, and are committed to an ongoing involvement in evaluation. Projects will also be required to produce a final report and make pilot data available to HEFCE for analysis.
20. We anticipate that pilots that involve a cross-sectional approach will run for approximately a year, while those using a longitudinal approach will last for up to three years.
Application process, selection and timescales
21. Expressions of interest should be submitted by email to email@example.com by noon on 5 May 2015, using the template at Annex A (available on the HEFCE website alongside this letter). Expressions of interest will then be considered against the selection criteria by a review panel, and successful applicants will be notified by the beginning of June 2015. The approximate timetable for the initial stages of the piloting and testing stage of the project is below.
|27 March||Call for expressions of interest released|
|5 May||Close of call for expressions of interest|
|June||Successful proposals notified|
|September||First Capacity Building and Evaluation Network meeting|
Freedom of Information Act 2000
22. HEFCE is subject to the Freedom of Information Act 2000, which gives a public right of access to information held by a public authority. This may mean that information supplied to us in the expressions of interest is subject to disclosure if a valid request is made to us. We will comply with such requests in accordance with the legislation and our own policies.
23. Institutions can, if they wish, provide potentially sensitive information (such as information relating to commercial interests) in a separate annex attached to the application form. This will highlight to us that there are concerns over disclosure. With annexes, the proposal must not exceed the maximum length as stated in the application template. Where we consider it to be appropriate and practicable we will seek the views of applicants before disclosing this information. The applicant acknowledges that information provided in the annex is of indicative value only and that HEFCE may nevertheless be obliged to disclose this information. Our assumption will be that all information in the main application documents can be disclosed on request.
24. If you have queries, please contact Ross Hudson, (tel 0117 931 7374 email firstname.lastname@example.org)
Professor Madeleine Atkins