Monitoring is essentially concerned with accountability and performance against milestones, and will be supplemented through intelligence and information gathered in the ongoing account management relationship between each consortium and HEFCE.
The evaluation is predominantly concerned with evidencing outcomes and impact. It will therefore need to consider the key factors that have influenced whether consortia are successful, and to ascertain the immediate and potential longer-term economic, social and cultural benefits.
We are evaluating the programme at a national level. Consortia will evaluate their projects on a local level, and should:
- adopt a locally determined approach as set out in their operating plans
- adapt to take account of national evaluation team recommendations
- track their individual target learners through the Higher Education Access Tracker (HEAT) or a similar mechanism.
What the evaluation will assess
The evaluation will assess how successfully the NCOP programme meets its aims. Read more about the aims of the NCOP.
The evaluation will also consider:
- changes to the education sector such as: level 3 qualification reform, emerging schools policy, FE area reviews and growth in apprenticeship routes
- alignment with, and impact on, the broader coverage of outreach activity and investment through access agreements.
This requires a robust and wide-ranging evaluation that builds on the recommendations from independent research for HEFCE in 2015.
Objectives of the evaluation
- Accountability for public funds
Ensure funding is spent according to plan.
- Impact assessment
Enable an overall assessment of the difference to the student, society and economy outcomes that can be attributed to NCOP funding.
- Return on investment
Demonstrate the value of any impact at individual, provider, consortium, and national levels.
- Benchmarking of outcomes
Identify differences between consortia approaches to see if these differences are associated with differential participation rates and progress.
- What works, why and in what circumstances
Establish the causal effect of different types of NCOP interventions.
How will NCOP be evaluated nationally?
HEFCE will evaluate NCOP nationally through:
- Regular monitoring and account management by HEFCE policy advisers
- Quantitative analysis of national administrative data sets to measure progression into pathways at Key Stage 5, and HE entry and progression rates. This includes econometric analysis on the return on investment.
- Longitudinal tracking of cohorts involved in NCOP. HEFCE will work with HEAT and other providers to evaluate the impact of particular interventions on NCOP learners.
HEFCE has commissioned the following work to contribute to the evaluation:
- A formative and capacity building evaluation with summative reports at key points in the programme. This will include support for local consortia in developing and implementing high quality evaluation plans that use robust approaches to evaluation of outreach activity, and a systematic review of the evidence emerging from local consortia. This work will be conducted by CFE Research in conjunction with Sheffield Hallam University.
- An impact evaluation that assesses the consequential changes resulting from the diversity of NCOP interventions, by using a range of quantitative, experimental methodologies. This work will be conducted by CFE Research in conjunction with the Behavioural Insights Team, University of Sheffield and the London Schools of Economics and Political Science.
We will use emerging findings to:
- provide evidence of progress and success
- support funding decisions for year 3 and 4 of the programme
- support evidence-based approaches to refinement of effective practice at the local level.
We will work closely with the evaluators to ensure timely and effective communication with consortia and stakeholders throughout the evaluation.
Table of key elements of the evaluation of NCOP
How to submit monitoring information
Consortia should submit monitoring information by downloading the appropriate forms from HEFCE’s extranet and submitting them through HEFCE’s extranet.
Consortia should submit monitoring information by the dates listed below.
Which information should be submitted?
There are three types of monitoring return, each requiring different information.
|Monitoring return type||
Quarterly financial updates to ensure that financial profiles and budgets are on track.
The same as a financial update, plus bi-annual updates on consortias’ progress against their project plans, objectives and milestones.
End of year monitoring
The same as a financial update, plus a full review of the operating plan.