University of East London
By using a number of methodological approaches, this project will analyse students’ skills development. It will include academic skills tests, and self-assessment of confidence measures with reference to academic study and career development.
Partners: Brunel University London; University of Roehampton
Project methodologies: Surveys; Standardised test; Mixed methods
Omnibus learning gain study
The Omnibus learning gain study takes a mixed methods approach to the development of inclusive learning metrics across three universities in the M25 region: University of East London (UEL), the University of Roehampton and Brunel University London.
Aims and objectives
- To establish a definition of learning gain that is relevant to UEL, Roehampton and Brunel’s students and staff and contributes to the development of meaningful national definitions
- To develop a robust and valid set of indicators of learning gain, concentrating on academic and employability skills
- To establish an approach that is sustainable across the institutions once additional funding ends.
Experiences and outcomes
From the student perspective:
Students are now more aware of how metrics like these are produced, tested (on them) and analysed.
Recruiting participants has been challenging, having to negotiate access through academic colleagues, who are quite rightly mindful of any activities which may impact on teaching time.
Administering the survey (paper based hand out/hand back), the team are conscious of the need to revise question wording since much of the language of established instruments is distant in time and space from educational experience of learners.
Focus groups provided us with a wealth of learning on their engagement, confidence and need for cognition. A few groups were given micro lectures on the analysis of the data as part of their HE experience. We will learn more about this in the second year as we intervene more in a teaching sense.
From the institutional perspective:
Feedback on student learning might be perceptible through lectures taking on board findings before designing and delivering subsequent years' teaching approach. However, there is a lot already going on in the Teaching and Learning space so attribution would be difficult. By producing wide staff and student participation - as well as an engaging dataset - the project has informed discussions about the TEF, student engagement retention and attainment.
Our partners at Brunel have used the opportunity to measure the impact of their weeklong HeadStart enhanced induction for students with widening participation characteristics.
Second wave data analysis is underway, so we should have a lot more to report following the project's completion and staff engagement.
We have tested the Academic Behaviour Confidence (ABC), partial United Kingdom Engagement Survey (UKES) and Need for Cognition (NfC) scales on a relatively large scale across very different student populations.
This has allowed us to understand the dynamics of engagement, cognitive energy amongst students with sub-annual resolution. We will create an omnibus instrument out of this.
Quotes about the impact of the project
"This project challenges us to contribute socially and intellectually just learning metrics that work across our hyper-diverse learning environments while maintaining the support of our participants."
Accessing student institutional data required persistence as HEIs are differentially capable with data. We recommend establishing data relationships early on and adapting to the gaps and the annual pattern of HESA and other returns.
It is a challenge to maintain high student engagement, and response rates from beginning to end of the academic year. HEIs are recommended to find ways which are resilient to student attendance drop-offs, e.g. compulsory electronic completion.
Recruiting students for focus groups was difficult, but made easier when working in partnership with recent students.
- Academic Behaviour Confidence
- Need for Cognition
- United Kingdom Engagement Survey (partial)
- Academic Motivation Scale
Publications and forums
Our work has been publicised mainly within partner universities for staff engagement. For example:
- two papers to UEL’s Learning & Teaching Committee
- presentation to Directors of Learning Teaching (UEL).
Academic publications since 2016: Presentation – Forum for Access & Continuing Education Conference (Queen’s University, Belfast)
Contact the case study author:
- Dr Fuad Ali, email email@example.com, tel 0208 223 4768