You need cookies enabled

Cookies

You need cookies enabled

Executive summary

Purpose

1. This report describes the findings of a research study to inform the development of the 2017 National Student Survey (NSS). The research involved running a pilot questionnaire, which was offered to students on an opt-in basis after they had completed the 2016 NSS. The pilot combined existing NSS questions with new and revised questions developed following a consultation held as part of the review of the provision of information. The aim was to examine whether the revised questionnaire was valid and reliable across a range of students, provision and institution types, and whether the survey worked as a whole.

2. The research tests the robustness of the questions and whether the new question scales work. The actual responses of the students have not been analysed.

3. Since the online version of the NSS may be taken on multiple types of device, the pilot survey also tested alternative interfaces and response scales in an attempt to find the most suitable layout for each. Research on the different interface and response scale options is included in analysis of the cognitive testing of the 2016 pilot NSS carried out by IFF Research Ltd. The investigation of the new interfaces and response scales is therefore not covered in this report.

4. This report belongs to a series published as part of the review of information about learning and teaching, and the student experience.

Key points

The questions are all broadly relevant to most people

5. Questions 14 (‘Good advice was available when I needed to make study choices’), 20 (‘I have been able to access subject specific resources […] when I needed to’) and 26 (‘Students’ academic interests on my course are effectively represented by the Students’ Union […]’) have the highest proportion of ‘not applicable’ responses. Questions 25 (‘It is clear how students’ feedback on the course has been acted on’) and 26 are answered the least positively overall.

The new question scales are working as expected

6. When split into eight groups, based on students’ responses, the eight question scales are identified as the most appropriate groupings for the questions.

7. Question 26 does not display particularly high correlation with any other question, although it is most closely related to Question 25.

8. This analysis excludes ‘not applicable’ responses, and thus over half of the distance learners who responded. When their responses are included, the scales no longer work as expected, meaning that the scales may not work in the same way for distance learners.

Removing the students’ union question improves the internal validity of the survey

9. All scales are found to have acceptable levels of internal consistency. However, reliability analysis suggests that the final question scale, ‘Student voice’, would work better if Question 26 was removed.

10. The analysis implies that removing any other question from the survey would only weaken the question scale in which it sits.

The ‘Learning community’ and ‘Student voice’ question scales are answered much less positively by distance learners than all other students

11. Distance learners are between 11 and 33 percentage points less satisfied than other students for Questions 21 to 26. Outside these two scales, distance learners’ level of agreement does not differ greatly from that of other students, except for ‘Assessment and feedback’, where (in line with the main survey results) distance learners have a considerably greater level of agreement.

12. In their free text responses, many distance learners mentioned that parts of the survey were not relevant to them. Over a quarter of distance learners students responded ‘not applicable’ to Question 26.

Action required

13. No action is required.

Date: 6 October 2016

Ref: 2016/27

To: Heads of HEFCE-funded further education colleges, Heads of HEFCE-funded higher education institutions, Heads of universities in Northern Ireland
Heads of higher education institutions in Scotland, Heads of higher education institutions in Wales, Heads of further education colleges in Wales, Heads of alternative providers in England, Heads of students’ unions and guilds at UK institutions, Students’ union officers and student representatives, Employer bodies, Professional, statutory and regulatory bodies, Other bodies with an interest in information about higher education, Careers advisers, Schools, sixth form colleges and their representative bodies, Other bodies and individuals involved in supplying advice and guidance to prospective higher education students

Of interest to those
responsible for:

Student information; Quality assurance; Admissions; Liaison with schools and colleges; Widening participation and marketing

Enquiries should be directed to:

Natasha Heaney, tel 0117 931 7395, e-mail n.heaney@hefce.ac.uk