1. This report describes the findings of a research study to inform the review of the National Student Survey (NSS). The research consisted of running a pilot questionnaire with students who had completed the main NSS. The pilot combined existing NSS questions with new and revised questions developed as part of the review. The aim was to examine whether the questions were valid and reliable across a range of students, provision and institution types.
2. As the research focuses solely on testing the robustness of the questions, the actual student responses have not been analysed.
3. This report is part of a series of reports published as part of the review of information.
4. Over 16,500 students participated in the pilot project, providing a representative sample of the students who participated in the main 2015 survey.
5. All questions were broadly understood by participants and no significant differences in understanding were found when analysed against different characteristics (disability, ethnicity, institution type, gender and subject). However, a small number of questions displayed higher use of the ‘Neither agree not disagree’ and ‘Not applicable’ responses. The reasons behind this should be considered further.
6. A negatively worded question was included in the pilot to assess the feasibility of using such questions to test whether students are considering their answers, as previous HEFCE analysis of the NSS shows an increasing proportion of students providing the same answer for all questions (known as ‘acquiescence bias’ or ‘yea-saying’). This question was the least well understood, albeit at a low proportion of 0.7 per cent responding ‘I do not understand the question’.
7. The extent of yea-saying in the pilot was examined both including and excluding the negatively worded question, and in both cases found to be lower than in the main NSS.
8. No action is required.