University of the Arts: National Student Survey analysis

The challenge

At the end of every academic year, all final year degree students at the University of the Arts London (UAL) complete a survey designed to gather opinions of the quality of degree programmes. Aside from 1 to 5 scoring of different aspects of their learning experience, the survey also offers respondents the opportunity to comment on their experiences and make suggestions for improvement, and it was this qualitative data that was examined during this study.

UAL was interested in exploring the qualitative data contained with the annual survey. In particular, UAL was keen for EdComs to conduct a thematic analysis across the data set which explored differences and commonalities with the findings from the quantitative data. The end goal was for UAL to use the findings to evolve its approach to improvement at the corporate level.

Our approach

A number of provisional key thematic areas that were likely to emerge were agreed with the client. The most appropriate methodology to analyse such a number of verbatim comments was to perform a simple content analysis on the text. To do this, we generated a number of keywords related to each thematic area that were used for a first level search and sifted through the dataset. Each comment was double-checked to ensure that it was relevant to the theme and sub-theme to which it was assigned.

However, given the plurality of ways in which opinions were expressed by respondents and the large number of textual inaccuracies, a large majority of comments could not be categorised through automated keyword searches. The judgement of the analyst was required. We therefore individually analysed the remaining comments and allocated each of them to a thematic area and to a specific sub-theme. The data was then coded by topics within each of the identified sub-themes.

An interim analysis of the findings of the research was presented at a meeting half way through the project. During this meeting, additional topics and sub-themes identified from the analysis and added to the original list of areas were presented. UAL discussed with EdComs the areas that were of particular interest for reporting and subsequently sub-themes and topics on which to focus were agreed.

Outcomes

We produced a report and summary tables, as well as giving a presentation to the senior management team at UAL. The report is being used to take urgent practical action on the most pressing issues emerging from the analysis.