Questionnaire development

This involves determining the rationale for the questionnaire and how it will answer the research question. It also involves consideration of the type of scale to be used and the response format. The questionnaire’s items are generated from a review of the literature.

The questionnaire was presented at a directorate research governance meeting where there were patient representatives. This helped frame the questionnaire layout and the questions asked. Patients were asked to rate their experiences of the care they had received. Items were drawn from empirical evidence concerning safety from the patients’ perspective – for example, questions focusing on patients’ experiences of and satisfaction with services. Account was taken of the National Inpatient Survey (Care Quality Commission 2014) when selecting items. This approach was informed by Rathert et al (2011), which used the validity of items in the Picker Patient Experience Questionnaire (Jenkinson et al 2002) to strengthen and determine items used in a new measurement tool.

Respondents would therefore respond to questions that resonated with them and were highly relevant to this research. Rattray and Jones’s (2007) approach of using open-ended questions early in the design stage was followed to generate other topics for the tool.

Pilot

This stage tests the content and validity of the tool. Ten inpatients from a range of demographic backgrounds (age, gender, ethnicity), who were more than 18 years old and who were due to be discharged within 48 hours were purposively selected.

Rattray and Jones’s (2007) framework recommends pilot testing questionnaires using cognitive interviews, to test face and content validity. Content validity can be determined by asking either an expert panel or a relevant group of participants – in our case, patients who had experienced care – whether the questionnaire captured important items. The Question Appraisal System (Willis and Lessler 1999) was used to look for those questions likely to be erroneous.

Respondents reported that the questionnaire was easy to understand, quick to complete and covered issues of safety that were significant for them. They all completed the questionnaire reflecting on their entire admission. They did not focus on one single event; however, some made comments about before-hospital experiences. We therefore modified the questionnaire to ask respondents to give answers only relevant to their current admission. The Likert scale used was reduced from ten to six ordinal responses to decrease the burden on respondents. Neutral responses to questions were removed, to encourage respondents to commit to either a positive or a negative response.

Cross-sectional study

This uses exploratory factor analysis to test the reliability of the tool. There is no definitive guidance on sample sizes for factor analysis, with rules proposed including 200 subjects or more or a ratio of ten subjects per item (Comfrey and Lee 1992). Parameters that remain unknown until the data have been analysed – for example, items per factor, communalities and item loading magnitudes – will affect the sample size required (Costello and Osborne 2005).

We considered a sample size of 150 to 200 would be achievable and large enough to establish whether the results of the study had statistical significance.

Patient characteristics

Tables 1a to 1e show important characteristics of the participants: age, gender, ethnic group, length of stay, mode of admission and IMD scores. The average age of participants was 56, there were similar numbers of female and male respondents, their mean length of stay was ten days, most respondents (91%) were emergency admissions and about two-thirds were of white British ethnicity. More than half of all participants (58%) were from the two most deprived quintiles.

Likert scale responses

Table 2 shows participants’ responses to each of the 13 items of the questionnaire. Patients were much more likely to agree than disagree with item statements. Patients most strongly agreed (70%) with item six – ‘I could have a member of my family or close friend for support when I wanted them’; they most strongly disagreed (13%) with item one – ‘I was allocated a bed straight way’.

Table 2.

Responses to the items of the KPSM

Of the 78 pairwise correlations, 27 (35%) were strong, two (3%) were weak and the remaining 49 (63%) correlations were moderate to fairly strong. The two weakest polychoric correlations occurred between ‘I was allocated a bed straight away’ and ‘Staff were familiar with equipment’ (0.24) and ‘Staff were familiar with procedures’ (0.26). Correlation between ‘Staff were familiar with equipment’ and ‘Staff were familiar with procedures’ was 0.88. There was a high correlation between ‘Staff listened carefully to what I had to say’ and ‘Staff explained things in a way that I could understand’ (0.81), as well as ‘Staff explained things in a way that I could understand’ and ‘I had confidence in the staff treating me’ (0.80).

The scree plot did not suggest the presence of more than two factors, as only two factors had eigenvalues greater than one (Figure 1).

Figure 1.

Scree plot of factors

Factor one had an eigenvalue of 7.64 and consisted of items one, two, three, four, five, seven, eight and nine (Table 3). All these items apart from item one related to communication with healthcare professionals.

Table 3.

Factor loadings

Factor two had an eigenvalue of 1.14 and consisted of items six, ten, 11, 12 and 13 (Table 3). These items related to the context in which care was delivered, having a family member or close friend for support, having enough staff on the ward, staff demonstrating they were familiar with procedures and equipment, and communication with staff regarding medication.

The correlation between the two factors was high (r=0.70). The two-factor solution did not convey any advantage over the simpler, single-factor solution and there were no strong theoretical grounds for opting for the former. All items for the single factor had loadings above 0.5. Cronbach’s alpha for the 13 items was high (0.91), providing further support for a single factor but suggesting an element of item redundancy. The mean inter-item Pearson correlation was 0.45, so within the 0.15 to 0.50 range. Items two to five were the most correlated of the 13 items.

Based on the general linear model, all covariate effects were not statistically significant at the 5% level. This demonstrated that patients’ characteristics did not influence how they perceived their safety. The correlation between mean safety score and patients’ overall rating of how safe they felt during their hospital stay was strong (r=0.65). This shows that the 13-item scale tapped into aspects of care that patients felt were important in making them feel safe.

Responses to open-ended questions

Patients felt safe when staff explained care to them in a way they could understand. Friendliness and kindness of staff was important to patients’ experience of feeling safe. Patients explained and recognised the effects of specialist expertise and multidisciplinary team working in making them feel safe. The importance of the hospital having enough staff for ‘constant visibility’ and ‘regular interaction’ was mentioned. Taking time to address patients’ medication queries was also cited as a feature of feeling safe.

Care that made patients feel unsafe was often the opposite of what made them feel safe. Patients recognised and described when communication between staff was unclear and how this made them feel unsafe. Staff being unaware of patients’ medication or past medical history were features of concern.

Patients had the opportunity to provide feedback about how staff responded and what staff could have done differently to make them feel safe. They said they wanted better communication, including staff listening more closely to them and providing reassurance.