• To understand the tasks and time required for budgeting and setting realistic project timelines and recruitment targets for research
• To gain insight into the workload associated with conducting qualitative research
• To appreciate the time and effort involved with conducting qualitative interviews with research participants
Background Appropriate costing and allocation of resources is vital to ensure that recruitment to a study is achieved on time and on target. However, there is little guidance concerning the workload associated with qualitative research.
Aim To review the planned versus actual workloads in a qualitative sub-study following elective cardiac surgery in children.
Discussion Parents of children approached for a clinical trial were invited to participate in a semi-structured interview to explore their views about making decisions concerning their children’s participation in the trial. A workload audit was conducted using anticipated points of contact with participants, and the duration of activities identified in the protocol and Health Research Authority statement of activities; these were compared with timed activities documented by the research team.
Conclusion The current system did not anticipate or capture the workload associated with conducting a relatively straightforward qualitative sub-study of a clinical trial with a research-engaged patient group.
Implications for practice Understanding the hidden workload associated with qualitative research is vital in ensuring that project timelines, recruitment targets and funding for research staff are realistic.
Nurse Researcher. doi: 10.7748/nr.2023.e1872
Peer reviewThis article has been subject to external double-blind peer review and checked for plagiarism using automated software
Correspondence Conflict of interestNone declared
Menzies JC, Tooke C, Jones TJ et al (2023) ‘Just one interview’: making visible the hidden workload associated with qualitative research. Nurse Researcher. doi: 10.7748/nr.2023.e1872
AcknowledgementsThe authors would like to thank Helen Winmill, Jenna Spry and colleagues in the paediatric intensive care unit research nursing team for assistance in recruitment, as well as the parents who gave their time to be interviewed for the study. This work was supported by a grant from Birmingham Children’s Hospital Research Foundation [BCHRF442]. Julie Menzies was an NIHR 70@70 senior nurse and midwife research leader. Nigel Drury was funded by an Intermediate Clinical Research Fellowship from the British Heart Foundation [FS/15/49/31612]. The views expressed are those of the authors and not necessarily those of the NHS, Department of Health and Social Care, National Institute for Health and Care Research, Birmingham Women’s and Children’s NHS Foundation Trust, University of Birmingham or the British Heart Foundation.
Open accessThis is an open access article distributed under the terms of the Creative Commons Attribution-Non Commercial 4.0 International (CC BY-NC 4.0) licence (see https://creativecommons.org/licenses/by-nc/4.0/) which permits others to copy and redistribute in any medium or format, remix, transform and build on this work non-commercially, provided appropriate credit is given and any changes made indicated.
Published online: 27 April 2023
Recruitment to and completion of studies on time and on target are important performance metrics in research (National Institute for Health Research (NIHR) 2021). Failing to recruit or retain participants can have serious implications – studies may require additional time or funding (Sully et al 2013) or even be discontinued, which wastes scarce resources (Kasenda et al 2014). The value of understanding participants’ views concerning research is therefore increasingly recognised, particularly in contexts regarded as challenging, such as paediatrics (Caldwell et al 2004, Knox and Burkhart 2007, Sammons et al 2007, Kanthimathinathan and Scholefield 2014).
Research nurses play a vital role in the day-to-day conduct and management of clinical research in the UK (Connolly et al 2004, Mori et al 2007, Pick et al 2011, Gibbs and Lowton 2012, Lawton et al 2012, Jones et al 2020). But despite this important role, there is a lack of data concerning optimum staffing for research and service capacity (Hong et al 2021) and tools to support workforce planning have focused primarily on oncology clinical trials (Good et al 2013, Milani et al 2017, Lee and Jeong 2018).
In the UK, the Schedule of Events Cost Attribution Template (SoECAT) (NIHR 2019) has been developed to identify NHS support and treatment costs in non-commercial research studies. However, it is not intended for costing studies and has limited applicability to qualitative research.
This article reports on a project conducted during a qualitative study in which concerns were raised about additional, unanticipated work that arose, associated with recruiting and retaining participants. The research team conducted an audit to identify the workload and time associated with recruitment and retention and compare the anticipated and actual activities and time required.
The qualitative study discussed in this article explored the views regarding participation in research of parents whose children were undergoing surgery for congenital heart disease (Drury et al 2021). It was a single-centre sub-study of the Bilateral Remote Ischaemic Conditioning in Children (BRICC) trial, which is a multi-centre, double-blind, randomised controlled trial (RCT) (Drury et al 2020).
Parents who consented or declined to join the trial were approached following their child’s discharge from the paediatric intensive care unit (PICU) and asked to participate in a face-to-face or telephone interview, either during admission or following hospital discharge (Drury et al 2021). Funding was obtained for a team of research nurses to support recruitment and an embedded researcher to conduct the interviews. Data were collected between September 2017 and June 2019.
The project reported in this article was an audit (Health Research Authority (HRA) 2021) comparing the activity planned for the researchers with the actual practice. Audits are beneficial as they identify where improvement is required and provide a baseline to evaluate if beneficial change occurs (Twycross and Shorten 2014, Limb et al 2017). The research team contemporaneously documented their actual workload and compared it retrospectively to the activity planned in the study protocol and the HRA statement of activities completed by the main researcher (the lead author). It is reported according to the SQUIRE 2 guidelines (Ogrinc et al 2016), in accordance with recommendations for reporting and publishing audits (Limb et al 2017).
Two main measures were assessed in the audit: the planned points of contact (POCs), which were defined as attempted or successful contact between the nurse research team/researchers and parents and the length of time spent on each activity. Two researchers from the study site independently timed all study activities using a stopwatch. The mean time for each activity was used to calculate an overall time per person and a total time for all documented activities.
Recruitment activities were defined as activities involving recruitment that took place from the first approach to a parent to discuss their participation until the parent signed the informed consent form. These activities were predominantly conducted by the nursing research team.
Retention activities were defined as any activity that took place after a parent provided consent until they had been interviewed. These activities were primarily performed by the lead author. If consent was obtained before the parent’s interview, this was recorded as 0 – no new POC for retention. However, the time to complete an interview was documented as retention activity.
The research team initially envisaged that interviews would take place within two weeks of children’s discharge from hospital. However, this was extended through an amendment to the study protocol to within six weeks of discharge.
The planned research activity involved three POCs: two for recruitment (first approach and obtaining informed consent); and one for retention (interview). The anticipated workload was 0.7 hours/participant for recruitment and 1.2 hours/participant for retention, for a total of 1.9 hours/participant. The planned sample size was 30 participants, so this equated to an anticipated workload of 56.9 hours or 7.6 full-time equivalent days (7.5 hours/day).
Documentation in the study site file was audited. The research team documented all attempted contacts with families, whether successful or unsuccessful. The data collected included the dates of approach, all contacts with parents, the dates of consent, the methods of communication, the locations of contacts and the dates of interview.
The activities were coded and entered on a spreadsheet for analysis (Table 1). The number of POCs and the time spent on recruitment and retention were identified for each person and the median and interquartile range calculated.
The audit did not review patients’ records, so no additional ethical review was required.
• There was a significant unanticipated workload in a qualitative research study associated with recruitment and retention
• Understanding this workload is important so that researchers can set appropriate recruitment targets and realistic project timelines, as well as ensure there is funding for sufficient staffing
• Further work is needed to develop templates to support workforce planning for qualitative and mixed methods research studies
A total of 46 families were approached about their children’s participation in BRICC trial. Of these, 38 were eligible for interview in the qualitative sub-study. The parents of 28 children consented to be interviewed, with 26 parents participating in 23 interviews (Figure 1).
The sub-study protocol stated that parents would be approached on the ward following discharge from the PICU between two and seven days following surgery, with consent obtained 24 hours later. Our review of the study documentation found parents were approached at a median of 4.0 days (IQR: 2.0-6.8), with consent obtained at a median of 18 days (IQR: 6.0-35.5).
We had anticipated two POCs taking 0.7 hours/person, but the median number of POCs was 3.0 (IQR: 2.0-4.8) taking 1.4 hours/participant (IQR: 1.0-1.7) (see Table 2). Recruitment activities for all 38 eligible families totalled 58.6 hours of work – 36.8 hours (4.9 working days) more than expected; this included ten families who declined an interview and required a median of 5.0 (IQR: 4.0-6.0) POCs at 1.4 hours/participant or 12.5 hours in total (Table 2).
The workload anticipated for retention was one POC at 1.2 hours/person, with an overall workload of 35.1 hours for 30 participants. However, we successfully completed 23 interviews with 26 parents, which required a median of 3.0 POCs (IQR: 0.8-4.0) and 1.3 hours/participant (IQR: 0.9-1.8). We conducted seven interviews with nine parents immediately after they had given consent, so there were no new POCs for retention with them, which we recorded as 0.
The median time from consent to interview was 33.5 days (IQR: 17.5-56.8). We conducted eight interviews after the intended six-week period, with the latest conducted at 133 days following discharge.
Six parents from five families who had agreed to participate withdrew their consent or were lost to follow-up without interview. We spent a median of 1.6 hours/person (IQR: 1.4-1.6) over 6.0 POCs (IQR: 4.0-9.0) in unsuccessful efforts to retain them.
The total workload for all 38 approached families was 5.0 POCs (IQR: 4.0-8.0). We spent a median 2.7 hours (IQR: 1.6-3.3) on recruitment and retention per person from first approach to completion of an interview, which equated to 99.2 hours or 13.2 working days in total. This exceeded the anticipated workload by 42.3 hours (75%).
The predominant form of contact between the research team and parents during recruitment was a ward visit (n=71, 53%) (Table 3). There were 49 visits where the family was present and a researcher spoke to them, and 22 visits where the family was not present. We made 32 (24%) attempts to contact people by phone. Ten of these attempts were unsuccessful. Twenty POCs (15%) were in the outpatient department, with only one parent failing to attend. This meant we spent 6.3 hours on failed visits or phone calls.
The main method of contact with participants during retention following hospital discharge was a phone call (n=69, 70%), of which 29 calls (42%) failed to establish contact with participants (Table 3). Unsuccessful phone calls took 3.7 hours in total.
Participants were offered a choice of method and location for their interviews: phone, in-person during a hospital clinic appointment or in-person at a dedicated home visit. We conducted most interviews by phone (n=14, 61%), five (22%) in-person at a clinic appointment and four (17%) during a home visit. The last of these required a considerable amount of time because of the additional travel needed, taking a median 2.2 hours (IQR: 1.7-2.6) on average to complete.
Overall, few reasons were captured about recruitment and retention challenges. Ten families declined to participate in the sub-study: eight who had consented to the BRICC trial and two who had declined. Six of the families provided no reason for their decision, two said they were not interested and two said they did not have time.
Two of the six parents who consented to be interviewed did not eventually participate: one withdrew due to complications following surgery and one was unavailable for interview, although their partner still participated. The other three parents failed to respond to repeated attempts by the research team to contact them, so were classed as ‘lost to follow-up.’
The research team initially planned that all contact with parents would be face-to-face. However, we amended the protocol as the study began to allow other forms of contact to assist in communication. This proved to be prudent for retention, as 98 POCs were needed to arrange the interviews, of which 88 (90%) were by phone call, text message, email or letter. Reminders and multiple methods to contact and communicate with participants are important to maximise engagement (Robinson et al 2007), even in a simple study involving only one follow-up intervention such as an interview.
The reasons parents gave for their declining participation reflected the lack of time and support they had for childcare. This is consistent with previous studies investigating the challenges faced by families of children who are undergoing surgery for congenital heart disease (Wray et al 2018).
Researchers wanting to engage with families need to offer flexibility in the timing and method of communication. Research ethics committees often suggest that study protocols should include a strict time frame for recruitment or limits on the number of attempted or actual contact points with participants. However, we caution against this: 48% of interviews in our study required six or more POCs and we conducted 35% more than six weeks after discharge from hospital. If we had not been flexible and considered families’ social circumstances, many would have been excluded and the perspective of seldom-heard participants would have been lost.
Accommodating participants’ needs is also important when deciding the location for interviews and how they will be conducted. We initially envisaged that we would interview all participants face-to-face; however, 61% of participants opted for a telephone interview, once we offered this method of contact.
We conducted this study before the COVID-19 pandemic and virtual approaches using online video platforms have subsequently proved to be acceptable to participants, as they provide a more time- and cost-effective method of gathering qualitative data (Schlegel et al 2021). Offering a range of options helps to involve seldom-heard participants and we advise offering a wide range of methods from the outset, as later amendments may require additional research ethics committee approval.
Families became eligible for the sub-study once their child had been discharged from the PICU to the ward following congenital heart surgery. They often experience elevated levels of distress and anxiety at this time (Jackson et al 2015, Woolf-King et al 2017), and being approached about research is an additional stressor (Menon et al 2012, Kanthimathinathan and Scholefield 2014).
The initial approach about research is extremely important with families of infants and children who are critically ill (Wilman et al 2015, Menzies 2018), which was the case in our study (Drury et al 2021). Demonstrating sensitivity to potential participants is crucial because parents who perceive a research team positively are more likely to consent to participate in a study (Tait et al 2003, Hoberman et al 2013).
Research nurses are ideally placed to provide insight into the purpose of a study and support the obtaining of informed consent (Mori et al 2007, Pick et al 2011, O’ Sullivan et al 2021). Our audit provides evidence of their ability to sensitively navigate recruitment and retention with families who have experienced the challenges of their children undergoing cardiac surgery. More POCs and time were necessary than we initially envisaged, but 74% of approached parents were successfully recruited and 82% of recruited parents went on to complete an interview. The research nurses’ efforts enabled the qualitative research study to be conducted and provided insights into the views of not only parents who consented to the trial but also those who declined to participate – a group that is seldom heard (Drury et al 2021).
The flexibility offered by the team was beneficial to the families but made it harder to predict workload. The research team could set aside time to speak to families or to arrange interviews, only to find no one was available or the researcher or family felt it was not an appropriate moment. Conversely, families who had not been engaging with the study or answering phone calls would answer the phone and decide this was an optimum moment to conduct an interview.
Our study team could support this enhanced level of involvement for free, but this level of resource and flexibility is not commonly available. In the UK, only 11 (52%) of the 21 responding PICUs had permanent, funded research delivery roles and only two (5%) including our unit offered cover seven days a week (Menzies et al 2022). This level of support may therefore not be replicable elsewhere.
We conducted this audit retrospectively and it only reflects documented research activity, so it is possible it missed some activities. There may therefore be an even greater difference between planned and actual workloads.
We estimated the times for when research activities were conducted using times specific to our IT systems and office location, which may not be directly applicable to other settings. In addition, the audit was conducted in a tertiary paediatric centre with a well-staffed research team, so may not be applicable to all centres or target populations.
Our approach to recruiting and retaining participants may have evolved during the study. We may have needed less time as we gained greater familiarity with the study’s processes but more POCs as we knew that greater persistence would often pay off.
This article contributes a valuable insight into the workload involved in conducting qualitative research and supports future research into better understanding the processes involved. We were conducting a relatively straightforward qualitative sub-study with a research-engaged patient group, but our audit nevertheless found there was a significant hidden workload, with the number of POCs and the time required for recruitment and retention much greater than existing systems had anticipated and accounted for – and that we had not anticipated when writing the study protocol and obtaining funding.
Researchers are typically aware of important stages in research such as obtaining informed consent. But intermediate steps and decision-making are often poorly articulated and understood (Skea et al 2017). Understanding the workload associated with research is vital to ensure that staffing is planned appropriately and helps to make a case for additional recruitment (Gilardi et al 2014).
Existing templates have been designed for clinical trials (Good et al 2013, Lee and Jeong 2018) or to consider treatment costs (NIHR 2019) but cannot capture the workload associated with conducting qualitative or mixed methods research. Enhancing the understanding of research workload is invaluable to ensure there is sufficient funding for staff and that project timelines and recruitment targets are realistic, as well as reduce research waste (Kitterman et al 2011, Kasenda et al 2014). If research is to truly become a core part of healthcare, it is vital that workload templates are developed to assist in planning and funding research appropriately as well as recruiting staff.
13 March 2024 / Vol 32 issue 1
Exploring the role and expertise of ward-based oncology clinicians
Aim The aim of this study was to involve ward-based band 6...
Effectiveness of a neutropenic sepsis clinical pathway
This article reports the findings of a re-audit across a...
Identifying patients at risk of neutropenic sepsis
An audit of mortality figures for patients undergoing...
The biology of cancer
Cancer research is moving fast. Understanding of the biology...
Compliance with swift communication of diagnosis to GPs
Until this year the National Cancer Peer Review Programme...
