Selection, collection and analysis as sources of evidence in case study research
Evidence & Practice Previous    

Open Access Selection, collection and analysis as sources of evidence in case study research

Catherine Houghton Lecturer, School of Nursing and Midwifery, National University of Ireland Galway, Republic of Ireland
Dympna Casey Senior lecturer, National University of Ireland Galway, Republic of Ireland
Siobhan Smyth Lecturer, National University of Ireland Galway, Republic of Ireland

Background Case study research is a valuable way to explore and describe nursing phenomena in their natural contexts. Multiple sources of evidence are critical in this approach. It is imperative that the strategies for selection, collection and analysis of cases are considered and articulated in the early stages of planning, to avoid having large datasets which cannot be harmonised.

Aim To critically examine what is meant by ‘multiple sources of evidence’ and how they can be used in case study research. Two examples of case study research are used to illustrate the decisions the authors made during the selection, collection and analysis stages of the research.

Discussion These decisions included what sources would be used, rationales for their use, and how the data would be collected and analysed. In addition, multiple sources of evidence can result in large amounts of data so the use of NVivo to manage the data is described.

Conclusion Each source of evidence selected must have a clear purpose and relate to the study’s objectives. Clarification of this during the early planning of any research is imperative.

Implications for practice The authors hope that the examples provided to illustrate how multiple sources of evidence are used will guide researchers conducting case study research.

Nurse Researcher. 24, 4, 36-41. doi: 10.7748/nr.2017.e1482

Correspondence

catherine.houghton@nuigalway.ie

Peer review

This article has been subject to external double-blind peer review and checked for plagiarism using automated software

Conflict of interest

None declared

Received: 13 April 2016

Accepted: 27 September 2016

Published in print: 22 March 2017

Introduction

Case study research (CSR) is becoming increasingly popular in nursing research. One of its strengths is the opportunity to use multiple sources of evidence (Yin 2003, Burns and Grove 2005, McGloin 2008, Ryan-Nicholls and Will 2009). These can provide in-depth and multifaceted explorations of phenomena. However, selecting, collecting and analysing multiple sources of evidence can be challenging and complex. The purpose of this paper is to explore, using two examples of CSR, some of the practical issues in working with multiple sources of evidence and how to address them.

Multiple sources of evidence

CSR is useful for studying a phenomenon in its natural context (Stake 1995, Yin 2003). It is used to explore real-life experiences and situations, when the phenomenon and the context in which it occurs are of interest (Yin 2003, Luck et al 2006, Salminen et al 2006). It is an appropriate design for exploring events, situations, programmes and activities (Hancock and Algozzine 2006, Berg 2009).

CSR is considered suitable for nursing research because it can explore and explain contemporary phenomena (Pegram 2000). However, increasing complexity in healthcare, coupled with increasing use of case study designs in nursing research, supports the need for an examination of this methodology (Anthony and Jack 2009).

CSR is classified in several ways (Casey and Houghton 2010): a case study can be exploratory, explanatory or descriptive (Yin 1994); and it can be holistic or embedded, single or multiple, instrumental or intrinsic (Yin 1994, Stake 1995). However, a critical characteristic common to all types of CSR is the use of multiple sources of evidence (Stake 1995, Bergin and While 2000, Yin 2003). This captures the complexities of phenomena, thus making the case more complete (Walshe et al 2004).

Another objective of using multiple sources of evidence is to enhance rigour (Houghton et al 2013a). Biases in the researcher’s collection and analysis of data can be counteracted by using multiple sources of evidence (Darke et al 1998, Yin 2003), making the findings more credible (Lincoln and Guba 1985, Houghton et al 2013a).

The most common sources of evidence in CSR are interviews, observations and documents (Yin 1994, Stake 1995, Keyzer 2000); others include surveys, physical artefacts and archival records (Moriarty et al 2007, Baxter and Jack 2008). With such a broad range of sources available, efficient and effective collection of data is imperative (Darke et al 1998).

There are challenges associated with using multiple sources. There can be overwhelming amounts of data that require management and researchers can sometimes get ‘lost’ (Baxter and Jack 2008). In addition, it can be difficult to bring the data together during analysis for reporting (Baxter and Jack 2008).

Two examples of CSR will be presented to illustrate how the authors overcame these and other challenges, emphasising the need for appropriate strategies for selection, collection and analysis of data.

Example 1

The first example of CSR was a multiple case study exploring the role of the clinical skills laboratory (CSL) in preparing nursing students for clinical practice. Five cases were identified; each comprised an institution offering the bachelor of nursing degree and its affiliated hospital. The main objectives were twofold: to describe the strategies for teaching and assessing in the CSL; and to explore the factors that could help or hinder how nursing students implement their learned skills in clinical settings. The overall aim was then to explore how the CSL could prepare students for that experience through teaching and assessment strategies. Identification of the case at the outset ensured a well-defined and bounded context (Stake 1995, Casey and Houghton 2010). The study was qualitative and located in an interpretivist paradigm (Houghton et al 2013a).

Selection

To answer the questions posed, multiple sources of evidence were needed. Three were chosen: interviews, observations and documents. Each method of collecting data needed to have a specific purpose. Semi-structured interviews were critical for gathering views about teaching and assessing in the CSL, as well as staff and students’ perceptions of factors that influence students’ implementation of skills in clinical practice. Non-participant observation was conducted to identify what happened when students implemented clinical skills in practice. Documents provided the context and background to how the teaching and learning of clinical skills occurred in the CSL and in clinical practice, as well as what was expected of students in terms of competency throughout their education and on qualification. Each source had a specific function but with triangulation could provide a comprehensive account of each case and confirm the findings of the phenomenon of interest (Murphy and Casey 2009).

Collection

A total of 58 semi-structured interviews were conducted by the primary researcher (CH) across the five case study sites between 2008 and 2009. Interviews are one of the most important sources of case study information (Yin 2003). They can provide rich, detailed data and gather information that is not overtly observable (Patton 2002, Shah and Corley 2006).

Participants included educators, CSL managers and technicians, clinical nurse managers, staff nurse preceptors, clinical placement coordinators, nursing students across the four years of the nursing degree programme, and newly qualified staff nurses who had recently completed the programme.

Efforts were made to ensure the equal representation of all types of participant across the five case study sites. The researcher asked participants about their perceptions and experiences of the CSL, as well as of students implementing clinical skills in practice.

The most challenging aspect of this was obtaining access. The researcher needed to establish an open, transparent relationship with gatekeepers to educational and clinical areas across the five sites (Simons 2009). Regular meetings were held, and gatekeepers and participants needed to be reassured that there was no intention to audit practice in either the CSLs or the clinical setting. Ethical approval was obtained from the research ethics committees of each institution and their affiliated hospitals. Informed consent was obtained and confidentiality ensured.

Observational data provided the opportunity to confirm verbal reports (Thompson et al 2005, Frankfort-Nachmias and Nachmias 2008). Observation is commonly used in CSR as it offers insight into behaviours and practices as they occur in their natural settings. Non-participant observation was conducted in the clinical settings of the five sites (Hancock and Algozzine 2006, Watson and Whyte 2006, Simons 2009).

A pilot was used to make important decisions about optimal positioning, length of time and the method of documenting the observations (Casey 2004).

Observations in the main study were conducted by the primary researcher (CH) at each site over a 12- were used to determine students’ implementation of clinical skills throughout the day. Mobile positioning and event sampling were used. Mobile positioning involved following a particular student to certain clinical areas while they performed certain skills, as advocated by Polit and Tatano Beck (2014).

Event sampling involved observing a particular event (Casey 2004), in this case meal time and medication rounds. This method gleaned rich data that would not have been gathered through interviews alone. Observing in a clinical setting where patients, other staff and members of the public are present posed ethical challenges which needed to be addressed (Houghton et al 2010).

The third source of evidence was documents. Simons (2009) defined a document as anything written or produced about the research context or site. Documents are important for augmenting evidence from other sources and are most often used in conjunction with interviews and observations to develop a better understanding of the phenomenon of interest (Merriam 1998, Yin 1998, Shah and Corley 2006).

In this example, documents included national and international guidelines for nurse education and how best to help students learn. Competency assessments from each site were examined to understand what students were expected to learn in clinical practice.

Finally, clinical placement timetables were accessed to ascertain when students learned skills in the CSL in relation to when they went out on placement. Table 1 summarises the sources of evidence used for Example 1.

Table 1

Overview of sources of evidence in Example 1

Semi-structured one-to-one interviews (n=58)Non–participant observation (five days)Documents
Junior nursing students (n=10)One day (12-hour shift) per site conducted in two-hour observation sessionsNational and international organisational guidelines
Senior nursing students (n=10) Competency assessment documents
Newly qualified staff nurses (n=8) Clinical placement timetables
Clinical staff (n=15)  
Academic staff (n=15)  

Analysis

The three methods used created a large volume of data from five different sites and from many different perspectives and sources. In multiple CSR, ‘within-case’ and ‘cross-case’ analysis are needed (Miles and Huberman 1994, Stake 2006, Creswell 2013): within-case analysis provides a comprehensive description of each case (Creswell 2013, Houghton et al 2015); cross-case analysis examines themes across the cases to deepen understanding and explanations (Eisenhardt 1989, Miles and Huberman 1994).

Analysis was conducted, guided by Morse’s framework (1994) and implementing strategies proposed by Miles and Huberman (1994) (Houghton et al 2015). NVivo 8 software was used to manage the large dataset and helped with cross-case and within-case analysis (Bassett 2009).

NVivo enabled the analysis of multiple sources of evidence together. In addition, using ‘queries’ to check the frequency and source of propositions made from the data ensured the confirmability of findings (Houghton et al 2013b).

It is important to note that in the final presentation of findings, only the cross case-analysed data were presented, to preserve the confidentiality of the research sites as they may have been identifiable from the within-case analysis.

Comparing data gathered from multiple sources can confirm the extent to which findings can be verified (Murphy and Casey 2009). NVivo provided an audit trail, which demonstrated how the data were integrated and analysed together within cases and across cases. Furthermore, query tools ensured that any issue described in the findings could be examined and compared from different sources, types of participant or research site (Houghton et al 2013b).

Example 2

The second example was a multiple case study which explored the experiences of mental health nurses (MHNs) trained in using psychosocial interventions (PSIs) in the care of patients with mental health problems. Consistent with the goal of understanding experience, a qualitative approach was adopted using a multiple case study design. Four public sector Health Service Executive (HSE) sites were used in the study.

Selection

As in example 1, multiple sources of evidence provided answers to the overall aim and addressed the objectives of the research. In this study, the data sources comprised nonparticipant observations and one-to-one semi-structured interviews, supplemented by field notes. The observations captured potential PSIs that nurses use in practice.

The interviews allowed the participants to talk freely about the topic (Polit and Tatano Beck 2014), while the supplementary field notes helped to make sense of the observations and the contexts in which the nurses worked (Gerrish and Lacey 2010). The field notes also helped in obtaining enriched meanings for the interview data.

Collection

The primary researcher (SS) gathered the interview data between 2012 and 2013 from 40 PSI-trained nurses across inpatient and community settings: 33 from community settings, seven from inpatient settings. In total, 16 of the nurses were clinical nurse specialists, 12 were staff nurses, 10 were clinical nurse managers, and two were advanced nurse practitioners. A multidisciplinary team meeting was also observed to help keep the findings related to the context. Observational data were collected through continuous note-taking during and after the observations, followed by reflective commentaries. In total, 19 hours and 20 minutes of non-participant observations were completed across the four cases. These were undertaken at convenient times for the participants.

Ethical approval was initially sought from the research ethical committee of the institution where the researcher was studying and from the four different hospital research ethics committees. Informed consent was obtained and participants’ well-being was always given priority over the study. Table 2 provides an overview of the different types of participant and the numbers of interviews and observations in this study.

Table 2

Overview of sources of evidence in Example 2

Data collection methodsParticipantsCommunity PSI-trained MHNsInpatient PSI-trained MHNs
Semi-structured one-to-one interviewsPSI-trained MHNs (40 interviews)337
Non–participant observationPSI-nurses (19 hours, 20 minutes) and multidisciplinary meetings5
1
2
Field notes (reflective notes: supplementary)

PSI can refer to a wide range of interventions, so a semi-structured observation tool was adapted from the literature (Spradley 1980), based on the researcher’s clinical experience as a MHN. This provided a focus to achieve a more comprehensive understanding of the participants’ experiences of using PSIs in practice. An interview schedule was used to guide the interviews. The field notes were managed by adapting Spradley’s (1980) format, which considers three sorts of field note: condensed accounts, expanded account and a fieldwork diary. Writing field notes during the study helped to identify patterns in the data and served to increase the quality of the research’s findings (Stake 1995).

For example, the observational data enabled the descriptions to be checked against facts, as it was possible to clarify afterwards in the interviews any discrepancies regarding how nurses use PSI in practice. Therefore, by observing and interviewing the same participants, the dissimilarities between what participants said and did could be acknowledged (Mays and Pope 1995).

Houghton et al (2013b) stated that when data are gathered from multiple sources, comparisons can be made to establish the degree to which findings can be confirmed. The field notes helped to provide contextual data; they served to support the observations and interviews by enhancing meaning and generating a more holistic and complete account of the phenomenon. Therefore, each modality assisted in providing the indepth information required to address the research’s aim and objectives. However, one challenge was the use of the non-participant observations. It was clear from when MHNs were approached about the study that most of them were reluctant to be observed. Many MHNs had queries about consent, particularly from the client’s perspective. Mulhall (2003) said these issues are compounded by the unpredictability of observation work.

Participants were reassured that it was unnecessary to gain written consent from the clients, because they were the focus of the research, not the clients. However, verbal consent was obtained from all clients interacting with the MHNs being observed.

Analysis

Ritchie and Spencer’s (1994) framework approach was adapted as it enabled flexibility throughout analysis, to detect important issues in the data and interpret the data’s meaning. This approach is consistent with multiple CSR. It uses five stages, each stage consisting of several cycles, with the researcher able to use two types of analysis: within-case and cross-case (Miles and Huberman 1994, Stake 2006, Creswell 2013). Within-case analysis involved outlining a thorough description of each case and its themes; cross-case analysis allowed themes to be analysed across the cases and the researcher could search for similarities and differences between the cases.

Arguably, an audit trail is the most important criterion for establishing the trustworthiness and credibility of a study. In this study, every step of the analysis was accounted for because of the diagrams and text used during the various cycles of the analysis.

Conclusion

Multiple sources of evidence are an important feature of CSR and can enhance the rigour of a study. This paper examined the practical challenges involved when collecting multiple sources of evidence. Each source of evidence must have a clear purpose and relate to the study’s aims and objectives. Articulation of these purposes when planning the study can help the researcher to fully explore the phenomenon of interest, but in a bounded context. It may also prevent a novice researcher using CSR from getting lost in irreconcilable data.

It is the very purpose of multiple sources that can pose the greatest challenge: multiple sources provide a comprehensive picture of a case, but it is that thoroughness that can potentially make gatekeepers and participants wary. The researcher should make the nature and purpose of the research explicit to key individuals openly and transparently; the way in which the data will be presented must also be determined at an early stage from an ethical perspective, so that gatekeepers and participants can be reassured that they will not be recognisable in the final report.

An additional challenge illustrated in the examples pertains to the complexity of managing data from different sources. NVivo or similar qualitative analysis software is therefore a crucial way to store and manage the large datasets generated.

Ultimately, careful consideration of how each source of evidence will help to answer the research question ensures that the multiple sources used are appropriate. Planning for how the data will be managed will prevent researchers from becoming overwhelmed during analysis. For novice CSR researchers, using the term ‘selection, collection and analysis’ in articulating how the multiple sources of evidence are selected may help to structure and make more explicit the decisions required to answer the research question.

References

  1. Anthony S, Jack S (2009) Qualitative case study methodology in nursing research: an integrative review. Journal of Advanced Nursing. 65, 6, 1171-1181.
  2. Bassett BR (2009) Computer-based analysis of qualitative data: NVIVO. In Mills AJ, Eurepos G, Wiebe E (Eds) Encyclopedia of Case Study Research. Sage Publications, Thousand Oaks CA.
  3. Baxter P, Jack S (2008) Qualitative case study methodology: study design and implementation for novice researchers. The Qualitative Report. 13, 4, 544-559.
  4. Berg BL (2009) Qualitative Research Methods for the Social Sciences. Seventh edition. Allyn and Bacon, Boston MA.
  5. Bergin A, While A (2000) A case for case studies: exploring the use of case study design in community nursing research. Journal of Advanced Nursing. 31, 4, 926-934.
  6. Burns N, Grove S (2005) The Practice of Nursing Research. Conduct, Critique, and Utilisation. Fifth edition. Elsevier Saunders, St Louis MO.
  7. Casey D (2004) Challenges of collecting data in the clinical setting. Nursing Times Research. 9, 2, 131-141.
  8. Casey D, Houghton C (2010) Clarifying case study research: examples from practice. Nurse Researcher. 17, 3, 41-51.
  9. Creswell JW (2013) Qualitative Inquiry & Research Design: Choosing Among Five Approaches. Third edition. Sage Publications, Thousand Oaks CA.
  10. Darke P, Shanks G, Broadbent M (1998) Successfully completing case study research: combining rigour, relevance and pragmatism. Information Systems Journal. 8, 4, 273-289.
  11. Eisenhardt KM (1989) Building theories from case study research. The Academy of Management Review. 14, 4, 532-550.
  12. Frankfort-Nachmias C, Nachmias D (2008) Research Methods in the Social Sciences. Seventh edition. Worth Publishers, New York NY.
  13. Gerrish K, Lacey A (Eds) (2010) The Research Process in Nursing. Sixth Edition. John Wiley and Sons, Chichester.
  14. Hancock DR, Algozzine B (2006) Doing Case Study Research. First edition. Teachers College Press, New York NY.
  15. Houghton CE, Casey D, Shaw D et al (2010) Ethical challenges in qualitative research: examples from practice. Nurse Researcher. 18, 1, 15-25.
  16. Houghton CE, Casey D, Shaw D et al (2013a) Students’ experiences of implementing clinical skills in the real world of practice. Journal of Clinical Nursing. 22, 13-14, 1961-1969.
  17. Houghton C, Casey D, Shaw D et al (2013b) Rigour in qualitative case study research. Nurse Researcher. 20, 4, 12-17.
  18. Houghton C, Casey D, Shaw D et al (2015) Qualitative case study data analysis: an example from practice. Nurse Researcher 22, 5, 8-12.
  19. Keyzer DM (2000) Nursing research in practice: the case study revisited. The Australian Journal of Rural Health. 8, 5, 266-270.
  20. Lincoln YS, Guba EG (1985) Naturalistic Inquiry. Sage Publications, Newbury Park CA.
  21. Luck L, Jackson D, Usher K (2006) Case study: a bridge across the paradigms. Nursing Inquiry. 13, 2, 103-109.
  22. Mays N, Pope C (1995) Qualitative research: rigour and qualitative research. British Medical Journal. 311, 6997, 109-112.
  23. McGloin S (2008) The trustworthiness of case study methodology. Nurse Researcher. 16, 1, 45-55.
  24. Merriam SB (1998) Qualitative Research and Case Study Applications in Education. Revised edition. Jossey-Bass Publishers, San Francisco CA.
  25. Miles MB, Huberman AM (1994) Qualitative Data Analysis: An Expanded Sourcebook. Second edition. Sage Publications, Thousand Oaks CA.
  26. Moriarty D, O’Hara A, Byron S (2007) Macmillan nurse facilitators for palliative care: evaluation of a pilot project. International Journal of Palliative Nursing. 13, 7, 334-343.
  27. Morse JM (1994) “Emerging from the data”: the cognitive processes of analysis in qualitative inquiry. In Morse JM (Ed) Critical Issues in Qualitative Research Methods. Sage Publications, Thousand Oaks CA, 23-43.
  28. Mulhall A (2003) In the field: notes on observation in qualitative research. Journal of Advanced Nursing. 41, 3, 306-313.
  29. Murphy K, Casey D (2009) Issues in using methodological triangulation in research. Nurse Researcher. 16, 4, 40-55.
  30. Patton MQ (2002) Qualitative Research and Evaluation Methods. Third edition. Sage Publications, Thousand Oaks CA.
  31. Pegram A (2000) What is case study research? Nurse Researcher. 7, 2, 5-16.
  32. Polit DF, Tatano Beck C (2014) Essentials of Nursing Research: Appraising Evidence for Nursing Practice. Eighth edition. Lippincott Williams & Wilkins, Wolters Kluwer Health, Philadelphia PA.
  33. Ritchie J, Spencer L (1994) Qualitative data analysis for applied policy research. In Bryman A, Burgess RG (Eds) Analyzing Qualitative Data. Routledge, Abingdon, 173-194.
  34. Ryan-Nicholls KD, Will CI (2009) Rigour in qualitative research: mechanisms for control. Nurse Researcher. 16, 3, 70-85.
  35. Salminen A-L, Harra T, Lautamo T (2006) Conducting case study research in occupational therapy. Australian Occupational Therapy Journal. 53, 1, 3-8.
  36. Shah SK, Corley KG (2006) Building better theory by bridging the quantitative-qualitative divide. Journal of Management Studies. 43, 8, 1821-1835.
  37. Simons H (2009) Case Study Research in Practice. Sage Publications, London.
  38. Spradley JP (1980) Participant Observation. Wadsworth, Belmont CA.
  39. Stake RE (1995) The Art of Case Study Research. Sage Publications, Thousand Oaks CA.
  40. Stake RE (2006) Multiple Case Study Analysis. The Guilford Press, New York NY.
  41. Thompson C, McCaughan D, Cullum N et al (2005) Barriers to evidence-based practice in primary care nursing – why viewing decision-making as context is helpful. Journal of Advanced Nursing. 52, 4, 432-444.
  42. Walshe CE, Caress AL, Chew-Graham C et al (2004) Case studies: a research strategy appropriate for palliative care? Palliative Medicine. 18, 8, 677-684.
  43. Watson H, Whyte R (2006) Using observation. In Gerrish K, Lacey A (Eds) The Research Process in Nursing. Fifth edition. Blackwell Publishing, Oxford, 383-389.
  44. Yin RK (1994) Case Study Research: Design and Methods. Second edition. Sage Publications, London.
  45. Yin RK (1998) The abridged version of case study research. In Bickman L, Rog DJ (Eds) Handbook of Applied Social Research Methods. Sage Publications, Thousand Oaks CA, 229-260.
  46. Yin RK (2003) Case Study Research: Designs and Methods. Third Edition. Sage Publications, London.