Background The authors designed the evidence-based nursing practice assessment tool (EBNPAT) to measure the competence of nurses in using the five-step evidence-based practice model: ask, acquire, appraise, apply and assess. However, they needed to assess its content validity to show it is a valid tool. Assessing the content validity of a newly developed measurement scale is often challenging. Although the content validity index (CVI) is widely used in nursing research, it is limited to evaluating the chance of variation in agreement and consistency among reviewers of the content.

Aim To demonstrate the content validity of EBNPAT.

Discussion An expert panel of 42 nursing and health academics and researchers evaluated the representativeness, comprehensiveness and clarity of each construct item on EBNPAT. They then rated their agreement on a four-point ordinal scale. The authors used the item-CVI (I-CVI) augmented with the intraclass correlation coefficient (ICC) and Cronbach’s to assess the results and determine the content validity of EBNPAT. All I-CVIs for representativeness, comprehensiveness and clarity of the EBNPAT items were greater than 0.9 and the overall agreement and consistency among experts were greater than 0.65. The strengths of this study were the good sample size and the substantial consensus among experts in their view of assessing competence in evidence-based practice.

Conclusion High content validity of all construct items supports the progression to the next step in psychometric evaluation of EBNPAT.

Implications for practice This paper has demonstrated a sophisticated statistical approach for evaluating content validity that may be of use to developers of measurement scales in the future.