Value of a Pilot Study

The author has disclosed no potential conflicts of interest, financial or otherwise.

How often has a study ended up underpowered because of issues with the recruitment and retention of participants? How often has an educational intervention been designed, only to discover there were conceptual and methodological issues that surfaced when the study was underway? Hopefully, the response is never or infrequently because a pilot study had been planned and executed to address some of these potential issues before implementing the larger study. Pilot studies can contribute invaluable information to assist with the conduct of important inquiry.

Terms such as pilot study or feasibility study are frequently used interchangeably (Arain et al., 2010; Leon et al., 2011). Some authors differentiate between undertaking a pilot study and undertaking a feasibility study. For example, Thabane et al. (2010) indicated that the goal of a pilot study is to assess the feasibility of the proposed study “so as to avoid potentially disastrous consequences of embarking on a large study, which could potentially ‘drown’ the whole research effort” (p. 1). Conversely, Arain et al. (2010) indicated that a feasibility study is undertaken to determine important components critical to the development of the main study, whereas a pilot study is the conduct of the main study in miniature. Thus, according to Arain et al., such issues as participant willingness to be randomized, team member willingness to recruit participants, and participant responsiveness to follow-up measures are considered in a feasibility study; when this information has been determined, the main study is designed, and the assessment of how well the study components work is through means of a pilot study.

Nurse scientists appreciate that the purpose of a pilot study is not generally hypothesis testing (Leon et al., 2011). Of note, in their review of seven medical journals in the period of 2007 to 2008, Arain et al. (2010) found that hypothesis testing was undertaken either to evaluate the effectiveness of an intervention or to determine potentially relevant associations. An important consideration is that most pilot or feasibility studies will have sample sizes that are too small to detect differences. In addition, estimates of sample size, which are determined based on pilot data, may lead to significant miscalculations. Thus, caution should be undertaken when determining sample size for the main study.

Although nurse investigators may be tempted not to consider publishing the results of a pilot study, Thabane et al. (2010) argued that investigators should be proactive in sharing their results, as “researchers have an ethical and [a] scientific obligation to attempt publishing the results of every research endeavor” (p. 6). In addition, sharing such information can have an impact on research resources or prevent unnecessary duplication of effort. In fact, Arain et al. (2010) advocated for their publication, stating that “publishing the results of well conducted pilot or feasibility studies is important for research, irrespective of outcome” (p. 6).

The Research Brief section of the Journal of Nursing Education provides a venue in which to share the results of pilot work of nurse scientists investigating nursing education issues. Such an opportunity carries with it the associated responsibilities to be true to the purposes of pilot studies. Thus, when writing the report of a pilot study, in addition to typical research components, nurse scientists should consider the following:

Background. Include a reason for undertaking the pilot study in this section of the paper (Thabane et al., 2010).

If conducting an intervention, indicate how feasibility will be assessed. Some would question inclusion of a control group during a pilot study; however, doing so helps to address issues such as recruitment (Leon et al., 2011).

Informed consent. Describe how participants were informed about the feasibility nature of the study (Thabane et al., 2010).

Interpret results within the context of feasibility and include the measures that need to be implemented to make the proposed study feasible (Arain et al., 2010; Thabane et al., 2010). Think of this section as the lessons learned to design a more cohesive study with greater potential to contribute to the science of nursing education. Being as explicit as possible will increase the contribution made by reporting the pilot study.

Engaging in the practice of conducting a pilot study prior to a larger study should identify major issues that could influence the outcome of the larger study and enable nurse scientists to address them expeditiously. Doing so enhances the possibilities of generating nursing education research that can accelerate the forward movement of the science of nursing education—an outcome to which we are all committed!