Lauren Bresee
I have been fortunate to work with a number of novice researchers during my career, including pharmacy residents, graduate students, and pharmacists. Supporting newer researchers is one of my favourite activities: it is very fulfilling to mentor individuals who are interested in conducting research. During my work with novice researchers, they have shared some misconceptions related to research. One of the most common of these misconceptions is the belief that a study without positive results was unsuccessful and therefore uninformative.
For the purpose of this editorial, a positive study is defined as one in which the intervention has statistically significant beneficial results relative to the comparator, a negative study is defined as one in which a statistically significant harm was found, and a neutral study is defined as one in which there was no statistically significant difference between the intervention and the comparator.1 The caveat for all these definitions is that the study must have been properly designed and conducted: specifically, the study was adequately powered to detect a statistical difference if one exists, an appropriate comparator was used, and the correct statistical tests were used for the particular study design and the data being analyzed.
Publication bias refers to the likelihood that studies with positive results are more likely to be published than studies with negative or neutral results.2 The issue of publication bias is well known, and its manifestations can range from purposely withholding the dissemination of negative studies to the inability to get negative or neutral studies published in peer-reviewed journals because they are rejected before or after peer review.3 It is difficult to quantify the frequency of the reasons for publication bias relating to negative and neutral studies; however, the belief that a study will not be accepted for publication because of its results likely contributes to publication bias through lack of submission of these manuscripts to peer-reviewed journals.2
Why are negative studies important? As health care providers, we instinctively want to offer our patients the most effective technologies to support them in achieving their goals. We also must be realistic about the finite resources available for health care in Canada, a constraint that further emphasizes the need to provide patients with interventions that will truly benefit them. Neutral and negative studies help us to identify therapies that are ineffective or harmful, so that we can ensure our patients do not receive these therapies. As a result, these studies play an integral role in helping us to determine the appropriate care for our patients.
How can we address negative attitudes toward negative studies? Of the recommendations listed below, the first two apply to all research, but are particularly important when considering studies with a negative or neutral outcome. First, researchers must properly plan and conduct their studies, to ensure the validity of the results. If a negative or neutral result is obtained, proper conduct of the study allows us to be confident that the result is valid and not due to a fatal flaw, such as inadequate sample size. Second, researchers must approach their investigations with personal equipoise, meaning that the researcher “has no personal preconceived preferences toward the ability of one or more of the interventions to have a better outcome than another”.4 Approaching a research project with equipoise may help to ensure there are no preconceived beliefs that the research can be considered successful only if a benefit is found, and hopefully will encourage the researcher to recognize that the study results are important regardless of the outcome, and therefore should be disseminated. This path can be particularly challenging when the researcher has a personal stake in the intervention being evaluated. An example would be the case of a research team evaluating the effectiveness of a program that they have designed for their patients’ benefit. If such a program is found to be ineffective or harmful, the researchers may be hesitant to release the study results. However, it is imperative to share these results with other health care providers who may be offering similar programs to their patients, so that those health care providers can make an informed decision as to whether to continue offering the similar program. The third recommendation is perhaps obvious, but researchers should submit their negative and neutral studies for publication. Journal editors are becoming more aware of the issue of publication bias and are therefore interested in publishing negative and neutral studies. As one example, the American Journal of Gastroenterology published a “negative issue” in November 2016, in which all of the articles had statistically negative or neutral outcomes.5 The Canadian Journal of Hospital Pharmacy welcomes the submission of negative and neutral studies. Finally, we must increase the recognition among all clinicians that negative studies are necessary to the advancement of clinical practice. We cannot expect all interventions to be beneficial, and we therefore need negative and neutral studies to help us to identify the most effective interventions for our patients.
1 Axelson O. Negative and non-positive epidemiological studies. Int J Occup Med Environ Health. 2004;17(1):115–21.
2 Olson CM, Rennie D, Cook D, Dickersin K, Flanagin A, Hogan JW, et al. Publication bias in editorial decision making. JAMA. 2002;287(21):2825–8.
3 Dwan K, Altman DG, Arnaiz JA, Bloom J, Chan AW, Cronin E, et al. Systematic review of the empirical evidence of study publication bias and outcome reporting bias. PLoS One. 2008;3(8):e3081. 
4 Cook C, Sheets C. Clinical equipoise and personal equipoise: two necessary ingredients for reducing bias in manual therapy trials. J Man Manip Ther. 2011;19(1):55–7.
5 The American Journal of Gastroenterology presents “The Negative Issue”. Twenty-five negative studies that remind readers that “negative is positive” [news release]. Bethesda (MD): American College of Gastroenterology; 2016 [cited 2017 Aug 23]. Available from: www.prnewswire.com/news-releases/the-american-journal-of-gastroenterology-presents-the-negative-issue300356821.html
Competing interests: None declared. ( Return to Text )
Canadian Journal of Hospital Pharmacy, VOLUME 70, NUMBER 6, November-December 2017