Figure 1 Projects and scholarly outcomes from the 34 research grants awarded by the Research and Education Foundation of the Canadian Society of Hospital Pharmacists, 1995–2008.
Peter J Zed , Mary H H Ensom , Richard S Slavik , Kerry Wilbur , Salmaan Kanji , Sheri L Koshman , Janice Irvine-Meek , Marc Perreault , Sheryl ZelenitskyABSTRACT
Background:
Pharmacist-led research has grown substantially over the past 10 to 15 years. The Research Grant Program of the Research and Education Foundation of the Canadian Society of Hospital Pharmacists (CSHP Foundation), initiated in 1992, is the only funding opportunity available specifically to members of the Society.
Objective:
To evaluate the status of research projects funded by the Research Grant Program of the CSHP Foundation, to examine the outcomes of these projects, and to determine the opinions of grant recipients regarding this competition.
Methods:
An e-mail survey was sent to each of the 34 hospital pharmacist researchers who received funding from the Research Grant Program of the CSHP Foundation during the period 1995 to 2008. Survey questions sought to evaluate scholarly outcomes (i.e., publications and presentations) from funded projects. The opinions of grant recipients about the value of the program were also solicited.
Results:
One of the potential respondents had returned the grant money and was ineligible for the survey. Of the 33 potential respondents, 30 (91%) responded to the survey. Overall, 24 of the projects had been completed at the time of the survey, and 19 of these had been published, resulting in a total of 26 manuscripts. Abstracts had been presented for 21 of the projects. In total, 49 abstracts had been presented at national (22), international (13), provincial (7) and local (7) conferences. The median award was $5000 (interquartile range $5000 to $7500). Eleven of the projects had received additional funding, primarily from the recipient’s hospital or health authority or from university sources. The survey respondents indicated that the grant from the CSHP Foundation had been critical to completion of their projects and had been of assistance in securing additional funding, when such funding was necessary. Respondents felt that dedicated research funding for hospital pharmacists in Canada should continue.
Conclusions:
The Research Grant Program of the CSHP Foundation has been important to hospital pharmacists, enabling a variety of research projects to be initiated and completed. The high rate of project completion and the large number of publications and presentations resulting from this work speak to both the quality of the work and the dedication of the research teams. The CSHP Foundation should continue to fund this competition and should explore a more robust model, with larger awards and more funded projects each year.
KEYWORDS: hospital pharmacy , research , funding , grants , survey
RÉSUMÉ
Contexte:
Depuis les 10 à 15 dernières années, les recherches entreprises par les pharmaciens sont considérablement plus nombreuses. Le programme de bourses de recherche offert par la Fondation pour la recherche et l’éducation de la Société canadienne des pharmaciens d’hôpitaux (Fondation de la SCPH), lancé en 1992, constitue la seule source de financement disponible spécifiquement pour les membres de la Société.
Objectifs:
Évaluer l’état des projets de recherche financés par le programme de bourses de recherche offert par la Fondation de la SCPH, analyser les résultats de ces projets et obtenir les opinions des boursiers relatives à ce concours.
Méthodes:
Un sondage par courriel a été envoyé à chacun des 34 pharmaciens d’hôpitaux chercheurs qui ont obtenu du financement du programme de bourses de recherche offert par la Fondation de la SCPH entre 1995 et 2008. Les questions du sondage visaient à évaluer si les résultats du projet de recherche financé avaient fait l’objet d’une publication ou d’une présentation scientifiques. On a également sollicité les opinions des boursiers quant à la valeur du programme.
Résultats:
Un des répondants potentiels ne s’est pas qualifié pour le sondage parce qu’il avait rendu sa bourse. Des 33 répondants potentiels, 30 (91 %) ont répondu au sondage. De l’ensemble des projets, 24 avaient été menés à terme au moment du sondage et 19 de ces derniers avaient fait l’objet d’une publication (pour un total de 26 manuscrits). Des résumés ont été présentés pour 21 de ces projets. Au total, 49 résumés ont été présentés dans le cadre de conférences nationales (22), internationales (13), provinciales (7) et locales (7). Le montant médian de la bourse était de 5000 $ (écart interquartile de 5000 $ à 7500 $). Onze de ces projets ont reçu un financement supplémentaire, principalement de l’établissement ou de la régie de la santé du récipiendaire ou de sources universitaires. Les répondants ont affirmé que la bourse de la Fondation de la SCPH avait été essentielle pour mener à terme leur projet et les a aidés à obtenir du financement additionnel, au besoin. Les répondants ont estimé qu’il faut continuer à allouer du financement aux pharmaciens d’hôpitaux du Canada pour mener leurs projets de recherche.
Conclusion:
Le programme de bourses de recherche offert par la Fondation de la SCPH a été un instrument important pour les pharmaciens d’hôpitaux, leur permettant d’entreprendre et de mener à terme divers projets de recherche. Le taux élevé de projets menés à terme et le grand nombre de publications et de présentations issues de leur travail révèlent la qualité de leur travail et la détermination des équipes de recherche. La Fondation de la SCPH devrait poursuivre le financement de ce concours et étudier un modèle plus généreux, permettant d’octroyer des sommes plus importantes et de financer plus de projets chaque année.
MOTS CLÉS: pharmacie d’hôpital , recherche , financement , bourses , sondage
[Traduction par l’éditeur]
Pharmacist-led research has undergone significant growth in the past 10–15 years.1–3 With this growth has come not only an increasing need to support the education and research training of pharmacists, but also a need for funding opportunities to support this research. In Canada, there are few funding opportunities directed specifically to pharmacists. The Research Grant Program of the Research and Education Foundation of the Canadian Society of Hospital Pharmacists (CSHP Foundation), initiated in 1992, remains a funding opportunity unique to CSHP members.4,5 Annually, a total of approximately $20 000 to $30 000 is available for the competition, and grants are awarded after a peer-review process conducted by the CSHP Grant Review Subcommittee, comprising members of the CSHP Research Committee.
At the time this project was conceived, the outcomes of the research projects funded by the Research Grant Program of the CSHP Foundation were unknown, as were recipients’ perceptions of the benefits of this program in developing their future research initiatives. The objectives of this study were to determine the status of research projects funded by the CSHP Foundation, to examine the outcomes of these projects, and to determine the opinions of grant recipients about this competition.
An e-mail–based survey of pharmacists who had received project funding from the Research Grant Program of the CSHP Foundation was performed. A presurvey e-mail message was first sent to each of the 34 people who had received a research grant from the CSHP Foundation (i.e., the designated principal investigator at the time the grant was awarded) between 1995 and 2008. Three days later, an English-language electronic cover letter and a brief survey (see Appendix 1, available online at www.cjhp-online.ca/index.php/cjhp/issue/view/84/showToc) were distributed to all of the grant recipients. Up to 3 reminders were sent to potential respondents if the survey was not completed and returned by the predefined deadline. Administrative approval was obtained from the CSHP Executive Committee and the CSHP Research and Education Foundation. Ethics approval was obtained from the Capital Health Research Ethics Board, Halifax, Nova Scotia. The consent of respondents was implied if the survey was completed and returned.
The survey was adapted, with permission, from the grant competition survey of the Canadian Association of Emergency Physicians (CAEP)6 (Dr Brian Rowe, Faculty of Medicine and Dentistry, University of Alberta, personal communication, June 2010). The survey consisted of 2 parts: first, a series of questions about the funded project itself (e.g., publications and/or presentations arising from the project; additional funding received), and second, a series of questions to elicit the opinions of the research investigator (i.e., the survey respondent). Specifically, respondents were asked to use a 7-point Likert scale (where 1 = strongly disagree and 7 = strongly agree) to answer a series of questions about the status of their research projects and their opinions regarding the grant competition (see Appendix 1, available online at www.cjhp-online.ca/index.php/cjhp/issue/view/84/showToc).
The research coordinator for the study sent the survey to each grant recipient by e-mail. Each grant recipient was encouraged to complete the survey and return it by e-mail directly to the research coordinator. Each survey was assigned a unique study number, which was used to allow anonymous entry of survey results into an Excel (Microsoft Corp, Redmond, Washington) database. A research assistant performed the data entry, and the data were de-identified before analysis.
The impact factor is a measure used in rating the quality of many biomedical journals. For each journal in which a research project was published, as reported by survey respondents, the impact factor for 2009 was recorded, if available.
The data are reported as descriptive statistics, either as numeric values with respective percentages or as medians with interquartile ranges (IQRs) for data that were not normally distributed. The data were analyzed with Stata Statistical Software, release 10.0 (Stata Corporation, College Station, Texas).
Of the 34 grant recipients to whom a survey was sent, one reported that the funding had been returned to the CSHP Foundation. This person was therefore ineligible for the study. Of the remaining 33 eligible funding recipients, 30 returned the survey, for a response rate of 91%. Twenty-four (80%) of these 30 projects had been completed at the time of the survey (Figure 1). The 6 incomplete projects (4 randomized trials, 1 retrospective chart analysis, and 1 survey) had been terminated by the investigator. Reasons for termination were inability to recruit the necessary patient sample ( n = 1), change in a standard of practice relevant to the study ( n = 2), investigator(s) left clinical practice to pursue a new job or a different practice ( n = 2), competing research priorities ( n = 2), and inability to secure additional funding ( n = 2). Some projects were terminated for multiple reasons.
|
|
||
|
Figure 1 Projects and scholarly outcomes from the 34 research grants awarded by the Research and Education Foundation of the Canadian Society of Hospital Pharmacists, 1995–2008. |
||
For the evaluation period, 1995 to 2008, the Research Grant Program of the CSHP Foundation awarded a total of $223 877, with a median award value of $5000 (IQR $5000–$7500) (data from CSHP National Office). Grants were awarded primarily to those whose highest level of education was PhD (2 [7%]), PharmD (16 [53%]) or residency (9 [30%]) (Table 1). Researchers in Ontario (10 [33%]), British Columbia (8 [27%]), and Quebec (5 [17%]) received the majority of the grants awarded.
Table 1
Characteristics of Researchers and Outcomes of Projects Funded through the Research Grant Program of the Research and Education Foundation of the Canadian Society of Hospital Pharmacists (1995–2008)
Overall, 26 publications and 49 abstract presentations were reported by the 30 respondents. Among the 24 completed projects, 19 (79%) were published; respondents for the remaining 5 projects (21%) indicated that there had been no attempt to publish their results. Reasons for not attempting to publish completed projects included investigator’s relocation to a new job or practice following completion of the project ( n = 3) and lack of statistical support for the data analysis ( n = 1). In one case, the respondent indicated that a manuscript was in preparation, but it had not been submitted. Of the 5 unpublished projects, 4 had been presented as abstracts.
The 26 published articles appeared in 16 different journals (Table 1). Fifteen of these manuscripts were published in a total of 6 journals for which impact factor is not calculated. Of the remaining 11 manuscripts, the median journal impact factor was 3.093 (IQR 1.598–5.873).
Additional funding was obtained from a total of 14 funding sources for 11 (37%) of the projects. The primary sources for this additional funding were hospitals or health care authorities ( n = 5) and universities ( n = 5). Overall, the median additional funding per source was $6500 (IQR $4538–$12 500).
Table 2 outlines respondents’ opinions about the Research Grant Program of the CSHP Foundation. Overall, respondents felt that their study results were important, and many described them as practice-changing. The grant program of the CSHP Foundation was deemed important to the performance of these projects and also helped the researchers to obtain additional or new research funding. There was very strong support for the continuation of designated research funding opportunities for hospital pharmacists in Canada. Finally, respondents indicated that they were likely to apply to the Research Grant Program of the CSHP Foundation again in the future.
Table 2
Opinions of Recipients of Research Grants from the Research and Education Foundation of the Canadian Society of Hospital Pharmacists (CSHP Foundation)
This study represents the first formal evaluation of the scholarly activity and outcomes of projects funded by the Research Grant Program of the CSHP Foundation and of the opinions of funding recipients. For the period 1995–2008, 80% of funded projects were completed, and among those completed, 79% were published, yielding a total of 26 published articles. The articles were published in journals with a relatively high impact factor (median 3.093, for journals for which the impact factor is available). In addition, a total of 49 abstracts had been presented for 87% of the projects, predominantly at national and international conferences.
The opinions of funding recipients indicated that grant funding from the CSHP Foundation was critical to completion of their projects; that it had assisted them in securing additional funding, when necessary; and that dedicated research funding for hospital pharmacists in Canada should continue.
The median award value was $5000, with individual grants ranging from $1000 to $14 000. Additional funding (median per source $6500) was obtained for 37% of the projects, primarily from hospitals and health care authorities or universities. This finding suggests that even a funding program with a small median research award can facilitate successful completion of research projects and can result in substantial academic output and perceived practice change. CSHP Foundation grants were also identified, where applicable, to have had a positive impact in terms of the researcher’s ability to secure additional and/or new funding. However, the fact that over one-third of the projects required additional funding (of similar value to the CSHP Foundation grant) suggests that typical research projects in hospital pharmacy need more funding than has traditionally been available through the Research Grant Program of the CSHP Foundation. This study did not explore reasons why additional funding was needed. However, we postulate that researchers may have had pre-existing funding from another source which was known at the time of the CSHP Foundation grant application, or they may have sought and obtained additional funding (after the CSHP Foundation grant was awarded) to allow completion of the project. Regardless, the CSHP Foundation may wish to explore the possibility of increasing the size of individual grant awards to ensure that funded projects can be completed without the need to seek additional funding.
The rate of noncompletion of projects was relatively high (20%). All of these studies had been terminated by the investigator. Inability to recruit sufficient subjects, changes in the standard of practice, changes in the investigator’s job or clinical practice, competing research priorities, and inability to secure additional funding were the reasons for project termination (with several projects being affected by more than one of these factors). Interestingly, 4 of the 6 terminated projects were randomized controlled trials, which may suggest that the complexity of conducting this type of research also contributed to discontinuation, particularly if resources were limited or exhausted before completion of the project, or if study logistics were challenged by poor enrolment of participants or lack of dedication to the project by the research term.
Equally concerning was the fact that 21% of completed projects had not been published in full article form, particularly given that none of these award recipients reported any attempts to publish their results, with only one recipient reporting that a manuscript was in progress (but not yet submitted). Although 4 of these 5 projects had been presented in abstract form, it remains imperative that research results, whether positive, negative, or neutral, be disseminated through publication of complete manuscripts. Cited reasons for failure to publish completed work were not usually related to the results of the studies themselves, but rather to factors related to the investigator or research team. However, part of a researcher’s obligation to any funding agency, and one aspect of the perceived return on investment for the sponsor and the study participants (i.e., patients) is to ensure that the results are published. In fact, applicants to the Research Grant Program of the CSHP Foundation must agree to such a stipulation when they submit their grant applications.
This analysis of outcomes of a grant funding program for pharmacy research is unique, and we were unable to identify a similar study by any other pharmacy funding agency in Canada or the United States for benchmarking purposes. However, we were able to compare our results with those of a survey evaluating the 60 projects funded by the CAEP grant program over the period 1996–2005.6 The results of that analysis were similar to the findings reported here. Overall, 88% of funded projects were completed, of which 76% were published (in journals with a median impact factor of 3.755), and 92% were presented in abstract form at national or international conferences. The median award was $4700, and 33% of the projects had additional funding, primarily from governments or charitable organizations (median additional funding per source $5116).6
The strength of the study reported here was its high response rate, with 91% of grant recipients responding to the survey. Also, the ability to objectively confirm published articles and abstracts reduced the effects of recall bias and allowed accurate reporting of the scholarly output of each project. Finally, the availability of accurate records of all previous award winners, with up-to-date contact information, from the CSHP National Office ensured that all award recipients were invited to participate.
This study did have limitations. First, given the time frame of the survey, there appeared to be a degree of recall bias, with some investigators being unable to accurately outline details of projects performed many years ago. This problem arose primarily with regard to details about the value of additional funding, which would not affect our main findings but precluded a complete assessment of this aspect of the projects. Second, use of the journal impact factor as a measure of the quality and impact of individual studies is controversial. In addition, most of the projects were published in journals for which impact factor is not calculated, including the Canadian Journal of Hospital Pharmacy , where 9 of the projects were published. Finally, although we did explore grant recipients’ perception of the value of the funding program, we did not formally assess the role of the program in subsequent projects or the research career of grant recipients.
The CSHP Foundation Research Grant Program has been important to hospital pharmacists, enabling a variety of research projects to be initiated and completed. The substantial scholarly activity (i.e., publications and presentations) associated with the completed projects speaks to the quality of the work and also to the dedication of these research teams. CSHP members who have benefited from grant funding value the Foundation’s Research Grant Program and believe that this unique funding opportunity enabled completion of their projects. The perceived value of this program and the fact that many projects required additional funding beyond the CSHP Foundation grant award would suggest that the CSHP Foundation should continue to fund this competition, exploring a more robust funding model, with larger awards and more funded projects each year.
1
Marinac JS, Kuo GM. Characterizing the American College of Clinical Pharmacy practice-based research network.
Pharmacotherapy
2010; 30(8):264e–273e.
2
American College of Clinical Pharmacy Research Affairs Committee; Fagan SC, Touchette D, Smith JA, Sowinski KM, Dolovich L, Olson KL, et al. The state of science and research in clinical pharmacy.
Pharmacotherapy
2006;26(7):1027–1040. 
3
Touchette DR, Bearden DT, Ottum SA. Research publication by pharmacist authors in major medical journals: changes over a 10-year interval.
Pharmacotherapy
2008;28(5):584–590.
4 Canadian Society of Hospital Pharmacists, Research and Education Foundation. Application for grants. Ottawa (ON): The Society; [cited 2011 Mar 29]. Available from: http://cshpfoundation.ca/web/la/en/pa/1A15033B82CA4C06BCC18C6179E0DCEB/template.asp
5 Canadian Society of Hospital Pharmacists, Research and Education Foundation [home page]. Ottawa (ON): The Society; [cited 2011 Mar 29]. Available from: www.cshpfoundation.ca/
6
Bawden J, Manouchehri N, Villa-Roel C, Grafstein E, Rowe BH. Important return on investment: an evaluation of a national grants competition in emergency medicine.
CJEM
2010;12(1):33–38.
The authors would like to acknowledge the Board of the Canadian Society of Hospital Pharmacists Research and Education Foundation and the Executive Committee of the Society for support to carry out this project; Corinne DeMone, Department of Emergency Medicine, Dalhousie University, for administrative support as Research Coordinator for this study; Dr Emily Black, who (at the time of the study) was a Doctor of Pharmacy student, Faculty of Pharmaceutical Sciences, The University of British Columbia, for assistance in carrying out this survey; and Dr Brian Rowe, Faculty of Medicine and Dentistry, University of Alberta, and the Canadian Association of Emergency Physicians (CAEP), for supplying permission to adapt the survey tool.
Canadian Journal of Hospital Pharmacy , VOLUME 64 , NUMBER 6 , November-December 2011