Donna K Buna , Rumi McGloin , Peter S Loewen , Aleisha K Thornhill , Edward D Dillon , Jennifer HawkesABSTRACT
Background:
The Canadian Hospital Pharmacy Residency Board (CHPRB) first introduced accreditation standards to guide the development and maintenance of pharmacy residency programs in the 1970s. These standards have evolved over the years, including a comprehensive revision in 1998. Ongoing quality assessment of residency training is necessary to identify areas for improvement and to ensure that residency training keeps pace with continual changes in pharmacists’ roles.
Objective:
To characterize the practices of graduates of the British Columbia Pharmacy Practice Residency Programs (BCPPRPs), to review the effect of residency training on their careers, and to assess whether the BCPPRPs achieved their educational goals with respect to the 2010 CHPRB accreditation standards.
Methods:
An 18-question electronic survey was sent by e-mail to all graduates of the BCPPRPs from inception in 1973 to 2009. Both quantitative and qualitative data were collected, so descriptive, narrative, and formal mixed-methods approaches were used for the analysis.
Results:
Of the estimated 490 graduates, 144 (29%) completed the survey. About 95% of respondents (125 of 131) agreed that the BCPPRPs had adequately prepared them for a career in hospital pharmacy practice. A majority strongly or somewhat agreed that their respective programs had allowed them to aquire all core competencies. There was less agreement about the achievement of project management and leadership skills. Analysis of these results by eras (specifically, before and after the 1998 revision of the CHPRB accreditation standards) showed statistically significant improvements in achievement for 2 of the educational outcomes, “providing evidence-based direct patient care as a member of an inter-professional team” ( p = 0.001) and “adequate training in literature appraisal” ( p = 0.005).
Conclusions:
A majority of respondents recognized that their residency program had a substantial impact on their hospital pharmacy careers, especially in terms of developing direct patient care skills. The 2010 standards have introduced a specific outcome related to leadership skills. These results indicate that both the CHPRB and specific residency programs would benefit from discussions clarifying outcomes related to project management skills.
KEYWORDS: pharmacy residency , survey , hospital residency
RÉSUMÉ
Contexte:
Le Conseil canadien de la résidence en pharmacie d’hôpital (CCRPH) a introduit au cours des années 1970 des normes d’agrément visant à guider le développement et le maintien des programmes de résidence en pharmacie. Ces normes ont évolué au cours des ans, dont une révision complète en 1998. L’évaluation continue de la qualité de la formation des résidents est nécessaire afin de cerner les points à améliorer et de veiller à ce que la formation tienne compte des changements continus dans les rôles des pharmaciens.
Objectif:
Caractériser les pratiques des diplômés des British Columbia Pharmacy Practice Residency Programs (BCPPRP), examiner l’influence de la résidence sur leurs carrières et déterminer si les BCPPRP ont atteint leurs objectifs éducatifs en regard des normes d’agrément de 2010 du CCRPH.
Méthodes:
Un sondage électronique comportant 18 questions a été envoyé par courriel à tous les diplômés des BCPPRP depuis le début des programmes en 1973 jusqu’en 2009. Des données quantitatives et qualitatives ont été recueillies et par conséquent l’analyse a reposé sur une approche descriptive, narrative et de méthode mixte.
Résultats:
Du nombre total de diplômés estimé à 490, 144 (29%) ont répondu au sondage. Environ 95 % des répondants (125 sur 131) ont affirmé que les BCPPRP les avaient préparés adéquatement à exercer leur profession de pharmacien d’hôpital. Une majorité était entièrement ou plutôt d’accord pour dire que leur programme respectif leur avait permis d’acquérir toutes les compétences principales. Le degré d’assentiment était toutefois moins élevé en ce qui a trait à l’acquisition des compétences en gestion de projet et des compétences de leadership. L’analyse de ces résultats par ères (particulièrement, avant et après la révision de 1998 des normes d’agrément du CCRPH) a révélé des améliorations statistiquement significatives dans l’atteinte de deux résultats pédagogiques, soit « la prestation de soins directs aux patients en tant que membre d’une équipe interprofessionnelle » ( p = 0.001) et « la formation adéquate en évaluation de la littérature » ( p = 0,005).
Conclusions:
Une majorité de répondants ont reconnu l’influence considérable qu’a eue le programme de résidence sur leur carrière en pharmacie d’hôpital, particulièrement pour ce qui est de développer des compétences en soins directs aux patients. Les normes de 2010 ont introduit un critère propre aux compétences de leadership. Ces résultats signalent que le CCRPH aussi bien que les programmes de résidence individuels auraient avantage à préciser les attentes souhaitées en matière de gestion de projet.
MOTS CLÉS: résidence en pharmacie , sondage , résidence hospitalière
[Traduction par l’éditeur]
Pharmacy residency training in Canada has undergone some major transformations since its inception in the 1970s. The focus of the initial residency programs was to prepare pharmacists for hospital practice. The Canadian Hospital Pharmacy Residency Board (CHPRB) first introduced a set of accreditation standards to guide the development and maintenance of pharmacy residency programs in the 1970s.1 These standards have evolved over the years, and in 1998, the standards underwent a major revision to place more emphasis on direct patient care skills. Ongoing quality assessment of residency training is necessary to identify areas for improvement and to ensure that residency training keeps pace with the continual changes in pharmacists’ roles. For example, in 2001, the CHPRB conducted a learning needs assessment of residents who had completed an accredited program in 1998, 1999, and 2000.2 The results of this needs assessment demonstrated strong agreement between the revised accreditation standards and the achievement of knowledge and skills in direct patient care, drug distribution, drug information, and drug use control. However, many of the residency graduates indicated that their residency program had not offered adequate opportunities for developing teaching skills. Teaching skills were subsequently incorporated into the standards (2010 version).
Since the early 1980s, the British Columbia Pharmacy Practice Residency Programs (BCPPRPs) have been cooperatively coordinated by a committee consisting of coordinators from the individual programs. The committee members share resources and expertise, provide support and consultation to one another, and ensure quality and uniformity in residency training in British Columbia. Residents, preceptors, residency coordinators, and directors of pharmacy continuously and rigorously evaluate the programs during and immediately following completion of each residency year. However, to date, the results of these evaluations have not been assessed longitudinally in a systematic manner. In preparation for the development of the 2010 CHPRB accreditation standards, the BCPPRP committee conducted an environmental scan by surveying all graduates of BC residency programs. The overall goal of this survey was to determine the degree to which the BCPPRPs prepared graduates for practice.
More specifically, the BCPPRP committee set out to characterize the practices of BCPPRP graduates, to review the impact of residency training on their careers, and to assess whether the BCPPRPs had achieved their educational goals. Because the potential participants in this study had graduated over a period of many years and had received their training under the influence of a variety of historical standards, the educational outcomes of the 2010 CHPRB standards were used as the benchmarks for comparison.
At the time of the survey, there were 5 pharmacy practice residency programs in British Columbia, with the total number of residents ranging from 26 to 30 per year. The 5 programs were run by Vancouver Coastal Health/Providence Health Care, Fraser Health Authority, Children’s & Women’s Health Centre of British Columbia, Northern Health Authority, and Vancouver Island Health Authority. The Interior Health Authority was in the process of establishing a new residency program for the 2011/2012 year.
An electronic survey was developed by the BCPPRP committee to gather data related to the study objectives.
The survey consisted of 18 questions (Appendix 1, available online at www.cjhp-online.ca/index.php/cjhp/issue/view/84/showToc). The first 4 questions established the demographic characteristics of the respondent. Questions 5 to 10 explored the achievement of key educational outcomes and skill sets and were based, in part, on the 2010 CHPRB standards. This section of the survey also included a question regarding attainment of literature appraisal skills, even though such skills were not included in the 2010 CHPRB standards, because they were deemed to represent a key competency in providing evidence-based direct patient care. Questions 11 to 15 characterized the respondent’s current practice area and his or her pursuit of further education. Questions 16 and 17 explored the effects on past graduates of the controversial new service contracts, which were introduced in 2009 for selected health authorities,3,4 as well as respondents’ opinions about the contracts. Question 18 invited additional comments. A draft of the survey questions was reviewed by a random sample of pharmacists in various BC health authorities to assess formatting, readability, and face validity.
The study population consisted of BCPPRP graduates from the year of the program’s inception (1973) to 2009 (i.e., graduating cohort of the 2008/2009 residency year). No formal database of this population existed, so the coordinators of the individual programs attempted to collect e-mail contact information through various means for as many graduates as possible. The BCPPRPC committee sent an invitation to the clinical leaders of each BC health authority to participate in the survey. These individuals then forwarded the survey via e-mail to all pharmacy staff within their respective health authorities. It was recognized that this method would miss graduates practising out of province at the time of the survey. To improve the response rate, the survey was sent out 2 additional times, as a reminder to those who had already received the survey but had not completed it and to capture respondents who may have been missed with previous distributions.
Both quantitative and qualitative data were collected, so descriptive, narrative, and formal mixed-methods approaches were used for the analyses.
The mixed-methods approach had the following characteristics: concurrent data collection, with priority given to quantitative data because the qualitative submissions were voluntary and because the qualitative results were partially integrated during the analysis stage.5 Where relevant, qualitative responses were stratified according to respondents’ levels of agreement with the question and were then analyzed to identify themes.
Percentages were calculated on the basis of the number of respondents with similar characteristics in their career path and to indicate the proportion who “strongly agreed” or “somewhat agreed” that their residency had achieved specified educational outcomes, with the 2010 CHPRB standards as a single point of reference.
Because the standards had undergone a major revision in 1998 to place stronger emphasis on direct patient care, responses related to educational outcomes were compared between respondents who graduated in 1998 and earlier and those who graduated after 1998. Two-sided Pearson χ2 analysis was used to compare responses concerning educational outcomes between respondents from the different eras.
Opportunities to improve the quality of the residency programs were sought by inviting respondents to provide comments.
It was estimated that the total size of the survey population was 490 graduates. Of these, 144 (29%) responded to the survey. About two-thirds (96/142 [68%]) of the respondents had graduated with a Bachelor of Science degree within the previous 1 or 2 years and most (131/142 [92%]) had graduated from the University of British Columbia. The demographic characteristics of respondents are presented in Table 1.
Table 1
Demographic Characteristics of Survey Population*
Sixty-four percent (72/112) of the respondents spent at least half of their time in a clinical role, and 12% (13/112) had all of their time allocated to a clinical role. Seventeen percent (16/93) of respondents were in a dedicated administrative role, whereas only 4% (4/102) were in a dedicated distribution role. Further details on the career paths of past BCPPRP graduates appear in Table 2.
Table 2
Characteristics of Career Paths Followed after Completion of Residency Program
The majority of respondents agreed that the BCPPRPs had adequately prepared them for a career in hospital pharmacy practice (125/131 [95%]) and that the program had allowed them to acquire the core competencies described in the 2010 CHPRB accreditation standards (Table 3). In addition, 65% of respondents (85/131) agreed that the BCPPRPs had provided them with adequate training in literature appraisal. Analysis of these results by era (graduating year 1973–1998 versus 1999–2009) showed only one CHPRB educational outcome for which achievement was significantly different before and after the 1998 revision of accreditation standards (“provide evidence-based direct patient care as a member of an inter-professional team”; 70% for 1973–1998 cohort versus 91% for 1999–2009 cohort, p = 0.001). In addition, there was a statistically significant difference between eras in terms of adequacy of training in literature appraisal (55% of 1973–1998 cohort versus 80% of 1999–2009 cohort, p = 0.005) (Table 4).
Table 3
Respondents’ Views of Extent to Which Their Respective Residency Programs Met Educational Standards of the Canadian Hospital Pharmacy Residency Board (CHPRB)
Table 4
Educational Outcomes by Era*
For the 4 questions for which qualitative data were elicited from respondents, the thematic analysis is presented in Table 5. The following 3 main themes were deduced from the responses: since the inception of the BCPPRPs, graduates’ skills in literature evaluation, teaching, and leadership and administration have improved; the BCPPRPs have evolved but sometimes fall short in terms of meeting the current demands of the profession; and service contracts remain controversial, and it is unclear how they will affect the applicant pool and matching process in the future.
Most of the survey respondents had graduated in the 10 years preceding the survey and were less than 25 years old when they completed the residency program. The relative abundance of recent graduates can be attributed, in part, to an increase in the number of residency positions over the 10 years preceding the survey, specifically through new program starts in the Vancouver Island Health Authority and the Northern Health Authority. Almost all of the respondents had gone on to practise at a hospital site with clinical and/or administrative responsibilities. This finding highlights the importance of the role that residency programs continue to play in training pharmacists for practice in the hospital setting. A large percentage of respondents had primary responsibilities in clinical practice rather than administration. This may be a consequence of the relatively young age of the respondents: younger practitioners have yet to evolve and pursue higher leadership and managerial roles.
Most respondents (95%) agreed that the residency had adequately prepared them for hospital practice. This result aligns with the learning needs assessment carried out by the CHPRB in 2001.2 However, less than 80% of the cohort of residents who graduated from 1999 to 2009 agreed that the BCPPRPs had adequately achieved the educational outcome of “exercise leadership”. The concern about lack of adequate preparation in leadership skills was also evident in the thematic analysis. Nonetheless, there was some indication that these deficiencies had been addressed in more recent years. With the amalgamation of 3 of the oldest programs in the province into Lower Mainland Pharmacy Services, attempts are being made to increase and assimilate exposure time to administration activities such as evaluation of medication use. The Vancouver Island Health Authority has adopted a longitudinal schedule for leadership and management activities that is more in line with “real-life” scenarios. Although residents are often encouraged and expected to take on leadership roles, and many of them become leaders of the profession after completion of their residency, formal and dedicated objectives related to skills and competencies in leadership and management have been minimal in past accreditation standards. There is substantial concern about the declining interest in pharmacy management and the ongoing vacancies in formal leadership positions. The lack of leaders and managers who could act as role models and provide learning opportunities for residents may explain why respondents felt less prepared for these roles. In 2008, the Canadian Society of Hospital Pharmacists Hospital Pharmacy Management Task Force published a preliminary report outlining factors contributing to this decline.6 The Task Force suggested some strategies to address this concern, one of which—to include standards related to leadership and management skills—was directed to the CHPRB. Such a standard was incorporated into the 2010 CHPRB accreditation standards.
In both the 1973–1998 and 1999–2009 graduation cohorts, only about three-quarters of respondents agreed that they had acquired project management skills during their residencies. Whereas this survey elicited specific feedback related to the CHPRB’s 2010 accreditation standards, the CHPRB’s 2001 survey2 expanded on the perceived value of the residency project. Results from the earlier survey showed that the residency project was seen as a valuable component of the program, but there was a desire for clearer guidelines and support in conducting research. The 2010 standards specify that “the resident use effective project management skills to undertake, conduct and successfully complete a project related to pharmacy practice”. The requirements of this standard include involvement in project development and collection, analysis, and interpretation of data; submission of a written report in a format suitable for publication in a peer-reviewed journal; and presentation and defence of the outcomes of the project. The achievement of all of these objectives within a single residency year is often difficult and a source of significant stress for many residents. This may be an area that requires further exploration and standardization among the BCPPRPs. If the educational goal is to allow the resident to develop research and publication skills, the applicable accreditation standards and requirements may require revision. If the educational goal is to allow the resident to develop project management skills, the accreditation standards may have to be more general.
The analysis of educational outcomes by era showed that the program was significantly more effective in preparing later graduates (1999–2009 era) for the standard of “providing evidence-based direct patient care as a member of inter-professional teams” and in “adequate training in literature appraisal”. These results demonstrate that the establishment of specific educational goals and objectives under the new 1998 accreditation guidelines had a significant impact on subsequent training and practice of residency graduates.
Although there is no specific educational standard related to literature appraisal, it is a critical skill needed to practise evidence-based direct patient care. The evolution of literature appraisal skills to support evidence-based practice has grown substantially over the past 10 years. The thematic analysis showed that respondents recognized that this deficiency in training had been addressed in recent years.
Service contracts were first introduced at the start of the 2009/2010 academic year in 3 of the health authorities: Vancouver Coastal Health/Providence Health Care, Fraser Health Authority, and Vancouver Island Health Authority. A service contract is a training and employment agreement between the resident and the health authority. In the contract, the health authority agrees to provide 1 year of training in an accredited program, which includes a repayable salary. Repayment of the salary is waived if the resident completes a minimum of 1 year of “service” or employment after graduation. It appears that a service contract would have influenced matching decisions for the majority of respondents for a variety of reasons, including uncertainty and loss of control over choices after graduation and the perceived shift from an educational or academic focus to a recruitment focus. Although the majority of respondents indicated that a service contract would have decreased the appeal of a residency in British Columbia or delayed their decision to pursue a residency in this province, many of them would still have applied to a BC program. Service contracts remain a part of the residency program in Lower Mainland Pharmacy Services (an amalgamation of the programs of Children’s & Women’s Health Centre of British Columbia, Vancouver Coastal Health/Providence Health Care, and the Fraser Health Authority), but they are not part of the residency programs of Vancouver Island Health Authority, the Northern Health Authority, or the Interior Health Authority. Despite the use of service contracts by Lower Mainland Pharmacy Services, interest in all BC programs remains high, and the quality of applicants is assured by the rigorous application and interview process.
The primary limiting factor of this analysis was the low response rate. An estimated 490 residents have completed the various BC residency programs since their inception, but only 144 of those graduates responded to the survey. The low response rate might have been due, in part, to the fact that the survey was forwarded to a liaison in each of the province’s 6 health authorities, who was then required to forward the survey to pharmacy departments within the respective region. This sampling strategy may seem haphazard, but it was the most efficient way to reach as many residency graduates as possible. We recognize that with this approach, we may have missed out-of-province practitioners and graduates who changed career paths from hospital pharmacy to industry or community practice. As previously mentioned, the BCPPRPs have adapted to the ever-changing practice of pharmacy in the hospital setting. Thus, respondents from the early programs were not trained to the current educational outcomes outlined in our survey. This may have resulted in an inherent recall and selection bias.
In this survey of pharmacy practice residency graduates, the majority of respondents recognized the significant impact of the residency on their hospital careers. Despite ongoing revisions in targeted educational outcomes, the majority of pharmacists reported that their educational needs had been met during their residency year. Many pharmacists who had completed one of the BCPPRPs went on to further education, particularly in advanced clinical practice. A select few pursued administrative and leadership roles, an area that was previously identified as needing enhancement in the program. The new 2010 standards have specifically identified leadership skills as a key educational focus. Project management was another area that was previously identified as needing further development. Subsequent delineation of feasible learning outcomes for this standard improved residents’ experiences. Lastly, educational activities related to literature appraisal should continue to be a focus of all programs, to meet the standard of delivering evidence-based direct patient care. These observations provide a foundation for comparing the impact of future changes to pharmacy residency programs, especially as pharmacists’ scope of practice continues to evolve.
1 Stieb E. The CCPF participates jointly with the CSHP in the CHPRB. In: Riedel B, Stieb E, editors. A history of the Association of Faculties of Pharmacy of Canada. The first fifty years 1944–1994 . Edmonton (AB): Association of Faculties of Pharmacy of Canada; 2001. p. 85-87.
2 Moy D, Musing E. Canadian pharmacy practice residencies: a learning needs assessment. Can J Hosp Pharm 2003;56(5):259-266.
3 Ensom RJ. Should service contracts be invoked after completion of a hospital pharmacy residency program? The “pro” side. Can J Hosp Pharm 2008;61(4):278-279.
4 Koleba T. Should service contracts be invoked after completion of a hospital pharmacy residency program? The “con” side. Can J Hosp Pharm 2008;61(4):279-280.
5 Creswell J. Research design: qualitative, quantitative and mixed methods approaches . 2nd ed. Thousand Oaks (CA): Sage; 2003.
6 CSHP Hospital Pharmacy Management Task Force. Planning for hospital pharmacy management, now and in the future. Can J Hosp Pharm 2008;61(5):374-384.
Canadian Journal of Hospital Pharmacy , VOLUME 64 , NUMBER 6 , November-December 2011
