FIGURE 1 Flow diagram for literature review. The sum of reasons for exclusion after full-text review (902) is greater than the number of articles excluded at this step (762) because some articles were excluded for more than one reason.
Patrick JF Deschênes, My Hanh Luu Hoai, Justine Rinfret, Sarah Pelletier, Alessandra Stortini, Rachel Therrien, Marie-Kim Héraut, Pascaline Bernier, and John Nam-Kha NguyenTo cite: Deschênes PJF, Luu Hoai MH, Rinfret J, Pelletier S, Stortini A, Therrien R, et al. Development of a competency assessment framework for pharmacy residents participating in high-fidelity simulation and creation of the SIMPHARM assessment tool. Can J Hosp Pharm. 2024;77(3):e3552. doi: 10.4212/cjhp.3552
ABSTRACT
Background
Simulations are used to improve professional practice across multiple health professions; however, the application of simulations in pharmacy curricula has been limited by a lack of evidence.
Objectives
To delineate the competencies of pharmacy residents needed to assess their progress while participating in a high-fidelity simulation training program and to create a reliable assessment tool based on these competencies.
Methods
A literature review was conducted, followed by application of a modified Delphi method. An assessment tool based on the results of these activities was drafted (in French). A second modified Delphi process was carried out to assess the reliability of the tool, and the tool was modified accordingly.
Results
The literature search yielded a total of 2670 articles, of which 289 were included for analysis. The first modified Delphi process involved 19 experts in the first round and 10 experts in the second round. The Cronbach α was 0.866 (95% confidence interval [CI] 0.713–0.960), indicating good reliability. A total of 7 competencies were retained for inclusion in the SIMPHARM assessment tool: professionalism, leadership, teamwork, communication, critical thinking, preparation and packaging of medications, and pharmaceutical calculations. The second modified Delphi process involved 9 experts in the first round and 11 experts in the second round. The final Cronbach α was 0.877 (95% CI 0.741–0.960).
Conclusions
To the authors’ knowledge, this was one of the largest studies exploring pharmacy competencies in the context of simulations. This work yielded a reliable framework for the assessment of pharmacy residents’ competencies. This assessment tool may help evaluators in assessing the competencies of pharmacy trainees after simulation training.
KEYWORDS: assessment tool, competencies, high-fidelity simulation, nontechnical skills, pharmacy
RÉSUMÉ
Contexte
Les simulations sont utilisées dans le but d’améliorer la pratique professionnelle de plusieurs professions de la santé; cependant, le manque de données probantes limite leur application dans les programmes d’enseignement en pharmacie.
Objectifs
Décrire les compétences nécessaires pour évaluer le progrès des résidents en pharmacie qui participent à une formation basée sur un programme de simulation haute-fidélité; et mettre au point un outil d’évaluation fiable qui se base sur ces compétences.
Méthodologie
Une revue de littérature a été effectuée, avant d’appliquer une méthode Delphi modifiée et de faire l’ébauche d’un outil d’évaluation (en français) basé sur les résultats de ces activités. Une deuxième méthode Delphi modifiée a été appliquée pour évaluer la fiabilité de l’outil et celui-ci a fait l’objet de modifications en conséquence.
Résultats
La revue de littérature a donné un total de 2670 articles, parmi lesquels 289 ont été inclus à des fins d’analyse. La première méthode Delphi modifiée a impliqué 19 experts pour la première phase, et 10 pour la seconde. Le coefficient alpha de Cronbach était de 0,866 (intervalle de confiance [IC] à 95 % 0,713–0,960), indiquant une bonne fiabilité. Au total, 7 compétences à inclure dans l’outil d’évaluation ont été retenues: professionnalisme, leadership, travail d’équipe, communication, pensée critique, préparation et conditionnement des médicaments, et calculs pharmaceutiques. La deuxième méthode Delphi modifiée impliquait quant à elle 9 experts pour la première phase et 11 pour la seconde. Le coefficient alpha de Cronbach final était de 0,877 (IC à 95 % 0,741–0,960).
Conclusions
À la connaissance des auteurs, cette étude était l’une des plus importantes se penchant sur les compétences en pharmacie dans le contexte des simulations. Ce travail a donné lieu à un cadre de référence fiable pour évaluer les compétences des résidents en pharmacie. Cet outil d’évaluation pourrait aider les évaluateurs à évaluer les compétences des stagiaires à la suite d’une formation par simulation.
Mots-clés: outil d’évaluation, compétences, simulation haute-fidélité, compétences non techniques, pharmacie
Simulations are educational tools consisting of fictitious (mock) scenarios with various levels of difficulty. In the context of pharmacy, they can improve professional practice by exposing learners to infrequent yet complex pharmacotherapeutic problems in a safe and controlled environment, possibly reducing the number of medical and medication-related errors in future practice.1–4 Overall, the primary focus of simulations is the acquisition of competencies, rather than solely knowledge.1 Although still used predominantly in medical programs, high-fidelity simulations, a form of simulation defined by a high level of realism, have gradually been integrated into nursing and, more sporadically, pharmacy program curricula in North America.5–7
Since 2016, pharmacy residents practising at the Centre hospitalier de l’Université de Montréal have benefited from an extracurricular high-fidelity simulation program. The simulation program aims to complement the pharmacy residents’ curriculum. It allows trainees to experience realistic acute care scenarios without the risk of endangering actual patients. However, the program’s prohibitive cost (up to $800 per simulation, which covers costs for 2 pharmacists, the simulation amphitheatre, and technical support staff) and the lack of proven benefit of simulations in pharmacy have limited the inclusion of this educational tool in the official academic curriculum. Moreover, there exists no valid assessment tool that would allow us to evaluate this simulation program. Of the studies that have evaluated the performance of pharmacy students and residents participating in high-fidelity simulation programs, most have assessed completion of tasks, participants’ perceptions, and knowledge acquisition through self-reported surveys.6 Few have assessed the development of pharmacy trainees’ competencies.8,9
To our knowledge, no studies have used an assessment tool specifically designed to assess pharmacy learners’ competencies in the context of high-fidelity simulations, let alone to evaluate the progression of competencies among pharmacy trainees participating in a high-fidelity simulation training program. Despite the substantial time requirements, financial resources, and human resources needed to orchestrate a pharmacy simulation program, the development and progression of pharmacy residents’ competencies has yet to be proven. To do so, parameters allowing the evaluation of competency acquisition must be determined. The aim of this study was to delineate the competencies suitable for assessing the progress of pharmacy residents participating in a high-fidelity simulation training program and to create a reliable assessment tool that could be used to objectively evaluate such progress.
The local institutional review board granted approval before the start of each phase of the study, and the research was conducted in accordance with the Helsinki Declaration.
Phase 1 of the study involved a literature review and subsequent application of a modified Delphi method. To determine the competencies that must be evaluated to assess learners’ progress, the following databases were searched: MEDLINE, Embase, Education Resources Information Center (ERIC), and Google Scholar. The 4 main themes covered by the search strategy were health sciences students or residents, professional competencies, simulation training, and educational assessment techniques.
Articles assessing the competencies of trainees from any health professional program during simulation training, published between January 2000 and March 2020 in English or French, were eligible for inclusion. The competency frameworks established by North American pharmaceutical associations, as well as by faculties of pharmacy, were also eligible. Included articles were not limited to the original publications of competency assessment tools. However, articles that did not showcase simulations or competencies were excluded, as were conference abstracts and duplicate reports.
Titles and abstracts were independently screened by 2 of the authors (P.J.F.D. and S.P.) to ensure article relevancy. Any disagreement was resolved by a third author (M.H.L.H. or J.R.). The full-text articles were then screened independently according to the inclusion and exclusion criteria in alternating order by 2 pairs of authors (P.J.F.D., SP or M.H.L.H., J.R.). Any disagreements were resolved by a third author (P.J.F.D., M.H.L.H., J.R., or S.P.). One author proceeded to data extraction (S.P.), and a second author (P.J.F.D.) reviewed extracted data in cases of uncertainty. Extracted data consisted of the first author’s name, year of publication, article title, language, country, field of study, level of education of the trainees (student, resident, or fellow), number of participants, and type of simulation. Qualitative data (domains, themes, items, and anchors), as well as quantitative data, were extracted from the assessment tools. A quality assessment was not performed. The qualitative data were analyzed with NVivo software (Lumivero, formerly QSR International).
From the competencies identified in the literature review, a 2-round modified Delphi process was conducted to determine the competencies to be included in the final assessment tool. Based on a preliminary literature review of other Delphi studies,10–12 we decided on a sample of at least 10 to 15 experts for the Delphi rounds in our study. Delphi participants had to meet the following criteria: health care professionals with experience in high-fidelity simulation, a background in education, or both. To ensure a stronger understanding of competencies specific to pharmacy practice, the majority of experts chosen were pharmacists. The first Delphi round was an online survey sent to participants by email. Participants were given 3 weeks to respond. The objectives were to determine the experts’ opinions on the competencies found in the literature review and to collect new ones. To address disagreements in the first round and obtain a final consensus, a second real-time Delphi round was carried out. To date, no optimal consensus threshold has been described in the literature, with suggestions for the consensus threshold ranging from 51% to 80%.13 For the current study, an arbitrary consensus threshold of 70% was established.
For both rounds, the experts were asked to rate the competency statements on a 4-point Likert scale (from “Totally agree” to “Totally disagree”). Experts who did not participate in the first round were excluded from participating in the second round.
A modified Delphi process was carried out to establish the reliability of the assessment tool developed in phase 1. Health care professionals with experience as simulation educators were invited to participate in the Delphi process. Participants received an explanation of how to use the assessment tool before watching a simulation video featuring 2 pharmacy residents in a high-fidelity simulation. Each participant was asked to use the assessment tool to evaluate the performance of the same pharmacy resident. Once all participants had completed their assessments, Fleiss weighted κ and Cronbach α statistics were calculated immediately. If the statistics did not reach the pre-established thresholds, as specified below, the participants were invited to discuss, debate, and modify their answers to achieve consensus. Comments were collected to be used in modifying the tool.
To determine the reliability of the tool, 2 complementary statistics were calculated: the Cronbach α (SPSS software, IBM) for inter-item consistency and the Fleiss weighted κ (Microsoft Excel software) for inter-rater consistency. Values of Cronbach α greater than 0.7 were considered to represent an acceptable level of consensus.14–16 Values of the Fleiss weighted κ greater than 0.2 were considered to represent a fair level of reliability for the agreement between a fixed number of raters when assigning ordinal ratings to a number of items or classifying items.17 Two pharmacists with expertise in Delphi software development (M.H.L.H. and J.R.) generated the data collection tool that was used during the Delphi process carried out in this study. The tool allowed for real-time calculation of statistics, which in turn allowed the Delphi participants to receive rapid feedback and modify their responses if needed.
The literature search yielded 2670 publications. After filtering for duplicates and application of the selection criteria, a total of 289 articles were included (Figure 1). Thirteen of the articles did not involve an assessment tool, so were not included in the quantitative analysis; however, they were retained for the qualitative analysis because they discussed competency frameworks. The characteristics of included articles are shown in Table 1. Overall, 23 (8%) of the articles considered competencies specific to pharmacy practice.
|
| ||
|
FIGURE 1 Flow diagram for literature review. The sum of reasons for exclusion after full-text review (902) is greater than the number of articles excluded at this step (762) because some articles were excluded for more than one reason. | ||
TABLE 1 Characteristics of the Included Literature
Among the individual assessment tools identified in the literature search, the global rating scale (GRS) was the most common type (170/226, 75%). In contrast, among tools assessing competencies specific to pharmacy practice, the most commonly employed assessment were checklists (8/23, 35%) followed by GRS (4/23, 17%). Only 61 (27%) of the 226 assessment tools used a global performance score; however, a majority of the assessment tools (156/226, 69%) provided a weighting of assessment tool items, which allowed calculation of an overall score. Only 5 (2%) of the 226 assessment tools involved evaluation of participants during a debriefing session.
The assessment tools most frequently employed were the Anaesthetists Non-Technical Skills tool18 (reported in 12 articles), the Ottawa GRS19 (12 articles), the Objective Structured Assessment of Technical Skills20 (12 articles), the Non-Technical Skills for Surgeons tool21 (7 articles), and the Lasater Clinical Judgment Rubric22 (7 articles).
The qualitative analysis yielded hierarchy charts of different technical and nontechnical skills and items describing the various skills, as well as specific and nonspecific anchors assessing the level of trainee performance. A hierarchy chart was constructed for our assessment tool (Figure 2). The main nontechnical skills were communication, leadership, professionalism, teamwork, task management, stress management, systems-based practice, critical thinking, and conflict management. The technical skills that stood out were preparation of medicines, dispensing of medications, administration of medications, and pharmaceutical calculations. Discrimination for the level of competency based on complexity of care as well as global performance was also elicited in the qualitative analysis.
|
| ||
|
FIGURE 2 Competency mind map. | ||
We initially identified 34 experts as potential participants in the 2-round modified Delphi process, of whom 22 agreed to participate. Nineteen (86%) completed the first round on June 19, 2020, and 10 (53%) of these 19 completed the second round on August 11, 2020. The experts who participated in the first round included health care professionals from 3 different fields (pharmacy [n = 11], medicine [n = 7], and nursing [n = 1]) and 2 countries (Canada and France). For the second round, the number of pharmacists remained greater than the number of physicians (9 and 1, respectively). Five of the authors (P.J.F.B., M.K.H., A.S., R.T., and J.N.-K.N.) participated in both rounds of the Delphi process. The 10 experts in the second round had a mean of 4 years (range 1–11 years) of experience with high-fidelity simulations, and 8 of them had received high-fidelity simulation training themselves. For the second round, the intraclass correlation coefficient based on the Cronbach α was 0.866 (95% confidence interval [CI] 0.713–0.960; p < 0.001).
Following phase 1 of the study, a total of 7 competencies were retained, categorized as nontechnical and technical skills. The 5 nontechnical skills were professionalism, leadership, teamwork, communication, and critical thinking. The 2 technical skills were preparation and packaging of medications and pharmaceutical calculations. Assessment of debriefing participation and level of entrustment were also added to the assessment tool as evaluation criteria. The experts agreed upon using a 5-point Likert-type scale with specific anchors.
For the assessment of reliability, we invited 28 experts to participate in the first round of the modified Delphi process on February 23, 2023, of whom 9 agreed to participate (all pharmacists from either Canada or France). Five of these participants had experience as simulation educators, and 7 had themselves received simulation training. Six of the authors (P.J.F.D., M.H.L.H., J.R., A.S., R.T., and P.B.) participated in the first round. The intraclass correlation coefficient based on the Cronbach α of 0.581 (95% CI 0.400–0.888; p = 0.02) and the Fleiss weighted κ of 0.031 (95% CI 0.001–0.061) did not reach the prespecified thresholds of 0.7 and 0.2, respectively, after the first round.
Modifications were made to the assessment tool according to participant feedback following the first Delphi round, and a second Delphi round was convened on April 27, 2023. For the second round, 52 experts were invited, of whom 11 participated. These 11 health care professionals were from 4 different fields (pharmacy [n = 8], medicine [n = 1], nursing [n = 1], and respiratory therapy [n = 1]) and 2 countries (Canada and France). Nine of the participants had experience as simulation educators, and these 9 participants had themselves received simulation training. Three of the authors (P.J.F.D., M.H.L.H., and P.B.) participated in the second round of the Delphi process. The intraclass correlation coefficient based on the Cronbach α was 0.877 (95% CI 0.741–0.960; p < 0.001), and the Fleiss weighted κ was 0.079 (95% CI 0.059–0.099). The modified Delphi process was stopped after the second round, because the predefined consensus threshold for the Cronbach α had been reached.
The SIMPHARM assessment tool, developed and validated in French only, is presented in Appendix 1.
To our knowledge, this is the first study outlining the competencies that should be evaluated in order to assess the development of trainee competencies in the context of a high-fidelity simulation program designed for pharmacy residents. Moreover, this study is the first attempting to create a global competency assessment tool for pharmacy residents. Among the studies identified by our literature search, only 3 assessed competencies in pharmaceutical practice: Bajis and others23 used a pretested marking rubric to evaluate communication skills during the medication history interview; Doloresco and others24 evaluated communication skills using a GRS that had been validated for its psychometric properties; and Bailey and others25 evaluated collaboration and communication using a validated assessment tool in the context of motivational interviews. The present study goes beyond communication and interpersonal skills by attempting to identify the full arsenal of competencies that pharmacy residents should master. Among studies specifically assessing pharmacy-related competencies, most evaluated the completion of tasks through the use of checklists rather than through demonstration of specific competencies.
In phase 1 of the current study, fewer experts participated in the second Delphi round, the main reason being that the second round was a real-time process with a predetermined date, time, and non-negligible duration. These restrictions may have limited the experts’ availability. Despite the loss of participants between the first and second rounds, the number of participants remained within the required sample size. All experts, as determined by selection criteria, had either training in high-fidelity simulation or a background in education. The experts’ credentials contributed to the credibility of the Delphi process. The variety in participants’ fields of practice yielded different perceptions on the assessment of pharmacy trainees. Pharmacists and nonpharmacists sometimes had differing opinions on which competencies should be included in the assessment tool. For instance, physicians were more inclined than pharmacists to include knowledge assessment, perhaps because their expectations of pharmacy trainees differed from those of pharmacists. Additionally, physicians’ objectives for carrying out simulations may differ from those of pharmacists. Specific management competencies found in the literature review (self-management, task management, conflict management, and health system management) were excluded by the experts who participated in our study because they could be assessed within broader competencies such as professionalism, leadership, and teamwork.
For phase 2 of the study, recruitment of participants was again challenging due to the real-time Delphi format. For the first round, last-minute cancellations obliged us to proceed with a Delphi group having fewer than 10 participants, which likely led to the lower reliability that we observed. After the first Delphi round, we amended the protocol to expand the pool of potential participants. Variation among participants in their interpretation of the tool also contributed to lower reliability during the first Delphi round. To address this issue, the anchors were further clarified, and a 20-minute training session was presented to the participants. During this session, participants viewed a short video, and proper use of the tool was demonstrated. Another point of possible misunderstanding was the “not applicable” option for each skill. Some participants mentioned that they chose this option when the learner did not demonstrate the skill. However, the “not applicable” option was intended for use when it was not possible to assess certain skills because the learners did not demonstrate them during the scenario, even though they were defined as learning objectives for the scenario and should have been performed. To clarify this point, participants were informed, during the training session for the second round, of the teaching objectives of the scenario, as well as the specific skills that could be assessed. Additionally, a general assessment item for each competency was added to the assessment tool.
This study had several limitations. In the literature review, certain variables such as the number of domains and number of themes may have been interpreted differently from one author to the next because of overlapping definitions. Arguably, little discrepancy remained after data extraction, given that other variables were not subject to personal interpretation and given that the domains, themes, and items remained consistent throughout data extraction. Also, a second author managed discrepancies as well as data extraction from articles for which the first author was unsure.
Overall, few pharmacists have accreditation and expertise in high-fidelity simulations, which was a limiting factor in the recruitment of pharmacists for the Delphi process. To minimize bias, authors participating as experts were not involved in organizing and running the Delphi processes.
The Fleiss weighted κ indicated only slight agreement among the Delphi participants. For the purpose of this study, it was decided that demonstration of good internal consistency through the Cronbach α would be sufficient to allow discrimination between good performance and poor performance with regard to competency progression over time, because this statistic determines whether a set of items evaluated by different individuals tends in the same direction. Currently, there is no plan to modify the assessment tool; however, it may be of interest in a subsequent study to assess reliability with a greater sample size, as more pharmacy simulation educators are trained in and perform simulation activities.
Considering the breadth of the current literature review, in terms of the variety of health professions and simulated scenarios represented, we believe the resulting assessment tool can be generalized to assessment of pharmacy learners participating in high-fidelity simulations. It may also be generalizable to other health professions. Although only 8% of articles in the current literature review addressed competencies specific to pharmacy practice, a properly conducted Delphi method ensured that the resulting assessment tool would outline key competencies and would be congruent with the practice of pharmacy. Notably, the role of the pharmacist may vary from one geographic region to another, and the competencies identified in the present study may not be generalizable to all jurisdictions. Because reliability was assessed with use of an adjunct training video, this video must be presented to first-time evaluators to ensure reproducibility.
The current study is, to our knowledge, one of the largest studies exploring pharmacy competencies in the context of simulations. A comprehensive literature review and Delphi process allowed us to trace the competencies required to evaluate the progress of pharmacy residents participating in a high-fidelity simulation training program. The assessment tool that was generated through this work has potentially broad application in helping pharmacy simulation evaluators and possibly other health care simulation evaluators in their assessment of trainees’ competencies. This may in turn aid in the evaluation of pharmacy simulation programs by assessing residents’ progress as well as summative and formative evaluation of trainees.
1 Issenberg SB, McGaghie WC, Petrusa ER, Gordon DL, Scalese RJ. Features and uses of high-fidelity medical simulations that lead to effective learning: a BEME systematic review. Med Teach. 2005;27(1):10–28.
Crossref PubMed
2 Ford DG, Seybert AL, Smithburger PL, Kobulinksy LR, Samosky JT, Kane-Gill SL. Impact of simulation-based learning on medication error rates in critically ill patients. Intensive Care Med. 2010;36(9):1526–31.
Crossref PubMed
3 Vyas D, Wombwell E, Russell E, Caligiuri F. High-fidelity patient simulation series to supplement introductory pharmacy practice experiences. Am J Pharm Educ. 2010;74(9):169.
Crossref
4 Robinson JD, Bray BS, Willson MN, Weeks DL. Using human patient simulation to prepare student pharmacists to manage medical emergencies in an ambulatory setting. Am J Pharm Educ. 2011;75(1):3.
Crossref PubMed PMC
5 Owen H. Chapter 3: Historical practices in healthcare simulation: what we still have to learn. In: Nestel D, Kelly M, Jolly B, Watson M, editors. Healthcare simulation education: evidence theory and practice. Wiley Online Library; 2017 [cited 2024 May 14]. pp. 16–22. Available from: https://onlinelibrary.wiley.com/doi/abs/10.1002/9781119061656.ch3
Crossref
6 Bray BS, Schwartz CR, Odegard PS, Hammer DP, Seybert AL. Assessment of human patient simulation-based learning. Am J Pharm Educ. 2011;75(10):208.
Crossref
7 Kane-Gill SL, Smithburger PL. Transitioning knowledge gained from simulation to pharmacy practice. Am J Pharm Educ. 2011;75(10):210.
Crossref
8 Barnett SG, Allen SM, Bastianelli KM, Chen JS, Clark Dula CA, Kachlic MD, et al. Determination of interrater reliability of a universal evaluator rubric to assess student pharmacist communication skills. Am J Pharm Educ. 2022;86(1):8544.
Crossref PMC
9 Schroeder MN, Lengel AJ. Evaluation of a rubric to assess pharmacy student performance in a telehealth simulation exercise. Am J Pharm Educ. 2022;86(8):ajpe8778.
Crossref
10 de Villiers MR, de Villiers PJT, Kent AP. The Delphi technique in health sciences education research. Med Teach. 2005;27(7):639–43.
Crossref PubMed
11 Gordon T, Pease A. RT Delphi: an efficient, “round-less” almost real time Delphi method. Technol Forecast Soc Change. 2006;73(4):321–33.
Crossref
12 Bujang MA, Omar ED, Baharum NA. A review on sample size determination for Cronbach’s alpha test: a simple guide for researchers. Malays J Med Sci. 2018;25(6):85–99.
13 Keeney S. The Delphi technique in nursing and health research. Wiley-Blackwell; 2011.
Crossref
14 Bravo G, Potvin L. Estimating the reliability of continuous measures with Cronbach’s alpha or the intraclass correlation coefficient: toward the integration of two traditions. J Clin Epidemiol. 1991;44(4–5):381–90.
Crossref
15 Cortina JM. What is coefficient alpha? An examination of theory and applications. J Appl Psychol. 1993;78(1):98–104.
Crossref
16 Tavakol M, Dennick R. Making sense of Cronbach’s alpha. Int J Med Educ. 2011;2:53–5.
Crossref PubMed PMC
17 Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977;33(1):159–74.
Crossref PubMed
18 Flin R, Patey R. Non-technical skills for anaesthetists: developing and applying ANTS. Best Pract Res Clin Anaesthesiol. 2011;25(2):215–27.
Crossref PubMed
19 Kim J, Neilipovitz D, Cardinal P, Chiu M, Clinch J. A pilot study using high-fidelity simulation to formally evaluate performance in the resuscitation of critically ill patients: the University of Ottawa Critical Care Medicine, High-Fidelity Simulation, and Crisis Resource Management I study. Crit Care Med. 2006;34(8):2167–74.
Crossref PubMed
20 Martin JA, Regehr G, Reznick R, MacRae H, Murnaghan J, Hutchison C, et al. Objective structured assessment of technical skill (OSATS) for surgical residents. Br J Surg. 1997;84(2):273–8.
PubMed
21 Yule S, Flin R, Paterson-Brown S, Maran M, Rowley D. Development of a rating system for surgeons’ non-technical skills. Med Educ. 2006; 40(11):1098–104.
Crossref PubMed
22 Lasater K. Clinical judgment: the last frontier for evaluation. Nurse Educ Pract. 2011;11(2):86–92.
Crossref PubMed
23 Bajis D, Chaar B, Basheti IA, Moles R. Pharmacy students’ medication history taking competency: simulation and feedback learning intervention. Curr Pharm Teach Learn. 2019;11(10):1002–15.
Crossref PubMed
24 Doloresco F, Maerten-Rivera J, Zhao Y, Foltz-Ramos K, Fusco NM. Pharmacy students’ standardized self-assessment of interprofessional skills during an objective structured clinical examination. Am J Pharm Educ. 2019;83(10):7439.
Crossref
25 Bailey L, Curington R, Brown B, Hegener M, Espel M. Motivational interviewing education: creation and assessment of a learning module implemented among advanced pharmacy practice students. Curr Pharm Teach Learn. 2017;9(5):786–93.
Crossref PubMed
Patrick J F Deschênes, My Hanh Luu Hoai, Justine Rinfret, Sarah Pelletier, and John Nam-Kha Nguyen contributed equally to the writing of this article.
The tool was developed and validated in French, as shown; to date, no English version has been developed. © 2020 The Authors (Patrick J F Deschênes, My Hanh Luu Hoai, Justine Rinfret, Sarah Pelletier, Alessandra Stortini, Rachel Therrien, Marie-Kim Héraut, Pascaline Bernier, and John Nam-Kha Nguyen).
Address correspondence to: Patrick J F Deschenes, McGill University Health Centre, 1001 Decarie Boulevard, Montréal QC H4A 3J1, email: deschenespat@hotmail.com
Competing interests: For work unrelated to the study reported here, Rachel Therrien has received speaker’s honoraria from Gilead and Viiv Healthcare. No other competing interests were declared.
Funding: For work on this project, My Hanh Luu Hoai, Justine Rinfret, and John Nam-Kha Nguyen were compensated by the Pineau-Schubert Fund, Faculty of Pharmacy, Université de Montréal (distributed by Patrick Deschênes). For presentation of results at the SoFraSims conference (2023), Patrick Deschênes received travel support from the Department of Pharmacy, McGill University Health Centre (as per agreement between the Association des pharmaciens d’établissements de santé du Québec and the Ministère de la santé et des services sociaux) and waiver of conference registration fees.
Acknowledgements: The authors acknowledge the contributions of the Delphi participants, Patrice Chrétien-Raymer, Gilles Leclerc, Marie-Laurence Tremblay, Dr Sébastien Béland, and Nathalie Letarte.
Submitted: September 28, 2023
Accepted: January 16, 2024
Published: July 10, 2024
© 2024 Canadian Society of Hospital Pharmacists | Société canadienne des pharmaciens d’hôpitaux
Canadian Journal of Hospital Pharmacy, VOLUME 77, NUMBER 3, 2024
