A Framework for Evaluating the Implementation of Biosimilar Drugs


Lisa Milgram, Sarah Wheeler, Andrea Adamic, Mirhad Loncar, Micheal Guirguis, Betty Jo McCabe

DOI: https://doi.org/10.4212/cjhp.3272

ABSTRACT

Background

The introduction of biosimilar drugs has significant effects on health care systems, and a variety of approaches are required to support acceptance, adoption, and use of these drugs. Literature exists on the enablers of, and barriers to, biosimilar implementation, but frameworks that support the evaluation of biosimilar implementation strategies are currently lacking.

Objective

To develop an evaluation framework for assessing the effects of biosimilar implementation strategies on patients, clinicians, and publicly funded drug programs.

Methods

The scope of the evaluation was determined by a pan-Canadian working group through the creation of a logic model of activities and expected outcomes associated with biosimilar implementation. Each component of the logic model was considered under the RE-AIM framework, which led to a set of evaluation questions and indicators. Feedback to inform the final framework was sought from stakeholders through focus group sessions and written responses.

Results

An evaluation framework was created that articulates evaluation questions and indicators across 5 priority areas: stakeholder engagement, patient experience, patient outcomes, clinician experience, and system sustainability and affordability. Stakeholder feedback was obtained through 9 focus group sessions with a total of 87 participants. Feedback was used to refine the framework on the basis of stakeholder priorities and feasibility.

Conclusions

Through extensive stakeholder consultation, an evaluation framework was developed to measure and monitor the effects of biosimilar implementation on the 5 identified priority areas, as well as to inform future biosimilar implementations. This framework can be used as a starting point for evaluating the implementation of biosimilars across health care systems.

KEYWORDS: biosimilar drugs, evaluation framework, biosimilar implementation, indicators

RÉSUMÉ

Contexte

L’apparition de médicaments biosimilaires a eu et continue d’avoir des effets importants sur les systèmes de soins de santé et diverses approches doivent être mises en place pour qu’ils soient acceptés, adoptés et utilisés. Il existe de la documentation sur les catalyseurs et les obstacles à leur mise en œuvre, mais les cadres entourant l’évaluation des stratégies de mise en œuvre des médicaments biosimilaires font actuellement défaut.

Objectif

Développer un cadre d’évaluation pour estimer les retombées des stratégies de mise en œuvre des biosimilaires sur les patients, les cliniciens et les programmes de médicaments financés par les deniers publics.

Méthodes

Un groupe de travail pancanadien a déterminé la portée de l’évaluation à l’aide d’un modèle logique des activités et des résultats attendus associés à la mise en œuvre des biosimilaires. Chaque composante du modèle logique a été examinée dans le cadre RE-AIM, ce qui a donné lieu à un ensemble de questions d’évaluation et des indicateurs d’évaluation. Des commentaires pour éclairer le cadre final ont été sollicités auprès des parties prenantes au moyen de groupes de discussion et de réponses écrites.

Résultats

Un cadre d’évaluation a été défini. Il articule les questions d’évaluation et des indicateurs d’évaluation dans 5 domaines prioritaires: l’engagement des intervenants, l’expérience des patients, les résultats des patients, l’expérience des cliniciens et la durabilité et l’abordabilité du système. Les commentaires des intervenants ont été obtenus au cours de 9 séances de groupes de discussion avec un total de 87 participants. Les commentaires ont été utilisés pour affiner le cadre sur la base des priorités des parties prenantes et de la faisabilité.

Conclusions

Une vaste consultation des parties prenantes a permis de définir un cadre d’évaluation pour mesurer et surveiller les effets de la mise en œuvre des biosimilaires sur les 5 domaines prioritaires identifiés, ainsi que pour éclairer les futures mises en œuvre des biosimilaires. Ce cadre peut être utilisé comme point de départ pour évaluer la mise en œuvre des biosimilaires dans les systèmes de soins de santé.

MOTS CLÉS: médicaments biosimilaires, cadre d’évaluation, mise en œuvre des biosimilaires, indicateurs

INTRODUCTION

Biologic drugs are expensive and represent a growing segment of the pharmaceutical market. Global biologics sales increased by 70% between 2011 and 2016.1 Canada has the second-highest per capita spending on biologics within the member countries of the Organisation for Economic Co-operation and Development, not accounting for confidential price rebates resulting from product listing agreements.2 In 2018, Canadian sales of biologics reached $7.7 billion, representing 30.1% of the country’s total pharmaceutical sales.2 However, the biologic shares of claims in Canadian public and private plans were much lower, at 1.5% and 1.9%, respectively.2 Health care payers are looking for ways to contain costs in light of limited budgets and the need for health care system sustainability. Biosimilars represent lower-cost alternatives to existing biologic drugs. A biosimilar is a biologic drug that is highly similar to a biologic drug already authorized for sale (commonly referred to as the reference biologic), with no expected clinically meaningful differences in efficacy or safety.3 Biosimilars offer an opportunity for significant cost savings, because they enter the market after the reference biologic drug’s patents and data protection have expired.4

Canada has a mixed system of private and public drug coverage. Each of the 10 provinces and 3 territories has its own drug funding policies for nonhospitalized patients, which differ between oncology and non-oncology therapeutic areas. Oncology biologic drugs are primarily publicly funded and are usually administered in a hospital outpatient setting, whereas non-oncology biologic drugs are funded through a mix of public insurance, private insurance, manufacturer-sponsored patient support programs, and out-of-pocket payment, and they are typically administered in private clinics. The pan-Canadian Pharmaceutical Alliance, a network of representatives from the provincial, territorial, and federal governments, guides and defines the process of how prices for biologic and biosimilar products are negotiated for public drug plans. Federal, provincial, and territorial drug plan managers make independent decisions on funding policies and coverage. Since 2018, a variety of approaches stemming from the action plan for oncology biosimilar implementation5 have been implemented across Canadian jurisdictions and therapeutic areas to support the appropriate use of biosimilars and related reference biologics and to enhance patients’ access to clinically relevant and cost-effective treatment options. These approaches have included engagement with stakeholders throughout implementation efforts,6,7 development and dissemination of educational resources for patients and providers,8,9 identification and development of funding policies1015 to promote uptake of biosimilars, and development of recommendations to support changes to the practices of heath care providers.16

Extant literature on the enablers of, and barriers to, biosimilar implementation1721 focuses largely on clinician perspectives and features the need for clinician-directed education about biosimilars and facilitation of administrative processes related to prescribing them. Literature was also found on patient perspectives before, during, and after the implementation of nonmedical switching policies; this literature highlights patient concerns, enablers of and barriers to implementation, and impacts of policy changes on affected patients.2224 However, no comprehensive evaluation framework was found in the literature to assess the effects of enablers, barriers, and implementation strategies on biosimilar implementation. As such, this study was undertaken to develop, through extensive stakeholder consultation, an evaluation framework for effectively and objectively evaluating the impacts of implementation approaches on drug utilization and uptake, cost savings, patient experiences and outcomes, clinician experiences, and education and resource needs.

METHODS

A pan-Canadian Evaluation Working Group (EWG) was established to help with this initiative. EWG participants represented ministries of health and cancer agencies from across Canada, health technology assessment organizations, a provincial health authority, Health Canada, and the Canadian Association of Provincial Cancer Agencies. EWG participants were nominated through jurisdictional representatives of the pan-Canadian Pharmaceutical Alliance and included a mix of pharmacists, physicians, nurses, health economists, policy advisors, and drug formulary managers. At monthly teleconferences, the project team presented on the progress of their work and facilitated generative discussion for the EWG to identify priorities for evaluation and to provide input on the feasibility of data collection and analysis. The EWG determined the scope of the evaluation by developing a logic model showing the activities and expected outcomes associated with the various approaches to biosimilar implementation across Canada. Box 1 shows a summary of the Biosimilars Implementation Logic Model.

BOX 1 Summary of Biosimilars Implementation Logic Model




 

Evaluation Framework

The RE-AIM framework is a tool that helps program planners, evaluators, funders, and policy-makers develop effective, sustainable health programs and interventions.25 When applied for purposes of evaluation, the framework proposes that different perspectives be used to evaluate the success of a program or intervention. Each activity and expected outcome identified within the Biosimilars Implementation Logic Model was considered from the 5 perspectives of the RE-AIM framework—Reach, Effectiveness, Adoption, Implementation, and Maintenance—as facilitated by the RE-AIM planning and evaluation tool.26 This process generated an extensive list of evaluation questions and indicators that formed the first draft of the evaluation framework. The draft framework was reviewed by the EWG, who provided feedback on its alignment with jurisdictional evaluation priorities and on the feasibility of data collection and analysis. EWG feedback also ensured that the framework was comprehensive and that irrelevant or infeasible questions and indicators were excluded.

Stakeholder Consultations

To ensure that the framework reflected the priorities and perspectives of the stakeholders most affected by changes in biosimilar policy, consultations on the draft framework were conducted through focus group sessions and requests for written comments from key stakeholder groups. Invitations were extended to 26 patient groups and 13 clinician groups across disease areas where biosimilars were approved for use (oncology, rheumatology, dermatology, gastroenterology, endocrinology, ophthalmology, and rare disorders), 4 pharmaceutical industry groups, 3 providers of patient support programs with private infusion clinics, 1 organization representing private health insurers, and 13 Canadian drug plan managers representing federal, provincial, and territorial ministries of health and cancer agencies. Organizations were identified by reviewing lists of participation in prior pan-Canadian biosimilar consultation sessions and through an online search of additional Canadian national organizations with an overt interest in biosimilar implementation. Invitations were sent to the leaders of the organizations, with a request to identify 1 or 2 representatives to participate in the focus group sessions.

Nine focus group sessions involving a total of 87 participants were conducted via Microsoft Teams videoconferencing software (Microsoft Corporation) in November 2020, 4 with patient groups (grouped by therapeutic area) and 1 each with clinician groups, the pharmaceutical industry, providers of patient support programs, private health insurance representatives, and public drug plan managers. Box 2 lists the participating organizations. Participants were not compensated for their time. Each session was facilitated by 2 of the authors (L.M., a program manager, and S.W., a methodologist) from the core project team, who had no known conflicts of interest. Meetings were recorded to facilitate transcription and subsequent analysis of the discussion.

BOX 2 Organizations that Participated in Focus Groups




 

To ensure efficient use of time during focus group sessions, evaluation questions and indicators that were most aligned with the perspectives of the various groups were prioritized for discussion. Clinician sessions prioritized themes of local implementation and education; patient group sessions focused on funding policies, local implementation, and education; industry sessions prioritized funding policies and education; and payer sessions focused on funding policies. All groups were given time to discuss questions and indicators related to stakeholder engagement. Time was reserved at the end of each session for discussion on any of the other evaluation questions or indicators, so that all stakeholders could comment on any aspect in the draft framework. Following the focus group sessions, stakeholders had the opportunity to provide additional feedback on the draft framework through the electronic platform Microsoft Forms (Microsoft Corporation). This ensured that all focus group participants had the opportunity to be heard on all issues.

Analysis

Transcripts from the focus group sessions were analyzed using NVivo qualitative data analysis software (QSR International Pty Ltd; version 11, 2015) to identify areas of importance for each stakeholder group. All feedback, including focus group discussions and written responses, was synthesized and thematically analyzed to inform the final list of indicators and evaluation questions. The final evaluation plan was developed from the draft plan by first removing evaluation questions and indicators that were not important to any stakeholder group and then expanding and clarifying the questions and indicators in the areas that stakeholder groups felt were most relevant to understanding the implementation of biosimilars. Priorities emerging from each stakeholder perspective were equally weighted, and thus emergent themes with high priority for any particular stakeholder group were included, even if other stakeholder groups did not consider the area to be a priority. For example, if patient support programs and private health insurance representatives prioritized different areas, both sets of priorities were included in the framework.

RESULTS

Through the process of considering the activities and outcomes of biosimilar implementation under the RE-AIM framework and incorporating the EWG’s feedback regarding relevance and feasibility, a draft framework was developed that contained the evaluation questions and indicators pertinent to assessing the effects of biosimilar implementation. In this framework, evaluation questions and indicators were thematically grouped into 5 priority areas: stakeholder engagement, patient experience, patient outcomes, clinician experience, and system sustainability and affordability.

Feedback received during the focus group sessions and through written feedback mechanisms highlighted the evaluation and implementation priorities for the various stakeholders and clarified which aspects were most relevant to measure. Feedback on the priority areas is summarized below.

Feedback on Stakeholder Engagement

All participants described the importance of collaborative engagement and consultation with stakeholders for the successful implementation of biosimilars. The following are key aspects that were identified to help measure the scope and effect of stakeholder engagement:

Feedback on Patient Experience

Patient groups indicated that clear and objective information on biosimilars was helpful to support acceptance and comfort with biosimilar therapy. However, in some jurisdictions and therapeutic areas, patients were faced with changes in out-of-pocket expenses and changes in treatment location. The following key aspects were identified to help measure the effects of patient experiences:

Feedback on Patient Outcomes

Some patient groups and clinicians expressed a lack of confidence in the safety and effectiveness of biosimilar drugs when a patient was switched from a reference biologic to a biosimilar. These concerns may stem from limited evidence in support of switching. Participants suggested that real-world data be collected to measure and monitor patient outcomes compared with historical cohorts (i.e., patients on the reference biologic), such as the following:

Feedback on Clinician Experience

Clinicians (physicians, pharmacists, nurses) indicated that policy and process changes resulted in increased workload when they were prescribing or administering biosimilars to patients. They wanted to examine these changes in early phases of biosimilar implementation to improve the efficiency of future implementations. Clinicians also indicated that the quantity, quality, and availability of credible, objective, evidence-based information on biosimilars differed across diseases and that it is important to understand where knowledge gaps exist. The key aspects that were identified to help measure the effects on clinician experiences included the following:

Feedback on System Sustainability and Affordability

Some stakeholders identified cost savings as a driving force for biosimilar implementation. Other stakeholders cautioned that a focus on costs alone does not align with the goals of better patient care and increased treatment options. A focus on driving down drug costs may also lead to decreased manufacturer profit margins and a disincentive for manufacturers to remain in the biosimilars market, resulting in potential drug shortages, supply interruptions, or fluctuations in pricing. The following key aspects were identified to help measure the effects on market and financial sustainability in a publicly funded system:

Biosimilars Implementation Evaluation Framework

The stakeholder feedback received through the focus groups and written contributions, summarized above, was used to refine and develop the Biosimilars Implementation Evaluation Framework presented in Appendix 1 (available from https://www.cjhp-online.ca/index.php/cjhp/issue/view/214). Data to support these indicators may come from a variety of sources, including existing administrative data sets (information routinely collected about program operations that is used for performance management, funding, or reporting), new organizational data sources (information about how a program is constructed and operates, which is used to understand how a program is implemented), and qualitative methods (information about context that is used to understand why a program worked or did not work well) such as informational interviews, focus groups, or surveys. The evaluation framework also includes supporting questions to further explore perspectives related to the qualitative indicators (Box 3). Because of differences in drug funding policies and the delivery of care across Canada, stakeholders recommended stratifying analysis of the indicators across therapeutic areas, jurisdictions, and care settings, when necessary, to demonstrate different effects and outcomes.

BOX 3 Qualitative Questions and Probes




 

DISCUSSION

The Biosimilars Implementation Evaluation Framework consists of evaluation questions and indicators to measure and monitor the effects of biosimilar funding policies and implementation strategies on patients, clinicians, and drug programs. Stakeholders who are most strongly affected by biosimilar policy changes helped to refine the indicators and improve the robustness and relevance of the evaluation framework. The EWG’s input on the feasibility of data collection and reporting helped support jurisdictional buy-in from those who may be responsible for executing components of the framework. Most stakeholders were interested in participating in the process, as they were given an opportunity to share their feedback on what would be most important to measure.

Similar to what has been reported in the literature,1721 stakeholders identified the need for more education, such as a centralized source of accredited, evidence-informed, objective educational material for physicians. They also identified the need for patient information, purposeful stakeholder engagement, and clear and timely communication to support the use and acceptance of biosimilars. In addition, stakeholders identified the need for clear and transparent policies for switching and substitution of biologic drugs.

Through the inclusion of diverse stakeholder perspectives, such as patient and industry groups, additional areas for evaluation were articulated. System sustainability and affordability were of particular importance to policy-and decision-makers, given the nature of public drug funding in Canada and the increasing cost of biologic drugs. Patient and industry groups acknowledged the importance of the future sustainability of the health care system and also highlighted the need for a sustainable market where multiple biosimilar brands can coexist. Patient groups expressed interest in additional safety and effectiveness data on switching (from reference biologic to biosimilar and from one biosimilar to another), psychological supports, and genuine collaboration and transparency to improve patients’ and clinicians’ experiences. More information about these consultations in Canada, as well as a Canadian-specific evaluation framework, is available by contacting the corresponding author.

There were some limitations in the development of the Biosimilars Implementation Evaluation Framework. First, although a concerted effort was made to consult with a diverse set of stakeholders, it is recognized that some perspectives may not have been captured. Although all stakeholders had prior knowledge of biosimilars and were engaged in various consultations and activities in preparation for the introduction of biosimilars and funding policies in Canada, only a few stakeholders had lived experienced with mandated switching. However, given the diversity of organizations that were invited for engagement, a wide range of perspectives was included. Second, although all participants had the opportunity to comment on the entire draft framework, participants may not have discussed every indicator directly, as a subset of indicators were prioritized for discussion at each focus group session. In addition, new indicators that were suggested at focus group sessions or through written feedback were not subsequently reviewed by all focus group participants. Therefore, the EWG had to decide whether to include those new indicators in the final evaluation framework without additional consultation. Decisions were based on expected value of the indicator, importance to stakeholders, and feasibility of data collection. Lastly, no prioritization exercise was conducted to reduce the number of indicators included in the framework. Some stakeholders expressed concern that the draft evaluation framework would be expensive and time-consuming to execute, given the large number of indicators. The framework, however, can be used as a toolkit, which can be customized or narrowed in scope, to align with the specific considerations of an individual jurisdiction or organization.

CONCLUSION

The introduction of biosimilar drugs has significant effects on health care systems. Jurisdictions across Canada have taken a variety of approaches to funding policies and implementation strategies, which will affect patients, clinicians, and drug programs. The Biosimilars Implementation Evaluation Framework contains evaluation questions and indicators to measure and monitor the introduction of biosimilar drugs. This framework may be used as a starting point for jurisdictions evaluating the implementation of biosimilars in their health care systems.

References

1 Kent D, Rickwood S, Di Biase S. Disruption and maturity: the next phase of biologics. IQVIA; 2017 [cited 2021 Aug 27]. Available from: https://www.iqvia.com/-/media/iqvia/pdfs/nemea/uk/disruption_and_maturity_the_next_phase_of_biologics.pdf

2 Biologics in Canada. Part 1: market trends, 2018 [slide show]. Patented Medicine Prices Review Board; 2020 [cited 2021 Aug 27]. Available from: https://www.canada.ca/content/dam/pmprb-cepmb/documents/reports-and-studies/chartbooks/biologics-part1-market-trends.pdf

3 Biosimilar biologic drugs in Canada: fact sheet. Health Canada; 2019 [cited 2021 Aug 27]. Available from: https://www.canada.ca/en/health-canada/services/drugs-health-products/biologics-radiopharmaceuticals-genetic-therapies/applications-submissions/guidance-documents/fact-sheet-biosimilars.html

4 Biologics in Canada. Part 2: biosimilar savings, 2018 [slide show]. Patented Medicine Prices Review Board; 2020 [cited 2021 Aug 27]. Available from: https://www.canada.ca/content/dam/pmprb-cepmb/documents/reports-and-studies/chartbooks/biologics-part2-biosimilar-savings2018.pdf

5 Pan-Canadian Oncology Biosimilars Initiative action plan. Cancer Care Ontario; 2019 [cited 2021 Aug 14]. Available from: https://www.cancercareontario.ca/sites/ccocancercare/files/assets/PanCanadianOncologyBiosimilarsActionPlan_0.pdf

6 Pan-Canadian oncology biosimilars summit proceedings report. Cancer Care Ontario; 2019 [cited 2021 Aug 27]. Available from: https://www.cancercareontario.ca/sites/ccocancercare/files/assets/PanCanadianBiosimilarsSummitProceedingsReport_0.pdf

7 National consultation on the use and implementation of biosimilars. Online consultation summary report. CADTH; 2020 Jan 30 [cited 2021 Aug 27]. Available from: https://www.pcpacanada.ca/sites/default/files/aoda/National_Consultation_on_the_Use_and_Implementation_of_Biosimilars__FINAL-s.pdf

8 Biosimilars. Cancer Care Ontario; 2019 Sep 12 [cited 2021 Aug 27]. Available from: https://www.cancercareontario.ca/en/cancer-treatments/biologic-biosimilar-therapy/biosimilars

9 Oncology biosimilars resources for healthcare providers. Cancer Care Ontario; 2019 Sep 13 [updated 2020 Feb 12; cited 2021 Aug 27]. Available from: https://www.cancercareontario.ca/en/guidelines-advice/modality/biological-therapy/biosimilars-providers

10 Bulletin #100. Tiered biologics reimbursement policy. Government of Manitoba; 2018 [cited 2021 Aug 27]. Available from: https://www.gov.mb.ca/health/mdbif/docs/bulletins/bulletin100.pdf

11 Biosimilar drugs. Government of Alberta; 2021 [cited 2021 Aug 27]. Available from: https://www.alberta.ca/biosimilar-drugs.aspx

12 Biosimilars Initiative. Government of New Brunswick; 2021 [cited 2021 Aug 27]. Available from: https://www2.gnb.ca/content/gnb/en/departments/health/MedicarePrescriptionDrugPlan/NBDrugPlan/biosimilars.html

13 Biosimilar drugs. Gouvernement du Québec; 2021 [cited 2021 Aug 27]. Available from: https://www.quebec.ca/en/health/health-system-and-services/biosimilar-drugs

14 Biosimilars Initiative for patients. Province of British Columbia; 2021 [cited 2021 Aug 27]. Available from: https://www2.gov.bc.ca/gov/content/health/health-drug-coverage/pharmacare-for-bc-residents/what-we-cover/biosimilars-initiative-patients

15 Pant S, de Léséleuc L, Spry C. International policies on the appropriate use of biosimilar drugs [environmental scan]. CADTH; 2018 [cited 2021 Aug 27]. Available from: https://www.cadth.ca/sites/default/files/pdf/es0333_international-policies-on-use-of-biosimilar-drugs.pdf

16 Position statements for the implementation of oncology biosimilars from the pan-Canadian Clinical Operations Working Group. Cancer Care Ontario; 2019 [cited 2021 Aug 27]. Available from: https://www.cancercareontario.ca/en/guidelines-advice/types-of-cancer/60296

17 Greene L, Singh RM, Carden MJ, Pardo CO, Lichtenstein GR. Strategies for overcoming barriers to adopting biosimilars and achieving goals of the Biologics Price Competition and Innovation Act: a survey of managed care and specialty pharmacy professionals. J Manag Care Spec Pharm. 2019;25(8):904–12.
PubMed  

18 Sarnola K, Merikoski M, Jyrkkä J, Hämeen-Anttila K. Physicians’ perceptions of the uptake of biosimilars: a systematic review. BMJ Open. 2020;10(5):e034183.
Crossref  PubMed  PMC  

19 Edgar BS, Cheifetz AS, Helfgott SM, Smith GP, Bandekar T, Hoye-Simek A, et al. Overcoming barriers to biosimilar adoption: real-world perspectives from a national payer and provider initiative. J Manag Care Spec Pharm. 2021;27(8):1129–35.
PubMed  

20 Leonard E, Wascovich M, Oskouei S, Gurz P, Carpenter D. Factors affecting health care provider knowledge and acceptance of biosimilar medicines: a systematic review. J Manag Care Spec Pharm. 2018;25(1): 102–12.
PubMed  

21 Teeple A, Ellis LA, Huff L, Reynolds C, Ginsburg S, Howard L, et al. Physician attitudes about non-medical switching to biosimilars: results from an online physician survey in the United States. Curr Med Res Opin. 2019;35(4):611–7.
Crossref  PubMed  

22 Chew C, Aguiar M, Bansback N, Law MR, Harrison M. Patient perspectives on the British Columbia Biosimilars Initiative: a qualitative descriptive study. Rheumatol Int. 2022;42(10):1831–42.
Crossref  

23 Teeple A, Ginsburg S, Howard L, Huff L, Reynolds C, Walls D, et al. Patient attitudes about non-medical switching to biosimilars: results from an online patient survey in the United States. Curr Med Res Opin. 2019;35(4):603–9.
Crossref  PubMed  

24 Frantzen L, Cohen JD, Tropé S, Beck M, Munos A, Sittler MA, et al. Patients’ information and perspectives on biosimilars in rheumatology: a French nation-wide survey. Joint Bone Spine. 2019;86(4):491–6.
Crossref  PubMed  

25 What is RE-AIM? RE-AIM; 2021 [cited 2021 Aug 27]. Available from: https://www.re-aim.org/about/what-is-re-aim/

26 RE-AIM planning tool. RE-AIM; 2021 [cited 2023 Feb 6]. Available from: https://re-aim.org/resources-and-tools/measures-and-checklists/


Lisa Milgram, MBA, is with the Provincial Drug Reimbursement Programs, Ontario Health (Cancer Care Ontario), Toronto, Ontario
Sarah Wheeler, PhD, is with Quality Measurement and Evaluation, Ontario Health (Cancer Care Ontario), Toronto, Ontario
Andrea Adamic, BA, is with the Provincial Drug Reimbursement Programs, Ontario Health (Cancer Care Ontario), Toronto, Ontario
Mirhad Loncar, MSc, is with Pharmaceutical Reviews, CADTH, Ottawa, Ontario
Micheal Guirguis, BSc(Pharm), PhD, is with the University of Alberta Hospital, Alberta Health Services, Edmonton, Alberta
Betty Jo McCabe, MEd, is with Quality Measurement and Evaluation, Ontario Health (Cancer Care Ontario), Toronto, Ontario

Competing interests: Other than funding (see below), no competing interests were declared. ( Return to Text )


Address correspondence to: Andrea Adamic, Provincial Drug Reimbursement Programs Ontario Health (Cancer Care Ontario) 525 University Avenue, Toronto ON M5G 2L3, email: andrea.adamic@ontariohealth.ca

(Return to Top)


Funding: The development of this evaluation framework was supported by the Pan-Canadian Pharmaceutical Alliance. ( Return to Text )


Acknowledgements

The authors would like to thank Jessica Arias and Scott Gavura, of Ontario Health (Cancer Care Ontario), for their contributions to discussions on the evaluation framework and for writing assistance on this manuscript. They would also like to thank the members of the Evaluation Working Group and the focus group participants for their invaluable contributions to this work.


Canadian Journal of Hospital Pharmacy, VOLUME 76, NUMBER 2, Spring 2023