Ross T Tsuyuki
The purpose of this paper is to give an overview of the components of a pharmacy practice trial. Many of the issues in the design of pharmacy practice research studies are similar to those for clinical trials, and the reader is referred to textbooks on clinical trial design for more in-depth information.1,2
Pharmacy practice research is a type of health services research that focuses on pharmacist care and its effect on patient outcomes. Although pharmacy practice research also deals with broader issues related to training for, as well as preparation and implementation of, pharmacist interventions, the focus of this chapter is on the design of trials of pharmacist care (i.e., “interventions”).
As with other forms of research, a good research project starts with a well-articulated research question. A more complete discussion of how to develop an appropriate research question appears in an earlier article in this series,3 but a few points are worth reiterating.
A good research question should specify the patient population, the intervention (and control, if applicable), and the outcome of interest.1 The question can be restated in the form of the study objective, for example, “The objective of this study is to determine the effect of pharmacist prescribing, relative to usual physician care, on the proportion of patients with stroke reaching their blood pressure target.” The acronym PICO is often used in crafting the research question, where PICO refers to Population, Intervention, Control, and Outcome. Some researchers also like to mention the research design in the research question.
If you do not start with a clear research question, it is unlikely that your proposed study will receive funding or generate interpretable results. In the case of a grant submission, a reviewer who does not understand what you are proposing to study cannot make judgments about the methods you propose and will usually conclude that your methods are unsound.
In a situation where you are designing (or helping to design) a study, all of your decisions about methods (as described below) depend upon your research question. As such, an unclear research question will lead to unclear methods.
The choice of study design is a fundamental first step.
Many pharmacy practice researchers use a “before–after” design, comparing patients’ data at enrolment (baseline) with data collected at the end of follow-up. Although a before–after study is simple to conduct, this design suffers from low causal inference, since the researcher cannot say with any degree of certainty that it is the intervention that led to any observed change in outcome. There are always other external factors that might have caused the change.
Randomized designs have the highest level of causal inference. In a patient-level randomized controlled trial, patients are randomly assigned (usually by means of a computer-generated sequence) to one or more treatments (interventions) or control. Each group is followed for a specified period of time, and the outcomes are compared between groups. However, in practice research, it may not be ideal for the same pharmacists to provide care to both the intervention and usual-care groups. That is because doing so can lead to “contamination” (whereby control patients receive some or all of the intervention), which can in turn reduce the difference between the intervention and control groups, leading to a false conclusion that the intervention is ineffective compared with control.
An alternative to randomization by patient is to adopt a clustered design. Cluster randomized trials also have a high level of causal inference. In cluster randomized trials, the unit of randomization is not the patient, but rather an organizational unit (e.g., a pharmacy, a city, or a region). For example, Pharmacy A (and all of its pharmacists) is assigned to provide the intervention to its patients, while Pharmacy B is assigned to provide usual care to its patients. Both pharmacies follow their respective patients, and the researchers compare the outcomes between intervention pharmacies and control (usual-care) pharmacies. This design can mitigate some of the concerns over contamination, since all of Pharmacy A’s patients receive the intervention and all of Pharmacy B’s patients receive the control. The disadvantages of this design are that it is not as statistically efficient, and larger numbers of patients must be recruited. Another disadvantage is that pharmacists working for a pharmacy assigned to provide usual care to all of its patients may lose interest and not recruit any patients. If this occurs, then the research design becomes invalid, as there is effectively no control group.
When considering a randomized trial design, ask participating pharmacists if they are willing to provide usual care. You should also ask what constitutes their usual level of care (i.e., what they already do for their patients). For example, if you are planning a diabetes intervention, random assignment of patients to the intervention or usual care might be problematic if one of the pharmacists is a certified diabetes educator and already intervenes in the care of all patients with diabetes. If you discover that certain pharmacists are not willing to provide usual care (as you define it), then it may be best to not have them participate.
The inclusion criteria for the study define the patients who will be recruited. Make these criteria as specific as necessary to ensure you get participants who have the condition of interest and will be responsive to the planned intervention (e.g., “patients with diabetes whose A1C is above the target of 7.0%”).
The exclusion criteria define those who will not be studied, because they do not have the condition of interest, are unlikely to respond to the intervention, or are unlikely to provide good data (e.g., cannot communicate in English or French if a survey is involved or unable to participate in follow-up, as may be the case for people with severe alcoholism or psychiatric disorders, those who are homeless, or those who live far away).
Structuring the inclusion criteria too narrowly (i.e., with a lot of restrictions) will limit study generalizability (applicability) to a wider population and will also make it difficult to recruit sufficient numbers of patients. Specifying the inclusion criteria too broadly may lead to inclusion of some patients who do not have the condition of interest or will not be responsive to the intervention.
You must specify how you will recruit patients into your study, a step that is often taken for granted. A well-designed research study includes a recruitment plan that details where and how participants will be recruited.
Pharmacy practice research is particularly susceptible to recruitment problems because pharmacist investigators are busy and do not have the time and/or expertise in conducting research. Pharmacy practice research depends upon pharmacists being able to identify patients who meet the inclusion criteria. It is important that this be done exactly as outlined by the inclusion and exclusion criteria, without any subjective judgment about whether people are likely to benefit or not, as such judgment would introduce bias.
The recruitment plan could be as simple as “Pharmacists will generate a list of all patients receiving metformin (as a marker for type 2 diabetes) from their computer system and will systematically approach all of these patients to invite them to participate in the study.” Multiple methods (or sources of participants) may sometimes be needed to reach recruitment targets, but the inclusion and exclusion criteria must be applied consistently.
You will need to describe the intervention being studied in enough detail to allow a pharmacist to apply it to his or her patient. This means specifying all components of the intervention, such as patient education, additions to or modifications of therapy, interactions with other health care professionals, measurements to be made, and timing and frequency of follow-up visits.
As with a drug intervention, an intervention in practice research should be “strong enough” to affect the condition of interest. Ideally, you will have some preliminary data to suggest that your intervention may be effective.
In many pharmacy practice trials, the control group is “usual care”, i.e., the care that patients would receive had there been no study underway. Of course, you cannot “forbid” any care that a patient would otherwise receive, either from a pharmacist or from a physician or other health care provider, but it is useful to know what the standard of care is among practitioners participating in the study.
In pharmacy practice research, many pharmacists who self-identify as being interested in research participation do so because they are interested in the topic under study. In fact, their usual care may include many components of your intervention. If so, it is important to know this before proceeding.
You must specify when the follow-up visits will occur, because this also determines when the study outcomes will be measured. You must also decide whether the schedule of follow-up visits will be the same for intervention and control patients. It needn’t be, but at the very least, all patients should have a final follow-up visit to ascertain outcomes.
As described below under “Bias and Confounding”, losses to follow-up are a major threat to validity and, in my experience, seem to be worse in pharmacy practice research (perhaps because patients feel less inclined to follow through with a pharmacist than with a physician, or because pharmacists may be less diligent in ensuring follow-up with their patients). Your plan to minimize losses to follow-up2 should include procedures to keep study participants and investigators interested and engaged with the study and ways to keep study-related visits (and time) to a minimum. Even if a study participant does withdraw consent, it may be possible to ask for a final follow-up visit to ascertain study outcomes.
Many decisions in the design of a research study relate to the outcome measure (or measures) chosen. Such decisions will affect the study budget, study procedures, and the impact of the trial on clinical practice. It is therefore strongly recommended to choose an outcome that is important in some respect. This could be a clinical outcome (such as mortality, readmission, or relapse) or a validated surrogate outcome (such as blood pressure). Generally, a single outcome should be defined as the primary outcome. The primary outcome is usually the outcome thought to be most responsive to the intervention and the most important. By definition, sample size is calculated on the basis of the primary outcome. Other (secondary) outcomes can also be defined, to provide additional information on the value of the intervention.
As a general rule, outcome measures that are continuous (measures on a known scale, such as weight, blood pressure, or forced expiratory volume in the first second) are more powerful1 and provide more information than dichotomous (yes/no) outcomes. Sometimes, however, a dichotomous outcome, such as readmission to hospital within 30 days (reported as the proportion of patients readmitted), is the most relevant outcome.
For each outcome, indicate how and when it will be measured, and, if necessary, cite relevant literature to show that it is an accepted and valid measure.
The outcome measures chosen for pharmacy practice research are often process-oriented, such as number of drug-related problems identified by the pharmacist. Although these variables do provide some information on the effect of an intervention, they are not well-validated outcome measures, in that they have typically not been shown to correlate with patient outcomes.
It is beyond the scope of this article to discuss methods of estimating sample size, but suffice it to say here that you must calculate the required sample size.2 Sample size drives the costs of and resources needed for the study; without it, the study will likely be underpowered (i.e., a study that cannot possibly answer your research question, which is both unethical and foolhardy).
As noted above, the sample size is calculated on the basis of the primary outcome. In the sample size calculation, you must justify all of the assumptions made to arrive at that figure, for example, “We assumed a rate of hospital admission in the control group of 20% over 6 months (based upon the work of Smith and others) and a reduction in hospital admission for heart failure with enhanced pharmacist care of 30% (based upon the systematic review of Jones and others).
The analytical plan is an essential part of the study protocol and cannot be developed “after the fact”, so you will need to talk with a biostatistician to establish the analytical plan before you begin. Many investigators omit this step because they themselves are uncomfortable with biostatistics, but this is ill advised. Get some help by speaking to a more experienced researcher or a biostatistician. You will not be able to compensate for a missing or inadequate analytical plan once data collection is complete.
Bias and confounding are factors that can affect study outcomes (sometimes even as much as the intervention) and that may “interfere” with assessment of the intervention. It is beyond the scope of this article to address all of the issues related to bias and confounding, but a few are worth mentioning.
Most pharmacy practice research cannot be blinded. Without blinding, patients and providers know the study group to which they have been assigned and can take extra steps. For example, patients assigned to “usual care” in a randomized dyslipidemia study might take some extra steps to see their physician and ask about cholesterol, and participants in the control group may thus receive an intervention that they might not otherwise have received. Some ways to mitigate this problem would be to provide a “sham” (weaker) intervention to the control group (e.g., a pamphlet on heart disease) or to delay delivering the intervention; in the latter situation, the control group eventually receives the intervention, after the comparative period of the trial has been completed. This is known as a waiting group control.
Ideally, outcome information will be collected by an investigator who is blinded to (unaware of) the treatment allocation. For example, a clerk who is not aware of treatment allocation could administer quality-of-life surveys to participants. However, it is sometimes impractical to implement blinded data collection.
Loss to follow-up is a potentially serious source of bias. Losses to follow-up can be minimized by keeping participants engaged with the research and the investigator. Losses to follow-up are especially damaging to a study if more patients are lost from one treatment arm than the other. In practice research done in the community, it is not unusual to lose 15% of patients to follow-up. Large losses to follow-up are a threat to validity, as the patients lost to follow-up are probably different from those who stayed in the trial. Guidance is that at least 80% of patients should have follow-up for a trial to be considered of good quality.
Pharmacy practice research is vital to the future of the pharmacy profession. Having good evidence is paramount as we strive to improve patient outcomes, expand scopes of practice, and justify new remuneration models.
The foregoing are a few simple guidelines for conducting a randomized controlled trial of a pharmacy intervention. The wise researcher, novice or otherwise, will not be intimidated by these guidelines but rather will consider, apply, and learn from them. If you do decide to undertake a pharmacy practice research study, don’t work alone. Get some assistance from an experienced researcher. Most would be happy to help.
1. Hulley SB, Cummongs SR, Browner WS, Grady DG, Newman TB. Designing clinical research. 3rd ed. Philadelphia (PA): Lippincott Williams & Wilkins; 2007.
2. Friedman LM, Furberg CD, DeMets DL. Fundamentals of clinical trials. 4th ed. New York (NY): Springer; 2010.
3. Tully MP. Research: articulating questions, generating hypotheses, and choosing study designs. Can J Hosp Pharm. 2014;67(1):31–4.
Competing interests: Dr Tsuyuki has received fees from BMSAstraZeneca (Advisory Board for Access with Evidence Development Programs), PharmaSmart International (consulting services, Pharmacy Advisory Board), Merck (consulting services), Abbott (Mavikontrol advisory board), and Boehringer-Ingelheim (data monitoring board for dabigatran studies). He has also received grants from Sanofi, Merck and AstraZeneca for investigator-initiated trials.
Canadian Journal of Hospital Pharmacy , VOLUME 67 , NUMBER 3 , May-June 2014