Figure 1 Results of the literature search. ADD = automated dispensing devices, BCMA = bar code medication administration, CPOE = computerized prescriber order entry.
Kieran Shah, Clifford Lo, Michele Babich, Nicole W Tsao, Nick J Bansback
Medication errors (any preventable event that may cause or lead to inappropriate medication use or patient harm while the medication is in the control of the health care professional, patient, or consumer) that lead to adverse drug events (any undesirable experience associated with a patient’s use of a drug) are known to represent a major threat to patient safety, despite widespread preventive programs and extensive education of hospital personnel.1–4 It has been estimated that when adverse drug events occur in the hospital setting, they increase the patient’s length of stay by an average of 4.6 days, and the cost to the Canadian health care system is $4685 per event4 ($6655 in 2016 Canadian dollars, adjusted for inflation). Fortunately, many medication errors are preventable, and the implementation of health information technologies, such as bar code medication administration (BCMA) systems, is increasingly being considered as one solution.4–6 In fact, the American Society of Health-System Pharmacists and the Healthcare Information and Management Systems Society both recommend the use of BCMA.7,8
BCMA systems reduce medication errors by electronically verifying the ‘5 rights’ of medication administration—right patient, right dose, right drug, right time, right route—at the patient’s bedside.7 For example, when a nurse scans a bar code on his or her identification badge, on the patient’s wristband, and on the medication to be administered, the data are delivered to a computer software system where algorithms check various databases and generate real-time warnings or approvals.7 Most systems then automatically document, in real time, the administration of the medication in an electronic medication administration record (eMAR).
Other than cost, one of the barriers to widespread adoption of BCMA technology is the lack of definitive evidence that BCMA actually reduces preventable medication errors, especially in hospitals that are already using other safety systems, such as computerized prescriber order entry (CPOE) and automated dispensing devices (ADDs).7,9 The objective of this systematic review was to determine the impact of BCMA on medication errors when used as part of a closed-loop medication administration system (i.e., BCMA with CPOE and ADD).
A comprehensive search, covering the years 1992 to 2015, was conducted within the MEDLINE, PubMed, and Embase databases, for English-language articles reporting on medication errors with the use of BCMA systems combined with CPOE and ADDs in hospital wards. The keywords ‘bar code’, ‘bar codes’, ‘bar coding’, and ‘barcoding’ generated the Medical Subject Heading (MeSH) terms ‘automatic data processing’, ‘medication errors’, and ‘medication systems, hospital’. The MeSH terms ‘systems analysis’ and ‘medication systems,’ adapted from Young and others,9 were used to broaden the search. Related articles identified by using the function ‘similar articles’ or ‘related articles’ in each database, pertaining to systematic reviews or other studies found to be relevant to this literature review, were also reviewed. This additional step helped to incorporate any other studies not found using the specific search terms. Finally, the reference lists of any relevant summaries, systematic reviews, and articles were reviewed to ensure that relevant articles not identified by the above search strategy were included.
All articles reporting on the use of BCMA at the point of care (i.e., the patient’s bedside) in a hospital setting, including randomized controlled trials, observational studies (cohort and case–control), and before-and-after studies, were considered for inclusion.
Studies were excluded if they examined the use of any bar code–based technologies used in other areas of the hospital, such as the pharmacy department, or in non–medication-related applications. Studies that did not report the impact of BCMA technology on medication error rates were also excluded. Studies that did not include the BCMA technology as a closed-loop medication process (i.e., in addition to CPOE and ADD systems) were excluded.
All relevant abstracts and titles were screened to assess the eligibility of studies for inclusion. Two reviewers (K.S. and C.L.), working independently, used a standardized data extraction form to extract information from the articles, such as study design, sample size, inclusion and exclusion criteria for the individual study, interventions, outcomes, and results. These data were used in a critical appraisal of the studies, whereby the strengths and weaknesses of the studies, their sources of bias, and their overall quality and reliability were determined, by overall consensus, using the Newcastle–Ottawa Scale.
A total of 430 citations were found, of which 393 were excluded at the abstract review level (Figure 1). These articles were excluded because they did not include the specified complementary technologies (CPOE and ADD), did not involve use of BCMA at the patient’s bedside, did not report the impact of BCMA on medication error rates, or reported only preliminary results on medication error rates. Of the 37 articles selected for full-text review, 5 met the inclusion criteria for evidence synthesis. Three of these studies used direct observation to determine medication errors,10–12 whereas the other 2 studies relied on self-reporting.13,14 Direct-observation studies are considered more reliable than those based on self-reporting15; however, both types of data collection are commonly used in studies examining medication errors. Three of the studies investigated the outcomes when BCMA technology was added to existing ADD and CPOE systems,10,11,13 one study examined a setting where all 3 technologies were implemented at once,12 and the final study investigated a setting where BCMA was added to existing ADDs, followed by implementation of CPOE.14 Given variations among the studies in terms of their methods, periods between data collection, populations, and care settings, we were unable to perform a pooled quantitative analysis incorporating all of the results. In general, the studies focused on 3 categories of errors: administration errors (timing or nontiming), transcription errors, and total medication errors. The study characteristics are summarized in Table 1, and overall results are summarized in Table 2.
|
|
||
|
Figure 1 Results of the literature search. ADD = automated dispensing devices, BCMA = bar code medication administration, CPOE = computerized prescriber order entry. |
||
Table 1 Characteristics of Included Studies
Table 2 Effect of BCMA on Medication Errors
The 3 studies that used direct-observation methods and a prospective before-and-after design examined differences in medication administration error rates.10–12 Two of these studies concluded that BCMA reduced the absolute rate of nontiming errors by 4.6%11 or 4.7%,10 but their findings on timing-related medication administration errors were conflicting.
Poon and others10 studied the impact of BCMA technology on patient safety in medical and surgical wards and intensive care units (ICUs) where a CPOE and ADD system was already established. They found that after implementation of BCMA, nontiming errors were reduced from 11.5% to 6.8%, a 41.4% relative risk reduction (RRR) (95% confidence interval [CI] −34.2% to −47.1%; p < 0.001). The nontiming errors were also analyzed by subtype. Wrong medication errors were reduced from 1.0% to 0.4% (RRR 57.4%, 95% CI −39.2% to −79.3%; p < 0.001), wrong dose errors from 2.0% to 1.1% (RRR 41.9%, 95% CI −27.9% to −58.7%; p < 0.001), wrong route of administration errors from 0.3% to 0.1% (RRR 68%, 95% CI −37.4% to −97.7%; p < 0.001), and administration documentation errors from 2.9% to 0.6% (RRR 80.3%, 95% CI −73.7% to −87.0%; p < 0.001). Potential adverse drug events due to nontiming administration errors decreased from 3.1% to 1.6% (RRR 50.8%, 95% CI −39.1% to −61.7%; p < 0.001). Specifically, there were RRRs of 48.5% (95% CI −33.9% to −64%; p < 0.001) and 54.1% (95% CI −36.8% to −70.4%; p < 0.001) for ‘significant’ and ‘serious’ potential adverse drug events, respectively, as adjudicated by a multidisciplinary panel of physicians, nurses, and pharmacists. However, there was no significant reduction in potential adverse drug events categorized as life-threatening.
Helmons and others11 examined medication administration error rates, as well as the accuracy of medication administration, in 2 medical–surgical units and 2 ICUs in a 386-bed teaching hospital in the United States. The incorporation of BCMA technology into an established CPOE and ADD system decreased medication administration errors in the medical–surgical units from 8% to 3.4%, representing a 56.9% RRR (p < 0.0001); however, no change in error rates was observed in the ICUs. This difference in findings for different settings within the hospital was largely attributed to a decrease in omission errors in the medical–surgical units, a type of error that did not occur frequently in the ICUs. The accuracy of medication administration was measured with the 9-point accuracy indicator system of the California Nursing Outcomes Coalition.17 One of the indicators, ‘two forms of identity not checked (orally confirming patient identity and scanning the bar code on the patient’s wristband)’, decreased from 13.4% to 6.9% (p < 0.0001) in the medical–surgical units.11 However, the use of BCMA led to increases in distractions or interruptions (from 15.5% to 25.2%; p < 0.0001) and in medications given without explanation to the patient (from 10.9% to 14.9%; p = 0.045).11 In the ICUs, none of the accuracy indicators improved after implementation of BCMA, except noncompliance with medication charting, which declined from 24.4% to 6.7% (p < 0.0001).11
Poon and others10 were the only authors to conclude that BCMA reduces wrong time errors. This type of error, defined as medication administration that was early or late by more than 1 h, decreased from 16.7% to 12.2% (RRR 27.3%, 95% CI −21.0% to −33.8%; p = 0.001). However, there was no significant reduction in potential adverse drug events as a result of wrong time errors. In contrast, Helmons and others11 found that wrong time errors increased after BCMA implementation in both the medical–surgical units (from 2.7% to 4.5%; p < 0.05) and the ICUs (no statistically significant difference).
Franklin and others12 conducted their before-and-after direct-observation study in a 28-bed surgical ward of a teaching hospital in London, UK. These authors investigated the impact of a closed-loop medication administration (CPOE, ADDs, and BCMA) on medication administration errors and prescribing errors; however, they did not report their findings on timing and nontiming medication administration errors separately. There was a statistically significant reduction in non-IV medication administration errors, from 7.0% to 4.3% (absolute difference 3.7%, 95% CI −0.9% to −4.5%; p = 0.005), after implementation of a closed-loop medication administration system. However, the reduction in mean clinical severity score (assessed by judges on a scale of 0 [no harm] to 10 [death], according to a validated method) was nonsignificant. The predominant types of errors that were reduced were wrong dose errors (1.8% before versus 0.4% after implementation; no p value reported) and omission errors not due to nonavailability of the drug (2.6% before versus 0.9% after implementation; no p value reported). Furthermore, the authors found a statistically significant reduction in prescribing errors, from 3.8% to 2.0% (absolute difference 1.8%, 95% CI −0.9% to −2.7%; p < 0.001), with no differences in mean clinical severity score; this reduction was likely the result of concurrent implementation of CPOE, rather than a direct consequence of BCMA. There was a nonsignificant trend for more prescribing errors to be resolved before reaching the patient (48% before versus 67% after implementation).
Finally, Franklin and others12 found that not checking patient identity before medication administration was significantly reduced, from 82.6% to 18.9% (absolute difference 63.7%, 95% CI 60.8% to 66.6%; p < 0.001), after implementation of the closed-loop medication administration system. The authors noted that full compliance in checking patient identity before each medication administration was not achieved because of informal practices, such as affixing bar codes to patients’ furniture, with the furniture, rather than the patient’s wristband, being scanned.
Although eMARs were implemented along with BCMA in 3 of the studies,10,12,14 Poon and others10 were the only authors to report the impact of these technologies on transcription errors. Transcription errors, defined as errors in the transcription of physicians’ orders onto the MAR for medications administered during the observation period, occurred at a baseline rate of 6.1%. Of these, 48% were classified as potential adverse drug events, with 25% being classified as ‘significant’ and 22% classified as ‘serious’ in severity.10 The types of transcription errors included directions stated in the physician’s order incompletely or incorrectly transcribed onto the MAR, physician’s order not transcribed onto the MAR at all, and incorrect formulations transcribed onto the MAR. Once BCMA with eMAR was deployed, such transcription errors were completely eliminated.
Two of the studies, based on self-reporting methods, reported RRRs for total medication errors of 49%13 and 75%.14 Higgins and other13 studied the incidence of total medication errors (specifically medication dispensing and administration errors) before and after addition of BCMA to an established CPOE and ADD system in the emergency department of a 655-bed teaching hospital in the United States, using data from an existing anonymous safety reporting system. They categorized the errors as ‘near-miss’ events (a situation with potential to cause harm or unsafe conditions that was noted by a provider, but corrected before reaching the patient) and ‘errors that reached the patient’.13 Interestingly, they found a 90% increase in near-miss events after implementation of BCMA (20 administration errors per million doses dispensed versus 38 administration errors per million doses dispensed; p < 0.05). When they separated the low-severity errors (identified before medication administration) from those that reached a patient, possibly necessitating monitoring or treatment for harm, they found a statistically significant relative reduction of 75% in errors reaching the patient (3.26 per million doses dispensed to 0.8 per million doses dispensed; p < 0.05). This error reduction was sustained for 15 months after BCMA implementation.
Richardson and others14 described the experience of a small New England hospital that added BCMA technology, followed by CPOE, to an established ADD system over a 3-year period. Self-reported data supplied by nurses showed a trend toward a reduction in total medication errors (types not defined), from 2.89 errors per 10 000 doses to 1.48 errors per 10 000 doses. Furthermore, the rate of bar code scanning by nurses increased from 94% at the end of the first year to 98% at the end of the study. Unfortunately, no analysis was performed to determine statistical significance.
To the authors’ knowledge, this is the first systematic review to investigate the effects of BCMA on patient safety and medication administration errors when used in conjunction with CPOE and ADDs. In a previous systematic review, Young and others9 included studies that used BCMA alongside CPOE and ADDs, BCMA with one other technology, or BCMA on its own. Their broad inclusion criteria made it difficult to isolate the magnitude of benefit provided by BCMA within a closed-loop medication administration system.9 In addition, their search covered a narrower period (1999–2009), whereas the current systematic review captured articles published between 1992 and 2015. This longer search period resulted in the inclusion of 3 new articles, all published after 2009, which allowed for an updated analysis using more homogeneous data. Two of these new studies used direct-observation methodology, which addressed one of the limitations identified by Young and others.9 Although no studies published between 1992 and 1999 met our inclusion criteria, we included those years in the literature search because BCMA technology was developed during this period.
The ability of BCMA to reduce nontiming-related administration errors was evident and generally accepted in the 3 studies that investigated this type of error.10–12 Poon and others10 found that dosing, incorrect medication, and wrong route errors were all reduced. Similarly, Franklin and others12 reported that dosing errors were one of the predominant types of error reduced by BCMA, and Helmons and others11 and Franklin and others12 found reductions in omitted doses. Because of the direct-observation design of these 3 studies, it is unclear whether reported omission errors were in fact wrong time errors, with the medications being given at another time but not observed. All 3 studies showed that, in addition to ensuring that patients received their medications, BCMA technology reduced errors resulting in administration of a wrong dose or wrong medication, as well as errors involving medication being given by the wrong route.10–12 These results are logical, given that the BCMA technology checks the ‘5 rights’ of medication administration at the patient’s bedside.
Wrong time errors are generally considered less severe than other types of errors. That is why some studies have reported wrong time errors separately from medication administration errors11 or have excluded them entirely.18 Two of the studies included in the current review10,11 reported conflicting data on wrong time errors associated with BCMA. The increase in wrong time errors in the study by Helmons and others11 was not explained by the new technology causing nurses to spend more time on medication administration, because the median duration of medication administration did not change after BCMA implementation. However, unless there were efficiency gains, the reduction in wrong time errors in the study by Poon and others10 could have been explained by the accompanying eMAR technology, since some eMARs display a visual status board of actions required for each patient. Therefore, the net effect of BCMA on wrong time errors, whether a decrease or an increase, is inconclusive but likely depends on the implementation and design of the particular closed-loop system. Further research is needed to determine the specific implications of BCMA for this type of error.
Two of the studies10,12 reported conflicting results in terms of the severity of potential medication administration errors prevented by BCMA, albeit using different methods and vague definitions to judge clinical severity. Poon and others10 found that the potential errors reduced by BCMA were ‘significant’ or ‘serious’ but not ‘life-threatening’. Conversely, Franklin and others12 found that BCMA did not significantly reduce the mean severity score of medication administration errors prevented; however, their small study in a single unit was likely insufficiently powered to evaluate serious medication errors. Further research (involving larger studies over longer study periods) is needed to determine the impact on life-threatening medication errors of BCMA within a closed-loop medication administration system. In particular, institutions that were early adopters of this technology are encouraged to publish their safety data.
Two of the studies found an increase in the percentage of doses for which a patient’s identity was checked before medication administration following implementation of BCMA in medical–surgical11 or surgical12 wards. However, this benefit may be offset by nurses being less likely to explain the side effects of a medication to the patient, possibly because there may be more distractions and interruptions after BCMA implementation.11 In addition, one of these studies found no significant improvement in the rate of checking 2 forms of identity in the ICU.11 The authors postulated that baseline compliance with the requirement to check 2 forms of identity is low in the ICU because most patients are unconscious, meaning that oral verification of a patient’s identity is impossible.11 Furthermore, visually checking the patient’s name and medical record number on the wristband and then scanning the wristband as a dual method of checking the patient’s identity was likely not performed in the ICU, because each nurse was assigned to the same patient for the entire shift.11 Therefore, checking 2 forms of identity may not be the best indicator of medication accuracy in all settings.
Poon and others10 were the only authors to conclude that BCMA completely eliminates transcription errors. Each of the transcription errors that they identified could have led to potential adverse events, but elimination of these errors was likely a result of the accompanying eMAR technology and a reduction in the need for clerical MAR entries, rather than being directly attributable to BCMA. Similarly, Helmons and others11 found that compliance with charting of medication administration on the MAR increased significantly in the ICU after implementation of BCMA, but this outcome may have been related in part to the relatively low baseline compliance. Taken together, these studies indicate that not only does the use of BCMA technology have the potential to improve the accuracy of the MAR, it facilitates nurses’ compliance with MAR charting.10,11 However, the impact on both of these error types will depend on each institution’s current practices and how it implements and configures BCMA.
Higgins and others13 and Richardson and others14 reported a reduction in total medication errors using self-reporting methods. Direct observation is considered more reliable than self-reporting,15 but the latter is a pragmatic method of determining error rates in hospitals. Its major weaknesses are the potential for under-reporting and the inability to distinguish between an increase in error rates and an increase in reporting rates. These reasons may explain why Higgins and others13 found a significant reduction in total medication errors reaching the patient but also reported an increase in near-miss errors after implementation of BCMA: an increase in self-reported near-miss medication errors should be expected when BCMA technology is first deployed.
Human factors and technical issues are important considerations for BCMA technology. Every study included in this systematic review reported an inability to completely eliminate medication administration errors and an inability to achieve 100% scanning rates,10–14 although Richardson and others14 came close to the latter goal, with a 97% scanning rate after 36 months of rapid quality improvement cycles. Workarounds by nurses and technical issues contributed to the incomplete scanning rates.10,12–14 Technical issues included smudged bar code labels, lack of updating of bar codes with a new pharmacy inventory, and activation of alerts despite correctly delivered care, all of which can result in increased scanning failures and, consequently, near-miss events.12,13
Despite these limitations, there are no data in this review citing BCMA as a direct cause of medication administration errors. In fact, all of the benefits reported—reductions in administration errors, transcription errors, and total medication errors, as well as reductions in severity of errors—were observed despite nursing workarounds and technical issues. Poon and others10 concluded that implementation of BCMA should not be regarded as a single event, but rather an ongoing process that requires training and education, along with improvements and modifications. Therefore, we encourage institutions that have adopted this technology to share their experiences. We also encourage the authors of studies using direct-observation methodology to perform follow-up analyses to determine whether the benefits of BCMA are sustained over time.
The Newcastle–Ottawa Scale16 was used to assess the quality of the nonrandomized trials included in this study; this validated tool is recommended for this purpose by the Cochrane Handbook for Systematic Reviews of Interventions.19 The maximum score for any individual study is 9, and the results of this analysis are presented in Table 1.
In terms of the selection criteria, the studies included in the current systematic review had representative populations, as they were conducted in tertiary care hospitals and included multiple sites, such as medical–surgical units, general medicine wards, and ICUs.10,11,13,14 Only one study focused on a single (surgical) ward,12 which was less desirable.
Given the observational design of these studies, the largest and most consistent limitation is the theoretical risk of confounding. None of the studies performed statistical modelling to control for potential confounding variables; therefore, no points were awarded in the comparability category for design and analysis.
In terms of outcome measures, major factors that reduced study quality included the use of self-reporting methodology, rather than direct observation of administration errors. Although studies that used self-reporting13,14 were rated less favourably, the self-reporting methodology did allow for a longer duration of follow-up (8 months to 1 year before implementation; 2 to 4 years after implementation) relative to direct observation, which typically occurred over only a few days. Two of the studies that used direct observation did not specify the duration of follow-up before and after the intervention, which led to less favourable ratings.10,11 For the single direct-observation study that did specify the follow-up period, this duration was only 2 weeks.12
Finally, the outcomes of interest in all studies (administration errors) were readily available from direct observation and self-reports, which made loss of data or attrition bias unlikely. Overall, the quality of studies included in this review (total score 6 for every study) was typical of observational studies conducted with medication management technology and automation. In a utopian world, we would call for randomized controlled data, but from a pragmatic perspective, the return on investment with this type of evaluation is low, and such studies will likely never be done. Instead, we encourage those who have implemented BCMA technology to share their experiences.
With regard to the search methods, included studies were restricted to those published in English, as we did not have the resources to translate articles published in other languages. Unfortunately, we were unable to assess publication bias because of the paucity of published studies with unfavourable results. We did not include any unpublished studies, since such studies have not undergone peer review and their reliability is uncertain. Nevertheless, our literature search was thorough and robust, and detailed data were extracted from each study and then synthesized to arrive at the most conclusive outcomes.
Comparative evidence providing clinical justification of BCMA with its complementary technologies is limited. Results from the 5 studies included in this review suggest that BCMA has the potential to reduce nontiming administration errors, transcription errors, and total medication errors. Its impact on wrong time errors, an error type that is less clinically significant, is unclear. Additionally, BCMA has the potential to improve compliance with the requirements to check patient identity before administering medications and to chart the administration of medications on the MAR. Although BCMA has been shown to reduce serious and significant nontiming medication administration errors, more longitudinal studies are required to capture data on life-threatening errors. Institutions that were early adopters of this technology are encouraged to publish their long-term data and to share their experience in managing human factors and technical issues that are barriers to completely eliminating medication administration errors and achieving 100% bar-code scanning rates. Finally, future research should focus on the economic impact of using BCMA (for example, through a full cost–benefit analysis incorporating all direct, indirect, and intangible costs and benefits) to further facilitate the assessment of its use in Canadian hospitals.
1 Leape LL. Systems analysis of adverse drug events. JAMA. 1995; 274(1):35.
2 Kohn LT, Corrigan JM, Donaldson MS, editors. To err is human: building a safer health system. Washington (DC): National Academy Press, Institute of Medicine, Committee on Quality of Health Care in America; 2000 [cited 2013 Jun 18]. Available from: www.nap.edu/openbook.php?isbn=0309068371
3 Baker GR. The Canadian Adverse Events Study: the incidence of adverse events among hospital patients in Canada. CMAJ. 2004;170(11):1678–86. 
4 Bates DW, Spell N, Cullen DJ, Burdick E, Laird N, Petersen LA, et al. The costs of adverse drug events in hospitalized patients. Adverse Drug Events Prevention Study Group. JAMA. 1997;277(4):307–11.
5 Bates DW. Using information technology to reduce rates of medication errors in hospitals. BMJ. 2000;320(7237):788–91.
6 Bates DW, Gawande AA. Improving safety with information technology. N Engl J Med. 2003;348(25):2526–34.
7 American Society of Health-System Pharmacists, Section of Pharmacy Informatics and Technology. ASHP statement on bar-code-enabled medication administration technology. Am J Health Syst Pharm 2009; 66(6):588–90.
8 Electronic medical record adoption model (EMRAM)SM [website]. Chicago (IL): HIMSS Analytics; [cited 2013 Jul 23]. Available from: www.himssanalytics.org/emram/emram.aspx
9 Young J, Slebodnik M, Sands L. Bar code technology and medication administration error. J Patient Saf. 2010;6(2):115–20.
10 Poon EG, Keohane CA, Yoon CS, Ditmore M, Bane A, Levtzion-Korach O, et al. Effect of bar-code technology on the safety of medication administration. N Engl J Med. 2010;362(18):1698–707.
11 Helmons PJ, Wargel LN, Daniels CE. Effect of bar-code-assisted medication administration on medication administration errors and accuracy in multiple patient care areas. Am J Health Syst Pharm. 2009;66(13):1202–10.
12 Franklin BD, O’Grady K, Donyai P, Jacklin A, Barber N. The impact of a closed-loop electronic prescribing and administration system on prescribing errors, administration errors and staff time: a before-and-after study. Qual Saf Health Care. 2007;16(4):279–84.
13 Higgins T, Heelon M, Siano B, Douglass L, Liebro P, Spath B, et al. Medication safety improves after implementation of positive patient identification. Appl Clin Inform. 2010;1(3):213–20.
14 Richardson B, Bromirski B, Hayden A. Implementing a safe and reliable process for medication administration. Clin Nurse Spec. 2012;26(3): 169–76.
15 Edlavitch SA. Adverse drug event reporting: improving the low US reporting rates. Arch Intern Med. 1988;148(7):1499–503.
16 Wells GA, Shea B, O’Connell D, Peterson J, Welch V, Losos M, et al. The Newcastle–Ottawa Scale (NOS) for assessing the quality of nonrandomised studies in meta-analyses. Ottawa (ON): Ottawa Hospital Research Institute; 2014 [cited 2016 Jun 9]. Available from: www.ohri.ca/programs/clinical_epidemiology/oxford.asp
17 Clarke SP, Donaldson NE. Nurse staffing and patient care quality and safety. In: Hughes RG, editor. Patient safety and quality: an evidence-based handbook for nurses. Rockville (MD): Agency for Healthcare Research and Quality; 2008 [cited 2013 Aug 16]. Available from: www.ncbi.nlm.nih.gov/books/NBK2676/
18 Allan EL, Barker KN. Fundamentals of medication error research. Am J Hosp Pharm. 1990;47(3):555–71.
19 Higgins JPT, Green S. Cochrane handbook for systematic reviews of interventions. Version 5.1.0. The Cochrane Collaboration; 2011 Mar [cited 2014 Jul 28]. Available from: http://handbook.cochrane.org/
Competing interests: None declared. ( Return to Text )
Funding: None received. ( Return to Text )
Canadian Journal of Hospital Pharmacy, VOLUME 69, NUMBER 5, September-October 2016