Janice M Irvine-Meek , Odette N GouldABSTRACT
Background:
Most community-dwelling older adults are engaged in medication self-management activities. Deviation in these activities can lead to adverse outcomes for patients and an increased burden on the health care system. Successful medication self-management involves a complex interaction among cognitive, functional, and psychosocial variables. Several assessment instruments have been developed, but there remains a need for an effective and comprehensive tool.
Objective:
To evaluate the psychometric properties (inter-rater reliability, test–retest reliability, and validity), as well as the usability, of the Self-Medication Assessment Tool (SMAT), an instrument designed to measure elderly patients’ ability to manage their medications.
Methods:
The study enrolled patients 65 years of age or older who were living independently and were admitted to family medicine beds in a community hospital in eastern Canada. Three subsamples of the population were identified. The inter-rater reliability group was videotaped and scored independently by 2 pharmacists. The test–retest reliability group was tested with the SMAT and was retested with the same tool a week later. The usability group was interviewed after using the SMAT to determine their satisfaction. Standard neuropsychological measures (Cognitive Competency Test, clock-drawing test, and Mini Mental State Examination [MMSE]) were used to determine convergent and divergent validity. Pill counts, refill rates, and use of adherence aids or reminders before study enrolment were used as measures of concurrent validity.
Results:
A total of 121 patients (mean age 81.5 years) were enrolled. The scales of the SMAT were determined to have good internal consistency and high inter-rater and test–retest reliability. Convergent validity was evidenced by the high positive correlation between the functional scale of the SMAT and the results of the clock-drawing and Cognitive Competency tests ( p < 0.01) and between the cognitive and recall scales of the SMAT and the results of the clock-drawing test ( p < 0.05), the MMSE ( p < 0.01), and the Cognitive Competency Test ( p < 0.01). Patients reported being highly satisfied with their experience.
Conclusion:
The SMAT is a practical, reliable, comprehensive instrument with demonstrated convergent validity, strong patient acceptability, and various internally consistent scales that assess multiple dimensions of elderly patients’ ability to self-manage their medications. Further testing is required to show that the SMAT correlates with medication adherence.
KEYWORDS: medication therapy management , geriatric assessment , psychometrics , self-care , medication adherence
RÉSUMÉ
Contexte :
La plupart des adultes âgés vivant dans la communauté gèrent eux-mêmes les activités liées à la prise de leurs médicaments. Tout changement dans ces activités peut entraîner des résultats indésirables pour les patients et un fardeau supplémentaire pour le système de santé. La réussite de l’autogestion des médicaments fait appel à une interaction complexe entre diverses variables cognitives, fonctionnelles et psychosociales. Aucun des nombreux outils mis au point pour évaluer le degré de réussite de l’autogestion des médicaments chez ces personnes ne s’est encore révélé efficace ni complet.
Objectif :
Évaluer les propriétés psychométriques (la fidélité interévaluateurs, la constance et la validité) ainsi que la convivialité de l’outil d’évaluation de l’automédication (OEAM), conçu pour mesurer la capacité des patients âgés à autogérer la prise de leurs médicaments.
Méthodes :
On a mené une étude chez des patients âgés d’au moins 65 ans vivant de façon autonome qui ont été hospitalisés dans une unité de médecine familiale d’un hôpital communautaire de l’Est du Canada. Trois sous-échantillons de la population à l’étude ont été déterminés. Le groupe fidélité interévaluateurs a été filmé puis évalué indépendamment par deux pharmaciens. Le groupe constance a fait l’objet d’une première évaluation au moyen de l’OEAM, puis d’une deuxième évaluation avec ce même outil après une semaine. Pour mesurer la satisfaction des patients, le groupe convivialité a été interviewé après l’évaluation au moyen de l’OEAM. Des mesures neuropsychologiques standards (test d’évaluation cognitive, test de l’horloge et le mini-examen de l’état mental [MEEM]) ont été utilisées pour déterminer la validité convergente et divergente. Le comptage des comprimés, le taux de renouvellement des ordonnances et l’emploi d’outils d’aide à l’observance ou de mémentos avant l’inscription à l’étude ont été utilisés pour mesurer la validité concourante.
Résultats :
Un total de 121 patients (âge moyen de 81,5 ans) ont été inscrits à l’étude. Les échelles de l’OEAM ont montré une bonne cohérence interne ainsi qu’une fidélité interévaluateurs et une constance élevées. La validité convergente a été attestée par une forte corrélation positive entre l’échelle fonctionnelle de l’OEAM et les résultats du test de l’horloge et du test d’évaluation cognitive ( p < 0,01) et entre les échelles cognitives et d’évocation de l’OEAM et les résultats du test de l’horloge ( p < 0,05), du MEEM ( p < 0,01) et du test d’évaluation cognitive ( p < 0,01). Les patients ont déclaré être très satisfaits de leur expérience.
Conclusion :
L’OEAM est un outil pratique, fiable et complet avec une validité convergente démontrée, une forte acceptabilité par les patients et diverses échelles dotées de cohérence interne évaluant des aspects multiples de la capacité des patients âgés à autogérer leurs médicaments. D’autres évaluations sont cependant nécessaires pour démontrer la corrélation entre l’OEAM et l’observance thérapeutique.
MOTS CLÉS : gestion du traitement médicamenteux , évaluation gériatrique , psychométrie , autosoins , observance du traitement médicamenteux
[Traduction par l’éditeur]
A person’s capacity for medication management is defined as the “cognitive and functional ability to self-administer a medication regimen as it has been prescribed”.1 The majority of community-dwelling older adults are engaged in the self-care activity of managing a medication regimen. There is considerable potential for deviations in this self-care activity, and the consequences to the independence of older adults and to the health-care system can be substantial. It has been estimated that 50% to 75% of people do not take their medications as prescribed,2 and drug-related problems are reported to be a major or contributing cause for 19% to 28% of hospital admissions among people over age 50 years.3–6 Furthermore, a change in the capacity of older adults to self-administer a medication regimen over 12 months was associated with an increase in the frequency of visits to the emergency department and with a move to an assisted living situation.7,8
Medication self-administration is a complex task that involves cognitive, functional, and psychosocial variables.9,10 Several studies have employed various tests of cognitive capacity and memory in an attempt to predict success in medication self-administration.11–18 The Drug Regimen Unassisted Grading Scale (DRUGS)7,8 and the Brief Medication Questionnaire19 were designed to measure various aspects of capacity for medication self-management. The DRUGS tool and the Medication Management Ability Assessment were highly correlated with each other and with cognitive function, but they lacked predictability.20 A comprehensive tool to evaluate elderly patients’ ability to manage their medications is still needed.21,22 The deficiencies in assessment tools published to date are the lack of capacity for scoring or otherwise quantifying the results, poor inter-rater reliability, undetermined reproducibility of the results, and lack of data on their acceptability to patients.
The present report describes an evaluation of the Self-Medication Assessment Tool (SMAT), a comprehensive instrument that can be used to screen for deficits in medication self-management and to facilitate targeted interventions in elderly patient populations. An experienced pharmacist specializing in geriatrics (J.I.-M.) and a cognitive psychologist specializing in research about aging (O.N.G.) developed the SMAT at a community hospital in eastern Canada. The SMAT includes functional, cognitive, recall, self-reported adherence, and purposeful nonadherence items selected to quantify patients’ capacity for medication self-management. Thus, the instrument combined elements from existing tools7,8,11,18,19 with new elements not previously used. The development of the SMAT and preliminary testing for face validity and usability were described in a previous report.23 The objectives of the present study were to test the inter-rater reliability, test–retest reliability, and validity of the SMAT.
The study was conducted at The Moncton Hospital (now part of Horizon Health Network), a community hospital in eastern Canada. Ethics approval was obtained from both the Southeast Regional Health Authority (now Horizon Health Network) Research Ethics Board and the Mount Allison University Research Ethics Board.
Patients were eligible for inclusion in the study if they had been admitted to a family medicine bed at the hospital, were 65 years of age or older, spoke English, were taking 2 or more prescribed medications before admission, had intact verbal communication ability, were living independently in the community before admission, and had undergone pharmacist-verified medication reconciliation at the time of admission.
Patients were excluded from the study if they were living in a long-term care facility or had extensive assistance from family members or caregivers in managing their medications. Patients with a medical condition of high acuity that would have prevented participation in the testing were also excluded.
A trained research assistant screened the daily admission records for patients who met the inclusion criteria and approached eligible patients to invite them to participate in the study. All participants provided written informed consent. The study goal was to test a minimum of 120 patients. Patients were initially asked to participate in the inter-rater reliability group. Patients who refused to be videotaped but who agreed to participate in the study were enrolled in the general sample. The original goal was to include one-quarter of the study patients in the inter-rater reliability group.24 However, preliminary analyses indicated that given the large effect sizes, sufficient statistical power would be obtained with a smaller sample. In the second stage of enrolment, patients who met the inclusion criteria (and who were not included in the inter-rater reliability subsample) were asked to participate in the test–retest reliability group. Patients who refused to be tested twice but agreed to participate in the study were enrolled in the general sample. Again, preliminary analyses indicated that sufficient statistical power was achieved before the original goal of testing one-quarter of the patients was reached. In the third stage of enrolment, a sample of the study participants were asked to participate in the usability group, with the goal of obtaining at least one-quarter of the patients in this subsample.25 Finally, the rest of the sample served as a comparison group for the reliability and usability subsamples.
Inter-rater reliability was tested with the first subsample of patients, as described above. This part of the study involved videotaping of face-to-face interviews during which one of the study pharmacists conducted the SMAT. This pharmacist then scored the patient off-camera. A second study pharmacist subsequently and independently viewed and scored the videotaped interview. The 2 pharmacists received the same training in administration of the SMAT.
Test–retest reliability was determined with the test–retest subsample, described above. For each of these patients, a single pharmacist administered the SMAT twice, with an interval of 1 week between testing. Participants’ responses during initial administration of the SMAT were compared with their responses on repeat administration. A total of 4 pharmacists participated in the test–retest phase, all of whom had received standardized training in administration of the instrument.
To determine the acceptability of the SMAT and to evaluate both the potential for misinterpretation of the questions and the potential that the questions were invalid for the target population, brief face-to-face post-test interviews were carried out with a number of patients.
Pill counts were conducted for patients whose medication supplies from home were available. For each medication, the percent adherence was calculated with the following equation: (no. of tablets taken/no. of tablets that should have been taken, based on prescription fill date and label instructions) × 100. Average percent adherence was determined by dividing the sum of all calculated percent adherence values by the total number of medications.26
To determine pharmacy refill rates, the patient’s community pharmacy was asked to provide a prescription dispensing profile for the 6 months before admission. Self-reported home medications were matched with those documented in the profile. The percent adherence with refills was calculated as follows: (no. of days supply for each prescription/no. of days between refills) × 100. The average percent adherence was calculated as the sum of all percent adherence scores divided by the total number of medications refilled.27
Absolute adherence scores were also calculated for both pill counts and refill rates. For example, for a patient who took 20% too many or too few doses, a score of 80% adherence was recorded.
A trained research assistant reviewed each patient’s medication regimen and calculated the Medication Complexity Index.28 The research assistant also applied the Beers criteria3 and identified the number and type of high-risk medications. The total Beers score for each patient was calculated by summing weighted scores, whereby high-severity medications were multiplied by 2 and low-severity medications by 1.
Occupational therapists administered the Mini Mental State Exam (MMSE),29 the clock-drawing test,30 and the Cognitive Competency Test (CCT),31 either just before or after the SMAT was administered. A total of 3 occupational therapists were involved in the study, and for each patient, a single therapist performed all of the cognitive assessments.
The SMAT, which was developed as part of this project, included scales measuring functional, cognitive, recall, self-reported adherence, and purposeful nonadherence dimensions.23 A standard set of stimuli (pill bottles, labels, etc.) was used for all patients. A percentage score was obtained for each dimension, higher score indicating better performance. The functional scale measured sensory, perceptual, and physical abilities by means of items such as the ability to read pill-bottle labels in different font sizes; to open different types of prescription vials; to remove tablets from a vial, a dosette, and a bubble pack; and to identify tablets by colour. The cognitive scale measured the ability to make judgments, manipulate information, and interpret instructions through tasks such as devising a plan of action for a medication regimen based on one or multiple pill bottle labels, correctly filling a weekly pill organizer using tablets from multiple vials, and locating the medication for a specific day and time in 2 types of compliance aid. The recall scale required the patient to recall his or her own medications, including drug names, indications, dosing, and descriptors such as colour, shape, and strength. The self-reported adherence scale included patients’ self-evaluation of their adherence in terms of medication timing, frequency, and dose. The purposeful nonadherence scale combined self-reporting of whether a medication was problematic, whether a medication was helpful, and the frequency and causes of purposeful nonadherence.
For each patient tested, the study pharmacist assigned to that patient was asked to record his or her clinical recommendations on the degree of supervision and the use of adherence aids needed to optimize future adherence to medication regimens. Supervision was ranked as 0 for no supervision required, 1 for minimal supervision, 2 for moderate supervision, and 3 for full supervision.
The research assistant contacted a sample of participants up to 1 week after completion of the SMAT, according to availability (i.e., not discharged from hospital), and administered a post-assessment questionnaire. Items included satisfaction with the length of the interview and the clarity of instructions, distress evoked by the testing, and perceived usefulness in identifying adherence concerns. A 7-point Likert scale was used for each item.
Descriptive analyses based on the demographic and neuropsychological measures were performed for the entire sample. The internal consistency of each scale of the SMAT was calculated as Cronbach’s α. This value provides a measure of whether each of a set of questions measures the same general construct and thus whether the questions “belong together” to form a scale within the test. Generally, internal consistency is considered acceptable if Cronbach’s α is greater than 0.8.
The inter-relationships between the scales were calculated as Pearson’s correlation coefficient. This measure indicates whether each scale is accessing different dimensions of performance. This technique is used to show that the scales are not all measuring the same construct and that it is reasonable to consider the tool as having multiple scales.
One-way analysis of variance (ANOVA) tests were carried out on the inter-rater, test–retest, and main samples to establish that the 3 samples did not differ in terms of demographic and neuropsychological measures.
Inter-rater reliability was determined for each scale of the SMAT, by determining rates of agreement between the 2 pharmacists who scored the videotaped assessments. Reliability was expressed as percent agreement, correlations, and root mean square deviations (RMSDs). The RMSD is the average value by which the 2 scorers disagreed using each of the SMAT scales. For example, an RMSD of 1 means that the 2 scorers differed, on average, by 1 point.
Test–retest reliability was assessed with the same statistical measures as inter-rater reliability and involved comparing patients’ scores obtained during the first and second testing sessions.
The convergent and divergent validity of the SMAT were tested by means of Pearson correlations. Convergent validity measures whether the SMAT scores agreed with other established relevant tests. Divergent validity is meant to show that the SMAT scales are not redundant with other tests. Pearson correlations were used to investigate the relationships between the various scales of the SMAT and the Medication Complexity Test, the clock-drawing test, the MMSE, the CCT, and the patient’s age. Specifically, it was expected that the results for scales of the test tapping into cognitive abilities would correlate with the results of neuropsychological tests and that results for scales not tapping into cognitive skills (such as purposeful nonadherence) would not.
Concurrent validity, a measure of whether the SMAT scores usefully predict patient behaviours, was evaluated by examining the relationship between SMAT scores and pill counts, refill history, and use of adherence aids or reminders before study enrolment. The post-SMAT recommendations of the pharmacist were also compared with behaviours such as use of adherence aids and other assistance before study enrolment. For this analysis, individual split-half groups were created for each of the SMAT scales. Then, for each adherence aid used before admission (i.e., dosette, medication calendar, blister pack, routine, reminders from others), a χ2 test was used to determine if a relationship existed between use of the aid and the score on the SMAT scale. Pearson correlations were also used to establish the relationships between scores on the SMAT scales and measures of validity.
Patients’ satisfaction with the assessment process was evaluated by comparing group means to the middle point on the 7-point Likert scale using 1-sample t tests.
All analyses were carried out with SPSS (version 16.0) software (IBM Corporation, Somers, New York), and a significance level of p < 0.05 was applied.
The study sample of 121 patients is described in Table 1. According to the 70–89 year norms for the clock-drawing test, 40 patients (33.1%) fell within normal limits, 34 (28.1%) had mild deficits, 24 (19.8%) had moderate deficits, and 15 (12.4%) had severe deficits; data were missing for 8 (6.6%) of the patients. When normative values for the MMSE were applied, 80 patients (66.1%) had MMSE in the range of 26–30 (normal), 36 patients (29.8%) had MMSE in the range of 20–25 (mild cognitive impairment), and 3 patients (2.5%) had MMSE in the range of 10–19 (moderately impaired). An MMSE value could not be obtained for 2 (1.7%) of the patients.
Table 1.
Demographic Characteristics, Cognitive Status, and Medication Management Indicators of 121 Participants
One-way ANOVA showed that the 2 subsamples (inter-rater reliability group, n = 25; test–retest reliability group, n = 20) did not differ from each other or from the rest of the sample in terms of age (F 2,118 = 0.61, p = 0.54, η2 = 0.01), MMSE scores, (F 2,116 = 0.14, p = 0.87, η2 = 0.002), or the clock-drawing test (F 2,110 = 1.5, p = 0.24, η2= 0.03) (where η2 [eta-squared] is a measure of effect size and indicates the proportion of the variance explained by the independent variable). The groups were also similar in terms of their distribution by sex (χ2(2, n = 121) = 4.40, p = 0.11) and education (χ2(6, n = 110) = 6.17, p = 0.40).
An additional analysis was done to compare the SMAT scores, the MMSE, clock-drawing, and CCT scores, and the demographic characteristics of the 42 patients who underwent a post-assessment interview with the balance of the study cohort, who did not participate in this aspect of the study. No significant differences were found between the groups.
The entire sample of 121 patients was used to establish the internal consistency of the 5 scales of the SMAT, on the basis of Cronbach’s α, although data were missing for some items (as indicated by sample sizes provided here). The functional scale of the SMAT comprised 22 items (α = 0.81, n = 121). Twelve items were scored on a 3-point scale (able, difficulty, unable), and the 10 colour-recognition items were scored on a 2-point scale (able, unable). All 22 cognitive scale items were scored on a 3-point scale (able, difficulty, unable) (α = 0.92, n = 121). The recall score was calculated from responses to 4 descriptors of medications applied to each of a patient’s medications. For each descriptor for each medication, the patient was scored as either able or unable to recall (α = 0.95, n = 120). For each of the 4 questions in the self-reported adherence section, the patient was scored as either able or unable for each medication (α = 0.98, n = 119). The purposeful nonadherence score was calculated from responses to 3 different descriptors of medication response (α = 0.51, n = 117).
The inter-relationships among the scales are presented in Table 2. There was a strong positive correlation between the cognitive scale and each of the functional, recall, and self-reported adherence scales. There was also a strong correlation between the recall and self-reported adherence scales.
Table 2.
Inter-relationships among Scales of the Self-Medication Assessment Test
The subsample used for determining inter-rater reliability consisted of 25 patients. Correlations and RMSD values are reported in Table 3. For the functional scale, there was 93.2% agreement, and in all but one of the disagreements, the 2 raters disagreed by only 1 point on the 3-point scale. For the cognitive scale, there was 78.2% agreement, and in 8 of the disagreements, the 2 raters disagreed by 2 points on the 3-point scale. For the remainder of the disagreements, they disagreed by 1 point. For the recall scale, there was 87.4% agreement and for the self-reported adherence scale, there was 91.7% agreement. For the purposeful nonadherence scale, it was not possible to calculate an overall percent agreement because of variation in scoring (2 items on the scale used a 4-point score and the third used a 5-point score, with each patient being scored in relation to his or her own medication list). However, an overall percentage score was calculated for the purposeful nonadherence scale using each patient’s responses for each medication divided by the total number of medications. Correlations based on the overall purposeful nonadherence score assigned to each patient were used to calculate reliability.
Table 3.
Reliability of Scales within the Self-Medication Assessment Test (SMAT)
The subsample used for determining test–retest reliability consisted of 20 patients. Correlations and RMSD values are reported in Table 3. For the functional scale, there was 82.3% agreement between the initial test and the retest. Performance improved (i.e., higher scores were obtained on retesting) for 37.2% of the scores that changed. For the cognitive scale, there was 76.6% agreement, and performance improved for 63.0% of the scores that changed. For the recall scale, there was 88.8% agreement and for the self-reported adherence scale, 95.3% agreement. As described above, agreement rates could not be obtained for the purposeful nonadherence scale.
The correlations between the various SMAT scales and validity indices are provided in Table 4. There were significant positive correlations between the functional scale and the results of both the clock-drawing test and the CCT ( p < 0.01). There were also significant positive correlations between the cognitive and recall scales and the MMSE, the clock-drawing test, and the CCT. A significant negative correlation was found between increasing age and performance on the functional and cognitive scales.
Table 4.
Correlations between Scales of the Self-Medication Assessment Test (SMAT) and Validity Indices
No patterns of relationship were statistically significant in comparisons of the cognitive scale and the purposeful nonadherence scale with patient behaviours related to adherence. Patients who scored lower on the recall scale were more likely to have used a blister pack (χ2 (1, n = 119) = 4.8, p = 0.03) and were more likely to have been reminded by others to take their medication (χ2 (1, n = 120) = 4.3, p = 0.04). Patients who scored lower on the functional scale (χ2 (1, n = 121) = 4.8, p = 0.03) and on the self-reported adherence scale (χ2 (1, n = 119) = 4.8, p = 0.03) were also more likely to have received reminders.
Pill counts were obtained on admission to the study for the 55 patients whose medication supplies from home were available and for whom pill counts were feasible. Prescription refill rates were available for 106 of the study participants. None of the correlations between these adherence measures and SMAT scores were statistically significant. There were also no correlations between the adherence measures and the neuropsychological measures (MMSE, clock-drawing test, CCT) or the Medication Complexity Index.
The pharmacists’ recommendations were compared to the amount of supervision that the patients reported having before hospital admission. Specifically, patients were asked if anyone routinely reminded them to take their medications. Of the 53 patients rated as needing no supervision, 1 (2%) had been receiving reminders at home. Of the 16 patients rated as needing minimal supervision, 3 (19%) had been receiving reminders. Of the 32 patients judged to need moderate supervision, 4 (12%) were already receiving reminders. Finally, for the 20 patients judged to need full supervision, 8 (40%) were already receiving reminders.
During post-assessment interviews conducted with 42 participants across the 3 subsamples, a 7-point Likert scale was used to code responses, and mean responses (± standard deviation) were compared to the midpoint on the scale by means of 1-sample t tests. Patients disagreed that the testing took too long (1.8 ± 1.7, t 41 = −8.0, p < 0.001). They felt that the instructions were clear (6.26 ± 1.5, t 41 = 9.5, p < 0.001) and that the test was useful for identifying areas of difficulty in managing their medications (5.8 ± 1.81, t 41 = 6.58, p < 0.001). Finally, patients reported experiencing low rates of distress as a result of the SMAT assessment (1.69 ± 1.45, t 41 = −10.28, p < 0.001).
The goal of this study was to test a new instrument for assessing elderly patients’ ability to manage their medications. An important aspect of developing a new test is showing that different users of the test will be able to use the test effectively and similarly (i.e., inter-rater reliability) and that scores will be relatively stable over time (i.e., test–retest reliability). Both forms of reliability were quite strong for the SMAT. Moreover, the internal validity of the instrument was also high, even though the sample was relatively small. The purposeful nonadherence scale had a relatively low consistency score (Cronbach’s α = 0.51), but this scale has only 3 items, and it is well established that it is difficult to obtain high levels of consistency with so few test items. However, we concluded that this scale did not show internal consistency and that each of the 3 items should be considered independently during interpretation of SMAT results.
The validity of an instrument indicates whether it is accurately measuring what it is meant to measure. In terms of convergent validity, we were able to show that the scales with a stronger cognitive aspect related well to neuropsychological measures that address cognitive function. The potential utility of the SMAT to predict patients’ outcomes was partly demonstrated when the medication-related behaviours that patients used at home were compared with their SMAT scores. Even more compelling was the relationship between lower scores on the recall, functional, and self-reported adherence scales and reminders from others. This suggests that persons close to the patient had recognized the adherence deficits identified by the SMAT.
We were not able to establish that a relationship existed between scores on the SMAT scales and the adherence measures chosen to support concurrent validity of the instrument (pill counts and pharmacy refill rates). These adherence measures reflected performance in the months preceding admission to hospital, whereas the SMAT testing was carried out after the medical event leading to the admission. As such, patients’ status at the time of testing might have been quite different from their status before entry into hospital.
Another issue is the use of pill counts and refill rates as indicators of adherence. In a recent Canadian study that used pill counts and home visits to determine adherence in a community population, pill counts were either not feasible or were considered inaccurate for 34.7% of the participants and 47.5% of eligible drugs.26 Factors affecting the accuracy or feasibility of these pill counts included information missing from labels, variability between dispensing and start dates, and the combining of old and new supplies. These issues were also present in our study, and they help to explain the limited number of patients for whom pill counts were feasible. Additional factors affecting the feasibility of pill counts were the hospital’s policy to use patients’ home medication supplies, prepackaging in blister packages by community pharmacists, and supplies left at home in other compliance aids such as weekly pill organizers.
In exploratory analyses, we investigated the relationships between these presumed measures of adherence and factors found to affect adherence in past research.32 Surprisingly, there was no relationship between a patient’s adherence scores as measured by pill counts and refill rates ( r < 0.01). Similarly, patients’ medication complexity (as indicated by the Medication Complexity Index) did not relate to either pill counts ( r = −0.16, p = 24) or refill rates ( r = −0.02, p = 0.87), despite the relatively high scores for medication complexity in this sample. Thus, the lack of expected relationships between these variables suggests that the measures of adherence obtained here may be inaccurate.
The prescription refill rates from community pharmacies were also problematic. The refill rates used in this study were obtained from a heterogeneous group of community pharmacies that used inconsistent report designs. Although refill rates have been used to successfully predict adherence, this success occurred primarily in studies that used central databases (such as those for prescription claims payment), which provide greater standardization of data. Ultimately, the acquisition of medications as identified by refill records does not reflect patients’ consumption of medication. Other concepts such as medication possession ratio and persistence (average number of days per month on therapy) may be superior indicators of adherence.33,34 Prospective measures of clinical outcomes associated with good medication adherence (e.g., blood pressure control or lipid reduction)35 may better establish predictive validity of the SMAT. Overall, the present data do not permit the conclusion that the SMAT yields strong concurrent validity. Future research is needed to address this issue.
A relatively small sample was used for measuring the psychometric properties of the instrument, such that norms for the scale scores could not be established. The study location is a tertiary care community hospital (with about 16 000 admissions per year) that serves a catchment area of 122 000, with both inpatient and outpatient services. The nature of the study design (taped interviews and assessment by occupational therapists and pharmacists) sometimes presented challenges for obtaining consent from participants. However, the hospital-based sample arguably reflects the reality of the elderly population better than would a volunteer community sample. The patients in our sample were relatively older, had more underlying illnesses, and varied widely in terms of cognitive status. These factors also made it difficult to obtain a large sample in a timely manner.
It was encouraging to learn that the acceptability of the SMAT to patients was high. Despite relatively thorough testing of multiple dimensions of medication self-management, patients generally felt that the time taken for testing was acceptable. More-over, they found the test useful for identifying problems. These findings, along with previous reports that the SMAT has high face validity among pharmacists,23 support future clinical use of the tool.
The SMAT builds upon and extends well-established findings of elements that relate to medication management. The use of videotaped interviews for measuring inter-rater agreement, along with a test–retest method for measuring the stability of the scale, provided strong measures of reliability. Validity was demonstrated by the significant correlations obtained with relevant neuropsychological and clinical indicators. The pharmacists participating in the testing noted that use of the instrument was particularly beneficial for establishing patient-specific targeted interventions.
Further research should use a prospective approach for establishing the relationship between adherence levels and scores on the SMAT scale. The usefulness of the SMAT in community pharmacies was not addressed but would also be of interest for future research.
The SMAT is a practical, reliable, and comprehensive instrument with demonstrated acceptability to patients. The strong internal consistency of the multiple scales indicates that the SMAT measures the multiple dimensions that are known to influence medication self-management ability in elderly patients.
1.
Maddigan SL, Farris KB, Keating N, Wiens CA, Johnson JA. Predictors of older adults’ capacity for medication management in a self-medication program: a retrospective chart review.
J Aging Health
2003;15(2):332–352.
2.
O’Brien MK, Petrie K, Raeburn J. Adherence to medication regimens: updating a complex medical issue.
Med Care Rev
1992;49(4):435–454.
3.
Beers MH. Explicit criteria for determining potentially inappropriate medication use by the elderly. An update.
Arch Intern Med
1997;157(14): 1531–1536.
4.
Hanlon JT, Shimp LA, Semla TP. Recent advances in geriatrics: drug-related problems in the elderly.
Ann Pharmacother
2000;34(3):360–365.
5.
Hanlon JT, Lindblad CI, Hajjar ER, McCarthy TC. Update on drug-related problems in the elderly.
Am J Geriatr Pharmacother
2003;1(1):38–43.
6.
Aparasu RR, Mort JR. Inappropriate prescribing for the elderly: Beers criteria-based review.
Ann Pharmacother
2000;34(3):338–346.
7. Edelberg HK, Shallenberger E, Wei JY. Medication management capacity in highly functioning community-living older adults: detection of early deficits. J Am Geriatr Soc 1999;47(5):592–596.
8. Edelberg HK, Shallenberger E, Hausdorff JM, Wei JY. One-year follow-up of medication management capacity in highly functioning older adults. J Gerontol A Biol Sci Med Sci 2000;55(10):M550–M553.
9. Gould ON. Cognition and affect in medication adherence. In: Park DC, Morrell RW, Shifren K, editors. Processing of medical information in aging patients. Mahwah (NJ): Lawrence Erlbaum Associates; 1999. p. 167–181.
10. Rudd, P. The measurement of compliance: medication taking. In: Krasnegor NA, Epstein L, Johnson S, Yaffe SJ, editors. Developmental aspects of health compliance behavior . Hillsdale (NJ): Lawrence Erlbaum Associates; 1993. p. 185–213.
11. Meyer ME, Schuna AA. Assessment of geriatric patients’ functional ability to take medication. DICP 1989;23(2):171–174.
12.
Levy HB. Self-administered medication-risk questionnaire in an elderly population.
Ann Pharmacother
2003;37(7–8):982–987.
13. Schepers GP, Won HK, Bieliauskas LA, Galecki AT, Hogikyan RV. A long-term-care setting pilot study evaluating predictors of success in medication self-administration. J Am Med Dir Assoc 2000;1(3):103–108.
14. Pereles L, Romonko L, Murzyn T, Hogan D, Silvius J, Stokes E, et al. Evaluation of a self-medication program. J Am Geriatr Soc 1996;44(2): 161–165.
15. Palmer HM, Dobson KS. Self-medication and memory in an elderly Canadian sample. Gerontologist 1994;34(5):658–664.
16. Ruscin JM, Semla TP. Assessment of medication management skills in older outpatients. Ann Pharmacother 1996;30(10):1083–1088.
17.
Gray SL, Mahoney JE, Blough DK. Medication adherence in elderly patients receiving home health services following hospital discharge.
Ann Pharmacother
2001;35(5):539–545.
18. Orwig D, Brandt N, Gruber-Baldini AL. Medication management assessment for older adults in the community. Gerontologist 2006;46(5):661–668.
19.
Svarstad BL, Chewning BA, Sleath BL, Claesson C. The brief medication questionnaire: a tool for screening patient adherence and barriers to adherence.
Patient Educ Couns
1999;37(2):113–124.
20.
Hutchison LC, Jones SK, West DS, Wei JY. Assessment of medication management by community-living elderly persons with two standardized assessment tools: a cross-sectional study.
Am J Geriatr Pharmacother
2006;4(2):144–153.
21.
Farris KB, Phillips BB. Instruments assessing capacity to manage medications.
Ann Pharmacother
2008;42(7):1026–1036.
22.
Elliott RA, Marriott JL. Standardised assessment of patients’ capacity to manage medications: a systematic review of published instruments.
BMC Geriatr
2009;9:27–37. 
23. Irvine-Meek J, Gould ON, Wheaton H, Todd LE. Acceptability and face validity of a geriatric Self-Medication Assessment Tool. Can J Hosp Pharm 2010;63(3):225–232.
24.
Johanson GA, Brooks GP. Initial scale development: sample size for pilot studies.
Educ Psychol Meas
2010;70:394–400.
25. VanVoorhis CRW, Morgan BL. Understanding power and rules of thumb for determining sample sizes. Tutor Quant Methods Psychol 2007;3(2): 43–50.
26. Vik SA, Maxwell CJ, Hogan DB, Patten SB, Johnson JA, Romonko-Slack L. Assessing medication adherence among older persons in community settings. Can J Clin Pharmacol 2005;12(1):e152–e164.
27.
Rijcken CAW, Tobi H, Vergouwen ACM, de Jongvan den Berg LT. Refill rate of antipsychotic drugs: an easy and inexpensive method to monitor patients’ compliance by using computerised pharmacy data.
Pharmacoepidemiol Drug Saf
2004;13(6):365–370.
28.
George J, Phun YT, Bailey MJ, Kong DCM, Steward K. Development and validation of the medication regimen complexity index.
Ann Pharmacother
2004;38(9):1369–1376.
29. Molloy DW, Alemayehu E, Roberts R. Reliability of a Standardized Mini-Mental State Examination compared with the traditional Mini-Mental State Examination. Am J Psychiatry 1991;148(1):102–105.
30.
Shulman KI. Clock-drawing: Is it the ideal cognitive screening test?
Int J Geriatr Psychiatry
2000;15(6):548–561.
31. Aiken LR. Psychological testing and assessment. 8th ed. Boston (MA): Allyn & Bacon; 1994.
32.
MacLaughlin EJ, Raehl C, Treadway A, Sterling TL, Zoller DP, Bond CA. Assessing medication adherence in the elderly: Which tools to use in clinical practice?
Drugs Aging
2005;22(3):231–255.
33.
Steiner JF, Prochazka AV. The assessment of refill compliance using pharmacy records: methods, validity and applications.
J Clin Epidemiol
1997;50(1):105–116.
34. Sikka R, Xia F, Aubert RE. Estimating medication persistency using administrative claims data. Am J Manag Care 2005;11(7):449–457.
35.
Briesacher BA, Andrade SE, Fouayzi H, Chan KA. Comparison of drug adherence rates among patients with seven different medical conditions.
Pharmacotherapy
2008;28(4):437–443.
The authors would like to thank all of the study participants, as well as the following individuals for their help in data collection: Erin Clarke, BSc(Pharm); Lisa Duplessis, Pharmacy Technician, BN; Michelle Lemoine, BSc(OT); Chantal Michaud, BPharm, ACPR, BCPS, CDE; Gisia Pisegna, BSc(Chemistry), MSc(Chemistry), BSc(Pharm), ACPR; Allison Post, Pharmacy Technician; Jane Soucie, BSc(OT); Heather Swan, BSc(OT); Melissa White, Pharmacy Technician.
The authors would also like to thank the following funding agencies for their financial support: Medbuy Corporation: Research, Education and Development Funds; Canadian Society of Hospital Pharmacists Research and Education Foundation; the Dalhousie Pharmacy Endowment Fund; and the Friends of The Moncton Hospital.
Canadian Journal of Hospital Pharmacy , VOLUME 64 , NUMBER 1 , 2011