METHODSEthics StatementMACH 14 is a multisite collaboration. Each of the 16 studies that make up this collaboration was approved by that institution's Institutional Review Board.
Studies and patient selectionMACH 14 included both observational (n=4) and intervention (n=12) studies. The process is described in detail elsewhere.18-21 To be included in this analysis, monitoring had to be continuous and data on whether patients were ART naïve had to be available. This excluded four studies that assessed adherence only in the weeks prior to a study visit and one study without ART naïve information, leaving 11 eligible studies. We use the term “study” and not “site” to avoid confusion between the sites where the study took place and the sites where patients received HIV clinical care, which was not necessarily the same for the studies that recruited patients from multiple care sites.
Because we were interested in the natural history of changes in ART adherence over time, we only included control patients in adherence intervention trials. Of the 1456 patients in the 11 eligible studies, 916 were either in observational studies or in control arms of intervention studies, and formed the analytic sample. The number of patients, intervention status, and length of follow up for each of the 11 included studies are shown in Table 1. Patients were followed for up to 12 months.
VariablesWe defined the dependent variable as ART adherence. For each patient, the time of observation was divided into one month periods. Adherence was operationalized as the number of observed openings divided by the number of prescribed doses. Typically one antiretroviral was monitored using EDM, but when more than one was monitored, we calculated an average from all monitored medications over the month. Adherence was summarized by month across the whole study population, and also by study.
The major independent variable was time measured in months. Up to 12 months of observations were assessed for each study. Studies varied in the amount of time observed. The shortest study followed patients for three months, one study followed patients for four months, two studies followed patients for six months, two studies followed patients for nine months, and six studies followed patients for 12 months.
Covariates included age in years, gender, education (high school graduate or less vs. more than high school education), race (White, African-American, Hispanic, or other), antiretroviral regimen (classified as NNRTI-based, PI-based, boosted PI-based, or other), history of substance abuse (yes/no), and whether the patient was ART naïve (yes/no).
AnalysesWe plotted adherence by month, and then adherence by month for each individual study in order to visualize between study differences. Because these graphs suggested that there were considerable between study differences, and also that the relationship might be nonlinear, next we modeled the relationship between adherence and time using a cubic spline model with 12 knots. To estimate the possible underlying non-linear relationships, we used the generalized additive mixed model (GAMM) that is an extension of generalized linear mixed models (GLMMs). GAMM relaxes the assumptions of normality and linearity inherent in linear regression and allows the parametric fixed effects to be modeled non-parametrically using additive smooth functions. The flexibility of non-parametric regression for the continuous predictors coupled with linear models for predictors provides a way to uncover structure within the data that may be missed using linear assumptions. GAMM modeling was carried out using the R Statistical Language and the MGCV Package.22 The model allows the assessment of the non-linearity of trend in the adherence curve, which is shown by the effective degrees of freedom (EDF) term from the model. An EDF of 1.0 denotes linearity, and values greater than 1.0 indicate non-linearity. The higher the edf, the more non-linear is the smoothing spline. The model also allows for assessment of the directional trend of the curve, that is, whether it is downward, flat, or upward trending.
We used a manual forward selection approach to select covariates, starting with a model that included study and a study by time interaction, and then sequentially testing the following variables: race, calendar year, regimen, ART naïve, and ART naïve by regimen. In each case, we used the likelihood ratio test to determine whether the added variable contributed significantly (p<0.05) to the model. For model diagnostics, we assessed 1) normality (the QQ-plot and the histogram of residuals), 2) homogeneity (residuals versus predictor plot, and residuals versus fitted values plot, also called the linear predictor plot for the Gaussian distribution with identity link), and 3) model fitting (fitted values versus observed values plot).
Formal modeling of missing data in longitudinal studies is complex.23 We were interested in determining whether those who dropped out of the 11 studies in our analysis were different from those who did not drop out, or, more formally, whether they were missing not at random.24 To test this, we estimated a Cox proportional hazards model with dropout as the dependent variable. Note that “dropout” here refers to study dropout, not treatment dropout (i.e., ARV non-persistence). The principal independent variable in the model was medication adherence. For those who did not drop out we used adherence in the last study month; for those who did drop out we used adherence in the month prior to dropout. Covariates included the same variables that were used in the models in which adherence was the dependent variable. This approach allows us to determine whether adherence prior to dropout predicted drop out.
RESULTSPatient characteristicsOf the 916 patients studied, mean age was 41 years, and 35% were female (Table 2). Twenty-seven percent were White, 46% African-American, 21% were Hispanic, and 6% were classified as other. Eight-eight percent had a high school education or less. Seventy-seven percent had a history of substance abuse. In approximately 27% an NNRTI was the monitored antiretroviral, in 17% it was a boosted PI, in 28% it was a PI, and in 28% it was an antiretroviral classified as other.
Changes in adherence over timeUnadjusted changes in adherence by month are shown in Figures 1 (all 11 studies aggregated) and 2 (each study individually). The base GAMM model that included only the spline terms (12 knots) is shown in Figure 3. The EDF of the smoothed term for month in this model was 4.63, signifying high non-linearity. The model shows that the curve is down sloping (F=7.3, p<0.0001). The R-square of this base model was <0.01.
Variables that were statistically significant in the final, multivariable GAMM model included study, the study by time interaction, race, and calendar year. The R-squared of this final model was 0.14. These results are shown graphically in Figure 4, where the fitted adherence values for each study are plotted against time. The scale of both axes varies for each plot to allow appreciation of the study-by-study variation. For 3 of the studies (7,10, and 12), the EDF was 1.0, denoting a linear relationship between adherence and time. For 2 of these studies (7 and 12), there was also a significant downward trend (p<0.0001), but for study 10, adherence did not decline (p>0.05). The relationship of adherence to time was non-linear for the remaining studies. For 6 of these (studies 1, 2, 8, 11, 13, and 15), there was a significant downward trend (p<0.05), and for 2 of these studies (5, and 6), there was no significant change in adherence over time (p>0.05). Model diagnostics, including normality, heterogeneity, and fit were acceptable. In the final model, African Americans had significantly worse adherence compared with Whites (7.9 points, p=0.0013). Calendar year was also significantly associated with adherence. Compared with year 1998, adherence was significantly better for years 1999 through 2006 and 2008, but was not significantly different for 2007.
We also tested a model that included an indicator variable for whether the study was a randomized trial or an observational study. The p-value for the indicator was 0.72.
Bias and missingness analysisIn a Cox proportional hazards model predicting study dropout that controlled for antiretroviral regimen and study, adherence was not associated with dropout (Hazard Ratio 1.0, 95% CI 0.74-1.33, p=0.97).
DISCUSSIONThere were three main findings from this research. First, across the 11 studies we studied, adherence declined with time, but this overall effect hides the fact that there was heterogeneity among studies in this decline. Second, the relationship between adherence and time was non-linear both overall and in the majority of individual studies. Third, covariate adjustment did not eliminate this heterogeneity.
Many of the previously published studies that address changes in adherence over time included patients from multiple clinical care sites.3,5,7,9,10,13,14,17 Only two of these studies included site as a variable in analyses. Maqutu et al.13 studied two sites in South Africa, and found that the rural site had adherence levels that were approximately 20 percentage points lower at study initiation than the urban site. They also found that the rate of increase in adherence in the rural site was significantly higher than that seen in the urban site (adjusted odds ratio for the rate of change was 1.06, p<0.004), such that both sites had approximately 90% adherence after 18 months of follow-up. Muyingo et al.14 studied four sites in Uganda and Zimbabwe. While they did not present data on changes over time by site, two of the sites had significantly higher (p<0.001) baseline adherence than the reference site.
Our data in combination with these other two studies that examined adherence levels by site suggest that aggregate data may often hide important study-level variation. This is not surprising. The 11 studies that we analyzed included both observational and intervention studies, and we have only limited data on the nature of adherence support for patients in these studies. While aggregated data is important for policy discussions, quality improvement requires care site specific data. That is, because historical factors, available resources, structural aspects of the delivery system, patient characteristics, and even provider incentives can differ widely from site to site, analyses of correlates or predictors of adherence and changes in adherence over time may yield site specific answers. We cannot know from these data why adherence in some studies declined with time and other studies did not. Answering this important question would require a study that collected a more extensive list of potential explanatory variables and that identified and characterized the care sites from which patients were recruited.
Overall, the relationship between adherence and time was non-linear, and it was non-linear for most of the studies. Existing studies use a variety of approaches to understanding changes in adherence with time, including descriptive measures,5-8,11,12 and more complex multivariable methods, often using generalized estimating equations (GEE) to account for repeated measures,1-3,9,10,13-17 and survival analysis.4 GEE's work on the population level, but their estimates of associations in smaller datasets are inefficient and sometimes inconsistent. GLMMs can model the adherence change on an individual level, but GAMMs allow flexible and non-linear specification of the dependence of the response variable on a set of temporal and/or spatial covariates without having to specify the model in terms of detailed parametric relationships, which made them more useful for the purposes of this analysis.
Another advantage of using models that capture nonlinearity is that we can address other hypotheses about how adherence as measured by EDM changes over time. For example, another possible interpretation for the decline in adherence that we saw is that in studies that use EDM there may be a Hawthorne effect. That is, participants who start using EDM have higher adherence than they would otherwise have by virtue of the fact that they know that their adherence is being measured or observed. Some studies support the existence of such an effect,25-28 but others do not.29 A pattern of adherence in which there was an increase in adherence followed by a decline (e.g., studies 6 and 8, Fig 4), or steeper declines in adherence in the first several months of treatment than what was seen later (studies 5, 13, and 15, Fig 4) would support the existence of Hawthorne effects. Thus our data provide only limited support for the assertion that Hawthorne effects are necessarily seen when EDM is used.
We were not surprised that adjustment for race and other covariates did not eliminate differences between sites. The finding that adherence was lower in African-Americans than Whites has been demonstrated in multiple other reports,30-35 including one from the MACH14 cohort.21
There is variability in how prior researchers have dealt with loss to follow up. Some studies either do not mention loss to follow up or note that those lost to follow-up were excluded from analyses.1-4,7,9,10,12,14,15,17 Others presented descriptive information about rates of loss to follow-up or missing data, or in some way compare the full sample of those with missing data with those who had complete data or no loss to follow-up.5,6,8,11,16 Only one study explicitly analyzed loss to follow up.13 In our study, and also in Maqutu et al.,13 analyses suggested that missing adherence values were missing completely at random, and therefore do not bias analyses, but this is a strong assumption, and we recommend that longitudinal studies of adherence formally examine the potential impact of loss to follow-up on their findings.
There were several study limitations. Regarding internal validity, we could not adjust for time on antiretroviral therapy prior to study entry, which, if it varied systematically between studies, could explain some of the observed variation. We also could not adjust for potentially important patient-level covariates including depression, income, social support, and patient provider relationship quality because these variables were not measured in all studies. Regarding generalizability, we studied a predominately treatment-experienced cohort cared for in the United States, and included only patients who agreed to use EDM. Thus our findings may not be generalizable to other populations or care settings. The studies we included were conducted over a time period during which there were important changes in ART regimens, particularly the addition of boosted-PIs and NNRTI's. However, our analyses controlled for calendar year, and regimen type was not associated with changes in adherence over time in our models, so we do not believe that this is an important limitation. Finally, the studies we analyzed were potentially different on multiple dimensions which we did not analyze, including the focus of the study (e.g., adherence in patients receiving methadone therapy), entry criteria (e.g., if only those with detectable viral loads were eligible to participate in an intervention trial), the nature of adherence support in study control arms, and the number of clinical care sites from which they recruited patients.
In conclusion, we found that, overall, adherence declined with time, but this was not consistent across studies. Studies that identify clinical and organizational factors associated with these different patterns are needed. Because this variability was not explained by the covariates we assessed, studies that look more carefully at how specific care processes may vary in the populations recruited by individual studies (e.g., availability of mental health services) are warranted. In conducting longitudinal analyses of adherence, our analysis suggests that investigators should take account of site and/or study effects and consider the use of non-linear models.