Our DSMES intervention used a pretest–posttest design and was implemented in rural Guatemala from November 2018 through December 2020. The study was conducted by Maya Health Alliance, the Inclusive Health Institute, and the Institute of Nutrition of Central America and Panama. This was a pragmatic study that focused on evaluating DSMES in real-world routine conditions; as a result, we did not perform a sample size calculation.

The study was conducted in 8 rural municipalities in a single province (Chimaltenango) in the Central Highlands region. We chose this province because it is where Maya Health Alliance’s main office is located. The population is predominantly Indigenous Maya (17), and most live below the national poverty line (18). Each municipality has a public health district operated by the Ministry of Health, as well as private and nongovernmental biomedical clinics and nonbiomedical traditional healers. In the public health sector, diabetes care is delivered at a physician-staffed health center. Free services at health centers include blood glucose monitoring and oral glucose-lowering drugs. Patients requiring laboratory assessments, insulin therapy, or specialist management are referred to regional referral hospitals. Delivery of DSMES is limited in this system (8).

We used broad participant inclusion criteria: 1) being aged 18 years or older and 2) having a glycated hemoglobin (HbA_1c) ≥6.5% or diagnosis of diabetes within the preceding 12 months. We excluded individuals who were pregnant or had type 1 diabetes.

Any health facility in included municipalities was eligible to refer patients. In each of the 8 municipalities, we approached all public health centers and selected public hospitals, private clinics, nongovernmental clinics, and pharmacies. At public health centers, study staff also actively recruited patients from diabetes peer-group meetings. Other recruitment activities included approaching known patients from Maya Health Alliance, word-of-mouth from enrolled participants, door-to-door visits, and public fliers.

The intervention was based on our previous pilot and delivered by community health workers at the participants’ homes (9). The intervention was a public–private partnership whereby community health workers paid by Maya Health Alliance worked within the public health system. The curriculum was adapted for low-literacy Mayan-speaking populations and based on a Guatemalan version of the US National Heart, Lung, and Blood Institute’s Salud Para Su Corazón (Health for Your Heart) model for Latinx populations (19,20). Prior adaptions by our group to this curriculum included diabetes-specific content, home visits with family participation, minimal written text, and culturally relevant drawings, props, and games (9). The intervention consists of a screening visit, 6 monthly education visits, and a closing visit (Box). The curriculum focuses on the “4 pillars” of diabetes control: 1) regular medical appointments, 2) adherence to medications prescribed by health care providers, 3) regular physical activity, and 4) a healthy diet that reduces intake of carbohydrates. At each visit, study educators review achievements from prior visits, assess individualized milestones, and use motivational interviewing to guide participants on overcoming barriers to behavior change. We expected that the total intervention time per participant would be 8 months with each monthly visit lasting 1 hour. We did not provide any incentives to increase participation or adherence.

Impact of COVID-19. We halted study enrollment in March 2020 when community-based transmission of COVID-19 was reported. Because of safety concerns and mandated travel restrictions, the study transitioned to telephone visits. Participants who had not finished before March 15, 2020, had end-point psychometric data but no end-point clinical data collected.

We conducted an explanatory sequential mixed-methods evaluation (21) guided by the RE-AIM (Reach, Effectiveness, Adoption, Implementation, Maintenance) Qualitative Evaluation for Systematic Translation (RE-AIM QuEST) mixed-methods framework (13). The project evaluation plan and data sources are available elsewhere (Appendixes 3–4 [16]).

Quantitative data were entered in real time using smartphones and data capture software (REDCap). Quantitative data included participant sociodemographic characteristics, clinical and psychometric outcomes, implementation metrics, and costs. All quantitative data were collected in participants’ homes by 1 trained research assistant. Qualitative data consisted of semistructured interviews with study participants (n = 12), intervention staff (n = 5), and staff members at health centers (n = 6). All intervention staff members and staff members from 6 of 8 health centers were interviewed; 2 health center staff members did not respond. We purposefully sampled interview participants with low and high effect size (change in HbA_1c) and engagement (average visit time), including at least 1 participant of each sex in each group. The interview guides were designed to explore elements of the quantitative analysis, following dimensions of the RE-AIM framework tailored to each group (13). Example probes included perceptions of project utility, determinants of clinical benefit, impact of COVID-19 and transition to telephone visits, role of sex and other determinants of participation, and family involvement. The full interview guide is available elsewhere (Appendix 5 [16]). Interviews in Spanish lasted approximately 30 minutes. Interviews in Maya Kaqchikel used an interpreter and lasted approximately 1 hour. Interviews were recorded and then translated and transcribed in Spanish. All qualitative data were collected by 1 study author (A.A.), a trained anthropologist, and were conducted via telephone because of COVID-19.

Reach. Reach references the absolute number, proportion, and representativeness of study participants. Quantitatively, we examined lost-to-follow-up and compared the characteristics of study participants with the characteristics of participants with diabetes in a contemporaneous population-representative chronic disease survey conducted by the authors in 2018 and 2019 in one of the study’s municipalities (22). Qualitatively, we focused on barriers to enrollment, especially for men.

Effectiveness. Effectiveness references the effect of the intervention on study outcomes. Primary clinical outcomes were HbA_1c, systolic blood pressure, diastolic blood pressure, and body mass index (BMI). HbA_1c was assessed with a point-of-care device (A1CNow, PTS Diagnostics). Seated arterial blood pressure was assessed in triplicate after 15 minutes with an Omron 7 digital cuff and estimated as the mean of 3 measurements. Secondary psychometric outcomes were diabetes knowledge, diabetes distress, and self-management. Diabetes knowledge was measured by the Diabetes Knowledge Questionnaire (DKQ-24); scores range from 0 to 24, with higher scores indicating more knowledge (23). Diabetes distress was measured by the Diabetes Distress Scale (DDS); scores range from 1 to 6, with higher scores indicating more distress (24). Self-management was measured by selected questions from the Summary of Diabetes Self-Care Activities instrument (SDSCA) (25). We previously validated the DKQ-24 and SDSCA during our pilot (9). Qualitatively, we investigated the mechanisms influencing effectiveness and potential explanations of differences between participants.

Adoption. Adoption references the absolute number, proportion, and representativeness of staff, providers, and organizations. We calculated the proportion of participants who were enrolled by type of health facility (public facilities, private clinics, nongovernmental clinics, pharmacies). We also examined the proportion of participating versus invited facilities. Qualitatively, our interviews explored factors that affected facility participation.

Implementation. Implementation references how accurately and consistently the intervention was carried out, including adaptations and cost. We examined total time between screening and closing visits, between first and last education visits, average visit duration, and costs. We also calculated the proportion of the suggested curriculum that was completed during each visit. Qualitatively, we explored barriers to fidelity, strategies to overcome barriers, and intervention modifications.

Maintenance. Maintenance references the extent that the intervention and intervention outcomes are sustained after the study. In interviews we explored factors leading to high or low levels of engagement and intent to continue.

Quantitative. We used Stata version 16 (StataCorp LLC) for analyses. We compared participant baseline characteristics with the population-representative sample using the Student t test for continuous data and the proportion test for categorical data. Baseline participant characteristics of retained study participants and participants lost to follow-up were also compared using the Student t test and proportion test. For clinical outcomes, we constructed multilevel mixed-effects models for HbA_1c, blood pressure, BMI, the DKQ-24, the DDS, and the SDSCA. Models were prespecified to include random effects for study participant and fixed effects for age, sex, ethnicity, education level, time since diagnosis, difficulty paying for medications, and baseline value.

To investigate the impact of missing data due to COVID-19 and other causes, we conducted a sensitivity analysis using multiple imputation with chained equations and 100 imputations (26). We conducted a second sensitivity analysis of the impact of conducting the intervention virtually during the COVID-19 pandemic on psychometric outcomes.

Qualitative. We analyzed interviews using Dedoose (Sociocultural Research Consultants). We conducted a thematic framework analysis using an inductive approach. We first developed a codebook by analyzing 2 interviews from each group. Responses were then coded by 2 authors (S.T. and D.F.), and grouped by RE-AIM dimensions. Differences were resolved through consensus. The final coded interviews added less than 5% new information, indicating thematic saturation (27).

Mixed methods. We integrated our findings using a joint display of quantitative and qualitative findings and meta-inferences (21). Meta-inferences are interpretations that emerge from the integrated analysis of the quantitative and qualitative data. They were generated iteratively by discussion among the team (27).

Quantitative. In the comparison of baseline characteristics of participants and the total diabetes population, important differences included the overrepresentation of women, greater preference for a Mayan language, lower levels of education, and higher baseline values for HbA_1c and blood pressure (Table 1). Compared with participants who completed the study, participants lost to follow-up were more likely to speak a Mayan language, but otherwise we found no significant baseline sociodemographic or clinical differences, including for self-identified Maya ethnicity. A full comparison of baseline characteristics between participants lost to follow-up and those retained and additional health service data characteristics are available elsewhere (Appendixes 6–7 [16]).

Qualitative. In the investigation into reasons for the low levels of enrollment among men, a common theme was that men often leave for work early in the morning and return late, after clinics close. In addition, “machismo” (an exaggerated sense of masculinity) negatively affected men’s self-care and their willingness to participate. As one educator reported,

I think it is because of the lack of time men have. The men are the ones who go out to work; they have to go out to the fields to work which doesn’t give them time . . . and I think they are closed in, they don’t like to get checkups often . . . they are embarrassed to say that they have some illness.

Quantitative. In adjusted multilevel models, mean change in HbA_1c was −0.4% (95% CI, −0.6% to −0.3%; P < .001), mean change in systolic blood pressure was −5.0 mm Hg (95% CI, −6.4 to −3.7 mm Hg; P < .001), mean change in diastolic blood pressure was −2.6 mm Hg (95% CI, −3.4 to −1.9 mm Hg; P < .001), and mean change in BMI was 0.5 (95% CI, 0.3 to 0.6; P < .001) (Table 2). Mean change in diabetes knowledge assessed using the DKQ-24 was 3.9 (95% CI, 3.6 to 4.1; P < .001) and mean change in diabetes distress using the DDS was −0.4 (95% CI, −0.4 to −0.3; P < .001). We also found significant improvements in most self-care activities.

The results from the sensitivity analysis investigating the impact of missing data through multiple imputation were similar to the results of the primary analysis; details are available elsewhere (Appendix 8 [16]). All missing data for this exercise were imputed, except for binary SDSCA outcomes because of multicollinearity. In the second sensitivity analysis, which assessed the impact of conducting the intervention virtually during the COVID-19 pandemic, improvements in knowledge, distress, and diet outcomes were similar before and during the pandemic; details are available elsewhere (Appendix 9 [16]). However, we did not observe improvements in physical activity outcomes during the pandemic.

Qualitative. The primary mechanism affecting effectiveness was the personalized nature of visits, which addressed participants’ specific needs while building trust. Themes mentioned included 1) the tailoring of educational content to each participant, 2) the supportiveness of educators, 3) the favorability of home-based visits, and 4) the patients’ ability to choose a preferred language. As 1 study educator explained,

If the patient preferred to speak in Kaqchikel, I would speak to them in Kaqchikel; if they wanted to speak in Spanish, then I would speak Spanish. I think it is very important that participants receive the education in their preferred language. This gives them more confidence . . . it is much better to have personalized education because the participant can express their doubts and not be embarrassed or worried about what their peers hear . . . in our program we go step by step, theme by theme, personalized to the participant.

Quantitative. Of the 612 participants, 386 (63.1%) were referred from health centers; 31.4% (192 of 612) from Maya Health Alliance programs; 4.2% (26 of 612) from private clinics; 1.0% (6 of 612) from the regional public hospital; and 0.3% (2 of 612) through door-to-door promotion.

All 44 health facilities approached agreed to participate. Of 10 public health facilities, 8 were health centers and 2 were hospitals. Seven of 8 health centers and 1 of 2 public hospitals referred patients. Of 24 private clinics, only 3 referred patients. None of the 8 pharmacies who agreed to participate referred participants.

Qualitative. Interviews highlighted partial adoption by participating health centers. Health center staff allowed recruitment of participants attending diabetes peer groups but were otherwise not active. One study educator acknowledged this lack of integration: “All we did was present the project to the directors to get approval. They gave us 10 to 15 minutes to present our project at the diabetes club meetings and that was it.”

One identified barrier that prevented adoption was the lack of a training program for health center staff. One study educator noted this could be improved: “In the future we could coordinate with the health centers to do trainings with the staff working in diabetes. We could train the staff and also the health center directors before the intervention so that they are more involved.”

Quantitative. The median time between baseline and end-point data collection was 268 (interquartile range, 225–343) days. The median time between first and last education visits was 155 (interquartile range, 144–182) days. During the intervention (pre– and post–COVID-19), 83.7% (525 of 627) of participants completed all 6 education visits. The mean (SD) duration of home visits was 70.9 (15.4) minutes. The mean (SD) duration of telephone visits was 41.4 (13.2) minutes.

Direct intervention costs were US$90.19 per participant (Appendix 10 [16]). In comparison, government health expenditure per capita is US$94.49, and total current health expenditure per capita is US$259.62, with 57.5% of costs being out of pocket (28).

The median of suggested curriculum elements that were completed for all visits was 94.3% (interquartile range, 91.8%–96.5%).

Qualitative. The main intervention modifications were caused by the COVID-19 pandemic. Implementing the intervention was more difficult after the transition to virtual visits. One positive aspect was the ability to schedule visits during nonworking hours. One study educator summarized:

One challenge was that cellular reception was very bad . . . I had to call 3 to 4 times to finish a study visit. Also, the phone numbers we had were often not the participants’. . . . When we called, the participant would not be home, and it was uncomfortable for the family member or neighbor. This gave us less time for the visit. Phone visits had fewer questions than home visits because we could not show them pictures, which helped generate a lot of questions. . . . We did cover all the topics, but the patients were a little more closed.

Qualitative. All interview participants desired to continue practicing what they had learned during the intervention. A common theme was that changes were difficult but became easier over time: “[T]he beginning was the most difficult because humans are used to eating what they want to. You have no diet, you eat everything. But later you start adapting to the diet and eventually you are used to it and it is easier.”

At the organizational level, all health center staff expressed support. The main barrier to continuing the intervention was lack of time: “I think that our availability, our time would be the biggest challenge. I don’t think that the intervention would be difficult for us to do, but the time we have is what would be difficult.”

We summarized the quantitative and qualitative findings and meta-inferences globally because we found no significant differences between the purposefully sampled groups (Table 3). First, DSMES interventions struggle to enroll men when they lack strategies that accommodate work schedules and address cultural barriers to self-care and education. Second, although all types of health facilities were eager to participate in identifying and referring participants, their day-to-day participation was limited. Better integration is critical in scaling up DSMES interventions in the public health system. Third, although the public health center system has interest in “personalized” DSMES, achieving sustainability requires addressing budgetary and time constraints. A key sustainability limitation of the public–private approach used here was that the public sector’s capacity to adequately support the cost of DSMES staff was unclear.