Kaiser Permanente Northern California is an integrated health care system serving more than 3.6 million people in California, including more than 260,000 members with diabetes who are automatically enrolled in the Kaiser Permanente Northern California diabetes registry.^[11]

Usual care for patients with diabetes and related chronic conditions in this setting includes a robust panel management approach that leverages performance feedback, system-wide efficiencies, disease registries and evidence-based practice.^[12] The electronic medical record at Kaiser Permanente Northern California is a critical component of this comprehensive disease management strategy.^[13,14]

Given the pragmatic nature of this trial, we engaged patients with diabetic peripheral neuropathy and clinical stakeholders in all aspects of the research from the identification of the research question to the design of the intervention and dissemination of research findings. Our stakeholder panel includes three patients, three primary care physicians (one of whom is an expert in electronic medical record physician prompts), one nurse case manager, two pharmacists, one health educator, and one endocrinologist. Our approach to stakeholder engagement is grounded in a framework developed by one of the authors (Schmittdiel) on system-based participatory research in health care.^[15] The four principles guiding our efforts are: 1) partnership and collaboration in all phases of research; 2) building upon existing resources and goals, 3) creating and investing in long term robust partnerships and 4) engaging in research as a cyclical and iterative process.^[15]

We randomized at the physician level to reduce the likelihood of contamination among patients seen by the same physician. One thousand eight hundred and thirty four Kaiser Permanente Northern California primary care physicians with adult patients in the Kaiser Permanente Northern California diabetes registry were randomized to the intervention or control group prior to patient eligibility determination and recruitment using a randomization procedure available in the SAS^® software package.^[16] Of those randomized, 1,714 physicians had patients who met the initial inclusion and exclusion criteria. In October 2014, we notified these physicians that their patients may be eligible for a new clinical trial, explained the trial, and invited them to opt out any patient due to illness or other factors that would make them ineligible for participation. Among diabetes patients not opted out by their physicians, we use the electronic health record to prospectively identify patients who start a new prescription (i.e., no use in the previous 12 months) for medications commonly used to treat diabetic peripheral neuropathy symptoms (amitriptyline, nortriptyline, imipramine, desipramine, duloxetine, paroxetine, citalopram, pregabalin, venlafaxine, or gabapentin) between September 2014-November 2015 and who have screened positive for DPN symptoms at least once during a routine visit occurring between April 2012 (when the screener was implemented system-wide) and the time they started medication.

Research associates send recruitment letters describing the study and potential risks and benefits of participation. The materials sent to patients in the treatment and control arms were identical. Patients are contacted by an interviewer by phone one week later to obtain phone consent and to conduct the baseline interview. To reduce the risk of selection bias, research associates and interviewers are blinded to treatment assignment from consent through outcomes assessment. Patients are blinded to their treatment status at the time of recruitment. The procedures followed are in accordance with the ethical standards of the responsible committee on human experimentation, the Kaiser Permanente Internal Review Board, which has reviewed and approved the study protocol. The study is registered at ClinicalTrials.gov (CE-1304-7250).

To be eligible for inclusion, identified patients have to be 18 years old at the time they start therapy, be continuously enrolled in Kaiser Permanente Northern California during the 12 months prior to starting study medications, and speak English or Spanish. We exclude patients who have a diagnosis of gestational diabetes, dementia or substance abuse disorder during the previous 12 months or who have any opioid use during the 90 days prior to starting diabetic peripheral neuropathy treatment. [Figure I]

As we identify potential participants, study telephone interviewers confirm that they have active diabetic peripheral neuropathy symptoms and that the prescribed study medications are for diabetic peripheral neuropathy and not some other condition. In addition, the interviewers identify other factors that may impede participation in this telephone based study such as undiagnosed cognitive deficits or use of rotary phones.

Patients of physicians allocated to the intervention arm receive all usual care for diabetes and diabetic peripheral neuropathy, plus three five-minute interactive voice response calls spaced two months apart over the six-month period following start of diabetic peripheral neuropathy treatment. These interactive calls systematically collect patient information on symptom relief, medication use, titration, discontinuation and side effects. Patient responses are then manually entered into the electronic health record by a trained research assistant. Responses indicating dissatisfaction with the level of symptom relief, side effects, and/or discontinuation of (or failing to start) the medication are forwarded directly to the physician for more rapid physician notification and response. Responses from patients who report no major problems with their diabetic peripheral neuropathy medication are entered into the health record but not forwarded to the physician since no immediate follow-up is required. (Table 1)

Patients of physicians allocated to the control arm receive all usual care for diabetes and diabetic peripheral neuropathy, plus three non-interactive, automated telephone calls spaced two months apart over the six months following start of a diabetic peripheral neuropathy treatment. The automated telephone calls consist of three separate diabetes-related educational messages about physical activity, dietary changes, and the importance of regular foot checks and last about two minutes or less. The goal of providing these non-interactive calls is to isolate the marginal impact of receiving an automated, diabetes-related phone call in the intervention arm. Intervention and control calls are offered in English and Spanish.

The primary outcome measure is quality of life as measured by the Global Health Scale, a 10-item PROMIS^® measure developed and validated in patients with neuropathy as part of the Quality of Life in Neurological Disorders Measures.^[16] The questionnaire includes questions about quality of life, mood, physical and social functioning and pain severity. Data are collected by trained researchers during 20-minute computer assisted telephone interviews at approximately 30 days before the first and 30 days after the final interactive voice response/control call. (Table 2)

To assess the mechanisms by which the intervention may impact quality of life, we are also capturing the following secondary outcome measures using instruments from the Quality of Life in Neurological Disorders Measures: Pain Interference, Sleep Disturbance, Depression, Ability to Participate, and Lower Extremity Functioning.^[17] In addition, we will evaluate patient-perceived changes in communication between patients and their doctors using the six-item Doctor Communication Composite from the Consumer Assessment of Health Plans Survey. This instrument is used routinely in this setting and refers to the patient’s most recent visit.^[18] Using the electronic health record, we will also assess changes in medication use and physician prescribing patterns.

We will evaluate the intention-to-treat effect of the interactive voice response intervention compared to the control intervention on the change in quality of life measurements collected at baseline and 30 days after the final interactive voice response/control call using generalized estimating equations to account for repeated (patient) measures within physician clusters .^[19,20] Double robust estimation will be implemented in secondary analyses to address any observed imbalance in characteristics between the control and intervention groups at baseline ^[21] and to address any evidence of differential rates of missing outcomes due to dropout between the two arms .^[22–24] Generalized estimating equations analyses will be conducted using the GENMOD procedure in SAS^® for fitting a linear model with repeated measurements. Doubly robust estimation will be conducted using the ltmle R package (available at: https://cran.r-project.org/web/packages/ltmle/index.html).

Our protocol includes testing for heterogeneity in study outcomes by patient desire for shared decision making. However, the high response rate will enable for additional secondary analyses of physician characteristics such as age, gender, and years at Kaiser Permanente Northern California, as well as patient’s perception of physician communication skills, as potential moderators of intervention effects.

Estimating the minimum detectable effect was challenging in this prospective pragmatic trial because we could not know in advance how many patients would become eligible during the enrollment period. Based on our prior studies, we estimated that 118 patients would be newly started on diabetic peripheral neuropathy treatment each month (est. n=2,124 over 18 months) and that, on average, most physicians would have one patient enrolled only. Assuming 1,062 patients are approached to participate in each arm, that none of these patients share the same primary care physician, a 40% participation rate and 20% loss to follow-up (complete data for 340/arm), the minimum detectable mean score difference between the two groups with alpha=0.05, beta=0.2, and an outcome with standard deviation of 0.143 is 0.030.^[25,26,27] Previous studies have estimated a minimally meaningful difference in the quality of life measure to be 0.074,^[28] indicating that we should be able to evaluate clinically meaningful changes in quality of life. To ensure our ability to conduct secondary analyses, we set a target enrollment of 1,190 patients or 595 per arm.

Given an average cluster size of 1.5 patients per physician and a commonly assumed intraclass correlation coefficient of 0.05, accounting for clustering would amount to a negligible variance inflation factor of 1.025, resulting in a minimum detectable effect of 0.0305 (which also results in an increase of 17 patients for a total of 697 patients with complete data required for detecting an effect of 0.030). Given that we have already collected baseline data on more than 1,200 patients and have a drop-out rate of less than 2%, we should have sufficient sample size for both primary and secondary outcomes even when clustering is present.

Recruitment of patients began in October 2014, approximately four months later than initially planned. However, the rate of new medication starts has been higher than anticipated. By October 27, 2015, we had screened 2,078 patients, 1,447 of whom were eligible for participation. Among those eligible, 877 unique primary care physicians were represented. The most common reasons for ineligibility were use of study medications for conditions other than diabetic peripheral neuropathy, absence of active symptoms, and lack of proficiency in Spanish or English language. We consented and enrolled 1,206 or 83% of those eligible, representing 792 unique primary care physicians and 54 facilities. Recruitment rates are similar across the two arms (control=82.7%; treatment=83.9%). Common reasons for refusal included lack of interest in participating in a study, being too sick to participate and being too busy.

Since the start of recruitment, twenty patients (<2%) have dropped out of the study. The most common reasons for drop out were worsening health status, caregiving responsibilities and challenges with the interactive voice response system such as missed calls.

Among the 1,206 enrolled as of October 27, 2015, 53% are women and the mean age is 67 (standard deviation: 12). The racial ethnic make-up is 56% white, 8% Asian, 13% black or African American, 19% Hispanic or Latino, 1% Native Hawaiian, Pacific Islander, or Native American, and 1% unknown. We have collected baseline data on 1,191 patients, 71 of whom conducted the survey in Spanish. Close out surveys have been completed by 92% (589) of those who have completed the intervention. We identified no statistically significant differences in the demographic characteristics or in baseline quality of life between the treatment and control arms.(Table 3)

This pragmatic, cluster randomized study is designed to be maximally generalizable to the patients and clinical settings in which the intervention is being tested.^[29] However, the pragmatic nature of the study design also presented several challenges that were addressed through collaboration with patient and provider stakeholders: