We conducted 54 interviews with 9 clinic staff across the 4 clinics. The PHC onboarder and clinic champion at each site were interviewed because of their involvement in PHC and their ability to provide relevant feedback. Each staff member was interviewed 6 times, including 1 face-to-face interview before implementation, 4 weekly interviews by telephone during implementation, and 1 wrap-up interview by telephone after implementation. Staff were asked about clinic preparedness; satisfaction with training; barriers and facilitators; contextual factors affecting implementation; appropriateness, acceptability, and feasibility of PHC; perceived sustainability; and suggestions for improvements for PHC. Four trained coders coded notes from the interviews using a topical codebook [14]. We used a framework analysis approach to identify themes by site, staff type, and time points [24].

We reviewed assessments completed by the clinics’ clinic champions and onboarders in preparation for the pilot implementation to identify potential barriers and facilitators. The clinic workflow assessment asked clinics to document the processes that occur during a typical patient visit and to assign key PHC pilot implementation roles to clinic staff members. The informational technology (IT) assessment asked each clinic to confirm that their IT infrastructure met the minimum requirements of (1) having Internet Explorer (9, 10, or 11) or Chrome (Version 34) installed on a clinic computer; (2) a minimum Internet download speed of 10 Mbps; (3) a minimum wireless connection of 802.11g, n, or ac; and (4) wireless routers that support 802.11g, n, or ac standards. The implementation tailoring worksheet collected information on potential clinic-specific adaptations to generic implementation procedures, as well as barriers and facilitators.

We reviewed standard operating procedures for training, TA, and site visits and other documentation of interactions between the research team and clinic staff, including site visit and meeting notes, to identify implementation strategies used in the pilot implementation.

The clinic workflow was a barrier at all sites. All sites reported that there was not enough time to complete PHC between the patient’s arrival and when they saw their provider. The amount of time required to onboard patients was cited as a barrier and was exacerbated by the learning curve to implement PHC. Onboarders at Clinics A, B, and C felt that the onboarding process took less time as they became more comfortable with implementation.

Clinics A, B, and C reported engagement with staff not involved in implementation as both a barrier and a facilitator. The goal of staff engagement is to generate buy-in from all clinic staff to facilitate integration of the intervention into the clinic workflow. Integration includes accommodating workflow changes and partnering with implementation staff to recruit patients to use the intervention. At Clinic A, staff engagement facilitated implementation because the onboarder had generated buy-in by meeting with each provider and demonstrating PHC. At Clinic C, staff engagement was both a barrier and facilitator. Clinic C reported that high staff engagement allowed the onboarder, who was brought in exclusively for the pilot implementation, to become integrated into the clinic. However, the onboarder was not able to engage other clinic staff to allow the patients to complete PHC before being seen; as a result, patients at Clinic C had low PHC completion rates. Similarly, Clinic B reported that clinic staff had not come to an agreement beforehand regarding when patients should use PHC. As a result, patients were frequently interrupted while using PHC.

The digital literacy level of patients acted as both a barrier and facilitator to PHC. The password generation process in particular was a barrier during the onboarding process. For privacy reasons, PHC requires that patients generate complex passwords with a combination of symbols, numbers, and uppercase and lowercase letters. Staff at Clinics A, C, and D noted this was a time-consuming process because onboarders had to help patients come up with their own password. Additionally, Clinic D reported that the onboarder had to assist patients with PHC because they were not familiar with using tablet computers. Clinic B did not find password generation to be a barrier because they had a younger patient population with a higher level of digital literacy. Digital literacy facilitated implementation at Clinic C because patients were routinely approached to use tablet interventions for research.

At Clinics B and C, patients were frequently interrupted while using PHC to meet with nurses and providers and receive other services. These interruptions were attributed in part to insufficient physical space. At Clinic B, providers assumed that patients were available because they were in an exam room. Clinic B required that patients use PHC in exam rooms because of concerns about privacy in a communal waiting room and the impracticality of using the clinic conference room as an alternative. At Clinic C, providers could not delay seeing patients to allow them to complete PHC because of a shortage of exam rooms. In contrast, Clinics A and D had extra space, which allowed patients to use PHC without disruption.

The physical environment also hindered delivery of handouts at Clinics B and C. Fax machines were located in inconvenient spaces that prevented the onboarder from either knowing that the handout was printed or added substantial time between printing and delivery so that patients were already seeing their provider.

The technological environment was intermittently a barrier to implementation at Clinics A, B, and C. At Clinic C, handout printing was delayed by up to 40 minutes because of an overburdened fax machine. As a result, patients did not receive their handouts before their appointment. At Clinics B and C, user error, poor Internet connections, or other technological issues at times required the onboarder to intervene and reset the tablet or PHC.

Onboarders at all clinics reported being satisfied with the training and TA that they received. Providers at Clinics A, C, and D reported being satisfied with the in-person training; the provider and clinic champion at Clinic B felt that they would need to review the training materials again to determine their satisfaction with training.

What Implementation Strategies Were Used to Implement PHC?

Of the 73 discrete strategies identified by Powell et al. [3], we used 19 strategies from the plan (8 strategies), educate (5 strategies), finance (1 strategy), restructure (1 strategy), and quality management (4 strategies) categories specified by Powell et al. [2] in the pilot implementation of PHC. We describe the operationalization of each strategy used in the PHC pilot implementation in Table 3.

We used 8 planning strategies to prepare clinics for implementation. Before establishing contracts, each clinic completed a written assessment that asked for the percentage of their patients who met the criteria for the intervention’s target populations (Strategy 1). The purpose of this assessment was to confirm that PHC would be relevant to the clinic’s patient population and that the clinic was representative of sites that would potentially implement PHC during dissemination. After each clinic completed the application, we established a contractual agreement that required the clinic to implement PHC for 4 weeks (Strategy 8). During this process, a clinic champion was identified at each clinic (Strategy 6). The champion assisted with receiving the necessary approvals and identifying staff who would implement the intervention.

During the preimplementation period, the clinic staff responsible for implementing the intervention completed 3 written assessments that were designed to assess for readiness and identify barriers (Strategy 2) and to promote adaptability of intervention procedures (Strategy 7). We intended to conduct a quick confirmation of these assessments at the in-person site visits (Strategy 5); however, at the first site visit it became clear that the written assessments were not sufficient to capture the clinic’s workflows accurately and to identify potential barriers. As a result, clinic staff at each site completed a physical walk-through with TA staff during which they discussed barriers, patient and staff movement through the clinic, and implementation procedures. Based on the information gathered from the assessments and site visit, we generated a tailored implementation plan for each clinic (Strategy 3).

We used 5 educate strategies to train staff implementing the intervention and to educate other staff members about the intervention and pilot implementation. We developed educational handouts and an intervention package that included an in-depth implementation manual and a 1-page quick guide that described key implementation steps (Strategy 9). During staff interviews, staff reported frequently using the quick guides, while the in-depth manual was used only occasionally for reference. These materials and others were distributed in physical and electronic forms at the site visit (Strategy 10). Initially, we planned to host in-service trainings for all clinic staff during each site visit (Strategy 11). However, it was not possible to host this training at all clinics because of scheduling conflicts. To train implementation staff, we had developed PowerPoint presentations to describe the intervention and implementation procedures as part of the intervention package. At the first site visit, however, we found that it was more effective and engaging to conduct walk-through and role-playing exercises with staff as part of a dynamic training approach with the PowerPoint and intervention package as reference materials (Strategy 13). Subsequent site visits used the dynamic training approaches.

We used 1 restructure strategy to provide clinics with the equipment necessary to implement PHC (Strategy 14). We initially intended to provide all clinics with 3 iOS or Android tablets. We ultimately supplied all sites with privacy screens and headphones to maintain patient privacy when the intervention was used in a communal space. We also provided hotspots to 2 sites to supplement insufficient Internet bandwidth or dead zones and wireless printers to 2 sites to address issues with fax machines.

We used 1 finance strategy to fund and contract for the clinical innovation (Strategy 15). This strategy did not change during the pilot implementation.

We used 4 quality management strategies to monitor implementation and provide TA. The centralized TA team structure (Strategy 16) allowed liaisons to identify common barriers across sites and to share solutions that were implemented at 1 site with other clinics that were experiencing the same challenge (Strategy 17). We provided clinic staff with access to the CWA to monitor implementation fidelity and progress (Strategy 19). However, clinics rarely used the reporting and data export functions to examine implementation. The TA team purposefully re-examined implementation (Strategy 18) by using the data provided in the CWA and discussing implementation during the weekly interviews with key staff. The weekly interviews became an integral part of TA efforts because implementation barriers, adaptations, and solutions were frequently discussed during these calls.

Insufficient wireless Internet bandwidth, wireless Internet dead zones, overburdened fax machines, inconvenient access to fax machines, and limited availability of private spaces for patients to use the intervention were barriers addressed by providing clinics with the appropriate equipment. However, not all environmental barriers could be addressed. For example, patients at Clinics B and C still experienced disruptions while using PHC because of the limited availability of space.

Implementation at the clinics was staggered by approximately 1 week. This allowed us to refine implementation strategies, such as introducing dynamic training and providing wireless printers and hotspots, to better prepare staff at subsequent sites. This approach combined with centralized TA also allowed us to share knowledge between sites. For example, Clinics A and D experienced barriers with low digital literacy. Clinic A developed a script to help patients generate acceptable passwords; this script was shared with all other sites through liaisons and implemented at other clinics.

All of the quality management strategies used in the pilot implementation contributed to the sufficient TA the clinics received. Through the centralized TA strategy, liaisons were able to identify and respond to clinic TA needs with in-depth knowledge of the clinic context. Lessons learned from 1 clinic were applied to other clinics through the centralized TA. Purposefully re-examining the intervention during weekly interviews and using CWA data were important opportunities for liaisons to identify the clinics’ TA needs. Implementation staff discussed barriers and adaptations made to implementation procedures on the weekly calls; liaisons helped troubleshoot barriers and suggest alternative adaptations when changes to procedures threatened fidelity. Using the CWA, liaisons were able to examine the number of patients using PHC and the extent to which it was implemented with fidelity. Liaisons used these metrics to work with clinic staff to identify barriers.

We found dynamic training to be an effective and engaging method of training implementation staff. In the interviews, implementation staff at all sites reported having received sufficient training. However, despite adequate training, 3 of the 4 sites still experienced a learning curve during the pilot implementation period.

We promoted adaptation of implementation procedures as a way to address clinic workflow periods throughout the pilot implementation. TA was often focused on how to address these barriers through adaptations. Despite these efforts, all clinics reported having insufficient time for patients to use PHC before their appointment in part because of intervention characteristics. Three clinics reported the length of time required for the onboarding process as a barrier, and 3 clinics reported the time required to generate the password as a barrier.

Changing the clinic workflow to accommodate PHC required engagement from other clinic staff (e.g., providers and staff checking in patients). Implementation staff reported that low engagement from clinic staff hindered implementation at 2 clinics and high engagement facilitated implementation at 2 clinics. The impact of our educational outreach strategy was likely limited. Although a contractual agreement was established at all sites, 2 sites reported insufficient engagement to prioritize the intervention in the clinic workflow. The implementation staff at the 2 clinics that reported high staff engagement reported building staff buy-in outside of the in-service training offered during the site visits.