SCPs have only recently been used for childhood cancer survivors and have been studied by our team. SCPs are a tool to improve health-related knowledge, engagement, and patient-provider communication (Shalom, Hahn, Casillas, & Ganz, 2011). They contain information about cancer treatment history, risks for late effects, and resources and strategies for health promotion. Only 38% of adult oncology patients reported receiving treatment summaries, a key component of SCPs (Sabatino et al., 2013), and about half of survivors of childhood cancer report receiving a survivorship care plan (Casillas et al., 2011). The University of Pennsylvania developed OncoLife, a publicly available SCP generating tool for adult cancer survivors that has generated over 60,000 SCPs to date (Hill-Kayser, Vachani, Hampshire, Jacobs, & Metz, 2009). Smart-ALACC (Adult Living After Childhood Cancer) was adapted from OncoLife to serve as a digital health tool for generation of SCPs for AYA survivors of childhood cancer (Szalda et al., 2016). Smart-ALACC content was written by a team of oncology physicians, nurses, psychosocial team members and a young adult to incorporate recommendations from the Children’s Oncology Group guidelines and other AYA resources. In pilot testing, Smart-ALACC demonstrated high feasibility (easy to use, taking less than 10 minutes to complete) and acceptability (Szalda et al., 2016). The incorporation of Smart-ALACC into an app intended to enhance uptake of the SCP is one aspect of AYA STEPS.

Our prior texting intervention, which also informed AYA STEPS, is called THRIVE (Texting Health Resources to Inform, motiVate, and Engage; (Schwartz et al., Submitted). THRIVE was tested through a pilot randomized controlled trial (RCT) for AYA survivors within one year after completing treatment. It was informed by our SMART framework of transition readiness (Schwartz et al., 2013), as well as the Health Belief Model and Social Cognitive Theory (Glanz & Rimer, 2005). Daily text messages were intended to enhance awareness of health vulnerability and the importance of continued engagement in care, while providing support and encouraging positive health. Text messages were tailored to patient age, a selected health goal (e.g., increase physical activity), medication adherence (yes/no), medical appointment reminders, and included general content on health-promoting behaviors, academic and social adjustment, and engagement in follow-up medical care. THRIVE demonstrated high acceptability and feasibility and preliminary efficacy related to improvements in health knowledge, health behaviors, and perceptions of health vulnerability.

For the current study, we combined Smart-ALACC and THRIVE into a mobile app called AYA STEPS. It contains individualized SCPs generated via Smart-ALACC and delivers daily tailored messages within the app to improve knowledge of personal health history and risk as well as support adherence to recommendations within the SCPs. To integrate and optimize these interventions, we adopted an agile science framework (Hekler et al., 2016). Agile science is an iterative process of intervention development, which involves ongoing optimization based on the fit between the individuals, contexts, and intervention in order to produce interventions intended for real-world use (Hekler et al., 2016). This framework borrows from agile software development, user-centered design (McCurdie et al., 2012), and implementation science methods such as pragmatic trials (Glasgow, Phillips, & Sanchez, 2014). Compared to traditional phase-based models of intervention development, agile science adopts small but rapid steps to specify, iteratively improve, and optimize intervention ideas (Patrick et al., 2016). Thus, agile science is especially well-suited for mHealth intervention development, as technology often changes at a faster rate than the scientific process. The ultimate goal is transparency of the process to determine components and when, where, for whom, and in what state to use a given tool (Murray et al., 2016). Behavioral scientists and clinical researchers have increasingly incorporated aspects of the agile process into their development of mHealth interventions, but none have clearly articulated the steps of the agile process (Casillas et al., 2017; Jibb et al., 2017). The complete steps to development of AYA STEPS, informed by the agile science framework, are described below.

Building on Smart-ALACC and THRIVE, we began with 1) developing a multifunctional mHealth app that could display the SCP document and allow two-way tailored messaging for AYA survivors, and 2) modifying and expanding the THRIVE text bank to reinforce, and enhance the uptake of, individual SCP recommendations. This involved the creation of new treatment-specific text messages and algorithms, by which the cancer treatment history would inform both the content of their SCP as well as the text messages they received within the app (see Figure 2). We convened a multidisciplinary working group of cancer survivorship researchers and providers [1 adult MD, 4 pediatric MD specialists (one of which is trained in medicine-pediatrics), 3 psychologists, 2 advanced practice nurses), technology specialists (LST team, 4 team members with mHealth expertise), and AYA volunteers (2 paid and 3 unpaid research assistants)] to determine tailoring algorithms and create or modify text messages from THRIVE. Each individual text message was developed with consideration of guiding theoretical models and prior acceptability data.

We identified a vendor (LifeScience Technologies; LST) who had an existing, yet modifiable, HIPAA-compliant disease self-management app (LSTCare). LST was selected due to optimal features of their automated app that included: 1) an integrated messenger applet that allows for two-way text messaging tailored to specific patient attributes, 2) capability of interfacing with the electronic health record (EHR) to upload information about appointments, medications, and so forth, 3) it has a configurable set of 50 health-related applets designed to facilitate self-management, including various applets for tracking health behaviors (e.g., step counts), medication reminders, capacity to upload and display documents (e.g., SCP) and videos, and 4) the ability to include gamification via a point system to reinforce engagement. Our multidisciplinary team selected applets hypothesized to facilitate survivorship self-management and support the aims of the study, while graphic designers at our institution were called on to make design modifications to customize the “look and feel” of the app for AYA survivors.

As depicted in Figure 2, the multidisciplinary team identified four major message tailoring components important for survivorship self-management and SCP uptake: time since completion of therapy (completed therapy in the last two years vs. long-term cancer survivor), age (15–17 years old vs. 18 years or older), personal health goal (e.g., getting more sleep), and treatment history (e.g., types of chemotherapy and location of radiation). After identifying these tailoring components, we modified existing THRIVE messages or created new messages to fulfill each of these components. Treatment history messages were developed to directly match with SCP content. For example, an AYA who received cisplatin, a type of chemotherapy drug that poses risks for hearing deficits, would receive a message related to getting hearing checked.

The development of text messages was done through group consensus during weekly meetings for three months, with team members suggesting and revising content to be theoretically grounded, developmentally appropriate, specific to intervention goals, and at a 6th grade reading level. The final text bank included 227 messages, about half of which were tailored to individual characteristics, with two-way functionality. Specifically, some messages invited a response (e.g., a trivia question or “text back for more information”), but also allowed free text to engage the team with questions or comments.

During this stage, weekly teleconferences with our technology vendor (LST) took place to review app navigation and applet functionality, user tracking and incentivization options tied to engagement in the app (gamification), and tracking of usage data. We then finalized the first prototype, which included the tailored text message bank, embedded SCP, selected applets for facilitating survivorship self-management (i.e., nutrition and activity tracking, goal tracking, points accrual, SCP content, messaging, appointment reminders and list of medications and allergies based on syncing with EHR), and branding/design elements (i.e., new color scheme and AYA STEPS logo).

Our next step was to solicit feedback about the first prototype from purposefully sampled AYA survivors (a.k.a., end users for this intervention) (see Figure 1). After obtaining IRB approval, we invited former THRIVE participants (n = 38, ages 15–25), who had experience with an AYA mHealth intervention within one year of completing treatment, to participate in a focus group or individual interview. Ten THRIVE participants participated in a focus group or individual interview. Consistent with an agile approach, flexibility and efficiency were optimized in the form of providing AYA the option of focus group or interview (Patel & Arya, 2017). The principal investigator (L.S.) led the focus group (n = 5); one of two trained study team members (A.P. and E.V.) conducted individual interviews (n = 5). A research assistant took notes during the group and interviews. A semi-structured interview guide included questions about experience with THRIVE, feedback about AYA STEPS and a sample SCP, and feedback about text message content (e.g., topic, wording). Participants downloaded AYA STEPS to their phone in order to provide real-time feedback on design, navigation ease, and content. A web-based program (Socrative; https://www.socrative.com) was employed to solicit feedback about message content, such as acceptability of certain text messages, recommended links, and other media. Through Socrative, participants could respond anonymously about what they liked or disliked (e.g., text messages, or graphics). We specifically had them review 34 messages representative of various types of messages on which the team decided further AYA input on acceptability/interest would be beneficial (e.g., messages representing quiz questions, stories, relaxation topics such as yoga, different types of media such as video or memes). Queries included, “Would you like text messages to be personalized with your name?”; “Would you be interested in cancer survivors’ stories?”

The group and interviews were audio-recorded and transcribed for review. Following an agile approach, the transcripts were not coded or analyzed using traditional qualitative methods. Rather, notes and transcripts were reviewed for feedback to improve the end user experience with the app and to assess if additional feedback was needed (e.g., if the feedback was too variable to draw conclusions on how to improve the app). Feedback informed refinements to the messages, while our technology vendor made recommended changes to the app design and functionality. These changes resulted in a second prototype of the AYA STEPS app.

Mean age of the 10 AYA providing feedback was 19 (Range = 15–21), 60% were female, 50% identified as non-Hispanic White (30% identified as non-Hispanic African American and 20% identified as Hispanic White), 80% had a prior leukemia/lymphoma diagnosis, and the average time since completing cancer therapy was 1.6 years (range 1–2 years). The feedback was constructive and actionable, and somewhat consistent across the 10 AYA, with interview responses overlapping with focus group feedback. Table 1 described feedback and subsequent modifications to the AYA STEPS prototype, including changes to text messages, functionality, and usability/design. For example, since AYA desired text messages to be tailored to their name, we added the participant’s own name to approximately one third of messages. We also modified several messages to be more salient to AYA values, such as independence and maturity. For example, a text message on sticking to a bedtime routine was edited to appeal to independence (i.e., …it not only takes discipline but shows that you are capable of taking care of your health) and preference for personalization with their name embedded.

Results of Stage 2 also informed further modifications to app functionality (e.g., adding a weekly survey assessing and graphing health goal progress; adding ability to sync fitness tracker data with app) and design (e.g., ability for users to display a photo). We also created an introductory video that showed two of the study doctors (LS and DS) welcoming participants to the study and orienting them to the app. While we were able to accommodate many suggestions for improvement, working with a shelf-ready app limited us in accommodating all areas of feedback (e.g., app functionality did not allow AYA to select their own color scheme).

During Stage 3, members of the study team (n = 6) tested the app on their phones, receiving messages for one month based on randomly chosen characteristics to create a “sample” patient. Once these initial issues were resolved (within the month), a pilot study was implemented with 14 AYA who were consecutively recruited in the Cancer Survivorship Program (all at least 2 years off treatment) as a convenience sample. The full procedures of the larger ongoing RCT were enacted (e.g., randomization, follow-up assessment) to test the feasibility of study procedures in their entirety; we only report here on 8 AYA who were randomized to the app. (The outcomes of the ongoing RCT will be reported when the trial is complete). AYA survivors, aged 15–29, who were treated for pediatric cancer, off treatment at the time of study participation, and able to read and speak English, were eligible for the AYA STEPS study. For those randomized to the intervention group, the app was manually downloaded on their phone by a team member, and was then relatively automated and did not require any further manipulation by the participant. The messages were delivered daily within the app and AYA received push notifications when new messages arrived in the app.

We assessed any problems with functionality and usability (e.g., ease of use, technology glitches) of the app and elements within it (e.g., text messages, SCP display, applets such as nutrition and step tracking, connection with electronic health record). Acceptability, including perceptions of the value, appropriateness, convenience, and helpfulness of AYA STEPS, as well as feedback on usability, was assessed with a 59-item survey created for the study containing both close-ended and open-ended questions. We report on 12 of those items related to specific experiences with the mobile app and/or text messages (other items pertained to general experiences or feedback on the SCP). We also assessed the end user usage data via the LST dashboard, which included frequency of views of the SCP and messages.

Prior to pilot participant enrollment, the 6 study team members discovered many technology and text message issues, such as typos in text messages and weekly goal and medical adherence surveys triggered at inconsistent or wrong times. We also tested with different types of devices (Android vs. iPhone) and remedied as needed. These issues were resolved within one month and we subsequently began recruiting pilot participants.

Data from 8 AYA survivors who were randomized to the intervention group and piloted the second AYA STEPS prototype (M age = 18, 88% female, 75% White/Non-Hispanic, M number of years since completing treatment = 8.5 with a range of 6–14, 88% history of leukemia/lymphoma) are reported here. Seven had iPhones and 1 had an Android. Responses on the acceptability survey revealed all AYA reported that the app was easy to use and it was easy to access the messages. All but one reported that messages were useful, they looked forward to receiving daily messages, and links and resources embedded in the app were helpful or interesting; that the app improved health knowledge and emotional and physical well-being, had positive impact on quality of life, related to own experiences, and provided new information not discussed with health providers; and that the applets were generally useful or interesting. In open ended comments, one participant reported losing the app. Some participants suggested syncing AYA STEPS with existing health apps rather than providing applets.

In terms of usage data, all eight AYA participants opened the app and engaged with it. Engagement with the app ranged from 19.3% to 98.2% of app intervention days, with a median of 63.9% of days actively using the app. All AYA viewed their SCP within the app (range = 2 to 9 times; M views per participant = 5.4). All participants read text messages in the app and 3 read all of them (range of % viewed = 24–100, median = 98.4). All AYA provided text responses (response to question in message or free text), ranging from 1 response to 46 responses.

From the pilot, we also discovered several functionality difficulties that were resolved before launching our ongoing RCT. For example, ensuring similar end user experiences across iPhone and Android devices was a significant challenge. For instance, iPhones needed app updates pushed, requiring an active update by the participant, while Android users’ apps could automatically update depending on their phone’s settings. Further, viewing the SCP on an Android was hindered by the lack of a native PDF reader, which complicated the enrollment process, since it required an additional app installation. A second functionality issue was variability in linking the AYA STEPS to outside applications. For example, a fitness device such as a FitBit needed to be registered, while iHealth (iPhone’s native fitness tracker) automatically synced with AYA STEPS. Third, several glitches occurred with specific app components, including functionality of graphics, repeated or missing text messages, and duplicate points for responding to text messages. We also found a variety of glitches in the usability/design of AYA STEPS, including individual user’s settings needed to be adjusted to ensure that push notifications, data sharing with iHealth, font sizes, default colors, and so forth worked as expected. Individual Wi-Fi connectivity issues caused delays in loading features (e.g., missing graphics). There were also issues with the EHR integration, such as problems with automating the integration process.