The study employed a convergent, parallel mixed-method research design (Creswell & Plano Clark, 2011). A one-group, posttest-only design was used for the quantitative strand of the study that included the collection of parent, staff, and management survey data. A practical qualitative approach was utilized for the qualitative strand of the study (Stufflebeam & Coryn, 2014). Institutional Review Boards at the University of Alabama at Birmingham, Indiana University, University of Iowa, and the University of Wisconsin-Madison approved, exempted, or determined that the project did not constitute research.

EHS programs were recruited from May to July 2018 and the study took place from August 2018 to March 2019. EHS programs were recruited through existing partnerships with the study team members and 48 were invited to participate in the study. In total, 11 EHS programs agreed to participate in the study, 7 operating in rural and 4 in urban areas. As seen in Table 1, programs operated multiple center-based, home visiting, and childcare collaborative programs. Most programs were able to distribute LTSAE materials to families in August or September 2018 at the beginning of the program year. Practical considerations such as program capacity to begin integrating materials and LTSAE materials delivery timing influenced the start dates. The duration of integration of materials in EHS programs ranged from 8 to 26 weeks. The survey and qualitative data collection took place from February to March 2019.

The study team and EHS program directors and managers collaboratively developed options for how EHS programs could integrate the LTSAE materials into existing practices. Either the study team or EHS management led trainings for staff on these options for integrating LTSAE materials. These trainings varied in length, with the average study team-conducted training lasting 52 minutes and the average EHS-led training lasting 15 minutes. All participating EHS programs used the milestone checklists, most often the printable single sheet and Milestone Moments Booklet versions. Some EHS programs elected to use promotional items such as children’s books, posters, and flyers to promote the Milestone Tracker App in addition to the milestone checklists. All EHS programs are required to use a curriculum, but the specific curriculum and accompanying parent handouts in use at the time of LTSAE integration was unknown. Programs had the options of providing LTSAE materials in packets at the beginning of the program year, at parent-teacher conferences, home visits, and at special parent events throughout the year. Staff were encouraged to ask parents to complete the milestone checklists and talk with families about them. If concerns about a child’s development arose, staff were to follow their program’s approved procedures for follow-up and referral.

English-speaking parents, staff, and management participated in surveys, interviews, and focus groups (see Table 2 for total participants by methodology). Across the four states, 25 focus groups were conducted, including 6 in Alabama, 10 in Indiana, 7 in Iowa, and 2 in Wisconsin. Fifteen individual interviews were conducted with six in Alabama, seven in Iowa, and two in Wisconsin. More than 90% of survey, focus group and interview respondents were female. Most parents who responded to the survey described themselves as white (49%) or black/African American (41%) and 12% indicated Hispanic or Latino origin. For the focus groups and interview participants, again most parents identified as white (53%) or black/ African American (40%). The race/ ethnicity of staff and managers in the focus groups and interviews was mostly white (67%) and black/ African American (26%) with 6% indicating Hispanic or Latino origin. See Tables 3 and 4 for additional characteristics of respondents.

Process data indicate that across all EHS programs sites, staff distributed 1,165 milestone checklists, 1,456 Milestone Moments Booklets, and 1,385 Milestone Tracker app flyers to parents during the months of LTSAE integration. During this time staff across all sites had 993 conversations with parents using completed milestone checklists. Additionally, from the staff survey, nearly all staff (99%) reported sharing the LTSAE materials with families and 70% reported reviewing checklists with most parents. More than half (58%) of staff members reported that they had completed a developmental milestone checklist for a child (separately from parents).

In total, 448 surveys from managers, staff, and parents were included in the study, with 36% from Alabama (n=162), 21% from Indiana (n=93), 22% from Iowa (n=99), and 21% from Wisconsin (n=94). Response rates for the managers, staff, and parent surveys were 71%, 61%, and 38% respectively. EHS staff surveyed were primarily teachers (55%) or home-based visitors (35%). EHS management was more diverse and often served multiple roles. These categories are therefore not mutually exclusive and included Directors (16), Child Development and Education Managers (5), Disability Services Managers (4), Health Services Manager (3), Teachers (2), Family and Community Partnerships Manager (1), Family Engagement Specialist (1), other managers and supervisors (3), and other (1).

About 90% of home visiting staff and 94% of center-based staff found the LTSAE activities very easy or easy to integrate into their work with families (see Figure 2). Similarly, most managers reported that it was very easy (35%) or easy (56%) to integrate LTSAE materials with staff in their programs. Managers provided narrative responses to support their answers on ease of integrating materials and the most common responses were that LTSAE materials aligned with their program, explained developmental milestones to parents, and were easy for the staff to use and understand. Three managers (9%) mentioned that it was difficult to make time to integrate the materials; one manager (4%) reported that staff integrated the materials but some parents did not use them, and another (4%) reported that staff were initially confused about how to integrate the materials.

Most managers (74%) reported that their program did not experience any barriers with the use of the LTSAE materials, while some (11%) reported they did experience barriers or were unsure (14%). Managers described the barriers to using LTSAE materials in narrative responses. Two managers (6%) said there was not enough time to use the materials, one (4%) reported that parents were not interested in using the information, and one (4%) reported that their staff thought the materials were inaccurate, believing the checklists were intended to reflect average ages of milestone attainment. Of staff who reported at least one barrier (55%), the four most common barriers identified were a lack of time to use the materials (26%), conflicting demands (26%), feeling that parents were unreceptive to materials (24%), and low literacy levels among parents (24%).

Most staff and managers strongly agreed or agreed with statements about the use of LTSAE increasing staff confidence in communicating with parents about development and developmental concerns, improvements to referral processes, as well as improvements in parent’s ability to complete developmental screenings (see Figure 3). Most parents (93%) reported that, because of the LTSAE materials, they talked to their child’s teacher or home visitor about their child’s developmental milestones. Most parents (72%) also agreed that they felt more comfortable talking to their child’s teacher or home visitor about any concerns they had about developmental milestones. The majority agreed that, because of the materials, they felt more confident tracking their child’s milestones, knew which milestones their child should have reached, felt confident talking to their child’s teacher about milestones, and knew what to do if they had a developmental concern (Figure 4).

Overall, 28% of parents had a concern about their child’s development and followed up with a professional about their concern. Half of the parents surveyed (51%) reported that the materials helped them identify a concern about how their child plays, learns, speaks, acts, or moves. Of these parents, about 68% said their teacher or home visitor suggested that they talk to someone (e.g., a doctor or other professional) about their concern and most of these parents (80%) reported that they did so. Of those who reported speaking with a professional about their concern, 52% thought the materials were very helpful when talking to their doctor, 26% thought they were helpful, 1% thought they were unhelpful, and 1% were not sure. Thirty-two parents provided narrative responses to explain what happened when they spoke to a professional about their concern. Reported outcomes for these conversations were that 28% of parents received additional information about their concern, 28% were referred to another professional, 25% were currently receiving help for their concern, 16% reported that their child was diagnosed with a developmental or physical delay, and 3% reported that they talked to a doctor but the doctor did not identify a developmental delay.

Most parents (92%), staff (85%), and managers (97%) indicated that their program should continue to offer LTSAE materials to families. Managers explained through narrative responses why their program should continue to offer LTSAE materials to families and the most common responses were that materials help parents track and understand their children’s development, were a useful resource for teachers to communicate with families, and generally enhanced the program.

The generalizability of these findings into other early childhood education settings may be limited as EHS has specific requirements and structures which may differ from other ECE programs. While conducting the project in multiple EHS programs and in four states allowed for collection of diverse opinions and experiences with LTSAE, the project had several limitations that affect how generalizable these findings may be to other EHS programs. While all parents of enrolled children were offered LTSAE materials, not all parents received the survey. Resource constraints limited the data collection to English-speaking parents. It is possible that non-English speaking parents had different experiences of the program not represented in these results. Additionally, two programs experienced challenges distributing the parent survey and did not offer it to all parents of children enrolled. The number and characteristics of parents in the EHS programs who did not receive the survey were not available to the study team. Also, the response rate to the parent survey was low as some programs found it challenging to motivate parents to complete the survey. Social desirability may have influenced self-report of behaviors such as following up on developmental concerns. Further, the participating EHS programs may not be representative of all EHS programs. The programs voluntarily participated in this project because managers were interested in utilizing the LTSAE materials and felt their program had some level of capacity to integrate materials and participate in data collection. These findings may be different in programs with different degrees of support from management and capacity to initiate a new activity. While some participants reported that parents received too many materials at once, the project could not control exactly when parents received LTSAE materials and did not have full information about other EHS materials that may have been shared at the same time. Variation across programs is a significant limitation. There were some notable differences in survey results between states and by program implementation time. However, the study design and inconsistency of the findings limit the ability to determine whether these differences are driven by state or population differences, length of program implementation, training or other factors (e.g., type of program, method of implementation). Also, while some staff reported low literacy as a barrier, the literacy level of parents who participated was not assessed. Lastly, the study used a single group, posttest-only design to accommodate the study timeframe and avoid overburdening participants with data collection. However, this design alone has considerable internal validity threats (Shadish et al., 2001), limiting the inferences that could be made about the program specifically causing change in behavior, attitudes or screening and referral outcomes.

To understand the impact of LTSAE, randomized controlled studies are needed that could more fully evaluate integration and LTSAE’s ability to improve early identification and children’s developmental outcomes. These studies are needed in EHS as well as other ECE settings and would ideally include parents who speak English, Spanish, and other languages. The challenges presented in this project to conducting evaluation research in real program settings, such as variability in how programs integrated LTSAE, data collection challenges, and brief study timeframes, could be addressed by working with only one or two ECE program partners with a single procedure for using LTSAE over two or more school years. More research is needed across early childhood program settings including EHS on early identification processes to target improvements and ensure children are accessing services as early as possible (Barger, Rice, Simmons, & Wolf, 2018). ECEs across early learning settings have an important role in early identification of developmental delays and disabilities and many are implementing practices aimed to identify children earlier (Boh & Johnson, 2018; Branson & Bingham, 2017; Ford & Johnson, 2017; Peterson et al., 2011). However, there is a dearth of research on the extent these activities are occurring across the field, and even less study about outcomes on early identification, child health, and wellbeing. Describing these processes would further the understanding of the needs of ECEs and their contributions to early identification in communities. Future controlled studies examining implementation of programs such as LTSAE could help fill these gaps if conducted over a sufficient period of time to measure change in developmental monitoring, developmental screening, referrals for developmental delay, and child outcomes related to early intervention.

This project was the first to examine the feasibility and value of integrating the use of LTSAE resources in EHS settings in the U.S. It was uncertain whether participants would value LTSAE in EHS settings where developmental screening is routine. This project showed that managers, staff and parents valued the LTSAE materials because they are designed for families and offered a common language as a foundation for discussions about development. Although some challenges were identified, results suggest integrating the use of LTSAE resources in a variety of EHS settings was generally feasible and acceptable, and might enhance EHS developmental monitoring, screening, and referral.