This mixed methods study was conducted from December 2008 to June 2010 at four urban pediatric primary care practices in Philadelphia. These practices experience more than 86,000 annual visits (22,500 under the age of five years). Children were eligible to participate if they were younger than 30 months old at the time of a visit, were greater than 36 weeks estimated gestational age, had no major congenital anomalies or genetic syndromes, were never placed in out-of-home foster care, and were not currently receiving Early Intervention Part C services. Clinicians were eligible if they were attending pediatricians, nurse practitioners, or pediatric residents at any of the participating practices. Medical students were excluded. The study was approved by the Institutional Review Board at the Children's Hospital of Philadelphia. Eligible participants including both parents and clinicians completed written informed consent. This study was a prelude to a clinical trial [25].

Prior to implementation of screening, we conducted two focus groups with parents and four focus groups with clinicians to identify perceptions related to developmental screening. Parents of children under the age of 5 were recruited to participate, as well as clinicians at the four participating practices. We used the Theory of Planned Behavior as a conceptual framework to guide our inquiry [26,27]. This model proposes that the strength of an individual’s intentions to adopt a new behavior, developmental screening in this case, is dependent on their attitudes toward the behavior, the subjective norms and beliefs of those around them, and their perceived behavioral control (i.e., challenges they believed prevented them from conducting developmental screening) [26,27].

Each focus group meeting consisted of four to eight participants [28]. Meetings lasted for approximately one hour and were led by trained facilitators. Each meeting consisted of open-ended questions followed by a summary of responses. The questions sought to identify perceptions related to the importance of developmental screening, current screening practices, and challenges to implementing universal screening in order to more accurately identify these challenges for implementation of developmental screening. For example, one question used in the focus groups for pediatricians was, “In the care of young children less than five years old in primary care settings, how important is periodic screening of the child’s development compared with other aspects of well child care?” Similar questions were adapted for the parent focus groups. To improve the validity of our findings, we summarized the main ideas and sought participant feedback at the conclusion of each meeting. An investigator (J.G.) was present at all meetings to record field notes.

All focus group meetings were audiotaped and transcribed. Transcripts were entered into Ethnograph 6.0 (Qualis Research Associates, Thousand Oaks, CA), a qualitative software program that allows users to code, organize, and conduct searches for themes across transcripts. Field notes compiled at the meetings were used to supplement information from the transcripts. Transcripts were initially read by seven investigators (D.M., S.P., A.B., N.B., M.G., J.P., J.G.) to identify major themes. This team of investigators included a diverse array of individuals from general pediatrics, developmental-behavioral pediatrics, psychology, and qualitative research methods. Reliability in the selection of themes was ensured during an investigator meeting that achieved consensus on the overall themes and code lists. Based on this list of themes, investigators (D.M. and A.B.) independently reread and coded all transcripts. The themes were coded using the constant comparative approach, and any differences were settled by consensus [29].

We developed an implementation strategy that encompassed the challenges identified in the focus groups including selection of developmental screening tools, clinician training methods, development of clinical workflow patterns, use of electronic decision support tools, and building a collaborative relationship with local early intervention (EI) agencies to share data. We garnered clinician input on the selection of screening tools through meetings at all four participating practices. After selection of tools, we developed an in-person training session to provide clinicians with an overview of the tools, information on administration and scoring of the tools, suggested text for interpreting positive screens with parents, and recommended referral procedures. Finally, we held meetings with staff from our local EI agency to develop agreements to share data on the status of EI referrals.

Our quantitative outcome measures were screening rates and clinician and parent satisfaction with screening processes. We developed ad hoc measures of satisfaction regarding developmental screening, ease and use of tools, challenges to screening, and overall satisfaction with screening. For each item, satisfaction was rated on a five-point likert scale from 1 = very dissatisfied to 5 = very satisfied. Clinicians were e-mailed a link to an electronic version of the survey following the completion of the study period. In the email, they were thanked for participating in the study, but were not provided with any incentives for completion of the survey. Parents were called by study staff and completed a phone survey similar to the clinician survey. Parents were queried concerning demographic characteristics, knowledge and attitudes towards screening, ease and use of tools, outcomes of screening, and overall satisfaction. Clinician and parent satisfaction surveys were entered into a Research Electronic Data Capture (REDCap) database at the Children’s Hospital of Philadelphia [30].

We collected information from our electronic health records (EHR) on the number of patient well child visits, developmental screening and surveillance tools completed, and the number of EI referrals made during the study period. We obtained EI referral data from the participating EI agency and merged it with EHR data using personal identifying information (medical record numbers, child name, child date of birth). Once data were merged, all personal health information was deleted prior to analysis. Prior to the screening intervention, referral to EI was approximately less than 10%, but no exact data existed at the local EI agency or at the Children’s Hospital of Philadelphia for accurate comparison.

We completed summary statistics on parent and clinician satisfaction surveys as well as EHR and EI referral data. The parents were not the same parents that were involved in the focus groups, however many of the same clinicians participated in both the focus groups and the satisfaction questionnaires. We categorized parent and clinician responses of “satisfied,” “very satisfied,” or “yes” as indicating agreement with each statement. We aggregated the proportion of patients who had a well visit, completed a developmental screening tool, had an EI referral, and completed an EI referral that included a multidisciplinary evaluation during the study period.

Both parents and clinicians endorsed the importance of discussing the child’s development during the well visit. However, parents felt pediatricians undervalued parental knowledge and concerns about child development (Table 2). Parents voiced a desire to provide greater input on their child’s development and share in treatment decisions. Conversely, clinicians perceived that parents in their practice lacked knowledge of normal development. They reported that they routinely did not rely solely on parental report of development, but utilized a combination of parental report, clinician observation, and clinician expertise to assess development (Table 2).

In these urban practices prior to this study, clinicians acknowledged that they frequently employed surveillance to assess development, relying on a set of age-specific milestones that were incorporated into the electronic health record to identify delays at well child visits. Clinicians reported validated developmental screening tools were used sporadically and only when parents raised concerns (Table 2). Likewise, parents felt clinicians did not spend adequate time to assess their children’s development. They desired more information on development for their children (Table 2). Parents preferred clinicians to be more proactive in referring their children to developmental services rather than using a watch-and-wait approach for screening and referral.

Clinicians identified the following challenges to screening: lack of time to conduct screening, lack of reimbursement for completing developmental screening tools, and lack of training in the use of developmental screening tools. Time constraints within busy well child visits were recognized, and clinicians perceived that they could not eliminate important aspects of well childcare to accommodate screening (Table 2). At the time of this study, developmental screening was not reimbursed in these four practices. Clinicians acknowledged that additional funding might help facilitate screening by paying clinic staff to assist with screening procedures or allotting extra time during well visits to conduct screening. Finally, clinicians perceived a lack of training on the administration, scoring, and interpretation of tools.

Using the information gathered from the focus groups, namely parents’ desire for more input on development and clinicians’ preference for a brief, validated, and global developmental screening tool, we made a list of validated screening tools for consideration in this urban clinic setting (Table 3). Table 3 illustrates our implementation of developmental screening strategy that includes suggestions from our focus groups. We convened an additional meeting with clinicians from participating practices in order to review the AAP recommendations for developmental screening and to provide an overview of a number of tools. Based on feedback from that meeting, we selected the Ages & Stages Questionnaires, Second Edition (ASQ-II) as a general developmental screener at the 9-, 18-, and 30-month well visits [31]. The ASQ-II accommodated parental self-report, clinicians’ preference for a milestone-based instrument, and adequate speech and language assessment that could also provide an educational tool for house staff, residents, and parents (Table 3). It was also recognized that the local EI agency used the ASQ-II as a first-stage screening tool for the evaluation of children with possible delays. We also selected the Modified Checklist for Autism in Toddlers (M-CHAT) at the 18- and 24-month well visits as it was a similarly validated parent self-report tool and was the only brief autism screener available in this age group [32].

We provided training and education of Pediatric Residents and Attendings at their discretion in order to meet the clinicians’ desires from the focus groups of providing sufficient training and resources on developmental screening (Table 3). Group meetings were conveniently scheduled at each participating practice to conduct clinician training. At these meetings, we reviewed developmental screening recommendations, provided an overview of the screening tools selected, and gave hands-on instruction in the administration, scoring, and interpretation of the tools. For those who could not attend a group training, a training video was developed that could be reviewed at their leisure. Continuing medical education credits were provided to Attendings as an incentive to complete training, and the developmental screening tools were incorporated into the overall residency curriculum. For additional assistance, on-site clinic staff was available in each clinic to assist clinicians with screening.

We developed and incorporated electronic clinical decision support tools to support developmental screening at recommended well visits in order to include the clinicians’ request to have better access to developmental screening resources (Table 3). Alerts were automatically generated to remind clinicians that screening was due at a particular well visit. For the ASQ-II, an age-specific portable document format (PDF) was available through a link within the EHR that could be printed and provided to parents to complete. Clinicians could transfer parent responses to the EHR by selecting the appropriate radio buttons conforming to responses on a scoring grid, and an automated scoring algorithm would tally the responses and provide an overall score to minimize errors. For the M-CHAT, an electronic interactive version of the questions was available within the EHR. Similarly, clinicians could transfer parent responses to an automated scoring grid, which would provide an overall score.

To facilitate implementation and improve clinician efficiency as identified in the focus groups, we developed a workflow procedure for the implementation of screening (Figure 1, Table 3). According to the workflow, 15 days prior to scheduled well visits, parents were mailed the age-appropriate screening tools with instructions to complete the tools and bring them to their child’s upcoming well visit. Automated phone calls were made one day prior to scheduled visits to remind parents of the visit and to complete the screening tools. On the day of the visit, front desk staff took completed screeners from parents at check-in. If parents did not bring their pre-mailed screening tool completed, office staff provided parents with available screeners. Parents completed or had assistance from office staff to complete screening tools. Completed screening tools were either provided to clinicians or results were entered into the EHR by office staff prior to clinicians seeing the patients.

We developed a letter of agreement with the county EI agency preceding the study to permit information on referral status to be faxed to EI agency staff (Table 3). The clinic staff maintained a monthly tracking spreadsheet that early intervention agencies would update on a quarterly basis. This spreadsheet included information on the child’s date of assessment, referral date, medical record number, age, the status of enrollment in EI services, and the scheduling and evaluation results of multidisciplinary evaluations (MDE). The EI agency agreed to accept the ASQ-II/M-CHAT results as part of their intake for developmental evaluations. EI staff reported on the status of all referrals: referral intake, completion of referrals, scheduling of MDE, and results of MDE and status of services.

One thousand three hundred ninety-seven eligible children under 31 months old were enrolled and followed for up to 18 months, and 1,184 (84.8%) parents/caregivers completed a developmental screening tool at least once during the study period (Table 4). Most children were male, African American, had mean family incomes of less than $30,000, and had a parent with greater than a high school education (see Additional file 1). There were no differences (p > 0.05) in demographic characteristics by practice site. Comparing the focus group caregiver participants to the screening intervention participants, the screening intervention parents were of slightly higher education levels, but of similar race and ethnicity (Table 1 and Additional file 1). Developmental screening resulted in 348 (24.9%) children being identified with developmental delays (Table 4). Two hundred fifty one children (18.0%) were referred for EI services, and 128 (9.2%) completed an EI referral (Table 4).

Once a parent completed a developmental screening visit, parents and clinicians were asked to complete satisfaction questionnaires, which typically occurred between one and 12 months after the well-child visit. Of the 1,184 who completed a developmental screen at a well visit and were eligible to participate in the survey, 1,016 parents (85.8%) completed the phone survey at the conclusion of the study. Most parents reported no difficulty completing the screens (98.6%), the screens covered important areas of child development (97.6%), and that the developmental screening tools helped them learn about their child’s strengths and challenges (88.3%) (Table 5). Of the 208 Attendings, Nurse Practitioners, and Residents, 123 (59.1%) completed the on-line survey. One-hundred sixteen clinicians (94.3%) felt that developmental assessment was an important part of well-child care (Table 6). Only 67 (54.5%) felt caregivers have a good understanding of typical child development. However, 120 (97.6%) valued the importance of seeking parental input regarding child development. Overall, most parents (98.5%) and clinicians (70.8%) reported satisfaction with developmental screening.