The American Academy of Pediatrics (AAP) recommends that all children be screened for autism spectrum disorder (ASD) in early childhood (Johnson and Myers 2007). Indeed, early identification of children with ASD is critical as it facilitates referral for early intervention services, which have been shown to improve outcomes (Connor 1998; Rogers 1998) and reduce the long term cost of care (Jacobson et al. 1998; Peters-Scheffer et al. 2012). However, there is little information about the performance of screeners for ASD in young children across demographic variables or by child characteristics.

Both the Social Communication Questionnaire (SCQ; Rutter et al. 2003) and Social Responsiveness Scale (SRS; Constantino and Gruber 2005) are questionnaires designed for detecting risk for ASD. The SCQ was originally designed as a screening tool for children 4 years of age or older enrolled in epidemiological research or for studies comparing individuals with ASD and other clinical groups (Berument et al. 1999; Rutter et al. 2003), and it is now used clinically. It was derived from the Autism Diagnostic Interview-Revised (ADI-R; Lord and Rutter 1994), a parent interview used to diagnose ASD in combination with other measures. The SCQ is strongly correlated with the ADI-R (r = .71, p < .001; Berument et al. 1999). The SRS is often used to identify the presence and severity of ASD-related behaviors (Constantino and Gruber 2005; Constantino et al. 2003a, 2004; Constantino and Todd 2005; Duvall et al. 2007; Frazier et al. 2014; Virkud et al. 2009).

Both the SCQ and SRS have good overall psychometric properties. The SCQ has high internal consistency (Cronbach’s alpha = 0.87; Rutter et al. 2003). It also has good discriminative validity when distinguishing between children with ASD and non-ASD diagnoses at all intelligence quotient levels, but particularly when used with children 4 years of age or older and children from clinical populations (Berument et al. 1999). The sensitivity of the SCQ, or ability to correctly classify children with ASD, is about 96% in samples of children without intellectual disability. The specificity of the SCQ, or ability to correctly classify children without ASD, is about 80% in samples of children without intellectual disability (Rutter et al. 2003). Specificity of the SCQ drops considerably when used with children who have intellectual impairments (Berument et al. 1999).

The SRS has high internal consistency (Cronbach’s alpha = 0.76), and has good discriminant validity (Constantino 2002). Sensitivity and specificity are 0.85 and 0.75, respectively, when the SRS total score of 75 is used as the cutoff (Bölte et al. 2011; Constantino 2002). Previous research also suggests that the SRS could be a cost-effective assessment for measuring ASD symptoms in school and clinical settings. Specifically, the SRS has been found to be useful at differentiating between children with ASD and those without ASD (Cholemkery et al. 2014; Wang et al. 2012). The SRS has also been used to track the severity of ASD symptoms over time (Constantino et al. 2009; Ho et al. 2005), and it has been found to be useful for assessing response to interventions (Constantino et al. 2009, 2007; Constantino and Todd 2003; Pine et al. 2006). However, SRS scores are known to increase in the presence of some child behavior challenges, such as Attention Deficit/Hyperactivity Disorder (Reiersen et al. 2007), and mood disorders (Pine et al. 2008). Further, child behavior problems as measured by the Child Behavior Checklist (CBCL; Achenbach and Rescorla 2001) account for a large proportion of the variance in SRS scores (Constantino et al. 2003b) suggesting that behavior problems may impact screening performance.

Although the SCQ performs well overall, there are variations in performance depending on child age and the cutoff score used. For instance, the SCQ has been found to be less effective when the recommended cutoff of ≥ 15 is used to differentiate children younger than 4 years of age with and without ASD (Allen et al. 2007; Corsello et al. 2007; Eaves et al. 2006; Snow and Lecavalier 2008; Wiggins et al. 2007), and may have a high false positive rate (Oosterling et al. 2010a). The performance of the SCQ seems to improve when a cutoff score of ≥ 11 or ≥ 13 is used in samples of young children (Allen et al. 2007; Corsello et al. 2007; Oosterling et al. 2010b; Snow and Lecavalier 2008; Wiggins et al. 2007). Because using a cut off of ≥ 11 has been shown to maximize sensitivity and specificity in younger children (Wiggins et al. 2007), this lower cut off has often been adopted by large public health research studies (Schendel et al. 2012; Wiggins et al. 2015). Also, most studies report that the SCQ shows an acceptable sensitivity (≈80%) and less than optimal specificity (≈60%) when used to distinguish children with ASD and other developmental disabilities (Allen et al. 2007; Eaves et al. 2006; Snow and Lecavalier 2008; Witwer and Lecavalier 2007).

Likewise, SRS scores are associated with variations in performance across child factors. Specifically, higher SRS scores suggest more risk for ASD and are associated with increased non-ASD behavior problems, older age, and more impaired language, and cognitive skills in children with ASD (Hus et al. 2013). Further, in a German sample, SRS scores were higher when used with children with conduct disorder and attention-deficit/hyperactivity disorder (Bölte et al. 2008). These findings suggest that behavior problems, increased age, and expressive language or cognitive level might reduce the specificity for these instruments (Charman et al. 2007). However, it is not known whether the SRS shows variability in performance by age as does the SCQ (Wiggins et al. 2007).

The current study expands upon past research by exploring the psychometric properties of the SCQ and the SRS in young children enrolled in the Study to Explore Early Development (SEED). Specifically, the sensitivity and specificity of the SCQ and SRS were estimated in this large community-based sample, and the relationships of SCQ and SRS to child and family demographic factors assessed. Child variables included several behavior problems as measured by the Child Behavior Checklist (Achenbach and Rescorla 2001) and developmental levels as measured by the Mullen Scales of Early Learning (Mullen 1995). Family characteristics included household income, maternal education, and maternal race and ethnicity. Based on research that has found variability in the performance for the SCQ and SRS, described above, as well as variability in the performance of other autism screeners (e.g., Scarpa et al. 2013) we expect to find that both child characteristics and family characteristics will impact screening performance of the SCQ and SRS.

This analysis found important differences in how the SRS and SCQ performed across several child and family characteristics. Specifically, both the SCQ and SRS instruments became less specific and more sensitive as maternal education and household income decreased. These relationships occurred at all cutoffs used for the SCQ. This adds to a growing body of literature that shows that ASD screener scores are associated with a number of child characteristics (Constantino et al. 2003b; Oosterling et al. 2010a; Pine et al. 2008; Reiersen et al. 2007) and that they may perform poorly in those from lower socioeconomic status (SES) backgrounds (Scarpa et al. 2013). Importantly, in the case of the SCQ, higher cutoffs resulted in somewhat higher specificity, but did not completely mitigate this problem. Specificity also dropped for children of non-white mothers; this was particularly pronounced for children of African American mothers as indicated by the non-overlapping CIs.

There were also noticeable drops in SCQ and SRS’s specificity for children with developmental or behavioral challenges as measured by the MSEL and CBCL as indicated by the non-overlapping CIs in Tables 3 and 4. That is, for every MSEL and CBCL subscale/composite score, the SRS and the SCQ showed reduced specificity for children who scored below average or in the clinical range relative to those who scored average or above. When stratified on MSEL scales, the SCQ with a cutoff of 15 had somewhat better performance in those children with below average development, with most sensitivities and specificities in the 70s. However, there was still an average drop of 19.5 percentage points in specificity relative to children without developmental challenges. By contrast, the SRS had specificities in the upper 40s to mid-50s and an average drop of 35 percentage points between those with and without developmental challenges. In the presence of some developmental challenge, both instruments seemed to capture those challenges as indicating ASD. Also, even though lower cutoffs on the SCQ have been shown to maximize sensitivity and specificity overall, we found that specificity suffered more at lower cutoffs. Those using the SCQ with lower cutoffs should, therefore, expect higher false positive rates in those with developmental concerns.

Similarly, when stratified on CBCL subscales, there were substantial drops in specificity and gains in sensitivity for both the SRS and for all cutoffs of the SCQ. The SRS showed the largest drop in specificity with most values in the 20s and 30s for children with CBCL scores in the clinical range (there were no overlapping CIs in this analysis). The average drop in specificity between the non-clinical and clinical groups was approximately 50 percentage points. The SCQ also had significant drops. Like the results stratified on the MSEL, the average drops in specificity were smaller for the cutoff of 15, and greater for cutoffs of 13 and 11. As expected, sensitivity tended to be lower and specificity was higher for children whose CBCL scores were in the non-clinical range. Thus, these screeners may be less able to identify children with ASD who score in the non-clinical range.

Taken together, these findings suggest that the SRS and SCQ effectively detect developmental and behavioral challenges, but may not reliably differentiate them from ASD. Screening individuals from minority and lower socioeconomic backgrounds is also problematic, as these screeners are likely to have unsatisfactory rates of false positives for these groups. If these screeners are routinely used with large populations of minority and low-income families, the likely high false positive rate could tax limited resources. Similarly, if these screeners are used for research, high false positive rates in these subgroups could skew results, or necessitate additional testing to exclude children who do not have ASD, again diverting scarce resources. This problem is particularly interesting given the known disparities in ASD identification for many minority groups (Autism and Developmental Disabilities Monitoring Network 2016). One could argue that higher false-positive rates are acceptable when used with under identified groups so as to increase detection of the condition. However, that we continue to observe these disparities suggests that additional research is needed to determine how to better identify ASD in diverse populations.

These findings do not mean that the SRS and SCQ are ineffective screeners when used in the general population. Rather, these data highlight the challenges of successfully identifying young children at risk for ASD from low SES backgrounds, as well as those with developmental or behavioral challenges. Indeed, these analyses revealed that both the SCQ and SRS performed adequately for the combined sample and equally well for both boys and girls overall. However, refining these screeners to be more effective, regardless of the child’s behavioral presentation and across cultures, could help reduce the impact of false positives.

Further, when used in clinical settings, additional testing is needed to inform the differential diagnosis. Likewise, researchers may want to avoid relying on screening results alone to confirm diagnosis, especially when cognitive impairment or co-occurring behavioral issues are present. Indeed, there has been a longstanding awareness that screening and monitoring for ASD is only the first step in the diagnostic process (Filipek et al. 1999). These results further highlight the need for a robust clinical evaluation following a positive screening that assesses multiple domains of functioning and includes a child observation and parent interview (Huerta and Lord 2012). Moreover, the impact of cultural variations and clinician interpretation on the clinical findings should be studied further given that the clinician’s judgement appears to be formed by more than just core ASD symptoms (Wiggins et al. 2017).

Although both the SCQ and SRS are validated ASD screeners, research and clinical programs that rely on these measures should recognize the potential for false positives. Specifically, high false positive rates could lead to misclassification if additional diagnostic testing is not performed. These challenges could be particularly problematic for community settings that have high proportions of minority families or families with low income or low maternal education. Our findings underscore the need to refine existing measures or develop new instruments to ensure they perform well across all groups.