IntroductionAutism spectrum disorder (ASD) is an early-emerging neurodevelopmental disorder defined by deficits in social communication and patterns of restricted or repetitive behaviors or interests (American Psychiatric Association, 2013). Although the average age of ASD diagnosis in the U.S. is 4–5 years (CDC, 2014; Shattuck et al., 2009), many parents report social or communication concerns as early as age 6 months (Bolton, Golding, Emond, & Steer, 2012). Recent estimates place ASD prevalence at 1.5% of the population (Christensen et al., 2016), but both age and rates of ASD diagnoses vary tremendously across different demographic groups. Perhaps best documented are sex differences, with a male-to-female ratio of 4.5:1 in ASD prevalence (Christensen et al., 2016), and an older mean age of diagnosis among girls compared to boys (Shattuck et al., 2009). Similarly, rates of ASD differ across racial and ethnic groups, with African American and Latino children being less likely to receive an ASD diagnosis than White children (Christensen et al., 2016). To date, such discrepancies have been attributed to true differences in ASD prevalence across subgroups (Werling & Geschwind, 2013), disparities in access to appropriate evaluation services (Zuckerman et al., 2014), and phenotypic differences that lead to under-identification among some children (Hiller, Young, & Weber, 2014; Kopp & Gillberg, 2011).
Like many psychiatric diagnoses, ASD is defined by a constellation of behavioral symptoms observed over the course of development. Initial diagnosis and subsequent evaluations of skills and deficits therefore rely upon observations and reports from multiple informants. Gold standard ASD evaluation integrates developmental information from caregivers with thorough observational assessments by trained clinicians (Ozonoff, Goodlin-Jones, & Solomon, 2005). Underlying this process is an understanding that caregivers and clinicians provide both unique and overlapping information necessary for understanding individuals’ unique profiles of strengths and difficulties (Risi et al., 2006). Although perspectives across reporters may match closely for some families, others may experience more striking differences between perspectives—for example, if a clinician perceives a difficulty that a family views as normative, or vice versa. Such divergences between reporters’ perspectives may place at least some families at risk for detrimental consequences across the lifespan. Perhaps most importantly, families for whom there is poorer agreement between reporter perspectives may be at elevated risk for (1) inaccurate diagnoses and inappropriate service provision, (2) poorer family experiences with healthcare professionals, and (3) underrepresentation in ASD research. Research in this area has been extremely limited to date, with very little exploration as to how convergence or divergence between caregivers’ and clinicians’ perspectives might influence individual and family outcomes. Thus, our understanding of this issue is far from complete.
Since demographic factors relate to ASD diagnosis, they may also relate to perceptions and interpretations of ASD-related behaviors more broadly, with some families more likely than others to experience significant divergences in perspectives. Two common and well-regarded measures of caregiver and clinician report of ASD symptoms are the Autism Diagnostic Interview–Revised (ADI-R; Lord, Rutter, & Le Couteur, 1994) and the Autism Diagnostic Observation Schedule (ADOS; Lord, Rutter, DiLavore, & Risi, 2003), respectively. Early work with these measures indicated fair agreement for both ASD symptom levels (Risi et al., 2006) and diagnoses (de Bildt et al., 2004), with correlations between caregiver and clinician reports in the range of r=.6–.7 (LeCouteur et al., 2008; Risi et al., 2006). However, studies also reveal marked variability in agreement across research samples (Gray et al., 2008; Ventola et al., 2006), from a low of r=.28 to a high of r=.95 based on sample characteristics (Risi et al., 2006). Child characteristics such as cognitive level appear to matter, as studies have suggested lower correlations between ADOS and ADI-R scores, and poorer sensitivity of clinical cut-offs on some measures, in samples with intellectual disability than in samples with average cognitive ability (Havdahl et al., 2016; Risi et al., 2006). Behaviorally, significant ADHD symptoms appear to increase the age at which children are diagnosed with ASD (Yee & Millichap, 2015). More generally, possible demographic moderators of reporter agreement include socioeconomic status and child race/ethnicity—both of which are associated with ability and desire to pursue ASD evaluations and the age at which an ASD diagnosis is received (Christensen et al., 2016; Emerson, Morrell, & Neece, 2016; Mazurek et al., 2014; Zuckerman et al., 2014).
Following from this discussion, our goal was to explore potential child- and family-level moderators of agreement between caregiver and clinician reports of ASD symptoms. We operationalize caregiver report with the Autism Diagnostic Interview—Revised (ADI-R; Lord et al., 1994), and clinician report with the Autism Diagnostic Observation Schedule (ADOS; Lord et al., 2003). Although diagnostic processes in the clinical and research communities vary and may not routinely include both measures, these instruments are standardized, well-researched, and frequently used in both clinical and research settings. Thus, they represent a solid foundation from which to explore caregiver-clinician agreement.
DiscussionOur study is among the first to examine correspondence and divergence between caregiver and clinician reports of ASD symptoms. We identify several child and family characteristics that moderate agreement between these reporters’ perspectives. For both younger (9 years and under) and older (10 years and older) children, caregivers and clinicians showed significant but modest correspondences between their reports of ASD symptoms, with caregiver reports accounting for approximately 15% of variance in clinician assessments. Caregiver-clinician agreement was moderated by a number of child and family factors that increased shared variance to approximately 40%. Moderating effects of child IQ, adaptive behavior, and behavioral difficulties were observed in both groups, such that children with lower IQs, lower adaptive behavior, or fewer behavioral difficulties had better correspondence between reporters. From a practical standpoint, children with the converse–higher IQs, stronger adaptive skills, and more behavioral concerns–were more likely to have poor agreement between caregiver reports and clinician assessments, and consequently may be at elevated risk for detrimental outcomes. To the extent that our findings extend beyond symptom description to include ASD diagnostic outcomes, a point to which we return below, these findings are consistent with concerns that current assessment practices might miss “high-functioning” individuals with ASD (those with strong cognitive and adaptive skills) as well as those with comorbid symptoms such as anxiety, inattention, or hyperactivity that might complicate differential diagnosis (Lai & Baron-Cohen, 2015; Yee & Millichap, 2015).
Beyond shared moderators, other factors were unique to our two age groups. For younger children, African American race was associated with lower agreement—an effect that remained over-and-above other moderators. Previous literature documents that African American children eventually diagnosed with ASD receive their diagnosis an average of 18 months later than White children (Mandell, Listerud, Levy, & Pinto-Martin, 2002), undergo more visits with clinicians before receiving a diagnosis (Mandell, Ittenbach, Levy, & Pinto-Martin, 2007), and more often receive a different diagnosis (e.g., ADHD) prior to ASD (Mandel et al., 2007). Our finding of poorer symptom agreement between caregivers and clinicians suggests one mechanism through which these discrepancies arise, placing African American children with ASD at a disadvantage for identifying areas of need and obtaining appropriate service referrals. With respect to initial diagnosis, it is difficult to overstate the implications of these findings—overwhelming evidence underscores the critical importance of early and intensive intervention for children with ASD (Dawson et al., 2010; Rogers et al., 2014), and such delays in diagnosis (and thus ASD-specific care) have meaningful effects on trajectories and ultimate outcomes of children’s social, communication, and cognitive skills (Dawson et al., 2010; Dawson et al., 2012).
A different set of family factors emerged as significant for older children and adolescents, perhaps suggesting an age-related shift in relative effects of different moderators. In our older subgroup, poorer agreement was associated with lower family income. This may reflect socioeconomic influences on parents’ perspectives on behaviors suggestive of ASD. For instance, although speculative, families with fewer economic resources might have limited access to routine medical care that would otherwise shape their expectations for their child’s development and thus their interpretation of behavior. Regardless of the cause, findings related to family income and child race are concerning, as they likely add a layer of complexity to disparities in accessing specialized evaluation, research, and support services for ASD, even at later ages.
We also observed an effect of fathers’ social difficulties in our older group, such that better agreement emerged when fathers were more competent socially, but we did not observe a comparable effect for mothers. Interpretation of this effect is tentative without knowledge of which parent provided ADI-R data; in some families, mothers reported on both child and father characteristics, with potential common reporter variance. Nonetheless, findings regarding fathers’ social difficulties present an interesting parallel to recent work revealing that fathers—but not mothers—underreport their own broader phenotype traits (Sasson, Faso, Parlier, Daniels, & Piven, 2014). Such findings highlight the importance of incorporating multiple sources of information across home and school settings (e.g., teachers) into diagnostic decisions, as each reporter’s social experiences and skills provide a unique interpretive lens for a child’s behavior.
Child age, sex, and birth order did not moderate agreement between reporters. These null findings are surprising, particularly regarding sex. Since its recognition (e.g., Kanner, 1943), ASD has been diagnosed more frequently in boys than girls, leading some researchers to suggest that current referral and diagnostic processes may overlook girls with significant symptoms (Lai, Baron-Cohen, 2015), perhaps due to sex-specific ASD phenotypes (Kopp & Gillberg, 2011), patterns of comorbidity (Hiller et al., 2014), molecular genetic substrates (Halladay et al., 2015), or behavioral expectations (Shattuck et al., 2009). Our current findings provide no evidence of an influence of sex in caregiver-clinician agreement.
LimitationsWe suspect that null findings, particularly with regard to biological sex, are due in part to the nature of our sample. By design, the final SSC sample included only children/adolescents who met strict and rigorous research inclusion standards, meeting ASD diagnostic thresholds on both the ADOS and the ADI-R. This approach yields a sample with high confidence of true ASD, but also introduces a potential floor effect because the range of ASD severity is somewhat truncated. As a result, we likely underestimate moderating influences of some variables. Our findings are conservative in that sense, and a broader sample including children without an established ASD diagnosis might reveal additional moderators.
Similarly, because of sample characteristics, our findings cannot speak directly to factors affecting initial diagnosis of ASD, given that children with wider discrepancies between reporters’ perspectives are less likely to have met the research inclusion diagnostic thresholds and consequently would have been excluded from the sample. Our analyses are best interpreted as relating to variability in symptom description within a sample that had strong agreement on child diagnosis. The implications for diagnostic outcomes are more speculative and will await exploration in additional, community-based samples where individuals might meet diagnostic criteria on one but not both measures. It is also the case that, although frequently used in research settings, the ADI-R and the ADOS are not used universally in clinical settings, due in part to constraints on financial resources, training commitments, and administration time. Thus, we cannot determine whether our results generalize to other caregiver and clinician assessment tools that are more common to clinical settings, particularly those which vary by format (e.g., questionnaire vs. interview, written vs. verbal) or length.
The simplex nature of our sample may also affect generalizability of conclusions. Consistent with the goals of the SSC project more broadly (Fischbach & Lord, 2010), families in the SSC dataset have only one child with ASD, presumed to have ASD of de novo genetic etiology. Moderating factors may differ in multiplex families (those in which a second child has ASD), as parents’ familiarity with ASD or family history of ASD might alter their perspective of their child’s behavior and development, and so alter relative contributions of moderators. Future research with multiplex families will be particularly interesting given phenotypic differences in parents’ social and communication skills across simplex vs. multiplex families (e.g., Gerdts, Bernier, Dawson, & Estes, 2013).
Finally, we must consider possible effects of measurement issues on our findings. Divergence between caregiver and clinician reports could reflect differences in perception (as we discuss here), true variability in children’s behavior across settings, or a combination of both, and our current analyses cannot differentiate between these possibilities. It may be that some children display greater variability in behavior from one setting to another, yielding caregiver and clinician reports that differ but are both accurate. This cannot be determined based on available data, although we emphasize that both child and parental characteristics appear to contribute to these differences. Similarly, although we use ADI-R scores for caregiver reports in this study, semi-structured interviews filter caregiver comments through clinician’s interpretations. Finally, scoring procedures on both the ADOS and the ADI-R yield scores of 0, 1, or 2 for the majority of items, which are then summed according to empirically derived algorithms (Gotham et al., 2007). Although higher algorithm scores are frequently interpreted as indicating ASD severity, correspondence between scores and ASD severity is not absolute, as higher totals may reflect a broader range of ASD symptoms rather than severity of symptoms. Our findings must be interpreted with this caveat in mind.
ImplicationsBecause our data were derived from a research diagnostic process, the most proximal implication is for scientific knowledge of ASD. Families for whom reporters diverge on ASD symptoms may be less likely to meet study inclusion criteria and qualify for research studies, and consequently may be poorly represented in the research literature. Many studies require participants to meet symptom thresholds on both caregiver-report and observational measures, and thus caregivers and clinicians must both indicate sufficient symptoms for study entry. If some subgroups within the population experience wider divergence in reporter perspective, they may be excluded systematically from research that could provide direct benefits to them, and provide more comprehensive understanding of ASD etiology and experience.
Extrapolating tentatively from our data into clinical diagnostic and referral processes, significant disagreement between caregivers and clinicians could promote a number of detrimental effects. Most immediately, marked disagreement in symptom reports at an initial evaluation could result in inaccurate diagnosis. For instance, a clinician may fail to recognize symptoms that are apparent to a caregiver, and may delay appropriate evaluation and supports (Zuckerman, Lindly, & Sinche, 2015). Even years after initial diagnosis, vastly different views of children’s skill deficits and strengths across caregivers and clinicians could result in inappropriate or inadequate supports and services that fail to meet individuals’ medical, educational, or behavioral needs.
More subtly, poor caregiver-clinician agreement may influence families’ experiences with healthcare professionals and attitude toward supports throughout development. Limited existing research addressing discrepancies between reporters suggests that clinicians might prioritize their own ADOS observations over caregiver reports (Risi et al., 2006). Evaluation for ASD is inherently stressful for families (Crane, Chester, Goddard, Henry, & Hill, 2015), and stress is likely exacerbated when caregivers and clinicians disagree about the presence or severity of ASD symptoms or proposed supports/interventions. Consistent with this, qualitative evidence reveals that pediatrician invalidation of parental concerns related to initial ASD diagnosis constitutes a marked source of stress and may affect parent mental health and trust in the provider (Zuckerman et al., 2014). Furthermore, parents’ experiences during the diagnostic process and satisfaction with its outcome likely influence their reception of the ASD diagnosis and their willingness to pursue treatment recommendations (Reed & Osborne, 2012).
Future DirectionsFollowing from this discussion, we highlight a number of important avenues for further examination. First, other potential moderators such as medical comorbidities or known genetic events should be considered, as families with ongoing medical or genetic issues may monitor and interpret their child’s behavior differently from those without such awareness. In addition, caregiver characteristics such as mental health concerns (e.g., depression, anxiety), language proficiency, familiarity with child development, or comfort interacting with medical personnel more broadly might influence how caregivers perceive and describe strengths and concerns related to ASD. Recent research suggests that language match between parents and children may be an important consideration as well (Vanegas, Magaña, Morales, & McNamara, 2016), as parents who have a different primary language from their child (e.g., parents for whom Spanish is primary but whose children speak English as their primary language) may interpret or report their child’s social and communication skills differently, tending to report greater skill and fewer deficits compared to clinician observation (Vanegas et al., 2016). Findings such as this underscore the need to consider individual differences among family members in addition to family-level factors, as well as to consider how these influence instrument validity (Magaña & Smith, 2013; Vanegas et al., 2016).
From a practical standpoint, critical questions remain regarding the degree to which moderators of caregiver-clinician agreement influence referral and diagnosis, and alter ongoing care and treatment over the course of development. Despite their potential importance, these questions have not been addressed directly in the literature. Yet, for families, agreement with their healthcare providers—and variables affecting that agreement—may be critical to their reception of and adjustment to an ASD diagnosis, motivation to pursue recommended intervention services, and ability to find a medical home for their child. If agreement is systematically poorer for some families, they may be at greater risk of negative outcomes in these important areas. Identifying factors that predict such outcomes could help clinicians and researchers more easily identify families at risk and adjust their clinical approach accordingly. For example, clinicians might shift the relative weight they attribute to a family’s report of their child’s behavior versus their own observations, engage in additional conversation about the nature and course of ASD, prioritize transparency in their assessments (e.g., having parents observe assessments with a professional who can explain procedures in the moment), or obtain information from additional observers (e.g., teachers, coaches). Similarly, researchers might consider whether their inclusion protocols bias their samples against families with characteristics identified here, such as low household income or African American ethnicity, and develop strategies to mitigate inequities. Such strategies have the potential to improve clinical outcomes as well as representativeness in research, two critical goals as we continue to enrich our understanding of causes, correlates, and outcomes related to ASD.