The factor structure, reliability, and construct validity of an abbreviated version of the Revised Dimensions of Temperament Survey (DOTS–R) were evaluated across Black, Hispanic, and White early adolescents. Primary caregivers reported on 5 dimensions of temperament for 4,701 children. Five temperament dimensions were identified via maximum likelihood exploratory factor analysis and were labeled flexibility, general activity level, positive mood, task orientation, and sleep rhythmicity. Multigroup mean and covariance structures analysis provided partial support for strong factorial invariance across these racial/ethnic groups. Mean level comparisons indicated that relative to Hispanics and Blacks, Whites had higher flexibility, greater sleep regularity, and lower activity. They also reported higher positive mood than Blacks. Blacks, relative to Hispanics, had higher flexibility and lower sleep regularity. Construct validity was supported as the 5 temperament dimensions were significantly correlated with externalizing problems and socioemotional competence. This abbreviated version of the DOTS–R could be used across racial/ethnic groups of early adolescents to assess significant dimensions of temperament risk that are associated with mental health and competent (healthy) functioning.

The Revised Dimensions of Temperament Survey (DOTS–R;

To develop a survey measure of these temperament dimensions that could be used across different age groups,

Data from the Healthy Passages Study (

The full-scale DOTS–R has desirable psychometric properties (e.g., high reliability and longitudinal stability, cross-cultural invariance, high heritability, moderate-to-high interrater agreement) and both short- and long-term predictive associations with substance use and mental health (

The DOTS–R has significantly predicted both internalizing and externalizing problems in children and adolescents (

A major focus of this study was the dimensional structure of the 23-item abbreviated DOTS–R used in the Healthy Passages study. The use of confirmatory item-factor analyses with a relatively large number of items and factors has commonly met with difficulties in achieving good model fit using conventional fit indexes and suggested cut points, and this issue can be influenced by larger sample sizes (

An alternative approach to estimating large inventory, multiple-factor models, and in our application with a large sample size, is the exploratory structural equation model (ESEM;

In this study we used ESEM to model the item-factor relationships of the abbreviated DOTS–R. Consistent with prior applications of ESEM (

Given the consistency of the factor structure of the DOTS–R across samples that have varied with regard to age, sex, and cultural group (

We also specified and tested a series of invariance hypotheses (

Healthy Passages is a longitudinal study of a cohort of 5,147 fifth-graders and their parents that explores health behaviors, outcomes, and related risk and protective factors using a multilevel approach (^{2}(2) = 5.78,

All three Healthy Passages research sites used standardized data collection materials and protocols, including training manuals, field manuals, and validation procedures. Both computer-assisted personal interviews (CAPI) and audio computer-assisted interviews (A-CASI) were used to collect data from participants. Institutional review boards at each study site and the Centers for Disease Control and Prevention approved the study. On average, it took about 3 hr for the field interviews to be completed. Primary caregivers were paid $50 and children were given a $20 gift card from a national chain store as reimbursement for completing the interview.

The DOTS–R (

The presence of symptoms of conduct disorder (8 items) and oppositional defiant disorder (10 items) in the past year was assessed by primary caregivers with 18 items adapted from the Diagnostic Interview Schedule for Children Predictive Scales (DPS;

The Social Skills scale from the Social Skills Rating System (

As described previously, initial analyses focused on evaluating the number of factors underlying this 23-item abbreviated version of the DOTS–R. Scree plots, parallel analysis, and findings from maximum likelihood factor analyses were used for this purpose. Then, guided by previous research on specifying, testing, and evaluating hypotheses about invariance relations across groups (

Analyses were conducted using the statistical software program ^{2}) that is asymptotically equivalent to the Yuan–Bentler T2* test statistic (

Evaluation of the specified models was based on multiple criteria that considered statistical, practical, and substantive fit. Following the recommendations of

In making comparisons among nested hierarchical models, it has been suggested that the chi-square difference test often detects inconsequentially small differences in loadings when sample sizes are large (

To determine whether the fit of more restrictive invariance models deteriorates significantly, we also used criteria regarding the magnitude of observed differences suggested by F.

Three methods were used to evaluate the adequacy of the number of factors to represent the DOTS–R items. First, based on an exploratory factor analysis, five eigenvalues exceed 1.0 (the values of these five were 3.85, 3.05, 2.22, 1.63, and 1.22) and accounted for 52% of the variance. Second, parallel analysis was performed, and five factors from the actual data with eigenvalues that were greater than those from random data were retained (

A five-factor representation consistent with the structure identified in the exploratory model was specified and estimated for a standard confirmatory factor analytic model and for an ESEM to evaluate the adequacy of this representation for the full sample. For the ESEM, 25 constraints (five per factor) were imposed on factor loadings that had the lowest estimated value in the full exploratory factor analytic solution. For the confirmatory factor model, the salient loadings (shown in bold in

A summary of the fit statistics to evaluate the sequential invariance hypotheses across groups is shown in

The strong invariance hypothesis, in which both the factor loadings and the intercepts were constrained to equivalence across groups, was weakly supported; the change in CFI (.023) exceeded the criterion established by F.

The range of standardized factor loadings for M4 was also very similar across the three groups and highly similar to those reported for the five-factor model of the full sample in

The internal consistency reliability coefficients (Cronbach’s alphas) for the five temperament dimensions are provided in

The numeric values of these coefficients largely fell within the acceptable .70 target value with the exception of rhythmicity sleep for the Hispanic sample, which was .53. Across dimensions, the average reliability for Hispanics was .66, for Blacks it was .72, and for Whites it was .78. Overall, these

The two factors of externalizing behaviors and socioemotional competence were selected to evaluate the construct validity of the DOTS–R. Consistent with prior theory and research (

This study used ESEMs with a large, multiracial community sample of fifth-graders to examine psychometric characteristics of an abbreviated version of the DOTS–R. A series of analyses were conducted, including an evaluation of the number of factors, the invariance of the factor structure, internal consistency estimates, and construct validity analyses. Findings indicated that the hypothesized five-factor structure of the DOTS–R provided an acceptable representation of the data across the full sample, and largely across all three racial/ethnic groups. For the multigroup comparisons, configural invariance was supported, indicating that the five-factor model provided an equally good representation for each of the three racial/ethnic groups. The weak factorial invariance model was also supported, indicating that the factor loadings could be constrained to equivalence across groups without significant departures in goodness of model fit. The strong factorial invariance model was also partially supported, indicating that the factor loadings and intercepts could be constrained to equivalence across groups without significant departures in goodness of model fit. However, to achieve optimal levels of model fit for the strong factorial invariance model, it was necessary to free five intercepts across groups; thus, there was only partial invariance for the intercepts across the three racial/ethnic groups as the intercepts for 18 items could be constrained to equivalence, but the intercepts for 5 items could not be constrained. This finding suggests some minor differential item functioning for these items across groups, with three of these items from the general activity factor and two from the rhythmicity sleep factor. Nevertheless, the overall factor structure for abbreviated factor structure of the DOTS–R was best represented by a five-factor model and invariant relationships for most parameters (i.e., all factor loadings and 18 of 23 intercepts) were supported across three racial/ethnic groups. This suggests that the underlying factor structure of the abbreviated DOTS–R as rated by primary caregivers was largely equivalent for Black, Hispanic, and White children.

To our knowledge, this is the first study to apply an invariance testing strategy in the form of mean and covariance structure analysis to the abbreviated DOTS–R to examine the comparability of its hypothesized factor structure across different racial/ethnic groups. The overall findings of this study were consistent with prior U.S. and international research (

In addition to tests regarding the number of factors and invariance issues across racial/ethnic groups, several other analyses were completed. Observed mean level analyses indicated that the three racial/ethnic groups did not differ with regard to task orientation. However, White early adolescents were rated by their primary caregivers as having higher levels of behavioral flexibility and sleep rhythmicity than Black or Hispanic early adolescents. They were also rated as having lower levels of general activity than Black early adolescents. Black early adolescents were also rated by their primary caregivers as having greater flexibility and lower sleep rhythmicity than Hispanic early adolescents. Hence, there were some overall differences in mean levels across racial/ethnic groups, especially for flexibility, general activity, and sleep rhythmicity, although effect sizes were typically small. The internal consistency estimates for the temperament dimensions of the abbreviated DOTS–R were somewhat higher for Whites and somewhat lower for Hispanic early adolescents, although the overall levels were highly similar to those reported with the full-scale DOTS–R, with levels appropriate for research purposes.

The construct validity findings for the abbreviated DOTS–R were consistent with prior theorizing and prior empirical findings with regard to similarities of low-to-moderate correlations between temperament scores and externalizing problems and socioemotional competence (

A noteworthy feature of this study was that primary caregivers were used as reporters to minimize potential monomethod reporter effects when investigators want to study relationships between temperament and other variables reported by children. Maternal reports have been used in the temperament literature for infants and children, but less often for children in, or nearing, early adolescence. Extant literature has demonstrated high rates of parent–child agreement for the DOTS–R (

Although findings of this study were largely supportive of the invariance of the abbreviated DOTS–R across three racial/ethnic groups, they should be viewed in the context of study limitations. First, the DOTS–R is a questionnaire (self- or other report) measure, and, as with all report instruments, responses might have been influenced by confounds such as informant bias (halo effects by primary caregivers regarding their children) and biases associated with psychopathological characteristics of the rater (e.g., maternal depression). Behavioral observation data or physiological assessments would enhance the value of temperament assessment and would facilitate further validation of this abbreviated measure. Second, the limited group sizes of some racial/ethnic groups (e.g., Asian Americans, subgroups of Hispanics from South American countries, Cuba, or Puerto Rico vs. our sample of primarily Mexican heritage) precluded systematic testing of weak and strong factorial invariance of the DOTS–R for these groups. Third, the item response options for the DOTS–R contain 4-point options, and it is possible that these data might be equally or better modeled as categorical with an alternative estimator such as robust weighted least squares. The impact of this issue in this application is reduced to some extent by the estimator we chose because it produces robust estimates even if data are nonnormally distributed. Furthermore, other characteristics of the data (e.g., sample size, symmetric vs. asymmetric data distribution) might affect the relative value of modeling data such as these as continuous versus categorical variables. Fourth, this was a study of reports by primary caregivers regarding their children. Although prior research has indicated high levels of interrater agreement between primary caregivers and their offspring with the DOTS–R (

Nevertheless, this study also had several strengths, including the large sample size and data from three racial/ethnic groups, which facilitated our efforts to address issues related to measurement invariance that are critical for making group comparisons. Further research on this issue should include similar analyses for other racial/ethnic groups, such as Asian Americans, the collection of validity data via other methods of assessment (e.g., physiological or behavioral assessments), and longitudinal associations between the DOTS–R dimensions and important health-related behaviors such as substance use, depression, and delinquency.

Funding

The Healthy Passages study was funded by the Centers for Disease Control and Prevention (Cooperative Agreements U48DP000046, U48DP000057, and U48DP000056). The findings and conclusions in this report are those of the author (s) and do not necessarily represent the official position of the Centers for Disease Control and Prevention.

Model fit of exploratory factor models ranging from one to six factors.

No. Factors | ^{2} | CFI | RMSEA | SRMR | |
---|---|---|---|---|---|

One-factor | 9,576.51 | 230 | .341 | .093 | .122 |

Two-factor | 5,889.26 | 208 | .599 | .076 | .083 |

Three-factor | 2,940.12 | 187 | .806 | .056 | .050 |

Four-factor | 1,697.60 | 167 | .892 | .044 | .032 |

Five-factor | 931.13 | 148 | .945 | .034 | .021 |

Six-factor | 617.11 | 130 | .966 | .028 | .017 |

^{2} = maximum likelihood chi-square test statistic; CFI = comparative fit index; RMSEA = root mean square error of approximation; SRMR = standardized root mean square residual.

Factor loadings and factor intercorrelations of five-factor exploratory model.

DOTS–R: Item description | Flexibility | Activity level | Rhythmicity sleep | Task orientation | Positive mood |
---|---|---|---|---|---|

1. Time to adjust to new thing in home | . | −.09 | .02 | −.09 | .05 |

7. Time to adjust to new schedules | . | −.19 | .10 | −.03 | .00 |

19. When things out of place, difficulty get used to | . | .03 | −.06 | .02 | −.02 |

21. Resists changes in routines | . | −.01 | −.08 | .04 | .01 |

2. Can’t stay still for long | .02 | . | .01 | −.01 | −.02 |

6. Gets restless if has to stay in one place | −.04 | . | −.02 | .02 | −.05 |

9. Stays still for long periods | .22 | . | −.02 | −.32 | −.01 |

12. Gets fidgety | −.09 | .00 | −.01 | .01 | |

15. Never seems to stop moving | −.03 | . | .05 | −.04 | .05 |

3. Wakes up at different times | .15 | −.29 | . | −.15 | −.03 |

13. Gets same amount of sleep each night | .02 | −.01 | . | .03 | .08 |

16. Gets sleepy at same time every night | −.06 | −.02 | . | −.02 | .10 |

17. When away from home, wakes up at same time | .02 | .02 | . | .05 | −.03 |

18. No matter goes to sleep, wakes up at same time | −.08 | .02 | . | .08 | −.05 |

4. Once involved, cannot distract | .02 | .04 | −.01 | . | −.03 |

5. Persists at tasks until finished | .10 | −.02 | .04 | . | .09 |

8. Something else occurring will not disturb focus | −.10 | .08 | −.04 | . | −.03 |

10. Other things happening will not distract | −.07 | .09 | −.01 | . | −.02 |

11. Once child initiates a task, stays with it | .14 | −.02 | .08 | . | .09 |

14. Smiles often | .05 | .11 | .10 | .05 | . |

20. Generally cheerful | .01 | −.02 | .06 | .02 | . |

22. Laughs frequently | −.01 | .04 | −.04 | −.02 | . |

23. Happy | −.03 | −.07 | −.05 | −.01 | . |

Flexibility | 1.0 | ||||

Activity level | −.45 | 1.0 | |||

Rhythmicity sleep | .07 | −.14 | 1.0 | ||

Task orientation | −.21 | −.06 | .21 | 1.0 | |

Positive mood | .19 | −.05 | .32 | .18 | 1.0 |

Fit statistics for exploratory, confirmatory, and exploratory structural equation models for full sample.

Model(M) | ^{2} | CFI | RMSEA | SRMR | Model comparison | ΔCFI | |
---|---|---|---|---|---|---|---|

M1: Exploratory model | 931.13 | 148 | .945 | .034 | .021 | — | — |

M2: Confirmatory model | 4,461.55 | 267 | .742 | .058 | .075 | 2 vs. 1 | .203 |

M3: ESEM | 985.85 | 154 | .941 | .034 | .024 | 3 vs. 1 | .003 |

M4: ESEM-2 correlated errors | 711.26 | 152 | .960 | .028 | .021 | — | — |

^{2} = robust maximum likelihood chi-square test statistic; CFI = comparative fit index; RMSEA = root mean square error of approximation; SRMR = standardized root mean square residual; ESEM = exploratory structural equation model.

Testing for invariance of five-factor model across White (

Model (M) | ^{2} | CFI | RMSEA | SRMR | Model comparison | ΔCFI | |
---|---|---|---|---|---|---|---|

Independent group | |||||||

Hispanic | 499.65 | 156 | .938 | .036 | .029 | — | — |

Black | 421.20 | 156 | .956 | .031 | .042 | — | — |

White | 431.45 | 156 | .951 | .038 | .061 | — | — |

M1E: Configural invariance | 1,352.34 | 468 | .949 | .035 | .044 | — | — |

M2: Weak factor invariance (FI) | 1,498.69 | 637 | .950 | .029 | .039 | 2 vs. 1 | .005 |

M3: Strong FI | 1,945.38 | 673 | .927 | .035 | .042 | 3 vs. 2 | .023 |

M4: Strong FI + 5 intercepts freed | 1,689.30 | 663 | .941 | .031 | .039 | 4 vs. 2 | .009 |

^{2} = robust maximum likelihood chi-square test statistic; CFI = comparative fit index; RMSEA = root mean square error of approximation; SRMR = standardized root mean square residual.

Observed mean level comparisons of temperament across racial and ethnic groups.

DOTS–R factors | Hispanic (H) | Black (B) | White (W) | F-statistic | Post-hoc tests and effect sizes |
---|---|---|---|---|---|

Flexibility | 11.27 | 12.25 | 12.86 | 123.04 | W > B, H (.12, .55); B > H (.34) |

General activity | 12.70 | 12.30 | 11.34 | 46.24 | H > B, W (.10, .35); B >W (.25) |

Sleep rhythmicity | 15.35 | 14.81 | 15.97 | 49.79 | W > H, B (.17, .19); H > B (.36) |

Positive mood | 14.55 | 14.52 | 14.72 | 4.27 | W > B (.10) |

Task orientation | 13.00 | 12.90 | 12.72 | 2.83 (ns) | ns |

Reliability coefficients (αs) for five temperament factors across racial and ethnic groups.

Temperament dimension | Hispanic | Black | White | Total | Original sample |
---|---|---|---|---|---|

Flexibility | .67 | .67 | .78 | .71 | .62 |

General activity | .68 | .80 | .85 | .78 | .75 |

Sleep rhythmicity | .53 | .69 | .73 | .65 | .69 |

Positive mood | .76 | .74 | .84 | .77 | .80 |

Task orientation | .65 | .68 | .71 | .67 | .70 |

reliability data (Cronbach’s alpha) for child sample from

Correlations between temperament factors, externalizing behaviors, and socioemotional competence.

Hispanic | Black | White | |
---|---|---|---|

Externalizing | |||

Flexibility | −.03 | −.28 | −.20 |

Activity level | .20 | .35 | .26 |

Rhythmicity sleep | −.15 | −.12 | −.21 |

Task orientation | −.15 | −.26 | −.20 |

Positive mood | −.22 | −.24 | −.40 |

Socioemotional competence | |||

Flexibility | .29 | .30 | .36 |

Activity level | −.15 | −.28 | −.27 |

Rhythmicity sleep | .20 | .24 | .25 |

Task orientation | .07 | .36 | .21 |

Positive mood | .26 | .29 | .46 |