The methods for this study have been previously published and they are briefly summarized here [8]. A self-report cross-sectional survey including the CDC 4-item HDCM and the 5-item HDSM was mailed to 4183 patients with self-reported arthritis in the fall of 2002. The participants are drawn from two ongoing cohorts: the North Carolina Family Medicine Resource Network (NC-FM-RN) and the musculoskeletal database (MSK). The original study was approved by the University of North Carolina Institutional Review Board. This secondary analysis was approved by the Columbia University Medical Center Institutional Review Board.

The NC-FM-RN is a practice-based network devoted to research on chronic diseases in primary care [12]. At the time of enrollment, participants confirmed their approval to be contacted for future studies. Participants were selected for this study if they self-reported osteoarthritis, rheumatoid arthritis, fibromyalgia, or reported any symptoms of pain, aching, or stiffness in or around a joint in the past thirty days. Out of the 4760 participants enrolled in the NC-FM-RN, 2182 were selected for this study based on these criteria. The MSK database enrolls consecutive adult patients from rheumatology (both academic and community clinics) and orthopedics. Participants in the MSK database can consent to be contacted for future studies. Participants (n = 2001) with osteoarthritis, rheumatoid arthritis and fibromyalgia were selected.

A total of 1820 participants completed surveys (1139 from the NC-FM-RN and 681 from the MSK). Details about the non-respondents and this response rate are published; a total of 631 participants were removed from the denominators of each sample due to incorrect addresses (n = 584) and deceased participants (n = 47) [8]. The response rate was 51 % when combining the completed surveys from both cohorts and this corrected denominator.

The HDCM includes: 1) Would you say that in general your health is :[five responses ranging from excellent to poor], 2) Now thinking about your physical health, which includes physical illness and injury, for how many days during the past 30 days was your physical health not good? [the number of days in the past 30], 3) Now thinking about your mental health, which includes stress, depression, and problems with emotions, for how many days during the past 30 days was your mental health not good? [the number of days in the past 30], 4) During the past 30 days for about how many days did poor physical or mental health keep you from doing your usual activities, such as self-care, work or recreation? [the number of days in the past 30], [From http://www.cdc.gov/hrqol/hrqol14_measure.htm; Access date 02/15/16] The HDSM asks: During the past 30 days, for about how many days 1) did PAIN make it hard for you to do your usual activities, such as self-care, work, or recreation?, 2) have your felt SAD, BLUE, or DEPRESSED?, 3) have you felt WORRIED, TENSE, or ANXIOUS?, 4) have your felt you did NOT get ENOUGH REST or SLEEP?, 5) have you felt VERY HEALTH AND FULL OF ENERGY? With the exception of the full of energy item, higher scores indicate poorer health. The full of energy item was reversed scored for all analyses.

The CDC-PH includes these five items described above: HDCM 1), 2) and 4) and HDSM 1) and 5) [8]. The CDC-MH includes the these three items described in the above section: HDCM 3) and HDSM 2) and 3). One item from the HDSM was dropped: During the past 30 days, for about how many days 4) have your felt you did NOT get ENOUGH REST or SLEEP [8]?

IRT is a set of models that describe the process by which individuals respond to items. Put another way, IRT is analogous to a factor analysis where the relationships between the measured variables and the latent construct are nonlinear [13]. One of the most widely used IRT models is the graded response model (GRM), which is appropriate when the responses are ordered categories [14]. The GRM has two types of parameters: discrimination and thresholds. The discrimination parameter provides information regarding how related each item is to the construct being measured. The threshold (or severity) parameters convey information related to the level of the underlying trait an individual would have to possess to choose any particular category with some probability. These parameters allow for the differential weighting of item responses when computing scale scores. The GRM is typically represented as:\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$ P\left({x}_j=c\left|\theta \right.\right)=\frac{1}{1+ \exp \left[-{a}_j\left(\theta -{b}_{cj}\right)\right]}-\frac{1}{1+ \exp \left[-{a}_j\left(\theta -{b}_{\left(c+1\right)j}\right)\right]} $$\end{document}Pxj=cθ=11+exp−ajθ−bcj−11+exp−ajθ−bc+1jwhere x_j is the observed response to item j, c is the particular response among the C response alternatives that was chosen, θ is the latent construct (singular in this case) being measures, a_j is the discrimination parameter for item j, and b_cj is one of the threshold parameters. For completeness we note that the first threshold is assumed to be minus infinity and the last infinity. This is a definitional part of the model and does not directly impact the estimated parameters. All IRT analyses in this paper were conducted using flexMIRT [15].

Some of the advantages of IRT include: 1) detailed item level information, 2) more accurate estimates of precision of individual scores, 3) item parameters that are not sample dependent, and 4) IRT is the segue into computerized adaptive testing [16]. Unfortunately, the CDC HRQOL scales pose a number of potential problems if the goal is to obtain item parameters via IRT modeling. First, the scales are relatively short, which has been shown to increase the difficultly in recovering parameters [17]. Next, as currently scored, there are a total of 31 response categories. While it is theoretically possible to use IRT with this many categories, it is rarely, if ever, done in practice. Lastly, when examining the questions, it seems possible that there could be local dependence among the items. Briefly, local dependence occurs when two (or more) items are more related to one another than the model would predict. Considered together, these impediments may preclude any IRT analysis of the CDC HRQOL scales. However, given the nature of our concerns weighed against the benefit of using an IRT approach, we decided to fit the models and then evaluate the stability/reasonableness of the resulting estimates.

The natural response scale for the HDCM and HDSM is number of the past 30 days that a statement was true. This leaves 31 possible response categories. An analysis of the observed responses suggests that the bulk of respondents are using far fewer than 31 categories. For example, roughly 72 % of the responses for the second item on the HDCM fell into one of seven categories (0,5,10,15,20,25,30). Similar trends were found in the other items considered here as well. We chose to recode the data into eight categories which, along with the original observed frequencies, are described in Table 1. Our recoding scheme acknowledges the observed clustering, but also adds an additional category for those who are greater than zero, but less than five.Table 1

Observed Frequencies and recoding scheme for the HDCM and HDSM

For the kind of IRT analysis described in this paper, a critical assumption is that of unidimensionality. If a scale is unidimensional, then responses to that scale arise from only one underlying trait. A closely related assumption is that of local independence. Local independence implies that, conditional on the latent trait being measured, item responses are independent from one another. The unidimensionality and local independence assumptions can be assessed using confirmatory factor analysis (CFA).

The CFA models were estimated in LISREL using polychoric correlations and diagonally weighted least squares (DWLS) [18]. Using the DWLS estimator allows us to obtain correct fit indices such as Root Mean Square Error of Approximation (RMSEA) and the Comparative Fit Index (CFI) in the presence of categorical data [19]. CFI values greater than 0.95 are generally regarded as indicating good model fit. Browne and Cudeck (1993) characterize RMSEA values less than 0.05 as indicating close fit, values greater than 0.05 but less than 0.08 indicating reasonable fit, values greater than 0.08 but less than 0.1 indicating mediocre fit, and values greater than 0.1 indicating unacceptable fit [19, 20].

After listwise deletion, the sample size for the CFA analyses was N = 1642. We began by fitting a one factor model to all nine items to see if this simple model could account for the relationship among the items. As expected, this model did not fit the observed data well. The RMSEA for the 1-factor model was 0.24 and the CFI was 0.88, both of which suggest poor model fit. We subsequently fit a 2-factor model that was based on the results of the Mielenz et al. (2006) principal components analysis [8]. We started using all nine items, allowing five items (general health, physical health, physical/mental health, pain, energy) to load on a first factor, three items (mental health, depressed, anxious) to load on a second factor, and a single item (rest) to load on both. This model provided a reasonable fit to the observed data (RMSEA = 0.056, CFI = 0.99). The item about rest had significant, but weak (~0.35) factor loadings on both factors. This means that although the item is related to both factor, it does not provide a large amount of information regarding respondents levels on either factors. To retain this item for the IRT analyses would require the use of advanced multidimensional models which did not seem warranted given the limited information the item provides. We decided to remove that item and re-run the 2-factor model on the remaining eight items. This model had a similar level of fit as the previous model (RMSEA = 0.058, CFI = 0.994), which suggests that it provides an adequate representation of the data. All items had significant and sizeable (>0.7) factor loadings on their respective items. The two factors, which we labeled physical and mental health, were correlated 0.67. This is a large correlation, but not so large to suggest that there aren’t two unique constructs being measured by the eight items. These results match with a reading of the content of the items. Perhaps the only surprising result is the extent to which the item regarding physical and mental health limiting activities seems to have no relationship to the mental health factor. The data suggest that respondents are more heavily weighting their physical health when responding to this item, as it is predominantly related to other items assessing physical health. We also examined each dimension separately to assess model fit. The physical health scale fit reasonably (RMSEA = 0.08 and CFI = 0.99) and the mental health scale fit perfectly as that model is saturated with only three indicators.

After the factor analyses described above, we were confident that the data could be modeled using a unidimensional IRT model. The CDC-MH and CDC-PH scales were analyzed separately in flexMIRT. Multidimensional calibrations were also conducted in flexMIRT and the impact on the estimated parameters was negligible, which is not surprising given the particular structure of these data. The estimated correlation between the two dimensions was 0.65, consistent with the CFA findings. The estimation procedure used in flexMIRT is able to accommodate missing data without resorting to list-wise (or pair-wise) deletion, so we did not have to remove participants who had missing data. This resulted in a sample size of N = 1790 for subsequent IRT analyses.

The general health item, which is the first item on the CDC-PH, has five response categories. All other items, in both the CDC-PH and CDC-MH, have eight response categories as detailed above. The GRM produces one fewer threshold parameter than number of categories, which means there are four threshold parameters for the general health item and seven threshold parameters for all remaining items. Each item also has a slope parameter, which in the GRM is allowed to freely vary over items. The parameter estimates for the mental and physical health items can be found in Table 2. All item-level marginal χ² values were non-significant and both models had an IRT-based RMSEA value of 0.05.Table 2

Parameter estimates from the GRM for the CDC-Physical and Mental Health

A primary interest in initially examining the item parameters in Table 2 is assessing the extent to which the results appear reasonable. While there are five items proposed to tap into physical health (CDC-PH), there are only three relating to mental health (CDC-MH). In the factor analytic framework (of which IRT is a special extension) a latent variable with three indicators is just identified statistically. This can lead to issues with stability of estimation or lead to latent variables that are very narrowly construed. The parameter estimates for the five CDC-PH items all appear reasonable and within the boundaries commonly seen. The slope for HDSM2, which is a CDC-MH item, is unusually large. This could indicate that the solution is not stable, or it could indicate that responses to this one item capture most of the variability in responses to all three. While we are confident in the quality of the estimates for the CDC-PH items, we advise caution in using and/or interpreting the results for the CDC-MH items.

Figure 1 contains a graphical display of the item parameters for the general health item from the HDCM (item 1 on the CDC-PH scale) and is called a trace line or option characteristic curves. These series of curves trace the probability that an individual will choose a particular category at a particular level of the latent construct. The x-axis represents the latent construct (physical health), which is assumed to follow a standard normal distribution. As coded, higher scores indicate less health, so someone with a latent trait score of two would be said to be two standard deviations above the population (from which the sample is drawn) in terms of health problems. Each line corresponds to one of the five possible response alternatives: excellent, very good, good, fair, poor. As someone moves from left to right on the x-axis their physical health problems are increasing and we see that the category they choose moves from those indicating good physical health to those indicating physical health problems. Someone with an average level of physical health problems is about equally likely to choose “good” or “fair” as their response to the general health item. It isn’t until someone is 2.5 standard deviations below the population average of physical health problems that they predominately rate their general health as “excellent”. On the other end of the spectrum, individuals with scores of 1.5 or more are most likely to choose the “poor” category.Fig. 1

Trace line plot for the general health item from the HDCM