There is an increased interest in how neighborhood social processes, such as collective efficacy, may protect mental health. Yet little is known about how stable these neighborhood processes are over time, or how to change them to influence other downstream factors. We used a population-based, repeat cross-sectional study of adults (n=5135) to assess stability of collective efficacy for families in 38 Boston neighborhoods across 4 years (2006, 2008, 2010) (the Boston Neighborhood Survey). We test temporal stability of collective efficacy for families across and within neighborhoods using 2-level random effects linear regression, fixed effects linear regression, T-tests, and Wilcoxon rank tests. Across the different methods, neighborhood collective efficacy for families remained stable across 4 years, after adjustment for neighborhood composition. If neighborhood collective efficacy is measured within 4 years of the exposure period of interest, assuming temporal stability may be valid.

A burgeoning literature has documented the association between individual mental health and neighborhood social and structural factors (

The bulk of this literature is relatively new, especially in regards to measuring social processes. Neighborhood collective efficacy is one such social process that has recently been linked to mental health. Studies have shown that collective efficacy interacts with other variables to predict depression (

A first step in understanding this causal process is to assess whether neighborhood collective efficacy exposures change over time, including to move beyond the predominantly cross-sectional research evidence. Some evidence of neighborhood effects on mental (and physical) health comes from longitudinal studies (see e.g.,

Theoretically, changes in neighborhood exposures may be generated by virtue of residents themselves moving to different neighborhoods (residential mobility), by place-based fluctuations in resident composition (driven by residential turnover), or by place-based physical or social changes (e.g. neighborhood revitalization), indicating several different approaches for modeling change. Although prior work has demonstrated that most neighborhoods are stable social systems that change slowly unless disrupted by ‘triggering events’ such as disinvestment or demolition (

Collective efficacy is a measure of how confident residents are that their neighbors trust each other and will work together for the good of the neighborhood (

Researchers applying ecometric methods (

Neighborhood collective efficacy may be dynamic across time as social relationships change or people in the neighborhood move, yet most studies using neighborhood resident surveys to operationalize social context are cross-sectional (

To date, only one study has examined the stability of neighborhood social processes over time. In a recent book on Chicago neighborhoods using data from the PHDCN, Sampson examined whether collective efficacy of Chicago neighborhoods remained stable or changed from 1995 to 2002 using two waves of data (

The overarching objective of this paper is to test whether neighborhood collective efficacy for families (

Data for this study come from the 2006, 2008, and 2010 Boston Neighborhood Survey (BNS). The BNS is a repeat cross-sectional, random-digit-dial telephone survey conducted by the opinion research firm Fact Finders, Inc (

Collective efficacy for families was measured by combining three validated neighborhood scales (social cohesion, informal social control, and intergenerational closure) (

To confirm that the 14 items across the 3 scales were unidimensional and reliable, we ran a factor analysis and calculated Cronbach’s alpha. Internal consistency reliability was very high, α=.93; moreover, items loaded on a single factor, with the first factor explaining 96% of the variance.

We then applied Item Response Theory (IRT) methods to calculate the collective efficacy measure, combining the three ordinal scales into a factor score with an approximately standard normal distribution. As such, a one-unit change in collective efficacy in our models corresponds to an approximately one standard deviation (SD) change (mean(SD)= −1.38E-04(.95)). IRT is a more flexible latent variable method for creating scales compared with simply summing items; IRT gives greater weight to items having a stronger relationship to the underlying construct and therefore decreases measurement error compared to simple mean scales (

We adjusted for year linearly (2006=0, 2008=1, 2010=2), sex, age, race/ethnicity, nativity (foreign-born vs. US-born), education, income, length of time in neighborhood (neighborhood tenure), and home ownership in multivariate analyses. All covariates had <2% missing except income (19%), which was modeled using a missing indicator. Cases with missing covariate data (except income), missing outcome data, or missing a neighborhood level identifier were excluded from multivariate analyses (n=232, 4.5% of sample).

The City of Boston conventionally divides its geography into 16 large neighborhoods. These divisions, which range in size up to 90,000 residents, are far too large for understanding the relationships of social processes such as collective efficacy, which are hypothesized to operate at smaller spatial scales. As such, as part of the larger project in which the BNS is embedded, the research team worked with key informants in sub-neighborhoods of the city who inspected maps and used their local knowledge to define 38 socially relevant “neighborhood clusters” comprised of multiple contiguous census blocks. Details of this neighborhood formulation process are described elsewhere (

We applied multiple methods to test the temporal stability of neighborhood collective efficacy for families, including comparing means over time, between and within neighborhoods. All hypothesis tests were conducted at nominal type 1 error=0.05. Individual-level data were used for some analyses; neighborhood-level measures were created for other analyses by aggregating data to calculate a mean collective efficacy value for each of the 38 neighborhood clusters for each year (2006, 2008, and 2010). We began with a basic test of difference across time, using a paired t-test to assess the change in collective efficacy from 2006 to 2010 across neighborhoods (i.e., using neighborhood-level data) and then separate t-tests within each of the 38 neighborhoods (i.e., using individual-level data stratified by neighborhood). Next, we used a Wilcoxon rank sum test to assess whether the rank ordering of neighborhoods changed for collective efficacy from 2006 to 2010 (i.e., using neighborhood-level data). Although these analyses provide simple tests of the null hypothesis that neighborhood collective efficacy for families remained stable over time, these analyses fail to adjust for covariates that might influence respondents’ reporting of their neighborhood social environment.

To obtain a more powerful test of neighborhood change, we employed a two-level random effects multi-level model, which explicitly accounted for the nesting of individuals within neighborhoods, to test whether there was neighborhood variation in change in collective efficacy over time. For each neighborhood, we had measurements of collective efficacy in 2006, 2008, and 2010, so we estimated a two-level model using SAS proc mixed, with respondents (level 1) nested in neighborhoods (level 2). We estimated both a crude model and a covariate-adjusted model to test for compositional effects of neighborhoods. The model contained a random intercept for neighborhoods, a fixed effect for year, and a random slope for year, which allows for neighborhood-specific rates of change that captures heterogeneity across neighborhoods. The equation for this model is:
_{00} is the overall neighborhood intercept, γ_{10} is the overall rate of change (fixed effect of year; a one unit change in this time variable is equal to a 2-year change), ζ_{0i} is the level 2 neighborhood random intercept variance, ζ_{1i} is the level 2 year random slope (a one unit change in this time variable is equal to a 2-year change), and Ɛ_{ij} is the level 1 residual error. The parameters of interest are the fixed effect of year and the variance-covariance matrix of random effects. We are particularly interested in whether there is evidence of moderate to large heterogeneity in the rate of change across neighborhoods, which would indicate that the time trend for some neighborhoods is significantly different from the average (fixed effect) of the time trend in collective efficacy for all of Boston.

The random effects model is a powerful framework for evaluating change, but it relies on the assumptions that unobserved heterogeneity is uncorrelated with other variables in the model and that the residuals are normally distributed (^{th} neighborhood), and an interaction term between year and each neighborhood indicator. The focus here was on the joint test of significance for the interaction terms, which tests the null hypothesis that there is no overall change across years across neighborhoods.

We began with t-tests to examine differences in collective efficacy for families using data from 2006 and 2010, both between and within neighborhood clusters (

A Wilcoxon rank sum test examined whether any neighborhoods differed in rank ordering on collective efficacy from 2006 to 2010, using neighborhood-level means. The test was non-significant (z-statistic= −1.12, p=.26), indicating that neighborhoods achieved no significant change in their neighborhood ranking across Boston during this time.

_{0}^{2}(SE)=.049(.018)), p=.004) indicating that neighborhoods significantly varied from one another in their 2006 cross-sectional collective efficacy score. However, we found no statistical evidence of a random slope for year (adjusted σ_{1}^{2}(SE)=.001(.003), p=.370), and no significant covariance between intercept and year slope (adjusted σ_{01}(SE)=-.002(.006), p=.805), indicating no change in collective efficacy across neighborhoods over time. We did find a statistically significant crude estimate of the fixed effect of year (b(SE)=.029(.015), p=.05) on collective efficacy, which suggests collective efficacy did change in Boston across this time; however, this effect disappeared upon adjusting for individual-level covariates (b(SE)=.004(.017), p=.827). Ten percent of the variation in the outcome was due to differences between neighborhood clusters (Intra-cluster correlation (ICC)=.103; calculated from unadjusted random intercept model). Although this is a larger ICC for a neighborhood study, it does indicate that most of the variation in collective efficacy for families was due to differences between individuals within neighborhoods, in line with other neighborhood ecometric studies (

Finally, we estimated fixed effects unadjusted and covariate-adjusted linear regression models, with an indicator for year, for each neighborhood, and for neighborhood-year interactions.

Our analysis of population-based neighborhood survey data in Boston tested whether collective efficacy for families remained stable across time, which has implications for causal questions regarding how neighborhood collective efficacy can potentially prevent mental health problems or other individual outcomes. Overall, our analyses could not reject the null hypothesis that collective efficacy for families in Boston neighborhoods remained stable over a 4-year period.

Although we found that Boston residents reported significantly higher collective efficacy in 2010 vs. 2006 in unadjusted analyses (mean comparison and fixed effect of year in multilevel regression), the time pattern was accounted for by compositional differences across time, which may be due to true population change across time, or to changes in BNS sample response patterns across time. Using covariate-adjusted regression, we found no statistical evidence of significant change in collective efficacy for families overall across neighborhood clusters. This suggests that it is important to adjust for population composition when assessing change not due to variation in person-level characteristics. However there also may be some argument for not adjusting for population composition. For example, if changing the population composition is one feasible way to increase (or decrease) collective efficacy, (suggesting that changing population composition is a cause of collective efficacy), then researchers may want to model it in its naturally occurring form.

The neighborhood fixed effects model was suggestive of potential variation across neighborhood cluster-years in change in collective efficacy for families 2006–2010 given the marginally-significant F-test result, but this was not confirmed by the random effects model. The fixed and random effects models are closely related in that they both account for the clustered nature of the data and incorporate the fact that observations within clusters are likely to be correlated. They differ, however, in one important respect: the random effects model incorporates intra-cluster correlation and assumes that it is uncorrelated with other variables in the model, and makes

Given the different assumptions and treatment of unobserved heterogeneity in fixed versus random effects models, it is not surprising that we found slightly divergent results in our models. Despite the divergence, the neighborhood-time fixed effect test was only marginally significant, so we cannot rule out that it may be due to chance. Taken together, these results suggest that collective efficacy for families is stable over this 4-year, 3-wave period in the city of Boston. This is consistent with recent findings for collective efficacy over a 7-year, 2-wave period in Chicago (

One potential explanation for our finding that collective efficacy for families was stable across time is that four years may be insufficient time for the construct to change, suggesting that studies that assume the stability of collective efficacy – at least over relatively short periods of time – may be justified in doing so. Higher levels of collective efficacy have been linked to positive neighborhood structural factors, such as having more parks and fewer alcohol outlets, after controlling for tract-level disadvantage and sociodemographic variables of individuals (

Collective efficacy is thought to be a modifiable neighborhood process (

Researchers commonly operationalize collective efficacy with two scales: social cohesion and informal social control (

We used a population-based survey, but cooperation rates (CR) were low, e.g. 39% in 2006, 32% in 2008, and 33% in 2010, calculated thus:

Our study also may not be generalizable to cities other than Boston. However, our finding that collective efficacy was stable in Boston neighborhoods across the 4-year period of our study, coupled with Sampson’s similar finding in Chicago over 7 years (

Analyzing change with mean-difference tests carries with it some problems (

Prior literature has provided very little information on how, and if, neighborhood social processes change over time (

Support for this publication was provided by NIH grant 1R01MD006064-01 (Dr. Osypuk, PI) and an award from the Centers for Disease Control and Prevention (U49-CE00740, Dr. Hemenway, PI). The Boston Youth Survey was conducted by the Harvard Youth Violence Prevention Center in collaboration with the City of Boston (the Honorable Thomas M. Menino, Mayor), the Boston Public Health Commission (Barbara Ferrer, Executive Director) and the Boston Public Schools.

Conflicts of Interest: The authors do not have any conflicts of interest to declare.

Author Contributions: Nicole M. Schmidt, PhD and Theresa L. Osypuk, ScD, SM conceived the hypotheses and interpreted results, drafted and revised the manuscript, and Dr. Schmidt additionally analyzed the data. Eric Tchetgen Tchetgen, PhD advised on the statistical analysis and interpretation of results, and edited the manuscript. Amy Ehntholt, MPH contributed to the data analysis and manuscript editing. Joanna Almeida, ScD and Quynh C. Nguyen, PhD contributed to interpreting results and editing. Beth E. Molnar, ScD and Deborah Azrael, PhD contributed to the design of the BNS study, the acquisition of the data for these analyses, interpretation of results, and editing. All authors approved the final submitted version.

Collective Efficacy Scale & Composite Scales Variable Coding

Scale | Higher Values Indicate | No. of | Cronbach's | Items | Rating Scale |
---|---|---|---|---|---|

Social Cohesion | More cohesion | 5 | 0.85 | In my neighborhood: people can be trusted; people are willing to help their neighbors; people know and like each other; people get along with each other; people share the same beliefs about what is right and wrong | 1=strongly agree to 4=strongly disagree; reverse coded |

Informal Social Control | More social control | 5 | 0.82 | In your neighborhood, how likely is it that your neighbors would: organize together to keep a fire station open that was going to close because of budget cuts; do something about neighborhood children skipping school and hanging out on a street corner; do something about a child showing disrespect to an adult; do something about a child spray-painting graffiti on a local building; do something if there was a fight in your neighborhood and someone was being beaten or threatened | 1=very likely to 4=very unlikely; reverse coded |

Intergenerational Closure | More connection between adults/children | 4 | 0.82 | In my neighborhood: there are adults that children can look up to; you can count on adults to watch out that children and teenagers are safe and stay out of trouble; parents know one another; parents know their children's friends | 1=strongly agree to 4=strongly disagree; reverse coded |

Collective Efficacy for families | More collective efficacy | 14 | 0.93 | All items described above | Described above |

Table Note

Scores for the Collective Efficacy for Families scale were recoded from the original scale items using IRT methods for the final models.

Individual-level descriptive statistics for Boston Neighborhoods Survey - 2006, 2008, and 2010

Variable | Subgroup | Total n | Mean | SD | Min | Max | |
---|---|---|---|---|---|---|---|

Male | 2123 | 5135 | 0.41 | 0.49 | 0 | 1 | |

Female | 3012 | 5135 | 0.59 | 0.49 | 0 | 1 | |

Age | -- | 5074 | 53.39 | 16.61 | 18 | 99 | |

Race/Ethnicity | |||||||

Black | 1013 | 5032 | 0.20 | 0.40 | 0 | 1 | |

White | 3145 | 5032 | 0.63 | 0.48 | 0 | 1 | |

Hispanic | 462 | 5032 | 0.09 | 0.29 | 0 | 1 | |

Other | 412 | 5032 | 0.08 | 0.27 | 0 | 1 | |

Foreign Born | 1078 | 5126 | 0.21 | 0.41 | 0 | 1 | |

Education | |||||||

< HS | 364 | 5096 | 0.07 | 0.26 | 0 | 1 | |

High school diploma or GED | 1049 | 5096 | 0.21 | 0.40 | 0 | 1 | |

Some College | 1205 | 5096 | 0.24 | 0.42 | 0 | 1 | |

Bachelors | 971 | 5096 | 0.19 | 0.39 | 0 | 1 | |

Graduate Study | 1507 | 5096 | 0.30 | 0.46 | 0 | 1 | |

Annual Income | |||||||

<20K | 790 | 5135 | 0.15 | 0.36 | 0 | 1 | |

20–40K | 875 | 5135 | 0.17 | 0.38 | 0 | 1 | |

40–80K | 1096 | 5135 | 0.21 | 0.41 | 0 | 1 | |

80–100K | 452 | 5135 | 0.09 | 0.28 | 0 | 1 | |

>100K | 944 | 5135 | 0.18 | 0.39 | 0 | 1 | |

Missing Income | 978 | 5135 | 0.19 | 0.39 | 0 | 1 | |

Neighborhood Tenure | -- | 5119 | 20.02 | 17.59 | 0 | 94 | |

Home Owner | 3016 | 5090 | 0.59 | 0.49 | 0 | 1 | |

Renter | 1865 | 5090 | 0.37 | 0.48 | 0 | 1 | |

Social Cohesion | -- | 5135 | 3.63 | 0.66 | 1 | 5 | |

Informal Social Control | -- | 5135 | 3.81 | 0.84 | 1 | 5 | |

Intergenerational Closure | -- | 5135 | 3.61 | 0.67 | 1 | 5 | |

Collective Efficacy for Families IRT | |||||||

Score | -- | 5125 | −1.38E-04 | 0.95 | −3.59 | 2.41 |

Before creating the mean score, items were recoded to a 5-point scale and missing data was imputed to the row-column mean (on average, 13% of scale items were missing).

T-tests between and within 38 Boston neighborhoods; Testing change in collective efficacy for families 2006 to 2010

3a. Between Neighborhoods Paired t-test; N=38 | |||||
---|---|---|---|---|---|

Mean (SD) | |||||

−.13(.42) | −.02(.37) | −2.15 | 0.04 |

3b. Within Neighborhoods t-test; N=5125, stratified by neighborhood | |||||
---|---|---|---|---|---|

Without Bonferroni | With Bonferroni | ||||

3 | 8% | 0.05 | 0.00 | 0% | 0.001 |

p<.10

p<.05

p<.01

p<.001

Bonferroni correction is .05 divided by the number of tests; for each neighborhood cluster (n=38) we are testing one outcome for 38 neighborhoods, so .05 is divided by 38.

NOTES: The sample size for section 3a is 38, because the t-tests are done using neighborhood level data; the sample size for section 3b is 5125, because the t-tests are done at the individual level, but stratified by neighborhood -- the neighborhood specific sample sizes range from 4 to 203 cases.

Two-level random effects model; Testing change in collective efficacy for families 2006, 2008, and 2010

Unadjusted Model | Covariate-Adjusted Model | |||||||
---|---|---|---|---|---|---|---|---|

Random Neighborhood Intercept | 0.097 | 0.029 | 0.001 | 0.049 | 0.018 | 0.004 | ||

Random Year Slope | −0.001 | 0.002 | 0.602 | 0.001 | 0.003 | 0.370 | ||

Covariance between Intercept-Slope | −0.001 | 0.007 | 0.907 | −0.002 | 0.006 | 0.805 | ||

Residual | 0.829 | 0.017 | <.0001 | 0.772 | 0.016 | <.0001 | ||

Year | 0.029 | 0.015 | 0.05 | 0.004 | 0.017 | 0.827 | ||

AIC | 13102.9 | 12806.1 | ||||||

N | 4903 | 4903 |

p<.10

p<.05

p<.01

p<.001

NOTE: Covariate-adjusted model adjusted for gender, age, race/ethnicity, foreign born status, education, income, neighborhood tenure, and home ownership

Neighborhood fixed effects model; Testing change in collective efficacy for families 2006, 2008, and 2010

Unadjusted Model | Covariate-Adjusted Model | |||||
---|---|---|---|---|---|---|

Linear Effect of Year b(SE) | −.016(.046) | .031(.016) | −0.064(.045) | .001(.016) | ||

Linear Effect of Year p-value | 0.74 | .05 | 0.16 | 0.93 | ||

R^{2} | 0.09 | 0.09 | 0.16 | 0.15 | ||

Joint F-Test of Interactions | 1.17 | 1.36 | ||||

p-value | 0.22 | .07 | ||||

N | 4907 | 4907 |

p<.10

p<.05

p<.01

p<.001

NOTES: Covariate-adjusted model adjusted for gender, age, race/ethnicity, foreign born status, education, income, neighborhood tenure, and home ownership. The F-test above is comparing a full model to a reduced model. The null hypothesis test is that the interaction terms do not significantly improve the model. In other words, the joint effect of the year*neighborhood interactions is equal to zero. The full model was estimated with post-estimation commands that give a sub-group analysis comparing, within each neighborhood, the effect of time on the construct of interest. The critical value with F(α=.05; df=37, 4907)=1.41. We tested for nonlinearity of year using a squared term and found no evidence of a non-linear effect of year.