03764226405PediatricsPediatricsPediatrics0031-40051098-427524187111450325310.1542/peds.2013-1021HHHSPA700126ArticleComponents Associated With Home Visiting Program Outcomes: A Meta-AnalysisFileneJill H.MPH1KaminskiJennifer W.PhD2ValleLinda AnnePhD2CachatPatriceMSW1James Bell Associates, Arlington, VACenters for Disease Control and Prevention, Atlanta, GAAddress correspondence to: Jill H. Filene, James Bell Associates, 3033 Wilson Boulevard, Suite 650, Arlington, Virginia 22201, Filene@jbassoc.com, 703-528-3230196201511201315720151320 2S100S109Introduction

Although several systematic reviews have concluded that home visiting has strong evidence of effectiveness, individual evaluations have produced inconsistent results across and within programs. We used a component-based, domain-specific approach to determine which program characteristics most strongly predict outcomes across a range of programs and models.

Methods

Medline and PsycINFO searches identified evaluations of universal and selected home visiting programs implemented in the United States. Coders trained to criterion coded characteristics of research design, program content, and service delivery. We conducted random-effects, inverse-variance-weighted linear regressions using program characteristics to predict effect sizes (ESs) on six outcome domains (birth outcomes, parenting behavior, maternal life course, child cognitive outcomes, child physical health, and child maltreatment).

Results

Aggregated to a single ES per study (k=51), the mean ES was 0.20 (95% CI = 0.14, 0.27), with a range of – 0.68 to 3.95. Mean ESs were significant and positive for three of the six outcome domains (maternal life course outcomes, child cognitive outcomes, and parent behaviors and skills), with marked heterogeneity of ESs in all six outcome domains. Research design characteristics generally did not predict ESs across the six outcome domains. No consistent pattern of effective components emerged across all outcome domains.

Conclusions

Home visiting programs evidenced small but significant overall effects, with wide variability in the size of domain-specific effects and in the components that significantly predicted domain-specific effects. Communities may need complementary or alternative strategies to home visiting programs to ensure widespread impact on these six important public health outcomes.

home visitingmeta-analysiseffectivenessinfancyearly childhoodmaternal life courseparenting behaviorchild cognitive developmentchild maltreatmentbirth outcomeschild physical health

Early childhood marks a period of rapid growth and development that lays the foundation for future health and success in school and life.1 Parents play a critical role in shaping children‘s early development, so interventions that reach families in these early years have great potential for producing long-term benefits.2 Prenatal and early childhood home visiting is a widely-endorsed method for delivering a vast array of preventive and early intervention services to families in need of support. By engaging families in home visiting programs during the prenatal or early childhood period, providers seek to improve children’s long-term developmental trajectories by fostering improved parenting knowledge and skills, social support, coping and problem-solving skills, and access to community and health services.3

Despite national and international endorsement of home visiting as a strategy to prevent child maltreatment and promote enhanced functioning and well-being for children and families,4–8 previous meta-analyses and literature reviews of home visiting programs across a wide range of outcomes suggest mixed, modest findings depending on the programs and outcomes examined.6,9–12 A recent review funded by the U.S. Department of Health and Human Services (HHS), the Home Visiting Evidence of Effectiveness (HomVEE) review, identified thirteen models that met the HHS criteria for effectiveness.13 Across and even within these “evidence-based” models, the findings have been inconsistent, leaving gaps in knowledge about the effectiveness of home visiting across various outcome domains. The mixed findings may be due to program design, the match between program components and expected outcomes, or the quality of implementation of the program or the evaluation. Alternatively, the differences in effects might simply be explained by the variation in the way home visiting programs are comprised and delivered.

Best practice recommendations concerning home visiting have generally either taken the form of suggesting wholesale adoption of models that have been shown to be effective (eg, HomVEE [homevee.acf.hhs.gov], Promising Practices Network [promisingpractices.net]) or have been based on clinical impression about particular approaches (eg, recommendations for a particular schedule of home visits). Although model ratings are important for guiding practitioners in adopting a program model, any particular program may not include the most effective combination of components to produce maximum results for a given population or community. In addition, as the Maternal, Infant, and Early Childhood Home Visiting Program (MIECHV)14 impels increased focus on outcomes, a pressing question is how to best build the effectiveness of a program model or enhance models that may already be in operation: what elements (eg, content, service delivery methods) in home visiting programs appear most important for program success?

Although two systematic reviews conducted prior to 2002 examined the relationship between parent and child outcomes and a small subset of program components, no reviews have fully disassembled home visiting programs into individual components or included studies conducted during the last decade.12,15 Therefore, a component analysis employing meta-analytic techniques was used to synthesize the results of published evaluations of home visiting programs to determine which individual home visiting program components have the most power to predict key parent and child outcomes.

METHODSSearch Strategy

In September 2010, the PsycINFO and MEDLINE databases were searched for literature published between 1979 and 2010 with evaluations of home visiting programs. Studies were limited to those published in English as a journal article, book, or book chapter, although programs could be implemented in any language. Details of the search strategy are outlined in Appendix 1. The initial search was designed to be very broadly inclusive of home visiting programs.

The original literature search resulted in 3,252 unduplicated studies. Of these, 49 were literature reviews and meta-analyses, from which we identified additional relevant publications. A secondary search was conducted on author names that appeared at least twice in the original search results. In addition, unduplicated studies from HomVEE were examined. These follow-up strategies yielded an additional 1875 records, providing 5,127 total abstracts for possible inclusion.

Study Selection

Inclusion criteria were selected to define the scope of the meta-analysis as evaluations of universal and selected (ie, for at-risk families) programs that used home visiting as a primary delivery strategy for pregnant women and families with children from birth through age three in the US. Programs that conducted only one or two home visits were excluded as dissimilar to the rest of the field. Home visiting programs targeting families for existing identified problems (eg, family preservation programs or programs that provided services to families with a substantiated child maltreatment case) were excluded. Similarly, criteria were selected to ensure that evaluation results could be generalized to a broad population of typically developing children and parents. Thus, we excluded programs that targeted parents or children because of developmental disabilities, chronic illness, feeding disorders or bereavement because the programs provide specialized components not found in the general field of home visiting.

Figure 1 presents the PRISMA flow diagram for study inclusion. Abstracts identified in the literature search were screened by two project staff to determine eligibility. A study was excluded at this point only if both staff agreed that it met none of the inclusion criteria; 525 documents were retrieved and reviewed in full text. To allow for calculation of comparable effect sizes, studies that utilized a single-case evaluation method, lacked a control or comparison group, or did not contain enough statistical information to calculate effect sizes were excluded. The resulting 126 studies were coded for meta-analysis; a subsample of the 51 articles including the six outcome measures (maternal life course, birth outcomes, parent behaviors and skills, child cognitive outcomes, child physical health, and child maltreatment) selected for this study were analyzed.

Data Abstraction

Coding forms adapted from Kaminski et al16 captured information about the document, author(s), home visiting program, participants, evaluation design, outcome measures, and statistical results. Table 1 lists and describes the variables coded for these analyses. Full coding forms are available from the first author. When an article referred to a secondary study or article providing additional program information, that secondary document was obtained, and the information was coded. Before coding independently, data abstractors were trained to criteria of coding three consecutive articles with greater than 90% accuracy.

Summary Measures

Effect sizes analogous to Cohen’s d17 were calculated from means and standard deviations whenever possible or from other reporting methods including categorical data, correlations, and odds ratios using Comprehensive Meta-Analysis 2 software.18 Effect sizes were calculated based on unadjusted data if available or adjusted data if not. Once effect sizes were calculated, they were exported into SPSS v20 for analyses using macros for multivariate analyses of effect sizes.19,20 We applied Hedges’ small sample correction to all effect sizes prior to analysis, and weighted each by the inverse of the variance.21

Within and across articles, some samples were represented multiple times (eg, the same sample assessed at different time points, assessed with different measures, or reported in different articles). Including all published reports of those samples would have allowed a small number of frequently published programs to bias the results. Thus for each analysis, we selected or aggregated effect sizes such that each sample (eg, a program implemented in a particular location) only provided a single effect size for that analysis. Data on birth outcomes at any time point in a study were included. For all other outcomes, immediate post-test assessments were preferred. If immediate post-test data were not available for a particular sample, we included assessments that occurred during the intervention but after two-thirds of the intervention was delivered. Follow-up data were excluded due to a lack of comparability in the length of follow-up periods. When “total” scores and “subscale” scores from particular measures were reported, preference was given to the total score if it fell within a single outcome category. When a single study included three or more study arms, the effect size most closely attributable to the effect of only the home visiting program (eg, treatment versus no-treatment comparison, or treatment plus enhancement versus enhancement only) was selected.

Analytic Plan

We first examined overall program effects on the six outcome categories by aggregating to a single effect size per study sample. We calculated overall weighted mean effect size, 95% confidence interval, and Q and I2 statistics.22 Following Kaminski et al.16 we next investigated outcome-specific mean effect sizes by aggregating to a single effect size per study sample for each outcome category, as well as confidence intervals and Q and I2 statistics. We used inverse-variance-weighted analyses of variance to examine the impact of four indicators of methodological rigor (random assignment, assessment of initial equivalence, using a pure no-treatment comparison group, and testing the effect of the home visiting program as a stand-alone intervention versus as part of a larger package of interventions) and timing of the outcome measure (prior to vs. at the end of treatment) on effect sizes for each outcome category. Finally, we used inverse-variance-weighted linear regression to test the impact of program components on effect sizes, to determine predictors of strongest program effects. Only components theoretically expected to contribute to particular outcomes were tested for those outcomes. As the intent of the analyses was to model variability among studies, all reported results were obtained via random-effects models.

RESULTS

The overall weighted effect size of the final set of 51 studies was 0.20 (95% CI = 0.14, 0.27). The 251 effect sizes ranged from – 0.68 to 3.95. The Q test of homogeneity of effect sizes was significant (p<.001), with an I2 value of 65%. Table 2 shows the number of studies and summary statistics by outcome category. Three outcome categories (maternal life course, child cognitive outcomes, and parent behaviors and skills) resulted in significant, positive average effect sizes. Average effects sizes were not significantly different from zero for birth outcomes, child physical health and child maltreatment. Between 52% and 86% of the heterogeneity observed for each outcome was attributable to true variance rather than to chance, suggesting the need to further examine the nature of the heterogeneity.

In the inverse-variance-weighted ANOVAs, only one research design variable was a significant predictor of any outcome: effect sizes of maternal life outcomes were higher among studies reporting outcomes during treatment (mean ES = 0.23, 95% CI = 0.13, 0.33) than studies reporting outcomes at immediate post-test (mean ES = 0.02, 95% CI = −0.11, 0.15). Measurement timing was therefore included as a covariate in the regression analysis of maternal life outcomes.

Results of the inverse-variance-weighted linear regressions assessing relationships between program components and effect sizes are presented in Table 3. Controlling for timing of assessment, no components significantly predicted maternal life outcomes. Effect sizes based on birth outcomes were significantly larger for programs using non-professional home visitors, programs that matched clients and home visitors on race and/or ethnicity, and programs that included problem-solving. Parent behaviors and skills effect sizes were significantly larger for programs that taught parents developmental norms and appropriate expectations, discipline and behavior management techniques, responsive and sensitive parenting practices, and programs that addressed parental substance use. Children’s cognitive outcomes were better in programs that taught parents responsive and sensitive parenting practices and programs reporting that they required parents to role-play or practice skills during home visits. Using professional home visitors was also a significant predictor of better child physical health outcomes, as was teaching discipline and behavior management techniques. However, providing parents with a support group was associated with smaller effect sizes on child physical health. Better child maltreatment outcomes were associated with teaching parents how to select alternate caregivers for children and problem solving.

To ensure that these results were not unduly influenced by effect sizes based on results reported in studies as adjusted statistics, we removed those effect sizes and re-examined regression analyses with significant components. Of the 14 components reported as significant above, three could not be analyzed without the adjusted effect sizes due to low frequency (the two components significant for child maltreatment outcomes and the relationship between child physical health outcomes and teaching discipline and behavior management techniques). Ten of the other 11 components maintained statistical significance in these sensitivity analyses. The effect of teaching parents problem solving strategies on birth outcomes was no longer significant, and thus may be a less robust finding than other component effects.

DISCUSSION

The overall effect size of home visiting programs (aggregated across the six selected outcome domains) was significant and equivalent to approximately one-fifth of a standard deviation favoring the intervention group. Translated to an odds ratio, such an effect is equivalent to the comparison group being approximately 1.5 times more likely to have poorer outcomes. Consistent with results of previous meta-analyses of home visiting programs,6,9,12,15 parents and children participating in home visiting programs achieved more positive outcomes overall than parents and children in control/comparison groups. However, outcome-specific mean effect sizes revealed significant, but small effects only on maternal life course, child cognitive outcomes, and parent behaviors and skills. In contrast, home visiting programs did not produce significant average effects on three frequent program targets of childbirth outcomes, child physical health, and child maltreatment, suggesting that programs were, on average, not effective in addressing these outcomes. The nonsignificant effect sizes, combined with the relatively small significant effect sizes, suggest that communities may need complementary or alternative strategies to home visiting programs in order to have a greater impact on these important public health outcomes.

Although surveillance bias (ie, program involvement increases the likelihood of detecting maltreatment) may partially explain the lack of a significant effect size on child maltreatment outcomes measured through child protective services data, previous studies have found surveillance bias effects to attenuate but not eliminate group differences where they exist.23,24 In addition, the present analyses included self-reports of abusive parenting practices in addition to child protective services reports. Thus, the presence of a surveillance bias would likely not fully explain the lack of statistical significance.

Research design variables were generally not significantly predictive of effect sizes, while many program components were. Similar to other systematic reviews, no clear and consistent pattern of effective home visiting program components emerged across outcome domains.12 Only three components were predictors of larger effects on more than one outcome; and one of those components was only robust for one outcome in the sensitivity analyses. All other significant components were only predictive of effect sizes for a single outcome domain. These results suggest that the “home visiting” label represents a diversity of approaches with differing effectiveness, and that attention to specific program content and delivery characteristics is critical to the effectiveness of these programs.

The components that emerged as significant for more than one outcome (teaching sensitive and responsive parenting, teaching discipline and behavior management techniques, and teaching problem-solving) make intuitive sense; teaching new parenting skills and behaviors was associated with greater effects on parenting behaviors, which may also translate into more positive impacts on other, sometimes more distal, outcomes, such as child cognitive development, child physical health, and child maltreatment. Using professional home visitors was unexpectedly associated with smaller program effects on birth outcomes but larger effects on child physical health outcomes. The inconsistency in these results may be due to the professional background or type of professional providing the services, as different professionals may be more or less effective with different health outcomes. Alternatively, the inconsistent results might be due to other differences not analyzed here between programs using professional and nonprofessional home visitors. Programs that enroll participants prenatally and use professional home visitors may want to look for ways to boost their effectiveness specifically on birth outcomes.

It is important to note that not all components were tested for each outcome, either because the components were not theoretically linked to the outcome or due to limited variability of the component among studies reporting a particular outcome. As well, nonsignificant components may be contributing to program outcomes (eg, as precursors to or in combination with other components) in interactive ways that cannot be tested with these analytic methods. The presence of a significant component thus indicates a robust effect, but the absence of significance for a component does not necessarily imply a lack of impact. We can only conclude that the nonsignificant components did not by themselves distinguish more successful programs from less successful programs on that outcome and are thus components that are unlikely to be sufficient to produce outcomes they did not significantly predict.

Our results for the impact of different components must be taken as correlational and not as an experimental manipulation. Our results are also based on published studies and are dependent on the completeness of reporting of components within each study. Many theoretically interesting and relevant program characteristics (eg, program dosage, sample demographics, fidelity of implementation, staff training, home visitor caseload, study or program attrition) could not be tested due to insufficient numbers of studies reporting those characteristics. For example, the timing of enrollment in home visiting programs during pregnancy might be associated with a program’s ability to promote positive birth outcomes; variability in gestation at enrollment could explain the lack of significance with birth outcomes. However, this relationship could not be tested due to insufficient reporting on initiation of services.

This meta-analysis marks a distinct departure from the common practice of recommending the wholesale adoption of evidence-based programs. Although model ratings are important for guiding practitioners in adopting a packaged program model, any particular program may not include the most effective combination of components to produce maximum results. Instead of considering each program as a black box, the coding scheme used in the current study allowed the authors to disassemble home visiting programs and examine the impact of specific components. The results suggest that certain existing components are more likely to be associated with positive impacts on specific outcomes. Although careful evaluation of modifications or adaptations to existing programs would be critical, changes to include more of the significant components identified are likely to produce programs that are more potent with respect to these parent and child outcomes. For other outcomes, components that significantly predict positive outcomes remain to be identified. Our findings point to new program and research opportunities within the home visiting field, whether through the development or selection of a home visiting program, or for improving programs already labeled efficacious or effective.

Funding Source: This study was supported by The Pew Center on the States. The views expressed are those of the author(s) and do not necessarily reflect the views of the Pew Center on the States or The Pew Charitable Trusts.

Financial Disclosure: The authors have no financial relationships relevant to this article to disclose.

Conflict of Interest: The authors have no conflicts of interest to disclose.

Contributor’s Statement:

Patrice Cachat: Ms. Cachat collected data, critically reviewed the manuscript, and approved the final manuscript as submitted.

Jill H. Filene: Ms. Filene conceptualized and designed the study, designed the data collection instruments, drafted the initial manuscript, and approved the final manuscript as submitted.

Jennifer W. Kaminski: Dr. Kaminski conceptualized and designed the study, designed the data collection instruments, carried out the analyses, drafted the initial manuscript, and approved the final manuscript as submitted.

Linda Anne Valle: Dr. Valle conceptualized and designed the study, designed the data collection instruments, reviewed and revised the manuscript, and approved the final manuscript as submitted.

The findings and conclusions in this report are those of the authors and do not necessarily represent the official position of the Centers for Disease Control and Prevention.

AbbreviationsHHS

Department of Health and Human Services

HomVEE

Home Visiting Evidence of Effectiveness

MIECHV

Maternal, Infant, and Early Childhood Home Visiting Program

U.S.

United States

REFERENCESShonkoffJPhillipsDFrom Neurons to Neighborhoods: The Science of Early Childhood Development2000Washington DCNational Academy PressBrooks-GunnJMarkmanLBThe contribution of parenting to ethnic and racial gaps in school readinessFuture Child200515113916816130545GutermanNStopping Child maltreatment Before it Starts: Emerging Horizons in Early Home Visitation Services2001Thousand Oaks, CASageKrugmanRDUniversal home visiting: A recommendation from the U.S. Advisory Board on Child Abuse and NeglectFuture Child19933184191American Academy of Pediatrics, Council on Child and Adolescent HealthThe role of home-visitation programs in improving health outcomes for children and familiesPediatrics199810486489BilukhaOHahnRACrosbyAThe effectiveness of early childhood home visitation in preventing violenceAm J Prev Med200528251113915698746National Governors Association, Center for Best PracticesThe Benefits and Financing of Home Visiting Programs2002Accessed August 23, 2008Available at: http://www.nga.org/Files/pdf/BENEFITSFINANCINGHOME.pdfButchartAHarveyAPMianMFurnissTPreventing child maltreatment: a guide to taking action and generating evidence2006Geneva, SwitzerlandWorld Health OrganizationGombyDSHome visitation in 2005: Outcomes for children and parents2005Washington, DCCommittee for Economic Development(Invest in Kids Working Paper No. 7).GutermanNBEnrollment strategies in early home visitation to prevent physical child abuse and neglect and the "universal versus targeted" debate: A meta-analysis of population-based and screening-based programsChild Abuse Negl19992386389010505901LayzerJIGoodsonBDBernsteinLPriceCNational evaluation of family support programs. Final Report Volume A: The meta-analysis2001Cambridge, MAAbt Associates, Inc.SweetMAAppelbaumMIIs home visiting an effective strategy? A meta-analytic review of home visiting programs for families with young childrenChild Dev20047551435145615369524AvellarSPaulsellDSama-MillerEDel GrossoPHome Visiting Evidence of Effectiveness Review: Executive Summary2012Washington, DCOffice of Planning, Research and Evaluation, Administration for Children and Families, U. S. Department of Health and Human ServicesAvailable at: http://homvee.acf.hhs.gov/HomVEE_Executive_Summary_2012.pdf.Health Resources Service AdministrationMaternal, Infant, and Early Childhood Home Visiting Program2011Accessed December 5, 2012Available at: http://mchb.hrsa.gov/programs/homevisiting.NievarMVan EgerenLAPollardSA meta-analysis of home visiting programs: Moderators of improvements in maternal behaviorInf Mental Hlth J2010315499520KaminskiJWValleLAFileneJHBoyleCLA meta-analytic review of components associated with parent training program effectivenessJ Abnorm Child Psychol20082656758918205039HedgesLVOlkinIStatistical methods for meta-analysis1985BostonAcademicBornsteinMHedgesLHigginsJRothsteinHComprehensive Meta-Analysis Version 2. Englewood2005Englewood, NJBiostat, Inc.LipseyMWWilsonDPractical Meta-Analysis2001Thousand Oaks, CASageWilsonDBMeta-analysis macros for SAS, SPSS, and Stata2005Accessed January 23, 2012Available at: http://mason.gmu.edu/~dwilsonb/ma.html.HedgesLVA random effects model for effect sizesPsychol Bull198393388395HigginsJPThompsonSGDeeksJJAltmanDGMeasuring inconsistency in meta-analysesBMJ200332755756012958120OldsDHendersonDRKitzmanHColeREffects of Prenatal and Infancy Nurse Home Visitation on Surveillance of Child MaltreatmentPediatrics19959533653727862474ChaffinMBardDImpact of Intervention Surveillance Bias on Analyses of Child Welfare Report OutcomesChild Maltreatment200611430131217043315Appendix 1

Search Strategy and Search Terms

Articles for this meta-analysis were identified from a literature search for a wider set of parenting interventions and thus returned a larger set of article citations than might have been returned by a more focused search only for home visiting programs. The overall search strategy built on the search for Kaminski, Valle, Filene & Boyle (2008) which was conducted in September 2002 and included articles published between 1990 and 2002. On September 16, 2010, we conducted a complementary, updated search to include studies published before 1990 and since 2002. The articles returned from the new search were added to the previous database. For simplicity of presentation, the search strategy shown below lists the terms and actions that would have returned the full set of publication years if the entire search had been conducted on September 16, 2010 instead of in two sections.

The search below was conducted via OvidSP using PsycInfo as the database. Terms in quotation marks were searched only as those explicit terms. Terms not in quotation marks were searched as a multipurpose term (ie, .mp) appearing in any relevant field. The option to conduct an “exploded” search was engaged whenever available for an entered search term. An * indicates a “wildcard” search wherein any possible endings of that term were included (eg, behavior* searched for behavior, behaviors, behavioral, etc). The search entries were repeated in Medline with necessary adjustments for that search engine. The returned articles from the two databases were then combined. Unpublished dissertations and duplicates were deleted from the final set of results.
Search StepSearch Entries
S1. Program/Evaluation Terms(parent and (training or education or program)) or ((support or treatment or intervention prevention) and ((parent or family) and results)
S2. General Program Target Terms(parenting skills or home environment or family relations or parent child relations or mother child relations or father child relations or childhood development or at risk protective or (resilient or resilience or resiliency) or child management or competence))
S3. Specific Child Outcome Terms((youth violence or juvenile delinquency or delinquent or conduct disorder or conduct problem or behavior problem or noncompliant or noncompliance or aggression or aggressive or (bully or bullying) or adhd or attention deficit disorder or academic problems or school adjustment or school problems or school dropout or impulsivity or impulse control or externalizing or prosocial or problem solving or communication skills or social skills or discipline or assertiveness or self esteem or drug abuse or substance abuse or alcohol or smoking or cigarette or sexual acting out or abuse or neglect or maltreatment or anxiety or depression or mental illness or suicide or eating disorder or internalizing or emotional adjustment or (Child* and (abuse or neglect or maltreatment or health or injury or violence or ingestion or poison* or attachment or immuniz* or “emergency department”)) or “infant mortality” or ((juvenile or adolescent) AND delinquen*) or (child and (cognit* or language or “social-emotional” or “socioemotional” or “socio-emotional” or physical or health) and development)) or “school readiness” or “school achievement” or “child development” or “developmental delay” or (child AND behavior*) or (child AND disab*) or ((Preterm or “pre-term” or premature) AND birth) or “low birth weight” or “low birthweight”
S4. Specific Maternal/Family Outcome Terms((parent* or family or matern* or mother* or father* or patern*) and (employment or career or stress or depress* or efficacy or “mental health” or health)) or ((subsequent or teen) AND (birth or pregnan*)) or “home environment” or “self sufficiency” or “self-sufficiency” or (parent* AND (skill* or ability*)) or (reduc* AND (crime or “domestic violence” or “family violence” or “intimate partner violence”) or ((community AND coordinate*) or referral*) or (smoking or tobacco)) and (parent and (training or education or program)))
S5. Compiling results from “General Program Target Terms,” “Specific Child Outcome Terms,” and “Specific Maternal/Family Outcome Terms” searchesS2 or S3 or S4
S6. Restricting Program Targets/Outcomes to Parenting Program EvaluationsS1 and S5
S7. Restricting Relevant Parenting Program Evaluations to those delivered in the homeS6 and “home”
S8. Restricting Relevant Home Visiting Program Evaluations to those published in EnglishLimit S7 to English language

PRISMA Flow Diagram for Study Inclusion

Variables Coded for the Analyses and Definitions

VariableDescription or Definition
Home Visiting Content/Delivery Components
Developmental norms and expectationsInformation on typical child development, developmental milestones, and child behavior
Developmentally appropriate care and routinesUsing developmentally appropriate behaviors related to satisfying a child’s primary needs (eg, diapering, dressing, bathing)
Safe or clean home environmentInformation or activities focused on home cleanliness, safety, accident prevention, and first aid
Stimulating home environmentOrganizing environment to promote development (eg, books)
Responsiveness, sensitivity to cues, and nurturingProviding developmentally appropriate responses to emotional needs, such as physical contact and affection
Discipline and behavior managementUsing age-appropriate discipline or management, including discipline-related communication skills
Promotion of child’s socioemotional developmentFostering children’s positive adjustment and well-being such as positive self-esteem, adaptability, creativity, and interpersonal comfort
Promotion of child’s cognitive developmentIncludes using naturally occurring opportunities to promote child language or knowledge by describing aspects of the child’s activity or environment and asking questions
Public assistanceInformation on obtaining or being directly taught to obtain housing or food assistance, SNAP, WIC, TANF, AFDC, welfare
Concrete or instrumental assistanceDirect provision of resources to address basic needs, including transportation services, respite or child care, grocery certificates
Selecting appropriate alternate caregiversInformation or activities related to finding capable child or respite caregivers
Parental relationshipsEnhancing parental relationship (eg, communication between parents)
Parental substance useProviding education or direct services related to substance use
Parental mental healthAddressing mental health issues or directly providing mental health services
Prenatal healthInformation or activities to promote prenatal health and behavior (eg, diet, nutrition, prenatal care, fetal development)
Family planning or birth spacingInformation or activities to promote family planning or birth spacing (eg, optimal intervals, contraception)
Self-, stress-, or anger-managementProviding services for stress-, anger-, or self-management (eg, self-sufficiency skills, such as time management)
Support groupDirectly providing a support group
Social support or social network (need for)Information and activities on the importance of and how to access social support (eg, teaching parents how to identify and access support groups or develop a support network)
Adult literacy or academic achievementInformation on obtaining GEDs, literacy, or other training or education
Problem solvingTeaching the use of problem-solving strategies
Goal settingTeaching parents to engage in goal setting
Case managementIdentifying and linking families to other services and resources (ie, hands-on assistance with contacting, making appointments, helping with forms or eligibility criteria, advocacy)
Rehearsal or role-playingUsing rehearsal, practice, or role playing of techniques or behaviors
Home visitor is professionalUsing professional home visitors, (eg, nurse, psychologist, social worker)
Match between home visitor and client: race/ethnicityPurposive matching of home visitor and client on race and/or ethnicity
Standardized curriculumUsing an established curriculum or curriculum adapted to family needs
Program delivered in language other than EnglishProgram delivered in language other than English
Research Design Characteristics
Random assignmentInvestigators randomly assigned individuals to treatment conditions prior to the intervention, and maintained group assignment in analyses (eg, studies in which intervention-assigned “non-attenders” were analyzed as comparison participants were coded as not using random assignment)
Assessment of initial equivalenceInvestigators reported assessment of group equivalence at baseline on either demographic or outcome measures
No-treatment comparison groupComparison group for a given effect size received no alternate treatment or services
Home visiting tested as a stand-alone interventionIntervention group for a given effect size received only the home visiting program (versus receiving the home visiting program as part of a broader package of interventions)
Timing of outcome assessmentOutcome was measured at 67%–90% of treatment implementation versus at immediate post-test
Outcome Measure Categories
Maternal Life CourseIndicators of maternal health, economic self-sufficiency, educational attainment, and other life outcomes, such as criminal behavior or subsequent pregnancies and births
Birth OutcomesIndicators of the absence of negative birth outcomes, such as prematurity, low birthweight, or childbirth complications
Parent Behaviors and skillsIndicators of parenting behaviors and practices, such as promoting a safe and stimulating home environment, positive parenting behaviors, well-child visits and immunizations
Child Cognitive OutcomesIndicators of cognitive and language development
Child Physical HealthIndicators of positive health outcomes, including the absence of child injury/ingestion, mortality, and illnesses
Child MaltreatmentIndicators of child maltreatment, including child protective services data and self-report of abusive/harsh parenting practices

Number of Studies, Mean Effect Sizes, and Results of Heterogeneity Analyses by Outcome Category

MaternalLifeCourseOutcomesBirthOutcomesParentBehaviors& SkillsChildCognitiveOutcomesChildPhysicalHealth,Illness &InjuryChildMaltreatment
Number of studies in analysis12143224159
Mean ES (95% CI)0.20 (0.07, 0.32)0.061 (−0.08, 0.20)0.23 (0.13, 0.33)0.25 (0.11, 0.38)0.11 (0.00, 0.22)−0.08 (−0.24, 0.07)
Heterogeneity Analysisp<0.02p<.0001p<.0001p<.0001p<.0001p<.01
I252%86%75%78%80%65%

Unstandardized Regression Coefficients and 95% Confidence Intervals from Individual Inverse-variance-weighted Linear Regressions Predicting Outcome Category from Program Components.

Maternal Life CourseOutcomesBirth OutcomesParent Behaviors &SkillsChild CognitiveOutcomesChild Physical Health,Illness & InjuryChild Maltreatment
Developmental norms and expectations0.34 (0.09, 0.60)0.24 (−0.05, 0.54)0.08 (−0.24, 0.39)--
Developmentally appropriate care and routines0.04 (−0.23, 0.31)0.00 (−0.30, 0.30)−0.01 (−0.26, 0.23)--
Safe or clean home environment−0.12 (−0.39, −0.15)−0.25 (−0.62, 0.12)0.04 (−0.21, 0.28)0.09 (−0.20, 0.38)
Stimulating home environment0.20 (−0.16, 0.56)0.44 (−0.08, 0.97)----
Responsiveness, sensitivity to cues, and nurturing0.47 (0.18, 0.76)0.38 (0.02, 0.74)--
Discipline and behavior management0.29 (0.06, 0.51)0.41 (0.13, 0.68)--
Promotion of child’s socio-emotional development−0.18 (−0.51, 0.15)0.21 (−0.09, 0.50)--
Promotion of child’s cognitive development0.01 (−0.26, 0.29)0.27 (−0.01, 0.55)--
Home management----
Public assistance--0.15 (−0.22, 0.53)−0.24 (−0.62, 0.14)--−0.06 (−0.36, 0.25)0.05 (−0.26, 0.36)
Concrete or instrumental assistance--−0.06 (−0.44, 0.33)−0.21 (−0.53, 0.12)0.15 (−0.32, 0.61)−0.07 (−0.35, 0.21)−0.19 (−0.49, 0.11)
Selecting appropriate alternate caregivers−0.06 (−0.23, 0.11)0.12 (−0.28, 0.55)−0.15 (−0.48, 0.19)−0.33 (−0.70, 0.05)0.09 (−0.17, 0.35)0.26 (0.01, 0.51)
Parental relationships--0.12 (−0.36, 0.59)−0.09 (−0.52, 0.35)----0.02 (−0.30, 0.35)
Parental substance use----0.32 (0.06, 0.60)0.07 (−0.35, 0.49)----
Parental mental health0.17 (−0.01, 0.36)--0.17 (−0.13, 0.47)−0.01 (−0.38, 0.36)0.08 (−0.23, 0.39)--
Prenatal health0.22 (−0.14, 0.57)−0.31 (−0.69, 0.07)0.17 (−0.09, 0.43)
Family planning or birth spacing0.01 (−0.17, 0.18)−0.08 (−0.42, 0.27)−0.05 (−0.37, 0.26)
Self-, stress-, or anger-management----−0.24 (−0.60, 0.13)----
Support group----−0.17 (−0.43, 0.09)0.01 (−0.30, 0.31)−0.28 (−0.47, − 0.10)−0.22 (−0.51, 0.06)
Social support or social network (need for)−0.01 (−0.18, 0.16)−0.05 (−0.43, 0.33)−0.20 (−0.46, 0.06)−0.07 (−0.38, 0.24)0.06 (−0.18, 0.30)0.01 (−0.29, 0.31)
Adult literacy or academic achievement−0.03 (−0.13, 0.20)−0.29 (−0.68, 0.09)0.05 (−0.26, 0.36)
Problem solving0.15 (−0.25, −0.56)0.38 (0.03, 0.73)−0.19 (−0.47, 0.10)0.01 (−0.34, 0.36)−0.04 (−0.29, 0.21)0.27 (0.03, 0.51)
Goal setting−0.14 (−0.30, 0.03)−0.01 (−0.46, 0.44)−0.14 (−0.44, 0.17)--0.05 (−0.23, 0.32)0.00 (−0.33, 0.33)
Case management----−0.15 (−0.49, 0.13)−0.17 (−0.48, 0.13)----
Rehearsal or role-playing----0.09 (−0.22, 0.40)0.52 (0.13, 0.91)----
Home visitor is professional−0.04 (−0.24, 0.15)−0.38 (− 0.70, −0.06)0.01 (−0.25, 0.28)0.02 (−0.29, 0.32)0.25 (0.04, 0.45)−0.02 (−0.33, 0.28)
Match between home visitor and client: race/ethnicity0.20 (−0.07, 0.46)0.39 (0.04, 0.75)0.09 (−0.20, 0.37)0.09 (−0.33, 0.50)--
Standardized curriculum0.20 (−0.03, 0.44)--0.14 (−0.13, 0.41)0.05 (−0.29, 0.38)−0.01 (−0.27, 0.26)--
Program delivered in language other than English0.19 (−0.01, 0.39)--0.03 (−0.24, 0.29)−0.21 (−0.50, 0.09)−0.20 (−0.42, 0.03)--

Note: Shaded cells represent component-outcome relationships that were not tested, as the particular component was not expected to contribute to that outcome. Cells containing “--” were unable to be tested due to insufficient variability on the component in programs measuring that outcome. Bold text indicates findings significant p<.05. For Maternal Life Course Outcomes only, the timing of the outcome assessment (ie, during the final third of treatment versus at the end of treatment) was entered as a covariate based on the results of the inverse-variance-weighted ANOVAs predicting effect sizes from study design variables.