What Should be Expected when Taking Home Visiting Programs to Scale?

CFRP Policy Brief | B.015.0415

April 2015
PDF version

Though home visiting programs have rapidly expanded across the country as an evidence-based policy choice for supporting families with young children, selecting an evidence-based model is not a guarantee of effectiveness. To guide expectations about the extent to which home visiting programs can affect significant and meaningful change when implemented at the community level, CFRP reviewed the effects of four evidence-based home visiting programs participating in the Maternal, Infant, and Early Childhood Home Visiting Program (MIECHV) on multiple aspects of early parenting. The findings presented in this brief describe potentially, the best possible outcomes after implementation—those that can be expected if the effects of large-scale implementation efforts mirrored those found in the programs’ randomized controlled trials (RCTs). This brief synthesizes the findings of the research to highlight which outcomes home visiting programs are the most likely to improve and for whom.

In 2010, as a part of the Patient Protection and Affordable Care Act, Congress established the Maternal, Infant, and Early Childhood Home Visiting Program (MIECHV), which offered states, jurisdictions, and American Indian tribes $1.5 billion in annual formula grant funding over five years to support the expansion and development of home visiting programs. Importantly, states were required to spend at least three-quarters of the federal funds on home visiting models that met federal standards of evidence-based effectiveness.1 As many policy scholars have noted, that a national initiative brought the importance of evidence-based practice to the forefront of public policy is a triumph for social science and demonstrates the importance of rigorous program evaluation.2 With that triumph, however, comes a responsibility to ensure that the public’s expectations for success of these programs are consistent with what researchers understand about the empirical evidence – will the same positive outcomes found in programs’ randomized controlled trials emerge when those programs are taken to scale? Concomitant with increased investment is an increased concern for the return on investment – policymakers and the public want to know if the programs being invested in produce the outcomes expected of them.3

The primary purpose of this brief is to guide expectations about the extent to which home visiting programs can demonstrate significant and meaningful change when implemented at the community level

To that end, the primary purpose of this brief is to guide expectations about the extent to which home visiting programs can demonstrate significant and meaningful change when implemented at the community level. This brief presents the findings of a larger research paper, currently under review, in which we review the evidence for four home visiting program being implemented with MIECHV funding and compare these effects to population averages and professional recommendations to provide a context in which to interpret those effects. Here, we highlight our review and synthesis of the evidence to illustrate the best possible outcomes that can be expected from home visiting programs given what we know about the prior research.

Method

Data and Sample
We consulted the federal Home Visiting Evidence of Effectiveness (HomVEE) project to identify the relevant research and the Design Options for Home Visiting Evaluation (DOHVE) project to identify the parenting outcomes prioritized by MIECHV. Our review focused on four widely-used home visiting program models that met the Department of Health and Human Services “evidence-based” criteria and that are included in the legislatively mandated, large-scale evaluation of the effectiveness of MIECHV-funded home visiting programs: Early Head Start-Home Based (EHS-HB), Healthy Families America (HFA), Nurse-Family Partnership (NFP), and Parents as Teachers (PAT).

For each of the selected home visiting program models, this study examined the evidence base for six parenting outcomes.

For each of the selected home visiting program models, this study examined the evidence base for six parenting outcomes (prenatal care, breastfeeding, well-child visits/immunizations, learning support behaviors, child maltreatment, and harsh discipline). Home visiting programs should have the greatest and most immediate impact on parenting outcomes relative to child outcomes because home visiting programs target parenting directly as a mechanism of change in children.

Review Process
An exhaustive search of the HomVEE website, program model websites, and government agency websites produced a combined total of approximately 60 research studies for review: 11 articles were reviewed for EHS-HB, 18 for HFA, 19 for NFP, and 10 for PAT. For each program model, we identified which, if any, of the parenting outcomes had been examined. Studies that did not include an examination of at least one of the six parenting outcomes of focus were excluded from our review. For each of the remaining studies, we identified how the outcome(s) was measured, whether there was a significant effect or not, and the size of the effect if evident.

Results

The results of our review are summarized for each parenting outcome below. In general, the research shows that home visiting programs have the greatest, albeit modest, effect on parents’ support for children’s learning and in reducing the prevalence of child maltreatment, but that these effects are strongest for the most disadvantaged program participants. The research provides less support for the effect of home visiting programs on early health behaviors including prenatal care, breastfeeding, or well-child visits, or on reducing the use of harsh parenting.

Prenatal Care
Across the four home visiting programs, the evidence linking home visiting programs to prenatal care is thin. The strongest, albeit still modest, evidence for a link comes from an evaluation of the NFP program, which shows impacts on attendance at child birth classes and knowledge of how to access prenatal care, but does not show an impact on actual prenatal care visits.4 More limited, but still promising, evidence comes from studies of EHS-HB and HFA.5,6 Evaluations of both programs suggest participation in the program is associated with accessing prenatal care, but neither provide enough information or rigor to draw valid conclusions about impacts. No evaluation of the PAT program has examined prenatal care.

b0150415_fig1
 

Breastfeeding

The evidence for an impact of home visiting programs on breastfeeding is driven almost entirely by the modest findings from an evaluation of NFP, in which nurse-visited mothers were significantly more likely to have attempted breastfeeding (though, as the authors report, there were no differences in duration of breastfeeding).7

b0150415_fig2

A study of HFA showed no significant impact on breastfeeding and the evidence provided in the EHS-HB evaluation does not lend itself to causal conclusions.8,9 Breastfeeding has not been examined as an outcome in evaluations of the PAT program.

Well-Child Visits and Immunizations

For the most part, home visiting programs have largely been shown to be ineffective at increasing rates of well-child visits and immunizations (Table 2). It is important to note, however, that the comparison group in many studies showed high levels of achieving adequate well-child visits or meeting immunization standards. This makes it harder to detect statistically significant differences, even if positive changes are seen among program participants. Studies of NFP and EHS-HB showed no statistically significant impact of the programs at increasing immunizations; nor is there any evidence that NFP has a statistically significant effect on well-child visits.10,11 There is some evidence that HFA may increase the number of well-child visits at age three, but there is little empirical support for the effectiveness of HFA on well-child visits or immunizations within the first two years.12 Two evaluations of PAT produced mixed findings, although one study did find a statistically significantly higher share of children who were fully-immunized for his/her age as compared to children in the comparison group.13,14

b0150415_fig3

 

Learning Support

Many home visiting programs aim to educate parents about the importance of supporting children’s early learning through frequent reading and a stimulating home environment and provide parents with the tools to support their children’s early learning.15 In general, evaluations of home visiting programs show fairly positive impacts on parents’ support for children’s learning, though the evidence is strongest for the most disadvantaged program participants (e.g., poor, unmarried teens; very-low income participants).

b0150415_fig4

For example, studies of EHS‐HB have found increased family support for children’s language and literacy learning at ages two and five, a greater percentage of parents who read to their children daily, and parents engaging in more teaching activities.16,17 Studies of NFP show positive results on the HOME Inventory and studies of PAT found that low‐income parents in the treatment group were more likely to read and sing with their child than parents in the comparison group.18,19 In contrast to the studies of EHS‐HB, NFP, and PAT, there is no support for an impact of HFA on parent’s support of their children’s learning. This may reflect the primary goal of HFA, which is to prevent child abuse and maltreatment through parental education and support rather than to promote children’s school readiness.

Child Maltreatment

The results from several evaluations and randomized control trials show that home visiting programs may be an effective approach to reducing the prevalence of child maltreatment, but generally only for the most disadvantaged or at-risk families (e.g., first-time mothers, families who previously interacted with child protective services, low-income families, and poor, unmarried teens). Studies of HFA, NFP, and PAT find effects for certain subgroups of their samples, but show no overall impact of program participation on reductions in child maltreatment. Mixed evidence emerges from the single EHS-HB study that has looked at child maltreatment.

b0150415_fig5
 

Harsh Discipline

In contrast to the support for reductions in child maltreatment, there is little support for a link between home visiting programs and a reduction in the frequency with which parents use spanking as a disciplinary technique. The research linking participation in EHS-HB and HFA with discipline practices is mixed. One small study of EHS-HB found that low-income families reported lower frequencies of spanking at age three,21 however, these results contradicted findings in other studies. Studies of HFA found that a larger share of mothers reported never slapping their children’s hand, fewer acts of minor physical aggression, and fewer acts of harsh parenting in the past week.22 There is little evidence to support an effect of NFP on discipline nor is there a research base for the link between PAT and harsh discipline.

b0150415_fig6
 

Conclusion and Implications

What, then, can be realistically expected from home visiting programs? The findings from the present review provide a basis to be optimistic about the improvement home visiting programs can have in some areas of child and family well-being. The findings also highlight, however there are other areas of child and family well-being where the evidence is not very strong, suggesting that expectations for success in these areas may need to be tempered.

The research suggests that these programs may positively influence parents’ support of their children’s learning and reduce rates of child maltreatment

If home visiting programs target the most at-risk families (i.e., young and poor parents, parents with a history of child maltreatment), the research suggests that these programs may positively influence parents’ support of their children’s learning and reduce rates of child maltreatment. Because the most at-risk families may also be the most difficult to reach in center-based settings, home visiting programs are at an advantage in that the very same high-risk families they need to target are the same families they may be better suited to reach.

To be the most effective, home visiting programs should be provided in conjunction with other services and support. The findings from this review indicate that these programs have the greatest potential to influence the processes that happen within the home environment, but are considerably more limited in their ability to influence outcomes that rely on resources and factors external to the home. Delivering home visiting programs within an organized system of early childhood services and support would likely allow home visiting programs to be more effective and allow families to be better served by them.

  • Citations
    1 U.S. Department of Health and Human Services, Administration for Children and Families, Administration on Children, Youth and Families. (2013). Home Visiting Evidence of Effectiveness. About Us: Executive Summary. Retrieved from http://homvee.acf.hhs.gov/document.aspx?rid=5&sid=20&mid=2.
    2 Astuto, J. & Allen, L. (2009). Home visitation and young children: An approach worth investing in? SRCD Social Policy Report, 23, 1‐24.
    3 Boller, K., Daro, D., Del Grosso, P., Cole, R., Paulsell, D., Hart, B., Coffee‐Borden, B., Strong, D., Zaveri, H., & Hargreaves, M. (2014). Making replication work: Building infrastructure to implement, scale‐up, and sustain evidence‐based early childhood home visiting programs with fidelity. Children’s Bureau, Administration for Children and Families, U.S. Department of Health and Human Services.
    4 Olds, Henderson, Tatelbaum, & Chamberlin, 1986
    5 Kisker, E., & Kuhns, C. (2004). Health and Disabilities Services in Early Head Start: Are Families Getting Needed Health Care Services? (No. 3983). Mathematica Policy Research
    6 Daro, D. A., & Harding, K. A. (1999). Healthy Families America: Using research to enhance practice. The Future of Children, 9(1), 152‐176.
    7 Kitzman, H., Olds, D. L., Henderson, C. R., Hanks, C., Cole, R., Tatelbaum, R., et al. (1997). Effect of prenatal and infancy home visitation by nurses on pregnancy outcomes, childhood injuries, and repeated childbearing. A randomized controlled trial. JAMA: The Journal of the American Medical Association, 278, 644–652.
    8 Mitchell‐Herzfeld, Izzo, Greene, Lee, & Lowenfels, 2005
    9 Kisker & Kuhns (2004).
    10 Kitzman et al. (1997).
    11 Chazan‐Cohen, R., Raikes, H. H., & Vogel, C. (2013). Program subgroups: Patterns of impacts for home‐based, center‐based, and mixed‐approach programs. In J. M. Love, R. Chazan‐Cohen, H. Raikes, & J. Brooks‐Gunn (Eds.), What makes a difference: Early Head Start Evaluation Findings in a Developmental Context. Monographs of the Society for Research in Child Development, 78, 93‐109.
    12 Caldera, D., Burrell, L., Rodriguez, K., Crowne, S. S., Rohde, C., & Duggan, A. (2007).Impact of a statewide home visiting program on parenting and on child health and development. Child Abuse & Neglect, 31(8), 829–852; Duggan, A. K., McFarlane, E. C., Windham, A. M., Rohde, C. A., Salkever, D. S., Fuddy, L., et al. (1999). Evaluation of Hawaii’s Healthy Start program. Future of Children, 9(1), 66–90; discussion 177–178; Mitchell‐Herzfeld, S., Izzo, C., Greene, R., Lee, E., & Lowenfels, A. (2005). Evaluation of Healthy Families New York (HFNY): First year program impacts. Albany, NY: University at Albany, Center for Human Services Research; Landsverk, J., Carrilio, T., Connelly, C. D., Ganger, W., Slymen, D., Newton, R., et al. (2002). Healthy Families San Diego clinical trial: Technical report. San Diego, CA: The Stuart Foundation, California Wellness Foundation, State of California Department of Social Services: Office of Child Abuse Prevention.
    13 Wagner, M., Clayton, S., Gerlach‐Downie, S., & McElroy, M. (1999). An evaluation of the northern California Parents as Teachersdemonstration. Menlo Park, CA: SRI International
    14 Wagner, M., Lida, E., Spiker, D., Hernandez, F., & Song, J. (2001) The multisite evaluation of Parents as Teachers home visiting program. Menlo Park, CA: SRI International.
    15 Astuto, J. & Allen, L. (2009). Home visitation and young children: An approach worth investing in? SRCD Social Policy Report, 23, 1‐24.
    16 Chazan‐Cohen et al. (2013).
    17 Jones Harden, B., Chazan‐Cohen, R., Raikes, H., & Vogel, C. (2012) Early Head Start Home Visitation: The role of implementation in bolstering program benefits. Journal of Community Psychology, 40(4), 438‐455.
    18 Kitzman et al. (1997)
    19 Wagner, M., Spiker, D., & Linn, M. I. (2002). The effectiveness of the Parents as Teachers program with low‐income parents and children. Topics in Early Childhood Special Education, 22(2),67–81.
    
    
  • PDF Version