CFRP Policy Brief | B.034.0118
The Program Implementation Evaluation (PIE) of the Texas Home Visiting (THV) Program aimed to better understand the factors that advance or limit the successful implementation of evidence-based home visiting programs in Texas, and the ability of these programs to produce positive outcomes for children and families. Over the course of the five-year study, THV served families in nine communities in 17 counties with three home visiting program models totaling 31 separate home visiting programs in high-need areas around Texas.
Regardless of the amount of prior experience primary contractors and/or subcontractors within each community had implementing home visiting programs at the onset of THV, after five years, each of the THV communities was successfully recruiting and enrolling high-risk families. All programs continued to struggle to deliver the intended number of visits and keep families enrolled for at least one year, but they managed to provide most of the needed services in the visits they did have with families. This brief highlights the lessons learned as the THV communities overcame challenges and barriers to get to where they are today.
Community Context Influences Programs’ Ability to Hire Staff and Serve Families
Many of the programs in THV initially had difficulty finding qualified staff, which limited some programs’ ability to serve the targeted number of families during the first year of implementation. Each home visiting program has different requirements for home visitors. For the Nurse-Family Partnership, hiring nurse visitors with a Bachelor of Science in Nursing (BSN) degree was particularly challenging for communities that do not have local BSN programs, and for communities in which the job market for nurses is competitive and the nurses can find higher salaries working for other organizations. Additionally, some home visiting programs in communities with a large Spanish-speaking population found that there was not a sufficient number of Spanish-speaking home visitors to serve their Spanish-speaking families.
The broader economic and political context surrounding several of the THV communities also played a role. The oil boom in West Texas drove up wages and housing prices in the area, making it difficult to compete with the much higher wages being offered by other employers in the area, which led to difficulty hiring and maintaining home visiting staff who demanded higher wages. This boom has recently been followed by a crash in the oil market, which has created issues related to retention in the home visiting programs—some families have moved outside of the community, and some programs report that moms are returning to work and do not have time for home visiting programs.
Immigration is also a major factor in some communities. One THV community had a large Burmese population settle in the area, and finding home visitors who could translate was a challenge. The counties in the Rio Grande Valley (Hidalgo/Willacy) have a large immigrant population from Mexico and Latin America, some of whom are undocumented, which presented a challenge in connecting those families to resources.
Word of Mouth Referrals are the Most Effective Long-Term
In the initial years of THV, recruitment efforts consisted of enrolling any and every possible family into the program to meet capacity numbers. As THV grew, the program developed the ability to rely significantly on referrals through word of mouth. Families who enrolled early on in THV often did not complete the program because they were overwhelmed by the commitment, unsatisfied with the program structure, or unconvinced of the program’s value or benefits. Families who did remain in the program, however, began to tell their friends and family members about it. Over time, staff noticed that families who were referred through word of mouth often “stuck better” and spent more time enrolled in the program. Although relying on word of mouth can take more time for new programs as they build their reputation in the community, we recommend that programs incentivize their current clients to refer their families and friends into the programs.
Retention Begins at Recruitment
Home visiting staff reported that taking the time during recruitment to explain the expectations, requirements, and benefits of the program to potential enrollees was a successful retention strategy. Although THV may lose families who choose not to enroll after learning all that the program entails, THV will benefit from enrolling committed families who understand their roles and responsibilities. Families who choose to enroll in the programs are less likely to become exhausted or dissatisfied with the programs and are more likely to stay longer. We recommend that programs take the time to prudently explain the programs’ expectations and commitment level to potential clients so that they can make a well-informed decision to enroll or not. This helps ensure that clients who do enroll understand the program requirements and are likely to see the program through to completion.
Flexibility Is Key to Serving High-Risk Families
Serving and retaining high-risk families is challenging because these families frequently cancel at the last minute because of conflicting schedules or family issues. Home visiting program staff consistently recommended flexibility as the key to keeping families engaged and enrolled in the program. Examples include giving families temporary breaks from the program, allowing schedule changes, and extending time to complete activities to help keep families engaged. Staff also were flexible with individual visit plans when crises arose and with meeting space to help families feel comfortable. Taking these steps helps families remain in the program, even when unforeseen events temporarily deter progress. We recommend that program staff follow the dosage prescribed by the model as closely as possible, but remain flexible to ensure families remain in the program long enough to benefit.
Data-Informed Decision-Making Requires Clear Definitions
Attrition is a common issue for home visiting programs, and THV is no exception. During the evaluation, the majority of THV families did not remain in the program for longer than one year. Accurate data on attrition are necessary to better inform decisions on how to improve retention in THV programs. The different program models are not consistent in how they define what it means for a family to “complete” the home visiting program across models, and at times, within program model. The models also are not clear in the protocol for exiting a family that has not had a home visit in several weeks or months, and whether to indicate that such a family has completed the program or not. To achieve reliable data on completion and attrition, home visitors must have clear and consistent guidance on when to exit a family and firm indicators for when a family has or has not completed the program. Accurate data on attrition will help inform the programs’ strategies to engage and retain families in their programs.
The Process for Monitoring Fidelity Needs to Be Clearly Defined
The state, communities, and programs are responsible for ensuring that the program sites strictly follow the program implementation models. Each of the program models in THV is evidence-based, but successful program outcomes are dependent upon proper application. If the programs do not produce the anticipated impacts in their communities, inconsistencies in implementation method likely will be the primary explanation as to why. The state must clearly define who is responsible for ensuring the programs are implemented with fidelity—whether that be the state program model leads, the primary contractors, or the home visiting program supervisors. The designated individual will guarantee that the same programs are implemented consistently across regions. If the responsibility is shared, then individual roles need to be explicitly defined. Further, the state should define how those tasked with monitoring fidelity are held accountable.
Community Coalitions Help Programs Better Serve Families
In addition to providing direct services to families through home visiting programs, THV communities also were required to develop or expand existing early childhood coalitions. Coalitions bring multiple services and programs in a community together to coordinate resources and encourage collaboration. Home visiting programs benefit from coalitions, which give them the opportunity to bring greater awareness to their programs, receive referrals from coalition partners, and recruit more families into their programs. Home visiting programs also gain awareness of other resources and programs in their communities and have a better sense of where to refer families with particular needs. We recommend that home visiting programs participate in local early childhood coalitions to better coordinate and link services.
Expectations for Program Success Should Align With the Evidence Base
The three program models currently implemented in THV meet the federal criteria for “evidence-based,” meaning the programs have produced positive impacts in areas of child and family wellbeing in rigorous evaluations. Just because a program is evidence-based, however, does not mean that it will be effective in all populations, for all outcomes, all the time. The different THV programs are based on evidence that varies widely in the populations for whom they have provided benefits and the outcomes on which they have produced an impact. No single home visiting program has evidence of positive impacts in every single benchmark area. We recommend that expectations for the programs’ success be tailored to reflect what each program model’s evidence base suggests is possible. Deliberate implementation is key—implement the program that has a proven track record for the desired outcome.
Though home visiting programs have expanded rapidly across the country as an evidence-based policy choice for supporting families with young children, selecting an evidence-based model is not a guarantee of effectiveness. Implementation is a key determinant of whether or not children and families benefit from home visiting programs. Additionally, the context in which home visiting services are provided can (and does) impact the outcomes achieved by the programs. Home visiting programs do not occur in a vacuum; they are embedded in broader developmental and ecological systems that shape the needs and resources available to families, which, in turn, affect the delivery and efficacy of home visiting programs.
Implementation must be intentional to reap the most benefits from the programs. The program models vary in their target population, the goals they prioritize, and thus in the impacts with which they are associated. To maximize the potential for success, implementation should be targeted—the program goals and evidence base should align with the desired population outcome. The programs should continue to undergo rigorous evaluations to learn which aspects of the program are critical to success and should be prioritized during implementation.