A Framework for Evidence-Based Systems-Level Change

CFRP Policy Brief | B.036.0718

July 2018
PDF version

Why Systems-Level Change?

Researchers argue that direct service programs should be embedded within a larger system of support to have an impact large enough to change community-level indicators. The complex needs of at-risk families often extend beyond what single programs can provide in isolation and many community-level issues negatively impacting families cannot be addressed with a service-level only approach. The most vulnerable families served may experience poverty, face difficulties accessing health services for family needs, lack access to early childhood care and other programs that can prepare children for school, and struggle to meet basic needs including food, housing, and electricity. Service coordination is an important step toward developing a comprehensive system for children and families within a community, but communities have to work beyond service coordination to implement system-level strategies that address broad policy, practice, or community infrastructure issues that impact young children or families and benefit the community-at-large (Figure 1).

CFRP Policy Brief B.036.0718 Framework for Evidence-Based Systems-Level Change

What is Evidence-Based Systems-Level Change?

Although many factors are important for making progress toward systems-level goals (including having clear, agreed-upon, and measurable goals; stakeholder organizations who are ready for collaboration; external funding; political champions), missing from most initiatives, but critical to success, is the need for communities to identify evidence-based drivers of change when identifying strategies to achieve their goals. In other words, when communities decide upon their goals, they need to identify the known predictors of their goals and align their strategies and action steps accordingly. Many coalitions attempt to reinvent the wheel—often haphazardly hypothesizing drivers of change, when decades of research and prior collective impact initiatives have already identified drivers of change that are evidence-based (i.e., we know they work). Importantly, community experience and voice are evidence too, but should be complemented with research and practice. Identifying evidence-based drivers of change and developing strategies to move the needle on those drivers is key to success in making progress toward community goals.

The Framework

Evidence-based systems level change (SLC) occurs over the course of five phases, often iterative, beginning with identifying which community partners need to be at the table (Figure 2).

CFRP Policy Brief B.036.0718 Framework for Evidence-Based Systems-Level Change

Foundational Activities

In the first phase, foundational activities, stakeholders are identifying which community partners need to be at the table (Figure 3), whether partners are ready and able to collaborate, and developing the specific and measurable goals for the community.

CFRP Policy Brief B.036.0718 Framework for Evidence-Based Systems-Level Change

Importantly, community stakeholders are also identifying how to measure and track change for their agreed upon goals. It can take coalitions months or years to coalesce and agree on a set of measurable goals. As the systems-level work evolves, new stakeholders may need to be invited to the table, or priorities may need to be shifted depending on certain factors, including the local economic and political context.

Collaborative Activities

Once a coalition of stakeholders, who are ready to collaborate, develop and commit to a set of specific and measurable goals for the community, the collaborative activities phase begins. The collaborative activities phase includes identifying evidence-based drivers of change for each goal and assessing the status of each evidence-based driver in the community. Stakeholders review the research literature and examine other community-wide initiatives with similar goals to identify the known predictors/drivers of change for their goal (Figure 4).

CFRP Policy Brief B.036.0718 Framework for Evidence-Based Systems-Level Change

As an example, if a community wanted to improve children’s school readiness (and importantly, had a much more specific and measurable school readiness goal), they would first identify the key predictors of children’s school readiness (Figure 5).

CFRP Policy Brief B.036.0718 Framework for Evidence-Based Systems-Level Change

Assessing the status of each evidence-based driver of change in the community lays the groundwork for developing strategies and action steps. If, for example, local data indicate that fewer than half of children are being read to on a daily basis (part of a high-quality home learning environment), stakeholders can work to identify strategies (evidence-based to a large degree) and action steps to increase the prevalence of daily reading (which aligns with increasing supportive and enriching home environments). There may be barriers that need to be removed (e.g., limited access to the library) or existing supports that need to be strengthened (e.g., expanding an existing free book program). Identifying which stakeholders and coalition members are particularly well suited to work on specific strategies is also a key part of the collaborative activities phase.

Importantly, the collaborative activities phase is revisited regularly throughout the journey toward systems-level change. Stakeholders will need to come back together often to reevaluate community context to identify any existing strengths to leverage in their progress toward systems-level change, continuously reassess the status of each evidence-based driver and reflect on progress toward both interim and overall goals; develop new strategies and action steps based on reevaluations; and as stakeholders and coalition members come and go, reassign responsibilities to partners.

Indicators of Progress

The indicators of progress, phase three, provide stakeholders with a framework to develop strategies and action steps, and for assessing and measuring the interim changes occurring in a community as they work toward systems-level change. Each indicator of progress represents an aspect of the community that can be improved or enhanced in multiple ways to nudge the needle and move a community forward in their systems-level work. Each indicator should also be measurable. Finding ways to track the indicators of progress is key for demonstrating interim progress. The indicators of progress framework includes: 1) policies and practices; 2) physical environment; 3) public perception and awareness; 4) peer networks and support; 5) physical planning and space; and 6) programmatic availability and alignment (Table 1).

CFRP Policy Brief B.036.0718 Framework for Evidence-Based Systems-Level Change

Importantly, the indicators of progress align with the evidence-based drivers of change identified by the stakeholders. Once stakeholders have identified and agreed upon a set of evidence-based drivers of change for the larger goal, they can use the indicators of progress framework to develop strategies and action steps (Figure 6) to target each driver of change (and ultimately, the long-term goal). For example, for each evidence-based driver, stakeholders could identify policies or practices that need to change, develop new programming to meet an identified need in the community, and/or find ways to shift the public perception or understanding.

CFRP Policy Brief B.036.0718 Framework for Evidence-Based Systems-Level Change

Often times there is evidence for which strategies can support an evidence-based driver of change, but communities should think creatively here as well. Not all stakeholders or community members participating in an initiative will participate in developing strategies for each evidence-based driver, nor will they take responsibility for all strategies for a particular evidence-based driver. Stakeholders should identify who (based on members’ skillsets, interests, connections to the community, and available resources) should take responsibility for carrying out each strategy. By measuring and tracking the strategies, stakeholders can rely on this framework to demonstrate their progress. Actualized strategies become indicators of progress toward the larger goal.

Using the example of a community who has selected increasing children’s school readiness as their long-term goal, one can see how the indicators of progress framework provides structure for identifying strategies and measuring progress in Figure 7.

CFRP Policy Brief B.036.0718 Framework for Evidence-Based Systems-Level Change

In this example, the community identified access to healthy food as an important evidence-based predictor of children’s school readiness (i.e., an evidence-based driver of change). For the access to healthy food evidence-based driver, the community identified two potential strategies:

  1. A physical environment strategy to develop a community garden to increase access to healthy food; and
  2. A public perception/awareness strategy of launching a public awareness campaign on nutrition’s role in healthy child development.

As these strategies are accomplished, they become indicators of progress toward the larger goal. Even before the larger school readiness goal is met, the community can show that by developing 10 new community gardens, and launching public awareness campaigns that reached 10,000 people, they are improving children’s access to healthy foods, which will ultimately increase children’s school readiness.

Outputs

Outputs represent the short-term metrics indicating a community is on the right track. Outputs represent metrics for the indicators of progress. For the community with a school readiness goal, they may measure the number of books provided to families each month or the number of families who have received at least one book. Outputs help quantify a community’s progress toward their goals.

Outcomes

The measurable outcomes are the community’s overarching measurable goals. Outcomes can also be examined more broadly by identifying what the community would like had the coalition not existed; what policies or programs would not exist but not for the coalition, and if the coalition went away, what would remain in the community? The outcome in the school readiness example would be the percentage increase in children who pass their Kindergarten screener and are identified as school ready after a specified amount of time has passed (e.g., five years)

Conclusion

Decades of research into the predictors of many aspects of child and family wellbeing suggest that we often know the evidence-based drivers of change for the intended outcome and the evidence-based programs or practices that affect these evidence-based drivers of change. Instead of recreating the wheel with every initiative, stakeholders should take advantage of what we already know and develop strategies that target evidence-based drivers of change to help ensure later success in reaching the long-term goal. Leveraging known predictors of a specific goal sets communities up for success because it helps focus often-limited resources in the most efficient way possible.

A community’s interim progress toward systems-level change is measurable and should be measured. The indicators of progress developed here reflect key aspects of a community that need to change to move a community forward in their systems-level work. The indicators of progress serve a dual purpose in helping communities first identify and develop strategies to meet their goals and second, assess interim progress toward the larger goal.