On August 17-18 in Chicago, with the generous support of the Annie E. Casey Foundation, NASBO convened state budget directors and analysts from 34 states for insightful discussions and presentations on how states can advance the use of performance data and evidence in budgeting, planning and management. Throughout the course of the meeting, participants heard from their peers in other states as well as experts from several nonprofit organizations about the challenges and opportunities associated with the use of evidence-based policymaking and data analytics in state government. Some common themes, lessons, and promising practices also emerged from the sessions.
Evidence-Based Policymaking & the Budget Process
With a rising number of rigorous evaluation studies available and more accessible, states are increasing their use of evidence in decision-making. Sara Dube and Nick Dantzer of the Pew-MacArthur Results First Initiative kicked off the meeting with a national overview of where states are with respect to evidence-based policymaking, drawing on the findings from their 50-state report released earlier this year. They also shared some practical lessons on implementing and executing Results First (or a similar model), pointing to state-specific examples of how different governmental entities use evidence in the budget process, including the executive budget office in Colorado, the legislature in Mississippi, and a criminal justice agency in New York State. Embedding evidence requirements into budget instructions, establishing a percentage of budgeted funds to be designated to evidence-based programs, and incentivizing providers to use evidence-based programs through contracts and grants are a few of the tools states are using to advance evidence-based policymaking.
By developing program inventories, determining which programs are evidence-based and what their expected benefits are, states can conduct cost-benefit analysis to measure likely return on investment of different programs and compare alternatives. This fuller understanding of program benefits (and costs) can help ensure scarce resources are distributed most effectively and efficiently. And in difficult budget times, programs that can demonstrate evidence of effectiveness may be able to avoid funding cuts.
The Potential of Integrated Data Systems
Dennis Culhane from Actionable Intelligence for Social Policy at the University of Pennsylvania and Bob Goerge from Chapin Hall at the University of Chicago spoke about how states and other jurisdictions are developing and using integrated data systems (IDS) to harness the power of administrative data. Most often, an IDS will include administrative data across a variety of state agencies, including child welfare, juvenile and adult justice, health care and Medicaid, housing, education, behavioral health, workforce development, employment and others. When individual level (deidentified) data are linked, this can enable research and program evaluation to help address complex, multi-sector social problems affecting children and families, such as educational achievement gaps, prisoner recidivism, and the opioid crisis, which are not easily solved when agencies work in silos.
An integrated data system is not primarily an IT project, but rather about setting up a governance structure that establishes and fosters collaboration and data sharing across agencies. To establish an IDS, policymakers and researchers must grapple with issues around governance and ethics, legality, technology and data security, and data standards. However, if a state can overcome these hurdles, an IDS can be a fairly low-cost tool with the power to yield high returns if implemented and used effectively. Interest in creating an integrated data system is spreading, and nine states are already signed up to participate in AISP’s Learning Communities, designed to help new sites launch an IDS.
Lessons from States on Using Evidence and Data
Participants also had the chance to hear directly from states that are at various stages of implementing evidence-based approaches in the budget process as well as other data-informed efforts to improve government efficiency and effectiveness. Minnesota explained how their budget office has implemented the Results First model, including completing program inventories and conducting cost-benefit analysis, and shared some early lessons learned from the process. Illinois discussed the activities and impact so far of the Budgeting for Results Commission in the state, established by legislation in 2010. Utah talked about how the budget office has helped state agencies implement the SUCCESS framework to improve government performance and operational capacity, as well as requiring agencies to provide information on expected outcomes and evidence of program effectiveness in budget requests. Michigan talked about executive branch efforts to consolidate core “good government” functions under the new statewide Office of Performance and Transformation, created by executive order in 2016. Delaware shared more about its new Government Efficiency and Accountability Review (GEAR) Board, also created by executive order, to improve strategic planning and government performance. Many other participants in the room chimed in with questions, examples, and advice based on their own states’ experiences and efforts to use data and evidence in governing.
Overall, several common themes, key lessons and promising practices emerged from these discussions based on state experiences using evidence and data in budgeting and policymaking. For example:
- Change is incremental: Integrating performance measures and evidence-based approaches into budgeting can be a lengthy process, and will not happen overnight. Implementation tends to be evolutionary and happens in stages.
- Statutory framework can be helpful: States that have passed legislation to require performance management, evidence-based policymaking, and other data-driven efforts have generally found that having a statute in place helps legitimize and sustain an initiative, particularly beyond the current administration. It is also helpful when the legislature provides funding for dedicated staff resources to support the effort.
- Fidelity monitoring: To achieve the goals of evidence-based policymaking – more effective, efficient allocation of resources that ultimately leads to better outcomes – it is important to ensure that effective evidence-based programs are not only funded, but also implemented as intended. Establishing requirements for and dedicating resources to implementation oversight, quality assurance and program evaluation can support fidelity monitoring efforts.
- Consolidating “good government” efforts: Some states have moved towards centralizing various initiatives aimed at improving government efficiency and effectiveness (such as continuous improvement, performance management, budgeting for results) under one statewide office or commission, and appointing agency representatives to coordinate those efforts at the agency level.
- Approach needs to be tailored to fit the state: States differ in their appropriation laws and procedures, elected official term limits, whether they have a contractual or county-delegated system for delivering social services, gubernatorial budget authorities, and so on. When looking to another state’s approach to evidence-based policymaking or performance budgeting as a model, it is important to be mindful of how the approach might need to be tweaked (or substantially revised) to work within another state.
- Another “tool in the toolbox”: Cost-benefit analysis and other evidence-based approaches are tools to better inform decision-making, but they will never be the sole – or even primary – basis for budget decisions. Constitutional and statutory requirements, political priorities, distributional concerns, resource constraints, federal regulations and other considerations will always play a role.
The meeting sessions and suggestions from participants provided NASBO a number of ideas for future research and activities. Going forward, NASBO plans to develop a working inventory of various state-level initiatives, programs and organizational structures focused on using evidence and data in decision-making, with other state-specific information to give context for making informed comparisons. NASBO will also explore more ways to highlight best practices and successful models for specific topics of interest in this area, such as sustaining an evidence-based initiative across administration changes, advancing the use of data analytics, and leveraging local research communities. And we look forward to convening state budget officers and analysts together for more fruitful discussions and trainings
in the future, as we expect interest in how to integrate evidence, performance measures, and “big data” into state budgeting and management decisions will only continue to grow.
For more resources on this topic from NASBO, check out:
If you have questions or would like more information on this topic, please contact Kathryn White at email@example.com or 202-624-5949.