Building Capacity for Evidence Use: CQI Practicum

“I came back from each session and shared what I learned because it was so unique from anything my colleagues or I had been to before…”

This past year, a select group of county child welfare managers from around the state of California convened at U.C. Davis to participate in a five-part CQI practicum focused on reentry to foster care. The purpose of this cohort-based, long term seminar was to support county leaders in using evidence to build and execute CQI plans aimed at reducing reentry. The practicum was hosted by the Northern California Training Academy and designed and delivered by Data Center researchers in partnership with Renée Boothroyd from the UNC Frank Porter Graham Child Development Institute.

Between December 2016 and June 2017, participants traveled to Davis for five separate full-day sessions, each centering on the use of evidence at a different point in the Continuous Quality Improvement (CQI) planning process. We used a four part structure to break down the process and teach the evidence use skills required at each stage:

Session 1: “I observe that…” – Analyzing administrative data to make an observation about reentry and identify opportunities for improvement.

Session 2: “I think it’s because…” – Developing a hypothesis about what might be driving the reentry rate and supporting that hypothesis with evidence.

Session 5: “…which I think will result in…” – Measuring the impact of innovation and change over time.

To maximize job-relevance and the transfer of learning, sessions were linked together through county-specific project work. Using county- and state-level data from Chapin Hall’s Multistate Foster Care Data Archive and real-life measures of reentry (see our latest Webtool Recipe for more on how to create these on your own) participants identified a population within their own county for whom reentry could be improved. From there, instructors coached participants through each of the four phases of the CQI planning process: defining the problem, hypothesizing as to its cause, identifying a solution, and measuring effectiveness. At each stage, students learned best practices for evidence use: articulating analytic questions, identifying the correct analytic population and method, and specifying periods for measuring change over time, to name a few. Homework assignments moved participants through the process and instructors provided customized feedback on each.

Practicum participants came away from the experience with new knowledge and valuable skills to bring back to their home agencies. Scroll down to hear their reactions in their own words.

To stay updated on details for future class offerings, subscribe to the Data Center’s newsletter or contact us to learn more about bringing a course to your state.

From the participants…

“I came back from each session and shared what I learned because it was so unique from anything my colleagues or I had been to before. The difference between entry cohorts and exit cohorts had never been clear to me, particularly how the interpretation of evidence changes depending on which population you choose. Getting clarity on those concepts was an ‘aha’ that helps me in my work reviewing quarterly reports and updating our stakeholders on performance. I absolutely love the “but why” process of drilling into observations. Using the “I observe that…I think it’s because…so I plan to” format to clarify the rationale for a particular initiative, and then asking “but why?” to drill down to root causes was an absolutely brilliant way to come up with interventions that link directly back to the observed problem. I had the very best instructors walk me through my blind spots and ensured I stayed clear from traps.”

Cori L. Allen
Deputy Director
Tuolumne County, CA

“Folks at my level would be particularly well suited for a course like this. People come to those of us in management positions wanting an immediate response, and because of who we are, we try to give answers. Instead this class teaches us to step back and explore what is being asked and see if there are bigger issues beyond the question. I think people at my level need that thoughtful perspective.”

Guy Howard Klopp
Program Manager
Sacramento County, CA

“Too often, we do not take the time to sift through all the extraneous complicating factors and understand the county’s outcomes at its heart. This can unfortunately lead to creating strategies which are based on assumptions rather than empirical findings, resulting in us not having the effects on the outcomes that we are seeking. This course allows the participants to learn how to appropriately analyze data by using best practices in measurement and implementation science strategies. This approach helps the county create efficient strategies focused on the correct issues, and thereby positively affecting the county’s outcome measures.”

Amber Davis
Staff Services Manager
Del Norte County, CA

“The CQI practicum was a gift to me, as it had a significant impact on my own professional development. As a result of what I learned in the practicum, my entire approach to identifying and solving problems in child welfare has become much more disciplined, methodical, and scientific. I now feel much more confident in knowing how to engage and team with child welfare stakeholders around a much more structured continuous quality improvement process that is logical and practical. I was able to quickly apply what was learned and used that knowledge to develop an excellent and simple CQI tool for my managers to help them get into the habit of using a CQI point of view. The quality of the trainers, the materials, videos, tools and discussions were outstanding.”

“I learned how to use administrative data to identify a problem and to observe variation over time, and then to take it one step further – to hypothesize about the source of the problem and say “I think it’s because…” I’ve realized that this approach can be applied to all different kinds of projects; I’m using the process now for another project at work and everyone is excited about it.”

Nkoli Nwufo
Senior Administrative Analyst
Riverside County, CA

“Since my county sent both a program analyst and a data analyst to participate in the practicum, it made it easier to absorb the concepts and to find the value in applying the Plan/Do/Study/Act model to various county-specific situations. I will be applying the Plan/Do/Study/Act method to my county system improvement plan and strategies. I found the method gave me a better conceptual basis for Continuous Quality Improvement than the current method my county is utilizing.”