Organizations in the business of providing and/or supporting social programs are increasingly required by funders to engage in data collection, analysis, and reporting (processes involved in program evaluation), in order to provide evidence of program effectiveness. Beyond consistently demonstrating program outcomes, at Invest in Kids (IIK), we are committed to using program data more intentionally in ways that inform decision making and promote quality improvement; this practice is exemplified through the work of The Incredible Years® (IY) team. Our use of data to inform decision making is one of the Implementation Drivers: The Decision Support Data System.
The IY team at IIK provides high-quality implementation supports, including ongoing training and coaching for the teachers and Parent Program Facilitators delivering IY across Colorado, as well as regular observations of their fidelity to the program model. Fidelity refers to how closely the program being delivered in a classroom or parent group resembles the program as it was designed in a research setting. IY fidelity data is reviewed regularly during IIK-IY team meetings that take place throughout the program year. During these meetings the IY team reviews data, complied and analyzed at IIK, in various forms (i.e., graphs, charts, etc.), so that patterns can be identified and discussed by the entire IY team.
One of the main questions that we look to answer is whether there is a general pattern of improvement over time – in other words, does the average fidelity rating show an increase across the program year? This question is central to the team’s work of providing regular training and coaching with the purpose of promoting a high level of program fidelity. The team also uses the lessons learned during the fidelity data discussions to reflect on their coaching protocols and make any changes that would better support teachers or Parent Program Facilitators in delivering the program with fidelity. The benefit of regularly monitoring this data is that we are able to observe whether there are corresponding patterns of improvement in the fidelity ratings. This is just one example of how our IY team successfully leverages the fidelity data to improve our coaching and the implementer’s delivery of IY.
