How do we know that a program or initiative is making a difference? Evaluation can be complex and nuanced. In order to meet the diverse needs of clients and communities, CGR provides a wide range of services to funders, service providers, K-16 institutions and communities looking to maximize the value of their investments and do the most good. We work with clients to select the approach that best matches their program needs and development stage. This may include building evaluation capacity through creating or revising evaluation plans for programs at all stages of development (e.g. planning, early implementation, fully developed), and developing logic models, measurement frameworks, research questions, and methodological approaches (qualitative and/or quantitative). For some programs, a case study design is appropriate; those that are solidly implemented with a well executed data collection infrastructure may engage in more rigorous, outcome-focused evaluation, such as quasi-experimental evaluations comparing outcomes for program and control groups.
In all cases, evaluations need to have practical impact and be able to inform decisions, and they must represent the perspectives and needs of the people served by the program. Participatory evaluation, where program stakeholders play an active role in the evaluative activities, is often an excellent choice.
Featured Work
Monroe Community College Title III Evaluation
CGR was engaged by Monroe Community College (MCC) to serve as the external evaluator for the federally funded Title III: Advancing Pathways for Partnerships in Learning through Integration of Education and Data (MCC APPLIED) grant (2019-2024). MCC’s Title III grant plan is focused on creating an enhanced advisement system to improve student outcomes. The grant’s goal is the creation of a user-friendly platform available to students, staff and faculty that combines various types of student information previously found in separate applications into a single integrated system. CGR’s evaluation documented continuous implementation progress with all Year 1 – Year 3 program objectives being met despite challenges encountered during Covid-19.
Cullen Foundation Evaluation Capacity Building Support
Since 2021, CGR has partnered with the Cullen Foundation in Buffalo, NY to build evaluation capacity of non-profit organizations that provide Out of School Time (OST) programs in Western New York. Our support, based on organizational need, may focus on theory of change and logic model revision or development, measurement framework creation, data collection template design, survey development, academic and socio-emotional learning assessment selection, data dashboard design and data analysis. Past and current clients include the Belle Center, Big Brothers Big Sisters of Erie, Niagara and Southern Tier, Buffalo Maritime Center, Buffalo Niagara Waterkeeper, City Swim Project and Roswell Park Comprehensive Cancer Center, At the completion of each project, an organization is able to apply for additional Cullen Foundation funding, engage in systematic data collection, and/or conduct evaluation studies.
THRIVE Buffalo Evaluation
The THRIVE Buffalo initiative was established with Empire State Poverty Reduction Initiative (ESPRI) funds to reduce poverty, provide supports to address employment barriers (e.g. transportation and childcare) and increase access to jobs with family-sustaining wages via a holistic workforce development pipeline and supportive services approach. CGR used a mixed method research strategy to evaluate program outcomes. Participant data extracts were analyzed to ascertain background information, employment status, services and supports used and job placement outcomes. Stakeholder interviews with partner organizations, employers and participants provided insights about program implementation, quality of program components and impact. Surveys were also used to gauge participant experiences. THRIVE served 358 participants through recruitment and outreach activities. The majority of participants were African American (75%), female (60%), and employed either full or part-time (55%). Lack of information and transportation were the top two employment barriers. Wraparound supports provided the necessary resources needed to obtain and keep a job with over 40% of participants receiving at least one wraparound support (e.g. transportation and emergency funds). Thirty-eight percent (137 out of 358) of THRIVE participants were placed in jobs. All of the white program participants were placed in at least one job. In contrast, 39% of African Americans and 28% of Hispanics obtained one or more jobs. THRIVE’s job retention rate was 83% at 30 days, 49% at 60 days, 24% at 90 days and 8% at 180 days. Participant survey and interview responses indicate that THRIVE wraparound supports provided them with the most value. Financial education and coaching provided participants with important economic tools. THRIVE confronted a number of implementation challenges including lack of program coherence, leadership conflict, inconsistent program expectations, limited accountability, job availability and key partner departures.
Full Service Community Schools Evaluation
CGR is evaluating the United Way of Buffalo and Erie County’s Full-Service Community Schools (FSCS) program (2020 -2025) that provides a variety of student and family supports at two schools in Cleveland Hill Union Free School District located in Cheektowaga, New York. A key focus of this project is to eliminate academic and disciplinary disparities for Black students, English Language Learners and students with disabilities. CGR also evaluated the United Way of Buffalo and Erie County’s Full-Service Community Schools (2018-2021) in the Buffalo Public Schools. The Cleveland Hill and Buffalo Public evaluations show that students and families received necessary social emotional programs, wraparound services and academic support. However, improvements in academic outcomes, school-based disciplinary disparities and school attendance have been the most challenging objectives to achieve.
RCSD Summer Learning Study
CGR was engaged by the Rochester City School District (RCSD) with support from the Wallace Foundation to investigate, through the use of focus groups of parents and children, the reasons for good and poor student attendance at summer learning programs. CGR created interview protocols and worked with RCSD to design an approach intended to reach about 100 families, including 1) those whose children attended summer learning programs in 2016 and had high attendance, 2) those who enrolled but had poor attendance, and 3) those who did not attend any district-affiliated summer learning program. CGR found that factors contributing to attendance included the quality of the program, the mix of academic and non-academic activities, transportation and perceived quality of food served for lunch. Based on the findings, we recommended that the district work to improve parents’ access to information about summer offerings, improve the mix of academic and non-academic activities in some programs, and investigate the quality of programs as may be needed.
Children First Scholarship Evaluation
CGR was engaged by The John R. Oishei Foundation and the Health Foundation for Western and Central New York in 2015 to complete a multi-year evaluation of the Children First Scholarship Program. This program, which is funded by a number of foundations, provides scholarships to families with children in low-performing schools in the Buffalo City School District, so that these children can attend a private school of their choice in Buffalo and the surrounding areas. Our evaluation of the fourth year of the program used student achievement data and student, parent and teacher survey results to assess the impact of scholarships.
