Overview
Trewon conducted an impact evaluation of the Gateway to Growth Program, a vibrant, year-round, out-of-school time (OST) science technology, engineering, and math (STEM) program, for the City Gate Foundation of Washington, DC. Established in 2000, City Gate is a bulwark of the community it serves, maintaining a strong commitment to its mission to provide services to the underserved of the District of Columbia, particularly its youth and children. It currently supports a wide range of education programs and social services.
In 2012 City Gate received a three-year 21st Century Community Learning Center (CCLC) grant through the Office of the State Superintendent of Education to implement its multisite, STEM-based after school program to students in kindergarten through 12th grade. With associated activities for parents and guardians, the program works to build student capacity and interest in STEM, reading, and writing.
Using Program Goals to Measure Effectiveness of Implementation
With the understanding that early preparation and identification of early warning signs of academic difficulties can help stem future dropout, the program seeks to enhance the rate of students to successfully graduate high school graduation through academic support from grades K through 12. Trewon used the program’s objectives and activities to structure the evaluation and to answer to evaluation questions posed.
Triangulating Data
To assess this multi-site program Trewon worked with program staff to measure the implementation and impact of the STEM-based afterschool program on student and parental satisfaction and academic outcomes. We tracked change on multiple variables across two participant population groups (Grades K-2 and 3-5). Variables, such as, level of student attendance in OST activities, classroom attendance, student grades, math and language arts assessment scores, student engagement in the STEM program and school setting, community partnerships, and family member engagement in program events were measured using qualitative and quantitative data sources.
Working closely with program staff, we identified several avenues to collect primary data for this three month study. We developed instruments—fidelity checklists and participation guides, consent forms, and focus group interview protocols—to conduct on site observations of the program and focus groups with parents of student participants and staff involved in program implementation, and teacher perception of knowledge, attitude, and behavioral change in student participants. A teacher survey was administered to all public school teachers that had students in their classes who regularly attended the STEM program. The evaluation tapped into critical elements of the program and uncovered important changes, unexpected outcomes, and challenges that informed the Gateway to Growth program on its achievements and provided recommendations for its continued growth.