Trewon Conducts Evaluation of NSF INCLUDES

Overview

Our long-standing experience working with NSF is evident in the three projects awarded to Trewon in 2020, one of which was Evaluation Support for NSF INCLUDES. NSF INCLUDES is one of NSF’s 10 Big Ideas, long-term investments designed to fund transformative, convergent research that advances the frontiers of science and engineering. NSF INCLUDES aims to broaden the participation of underrepresented groups in STEM by catalyzing the STEM enterprise to work collaboratively for inclusive change. During this three-month project, Trewon provided the Directorate for Education and Human Resources (EHR) analytic and evaluative support services that extended past work. We addressed EHR’s evidence-building and evaluation needs and provided systematic answers to questions regarding the activities and outcomes of NSF INCLUDES.

Evidence mapping and report analysis

Trewon worked closely with EHR and the NSF INCLUDES implementation team lead to conduct two major activities. First, we completed a thorough assessment of the existing data available to answer the program’s learning agenda questions and provided NSF with a detailed mapping of the current evidence, gaps in knowledge, and potential avenues for future work. Second, we conducted an in-depth analysis of grantees’ annual progress and evaluation reports to provide NSF with an understanding of the activities, accomplishments, and outcomes of the program’s funded awards. We also explored the feasibility of using machine learning to automate some of the data extraction from the progress and evaluation reports and make the large-scale analysis of reports across the NSF portfolio more efficient.

Reporting

The Trewon evaluation team created high-quality, visually engaging reports that synthesized program evidence while being user-friendly and accessible to a variety of audiences.