8 Tips for TPP Rigorous Evaluation Planning: Highlights from the What I Wish I Knew Expert Panel Discussion with Center for Supportive Schools and The Policy & Research Group
Published:
Plan to over-recruit. There’s a good chance you will need more sites than you initially thought due to factors that are out of your control.Sherry Barr, Managing Director, Operations and Evaluation, Center for Supportive Schools
On December 5, 2023 in Washington, D.C., two adolescent health research experts presented on a panel along with federal staff and Mathematica staff to offer advice to newly-funded Teen Pregnancy Prevention Program (TPP) Tier 2 Rigorous Impact Evaluation projects.
The panel included Dr. Catie Henley, Senior Research Analyst with The Policy & Research Group (PRG), and Dr. Sherry Barr, Managing Director of Operations and Evaluation at Center for Supportive Schools (CSS), both TPP Tier 2 Rigorous Impact Evaluation grantees with decades of TPP evaluation knowledge. They shared tips based on their experiences evaluating new and innovative approaches to promote adolescent health and prevent teen pregnancies and sexually transmitted infections.
Specifically, Drs. Henley and Barr advised grantees to:
- Over-recruit participants, and recruit throughout the project year. These tactics help ensure a robust, diverse evaluation sample.
- Build in ample time for research planning. For example, overestimate the time needed to secure institutional review board (IRB) approval.
- Make sure recruitment materials match the priority populations. Generic messaging tends to be less effective.
- Stay flexible and pivot when needed. Project implementation rarely goes exactly as planned so recalibrating is often key.
- Ask research participants for feedback on any incentives you offer. This helps ensure that incentives are a good fit for the priority population.
- Call upon other evaluation experts. Evaluation colleagues and technical assistance (TA) providers—like the RHNTC and Mathematica—can help troubleshoot unexpected data collection challenges.
- Communicate openly with partners, fellow researchers, TA providers, and federal project officers. All panelists agreed on this point.
- Seek help when you need it. There’s no need to suffer silently. Transparency about challenges, roadblocks, and uncertainty is a pathway to stronger research outcomes.
As the session wrapped up, panelists returned to the theme of partnership. Dr. Henley captured the sentiment well: “Devote time and energy to building strong relationships with all partners on the project. They are the foundation for the success of your evaluation.”
Looking for resources to support some of these strategies? Check out the Expectations Tool for TPP23 Rigorous Impact Evaluation Project Directors.