Pace Center for Girls Flexes Its Feedback Muscle to Support Its Mission and Theory of Change
Feedback is core to Pace Center for Girls’ learning agenda and contributes to quality improvement efforts, innovation, and social impact. Listening to the girls we serve through our year-round academic and social services programming is a highlighted value in our mission and theory of change. And in recent years we’ve worked hard to get to this point of being able to flex our feedback muscle.
A nonprofit based in Florida, Pace provides girls and their families with services through a gender-responsive, trauma-informed, and strength-based lens. Program evaluation is important to us, and so in 2016 we set out to more formally incorporate feedback into our program evaluation toolbox.
Participating in Listen4Good (L4G), a capacity-building program started by the funding collaborative Fund for Shared Insight, helped us develop the capacity to listen, reflect, and act on feedback received from girls and their families, along with our team members and community partners. A few years later, we joined Shared Insight’s feedback research cohort, eager to explore the potential links between feedback and participant outcomes.
Since participating in L4G, we have incorporated feedback into our day-to-day work. Hearing from our participants brings their voices to the table when discussing issues affecting program implementation and improvement. Pace invites girls to share feedback through several formal and informal means.
Formally, Pace girls participate in program evaluation activities by vetting questions included in surveys and focus groups and supporting data collection activities. Once the information is collected, the Pace Girls Leadership Council (a selected group of girls working on developing their leadership qualities) engages in discussions about the findings to provide contextual meaning and develop potential solutions. Informally, and with team member encouragement, Pace girls progress in how they use their voices in the program. They learn to modulate their message, speak up for themselves, and advocate for others.
Importantly, feedback allows Pace to be responsive to participants’ needs and identify the contributors to a positive program experience. For example, Pace has incorporated youth perspectives in decision-making on issues such as listening, sharing feedback, and seeking participants’ perspectives in interpreting feedback, uncovering issues that, from an organizational standpoint, would not have been considered otherwise.
Girls’ feedback has also led Pace to collect information about girls’ relationships with team members. Survey results combined with participants’ insights highlighted the role of the relationship between team members and girls and how those relationships could impact outcomes for the girls. Girls reported valuing the development of trusting relationships with Pace team members and indicated that these relationships influence their program experience. This feedback on the relevance of relationships aligned with our theory of change and existing research. We know that connectedness is an essential protective factor for youth that can reduce the likelihood of various health risk behaviors.
We also thought it would be interesting and instructive to examine the potential link between the feedback on social connectedness we were hearing and program outcomes. Starting with a Shared Insight research grant in 2019, we began addressing this question of impact, thus testing a key idea outlined in Pace’s theory of change.
Pace theorized that girls with positive program experiences have better outcomes than girls with negative program experiences. We collaborated with Covian Consulting to examine whether feedback mediates (directly connects) and/or moderates (otherwise impacts) the associations of program engagement with program outcomes. Together, we applied a mixed methods approach to examine feedback processes, the value girls attribute to these opportunities, and how they translate into outcomes. Covian conducted statistical analyses, including Structural Equation Modeling, and regression analyses, finding that promoter status (a measure of whether a girl would refer Pace to a friend or family member) does not directly predict program outcomes. The analyses found, however, that promoter status does moderate the association between program engagement and program outcomes (e.g., the associations are stronger for program promoters than program non-promoters).
Further analysis revealed that program attendance predicts changes in girls’ self-efficacy (the ability for girls to believe that they can be successful based on whatever skills they possess) and this primary relationship can account for changes in feelings of social connection and girls’ perceptions of Pace. Put differently, girls with high attendance rates experience increases in feelings of social connection during their time at Pace, and girls who feel socially connected report more positive perceptions of Pace. Lastly, girls who report positive perceptions of Pace experience increases in self-efficacy, and this is only observed for girls who are identified as program promoters.
The survey and other listening work we do continue to inform how we understand the research. We now have evidence that the type of feedback we hear from the girls matters and impacts core outcomes. The findings highlight how feedback can be employed as a tool for gauging quality and they reaffirm Pace’s focus on girls’ development and empowerment through feedback opportunities.
Through our surveys, research, and partnership with participants, we have learned that when we listen, we not only gain valuable insights into how to make our programs stronger in the short-term, we’re also contributing to our participants’ sense of connectedness and self-efficacy, which supports stronger outcomes for them in the long-term.
Dr. Lymari Benítez supports the Pace Center for Girls’ program evaluation and learning agenda. She has built Pace’s data infrastructure and analytics program, supporting the sustainability and replication of the model using feedback and participatory approaches.
Dr. Abigail Todhunter-Reid, a consultant to Pace, conducts rigorous applied research and supports education leaders in establishing research protocols and systematic data practices. She specializes in the evaluation and cost-analysis of educational programs and has a passion for motivation theory and social-emotional learning.