Session Three: Measuring PYD Implementation
“feedback informs the work, forms the story”
Lily Robinson, age 3 and 1/2, had an idea to share. She thought that the markings found on the ‘tiger bread’ (a soft and fluffy loaf of bread that has a distinct pattern on the top due to rice flour paste) that her family purchased from Sainsbury’s resembled a giraffe more than a tiger. Lily shared this feedback with the bread company, suggesting the name be changed. She received a letter from Chris King, age 27 and 1/3, that her idea was brilliant. Due to Lily’s letter and Chris’s reply going viral, the supermarket began selling new loaves of ‘giraffe bread.’
Even the youngest participants in our programs have something to share. Gathering feedback in developmentally appropriate ways provides us as youth development professionals with information and data that can be used to tell our story, to be accountable, and to learn and improve.
Welcome to the third session of the Program Planning and Evaluation for PYD Impact Academy. This session will focus on the implementation and assessment of PYD as defined by the 4-H Thriving Model. Self-paced learning assignments include a review of the Developmental Context Layer of the 4-H Thriving Model and an introduction to methods for gathering participatory feedback. During the live webinar session, hosted by experienced PYD evaluators, you will learn more about monitoring performance objectives, discuss the application of feedback in a continuous improvement model, and be introduced to program quality observations.
Learning Objectives for Session 3
Welcome to the third session of the Program Planning and Evaluation for PYD Impact Academy. This session will focus on the implementation and assessment of PYD as defined by the 4-H Thriving Model. Self-paced learning assignments include a review of the Developmental Context Layer of the 4-H Thriving Model and an introduction to methods for gathering participatory feedback. During the live webinar session, hosted by experienced PYD evaluators, you will learn more about monitoring performance objectives, discuss the application of feedback in a continuous improvement model, and be introduced to program quality observations.
Learning Objectives for Session 3
- Consider what type of data you need to collect (and why)
- Understand why the developmental context portion of the thriving model is an essential place to focus evaluation efforts.
- Learn tools and processes for evaluating the developmental context elements.
- Learn methods for gathering participant feedback.
- Learn ways to use data to monitor program quality and to drive decision making and program improvement.
- Learn ways to communicate the importance of high-quality program settings and the impact they have on youth.
Before you begin – please register for one of the two required webinars for Session Three
Please complete the self-paced learning assignments listed below in the order they are presented.
Clicking on the button will open a new tab. To return to the session content, click on the PYD Academy in your browser’s tab list. This will take you back to where you left off.
Step One – Watch
Dr. Mary Arnold provides an introduction to the 4-H Thriving Model
The Thriving Model explains the process of Positive Youth Development in 4-H – how 4-H works its magic with youth. In this video, Dr. Mary Arnold, Professor and Youth Development Specialist at Oregon State University, shares how the 4-H Thriving Model was created, the parts of the model, the outcomes of positive youth development, and much more!
Take Away: 4-H is a powerful program for youth, but the impact can only happen if you remember that what you do matters. Take one program that you offer and consider planning it through the lens of the 4-H Thriving Model.
Step Two – Review
Tip Sheets for Creating High Quality Developmental Context
Research shows that youth development programs like 4-H must incorporate eight principle concepts in their structure in order to have a positive impact on youth. A variety of 4-H Thriving Model resources are available for Educators who work with youth and are designed to assist with the implementation of positive youth development experiences, and three of these tip sheets focus on creating high quality developmental context. Review the available tip sheets to learn more about how you as a youth development professional can form programs where youth can thrive.
Take Away: To successfully evaluate PYD, it is critical to understand what we are striving to achieve. These tip sheets will illustrate measurable standards for developmental context. As you review, ask yourself, how would I know if this was being achieved?
Step Three – Watch
“The Joy of Getting Feedback” by Joe Hirsch
Speaker and author Joe Hirsch specializes in helping schools and businesses master the art and science of high-performance feedback. He also had a fear of getting feedback – he was ashamed of his shortcomings; getting feedback meant coming to terms with himself on someone else’s terms, and that wasn’t something he could easily accept. In this video, Joe shares how moving from feedback to “feedforward” pushes us to see where we’re heading, not just where we’ve been; and that joy isn’t something we just find – it’s something we create.
Take Away: As Youth Development professionals, it can be easy to get busy with the doing of program delivery and welcoming feedback and making space for evaluation isn’t always top of mind. This video reminds us that high-quality program design, must include opportunity for intentional evaluation and feedback which is essential for program improvement. Reflect on instances when you might have missed an opportunity to move from feedback to feedforward.
Step Four – Read
“Age-Sensitive Instrument Design for Youth: A Developmental Approach”
Designing instruments for children and youth that results in reliable and valid data requires consideration beyond calculating grade-level equivalence of the text. Very little methodological research has been conducted on the survey response processes of children and youth. The authors of this paper review some information about child development in order to establish a set of theoretically-based recommendations proposed for designing or adapting instruments for children and youth that are 9 to 18 years old.
Take Away: Youth are often the best positioned to help tell the story of PYD. The article demonstrates the importance of applying what we know about the development of young people to selecting evaluation strategies that are age-appropriate. Make a short list of the most important things to consider when collecting data from youth.
Step Five – Read
“5 Tips for Continuous Youth Program Improvement”
In this blog entry from Youth Development Insight, Betsy Olson shares with readers how interactive evaluation or feedback mechanisms that enliven the young people we serve can help our programs continue to build and improve. Through communicating program and activity goals to youth consistently, providing more opportunities for small-group work, checking in regularly, putting youth in charge, and creating mutual accountability, youth will be positioned to share meaningful feedback with youth development professionals.
Take Away: This blog is a great reminder that continuous evaluation doesn’t have to be formal methods like surveys or interviews. Engaging youth in understanding where were we are headed allows them to speak-up if we’re off course and to celebrate when milestones are achieved. Plan to practice an informal check-in with youth at your next program.
Step Six – Read
“A Mixed-Methods Evaluation to Measure 4-H STEM Program Quality”
Over 150 4-H youth and parents, Extension 4-H professionals, corporate volunteers, and community volunteers engaging in the 4-H Science: Building a 4-H Career Pathway Initiative were participants in this study focused upon measuring 4-H STEM program quality. Joseph Donaldson and Karen Franck provide insight in this article about what was learned by throughout this mixed-methods evaluation process.
Take Away: This study is a great example of evaluation designed to measure program quality. While we’re often asked to focus on reporting program outcomes, understanding program quality has tremendous implications for the effectiveness of programming. Identify a new or emerging program you are working with that might benefit from a program quality assessment.
Step Seven- Reflect & Share
Take a moment and think through what you have learned so far, and use the Jam Board to record your responses to the following questions. Be sure to add your thoughts and comments to other’s posts too!
- What method(s) of gathering participant feedback do you plan to implement in your programming, and how will you use this information?
- What’s one question you have about what you have learned so far?
Step Eight – Attend ONE of the Live Webinar Sessions
Tuesday, November 7th or Thursday, November 9th (2 hours, select one)
Session 3 of the academy wraps up with a live webinar session. In this session, Celeste Janssen, a 4-H Positive Youth Development Research Coordinator at Oregon State University Extension, and Dr. Jill Lingard, an Extension Professor of 4-H Youth Development at the University of Nebraska-Lincoln, will share more about monitoring your performance objectives, applying feedback in a continuous improvement model, and observing program quality and a Developmental Context Fidelity measure. Please come prepared to learn!
Step Nine – Do the Evaluation
After you have participated in the live webinar there are two steps to complete before you move on to the next session – confirm completion of all session requirements and complete the session evaluation.
You will be directed to the confirmation and evaluation site at the end of the live webinar session. You can also return to this page to complete this required step.
Your input on the academy experience is critical to helping us shape the best possible academy going forward!
Step Ten – Additional Resources
Step Eleven- Go To Session Four