Session Four: Connecting Implementation to Outcomes
“one size doesn’t fit all”
Intentionality is important when designing an evaluation tool. If we think blindly about the program, and make assumptions about what we believe is going to happen, we may discover that there is ‘a problem with the nut,’ when the real problem is with the evaluation plan.
Welcome to the fourth session of the Program Planning and Evaluation for PYD Impact Academy. This session will connect the intentional “what” we do to “why” we do it and examine the measurement of outputs vs. outcomes. Self-paced learning assignments include a review of the youth thriving indicators and the developmental outcomes of the 4-H Thriving Model and the research establishing the connection between PYD and the learning outcomes of individual programs. During the live webinar session, hosted by experienced PYD evaluators, you will explore resources for measuring PYD as defined by the 4-H Thriving Model and discuss strategies for data collection, analysis, and reporting in their perspective programs.
Learning Objectives for Session 4
- Increase understanding and use of logic modeling to distinguish between program outputs and outcomes, include the process of PYD (thriving indicators), PYD developmental outcomes, and long-term outcomes.
- Distinguish between program content outcomes and PYD outcomes and make a connection between quality PYD and program outcomes.
- Develop program planning skills to connect learning activities to one or more of the thriving indicators in the 4-H Thriving Model.
- Learn resources for evaluating youth thriving.
Before you begin – please register for one of the two required webinars for Session 4
Please use the link provided below to register for the learning circle sessions, which will commence after the completion of Session 5
Please complete the self-paced learning assignments listed below in the order they are presented.
Clicking on the button will open a new tab. To return to the session content, click on the PYD Academy in your browser’s tab list. This will take you back to where you left off.
Step One – Watch
Understanding How It All Fits Together: The Logic Model
Planning and then adjusting is an important part of the programming for impact cycle. Using a logic model can show you how to develop your project objectives more effectively, it can also help you communicate the plan to funders and stakeholders.
Take Away: This quick refresher on the logic model reminds us that impact focused planning requires us to be “checking in” on our plan throughout the program cycle. Take this opportunity to review the layers of intended outcomes outlined in 4-H Thriving Model from the bottom of the flower model to the long-term impacts at the top.
Step Two – Read
“Measuring the Impact of 4-H: Common Measures 2.0 and the 4-H Thriving Model” Article
The development of the 4-H Thriving Model has advanced the measurement of the impact of 4-H by identifying the processes through which 4-H contributes to the positive development of youth. 4-H Common Measures and the 4-H Thriving Model align to provide evaluation options for youth development professionals focused upon measuring: 1) The 4-H Program Experience, 2) Youth Content Outcomes, 3) Positive Youth Development, and 4) Developmental Outcomes.
Take Away: The focus of this Academy has been on evaluating PYD as described by the 4-H Thriving Model. Our PYD impact story can and should be told in complement to our program content outcomes measures by 4-H Common Measures and other related instruments. As you read this paper consider opportunities where you can tell the story of PYD and your program content priority outcomes together.
Step Three – Read
“2022 National 4-H Index Study” Summary
The 2022 National 4-H Index Study is a point-in-time study conducted by the 4-H Thrive Model Team in collaboration with the 4-H Common Measures Team. This study is the first time that measurement of PYD with 4-H program outcomes measured by Common Measures were combined. The study also replicates the structure of the 4-H Thriving Model with a broader group of 4-H members and revealed that 4-H’ers are experiencing high-quality PYD programs that help them thrive.
Take Away: The National 4-H Annual Index Study establishes the relationship between high quality PYD and stronger 4-H program outcomes. Review and consider if where you might share this national story with your local stakeholders.
Step Four – Read
“Developing Youth Competencies: The Impact of Program Quality”
This study represents one of the first investigations in the literature into the connection between program quality and outcomes. Using data collected through the CYFAR (Children, Youth, and Families at Risk) Initiative, the authors share how youth’s attitudes and behaviors as they relate to the five common competencies of all CYFAR programs (caring, social conscience, personal values, decision making, and critical thinking) were evaluated and provide several implications for youth programming.
Take Away: This article is a good example of investigating program quality in relation to program outcomes. It underscores the importance of assessing for program quality to ensure program are maximizing the impacts they can have on youth they serve. Consider your own programs, and identify one place you might consider this type of investigation.
Step Five – Read
“Youth Program Quality Principles”
Eight critical principles of program quality have been formed to assist volunteers and staff who work directly with youth in 4-H programs. These principles provide a structure which can lead to the biggest impact when high-quality educational experiences for youth audiences are delivered.
Take Away: As we strive to help youth thrive and achieve outcomes at the top of the flower model, consider this question. How do youth program quality principles help us intentionally connect learning activities and support the development of the 4-H Thriving Indicators?
Step Six – Read
“The 4-H Thriving Model Program Evaluation Instruments: Information and Scoring”
Four 4-H Thriving Model Program Evaluation Instruments have been created and are recommended for use in program evaluation efforts with youth ages 13 and older at the end of the 4-H program. The instruments are intended to measure group trends to understand the quality and impact of the program.
After reading the Information and Scoring Guide, go to the following web site to download each instrument: https://helping-youth-thrive.extension.org/evaluation/ (Password: Thrive).
Take Away: As you work on your individual plans, identify one or more places where you will implement one more of the 4-H Thriving Model Evaluation Instruments. Throughout the course, we will introduce additional strategies, but we encourage you to consider where these tools fit into your evaluation plans.
Step Seven- Reflect & Share
Take a moment and think through what you have learned so far, and use the Jam Board to record your responses to the following questions. Be sure to add your thoughts and comments to other’s posts too!
- What evaluation tools and strategies do you plan to implement into youth-focused programming?
- What’s one question you have about what you have learned so far?
Step Eight – Attend ONE of the Live Webinar Sessions
Session 4 of the academy wraps up with a live webinar session. In this session, Dr. Rachel Guidugli, an Assistant Extension Director for Kentucky 4-H Youth Development, and Dr. Autumn Gain, a Quality Youth Programming Specialist at North Carolina State Extension, share more about monitoring your performance objectives, applying feedback in a continuous improvement model, and observing program quality and a Developmental Context Fidelity measure. Please come prepared to learn!
Step Nine – Do the Evaluation
After you have participated in the live webinar there are two steps to complete before you move on to the next session – confirm completion of all session requirements and complete the session evaluation.
You will be directed to the confirmation and evaluation site at the end of the live webinar session. You can also return to this page to complete this required step.
Your input on the academy experience is critical to helping us shape the best possible academy going forward!
Step Ten- Go To Session Five