Feb 28, 2024 – The Utah Educational Policy Center conducted a comprehensive evaluation of 21 grants made by the Idaho Out-of-School Network totaling $1.5 million for 2022-2023 School Year, for out-of-school time (OST) youth programs across the state. ION also supported those grantees with technical assistance, training and coaching on best practices for quality youth programming.
UEPC assessed the impacts and effectiveness of programs by reviewing all written materials and conducted surveys and interviews. This report includes survey data from 141 parents (and guardians) and 51 youth program staff. UEPC also interviewed 17 program directors and worked extensively with ION to review benchmarks of success. This report will be followed by a final report covering three years of ION’s granting program in fall 2024.
FINDINGS
#1 Grantees implemented services as described in their approved grant applications, offering a wide variety of program activities and unique opportunities for participants that meet community needs. Almost all family survey respondents indicated that programs provided activities that were appropriate for their child (96%); that offered new learning opportunities (97%); and new experiences (98%) for their child. 89% reported that their children attended the programs “all” or “most” of the days it was offered.
#2 Program administrators and staff welcomed support from the ION team in their efforts to implement quality services for students and families. Grantees appreciated the learning opportunities provided by the ION team, including virtual options.
#3 Grantees reported various student outcomes aligned with their goals, such as higher program and school day attendance, academic progress, character and social skill development, and improved behavior. 89% of staff survey respondents said that students showed improvement in behavior throughout the school year and social emotional learning (see graphic).
#4 Grantees successfully collaborated with community partners to support the implementation of program services, particularly in rural communities.
#5 Finding and retaining staff was a challenge for many programs, prompting directors to be creative with staff schedules and employee incentives. Comments focused on the need for adequate pay, affordable housing, and job sharing. 24% of the staff survey respondents were in their first year working for their programs.
#6 Several grantees used grant funds to intentionally increase program access for students and families in rural areas. Comments focused on many families’ lack of resources and transportation in rural areas.
#7 Grantees strategically used grant funds to provide services to students with specific needs, such as multilingual learners, students with disabilities, and students with mental health needs.
#8 Grantees developed strategic partnerships with local schools and felt that these partnerships enhanced the quality of their services for students. Comments focused on sharing resources such as a counselor and snacks from the cafeteria and partnering to support students during the summer.
#9 Grantees and families identified a variety of program elements that would be valuable for students and families in other settings, such as inclusive program cultures, adult and peer mentors, targeted academic assistance, and support for working families. 69% of family survey respondents said their child’s program took their work schedule into account when making the program schedule (e.g. start and end times, days of the events).
#10 Grantees explored opportunities to expand services to more students and families amid the challenges of securing additional funding opportunities. This included adding programs in nearby communities to reach more students.
#11 Grantees indicated that funding is needed to sustain staffing, particularly to expand the number of students served and to purchase necessary materials and supplies. 22% of staff survey respondents agreed that limited resources currently hinder their ability to achieve their program’s goals. Others commented that the wait list grows each year due to limited staff, that programs need additional funding to offer to more students, and noted the need for supplies and tools, including technology.
State-level Considerations: (For ION and granting agencies)
- Offer statewide or regional professional learning opportunities for program teams to learn from and with other teams about overcoming challenges by programs across the state.
- Develop a system to document successful program partnerships, such as those shared by program directors so other teams can use this resource as they recruit partners to meet needs of students and families.
- Provide opportunities for program teams to build on their successes with academic support by intentionally aligning these services with the school day using specific tools/resources.
- Engage in state-level discussions with SDE re: areas of the highest need for OST programs and explore resources to help sustain and expand services —a concern cited by Program Directors across state
- Support program teams in collecting data to document progress toward their goals which can be used to market successes with schools, partners, and funders and demonstrate the impact of OST programming across the state.
Program-level Consideration (for OST youth programs)
- As a program team, identify specific data to collect at regular intervals that show progress toward program/student goals so that program success and needs can be highlighted when seeking additional grant funding and developing new partnerships.
- Program teams should continue to explore current needs of program participants, families, and communities, and expand existing partnerships or recruit new partners to provide specific supports to meet their ongoing needs, such as those shared by program staff (e.g. mental health supports, language services.)
- Consider implementing creative solutions to staffing challenges that have been successful for other grantees, such as offering flexible schedules, sharing staff with nearby programs, working with local high schools, and providing stipends to retain quality staff.
- Develop strategies to share regular updates on program successes such as outcomes achieved, notable program events, and highlights of participating students/families with school and community partners, local media, and families as a way to secure additional resources and leverage opportunities to sustain programming.
- Consider providing or expanding out-of-school services on Fridays as more school districts shift to a four-day school week.
