Assignment #3 - Planning a PE

Planning document for the evaluation of a 

Saskatchewan based program called 

Recess Guardians 

1.           Engage Stakeholders

Who should be involved?

Stakeholders include Board of Directors, Executive Director (M. McDonald), Volunteer Coordinator, volunteer leaders, teachers, students, administration

How might they be engaged?

Stakeholders will participate in the process of determining the program’s effectiveness through the various data collected (surveys, interviews, anecdotal logs, etc.). They will also play an important role in offering suggestions for improvement

2.           Focus the Evaluation

What are you going to evaluate?  Describe program (logic model).

Effectiveness of the  Recess Guardians Program in increasing physical activity, positive behavior and attitudes and leadership of students during recess periods.

What is the purpose of the evaluation?

The purpose of the evaluation will be to determine effectiveness of current programming, and make recommendations for program improvement in goals with regards to student activity, attitudes and behavior during recess periods

Who will use the evaluation? How will they use it?

Who/users	How will they use the information?
Board of Directors	Evaluation will be used to determine modifications that should be made to program and/or use of available funding
Executive Director	Findings will assist in future planning of program operation and delivery. Also, to determine changes that should occur in programming itself, volunteer training, collaboration needed with schools, and decisions to expand or modify program.
Volunteer Coordinator	Data will help in future planning with regards to numbers and types of volunteers required, organization of volunteers and training needed
Volunteer Leaders	Evaluation can affirm role and importance of volunteers, and/or make suggestions for greater effectiveness
Teachers/Administrators (schools)	Evaluation can help determine whether schools will continue to utilize program, and see (and hopefully give input into) recommended changes for improvement
Students	To see data showing the changes (if any) to their activity levels, attitude and behaviors during recess periods

What questions will the evaluation seek to answer?

How does the program affect student engagement in activity during recess periods?

How is student attitude towards activity and sportsmanship impacted by the program?

How does the program foster the development of leadership skills in students?

What information do you need to answer the questions?

What I wish to know	Indicators – How will I know it?
Does the program increase student activity during recess periods?	- Observation of students pre-program and post-program - Increased amount of play on the playground; games taught through program and others - student journals (record of activities played and attitude/feelings towards activity)
Do students exhibit more positive attitudes and sportsmanship when utilizing games taught through the program?	- students demonstrating ability to work cooperatively and with positive sportsmanship - logs (teachers) – are students returning from recess with less incidents of dispute due to lack of fair play, or “getting into trouble” due to lack of productive activity?
Does the program improve student leadership (skills and attitudes)?	- Increased confidence in students working with other students to play together and teach each other - ability to problem solve and work together to play appropriately (less arguments, more working together) -students taking roles to facilitate play, rather than waiting for teacher direction

What evaluation design will you use?

The evaluation will be both formative and summative, utilizing a mix of Stufflebeam’s CIPP model along with Scriven’s purpose driven approach.

 3.         Collect the information 

What sources of information will you use?

Existing information:	Surveys of previous participants (volunteers, teachers, students), previous news articles
People:	Executive director, Volunteer coordinator, students, teachers, volunteers
Pictorial records and observations:	Lessons taught (in class instruction), playground carry-over (games being played, climate of sportsmanship,etc.)

X	Survey		Document review
X	Interview		Testimonials
X	Observation		Expert panel
	Group techniques		Simulated problems or situations
	Case study	X	Journal, log, diary
	Tests		Unobtrusive measures
X	Photos, videos		Other (list)

Instrumentation: What is needed to record the information?

Surveys, anecdotal observation records, video camera, Student journal/teacher logs

When will you collect data for each method you’ve chosen?

Method	Before program	During program	Immediately after	Later
Survey	X		X
Interview	X		X	X
Observation	X	X
Photos/Video		X
Student journal/teacher log		X	X

Will a sample be used? 

No
Yes	X	If yes, describe the procedure you will use.
Program delivery will be observed, and data collected (pre-program and post-program surveys, interviews, anecdotal student and teacher logs) from three Saskatoon schools. One similarly sized and grade grouped class will be used for the sample (example, 25 grade 4 students from each of three schools). The teacher for each classroom will also be asked to participate in a survey, interview, and in collecting anecdotal observations/tracking log

           Analyze and Interpret

4.  

How will the data be analyzed?

Data analysis methods:	All data (quantitative and qualitative) will be tabulated and sorted, with common themes and findings being noted. Also, unanticipated outcomes will be noted, to include in evaluation reporting
Who responsible:	Evaluator

How will the information be interpreted—by whom?

The evaluator will be responsible for creating surveys, interview questions, observations and analyzing the data

What did you learn? What are the limitations?

Ultimately, the hope is to learn if the program is being successful in meeting its objectives and offering suggestions that can further enhance the program.

Some limitations I foresee include that since the program runs for only two weeks within each school, the evaluator would need to ensure that all pre-program surveys, observations, teacher and student logs and post-program surveys were conducted in a timely fashion, recognizing the demands on teacher instructional time. Also, given the nature of our weather in Saskatchewan, observations may vary greatly if performed within different seasons (a factor to consider when comparing sample results).

	5.

 

    Use the Information

How will the evaluation be communicated and shared?

To whom	When/where/how to present
Board of Directors, Executive director, Volunteer Coordinator	Detailed written report of findings (are objectives being met) and recommendations for change
Teachers, administrators	Short summary statements of evaluation, including summarized suggestions for improvement acquired through student and teacher surveys
Classrooms of participating Students	Recognition and thank you for participation, evidence that their ideas have been heard (from student survey) – Pictures/video presentation

Next steps?

Recommendations given for modification and/or improvement of program

           Manage the evaluation                   Standards

X	Human subject’s protection	X	Utility
	Management chart	X	Feasibility
X	Timeline	X	Propriety
X	Responsibilities	X	Accuracy
X	Budget