1. Engage Stakeholders
Who should be involved?
|
Stakeholders
include Board of Directors, Executive Director (M. McDonald), Volunteer
Coordinator, volunteer leaders, teachers, students,
administration
|
How might they be
engaged?
|
Stakeholders
will participate in the process of determining the program’s effectiveness
through the various data collected (surveys, interviews, anecdotal logs,
etc.). They will also play an important role in offering suggestions for
improvement
|
2. Focus the Evaluation
What are you going to evaluate? Describe program (logic model).
|
Effectiveness
of the Recess Guardians Program in
increasing physical activity, positive behavior and attitudes and leadership
of students during recess periods.
|
What is the purpose of
the evaluation?
|
The
purpose of the evaluation will be to determine effectiveness of current
programming, and make recommendations for program improvement in goals with
regards to student activity, attitudes and behavior during recess periods
|
Who will use the evaluation? How
will they use it?
|
Who/users
|
How will
they use the information?
|
|
Board
of Directors
|
Evaluation
will be used to determine modifications that should be made to program and/or
use of available funding
|
|
Executive
Director
|
Findings
will assist in future planning of program operation and delivery. Also, to
determine changes that should occur in programming itself, volunteer
training, collaboration needed with schools, and decisions to expand or
modify program.
|
|
Volunteer
Coordinator
|
Data
will help in future planning with regards to numbers and types of volunteers
required, organization of volunteers and training needed
|
|
Volunteer
Leaders
|
Evaluation
can affirm role and importance of volunteers, and/or make suggestions for
greater effectiveness
|
|
Teachers/Administrators
(schools)
|
Evaluation
can help determine whether schools will continue to utilize program, and see
(and hopefully give input into) recommended changes for improvement
|
|
Students
|
To
see data showing the changes (if any) to their activity levels, attitude and
behaviors during recess periods
|
What questions will
the evaluation seek to answer?
|
How
does the program affect student engagement in activity during recess periods?
|
|
How
is student attitude towards activity and sportsmanship impacted by the
program?
|
|
How
does the program foster the development of leadership skills in students?
|
What information do you need to
answer the questions?
|
What I
wish to know
|
Indicators
– How will I know it?
|
|
Does
the program increase student activity during recess periods?
|
-
Observation of students pre-program and post-program
-
Increased amount of play on the playground; games taught through program and
others
-
student journals (record of activities played and attitude/feelings towards
activity)
|
|
Do
students exhibit more positive attitudes and sportsmanship when utilizing
games taught through the program?
|
-
students demonstrating ability to work cooperatively and with positive
sportsmanship
-
logs (teachers) – are students returning from recess with less incidents of
dispute due to lack of fair play, or “getting into trouble” due to lack of
productive activity?
|
|
Does
the program improve student leadership (skills and attitudes)?
|
-
Increased confidence in students working with other students to play together
and teach each other
-
ability to problem solve and work together to play appropriately (less
arguments, more working together)
-students
taking roles to facilitate play, rather than waiting for teacher direction
|
What evaluation design
will you use?
|
The
evaluation will be both formative and summative, utilizing a mix of
Stufflebeam’s CIPP model along with Scriven’s purpose driven approach.
|
3. Collect the information
What
sources of information will you use?
|
Existing
information:
|
Surveys
of previous participants (volunteers, teachers, students), previous news
articles
|
|
People:
|
Executive
director, Volunteer coordinator, students, teachers, volunteers
|
|
Pictorial
records and observations:
|
Lessons
taught (in class instruction), playground carry-over (games being played,
climate of sportsmanship,etc.)
|
|
X
|
Survey
|
|
Document review
|
|
X
|
Interview
|
|
Testimonials
|
|
X
|
Observation
|
|
Expert panel
|
|
|
Group techniques
|
|
Simulated problems or situations
|
|
|
Case study
|
X
|
Journal, log, diary
|
|
|
Tests
|
|
Unobtrusive measures
|
|
X
|
Photos, videos
|
|
Other (list)
|
Instrumentation: What is needed to record
the information?
|
Surveys,
anecdotal observation records, video camera, Student journal/teacher logs
|
|
|
When will you collect data for each
method you’ve chosen?
|
Method
|
Before
program
|
During
program
|
Immediately
after
|
Later
|
|
Survey
|
X
|
|
X
|
|
|
Interview
|
X
|
|
X
|
X
|
|
Observation
|
X
|
X
|
|
|
|
Photos/Video
|
|
X
|
|
|
|
Student
journal/teacher log
|
|
X
|
X
|
|
Will a sample be used?
|
No
|
|
|
|
Yes
|
X
|
If yes,
describe the procedure you will use.
|
|
Program
delivery will be observed, and data collected (pre-program and post-program
surveys, interviews, anecdotal student and teacher logs) from three Saskatoon
schools. One similarly sized and grade grouped class will be used for the
sample (example, 25 grade 4 students from each of three schools). The teacher
for each classroom will also be asked to participate in a survey, interview,
and in collecting anecdotal observations/tracking log
|
||
Analyze and Interpret
4.
How will the data be analyzed?
|
Data
analysis methods:
|
All
data (quantitative and qualitative) will be tabulated and sorted, with common
themes and findings being noted. Also, unanticipated outcomes will be noted,
to include in evaluation reporting
|
|
Who
responsible:
|
Evaluator
|
How will the information be interpreted—by whom?
|
The
evaluator will be responsible for creating surveys, interview questions, observations
and analyzing the data
|
|
|
What did you learn? What are the limitations?
|
Ultimately,
the hope is to learn if the program is being successful in meeting its
objectives and offering suggestions that can further enhance the program.
|
|
Some
limitations I foresee include that since the program runs for only two weeks
within each school, the evaluator would need to ensure that all pre-program
surveys, observations, teacher and student logs and post-program surveys were
conducted in a timely fashion, recognizing the demands on teacher
instructional time. Also, given the nature of our weather in Saskatchewan,
observations may vary greatly if performed within different seasons (a factor
to consider when comparing sample results).
|
| 5. |
Use the Information
How will the evaluation be communicated and shared?
|
To whom
|
When/where/how to present
|
|
Board
of Directors, Executive director, Volunteer Coordinator
|
Detailed
written report of findings (are objectives being met) and recommendations for
change
|
|
Teachers,
administrators
|
Short
summary statements of evaluation, including summarized suggestions for
improvement acquired through student and teacher surveys
|
|
Classrooms
of participating Students
|
Recognition
and thank you for participation, evidence that their ideas have been heard
(from student survey) – Pictures/video presentation
|
Next steps?
|
Recommendations
given for modification and/or improvement of program
|
Manage the evaluation Standards
|
X
|
Human subject’s protection
|
X
|
Utility
|
|
|
Management chart
|
X
|
Feasibility
|
|
X
|
Timeline
|
X
|
Propriety
|
|
X
|
Responsibilities
|
X
|
Accuracy
|
|
X
|
Budget
|
|
|
