Planning a Program Evaluation: Worksheet

Engage Stakeholders

Who should be involved?
Board Administration:
School based administration:
School based student services:
School based Learning Coordinators/ Learning Leaders : School based teaching and Educational assistance staff
Students

How might they be engaged?
Through participating in the design, implementation, and reporting of results.

Focus the Evaluation

What are you going to evaluate? Describe program (see logic model).

What is the purpose of the evaluation? _To determine the effectiveness of the relevance potion of the collegiate renewal program


Who will use the evaluation? How will they use it?

Who/users How will they use the information?

Students and teachers To inform their practice (what they are doing
during the evaluation)

Teachers and school based To inform and guide future practice
administration (what they will do post-evaluation)

School based administration To account for programs that are successful
and board administration so that they can be replicated, or
unsuccessful so that they can be addressed.


What questions will the evaluation seek to answer?

Does ____________ programming have an effect of student perception of their learning purpose and, if so, what is the effect and how was it made.
OR
Main Question:
Did students learn the relevance of schooling?

Sub Questions:
How aware were students of the relevance of schooling before it was addressed through collegiate renewal?

How often are teachers informing students of the relevance of what is being studied?

In what ways are teachers informing students of the relevance of what is being studied?

What support materials are provided and used by teachers and students to connect topics being studied and real-world application?

Is the attempt to make schooling more relevant successful?

Should increasing students’ awareness of relevance be a focus for 2010-2011?




What information do you need to answer the questions?

What I wish to know Indicators – How will I know it?

What are students’ understandings of their purpose in regarding to their education?

 Quotes will include indicator terms and phrases such as “I need my grade 12 to get a job” (indicated ritualistic participation) or personal relevance terms such as “like”, “enjoy”, “interested”, “understand” and “help”

How often are teachers informing students of the relevance of what is being studied

 Frequency charts, self reporting, interviews

In what ways are teachers informing students of the relevance of what is being studied?
 Frequency charts, self reporting, interviews

What support materials are provided and used by teachers and students to connect topics being studied and real-world application?
 Material and document review, observation

Is the attempt to make schooling more relevant successful?
 Students will be able to articulate how their engagement in school will affect their lives outside of school and into the future.
 Students will be able to articulate what skills they are being offered and how those skills are/will be useful in the future
 Students may be able to articulate how learning continues beyond the classroom and beyond their school years.
 Teachers will perceive a decreased need to “prod” or “cajole” students into engaging in a task or activity.
 Success/retention rates shown in number of credits earned year to year

Should increasing students’ awareness of relevance be a focus for 2010-2011?
 The success of moderate intervention during the school year

When is the evaluation needed? May 2010

What evaluation design will you use? Mixed Method Design

Collect the information
What sources of information will you use?
Existing information: _______Tell them From Me data__________________

People: ____ School based administration: School based student services:
School based Learning Coordinators/ Learning Leaders 1 :
School based teaching and Educational assistance staff
Students

Pictorial records and observations: __Observation logs from __________and _________

What data collection method(s) will you use?

Survey
Interview
Observation
Document review
Testimonials
Journal, log, diary
Other (list) ___Credit review

When will you collect data for each method you’ve chosen?

Method Before program During program Immediately after Later
Survey X X
Interview X X
Observation
(Administration
and School
counselor
to keep
visitation
logs) X X
Log X
Document review X
Credit Review X X

Instrumentation: What is needed to record the information? Computer
Flip Camera
Digital Audio Recorder
Transcription software

Will a sample be used? No___________________

Analyze and Interpret

How will the data be analyzed?
Data analysis methods:, video, textual and numerical data analysis

Who responsible: ___

How will the information be interpreted—by whom? The information will be interpreted in a report written by Lesley Walters and vetted by a selection of stakeholders

Use the Information

How will the evaluation be communicated and shared?

This has yet to be determined. While I can expect that the report will be shared with teaching, administration and para-professional staff, I has not been decided if this information will be appropriate beyond that scope. I have been counseled to keep the project in a limited sphere until it is completed and has been viewed by administration.


Notes:

1) Learning Coordinator is a position formerly referred to as Department Head
Learning Leader is a position created in 2007 to support the collegiate Renewal Initiative
2) Relevance
"Motivation is the reason for being engaged and relevance helps to provide that compelling reason. Learning is often engaging to the extent that students deem the learning as meaningful and interesting to self and world. “Contextualized learning involves being able to see the value and relevance of the skills [and understandings] being learned as opposed to learning that is abstract and divorced from real life (Feuerstein, 1980, Haywood, 1993). Learning is best placed in meaningful contexts that show its inherent utility and capitalizes on students’ interest” (McLean, 2003).

When will I ever use this in the “real world”? This is a perennial student question and thoughtful teachers consider relevance when viewing curriculum and considering instruction. The sentiment is reflected in our instructional planning question: “Why would students give their hearts and heads to this work?” For we know, learning is ultimately the choice of the learner. Still, we also understand that students must take responsibility “for generating interest, if they find the work boring, by finding ways to make it more challenging and worthwhile for themselves” (McLean, 2003). "
Taken from the Saskatoon Public schools Collegiate renewal website on October 15, 2009 http://schools.spsd.sk.ca/collegiaterenewal/