2.
Getting Started: Planning for Evaluation
The
framework described here is primarily intended to guide
formative evaluation of projects that apply technology
to teaching and learning activities in classrooms. It
is hoped that this framework will be valuable for a
variety of reasons:
- The
collective understanding of "how technology works"
to improve teaching and learning has expanded, and
old assumptions may no longer hold;
- Evaluation
can be a relatively expensive endeavor and is generally
under funded as a component of education technology
initiatives; and
- Granting
agencies are raising expectations of accountability
that come with technology-focused awards.
Although
internal, formative evaluation by a district or school
cannot replace a comprehensive, professional evaluation,
education programs can benefit tremendously from formative
feedback generated by those within the system.
The
CAPE framework provides a step-by-step approach to help
non-evaluators plan and implement efficient, well-founded,
theoretically sound evaluations of technology projects
in education settings by breaking a complex process
into manageable pieces:
- Planning
the evaluation
- Explaining
how the project is supposed to work
- Establishing
project goals, objectives, and strategies
- Developing
the basic components of an evaluation plan
- Identifying
data sources for the evaluation
- Implementing
the evaluation effort
- Communicating
the evaluation results
These
pages also provide tested resources, data-collection
instruments tailored to technology implementations,
and examples to guide the above steps.
Evaluation
Terms
It
might seem simplistic, but the biggest challenge in
developing a formative evaluation project plan might
be assuring that everyone involved in the process is
speaking the same language. The distinctions among terms
can be critical to understanding and to building consensus
necessary to design and implement a feasible, accurate,
useful evaluation. It is not necessary that all stakeholders
agree on a universal definition of a term like "objective,"
but it is extremely important that they accept common
word usage for the purposes of the evaluation effort.
One
possible first step toward understanding is the CAPE
Evaluation Term Matching Exercise (in PDF from).
Using this exercise, have evaluation planning team members
match evaluation terms with Thanksgiving Day metaphors,
and then compare responses to the Matching
Exercise Answer Key. Resolve differences in understanding
by discussion the resulting matches. Finally, record
definitions of these terms, as they will be applied
to your evaluation planning effort, and distribute to
all stakeholders.
Planning
for Evaluation
The
next step might be to establish and overview for planning
by reviewing and discussing Guided
Planning for Evaluation: An Overview (PDF).
This presentation was originally prepared at SERVE for
the Mississippi Department of Education's Evaluation
Institute for EETT Grant Coordinators to:
- Provide
an overview of the steps that grantees go through
in developing an evaluation plan
- Provide
an evaluation framework that will help evaluation
planners make informed decisions about project implementation
and impact
- Provide
examples of evaluation plan components
- Offer
worksheets for developing an effective evaluation
plan
The
process described in this presentation may also be supported
by referring to How
to develop and Evaluation Plan (PDF) - a flowchart
depicting a complete overview of the logical sequence
that applies the materials available in these web pages.
Finally,
a more comprehensive review is available in Planning
into Practice. This publication is grounded in SEIR*TEC's
work and details the process of creating a strategic
technology plan. Chapter
7 examines development of evaluation plans tailored
to technology implementations in teaching and learning,
describes options for collecting data, and suggests
methods that might be applied to analyze results.
The next major step is to precisely define the project
that will be evaluated. Again, it is absolutely essential
that all stakeholders agree about how the project works,
how it will be implemented, and what successful outcomes
"look like." Failure to work thorough this
often difficult process will complicate a project evaluation
- sometimes to the point of paralysis.
Next > Theory: Explaining How Your
Project Works |
