|
1.
Introduction: How to Use These Resources
The Capacity for Applying Project Evaluation (CAPE)
evaluation framework was developed to help school and
district staff members who are not trained evaluators
learn how to evaluate their own education initiatives
or projects. It was originally created by the Technology
in Learning group in the SERVE Center at UNC Greensboro,
to help recipients of Enhancing Education Through
Technology (EETT) grants evaluate their projects,
but it can be easily adapted for the evaluation of other
types of grants and school improvement efforts. These
materials may be used in two ways:
- If
you want to work through the entire evaluation planning
process - You can read through to the bottom
of this page, follow the link at the bottom to the
next section (the Overview), and repeat that process
for each subsequent piece of the framework.
- If
you have a specific evaluation planning need or interest - You can jump to any section of the CAPE evaluation
framework (below or on the menu at on the left), or
use the index provided in the Resources section to move freely about the information provided.
The
CAPE
Evaluation Framework |
1.
Introduction: How to Use These Resources
What's
on the site, why evaluation is important, types
of evaluation, and uses of evaluation. |
2.
Getting Started: Planning for Evaluation
Evaluation terminology, and how to develop
a useful, practical, appropriate, accurate project
evaluation plan. |
3.
Theory: Explaining How Your Project Works
How your project can achieve results -
an introduction to project theory models and logic
maps. |
4.
Outcomes: Goals and Objectives; and Their Relationship
to Strategies
Defining your project in terms of desired
outcomes and the activities that lead to successful
outcomes. |
5.
The Plan: Basic Components
The nuts and bolts of an evaluation plan
- questions, indicators, data collection methods
& measures, benchmarks, and uses of evaluation
findings. |
6. Data Sources: Some Examples
Data-collection tools, most of which have
been developed by SERVE specifically to support
the CAPE framework. |
7.
Implementation: Putting the Evaluation to Work
The work of a school or district evaluation
committee, and how to develop and implement and
evaluation management plan. |
8.
The Report: Communicating the Results
A sample reporting format (from the NC
Department of Public Instruction), and helpful tips
for writing an effective evaluation report. |
9.
Examples: Real Evaluation Plans and Reports
Evaluation plans from schools in North
Carolina and a school in Mississippi.
|
10.
Resources: Index of Materials Supporting Evaluation
An index of resources referred to on this
site, plus additional online information useful
to evaluation planning. |
 |
What
is Evaluation?
Evaluation
is the process of judging the value or worth of a product,
project, or program. Although the CAPE materials focus
on formative project evaluation - intended to help leaders
make decisions about revising or improving project implementation
- many of the concepts presented can be extended to
evaluations of other things (personnel, for example)
or for other purposes.
Why
is Evaluation Conducted?
Evaluation is typically conducted for three
reasons: 1) to determine what type of project should
be developed (often referred to broadly as needs assessment),
2) to determine if a project has been implemented as
intended and with quality (implementation evaluation),
and 3) to determine the effects of a project (impact
evaluation). There are other terms recognized by evaluators
as describing the same purposes, but these three terms
will be used here for the sake of consistency. Formative evaluation requires ascertaining
both the implementation quality and impact of a project.
Need
Assessment - Sometimes, evaluation is conducted
before a project is started (Rossi & Freeman, 1989).
This needs assessment (or conceptualization) phase involves
gathering data to determine gaps between the current
state of affairs in a particular situation (e.g., teachers'
uses of technology for instruction) and the desired
or optimal state. Project managers and other stakeholders
can then determine whether there is a gap and if so,
which are the most appropriate and pressing project
activities to foster and support.
To
assist with defining school technology needs, educators
might consider the School
Technology Needs Assessment (STNA, say "stenna"),
developed through a collaboration between SERVE and
the North Carolina Department of Public Instruction.
Another option is "TAGLIT:
A Tool for Measuring Project's Results" - an
assessment tool that can be used to determine how technology
is being used in a school.
Implementation
Evaluation - Evaluation is often conducted
to monitor the fidelity with which the activities of
a project are being implemented. A project manager who
is accountable to a funding agency may ne interested
in documenting internal processes and lessons learned,
or a project sponsor might want to verify that funded
projects are being implemented as promised. In such
cases, evaluation can help to verify that the specified
target populations are receiving the promised services,
or the sponsored activities are being carried out as
intended (e.g., "We said we should use technology,
and here is the evidence that we used technology.")
and with quality (e.g., technical support is provided
in a timely manner).
Impact
Evaluation - Evaluation is also conducted to
determine if a project has met its goals or has achieved
the desired outcomes. This type of evaluation is the
most powerful to argue for project continuation, as
it provides evidence that a particular initiative is
related to, or influenced, a positive change (i.e.,
"We said the project would improve student learning,
and here is the evidence that the project improved student
learning.").
When
is Evaluation Conducted?
The work of evaluation can span the life of
a project from pre-conceptualization, to monitoring
implemented initiatives, to judging any effects after
implementation.
Formative
evaluation is intended to determine if or how project
activities should be revised, or how resources might
be reallocated, while summative evaluation is applied to decide whether a project should be adopted
or continued. Both uses of evaluation findings may require
shorter- or longer-term evaluation efforts, or some
combination of the two. For example, a comprehensive
summative evaluation might include an examination of
immediate results (sometimes more specifically referred
to as project effects), and a longitudinal
study to determine long term impact.
The
terms formative and summative evaluation
are sometimes used primarily to describe when an evaluation
is conducted - during or after implementation. However,
this is not an entirely accurate use of these terms,
which more correctly refer to the intended purposes
described above.
How
is Evaluation Different from Assessment?
Evaluation is related to and often confused
with assessment. In the context of education,
assessment is the process used to determine how much
a student knows or how effectively a student can perform
some task relative to a criterion or standard. Evaluation
is intended to determine how well a project has achieved
the criteria or goals established for it. If a project's
goals include "improved student learning"
however, then the evaluation would likely utilize assessment
data. So while assessment and evaluation are different,
evaluations often utilize assessment data to help make
decisions about the value of the project under study.
Is
Evaluation the Same as Research?
Evaluation is related to research in that both
utilize similar data collection methods and statistical
analysis procedures. However, evaluation differs from
research in focus and involvement (Ohio Department of
Mental Health, 2002). research is based on existing
theory and attempts to tease out the relationship between
variables to add more knowledge to an existing field,
while evaluation seeks to determine project fidelity
of implementation and/or impact. Adding to theory or
developing a new theory is the focus of research, not
evaluation. This is not to say evaluation conclusions
have not been used to develop theory, just that the
primary focus of evaluation is checking for the attainment
of goals rather than contribution to theory.
Further,
individuals or groups who have reviewed the relevant
literature, designed a project, and wish to study its
effects, typically conduct research on their own intervention.
An evaluator, however, is often hired as an unbiased
third party to estimate the value of a project, from
a more distant perspective. A person working in this
capacity is typically called an external evaluator,
whereas someone conducting evaluation from within the
organization implementing the project under examination
is referred to as an internal evaluator.
Next > Getting Started |
