| What
is Scientifically Based Research?
Randomized
Experiments
One
of the most important goals of research applied to practice
is to test claims of effectiveness—the determination
that innovations cause certain predictable outcomes.
This kind of research is most effective in determining
that interventions actually cause outcomes and is the
research design most valued by the Institute of Education
Sciences. In experimental research designs, a well-defined
innovation is tested for effectiveness by exposing two
groups of participants to alternate experiences—the
treatment and the control
versions—and measuring the difference between
the groups on one or more post-treatment outcome measures.
Typically, the treatment version is the innovation under
study, while the control version represents commonly
accepted practice. Differences between the groups in
average outcomes, if they are statistically significant
and have a large enough effect size,
represent the effects or impacts of the intervention.
|
|
|
|
| For
differences in outcomes between treatment and control groups
to be seen as believable evidence for claims of effectiveness,
it is necessary to be sure that there are no other explanations
for the differences between experimental and control conditions,
such as ethnicity or socioeconomic factors. In a true experiment,
this is accomplished by random assignment
of participants to either the treatment or control conditions.
It is this random assignment that is the keystone of scientific
research.
There
are a number of other important considerations that researchers
must make when they test claims of effectiveness using experimental
research designs:
- For
randomization to function properly, sample sizes need to
be reasonably large. In small samples, random assignment
is not necessarily a guarantee of equivalence between groups.
- The
ability to apply experimental findings beyond the research
setting depends on the inclusion of important contextual
variables as factors in the study. For example,
the research design can incorporate such factors as ethnicity,
rurality, and socioeconomic level, and random assignment
to treatment conditions can be conducted within levels of
these contextual factors.
- Outcomes,
such as achievement test scores, should be objectively measured
and of high reliability and validity. Objectivity
means that the researcher should not have tried to influence
the outcomes of the study to get the result he or she wanted.
Reliability and validity
of outcome variables play an important role in drawing inferences
from experiments. To be valid, the research
has to measure the effect of the intervention on the outcomes
the researchers were looking for. To be reliable,
the outcomes of several trials of the intervention have
to render similar results.
|
