State Assessment

Criteria for Passing a State Test Differ

In the 1980s, test scores were often reported in a “norm-referenced” way such that the average score for a school’s fifth-graders in math might be the 63rd percentile, meaning that on average, students in that grade level scored better than 63% of another representative sample of students. Now, state tests usually report results by proficiency levels. Students are typically reported as either being below grade level, at grade level, or above grade level.

Test developers must make decisions about what kind of test performance is expected for each proficiency level category. They have various approaches to setting these “cut scores,” but it is always a matter of judgment. That’s “judgment” in the good sense: the professional judgment of educators about what students should know and be able to do. Scale scores answer the question, “How did my child do?” Adding cut scores to describe levels of performance adds, “How good is that compared with expectations in my state?”

Different ways of deciding on cut scores (called standard-setting methods) yield different results. Standard setting methods are of two general types: student-based or test-based. In student-based methods, educators judge the performance level of students they know and then see what scores these students obtain. In test-based methods, educators judge the test questions and estimate the likelihood that students in various performance levels would answer them correctly or incorrectly. One student-based approach that test developers used in North Carolina had teachers place each student in one of four categories and then compared these ratings to the students’ actual test scores to determine the cut points.

South Carolina’s accountability system provides a direct example of how performance categories are defined differently. South Carolina adopted the same categories as the National Assessment of Educational Progress (NAEP). Table 3 provides a comparison of the 2003 results for fourth-grade reading on the state test and the state’s performance on the NAEP test. Notice that about 4 in 10 students are considered “Below Basic” on the NAEP test, while only 2.5 out of 10 are “Below Basic” on the state test.

State tests can be compared to national tests like the NAEP as a way of judging how easy or hard their cut scores are. If the state test results show a much larger percentage of students at “proficient” and “advanced” levels than the NAEP results for their state, then their test results may be inflated.

  Back