In practice, standardized tests can be composed of multiple-choice and true-false questions. Such items can be tested inexpensively and quickly by scoring special answer sheets by computer or via computer-adaptive testing. Some tests also have short-answer or essay writing components that are assigned a score by independent evaluators. These can be graded by evaluators who use rubrics (rules or guidelines) and anchor papers (examples of papers for each possible score) to determine the grade to be given to a response. A number of assessments, however, are not scored by people. For example, the Graduate Record Exam is a computer-adaptive assessment that requires no scoring by people (except for the writing portion).
Scoring Issues
There can be problems with human scoring. For example, the Seattle Times reported that for Washington State's WASL, temporary employees were paid $10 an hour. They spent as little as 20 seconds on each math problem, 2 and 1/2 minutes on an essay on items which may determine if a student graduates from high school, which some believe is a matter of concern given the high stakes nature of such tests. Pearson scores many other state tests similarly. Agreement between scorers can vary between 60 to 85 percent depending on the test and the scoring session. Sometimes states pay to have two or more scorers read each paper to improve reliability, though this does not eliminate test responses getting different scores.
Score Reference
There are two types of standardized tests: norm-referenced tests and criterion-referenced tests, resulting in a norm-referenced score or a criterion-referenced score, respectively. Norm-referenced scores compare test-takers to a sample of peers. Criterion-referenced scores compare test-takers to a criterion, and may also be described as standards-based assessment as they are aligned with the standards-based education reform movement. Norm-referenced tests are associated with traditional education, which measures success by rank ordering students, while standards-based assessments are based on the egalitarian belief that all students can succeed if they are assessed against high standards which are required of all students regardless of ability or economic background.