Publication Date
4-2023
Document Type
Presented Paper
Abstract
In Facets models, raters should be connected, and there are multiple ways to connect raters. Keeping the number of ratings constant and two raters scoring each examinee, the standard error of both rater severity and examinee ability was higher when raters scored one examinee in common with many different raters than when they scored many examinees in common with two raters. However, the differences were small, especially for the standard error of examinee ability. Alternatively, when only a subset of examinees were scored by two or more raters, the smallest standard errors were achieved when all raters scored a common linking set, although standard errors were larger than they were when all examinees were scored by two raters. If the rating design only allows for a single rating of most examinees, it is preferable to link the metric by assigning all raters to rate the same set of linking examinees.
Creative Commons License
This work is licensed under a Creative Commons Attribution-Noncommercial-No Derivative Works 4.0 License.
Recommended Citation
DeMars, C. E., Shapovalov, Y. A., & Hathcoat, J. D. (2023, April 13-15). Many-Facet Rasch Designs: How Should Raters be Assigned to Examinees? [Paper presentation]. National Council on Measurement in Education annual meeting, Chicago.
