Preferred Name

Kathryn N Thompson

Creative Commons License

Creative Commons License
This work is licensed under a Creative Commons Attribution-Noncommercial-No Derivative Works 4.0 License.

Date of Graduation

5-8-2020

Document Type

Thesis

Degree Name

Master of Arts (MA)

Department

Department of Graduate Psychology

Advisor(s)

Brian C. Leventhal

Abstract

Distractors, or the incorrect options, are an important part of the multiple-choice item. Previous literature has supported the inclusion of distractors when estimating abilities. While the effects of well-functioning distractors on estimates of ability have been examined, research has neglected to examine the effects of undesirable distractors on estimates of ability. Undesirable distractors are defined as distractors that are opposite of what test-developers expect or want distractors to behave. For instance, an upper lure distractor is one that high ability examinees select rather than selecting the correct answer. A simulation study was employed to determine these effects by varying undesirable distractor type, percentage of items containing undesirable distractors, and test length. Item responses were generated using the Thissen-Steinberg multiple-choice model for simulating undesirable distractors and the three-parameter logistic model for simulating normal items. Following data generation, item responses were analyzed using the three-parameter logistic model in SAS. An analysis of covariance (ANCOVA) was used to examine the effects of undesirable distractors on estimates of ability for bias and standard error. Multiple significant interactions were identified for bias and standard error. One type of undesirable distractor that was especially problematic was the lower lure distractor, where high ability examinees have a slightly lower, but still high, probability of being selected in comparison to the correct answer. Additionally, a longer test resulted in the least amount of bias and standard error. Overall, test-developers should pay attention to the functioning of distractors, as there are effects of these undesirable distractors on estimates of ability.

Share

COinS
 
 

To view the content in your browser, please download Adobe Reader or, alternately,
you may Download the file to your hard drive.

NOTE: The latest versions of Adobe Reader do not support viewing PDF files within Firefox on Mac OS and if you are using a modern (Intel) Mac, there is no official plugin for viewing PDF files within the browser window.