Sonography certification examinations: the influence of clinical education and practice tests after controlling for self-selection bias
Faculty Advisor Name
Dr. S. Jeanne Horst
Department
Department of Graduate Psychology
Description
In order to be an ultrasound technician, an individual is required to pass a certification examination, and become recertified every 10 years. The certification and recertification processes are important for maintaining standards for patient care and holding healthcare providers accountable for their knowledgeability and professionalism. Licensure organizations therefore, have a responsibility to set standards for the field, and support sonography professionals in the process. Some of this support involves providing resources, such as online practice tests to assist students and clinicians preparing for their certification examinations. Practice tests serve as a means of formative self-assessment for the examinees. Ideally, formative assessments help learners determine where they are, knowledge-wise, and what they need to do to get where they need to go (Black & Wiliam, 2009). Doing so, not only helps learners attain the factual knowledge they need, but helps them develop skills for assessing their own knowledge (Nitko & Brookhart, 2011). For an assessment to be considered ‘formative,’ it needs to provide learners with information about the standard for performance, and how their own performance compares against the ideal standard (Nicol & Macfarlane-Dick, 2006; Shepard, 2005).
Recent literature examined the effect of online practice tests on students’ performance on the American Registry for Diagnostic Medical Sonography (ARDMS) examination. Researchers randomly assigned students to a treatment (practice test) group and a control (no practice test) group. The practice test was formative in that it provided students with a detailed report, including level of preparedness, and a comparison to the standard pass/fail score of 555. The treatment group scored significantly higher on the ARDMS examination than the control group, showing promise for the benefits of practice testing (Wilson, 2018).
In real-world circumstances, the practice test is an option for any person who registers to take a sonogram certification exam. Due to usage of random assignment, experimental research studies such as the above cannot fully account for the existence of differences in the type of individual who would choose to take a practice test (selection bias).
The current study intends to account for these systematic group differences due to self-selection through use of a quasi-experimental technique called propensity score matching (PSM). PSM provides statistical means of balancing the intervention (practice test) and comparison (no practice test) groups on covariates, such as gender and area of certification, in order to enhance the accuracy of inferences regarding group differences. Researchers will use archival sonogram examination data from a global sonogram certification organization based in the Northeastern United States. The ultimate goal of the research is to examine, after using PSM to control for selection bias (1) whether taking a practice test influences sonography certification exam score, and (2) whether the effectiveness differs based on an individual’s clinical education (e.g., outpatient clinic, acute care setting).
Sonography certification examinations: the influence of clinical education and practice tests after controlling for self-selection bias
In order to be an ultrasound technician, an individual is required to pass a certification examination, and become recertified every 10 years. The certification and recertification processes are important for maintaining standards for patient care and holding healthcare providers accountable for their knowledgeability and professionalism. Licensure organizations therefore, have a responsibility to set standards for the field, and support sonography professionals in the process. Some of this support involves providing resources, such as online practice tests to assist students and clinicians preparing for their certification examinations. Practice tests serve as a means of formative self-assessment for the examinees. Ideally, formative assessments help learners determine where they are, knowledge-wise, and what they need to do to get where they need to go (Black & Wiliam, 2009). Doing so, not only helps learners attain the factual knowledge they need, but helps them develop skills for assessing their own knowledge (Nitko & Brookhart, 2011). For an assessment to be considered ‘formative,’ it needs to provide learners with information about the standard for performance, and how their own performance compares against the ideal standard (Nicol & Macfarlane-Dick, 2006; Shepard, 2005).
Recent literature examined the effect of online practice tests on students’ performance on the American Registry for Diagnostic Medical Sonography (ARDMS) examination. Researchers randomly assigned students to a treatment (practice test) group and a control (no practice test) group. The practice test was formative in that it provided students with a detailed report, including level of preparedness, and a comparison to the standard pass/fail score of 555. The treatment group scored significantly higher on the ARDMS examination than the control group, showing promise for the benefits of practice testing (Wilson, 2018).
In real-world circumstances, the practice test is an option for any person who registers to take a sonogram certification exam. Due to usage of random assignment, experimental research studies such as the above cannot fully account for the existence of differences in the type of individual who would choose to take a practice test (selection bias).
The current study intends to account for these systematic group differences due to self-selection through use of a quasi-experimental technique called propensity score matching (PSM). PSM provides statistical means of balancing the intervention (practice test) and comparison (no practice test) groups on covariates, such as gender and area of certification, in order to enhance the accuracy of inferences regarding group differences. Researchers will use archival sonogram examination data from a global sonogram certification organization based in the Northeastern United States. The ultimate goal of the research is to examine, after using PSM to control for selection bias (1) whether taking a practice test influences sonography certification exam score, and (2) whether the effectiveness differs based on an individual’s clinical education (e.g., outpatient clinic, acute care setting).
