Do Different Racial/Ethnic Groups Perform Differently on Wellness Assessments? An Application of a Diagnostic Cognitive Model to Inform JMU's General Education Curriculum
Faculty Advisor Name
Yu Bao
Department
Department of Graduate Psychology
Description
The assessment of student learning outcomes (SLOs) in higher education is a crucial element in understanding the strength and weaknesses of academic and general education programs. When reporting assessment results, however, results are summarized in an aggregated manner as opposed to being reported by specific groups (e.g., racial/ethnic groups), which might impede the design of group-specific learning strategies. As an example, race/ethnicity has been found to be an important factor in health and wellness (e.g., Oliver et al., 2017). In addition, Van Dyke (2001) has found that different racial/ethnic groups perform differently across different wellness components. More specifically, part of Van Dyke’s (2001) results shows that Caucasians (Non-Hispanic) had, on average, better performance on the preventive behaviors wellness component than did Blacks /African Americans (Non-Hispanic) and Hispanics.
Hence, the purpose of this study is to examine performance and provide feedback for different racial/ethnic groups on a wellness assessment to potentially inform curriculum changes in the wellness domain of JMU's General Education program (fifth cluster).
The cognitive section of the Knowledge of Wellness and Health (KWH) test (28 items) was used to assess students' knowledge in the three wellness awareness SLOs covered in the fifth cluster courses. The KWH test was administered to the same group of students in the Fall 2019 and Spring 2021 semesters to assess students' initial performance and growth on the three SLOs. A Diagnostic cognitive model (DCM), a modern psychometric model, which aims to provide SLO-level and individual-level feedback on students' mastery of knowledge was used to measure students' initial performance and growth based on their pre- and post-assessment responses. Moreover, DCM was also used to disaggregate feedback by racial/ethnic groups. In this study, only students who identified themselves with the following racial/ethnic groups were examined: Caucasian (non-Hispanic), Black/African-American (non-Hispanic), Asian, and Hispanic. Students who identify themselves as Native Americans (or Alaska Natives), as Native Hawaiian (or other Pacific Islanders), or as multiracial were not examined in this study.
Results show an overall increase in mastery levels across the three wellness SLOs between Fall 2019 and Spring 2021 (SLO 1: 16%, SLO 2: 26 %, SLO 3: 15%). At pre-assessment, across the three wellness goals, Asians, Blacks/Africans (non-Hispanic), Hispanics, and Caucasians had an initial mastery performance of 64%, 36%, 49 %, and 65 % respectively. At post-assessment, a notable increase in performance is seen for Hispanics and Blacks/African-Americans with performance nearly equaling those of Caucasians and Asians (Asians: 89%, Blacks/African – American: 83%, Hispanics: 82%, Caucasians: 87%. Results suggest that JMU’s General Education program courses in the fifth cluster are effective across the different ethnic/racial groups analyzed. Item parameter estimation results also suggest the need for further investigations of the KWH assessment to improve item quality and test validity.
Do Different Racial/Ethnic Groups Perform Differently on Wellness Assessments? An Application of a Diagnostic Cognitive Model to Inform JMU's General Education Curriculum
The assessment of student learning outcomes (SLOs) in higher education is a crucial element in understanding the strength and weaknesses of academic and general education programs. When reporting assessment results, however, results are summarized in an aggregated manner as opposed to being reported by specific groups (e.g., racial/ethnic groups), which might impede the design of group-specific learning strategies. As an example, race/ethnicity has been found to be an important factor in health and wellness (e.g., Oliver et al., 2017). In addition, Van Dyke (2001) has found that different racial/ethnic groups perform differently across different wellness components. More specifically, part of Van Dyke’s (2001) results shows that Caucasians (Non-Hispanic) had, on average, better performance on the preventive behaviors wellness component than did Blacks /African Americans (Non-Hispanic) and Hispanics.
Hence, the purpose of this study is to examine performance and provide feedback for different racial/ethnic groups on a wellness assessment to potentially inform curriculum changes in the wellness domain of JMU's General Education program (fifth cluster).
The cognitive section of the Knowledge of Wellness and Health (KWH) test (28 items) was used to assess students' knowledge in the three wellness awareness SLOs covered in the fifth cluster courses. The KWH test was administered to the same group of students in the Fall 2019 and Spring 2021 semesters to assess students' initial performance and growth on the three SLOs. A Diagnostic cognitive model (DCM), a modern psychometric model, which aims to provide SLO-level and individual-level feedback on students' mastery of knowledge was used to measure students' initial performance and growth based on their pre- and post-assessment responses. Moreover, DCM was also used to disaggregate feedback by racial/ethnic groups. In this study, only students who identified themselves with the following racial/ethnic groups were examined: Caucasian (non-Hispanic), Black/African-American (non-Hispanic), Asian, and Hispanic. Students who identify themselves as Native Americans (or Alaska Natives), as Native Hawaiian (or other Pacific Islanders), or as multiracial were not examined in this study.
Results show an overall increase in mastery levels across the three wellness SLOs between Fall 2019 and Spring 2021 (SLO 1: 16%, SLO 2: 26 %, SLO 3: 15%). At pre-assessment, across the three wellness goals, Asians, Blacks/Africans (non-Hispanic), Hispanics, and Caucasians had an initial mastery performance of 64%, 36%, 49 %, and 65 % respectively. At post-assessment, a notable increase in performance is seen for Hispanics and Blacks/African-Americans with performance nearly equaling those of Caucasians and Asians (Asians: 89%, Blacks/African – American: 83%, Hispanics: 82%, Caucasians: 87%. Results suggest that JMU’s General Education program courses in the fifth cluster are effective across the different ethnic/racial groups analyzed. Item parameter estimation results also suggest the need for further investigations of the KWH assessment to improve item quality and test validity.
