Moderators of Male-Female and White-Black Subgroup Differences in Assessment Center Ratings: A Meta-Analysis
Faculty Advisor Name
Dr. Adam Vanhove
Department
School of Strategic Leadership Studies
Description
The assessment center (AC) method has become a popular tool in human resource management for making high-stakes selection and promotion decisions because of its ability to predict job performance while also minimizing gender and racial discrimination. Recent reviews highlight the variability in estimates of gender and racial subgroup differences observed across primary studies using the assessment center (AC) method. Using meta-analysis, we sought to explain this variability by testing the effects of a series of moderators using primary AC field studies. Among male-female comparison data, we examined: type of AC dimension (task-oriented versus interpersonal-oriented), the female-to-male proportion of assessees, the number of dimensions being rated, the mean correlation among dimension ratings, and the rigor of AC design and implementation. Among White-Black comparison data, we examined: type of AC dimension (cognitively loaded versus non-cognitively loaded), the Black-to-White proportion of assessees, the rigor of pre-testing procedures, the number of dimensions being rated, the mean correlation among dimension ratings, and the rigor of AC design and implementation. We found two statistically significant moderators of White-Black subgroup difference estimates: mean correlation between AC dimension ratings and methodological rigor with which ACs were designed and implemented. Both suggest that introducing greater opportunity for rating biases is associated with larger subgroup differences favoring White assessees. Findings regarding White-Black subgroup differences have important implications for designing and implementing ACs. ACs came to prominence as a viable alternative to cognitive ability testing based on evidence showing strong criterion-related validity as well as minimal risk for adverse impact. However, our findings show that the benefits of using ACs may be compromised to the extent that AC designers fail to sufficiently differentiate rating dimensions, follow methodological best practices, and perhaps minimize the cognitive loading of rating criteria. Given that the AC method is just that, a method, considerable care must be taken to ensure that operational ACs eliminate opportunities for subjective biases and minimize the cognitive loading of rating criteria if they are to provide the benefits that made ACs popular in the first place. Although we found many effects on the magnitude of male-female differences to be in the hypothesized direction, none reached the level of statistical significance, suggesting that moderating effects on gender differences may be nuanced. We discuss these findings in terms of the practical implications for minimizing racial subgroup differences, and theoretical implications for stereotype-fit models of gender discrimination.
Moderators of Male-Female and White-Black Subgroup Differences in Assessment Center Ratings: A Meta-Analysis
The assessment center (AC) method has become a popular tool in human resource management for making high-stakes selection and promotion decisions because of its ability to predict job performance while also minimizing gender and racial discrimination. Recent reviews highlight the variability in estimates of gender and racial subgroup differences observed across primary studies using the assessment center (AC) method. Using meta-analysis, we sought to explain this variability by testing the effects of a series of moderators using primary AC field studies. Among male-female comparison data, we examined: type of AC dimension (task-oriented versus interpersonal-oriented), the female-to-male proportion of assessees, the number of dimensions being rated, the mean correlation among dimension ratings, and the rigor of AC design and implementation. Among White-Black comparison data, we examined: type of AC dimension (cognitively loaded versus non-cognitively loaded), the Black-to-White proportion of assessees, the rigor of pre-testing procedures, the number of dimensions being rated, the mean correlation among dimension ratings, and the rigor of AC design and implementation. We found two statistically significant moderators of White-Black subgroup difference estimates: mean correlation between AC dimension ratings and methodological rigor with which ACs were designed and implemented. Both suggest that introducing greater opportunity for rating biases is associated with larger subgroup differences favoring White assessees. Findings regarding White-Black subgroup differences have important implications for designing and implementing ACs. ACs came to prominence as a viable alternative to cognitive ability testing based on evidence showing strong criterion-related validity as well as minimal risk for adverse impact. However, our findings show that the benefits of using ACs may be compromised to the extent that AC designers fail to sufficiently differentiate rating dimensions, follow methodological best practices, and perhaps minimize the cognitive loading of rating criteria. Given that the AC method is just that, a method, considerable care must be taken to ensure that operational ACs eliminate opportunities for subjective biases and minimize the cognitive loading of rating criteria if they are to provide the benefits that made ACs popular in the first place. Although we found many effects on the magnitude of male-female differences to be in the hypothesized direction, none reached the level of statistical significance, suggesting that moderating effects on gender differences may be nuanced. We discuss these findings in terms of the practical implications for minimizing racial subgroup differences, and theoretical implications for stereotype-fit models of gender discrimination.
