Unemployed and Turking: Is Amazon Mechanical Turk a Viable Source for Unemployment Research?
Faculty Advisor Name
Adam Vanhove
Description
Given the high accessibility and low cost associated with sampling via MTurk, research based on MTurk data is playing an increasing role in the development of knowledge and theory throughout the organizational sciences. Although most evidence regarding the validity of MTurk data for organizational science research has been positive, evidence has been primarily related to the reliability of results and demographic equivalence. This study provides additional evidence to the applicability of MTurk data for unemployment studies while raising questions to the applicability of data for employment studies.
Beyond the demographic comparisons typically completed in MTurk validation studies (Analysis 1), this study investigates whether respondent demographics, traits and psychosocial health relate to employment level in the general sample (Analyses 2 & 3) and to general well-being in the unemployed sample (Analysis 4). Hypotheses are guided by two theoretical perspectives. Jahoda’s (1981) latent deprivation model (LDM) suggests that job loss and unemployment deprives individuals of key psychosocial resources (latent benefits) causing a decline in well-being. Second, the drift hypothesis suggests preexisting psychosocial characteristics influence employability (Dooley, Catalano & Hough, 1992).
We suggest that MTurk samples may be better suited to study under/unemployment dynamics rather than measures related to employment. This recommendation is based upon several key results. First, the Unemployed subgroup showed demographics similar to the national unemployed subpopulation while there were demographic differences between the employed MTurk subgroup and national employed subpopulation. Second, the relationship between latent benefit variables on well-being replicated previous evidence and theory in the Unemployed subgroup. Third, there was evidence, albeit weaker, of a relationship between dispositional personality traits and employment stability. Finally, employment status was not predicted by expected differences in latent benefit variables which questioned the level of difference between the Unemployed and Employed subgroups.
This study provides evidence of the applicability of using crowdsourcing samples through Amazon MTurk for studying employment. It also highlights that organizational science researchers need to defend the appropriateness (beyond convenience) of using crowdsource workers’ data. Additionally, researchers must channel respondents based on whether they can provide relevant insight into the phenomenon under study.
Unemployed and Turking: Is Amazon Mechanical Turk a Viable Source for Unemployment Research?
Given the high accessibility and low cost associated with sampling via MTurk, research based on MTurk data is playing an increasing role in the development of knowledge and theory throughout the organizational sciences. Although most evidence regarding the validity of MTurk data for organizational science research has been positive, evidence has been primarily related to the reliability of results and demographic equivalence. This study provides additional evidence to the applicability of MTurk data for unemployment studies while raising questions to the applicability of data for employment studies.
Beyond the demographic comparisons typically completed in MTurk validation studies (Analysis 1), this study investigates whether respondent demographics, traits and psychosocial health relate to employment level in the general sample (Analyses 2 & 3) and to general well-being in the unemployed sample (Analysis 4). Hypotheses are guided by two theoretical perspectives. Jahoda’s (1981) latent deprivation model (LDM) suggests that job loss and unemployment deprives individuals of key psychosocial resources (latent benefits) causing a decline in well-being. Second, the drift hypothesis suggests preexisting psychosocial characteristics influence employability (Dooley, Catalano & Hough, 1992).
We suggest that MTurk samples may be better suited to study under/unemployment dynamics rather than measures related to employment. This recommendation is based upon several key results. First, the Unemployed subgroup showed demographics similar to the national unemployed subpopulation while there were demographic differences between the employed MTurk subgroup and national employed subpopulation. Second, the relationship between latent benefit variables on well-being replicated previous evidence and theory in the Unemployed subgroup. Third, there was evidence, albeit weaker, of a relationship between dispositional personality traits and employment stability. Finally, employment status was not predicted by expected differences in latent benefit variables which questioned the level of difference between the Unemployed and Employed subgroups.
This study provides evidence of the applicability of using crowdsourcing samples through Amazon MTurk for studying employment. It also highlights that organizational science researchers need to defend the appropriateness (beyond convenience) of using crowdsource workers’ data. Additionally, researchers must channel respondents based on whether they can provide relevant insight into the phenomenon under study.
