Investigating the Relationship between Examinee Effort and Examinee Emotions When Completing Low-Stakes Tests
Faculty Advisor Name
Dr. Sara J. Finney
Description
The current recommendation when collecting institutional accountability data in higher education includes also collecting effort data and using the effort data when interpreting results (AERA, APA, & NCME, 2014). There is variability in expended effort during a test that has no consequences for the student (Barry & Finney, 2016). Since effort has an effect on performance, ensuring students expend adequate levels of effort is of particular concern during tests that are low-stakes for the student (Wise & DeMars, 2005). Not only do levels of expended effort differ, expended effort may change over the course of a test (Wise & Smith, 2011). Additionally, emotions experienced while taking a test influence performance (McMillian, 2016). Similar to effort, emotions can change over the course of a test (Goetz, et al., 2007). To examine what impacts examinee effort, we investigated the relationship between the change in effort and the change in emotions while taking a test.
In the current study, 168 undergraduate students at James Madison University participated in institutional accountability testing. To measure the change in emotions and effort over the course of a test, we divided a 29-item cognitive test into three subtests (items 1-10 first “subtest”; items 11-20 second “subtest”; items 21-29 third “subtest”). After each third of the cognitive test, students completed measures of emotions and effort. The items used to measure emotions (anger, boredom, pride, and enjoyment) were adapted from the Test Emotions Questionnaire (TEQ) developed by Pekrun et al. (2005). The Student Opinion Scale (SOS; Sundre, 1999) is a measure of test importance and examinee motivation, which was used to measure effort.
Using Latent Growth Modeling (LGM), we found a significant linear decrease in effort as well as a significant linear increase in anger over the course of the test. Using multivariate LGM, we found a significant negative relationship between change in effort and change in anger. The more positive an examinee’s anger slope, the more negative the examinee’s effort slope. Examinees that experienced greater increases in anger experienced greater decreases in effort over the course of the test.
Since examinee effort impacts performance, it is pertinent to investigate constructs which impact effort. We found that the change in emotion (anger) over the course of the test occurred simultaneously with the change in effort over the course of the test. Assessment practitioners who are interested in ensuring examinees expend adequate effort during low-stakes tests should take steps to reduce anger levels experienced by students during the test. One possible strategy is to keep the test as short as possible to assess the content knowledge. Interventions to encourage positive emotions during a test may safeguard against the effects of negative emotions on examinee effort. Based on the results of this study, we argue for the consideration of emotions in addition to effort when interpreting test scores. Replication of this study with the addition of a reliable measure of performance is necessary to investigate the relationship between the change in emotions over the course of the test and performance.
Investigating the Relationship between Examinee Effort and Examinee Emotions When Completing Low-Stakes Tests
The current recommendation when collecting institutional accountability data in higher education includes also collecting effort data and using the effort data when interpreting results (AERA, APA, & NCME, 2014). There is variability in expended effort during a test that has no consequences for the student (Barry & Finney, 2016). Since effort has an effect on performance, ensuring students expend adequate levels of effort is of particular concern during tests that are low-stakes for the student (Wise & DeMars, 2005). Not only do levels of expended effort differ, expended effort may change over the course of a test (Wise & Smith, 2011). Additionally, emotions experienced while taking a test influence performance (McMillian, 2016). Similar to effort, emotions can change over the course of a test (Goetz, et al., 2007). To examine what impacts examinee effort, we investigated the relationship between the change in effort and the change in emotions while taking a test.
In the current study, 168 undergraduate students at James Madison University participated in institutional accountability testing. To measure the change in emotions and effort over the course of a test, we divided a 29-item cognitive test into three subtests (items 1-10 first “subtest”; items 11-20 second “subtest”; items 21-29 third “subtest”). After each third of the cognitive test, students completed measures of emotions and effort. The items used to measure emotions (anger, boredom, pride, and enjoyment) were adapted from the Test Emotions Questionnaire (TEQ) developed by Pekrun et al. (2005). The Student Opinion Scale (SOS; Sundre, 1999) is a measure of test importance and examinee motivation, which was used to measure effort.
Using Latent Growth Modeling (LGM), we found a significant linear decrease in effort as well as a significant linear increase in anger over the course of the test. Using multivariate LGM, we found a significant negative relationship between change in effort and change in anger. The more positive an examinee’s anger slope, the more negative the examinee’s effort slope. Examinees that experienced greater increases in anger experienced greater decreases in effort over the course of the test.
Since examinee effort impacts performance, it is pertinent to investigate constructs which impact effort. We found that the change in emotion (anger) over the course of the test occurred simultaneously with the change in effort over the course of the test. Assessment practitioners who are interested in ensuring examinees expend adequate effort during low-stakes tests should take steps to reduce anger levels experienced by students during the test. One possible strategy is to keep the test as short as possible to assess the content knowledge. Interventions to encourage positive emotions during a test may safeguard against the effects of negative emotions on examinee effort. Based on the results of this study, we argue for the consideration of emotions in addition to effort when interpreting test scores. Replication of this study with the addition of a reliable measure of performance is necessary to investigate the relationship between the change in emotions over the course of the test and performance.
