Stealth Assessment of Reading Comprehension Skill and Vocabulary Knowledge using Game Performance

The current study explores the extent to which literacy game performance can be used to assess reading comprehension skill and vocabulary knowledge. Standardized reading assessments have the benefit of years of validation across different age groups and reading comprehension levels, allowing teachers to evaluate students’ reading performance and relate it to a national standard. However, these assessments reduce classroom time for learning activities, which may be more authentic indicators of student progress. Students’ reading skills can be measured during learning activities by using game-based stealth assessment of literacy. Game-based assessment may be more enjoyable and less likely to invoke test anxiety than traditional assessments, but enjoyment may also impact the validity of the assessment. The current study recruited participants (n=405) to play five literacy games: CON-Artist, Paraphrase Quest, Fix It, Map Conquest, and Vocab Flash. Students also completed the Gates-MacGinitie Reading Test (GMRT), which serves as a validated measure of reading comprehension skill and vocabulary knowledge. Students answered enjoyment questions after each game and the GMRT, and they completed the Cognitive Test Anxiety questionnaire, which measures trait-level negative thoughts about test-taking. The results indicate that Vocab Flash predicted 31% of variance in reading comprehension and 21% of variance in vocabulary knowledge. The other games were not predictive beyond Vocab Flash, but each of them was weakly correlated with reading comprehension skill and vocabulary knowledge. Three games were more enjoyable than GMRT Reading Comprehension, but no games were more enjoyable than GMRT Vocabulary. Cognitive Test Anxiety was negatively correlated with the GMRT and Vocab Flash, but not with the other games. Game enjoyment moderated the relationship between game performance and reading skill, albeit in differing directions. Paraphrase Quest was less predictive of reading comprehension for students who enjoyed the game, and Vocab Flash was more predictive of reading comprehension for those who enjoyed the game. The findings of this study suggest that a simple vocabulary game can be used to measure reading comprehension skill and vocabulary knowledge. Future research is needed to better understand how game-based assessments can be designed to minimize the effects of test anxiety and enjoyment on performance.