TEST-TAKING EFFORT AS A PREDICTOR OF PERFORMANCE IN LOW-STAKES TESTS; pp. 433–448Full article in PDF format
| DOI: 10.3176/tr.2013.4.08
We studied how time measures can be used as predictors of test-taking performance in low-stakes tests. Our sample consisted of undergraduate students (N = 327) who took a computer-based cognitive abilities test. Our aim was to find how test-takers’ motivation manifests itself in test-taking effort. We found that a high test-taking speed is related to low test scores (the correlation between test score and Response Time Effort was r = .71). Also, the mean time for wrong answers per item was smaller than the time for right answers (mean effect size d = .22). We found that performance in low-stakes tests is influenced by two test-taking effort characteristics: the number of items the test-taker attempts to solve and the mean time that is devoted to solve an item (β = .4–.5).We suggest that test-taking motivation should be studied further as it may provide useful information for interpreting results of tests and examinations.
American Educational Research Association, American Psychological Association, and National Council for Measurement in Education (AERA, APA, and NCME) (1999) Standards for educational and psychological testing. Washington DC: American Educational Research Association.
Atkinson, John W. (1957) “Motivational determinants of risk-taking behaviour”. Psychological Review 64, 359–372.
Baumert, Jürgen and Anke Demmrich (2001) “Test motivation in the assessment of student skills: the effects of incentives on motivation and performance”. European Journal of Psychology of Education 16, 441–462.
Barry, Carol L. and Sara J. Finney (2009) “Exploring change in test-taking motivation”. Paper presented at the 2009 annual meeting of the Northeastern Educational Research Association, October, in Rocky Hill, CT.
Barry, Carol L., S. Jeanne Horst, Sara J. Finney, Allison R. Brown, and Jason P. Kopp (2010) “do examinees have similar test-taking effort? A high-stakes question for low-stakes testing”. International Journal of Testing 10, 342–363.
Brand, Christopher (1996) The g factor: general intelligence and its implications. New York: John Wiley and Sons Inc.
Cole, James S. and Steven J. Osterlind (2008) “Investigating differences between low- and high-stakes test performance on a general education exam”. The Journal of General Education 57, 119–130.
Cronbach, Lee J. (1984) Essentials of psychological testing. New York: Harper and Row.
Duckworth, Angela Lee, Patrick D. Quinn, Donald R. Lynam, Rolf Loeber, and Magda Stouthamer-Loeber (2011) “Role of test motivation in intelligence testing”. Proceedings of the National Academy of Sciences of the United States of America 108, 7716–7720.
Eccles, Jacquelynne S. and Allan Wigfield (2002) “Motivational beliefs, values, and goals”. Annual Review of Psychology 53, 109–132.
Eklöf, Hanna (2006) “Development and validation of scores from an instrument measuring student test-taking motivation”. Educational and Psychological Measurement 66, 4, 643–56.
Eklöf, Hanna (2010) “Skill and will: test-taking motivation and assessment quality”. Assessment in Education: Principles, Policy and Practice 17, 4, 345–356.
Furneaux, W. D. (1960) “Intellectual abilities and problem solving behaviour”. In Handbook of abnormal psychology, 167–192. H. J. Eysenck, ed. London: Pergamon Press.
Herrnstein, Richard J. and Charles Murray (1994) The Bell curve. New York: A Free Press.
Jensen, Arthur (1998) The g factor: the science of mental ability. Westport: Praeger.
Klein Entink, R., J. Kuhn, L. Hornke, and J. Fox (2009) “Evaluating cognitive theory: a joint modeling approach using responses and response times”. Psychological Methods 14, 54–75.
Mislevy, Robert J. (1995a) “What can we learn from international assessments?”. Educational Evaluation and Policy Analysis 17, 419–437.
Must, Olev and Jüri Allik (2002) “Tunne oma võimeid. Abivahend eneseanalüüsiks”. [Know your abilities: a tool for self-analysis.] Tartu: Tartu University Press.
Partchev, Ivailo, Paul De Boeck, and Rolf Steyer (2011) “How much power and speed is measured in this test?” Assessment 20, 242–252.
Schnipke, Deborah L. and David J. Scrams (1997) “Modeling item response times with a two-state mixture model: a new method of measuring speededness“. Journal of Educational Measurement 34, 3, 213–232.
Sheppard, Leah D. and Philip A. Vernon (2008) “Intelligence and speed of information-processing: a review of 50 years of research”. Personality and Individual Differences 44, 535–551.
Spearman, Charles E. (1927) The abilities of man: their nature and measurement. London: Macmillan and Co.
Sundre, Donna L. and Anastasia Kitsantas (2004) “An exploration of the psychology of the examinee: can examinee self-regulation and test-taking motivation predict consequential and non consequential test performance?”. Contemporary Educational Psychology 29, 1, 6–26.
Sundre, Donna L. and Deborah L. Moore (2002) “The student opinion scale: a measure of examinee motivation”. Assessment Update 14, 1, 8–9.
Thorndike, Edward L. (1927) The measurement of intelligence. New York: Bureau of Publications Teachers College, Columbia University.
Unt, Marge, Kadri Täht, Ellu Saar, and Jelena Helemäe (2013) “The expansion of higher education: devaluation or differentiation?” In Higher education at a crossroad: the case of Estonia, 367–390. Ellu Saar and René Mõttus, eds. Frankfurt am Main: Peter Lang.
van der Linden, Wim J. (2011) “Test design and speededness”. Journal of Educational Measurement 48, 44–60.
van der Maas, H., D. Molneaar, G. Maris, R. Kievit, and D. Borsboom (2011) “Cognitive psychology meets psychometric theory: on the relation between process models for decision making and latent variable models for individual differences”. Psychological Review 118, 339–356.
Wicherts, Jelte M. and Annemarie Zand Scholten (2010) “Test anxiety and the validity of cognitive tests: a confirmatory factor analysis perspective and some empirical findings”. Intelligence 38, 169–178.
Wigfield, Allan and Jacquelynne S. Eccles (2000) “Expectancy-value theory of achievement motivation”. Contemporary Educational Psychology 25, 68–81.
Wilhelm, Oliver and Ralf Schulze (2002) “The relation of speeded and unspeeded reasoning with mental speed”. Intelligence 30, 537–554.
Wise, Steven L. and Christine E. DeMars (2005) “Low examinee effort in low-stakes assessment: problems and potential solutions”. Educational Assessment 10, 1, 1–17.
Wise, Steven L. and Christine. E. DeMars (2010) “Examinee noneffort and the validity of program assessment results”. Educational Assessment 15, 1, 27–41.
Wise, Steven L. and Xiaojing Kong (2005) “Response time effort: a new measure of examinee motivation in computer-based tests”. Applied Measurement in Education 18, 163–183.
Wise, Steven L., Dena A. Pastor, and Xiaojing J. Kong (2009) “Correlates of rapid-guessing behaviour in low-stakes testing: implications for test development and measurement practice”. Applied Measurement in Education 22, 185–205.
Wolf, Lisa F., Jeffrey K. Smith, and Marilyn E. Birnbaum (1995) “Consequence of performance, test motivation, and mentally taxing items”. Applied Measurement in Education 8, 4, 341–351.
http://dx.doi.org/10.1207/s15324818ame0804_4Back to Issue