We studied how time measures can be used as predictors of test-taking performance in low-stakes tests. Our sample consisted of undergraduate students (N = 327) who took a computer-based cognitive abilities test. Our aim was to find how test-takers’ motivation manifests itself in test-taking effort. We found that a high test-taking speed is related to low test scores (the correlation between test score and Response Time Effort was r = .71). Also, the mean time for wrong answers per item was smaller than the time for right answers (mean effect size d = .22). We found that performance in low-stakes tests is influenced by two test-taking effort characteristics: the number of items the test-taker attempts to solve and the mean time that is devoted to solve an item (β = .4–.5).We suggest that test-taking motivation should be studied further as it may provide useful information for interpreting results of tests and examinations.
American Educational Research Association, American Psychological Association, and National Council for Measurement in Education (AERA, APA, and NCME) (1999) Standards for educational and psychological testing. Washington DC: American Educational Research Association.
Atkinson, John W. (1957) “Motivational determinants of risk-taking behaviour”. Psychological Review 64, 359–372.
http://dx.doi.org/10.1037/h0043445
Baumert, Jürgen and Anke Demmrich (2001) “Test motivation in the assessment of student skills: the effects of incentives on motivation and performance”. European Journal of Psychology of Education 16, 441–462.
http://dx.doi.org/10.1007/BF03173192
Barry, Carol L. and Sara J. Finney (2009) “Exploring change in test-taking motivation”. Paper presented at the 2009 annual meeting of the Northeastern Educational Research Association, October, in Rocky Hill, CT.
Barry, Carol L., S. Jeanne Horst, Sara J. Finney, Allison R. Brown, and Jason P. Kopp (2010) “do examinees have similar test-taking effort? A high-stakes question for low-stakes testing”. International Journal of Testing 10, 342–363.
http://dx.doi.org/10.1080/15305058.2010.508569
Brand, Christopher (1996) The g factor: general intelligence and its implications. New York: John Wiley and Sons Inc.
Cole, James S. and Steven J. Osterlind (2008) “Investigating differences between low- and high-stakes test performance on a general education exam”. The Journal of General Education 57, 119–130.
http://dx.doi.org/10.1353/jge.0.0018
Cronbach, Lee J. (1984) Essentials of psychological testing. New York: Harper and Row.
Duckworth, Angela Lee, Patrick D. Quinn, Donald R. Lynam, Rolf Loeber, and Magda Stouthamer-Loeber (2011) “Role of test motivation in intelligence testing”. Proceedings of the National Academy of Sciences of the United States of America 108, 7716–7720.
http://dx.doi.org/10.1073/pnas.1018601108
Eccles, Jacquelynne S. and Allan Wigfield (2002) “Motivational beliefs, values, and goals”. Annual Review of Psychology 53, 109–132.
http://dx.doi.org/10.1146/annurev.psych.53.100901.135153
Eklöf, Hanna (2006) “Development and validation of scores from an instrument measuring student test-taking motivation”. Educational and Psychological Measurement 66, 4, 643–56.
http://dx.doi.org/10.1177/0013164405278574
Eklöf, Hanna (2010) “Skill and will: test-taking motivation and assessment quality”. Assessment in Education: Principles, Policy and Practice 17, 4, 345–356.
http://dx.doi.org/10.1080/0969594X.2010.516569
Furneaux, W. D. (1960) “Intellectual abilities and problem solving behaviour”. In Handbook of abnormal psychology, 167–192. H. J. Eysenck, ed. London: Pergamon Press.
Herrnstein, Richard J. and Charles Murray (1994) The Bell curve. New York: A Free Press.
Jensen, Arthur (1998) The g factor: the science of mental ability. Westport: Praeger.
Klein Entink, R., J. Kuhn, L. Hornke, and J. Fox (2009) “Evaluating cognitive theory: a joint modeling approach using responses and response times”. Psychological Methods 14, 54–75.
http://dx.doi.org/10.1037/a0014877
Mislevy, Robert J. (1995a) “What can we learn from international assessments?”. Educational Evaluation and Policy Analysis 17, 419–437.
http://dx.doi.org/10.3102/01623737017004419
Must, Olev and Jüri Allik (2002) “Tunne oma võimeid. Abivahend eneseanalüüsiks”. [Know your abilities: a tool for self-analysis.] Tartu: Tartu University Press.
Partchev, Ivailo, Paul De Boeck, and Rolf Steyer (2011) “How much power and speed is measured in this test?” Assessment 20, 242–252.
http://dx.doi.org/10.1177/1073191111411658
Schnipke, Deborah L. and David J. Scrams (1997) “Modeling item response times with a two-state mixture model: a new method of measuring speededness“. Journal of Educational Measurement 34, 3, 213–232.
http://dx.doi.org/10.1111/j.1745-3984.1997.tb00516.x
Sheppard, Leah D. and Philip A. Vernon (2008) “Intelligence and speed of information-processing: a review of 50 years of research”. Personality and Individual Differences 44, 535–551.
http://dx.doi.org/10.1016/j.paid.2007.09.015
Spearman, Charles E. (1927) The abilities of man: their nature and measurement. London: Macmillan and Co.
Sundre, Donna L. and Anastasia Kitsantas (2004) “An exploration of the psychology of the examinee: can examinee self-regulation and test-taking motivation predict consequential and non consequential test performance?”. Contemporary Educational Psychology 29, 1, 6–26.
http://dx.doi.org/10.1016/S0361-476X(02)00063-2
Sundre, Donna L. and Deborah L. Moore (2002) “The student opinion scale: a measure of examinee motivation”. Assessment Update 14, 1, 8–9.
http://dx.doi.org/10.1002/au.144
Thorndike, Edward L. (1927) The measurement of intelligence. New York: Bureau of Publications Teachers College, Columbia University.
Unt, Marge, Kadri Täht, Ellu Saar, and Jelena Helemäe (2013) “The expansion of higher education: devaluation or differentiation?” In Higher education at a crossroad: the case of Estonia, 367–390. Ellu Saar and René Mõttus, eds. Frankfurt am Main: Peter Lang.
van der Linden, Wim J. (2011) “Test design and speededness”. Journal of Educational Measurement 48, 44–60.
http://dx.doi.org/10.1111/j.1745-3984.2010.00130.x
van der Maas, H., D. Molneaar, G. Maris, R. Kievit, and D. Borsboom (2011) “Cognitive psychology meets psychometric theory: on the relation between process models for decision making and latent variable models for individual differences”. Psychological Review 118, 339–356.
http://dx.doi.org/10.1037/a0022749
Wicherts, Jelte M. and Annemarie Zand Scholten (2010) “Test anxiety and the validity of cognitive tests: a confirmatory factor analysis perspective and some empirical findings”. Intelligence 38, 169–178.
http://dx.doi.org/10.1016/j.intell.2009.09.008
Wigfield, Allan and Jacquelynne S. Eccles (2000) “Expectancy-value theory of achievement motivation”. Contemporary Educational Psychology 25, 68–81.
http://dx.doi.org/10.1006/ceps.1999.1015
Wilhelm, Oliver and Ralf Schulze (2002) “The relation of speeded and unspeeded reasoning with mental speed”. Intelligence 30, 537–554.
http://dx.doi.org/10.1016/S0160-2896(02)00086-7
Wise, Steven L. and Christine E. DeMars (2005) “Low examinee effort in low-stakes assessment: problems and potential solutions”. Educational Assessment 10, 1, 1–17.
http://dx.doi.org/10.1207/s15326977ea1001_1
Wise, Steven L. and Christine. E. DeMars (2010) “Examinee noneffort and the validity of program assessment results”. Educational Assessment 15, 1, 27–41.
http://dx.doi.org/10.1080/10627191003673216
Wise, Steven L. and Xiaojing Kong (2005) “Response time effort: a new measure of examinee motivation in computer-based tests”. Applied Measurement in Education 18, 163–183.
http://dx.doi.org/10.1207/s15324818ame1802_2
Wise, Steven L., Dena A. Pastor, and Xiaojing J. Kong (2009) “Correlates of rapid-guessing behaviour in low-stakes testing: implications for test development and measurement practice”. Applied Measurement in Education 22, 185–205.
http://dx.doi.org/10.1080/08957340902754650
http://dx.doi.org/10.1207/s15324818ame0804_4