ESTONIAN ACADEMY
PUBLISHERS
eesti teaduste
akadeemia kirjastus
PUBLISHED
SINCE 1997
 
TRAMES cover
TRAMES. A Journal of the Humanities and Social Sciences
ISSN 1736-7514 (Electronic)
ISSN 1406-0922 (Print)
Impact Factor (2022): 0.2
TEST-TAKING EFFORT AS A PREDICTOR OF PERFORMANCE IN LOW-STAKES TESTS; pp. 433–448
PDF | DOI: 10.3176/tr.2013.4.08

Authors
Gerli Silm, Olev Must, Karin Täht
Abstract

We studied how time measures can be used as predictors of test-taking performance in low-stakes tests. Our sample consisted of undergraduate students (N = 327) who took a computer-based cognitive abilities test. Our aim was to find how test-takers’ motivation manifests itself in test-taking effort. We found that a high test-taking speed is related to low test scores (the correlation between test score and Response Time Effort was r = .71). Also, the mean time for wrong answers per item was smaller than the time for right answers (mean effect size d = .22). We found that performance in low-stakes tests is influenced by two test-taking effort characteristics: the number of items the test-taker attempts to solve and the mean time that is devoted to solve an item (β = .4–.5).We suggest that test-taking motivation should be studied further as it may provide useful information for interpreting results of tests and examinations.

References

American Educational Research Association, American Psychological Association, and National Council for Measurement in Education (AERA, APA, and NCME) (1999) Standards for educational and psychological testing. Washington DC: American Educational Research Association.

Atkinson, John W. (1957) “Motivational determinants of risk-taking behaviour”. Psychological Review 64, 359–372.
http://dx.doi.org/10.1037/h0043445

Baumert, Jürgen and Anke Demmrich (2001) “Test motivation in the assessment of student skills: the effects of incentives on motivation and performance”. European Journal of Psychology of Education 16, 441–462.
http://dx.doi.org/10.1007/BF03173192

Barry, Carol L. and Sara J. Finney (2009) “Exploring change in test-taking motivation”. Paper presented at the 2009 annual meeting of the Northeastern Educational Research Association, October, in Rocky Hill, CT.

Barry, Carol L., S. Jeanne Horst, Sara J. Finney, Allison R. Brown, and Jason P. Kopp (2010) “do examinees have similar test-taking effort? A high-stakes question for low-stakes testing”. International Journal of Testing 10, 342–363.
http://dx.doi.org/10.1080/15305058.2010.508569

Brand, Christopher (1996) The g factor: general intelligence and its implications. New York: John Wiley and Sons Inc.

Cole, James S. and Steven J. Osterlind (2008) “Investigating differences between low- and high-stakes test performance on a general education exam”. The Journal of General Education 57, 119–130.
http://dx.doi.org/10.1353/jge.0.0018

Cronbach, Lee J. (1984) Essentials of psychological testing. New York: Harper and Row.

Duckworth, Angela Lee, Patrick D. Quinn, Donald R. Lynam, Rolf Loeber, and Magda Stouthamer-Loeber (2011) “Role of test motivation in intelligence testing”. Proceedings of the National Academy of Sciences of the United States of America 108, 7716–7720.
http://dx.doi.org/10.1073/pnas.1018601108

Eccles, Jacquelynne S. and Allan Wigfield (2002) “Motivational beliefs, values, and goals”. Annual Review of Psychology 53, 109–132.
http://dx.doi.org/10.1146/annurev.psych.53.100901.135153

Eklöf, Hanna (2006) “Development and validation of scores from an instrument measuring student test-taking motivation”. Educational and Psychological Measurement 66, 4, 643–56.
http://dx.doi.org/10.1177/0013164405278574

Eklöf, Hanna (2010) “Skill and will: test-taking motivation and assessment quality”. Assessment in Education: Principles, Policy and Practice 17, 4, 345–356.
http://dx.doi.org/10.1080/0969594X.2010.516569

Furneaux, W. D. (1960) “Intellectual abilities and problem solving behaviour”. In Handbook of abnormal psychology, 167–192. H. J. Eysenck, ed. London: Pergamon Press.

Herrnstein, Richard J. and Charles Murray (1994) The Bell curve. New York: A Free Press.

Jensen, Arthur (1998) The g factor: the science of mental ability. Westport: Praeger.

Klein Entink, R., J. Kuhn, L. Hornke, and J. Fox (2009) “Evaluating cognitive theory: a joint model­ing approach using responses and response times”. Psychological Methods 14, 54–75.
http://dx.doi.org/10.1037/a0014877

Mislevy, Robert J. (1995a) “What can we learn from international assessments?”. Educational Evaluation and Policy Analysis 17, 419–437.
http://dx.doi.org/10.3102/01623737017004419

Must, Olev and Jüri Allik (2002) “Tunne oma võimeid. Abivahend eneseanalüüsiks”. [Know your abilities: a tool for self-analysis.] Tartu: Tartu University Press.

Partchev, Ivailo, Paul De Boeck, and Rolf Steyer (2011) “How much power and speed is measured in this test?” Assessment 20, 242–252.
http://dx.doi.org/10.1177/1073191111411658

Schnipke, Deborah L. and David J. Scrams (1997) “Modeling item response times with a two-state mixture model: a new method of measuring speededness“. Journal of Educational Measure­ment 34, 3, 213–232.
http://dx.doi.org/10.1111/j.1745-3984.1997.tb00516.x

Sheppard, Leah D. and Philip A. Vernon (2008) “Intelligence and speed of information-processing: a review of 50 years of research”. Personality and Individual Differences 44, 535–551.
http://dx.doi.org/10.1016/j.paid.2007.09.015

Spearman, Charles E. (1927) The abilities of man: their nature and measurement. London: Macmillan and Co.

Sundre, Donna L. and Anastasia Kitsantas (2004) “An exploration of the psychology of the examinee: can examinee self-regulation and test-taking motivation predict consequential and non consequential test performance?”. Contemporary Educational Psychology 29, 1, 6–26.
http://dx.doi.org/10.1016/S0361-476X(02)00063-2

Sundre, Donna L. and Deborah L. Moore (2002) “The student opinion scale: a measure of examinee motivation”. Assessment Update 14, 1, 8–9.
http://dx.doi.org/10.1002/au.144

Thorndike, Edward L. (1927) The measurement of intelligence. New York: Bureau of Publications Teachers College, Columbia University.

Unt, Marge, Kadri Täht, Ellu Saar, and Jelena Helemäe (2013) “The expansion of higher education: devaluation or differentiation?” In Higher education at a crossroad: the case of Estonia, 367–390. Ellu Saar and René Mõttus, eds. Frankfurt am Main: Peter Lang.

van der Linden, Wim J. (2011) “Test design and speededness”. Journal of Educational Measurement 48, 44–60.
http://dx.doi.org/10.1111/j.1745-3984.2010.00130.x

van der Maas, H., D. Molneaar, G. Maris, R. Kievit, and D. Borsboom (2011) “Cognitive psychology meets psychometric theory: on the relation between process models for decision making and latent variable models for individual differences”. Psychological Review 118, 339–356.
http://dx.doi.org/10.1037/a0022749

Wicherts, Jelte M. and Annemarie Zand Scholten (2010) “Test anxiety and the validity of cognitive tests: a confirmatory factor analysis perspective and some empirical findings”. Intelligence 38, 169–178.
http://dx.doi.org/10.1016/j.intell.2009.09.008

Wigfield, Allan and Jacquelynne S. Eccles (2000) “Expectancy-value theory of achievement motiva­tion”. Contemporary Educational Psychology 25, 68–81.
http://dx.doi.org/10.1006/ceps.1999.1015

Wilhelm, Oliver and Ralf Schulze (2002) “The relation of speeded and unspeeded reasoning with mental speed”. Intelligence 30, 537–554.
http://dx.doi.org/10.1016/S0160-2896(02)00086-7

Wise, Steven L. and Christine E. DeMars (2005) “Low examinee effort in low-stakes assessment: problems and potential solutions”. Educational Assessment 10, 1, 1–17.
http://dx.doi.org/10.1207/s15326977ea1001_1

Wise, Steven L. and Christine. E. DeMars (2010) “Examinee noneffort and the validity of program assessment results”. Educational Assessment 15, 1, 27–41.
http://dx.doi.org/10.1080/10627191003673216

Wise, Steven L. and Xiaojing Kong (2005) “Response time effort: a new measure of examinee motivation in computer-based tests”. Applied Measurement in Education 18, 163–183.
http://dx.doi.org/10.1207/s15324818ame1802_2

Wise, Steven L., Dena A. Pastor, and Xiaojing J. Kong (2009) “Correlates of rapid-guessing behaviour in low-stakes testing: implications for test development and measurement practice”. Applied Measurement in Education 22, 185–205.
http://dx.doi.org/10.1080/08957340902754650

Wolf, Lisa F., Jeffrey K. Smith, and Marilyn E. Birnbaum (1995) “Consequence of performance, test motivation, and mentally taxing items”. Applied Measurement in Education 8, 4, 341–351.
http://dx.doi.org/10.1207/s15324818ame0804_4

Back to Issue