Repeatedly unreliable, and unreliably repeatable: the trials-to-criterion fallacy.

Authors

Robert J Dugand

Published in

Animal cognition. Jul 13, 2026. Epub Jul 13, 2026.

Abstract

Trials-to-criterion (TTC) is a common method used to quantify learning speed and estimate intelligence. Often, individuals are presented with tasks that have two choices: one has a reward (pass), while the other does not (fail). The task is repeatedly presented until a pre-determined criterion is met (e.g., five passes in a row). The total number of trials to reach the criterion purportedly estimates intelligence, with lower scores reflecting higher intelligence. However, in any given trial, the probability of guessing correctly and passing is 50%, meaning that lucky streaks can readily result in individuals reaching the task criterion by chance. Although the criterion is intended to minimise lucky streaks and ensure that < 5% of tasks are completed by random chance, this is a long recognised, but underappreciated, fallacy of TTC. Using simulations, I show the extent to which random runs of passes, rather than deterministic reaching of the task completion criterion via learning, conflate 'smart' with 'lucky'. Moreover, occasional errors by intelligent individuals can dramatically inflate scores, conflating 'smart-less' with 'luck-less'. The extreme variability in outcomes generated by TTC for a fixed level of intelligence exposes its unreliability and suggests that inferences based on TTC data may often be misleading. I offer considerations for future studies that aim to reduce the risk of artefactual results, but ultimately caution against TTC.

PMID:
42440134
Bibliographic data and abstract were imported from PubMed on 13 Jul 2026.

Read full publication at:
Please sign in to see all details.

Stats

1-terrible, 9-excellent. How would you rate this publication? Sign in in to submit your rating.

Post a comment

You need to be signed in to post comments. You can sign in here.

Comments

There are no comments yet.