The Human as Delta-rule Learner

S. Lee, J.I. Gold, & J. W. Kable

Published in Decision

A long-standing debate in psychology concerns the best algorithmic description of learning. In delta-rule models, such as Rescorla-Wagner, beliefs are updated by a fixed proportion of errors in prediction. In contrast, alternative models, such as Pearce-Hall, posit that learning occurs more rapidly in response to surprising outcomes accompanied by large prediction errors. Recent studies that measure learning rates on a trial-by-trial basis have shown that humans adjust their beliefs to a greater degree in response to surprising outcomes, akin to Pearce-Hall, in environments where adjusting learning rates according to the size of the prediction error generates optimal predictions. Here we ask whether greater learning after surprising outcomes is a general feature of human belief updating, or whether human belief learning conforms to normative principles, exhibiting updates in fixed proportion to prediction errors, akin to Rescorla-Wagner, in environments where this is optimal. Specifically, normative predictions about variables that undergo Gaussian drift with noise require different fixed values of the learning rate depending on the drift rate (rate of environmental change) and noise (observation stochasticity). We found that human participants in such environments updated their beliefs in a trial-by-trial manner consistent with a fixed learning rate, the value of which was adjusted in the appropriate direction given changes in the drift rate or noise. However, learning rates were systematically higher than optimal, overweighting recent evidence. These results show that human belief updating conforms to the assumptions of widely-used delta-rule models of learning under the conditions for which such models provide normative predictions.

Sangil Lee2019, Decision