Exploration and recency as the main proximate causes of probability matching: a reinforcement learning analysis

Autor(es)

da Silva, Carolina Feher ^[1] ; Victorino, Camila Gomes ^[2] ; Caticha, Nestor ^[1] ; Baldo, Marcus Vinicius Chrysostomo ^[2]

Afiliação do(s) autor(es)

^[1] Univ Sao Paulo, Inst Phys, Dept Gen Phys, Rua Matao 1371, Cidade Univ, BR-05508090 Sao Paulo, SP - Brazil

^[2] Univ Sao Paulo, Inst Biomed Sci, Dept Physiol & Biophys, Av Prof Lineu Prestes 1524, ICB 1, Cidade Univ, BR-05508000 Sao Paulo, SP - Brazil

• Tipo de documento: Artigo Científico
• Fonte: SCIENTIFIC REPORTS; v. 7, NOV 10 2017.
• Citações Web of Science: 0

Resumo

Research has not yet reached a consensus on why humans match probabilities instead of maximise in a probability learning task. The most influential explanation is that they search for patterns in the random sequence of outcomes. Other explanations, such as expectation matching, are plausible, but do not consider how reinforcement learning shapes people's choices. We aimed to quantify how human performance in a probability learning task is affected by pattern search and reinforcement learning. We collected behavioural data from 84 young adult participants who performed a probability learning task wherein the majority outcome was rewarded with 0.7 probability, and analysed the data using a reinforcement learning model that searches for patterns. Model simulations indicated that pattern search, exploration, recency (discounting early experiences), and forgetting may impair performance. Our analysis estimated that 85% (95% HDI {[}76, 94]) of participants searched for patterns and believed that each trial outcome depended on one or two previous ones. The estimated impact of pattern search on performance was, however, only 6%, while those of exploration and recency were 19% and 13% respectively. This suggests that probability matching is caused by uncertainty about how outcomes are generated, which leads to pattern search, exploration, and recency. (AU)

• Processo FAPESP: 13/10694-0 - Papel da recompensa e dos núcleos da base na tomada de decisão
• Beneficiário: Carolina Feher da Silva
• Modalidade de apoio: Bolsas no Brasil - Pós-Doutorado
• Processo FAPESP: 13/13352-2 - Escolhas binárias repetidas como modelo de tomada de decisão em dimensões perceptuais não espaciais
• Beneficiário: Marcus Vinícius Chrysóstomo Baldo
• Modalidade de apoio: Bolsas no Exterior - Pesquisa