Decomposing the influences of uncertainty on learning
Rasmus Bruckner · Refubium (Universitätsbibliothek der Freien Universität Berlin) · 2021
Learning often takes place in environments with uncertainty about current and future outcomes. To behave adaptively in these circumstances, people need to learn beliefs from past experiences, based on which they can predict future outcomes. In my dissertation, I examine: (1) Normative computations that should determine learning under uncertainty. (2) Uncertainty biases that lead to deviations from normative learning. (3) Age-related differences in learning under uncertainty that are characteristic across the lifespan. Here, the term normative computations from the field of computational neuroscience refers to computations that provide an optimal solution to a learning and decision-making problem. My dissertation studies draw on computational models that implement normative computations and formally define uncertainty. Based on these models, the studies systematically investigated to what degree younger adults and people across the lifespan consider uncertainty when learning from their experiences. I begin by illustrating that adaptive behavior consists of several related steps, including a representation of the environment, decision making, and learning (Introduction). Based on this, I present a framework that decomposes uncertainty into three forms: perceptual uncertainty, expected uncertainty, and unexpected uncertainty (Normative computations). Perceptual uncertainty is related to sensory information processing, expected uncertainty arises from outcome variability, and unexpected uncertainty is the consequence of changes in the environment. For each form, I describe how individuals should learn under uncertainty based on normative computations. I then show that biases, that is, deviations from a normative consideration of uncertainty, are characteristic of human learning behavior (Uncertainty biases). Finally, I motivate why capturing these biases in computational models of cognition can improve our understanding of age-related lifespan differences in learning under uncertainty (Lifespan differences). The first dissertation study (Bruckner et al., 2020a) examined which normative computations should guide learning under perceptual uncertainty, to which degree humans regulate learning accordingly, and how past perceptual choices bias this process. The second study (Nassar et al., 2016) investigated expected and unexpected uncertainty in younger and older adults, particularly how biases in the consideration of uncertainty explain age-related learning differences. The third study (Bruckner et al., 2020b) built upon this and examined the role of simplified learning strategies across the lifespan. Finally, the fourth study (Van den Bos et al., 2018) was an opinion paper on how applying computational cognitive models advances our understanding of age-related lifespan differences in learning and decision making. In the following, I briefly summarize the results of the dissertation studies mentioned above. In Bruckner et al. (2020a), we showed that perceptual uncertainty often corrupts learning because of misinterpreted perceptual information. Learning behavior under perceptual uncertainty should be more cautious than in perceptually clear situations to avoid such misinterpretations. We found that humans consider perceptual uncertainty during learning. However, we also identified learning biases driven by previous perceptual choices, which led to a less cautious regulation of learning. In Nassar et al. (2016), our results suggested that age-related learning differences are related to the adjustment to expected uncertainty. In particular, we found that older adults (60 to 80 years) exhibit a bias to underestimate uncertainty about their beliefs compared to younger adults (20 to 30 years). This form of uncertainty underestimation leads to less flexible learning behavior compared to younger adults. In Bruckner et al. (2020b), we found that age-related impairments in learning under uncertainty often arise because children (7 to 11 years) and older adults resort to simplified learning strategies that lead to more repetitive responding (perseveration) and stronger environmental influences on behavior (environmental control) compared to younger adults. Finally, in Van den Bos et al. (2018), we argued that computational cognitive models are an essential tool to gain a better understanding of age-related learning and decision-making differences. In particular, we illustrated both promises of the application of computational models to study age-related behavioral differences (concerning risk-taking, strategy selection, and reinforcement learning) and potential pitfalls. After discussing the implications of these studies (General discussion and future directions), I propose a cognitive model of learning under uncertainty based on the new insights of my studies and previous work in the literature (Uncertainty in the cycle of adaptive behavior). In summary, the dissertation highlights that learning is a dynamic process that is influenced by multiple forms of uncertainty. People take uncertainty into account during learning but show inherent uncertainty biases that substantially change across the lifespan.