Structured, uncertainty-driven exploration in real-world consumer choice

Multiple and subject-specific roles of uncertainty in reward-guided decision-making

Decision-making in noisy, changing, and partially observable environments entails a basic tradeoff between immediate reward and longer-term information gain, known as the exploration-exploitation dilemma. Computationally, an effective way to balance this tradeoff is by leveraging uncertainty to guide exploration. Yet, in humans, empirical findings are mixed, from suggesting uncertainty-seeking to indifference and avoidance. In a novel bandit task that better captures uncertainty-driven behavior, we find multiple roles for uncertainty in human choices. First, stable and psychologically meaningful individual differences in uncertainty preferences actually range from seeking to avoidance, which can manifest as null group-level effects. Second, uncertainty modulates the use of basic decision heuristics that imperfectly exploit immediate rewards: a repetition bias and win-stay-lose-shift heuristic. These heuristics interact with uncertainty, favoring heuristic choices under higher uncertainty. These results, highlighting the rich and varied structure of reward-based choice, are a step to understanding its functional basis and dysfunction in psychopathology.

Testing the convergent validity, domain generality, and temporal stability of selected measures of people's tendency to explore

Given the ubiquity of exploration in everyday life, researchers from many disciplines have developed methods to measure exploratory behaviour. There are therefore many ways to quantify and measure exploration. However, it remains unclear whether the different measures (i) have convergent validity relative to one another, (ii) capture a domain general tendency, and (iii) capture a tendency that is stable across time. In a sample of 678 participants, we found very little evidence of convergent validity for the behavioural measures (Hypothesis 1); most of the behavioural measures lacked sufficient convergent validity with one another or with the self-reports. In psychometric modelling analyses, we could not identify a good fitting model with an assumed general tendency to explore (Hypothesis 2); the best fitting model suggested that the different behavioural measures capture behaviours that are specific to the tasks. In a subsample of 254 participants who completed the study a second time, we found that the measures had stability across an 1 month timespan (Hypothesis 3). Therefore, although there were stable individual differences in how people approached each task across time, there was no generalizability across tasks, and drawing broad conclusions about exploratory behaviour from studies using these tasks may be problematic. The Stage 1 protocol for this Registered Report was accepted in principle on 2nd December 2022 https://doi.org/10.6084/m9.figshare.21717407.v1 . The protocol, as accepted by the journal, can be found at https://doi.org/10.17605/OSF.IO/64QJU .

Political reinforcement learners

Politics can seem home to the most calculating and yet least rational elements of humanity. How might we systematically characterize this spectrum of political cognition? Here, we propose reinforcement learning (RL) as a unified framework to dissect the political mind. RL describes how agents algorithmically navigate complex and uncertain domains like politics. Through this computational lens, we outline three routes to political differences, stemming from variability in agents' conceptions of a problem, the cognitive operations applied to solve the problem, or the backdrop of information available from the environment. A computational vantage on maladies of the political mind offers enhanced precision in assessing their causes, consequences, and cures.

Information foraging with an oracle

During ecological decisions, such as when foraging for food or selecting a weekend activity, we often have to balance the costs and benefits of exploiting known options versus exploring novel ones. Here, we ask how individuals address such cost-benefit tradeoffs during tasks in which we can either explore by ourselves or seek external advice from an oracle (e.g., a domain expert or recommendation system). To answer this question, we designed two studies in which participants chose between inquiring (at a cost) for expert advice from an oracle, or to search for options without guidance, under manipulations affecting the optimal choice. We found that participants showed a greater propensity to seek expert advice when it was instrumental to increase payoff (study A), and when it reduced choice uncertainty, above and beyond payoff maximization (study B). This latter result was especially apparent in participants with greater trait-level intolerance of uncertainty. Taken together, these results suggest that we seek expert advice for both economic goals (i.e., payoff maximization) and epistemic goals (i.e., uncertainty minimization) and that our decisions to ask or not ask for advice are sensitive to cost-benefit tradeoffs.

The scaling of mental computation in a sorting task

Many cognitive models provide valuable insights into human behavior. Yet the algorithmic complexity of candidate models can fail to capture how human reaction times scale with increasing input complexity. In the current work, we investigate the algorithms underlying human cognitive processes. Computer science characterizes algorithms by their time and space complexity scaling with problem size. We propose to use participants' reaction times to study how human computations scale with increasing input complexity. We tested this approach in a task where participants had to sort sequences of rectangles by their size. Our results showed that reaction times scaled close to linearly with sequence length and that participants learned and actively used latent structure whenever it was provided. This behavior was in line with a computational model that used the observed sequences to form hypotheses about the latent structures, searching through candidate hypotheses in a directed fashion. These results enrich our understanding of plausible cognitive models for efficient mental sorting and pave the way for future studies using reaction times to investigate the scaling of mental computations across psychological domains.

PET-measured human dopamine synthesis capacity and receptor availability predict trading rewards and time-costs during foraging

Foraging behavior requires weighing costs of time to decide when to leave one reward patch to search for another. Computational and animal studies suggest that striatal dopamine is key to this process; however, the specific role of dopamine in foraging behavior in humans is not well characterized. We use positron emission tomography (PET) imaging to directly measure dopamine synthesis capacity and D1 and D2/3 receptor availability in 57 healthy adults who complete a computerized foraging task. Using voxelwise data and principal component analysis to identify patterns of variation across PET measures, we show that striatal D1 and D2/3 receptor availability and a pattern of mesolimbic and anterior cingulate cortex dopamine function are important for adjusting the threshold for leaving a patch to explore, with specific sensitivity to changes in travel time. These findings suggest a key role for dopamine in trading reward benefits against temporal costs to modulate behavioral adaptions to changes in the reward environment critical for foraging.

Empowerment contributes to exploration behaviour in a creative video game

Studies of human exploration frequently cast people as serendipitously stumbling upon good options. Yet these studies may not capture the richness of exploration strategies that people exhibit in more complex environments. Here we study behaviour in a large dataset of 29,493 players of the richly structured online game 'Little Alchemy 2'. In this game, players start with four elements, which they can combine to create up to 720 complex objects. We find that players are driven not only by external reward signals, such as an attempt to produce successful outcomes, but also by an intrinsic motivation to create objects that empower them to create even more objects. We find that this drive for empowerment is eliminated when playing a game variant that lacks recognizable semantics, indicating that people use their knowledge about the world and its possibilities to guide their exploration. Our results suggest that the drive for empowerment may be a potent source of intrinsic motivation in richly structured domains, particularly those that lack explicit reward signals.

Understanding the first-offer conundrum: How buyer offers impact sale price and impasse risk in 26 million eBay negotiations

How low is the ideal first offer? Prior to any negotiation, decision-makers must balance a crucial tradeoff between two opposing effects. While lower first offers benefit buyers by anchoring the price in their favor, an overly ambitious offer increases the impasse risk, thus potentially precluding an agreement altogether. Past research with simulated laboratory or classroom exercises has demonstrated either a first offer's anchoring benefits or its impasse risk detriments, while largely ignoring the other effect. In short, there is no empirical answer to the conundrum of how low an ideal first offer should be. Our results from over 26 million incentivized real-world negotiations on eBay document (a) a linear anchoring effect of buyer offers on sales price, (b) a nonlinear, quartic effect on impasse risk, and (c) specific offer values with particularly low impasse risks but high anchoring benefits. Integrating these findings suggests that the ideal buyer offer lies at 80% of the seller's list price across all products-although this value ranges from 33% to 95% depending on the type of product, demand, and buyers' weighting of price versus impasse risk. We empirically amend the well-known midpoint bias, the assumption that buyer and seller eventually meet in the middle of their opening offers, and find evidence for a "buyer bias." Product demand moderates the (non)linear effects, the ideal buyer offer, and the buyer bias. Finally, we apply machine learning analyses to predict impasses and present a website with customizable first-offer advice configured to different products, prices, and buyers' risk preferences.

Adaptive tuning of human learning and choice variability to unexpected uncertainty

Human value-based decisions are notably variable under uncertainty. This variability is known to arise from two distinct sources: variable choices aimed at exploring available options and imprecise learning of option values due to limited cognitive resources. However, whether these two sources of decision variability are tuned to their specific costs and benefits remains unclear. To address this question, we compared the effects of expected and unexpected uncertainty on decision-making in the same reinforcement learning task. Across two large behavioral datasets, we found that humans choose more variably between options but simultaneously learn less imprecisely their values in response to unexpected uncertainty. Using simulations of learning agents, we demonstrate that these opposite adjustments reflect adaptive tuning of exploration and learning precision to the structure of uncertainty. Together, these findings indicate that humans regulate not only how much they explore uncertain options but also how precisely they learn the values of these options.

Microfoundations of Adaptive Search in Complex Tasks: The Role of Cognitive Abilities and Styles

Problem-solving in complex environments requires a cognitively demanding search for task solutions. Managing this search process presents a major challenge in organizations. We contribute to the literature on this topic by providing new evidence on the cognitive antecedents that shape how individuals search when engaged in complex problem-solving tasks. We present results from three laboratory studies, wherein 335 individuals solved a complex task. In doing so, they generated behavioral data coupled with survey-based measurements of the individuals’ cognitive styles and performance-based tests of their cognitive abilities. Our data analysis contributes to the current literature by documenting systematic heterogeneity in the persistence and distance of search that can be explained by the participants’ level of creativity, attention to detail, and executive functions. We extend the research on the microfoundations of adaptive search by linking cognitive antecedents with a complex search task, widening our insight into what search behavior certain cognitive microfoundations lead to, and showing how managers can more effectively shape organizational search.

History: This paper has been accepted for the Organization Science Special Issue on Experiments in Organizational Theory.

Supplemental Material: The online appendix is available at https://doi.org/10.1287/orsc.2023.1654 .

Individual differences in naturalistic learning link negative emotionality to the development of anxiety

Organisms learn from prediction errors (PEs) to predict the future. Laboratory studies using small financial outcomes find that humans use PEs to update expectations and link individual differences in PE-based learning to internalizing disorders. Because of the low-stakes outcomes in most tasks, it is unclear whether PE learning emerges in naturalistic, high-stakes contexts and whether individual differences in PE learning predict psychopathology risk. Using experience sampling to assess 625 college students' expected exam grades, we found evidence of PE-based learning and a general tendency to discount negative PEs, an "optimism bias." However, individuals with elevated negative emotionality, a personality trait linked to the development of anxiety disorders, displayed a global pessimism and learning differences that impeded accurate expectations and predicted future anxiety symptoms. A sensitivity to PEs combined with an aversion to negative PEs may result in a pessimistic and inaccurate model of the world, leading to anxiety.

Computational meaningfulness as the source of beneficial cognitive biases

The human brain has evolved to solve the problems it encounters in multiple environments. In solving these challenges, it forms mental simulations about multidimensional information about the world. These processes produce context-dependent behaviors. The brain as overparameterized modeling organ is an evolutionary solution for producing behavior in a complex world. One of the most essential characteristics of living creatures is that they compute the values of information they receive from external and internal contexts. As a result of this computation, the creature can behave in optimal ways in each environment. Whereas most other living creatures compute almost exclusively biological values (e.g., how to get food), the human as a cultural creature computes meaningfulness from the perspective of one's activity. The computational meaningfulness means the process of the human brain, with the help of which an individual tries to make the respective situation comprehensible to herself to know how to behave optimally. This paper challenges the bias-centric approach of behavioral economics by exploring different possibilities opened up by computational meaningfulness with insight into wider perspectives. We concentrate on confirmation bias and framing effect as behavioral economics examples of cognitive biases. We conclude that from the computational meaningfulness perspective of the brain, the use of these biases are indispensable property of an optimally designed computational system of what the human brain is like. From this perspective, cognitive biases can be rational under some conditions. Whereas the bias-centric approach relies on small-scale interpretable models which include only a few explanatory variables, the computational meaningfulness perspective emphasizes the behavioral models, which allow multiple variables in these models. People are used to working in multidimensional and varying environments. The human brain is at its best in such an environment and scientific study should increasingly take place in such situations simulating the real environment. By using naturalistic stimuli (e.g., videos and VR) we can create more realistic, life-like contexts for research purposes and analyze resulting data using machine learning algorithms. In this manner, we can better explain, understand and predict human behavior and choice in different contexts.

Counterfactual curiosity: motivated thinking about what might have been

Counterfactual information, information about what might have been, forms the content of counterfactual thoughts and emotions like regret and relief. Recent research suggests that human adults and children, as well as rhesus monkeys, demonstrate 'counterfactual curiosity': they are motivated to seek out counterfactual information after making decisions. Based on contemporary theories of curiosity and information seeking and a broad range of empirical literature, we suggest multiple heterogeneous psychological processes that contribute to people's motivation for counterfactual information. This includes processes that are identified in the curiosity literature more generally-the potential use of counterfactual information for adaptive decision making (its long-term instrumental value) and the drive to reduce uncertainty. Additionally, we suggest that counterfactual information may be particularly alluring because of its role in causal reasoning; its relationship with prediction and decision making; and its potential to fulfil emotion regulation and self-serving goals. Some future directions have been suggested, including investigating the role of individual differences in counterfactual curiosity on learning and wellbeing. This article is part of the theme issue 'Thinking about possibilities: mechanisms, ontogeny, functions and phylogeny'.

Humans adaptively resolve the explore-exploit dilemma under cognitive constraints: Evidence from a multi-armed bandit task

When navigating uncertain worlds, humans must balance exploring new options versus exploiting known rewards. Longer horizons and spatially structured option values encourage humans to explore, but the impact of real-world cognitive constraints such as environment size and memory demands on explore-exploit decisions is unclear. In the present study, humans chose between options varying in uncertainty during a multi-armed bandit task with varying environment size and memory demands. Regression and cognitive computational models of choice behavior showed that with a lower cognitive load, humans are more exploratory than a simulated value-maximizing learner, but under cognitive constraints, they adaptively scale down exploration to maintain exploitation. Thus, while humans are curious, cognitive constraints force people to decrease their strategic exploration in a resource-rational-like manner to focus on harvesting known rewards.

A guide to area-restricted search: a foundational foraging behaviour

Area-restricted search is the capacity to change search effort adaptively in response to resource encounters or expectations, from directional exploration (global, extensive search) to focused exploitation (local, intensive search). This search pattern is used by numerous organisms, from worms and insects to humans, to find various targets, such as food, mates, nests, and other resources. Area-restricted search has been studied for at least 80 years by ecologists, and more recently in the neurological and psychological literature. In general, the conditions promoting this search pattern are: (1) clustered resources; (2) active search (e.g. not a sit-and-wait predator); (3) searcher memory for recent target encounters or expectations; and (4) searcher ignorance about the exact location of targets. Because area-restricted search adapts to resource encounters, the search can be performed at multiple spatial scales. Models and experiments have demonstrated that area-restricted search is superior to alternative search patterns that do not involve a memory of the exact location of the target, such as correlated random walks or Lévy walks/flights. Area-restricted search is triggered by sensory cues whereas concentrated search in the absence of sensory cues is associated with other forms of foraging. Some neural underpinnings of area-restricted search are probably shared across metazoans, suggesting a shared ancestry and a shared solution to a common ecological problem of finding clustered resources. Area-restricted search is also apparent in other domains, such as memory and visual search in humans, which may indicate an exaptation from spatial search to other forms of search. Here, we review these various aspects of area-restricted search, as well as how to identify it, and point to open questions.

People adaptively use information to improve their internal states and external outcomes

Information can strongly impact people's affect, their level of uncertainty and their decisions. It is assumed that people seek information with the goal of improving all three. But are they successful at achieving this goal? Answering this question is important for assessing the impact of self-driven information consumption on people's well-being. Here, over five experiments (total N = 727) we show that participants accurately predict the impact of information on their internal states (e.g., affect and cognition) and external outcomes (e.g., material rewards), and use these predictions to guide information-seeking choices. A model incorporating participants' subjective expectations regarding the impact of information on their affective, cognitive, and material outcomes accounted for information-seeking choices better than a model that included only objective proxies of those measures. This model also accounted for individual differences in information-seeking choices. By balancing considerations of the impact of information on affective, cognitive and material outcomes when seeking knowledge, participants became happier, more certain and made better decisions when they sought information relative to when they did not, suggesting that the actual consequences of receiving information aligned with their subjective expectations.

Unhealthy Food at Your Fingertips: Cross-Sectional Analysis of the Nutritional Quality of Restaurants and Takeaway Outlets on an Online Food Delivery Platform in New Zealand

Online food delivery (OFD) platforms have become increasingly popular due to advanced technology, which is changing the way consumers purchase food prepared outside of the home. There is limited research investigating the healthiness of the digital food environment and its influence on consumer choice and dietary behaviours. This study is the first to examine the nutritional quality and marketing attributes of menu items from popular independent and franchise restaurants and takeaway outlets on New Zealand’s market leading OFD platform (UberEATS®). A total of 374 popular independent and franchise restaurants and takeaway outlets were identified to form a database of complete menus and marketing attributes. All 25,877 menu items were classified into 38 food and beverage categories based on the Australian Dietary Guidelines. Of complete menus, 73.3% (18,955/25,877) were discretionary. Thirty-six percent (9419/25,877) were discretionary cereal-based mixed meals, the largest of the 38 categories. Discretionary menu items were more likely to be categorized as most popular (OR: 2.0, 95% CI 1.7−2.2), accompanied by a photo (OR: 1.7, 95% CI 1.6−1.8), and offered as a value bundle (OR: 4.6, 95% CI 3.2−6.8). Two of the three discretionary mixed meal categories were significantly less expensive than their healthier counterparts (p < 0.001). The overwhelming availability and promotion of discretionary choices offered by restaurants and takeaway outlets on OFD platforms have implications for public health policy. Further research to explore direct associations between nutritional quality and consumers’ dietary choices is required.

Context-dependent choice and evaluation in real-world consumer behavior

A body of work spanning neuroscience, economics, and psychology indicates that decision-making is context-dependent, which means that the value of an option depends not only on the option in question, but also on the other options in the choice set-or the 'context'. While context effects have been observed primarily in small-scale laboratory studies with tightly constrained, artificially constructed choice sets, it remains to be determined whether these context effects take hold in real-world choice problems, where choice sets are large and decisions driven by rich histories of direct experience. Here, we investigate whether valuations are context-dependent in real-world choice by analyzing a massive restaurant rating dataset as well as two independent replication datasets which provide complementary operationalizations of restaurant choice. We find that users make fewer ratings-maximizing choices in choice sets with higher-rated options-a hallmark of context-dependent choice-and that post-choice restaurant ratings also varied systematically with the ratings of unchosen restaurants. Furthermore, in a follow-up laboratory experiment using hypothetical choice sets matched to the real-world data, we find further support for the idea that subjective valuations of restaurants are scaled in accordance with the choice context, providing corroborating evidence for a general mechanistic-level account of these effects. Taken together, our results provide a potent demonstration of context-dependent choice in real-world choice settings, manifesting both in decisions and subjective valuation of options.

Superstitious learning of abstract order from random reinforcement

Humans and other animals often infer spurious associations among unrelated events. However, such superstitious learning is usually accounted for by conditioned associations, raising the question of whether an animal could develop more complex cognitive structures independent of reinforcement. Here, we tasked monkeys with discovering the serial order of two pictorial sets: a "learnable" set in which the stimuli were implicitly ordered and monkeys were rewarded for choosing the higher-rank stimulus and an "unlearnable" set in which stimuli were unordered and feedback was random regardless of the choice. We replicated prior results that monkeys reliably learned the implicit order of the learnable set. Surprisingly, the monkeys behaved as though some ordering also existed in the unlearnable set, showing consistent choice preference that transferred to novel untrained pairs in this set, even under a preference-discouraging reward schedule that gave rewards more frequently to the stimulus that was selected less often. In simulations, a model-free reinforcement learning algorithm (Q-learning) displayed a degree of consistent ordering among the unlearnable set but, unlike the monkeys, failed to do so under the preference-discouraging reward schedule. Our results suggest that monkeys infer abstract structures from objectively random events using heuristics that extend beyond stimulus-outcome conditional learning to more cognitive model-based learning mechanisms.

Drafting strategies in fantasy football: A study of competitive sequential human decision making

Drafting is a competitive task in which a set of decision makers choose from a set of resources sequentially, with each resource becoming unavailable once selected. How people make these choices raises basic questions about human decision making, including people’s sensitivity to the statistical regularities of the resource environment, their ability to reason about the behavior of their competitors, and their ability to execute and adapt sophisticated strategies in dynamic situations involving uncertainty. Sports provides one real-world example of drafting behavior, in which a set of teams draft players from an available pool in a well-regulated way. Fantasy sport competitions provide potentially large data sets of drafting behavior. We study fantasy football drafting behavior from the 2017 National Football League (NFL) season based on 1350 leagues hosted by the http://sleeper.app platform. We find people are sensitive to some important environmental regularities in the order in which they draft players, but also present evidence that they use a more narrow range of strategies than is likely optimal in terms of team composition. We find little to no evidence for the use of the complicated but well-documented strategy known as handcuffing, and no evidence of irrational influence from individual-level biases for different NFL teams. We do, however, identify a set of circumstances for which there is clear evidence that people’s choices are strongly influenced by the immediately preceding choice made by a competitor.

Resting functional connectivity of the left inferior frontal gyrus with the dorsomedial prefrontal cortex and temporoparietal junction reflects the social network size for active interactions

The size of an individual active social network is a key parameter of human social behavior and is correlated with subjective well-being. However, it remains unknown how the social network size of active interactions is represented in the brain. Here, we examined whether resting-state functional magnetic resonance imaging (fMRI) connectivity is associated with the social network size of active interactions using behavioral data of a large sample (N = 222) on Twitter. Region of interest (ROI)-to-ROI analysis, graph theory analysis, seed-based analysis, and decoding analysis together provided compelling evidence that people who have a large social network size of active interactions, as measured by "reply," show higher fMRI connectivity of the left inferior frontal gyrus with the dorsomedial prefrontal cortex and temporoparietal junction, which represents the core of the theory of mind network. These results demonstrated that people who have a large social network size of active interactions maintain activity of the identified functional connectivity in daily life, possibly providing a mechanism for efficient information transmission between the brain networks related to language and theory-of-mind.

Time pressure changes how people explore and respond to uncertainty

How does time pressure influence exploration and decision-making? We investigated this question with several four-armed bandit tasks manipulating (within subjects) expected reward, uncertainty, and time pressure (limited vs. unlimited). With limited time, people have less opportunity to perform costly computations, thus shifting the cost-benefit balance of different exploration strategies. Through behavioral, reinforcement learning (RL), reaction time (RT), and evidence accumulation analyses, we show that time pressure changes how people explore and respond to uncertainty. Specifically, participants reduced their uncertainty-directed exploration under time pressure, were less value-directed, and repeated choices more often. Since our analyses relate uncertainty to slower responses and dampened evidence accumulation (i.e., drift rates), this demonstrates a resource-rational shift towards simpler, lower-cost strategies under time pressure. These results shed light on how people adapt their exploration and decision-making strategies to externally imposed cognitive constraints.

Advances in modeling learning and decision-making in neuroscience

An organism's survival depends on its ability to learn about its environment and to make adaptive decisions in the service of achieving the best possible outcomes in that environment. To study the neural circuits that support these functions, researchers have increasingly relied on models that formalize the computations required to carry them out. Here, we review the recent history of computational modeling of learning and decision-making, and how these models have been used to advance understanding of prefrontal cortex function. We discuss how such models have advanced from their origins in basic algorithms of updating and action selection to increasingly account for complexities in the cognitive processes required for learning and decision-making, and the representations over which they operate. We further discuss how a deeper understanding of the real-world complexities in these computations has shed light on the fundamental constraints on optimal behavior, and on the complex interactions between corticostriatal pathways to determine such behavior. The continuing and rapid development of these models holds great promise for understanding the mechanisms by which animals adapt to their environments, and what leads to maladaptive forms of learning and decision-making within clinical populations.

From exploration to exploitation: a shifting mental mode in late life development

Changes in cognition, affect, and brain function combine to promote a shift in the nature of mentation in older adulthood, favoring exploitation of prior knowledge over exploratory search as the starting point for thought and action. Age-related exploitation biases result from the accumulation of prior knowledge, reduced cognitive control, and a shift toward affective goals. These are accompanied by changes in cortical networks, as well as attention and reward circuits. By incorporating these factors into a unified account, the exploration-to-exploitation shift offers an integrative model of cognitive, affective, and brain aging. Here, we review evidence for this model, identify determinants and consequences, and survey the challenges and opportunities posed by an exploitation-biased mental mode in later life.

Humans monitor learning progress in curiosity-driven exploration

Curiosity-driven learning is foundational to human cognition. By enabling humans to autonomously decide when and what to learn, curiosity has been argued to be crucial for self-organizing temporally extended learning curricula. However, the mechanisms driving people to set intrinsic goals, when they are free to explore multiple learning activities, are still poorly understood. Computational theories propose different heuristics, including competence measures (e.g., percent correct) and learning progress, that could be used as intrinsic utility functions to efficiently organize exploration. Such intrinsic utilities constitute computationally cheap but smart heuristics to prevent people from laboring in vain on unlearnable activities, while still motivating them to self-challenge on difficult learnable activities. Here, we provide empirical evidence for these ideas by means of a free-choice experimental paradigm and computational modeling. We show that while humans rely on competence information to avoid easy tasks, models that include a learning-progress component provide the best fit to task selection data. These results bridge the research in artificial and biological curiosity, reveal strategies that are used by humans but have not been considered in computational research, and introduce tools for probing how humans become intrinsically motivated to learn and acquire interests and skills on extended time scales.

Communicating Compositional Patterns

How do people perceive and communicate structure? We investigate this question by letting participants play a communication game, where one player describes a pattern, and another player redraws it based on the description alone. We use this paradigm to compare two models of pattern description, one compositional (complex structures built out of simpler ones) and one noncompositional. We find that compositional patterns are communicated more effectively than noncompositional patterns, that a compositional model of pattern description predicts which patterns are harder to describe, and that this model can be used to evaluate participants’ drawings, producing humanlike quality ratings. Our results suggest that natural language can tap into a compositionally structured pattern description language.

"Is it Reasonable to Study Decision-Making Quantitatively?"

Scientists studying decision-making often provide a set of choices, each specified with values or distributions of values, and probabilities or distributions of probabilities. For example, "Would you prefer $100 with probability 1.0 or $1 with probability .9 and $1,000 with probability 0.1?" Other decision research examines choices made in the absence of most quantitative information; for example, "Would you prefer a Ford now or a Porsche a year from now?," "Which food would you prefer," but models the findings with precise quantitative assumptions. Yet other research does neither; for example, modeling verbally stated choices with verbally stated heuristics. This article asks about the relevance of the first two research approaches for much of the decision-making made in life. The use of quantitative research and modeling is unsurprising, given that this approach underlies most of science. In life, values and probabilities are almost always partly or wholly vague and qualitative rather than quantitative. For example, when deciding which house to buy, there are relevant features such as size, color, neighborhood schools, construction materials, attractiveness, and many more, but the decision-maker finds it difficult and of little use to assign these precise values or weights. Nonetheless, humans have evolved to make decisions in such vaguely specified settings. I provide an example showing how a very high degree of uncertainty can defeat the application of quantitative decision-making, but such a demonstration is not critical if quantitative research and modeling produce a good understanding of and a good approximation to decision-making in the natural environment. This perspective addresses these issues.

Balancing exploration and exploitation with information and randomization

Explore-exploit decisions require us to trade off the benefits of exploring unknown options to learn more about them, with exploiting known options, for immediate reward. Such decisions are ubiquitous in nature, but from a computational perspective, they are notoriously hard. There is therefore much interest in how humans and animals make these decisions and recently there has been an explosion of research in this area. Here we provide a biased and incomplete snapshot of this field focusing on the major finding that many organisms use two distinct strategies to solve the explore-exploit dilemma: a bias for information ('directed exploration') and the randomization of choice ('random exploration'). We review evidence for the existence of these strategies, their computational properties, their neural implementations, as well as how directed and random exploration vary over the lifespan. We conclude by highlighting open questions in this field that are ripe to both explore and exploit.

Hunger for Home Delivery: Cross-Sectional Analysis of the Nutritional Quality of Complete Menus on an Online Food Delivery Platform in Australia

Online food delivery (OFD) platforms have changed how consumers purchase food prepared outside of home by capitalising on convenience and smartphone technology. Independent food outlets encompass a substantial proportion of partnering outlets, but their offerings’ nutritional quality is understudied. Little is also known as to how OFD platforms influence consumer choice. This study evaluated the nutritional quality and marketing attributes of offerings from independent takeaway outlets available on Sydney’s market-leading OFD platform (UberEats^®). Complete menus and marketing attributes from 202 popular outlets were collected using web scraping. All 13841 menu items were classified into 38 food and beverage categories based on the Australian Dietary Guidelines. Of complete menus, 80.5% (11,139/13,841) were discretionary and 42.3% (5849/13,841) were discretionary cereal-based mixed meals, the largest of the 38 categories. Discretionary menu items were more likely to be categorised as most popular (OR: 2.5, 95% CI 1.9–3.2), accompanied by an image (OR: 1.3, 95% CI 1.2–1.5) and offered as a value bundle (OR: 6.5, 95% CI 4.8–8.9). Two of the three discretionary food categories were more expensive than their healthier Five Food Group counterparts (p < 0.02). The ubiquity of discretionary choices offered by independent takeaways and the marketing attributes employed by OFD platforms has implications for public health policy. Further research on the contribution of discretionary choices and marketing attributes to nutritional intakes is warranted.

A computational reward learning account of social media engagement

Social media has become a modern arena for human life, with billions of daily users worldwide. The intense popularity of social media is often attributed to a psychological need for social rewards (likes), portraying the online world as a Skinner Box for the modern human. Yet despite such portrayals, empirical evidence for social media engagement as reward-based behavior remains scant. Here, we apply a computational approach to directly test whether reward learning mechanisms contribute to social media behavior. We analyze over one million posts from over 4000 individuals on multiple social media platforms, using computational models based on reinforcement learning theory. Our results consistently show that human behavior on social media conforms qualitatively and quantitatively to the principles of reward learning. Specifically, social media users spaced their posts to maximize the average rate of accrued social rewards, in a manner subject to both the effort cost of posting and the opportunity cost of inaction. Results further reveal meaningful individual difference profiles in social reward learning on social media. Finally, an online experiment (n = 176), mimicking key aspects of social media, verifies that social rewards causally influence behavior as posited by our computational account. Together, these findings support a reward learning account of social media engagement and offer new insights into this emergent mode of modern human behavior.

Emerging Applications of Machine Learning in Food Safety

Food safety continues to threaten public health. Machine learning holds potential in leveraging large, emerging data sets to improve the safety of the food supply and mitigate the impact of food safety incidents. Foodborne pathogen genomes and novel data streams, including text, transactional, and trade data, have seen emerging applications enabled by a machine learning approach, such as prediction of antibiotic resistance, source attribution of pathogens, and foodborne outbreak detection and risk assessment. In this article, we provide a gentle introduction to machine learning in the context of food safety and an overview of recent developments and applications. With many of these applications still in their nascence, general and domain-specific pitfalls and challenges associated with machine learning have begun to be recognized and addressed, which are critical to prospective use and future deployment of large data sets and their associated machine learning models for food safety applications.

Dogmatism manifests in lowered information search under uncertainty

When knowledge is scarce, it is adaptive to seek further information to resolve uncertainty and obtain a more accurate worldview. Biases in such information-seeking behavior can contribute to the maintenance of inaccurate views. Here, we investigate whether predispositions for uncertainty-guided information seeking relate to individual differences in dogmatism, a phenomenon linked to entrenched beliefs in political, scientific, and religious discourse. We addressed this question in a perceptual decision-making task, allowing us to rule out motivational factors and isolate the role of uncertainty. In two independent general population samples (n = 370 and n = 364), we show that more dogmatic participants are less likely to seek out new information to refine an initial perceptual decision, leading to a reduction in overall belief accuracy despite similar initial decision performance. Trial-by-trial modeling revealed that dogmatic participants placed less reliance on internal signals of uncertainty (confidence) to guide information search, rendering them less likely to seek additional information to update beliefs derived from weak or uncertain initial evidence. Together, our results highlight a cognitive mechanism that may contribute to the formation of dogmatic worldviews.

Impact of incentive and selection strength on green technology innovation in Moran process

Methods of previous researches on green technology innovation will have difficulty in finite population. One solution is the use of stochastic evolutionary game dynamic-Moran process. In this paper we study stochastic dynamic games about green technology innovation with a two-stage free riding problem. Results illustrate the incentive and selection strength play positive roles in promoting participant to be more useful to society, but with threshold effect: too slighted strength makes no effect due to the randomness of the evolution process in finite population. Two-stage free riding problem can be solved with the use of inequality incentives, however, higher inequality can make policy achieves faster but more unstable, so there would be an optimal range. In this paper we provided the key variables of green technology innovation incentive and principles for the environmental regulation policy making. Also reminded that it’s difficult to formulate policies reasonably and make them achieve the expected results.

How decisions and the desire for coherency shape subjective preferences over time

Recent findings suggest a bidirectional relationship between preferences and choices such that what is chosen can become preferred. Yet, it is still commonly held that preferences for individual items are maintained, such as caching a separate value estimate for each experienced option. Instead, we propose that all possible choice options and preferences are represented in a shared, continuous, multidimensional space that supports generalization. Decision making is cast as a learning process that seeks to align choices and preferences to maintain coherency. We formalized an error-driven learning model that updates preferences to align with past choices, which makes repeating those and related choices more likely in the future. The model correctly predicts that making a free choice increases preferences along related attributes. For example, after choosing a political candidate based on trivial information (e.g., they like cats), voters' views on abortion, immigration, and trade subsequently shifted to match their chosen candidate.

Scaling up psychology via Scientific Regret Minimization

Do large datasets provide value to psychologists? Without a systematic methodology for working with such datasets, there is a valid concern that analyses will produce noise artifacts rather than true effects. In this paper, we offer a way to enable researchers to systematically build models and identify novel phenomena in large datasets. One traditional approach is to analyze the residuals of models-the biggest errors they make in predicting the data-to discover what might be missing from those models. However, once a dataset is sufficiently large, machine learning algorithms approximate the true underlying function better than the data, suggesting, instead, that the predictions of these data-driven models should be used to guide model building. We call this approach "Scientific Regret Minimization" (SRM), as it focuses on minimizing errors for cases that we know should have been predictable. We apply this exploratory method on a subset of the Moral Machine dataset, a public collection of roughly 40 million moral decisions. Using SRM, we find that incorporating a set of deontological principles that capture dimensions along which groups of agents can vary (e.g., sex and age) improves a computational model of human moral judgment. Furthermore, we are able to identify and independently validate three interesting moral phenomena: criminal dehumanization, age of responsibility, and asymmetric notions of responsibility.

The Role of Supportive Food Environments to Enable Healthier Choices When Eating Meals Prepared Outside the Home: Findings from Focus Groups of 18 to 30-Year-Olds

Young adults are the highest consumers of food prepared outside home, which has been linked to weight gain. The aim of this qualitative research was to gather opinions from young adults about what influences their food choice when eating out and if they so desired, what might enable healthy choices. Thirty-one adults aged 18 to 30 years participated in four focus groups (females = 24). Predetermined questions were used to guide discussions which were audiotaped then transcribed. The content was organized into themes and sub-themes using NVivo software. Two broad groupings arose—personal behaviors and changes to physical and social food environments. For many, eating out was viewed as a special occasion so that healthy food was not a priority and despite understanding health consequences of poor diets this was not an immediate concern. Price discounts and menu-labelling were suggested and trust in credible organizations and peers’ endorsement of meals and venues expressed. The role of smartphones in the modern food environment emerged as a tool to enable immediate access to many restaurants to order food and access reviews and social media as a source of persuasive food imagery. Current menu-labelling initiatives should continue, food pricing be explored and influence of smartphones on diet further researched. However, these findings may be limited by the high proportion of women and higher socioeconomic status and urban residence of many participants.