51
|
Liveringhouse C, Robinson T, Garcia G, Peters J, Kim S, Latifi K. Dosimetric Comparison of Volumetric Modulated Arc Therapy with Tomotherapy Based Total Body Irradiation for Patients Undergoing Conditioning Prior to Hematopoietic Stem Cell Transplantation for Acute Lymphocytic Leukemia. Int J Radiat Oncol Biol Phys 2021. [DOI: 10.1016/j.ijrobp.2021.07.950] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/20/2022]
|
52
|
Wolanin J, Michel L, Tabacchioni D, Zanotti JM, Peters J, Imaz I, Coasne B, Plazanet M, Picard C. Heterogeneous Microscopic Dynamics of Intruded Water in a Superhydrophobic Nanoconfinement: Neutron Scattering and Molecular Modeling. J Phys Chem B 2021; 125:10392-10399. [PMID: 34492185 DOI: 10.1021/acs.jpcb.1c06791] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/28/2022]
Abstract
With their strong confining porosity and versatile surface chemistry, zeolitic imidazolate frameworks-including the prototypical ZIF-8-display exceptional properties for various applications. In particular, the forced intrusion of water at high pressure (∼25 MPa) into ZIF-8 nanopores is of interest for energy storage. Such a system reveals also ideal to study experimentally water dynamics and thermodynamics in an ultrahydrophobic confinement. Here, we report on neutron scattering experiments to probe the molecular dynamics of water within ZIF-8 nanopores under high pressure up to 38 MPa. In addition to an overall confinement-induced slowing down, we provide evidence for strong dynamical heterogeneities with different underlying molecular dynamics. Using complementary molecular simulations, these heterogeneities are found to correspond to different microscopic mechanisms inherent to vicinal molecules located in strongly adsorbing sites (ligands) and other molecules nanoconfined in the cavity center. These findings unveil a complex microscopic dynamics, which results from the combination of surface residence times and exchanges between the cavity surface and center.
Collapse
|
53
|
Peters J, Köhler HC, Oltmanns K, Besselmann M, Zwaan M, Gutcke A, Rüttermann M. Erratum: Heterotope Ossifikationen nach Langzeitbeatmung bei Covid-19 Erkrankung. REHABILITATION 2021; 60:e1. [PMID: 34488235 DOI: 10.1055/a-1612-8527] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/20/2022]
|
54
|
Peters J, Köhler HC, Oltmanns K, Besselmann M, Zwaan M, Gutcke A, Rüttermann M. Heterotope Ossifikationen nach Langzeitbeatmung bei Covid-19 Erkrankung. REHABILITATION 2021; 60:231-234. [PMID: 34428803 DOI: 10.1055/a-1339-5365] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/20/2022]
|
55
|
Zaccaria J, Lent D, Peters J. Einseitige Mikrophthalmie bei einem 4 Monate alten Säugling. Monatsschr Kinderheilkd 2021. [DOI: 10.1007/s00112-020-01101-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/28/2022]
|
56
|
Tanneberg D, Ploeger K, Rueckert E, Peters J. SKID RAW: Skill Discovery From Raw Trajectories. IEEE Robot Autom Lett 2021. [DOI: 10.1109/lra.2021.3068891] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/08/2022]
|
57
|
Peters J, Zwaan M, Wrobel M, Gutcke A, Rüttermann M. [Painful swelling of the hand after contrast agent CT of the abdomen]. Radiologe 2021; 61:60-63. [PMID: 33184679 DOI: 10.1007/s00117-020-00773-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/23/2022]
|
58
|
Schüller CB, Wagner BJ, Schüller T, Baldermann JC, Huys D, Kerner auch Koerner J, Niessen E, Münchau A, Brandt V, Peters J, Kuhn J. Temporal discounting in adolescents and adults with Tourette syndrome. PLoS One 2021; 16:e0253620. [PMID: 34143854 PMCID: PMC8213148 DOI: 10.1371/journal.pone.0253620] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/01/2021] [Accepted: 06/08/2021] [Indexed: 12/18/2022] Open
Abstract
Tourette syndrome is a neurodevelopmental disorder associated with hyperactivity in dopaminergic networks. Dopaminergic hyperactivity in the basal ganglia has previously been linked to increased sensitivity to positive reinforcement and increases in choice impulsivity. In this study, we examine whether this extends to changes in temporal discounting, where impulsivity is operationalized as an increased preference for smaller-but-sooner over larger-but-later rewards. We assessed intertemporal choice in two studies including nineteen adolescents (age: mean[sd] = 14.21[±2.37], 13 male subjects) and twenty-five adult patients (age: mean[sd] = 29.88 [±9.03]; 19 male subjects) with Tourette syndrome and healthy age- and education matched controls. Computational modeling using exponential and hyperbolic discounting models via hierarchical Bayesian parameter estimation revealed reduced temporal discounting in adolescent patients, and no evidence for differences in adult patients. Results are discussed with respect to neural models of temporal discounting, dopaminergic alterations in Tourette syndrome and the developmental trajectory of temporal discounting. Specifically, adolescents might show attenuated discounting due to improved inhibitory functions that also affect choice impulsivity and/or the developmental trajectory of executive control functions. Future studies would benefit from a longitudinal approach to further elucidate the developmental trajectory of these effects.
Collapse
|
59
|
Kollegger G, Wiemeyer J, Ewerton M, Peters J. Perception and prediction of the putting distance of robot putting movements under different visual/viewing conditions. PLoS One 2021; 16:e0249518. [PMID: 33891623 PMCID: PMC8064581 DOI: 10.1371/journal.pone.0249518] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/28/2020] [Accepted: 03/22/2021] [Indexed: 11/19/2022] Open
Abstract
The purpose of this paper is to examine, whether and under which conditions humans are able to predict the putting distance of a robotic device. Based on the “flash-lag effect” (FLE) it was expected that the prediction errors increase with increasing putting velocity. Furthermore, we hypothesized that the predictions are more accurate and more confident if human observers operate under full vision (F-RCHB) compared to either temporal occlusion (I-RCHB) or spatial occlusion (invisible ball, F-RHC, or club, F-B). In two experiments, 48 video sequences of putt movements performed by a BioRob robot arm were presented to thirty-nine students (age: 24.49±3.20 years). In the experiments, video sequences included six putting distances (1.5, 2.0, 2.5, 3.0, 3.5, and 4.0 m; experiment 1) under full versus incomplete vision (F-RCHB versus I-RCHB) and three putting distances (2. 0, 3.0, and 4.0 m; experiment 2) under the four visual conditions (F-RCHB, I-RCHB, F-RCH, and F-B). After the presentation of each video sequence, the participants estimated the putting distance on a scale from 0 to 6 m and provided their confidence of prediction on a 5-point scale. Both experiments show comparable results for the respective dependent variables (error and confidence measures). The participants consistently overestimated the putting distance under the full vision conditions; however, the experiments did not show a pattern that was consistent with the FLE. Under the temporal occlusion condition, a prediction was not possible; rather a random estimation pattern was found around the centre of the prediction scale (3 m). Spatial occlusion did not affect errors and confidence of prediction. The experiments indicate that temporal constraints seem to be more critical than spatial constraints. The FLE may not apply to distance prediction compared to location estimation.
Collapse
|
60
|
Sun F, Peters J, Thullner M, Cirpka OA, Elsner M. Magnitude of Diffusion- and Transverse Dispersion-Induced Isotope Fractionation of Organic Compounds in Aqueous Systems. ENVIRONMENTAL SCIENCE & TECHNOLOGY 2021; 55:4772-4782. [PMID: 33729766 PMCID: PMC8154364 DOI: 10.1021/acs.est.0c06741] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/12/2023]
Abstract
Determining whether aqueous diffusion and dispersion lead to significant isotope fractionation is important for interpreting the isotope ratios of organic contaminants in groundwater. We performed diffusion experiments with modified Stokes diaphragm cells and transverse-dispersion experiments in quasi-two-dimensional flow-through sediment tank systems to explore isotope fractionation for benzene, toluene, ethylbenzene, 2,6-dichlorobenzamide, and metolachlor at natural isotopic abundance. We observed very small to negligible diffusion- and transverse-dispersion-induced isotope enrichment factors (ε < -0.4 ‰), with changes in carbon and nitrogen isotope values within ±0.5‰ and ±1‰, respectively. Isotope effects of diffusion did not show a clear correlation with isotopologue mass with calculated power-law exponents β close to zero (0.007 < β < 0.1). In comparison to ions, noble gases, and labeled compounds, three aspects stand out. (i) If a mass dependence is derived from collision theory, then isotopologue masses of polyatomic molecules would be affected by isotopes of multiple elements resulting in very small expected effects. (ii) However, collisions do not necessarily lead to translational movement but can excite molecular vibrations or rotations minimizing the mass dependence. (iii) Solute-solvent interactions like H-bonds can further minimize the effect of collisions. Modeling scenarios showed that an inadequate model choice, or erroneous choice of β, can greatly overestimate the isotope fractionation by diffusion and, consequently, transverse dispersion. In contrast, available data for chlorinated solvent and gasoline contaminants at natural isotopic abundance suggest that in field scenarios, a potential additional uncertainty from aqueous diffusion or dispersion would add to current instrumental uncertainties on carbon or nitrogen isotope values (±1‰) with an additional ±1‰ at most.
Collapse
|
61
|
Muratore F, Gienger M, Peters J. Assessing Transferability From Simulation to Reality for Reinforcement Learning. IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE 2021; 43:1172-1183. [PMID: 31722475 DOI: 10.1109/tpami.2019.2952353] [Citation(s) in RCA: 7] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/10/2023]
Abstract
Learning robot control policies from physics simulations is of great interest to the robotics community as it may render the learning process faster, cheaper, and safer by alleviating the need for expensive real-world experiments. However, the direct transfer of learned behavior from simulation to reality is a major challenge. Optimizing a policy on a slightly faulty simulator can easily lead to the maximization of the 'Simulation Optimization Bias' (SOB). In this case, the optimizer exploits modeling errors of the simulator such that the resulting behavior can potentially damage the robot. We tackle this challenge by applying domain randomization, i.e., randomizing the parameters of the physics simulations during learning. We propose an algorithm called Simulation-based Policy Optimization with Transferability Assessment (SPOTA) which uses an estimator of the SOB to formulate a stopping criterion for training. The introduced estimator quantifies the over-fitting to the set of domains experienced while training. Our experimental results on two different second order nonlinear systems show that the new simulation-based policy search algorithm is able to learn a control policy exclusively from a randomized simulator, which can be applied directly to real systems without any additional training.
Collapse
|
62
|
Muratore F, Eilers C, Gienger M, Peters J. Data-Efficient Domain Randomization With Bayesian Optimization. IEEE Robot Autom Lett 2021. [DOI: 10.1109/lra.2021.3052391] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/09/2022]
|
63
|
Wiehler A, Chakroun K, Peters J. Attenuated Directed Exploration during Reinforcement Learning in Gambling Disorder. J Neurosci 2021; 41:2512-2522. [PMID: 33531415 PMCID: PMC7984586 DOI: 10.1523/jneurosci.1607-20.2021] [Citation(s) in RCA: 19] [Impact Index Per Article: 6.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/26/2020] [Revised: 01/18/2021] [Accepted: 01/22/2021] [Indexed: 12/30/2022] Open
Abstract
Gambling disorder (GD) is a behavioral addiction associated with impairments in value-based decision-making and behavioral flexibility and might be linked to changes in the dopamine system. Maximizing long-term rewards requires a flexible trade-off between the exploitation of known options and the exploration of novel options for information gain. This exploration-exploitation trade-off is thought to depend on dopamine neurotransmission. We hypothesized that human gamblers would show a reduction in directed (uncertainty-based) exploration, accompanied by changes in brain activity in a fronto-parietal exploration-related network. Twenty-three frequent, non-treatment seeking gamblers and twenty-three healthy matched controls (all male) performed a four-armed bandit task during functional magnetic resonance imaging (fMRI). Computational modeling using hierarchical Bayesian parameter estimation revealed signatures of directed exploration, random exploration, and perseveration in both groups. Gamblers showed a reduction in directed exploration, whereas random exploration and perseveration were similar between groups. Neuroimaging revealed no evidence for group differences in neural representations of basic task variables (expected value, prediction errors). Our hypothesis of reduced frontal pole (FP) recruitment in gamblers was not supported. Exploratory analyses showed that during directed exploration, gamblers showed reduced parietal cortex and substantia-nigra/ventral-tegmental-area activity. Cross-validated classification analyses revealed that connectivity in an exploration-related network was predictive of group status, suggesting that connectivity patterns might be more predictive of problem gambling than univariate effects. Findings reveal specific reductions of strategic exploration in gamblers that might be linked to altered processing in a fronto-parietal network and/or changes in dopamine neurotransmission implicated in GD.SIGNIFICANCE STATEMENT Wiehler et al. (2021) report that gamblers rely less on the strategic exploration of unknown, but potentially better rewards during reward learning. This is reflected in a related network of brain activity. Parameters of this network can be used to predict the presence of problem gambling behavior in participants.
Collapse
|
64
|
O'Connor DA, Janet R, Guigon V, Belle A, Vincent BT, Bromberg U, Peters J, Corgnet B, Dreher JC. Rewards that are near increase impulsive action. iScience 2021; 24:102292. [PMID: 33889815 PMCID: PMC8050375 DOI: 10.1016/j.isci.2021.102292] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/15/2020] [Revised: 01/13/2021] [Accepted: 03/05/2021] [Indexed: 01/26/2023] Open
Abstract
In modern society, the natural drive to behave impulsively in order to obtain rewards must often be curbed. A continued failure to do so is associated with a range of outcomes including drug abuse, pathological gambling, and obesity. Here, we used virtual reality technology to investigate whether spatial proximity to rewards has the power to exacerbate the drive to behave impulsively toward them. We embedded two behavioral tasks measuring distinct forms of impulsive behavior, impulsive action, and impulsive choice, within an environment rendered in virtual reality. Participants responded to three-dimensional cues representing food rewards located in either near or far space. Bayesian analyses revealed that participants were significantly less able to stop motor actions when rewarding cues were near compared with when they were far. Since factors normally associated with proximity were controlled for, these results suggest that proximity plays a distinctive role in driving impulsive actions for rewards.
Collapse
|
65
|
Koert D, Kircher M, Salikutluk V, D'Eramo C, Peters J. Multi-Channel Interactive Reinforcement Learning for Sequential Tasks. Front Robot AI 2021; 7:97. [PMID: 33501264 PMCID: PMC7805623 DOI: 10.3389/frobt.2020.00097] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/05/2020] [Accepted: 06/15/2020] [Indexed: 11/13/2022] Open
Abstract
The ability to learn new tasks by sequencing already known skills is an important requirement for future robots. Reinforcement learning is a powerful tool for this as it allows for a robot to learn and improve on how to combine skills for sequential tasks. However, in real robotic applications, the cost of sample collection and exploration prevent the application of reinforcement learning for a variety of tasks. To overcome these limitations, human input during reinforcement can be beneficial to speed up learning, guide the exploration and prevent the choice of disastrous actions. Nevertheless, there is a lack of experimental evaluations of multi-channel interactive reinforcement learning systems solving robotic tasks with input from inexperienced human users, in particular for cases where human input might be partially wrong. Therefore, in this paper, we present an approach that incorporates multiple human input channels for interactive reinforcement learning in a unified framework and evaluate it on two robotic tasks with 20 inexperienced human subjects. To enable the robot to also handle potentially incorrect human input we incorporate a novel concept for self-confidence, which allows the robot to question human input after an initial learning phase. The second robotic task is specifically designed to investigate if this self-confidence can enable the robot to achieve learning progress even if the human input is partially incorrect. Further, we evaluate how humans react to suggestions of the robot, once the robot notices human input might be wrong. Our experimental evaluations show that our approach can successfully incorporate human input to accelerate the learning process in both robotic tasks even if it is partially wrong. However, not all humans were willing to accept the robot's suggestions or its questioning of their input, particularly if they do not understand the learning process and the reasons behind the robot's suggestions. We believe that the findings from this experimental evaluation can be beneficial for the future design of algorithms and interfaces of interactive reinforcement learning systems used by inexperienced users.
Collapse
|
66
|
Veiga F, Akrour R, Peters J. Hierarchical Tactile-Based Control Decomposition of Dexterous In-Hand Manipulation Tasks. Front Robot AI 2021; 7:521448. [PMID: 33501302 PMCID: PMC7805629 DOI: 10.3389/frobt.2020.521448] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/19/2019] [Accepted: 10/15/2020] [Indexed: 11/13/2022] Open
Abstract
In-hand manipulation and grasp adjustment with dexterous robotic hands is a complex problem that not only requires highly coordinated finger movements but also deals with interaction variability. The control problem becomes even more complex when introducing tactile information into the feedback loop. Traditional approaches do not consider tactile feedback and attempt to solve the problem either by relying on complex models that are not always readily available or by constraining the problem in order to make it more tractable. In this paper, we propose a hierarchical control approach where a higher level policy is learned through reinforcement learning, while low level controllers ensure grip stability throughout the manipulation action. The low level controllers are independent grip stabilization controllers based on tactile feedback. The independent controllers allow reinforcement learning approaches to explore the manipulation tasks state-action space in a more structured manner. We show that this structure allows learning the unconstrained task with RL methods that cannot learn it in a non-hierarchical setting. The low level controllers also provide an abstraction to the tactile sensors input, allowing transfer to real robot platforms. We show preliminary results of the transfer of policies trained in simulation to the real robot hand.
Collapse
|
67
|
Rueckert E, Čamernik J, Peters J, Babič J. Author Correction: Probabilistic Movement Models Show that Postural Control Precedes and Predicts Volitional Motor Control. Sci Rep 2020; 10:6694. [PMID: 32300170 PMCID: PMC7162845 DOI: 10.1038/s41598-020-63129-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/17/2022] Open
|
68
|
Tanneberg D, Rueckert E, Peters J. Evolutionary training and abstraction yields algorithmic generalization of neural computers. NAT MACH INTELL 2020. [DOI: 10.1038/s42256-020-00255-1] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/09/2022]
|
69
|
Lauri M, Pajarinen J, Peters J, Frintrop S. Multi-Sensor Next-Best-View Planning as Matroid-Constrained Submodular Maximization. IEEE Robot Autom Lett 2020. [DOI: 10.1109/lra.2020.3007445] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/08/2022]
|
70
|
Pajarinen J, Arenz O, Peters J, Neumann G. Probabilistic Approach to Physical Object Disentangling. IEEE Robot Autom Lett 2020. [DOI: 10.1109/lra.2020.3006789] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/05/2022]
|
71
|
Ewerton M, Arenz O, Peters J. Assisted teleoperation in changing environments with a mixture of virtual guides. Adv Robot 2020. [DOI: 10.1080/01691864.2020.1785326] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/23/2022]
|
72
|
Peters J, Vega T, Weinstein D, Mitchell J, Kayser A. Dopamine and Risky Decision-Making in Gambling Disorder. eNeuro 2020; 7:ENEURO.0461-19.2020. [PMID: 32341121 PMCID: PMC7294471 DOI: 10.1523/eneuro.0461-19.2020] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/05/2019] [Revised: 04/15/2020] [Accepted: 04/17/2020] [Indexed: 12/03/2022] Open
Abstract
Gambling disorder is a behavioral addiction associated with impairments in value-based decision-making and cognitive control. These functions are thought to be regulated by dopamine within fronto-striatal circuits, but the role of altered dopamine neurotransmission in the etiology of gambling disorder remains controversial. Preliminary evidence suggests that increasing frontal dopamine tone might improve cognitive functioning in gambling disorder. We therefore examined whether increasing frontal dopamine tone via a single dose of the catechol-O-methyltransferase (COMT) inhibitor tolcapone would reduce risky choice in human gamblers (n = 14) in a randomized double-blind placebo-controlled crossover study. Data were analyzed using hierarchical Bayesian parameter estimation and a combined risky choice drift diffusion model (DDM). Model comparison revealed a nonlinear mapping from value differences to trial-wise drift rates, confirming recent findings. An increase in risk-taking under tolcapone versus placebo was about five times more likely, given the data, than a decrease [Bayes factor (BF) = 0.2]. Examination of drug effects on diffusion model parameters revealed that an increase in the value dependency of the drift rate under tolcapone was about thirteen times more likely than a decrease (BF = 0.073). In contrast, a reduction in the maximum drift rate under tolcapone was about seven times more likely than an increase (BF = 7.51). Results add to previous work on COMT inhibitors in behavioral addictions and to mounting evidence for the applicability of diffusion models in value-based decision-making. Future work should focus on individual genetic, clinical and cognitive factors that might account for heterogeneity in the effects of COMT inhibition.
Collapse
|
73
|
Chakroun K, Mathar D, Wiehler A, Ganzer F, Peters J. Dopaminergic modulation of the exploration/exploitation trade-off in human decision-making. eLife 2020; 9:e51260. [PMID: 32484779 PMCID: PMC7266623 DOI: 10.7554/elife.51260] [Citation(s) in RCA: 45] [Impact Index Per Article: 11.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/21/2019] [Accepted: 05/01/2020] [Indexed: 01/15/2023] Open
Abstract
Involvement of dopamine in regulating exploration during decision-making has long been hypothesized, but direct causal evidence in humans is still lacking. Here, we use a combination of computational modeling, pharmacological intervention and functional magnetic resonance imaging to address this issue. Thirty-one healthy male participants performed a restless four-armed bandit task in a within-subjects design under three drug conditions: 150 mg of the dopamine precursor L-dopa, 2 mg of the D2 receptor antagonist haloperidol, and placebo. Choices were best explained by an extension of an established Bayesian learning model accounting for perseveration, directed exploration and random exploration. Modeling revealed attenuated directed exploration under L-dopa, while neural signatures of exploration, exploitation and prediction error were unaffected. Instead, L-dopa attenuated neural representations of overall uncertainty in insula and dorsal anterior cingulate cortex. Our results highlight the computational role of these regions in exploration and suggest that dopamine modulates how this circuit tracks accumulating uncertainty during decision-making.
Collapse
|
74
|
Peters J, D’Esposito M. The drift diffusion model as the choice rule in inter-temporal and risky choice: A case study in medial orbitofrontal cortex lesion patients and controls. PLoS Comput Biol 2020; 16:e1007615. [PMID: 32310962 PMCID: PMC7192518 DOI: 10.1371/journal.pcbi.1007615] [Citation(s) in RCA: 28] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/01/2019] [Revised: 04/30/2020] [Accepted: 12/19/2019] [Indexed: 01/20/2023] Open
Abstract
Sequential sampling models such as the drift diffusion model (DDM) have a long tradition in research on perceptual decision-making, but mounting evidence suggests that these models can account for response time (RT) distributions that arise during reinforcement learning and value-based decision-making. Building on this previous work, we implemented the DDM as the choice rule in inter-temporal choice (temporal discounting) and risky choice (probability discounting) using hierarchical Bayesian parameter estimation. We validated our approach in data from nine patients with focal lesions to the ventromedial prefrontal cortex / medial orbitofrontal cortex (vmPFC/mOFC) and nineteen age- and education-matched controls. Model comparison revealed that, for both tasks, the data were best accounted for by a variant of the drift diffusion model including a non-linear mapping from value-differences to trial-wise drift rates. Posterior predictive checks confirmed that this model provided a superior account of the relationship between value and RT. We then applied this modeling framework and 1) reproduced our previous results regarding temporal discounting in vmPFC/mOFC patients and 2) showed in a previously unpublished data set on risky choice that vmPFC/mOFC patients exhibit increased risk-taking relative to controls. Analyses of DDM parameters revealed that patients showed substantially increased non-decision times and reduced response caution during risky choice. In contrast, vmPFC/mOFC damage abolished neither scaling nor asymptote of the drift rate. Relatively intact value processing was also confirmed using DDM mixture models, which revealed that in both groups >98% of trials were better accounted for by a DDM with value modulation than by a null model without value modulation. Our results highlight that novel insights can be gained from applying sequential sampling models in studies of inter-temporal and risky decision-making in cognitive neuroscience.
Collapse
|
75
|
Lockel S, Peters J, van Vliet P. A Probabilistic Framework for Imitating Human Race Driver Behavior. IEEE Robot Autom Lett 2020. [DOI: 10.1109/lra.2020.2970620] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/05/2022]
|