Active afforestation of drained peatlands is not a viable option under the EU Nature Restoration Law

The EU Nature Restoration Law (NRL) is critical for the restoration of degraded ecosystems and active afforestation of degraded peatlands has been suggested as a restoration measure under the NRL. Here, we discuss the current state of scientific evidence on the climate mitigation effects of peatlands under forestry. Afforestation of drained peatlands without restoring their hydrology does not fully restore ecosystem functions. Evidence on long-term climate benefits is lacking and it is unclear whether CO2 sequestration of forest on drained peatland can offset the carbon loss from the peat over the long-term. While afforestation may offer short-term gains in certain cases, it compromises the sustainability of peatland carbon storage. Thus, active afforestation of drained peatlands is not a viable option for climate mitigation under the EU Nature Restoration Law and might even impede future rewetting/restoration efforts. Instead, restoring hydrological conditions through rewetting is crucial for effective peatland restoration.

On the Benefit of Optimal Transport for Curriculum Reinforcement Learning

Curriculum reinforcement learning (CRL) allows solving complex tasks by generating a tailored sequence of learning tasks, starting from easy ones and subsequently increasing their difficulty. Although the potential of curricula in RL has been clearly shown in various works, it is less clear how to generate them for a given learning environment, resulting in various methods aiming to automate this task. In this work, we focus on framing curricula as interpolations between task distributions, which has previously been shown to be a viable approach to CRL. Identifying key issues of existing methods, we frame the generation of a curriculum as a constrained optimal transport problem between task distributions. Benchmarks show that this way of curriculum generation can improve upon existing CRL methods, yielding high performance in various tasks with different characteristics.

Variational Hierarchical Mixtures for Probabilistic Learning of Inverse Dynamics

Well-calibrated probabilistic regression models are a crucial learning component in robotics applications as datasets grow rapidly and tasks become more complex. Unfortunately, classical regression models are usually either probabilistic kernel machines with a flexible structure that does not scale gracefully with data or deterministic and vastly scalable automata, albeit with a restrictive parametric form and poor regularization. In this paper, we consider a probabilistic hierarchical modeling paradigm that combines the benefits of both worlds to deliver computationally efficient representations with inherent complexity regularization. The presented approaches are probabilistic interpretations of local regression techniques that approximate nonlinear functions through a set of local linear or polynomial units. Importantly, we rely on principles from Bayesian nonparametrics to formulate flexible models that adapt their complexity to the data and can potentially encompass an infinite number of components. We derive two efficient variational inference techniques to learn these representations and highlight the advantages of hierarchical infinite local regression models, such as dealing with non-smooth functions, mitigating catastrophic forgetting, and enabling parameter sharing and fast predictions. Finally, we validate this approach on large inverse dynamics datasets and test the learned models in real-world control scenarios.

Decomposition of Reinforcement Learning Deficits in Disordered Gambling via Drift Diffusion Modeling and Functional Magnetic Resonance Imaging

Gambling disorder is associated with deficits in reward-based learning, but the underlying computational mechanisms are still poorly understood. Here, we examined this issue using a stationary reinforcement learning task in combination with computational modeling and functional resonance imaging (fMRI) in individuals that regular participate in gambling (n = 23, seven fulfilled one to three DSM 5 criteria for gambling disorder, sixteen fulfilled four or more) and matched controls (n = 23). As predicted, the gambling group exhibited substantially reduced accuracy, whereas overall response times (RTs) were not reliably different between groups. We then used comprehensive modeling using reinforcement learning drift diffusion models (RLDDMs) in combination with hierarchical Bayesian parameter estimation to shed light on the computational underpinnings of this performance deficit. In both groups, an RLDDM in which both non-decision time and decision threshold (boundary separation) changed over the course of the experiment accounted for the data best. The model showed good parameter and model recovery, and posterior predictive checks revealed that, in both groups, the model accurately reproduced the evolution of accuracies and RTs over time. Modeling revealed that, compared to controls, the learning impairment in the gambling group was linked to a more rapid reduction in decision thresholds over time, and a reduced impact of value-differences on the drift rate. The gambling group also showed shorter non-decision times. FMRI analyses replicated effects of prediction error coding in the ventral striatum and value coding in the ventro-medial prefrontal cortex, but there was no credible evidence for group differences in these effects. Taken together, our findings show that reinforcement learning impairments in disordered gambling are linked to both maladaptive decision threshold adjustments and a reduced consideration of option values in the choice process.

Identification, semi-quantification and risk assessment of contaminants of emerging concern in Flemish indoor dust through high-resolution mass spectrometry

Indoor dust can contribute substantially to human exposure to known and contaminants of emerging concern (CECs). Novel compounds with high structural variability and different homologues are frequently discovered through screening of the indoor environment, implying that constant monitoring is required. The present study aimed at the identification and semi-quantification of CECs in 46 indoor dust samples collected in Belgium by liquid chromatography high-resolution mass spectrometry. Samples were analyzed applying a targeted and suspect screening approach; the latter based on a suspect list containing >4000 CECs. This allowed the detection of a total of 55 CECs, 34 and 21 of which were identified with confidence level (CL) 1/2 or CL 3, respectively. Besides numerous known contaminants such as di(2-ethylhexyl) phthalate (DEHP), di(2-ethylhexyl) adipate (DEHA) or tris(2-butoxyethyl) phosphate (TBOEP) which were reported with detection frequencies (DFs) > 90%, several novel CECs were annotated. These included phthalates with differing side chains, such as decyl nonyl and decyl undecyl phthalate detected with DFs >80% and identified through the observation of characteristic neutral losses. Additionally, two novel organophosphate flame retardants not previously described in indoor dust, i.e. didecyl butoxyethoxyethyl phosphate (DDeBEEP) and bis(butoxyethyl) butyl phosphate (BBEBP), were identified. The implementation of a dedicated workflow provided semi-quantitative concentrations for a set of suspects. Such data obtained for novel phthalates were in the same order of magnitude as the concentrations observed for legacy phthalates indicating their high relevance for human exposure. From the semi-quantitative data, estimated daily intakes and resulting hazard quotients (HQs) were calculated to estimate the exposure and potential health effects. Neither of the obtained HQ values exceeded the risk threshold, indicating no expected adverse health effects.

The landscape of open science in behavioral addiction research: Current practices and future directions

Open science refers to a set of practices that aim to make scientific research more transparent, accessible, and reproducible, including pre-registration of study protocols, sharing of data and materials, the use of transparent research methods, and open access publishing. In this commentary, we describe and evaluate the current state of open science practices in behavioral addiction research. We highlight the specific value of open science practices for the field; discuss recent field-specific meta-scientific reviews that show the adoption of such practices remains in its infancy; address the challenges to engaging with open science; and make recommendations for how researchers, journals, and scientific institutions can work to overcome these challenges and promote high-quality, transparently reported behavioral addiction research. By collaboratively promoting open science practices, the field can create a more sustainable and productive research environment that benefits both the scientific community and society as a whole.

PAC-Bayes Bounds for Bandit Problems: A Survey and Experimental Comparison

PAC-Bayes has recently re-emerged as an effective theory with which one can derive principled learning algorithms with tight performance guarantees. However, applications of PAC-Bayes to bandit problems are relatively rare, which is a great misfortune. Many decision-making problems in healthcare, finance and natural sciences can be modelled as bandit problems. In many of these applications, principled algorithms with strong performance guarantees would be very much appreciated. This survey provides an overview of PAC-Bayes bounds for bandit problems and an experimental comparison of these bounds. On the one hand, we found that PAC-Bayes bounds are a useful tool for designing offline bandit algorithms with performance guarantees. In our experiments, a PAC-Bayesian offline contextual bandit algorithm was able to learn randomised neural network polices with competitive expected reward and non-vacuous performance guarantees. On the other hand, the PAC-Bayesian online bandit algorithms that we tested had loose cumulative regret bounds. We conclude by discussing some topics for future work on PAC-Bayesian bandit algorithms.

Erotic cue exposure increases physiological arousal, biases choices toward immediate rewards, and attenuates model-based reinforcement learning

Computational psychiatry focuses on identifying core cognitive processes that appear altered across distinct psychiatric disorders. Temporal discounting of future rewards and model-based control during reinforcement learning have proven as two promising candidates. Despite its trait-like stability, temporal discounting may be at least partly under contextual control. Highly arousing cues were shown to increase discounting, although evidence to date remains somewhat mixed. Whether model-based reinforcement learning is similarly affected by arousing cues remains unclear. Here, we tested cue-reactivity effects (erotic pictures) on subsequent temporal discounting and model-based reinforcement learning in a within-subjects design in n = 39 healthy heterosexual male participants. Self-reported and physiological arousal (cardiac activity and pupil dilation) were assessed before and during cue exposure. Arousal was increased during exposure of erotic versus neutral cues both on the subjective and autonomic level. Erotic cue exposure increased discounting as reflected by more impatient choices. Hierarchical drift diffusion modeling (DDM) linked increased discounting to a shift in the starting point bias of evidence accumulation toward immediate options. Model-based control during reinforcement learning was reduced following erotic cues according to model-agnostic analysis. Notably, DDM linked this effect to attenuated forgetting rates of unchosen options, leaving the model-based control parameter unchanged. Our findings replicate previous work on cue-reactivity effects in temporal discounting and for the first time show similar effects in model-based reinforcement learning in a heterosexual male sample. This highlights how environmental cues can impact core human decision processes and reveal that comprehensive modeling approaches can yield novel insights in reward-based decision processes.

A human-centered safe robot reinforcement learning framework with interactive behaviors

Deployment of Reinforcement Learning (RL) algorithms for robotics applications in the real world requires ensuring the safety of the robot and its environment. Safe Robot RL (SRRL) is a crucial step toward achieving human-robot coexistence. In this paper, we envision a human-centered SRRL framework consisting of three stages: safe exploration, safety value alignment, and safe collaboration. We examine the research gaps in these areas and propose to leverage interactive behaviors for SRRL. Interactive behaviors enable bi-directional information transfer between humans and robots, such as conversational robot ChatGPT. We argue that interactive behaviors need further attention from the SRRL community. We discuss four open challenges related to the robustness, efficiency, transparency, and adaptability of SRRL with interactive behaviors.

[Proximal femoral fracture with bony ankylosis of the hip joint : Case report of difficult surgical treatment]

A 78-year-old female patient sustained a left-sided proximal femoral fracture due to a fall. Since childhood the patient had suffered from ankylosis of unknown origin in the ipsilateral hip joint. The recommended intramedullary osteosynthesis could restore the original status quo before the fracture. It needs less time in the operating room (OR) and includes a lower risk for intraoperative complications than a hip joint endoprosthesis. Arthroplasty can lead to a significant improvement in mobility of the hip joint but requires more time in the OR for the surgical procedure and anesthesia time and includes a higher risk for intraoperative and postoperative complications. The antetorsion angle of 76° in this patient deviated extremely from the standard and made the intramedullary osteosynthesis for this type of fracture a challenge. Using improvised techniques led to a load-stable fracture treatment that ultimately led to a satisfactory result.

Chronic Deep Brain Stimulation of the Human Nucleus Accumbens Region Disrupts the Stability of Intertemporal Preferences

When choosing between rewards that differ in temporal proximity (intertemporal choice), human preferences are typically stable, constituting a clinically relevant transdiagnostic trait. Here we show, in female and male human patients undergoing deep brain stimulation (DBS) of the anterior limb of the internal capsule/NAcc region for treatment-resistant obsessive-compulsive disorder, that long-term chronic (but not phasic) DBS disrupts intertemporal preferences. Hierarchical Bayesian modeling accounting for temporal discounting behavior across multiple time points allowed us to assess both short-term and long-term reliability of intertemporal choice. In controls, temporal discounting was highly reliable, both long-term (6 months) and short-term (1 week). In contrast, in patients undergoing DBS, short-term reliability was high, but long-term reliability (6 months) was severely disrupted. Control analyses confirmed that this effect was not because of range restriction, the presence of obsessive-compulsive disorder symptoms or group differences in choice stochasticity. Model-agnostic between- and within-subject analyses confirmed this effect. These findings provide initial evidence for long-term modulation of cognitive function via DBS and highlight a potential contribution of the human NAcc region to intertemporal preference stability over time.SIGNIFICANCE STATEMENT Choosing between rewards that differ in temporal proximity is in part a stable trait with relevance for many mental disorders, and depends on prefrontal regions and regions of the dopamine system. Here we show that chronic deep brain stimulation of the human anterior limb of the internal capsule/NAcc region for treatment-resistant obsessive-compulsive disorder disrupts the stability of intertemporal preferences. These findings show that chronic stimulation of one of the brain's central motivational hubs can disrupt preferences thought to depend on this circuit.

[Salivary gland ultrasound or biopsy? : Comparison of methods based on case examples]

BACKGROUND

Ultrasound examination of the salivary glands (SG) is a quick and noninvasive method to detect and semiquantitatively estimate typical changes in the large SG in Sjögren's syndrome (SS). The differential diagnosis of SS is difficult because several diseases and adverse effects of treatment have a similar clinical picture as SS with sicca syndrome and can even induce alterations in the SG (mimic diseases). Hence, for a long time an SG biopsy was regarded as the diagnostic procedure of choice, especially in SS‑A negative patients, whereas the significance of SD sonography is still controversially discussed.

OBJECTIVE

Comparison of typical and atypical changes for SS in the salivary glands in ultrasound and associated histological sections.

MATERIAL AND METHODS

This article describes six patient cases with antibody positive or negative SS with and without typical SS ultrasound patterns, SS-associated lymphoma, sarcoidosis and IgG4-associated disease. The findings of the sonographic examination of the parotid glands and the associated histology of the SD are explained and put into context.

RESULTS

The SSA antibody positive patients with SS show a typical sonographic pattern with hypoechoic foci, especially if the disease has been present for a long time. This pattern can help support the diagnosis of SS. The ultrasound patterns of the mimic diseases sometimes differ significantly from the typical patterns of pSS. The histological examination of the SG helps to corroborate the diagnosis but low histological focus scores, in particular, require a critical synopsis of the clinical, serological and imaging findings.

CONCLUSION

Both salivary gland ultrasound and the histological examination of SG biopsies are justified in the diagnostics and differential diagnosis of SS and sicca syndrome.

Dopamine regulates decision thresholds in human reinforcement learning in males

Dopamine fundamentally contributes to reinforcement learning, but recent accounts also suggest a contribution to specific action selection mechanisms and the regulation of response vigour. Here, we examine dopaminergic mechanisms underlying human reinforcement learning and action selection via a combined pharmacological neuroimaging approach in male human volunteers (n = 31, within-subjects; Placebo, 150 mg of the dopamine precursor L-dopa, 2 mg of the D2 receptor antagonist Haloperidol). We found little credible evidence for previously reported beneficial effects of L-dopa vs. Haloperidol on learning from gains and altered neural prediction error signals, which may be partly due to differences experimental design and/or drug dosages. Reinforcement learning drift diffusion models account for learning-related changes in accuracy and response times, and reveal consistent decision threshold reductions under both drugs, in line with the idea that lower dosages of D2 receptor antagonists increase striatal DA release via an autoreceptor-mediated feedback mechanism. These results are in line with the idea that dopamine regulates decision thresholds during reinforcement learning, and may help to bridge action selection and response vigor accounts of dopamine.

Die Neustrukturierung der Notaufnahme gemäß dem G-BA-Beschluss zu einem gestuften System von Notfallstrukturen

BACKGROUND

There is no assessment of the effects regarding the decision of the Federal Joint Committee from April 19th, 2018 which demands a central and interdisciplinary emergency room system under the control of a professionally and organizationally independent administration. The aim of this study was to solicit the current opinion of leading orthopedic surgeons regarding the above-mentioned decision.

MATERIAL AND METHODS

An online questionnaire was sent to all 731 members of the Verband Leitender Orthopäden und Unfallchirurgen (www.vlou.de) to compare the opinion of these leading physicians on their current and future emergency room system.

RESULTS

The majority of the 179 responding surgeons consider emergency room systems with primary treatment of orthopedic patients by orthopedic surgeons in certain aspects to be more appropriate suitable than systems without such primary treatment. It is also not yet clear how work-related accidents should be treated according to the specifications of the German statutory accident insurance (DGUV®). The provision of training content could become more difficult if the ER rotation of orthopedic surgeons is not any longer part of these new ER concepts.

CONCLUSION

The consequences of the decision of the Federal Joint Committee from April 19th, 2018 need to be reevaluated continuously and, if necessary, optimised accordingly, taking into account competing requirements, such as training regulations and the SGB VII.

Large-Scale Web Scraping for Problem Gambling Research: A Case Study of COVID-19 Lockdown Effects in Germany

The COVID-19 pandemic and the measures to prevent its spread have had a negative impact on substance use behaviour. It is likely that social distancing and lockdown measures have also altered gambling behaviour, for instance shifting from land-based to online gambling. We used large-scale web scraping to analyse posting behaviour on a major German online gambling forum, gathering a database of more than 200k posts. We examined the usage of different subforums, i.e. terrestrial, online gambling and problem gambling sections, and changes in posting behaviour related to the casino closures that were part of the nationwide restrictions in Germany in 2020. There was a marked increase in newly registered users during the first lockdown compared to the preceding weeks, an increase in the number of posts in the online gambling subforum and concurrent decrease in the terrestrial gambling subforum. Further, the number of short-latency replies was higher during the first lockdown compared to the preceding weeks. Many users who posted in both the online and terrestrial forum contributed at least once to the problem gambling subforum, implying that the topic of problem gambling is widely discussed. Our findings may indicate a shift from terrestrial to online gambling during lockdown, and mirror the general increase in screen time and usage of online platforms after the onset of the COVID-19 pandemic. The analyses help to identify lockdown-related effects on gambling behaviour. These potentially detrimental effects pose a special threat for individuals at risk and may require monitoring and special public health measures.

Whether cycling around the city is in fact healthy in the light of air quality - Results of black carbon

Studying the air quality and exposure of the inhabitants of urban agglomerations to pollution is the basis for the creation and development of more sustainable cities. Although research on black carbon (BC) has not yet reached the official acceptable levels and guidelines, the World Health Organization clearly indicates the need to measure and control the level of this pollutant. In Poland, monitoring of the level of BC concentration is not included in the air quality monitoring network. To estimate the extent of this pollutant to which pedestrians and cyclists are exposed, mobile measurements were carried out on over 26 km of bicycle paths in Wrocław. The obtained results indicate the influence of urban greenery next to the bicycle path (especially if the cyclist is separated from the street lane by hedges or other tall plants) and the 'breathability' (i.e., associated with surrounding infrastructure) of the area on the obtained concentrations; the average concentration of BC in such places ranged from 1.3 to 2.2 μg/m³, whereas a cyclist riding directly on bike paths adjacent to the main roads in the city center is exposed to concentrations in the range of 2.3-14 μg/m³. The results of the measurements, also related to stationary measurements made at a selected point of one of the routes, clearly indicate the importance of the infrastructure surrounding the bicycle paths, their location, and the impact of urban traffic on the obtained BC concentrations. The results presented in our study are based only on short-term-field campaigns preliminary studies. To determine the quantitative impact of the characteristics of the bicycle route on the concentration of pollutants, and thus the exposure of users, the systematized research should cover a greater part of the city and be representative in terms of various hours of the day.

Identification and characterization of quaternary ammonium compounds in Flemish indoor dust by ion-mobility high-resolution mass spectrometry

Quaternary ammonium compounds (QACs) are a class of surfactants commonly used in disinfecting and cleaning products. Their use has substantially increased during the COVID-19 pandemic leading to increasing human exposure. QACs have been associated with hypersensitivity reactions and an increased risk of asthma. This study introduces the first identification, characterization and semi-quantification of QACs in European indoor dust using ion-mobility high-resolution mass spectrometry (IM-HRMS), including the acquisition of collision cross section values (^DTCCS_N2) for targeted and suspect QACs. A total of 46 indoor dust samples collected in Belgium were analyzed using target and suspect screening. Targeted QACs (n = 21) were detected with detection frequencies ranging between 4.2 and 100 %, while 15 QACs showed detection frequencies > 90 %. Semi-quantified concentrations of individual QACs showed a maximum of 32.23 µg/g with a median ∑QAC concentration of 13.05 µg/g and allowed the calculation of Estimated Daily Intakes for adults and toddlers. Most abundant QACs matched the patterns reported in indoor dust collected in the United States. Suspect screening allowed the identification of 17 additional QACs. A dialkyl dimethyl ammonium compound with mixed chain lengths (C16:C18) was characterized as a major QAC homologue with a maximum semi-quantified concentration of 24.90 µg/g. The high detection frequencies and structural variabilities observed call for more European studies on potential human exposure to these compounds. For all targeted QACs, drift tube IM-HRMS derived collision cross section values (^DTCCS_N2) are reported. Reference ^DTCCS_N2 values allowed the characterization of CCS-m/z trendlines for each of the targeted QAC classes. Experimental CCS-m/z ratios of suspect QACs were compared with the CCS-m/z trendlines. The alignment between the two datasets served as an additional confirmation of the assigned suspect QACs. The use of the 4bit multiplexing acquisition mode with consecutive high-resolution demultiplexing confirmed the presence of isomers for two of the suspect QACs.

[Position of trauma surgery in emergency medicine]

BACKGROUND

The decision of the Federal Joint Committee has resulted in the further development of in-hospital emergency medicine through the establishment of central emergency departments and staged emergency care. In addition, the additional training in clinical acute and emergency medicine was established.

AIM

The aim of this work is on the one hand to make trauma surgeons aware of these structural changes. On the other hand, we would like to evaluate an opinion and discuss the position of trauma surgery in emergency medicine.

METHODS

A web-based online survey was conducted at the Trauma Network and Trauma Registry Congress to collect participants' opinions on the position of trauma surgery in emergency medicine.

RESULTS

Of 143 congress participants, 98 (67%) responded to the survey. The majority of participants were male (n = 78, 80%), over 40 years of age (n = 62, 63%), and in a professional position with staff responsibility (n = 73, 75%). Emergency medicine (mean: 84.8; SD: 18.7) and intensive care medicine (mean: 78.3; SD: 20.4) training appears important. On the other hand, subsequent work in these areas appears less important (prehospital emergency medicine: mean: 65.1; SD: 28.0; ICU: mean: 53.7, SD: 30.3); however, activity in an emergency department is rated higher (MW: 87.0; SD: 18.7). There is high agreement that the trauma leader should be a trauma surgeon (mean 87.9; SD: 19.7).

DISCUSSION

A high volume of emergency trauma surgery patients and the care of severely injured patients in designated trauma centers show that trauma surgery expertise is mandatory in a central emergency department. Senior positions should also be sought to ensure high quality standards.

SensEURCity: A multi-city air quality dataset collected for 2020/2021 using open low-cost sensor systems

Low-cost air quality sensor systems can be deployed at high density, making them a significant candidate of complementary tools for improved air quality assessment. However, they still suffer from poor or unknown data quality. In this paper, we report on a unique dataset including the raw sensor data of quality-controlled sensor networks along with co-located reference data sets. Sensor data are collected using the AirSensEUR sensor system, including sensors to monitor NO, NO₂, O₃, CO, PM_2.5, PM₁₀, PM₁, CO₂ and meteorological parameters. In total, 85 sensor systems were deployed throughout a year in three European cities (Antwerp, Oslo and Zagreb), resulting in a dataset comprising different meteorological and ambient conditions. The main data collection included two co-location campaigns in different seasons at an Air Quality Monitoring Station (AQMS) in each city and a deployment at various locations in each city (also including locations at other AQMSs). The dataset consists of data files with sensor and reference data, and metadata files with description of locations, deployment dates and description of sensors and reference instruments.

Continuous-Time Fitted Value Iteration for Robust Policies

Solving the Hamilton-Jacobi-Bellman equation is important in many domains including control, robotics and economics. Especially for continuous control, solving this differential equation and its extension the Hamilton-Jacobi-Isaacs equation, is important as it yields the optimal policy that achieves the maximum reward on a give task. In the case of the Hamilton-Jacobi-Isaacs equation, which includes an adversary controlling the environment and minimizing the reward, the obtained policy is also robust to perturbations of the dynamics. In this paper we propose continuous fitted value iteration (cFVI) and robust fitted value iteration (rFVI). These algorithms leverage the non-linear control-affine dynamics and separable state and action reward of many continuous control problems to derive the optimal policy and optimal adversary in closed form. This analytic expression simplifies the differential equations and enables us to solve for the optimal value function using value iteration for continuous actions and states as well as the adversarial case. Notably, the resulting algorithms do not require discretization of states or actions. We apply the resulting algorithms to the Furuta pendulum and cartpole. We show that both algorithms obtain the optimal policy. The robustness Sim2Real experiments on the physical systems show that the policies successfully achieve the task in the real-world. When changing the masses of the pendulum, we observe that robust value iteration is more robust compared to deep reinforcement learning algorithm and the non-robust version of the algorithm. Videos of the experiments are shown at https://sites.google.com/view/rfvi.

A Deterministic Approximation to Neural SDEs

Neural Stochastic Differential Equations (NSDEs) model the drift and diffusion functions of a stochastic process as neural networks. While NSDEs are known to make accurate predictions, their uncertainty quantification properties have been remained unexplored so far. We report the empirical finding that obtaining well-calibrated uncertainty estimations from NSDEs is computationally prohibitive. As a remedy, we develop a computationally affordable deterministic scheme which accurately approximates the transition kernel, when dynamics is governed by a NSDE. Our method introduces a bidimensional moment matching algorithm: vertical along the neural net layers and horizontal along the time direction, which benefits from an original combination of effective approximations. Our deterministic approximation of the transition kernel is applicable to both training and prediction. We observe in multiple experiments that the uncertainty calibration quality of our method can be matched by Monte Carlo sampling only after introducing high computational cost. Thanks to the numerical stability of deterministic training, our method also improves prediction accuracy.

Mobile measurements of black carbon: Comparison of normal traffic with reduced traffic conditions during COVID-19 lock-down

A mobile monitoring campaign was conducted (by bicycle) to assess the black carbon (BC) concentrations in Cluj-Napoca city, Romania, in 2020, before, during and after COVID-19 lock-down. Over the entire study period, the BC concentrations ranged between 1.0 and 25.9 μg/m³ (averaged per street section and period characterized by different traffic conditions). Marked spatial and temporal differences were observed. Observed differences in BC concentrations between locations are attributed to traffic intensities, with average BC concentrations, under normal circumstances, of 6.6-14.3 μg/m³ at roads with high to intense traffic, compared to 2.8-3.1 μg/m³ at areas with reduced traffic, such as residential areas, parks and pedestrian streets. The COVID-19 measures impacted traffic volumes, and hence average BC concentrations decreased from 5.9 μg/m³ to 3.0 μg/m³ during lock-down and in a lower extent to 3.4 μg/m³ and 4.4 μg/m³ in post-lockdown periods with reduced and more normalized traffic. Two approaches to account for variations in background concentrations when comparing different situations in time are assessed. Subtracting background concentrations that are measured at background sites along the monitoring route is an appropriate method to assess spatio-temporal differences in concentrations. A reduction of about 1-2 μg/m³ was observed for the streets with low to medium traffic, and up to 6 μg/m³ at high traffic locations under lockdown. The approach presented in this study, using mobile measurements, is useful to understand the personal exposure to BC along the roads in different seasons and the influence of traffic reduction on BC pollution during prolonged restrictions. All these will support policymakers to reduce pollution and achieve EU directives targets and WHO recommendations.

The catecholamine precursor Tyrosine reduces autonomic arousal and decreases decision thresholds in reinforcement learning and temporal discounting

Supplementation with the catecholamine precursor L-Tyrosine might enhance cognitive performance, but overall findings are mixed. Here, we investigate the effect of a single dose of tyrosine (2g) vs. placebo on two catecholamine-dependent trans-diagnostic traits: model-based control during reinforcement learning (2-step task) and temporal discounting, using a double-blind, placebo-controlled, within-subject design (n = 28 healthy male participants). We leveraged drift diffusion models in a hierarchical Bayesian framework to jointly model participants' choices and response times (RTS) in both tasks. Furthermore, comprehensive autonomic monitoring (heart rate, heart rate variability, pupillometry, spontaneous eye blink rate) was performed both pre- and post-supplementation, to explore potential physiological effects of supplementation. Across tasks, tyrosine consistently reduced participants' RTs without deteriorating task-performance. Diffusion modeling linked this effect to attenuated decision-thresholds in both tasks and further revealed increased model-based control (2-step task) and (if anything) attenuated temporal discounting. On the physiological level, participants' pupil dilation was predictive of the individual degree of temporal discounting. Tyrosine supplementation reduced physiological arousal as revealed by increases in pupil dilation variability and reductions in heart rate. Supplementation-related changes in physiological arousal predicted individual changes in temporal discounting. Our findings provide first evidence that tyrosine supplementation might impact psychophysiological parameters, and suggest that modeling approaches based on sequential sampling models can yield novel insights into latent cognitive processes modulated by amino-acid supplementation.

[Atraumatic rupture of the extensor hallucis longus tendon]

An atraumatic rupture of the extensor hallucis longus tendon is a rare injury. Chronic overload due to contact sports, bony anomalies, previous operations, and drug injections close to the tendon have been reported as a cause. We report the case of a young patient who presented with an acute loss of function in his extensor hallucis longus tendon for no apparent reason.

Batch Reinforcement Learning With a Nonparametric Off-Policy Policy Gradient

Off-policy reinforcement learning (RL) holds the promise of better data efficiency as it allows sample reuse and potentially enables safe interaction with the environment. Current off-policy policy gradient methods either suffer from high bias or high variance, delivering often unreliable estimates. The price of inefficiency becomes evident in real-world scenarios such as interaction-driven robot learning, where the success of RL has been rather limited, and a very high sample cost hinders straightforward application. In this paper, we propose a nonparametric Bellman equation, which can be solved in closed form. The solution is differentiable w.r.t the policy parameters and gives access to an estimation of the policy gradient. In this way, we avoid the high variance of importance sampling approaches, and the high bias of semi-gradient methods. We empirically analyze the quality of our gradient estimate against state-of-the-art methods, and show that it outperforms the baselines in terms of sample efficiency on classical control tasks.

Continuous Action Reinforcement Learning From a Mixture of Interpretable Experts

Reinforcement learning (RL) has demonstrated its ability to solve high dimensional tasks by leveraging non-linear function approximators. However, these successes are mostly achieved by 'black-box' policies in simulated domains. When deploying RL to the real world, several concerns regarding the use of a 'black-box' policy might be raised. In order to make the learned policies more transparent, we propose in this paper a policy iteration scheme that retains a complex function approximator for its internal value predictions but constrains the policy to have a concise, hierarchical, and human-readable structure, based on a mixture of interpretable experts. Each expert selects a primitive action according to a distance to a prototypical state. A key design decision to keep such experts interpretable is to select the prototypical states from trajectory data. The main technical contribution of the paper is to address the challenges introduced by this non-differentiable prototypical state selection procedure. Experimentally, we show that our proposed algorithm can learn compelling policies on continuous action deep RL benchmarks, matching the performance of neural network based policies, but returning policies that are more amenable to human inspection than neural network or linear-in-feature policies.

HEBO: An Empirical Study of Assumptions in Bayesian Optimisation

In this work we rigorously analyse assumptions inherent to black-box optimisation hyper-parameter tuning tasks. Our results on the Bayesmark benchmark indicate that heteroscedasticity and non-stationarity pose significant challenges for black-box optimisers. Based on these findings, we propose a Heteroscedastic and Evolutionary Bayesian Optimisation solver (HEBO). HEBO performs non-linear input and output warping, admits exact marginal log-likelihood optimisation and is robust to the values of learned parameters. We demonstrate HEBO’s empirical efficacy on the NeurIPS 2020 Black-Box Optimisation challenge, where HEBO placed first. Upon further analysis, we observe that HEBO significantly outperforms existing black-box optimisers on 108 machine learning hyperparameter tuning tasks comprising the Bayesmark benchmark. Our findings indicate that the majority of hyper-parameter tuning tasks exhibit heteroscedasticity and non-stationarity, multiobjective acquisition ensembles with Pareto front solutions improve queried configurations, and robust acquisition maximisers afford empirical advantages relative to their non-robust counterparts. We hope these findings may serve as guiding principles for practitioners of Bayesian optimisation.

Motor response vigour and visual fixation patterns reflect subjective valuation during intertemporal choice

Value-based decision-making is of central interest in cognitive neuroscience and psychology, as well as in the context of neuropsychiatric disorders characterised by decision-making impairments. Studies examining (neuro-)computational mechanisms underlying choice behaviour typically focus on participants’ decisions. However, there is increasing evidence that option valuation might also be reflected in motor response vigour and eye movements, implicit measures of subjective utility. To examine motor response vigour and visual fixation correlates of option valuation in intertemporal choice, we set up a task where the participants selected an option by pressing a grip force transducer, simultaneously tracking fixation shifts between options. As outlined in our preregistration (https://osf.io/k6jct), we used hierarchical Bayesian parameter estimation to model the choices assuming hyperbolic discounting, compared variants of the softmax and drift diffusion model, and assessed the relationship between response vigour and the estimated model parameters. The behavioural data were best explained by a drift diffusion model specifying a non-linear scaling of the drift rate by the subjective value differences. Replicating previous findings, we found a magnitude effect for temporal discounting, such that higher rewards were discounted less. This magnitude effect was further reflected in motor response vigour, such that stronger forces were exerted in the high vs. the low magnitude condition. Bayesian hierarchical linear regression further revealed higher grip forces, faster response times and a lower number of fixation shifts for trials with higher subjective value differences. An exploratory analysis revealed that subjective value sums across options showed an even more pronounced association with trial-wise grip force amplitudes. Our data suggest that subjective utility or implicit valuation is reflected in motor response vigour and visual fixation patterns during intertemporal choice. Taking into account response vigour might thus provide deeper insight into decision-making, reward valuation and maladaptive changes in these processes, e.g. in the context of neuropsychiatric disorders.

POS0774 ANTI-Sm AUTOANTIBODIES IDENTIFY A PHENOTYPE OF SEVERE SLE WITH AN ASSOCIATED SERUM BIOMARKER PROFILE

Background

Antibodies to Smith (Sm) have been described as one of the most specific autoantibodies for systemic lupus erythematosus (SLE). Other than its association with lupus nephritis, there is, however, limited understanding of its clinical significance1,2.

Objectives

To describe clinical associations and serum protein profiles of anti-Sm positivity in SLE.

Methods

Patients fulfilling SLE classification criteria who were followed longitudinally in a prospective multicentre cohort were studied according to their baseline anti-Sm antibody status. Comparison between Sm+ and Sm- patients was made using descriptive statistics. Clinical associations of Sm positivity with patient disease characteristics were studied using logistic regression. In a subset, 211 serum analytes were measured using Quantibody, Luminex and ELISA assays. Associations between serum proteins and Sm positivity were studied using Least Absolute Shrinkage and Selection Operator (LASSO) penalised regression, adjusting for demographics (age, sex, ethnicity) and medication use

Results

383 patients were studied with median (IQR) follow-up of 4.9 (2,9) years; 65 (17%) had positive anti-Sm antibodies. Sm+ patients were significantly more likely to be of non-European ancestry (OR 2.73, 95% CI 1.55-4.82, p<0.001), and to be positive for anti-dsDNA antibodies (OR 2.8, 95% CI 2.3-3.4, p<0.001), anti-RNP antibodies (OR 15.7, 95% CI 13.9-17.8, p<0.001), direct anti-globulin test (OR 2.36, 95% CI 2.07-2.7, p<0.001) and hypocomplementemia (OR 7.73, 95% CI 5.1-11.7, p<0.001). Sm+ patients were significantly more likely to have active disease during the observation period in a range of organ domains, including mucocutaneous, renal, vasculitis and fever.

More Sm+ patients had episodes of High Disease Activity Status (HDAS, SLEDAI-2K ≧10)3 (OR 3.07, 95% CI 1.70-5.54, p<0.001) and persistent active disease (time-adjusted mean SLEDAI-2K > 4) (OR 3.23. 95% CI 1.84-5.70, p<0.001). Conversely, fewer Sm+ patients attained LLDAS for ≥50% observed time (19.7% vs 41.8%, p=0.002). Sm+ patients were more likely to be treated with glucocorticoids, immunosuppressants, and rituximab. There was no significant difference in damage accrual between Sm + and Sm - patients.

In serum protein analysis (n=197, 29 Sm+), LASSO modelling retained 3 proteins associated with Sm+ status, CXCL13, IL1RL1 and FLT1, along with Asian ethnicity and age. In analysis including pairwise interaction between predictors, 28 Sm+ associated proteins were identified, including CCL4, VCAM1, IL1RL1, Fcg R IIB/C, TDGF1, CEACAM1, TIMP1, BMP5, GDF15, and TNFRSF17.

Conclusion

Anti-Sm autoantibodies, present in 17% of SLE patients, were strongly associated with classical disease manifestations, more severe disease activity, and a specific serological and proteomic profile. These findings suggest anti-Sm+ SLE as a specific disease subset.

References

[1]Barada, FA., B.S. Andrews, J.S. Davis, R.P. Taylor, Antibodies to Sm in patients with systemic lupus erythematosus. Correlation of Sm antibody titers with disease activity and other laboratory parameters. Arthritis Rheum, 1981. 24:1236-1244

[2]Arroyo-Avilla, M, Y. Santiago-Casas, G.McGwin, R.S. Cantor, M. Petri, R. Ramsey-Goldman, J.D. Reveille, R.P.Kimberly, G.S. Alarcon, L.M.Vila, E.E. Brown. Clinical Associations of anti-Smith antibodies in PROFILE: a multi-ethnic lupus cohort. 2015. 34:1217-1223

[3]Koelmeyer, R., H.T. Nim, M. Nikpour, Y.B. Sun, A. Kao, O. Guenther, E. Morand, and A. Hoi, High disease activity status suggests more severe disease and damage accrual in systemic lupus erythematosus. Lupus Sci Med, 2020. 7(1).

Acknowledgements

I would like to acknowledge participants and clinicians involved with the Australian Lupus Registry & Biobank

Disclosure of Interests

None declared

Category-sensitive incidental reinstatement in medial temporal lobe subregions during word recognition

During associative retrieval, the brain reinstates neural representations that were present during encoding. The human medial temporal lobe (MTL), with its subregions hippocampus (HC), perirhinal cortex (PRC), and parahippocampal cortex (PHC), plays a central role in neural reinstatement. Previous studies have given compelling evidence for reinstatement in the MTL during explicitly instructed associative retrieval. High-confident recognition may be similarly accompanied by recollection of associated information from the encoding context. It is unclear, however, whether high-confident recognition memory elicits reinstatement in the MTL even in the absence of an explicit instruction to retrieve associated information. Here, we addressed this open question using high-resolution fMRI. Twenty-eight male and female human volunteers engaged in a recognition memory task for words that they had previously encoded together with faces and scenes. Using complementary univariate and multivariate approaches, we show that MTL subregions including the PRC, PHC, and HC differentially reinstate category-sensitive representations during high-confident word recognition, even though no explicit instruction to retrieve the associated category was given. This constitutes novel evidence that high-confident recognition memory is accompanied by incidental reinstatement of associated category information in MTL subregions, and supports a functional model of the MTL that emphasizes content-sensitive representations during both encoding and retrieval.

Fixing a Subtrochanteric Femoral Fracture with a Humerus Nail

A 17-year-old paraplegic patient sustained a subtrochanteric femoral fracture due to inadequate trauma. The unusual anatomical conditions associated with his congenital paraplegia did not allow treatment with a standard intramedullary implant for the femur. Because his soft tissues were already compromised, alternative options like plate osteosynthesis were considered unfavourable as a salvage procedure. Therefore, we used an implant designed for the humerus. A satisfactory result of osteosynthesis was achieved despite varus deformity, shortening and rotational error.

Robot Learning From Randomized Simulations: A Review

The rise of deep learning has caused a paradigm shift in robotics research, favoring methods that require large amounts of data. Unfortunately, it is prohibitively expensive to generate such data sets on a physical platform. Therefore, state-of-the-art approaches learn in simulation where data generation is fast as well as inexpensive and subsequently transfer the knowledge to the real robot (sim-to-real). Despite becoming increasingly realistic, all simulators are by construction based on models, hence inevitably imperfect. This raises the question of how simulators can be modified to facilitate learning robot control policies and overcome the mismatch between simulation and reality, often called the “reality gap.” We provide a comprehensive review of sim-to-real research for robotics, focusing on a technique named “domain randomization” which is a method for learning from randomized simulations.

Trial-wise exposure to visual emotional cues increases physiological arousal but not temporal discounting

Humans and many animals devalue future rewards as a function of time (temporal discounting). Increased discounting has been linked to various psychiatric conditions, including substance-use-disorders, behavioral addictions, and obesity. Despite its high intra-individual stability, temporal discounting is partly under contextual control. One prominent manipulation that has been linked to increases in discounting is the exposure to highly arousing appetitive cues. However, results from trial-wise cue exposure studies appear highly mixed, and changes in physiological arousal were not adequately controlled. Here we tested the effects of appetitive (erotic), aversive, and neutral visual cues on temporal discounting in 35 healthy male participants. The contribution of single-trial physiological arousal was assessed using comprehensive monitoring of autonomic activity (pupil size, heart rate, electrodermal activity). Physiological arousal was elevated following aversive and in particular erotic cues. In contrast to our pre-registered hypothesis, steepness of temporal discounting was not significantly affected by emotional cues of either valence. Aversive cues tended to increase decision noise. Computational modeling revealed that trial-wise arousal only accounted for minor variance over and above aversive and erotic condition effects, arguing against a general effect of physiological arousal on temporal discounting.

ExGenNet: Learning to Generate Robotic Facial Expression Using Facial Expression Recognition

The ability of a robot to generate appropriate facial expressions is a key aspect of perceived sociability in human-robot interaction. Yet many existing approaches rely on the use of a set of fixed, preprogrammed joint configurations for expression generation. Automating this process provides potential advantages to scale better to different robot types and various expressions. To this end, we introduce ExGenNet, a novel deep generative approach for facial expressions on humanoid robots. ExGenNets connect a generator network to reconstruct simplified facial images from robot joint configurations with a classifier network for state-of-the-art facial expression recognition. The robots' joint configurations are optimized for various expressions by backpropagating the loss between the predicted expression and intended expression through the classification network and the generator network. To improve the transfer between human training images and images of different robots, we propose to use extracted features in the classifier as well as in the generator network. Unlike most studies on facial expression generation, ExGenNets can produce multiple configurations for each facial expression and be transferred between robots. Experimental evaluations on two robots with highly human-like faces, Alfie (Furhat Robot) and the android robot Elenoide, show that ExGenNet can successfully generate sets of joint configurations for predefined facial expressions on both robots. This ability of ExGenNet to generate realistic facial expressions was further validated in a pilot study where the majority of human subjects could accurately recognize most of the generated facial expressions on both the robots.

The WHO 2016 diagnostic criteria for Acute Myeloid leukemia with myelodysplasia related changes (AML-MRC) produce a very heterogeneous entity: A retrospective analysis of the FAB subtype RAEB-T

We studied 79 patients with AML-MRC or RAEB-T, who were later reclassified according to the WHO classification. Marrow slides were examined cytomorphologically with regard to dysplasia. Patients were followed up until March 2020. Thirty-one patients underwent allogeneic stem cell transplantation (median survival (ms) 16 months), 14 were treated with induction chemotherapy (ms 8.4 months), 18 received hypomethylating agents (ms 9.2 months), 16 received low dose chemotherapy or best supportive care (ms 2.4 months). Only 30.4 % fulfilled the morphologic WHO criteria. 46.8 % were classified as AML-MRC by an antecedent MDS, 54.4 % of the pts were classified by MDS-related chromosomal abnormalities. 5 % did not fulfill any of the criteria and were entered based on 20-29 % medullary blasts. There was no difference in ms between pts presenting with > 50 % dysplasia as compared to pts with dysplasia between 10 % and 50 % (ms 9.1 vs 9.9 months, p = n.s.) or for pts with antecedent MDS (ms 9.1 vs 8.9 months, p = n.s.). Myelodysplasia-related cytogenetic abnormalities were associated with a worse outcome (ms 8.1 vs 13.5 months, p = 0.026). AML-MRC in its current definition is a heterogenous entity. Dysplasia of ≥ 50 % in ≥ two lineages is not helpful for diagnostics and prognostication and therefore should be deleted in future classifications. We recommend utilizing the WHO guidelines for defining dysplasia (10 % or greater in ≥ 1 of the three myeloid cell lines) assisting in establishing the diagnosis of MDS.

Heterogeneous Microscopic Dynamics of Intruded Water in a Superhydrophobic Nanoconfinement: Neutron Scattering and Molecular Modeling

With their strong confining porosity and versatile surface chemistry, zeolitic imidazolate frameworks-including the prototypical ZIF-8-display exceptional properties for various applications. In particular, the forced intrusion of water at high pressure (∼25 MPa) into ZIF-8 nanopores is of interest for energy storage. Such a system reveals also ideal to study experimentally water dynamics and thermodynamics in an ultrahydrophobic confinement. Here, we report on neutron scattering experiments to probe the molecular dynamics of water within ZIF-8 nanopores under high pressure up to 38 MPa. In addition to an overall confinement-induced slowing down, we provide evidence for strong dynamical heterogeneities with different underlying molecular dynamics. Using complementary molecular simulations, these heterogeneities are found to correspond to different microscopic mechanisms inherent to vicinal molecules located in strongly adsorbing sites (ligands) and other molecules nanoconfined in the cavity center. These findings unveil a complex microscopic dynamics, which results from the combination of surface residence times and exchanges between the cavity surface and center.