MOOD 2020: A Public Benchmark for Out-of-Distribution Detection and Localization on Medical Images

Detecting Out-of-Distribution (OoD) data is one of the greatest challenges in safe and robust deployment of machine learning algorithms in medicine. When the algorithms encounter cases that deviate from the distribution of the training data, they often produce incorrect and over-confident predictions. OoD detection algorithms aim to catch erroneous predictions in advance by analysing the data distribution and detecting potential instances of failure. Moreover, flagging OoD cases may support human readers in identifying incidental findings. Due to the increased interest in OoD algorithms, benchmarks for different domains have recently been established. In the medical imaging domain, for which reliable predictions are often essential, an open benchmark has been missing. We introduce the Medical-Out-Of-Distribution-Analysis-Challenge (MOOD) as an open, fair, and unbiased benchmark for OoD methods in the medical imaging domain. The analysis of the submitted algorithms shows that performance has a strong positive correlation with the perceived difficulty, and that all algorithms show a high variance for different anomalies, making it yet hard to recommend them for clinical practice. We also see a strong correlation between challenge ranking and performance on a simple toy test set, indicating that this might be a valuable addition as a proxy dataset during anomaly detection algorithm development.

Comparative analysis of machine learning algorithms for computer-assisted reporting based on fully automated cross-lingual RadLex mappings

Computer-assisted reporting (CAR) tools were suggested to improve radiology report quality by context-sensitively recommending key imaging biomarkers. However, studies evaluating machine learning (ML) algorithms on cross-lingual ontological (RadLex) mappings for developing embedded CAR algorithms are lacking. Therefore, we compared ML algorithms developed on human expert-annotated features against those developed on fully automated cross-lingual (German to English) RadLex mappings using 206 CT reports of suspected stroke. Target label was whether the Alberta Stroke Programme Early CT Score (ASPECTS) should have been provided (yes/no:154/52). We focused on probabilistic outputs of ML-algorithms including tree-based methods, elastic net, support vector machines (SVMs) and fastText (linear classifier), which were evaluated in the same 5 × fivefold nested cross-validation framework. This allowed for model stacking and classifier rankings. Performance was evaluated using calibration metrics (AUC, brier score, log loss) and -plots. Contextual ML-based assistance recommending ASPECTS was feasible. SVMs showed the highest accuracies both on human-extracted- (87%) and RadLex features (findings:82.5%; impressions:85.4%). FastText achieved the highest accuracy (89.3%) and AUC (92%) on impressions. Boosted trees fitted on findings had the best calibration profile. Our approach provides guidance for choosing ML classifiers for CAR tools in fully automated and language-agnostic fashion using bag-of-RadLex terms on limited expert-labelled training data.

The Functional Role of Large-scale Brain Network Coordination in Placebo-induced Anxiolysis

Anxiety reduction through mere expectation of anxiolytic treatment effects (placebo anxiolysis) has enormous clinical importance. Recent behavioral and electrophysiological data suggest that placebo anxiolysis involves reduced vigilance and enhanced internalization of attention; however, the underlying neurobiological mechanisms are not yet clear. Given the fundamental function of intrinsic connectivity networks (ICNs) in basic cognitive processes, we investigated ICN activity patterns associated with externally and internally directed mental states under the influence of an anxiolytic placebo medication. Based on recent findings, we specifically analyzed the functional role of the rostral anterior cingulate cortex (rACC) in coordinating placebo-dependent cue-related (phasic) and cue-unrelated (sustained) network activity. Under placebo, we observed a down-regulation of the entire salience network (SN), particularly in response to threatening cues. The rACC exhibited enhanced cue-unrelated functional connectivity (FC) with the SN, which correlated with reductions in tonic arousal and anxiety. Hence, apart from the frequently reported modulation of aversive cue responses, the rACC appears to be crucially involved in exerting a tonically dampening control over salience-responsive structures. In line with a more internally directed mental state, we also found enhanced FC within the default mode network (DMN), again predicting reductions in anxiety under placebo.

Beyond scale-free small-world networks: cortical columns for quick brains

We study to what extent cortical columns with their particular wiring boost neural computation. Upon a vast survey of columnar networks performing various real-world cognitive tasks, we detect no signs of enhancement. It is on a mesoscopic--intercolumnar--scale that the existence of columns, largely irrespective of their inner organization, enhances the speed of information transfer and minimizes the total wiring length required to bind distributed columnar computations towards spatiotemporally coherent results. We suggest that brain efficiency may be related to a doubly fractal connectivity law, resulting in networks with efficiency properties beyond those by scale-free networks.