A survey of human judgement and quantitative forecasting methods

Predicting pharmaceutical supply chain disruptions before and during the COVID-19 pandemic

Disruptions to the pharmaceutical supply chain (PSC) have negative implications for patients, motivating their prediction to improve risk mitigation. Although data analytics and machine learning methods have been proposed to support the characterization of probabilities to inform decisions and risk mitigation strategies, their application in the PSC has not been previously described. Further, it is unclear how well these models perform in the presence of emergent events representing deep uncertainty such as the COVID-19 pandemic. This article examines the use of data-driven models to predict PSC disruptions before and during the COVID-19 pandemic. Using data on generic drugs from the pharmacy supply chain division of a Fortune 500 pharmacy benefit management firm, we have developed predictive models based on the naïve Bayes algorithm, where the models predict whether a specific supplier or whether a specific product will experience a supply disruption in the next time period. We find statistically significant changes in the relationships of nearly all variables associated with product supply disruptions during the pandemic, despite pre-pandemic stability. We present results showing how the sensitivity, specificity, and false positive rate of predictive models changed with the onset of the COVID-19 pandemic and show the beneficial effects of regular model updating. The results show that maintaining model sensitivity is more challenging than maintaining specificity and false positive rates. The results provide unique insight into the pandemic's effect on risk prediction within the PSC and provide insight for risk analysts to better understand how surprise events and deep uncertainty affect predictive models.

How will technology change people's home care in the next 20 years? A strategic foresight study

The rapid expansion of home health care has raised many unresolved issues and will have far-reaching consequences that can only be overcome with a holistic approach to help build and use collective intelligence in a structured, systemic way to anticipate developments. In this frame, the set of issues covered by the human factors research field will significantly impact the safety, quality, and effectiveness of home health care. However, only with a gaze of strategic foresight will we be capable of exploring, anticipating, and shaping the future. A group of researchers from the Italian Society of Ergonomics and Human Factors (SIE) has developed a road map to help all the stakeholders involved in this process.

Compromising improves forecasting

Predicting the future can bring enormous advantages. Across the ages, reliance on supernatural foreseeing was substituted by the opinion of expert forecasters, and now by collective intelligence approaches which draw on many non-expert forecasters. Yet all of these approaches continue to see individual forecasts as the key unit on which accuracy is determined. Here, we hypothesize that compromise forecasts, defined as the average prediction in a group, represent a better way to harness collective predictive intelligence. We test this by analysing 5 years of data from the Good Judgement Project and comparing the accuracy of individual versus compromise forecasts. Furthermore, given that an accurate forecast is only useful if timely, we analyze how the accuracy changes through time as the events approach. We found that compromise forecasts are more accurate, and that this advantage persists through time, though accuracy varies. Contrary to what was expected (i.e. a monotonous increase in forecasting accuracy as time passes), forecasting error for individuals and for team compromise starts its decline around two months prior to the event. Overall, we offer a method of aggregating forecasts to improve accuracy, which can be straightforwardly applied in noisy real-world settings.

To deliberate or not? The role of intuition and deliberation when controlling for irrelevant information in selection decisions

In selection decisions, decision makers often struggle to ignore irrelevant information, such as candidates' age, gender and attractiveness, which can lead to suboptimal decisions. One way to correct the effects of these irrelevant attributes is to consider them as suppressor variables, and penalize individuals who unjustifiably benefit from them. Previous research demonstrated that people have difficulties doing so. In five experiments (N = 1325), we examined the mechanism at the core of people's ability to do so. We found that triggering System 2 did not improve participants' ability to correct for this bias. The majority of those who were successful did so even when denied the opportunity to deliberate. We suggest that logic intuition-not deliberation-is the basis for successfully considering irrelevant information as suppressors. Our results are in line with a revised dual-process approach, in which solving reasoning problems can occur directly through System 1 and does not require an override by a System 2's-based process.

The simple observational critical care studies: estimations by students, nurses, and physicians of in-hospital and 6-month mortality

Background

Prognostic assessments of the mortality of critically ill patients are frequently performed in daily clinical practice and provide prognostic guidance in treatment decisions. In contrast to several sophisticated tools, prognostic estimations made by healthcare providers are always available and accessible, are performed daily, and might have an additive value to guide clinical decision-making. The aim of this study was to evaluate the accuracy of students’, nurses’, and physicians’ estimations and the association of their combined estimations with in-hospital mortality and 6-month follow-up.

Methods

The Simple Observational Critical Care Studies is a prospective observational single-center study in a tertiary teaching hospital in the Netherlands. All patients acutely admitted to the intensive care unit were included. Within 3 h of admission to the intensive care unit, a medical or nursing student, a nurse, and a physician independently predicted in-hospital and 6-month mortality. Logistic regression was used to assess the associations between predictions and the actual outcome; the area under the receiver operating characteristics (AUROC) was calculated to estimate the discriminative accuracy of the students, nurses, and physicians.

Results

In 827 out of 1,010 patients, in-hospital mortality rates were predicted to be 11%, 15%, and 17% by medical students, nurses, and physicians, respectively. The estimations of students, nurses, and physicians were all associated with in-hospital mortality (OR 5.8, 95% CI [3.7, 9.2], OR 4.7, 95% CI [3.0, 7.3], and OR 7.7 95% CI [4.7, 12.8], respectively). Discriminative accuracy was moderate for all students, nurses, and physicians (between 0.58 and 0.68). When more estimations were of non-survival, the odds of non-survival increased (OR 2.4 95% CI [1.9, 3.1]) per additional estimate, AUROC 0.70 (0.65, 0.76). For 6-month mortality predictions, similar results were observed.

Conclusions

Based on the initial examination, students, nurses, and physicians can only moderately predict in-hospital and 6-month mortality in critically ill patients. Combined estimations led to more accurate predictions and may serve as an example of the benefit of multidisciplinary clinical care and future research efforts.

Supplementary Information

The online version contains supplementary material available at 10.1186/s13054-021-03809-w.

A Brief Taxonomy of Hybrid Intelligence

As artificial intelligence becomes ubiquitous in our lives, so do the opportunities to combine machine and human intelligence to obtain more accurate and more resilient prediction models across a wide range of domains. Hybrid intelligence can be designed in many ways, depending on the role of the human and the algorithm in the hybrid system. This paper offers a brief taxonomy of hybrid intelligence, which describes possible relationships between human and machine intelligence for robust forecasting. In this taxonomy, biological intelligence represents one axis of variation, going from individual intelligence (one individual in isolation) to collective intelligence (several connected individuals). The second axis of variation represents increasingly sophisticated algorithms that can take into account more aspects of the forecasting system, from information to task to human problem-solvers. The novelty of the paper lies in the interpretation of recent studies in hybrid intelligence as precursors of a set of algorithms that are expected to be more prominent in the future. These algorithms promise to increase hybrid system’s resilience across a wide range of human errors and biases thanks to greater human-machine understanding. This work ends with a short overview for future research in this field.