Missing the missing values: The ugly duckling of fairness in machine learning

Adaptive predictor-set linear model: An imputation-free method for linear regression prediction on data sets with missing values

Linear regression (LR) is vastly used in data analysis for continuous outcomes in biomedicine and epidemiology. Despite its popularity, LR is incompatible with missing data, which frequently occur in health sciences. For parameter estimation, this shortcoming is usually resolved by complete-case analysis or imputation. Both work-arounds, however, are inadequate for prediction, since they either fail to predict on incomplete records or ignore missingness-induced reduction in prediction accuracy and rely on (unrealistic) assumptions about the missing mechanism. Here, we derive adaptive predictor-set linear model (aps-lm), capable of making predictions for incomplete data without the need for imputation. It is derived by using a predictor-selection operation, the Moore-Penrose pseudoinverse, and the reduced QR decomposition. aps-lm is an LR generalization that inherently handles missing values. It is applied on a reference data set, where complete predictors and outcome are available, and yields a set of privacy-preserving parameters. In a second stage, these are shared for making predictions of the outcome on external data sets with missing entries for predictors without imputation. Moreover, aps-lm computes prediction errors that account for the pattern of missing values even under extreme missingness. We benchmark aps-lm in a simulation study. aps-lm showed greater prediction accuracy and reduced bias compared to popular imputation strategies under a wide range of scenarios including variation of sample size, goodness of fit, missing value type, and covariance structure. Finally, as a proof-of-principle, we apply aps-lm in the context of epigenetic aging clocks, linear models that predict a person's biological age from epigenetic data with promising clinical applications.

Electronic health records and stratified psychiatry: bridge to precision treatment?

The use of a stratified psychiatry approach that combines electronic health records (EHR) data with machine learning (ML) is one potentially fruitful path toward rapidly improving precision treatment in clinical practice. This strategy, however, requires confronting pervasive methodological flaws as well as deficiencies in transparency and reporting in the current conduct of ML-based studies for treatment prediction. EHR data shares many of the same data quality issues as other types of data used in ML prediction, plus some unique challenges. To fully leverage EHR data's power for patient stratification, increased attention to data quality and collection of patient-reported outcome data is needed.

Augmented Behavioral Annotation Tools, with Application to Multimodal Datasets and Models: A Systematic Review

Annotation tools are an essential component in the creation of datasets for machine learning purposes. Annotation tools have evolved greatly since the turn of the century, and now commonly include collaborative features to divide labor efficiently, as well as automation employed to amplify human efforts. Recent developments in machine learning models, such as Transformers, allow for training upon very large and sophisticated multimodal datasets and enable generalization across domains of knowledge. These models also herald an increasing emphasis on prompt engineering to provide qualitative fine-tuning upon the model itself, adding a novel emerging layer of direct machine learning annotation. These capabilities enable machine intelligence to recognize, predict, and emulate human behavior with much greater accuracy and nuance, a noted shortfall of which have contributed to algorithmic injustice in previous techniques. However, the scale and complexity of training data required for multimodal models presents engineering challenges. Best practices for conducting annotation for large multimodal models in the most safe and ethical, yet efficient, manner have not been established. This paper presents a systematic literature review of crowd and machine learning augmented behavioral annotation methods to distill practices that may have value in multimodal implementations, cross-correlated across disciplines. Research questions were defined to provide an overview of the evolution of augmented behavioral annotation tools in the past, in relation to the present state of the art. (Contains five figures and four tables).

Optimizing Equity: Working towards Fair Machine Learning Algorithms in Laboratory Medicine

BACKGROUND

Methods of machine learning provide opportunities to use real-world data to solve complex problems. Applications of these methods in laboratory medicine promise to increase diagnostic accuracy and streamline laboratory operations leading to improvement in the quality and efficiency of healthcare delivery. However, machine learning models are vulnerable to learning from undesirable patterns in the data that reflect societal biases. As a result, irresponsible application of machine learning may lead to the perpetuation, or even amplification, of existing disparities in healthcare outcomes.

CONTENT

In this work, we review what it means for a model to be unfair, discuss the various ways that machine learning models become unfair, and present engineering principles emerging from the field of algorithmic fairness. These materials are presented with a focus on the development of machine learning models in laboratory medicine.

SUMMARY

We hope that this work will serve to increase awareness, and stimulate further discussion, of this important issue among laboratorians as the field moves forward with the incorporation of machine learning models into laboratory practice.

The Development of the Existential Empathy Questionnaire

Therapist empathy toward existential concerns might be a critical component of clinical practice. This study aims to explore the psychometric properties of the Existential Empathy Questionnaire (EEQ), a self-report instrument developed to measure levels of existential empathy among mental health professionals. The EEQ was completed by a sample of 393 therapists recruited in Belgium, along with measures of general empathy, experiential avoidance, and existential avoidance. To assess the test–retest reliability, 353 participants of the same sample completed the EEQ a second time 2 weeks later. Clinical experience and therapeutic theoretical background were assessed to inform professional characteristics. The results support the use of the EEQ as a unidimensional measure of existential empathy. It demonstrates good internal reliability and temporal stability. A principal components analysis indicates three components with small to moderate intercorrelations, labeled as “Communication,” “Avoidance and Overwhelming Feelings,” and “Resonance and Presence.” EEQ total scores show a moderate positive association with general empathy and a moderate negative association with experiential and existential avoidance. Furthermore, the EEQ significantly uniquely predicts experiential and existential avoidance after controlling for general empathy. Years of clinical practice and a humanistic-oriented therapeutic approach correlated moderately with high EEQ scores.

Can language models automate data wrangling?

The automation of data science and other data manipulation processes depend on the integration and formatting of ‘messy’ data. Data wrangling is an umbrella term for these tedious and time-consuming tasks. Tasks such as transforming dates, units or names expressed in different formats have been challenging for machine learning because (1) users expect to solve them with short cues or few examples, and (2) the problems depend heavily on domain knowledge. Interestingly, large language models today (1) can infer from very few examples or even a short clue in natural language, and (2) can integrate vast amounts of domain knowledge. It is then an important research question to analyse whether language models are a promising approach for data wrangling, especially as their capabilities continue growing. In this paper we apply different variants of the language model Generative Pre-trained Transformer (GPT) to five batteries covering a wide range of data wrangling problems. We compare the effect of prompts and few-shot regimes on their results and how they compare with specialised data wrangling systems and other tools. Our major finding is that they appear as a powerful tool for a wide range of data wrangling tasks. We provide some guidelines about how they can be integrated into data processing pipelines, provided the users can take advantage of their flexibility and the diversity of tasks to be addressed. However, reliability is still an important issue to overcome.

Mitigating Bias in Radiology Machine Learning: 1. Data Handling

Minimizing bias is critical to adoption and implementation of machine learning (ML) in clinical practice. Systematic mathematical biases produce consistent and reproducible differences between the observed and expected performance of ML systems, resulting in suboptimal performance. Such biases can be traced back to various phases of ML development: data handling, model development, and performance evaluation. This report presents 12 suboptimal practices during data handling of an ML study, explains how those practices can lead to biases, and describes what may be done to mitigate them. Authors employ an arbitrary and simplified framework that splits ML data handling into four steps: data collection, data investigation, data splitting, and feature engineering. Examples from the available research literature are provided. A Google Colaboratory Jupyter notebook includes code examples to demonstrate the suboptimal practices and steps to prevent them. Keywords: Data Handling, Bias, Machine Learning, Deep Learning, Convolutional Neural Network (CNN), Computer-aided Diagnosis (CAD) © RSNA, 2022.

Duration of Care and Operative Time Are the Primary Drivers of Total Charges After Ambulatory Hip Arthroscopy: A Machine Learning Analysis

PURPOSE

To develop a machine learning algorithm to predict total charges after ambulatory hip arthroscopy and create a risk-adjusted payment model based on patient comorbidities.

METHODS

A retrospective review of the New York State Ambulatory Surgery and Services database was performed to identify patients who underwent elective hip arthroscopy between 2015 and 2016. Features included in initial models consisted of patient characteristics, medical comorbidities, and procedure-specific variables. Models were generated to predict total charges using 5 algorithms. Model performance was assessed by the root-mean-square error, root-mean-square logarithmic error, and coefficient of determination. Global variable importance and partial dependence curves were constructed to show the impact of each input feature on total charges. For performance benchmarking, the best candidate model was compared with a multivariate linear regression using the same input features.

RESULTS

A total of 5,121 patients were included. The median cost after hip arthroscopy was $19,720 (interquartile range, $12,399-$26,439). The gradient-boosted ensemble model showed the best performance (root-mean-square error, $3,800 [95% confidence interval, $3,700-$3,900]; logarithmic root-mean-square error, 0.249 [95% confidence interval, 0.24-0.26]; R² = 0.73). Major cost drivers included total hours in facility less than 12 or more than 15, longer procedure time, performance of a labral repair, age younger than 30 years, Elixhauser Comorbidity Index (ECI) of 1 or greater, African American race, residence in extreme urban and rural areas, and higher household and neighborhood income.

CONCLUSIONS

The gradient-boosted ensemble model effectively predicted total charges after hip arthroscopy. Few modifiable variables were identified other than anesthesia type; nonmodifiable drivers of total charges included duration of care less than 12 hours or more than 15 hours, operating room time more than 100 minutes, age younger than 30 years, performance of a labral repair, and ECI greater than 0. Stratification of patients based on the ECI highlighted the increased financial risk borne by physicians via flat reimbursement schedules given variable degrees of comorbidities.

LEVEL OF EVIDENCE

Level III, retrospective cohort study.