Machine learning for healthcare that matters: Reorienting from technical novelty to equitable impact

Despite significant technical advances in machine learning (ML) over the past several years, the tangible impact of this technology in healthcare has been limited. This is due not only to the particular complexities of healthcare, but also due to structural issues in the machine learning for healthcare (MLHC) community which broadly reward technical novelty over tangible, equitable impact. We structure our work as a healthcare-focused echo of the 2012 paper "Machine Learning that Matters", which highlighted such structural issues in the ML community at large, and offered a series of clearly defined "Impact Challenges" to which the field should orient itself. Drawing on the expertise of a diverse and international group of authors, we engage in a narrative review and examine issues in the research background environment, training processes, evaluation metrics, and deployment protocols which act to limit the real-world applicability of MLHC. Broadly, we seek to distinguish between machine learning ON healthcare data and machine learning FOR healthcare-the former of which sees healthcare as merely a source of interesting technical challenges, and the latter of which regards ML as a tool in service of meeting tangible clinical needs. We offer specific recommendations for a series of stakeholders in the field, from ML researchers and clinicians, to the institutions in which they work, and the governments which regulate their data access.

Does diversity beget diversity? A scientometric analysis of over 150,000 studies and 49,000 authors published in high-impact medical journals between 2007 and 2022

Background

Health research that significantly impacts global clinical practice and policy is often published in high-impact factor (IF) medical journals. These outlets play a pivotal role in the worldwide dissemination of novel medical knowledge. However, researchers identifying as women and those affiliated with institutions in low- and middle-income countries (LMIC) have been largely underrepresented in high-IF journals across multiple fields of medicine. To evaluate disparities in gender and geographical representation among authors who have published in any of five top general medical journals, we conducted scientometric analyses using a large-scale dataset extracted from the New England Journal of Medicine (NEJM), Journal of the American Medical Association (JAMA), The British Medical Journal (BMJ), The Lancet, and Nature Medicine.

Methods

Author metadata from all articles published in the selected journals between 2007 and 2022 were collected using the DimensionsAI platform. The Genderize.io API was then utilized to infer each author's likely gender based on their extracted first name. The World Bank country classification was used to map countries associated with researcher affiliations to the LMIC or the high-income country (HIC) category. We characterized the overall gender and country income category representation across the medical journals. In addition, we computed article-level diversity metrics and contrasted their distributions across the journals.

Findings

We studied 151,536 authors across 49,764 articles published in five top medical journals, over a long period spanning 15 years. On average, approximately one-third (33.1%) of the authors of a given paper were inferred to be women; this result was consistent across the journals we studied. Further, 86.6% of the teams were exclusively composed of HIC authors; in contrast, only 3.9% were exclusively composed of LMIC authors. The probability of serving as the first or last author was significantly higher if the author was inferred to be a man (18.1% vs 16.8%, P < .01) or was affiliated with an institution in a HIC (16.9% vs 15.5%, P < .01). Our primary finding reveals that having a diverse team promotes further diversity, within the same dimension (i.e., gender or geography) and across dimensions. Notably, papers with at least one woman among the authors were more likely to also involve at least two LMIC authors (11.7% versus 10.4% in baseline, P < .001; based on inferred gender); conversely, papers with at least one LMIC author were more likely to also involve at least two women (49.4% versus 37.6%, P < .001; based on inferred gender).

Conclusion

We provide a scientometric framework to assess authorship diversity. Our research suggests that the inclusiveness of high-impact medical journals is limited in terms of both gender and geography. We advocate for medical journals to adopt policies and practices that promote greater diversity and collaborative research. In addition, our findings offer a first step towards understanding the composition of teams conducting medical research globally and an opportunity for individual authors to reflect on their own collaborative research practices and possibilities to cultivate more diverse partnerships in their work.

Peer review of GPT-4 technical report and systems card

The study provides a comprehensive review of OpenAI's Generative Pre-trained Transformer 4 (GPT-4) technical report, with an emphasis on applications in high-risk settings like healthcare. A diverse team, including experts in artificial intelligence (AI), natural language processing, public health, law, policy, social science, healthcare research, and bioethics, analyzed the report against established peer review guidelines. The GPT-4 report shows a significant commitment to transparent AI research, particularly in creating a systems card for risk assessment and mitigation. However, it reveals limitations such as restricted access to training data, inadequate confidence and uncertainty estimations, and concerns over privacy and intellectual property rights. Key strengths identified include the considerable time and economic investment in transparent AI research and the creation of a comprehensive systems card. On the other hand, the lack of clarity in training processes and data raises concerns about encoded biases and interests in GPT-4. The report also lacks confidence and uncertainty estimations, crucial in high-risk areas like healthcare, and fails to address potential privacy and intellectual property issues. Furthermore, this study emphasizes the need for diverse, global involvement in developing and evaluating large language models (LLMs) to ensure broad societal benefits and mitigate risks. The paper presents recommendations such as improving data transparency, developing accountability frameworks, establishing confidence standards for LLM outputs in high-risk settings, and enhancing industry research review processes. It concludes that while GPT-4's report is a step towards open discussions on LLMs, more extensive interdisciplinary reviews are essential for addressing bias, harm, and risk concerns, especially in high-risk domains. The review aims to expand the understanding of LLMs in general and highlights the need for new reflection forms on how LLMs are reviewed, the data required for effective evaluation, and addressing critical issues like bias and risk.

Steering clear of Akrasia: An integrative review of self-binding Ulysses Contracts in clinical practice

In many jurisdictions, legal frameworks afford patients the opportunity to make prospective medical decisions or to create directives that contain a special provision forfeiting their own ability to object to those decisions at a future time point, should they lose decision-making capacity. These agreements have been described with widely varying nomenclatures, including Ulysses Contracts, Odysseus Transfers, Psychiatric Advance Directives with Ulysses Clauses, and Powers of Attorney with Special Provisions. As a consequence of this terminological heterogeneity, it is challenging for healthcare providers to understand the terms and uses of these agreements and for ethicists to engage with the nuances of clinical decision-making with such unique provisions surrounding patient autonomy. In theory, prospective self-binding agreements may safeguard patient's "authentic" wishes from future "inauthentic" changes of mind. In practice, it is unclear what may be comprised within these agreements or how-and to what effect-they are used. The primary focus of this integrative review is to curate the existing literature describing Ulysses Contracts (and analogous decisions) used in the clinical arena, in order to empirically synthesize their shared essence and provide insights into the traditional components of these agreements when used in practice, the requirements of their consent processes, and the outcomes of their utilization.

Developing and Validating a Prediction Model For Death or Critical Illness in Hospitalized Adults, an Opportunity for Human-Computer Collaboration

Hospital early warning systems that use machine learning (ML) to predict clinical deterioration are increasingly being used to aid clinical decision-making. However, it is not known how ML predictions complement physician and nurse judgment. Our objective was to train and validate a ML model to predict patient deterioration and compare model predictions with real-world physician and nurse predictions.

DESIGN

Retrospective and prospective cohort study.

SETTING

Academic tertiary care hospital.

PATIENTS

Adult general internal medicine hospitalizations.

MEASUREMENTS AND MAIN RESULTS

We developed and validated a neural network model to predict in-hospital death and ICU admission in 23,528 hospitalizations between April 2011 and April 2019. We then compared model predictions with 3,374 prospectively collected predictions from nurses, residents, and attending physicians about their own patients in 960 hospitalizations between April 30, and August 28, 2019. ML model predictions achieved clinician-level accuracy for predicting ICU admission or death (ML median F1 score 0.32 [interquartile range (IQR) 0.30-0.34], AUC 0.77 [IQ 0.76-0.78]; clinicians median F1-score 0.33 [IQR 0.30-0.35], AUC 0.64 [IQR 0.63-0.66]). ML predictions were more accurate than clinicians for ICU admission. Of all ICU admissions and deaths, 36% occurred in hospitalizations where the model and clinicians disagreed. Combining human and model predictions detected 49% of clinical deterioration events, improving sensitivity by 16% compared with clinicians alone and 24% compared with the model alone while maintaining a positive predictive value of 33%, thus keeping false alarms at a clinically acceptable level.

CONCLUSIONS

ML models can complement clinician judgment to predict clinical deterioration in hospital. These findings demonstrate important opportunities for human-computer collaboration to improve prognostication and personalized medicine in hospital.

Large scale genotype- and phenotype-driven machine learning in Von Hippel-Lindau disease

Von Hippel-Lindau (VHL) disease is a hereditary cancer syndrome where individuals are predisposed to tumor development in the brain, adrenal gland, kidney, and other organs. It is caused by pathogenic variants in the VHL tumor suppressor gene. Standardized disease information has been difficult to collect due to the rarity and diversity of VHL patients. Over 4100 unique articles published until October 2019 were screened for germline genotype-phenotype data. Patient data were translated into standardized descriptions using Human Genome Variation Society gene variant nomenclature and Human Phenotype Ontology terms and has been manually curated into an open-access knowledgebase called Clinical Interpretation of Variants in Cancer. In total, 634 unique VHL variants, 2882 patients, and 1991 families from 427 papers were captured. We identified relationship trends between phenotype and genotype data using classic statistical methods and spectral clustering unsupervised learning. Our analyses reveal earlier onset of pheochromocytoma/paraganglioma and retinal angiomas, phenotype co-occurrences and genotype-phenotype correlations including hotspots. It confirms existing VHL associations and can be used to identify new patterns and associations in VHL disease. Our database serves as an aggregate knowledge translation tool to facilitate sharing information about the pathogenicity of VHL variants.

Underrecognition of Dysmenorrhea Is an Iatrogenic Harm

Many patients face years of recurrent and debilitating menstrual pain that affects their ability to work and study. Patients often normalize their severe pain as an expected part of menses. Both underrecognition and lack of awareness of available therapies for this remediable condition serve as a quintessential example of hermeneutic injustice. Hermeneutic injustice describes a structural lack of access to epistemic resources, such as shared concepts and knowledge. Pervasive menstrual stigma further discourages people with dysmenorrhea from discussing their symptoms and seeking health care. A lack of respect for women's experiences of pain in clinical encounters acts to worsen these issues and should be considered a source of iatrogenic harm. Health care workers can promote hermeneutic justice by preemptively destigmatizing discussions about menstruation and validating patients' concerns. On a systemic level, there should be greater awareness of dysmenorrhea and the various treatments availabe for it.

An ethical analysis of clinical triage protocols and decision-making frameworks: what do the principles of justice, freedom, and a disability rights approach demand of us?

Background

The expectation of pandemic-induced severe resource shortages has prompted authorities to draft and update frameworks to guide clinical decision-making and patient triage. While these documents differ in scope, they share a utilitarian focus on the maximization of benefit. This utilitarian view necessarily marginalizes certain groups, in particular individuals with increased medical needs.

Main body

Here, we posit that engagement with the disability critique demands that we broaden our understandings of justice and fairness in clinical decision-making and patient triage. We propose the capabilities theory, which recognizes that justice requires a range of positive capabilities/freedoms conducive to the achievement of meaningful life goals, as a means to do so. Informed by a disability rights critique of the clinical response to the pandemic, we offer direction for the construction of future clinical triage protocols which will avoid ableist biases by incorporating a broader apprehension of what it means to be human.

Conclusion

The clinical pandemic response, codified across triage protocols, should embrace a form of justice which incorporates a vision of pluralistic human capabilities and a valuing of positive freedoms.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12910-022-00749-0.

Believing in Black Boxes: Machine Learning for Healthcare Does Not Need Explainability to be Evidence-Based

OBJECTIVE

To examine the role of explainability in machine learning for healthcare (MLHC), and its necessity and significance with respect to effective and ethical MLHC application.

STUDY DESIGN AND SETTING

This commentary engages with the growing and dynamic corpus of literature on the use of MLHC and artificial intelligence (AI) in medicine, which provide the context for a focused narrative review of arguments presented in favour of and opposition to explainability in MLHC.

RESULTS

We find that concerns regarding explainability are not limited to MLHC, but rather extend to numerous well-validated treatment interventions as well as to human clinical judgment itself. We examine the role of evidence-based medicine in evaluating inexplicable treatments and technologies, and highlight the analogy between the concept of explainability in MLHC and the related concept of mechanistic reasoning in evidence-based medicine.

CONCLUSION

Ultimately, we conclude that the value of explainability in MLHC is not intrinsic, but is instead instrumental to achieving greater imperatives such as performance and trust. We caution against the uncompromising pursuit of explainability, and advocate instead for the development of robust empirical methods to successfully evaluate increasingly inexplicable algorithmic systems.

From Clinic to Computer and Back Again: Practical Considerations When Designing and Implementing Machine Learning Solutions for Pediatrics

Purpose of review

Machine learning (ML), a branch of artificial intelligence, is influencing all fields in medicine, with an abundance of work describing its application to adult practice. ML in pediatrics is distinctly unique with clinical, technical, and ethical nuances limiting the direct translation of ML tools developed for adults to pediatric populations. To our knowledge, no work has yet focused on outlining the unique considerations that need to be taken into account when designing and implementing ML in pediatrics.

Recent findings

The nature of varying developmental stages and the prominence of family-centered care lead to vastly different data-generating processes in pediatrics. Data heterogeneity and a lack of high-quality pediatric databases further complicate ML research. In order to address some of these nuances, we provide a common pipeline for clinicians and computer scientists to use as a foundation for structuring ML projects, and a framework for the translation of a developed model into clinical practice in pediatrics. Throughout these pathways, we also highlight ethical and legal considerations that must be taken into account when working with pediatric populations and data.

Summary

Here, we describe a comprehensive outline of special considerations required of ML in pediatrics from project ideation to implementation. We hope this review can serve as a high-level guideline for ML scientists and clinicians alike to identify applications in the pediatric setting, generate effective ML solutions, and subsequently deliver them to patients, families, and providers.

Characterizing early Canadian federal, provincial, territorial and municipal nonpharmaceutical interventions in response to COVID-19: a descriptive analysis

BACKGROUND

Nonpharmaceutical interventions (NPIs) are the primary tools to mitigate early spread of the coronavirus disease 2019 (COVID-19) pandemic; however, such policies are implemented variably at the federal, provincial or territorial, and municipal levels without centralized documentation. We describe the development of the comprehensive open Canadian Non-Pharmaceutical Intervention (CAN-NPI) data set, which identifies and classifies all NPIs implemented in regions across Canada in response to COVID-19, and provides an accompanying description of geographic and temporal heterogeneity.

METHODS

We performed an environmental scan of government websites, news media and verified government social media accounts to identify NPIs implemented in Canada between Jan. 1 and Apr. 19, 2020. The CAN-NPI data set contains information about each intervention's timing, location, type, target population and alignment with a response stringency measure. We conducted descriptive analyses to characterize the temporal and geographic variation in early NPI implementation.

RESULTS

We recorded 2517 NPIs grouped in 63 distinct categories during this period. The median date of NPI implementation in Canada was Mar. 24, 2020. Most jurisdictions heightened the stringency of their response following the World Health Organization's global pandemic declaration on Mar. 11, 2020. However, there was variation among provinces or territories in the timing and stringency of NPI implementation, with 8 out of 13 provinces or territories declaring a state of emergency by Mar. 18, and all by Mar. 22, 2020.

INTERPRETATION

There was substantial geographic and temporal heterogeneity in NPI implementation across Canada, highlighting the importance of a subnational lens in evaluating the COVID-19 pandemic response. Our comprehensive open-access data set will enable researchers to conduct robust interjurisdictional analyses of NPI impact in curtailing COVID-19 transmission.

What do medical students actually need to know about artificial intelligence?

With emerging innovations in artificial intelligence (AI) poised to substantially impact medical practice, interest in training current and future physicians about the technology is growing. Alongside comes the question of what, precisely, should medical students be taught. While competencies for the clinical usage of AI are broadly similar to those for any other novel technology, there are qualitative differences of critical importance to concerns regarding explainability, health equity, and data security. Drawing on experiences at the University of Toronto Faculty of Medicine and MIT Critical Data's "datathons", the authors advocate for a dual-focused approach: combining robust data science-focused additions to baseline health research curricula and extracurricular programs to cultivate leadership in this space.