The PLOS ONE collection on machine learning in health and biomedicine: Towards open code and open data

A multimodal framework for extraction and fusion of satellite images and public health data

In low- and middle-income countries, the substantial costs associated with traditional data collection pose an obstacle to facilitating decision-making in the field of public health. Satellite imagery offers a potential solution, but the image extraction and analysis can be costly and requires specialized expertise. We introduce SatelliteBench, a scalable framework for satellite image extraction and vector embeddings generation. We also propose a novel multimodal fusion pipeline that utilizes a series of satellite imagery and metadata. The framework was evaluated generating a dataset with a collection of 12,636 images and embeddings accompanied by comprehensive metadata, from 81 municipalities in Colombia between 2016 and 2018. The dataset was then evaluated in 3 tasks: including dengue case prediction, poverty assessment, and access to education. The performance showcases the versatility and practicality of SatelliteBench, offering a reproducible, accessible and open tool to enhance decision-making in public health.

Peroxynitrite-Triggered Carbon Monoxide Donor Improves Ischemic Stroke Outcome by Inhibiting Neuronal Apoptosis and Ferroptosis

Cerebral ischemia-reperfusion injury produces excessive reactive oxygen and nitrogen species, including superoxide, nitric oxide, and peroxynitrite (ONOO-). We recently developed a new ONOO--triggered metal-free carbon monoxide donor (PCOD585), exhibiting a notable neuroprotective outcome on the rat middle cerebral artery occlusion model and rendering an exciting intervention opportunity toward ischemia-induced brain injuries. However, its therapeutic mechanism still needs to be addressed. In the pharmacological study, we found PCOD585 inhibited neuronal Bcl2/Bax/caspase-3 apoptosis pathway in the peri-infarcted area of stroke by scavenging ONOO-. ONOO- scavenging further led to decreased Acyl-CoA synthetase long-chain family member 4 and increased glutathione peroxidase 4, to minimize lipoperoxidation. Additionally, the carbon monoxide release upon the ONOO- reaction with PCOD585 further inhibited the neuronal Iron-dependent ferroptosis associated with ischemia-reperfusion. Such a synergistic neuroprotective mechanism of PCOD585 yields as potent a neuroprotective effect as Edaravone. Additionally, PCOD585 penetrates the blood-brain barrier and reduces the degradation of zonula occludens-1 by inhibiting matrix metalloproteinase-9, thereby protecting the integrity of the blood-brain barrier. Our study provides a new perspective for developing multi-functional compounds to treat ischemic stroke.

German funders' data sharing policies-A qualitative interview study

BACKGROUND

Data sharing is commonly seen as beneficial for science but is not yet common practice. Research funding agencies are known to play a key role in promoting data sharing, but German funders' data sharing policies appear to lag behind in international comparison. This study aims to answer the question of how German data sharing experts inside and outside funding agencies perceive and evaluate German funders' data sharing policies and overall efforts to promote data sharing.

METHODS

This study is based on sixteen guided expert interviews with representatives of German funders and German research data experts from stakeholder organisations, who shared their perceptions of German' funders efforts to promote data sharing. By applying the method of qualitative content analysis to our interview data, we categorise and describe noteworthy aspects of the German data sharing policy landscape and illustrate our findings with interview passages.

RESULTS

We present our findings in five sections to distinguish our interviewees' perceptions on a) the status quo of German funders' data sharing policies, b) the role of funders in promoting data sharing, c) current and potential measures by funders to promote data sharing, d) general barriers to those measures, and e) the implementation of more binding data sharing requirements.

DISCUSSION AND CONCLUSION

Although funders are perceived to be important promoters and facilitators of data sharing throughout our interviews, only few German funding agencies have data sharing policies in place. Several interviewees stated that funders could do more, for example by providing incentives for data sharing or by introducing more concrete policies. Our interviews suggest the academic freedom of grantees is widely perceived as an obstacle for German funders in introducing mandatory data sharing requirements. However, some interviewees stated that stricter data sharing requirements could be justified if data sharing is a part of good scientific practice.

A critical moment in machine learning in medicine: on reproducible and interpretable learning

Over the past two decades, advances in computational power and data availability combined with increased accessibility to pre-trained models have led to an exponential rise in machine learning (ML) publications. While ML may have the potential to transform healthcare, this sharp increase in ML research output without focus on methodological rigor and standard reporting guidelines has fueled a reproducibility crisis. In addition, the rapidly growing complexity of these models compromises their interpretability, which currently impedes their successful and widespread clinical adoption. In medicine, where failure of such models may have severe implications for patients' health, the high requirements for accuracy, robustness, and interpretability confront ML researchers with a unique set of challenges. In this review, we discuss the semantics of reproducibility and interpretability, as well as related issues and challenges, and outline possible solutions to counteracting the "black box". To foster reproducibility, standard reporting guidelines need to be further developed and data or code sharing encouraged. Editors and reviewers may equally play a critical role by establishing high methodological standards and thus preventing the dissemination of low-quality ML publications. To foster interpretable learning, the use of simpler models more suitable for medical data can inform the clinician how results are generated based on input data. Model-agnostic explanation tools, sensitivity analysis, and hidden layer representations constitute further promising approaches to increase interpretability. Balancing model performance and interpretability are important to ensure clinical applicability. We have now reached a critical moment for ML in medicine, where addressing these issues and implementing appropriate solutions will be vital for the future evolution of the field.

Technical/Algorithm, Stakeholder, and Society (TASS) barriers to the application of artificial intelligence in medicine: A systematic review

INTRODUCTION

The use of artificial intelligence (AI), particularly machine learning and predictive analytics, has shown great promise in health care. Despite its strong potential, there has been limited use in health care settings. In this systematic review, we aim to determine the main barriers to successful implementation of AI in healthcare and discuss potential ways to overcome these challenges.

METHODS

We conducted a literature search in PubMed (1/1/2001-1/1/2023). The search was restricted to publications in the English language, and human study subjects. We excluded articles that did not discuss AI, machine learning, predictive analytics, and barriers to the use of these techniques in health care. Using grounded theory methodology, we abstracted concepts to identify major barriers to AI use in medicine.

RESULTS

We identified a total of 2,382 articles. After reviewing the 306 included papers, we developed 19 major themes, which we categorized into three levels: the Technical/Algorithm, Stakeholder, and Social levels (TASS). These themes included: Lack of Explainability, Need for Validation Protocols, Need for Standards for Interoperability, Need for Reporting Guidelines, Need for Standardization of Performance Metrics, Lack of Plan for Updating Algorithm, Job Loss, Skills Loss, Workflow Challenges, Loss of Patient Autonomy and Consent, Disturbing the Patient-Clinician Relationship, Lack of Trust in AI, Logistical Challenges, Lack of strategic plan, Lack of Cost-effectiveness Analysis and Proof of Efficacy, Privacy, Liability, Bias and Social Justice, and Education.

CONCLUSION

We identified 19 major barriers to the use of AI in healthcare and categorized them into three levels: the Technical/Algorithm, Stakeholder, and Social levels (TASS). Future studies should expand on barriers in pediatric care and focus on developing clearly defined protocols to overcome these barriers.

A scoping review of the landscape of health-related open datasets in Latin America

Artificial intelligence (AI) algorithms have the potential to revolutionize healthcare, but their successful translation into clinical practice has been limited. One crucial factor is the data used to train these algorithms, which must be representative of the population. However, most healthcare databases are derived from high-income countries, leading to non-representative models and potentially exacerbating health inequities. This review focuses on the landscape of health-related open datasets in Latin America, aiming to identify existing datasets, examine data-sharing frameworks, techniques, platforms, and formats, and identify best practices in Latin America. The review found 61 datasets from 23 countries, with the DATASUS dataset from Brazil contributing to the majority of articles. The analysis revealed a dearth of datasets created by the authors themselves, indicating a reliance on existing open datasets. The findings underscore the importance of promoting open data in Latin America. We provide recommendations for enhancing data sharing in the region.

The composition of data economy: a bibliometric approach and TCCM framework of conceptual, intellectual and social structure

Purpose

The data economy mainly relies on the surveillance capitalism business model, enabling companies to monetize their data. The surveillance allows for transforming private human experiences into behavioral data that can be harnessed in the marketing sphere. This study aims to focus on investigating the domain of data economy with the methodological lens of quantitative bibliometric analysis of published literature.

Design/methodology/approach

The bibliometric analysis seeks to unravel trends and timelines for the emergence of the data economy, its conceptualization, scientific progression and thematic synergy that could predict the future of the field. A total of 591 data between 2008 and June 2021 were used in the analysis with the Biblioshiny app on the web interfaced and VOSviewer version 1.6.16 to analyze data from Web of Science and Scopus.

Findings

This study combined findable, accessible, interoperable and reusable (FAIR) data and data economy and contributed to the literature on big data, information discovery and delivery by shedding light on the conceptual, intellectual and social structure of data economy and demonstrating data relevance as a key strategic asset for companies and academia now and in the future.

Research limitations/implications

Findings from this study provide a steppingstone for researchers who may engage in further empirical and longitudinal studies by employing, for example, a quantitative and systematic review approach. In addition, future research could expand the scope of this study beyond FAIR data and data economy to examine aspects such as theories and show a plausible explanation of several phenomena in the emerging field.

Practical implications

The researchers can use the results of this study as a steppingstone for further empirical and longitudinal studies.

Originality/value

This study confirmed the relevance of data to society and revealed some gaps to be undertaken for the future.

Reproducibility of prediction models in health services research

The field of health services research studies the health care system by examining outcomes relevant to patients and clinicians but also health economists and policy makers. Such outcomes often include health care spending, and utilization of care services. Building accurate prediction models using reproducible research practices for health services research is important for evidence-based decision making. Several systematic reviews have summarized prediction models for outcomes relevant to health services research, but these systematic reviews do not present a thorough assessment of reproducibility and research quality of the prediction modelling studies. In the present commentary, we discuss how recent advances in prediction modelling in other medical fields can be applied to health services research. We also describe the current status of prediction modelling in health services research, and we summarize available methodological guidance for the development, update, external validation and systematic appraisal of prediction models.

Outcome Analysis in Elective Electrical Cardioversion of Atrial Fibrillation Patients: Development and Validation of a Machine Learning Prognostic Model

Background: The integrated approach to electrical cardioversion (EC) in atrial fibrillation (AF) is complex; candidates can resolve spontaneously while waiting for EC, and post-cardioversion recurrence is high. Thus, it is especially interesting to avoid the programming of EC in patients who would restore sinus rhythm (SR) spontaneously or present early recurrence. We have analyzed the whole elective EC of the AF process using machine-learning (ML) in order to enable a more realistic and detailed simulation of the patient flow for decision making purposes. Methods: The dataset consisted of electronic health records (EHRs) from 429 consecutive AF patients referred for EC. For analysis of the patient outcome, we considered five pathways according to restoring and maintaining SR: (i) spontaneous SR restoration, (ii) pharmacologic-cardioversion, (iii) direct-current cardioversion, (iv) 6-month AF recurrence, and (v) 6-month rhythm control. We applied ML classifiers for predicting outcomes at each pathway and compared them with the CHA2DS2-VASc and HATCH scores. Results: With the exception of pathway (iii), all ML models achieved improvements in comparison with CHA2DS2-VASc or HATCH scores (p < 0.01). Compared to the most competitive score, the area under the ROC curve (AUC-ROC) was: 0.80 vs. 0.66 for predicting (i); 0.71 vs. 0.55 for (ii); 0.64 vs. 0.52 for (iv); and 0.66 vs. 0.51 for (v). For a threshold considered optimal, the empirical net reclassification index was: +7.8%, +47.2%, +28.2%, and +34.3% in favor of our ML models for predicting outcomes for pathways (i), (ii), (iv), and (v), respectively. As an example tool of generalizability of ML models, we deployed our algorithms in an open-source calculator, where the model would personalize predictions. Conclusions: An ML model improves the accuracy of restoring and maintaining SR predictions over current discriminators. The proposed approach enables a detailed simulation of the patient flow through personalized predictions.

The new normal? Redaction bias in biomedical science

A concerning amount of biomedical research is not reproducible. Unreliable results impede empirical progress in medical science, ultimately putting patients at risk. Many proximal causes of this irreproducibility have been identified, a major one being inappropriate statistical methods and analytical choices by investigators. Within this, we formally quantify the impact of inappropriate redaction beyond a threshold value in biomedical science. This is effectively truncation of a dataset by removing extreme data points, and we elucidate its potential to accidentally or deliberately engineer a spurious result in significance testing. We demonstrate that the removal of a surprisingly small number of data points can be used to dramatically alter a result. It is unknown how often redaction bias occurs in the broader literature, but given the risk of distortion to the literature involved, we suggest that it must be studiously avoided, and mitigated with approaches to counteract any potential malign effects to the research quality of medical science.

Effect of data leakage in brain MRI classification using 2D convolutional neural networks

In recent years, 2D convolutional neural networks (CNNs) have been extensively used to diagnose neurological diseases from magnetic resonance imaging (MRI) data due to their potential to discern subtle and intricate patterns. Despite the high performances reported in numerous studies, developing CNN models with good generalization abilities is still a challenging task due to possible data leakage introduced during cross-validation (CV). In this study, we quantitatively assessed the effect of a data leakage caused by 3D MRI data splitting based on a 2D slice-level using three 2D CNN models to classify patients with Alzheimer's disease (AD) and Parkinson's disease (PD). Our experiments showed that slice-level CV erroneously boosted the average slice level accuracy on the test set by 30% on Open Access Series of Imaging Studies (OASIS), 29% on Alzheimer's Disease Neuroimaging Initiative (ADNI), 48% on Parkinson's Progression Markers Initiative (PPMI) and 55% on a local de-novo PD Versilia dataset. Further tests on a randomly labeled OASIS-derived dataset produced about 96% of (erroneous) accuracy (slice-level split) and 50% accuracy (subject-level split), as expected from a randomized experiment. Overall, the extent of the effect of an erroneous slice-based CV is severe, especially for small datasets.

An open-source data set of anti-VEGF therapy in diabetic macular oedema patients over 4 years and their visual acuity outcomes

OBJECTIVES

The objective of this paper is to evaluate visual acuity (VA) outcomes of intravitreal anti-vascular endothelial growth factor (VEGF) in diabetic macular oedema (DMO).

METHODS

In this retrospective cohort study, electronic medical records for all patients undergoing intravitreal injections in a tertiary referral centre between March 2013 and October 2018 were analysed. Treatment response in terms of VA outcomes was reported for all eyes over a 4-year observation period.

RESULTS

Our cohort includes 2614 DMO eyes of 1964 patients over 48 months. Cox proportional-hazards modelling identified injection number (hazard ratio (HR) = 1.18), male gender (HR = 1.13) and baseline VA (HR = 1.09) as independent predictors to reach a favourable visual outcome of more than 70 Early Treatment Diabetic Retinopathy Study letters. Half of our cohort reached 70 letters 1.9 months after starting anti-VEGF therapy. Of those that reached 70 letters, 50% fell below 70 letters by 14.7 months.

CONCLUSION

To date, this is the largest single centre cohort study and over the longest observation period reporting on real-life outcomes of anti-VEGF in DMO. We have made an anonymised version of our data set available on an open-source data repository as a resource for clinical researchers globally.

Mobile Apps for Helping Patient-Users: Is It Still Far-Fetched?

Emergence of health-related smartphone applications and their wide dissemination in public as well as healthcare practitioners has undergone criticism under the scope of public health. Still, despite methodological issues curbing the initial enthusiasm, availability, safety and, in certain cases, documented efficacy of these measures has secured regulatory approval. Bearing in mind these pitfalls, we describe the necessary steps towards implementation of deep learning techniques in the specific clinical context of women’s health and infertility in particular.

Quality and Safety in eHealth: The Need to Build the Evidence Base

Research in the area of health technology safety has demonstrated that technology may both improve patient safety and introduce new types of technology-induced errors. Thus, there is a need to publish safety science literature to develop an evidence-based research base, on which we can continually develop new, safe technologies and improve patient safety. The aim of this viewpoint is to argue for the need to advance evidence-based research in health informatics, so that new technologies can be designed, developed, and implemented for their safety prior to their use in health care. This viewpoint offers a historical perspective on the development of health informatics and safety literature in the area of health technology. I argue for the need to conduct safety studies of technologies used by health professionals and consumers to develop an evidence base in this area. Ongoing research is necessary to improve the quality and safety of health technologies. Over the past several decades, we have seen health informatics emerge as a discipline, with growing research in the field examining the design, development, and implementation of different health technologies and new challenges such as those associated with the quality and safety of technology use. Future research will need to focus on how we can continually extend safety science in this area. There is a need to integrate evidence-based research into the design, development, and implementation of health technologies to improve their safety and reduce technology-induced errors.

Looking beyond the hype: Applied AI and machine learning in translational medicine

Big data problems are becoming more prevalent for laboratory scientists who look to make clinical impact. A large part of this is due to increased computing power, in parallel with new technologies for high quality data generation. Both new and old techniques of artificial intelligence (AI) and machine learning (ML) can now help increase the success of translational studies in three areas: drug discovery, imaging, and genomic medicine. However, ML technologies do not come without their limitations and shortcomings. Current technical limitations and other limitations including governance, reproducibility, and interpretation will be discussed in this article. Overcoming these limitations will enable ML methods to be more powerful for discovery and reduce ambiguity within translational medicine, allowing data-informed decision-making to deliver the next generation of diagnostics and therapeutics to patients quicker, at lowered costs, and at scale.