Development of an Artificial Intelligence-Based Automated Recommendation System for Clinical Laboratory Tests: Retrospective Analysis of the National Health Insurance Database

Integrating ensemble and machine learning models for early prediction of pneumonia mortality using laboratory tests

Background

The recent use of artificial intelligence (AI) in medical research is noteworthy. However, most research has focused on medical imaging. Although the importance of laboratory tests in the clinical field is acknowledged by clinicians, they are undervalued in medical AI research. Our study aims to develop an early prediction AI model for pneumonia mortality, primarily using laboratory test results.

Materials and methods

We developed a mortality prediction model using initial laboratory results and basic clinical information of patients with pneumonia. Several machine learning (ML) models and a deep learning method-multilayer perceptron (MLP)-were selected for model development. The area under the receiver operating characteristic curve (AUROC) and F1-score were optimized to improve model performance. In addition, an ensemble model was developed by blending several models to improve the prediction performance. We used 80,940 data instances for model development.

Results

Among the ML models, XGBoost exhibited the best performance (AUROC = 0.8989, accuracy = 0.88, F1-score = 0.80). MLP achieved an AUROC of 0.8498, accuracy of 0.86, and F1-score of 0.75. The performance of the ensemble model was the best among the developed models, with an AUROC of 0.9006, accuracy of 0.90, and F1-score of 0.81. Several laboratory tests were conducted to identify risk factors that affect pneumonia mortality using the "Feature importance" technique and SHapley Additive exPlanations. We identified several laboratory results, including systolic blood pressure, serum glucose level, age, aspartate aminotransferase-to-alanine aminotransferase ratio, and monocyte-to-lymphocyte ratio, as significant predictors of mortality in patients with pneumonia.

Conclusions

Our study demonstrates that the ensemble model, incorporating XGBoost, CatBoost, and LGBM techniques, outperforms individual ML and deep learning models in predicting pneumonia mortality. Our findings emphasize the importance of integrating AI techniques to leverage laboratory test data effectively, offering a promising direction for advancing AI applications in medical research and clinical decision-making.

Working with the Electronic Health Record and Laboratory Information System to Maximize Ordering and Reporting of Molecular Microbiology Results

Molecular microbiology assays have a higher cost of testing compared to traditional methods and need to be utilized appropriately. Results from these assays may also require interpretation and appropriate follow-up. Electronic tools available in the electronic health record and laboratory information system can be deployed both preanalytically and postanalytically to influence ordering behaviors and positively impact diagnostic stewardship. Next generation technologies, such as machine learning and artificial intelligence, have the potential to expand upon the capabilities currently available and warrant additional study and development but also require regulation around their use in health care.

The application of artificial intelligence in health financing: a scoping review

INTRODUCTION

Artificial Intelligence (AI) represents a significant advancement in technology, and it is crucial for policymakers to incorporate AI thinking into policies and to fully explore, analyze and utilize massive data and conduct AI-related policies. AI has the potential to optimize healthcare financing systems. This study provides an overview of the AI application domains in healthcare financing.

METHOD

We conducted a scoping review in six steps: formulating research questions, identifying relevant studies by conducting a comprehensive literature search using appropriate keywords, screening titles and abstracts for relevance, reviewing full texts of relevant articles, charting extracted data, and compiling and summarizing findings. Specifically, the research question sought to identify the applications of artificial intelligence in health financing supported by the published literature and explore potential future applications. PubMed, Scopus, and Web of Science databases were searched between 2000 and 2023.

RESULTS

We discovered that AI has a significant impact on various aspects of health financing, such as governance, revenue raising, pooling, and strategic purchasing. We provide evidence-based recommendations for establishing and improving the health financing system based on AI.

CONCLUSIONS

To ensure that vulnerable groups face minimum challenges and benefit from improved health financing, we urge national and international institutions worldwide to use and adopt AI tools and applications.

Artificial Intelligence in Dementia: A Bibliometric Study

The applications of artificial intelligence (AI) in dementia research have garnered significant attention, prompting the planning of various research endeavors in current and future studies. The objective of this study is to provide a comprehensive overview of the research landscape regarding AI and dementia within scholarly publications and to suggest further studies for this emerging research field. A search was conducted in the Web of Science database to collect all relevant and highly cited articles on AI-related dementia research published in English until 16 May 2023. Utilizing bibliometric indicators, a search strategy was developed to assess the eligibility of titles, utilizing abstracts and full texts as necessary. The Bibliometrix tool, a statistical package in R, was used to produce and visualize networks depicting the co-occurrence of authors, research institutions, countries, citations, and keywords. We obtained a total of 1094 relevant articles published between 1997 and 2023. The number of annual publications demonstrated an increasing trend over the past 27 years. Journal of Alzheimer's Disease (39/1094, 3.56%), Frontiers in Aging Neuroscience (38/1094, 3.47%), and Scientific Reports (26/1094, 2.37%) were the most common journals for this domain. The United States (283/1094, 25.86%), China (222/1094, 20.29%), India (150/1094, 13.71%), and England (96/1094, 8.77%) were the most productive countries of origin. In terms of institutions, Boston University, Columbia University, and the University of Granada demonstrated the highest productivity. As for author contributions, Gorriz JM, Ramirez J, and Salas-Gonzalez D were the most active researchers. While the initial period saw a relatively low number of articles focusing on AI applications for dementia, there has been a noticeable upsurge in research within this domain in recent years (2018-2023). The present analysis sheds light on the key contributors in terms of researchers, institutions, countries, and trending topics that have propelled the advancement of AI in dementia research. These findings collectively underscore that the integration of AI with conventional treatment approaches enhances the effectiveness of dementia diagnosis, prediction, classification, and monitoring of treatment progress.

Department of Medicine 2040: Implementing a Constrained Disorder Principle-Based Second-Generation Artificial Intelligence System for Improved Patient Outcomes in the Department of Internal Medicine

Internal medicine departments must adapt their structures and methods of operation to accommodate changing healthcare systems. The present paper discusses some challenges departments of medicine face as healthcare providers and consumers continue to change. A co-pilot model is described in this article for augmenting physicians rather than replacing them. The paper presents the co-pilot models to improve diagnoses, treatments, and monitoring. Personalized variability patterns based on the constrained-disorder principle (CDP) are described to assess chronic therapies' effectiveness in improving patient outcomes. Based on CDP-based enhanced digital twins, this paper presents personalized treatments and follow-ups that improve diagnosis accuracy and therapy outcomes. While maintaining their professional values, departments of internal medicine must respond proactively to the needs of patients and healthcare systems. To meet the needs of patients and healthcare systems, they must strive for medical professionalism and adapt to the dynamic environment.

Diagnostic Mycology Laboratories Should Have a Central Role for the Management of Fungal Disease

The absence of awareness of fungal diseases as part of the differential diagnosis in at-risk populations has severe consequences. Here, we show how the active role of laboratories can improve patients’ survival. Recently, major advances have been made in non-culture-based assays for fungal diseases, improving accuracy and turnaround time. Furthermore, with the introduction of proficiency control systems, laboratories are an easily monitored environment with good analytical accuracy. Diagnostic packages for opportunistic infections can overcome many deficiencies caused by the absence of awareness. In Guatemala, to make diagnosis accessible, we set up a diagnostic laboratory hub (DLH) providing screening for cryptococcosis, histoplasmosis and tuberculosis to a network of 13 healthcare facilities attending people living with HIV (PLWHIV). In two years, we screened 2127 newly HIV-diagnosed patients. The frequency of opportunistic infections was 21%, rising to 30.3% in patients with advanced HIV disease (<200 CD4); 8.1% of these patients had more than one infection. With the implementation of this diagnostic package, mortality decreased by 7%, a key goal of many public health interventions. Screening for serious infection in high-risk populations can partially overcome training or experiential deficiencies among clinicians for life-threatening fungal diseases.

Applications of machine learning in routine laboratory medicine: Current state and future directions

Machine learning is able to leverage large amounts of data to infer complex patterns that are otherwise beyond the capabilities of rule-based systems and human experts. Its application to laboratory medicine is particularly exciting, as laboratory testing provides much of the foundation for clinical decision making. In this article, we provide a brief introduction to machine learning for the medical professional in addition to a comprehensive literature review outlining the current state of machine learning as it has been applied to routine laboratory medicine. Although still in its early stages, machine learning has been used to automate laboratory tasks, optimize utilization, and provide personalized reference ranges and test interpretation. The published literature leads us to believe that machine learning will be an area of increasing importance for the laboratory practitioner. We envision the laboratory of the future will utilize these methods to make significant improvements in efficiency and diagnostic precision.

A Data-Driven Algorithm to Recommend Initial Clinical Workup for Outpatient Specialty Referral: Algorithm Development and Validation Using Electronic Health Record Data and Expert Surveys

BACKGROUND

Millions of people have limited access to specialty care. The problem is exacerbated by ineffective specialty visits due to incomplete prereferral workup, leading to delays in diagnosis and treatment. Existing processes to guide prereferral diagnostic workup are labor-intensive (ie, building a consensus guideline between primary care doctors and specialists) and require the availability of the specialists (ie, electronic consultation).

OBJECTIVE

Using pediatric endocrinology as an example, we develop a recommender algorithm to anticipate patients' initial workup needs at the time of specialty referral and compare it to a reference benchmark using the most common workup orders. We also evaluate the clinical appropriateness of the algorithm recommendations.

METHODS

Electronic health record data were extracted from 3424 pediatric patients with new outpatient endocrinology referrals at an academic institution from 2015 to 2020. Using item co-occurrence statistics, we predicted the initial workup orders that would be entered by specialists and assessed the recommender's performance in a holdout data set based on what the specialists actually ordered. We surveyed endocrinologists to assess the clinical appropriateness of the predicted orders and to understand the initial workup process.

RESULTS

Specialists (n=12) indicated that <50% of new patient referrals arrive with complete initial workup for common referral reasons. The algorithm achieved an area under the receiver operating characteristic curve of 0.95 (95% CI 0.95-0.96). Compared to a reference benchmark using the most common orders, precision and recall improved from 37% to 48% (P<.001) and from 27% to 39% (P<.001) for the top 4 recommendations, respectively. The top 4 recommendations generated for common referral conditions (abnormal thyroid studies, obesity, amenorrhea) were considered clinically appropriate the majority of the time by specialists surveyed and practice guidelines reviewed.

CONCLUSIONS

 An item association-based recommender algorithm can predict appropriate specialists' workup orders with high discriminatory accuracy. This could support future clinical decision support tools to increase effectiveness and access to specialty referrals. Our study demonstrates important first steps toward a data-driven paradigm for outpatient specialty consultation with a tier of automated recommendations that proactively enable initial workup that would otherwise be delayed by awaiting an in-person visit.

Rise of the Machines: The Inevitable Evolution of Medicine and Medical Laboratories Intertwining with Artificial Intelligence-A Narrative Review

Laboratory medicine has evolved from a mainly manual profession, providing few selected test results to a highly automated and standardized medical discipline, generating millions of test results per year. As the next inevitable evolutional step, artificial intelligence (AI) algorithms will need to assist us in structuring and making sense of the masses of diagnostic data collected today. Such systems will be able to connect clinical and diagnostic data and to provide valuable suggestions in diagnosis, prognosis or therapeutic options. They will merge the often so separated worlds of the laboratory and the clinics. When used correctly, it will be a tool, capable of freeing the physicians time so that he/she can refocus on the patient. In this narrative review I therefore aim to provide an overview of what AI is, what applications currently are available in healthcare and in laboratory medicine in particular. I will discuss the challenges and pitfalls of applying AI algorithms and I will elaborate on the question if healthcare workers will be replaced by such systems in the near future.

Laboratory Demand Management Strategies-An Overview

Inappropriate laboratory test selection in the form of overutilization as well as underutilization frequently occurs despite available guidelines. There is broad approval among laboratory specialists as well as clinicians that demand management strategies are useful tools to avoid this issue. Most of these tools are based on automated algorithms or other types of machine learning. This review summarizes the available demand management strategies that may be adopted to local settings. We believe that artificial intelligence may help to further improve these available tools.