Diagnostic quality model (DQM): an integrated framework for the assessment of diagnostic quality when using AI/ML

Translating the regulatory landscape of medical devices to create fit-for-purpose artificial intelligence (AI) cytometry solutions

The implementation of medical software and artificial intelligence (AI) algorithms into routine clinical cytometry diagnostic practice requires a thorough understanding of regulatory requirements and challenges throughout the cytometry software product lifecycle. To provide cytometry software developers, computational scientists, researchers, industry professionals, and diagnostic physicians/pathologists with an introduction to European Union (EU) and United States (US) regulatory frameworks. Informed by community feedback and needs assessment established during two international cytometry workshops, this article provides an overview of regulatory landscapes as they pertain to the application of AI, AI-enabled medical devices, and Software as a Medical Device in diagnostic flow cytometry. Evolving regulatory frameworks are discussed, and specific examples regarding cytometry instruments, analysis software and clinical flow cytometry in-vitro diagnostic assays are provided. An important consideration for cytometry software development is the modular approach. As such, modules can be segregated and treated as independent components based on the medical purpose and risk and become subjected to a range of context-dependent compliance and regulatory requirements throughout their life cycle. Knowledge of regulatory and compliance requirements enhances the communication and collaboration between developers, researchers, end-users and regulators. This connection is essential to translate scientific innovation into diagnostic practice and to continue to shape the development and revision of new policies, standards, and approaches.

Analytical Performance Specifications for Input Variables: Investigation of the Model of End-Stage Liver Disease

BACKGROUND

Artificial intelligence models constitute specific uses of analysis results and, therefore, necessitate evaluation of analytical performance specifications (APS) for this context specifically. The Model of End-stage Liver Disease (MELD) is a clinical prediction model based on measurements of bilirubin, creatinine, and the international normalized ratio (INR). This study evaluates the propagation of error through the MELD, to inform choice of APS for the MELD input variables.

METHODS

A total of 6093 consecutive MELD scores and underlying analysis results were retrospectively collected. "Desirable analytical variation" based on biological variation as well as current local analytical variation was simulated onto the data set as well as onto a constructed data set, representing a worst-case scenario. Resulting changes in MELD score and risk classification were calculated.

RESULTS

Biological variation-based APS in the worst-case scenario resulted in 3.26% of scores changing by ≥1 MELD point. In the patient-derived data set, the same variation resulted in 0.92% of samples changing by ≥1 MELD point, and 5.5% of samples changing risk category. Local analytical performance resulted in lower reclassification rates.

CONCLUSIONS

Error propagation through MELD is complex and includes population-dependent mechanisms. Biological variation-derived APS were acceptable for all uses of the MELD score. Other combinations of APS can yield equally acceptable results. This analysis exemplifies how error propagation through artificial intelligence models can become highly complex. This complexity will necessitate that both model suppliers and clinical laboratories address analytical performance specifications for the specific use case, as these may differ from performance specifications for traditional use of the analyses.

Insights from semi-structured interviews on integrating artificial intelligence in clinical chemistry laboratory practices

BACKGROUND

Artificial intelligence (AI) is gradually transforming the practises of healthcare providers. Over the last two decades, the advent of AI into numerous aspects of pathology has opened transformative possibilities in how we practise laboratory medicine. Objectives of this study were to explore how AI could impact the clinical practices of professionals working in Clinical Chemistry laboratories, while also identifying effective strategies in medical education to facilitate the required changes.

METHODS

From March to August 2022, an exploratory qualitative study was conducted at the Section of Clinical Chemistry, Department of Pathology and Laboratory Medicine, Aga Khan University, Karachi, Pakistan, in collaboration with Keele University, Newcastle, United Kingdom. Semi-structured interviews were conducted to collect information from diverse group of professionals working in Clinical Chemistry laboratories. All interviews were audio recorded and transcribed verbatim. They were asked what changes AI would involve in the laboratory, what resources would be necessary, and how medical education would assist them in adapting to the change. A content analysis was conducted, resulting in the development of codes and themes based on the analyzed data.

RESULTS

The interviews were analysed to identify three primary themes: perspectives and considerations for AI adoption, educational and curriculum adjustments, and implementation techniques. Although the use of diagnostic algorithms is currently limited in Pakistani Clinical Chemistry laboratories, the application of AI is expanding. All thirteen participants stated their reasons for being hesitant to use AI. Participants stressed the importance of critical aspects for effective AI deployment, the need of a collaborative integrative approach, and the need for constant horizon scanning to keep up with AI developments.

CONCLUSIONS

Three primary themes related to AI adoption were identified: perspectives and considerations, educational and curriculum adjustments, and implementation techniques. The study's findings give a sound foundation for making suggestions to clinical laboratories, scientific bodies, and national and international Clinical Chemistry and laboratory medicine organisations on how to manage pathologists' shifting practises because of AI.

Artificial intelligence in the risk prediction models of cardiovascular disease and development of an independent validation screening tool: a systematic review

BACKGROUND

A comprehensive overview of artificial intelligence (AI) for cardiovascular disease (CVD) prediction and a screening tool of AI models (AI-Ms) for independent external validation are lacking. This systematic review aims to identify, describe, and appraise AI-Ms of CVD prediction in the general and special populations and develop a new independent validation score (IVS) for AI-Ms replicability evaluation.

METHODS

PubMed, Web of Science, Embase, and IEEE library were searched up to July 2021. Data extraction and analysis were performed for the populations, distribution, predictors, algorithms, etc. The risk of bias was evaluated with the prediction risk of bias assessment tool (PROBAST). Subsequently, we designed IVS for model replicability evaluation with five steps in five items, including transparency of algorithms, performance of models, feasibility of reproduction, risk of reproduction, and clinical implication, respectively. The review is registered in PROSPERO (No. CRD42021271789).

RESULTS

In 20,887 screened references, 79 articles (82.5% in 2017-2021) were included, which contained 114 datasets (67 in Europe and North America, but 0 in Africa). We identified 486 AI-Ms, of which the majority were in development (n = 380), but none of them had undergone independent external validation. A total of 66 idiographic algorithms were found; however, 36.4% were used only once and only 39.4% over three times. A large number of different predictors (range 5-52,000, median 21) and large-span sample size (range 80-3,660,000, median 4466) were observed. All models were at high risk of bias according to PROBAST, primarily due to the incorrect use of statistical methods. IVS analysis confirmed only 10 models as "recommended"; however, 281 and 187 were "not recommended" and "warning," respectively.

CONCLUSION

AI has led the digital revolution in the field of CVD prediction, but is still in the early stage of development as the defects of research design, report, and evaluation systems. The IVS we developed may contribute to independent external validation and the development of this field.

Diagnostic Test Utilization Management Strategies as an Opportunity for Equitable Access to Molecularly Informed Clinical Care

BACKGROUND

Companion diagnostics are an essential component of oncology. Timing, cost, and adaptability to new drug/biomarker approvals represent challenges in assuring value-based care. Overcoming these challenges requires strategies for equitable access and efficient integration.

METHODS

Based on prior laboratory improvements and payor policy implementations, we define equitable access in laboratory testing and conceptualized a framework for initiatives that optimize diagnostic performance.

RESULTS

We define equitable access as an imperative goal seeking to remove disparities that may arise due to financial hardships, geographical isolation, cultural differences, or other social determinants of health. We distinguish (a) utilization, as the practice pattern of ordered tests, (b) utilization management, as the evidence-based guidance of the utilization decisions, and (c) utilization management strategies, defined as the tools and techniques used to influence decision-making. These 3 dimensions establish a standardized vocabulary to clarify equitable alignment of strategies in specific care pathways. Alignment of logistic, administrative, and financial incentive structures is paramount when creating sustainable personalized care pathway programs.

CONCLUSIONS

Strategies to accomplish equitable and meaningful use of diagnostic tests can help enhance access to timely and accurate diagnoses, ultimately leading to improved patient outcomes.

Computational pathology in cancer diagnosis, prognosis, and prediction - present day and prospects

Computational pathology refers to applying deep learning techniques and algorithms to analyse and interpret histopathology images. Advances in artificial intelligence (AI) have led to an explosion in innovation in computational pathology, ranging from the prospect of automation of routine diagnostic tasks to the discovery of new prognostic and predictive biomarkers from tissue morphology. Despite the promising potential of computational pathology, its integration in clinical settings has been limited by a range of obstacles including operational, technical, regulatory, ethical, financial, and cultural challenges. Here, we focus on the pathologists' perspective of computational pathology: we map its current translational research landscape, evaluate its clinical utility, and address the more common challenges slowing clinical adoption and implementation. We conclude by describing contemporary approaches to drive forward these techniques. © 2023 The Authors. The Journal of Pathology published by John Wiley & Sons Ltd on behalf of The Pathological Society of Great Britain and Ireland.

Artificial Intelligence and laboratory data in rheumatic diseases

Artificial intelligence (AI)-based medical technologies are rapidly evolving into actionable solutions for clinical practice. Machine learning (ML) algorithms can process increasing amounts of laboratory data such as gene expression immunophenotyping data and biomarkers. In recent years, the analysis of ML has become particularly useful for the study of complex chronic diseases, such as rheumatic diseases, heterogenous conditions with multiple triggers. Numerous studies have used ML to classify patients and improve diagnosis, to stratify the risk and determine disease subtypes, as well as to discover biomarkers and gene signatures. This review aims to provide examples of ML models for specific rheumatic diseases using laboratory data and some insights into relevant strengths and limitations. A better understanding and future application of these analytical strategies could facilitate the development of precision medicine for rheumatic patients.