1
|
Castellana R, Fanni SC, Roncella C, Romei C, Natrella M, Neri E. Radiomics and deep learning models for CT pre-operative lymph node staging in pancreatic ductal adenocarcinoma: A systematic review and meta-analysis. Eur J Radiol 2024; 176:111510. [PMID: 38781919 DOI: 10.1016/j.ejrad.2024.111510] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/29/2024] [Revised: 04/23/2024] [Accepted: 05/13/2024] [Indexed: 05/25/2024]
Abstract
PURPOSE To evaluate the diagnostic accuracy of computed tomography (CT)-based radiomic algorithms and deep learning models to preoperatively identify lymph node metastasis (LNM) in patients with pancreatic ductal adenocarcinoma (PDAC). METHODS PubMed, CENTRAL, Scopus, Web of Science and IEEE databases were searched to identify relevant studies published up until February 11, 2024. Two reviewers screened all papers independently for eligibility. Studies reporting the accuracy of CT-based radiomics or deep learning models for detecting LNM in PDAC, using histopathology as the reference standard, were included. Quality was assessed using the Quality Assessment of Diagnostic Accuracy Studies 2, the Radiomics Quality Score (RQS) and the the METhodological RadiomICs Score (METRICS). Overall sensitivity (SE), specificity (SP), diagnostic odds ratio (DOR), and the area under the curve (AUC) were calculated. RESULTS Four radiomics studies comprising 213 patients and four deep learning studies with 272 patients were included. The average RQS total score was 12.00 ± 3.89, corresponding to an RQS percentage of 33.33 ± 10.80, while the average METRICS score was 63.60 ± 10.88. A significant and strong positive correlation was found between RQS and METRICS (p = 0.016; r = 0.810). The pooled SE, SP, DOR, and AUC of all the studies were 0.83 (95 %CI = 0.77-0.88), 0.76 (95 %CI = 0.62-0.86), 15.70 (95 %CI = 8.12-27.50) and 0.85 (95 %CI = 0.77-0.88). Meta-regression analysis results indicated that neither the study type (radiomics vs deep learning) nor the dataset size of the studies had a significant effect on the DOR (p = 0.09 and p = 0.26, respectively). CONCLUSION Based on our meta-analysis findings, preoperative CT-based radiomics algorithms and deep learning models demonstrate favorable performance in predicting LNM in patients with PDAC, with a strong correlation between RQS and METRICS of the included studies.
Collapse
Affiliation(s)
- Roberto Castellana
- Diagnostic and Interventional Radiology, "Parini" Regional Hospital, Azienda USL della Valle d'Aosta, Viale Ginevra 3 11100, Aosta, Italy.
| | - Salvatore Claudio Fanni
- Department of Translational Research, Academic Radiology, University of Pisa, Via Paradisa 2, 56124 Pisa, Italy
| | - Claudia Roncella
- Radiology Unit, Apuane Hospital, Azienda USL Toscana Nord Ovest, Via Mattei 21, 54100, Massa, Italy
| | - Chiara Romei
- Department of Diagnostic Imaging, Diagnostic Radiology 2, Pisa University Hospital, Via Paradisa 2, 56124, Pisa, Italy
| | - Massimiliano Natrella
- Diagnostic and Interventional Radiology, "Parini" Regional Hospital, Azienda USL della Valle d'Aosta, Viale Ginevra 3 11100, Aosta, Italy
| | - Emanuele Neri
- Department of Translational Research, Academic Radiology, University of Pisa, Via Paradisa 2, 56124 Pisa, Italy
| |
Collapse
|
2
|
Kondylakis H, Catalan R, Alabart SM, Barelle C, Bizopoulos P, Bobowicz M, Bona J, Fotiadis DI, Garcia T, Gomez I, Jimenez-Pastor A, Karatzanis G, Lekadir K, Kogut-Czarkowska M, Lalas A, Marias K, Marti-Bonmati L, Munuera J, Nikiforaki K, Pelissier M, Prior F, Rutherford M, Saint-Aubert L, Sakellariou Z, Seymour K, Trouillard T, Votis K, Tsiknakis M. Documenting the de-identification process of clinical and imaging data for AI for health imaging projects. Insights Imaging 2024; 15:130. [PMID: 38816658 PMCID: PMC11139818 DOI: 10.1186/s13244-024-01711-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/16/2024] [Accepted: 04/26/2024] [Indexed: 06/01/2024] Open
Abstract
Artificial intelligence (AI) is revolutionizing the field of medical imaging, holding the potential to shift medicine from a reactive "sick-care" approach to a proactive focus on healthcare and prevention. The successful development of AI in this domain relies on access to large, comprehensive, and standardized real-world datasets that accurately represent diverse populations and diseases. However, images and data are sensitive, and as such, before using them in any way the data needs to be modified to protect the privacy of the patients. This paper explores the approaches in the domain of five EU projects working on the creation of ethically compliant and GDPR-regulated European medical imaging platforms, focused on cancer-related data. It presents the individual approaches to the de-identification of imaging data, and describes the problems and the solutions adopted in each case. Further, lessons learned are provided, enabling future projects to optimally handle the problem of data de-identification. CRITICAL RELEVANCE STATEMENT: This paper presents key approaches from five flagship EU projects for the de-identification of imaging and clinical data offering valuable insights and guidelines in the domain. KEY POINTS: ΑΙ models for health imaging require access to large amounts of data. Access to large imaging datasets requires an appropriate de-identification process. This paper provides de-identification guidelines from the AI for health imaging (AI4HI) projects.
Collapse
Affiliation(s)
| | - Rocio Catalan
- La Fe University and Polytechnic Hospital, La Fe Health Research Institute, Valencia, Spain
| | | | | | - Paschalis Bizopoulos
- Centre for Research & Technology Hellas, Information Technologies Institute (CERTH-ITI), Central Directorate, Thermi, Thessaloniki, Greece
| | | | - Jonathan Bona
- University of Arkansas for Medical Sciences, Little Rock, AR, USA
| | - Dimitrios I Fotiadis
- Unit of Medical Technology and Intelligent Information Systems, Department of Materials Science and Engineering, University of Ioannina, Ioannina, Greece
| | - Teresa Garcia
- Centre for Genomic Regulation (CRG), The Barcelona Institute of Science and Technology, Barcelona, Spain
| | - Ignacio Gomez
- La Fe University and Polytechnic Hospital, La Fe Health Research Institute, Valencia, Spain
| | | | | | - Karim Lekadir
- Artificial Intelligence in Medicine Labm Universitat de Barcelona, Barcelona, Spain
| | | | - Antonios Lalas
- Centre for Research & Technology Hellas, Information Technologies Institute (CERTH-ITI), Central Directorate, Thermi, Thessaloniki, Greece
| | | | - Luis Marti-Bonmati
- Hospital Universitario y Politécnico La Fe, Grupo de Investigación Biomédica en Imagen IIS La Fe, Valencia, España
| | - Jose Munuera
- Quantitative Imaging Biomarkers in Medicine, Quibim, Valencia, Spain
| | | | | | - Fred Prior
- University of Arkansas for Medical Sciences, Little Rock, AR, USA
| | | | | | - Zisis Sakellariou
- Centre for Research & Technology Hellas, Information Technologies Institute (CERTH-ITI), Central Directorate, Thermi, Thessaloniki, Greece
| | | | | | - Konstantinos Votis
- Centre for Research & Technology Hellas, Information Technologies Institute (CERTH-ITI), Central Directorate, Thermi, Thessaloniki, Greece
| | | |
Collapse
|
3
|
Crombé A, Spinnato P, Italiano A, Brisse HJ, Feydy A, Fadli D, Kind M. Radiomics and artificial intelligence for soft-tissue sarcomas: Current status and perspectives. Diagn Interv Imaging 2023; 104:567-583. [PMID: 37802753 DOI: 10.1016/j.diii.2023.09.005] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/10/2023] [Revised: 09/18/2023] [Accepted: 09/19/2023] [Indexed: 10/08/2023]
Abstract
This article proposes a summary of the current status of the research regarding the use of radiomics and artificial intelligence to improve the radiological assessment of patients with soft tissue sarcomas (STS), a heterogeneous group of rare and ubiquitous mesenchymal malignancies. After a first part explaining the principle of radiomics approaches, from raw image post-processing to extraction of radiomics features mined with unsupervised and supervised machine-learning algorithms, and the current research involving deep learning algorithms in STS, especially convolutional neural networks, this review details their main research developments since the formalisation of 'radiomics' in oncologic imaging in 2010. This review focuses on CT and MRI and does not involve ultrasonography. Radiomics and deep radiomics have been successfully applied to develop predictive models to discriminate between benign soft-tissue tumors and STS, to predict the histologic grade (i.e., the most important prognostic marker of STS), the response to neoadjuvant chemotherapy and/or radiotherapy, and the patients' survivals and probability for presenting distant metastases. The main findings, limitations and expectations are discussed for each of these outcomes. Overall, after a first decade of publications emphasizing the potential of radiomics through retrospective proof-of-concept studies, almost all positive but with heterogeneous and often non-replicable methods, radiomics is now at a turning point in order to provide robust demonstrations of its clinical impact through open-science, independent databases, and application of good and standardized practices in radiomics such as those provided by the Image Biomarker Standardization Initiative, without forgetting innovative research paths involving other '-omics' data to better understand the relationships between imaging of STS, gene-expression profiles and tumor microenvironment.
Collapse
Affiliation(s)
- Amandine Crombé
- Department of Radiology, Pellegrin University Hospital, 33000 Bordeaux, France; Department of Oncologic Imaging, Bergonié Institute, 33076 Bordeaux, France; 'Sarcotarget' team, BRIC INSERM U1312 and Bordeaux University, 33000 Bordeaux France.
| | - Paolo Spinnato
- Diagnostic and Interventional Radiology, IRCCS Istituto Ortopedico Rizzoli, Bologna 40136, Italy
| | | | | | - Antoine Feydy
- Department of Radiology, Hopital Cochin-AP-HP, 75014 Paris, France; Université Paris Cité, Faculté de Médecine, 75006 Paris, France
| | - David Fadli
- Department of Radiology, Pellegrin University Hospital, 33000 Bordeaux, France
| | - Michèle Kind
- Department of Oncologic Imaging, Bergonié Institute, 33076 Bordeaux, France
| |
Collapse
|
4
|
Mello-Thoms C, Mello CAB. Clinical applications of artificial intelligence in radiology. Br J Radiol 2023; 96:20221031. [PMID: 37099398 PMCID: PMC10546456 DOI: 10.1259/bjr.20221031] [Citation(s) in RCA: 6] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/03/2022] [Revised: 03/28/2023] [Accepted: 03/28/2023] [Indexed: 04/27/2023] Open
Abstract
The rapid growth of medical imaging has placed increasing demands on radiologists. In this scenario, artificial intelligence (AI) has become an attractive partner, one that may complement case interpretation and may aid in various non-interpretive aspects of the work in the radiological clinic. In this review, we discuss interpretative and non-interpretative uses of AI in the clinical practice, as well as report on the barriers to AI's adoption in the clinic. We show that AI currently has a modest to moderate penetration in the clinical practice, with many radiologists still being unconvinced of its value and the return on its investment. Moreover, we discuss the radiologists' liabilities regarding the AI decisions, and explain how we currently do not have regulation to guide the implementation of explainable AI or of self-learning algorithms.
Collapse
Affiliation(s)
| | - Carlos A B Mello
- Centro de Informática, Universidade Federal de Pernambuco, Recife, Brazil
| |
Collapse
|
5
|
Kondylakis H, Kalokyri V, Sfakianakis S, Marias K, Tsiknakis M, Jimenez-Pastor A, Camacho-Ramos E, Blanquer I, Segrelles JD, López-Huguet S, Barelle C, Kogut-Czarkowska M, Tsakou G, Siopis N, Sakellariou Z, Bizopoulos P, Drossou V, Lalas A, Votis K, Mallol P, Marti-Bonmati L, Alberich LC, Seymour K, Boucher S, Ciarrocchi E, Fromont L, Rambla J, Harms A, Gutierrez A, Starmans MPA, Prior F, Gelpi JL, Lekadir K. Data infrastructures for AI in medical imaging: a report on the experiences of five EU projects. Eur Radiol Exp 2023; 7:20. [PMID: 37150779 PMCID: PMC10164664 DOI: 10.1186/s41747-023-00336-x] [Citation(s) in RCA: 8] [Impact Index Per Article: 8.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/25/2023] [Accepted: 03/02/2023] [Indexed: 05/09/2023] Open
Abstract
Artificial intelligence (AI) is transforming the field of medical imaging and has the potential to bring medicine from the era of 'sick-care' to the era of healthcare and prevention. The development of AI requires access to large, complete, and harmonized real-world datasets, representative of the population, and disease diversity. However, to date, efforts are fragmented, based on single-institution, size-limited, and annotation-limited datasets. Available public datasets (e.g., The Cancer Imaging Archive, TCIA, USA) are limited in scope, making model generalizability really difficult. In this direction, five European Union projects are currently working on the development of big data infrastructures that will enable European, ethically and General Data Protection Regulation-compliant, quality-controlled, cancer-related, medical imaging platforms, in which both large-scale data and AI algorithms will coexist. The vision is to create sustainable AI cloud-based platforms for the development, implementation, verification, and validation of trustable, usable, and reliable AI models for addressing specific unmet needs regarding cancer care provision. In this paper, we present an overview of the development efforts highlighting challenges and approaches selected providing valuable feedback to future attempts in the area.Key points• Artificial intelligence models for health imaging require access to large amounts of harmonized imaging data and metadata.• Main infrastructures adopted either collect centrally anonymized data or enable access to pseudonymized distributed data.• Developing a common data model for storing all relevant information is a challenge.• Trust of data providers in data sharing initiatives is essential.• An online European Union meta-tool-repository is a necessity minimizing effort duplication for the various projects in the area.
Collapse
Affiliation(s)
| | | | | | - Kostas Marias
- FORTH-ICS, FORTH-ICS, N. Plastira 100, Heraklion, Crete, Greece
| | | | | | | | | | | | | | | | | | - Gianna Tsakou
- MAGGIOLI S.P.A., Research and Development Lab, Marousi, Greece
| | - Nikolaos Siopis
- Centre of Research & Technology - Hellas, Information Technologies Institute, Thermi - Thessaloniki, Greece
| | - Zisis Sakellariou
- Centre of Research & Technology - Hellas, Information Technologies Institute, Thermi - Thessaloniki, Greece
| | - Paschalis Bizopoulos
- Centre of Research & Technology - Hellas, Information Technologies Institute, Thermi - Thessaloniki, Greece
| | - Vicky Drossou
- Centre of Research & Technology - Hellas, Information Technologies Institute, Thermi - Thessaloniki, Greece
| | - Antonios Lalas
- Centre of Research & Technology - Hellas, Information Technologies Institute, Thermi - Thessaloniki, Greece
| | - Konstantinos Votis
- Centre of Research & Technology - Hellas, Information Technologies Institute, Thermi - Thessaloniki, Greece
| | - Pedro Mallol
- La Fe Health Research Institute, Valencia, Spain
| | | | | | | | | | | | - Lauren Fromont
- European Genome-Phenome Archive, Centre for Genomic Regulation, Barcelona, Spain
| | - Jordi Rambla
- European Genome-Phenome Archive, Centre for Genomic Regulation, Barcelona, Spain
| | | | | | | | - Fred Prior
- Department of Biomedical Informatics, University of Arkansas for Medical Sciences, Little Rock, AR, USA
| | | | | |
Collapse
|
6
|
Gorman C, Punzo D, Octaviano I, Pieper S, Longabaugh WJR, Clunie DA, Kikinis R, Fedorov AY, Herrmann MD. Interoperable slide microscopy viewer and annotation tool for imaging data science and computational pathology. Nat Commun 2023; 14:1572. [PMID: 36949078 PMCID: PMC10033920 DOI: 10.1038/s41467-023-37224-2] [Citation(s) in RCA: 5] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/09/2022] [Accepted: 03/08/2023] [Indexed: 03/24/2023] Open
Abstract
The exchange of large and complex slide microscopy imaging data in biomedical research and pathology practice is impeded by a lack of data standardization and interoperability, which is detrimental to the reproducibility of scientific findings and clinical integration of technological innovations. We introduce Slim, an open-source, web-based slide microscopy viewer that implements the internationally accepted Digital Imaging and Communications in Medicine (DICOM) standard to achieve interoperability with a multitude of existing medical imaging systems. We showcase the capabilities of Slim as the slide microscopy viewer of the NCI Imaging Data Commons and demonstrate how the viewer enables interactive visualization of traditional brightfield microscopy and highly-multiplexed immunofluorescence microscopy images from The Cancer Genome Atlas and Human Tissue Atlas Network, respectively, using standard DICOMweb services. We further show how Slim enables the collection of standardized image annotations for the development or validation of machine learning models and the visual interpretation of model inference results in the form of segmentation masks, spatial heat maps, or image-derived measurements.
Collapse
Affiliation(s)
- Chris Gorman
- Department of Pathology, Massachusetts General Hospital and Harvard Medical School, Boston, MA, USA
| | | | | | | | | | | | - Ron Kikinis
- Department of Radiology, Brigham and Women's Hospital and Harvard Medical School, Boston, MA, USA
| | - Andrey Y Fedorov
- Department of Radiology, Brigham and Women's Hospital and Harvard Medical School, Boston, MA, USA.
| | - Markus D Herrmann
- Department of Pathology, Massachusetts General Hospital and Harvard Medical School, Boston, MA, USA.
| |
Collapse
|
7
|
Cerdá-Alberich L, Solana J, Mallol P, Ribas G, García-Junco M, Alberich-Bayarri A, Marti-Bonmati L. MAIC-10 brief quality checklist for publications using artificial intelligence and medical images. Insights Imaging 2023; 14:11. [PMID: 36645542 PMCID: PMC9842808 DOI: 10.1186/s13244-022-01355-9] [Citation(s) in RCA: 11] [Impact Index Per Article: 11.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/07/2022] [Accepted: 12/20/2022] [Indexed: 01/17/2023] Open
Abstract
The use of artificial intelligence (AI) with medical images to solve clinical problems is becoming increasingly common, and the development of new AI solutions is leading to more studies and publications using this computational technology. As a novel research area, the use of common standards to aid AI developers and reviewers as quality control criteria will improve the peer review process. Although some guidelines do exist, their heterogeneity and extension advocate that more explicit and simple schemes should be applied on the publication practice. Based on a review of existing AI guidelines, a proposal which collects, unifies, and simplifies the most relevant criteria was developed. The MAIC-10 (Must AI Criteria-10) checklist with 10 items was implemented as a guide to design studies and evaluate publications related to AI in the field of medical imaging. Articles published in Insights into Imaging in 2021 were selected to calculate their corresponding MAIC-10 quality score. The mean score was found to be 5.6 ± 1.6, with critical items present in most articles, such as "Clinical need", "Data annotation", "Robustness", and "Transparency" present in more than 80% of papers, while improvements in other areas were identified. MAIC-10 was also observed to achieve the highest intra-observer reproducibility when compared to other existing checklists, with an overall reduction in terms of checklist length and complexity. In summary, MAIC-10 represents a short and simple quality assessment tool which is objective, robust and widely applicable to AI studies in medical imaging.
Collapse
Affiliation(s)
- Leonor Cerdá-Alberich
- grid.84393.350000 0001 0360 9602Clinical Medical Imaging Area and Biomedical Imaging Research Group (GIBI230-PREBI), Hospital Universitario y Politécnico La Fe – Instituto de Investigación Sanitaria La Fe, Valencia, Spain
| | - Jimena Solana
- grid.84393.350000 0001 0360 9602Clinical Medical Imaging Area and Biomedical Imaging Research Group (GIBI230-PREBI), Hospital Universitario y Politécnico La Fe – Instituto de Investigación Sanitaria La Fe, Valencia, Spain
| | - Pedro Mallol
- grid.84393.350000 0001 0360 9602Clinical Medical Imaging Area and Biomedical Imaging Research Group (GIBI230-PREBI), Hospital Universitario y Politécnico La Fe – Instituto de Investigación Sanitaria La Fe, Valencia, Spain
| | - Gloria Ribas
- grid.84393.350000 0001 0360 9602Clinical Medical Imaging Area and Biomedical Imaging Research Group (GIBI230-PREBI), Hospital Universitario y Politécnico La Fe – Instituto de Investigación Sanitaria La Fe, Valencia, Spain
| | - Miguel García-Junco
- grid.84393.350000 0001 0360 9602Clinical Medical Imaging Area and Biomedical Imaging Research Group (GIBI230-PREBI), Hospital Universitario y Politécnico La Fe – Instituto de Investigación Sanitaria La Fe, Valencia, Spain
| | - Angel Alberich-Bayarri
- grid.84393.350000 0001 0360 9602Clinical Medical Imaging Area and Biomedical Imaging Research Group (GIBI230-PREBI), Hospital Universitario y Politécnico La Fe – Instituto de Investigación Sanitaria La Fe, Valencia, Spain ,Quantitative Imaging Biomarkers in Medicine, Quibim SL, Valencia, Spain
| | - Luis Marti-Bonmati
- grid.84393.350000 0001 0360 9602Clinical Medical Imaging Area and Biomedical Imaging Research Group (GIBI230-PREBI), Hospital Universitario y Politécnico La Fe – Instituto de Investigación Sanitaria La Fe, Valencia, Spain
| |
Collapse
|
8
|
Eysenbach G, Leung T, Schneider G, Heinze O. Exploring Stakeholder Requirements to Enable the Research and Development of Artificial Intelligence Algorithms in a Hospital-Based Generic Infrastructure: Protocol for a Multistep Mixed Methods Study. JMIR Res Protoc 2022; 11:e42208. [PMID: 36525300 PMCID: PMC9804098 DOI: 10.2196/42208] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/26/2022] [Revised: 10/12/2022] [Accepted: 10/18/2022] [Indexed: 11/05/2022] Open
Abstract
BACKGROUND In recent years, research and developments in advancing artificial intelligence (AI) in health care and medicine have increased. High expectations surround the use of AI technologies, such as improvements for diagnosis and increases in the quality of care with reductions in health care costs. The successful development and testing of new AI algorithms require large amounts of high-quality data. Academic hospitals could provide the data needed for AI development, but granting legal, controlled, and regulated access to these data for developers and researchers is difficult. Therefore, the German Federal Ministry of Health supports the Protected Artificial Intelligence Innovation Environment for Patient-Oriented Digital Health Solutions for Developing, Testing, and Evidence-Based Evaluation of Clinical Value (pAItient) project, aiming to install the AI Innovation Environment at the Heidelberg University Hospital in Germany. The AI Innovation Environment was designed as a proof-of-concept extension of the already existing Medical Data Integration Center. It will establish a process to support every step of developing and testing AI-based technologies. OBJECTIVE The first part of the pAItient project, as presented in this research protocol, aims to explore stakeholders' requirements for developing AI in partnership with an academic hospital and granting AI experts access to anonymized personal health data. METHODS We planned a multistep mixed methods approach. In the first step, researchers and employees from stakeholder organizations were invited to participate in semistructured interviews. In the following step, questionnaires were developed based on the participants' answers and distributed among the stakeholders' organizations to quantify qualitative findings and discover important aspects that were not mentioned by the interviewees. The questionnaires will be analyzed descriptively. In addition, patients and physicians were interviewed as well. No survey questionnaires were developed for this second group of participants. The study was approved by the Ethics Committee of the Heidelberg University Hospital (approval number: S-241/2021). RESULTS Data collection concluded in summer 2022. Data analysis is planned to start in fall 2022. We plan to publish the results in winter 2022 to 2023. CONCLUSIONS The results of our study will help in shaping the AI Innovation Environment at our academic hospital according to stakeholder requirements. With this approach, in turn, we aim to shape an AI environment that is effective and is deemed acceptable by all parties. INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID) DERR1-10.2196/42208.
Collapse
Affiliation(s)
| | | | - Gerd Schneider
- Institute of Medical Informatics, Heidelberg University Hospital, Heidelberg, Germany
| | - Oliver Heinze
- Institute of Medical Informatics, Heidelberg University Hospital, Heidelberg, Germany
| |
Collapse
|
9
|
Weinert L, Klass M, Schneider G, Heinze O. Exploring Stakeholder Requirements to enable research and development of AI algorithms in a hospital based generic infrastructure: Results of a Multi-step mixed-methods Study (Preprint). JMIR Form Res 2022; 7:e43958. [PMID: 37071450 PMCID: PMC10155093 DOI: 10.2196/43958] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/31/2022] [Revised: 01/29/2023] [Accepted: 02/22/2023] [Indexed: 02/24/2023] Open
Abstract
BACKGROUND Legal, controlled, and regulated access to high-quality data from academic hospitals currently poses a barrier to the development and testing of new artificial intelligence (AI) algorithms. To overcome this barrier, the German Federal Ministry of Health supports the "pAItient" (Protected Artificial Intelligence Innovation Environment for Patient Oriented Digital Health Solutions for developing, testing and evidence-based evaluation of clinical value) project, with the goal to establish an AI Innovation Environment at the Heidelberg University Hospital, Germany. It is designed as a proof-of-concept extension to the preexisting Medical Data Integration Center. OBJECTIVE The first part of the pAItient project aims to explore stakeholders' requirements for developing AI in partnership with an academic hospital and granting AI experts access to anonymized personal health data. METHODS We designed a multistep mixed methods approach. First, researchers and employees from stakeholder organizations were invited to participate in semistructured interviews. In the following step, questionnaires were developed based on the participants' answers and distributed among the stakeholders' organizations. In addition, patients and physicians were interviewed. RESULTS The identified requirements covered a wide range and were conflicting sometimes. Relevant patient requirements included adequate provision of necessary information for data use, clear medical objective of the research and development activities, trustworthiness of the organization collecting the patient data, and data should not be reidentifiable. Requirements of AI researchers and developers encompassed contact with clinical users, an acceptable user interface (UI) for shared data platforms, stable connection to the planned infrastructure, relevant use cases, and assistance in dealing with data privacy regulations. In a next step, a requirements model was developed, which depicts the identified requirements in different layers. This developed model will be used to communicate stakeholder requirements within the pAItient project consortium. CONCLUSIONS The study led to the identification of necessary requirements for the development, testing, and validation of AI applications within a hospital-based generic infrastructure. A requirements model was developed, which will inform the next steps in the development of an AI innovation environment at our institution. Results from our study replicate previous findings from other contexts and will add to the emerging discussion on the use of routine medical data for the development of AI applications. INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID) RR2-10.2196/42208.
Collapse
Affiliation(s)
- Lina Weinert
- Institute of Medical Informatics, Heidelberg University Hospital, Heidelberg, Germany
- Section for Translational Health Economics, Department for Conservative Dentistry, Heidelberg University Hospital, Heidelberg, Germany
| | - Maximilian Klass
- Institute of Medical Informatics, Heidelberg University Hospital, Heidelberg, Germany
| | - Gerd Schneider
- Institute of Medical Informatics, Heidelberg University Hospital, Heidelberg, Germany
| | - Oliver Heinze
- Institute of Medical Informatics, Heidelberg University Hospital, Heidelberg, Germany
| |
Collapse
|
10
|
Kondylakis H, Ciarrocchi E, Cerda-Alberich L, Chouvarda I, Fromont LA, Garcia-Aznar JM, Kalokyri V, Kosvyra A, Walker D, Yang G, Neri E. Position of the AI for Health Imaging (AI4HI) network on metadata models for imaging biobanks. Eur Radiol Exp 2022; 6:29. [PMID: 35773546 PMCID: PMC9247122 DOI: 10.1186/s41747-022-00281-1] [Citation(s) in RCA: 7] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/23/2021] [Accepted: 04/20/2022] [Indexed: 11/10/2022] Open
Abstract
A huge amount of imaging data is becoming available worldwide and an incredible range of possible improvements can be provided by artificial intelligence algorithms in clinical care for diagnosis and decision support. In this context, it has become essential to properly manage and handle these medical images and to define which metadata have to be considered, in order for the images to provide their full potential. Metadata are additional data associated with the images, which provide a complete description of the image acquisition, curation, analysis, and of the relevant clinical variables associated with the images. Currently, several data models are available to describe one or more subcategories of metadata, but a unique, common, and standard data model capable of fully representing the heterogeneity of medical metadata has not been yet developed. This paper reports the state of the art on metadata models for medical imaging, the current limitations and further developments, and describes the strategy adopted by the Horizon 2020 "AI for Health Imaging" projects, which are all dedicated to the creation of imaging biobanks.
Collapse
Affiliation(s)
| | - Esther Ciarrocchi
- grid.5395.a0000 0004 1757 3729Department of Translational Research, University of Pisa, Pisa, Italy
| | | | - Ioanna Chouvarda
- grid.4793.90000000109457005Aristotle University of Thessaloniki, Thessaloniki, Greece
| | - Lauren A. Fromont
- grid.11478.3b0000 0004 1766 3695Centre for Genomic Regulation (CRG), The Barcelona Institute of Science and Technology, Barcelona, Spain
| | | | - Varvara Kalokyri
- grid.5395.a0000 0004 1757 3729Department of Translational Research, University of Pisa, Pisa, Italy
| | - Alexandra Kosvyra
- grid.4793.90000000109457005Aristotle University of Thessaloniki, Thessaloniki, Greece
| | - Dawn Walker
- grid.11835.3e0000 0004 1936 9262Department of Computer Science and Insigneo Institute of in silico Medicine, University of Sheffield, Sheffield, UK
| | - Guang Yang
- grid.7445.20000 0001 2113 8111National Heart and Lung Institute, Imperial College London, London, UK
| | - Emanuele Neri
- grid.5395.a0000 0004 1757 3729Department of Translational Research, University of Pisa, Pisa, Italy
| | | |
Collapse
|