How to read and review papers on machine learning and artificial intelligence in radiology: a survival guide to key methodological concepts

An interpretable clinical ultrasound-radiomics combined model for diagnosis of stage I cervical cancer

Objective

The purpose of this retrospective study was to establish a combined model based on ultrasound (US)-radiomics and clinical factors to predict patients with stage I cervical cancer (CC) before surgery.

Materials and methods

A total of 209 CC patients who had cervical lesions found by transvaginal sonography (TVS) from the First Affiliated Hospital of Anhui Medical University were retrospectively reviewed, patients were divided into the training set (n = 146) and internal validation set (n = 63), and 52 CC patients from Anhui Provincial Maternity and Child Health Hospital and Nanchong Central Hospital were taken as the external validation set. The clinical independent predictors were selected by univariate and multivariate logistic regression analyses. US-radiomics features were extracted from US images. After selecting the most significant features by univariate analysis, Spearman's correlation analysis, and the least absolute shrinkage and selection operator (LASSO) algorithm, six machine learning (ML) algorithms were used to build the radiomics model. Next, the ability of the clinical, US-radiomics, and clinical US-radiomics combined model was compared to diagnose stage I CC. Finally, the Shapley additive explanations (SHAP) method was used to explain the contribution of each feature.

Results

Long diameter of the cervical lesion (L) and squamous cell carcinoma-associated antigen (SCCa) were independent clinical predictors of stage I CC. The eXtreme Gradient Boosting (Xgboost) model performed the best among the six ML radiomics models, with area under the curve (AUC) values in the training, internal validation, and external validation sets being 0.778, 0.751, and 0.751, respectively. In the final three models, the combined model based on clinical features and rad-score showed good discriminative power, with AUC values in the training, internal validation, and external validation sets being 0.837, 0.828, and 0.839, respectively. The decision curve analysis validated the clinical utility of the combined nomogram. The SHAP algorithm illustrates the contribution of each feature in the combined model.

Conclusion

We established an interpretable combined model to predict stage I CC. This non-invasive prediction method may be used for the preoperative identification of patients with stage I CC.

Reproducible Radiomics Features from Multi-MRI-Scanner Test-Retest-Study: Influence on Performance and Generalizability of Models

BACKGROUND

Radiomics models trained on data from one center typically show a decline of performance when applied to data from external centers, hindering their introduction into large-scale clinical practice. Current expert recommendations suggest to use only reproducible radiomics features isolated by multiscanner test-retest experiments, which might help to overcome the problem of limited generalizability to external data.

PURPOSE

To evaluate the influence of using only a subset of robust radiomics features, defined in a prior in vivo multi-MRI-scanner test-retest-study, on the performance and generalizability of radiomics models.

STUDY TYPE

Retrospective.

POPULATION

Patients with monoclonal plasma cell disorders. Training set (117 MRIs from center 1); internal test set (42 MRIs from center 1); external test set (143 MRIs from center 2-8).

FIELD STRENGTH/SEQUENCE

1.5T and 3.0T; T1-weighted turbo spin echo.

ASSESSMENT

The task for the radiomics models was to predict plasma cell infiltration, determined by bone marrow biopsy, noninvasively from MRI. Radiomics machine learning models, including linear regressor, support vector regressor (SVR), and random forest regressor (RFR), were trained on data from center 1, using either all radiomics features, or using only reproducible radiomics features. Models were tested on an internal (center 1) and a multicentric external data set (center 2-8).

STATISTICAL TESTS

Pearson correlation coefficient r and mean absolute error (MAE) between predicted and actual plasma cell infiltration. Fisher's z-transformation, Wilcoxon signed-rank test, Wilcoxon rank-sum test; significance level P < 0.05.

RESULTS

When using only reproducible features compared with all features, the performance of the SVR on the external test set significantly improved (r = 0.43 vs. r = 0.18 and MAE = 22.6 vs. MAE = 28.2). For the RFR, the performance on the external test set deteriorated when using only reproducible instead of all radiomics features (r = 0.33 vs. r = 0.44, P = 0.29 and MAE = 21.9 vs. MAE = 20.5, P = 0.10).

CONCLUSION

Using only reproducible radiomics features improves the external performance of some, but not all machine learning models, and did not automatically lead to an improvement of the external performance of the overall best radiomics model.

LEVEL OF EVIDENCE: 3

TECHNICAL EFFICACY

Stage 2.

Machine Learning Models in Prediction of Treatment Response After Chemoembolization with MRI Clinicoradiomics Features

PURPOSE

To evaluate machine learning models, created with radiomics and clinicoradiomics features, ability to predict local response after TACE.

MATERIALS AND METHODS

188 treatment-naïve patients (150 responders, 38 non-responders) with HCC who underwent TACE were included in this retrospective study. Laboratory, clinical and procedural information were recorded. Local response was evaluated by European Association for the Study of the Liver criteria at 3-months. Radiomics features were extracted from pretreatment pre-contrast enhanced T1 (T1WI) and late arterial-phase contrast-enhanced T1 (CE-T1) MRI images. After data augmentation, data were split into training and test sets (70/30). Intra-class correlations, Pearson's correlation coefficients were analyzed and followed by a sequential-feature-selection (SFS) algorithm for feature selection. Support-vector-machine (SVM) models were trained with radiomics and clinicoradiomics features of T1WI, CE-T1 and the combination of both datasets, respectively. Performance metrics were calculated with the test sets. Models' performances were compared with Delong's test.

RESULTS

1128 features were extracted. In feature selection, SFS algorithm selected 18, 12, 24 and 8 features in T1WI, CE-T1, combined datasets and clinical features, respectively. The SVM models area-under-curve was 0.86 and 0.88 in T1WI; 0.76, 0.71 in CE-T1 and 0.82, 0.91 in the combined dataset, with and without clinical features, respectively. The only significant change was observed after inclusion of clinical features in the combined dataset (p = 0.001). Higher WBC and neutrophil levels were significantly associated with lower treatment response in univariant analysis (p = 0.02, for both).

CONCLUSION

Machine learning models created with clinical and MRI radiomics features, may have promise in predicting local response after TACE.

LEVEL OF EVIDENCE

Level 4, Case-control study.

Construction and validation of infection risk model for patients with external ventricular drainage: a multicenter retrospective study

PURPOSE

External ventricular drainage (EVD) is a life-saving neurosurgical procedure, of which the most concerning complication is EVD-related infection (ERI). We aimed to construct and validate an ERI risk model and establish a monographic chart.

METHODS

We retrospectively analyzed the adult EVD patients in four medical centers and split the data into a training and a validation set. We selected features via single-factor logistic regression and trained the ERI risk model using multi-factor logistic regression. We further evaluated the model discrimination, calibration, and clinical usefulness, with internal and external validation to assess the reproducibility and generalizability. We finally visualized the model as a nomogram and created an online calculator (dynamic nomogram).

RESULTS

Our research enrolled 439 EVD patients and found 75 cases (17.1%) had ERI. Diabetes, drainage duration, site leakage, and other infections were independent risk factors that we used to fit the ERI risk model. The area under the receiver operating characteristic curve (AUC) and the Brier score of the model were 0.758 and 0.118, and these indicators' values were similar when internally validated. In external validation, the model discrimination had a moderate decline, of which the AUC was 0.720. However, the Brier score was 0.114, suggesting no degradation in overall performance. Spiegelhalter's Z-test indicated that the model had adequate calibration when validated internally or externally (P = 0.464 vs. P = 0.612). The model was transformed into a nomogram with an online calculator built, which is available through the website: https://wang-cdutcm.shinyapps.io/DynNomapp/ .

CONCLUSIONS

The present study developed an infection risk model for EVD patients, which is freely accessible and may serve as a simple decision tool in the clinic.

Machine learning item selection for short scale construction: A proof-of-concept using the SIMS

ObjectiveThis proof-of-concept paper provides evidence to support machine learning (ML) as a valid alternative to traditional psychometric techniques in the development of short forms of longer parent psychological tests. ML comprises a variety of feature selection techniques that can be efficiently applied to identify the set of items that best replicates the characteristics of the original test. MethodsIn the present study, we integrated a dataset of 329 participants from published and unpublished datasets used in previous research on the Structured Inventory of Malingered Symptomatology (SIMS) to develop a short version of the scale. The SIMS is a multi-axial self-report questionnaire and a highly efficient psychometric measure of symptom validity, which is frequently applied in forensic settings. Results State-of-the-art ML item selection techniques achieved a 72% reduction in length while capturing 92% of the variance of the original SIMS. The new SIMS short form now consists of 21 items. ConclusionsThe results suggest that the proposed ML-based item selection technique represents a promising alternative to standard psychometric correlation-based methods (i.e. item selection, item response theory), especially when selection techniques (e.g. wrapper) are employed that evaluate global, rather than local, item value.

Radiologist's Guide to Evaluating Publications of Clinical Research on AI: How We Do It

Literacy in research studies of artificial intelligence (AI) has become an important skill for radiologists. It is required to make a proper assessment of the validity, reproducibility, and clinical applicability of AI studies. However, AI studies are generally perceived to be more difficult for clinician readers to evaluate than traditional clinical research studies. This special report-as an effective, concise guide for readers-aims to assist clinical radiologists in critically evaluating different types of clinical research articles involving AI. It does not intend to be a comprehensive checklist or methodological summary for complete clinical evaluation of AI or a reporting guideline. Ten key items for readers to check are described, regarding study purpose, function and clinical context of AI, training data, data preprocessing, AI modeling techniques, test data, AI performance, helpfulness and value of AI, interpretability of AI, and code sharing. The important aspects of each item are explained for readers to consider when reading publications on AI clinical research. Evaluating each item can help radiologists assess the validity, reproducibility, and clinical applicability of clinical research articles involving AI.

The Quality and Utility of Artificial Intelligence in Patient Care

BACKGROUND

Artificial intelligence (AI) is increasingly being used in patient care. In the future, physicians will need to understand not only the basic functioning of AI applications, but also their quality, utility, and risks.

METHODS

This article is based on a selective review of the literature on the principles, quality, limitations, and benefits AI applications in patient care, along with examples of individual applications.

RESULTS

The number of AI applications in patient care is rising, with more than 500 approvals in the United States to date. Their quality and utility are based on a number of interdependent factors, including the real-life setting, the type and amount of data collected, the choice of variables used by the application, the algorithms used, and the goal and implementation of each application. Bias (which may be hidden) and errors can arise at all these levels. Any evaluation of the quality and utility of an AI application must, therefore, be conducted according to the scientific principles of evidence-based medicine-a requirement that is often hampered by a lack of transparency.

CONCLUSION

AI has the potential to improve patient care while meeting the challenge of dealing with an ever-increasing surfeit of information and data in medicine with limited human resources. The limitations and risks of AI applications require critical and responsible consideration. This can best be achieved through a combination of scientific.

Automatic segmentation and radiomic texture analysis for osteoporosis screening using chest low-dose computed tomography

OBJECTIVE

This study developed a diagnostic tool combining machine learning (ML) segmentation and radiomic texture analysis (RTA) for bone density screening using chest low-dose computed tomography (LDCT).

METHODS

A total of 197 patients who underwent LDCT followed by dual-energy X-ray absorptiometry were analyzed. First, an autosegmentation model was trained using LDCT to delineate the thoracic vertebral body (VB). Second, a two-level classifier was developed using radiomic features extracted from VBs for the hierarchical pairwise classification of each patient's bone status. All the patients were initially classified as either normal or abnormal, and all patients with abnormal bone density were then subdivided into an osteopenia group and an osteoporosis group. The performance of the classifier was evaluated through fivefold cross-validation.

RESULTS

The model for automated VB segmentation achieved a Sorenson-Dice coefficient of 0.87 ± 0.01. Furthermore, the area under the receiver operating characteristic curve scores for the two-level classifier were 0.96 ± 0.01 for detecting abnormal bone density (accuracy = 0.91 ± 0.02; sensitivity = 0.93 ± 0.03; specificity = 0.89 ± 0.03) and 0.98 ± 0.01 for distinguishing osteoporosis (accuracy = 0.94 ± 0.02; sensitivity = 0.95 ± 0.03; specificity = 0.93 ± 0.03). The testing prediction accuracy levels for the first- and second-level classifiers were 0.92 ± 0.04 and 0.94 ± 0.05, respectively. The overall testing prediction accuracy of our method was 0.90 ± 0.05.

CONCLUSION

The combination of ML segmentation and RTA for automated bone density prediction based on LDCT scans is a feasible approach that could be valuable for osteoporosis screening during lung cancer screening.

KEY POINTS

• This study developed an automatic diagnostic tool combining machine learning-based segmentation and radiomic texture analysis for bone density screening using chest low-dose computed tomography. • The developed method enables opportunistic screening without quantitative computed tomography or a dedicated phantom. • The developed method could be integrated into the current clinical workflow and used as an adjunct for opportunistic screening or for patients who are ineligible for screening with dual-energy X-ray absorptiometry.

CheckList for EvaluAtion of Radiomics research (CLEAR): a step-by-step reporting guideline for authors and reviewers endorsed by ESR and EuSoMII

Even though radiomics can hold great potential for supporting clinical decision-making, its current use is mostly limited to academic research, without applications in routine clinical practice. The workflow of radiomics is complex due to several methodological steps and nuances, which often leads to inadequate reporting and evaluation, and poor reproducibility. Available reporting guidelines and checklists for artificial intelligence and predictive modeling include relevant good practices, but they are not tailored to radiomic research. There is a clear need for a complete radiomics checklist for study planning, manuscript writing, and evaluation during the review process to facilitate the repeatability and reproducibility of studies. We here present a documentation standard for radiomic research that can guide authors and reviewers. Our motivation is to improve the quality and reliability and, in turn, the reproducibility of radiomic research. We name the checklist CLEAR (CheckList for EvaluAtion of Radiomics research), to convey the idea of being more transparent. With its 58 items, the CLEAR checklist should be considered a standardization tool providing the minimum requirements for presenting clinical radiomics research. In addition to a dynamic online version of the checklist, a public repository has also been set up to allow the radiomics community to comment on the checklist items and adapt the checklist for future versions. Prepared and revised by an international group of experts using a modified Delphi method, we hope the CLEAR checklist will serve well as a single and complete scientific documentation tool for authors and reviewers to improve the radiomics literature.

Prediction of Response of Hepatocellular Carcinoma to Radioembolization: Machine Learning Using Preprocedural Clinical Factors and MR Imaging Radiomics

PURPOSE

To create and evaluate the ability of machine learning-based models with clinicoradiomic features to predict radiologic response after transarterial radioembolization (TARE).

MATERIALS AND METHODS

82 treatment-naïve patients (65 responders and 17 nonresponders; median age: 65 years; interquartile range: 11) who underwent selective TARE were included. Treatment responses were evaluated using the European Association for the Study of the Liver criteria at 3-month follow-up. Laboratory, clinical, and procedural information were collected. Radiomic features were extracted from pretreatment contrast-enhanced T1-weighted magnetic resonance images obtained within 3 months before TARE. Feature selection consisted of intraclass correlation, followed by Pearson correlation analysis and finally, sequential feature selection algorithm. Support vector machine, logistic regression, random forest, and LightGBM models were created with both clinicoradiomic features and clinical features alone. Performance metrics were calculated with a nested 5-fold cross-validation technique. The performances of the models were compared by Wilcoxon signed-rank and Friedman tests.

RESULTS

In total, 1,128 features were extracted. The feature selection process resulted in 12 features (8 radiomic and 4 clinical features) being included in the final analysis. The area under the receiver operating characteristic curve values from the support vector machine, logistic regression, random forest, and LightGBM models were 0.94, 0.94, 0.88, and 0.92 with clinicoradiomic features and 0.82, 0.83, 0.82, and 0.83 with clinical features alone, respectively. All models exhibited significantly higher performances when radiomic features were included (P = .028, .028, .043, and .028, respectively).

CONCLUSIONS

Based on clinical and imaging-based information before treatment, machine learning-based clinicoradiomic models demonstrated potential to predict response to TARE.

The application of machine learning to imaging in hematological oncology: A scoping review

Background

Here, we conducted a scoping review to (i) establish which machine learning (ML) methods have been applied to hematological malignancy imaging; (ii) establish how ML is being applied to hematological cancer radiology; and (iii) identify addressable research gaps.

Methods

The review was conducted according to the Preferred Reporting Items for Systematic Reviews and Meta-Analysis Extension for Scoping Reviews guidelines. The inclusion criteria were (i) pediatric and adult patients with suspected or confirmed hematological malignancy undergoing imaging (population); (ii) any study using ML techniques to derive models using radiological images to apply to the clinical management of these patients (concept); and (iii) original research articles conducted in any setting globally (context). Quality Assessment of Diagnostic Accuracy Studies 2 criteria were used to assess diagnostic and segmentation studies, while the Newcastle-Ottawa scale was used to assess the quality of observational studies.

Results

Of 53 eligible studies, 33 applied diverse ML techniques to diagnose hematological malignancies or to differentiate them from other diseases, especially discriminating gliomas from primary central nervous system lymphomas (n=18); 11 applied ML to segmentation tasks, while 9 applied ML to prognostication or predicting therapeutic responses, especially for diffuse large B-cell lymphoma. All studies reported discrimination statistics, but no study calculated calibration statistics. Every diagnostic/segmentation study had a high risk of bias due to their case-control design; many studies failed to provide adequate details of the reference standard; and only a few studies used independent validation.

Conclusion

To deliver validated ML-based models to radiologists managing hematological malignancies, future studies should (i) adhere to standardized, high-quality reporting guidelines such as the Checklist for Artificial Intelligence in Medical Imaging; (ii) validate models in independent cohorts; (ii) standardize volume segmentation methods for segmentation tasks; (iv) establish comprehensive prospective studies that include different tumor grades, comparisons with radiologists, optimal imaging modalities, sequences, and planes; (v) include side-by-side comparisons of different methods; and (vi) include low- and middle-income countries in multicentric studies to enhance generalizability and reduce inequity.

Evaluating artificial intelligence algorithms for use in veterinary radiology

Artificial intelligence is increasingly being used for applications in veterinary radiology, including detection of abnormalities and automated measurements. Unlike human radiology, there is no formal regulation or validation of AI algorithms for veterinary medicine and both general practitioner and specialist veterinarians must rely on their own judgment when deciding whether or not to incorporate AI algorithms to aid their clinical decision-making. The benefits and challenges to developing clinically useful and diagnostically accurate AI algorithms are discussed. Considerations for the development of AI research projects are also addressed. A framework is suggested to help veterinarians, in both research and clinical practice contexts, assess AI algorithms for veterinary radiology.

Machine learning-based prediction of cognitive outcomes in de novo Parkinson's disease

Cognitive impairment is a debilitating symptom in Parkinson's disease (PD). We aimed to establish an accurate multivariate machine learning (ML) model to predict cognitive outcome in newly diagnosed PD cases from the Parkinson's Progression Markers Initiative (PPMI). Annual cognitive assessments over an 8-year time span were used to define two cognitive outcomes of (i) cognitive impairment, and (ii) dementia conversion. Selected baseline variables were organized into three subsets of clinical, biofluid and genetic/epigenetic measures and tested using four different ML algorithms. Irrespective of the ML algorithm used, the models consisting of the clinical variables performed best and showed better prediction of cognitive impairment outcome over dementia conversion. We observed a marginal improvement in the prediction performance when clinical, biofluid, and epigenetic/genetic variables were all included in one model. Several cerebrospinal fluid measures and an epigenetic marker showed high predictive weighting in multiple models when included alongside clinical variables.

Mitigating Bias in Radiology Machine Learning: 1. Data Handling

Minimizing bias is critical to adoption and implementation of machine learning (ML) in clinical practice. Systematic mathematical biases produce consistent and reproducible differences between the observed and expected performance of ML systems, resulting in suboptimal performance. Such biases can be traced back to various phases of ML development: data handling, model development, and performance evaluation. This report presents 12 suboptimal practices during data handling of an ML study, explains how those practices can lead to biases, and describes what may be done to mitigate them. Authors employ an arbitrary and simplified framework that splits ML data handling into four steps: data collection, data investigation, data splitting, and feature engineering. Examples from the available research literature are provided. A Google Colaboratory Jupyter notebook includes code examples to demonstrate the suboptimal practices and steps to prevent them. Keywords: Data Handling, Bias, Machine Learning, Deep Learning, Convolutional Neural Network (CNN), Computer-aided Diagnosis (CAD) © RSNA, 2022.

Prediction of Lumbar Drainage-Related Meningitis Based on Supervised Machine Learning Algorithms

Background

Lumbar drainage is widely used in the clinic; however, forecasting lumbar drainage-related meningitis (LDRM) is limited. We aimed to establish prediction models using supervised machine learning (ML) algorithms.

Methods

We utilized a cohort of 273 eligible lumbar drainage cases. Data were preprocessed and split into training and testing sets. Optimal hyper-parameters were archived by 10-fold cross-validation and grid search. The support vector machine (SVM), random forest (RF), and artificial neural network (ANN) were adopted for model training. The area under the operating characteristic curve (AUROC) and precision-recall curve (AUPRC), true positive ratio (TPR), true negative ratio (TNR), specificity, sensitivity, accuracy, and kappa coefficient were used for model evaluation. All trained models were internally validated. The importance of features was also analyzed.

Results

In the training set, all the models had AUROC exceeding 0.8. SVM and the RF models had an AUPRC of more than 0.6, but the ANN model had an unexpectedly low AUPRC (0.380). The RF and ANN models revealed similar TPR, whereas the ANN model had a higher TNR and demonstrated better specificity, sensitivity, accuracy, and kappa efficiency. In the testing set, most performance indicators of established models decreased. However, the RF and AVM models maintained adequate AUROC (0.828 vs. 0.719) and AUPRC (0.413 vs. 0.520), and the RF model also had better TPR, specificity, sensitivity, accuracy, and kappa efficiency. Site leakage showed the most considerable mean decrease in accuracy.

Conclusions

The RF and SVM models could predict LDRM, in which the RF model owned the best performance, and site leakage was the most meaningful predictor.

Radiomic Analysis: Study Design, Statistical Analysis, and Other Bias Mitigation Strategies

Rapid advances in automated methods for extracting large numbers of quantitative features from medical images have led to tremendous growth of publications reporting on radiomic analyses. Translation of these research studies into clinical practice can be hindered by biases introduced during the design, analysis, or reporting of the studies. Herein, the authors review biases, sources of variability, and pitfalls that frequently arise in radiomic research, with an emphasis on study design and statistical analysis considerations. Drawing on existing work in the statistical, radiologic, and machine learning literature, approaches for avoiding these pitfalls are described.

Considerations for artificial intelligence clinical impact in oncologic imaging: an AI4HI position paper

To achieve clinical impact in daily oncological practice, emerging AI-based cancer imaging research needs to have clearly defined medical focus, AI methods, and outcomes to be estimated. AI-supported cancer imaging should predict major relevant clinical endpoints, aiming to extract associations and draw inferences in a fair, robust, and trustworthy way. AI-assisted solutions as medical devices, developed using multicenter heterogeneous datasets, should be targeted to have an impact on the clinical care pathway. When designing an AI-based research study in oncologic imaging, ensuring clinical impact in AI solutions requires careful consideration of key aspects, including target population selection, sample size definition, standards, and common data elements utilization, balanced dataset splitting, appropriate validation methodology, adequate ground truth, and careful selection of clinical endpoints. Endpoints may be pathology hallmarks, disease behavior, treatment response, or patient prognosis. Ensuring ethical, safety, and privacy considerations are also mandatory before clinical validation is performed. The Artificial Intelligence for Health Imaging (AI4HI) Clinical Working Group has discussed and present in this paper some indicative Machine Learning (ML) enabled decision-support solutions currently under research in the AI4HI projects, as well as the main considerations and requirements that AI solutions should have from a clinical perspective, which can be adopted into clinical practice. If effectively designed, implemented, and validated, cancer imaging AI-supported tools will have the potential to revolutionize the field of precision medicine in oncology.

Machine learning-based CT radiomics approach for predicting WHO/ISUP nuclear grade of clear cell renal cell carcinoma: an exploratory and comparative study

Purpose

To investigate the predictive performance of machine learning-based CT radiomics for differentiating between low- and high-nuclear grade of clear cell renal cell carcinomas (CCRCCs).

Methods

This retrospective study enrolled 406 patients with pathologically confirmed low- and high-nuclear grade of CCRCCs according to the WHO/ISUP grading system, which were divided into the training and testing cohorts. Radiomics features were extracted from nephrographic-phase CT images using PyRadiomics. A support vector machine (SVM) combined with three feature selection algorithms such as least absolute shrinkage and selection operator (LASSO), recursive feature elimination (RFE), and ReliefF was performed to determine the most suitable classification model, respectively. Clinicoradiological, radiomics, and combined models were constructed using the radiological and clinical characteristics with significant differences between the groups, selected radiomics features, and a combination of both, respectively. Model performance was evaluated by receiver operating characteristic (ROC) curve, calibration curve, and decision curve analyses.

Results

SVM-ReliefF algorithm outperformed SVM-LASSO and SVM-RFE in distinguishing low- from high-grade CCRCCs. The combined model showed better prediction performance than the clinicoradiological and radiomics models (p < 0.05, DeLong test), which achieved the highest efficacy, with an area under the ROC curve (AUC) value of 0.887 (95% confidence interval [CI] 0.798–0.952), 0.859 (95% CI 0.748–0.935), and 0.828 (95% CI 0.731–0.929) in the training, validation, and testing cohorts, respectively. The calibration and decision curves also indicated the favorable performance of the combined model.

Conclusion

A combined model incorporating the radiomics features and clinicoradiological characteristics can better predict the WHO/ISUP nuclear grade of CCRCC preoperatively, thus providing effective and noninvasive assessment.

Supplementary Information

The online version contains supplementary material available at 10.1186/s13244-021-01107-1.

Examination of the diaphragm in obstructive sleep apnea using ultrasound imaging

PURPOSE

The aim of this study was to analyze the effect of obstructive sleep apnea (OSA) on the ultrasound (US) features of the diaphragm and to determine if diaphragmatic US may be a useful screening tool for patients with possible OSA.

METHODS

Patients complaining of snoring were prospectively enrolled for overnight polygraphy using the ApneaLink Air device. Thickness and motion of the diaphragm during tidal and deep inspiration were measured. Logistic regression was used to assess parameters of the diaphragm associated with OSA.

RESULTS

Of 100 patients, 64 were defined as having OSA. Thicknesses of the left and right hemidiaphragms were significantly different between OSA and control groups. Using a combination of diaphragmatic dimensions, diaphragm dilation, age, sex, and BMI, we developed an algorithm that predicted the presence of OSA with 91% sensitivity and 81% specificity.

CONCLUSION

A combination of anthropometric measurements, demographic factors, and US imaging may be useful for screening patients for possible OSA. These findings need to be confirmed in larger sample sizes in different clinical settings.

Review of study reporting guidelines for clinical studies using artificial intelligence in healthcare

High-quality research is essential in guiding evidence-based care, and should be reported in a way that is reproducible, transparent and where appropriate, provide sufficient detail for inclusion in future meta-analyses. Reporting guidelines for various study designs have been widely used for clinical (and preclinical) studies, consisting of checklists with a minimum set of points for inclusion. With the recent rise in volume of research using artificial intelligence (AI), additional factors need to be evaluated, which do not neatly conform to traditional reporting guidelines (eg, details relating to technical algorithm development). In this review, reporting guidelines are highlighted to promote awareness of essential content required for studies evaluating AI interventions in healthcare. These include published and in progress extensions to well-known reporting guidelines such as Standard Protocol Items: Recommendations for Interventional Trials-AI (study protocols), Consolidated Standards of Reporting Trials-AI (randomised controlled trials), Standards for Reporting of Diagnostic Accuracy Studies-AI (diagnostic accuracy studies) and Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis-AI (prediction model studies). Additionally there are a number of guidelines that consider AI for health interventions more generally (eg, Checklist for Artificial Intelligence in Medical Imaging (CLAIM), minimum information (MI)-CLAIM, MI for Medical AI Reporting) or address a specific element such as the ‘learning curve’ (Developmental and Exploratory Clinical Investigation of Decision-AI). Economic evaluation of AI health interventions is not currently addressed, and may benefit from extension to an existing guideline. In the face of a rapid influx of studies of AI health interventions, reporting guidelines help ensure that investigators and those appraising studies consider both the well-recognised elements of good study design and reporting, while also adequately addressing new challenges posed by AI-specific elements.

Deep learning to automate the labelling of head MRI datasets for computer vision applications

Objectives

The purpose of this study was to build a deep learning model to derive labels from neuroradiology reports and assign these to the corresponding examinations, overcoming a bottleneck to computer vision model development.

Methods

Reference-standard labels were generated by a team of neuroradiologists for model training and evaluation. Three thousand examinations were labelled for the presence or absence of any abnormality by manually scrutinising the corresponding radiology reports (‘reference-standard report labels’); a subset of these examinations (n = 250) were assigned ‘reference-standard image labels’ by interrogating the actual images. Separately, 2000 reports were labelled for the presence or absence of 7 specialised categories of abnormality (acute stroke, mass, atrophy, vascular abnormality, small vessel disease, white matter inflammation, encephalomalacia), with a subset of these examinations (n = 700) also assigned reference-standard image labels. A deep learning model was trained using labelled reports and validated in two ways: comparing predicted labels to (i) reference-standard report labels and (ii) reference-standard image labels. The area under the receiver operating characteristic curve (AUC-ROC) was used to quantify model performance. Accuracy, sensitivity, specificity, and F1 score were also calculated.

Results

Accurate classification (AUC-ROC > 0.95) was achieved for all categories when tested against reference-standard report labels. A drop in performance (ΔAUC-ROC > 0.02) was seen for three categories (atrophy, encephalomalacia, vascular) when tested against reference-standard image labels, highlighting discrepancies in the original reports. Once trained, the model assigned labels to 121,556 examinations in under 30 min.

Conclusions

Our model accurately classifies head MRI examinations, enabling automated dataset labelling for downstream computer vision applications.

Key Points

• Deep learning is poised to revolutionise image recognition tasks in radiology; however, a barrier to clinical adoption is the difficulty of obtaining large labelled datasets for model training.

• We demonstrate a deep learning model which can derive labels from neuroradiology reports and assign these to the corresponding examinations at scale, facilitating the development of downstream computer vision models.

• We rigorously tested our model by comparing labels predicted on the basis of neuroradiology reports with two sets of reference-standard labels: (1) labels derived by manually scrutinising each radiology report and (2) labels derived by interrogating the actual images.

Supplementary Information

The online version contains supplementary material available at 10.1007/s00330-021-08132-0.

A scoping review on biomedical journal peer review guides for reviewers

Background

Peer review is widely used in academic fields to assess a manuscript’s significance and to improve its quality for publication. This scoping review will assess existing peer review guidelines and/or checklists intended for reviewers of biomedical journals and provide an overview on the review guidelines.

Methods

PubMed, Embase, and Allied and Complementary Medicine (AMED) databases were searched for review guidelines from the date of inception until February 19, 2021. There was no date restriction nor article type restriction. In addition to the database search, websites of journal publishers and non-publishers were additionally hand-searched.

Results

Of 14,633 database publication records and 24 website records, 65 publications and 14 websites met inclusion criteria for the review (78 records in total). From the included records, a total of 1,811 checklist items were identified. The items related to Methods, Results, and Discussion were found to be the highly discussed in reviewer guidelines.

Conclusion

This review identified existing literature on peer review guidelines and provided an overview of the current state of peer review guides. Review guidelines were varying by journals and publishers. This calls for more research to determine the need to use uniform review standards for transparent and standardized peer review.

Protocol registration

The protocol for this study has been registered at Research Registry (www.researchregistry.com): reviewregistry881.

Artificial intelligence and hybrid imaging: the best match for personalized medicine in oncology

Artificial intelligence (AI) refers to a field of computer science aimed to perform tasks typically requiring human intelligence. Currently, AI is recognized in the broader technology radar within the five key technologies which emerge for their wide-ranging applications and impact in communities, companies, business, and value chain framework alike. However, AI in medical imaging is at an early phase of development, and there are still hurdles to take related to reliability, user confidence, and adoption. The present narrative review aimed to provide an overview on AI-based approaches (distributed learning, statistical learning, computer-aided diagnosis and detection systems, fully automated image analysis tool, natural language processing) in oncological hybrid medical imaging with respect to clinical tasks (detection, contouring and segmentation, prediction of histology and tumor stage, prediction of mutational status and molecular therapies targets, prediction of treatment response, and outcome). Particularly, AI-based approaches have been briefly described according to their purpose and, finally lung cancer-being one of the most extensively malignancy studied by hybrid medical imaging-has been used as illustrative scenario. Finally, we discussed clinical challenges and open issues including ethics, validation strategies, effective data-sharing methods, regulatory hurdles, educational resources, and strategy to facilitate the interaction among different stakeholders. Some of the major changes in medical imaging will come from the application of AI to workflow and protocols, eventually resulting in improved patient management and quality of life. Overall, several time-consuming tasks could be automatized. Machine learning algorithms and neural networks will permit sophisticated analysis resulting not only in major improvements in disease characterization through imaging, but also in the integration of multiple-omics data (i.e., derived from pathology, genomic, proteomics, and demographics) for multi-dimensional disease featuring. Nevertheless, to accelerate the transition of the theory to practice a sustainable development plan considering the multi-dimensional interactions between professionals, technology, industry, markets, policy, culture, and civil society directed by a mindset which will allow talents to thrive is necessary.