A feed forward neural network for classification of bull's-eye myocardial perfusion images

Comparing various AI approaches to traditional quantitative assessment of the myocardial perfusion in [82Rb] PET for MACE prediction

Assessing the individual risk of Major Adverse Cardiac Events (MACE) is of major importance as cardiovascular diseases remain the leading cause of death worldwide. Quantitative Myocardial Perfusion Imaging (MPI) parameters such as stress Myocardial Blood Flow (sMBF) or Myocardial Flow Reserve (MFR) constitutes the gold standard for prognosis assessment. We propose a systematic investigation of the value of Artificial Intelligence (AI) to leverage [82 Rb] Silicon PhotoMultiplier (SiPM) PET MPI for MACE prediction. We establish a general pipeline for AI model validation to assess and compare the performance of global (i.e. average of the entire MPI signal), regional (17 segments), radiomics and Convolutional Neural Network (CNN) models leveraging various MPI signals on a dataset of 234 patients. Results showed that all regional AI models significantly outperformed the global model ( p < 0.001 ), where the best AUC of 73.9% (CI 72.5-75.3) was obtained with a CNN model. A regional AI model based on MBF averages from 17 segments fed to a Logistic Regression (LR) constituted an excellent trade-off between model simplicity and performance, achieving an AUC of 73.4% (CI 72.3-74.7). A radiomics model based on intensity features revealed that the global average was the least important feature when compared to other aggregations of the MPI signal over the myocardium. We conclude that AI models can allow better personalized prognosis assessment for MACE.

Preparing for the Artificial Intelligence Revolution in Nuclear Cardiology

A major opportunity in nuclear cardiology is the many significant artificial intelligence (AI) applications that have recently been reported. These developments include using deep learning (DL) for reducing the needed injected dose and acquisition time in perfusion acquisitions also due to DL improvements in image reconstruction and filtering, SPECT attenuation correction using DL without need for transmission images, DL and machine learning (ML) use for feature extraction to define myocardial left ventricular (LV) borders for functional measurements and improved detection of the LV valve plane and AI, ML, and DL implementations for MPI diagnosis, prognosis, and structured reporting. Although some have, most of these applications have yet to make it to widespread commercial distribution due to the recency of their developments, most reported in 2020. We must be prepared both technically and socio-economically to fully benefit from these and a tsunami of other AI applications that are coming.

Diagnostic performance of an artificial intelligence-driven cardiac-structured reporting system for myocardial perfusion SPECT imaging

OBJECTIVES

To describe and validate an artificial intelligence (AI)-driven structured reporting system by direct comparison of automatically generated reports to results from actual clinical reports generated by nuclear cardiology experts.

BACKGROUND

Quantitative parameters extracted from myocardial perfusion imaging (MPI) studies are used by our AI reporting system to generate automatically a guideline-compliant structured report (sR).

METHOD

A new nonparametric approach generates distribution functions of rest and stress, perfusion, and thickening, for each of 17 left ventricle segments that are then transformed to certainty factors (CFs) that a segment is hypoperfused, ischemic. These CFs are then input to our set of heuristic rules used to reach diagnostic findings and impressions propagated into a sR referred as an AI-driven structured report (AIsR). The diagnostic accuracy of the AIsR for detecting coronary artery disease (CAD) and ischemia was tested in 1,000 patients who had undergone rest/stress SPECT MPI.

RESULTS

At the high-specificity (SP) level, in a subset of 100 patients, there were no statistical differences in the agreements between the AIsr, and nine experts' impressions of CAD (P = .33) or ischemia (P = .37). This high-SP level also yielded the highest accuracy across global and regional results in the 1,000 patients. These accuracies were statistically significantly better than the other two levels [sensitivity (SN)/SP tradeoff, high SN] across all comparisons.

CONCLUSIONS

This AI reporting system automatically generates a structured natural language report with a diagnostic performance comparable to those of experts.

Classification of Polar Maps from Cardiac Perfusion Imaging with Graph-Convolutional Neural Networks

Myocardial perfusion imaging is a non-invasive imaging technique commonly used for the diagnosis of Coronary Artery Disease and is based on the injection of radiopharmaceutical tracers into the blood stream. The patient's heart is imaged while at rest and under stress in order to determine its capacity to react to the imposed challenge. Assessment of imaging data is commonly performed by visual inspection of polar maps showing the tracer uptake in a compact, two-dimensional representation of the left ventricle. This article presents a method for automatic classification of polar maps based on graph convolutional neural networks. Furthermore, it evaluates how well localization techniques developed for standard convolutional neural networks can be used for the localization of pathological segments with respect to clinically relevant areas. The method is evaluated using 946 labeled datasets and compared quantitatively to three other neural-network-based methods. The proposed model achieves an agreement with the human observer on 89.3% of rest test polar maps and on 91.1% of stress test polar maps. Localization performed on a fine 17-segment division of the polar maps achieves an agreement of 83.1% with the human observer, while localization on a coarse 3-segment division based on the vessel beds of the left ventricle has an agreement of 78.8% with the human observer. Our method could thus assist the decision-making process of physicians when analyzing polar map data obtained from myocardial perfusion images.

Clinical decision support systems in myocardial perfusion imaging

Diagnostic imaging is becoming more complicated, physicians are also required to master an ever-expanding knowledge base and take into account an ever increasing amount of patient-specific clinical information while the time available to master this knowledge base, assemble the relevant clinical data, and apply it to specific tasks is steadily shrinking. Compounding these problems, there is an ever increasing number of aging "Baby Boomers" who are becoming patients coupled with a declining number of cardiac diagnosticians experienced in interpreting these studies. Hence, it is crucial that decision support tools be developed and implemented to assist physicians in interpreting studies at a faster rate and at the highest level of up-to-date expertise. Such tools will minimize subjectivity and intra- and inter-observer variation in image interpretation, help achieve a standardized high level of performance, and reduce healthcare costs. Presently, there are many decision support systems and approaches being developed and implemented to provide greater automation and to further objectify and standardize analysis, display, integration, interpretation, and reporting of myocardial perfusion SPECT and PET studies. This review focuses on these systems and approaches.

Computer-assisted diagnosis in renal nuclear medicine: rationale, methodology, and interpretative criteria for diuretic renography

The goal of artificial intelligence, expert systems, decision support systems, and computer-assisted diagnosis (CAD) in imaging is the development and implementation of software to assist in the detection and evaluation of abnormalities, to alert physicians to cognitive biases, to reduce intraobserver and interobserver variability, and to facilitate the interpretation of studies at a faster rate and with a higher level of accuracy. These developments are needed to meet the challenges resulting from a rapid increase in the volume of diagnostic imaging studies coupled with a concurrent increase in the number and complexity of images in each patient data. The convergence of an expanding knowledge base and escalating time constraints increases the likelihood of physician errors. Errors are even more likely when physicians interpret low-volume studies such as technetium-99m-mercaptoacetyltriglycine diuretic scans where imagers may have had limited training or experience. Decision support systems include neural networks, case-based reasoning, expert systems, and statistical systems. iRENEX (renal expert) is an expert system for diuretic renography that uses a set of rules obtained from human experts to analyze a knowledge base of both clinical parameters and quantitative parameters derived from the renogram. Initial studies have shown that the interpretations provided by iRENEX are comparable to the interpretations of a panel of experts. iRENEX provides immediate patient-specific feedback at the time of scan interpretation, can be queried to provide the reasons for its conclusions, and can be used as an educational tool to teach trainees to better interpret renal scans. It also has the capacity to populate a structured reporting module and generate a clear and concise impression based on the elements contained in the report; adherence to the procedural and data entry components of the structured reporting module ensures and documents procedural competency. Finally, although the focus is CAD applied to diuretic renography, this review offers a window into the rationale, methodology, and broader applications of computer-assisted diagnosis in medical imaging.

Decision support systems in diuresis renography

The volume of diagnostic imaging studies performed in the United States is rapidly increasing resulting from an increase in the number of patients as well as an increase in the volume of studies per patient. Concurrently, the number and complexity of images in each patient data set are also increasing. Nuclear medicine physicians and radiologists are required to master an ever-expanding knowledge base whereas the hours available to master this knowledge base and apply it to specific tasks are steadily shrinking. The convergence of an expanding knowledge base and escalating time constraints increases the likelihood of physician errors. The problem is particularly acute for low-volume studies such as MAG3 diuresis renography where many imagers may have had limited training or experience. To address this problem, renal decision support systems (DSS) are being developed to assist physicians evaluate suspected obstruction in patients referred for diuresis renography. Categories of DSS include neural networks, case-based reasoning, expert systems and statistical systems; RENEX and CART are examples of renal DSS currently in development. RENEX (renal expert) uses a set of rules obtained from human experts to analyze a knowledge base of expanded quantitative parameters obtained from diuresis MAG3 scintigraphy whereas CART (classification and regression tree analysis) is a statistical method that grows and prunes a decision tree based on an analysis of these quantitative parameters in a training data set. RENEX can be queried to provide the reasons for its conclusions. Initial data show that the interpretations provided by RENEX and CART are comparable to the interpretations of a panel of experts blinded to clinical information. This project should serve as a benchmark for the scientific comparison and collaboration of these 2 fields of medical decision-making. Moreover, we anticipate that these DSS will better define the essential interpretative criteria, foster standardized interpretation, teach trainees to better interpret renal scans, enhance diagnostic accuracy and provide a methodology applicable to other diagnostic problems in radiology and medicine.

Diagnostic performance of an expert system for interpretation of 99mTc MAG3 scans in suspected renal obstruction

The purpose of the study was to compare diuresis renography scan interpretation generated by a renal expert system with the consensus interpretation of 3 expert readers.

METHODS

The expert system was evaluated in 95 randomly selected furosemide-augmented patient studies (185 kidneys) obtained for suspected obstruction; there were 55 males and 40 females with a mean age +/- SD of 58.6 +/- 16.5 y. Each subject had a baseline (99m)Tc-mercaptoacetyltriglycine ((99m)Tc-MAG3) scan followed by furosemide administration and a separate 20-min acquisition. Quantitative parameters were automatically extracted from baseline and furosemide acquisitions and forwarded to the expert system for analysis. Three experts, unaware of clinical information, independently graded each kidney as obstructed/probably obstructed, equivocal, and probably nonobstructed/nonobstructed; experts resolved differences by a consensus reading. These 3 expert categories were compared with the obstructed, equivocal, and nonobstructed interpretations provided by the expert system. Agreement was assessed using weighted kappa, and the predictive accuracy of the expert system compared with expert readers was assessed by the area under receiver-operating-characteristic (ROC curve) curves.

RESULTS

The expert system agreed with the consensus reading in 84% (101/120) of nonobstructed kidneys, in 92% (33/36) of obstructed kidneys, and in 45% (13/29) of equivocal kidneys. The weighted kappa between the expert system and the consensus reading was 0.72 and was comparable with the weighted kappa between experts. There was no significant difference in the areas under the ROC curves when the expert system was compared with each expert using the other 2 experts as the gold standard.

CONCLUSION

The renal expert system showed good agreement with the expert interpretation and could be a useful educational and decision support tool to assist physicians in the diagnosis of renal obstruction. To better mirror the clinical setting, algorithms to incorporate clinical data must be designed, implemented, and tested.

Evaluation of two diuresis renography decision support systems to determine the need for furosemide in patients with suspected obstruction

OBJECTIVE

The purpose of this study was to compare the decisions regarding the need for furosemide made by two independent renal decision support systems, RENEX and CAR-TAN, with the need for furosemide determined in clinical practice and by expert reviewers using the baseline plus furosemide protocol.

SUBJECTS AND METHODS

RENEX and CARTAN are independent decision support systems that reach their conclusions without operator input. RENEX is a knowledge-based system and CARTAN is a statistical decision support system. Both were trained using the same pilot group of 31 adult patients (61 kidneys) referred for suspected obstruction. Subsequently, both systems were prospectively applied to 102 patients (200 kidneys) of whom 70 received furosemide; decisions regarding the need for furosemide were compared with the clinical decisions and the decisions of three experts who independently scored each kidney on the need for furosemide. Differences were resolved by consensus.

RESULTS

RENEX agreed with the clinical and experts' decisions to give furosemide in 97% (68/70) and 98% (65/66) of patients, respectively, whereas CARTAN agreed in 90% (63/70) and 89% (59/66), respectively, p < 0.03. In contrast, CARTAN agreed with the experts' decision to withhold furosemide in 78% of kidneys (87/111), whereas RENEX agreed in only 69% of kidneys (77/111), p = 0.008.

CONCLUSION

Use of RENEX or CARTAN as decision support tools in the baseline plus furosemide protocol has the potential to help the radiologist avoid unnecessary imaging and reduce the technologist, computer, camera, and physician time required to perform the procedure.

The added value of ECG-gating for the diagnosis of myocardial infarction using myocardial perfusion scintigraphy and artificial neural networks

To assess the value of ECG-gating for the diagnosis of myocardial infarction using myocardial perfusion scintigraphy (MPS) and an artificial neural network. A total of 422 patients referred for MPS were studied using a one day (99m)Tc-tetrofosmin protocol. Adenosine stress combined with submaximal dynamic exercise was used. The images were interpreted by one of three experienced clinicians and these interpretations regarding the presence or absence of myocardial infarction were used as the standard. A fully automated method using artificial neural networks was compared with the clinical interpretation. Either perfusion data alone or a combination of perfusion and function from ECG-gated images were used as input to different artificial neural networks. After a training session, the two types of neural networks were evaluated in separate test groups using an eightfold cross-validation procedure. The neural networks trained with both perfusion and ECG-gated images had a 4-7% higher specificity compared with the corresponding networks using perfusion data only, in four of five segments compared at the same level of sensitivity. The greatest improvement in specificity, from 70% to 77%, was seen in the inferior segment. In the septal and lateral segments the specificity rose from 73% to 77% and from 81% to 85%, respectively. In the anterior segment, the increase in specificity from 93% to 94% by adding functional data was not significant. The addition of functional information from ECG-gated MPS is of value for the diagnosis of myocardial infarction using an automated method of interpreting myocardial perfusion images.

Artificial neural network modeling of stress single-photon emission computed tomographic imaging for detecting extensive coronary artery disease

Single-photon emission computed tomographic imaging is a useful, noninvasive method to detect coronary artery disease (CAD). We tested the hypothesis that artificial neural network modeling could predict CAD extent better than visual interpretation; 109 patients who underwent stress single-photon emission computed tomography and coronary angiography were selected. Twenty patients who had a <5% probability of CAD were also selected for calculation of normalcy rate. A model was trained for each vessel. Stress images were decreased to 25 points by pixel averaging the polar map. The model output was 1 for vessel stenosis >60% and 0 otherwise. Model sensitivities were 92% (55 of 60) for left anterior descending artery versus 62% (37 of 60) for visual interpretation (p = 0.0002), 69% (20 of 29) for left circumflex artery versus 55% for visual interpretation (p = 0.30), and 94% (45 of 48) for right coronary artery versus 78% for visual interpretation (p = 0.024). Model specificities and normalcy rates were 78% and 85% for the left anterior descending artery, 93% and 100% for the left circumflex artery, and 85% and 90% for the right circumflex artery, respectively. Single-vessel CAD was predicted in 27 of 28 patients (96%) by modeling versus 23 of 28 patients (82%) by visual interpretation (p = 0.11). Multivessel CAD was correctly predicted in 30 of 46 patients (65%) by modeling versus 16 of 46 patients (35%) by visual interpretation (p = 0.004). Thus, artificial neural network models can predict CAD from stress single-photon emission computed tomographic images when using separate models for the 3 major epicardial vessels. Because of their high sensitivity and specificity in detecting extensive CAD, these models have great promise as an aid to correctly identify patients at high risk for CAD.

WeAidU-a decision support system for myocardial perfusion images using artificial neural networks

This paper presents a computer-based decision support system for automated interpretation of diagnostic heart images (called WeAidU), which is made available via the Internet. The system is based on image processing techniques, artificial neural networks (ANNs) and large well-validated medical databases. We present results using artificial neural networks, and compare with two other classification methods, on a retrospective data set containing 1320 images from the clinical routine. The performance of the artificial neural networks detecting infarction and ischemia in different parts of the heart, measured as areas under the receiver operating characteristic curves, is in the range 0.83-0.96. These results indicate a high potential for the tool as a clinical decision support system.

Value of exercise data for the interpretation of myocardial perfusion SPECT

BACKGROUND

Artificial neural networks have successfully been applied for automated interpretation of myocardial perfusion images. So far the networks have used data from the myocardial perfusion images only. The purpose of this study was to investigate whether the automated interpretation of myocardial perfusion images with the use of artificial neural networks was improved if clinical data were assessed in addition to the perfusion images.

METHODS AND RESULTS

A population of 229 patients who had undergone both rest-stress myocardial perfusion scintigraphy in conjunction with an exercise test and coronary angiography, with no more than 3 months elapsing between the 2 examinations, were studied. The networks were trained to detect coronary artery disease or myocardial ischemia with the use of 2 different gold standards. The first was based on coronary angiography, and the second was based on all data available (including perfusion scintigrams, coronary angiography, exercise test, resting electrocardiography, patient history, etc). The performance of the neural networks was quantified as areas under the receiver operating characteristic curves. The results showed that the neural networks trained with perfusion images performed better than those trained with exercise data (0.78 vs 0.55, P <.0001), with coronary angiography used as the gold standard. Furthermore, the networks did not improve when data from the exercise test were used as input in addition to the perfusion images (0.78 vs 0.77, P =.6).

CONCLUSIONS

The results show that the clinically important information in combined exercise test and myocardial scintigraphy could be found in the perfusion images. Exercise test information did not improve upon the accuracy of automated neural network interpretation of myocardial perfusion images in a receiver operator characteristic analysis of test accuracy.

Scandinavian test of artificial neural network for classification of myocardial perfusion images

Artificial neural networks are systems of elementary computing units capable of learning from examples. They have been applied to automated interpretation of myocardial perfusion images and have been shown to perform even better than experienced physicians. It has been shown that physicians interpreting myocardial perfusion images benefit from the advice of such networks. These networks have been developed and validated in the same hospital. However, widespread use of neural networks will only take place if the networks can maintain a high accuracy in other hospitals, i.e. hospitals using different gamma cameras, different acquisition techniques, different study protocols, etc. The purpose of this study was to develop a neural network in one hospital and test it in another. An artificial neural network was trained to detect coronary artery disease using myocardial perfusion scintigrams from 135 patients at a Swedish hospital. Thereafter, this network was tested using scintigrams from 68 patients at a Danish hospital and compared to six criteria based on expert physician analysis and quantitative analysis by the CEqual program. The sensitivity of the network was significantly higher than that of one of the physician criteria (0. 92 versus 0.71) and two of the CEqual-based criteria (0.94 versus 0. 63 and 0.96 versus 0.65) compared at equal specificities. It was concluded that an artificial neural network can maintain high accuracy in a hospital other than the one where it was developed.

Myocardial SPET: artificial neural networks describe extent and severity of perfusion defects

Artificial neural networks are computer programs that learn from examples. They have been successfully used to detect coronary artery disease from myocardial perfusion images. The purpose of the present study was to develop neural networks that could classify myocardial scintigrams regarding reversibility, localization, severity and extent of perfusion defects. Rest/exercise technetium-99m sestamibi scintigrams from 338 patients were studied. The classifications of two experts were employed as the gold standard. Artificial neural networks were trained to classify both reversible (ischaemia) and non-reversible (infarct) defects in three vascular territories, corresponding to the main coronary arteries. The extent (small or large) and severity (mild or severe) of the defects were described by the networks. After the training process, separate test sets were used to compare the neural networks with one of the experts who reclassified the scintigrams two months later. The neural networks made correct classifications in 71% of the test cases and the human expert in 70% (P=0.10). It was concluded that artificial neural networks can be trained to make clinical interpretations of myocardial perfusion scintigrams. The results indicate that networks can assist physicians in achieving correct interpretations and thereby improve the diagnostic accuracy of medical imaging.

Quantitative SPECT techniques

Quantitative imaging involves first, a set of measurements that characterize an image. There are several variations of technique, but the basic measurements that are used for single photon emission computed tomography (SPECT) perfusion images are reasonably standardized. Quantification currently provides only relative tracer activity within the myocardial regions defined by an individual SPECT acquisition. Absolute quantification is still a work in progress. Quantitative comparison of absolute changes in tracer uptake comparing a stress and rest study or preintervention and postintervention study would be useful and could be done, but most commercial systems do not maintain the data normalization that is necessary for this. Measurements of regional and global function are now possible with electrocardiography (ECG) gating, and this provides clinically useful adjunctive data. Techniques for measuring ventricular function are evolving and promise to provide clinically useful accuracy. The computer can classify images as normal or abnormal by comparison with a normal database. The criteria for this classification involve more than just checking the normal limits. The images should be analyzed to measure how far they deviate from normal, and this information can be used in conjunction with pretest likelihood to indicate the level of statistical certainty that an individual patient has a true positive or true negative test. The interface between the computer and the clinician interpreter is an important part of the process. Especially when both perfusion and function are being determined, the ability of the interpreter to correctly assimilate the data is essential to the use of the quantitative process. As we become more facile with performing and recording objective measurements, the significance of the measurements in terms of risk evaluation, viability assessment, and outcome should be continually enhanced.

Scintigraphic diagnosis of coronary artery disease: myocardial bull's-eye images contain the important information

The bull's-eye image, also called polar map image, has been developed as an important display for the visual and quantitative analysis of myocardial perfusion scintigrams. Quantitative analysis can be performed for example by comparing areas in the bull's-eye image with normal limits or by processing it using artificial neural networks. The usefulness of such methods is highly dependent on the information content of the bull's-eye image. The purpose of this study was to investigate whether there is more diagnostically important information in a set consisting of the myocardial bull's-eye image plus tomographic slice image than in the bull's-eye image alone. A population of 135 patients who had undergone both myocardial scintigraphy and coronary angiography, with no more than 3 months elapsing between the two examinations, was studied retrospectively. Four experienced observers independently classified visually all scintigrams regarding the presence/absence of coronary artery disease in two vascular territories using a four-grade scale. The observers classified the scintigrams once viewing bull's-eye images only, and once viewing tomographic slices and bull's-eye images. Coronary angiography was used as gold standard. The classifications were evaluated using the areas under the receiver operating characteristics (ROC) curves. The classifications based on bull's-eye images only were slightly more accurate than those based on tomographic slices and bull's-eye images in one of the two vascular territories (ROC areas of 0.66 vs. 0.64). The opposite relationship was found in the other vascular territory (0.78 vs. 0.81). None of the differences was statistically significant. In conclusion, the diagnostically important information for the diagnosis of coronary artery disease by myocardial perfusion scintigraphy is present in the bull's-eye image.

Identification of a hypoperfused segment in bull's-eye myocardial perfusion images using a feed forward neural network

Artificial neural networks are computer systems which can be trained to recognize similarities in patterns and which learn by example; one of the more straightforward types being the feed forward neural network (FFNN). We previously reported the use of FFNNs for classification of hypoperfusion patterns in bull's-eye representation of 201Tl single photon emission tomography myocardial perfusion studies and showed that, when such an image was divided into 24 segments, FFNNs could detect perfusion defects without direct comparison to a normal data base. This has been extended in this investigation to assess the ability of an FFNN, trained on data in which only a single segment was hypoperfused, to detect this abnormal segment when the hypoperfusion pattern of the other segments in the image varied. The results indicated that the network could reliably determine whether a segment was normally or under perfused, with accuracies of 99% and 100%, respectively, if all other segments were normally perfused. It could also reliably detect a normally perfused segment, even if other segments were hypoperfused, with accuracies of 95% and 98%. The network was less reliable, however, in detecting a hypoperfused segment when other segments were also hypoperfused, showing accuracies of only 74% and 88%.