Computerized image-searching method for finding correct patients for misfiled chest radiographs in a PACS server by use of biological fingerprints

Gray-Scale Extraction of Bone Features from Chest Radiographs Based on Deep Learning Technique for Personal Identification and Classification in Forensic Medicine

Post-mortem (PM) imaging has potential for identifying individuals by comparing ante-mortem (AM) and PM images. Radiographic images of bones contain significant information for personal identification. However, PM images are affected by soft tissue decomposition; therefore, it is desirable to extract only images of bones that change little over time. This study evaluated the effectiveness of U-Net for bone image extraction from two-dimensional (2D) X-ray images. Two types of pseudo 2D X-ray images were created from the PM computed tomography (CT) volumetric data using ray-summation processing for training U-Net. One was a projection of all body tissues, and the other was a projection of only bones. The performance of the U-Net for bone extraction was evaluated using Intersection over Union, Dice coefficient, and the area under the receiver operating characteristic curve. Additionally, AM chest radiographs were used to evaluate its performance with real 2D images. Our results indicated that bones could be extracted visually and accurately from both AM and PM images using U-Net. The extracted bone images could provide useful information for personal identification in forensic pathology.

[Development of Auto Dense-breast Classification on Mammography Images Using Image Similarity]

PURPOSE

In Japan, radiologists perform qualitative visual classification to define four categories of mammary gland density. However, an objective estimation of mammary gland density is necessary. To address this, we developed an automatic classification software using image similarity.

METHODS

We prepared 741 cases of mediolateral oblique images (MLO) for evaluation, and they were diagnosed as normal among the mammography images taken at our hospital. Image matching was performed using the evaluation images and an image database for breast density determination. In this study, the image similarity used zero normalized cross-correlation (ZNCC) as an index. In addition, if the breast thickness is less than 30 mm and each breast density category ZNNC has the same value, the category is evaluated on the fat side. We compared the results of qualitative visual classification and automatic classification methods to assess consistency.

RESULTS

The agreement with the subjective breast composition classification was 78.5%, and the weighted kappa coefficient was 0.98. One mismatched case was evaluated on the higher density side with the same ZNCC value between categories and a breast thickness greater than 30 mm.

CONCLUSION

Image similarity provides an excellent estimation of quantification of breast density. This system could contribute to improving the efficiency of the mammography screening system.

Patient Re-Identification Based on Deep Metric Learning in Trunk Computed Tomography Images Acquired from Devices from Different Vendors

During radiologic interpretation, radiologists read patient identifiers from the metadata of medical images to recognize the patient being examined. However, it is challenging for radiologists to identify "incorrect" metadata and patient identification errors. We propose a method that uses a patient re-identification technique to link correct metadata to an image set of computed tomography images of a trunk with lost or wrongly assigned metadata. This method is based on a feature vector matching technique that uses a deep feature extractor to adapt to the cross-vendor domain contained in the scout computed tomography image dataset. To identify "incorrect" metadata, we calculated the highest similarity score between a follow-up image and a stored baseline image linked to the correct metadata. The re-identification performance tests whether the image with the highest similarity score belongs to the same patient, i.e., whether the metadata attached to the image are correct. The similarity scores between the follow-up and baseline images for the same "correct" patients were generally greater than those for "incorrect" patients. The proposed feature extractor was sufficiently robust to extract individual distinguishable features without additional training, even for unknown scout computed tomography images. Furthermore, the proposed augmentation technique further improved the re-identification performance of the subset for different vendors by incorporating changes in width magnification due to changes in patient table height during each examination. We believe that metadata checking using the proposed method would help detect the metadata with an "incorrect" patient identifier assigned due to unavoidable errors such as human error.

Patient Identification Based on Deep Metric Learning for Preventing Human Errors in Follow-up X-Ray Examinations

Biological fingerprints extracted from clinical images can be used for patient identity verification to determine misfiled clinical images in picture archiving and communication systems. However, such methods have not been incorporated into clinical use, and their performance can degrade with variability in the clinical images. Deep learning can be used to improve the performance of these methods. A novel method is proposed to automatically identify individuals among examined patients using posteroanterior (PA) and anteroposterior (AP) chest X-ray images. The proposed method uses deep metric learning based on a deep convolutional neural network (DCNN) to overcome the extreme classification requirements for patient validation and identification. It was trained on the NIH chest X-ray dataset (ChestX-ray8) in three steps: preprocessing, DCNN feature extraction with an EfficientNetV2-S backbone, and classification with deep metric learning. The proposed method was evaluated using two public datasets and two clinical chest X-ray image datasets containing data from patients undergoing screening and hospital care. A 1280-dimensional feature extractor pretrained for 300 epochs performed the best with an area under the receiver operating characteristic curve of 0.9894, an equal error rate of 0.0269, and a top-1 accuracy of 0.839 on the PadChest dataset containing both PA and AP view positions. The findings of this study provide considerable insights into the development of automated patient identification to reduce the possibility of medical malpractice due to human errors.

Development of individual identification method using thoracic vertebral features as biometric fingerprints

Identification of individuals is performed when a corpse is found after a natural disaster, incident, or accident. DNA and dental records are frequently used as biometric fingerprints; however, identification may be difficult in some cases due to decomposition or damage to the corpse. The present study aimed to develop an individual identification method using thoracic vertebral features as a biological fingerprint. In this method, the shortest diameter in height, width, and depth of the thoracic vertebrae in the postmortem image and a control antemortem were recorded and a database was compiled using this information. The Euclidean distance or the modified Hausdorff distance was calculated as the distance between two points on the three-dimensional feature space of these measurement data. The thoracic vertebrae T1-12 were measured and the pair with the smallest distance was considered to be from the same person. The accuracy of this method for identifying individuals was evaluated by matching images of 82 cases from a total of 702 antemortem images and showed a hit ratio of 100%. Therefore, this method may be used to identify individuals with high accuracy.

Biological fingerprint for patient verification using trunk scout views at various scan ranges in computed tomography

Immediate verification of whether a patient being examined is correct is desirable, even if the scan ranges change during different examinations for the same patient. This study proposes an advanced biological fingerprint technique for the rapid and reliable verification of various scan ranges in computed tomography (CT) scans of the torso of the same patient. The method comprises the following steps: geometric correction of different scans, local feature extraction, mismatch elimination, and similarity evaluation. The geometric magnification correction was aligned at the scanner table height in the first two steps, and the local maxima were calculated as the local features. In the third step, local features from the follow-up scout image are matched to those in the corresponding baseline scout image via template matching and outlier elimination via a robust estimator. We evaluated the correspondence rate based on the inlier ratio between corresponding scout images. The ratio of inliers between the baseline and follow-up scout images was assessed as the similarity score. The clinical dataset, including chest, abdomen-pelvis, and chest-abdomen-pelvis scans, included 600 patients (372 men, 68 ± 12 years) who underwent two routine torso CT examinations. The highest area under the receiver operating characteristic curve (AUC) was 0.996, which was sufficient for patient verification. Moreover, the verification results were comparable to the conventional method, which uses scout images in the same scan range. Patient identity verification was achieved before the main scan, even in follow-up torso CT, under different scan ranges.

Development of a computer-aided quality assurance support system for identifying hand X-ray image direction using deep convolutional neural network

The convenience of imaging has improved with digitization; however, there has been no progress in the methods used to prevent human error. Therefore, radiographic incidents and accidents are not prevented. In Japan, image interpretation is conducted for incident prevention; nevertheless, in some cases, incidents are overlooked. Thus, assistance from a computer-aided quality assurance support system is important. This study developed a method to identify hand image direction, which is an elementary technology of a computer-aided quality assurance support system. In total, 14,236 hand X-ray images were used to classify hand directions (upward, downward, rightward, and leftward) commonly evaluated in clinical settings. The accuracy of the conventional classification method using original images, classification method with histogram equation images, and a novel classification method using binarization images for background removal via U-Net segmentation was evaluated. The following classification accuracy rates were achieved: 89.20% if the original image was input, 99.10% if the histogram equation image was input, and 99.70% if binarization images for background removal via U-Net segmentation was input. Our computer-aided quality assurance support system can be used to identify hand direction with high accuracy.

[Evaluation of Radiograph Accuracy in Skull X-ray Images Using Deep Learning]

PURPOSE

Accurate positioning is essential for radiography, and it is especially important to maintain image reproducibility in follow-up observations. The decision on re-taking radiographs is entrusting to the individual radiological technologist. The evaluation is a visual and qualitative evaluation and there are individual variations in the acceptance criteria. In this study, we propose a method of image evaluation using a deep convolutional neural network (DCNN) for skull X-ray images.

METHOD

The radiographs were obtained from 5 skull phantoms and were classified by simple network and VGG16. The discrimination ability of DCNN was verified by recognizing the X-ray projection angle and the retake of the radiograph. DCNN architectures were used with the different input image sizes and were evaluated by 5-fold cross-validation and leave-one-out cross-validation.

RESULT

Using the 5-fold cross-validation, the classification accuracy was 99.75% for the simple network and 80.00% for the VGG16 in small input image sizes, and when the input image size was general image size, simple network and VGG16 showed 79.58% and 80.00%, respectively.

CONCLUSION

The experimental results showed that the combination between the small input image size, and the shallow DCNN architecture was suitable for the four-category classification in X-ray projection angles. The classification accuracy was up to 99.75%. The proposed method has the potential to automatically recognize the slight projection angles and the re-taking images to the acceptance criteria. It is considered that our proposed method can contribute to feedback for re-taking images and to reduce radiation dose due to individual subjectivity.

New solutions for automated image recognition and identification: challenges to radiologic technology and forensic pathology

This paper outlines the history of biometrics for personal identification, the current status of the initial biological fingerprint techniques for digital chest radiography, and patient verification during medical imaging, such as computed tomography and magnetic resonance imaging. Automated image recognition and identification developed for clinical images without metadata could also be applied to the identification of victims in mass disasters or other unidentified individuals. The development of methods that are adaptive to a wide range of recent imaging modalities in the fields of radiologic technology, patient safety, forensic pathology, and forensic odontology is still in its early stages. However, its importance in practice will continue to increase in the future.

Biological fingerprint using scout computed tomographic images for positive patient identification

PURPOSE

Management of patient identification is an important issue that should be addressed to ensure patient safety while using modern healthcare systems. Patient identification errors can be mainly attributed to human errors or system problems. An error-tolerant system, such as a biometric system, should be able to prevent or mitigate potential misidentification occurrences. Herein, we propose the use of scout computed tomography (CT) images for biometric patient identity verification and present the quantitative accuracy outcomes of using this technique in a clinical setting.

METHODS

Scout CT images acquired from routine examinations of the chest, abdomen, and pelvis were used as biological fingerprints. We evaluated the resemblance of the follow-up with the baseline image by comparing the estimates of the image characteristics using local feature extraction and matching algorithms. The verification performance was evaluated according to the receiver operating characteristic (ROC) curves, area under the ROC curves (AUC), and equal error rates (EER). The closed-set identification performance was evaluated according to the cumulative match characteristic curves and rank-one identification rates (R1).

RESULTS

A total of 619 (383 males, 236 females, age range 21-92 years) patients who underwent baseline and follow-up chest-abdomen-pelvis CT scans on the same CT system were analyzed for verification and closed-set identification. The highest performances of AUC, EER, and R1 were 0.998, 1.22%, and 99.7%, respectively, in the considered evaluation range. Furthermore, to determine whether the performance decreased in the presence of metal artifacts, the patients were classified into two groups, namely scout images with (255 patients) and without (364 patients) metal artifacts, and the significance test was performed for two ROC curves using the unpaired Delong's test. No significant differences were found between the ROC performances in the presence and absence of metal artifacts when using a sufficient number of local features. Our proposed technique demonstrated that the performance was comparable to that of conventional biometrics methods when using chest, abdomen, and pelvis scout CT images. Thus, this method has the potential to discover inadequate patient information using the available chest, abdomen, and pelvis scout CT image; moreover, it can be applied widely to routine adult CT scans where no significant body structure effects due to illness or aging are present.

CONCLUSIONS

Our proposed method can obtain accurate patient information available at the point-of-care and help healthcare providers verify whether a patient's identity is matched accurately. We believe the method to be a key solution for patient misidentification problems.

Clinical application of biological fingerprints extracted from averaged chest radiographs and template-matching technique for preventing left-right flipping mistakes in chest radiography

We aimed to evaluate the identification performance achieved using biological fingerprints extracted from averaged chest radiographs and template-matching techniques for the prevention of left-right flipping mistakes. We produced averaged chest radiographs for each sex by averaging 100 posteroanterior chest radiographs. Further, 400 and 566 chest radiographs were used in consistency and validation tests, respectively, and they were flipped horizontally to produce flipped chest radiographs under the assumption that the left-right flipping mistake occurred. The correlation values obtained with chest radiographs and those obtained with flipped chest radiographs were calculated. When we used correlation indices calculated from the correlation values from four biological fingerprints except for the lung apex, 96.5% (386/400) and 95.8% (542/566) of the left or right sides were identified correctly in the consistency and validation tests, respectively. This result indicates that our proposed method would be promising for the prevention of left-right flipping mistakes.

[Usefulness of Biological Fingerprints and Template Matching Techniques in Bedside Chest Radiography for Patient Identification and Preventing Filing Mistakes]

The purpose of this study was to investigate whether patients can be identified by using biological fingerprints extracted from bedside chest radiographs and template matching techniques for preventing filing mistakes in a picture archiving and communication system (PACS) server. A total of 400 bedside chest radiographs from 100 male and 100 female patients with current and previous images were used for evaluating patient identification performance. Five biological fingerprints were extracted from 200 previous images using the averaged bedside chest radiographs, produced for each sex and detector size. The correlation values of 200 same patients and 39,800 different patients were calculated as a similarity index, and used for the receiver operating characteristic (ROC) analysis. The patient identification performance was examined by using the correlation index calculated by the summation of correlation values obtained from five biological fingerprints. The sensitivity at 90.0% specificity was calculated using the correlation index. The correlation index for same patients was higher than that for different patients. The area under the ROC curve was 0.974. The patient identification performance was 76.0% (152/200), and the sensitivity at 90.0% specificity was 93.4% (37168/39800). Our results suggest that the proposed method may potentially be useful for preventing filing mistakes in bedside chest radiographs on a PACS server.

Usefulness of biological fingerprint in magnetic resonance imaging for patient verification

The purpose of our study is to investigate the feasibility of automated patient verification using multi-planar reconstruction (MPR) images generated from three-dimensional magnetic resonance (MR) imaging of the brain. Several anatomy-related MPR images generated from three-dimensional fast scout scan of each MR examination were used as biological fingerprint images in this study. The database of this study consisted of 730 temporal pairs of MR examination of the brain. We calculated the correlation value between current and prior biological fingerprint images of the same patient and also all combinations of two images for different patients to evaluate the effectiveness of our method for patient verification. The best performance of our system were as follows: a half-total error rate of 1.59 % with a false acceptance rate of 0.023 % and a false rejection rate of 3.15 %, an equal error rate of 1.37 %, and a rank-one identification rate of 98.6 %. Our method makes it possible to verify the identity of the patient using only some existing medical images without the addition of incidental equipment. Also, our method will contribute to patient misidentification error management caused by human errors.

[Comparative Study of Patient Identifications for Conventional and Portable Chest Radiographs Utilizing ROC Analysis]

To evaluate the patient identification ability of radiographers, previous and current chest radiographs were assessed with observer study utilizing a receiver operating characteristics (ROCs) analysis. This study included portable and conventional chest radiographs from 43 same and 43 different patients. The dataset used in this study was divided into the three following groups: (1) a pair of portable radiographs, (2) a pair of conventional radiographs, and (3) a combination of each type of radiograph. Seven observers participated in this ROC study, which aimed to identify same or different patients, using these datasets. ROC analysis was conducted to calculate the average area under ROC curve obtained by each observer (AUCave), and a statistical test was performed using the multi-reader multi-case method. Comparable results were obtained with pairs of portable (AUCave: 0.949) and conventional radiographs (AUCave: 0.951). In a comparison between the same modality, there were no significant differences. In contrast, the ability to identify patients by comparing a portable and conventional radiograph (AUCave: 0.873) was lower than with the matching datasets (p=0.002 and p=0.004, respectively). In conclusion, the use of different imaging modalities reduces radiographers' ability to identify their patients.

[Development of an automated patient recognition method for chest CT images using a template-matching technique]

If patient information, such as identification number or patient name, has been entered incorrectly in a picture archiving and communication system (PACS) environment, the image may be stored in the wrong place. To prevent such cases of misfiling, we have developed an automated patient recognition system for chest CT images. The image database consisted of 100 cases with present and previous chest CT images. A volume of interest (VOI) measuring 40 × 40 pixels was selected from the left lung region, bronchus region, and right lung region. Next, the overall lung region and these three regions in a current chest CT image were used as a template for determining the residual value with the corresponding four regions in previous chest CT images. To ensure separation between the same and different patients, we applied a combined analysis that employed the ruled-based plus artificial neural network (ANN) method. The overall performance of the method developed was examined in terms of receiver operating characteristic (ROC) curves. The performance of the rule-based plus ANN method using a combination of the four regions was higher than obtained using a rule-based method using these four regions separately. The automated patient recognition system using the rule-based plus ANN method achieved an area under the curve (AUC) value of 0.987. This automated patient recognition method for chest CT images is promising for helping to retrieve misfiled patient images, especially in a PACS environment.