1
|
Pastor M, Dabli D, Lonjon R, Serrand C, Snene F, Trad F, de Oliveira F, Beregi JP, Greffier J. Comparison between artificial intelligence solution and radiologist for the detection of pelvic, hip and extremity fractures on radiographs in adult using CT as standard of reference. Diagn Interv Imaging 2024:S2211-5684(24)00197-9. [PMID: 39299831 DOI: 10.1016/j.diii.2024.09.004] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/19/2024] [Revised: 09/03/2024] [Accepted: 09/04/2024] [Indexed: 09/22/2024]
Abstract
PURPOSE The purpose of this study was to compare the diagnostic performance of an artificial intelligence (AI) solution for the detection of fractures of pelvic, proximal femur or extremity fractures in adults with radiologist interpretation of radiographs, using standard dose CT examination as the standard of reference. MATERIALS AND METHODS This retrospective study included 94 adult patients with suspected bone fractures who underwent a standard dose CT examination and radiographs of the pelvis and/or hip and extremities at our institution between January 2022 and August 2023. For all patients, an AI solution was used retrospectively on the radiographs to detect and localize bone fractures of the pelvis and/or hip and extremities. Results of the AI solution were compared to the reading of each radiograph by a radiologist using McNemar test. The results of standard dose CT examination as interpreted by a senior radiologist were used as the standard of reference. RESULT A total of 94 patients (63 women; mean age, 56.4 ± 22.5 [standard deviation] years) were included. Forty-seven patients had at least one fracture, and a total of 71 fractures were deemed present using the standard of reference (25 hand/wrist, 16 pelvis, 30 foot/ankle). Using the standard of reference, the analysis of radiographs by the AI solution resulted in 58 true positive, 13 false negative, 33 true negative and 15 false positive findings, yielding 82 % sensitivity (58/71; 95 % confidence interval [CI]: 71-89 %), 69 % specificity (33/48; 95 % CI: 55-80 %), and 76 % accuracy (91/119; 95 % CI: 69-84 %). Using the standard of reference, the reading of the radiologist resulted in 65 true positive, 6 false negative, 42 true negative and 6 false positive findings, yielding 92 % sensitivity (65/71; 95 % CI: 82-96 %), 88 % specificity (42/48; 95 % CI: 75-94 %), and 90 % accuracy (107/119; 95 % CI: 85-95 %). The radiologist outperformed the AI solution in terms of sensitivity (P = 0.045), specificity (P = 0.016), and accuracy (P < 0.001). CONCLUSION In this study, the radiologist outperformed the AI solution for the diagnosis of pelvic, hip and extremity fractures of the using radiographs. This raises the question of whether a strong standard of reference for evaluating AI solutions should be used in future studies comparing AI and human reading in fracture detection using radiographs.
Collapse
Affiliation(s)
- Maxime Pastor
- IMAGINE UR UM 103, Montpellier University, Department of Medical Imaging, Nîmes University Hospital, 30900 Nîmes, France.
| | - Djamel Dabli
- IMAGINE UR UM 103, Montpellier University, Department of Medical Imaging, Nîmes University Hospital, 30900 Nîmes, France
| | - Raphaël Lonjon
- IMAGINE UR UM 103, Montpellier University, Department of Medical Imaging, Nîmes University Hospital, 30900 Nîmes, France
| | - Chris Serrand
- Department of Biostatistics, Epidemiology, Public Health and Innovation in Methodology, Nîmes University Hospital, Univ. Montpellier, 30900 Nîmes, France
| | - Fehmi Snene
- IMAGINE UR UM 103, Montpellier University, Department of Medical Imaging, Nîmes University Hospital, 30900 Nîmes, France
| | - Fayssal Trad
- IMAGINE UR UM 103, Montpellier University, Department of Medical Imaging, Nîmes University Hospital, 30900 Nîmes, France
| | - Fabien de Oliveira
- IMAGINE UR UM 103, Montpellier University, Department of Medical Imaging, Nîmes University Hospital, 30900 Nîmes, France
| | - Jean-Paul Beregi
- IMAGINE UR UM 103, Montpellier University, Department of Medical Imaging, Nîmes University Hospital, 30900 Nîmes, France
| | - Joël Greffier
- IMAGINE UR UM 103, Montpellier University, Department of Medical Imaging, Nîmes University Hospital, 30900 Nîmes, France
| |
Collapse
|
2
|
Dankelman LHM, Barvelink B, Verhofstad MHJ, Wijffels MME, Colaris JW. Traditional radiography versus computed tomography to assess reduced distal radius fractures. Eur J Trauma Emerg Surg 2024:10.1007/s00068-024-02598-5. [PMID: 38985187 DOI: 10.1007/s00068-024-02598-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/15/2024] [Accepted: 07/02/2024] [Indexed: 07/11/2024]
Abstract
INTRODUCTION This study compares computed tomography (CT) with plain radiography in its ability to assess distal radius fracture (DRF) malalignment after closed reduction and cast immobilization. METHODS Malalignment is defined as radiographic fracture alignment beyond threshold values according to the Dutch guideline encompassing angulation, inclination, positive ulnar variance and intra-articular step-off or gap. After identifying 96 patients with correct alignment on initial post-reduction radiographs, we re-assessed alignment on post-reduction CT scans. RESULTS Significant discrepancies were found between radiographs and CT scans in all measurement parameters. Notably, intra-articular step-off and gap variations on CT scans led to the reclassification of the majority of cases from correct alignment to malalignment. CT scans showed malalignment in 53% of cases, of which 73% underwent surgery. CONCLUSION When there is doubt about post-reduction alignment based on radiograph imaging, additional CT scanning often reveals malalignment, primarily due to intra-articular incongruency.
Collapse
Affiliation(s)
- Lente H M Dankelman
- Trauma Research Unit, Department of Surgery, Erasmus MC, University Medical Center Rotterdam, 3000 CA, P.O. Box 2040, Rotterdam, The Netherlands.
| | - Britt Barvelink
- Department of Orthopaedics, Erasmus University Medical Centre, Rotterdam, The Netherlands
| | - Michael H J Verhofstad
- Trauma Research Unit, Department of Surgery, Erasmus MC, University Medical Center Rotterdam, 3000 CA, P.O. Box 2040, Rotterdam, The Netherlands
| | - Mathieu M E Wijffels
- Trauma Research Unit, Department of Surgery, Erasmus MC, University Medical Center Rotterdam, 3000 CA, P.O. Box 2040, Rotterdam, The Netherlands
| | - Joost W Colaris
- Department of Orthopaedics, Erasmus University Medical Centre, Rotterdam, The Netherlands
| |
Collapse
|
3
|
Jacques T, Cardot N, Ventre J, Demondion X, Cotten A. Commercially-available AI algorithm improves radiologists' sensitivity for wrist and hand fracture detection on X-ray, compared to a CT-based ground truth. Eur Radiol 2024; 34:2885-2894. [PMID: 37919408 DOI: 10.1007/s00330-023-10380-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/28/2022] [Revised: 09/25/2023] [Accepted: 09/27/2023] [Indexed: 11/04/2023]
Abstract
OBJECTIVES Algorithms for fracture detection are spreading in clinical practice, but the use of X-ray-only ground truth can induce bias in their evaluation. This study assessed radiologists' performances to detect wrist and hand fractures on radiographs, using a commercially-available algorithm, compared to a computerized tomography (CT) ground truth. METHODS Post-traumatic hand and wrist CT and concomitant X-ray examinations were retrospectively gathered. Radiographs were labeled based on CT findings. The dataset was composed of 296 consecutive cases: 118 normal (39.9%), 178 pathological (60.1%) with a total of 267 fractures visible in CT. Twenty-three radiologists with various levels of experience reviewed all radiographs without AI, then using it, blinded towards CT results. RESULTS Using AI improved radiologists' sensitivity (Se, 0.658 to 0.703, p < 0.0001) and negative predictive value (NPV, 0.585 to 0.618, p < 0.0001), without affecting their specificity (Sp, 0.885 vs 0.891, p = 0.91) or positive predictive value (PPV, 0.887 vs 0.899, p = 0.08). On the radiographic dataset, based on the CT ground truth, stand-alone AI performances were 0.771 (Se), 0.898 (Sp), 0.684 (NPV), 0.915 (PPV), and 0.764 (AUROC) which were lower than previously reported, suggesting a potential underestimation of the number of missed fractures in the AI literature. CONCLUSIONS AI enabled radiologists to improve their sensitivity and negative predictive value for wrist and hand fracture detection on radiographs, without affecting their specificity or positive predictive value, compared to a CT-based ground truth. Using CT as gold standard for X-ray labels is innovative, leading to algorithm performance poorer than reported elsewhere, but probably closer to clinical reality. CLINICAL RELEVANCE STATEMENT Using an AI algorithm significantly improved radiologists' sensitivity and negative predictive value in detecting wrist and hand fractures on radiographs, with ground truth labels based on CT findings. KEY POINTS • Using CT as a ground truth for labeling X-rays is new in AI literature, and led to algorithm performance significantly poorer than reported elsewhere (AUROC: 0.764), but probably closer to clinical reality. • AI enabled radiologists to significantly improve their sensitivity (+ 4.5%) and negative predictive value (+ 3.3%) for the detection of wrist and hand fractures on X-rays. • There was no significant change in terms of specificity or positive predictive value.
Collapse
Affiliation(s)
- Thibaut Jacques
- Department of Musculoskeletal Radiology, Lille University Hospital, Rue du Professeur Emile Laine, 59000, Lille, France.
- IRIS Radiology - Clinique Lille Sud, SOS Hands and Fingers, 96 Rue Gustave Delory, 59810, Lesquin, France.
| | - Nicolas Cardot
- Department of Musculoskeletal Radiology, Lille University Hospital, Rue du Professeur Emile Laine, 59000, Lille, France
| | | | - Xavier Demondion
- Department of Musculoskeletal Radiology, Lille University Hospital, Rue du Professeur Emile Laine, 59000, Lille, France
- Lille University School of Medicine, 59000, Lille, France
| | - Anne Cotten
- Department of Musculoskeletal Radiology, Lille University Hospital, Rue du Professeur Emile Laine, 59000, Lille, France
- Lille University School of Medicine, 59000, Lille, France
- MABLab - Marrow Adiposity and Bone Lab ULR4490, University of Lille, 59000, Lille, France
| |
Collapse
|
4
|
Mørup SD, Mussmann B, Pedersen MRV, Rasumssen LM, Gaarde K, Jensen J. 3D wrist imaging - Is it time for superman to retire? J Clin Imaging Sci 2023; 13:39. [PMID: 38205276 PMCID: PMC10778063 DOI: 10.25259/jcis_64_2023] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/24/2023] [Accepted: 11/23/2023] [Indexed: 01/12/2024] Open
Abstract
Objectives Computed tomography (CT) of the wrist may be challenged, due to patients' inability to extend the arm for a "Superman pose" resulting in increased radiation dose due to scatter. Alternative positions and less dose administering modalities such as 3D Cone-beam CT (CBCT) and single-shot CT could be considered. This phantom study aimed to estimate scatter radiation dose in different phantom positions using helical and single-shot CT and 3D CBCT. Material and Methods Wireless electronic dosimeters attached to the head and chest of an anthropomorphic phantom in various clinically relevant positions were used to measure scatter radiation. In helical CT, the following positions were used: Superman pose, semi-superman pose, wrist on the abdomen, and single-shot CT with the patient sitting in front of and behind the gantry. In 3D CBCT, the phantom was in a supine position with the arm extended laterally. Results Helical CT using the Superman pose resulted in a total scattered radiation dose of 64.8 µGy. The highest total dose (269.7 µGy) was obtained with the wrist positioned on the abdomen while the lowest total dose was achieved in single-shot CT with the phantom sitting behind the gantry with the forearm placed inside the gantry (3.2 µGy). The total dose in 3D CBCT was 171.1 µGy. Conclusion The commonly used semi-superman and wrist-on-abdomen positions in CT administer the highest scattered doses and should be avoided when either single-shot CT or 3D CBCT is available. Radiographers should carefully consider alternatives when a patient referred for wrist CT cannot comply with the Superman position.
Collapse
Affiliation(s)
| | - Bo Mussmann
- Department of Radiology, Odense University Hospital, Odense, Denmark
| | - Malene Roland Vils Pedersen
- Department of Regional Health Research, Faculty of Health Sciences, University of Southern Denmark, Odense, Denmark
| | | | - Katrine Gaarde
- Health Sciences Research Centre, University College Lilleaelt, Odense, Denmark
| | - Janni Jensen
- Research and Innovation Unit of Radiology, University of Southern Denmark, Odense, Denmark
| |
Collapse
|
5
|
Bry K, Kortesniemi M, Koivikko M, Kerttula L. Comparison of cone beam computed tomography and plane radiographs of radial fractures as a basis for radiographical measurements. BMC Med Imaging 2023; 23:125. [PMID: 37710172 PMCID: PMC10500865 DOI: 10.1186/s12880-023-01093-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/25/2021] [Accepted: 08/31/2023] [Indexed: 09/16/2023] Open
Abstract
BACKGROUND The purpose of this study was to determine whether radiological measurements of radial fracture position made in cone beam computed tomography (CBCT) projection images are comparable to those made on traditional radiographs and could potentially substitute them. METHODS Sixteen patients with fractures of the distal radius referred for radiographs were recruited for an additional CBCT scan which was performed immediately afterwards. Projection images and volumetric data were saved from the CBCT scans. Measurements of ulnar variance, radial inclination and volar tilt were made from all three sets of images. RESULTS Agreement of projection image based measurements with radiographs was nearly as good as as the agreement of cross sectional image measurements with radiographs. The average difference between the results for projection images and radiographs were -1.2 mm (SD 1.9 mm), for radial inclination 0.7° (SD 2.9°) and for volar tilt 1.9° (SD 5.6°). CONCLUSION Differences between radiological measurements between the modalities studied are small and projection images could be used for the assessment of distal radial fractures.
Collapse
Affiliation(s)
- Kristian Bry
- Bridge Hospital, HUS, Haartmaninkatu 4, 00029 Helsinki, Finland
- Department of Radiology, Helsinki Medical Imaging Center, Helsinki University Hospital and University of Helsinki, 00029 Helsinki, Finland
| | - Mika Kortesniemi
- Department of Radiology, Medical Imaging Center, University of Helsinki and Helsinki University Hospital, P.O.Box 263, HUS, 00029 Helsinki, Finland
| | - Mika Koivikko
- Bridge Hospital, HUS, Haartmaninkatu 4, 00029 Helsinki, Finland
- Department of Radiology, Helsinki Medical Imaging Center, Helsinki University Hospital and University of Helsinki, 00029 Helsinki, Finland
| | - Liisa Kerttula
- Bridge Hospital, HUS, Haartmaninkatu 4, 00029 Helsinki, Finland
- Department of Radiology, Helsinki Medical Imaging Center, Helsinki University Hospital and University of Helsinki, 00029 Helsinki, Finland
| |
Collapse
|
6
|
A Minority Class Balanced Approach Using the DCNN-LSTM Method to Detect Human Wrist Fracture. LIFE (BASEL, SWITZERLAND) 2023; 13:life13010133. [PMID: 36676082 PMCID: PMC9861673 DOI: 10.3390/life13010133] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 11/24/2022] [Revised: 12/19/2022] [Accepted: 12/27/2022] [Indexed: 01/06/2023]
Abstract
The emergency department of hospitals receives a massive number of patients with wrist fracture. For the clinical diagnosis of a suspected fracture, X-ray imaging is the major screening tool. A wrist fracture is a significant global health concern for children, adolescents, and the elderly. A missed diagnosis of wrist fracture on medical imaging can have significant consequences for patients, resulting in delayed treatment and poor functional recovery. Therefore, an intelligent method is needed in the medical department to precisely diagnose wrist fracture via an automated diagnosing tool by considering it a second option for doctors. In this research, a fused model of the deep learning method, a convolutional neural network (CNN), and long short-term memory (LSTM) is proposed to detect wrist fractures from X-ray images. It gives a second option to doctors to diagnose wrist facture using the computer vision method to lessen the number of missed fractures. The dataset acquired from Mendeley comprises 192 wrist X-ray images. In this framework, image pre-processing is applied, then the data augmentation approach is used to solve the class imbalance problem by generating rotated oversamples of images for minority classes during the training process, and pre-processed images and augmented normalized images are fed into a 28-layer dilated CNN (DCNN) to extract deep valuable features. Deep features are then fed to the proposed LSTM network to distinguish wrist fractures from normal ones. The experimental results of the DCNN-LSTM with and without augmentation is compared with other deep learning models. The proposed work is also compared to existing algorithms in terms of accuracy, sensitivity, specificity, precision, the F1-score, and kappa. The results show that the DCNN-LSTM fusion achieves higher accuracy and has high potential for medical applications to use as a second option.
Collapse
|
7
|
Lei M, Zhang M, Li H, Liu J, Chen J, Xu R, Xiao M. The diagnostic performance of ultra-low-dose 320-row detector CT with different reconstruction algorithms on limb joint fractures in the emergency department. Jpn J Radiol 2022; 40:1079-1086. [PMID: 35588348 DOI: 10.1007/s11604-022-01290-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/12/2022] [Accepted: 04/26/2022] [Indexed: 11/27/2022]
Abstract
BACKGROUND The aim of the study was to evaluate whether ultra-low-dose computed tomography (ULD-CT) could replace conventional-dose CT (CD-CT) for diagnosis of acute wrist, ankle, knee, and shoulder fractures in emergency departments (ED). METHODS We developed CD-CT and ULD-CT scanning schemes for the various joints of the four limbs and scanned emergency patients prospectively. When performing CD-CT, a conventional bone reconstruction algorithm was used, while ULD-CT used both soft tissue and bone algorithms. A five-point scale was used to evaluate whether ULD-CT image quality affected surgical planning. The image quality and diagnostic performance of different types of scanned and reconstructed images for diagnosing fractures were evaluated and compared. Effective radiation dose of each group was calculated. RESULTS Our study included 56 normal cases and 185 fracture cases. The combination of bone and soft tissue algorithms on ULD-CT can improve diagnostic performance, such that on ULD-CT, the sensitivity improved from 96.7% to 98.9%, specificity from 98.2% to 100%, positive predictive value from 99.4% to 100%, negative predictive value from 90.2% to 96.6% and diagnostic accuracy ranged from 97.5% to 99.1%. There were no statistically significant differences between ULD-CT and CD-CT on diagnostic performance (p values, 0.40-1.00). The radiation doses for ULD-CT protocols were only 3.0-7.7% of those for CD-CT protocols (all p < 0.01). CONCLUSIONS In the emergency department, the 320-row detector ULD-CT could replace CD-CT in the diagnosis of limb joint fractures. The combination of bone algorithm with soft tissue algorithm reconstruction can further improve the image quality and diagnostic performance.
Collapse
Affiliation(s)
- Ming Lei
- Department of Radiology, Zhuhai Hospital, Guangdong Provincial Hospital of Traditional Chinese Medicine, 53 Jingle Road, Zhuhai, Guangdong, China
| | - Meng Zhang
- Department of Radiology, Zhuhai Hospital, Guangdong Provincial Hospital of Traditional Chinese Medicine, 53 Jingle Road, Zhuhai, Guangdong, China
| | - Hongyi Li
- Department of Radiology, Zhuhai Hospital, Guangdong Provincial Hospital of Traditional Chinese Medicine, 53 Jingle Road, Zhuhai, Guangdong, China
| | - Jingfeng Liu
- Department of Radiology, Zhuhai Hospital, Guangdong Provincial Hospital of Traditional Chinese Medicine, 53 Jingle Road, Zhuhai, Guangdong, China
| | - Jun Chen
- Department of Radiology, Zhuhai Hospital, Guangdong Provincial Hospital of Traditional Chinese Medicine, 53 Jingle Road, Zhuhai, Guangdong, China
| | - Rulin Xu
- Research Collaboration, Canon Medical Systems Co., Ltd, Rm 2906, R&F Centre, No.10 Huaxia Road, Guangzhou, Guangdong, China
| | - Mengqiang Xiao
- Department of Radiology, Zhuhai Hospital, Guangdong Provincial Hospital of Traditional Chinese Medicine, 53 Jingle Road, Zhuhai, Guangdong, China.
| |
Collapse
|
8
|
Fitzpatrick E, Sharma V, Rojoa D, Raheman F, Singh H. The use of cone-beam computed tomography (CBCT) in radiocarpal fractures: a diagnostic test accuracy meta-analysis. Skeletal Radiol 2022; 51:923-934. [PMID: 34542681 PMCID: PMC8930799 DOI: 10.1007/s00256-021-03883-9] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 05/17/2021] [Revised: 07/25/2021] [Accepted: 07/25/2021] [Indexed: 02/02/2023]
Abstract
OBJECTIVE Occult radiocarpal fractures often present a diagnostic challenge to the emergency department. Accurate diagnosis of these injuries is crucial as a missed fracture can lead to significant morbidity. Cone-beam CT (CBCT) scan is a novel imaging modality, with minimal radiation exposure and comparatively fast acquisition time. Our aim was to evaluate its use in the diagnosis of cortical fractures in the upper limb extremity. MATERIALS AND METHODS We conducted a systematic review of literature and included all studies that evaluated the use of CBCT in the diagnosis of radiocarpal fractures. We used a mixed-effects logistic regression bivariate model to estimate the summary sensitivity and specificity and constructed hierarchical summary receiver operative characteristic curves (HSROC). RESULTS We identified 5 studies, with 439 patients, and observed CBCT to be 87.7% (95% CI 77.6-93.6) sensitive and 99.2% (95% CI 92.6-99.9) specific for scaphoid fractures. For carpal fractures, CBCT was observed to have a pooled sensitivity and specificity of 90.6% (95% CI 72.7-97.2) and 100% (95% CI 99-100) respectively. For distal radius fractures, CBCT sensitivity was 90% (95% CI 67-98) and specificity was 100% (95% CI 10-100). The overall inter-rater agreement effect was shown to be 0.89 (95% CI 0.82-0.96), which is deemed to be almost perfect. CONCLUSION CBCT is an accurate diagnostic tool for occult radiocarpal cortical fractures, which could replace or supplement radiographs. We believe CBCT has a promising role in the acute radiocarpal fracture diagnostic algorithm in both emergency and trauma departments.
Collapse
Affiliation(s)
- Emma Fitzpatrick
- Department of Trauma and Orthopaedics, Leicester Royal Infirmary, University Hospitals of Leicester, Leicester, UK
| | - Vivek Sharma
- Department of Trauma and Orthopaedics, Leicester Royal Infirmary, University Hospitals of Leicester, Leicester, UK
| | - Djamila Rojoa
- Department of Plastics Surgery, Leicester Royal Infirmary, University Hospital of Leicester, Leicester, UK
| | - Firas Raheman
- Department of Trauma and Orthopaedics, Leicester Royal Infirmary, University Hospitals of Leicester, Leicester, UK
| | - Harvinder Singh
- Department of Trauma and Orthopaedics, Leicester Royal Infirmary, University Hospitals of Leicester, Leicester, UK
| |
Collapse
|
9
|
Snaith B, Harris M, Hughes J, Spencer N, Shinkins B, Tachibana A, Bessant G, Robertshaw S. Evaluating the potential for cone beam CT to improve the suspected scaphoid fracture pathway: InSPECTED - A single-centre feasibility study. J Med Imaging Radiat Sci 2021; 53:35-40. [PMID: 34802964 DOI: 10.1016/j.jmir.2021.10.002] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/26/2021] [Revised: 10/07/2021] [Accepted: 10/18/2021] [Indexed: 10/19/2022]
Abstract
BACKGROUND The suspected scaphoid fracture remains a diagnostic conundrum with over-treatment a common risk-averse strategy. Cross-sectional imaging remains the gold standard with MRI recommended but CT used by some because of easier access or limited MRI availability. The aim of this feasibility study was to evaluate whether cone beam computed tomography (CBCT) could support early diagnosis, or exclusion, of scaphoid fractures. METHODS Patients with a suspected scaphoid were recruited fracture between March and July 2020. All underwent a 4-view X-ray. If this examination was normal, they were immediately referred for a CBCT scan of the wrist. Those with a normal scan were discharged to research follow-up at 2 and 6-weeks. RESULTS 68 participants were recruited, 55 had a normal or equivocal X-ray and underwent CBCT. Nine additional radiocarpal fractures (16.2%) were demonstrated on CBCT, the remainder were discharged to research follow-up. Based on the 2-week and 6-week follow up three patients (4.4%) were referred for MRI to investigate persistent symptoms with no bony injuries identified. CONCLUSIONS CBCT scans enabled a rapid pathway for the diagnosis or exclusion of scaphoid fractures, identifying other fractures and facilitating early treatment. The rapid pathway also enabled those with no bony injury to start rehabilitation, suggesting that patients can be safely discharged with safety-net advice following a CBCT scan.
Collapse
Affiliation(s)
- Beverly Snaith
- Radiology, Mid Yorkshire Hospitals NHS Trust, Wakefield, UK; Faculty of Health Studies, University of Bradford, Richmond Road, Bradford, BD7 1DP, UK.
| | - Martine Harris
- Radiology, Mid Yorkshire Hospitals NHS Trust, Wakefield, UK
| | - James Hughes
- Radiology, Mid Yorkshire Hospitals NHS Trust, Wakefield, UK
| | | | - Bethany Shinkins
- Academic Unit of Health Economics, University of Leeds, Leeds, UK
| | | | - Gareth Bessant
- Emergency Department, Mid Yorkshire Hospitals NHS Trust, Wakefield, UK
| | - Sarah Robertshaw
- Emergency Department, Mid Yorkshire Hospitals NHS Trust, Wakefield, UK
| |
Collapse
|
10
|
Keller G, Götz S, Kraus MS, Grünwald L, Springer F, Afat S. Radiation Dose Reduction in CT Torsion Measurement of the Lower Limb: Introduction of a New Ultra-Low Dose Protocol. Diagnostics (Basel) 2021; 11:diagnostics11071209. [PMID: 34359292 PMCID: PMC8304839 DOI: 10.3390/diagnostics11071209] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/29/2021] [Revised: 06/28/2021] [Accepted: 07/01/2021] [Indexed: 11/26/2022] Open
Abstract
This study analyzed the radiation exposure of a new ultra-low dose (ULD) protocol compared to a high-quality (HQ) protocol for CT-torsion measurement of the lower limb. The analyzed patients (n = 60) were examined in the period March to October 2019. In total, 30 consecutive patients were examined with the HQ and 30 consecutive patients with the new ULD protocol comprising automatic tube voltage selection, automatic exposure control, and iterative image reconstruction algorithms. Radiation dose parameters as well as the contrast-to-noise ratio (CNR) and diagnostic confidence (DC; rated by two radiologists) were analyzed and potential predictor variables, such as body mass index and body volume, were assessed. The new ULD protocol resulted in significantly lower radiation dose parameters, with a reduction of the median total dose equivalent to 0.17 mSv in the ULD protocol compared to 4.37 mSv in the HQ protocol (p < 0.001). Both groups showed no significant differences in regard to other parameters (p = 0.344–0.923). CNR was 12.2% lower using the new ULD protocol (p = 0.033). DC was rated best by both readers in every HQ CT and in every ULD CT. The new ULD protocol for CT-torsion measurement of the lower limb resulted in a 96% decrease of radiation exposure down to the level of a single pelvic radiograph while maintaining good image quality.
Collapse
Affiliation(s)
- Gabriel Keller
- Department of Diagnostic and Interventional Radiology, University Hospital Tübingen, Eberhard Karls University Tübingen, 72076 Tübingen, Germany; (S.G.); (M.S.K.); (S.A.)
- Correspondence: (G.K.); (F.S.)
| | - Simon Götz
- Department of Diagnostic and Interventional Radiology, University Hospital Tübingen, Eberhard Karls University Tübingen, 72076 Tübingen, Germany; (S.G.); (M.S.K.); (S.A.)
| | - Mareen Sarah Kraus
- Department of Diagnostic and Interventional Radiology, University Hospital Tübingen, Eberhard Karls University Tübingen, 72076 Tübingen, Germany; (S.G.); (M.S.K.); (S.A.)
| | - Leonard Grünwald
- Department of Traumatology and Reconstructive Surgery, BG Trauma Center Tübingen, Eberhard Karls University Tübingen, 72076 Tübingen, Germany;
| | - Fabian Springer
- Department of Diagnostic and Interventional Radiology, University Hospital Tübingen, Eberhard Karls University Tübingen, 72076 Tübingen, Germany; (S.G.); (M.S.K.); (S.A.)
- Department of Diagnostic Radiology, BG Trauma Center Tübingen, Eberhard Karls University Tübingen, 72076 Tübingen, Germany
- Correspondence: (G.K.); (F.S.)
| | - Saif Afat
- Department of Diagnostic and Interventional Radiology, University Hospital Tübingen, Eberhard Karls University Tübingen, 72076 Tübingen, Germany; (S.G.); (M.S.K.); (S.A.)
| |
Collapse
|
11
|
Grunz JP, Pennig L, Fieber T, Gietzen CH, Heidenreich JF, Huflage H, Gruschwitz P, Kuhl PJ, Petritsch B, Kosmala A, Bley TA, Gassenmaier T. Twin robotic x-ray system in small bone and joint trauma: impact of cone-beam computed tomography on treatment decisions. Eur Radiol 2020; 31:3600-3609. [PMID: 33280057 PMCID: PMC8128787 DOI: 10.1007/s00330-020-07563-5] [Citation(s) in RCA: 14] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/11/2020] [Revised: 09/24/2020] [Accepted: 11/25/2020] [Indexed: 12/11/2022]
Abstract
Objectives Trauma evaluation of extremities can be challenging in conventional radiography. A multi-use x-ray system with cone-beam CT (CBCT) option facilitates ancillary 3-D imaging without repositioning. We assessed the clinical value of CBCT scans by analyzing the influence of additional findings on therapy. Methods Ninety-two patients underwent radiography and subsequent CBCT imaging with the twin robotic scanner (76 wrist/hand/finger and 16 ankle/foot/toe trauma scans). Reports by on-call radiologists before and after CBCT were compared regarding fracture detection, joint affliction, comminuted injuries, and diagnostic confidence. An orthopedic surgeon recommended therapy based on reported findings. Surgical reports (N = 52) and clinical follow-up (N = 85) were used as reference standard. Results CBCT detected more fractures (83/64 of 85), joint involvements (69/53 of 71), and multi-fragment situations (68/50 of 70) than radiography (all p < 0.001). Six fractures suspected in radiographs were ruled out by CBCT. Treatment changes based on additional information from CBCT were recommended in 29 patients (31.5%). While agreement between advised therapy before CBCT and actual treatment was moderate (κ = 0.41 [95% confidence interval 0.35–0.47]; p < 0.001), agreement after CBCT was almost perfect (κ = 0.88 [0.83–0.93]; p < 0.001). Diagnostic confidence increased considerably for CBCT studies (p < 0.001). Median effective dose for CBCT was 4.3 μSv [3.3–5.3 μSv] compared to 0.2 μSv [0.1–0.2 μSv] for radiography. Conclusions CBCT provides advantages for the evaluation of acute small bone and joint trauma by detecting and excluding extremity fractures and fracture-related findings more reliably than radiographs. Additional findings induced therapy change in one third of patients, suggesting substantial clinical impact. Key Points • With cone-beam CT, extremity fractures and fracture-related findings can be detected and ruled out more reliably than with conventional radiography. • Additional diagnostic information provided by cone-beam CT scans has substantial impact on therapy in small bone and joint trauma. • For distal extremity injury assessment, one-stop-shop imaging without repositioning is feasible with the twin robotic x-ray system.
Collapse
Affiliation(s)
- Jan-Peter Grunz
- Department of Diagnostic and Interventional Radiology, University Hospital Würzburg, Oberdürrbacher Straße 6, 97080, Würzburg, Germany.
| | - Lenhard Pennig
- Institute for Diagnostic and Interventional Radiology, Faculty of Medicine and University Hospital Cologne, University of Cologne, Kerpener Straße 62, 50937, Cologne, Germany
| | - Tabea Fieber
- Department of Trauma, Hand, Plastic and Reconstructive Surgery, University Hospital Würzburg, Oberdürrbacher Straße 6, 97080, Würzburg, Germany
| | - Carsten Herbert Gietzen
- Department of Diagnostic and Interventional Radiology, University Hospital Würzburg, Oberdürrbacher Straße 6, 97080, Würzburg, Germany
| | - Julius Frederik Heidenreich
- Department of Diagnostic and Interventional Radiology, University Hospital Würzburg, Oberdürrbacher Straße 6, 97080, Würzburg, Germany
| | - Henner Huflage
- Department of Diagnostic and Interventional Radiology, University Hospital Würzburg, Oberdürrbacher Straße 6, 97080, Würzburg, Germany
| | - Philipp Gruschwitz
- Department of Diagnostic and Interventional Radiology, University Hospital Würzburg, Oberdürrbacher Straße 6, 97080, Würzburg, Germany
| | - Philipp Josef Kuhl
- Department of Diagnostic and Interventional Radiology, University Hospital Würzburg, Oberdürrbacher Straße 6, 97080, Würzburg, Germany
| | - Bernhard Petritsch
- Department of Diagnostic and Interventional Radiology, University Hospital Würzburg, Oberdürrbacher Straße 6, 97080, Würzburg, Germany
| | - Aleksander Kosmala
- Department of Diagnostic and Interventional Radiology, University Hospital Würzburg, Oberdürrbacher Straße 6, 97080, Würzburg, Germany
| | - Thorsten Alexander Bley
- Department of Diagnostic and Interventional Radiology, University Hospital Würzburg, Oberdürrbacher Straße 6, 97080, Würzburg, Germany
| | - Tobias Gassenmaier
- Department of Diagnostic and Interventional Radiology, University Hospital Würzburg, Oberdürrbacher Straße 6, 97080, Würzburg, Germany
| |
Collapse
|
12
|
Diagnostic accuracy of ultra-low-dose CT for torsion measurement of the lower limb. Eur Radiol 2020; 31:3574-3581. [PMID: 33241518 PMCID: PMC8249276 DOI: 10.1007/s00330-020-07528-8] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/13/2020] [Revised: 10/24/2020] [Accepted: 11/16/2020] [Indexed: 01/12/2023]
Abstract
Objectives The study aimed to investigate the diagnostic performance of simulated ultra-low-dose CT (ULD-CT) for torsion measurement of the lower limb. Methods Thirty retrospectively identified patients were included (32.3 ± 14.2 years; 14 women, 16 men). ULD-CT simulations were generated at dose levels of 100%, 10%, 5%, and 1% using two reconstruction methods: standard filtered back projection (FBP) and iterative reconstruction (ADMIRE). Two readers measured the lower limb torsion in all data sets. The readers also captured image noise in standardized anatomical landmarks. All data sets were evaluated regarding subjective diagnostic confidence (DC; 5-point Likert scale). Effective radiation dose of the original data sets and the simulated ULD-CT was compared. Results There was no significant difference of measured lower limb torsion in any simulated dose level compared to the original data sets in both readers. Dose length product (DLP) of the original examinations was 402.1 ± 4.3 mGy cm, which resulted in an effective radiation dose of 4.00 ± 2.12 mSv. Calculated effective radiation dose in ULD-CT at 1% of the original dose was 0.04 mSv. Image noise increased significantly with dose reduction (p < 0.0001) and was dependent on the reconstructional method (p < 0.0001) with less noise using ADMIRE compared to FBP. Both readers rated DC at doses 100%, 10%, and 5% with 5.0/5: there were no ratings worse than 3/5 at 1% dose level. Conclusions The results suggest that radiation dose reduction down to 1% of original CT dose levels may be achieved in CT torsion measurements of the lower limb without compromising diagnostic accuracy. Key Points • Modern CT delivers exceptional high image quality in musculoskeletal imaging, especially for evaluation of osseous structures. • Usually, this high image quality is accompanied by significant radiation exposure to the patient and may not always be required for the intended purpose, e.g., pure delineation of cortical bone of the lower limb. • This study shows the tremendous prospects of radiation dose reduction without compromising diagnostic confidence in CT torsion measurement of the lower limb.
Collapse
|
13
|
Krastman P, Mathijssen NM, Bierma-Zeinstra SMA, Kraan G, Runhaar J. Diagnostic accuracy of history taking, physical examination and imaging for phalangeal, metacarpal and carpal fractures: a systematic review update. BMC Musculoskelet Disord 2020; 21:12. [PMID: 31910838 PMCID: PMC6947988 DOI: 10.1186/s12891-019-2988-z] [Citation(s) in RCA: 14] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 09/03/2019] [Accepted: 12/04/2019] [Indexed: 02/02/2023] Open
Abstract
BACKGROUND The standard diagnostic work-up for hand and wrist fractures consists of history taking, physical examination and imaging if needed, but the supporting evidence for this work-up is limited. The purpose of this study was to systematically examine the diagnostic accuracy of tests for hand and wrist fractures. METHODS A systematic search for relevant studies was performed. Methodological quality was assessed and sensitivity (Se), specificity (Sp), accuracy, positive predictive value (PPV) and negative predictive value (NPV) were extracted from the eligible studies. RESULTS Of the 35 eligible studies, two described the diagnostic accuracy of history taking for hand and wrist fractures. Physical examination with or without radiological examination for diagnosing scaphoid fractures (five studies) showed Se, Sp, accuracy, PPV and NPV ranging from 15 to 100%, 13-98%, 55-73%, 14-73% and 75-100%, respectively. Physical examination with radiological examination for diagnosing other carpal bone fractures (one study) showed a Se of 100%, with the exception of the triquetrum (75%). Physical examination for diagnosing phalangeal and metacarpal fractures (one study) showed Se, Sp, accuracy, PPV and NPV ranging from 26 to 55%, 13-89%, 45-76%, 41-77% and 63-75%, respectively. Imaging modalities of scaphoid fractures showed predominantly low values for PPV and the highest values for Sp and NPV (24 studies). Magnetic Resonance Imaging (MRI), Computed Tomography (CT), Ultrasonography (US) and Bone Scintigraphy (BS) were comparable in diagnostic accuracy for diagnosing a scaphoid fracture, with an accuracy ranging from 85 to 100%, 79-100%, 49-100% and 86-97%, respectively. Imaging for metacarpal and finger fractures showed Se, Sp, accuracy, PPV and NPV ranging from 73 to 100%, 78-100%, 70-100%, 79-100% and 70-100%, respectively. CONCLUSIONS Only two studies were found on the diagnostic accuracy of history taking for hand and wrist fractures in the current review. Physical examination was of moderate use for diagnosing a scaphoid fracture and of limited use for diagnosing phalangeal, metacarpal and remaining carpal fractures. MRI, CT and BS were found to be moderately accurate for the definitive diagnosis of clinically suspected carpal fractures.
Collapse
Affiliation(s)
- Patrick Krastman
- Department of General Practice, Erasmus MC University Medical Center Rotterdam, Room NA1911 PO Box 2040, 3000 CA Rotterdam, the Netherlands
| | - Nina M. Mathijssen
- Department of Orthopaedic Surgery, Reinier de Graaf Groep, Reinier de Graafweg 5-11, 2625 AD Delft, the Netherlands
| | - Sita M. A. Bierma-Zeinstra
- Department of Orthopaedics, Erasmus MC University Medical Center Rotterdam, Room NA1920 PO Box 2040, 3000 CA Rotterdam, the Netherlands
- Department of General Practice, Erasmus MC University Medical Center Rotterdam, Room NA1920 PO Box 2040, 3000 CA Rotterdam, the Netherlands
| | - Gerald Kraan
- Department of Orthopaedic Surgery, Reinier de Graaf Groep, Reinier de Graafweg 5-11, 2625 AD Delft, the Netherlands
| | - Jos Runhaar
- Department of General Practice, Erasmus MC University Medical Center Rotterdam, Room NA1911 PO Box 2040, 3000 CA Rotterdam, the Netherlands
| |
Collapse
|
14
|
Alagic Z, Bujila R, Enocson A, Srivastava S, Koskinen SK. Ultra-low-dose CT for extremities in an acute setting: initial experience with 203 subjects. Skeletal Radiol 2020; 49:531-539. [PMID: 31501959 PMCID: PMC7021773 DOI: 10.1007/s00256-019-03309-7] [Citation(s) in RCA: 14] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 05/03/2019] [Revised: 08/28/2019] [Accepted: 08/29/2019] [Indexed: 02/02/2023]
Abstract
OBJECTIVE The purpose of this study was to assess if ultra-low-dose CT is a useful clinical alternative to digital radiographs in the evaluation of acute wrist and ankle fractures. MATERIALS AND METHODS An ultra-low-dose protocol was designed on a 256-slice multi-detector CT. Patients from the emergency department were evaluated prospectively. After initial digital radiographs, an ultra-low-dose CT was performed. Two readers independently analyzed the images. Also, the radiation dose, examination time, and time to preliminary report was compared between digital radiographs and CT. RESULTS In 207 extremities, digital radiography and ultra-low-dose CT detected 73 and 109 fractures, respectively (p < 0.001). The odds ratio for fracture detection with ultra-low-dose CT vs. digital radiography was 2.0 (95% CI, 1.4-3.0). CT detected additional fracture-related findings in 33 cases (15.9%) and confirmed or ruled out suspected fractures in 19 cases (9.2%). The mean effective dose was comparable between ultra-low-dose CT and digital radiography (0.59 ± 0.33 μSv, 95% CI 0.47-0.59 vs. 0.53 ± 0.43 μSv, 95% CI 0.54-0.64). The mean combined examination time plus time to preliminary report was shorter for ultra-low-dose CT compared to digital radiography (7.6 ± 2.5 min, 95% CI 7.1-8.1 vs. 9.8 ± 4.7 min, 95% CI 8.8-10.7) (p = 0.002). The recommended treatment changed in 34 (16.4%) extremities. CONCLUSIONS Ultra-low-dose CT is a useful alternative to digital radiography for imaging the peripheral skeleton in the acute setting as it detects significantly more fractures and provides additional clinically important information, at a comparable radiation dose. It also provides faster combined examination and reporting times.
Collapse
Affiliation(s)
- Zlatan Alagic
- Functional Unit for Musculoskeletal Radiology Function Imaging and Physiology, Karolinska University Hospital, Karolinska Vägen Solna, 17176 Stockholm, Sweden ,Clinical Science, Intervention and Technology (CLINTEC), Karolinska Institutet, Stockholm, Sweden
| | - Robert Bujila
- Functional Unit for Medical Radiation Physics and Nuclear Medicine, Function Imaging and Physiology, Karolinska University Hospital, Stockholm, Sweden ,Department of Physics, Royal Institute of Technology, Stockholm, Sweden
| | - Anders Enocson
- Department of Molecular Medicine and Surgery, Karolinska University Hospital, Karolinska Institutet, Stockholm, Sweden
| | - Subhash Srivastava
- Functional Unit for Musculoskeletal Radiology Function Imaging and Physiology, Karolinska University Hospital, Karolinska Vägen Solna, 17176 Stockholm, Sweden
| | - Seppo K. Koskinen
- Functional Unit for Musculoskeletal Radiology Function Imaging and Physiology, Karolinska University Hospital, Karolinska Vägen Solna, 17176 Stockholm, Sweden ,Clinical Science, Intervention and Technology (CLINTEC), Karolinska Institutet, Stockholm, Sweden
| |
Collapse
|
15
|
Bruno F, Arrigoni F, Palumbo P, Natella R, Maggialetti N, Reginelli A, Splendiani A, Di Cesare E, Bazzocchi A, Guglielmi G, Masciocchi C, Barile A. The Acutely Injured Wrist. Radiol Clin North Am 2019; 57:943-955. [PMID: 31351543 DOI: 10.1016/j.rcl.2019.05.003] [Citation(s) in RCA: 14] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/11/2022]
Abstract
Wrist traumas are a frequent clinical emergency for which instrumental imaging assessment is required. The purpose of this article is to review the role of imaging assessment of traumatic wrist injuries, with particular reference to fractures and associated lesions.
Collapse
Affiliation(s)
- Federico Bruno
- Department of Biotechnological and Applied Clinical Sciences, University of L'Aquila, Via Vetoio 1, Coppito, L'Aquila (AQ) 67100, Italy
| | - Francesco Arrigoni
- Department of Biotechnological and Applied Clinical Sciences, University of L'Aquila, Via Vetoio 1, Coppito, L'Aquila (AQ) 67100, Italy
| | - Pierpaolo Palumbo
- Department of Biotechnological and Applied Clinical Sciences, University of L'Aquila, Via Vetoio 1, Coppito, L'Aquila (AQ) 67100, Italy
| | - Raffaele Natella
- Department of Precision Medicine, University of Campania "L.Vanvitelli", Via Santa Maria di Costantinopoli 104, 80138 Naples, Italy
| | - Nicola Maggialetti
- Department Life and Health "V. Tiberio", University of Molise, Via Francesco De Sanctis, 86100 Campobasso, Italy
| | - Alfonso Reginelli
- Department of Precision Medicine, University of Campania "L.Vanvitelli", Via Santa Maria di Costantinopoli 104, 80138 Naples, Italy
| | - Alessandra Splendiani
- Department of Biotechnological and Applied Clinical Sciences, University of L'Aquila, Via Vetoio 1, Coppito, L'Aquila (AQ) 67100, Italy
| | - Ernesto Di Cesare
- Department of Biotechnological and Applied Clinical Sciences, University of L'Aquila, Via Vetoio 1, Coppito, L'Aquila (AQ) 67100, Italy
| | - Alberto Bazzocchi
- Diagnostic and Interventional Radiology, IRCCS Istituto Ortopedico Rizzoli, Via C. Pupilli 1, 40136 Bologna, Italy
| | - Giuseppe Guglielmi
- Department of Radiology, University of Foggia, Viale Luigi Pinto 1, Foggia 71100, Italy
| | - Carlo Masciocchi
- Department of Biotechnological and Applied Clinical Sciences, University of L'Aquila, Via Vetoio 1, Coppito, L'Aquila (AQ) 67100, Italy
| | - Antonio Barile
- Department of Biotechnological and Applied Clinical Sciences, University of L'Aquila, Via Vetoio 1, Coppito, L'Aquila (AQ) 67100, Italy.
| |
Collapse
|