1
|
Wang DY, Liu SG, Ding J, Sun AL, Jiang D, Jiang J, Zhao JZ, Chen DS, Ji G, Li N, Yuan HS, Yu JK. A Deep Learning Model Enhances Clinicians' Diagnostic Accuracy to More Than 96% for Anterior Cruciate Ligament Ruptures on Magnetic Resonance Imaging. Arthroscopy 2024; 40:1197-1205. [PMID: 37597705 DOI: 10.1016/j.arthro.2023.08.010] [Citation(s) in RCA: 5] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 03/08/2023] [Revised: 08/02/2023] [Accepted: 08/03/2023] [Indexed: 08/21/2023]
Abstract
PURPOSE To develop a deep learning model to accurately detect anterior cruciate ligament (ACL) ruptures on magnetic resonance imaging (MRI) and to evaluate its effect on the diagnostic accuracy and efficiency of clinicians. METHODS A training dataset was built from MRIs acquired from January 2017 to June 2021, including patients with knee symptoms, irrespective of ACL ruptures. An external validation dataset was built from MRIs acquired from January 2021 to June 2022, including patients who underwent knee arthroscopy or arthroplasty. Patients with fractures or prior knee surgeries were excluded in both datasets. Subsequently, a deep learning model was developed and validated using these datasets. Clinicians of varying expertise levels in sports medicine and radiology were recruited, and their capacities in diagnosing ACL injuries in terms of accuracy and diagnosing time were evaluated both with and without artificial intelligence (AI) assistance. RESULTS A deep learning model was developed based on the training dataset of 22,767 MRIs from 5 centers and verified with external validation dataset of 4,086 MRIs from 6 centers. The model achieved an area under the receiver operating characteristic curve of 0.987 and a sensitivity and specificity of 95.1%. Thirty-eight clinicians from 25 centers were recruited to diagnose 3,800 MRIs. The AI assistance significantly improved the accuracy of all clinicians, exceeding 96%. Additionally, a notable reduction in diagnostic time was observed. The most significant improvements in accuracy and time efficiency were observed in the trainee groups, suggesting that AI support is particularly beneficial for clinicians with moderately limited diagnostic expertise. CONCLUSIONS This deep learning model demonstrated expert-level diagnostic performance for ACL ruptures, serving as a valuable tool to assist clinicians of various specialties and experience levels in making accurate and efficient diagnoses. LEVEL OF EVIDENCE Level III, retrospective comparative case series.
Collapse
Affiliation(s)
- Ding-Yu Wang
- Department of Sports Medicine, Peking University Third Hospital, Institute of Sports Medicine of Peking University, Beijing, China; Beijing Key Laboratory of Sports Injuries, Beijing, China; Engineering Research Center of Sports Trauma Treatment Technology and Devices, Ministry of Education, Beijing, China
| | - Shang-Gui Liu
- Department of Sports Medicine, Peking University Third Hospital, Institute of Sports Medicine of Peking University, Beijing, China; Beijing Key Laboratory of Sports Injuries, Beijing, China; Engineering Research Center of Sports Trauma Treatment Technology and Devices, Ministry of Education, Beijing, China
| | - Jia Ding
- Beijing Yizhun Medical AI Co., Ltd, Beijing, China
| | - An-Lan Sun
- Beijing Yizhun Medical AI Co., Ltd, Beijing, China
| | - Dong Jiang
- Department of Sports Medicine, Peking University Third Hospital, Institute of Sports Medicine of Peking University, Beijing, China; Beijing Key Laboratory of Sports Injuries, Beijing, China; Engineering Research Center of Sports Trauma Treatment Technology and Devices, Ministry of Education, Beijing, China
| | - Jia Jiang
- Department of Sports Medicine, Shanghai Jiaotong University Affiliated Sixth People's Hospital, Shanghai, China
| | - Jin-Zhong Zhao
- Department of Sports Medicine, Shanghai Jiaotong University Affiliated Sixth People's Hospital, Shanghai, China
| | - De-Sheng Chen
- Department of Sports Medicine and Arthroscopy, Tianjin Hospital of Tianjin University, Tianjin, China
| | - Gang Ji
- Department of Orthopaedic Surgery, Third Hospital of Hebei Medical University, Hebei, China
| | - Nan Li
- Research Center of Clinical Epidemiology, Peking University Third Hospital, Beijing, China
| | - Hui-Shu Yuan
- Department of Radiology, Peking University Third Hospital, Beijing, China
| | - Jia-Kuo Yu
- Department of Sports Medicine, Peking University Third Hospital, Institute of Sports Medicine of Peking University, Beijing, China; Beijing Key Laboratory of Sports Injuries, Beijing, China; Engineering Research Center of Sports Trauma Treatment Technology and Devices, Ministry of Education, Beijing, China.
| |
Collapse
|
2
|
Hu S, Wang X, Wang Q, Feng W. Lever sign test for anterior cruciate ligament injuries: a diagnostic meta-analysis. J Orthop Surg Res 2024; 19:155. [PMID: 38429720 PMCID: PMC10905791 DOI: 10.1186/s13018-024-04635-w] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 12/03/2023] [Accepted: 02/21/2024] [Indexed: 03/03/2024] Open
Abstract
BACKGROUND Sports-related ACL (anterior cruciate ligament) injuries are frequent. Successful management requires early diagnosis and treatment. One of the clinical tests used to identify ACL damage is the lever sign test. This meta-analysis aimed to assess the lever sign test's diagnostic efficacy for ACL injuries. METHODS An extensive investigation of the Cochrane Library, Embase, and PubMed databases was conducted until April 2023. Studies assessing the lever sign test's diagnostic efficacy for ACL injuries were also included. A bivariate random-effects model was employed to acquire the pooled estimates of diagnostic odds ratios, specificity, positive and negative likelihood ratios, sensitivity, and curves of the summary receiver operating characteristic (SROC). RESULTS The meta-analysis comprised twelve investigations with a total of 1365 individuals. The lever sign test's combined sensitivity and specificity for the purpose of diagnosing injuries to the ACL were 0.810 (95% confidence interval [CI] 0.686-0.893) and 0.784 (95% CI 0.583-0.904), respectively. The positive and negative likelihood ratios were 3.148 (95% CI 1.784-5.553) and 0.210 (95% CI 0.084-0.528), respectively. The study revealed a diagnostic odds ratio of 17.656, with a 95% CI ranging from 4.800 to 64.951. The SROC curve's area was determined to be 0.912 (95% CI 0.857-0.967). CONCLUSION With high specificity and sensitivity, the lever sign test is a reliable diagnostic modality for ACL injuries. However, the test should be used in combination with other diagnostic tests to increase the accuracy of the diagnosis. Further investigations are warranted to assess the clinical practicability of the lever sign test in various populations and settings.
Collapse
Affiliation(s)
- Shiqiang Hu
- Orthopaedics Department, Xiaolan People's Hospital of Zhongshan, Zhongshan, People's Republic of China
| | - Xiaoping Wang
- Orthopaedics Department, Xiaolan People's Hospital of Zhongshan, Zhongshan, People's Republic of China
| | - Qiyue Wang
- Sports Medicine Center, Department of Orthopaedics Surgery, First Affiliated Hospital of Shantou University Medical College, Shantou, People's Republic of China
| | - Weili Feng
- Orthopaedics Department, Xiaolan People's Hospital of Zhongshan, Zhongshan, People's Republic of China.
| |
Collapse
|
3
|
Cavelti D, Grehn H, Luomajoki H. [Reliability and validity of the Lever Sign Test in suspected ligament and/or meniscus injuries of the knee: a comparison between an orthopaedist and a physical therapist]. SPORTVERLETZUNG SPORTSCHADEN : ORGAN DER GESELLSCHAFT FUR ORTHOPADISCH-TRAUMATOLOGISCHE SPORTMEDIZIN 2023; 37:187-195. [PMID: 38048809 DOI: 10.1055/a-2153-1550] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/06/2023]
Abstract
INTRODUCTION The purpose of this study was to investigate the intertester reliability and validity of the Lever Sign Test. Intertester reliability has received little attention in previous studies. In order to make comparisons with other studies, the intertester reliability and validity of the Lachman and Anterior Drawer Tests were also calculated. PATIENTS/MATERIAL AND METHODS Patients between 18 and 50 years of age with suspected ligament and/or meniscus injuries were included. Exclusion criteria were suspected malignant, systemic, or central neurologic disease, acute cartilage injury, suspected fracture, or knowledge of the participant's knee findings. Testing was randomized and blinded by an experienced orthopaedic surgeon and a physical therapist. Cohen's kappa was calculated for intertester reliability. Sensitivity and specificity, positive and negative predictive value, and positive and negative likelihood ratio were calculated. MRI findings were used as the gold standard. RESULTS Thirty-six patients were enrolled in the study, 27 male (75%) and 9 female (25%). The prevalence of anterior cruciate ligament lesions was 44.4%. Cohen's kappa (κ) was κ=0.6 (CI 95% [0.29, 0.91]) for the Lever Sign Test, κ=0.64 (CI 95% [0.35, 0.93]) for the Lachman test, and κ=0.63 (CI 95% [0.3, 0.95]) for the Anterior Drawer Test. The sensitivity of the Lever Sign Test was 0.53 and 0.40 for the orthopaedic surgeon and physical therapist, respectively, specificity was 0.89 and 0.79, the positive predictive value was 0.80 and 0.60, the negative predictive value was 0.70 and 0.62, the positive likelihood ratio was 4.80 and 1.90, respectively, and the negative likelihood ratio was 0.76 and 0.53. CONCLUSION The estimated intertester reliability of the Lever Sign Test was good. The Lever Sign Test is suitable as a complement to the Lachman Test, which is considered the most valid test in the literature. The weaknesses and ambiguities of the operating mechanism of the Lever Sign Test should not be ignored and should be further explored.
Collapse
Affiliation(s)
- David Cavelti
- Physiotherapy, Kantonsspital Graubunden, Chur, SWITZERLAND
| | | | - Hannu Luomajoki
- Gesundheit, Institut für Physiotherapie, Zürcher Hochschule fur Angewandte Wissenschaften, Winterthur, SWITZERLAND
| |
Collapse
|
4
|
Allott NEH, Banger MS, McGregor AH. Evaluating the diagnostic pathway for acute ACL injuries in trauma centres: a systematic review. BMC Musculoskelet Disord 2022; 23:649. [PMID: 35799147 PMCID: PMC9261037 DOI: 10.1186/s12891-022-05595-0] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 11/18/2021] [Accepted: 06/22/2022] [Indexed: 11/12/2022] Open
Abstract
Objective This review sought to evaluate the literature on the initial assessment and diagnostic pathway for patients with a suspected Anterior Cruciate Ligament (ACL) tear. Methods MEDLINE, EMBASE, and CINAHL were systematically searched for eligible studies, PRISMA guidelines were followed. Studies were included if they used at least one assessment method to assess for ACL injury and participants were assessed at an acute trauma centre within 6-weeks of injury. Article quality was evaluated using the QUADAS-2 checklist. Results A total of 353 studies were assessed for eligibility, 347 were excluded for the following reasons: injuries were not assessed in an acute trauma setting, injuries were not acute, participants had previous ACL injuries or chronic joint deformities affecting the knee, participants were under 18, or participants included animals or cadavers. A total of six studies were included in the review. Common assessment methods included: laxity tests, joint effusion, inability to continue activity, and a history of a ‘pop’ and ‘giving way’ at the time of injury. Diagnostic accuracy varied greatly between the assessment method and the assessing clinician. Gold standard diagnostics were MRI and arthroscopy. A weighted meta-mean calculated the time to reach diagnosis to be 68.60 days [CI 23.94, 113.24]. The mean number of appointments to reach diagnosis varied from 2–5. Delay to surgery or surgical consultation ranged from 61 to 328 days. Conclusion Clinicians in the Emergency Department are not proficient in performing the assessment methods that are used for diagnosis in acute ACL injury. Reliance on specialist assessments or radiological methods inevitably increases the time to reach a diagnosis, which has repercussions on management options. There is an ever-growing demand to improve diagnostic accuracy and efficiency; further exploration into quantitative measures of instability would aid the assessment of peripheral joint assessment.
Collapse
Affiliation(s)
- Natasha E H Allott
- Sir Michael Uren Hub, Imperial College London, White City Campus, 86 Wood Lane, London, W12 0BZ, UK.
| | - Matthew S Banger
- Sir Michael Uren Hub, Imperial College London, White City Campus, 86 Wood Lane, London, W12 0BZ, UK
| | - Alison H McGregor
- Sir Michael Uren Hub, Imperial College London, White City Campus, 86 Wood Lane, London, W12 0BZ, UK
| |
Collapse
|
5
|
İpek D, Zehir S, Dündar A. Is Lever Test Superior to Lachman, Pivot Shift, Drawer Tests in Diagnosing Anterior Cruciate Ligament Injuries? Cureus 2022; 14:e22049. [PMID: 35340524 PMCID: PMC8915683 DOI: 10.7759/cureus.22049] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 02/09/2022] [Indexed: 12/03/2022] Open
Abstract
Introduction: The physical examination in anterior cruciate ligament (ACL) injuries is extremely important, and the Lever test is commonly utilized on ACL evaluation. However, the number and scope of studies on the Lever test is limited. In this prospective cross-sectional study, we aimed to evaluate the effectiveness of the diagnostic values of Lachman, Pivot Shift, Lever, and Anterior Drawer tests in terms of quadriceps atrophy and case phase in ACL injuries. Methods: In this prospective study, diagnostic values of Lachman, Pivot Shift, Lever, and Anterior Drawer tests were examined on 189 patients with positive MRI results as the gold standard. Results: Lever test positivity was significantly more frequent in the group with quadriceps atrophy preoperative and after sedation (p<0.05). Anterior Drawer test positivity was significantly more frequent in the group with positive quadriceps atrophy preoperatively, after sedation and after spinal anesthesia (p<0.05). Lever and Anterior Drawer tests were positively correlated with quadriceps atrophy preop and after sedation (p<0.05). Lever test before surgery, after sedation and after spinal anesthesia in the chronic patient group was more positive than in the acute and subacute groups (p<0.05). Lever test was positively correlated with phase preoperatively, after sedation and after spinal anesthesia (p<0.01). Conclusion: The presence or absence of quadriceps atrophy in patients with acute, sub-acute, or chronic ACL injury has a significant effect on the predictive value of the Lever test. We think that univariate analyzes may give incorrect results when demonstrating predictive value, and it would be more correct to perform multivariate analyzes.
Collapse
|
6
|
Sokal PA, Norris R, Maddox TW, Oldershaw RA. The diagnostic accuracy of clinical tests for anterior cruciate ligament tears are comparable but the Lachman test has been previously overestimated: a systematic review and meta-analysis. Knee Surg Sports Traumatol Arthrosc 2022; 30:3287-3303. [PMID: 35150292 PMCID: PMC9464183 DOI: 10.1007/s00167-022-06898-4] [Citation(s) in RCA: 13] [Impact Index Per Article: 6.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 09/20/2021] [Accepted: 01/20/2022] [Indexed: 11/29/2022]
Abstract
PURPOSE The diagnostic accuracy of clinical tests for anterior cruciate ligament injury has been reported in previous systematic reviews. Numerous studies in these reviews include subjects with additional knee ligament injury, which could affect the sensitivity of the tests. Meta-analyses have also been performed using methods that do not account for the non-independence of sensitivity and specificity, potentially overestimating diagnostic accuracy. The aim of this study was to report the diagnostic accuracy of clinical tests for anterior cruciate ligament tears (partial and complete) without concomitant knee ligament injury. METHODS A systematic review with meta-analysis was performed according to the PRISMA guidelines. Meta-analyses included studies reporting the specificity and/or sensitivity of tests with or without concomitant meniscal injury. Where possible, pooled diagnostic estimates were calculated with bivariate random-effects modelling to determine the most accurate effect sizes. Diagnostic accuracy values are presented for the anterior drawer, Lachman, Lever sign and pivot shift tests overall and in acute or post-acute presentations. RESULTS Pooled estimates using a bivariate model for overall sensitivity and specificity respectively were as follows: anterior drawer test 83% [95% CI, 77-88] and 85% [95% CI, 64-95]; Lachman test 81% [95% CI, 73-87] and 85% [95% CI, 73-92]; pivot shift test 55% [95% CI, 47-62] and 94% [95% CI, 88-97]; Lever sign test 83% [95% CI, 68-92] and 91% [95% CI, 83-95]. For specific presentations, the sensitivity and specificity of the Lachman test, respectively, were: complete tears 68% [95% CI, 54-79] and 79% [95% CI, 51-93]; post-acute injuries 70% [95% CI, 57-80] and 77% [95% CI, 53-91]. CONCLUSIONS The pivot shift and Lever sign were the best tests overall for ruling in or ruling out an anterior cruciate ligament tear, respectively. The diagnostic accuracy of the Lachman test, particularly in post-acute presentations and for complete tears, is lower than previously reported. Further research is required to establish more accurate estimates for the Lachman test in acute presentations and partial ligament tears using bivariate analysis. LEVEL OF EVIDENCE III.
Collapse
Affiliation(s)
- Pawel A. Sokal
- Department of Musculoskeletal and Ageing Science, Institute of Life Course and Medical Sciences, Faculty of Health and Life Sciences, University of Liverpool, William Henry Duncan Building, 6 West Derby Street, Liverpool, L7 8TX UK ,Department of Medicine, School of Medicine, University of Liverpool, Liverpool, L69 3GE UK
| | - Richard Norris
- Department of Musculoskeletal and Ageing Science, Institute of Life Course and Medical Sciences, Faculty of Health and Life Sciences, University of Liverpool, William Henry Duncan Building, 6 West Derby Street, Liverpool, L7 8TX UK ,Department of Trauma and Orthopaedics, Aintree University Hospital, Liverpool University Hospitals NHS Foundation Trust, Lower Lane, Fazakerley, Liverpool, L9 7AL UK
| | - Thomas W. Maddox
- Department of Musculoskeletal and Ageing Science, Institute of Life Course and Medical Sciences, Faculty of Health and Life Sciences, University of Liverpool, William Henry Duncan Building, 6 West Derby Street, Liverpool, L7 8TX UK ,Small Animal Teaching Hospital, Institute of Infection, Veterinary and Ecological Sciences, Faculty of Health and Life Sciences, University of Liverpool, Leahurst Campus, Chester High Road, Neston, CH64 7TE Wirral UK
| | - Rachel A. Oldershaw
- Department of Musculoskeletal and Ageing Science, Institute of Life Course and Medical Sciences, Faculty of Health and Life Sciences, University of Liverpool, William Henry Duncan Building, 6 West Derby Street, Liverpool, L7 8TX UK
| |
Collapse
|