7
|
Lim DSW, Makmur A, Zhu L, Zhang W, Cheng AJL, Sia DSY, Eide SE, Ong HY, Jagmohan P, Tan WC, Khoo VM, Wong YM, Thian YL, Baskar S, Teo EC, Algazwi DAR, Yap QV, Chan YH, Tan JH, Kumar N, Ooi BC, Yoshioka H, Quek ST, Hallinan JTPD. Improved Productivity Using Deep Learning-assisted Reporting for Lumbar Spine MRI. Radiology 2022; 305:160-166. [PMID: 35699577 DOI: 10.1148/radiol.220076] [Citation(s) in RCA: 10] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/11/2022]
Abstract
Background Lumbar spine MRI studies are widely used for back pain assessment. Interpretation involves grading lumbar spinal stenosis, which is repetitive and time consuming. Deep learning (DL) could provide faster and more consistent interpretation. Purpose To assess the speed and interobserver agreement of radiologists for reporting lumbar spinal stenosis with and without DL assistance. Materials and Methods In this retrospective study, a DL model designed to assist radiologists in the interpretation of spinal canal, lateral recess, and neural foraminal stenoses on lumbar spine MRI scans was used. Randomly selected lumbar spine MRI studies obtained in patients with back pain who were 18 years and older over a 3-year period, from September 2015 to September 2018, were included in an internal test data set. Studies with instrumentation and scoliosis were excluded. Eight radiologists, each with 2-13 years of experience in spine MRI interpretation, reviewed studies with and without DL model assistance with a 1-month washout period. Time to diagnosis (in seconds) and interobserver agreement (using Gwet κ) were assessed for stenosis grading for each radiologist with and without the DL model and compared with test data set labels provided by an external musculoskeletal radiologist (with 32 years of experience) as the reference standard. Results Overall, 444 images in 25 patients (mean age, 51 years ± 20 [SD]; 14 women) were evaluated in a test data set. DL-assisted radiologists had a reduced interpretation time per spine MRI study, from a mean of 124-274 seconds (SD, 25-88 seconds) to 47-71 seconds (SD, 24-29 seconds) (P < .001). DL-assisted radiologists had either superior or equivalent interobserver agreement for all stenosis gradings compared with unassisted radiologists. DL-assisted general and in-training radiologists improved their interobserver agreement for four-class neural foraminal stenosis, with κ values of 0.71 and 0.70 (with DL) versus 0.39 and 0.39 (without DL), respectively (both P < .001). Conclusion Radiologists who were assisted by deep learning for interpretation of lumbar spinal stenosis on MRI scans showed a marked reduction in reporting time and superior or equivalent interobserver agreement for all stenosis gradings compared with radiologists who were unassisted by deep learning. © RSNA, 2022 Online supplemental material is available for this article. See also the editorial by Hayashi in this issue.
Collapse
Affiliation(s)
- Desmond Shi Wei Lim
- From the Department of Diagnostic Imaging, National University Hospital, 5 Lower Kent Ridge Rd, Singapore 119074 (D.S.W.L., A.M., A.J.L.C., D.S.Y.S., S.E.E., H.Y.O., P.J., W.C.T., V.M.K., Y.M.W., Y.L.T., S.B., E.C.T., S.T.Q., J.T.P.D.H.); Department of Diagnostic Radiology (A.M., S.E.E., P.J., Y.L.T., S.T.Q., J.T.P.D.H.), NUS Graduate School, Integrative Sciences and Engineering Programme (L.Z.), Department of Computer Science, School of Computing (W.Z., B.C.O.), and Biostatistics Unit, Yong Loo Lin School of Medicine (Q.V.Y., Y.H.C.), National University of Singapore, Singapore; Department of Radiology, Qatif Central Hospital, Qatif, Saudi Arabia (D.A.R.A.); Department of Orthopaedic Surgery, National University Health System, Singapore (J.H.T., N.K.); and Department of Radiological Sciences, University of California, Irvine, Orange, Calif (H.Y.)
| | - Andrew Makmur
- From the Department of Diagnostic Imaging, National University Hospital, 5 Lower Kent Ridge Rd, Singapore 119074 (D.S.W.L., A.M., A.J.L.C., D.S.Y.S., S.E.E., H.Y.O., P.J., W.C.T., V.M.K., Y.M.W., Y.L.T., S.B., E.C.T., S.T.Q., J.T.P.D.H.); Department of Diagnostic Radiology (A.M., S.E.E., P.J., Y.L.T., S.T.Q., J.T.P.D.H.), NUS Graduate School, Integrative Sciences and Engineering Programme (L.Z.), Department of Computer Science, School of Computing (W.Z., B.C.O.), and Biostatistics Unit, Yong Loo Lin School of Medicine (Q.V.Y., Y.H.C.), National University of Singapore, Singapore; Department of Radiology, Qatif Central Hospital, Qatif, Saudi Arabia (D.A.R.A.); Department of Orthopaedic Surgery, National University Health System, Singapore (J.H.T., N.K.); and Department of Radiological Sciences, University of California, Irvine, Orange, Calif (H.Y.)
| | - Lei Zhu
- From the Department of Diagnostic Imaging, National University Hospital, 5 Lower Kent Ridge Rd, Singapore 119074 (D.S.W.L., A.M., A.J.L.C., D.S.Y.S., S.E.E., H.Y.O., P.J., W.C.T., V.M.K., Y.M.W., Y.L.T., S.B., E.C.T., S.T.Q., J.T.P.D.H.); Department of Diagnostic Radiology (A.M., S.E.E., P.J., Y.L.T., S.T.Q., J.T.P.D.H.), NUS Graduate School, Integrative Sciences and Engineering Programme (L.Z.), Department of Computer Science, School of Computing (W.Z., B.C.O.), and Biostatistics Unit, Yong Loo Lin School of Medicine (Q.V.Y., Y.H.C.), National University of Singapore, Singapore; Department of Radiology, Qatif Central Hospital, Qatif, Saudi Arabia (D.A.R.A.); Department of Orthopaedic Surgery, National University Health System, Singapore (J.H.T., N.K.); and Department of Radiological Sciences, University of California, Irvine, Orange, Calif (H.Y.)
| | - Wenqiao Zhang
- From the Department of Diagnostic Imaging, National University Hospital, 5 Lower Kent Ridge Rd, Singapore 119074 (D.S.W.L., A.M., A.J.L.C., D.S.Y.S., S.E.E., H.Y.O., P.J., W.C.T., V.M.K., Y.M.W., Y.L.T., S.B., E.C.T., S.T.Q., J.T.P.D.H.); Department of Diagnostic Radiology (A.M., S.E.E., P.J., Y.L.T., S.T.Q., J.T.P.D.H.), NUS Graduate School, Integrative Sciences and Engineering Programme (L.Z.), Department of Computer Science, School of Computing (W.Z., B.C.O.), and Biostatistics Unit, Yong Loo Lin School of Medicine (Q.V.Y., Y.H.C.), National University of Singapore, Singapore; Department of Radiology, Qatif Central Hospital, Qatif, Saudi Arabia (D.A.R.A.); Department of Orthopaedic Surgery, National University Health System, Singapore (J.H.T., N.K.); and Department of Radiological Sciences, University of California, Irvine, Orange, Calif (H.Y.)
| | - Amanda J L Cheng
- From the Department of Diagnostic Imaging, National University Hospital, 5 Lower Kent Ridge Rd, Singapore 119074 (D.S.W.L., A.M., A.J.L.C., D.S.Y.S., S.E.E., H.Y.O., P.J., W.C.T., V.M.K., Y.M.W., Y.L.T., S.B., E.C.T., S.T.Q., J.T.P.D.H.); Department of Diagnostic Radiology (A.M., S.E.E., P.J., Y.L.T., S.T.Q., J.T.P.D.H.), NUS Graduate School, Integrative Sciences and Engineering Programme (L.Z.), Department of Computer Science, School of Computing (W.Z., B.C.O.), and Biostatistics Unit, Yong Loo Lin School of Medicine (Q.V.Y., Y.H.C.), National University of Singapore, Singapore; Department of Radiology, Qatif Central Hospital, Qatif, Saudi Arabia (D.A.R.A.); Department of Orthopaedic Surgery, National University Health System, Singapore (J.H.T., N.K.); and Department of Radiological Sciences, University of California, Irvine, Orange, Calif (H.Y.)
| | - David Soon Yiew Sia
- From the Department of Diagnostic Imaging, National University Hospital, 5 Lower Kent Ridge Rd, Singapore 119074 (D.S.W.L., A.M., A.J.L.C., D.S.Y.S., S.E.E., H.Y.O., P.J., W.C.T., V.M.K., Y.M.W., Y.L.T., S.B., E.C.T., S.T.Q., J.T.P.D.H.); Department of Diagnostic Radiology (A.M., S.E.E., P.J., Y.L.T., S.T.Q., J.T.P.D.H.), NUS Graduate School, Integrative Sciences and Engineering Programme (L.Z.), Department of Computer Science, School of Computing (W.Z., B.C.O.), and Biostatistics Unit, Yong Loo Lin School of Medicine (Q.V.Y., Y.H.C.), National University of Singapore, Singapore; Department of Radiology, Qatif Central Hospital, Qatif, Saudi Arabia (D.A.R.A.); Department of Orthopaedic Surgery, National University Health System, Singapore (J.H.T., N.K.); and Department of Radiological Sciences, University of California, Irvine, Orange, Calif (H.Y.)
| | - Sterling Ellis Eide
- From the Department of Diagnostic Imaging, National University Hospital, 5 Lower Kent Ridge Rd, Singapore 119074 (D.S.W.L., A.M., A.J.L.C., D.S.Y.S., S.E.E., H.Y.O., P.J., W.C.T., V.M.K., Y.M.W., Y.L.T., S.B., E.C.T., S.T.Q., J.T.P.D.H.); Department of Diagnostic Radiology (A.M., S.E.E., P.J., Y.L.T., S.T.Q., J.T.P.D.H.), NUS Graduate School, Integrative Sciences and Engineering Programme (L.Z.), Department of Computer Science, School of Computing (W.Z., B.C.O.), and Biostatistics Unit, Yong Loo Lin School of Medicine (Q.V.Y., Y.H.C.), National University of Singapore, Singapore; Department of Radiology, Qatif Central Hospital, Qatif, Saudi Arabia (D.A.R.A.); Department of Orthopaedic Surgery, National University Health System, Singapore (J.H.T., N.K.); and Department of Radiological Sciences, University of California, Irvine, Orange, Calif (H.Y.)
| | - Han Yang Ong
- From the Department of Diagnostic Imaging, National University Hospital, 5 Lower Kent Ridge Rd, Singapore 119074 (D.S.W.L., A.M., A.J.L.C., D.S.Y.S., S.E.E., H.Y.O., P.J., W.C.T., V.M.K., Y.M.W., Y.L.T., S.B., E.C.T., S.T.Q., J.T.P.D.H.); Department of Diagnostic Radiology (A.M., S.E.E., P.J., Y.L.T., S.T.Q., J.T.P.D.H.), NUS Graduate School, Integrative Sciences and Engineering Programme (L.Z.), Department of Computer Science, School of Computing (W.Z., B.C.O.), and Biostatistics Unit, Yong Loo Lin School of Medicine (Q.V.Y., Y.H.C.), National University of Singapore, Singapore; Department of Radiology, Qatif Central Hospital, Qatif, Saudi Arabia (D.A.R.A.); Department of Orthopaedic Surgery, National University Health System, Singapore (J.H.T., N.K.); and Department of Radiological Sciences, University of California, Irvine, Orange, Calif (H.Y.)
| | - Pooja Jagmohan
- From the Department of Diagnostic Imaging, National University Hospital, 5 Lower Kent Ridge Rd, Singapore 119074 (D.S.W.L., A.M., A.J.L.C., D.S.Y.S., S.E.E., H.Y.O., P.J., W.C.T., V.M.K., Y.M.W., Y.L.T., S.B., E.C.T., S.T.Q., J.T.P.D.H.); Department of Diagnostic Radiology (A.M., S.E.E., P.J., Y.L.T., S.T.Q., J.T.P.D.H.), NUS Graduate School, Integrative Sciences and Engineering Programme (L.Z.), Department of Computer Science, School of Computing (W.Z., B.C.O.), and Biostatistics Unit, Yong Loo Lin School of Medicine (Q.V.Y., Y.H.C.), National University of Singapore, Singapore; Department of Radiology, Qatif Central Hospital, Qatif, Saudi Arabia (D.A.R.A.); Department of Orthopaedic Surgery, National University Health System, Singapore (J.H.T., N.K.); and Department of Radiological Sciences, University of California, Irvine, Orange, Calif (H.Y.)
| | - Wei Chuan Tan
- From the Department of Diagnostic Imaging, National University Hospital, 5 Lower Kent Ridge Rd, Singapore 119074 (D.S.W.L., A.M., A.J.L.C., D.S.Y.S., S.E.E., H.Y.O., P.J., W.C.T., V.M.K., Y.M.W., Y.L.T., S.B., E.C.T., S.T.Q., J.T.P.D.H.); Department of Diagnostic Radiology (A.M., S.E.E., P.J., Y.L.T., S.T.Q., J.T.P.D.H.), NUS Graduate School, Integrative Sciences and Engineering Programme (L.Z.), Department of Computer Science, School of Computing (W.Z., B.C.O.), and Biostatistics Unit, Yong Loo Lin School of Medicine (Q.V.Y., Y.H.C.), National University of Singapore, Singapore; Department of Radiology, Qatif Central Hospital, Qatif, Saudi Arabia (D.A.R.A.); Department of Orthopaedic Surgery, National University Health System, Singapore (J.H.T., N.K.); and Department of Radiological Sciences, University of California, Irvine, Orange, Calif (H.Y.)
| | - Vanessa Meihui Khoo
- From the Department of Diagnostic Imaging, National University Hospital, 5 Lower Kent Ridge Rd, Singapore 119074 (D.S.W.L., A.M., A.J.L.C., D.S.Y.S., S.E.E., H.Y.O., P.J., W.C.T., V.M.K., Y.M.W., Y.L.T., S.B., E.C.T., S.T.Q., J.T.P.D.H.); Department of Diagnostic Radiology (A.M., S.E.E., P.J., Y.L.T., S.T.Q., J.T.P.D.H.), NUS Graduate School, Integrative Sciences and Engineering Programme (L.Z.), Department of Computer Science, School of Computing (W.Z., B.C.O.), and Biostatistics Unit, Yong Loo Lin School of Medicine (Q.V.Y., Y.H.C.), National University of Singapore, Singapore; Department of Radiology, Qatif Central Hospital, Qatif, Saudi Arabia (D.A.R.A.); Department of Orthopaedic Surgery, National University Health System, Singapore (J.H.T., N.K.); and Department of Radiological Sciences, University of California, Irvine, Orange, Calif (H.Y.)
| | - Ying Mei Wong
- From the Department of Diagnostic Imaging, National University Hospital, 5 Lower Kent Ridge Rd, Singapore 119074 (D.S.W.L., A.M., A.J.L.C., D.S.Y.S., S.E.E., H.Y.O., P.J., W.C.T., V.M.K., Y.M.W., Y.L.T., S.B., E.C.T., S.T.Q., J.T.P.D.H.); Department of Diagnostic Radiology (A.M., S.E.E., P.J., Y.L.T., S.T.Q., J.T.P.D.H.), NUS Graduate School, Integrative Sciences and Engineering Programme (L.Z.), Department of Computer Science, School of Computing (W.Z., B.C.O.), and Biostatistics Unit, Yong Loo Lin School of Medicine (Q.V.Y., Y.H.C.), National University of Singapore, Singapore; Department of Radiology, Qatif Central Hospital, Qatif, Saudi Arabia (D.A.R.A.); Department of Orthopaedic Surgery, National University Health System, Singapore (J.H.T., N.K.); and Department of Radiological Sciences, University of California, Irvine, Orange, Calif (H.Y.)
| | - Yee Liang Thian
- From the Department of Diagnostic Imaging, National University Hospital, 5 Lower Kent Ridge Rd, Singapore 119074 (D.S.W.L., A.M., A.J.L.C., D.S.Y.S., S.E.E., H.Y.O., P.J., W.C.T., V.M.K., Y.M.W., Y.L.T., S.B., E.C.T., S.T.Q., J.T.P.D.H.); Department of Diagnostic Radiology (A.M., S.E.E., P.J., Y.L.T., S.T.Q., J.T.P.D.H.), NUS Graduate School, Integrative Sciences and Engineering Programme (L.Z.), Department of Computer Science, School of Computing (W.Z., B.C.O.), and Biostatistics Unit, Yong Loo Lin School of Medicine (Q.V.Y., Y.H.C.), National University of Singapore, Singapore; Department of Radiology, Qatif Central Hospital, Qatif, Saudi Arabia (D.A.R.A.); Department of Orthopaedic Surgery, National University Health System, Singapore (J.H.T., N.K.); and Department of Radiological Sciences, University of California, Irvine, Orange, Calif (H.Y.)
| | - Sangeetha Baskar
- From the Department of Diagnostic Imaging, National University Hospital, 5 Lower Kent Ridge Rd, Singapore 119074 (D.S.W.L., A.M., A.J.L.C., D.S.Y.S., S.E.E., H.Y.O., P.J., W.C.T., V.M.K., Y.M.W., Y.L.T., S.B., E.C.T., S.T.Q., J.T.P.D.H.); Department of Diagnostic Radiology (A.M., S.E.E., P.J., Y.L.T., S.T.Q., J.T.P.D.H.), NUS Graduate School, Integrative Sciences and Engineering Programme (L.Z.), Department of Computer Science, School of Computing (W.Z., B.C.O.), and Biostatistics Unit, Yong Loo Lin School of Medicine (Q.V.Y., Y.H.C.), National University of Singapore, Singapore; Department of Radiology, Qatif Central Hospital, Qatif, Saudi Arabia (D.A.R.A.); Department of Orthopaedic Surgery, National University Health System, Singapore (J.H.T., N.K.); and Department of Radiological Sciences, University of California, Irvine, Orange, Calif (H.Y.)
| | - Ee Chin Teo
- From the Department of Diagnostic Imaging, National University Hospital, 5 Lower Kent Ridge Rd, Singapore 119074 (D.S.W.L., A.M., A.J.L.C., D.S.Y.S., S.E.E., H.Y.O., P.J., W.C.T., V.M.K., Y.M.W., Y.L.T., S.B., E.C.T., S.T.Q., J.T.P.D.H.); Department of Diagnostic Radiology (A.M., S.E.E., P.J., Y.L.T., S.T.Q., J.T.P.D.H.), NUS Graduate School, Integrative Sciences and Engineering Programme (L.Z.), Department of Computer Science, School of Computing (W.Z., B.C.O.), and Biostatistics Unit, Yong Loo Lin School of Medicine (Q.V.Y., Y.H.C.), National University of Singapore, Singapore; Department of Radiology, Qatif Central Hospital, Qatif, Saudi Arabia (D.A.R.A.); Department of Orthopaedic Surgery, National University Health System, Singapore (J.H.T., N.K.); and Department of Radiological Sciences, University of California, Irvine, Orange, Calif (H.Y.)
| | - Diyaa Abdul Rauf Algazwi
- From the Department of Diagnostic Imaging, National University Hospital, 5 Lower Kent Ridge Rd, Singapore 119074 (D.S.W.L., A.M., A.J.L.C., D.S.Y.S., S.E.E., H.Y.O., P.J., W.C.T., V.M.K., Y.M.W., Y.L.T., S.B., E.C.T., S.T.Q., J.T.P.D.H.); Department of Diagnostic Radiology (A.M., S.E.E., P.J., Y.L.T., S.T.Q., J.T.P.D.H.), NUS Graduate School, Integrative Sciences and Engineering Programme (L.Z.), Department of Computer Science, School of Computing (W.Z., B.C.O.), and Biostatistics Unit, Yong Loo Lin School of Medicine (Q.V.Y., Y.H.C.), National University of Singapore, Singapore; Department of Radiology, Qatif Central Hospital, Qatif, Saudi Arabia (D.A.R.A.); Department of Orthopaedic Surgery, National University Health System, Singapore (J.H.T., N.K.); and Department of Radiological Sciences, University of California, Irvine, Orange, Calif (H.Y.)
| | - Qai Ven Yap
- From the Department of Diagnostic Imaging, National University Hospital, 5 Lower Kent Ridge Rd, Singapore 119074 (D.S.W.L., A.M., A.J.L.C., D.S.Y.S., S.E.E., H.Y.O., P.J., W.C.T., V.M.K., Y.M.W., Y.L.T., S.B., E.C.T., S.T.Q., J.T.P.D.H.); Department of Diagnostic Radiology (A.M., S.E.E., P.J., Y.L.T., S.T.Q., J.T.P.D.H.), NUS Graduate School, Integrative Sciences and Engineering Programme (L.Z.), Department of Computer Science, School of Computing (W.Z., B.C.O.), and Biostatistics Unit, Yong Loo Lin School of Medicine (Q.V.Y., Y.H.C.), National University of Singapore, Singapore; Department of Radiology, Qatif Central Hospital, Qatif, Saudi Arabia (D.A.R.A.); Department of Orthopaedic Surgery, National University Health System, Singapore (J.H.T., N.K.); and Department of Radiological Sciences, University of California, Irvine, Orange, Calif (H.Y.)
| | - Yiong Huak Chan
- From the Department of Diagnostic Imaging, National University Hospital, 5 Lower Kent Ridge Rd, Singapore 119074 (D.S.W.L., A.M., A.J.L.C., D.S.Y.S., S.E.E., H.Y.O., P.J., W.C.T., V.M.K., Y.M.W., Y.L.T., S.B., E.C.T., S.T.Q., J.T.P.D.H.); Department of Diagnostic Radiology (A.M., S.E.E., P.J., Y.L.T., S.T.Q., J.T.P.D.H.), NUS Graduate School, Integrative Sciences and Engineering Programme (L.Z.), Department of Computer Science, School of Computing (W.Z., B.C.O.), and Biostatistics Unit, Yong Loo Lin School of Medicine (Q.V.Y., Y.H.C.), National University of Singapore, Singapore; Department of Radiology, Qatif Central Hospital, Qatif, Saudi Arabia (D.A.R.A.); Department of Orthopaedic Surgery, National University Health System, Singapore (J.H.T., N.K.); and Department of Radiological Sciences, University of California, Irvine, Orange, Calif (H.Y.)
| | - Jiong Hao Tan
- From the Department of Diagnostic Imaging, National University Hospital, 5 Lower Kent Ridge Rd, Singapore 119074 (D.S.W.L., A.M., A.J.L.C., D.S.Y.S., S.E.E., H.Y.O., P.J., W.C.T., V.M.K., Y.M.W., Y.L.T., S.B., E.C.T., S.T.Q., J.T.P.D.H.); Department of Diagnostic Radiology (A.M., S.E.E., P.J., Y.L.T., S.T.Q., J.T.P.D.H.), NUS Graduate School, Integrative Sciences and Engineering Programme (L.Z.), Department of Computer Science, School of Computing (W.Z., B.C.O.), and Biostatistics Unit, Yong Loo Lin School of Medicine (Q.V.Y., Y.H.C.), National University of Singapore, Singapore; Department of Radiology, Qatif Central Hospital, Qatif, Saudi Arabia (D.A.R.A.); Department of Orthopaedic Surgery, National University Health System, Singapore (J.H.T., N.K.); and Department of Radiological Sciences, University of California, Irvine, Orange, Calif (H.Y.)
| | - Naresh Kumar
- From the Department of Diagnostic Imaging, National University Hospital, 5 Lower Kent Ridge Rd, Singapore 119074 (D.S.W.L., A.M., A.J.L.C., D.S.Y.S., S.E.E., H.Y.O., P.J., W.C.T., V.M.K., Y.M.W., Y.L.T., S.B., E.C.T., S.T.Q., J.T.P.D.H.); Department of Diagnostic Radiology (A.M., S.E.E., P.J., Y.L.T., S.T.Q., J.T.P.D.H.), NUS Graduate School, Integrative Sciences and Engineering Programme (L.Z.), Department of Computer Science, School of Computing (W.Z., B.C.O.), and Biostatistics Unit, Yong Loo Lin School of Medicine (Q.V.Y., Y.H.C.), National University of Singapore, Singapore; Department of Radiology, Qatif Central Hospital, Qatif, Saudi Arabia (D.A.R.A.); Department of Orthopaedic Surgery, National University Health System, Singapore (J.H.T., N.K.); and Department of Radiological Sciences, University of California, Irvine, Orange, Calif (H.Y.)
| | - Beng Chin Ooi
- From the Department of Diagnostic Imaging, National University Hospital, 5 Lower Kent Ridge Rd, Singapore 119074 (D.S.W.L., A.M., A.J.L.C., D.S.Y.S., S.E.E., H.Y.O., P.J., W.C.T., V.M.K., Y.M.W., Y.L.T., S.B., E.C.T., S.T.Q., J.T.P.D.H.); Department of Diagnostic Radiology (A.M., S.E.E., P.J., Y.L.T., S.T.Q., J.T.P.D.H.), NUS Graduate School, Integrative Sciences and Engineering Programme (L.Z.), Department of Computer Science, School of Computing (W.Z., B.C.O.), and Biostatistics Unit, Yong Loo Lin School of Medicine (Q.V.Y., Y.H.C.), National University of Singapore, Singapore; Department of Radiology, Qatif Central Hospital, Qatif, Saudi Arabia (D.A.R.A.); Department of Orthopaedic Surgery, National University Health System, Singapore (J.H.T., N.K.); and Department of Radiological Sciences, University of California, Irvine, Orange, Calif (H.Y.)
| | - Hiroshi Yoshioka
- From the Department of Diagnostic Imaging, National University Hospital, 5 Lower Kent Ridge Rd, Singapore 119074 (D.S.W.L., A.M., A.J.L.C., D.S.Y.S., S.E.E., H.Y.O., P.J., W.C.T., V.M.K., Y.M.W., Y.L.T., S.B., E.C.T., S.T.Q., J.T.P.D.H.); Department of Diagnostic Radiology (A.M., S.E.E., P.J., Y.L.T., S.T.Q., J.T.P.D.H.), NUS Graduate School, Integrative Sciences and Engineering Programme (L.Z.), Department of Computer Science, School of Computing (W.Z., B.C.O.), and Biostatistics Unit, Yong Loo Lin School of Medicine (Q.V.Y., Y.H.C.), National University of Singapore, Singapore; Department of Radiology, Qatif Central Hospital, Qatif, Saudi Arabia (D.A.R.A.); Department of Orthopaedic Surgery, National University Health System, Singapore (J.H.T., N.K.); and Department of Radiological Sciences, University of California, Irvine, Orange, Calif (H.Y.)
| | - Swee Tian Quek
- From the Department of Diagnostic Imaging, National University Hospital, 5 Lower Kent Ridge Rd, Singapore 119074 (D.S.W.L., A.M., A.J.L.C., D.S.Y.S., S.E.E., H.Y.O., P.J., W.C.T., V.M.K., Y.M.W., Y.L.T., S.B., E.C.T., S.T.Q., J.T.P.D.H.); Department of Diagnostic Radiology (A.M., S.E.E., P.J., Y.L.T., S.T.Q., J.T.P.D.H.), NUS Graduate School, Integrative Sciences and Engineering Programme (L.Z.), Department of Computer Science, School of Computing (W.Z., B.C.O.), and Biostatistics Unit, Yong Loo Lin School of Medicine (Q.V.Y., Y.H.C.), National University of Singapore, Singapore; Department of Radiology, Qatif Central Hospital, Qatif, Saudi Arabia (D.A.R.A.); Department of Orthopaedic Surgery, National University Health System, Singapore (J.H.T., N.K.); and Department of Radiological Sciences, University of California, Irvine, Orange, Calif (H.Y.)
| | - James Thomas Patrick Decourcy Hallinan
- From the Department of Diagnostic Imaging, National University Hospital, 5 Lower Kent Ridge Rd, Singapore 119074 (D.S.W.L., A.M., A.J.L.C., D.S.Y.S., S.E.E., H.Y.O., P.J., W.C.T., V.M.K., Y.M.W., Y.L.T., S.B., E.C.T., S.T.Q., J.T.P.D.H.); Department of Diagnostic Radiology (A.M., S.E.E., P.J., Y.L.T., S.T.Q., J.T.P.D.H.), NUS Graduate School, Integrative Sciences and Engineering Programme (L.Z.), Department of Computer Science, School of Computing (W.Z., B.C.O.), and Biostatistics Unit, Yong Loo Lin School of Medicine (Q.V.Y., Y.H.C.), National University of Singapore, Singapore; Department of Radiology, Qatif Central Hospital, Qatif, Saudi Arabia (D.A.R.A.); Department of Orthopaedic Surgery, National University Health System, Singapore (J.H.T., N.K.); and Department of Radiological Sciences, University of California, Irvine, Orange, Calif (H.Y.)
| |
Collapse
|
11
|
Hallinan JTPD, Zhu L, Yang K, Makmur A, Algazwi DAR, Thian YL, Lau S, Choo YS, Eide SE, Yap QV, Chan YH, Tan JH, Kumar N, Ooi BC, Yoshioka H, Quek ST. Deep Learning Model for Automated Detection and Classification of Central Canal, Lateral Recess, and Neural Foraminal Stenosis at Lumbar Spine MRI. Radiology 2021; 300:130-138. [PMID: 33973835 DOI: 10.1148/radiol.2021204289] [Citation(s) in RCA: 68] [Impact Index Per Article: 22.7] [Reference Citation Analysis] [Abstract] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/11/2022]
Abstract
Background Assessment of lumbar spinal stenosis at MRI is repetitive and time consuming. Deep learning (DL) could improve -productivity and the consistency of reporting. Purpose To develop a DL model for automated detection and classification of lumbar central canal, lateral recess, and neural -foraminal stenosis. Materials and Methods In this retrospective study, lumbar spine MRI scans obtained from September 2015 to September 2018 were included. Studies of patients with spinal instrumentation or studies with suboptimal image quality, as well as postgadolinium studies and studies of patients with scoliosis, were excluded. Axial T2-weighted and sagittal T1-weighted images were used. Studies were split into an internal training set (80%), validation set (9%), and test set (11%). Training data were labeled by four radiologists using predefined gradings (normal, mild, moderate, and severe). A two-component DL model was developed. First, a convolutional neural network (CNN) was trained to detect the region of interest (ROI), with a second CNN for classification. An internal test set was labeled by a musculoskeletal radiologist with 31 years of experience (reference standard) and two subspecialist radiologists (radiologist 1: A.M., 5 years of experience; radiologist 2: J.T.P.D.H., 9 years of experience). DL model performance on an external test set was evaluated. Detection recall (in percentage), interrater agreement (Gwet κ), sensitivity, and specificity were calculated. Results Overall, 446 MRI lumbar spine studies were analyzed (446 patients; mean age ± standard deviation, 52 years ± 19; 240 women), with 396 patients in the training (80%) and validation (9%) sets and 50 (11%) in the internal test set. For internal testing, DL model and radiologist central canal recall were greater than 99%, with reduced neural foramina recall for the DL model (84.5%) and radiologist 1 (83.9%) compared with radiologist 2 (97.1%) (P < .001). For internal testing, dichotomous classification (normal or mild vs moderate or severe) showed almost-perfect agreement for both radiologists and the DL model, with respective κ values of 0.98, 0.98, and 0.96 for the central canal; 0.92, 0.95, and 0.92 for lateral recesses; and 0.94, 0.95, and 0.89 for neural foramina (P < .001). External testing with 100 MRI scans of lumbar spines showed almost perfect agreement for the DL model for dichotomous classification of all ROIs (κ, 0.95-0.96; P < .001). Conclusion A deep learning model showed comparable agreement with subspecialist radiologists for detection and classification of central canal and lateral recess stenosis, with slightly lower agreement for neural foraminal stenosis at lumbar spine MRI. © RSNA, 2021 Online supplemental material is available for this article. See also the editorial by Hayashi in this issue.
Collapse
Affiliation(s)
- James Thomas Patrick Decourcy Hallinan
- From the Department of Diagnostic Imaging, National University Hospital, 5 Lower Kent Ridge Rd, Singapore 119074 (J.T.P.D.H., A.M., Y.L.T., S.L., Y.S.C., S.E.E., S.T.Q.); Department of Diagnostic Radiology, Yong Loo Lin School of Medicine, National University of Singapore, Singapore (J.T.P.D.H., A.M., Y.L.T., S.L., Y.S.C., S.E.E., S.T.Q.); NUS Graduate School, Integrative Sciences and Engineering Programme, National University of Singapore, Singapore (L.Z.); Department of Computer Science, School of Computing, National University of Singapore, Singapore (K.Y., B.C.O.); Department of Radiology, Dammam Medical Complex, Dammam, Saudi Arabia (D.A.R.A.); Biostatistics Unit, Yong Loo Lin School of Medicine, Singapore (Q.V.Y., Y.H.C.); University Spine Centre, Department of Orthopaedic Surgery, National University Health System, Singapore (J.H.T., N.K.); and Department of Radiological Sciences, University of California, Irvine, Orange, Calif (H.Y.)
| | - Lei Zhu
- From the Department of Diagnostic Imaging, National University Hospital, 5 Lower Kent Ridge Rd, Singapore 119074 (J.T.P.D.H., A.M., Y.L.T., S.L., Y.S.C., S.E.E., S.T.Q.); Department of Diagnostic Radiology, Yong Loo Lin School of Medicine, National University of Singapore, Singapore (J.T.P.D.H., A.M., Y.L.T., S.L., Y.S.C., S.E.E., S.T.Q.); NUS Graduate School, Integrative Sciences and Engineering Programme, National University of Singapore, Singapore (L.Z.); Department of Computer Science, School of Computing, National University of Singapore, Singapore (K.Y., B.C.O.); Department of Radiology, Dammam Medical Complex, Dammam, Saudi Arabia (D.A.R.A.); Biostatistics Unit, Yong Loo Lin School of Medicine, Singapore (Q.V.Y., Y.H.C.); University Spine Centre, Department of Orthopaedic Surgery, National University Health System, Singapore (J.H.T., N.K.); and Department of Radiological Sciences, University of California, Irvine, Orange, Calif (H.Y.)
| | - Kaiyuan Yang
- From the Department of Diagnostic Imaging, National University Hospital, 5 Lower Kent Ridge Rd, Singapore 119074 (J.T.P.D.H., A.M., Y.L.T., S.L., Y.S.C., S.E.E., S.T.Q.); Department of Diagnostic Radiology, Yong Loo Lin School of Medicine, National University of Singapore, Singapore (J.T.P.D.H., A.M., Y.L.T., S.L., Y.S.C., S.E.E., S.T.Q.); NUS Graduate School, Integrative Sciences and Engineering Programme, National University of Singapore, Singapore (L.Z.); Department of Computer Science, School of Computing, National University of Singapore, Singapore (K.Y., B.C.O.); Department of Radiology, Dammam Medical Complex, Dammam, Saudi Arabia (D.A.R.A.); Biostatistics Unit, Yong Loo Lin School of Medicine, Singapore (Q.V.Y., Y.H.C.); University Spine Centre, Department of Orthopaedic Surgery, National University Health System, Singapore (J.H.T., N.K.); and Department of Radiological Sciences, University of California, Irvine, Orange, Calif (H.Y.)
| | - Andrew Makmur
- From the Department of Diagnostic Imaging, National University Hospital, 5 Lower Kent Ridge Rd, Singapore 119074 (J.T.P.D.H., A.M., Y.L.T., S.L., Y.S.C., S.E.E., S.T.Q.); Department of Diagnostic Radiology, Yong Loo Lin School of Medicine, National University of Singapore, Singapore (J.T.P.D.H., A.M., Y.L.T., S.L., Y.S.C., S.E.E., S.T.Q.); NUS Graduate School, Integrative Sciences and Engineering Programme, National University of Singapore, Singapore (L.Z.); Department of Computer Science, School of Computing, National University of Singapore, Singapore (K.Y., B.C.O.); Department of Radiology, Dammam Medical Complex, Dammam, Saudi Arabia (D.A.R.A.); Biostatistics Unit, Yong Loo Lin School of Medicine, Singapore (Q.V.Y., Y.H.C.); University Spine Centre, Department of Orthopaedic Surgery, National University Health System, Singapore (J.H.T., N.K.); and Department of Radiological Sciences, University of California, Irvine, Orange, Calif (H.Y.)
| | - Diyaa Abdul Rauf Algazwi
- From the Department of Diagnostic Imaging, National University Hospital, 5 Lower Kent Ridge Rd, Singapore 119074 (J.T.P.D.H., A.M., Y.L.T., S.L., Y.S.C., S.E.E., S.T.Q.); Department of Diagnostic Radiology, Yong Loo Lin School of Medicine, National University of Singapore, Singapore (J.T.P.D.H., A.M., Y.L.T., S.L., Y.S.C., S.E.E., S.T.Q.); NUS Graduate School, Integrative Sciences and Engineering Programme, National University of Singapore, Singapore (L.Z.); Department of Computer Science, School of Computing, National University of Singapore, Singapore (K.Y., B.C.O.); Department of Radiology, Dammam Medical Complex, Dammam, Saudi Arabia (D.A.R.A.); Biostatistics Unit, Yong Loo Lin School of Medicine, Singapore (Q.V.Y., Y.H.C.); University Spine Centre, Department of Orthopaedic Surgery, National University Health System, Singapore (J.H.T., N.K.); and Department of Radiological Sciences, University of California, Irvine, Orange, Calif (H.Y.)
| | - Yee Liang Thian
- From the Department of Diagnostic Imaging, National University Hospital, 5 Lower Kent Ridge Rd, Singapore 119074 (J.T.P.D.H., A.M., Y.L.T., S.L., Y.S.C., S.E.E., S.T.Q.); Department of Diagnostic Radiology, Yong Loo Lin School of Medicine, National University of Singapore, Singapore (J.T.P.D.H., A.M., Y.L.T., S.L., Y.S.C., S.E.E., S.T.Q.); NUS Graduate School, Integrative Sciences and Engineering Programme, National University of Singapore, Singapore (L.Z.); Department of Computer Science, School of Computing, National University of Singapore, Singapore (K.Y., B.C.O.); Department of Radiology, Dammam Medical Complex, Dammam, Saudi Arabia (D.A.R.A.); Biostatistics Unit, Yong Loo Lin School of Medicine, Singapore (Q.V.Y., Y.H.C.); University Spine Centre, Department of Orthopaedic Surgery, National University Health System, Singapore (J.H.T., N.K.); and Department of Radiological Sciences, University of California, Irvine, Orange, Calif (H.Y.)
| | - Samuel Lau
- From the Department of Diagnostic Imaging, National University Hospital, 5 Lower Kent Ridge Rd, Singapore 119074 (J.T.P.D.H., A.M., Y.L.T., S.L., Y.S.C., S.E.E., S.T.Q.); Department of Diagnostic Radiology, Yong Loo Lin School of Medicine, National University of Singapore, Singapore (J.T.P.D.H., A.M., Y.L.T., S.L., Y.S.C., S.E.E., S.T.Q.); NUS Graduate School, Integrative Sciences and Engineering Programme, National University of Singapore, Singapore (L.Z.); Department of Computer Science, School of Computing, National University of Singapore, Singapore (K.Y., B.C.O.); Department of Radiology, Dammam Medical Complex, Dammam, Saudi Arabia (D.A.R.A.); Biostatistics Unit, Yong Loo Lin School of Medicine, Singapore (Q.V.Y., Y.H.C.); University Spine Centre, Department of Orthopaedic Surgery, National University Health System, Singapore (J.H.T., N.K.); and Department of Radiological Sciences, University of California, Irvine, Orange, Calif (H.Y.)
| | - Yun Song Choo
- From the Department of Diagnostic Imaging, National University Hospital, 5 Lower Kent Ridge Rd, Singapore 119074 (J.T.P.D.H., A.M., Y.L.T., S.L., Y.S.C., S.E.E., S.T.Q.); Department of Diagnostic Radiology, Yong Loo Lin School of Medicine, National University of Singapore, Singapore (J.T.P.D.H., A.M., Y.L.T., S.L., Y.S.C., S.E.E., S.T.Q.); NUS Graduate School, Integrative Sciences and Engineering Programme, National University of Singapore, Singapore (L.Z.); Department of Computer Science, School of Computing, National University of Singapore, Singapore (K.Y., B.C.O.); Department of Radiology, Dammam Medical Complex, Dammam, Saudi Arabia (D.A.R.A.); Biostatistics Unit, Yong Loo Lin School of Medicine, Singapore (Q.V.Y., Y.H.C.); University Spine Centre, Department of Orthopaedic Surgery, National University Health System, Singapore (J.H.T., N.K.); and Department of Radiological Sciences, University of California, Irvine, Orange, Calif (H.Y.)
| | - Sterling Ellis Eide
- From the Department of Diagnostic Imaging, National University Hospital, 5 Lower Kent Ridge Rd, Singapore 119074 (J.T.P.D.H., A.M., Y.L.T., S.L., Y.S.C., S.E.E., S.T.Q.); Department of Diagnostic Radiology, Yong Loo Lin School of Medicine, National University of Singapore, Singapore (J.T.P.D.H., A.M., Y.L.T., S.L., Y.S.C., S.E.E., S.T.Q.); NUS Graduate School, Integrative Sciences and Engineering Programme, National University of Singapore, Singapore (L.Z.); Department of Computer Science, School of Computing, National University of Singapore, Singapore (K.Y., B.C.O.); Department of Radiology, Dammam Medical Complex, Dammam, Saudi Arabia (D.A.R.A.); Biostatistics Unit, Yong Loo Lin School of Medicine, Singapore (Q.V.Y., Y.H.C.); University Spine Centre, Department of Orthopaedic Surgery, National University Health System, Singapore (J.H.T., N.K.); and Department of Radiological Sciences, University of California, Irvine, Orange, Calif (H.Y.)
| | - Qai Ven Yap
- From the Department of Diagnostic Imaging, National University Hospital, 5 Lower Kent Ridge Rd, Singapore 119074 (J.T.P.D.H., A.M., Y.L.T., S.L., Y.S.C., S.E.E., S.T.Q.); Department of Diagnostic Radiology, Yong Loo Lin School of Medicine, National University of Singapore, Singapore (J.T.P.D.H., A.M., Y.L.T., S.L., Y.S.C., S.E.E., S.T.Q.); NUS Graduate School, Integrative Sciences and Engineering Programme, National University of Singapore, Singapore (L.Z.); Department of Computer Science, School of Computing, National University of Singapore, Singapore (K.Y., B.C.O.); Department of Radiology, Dammam Medical Complex, Dammam, Saudi Arabia (D.A.R.A.); Biostatistics Unit, Yong Loo Lin School of Medicine, Singapore (Q.V.Y., Y.H.C.); University Spine Centre, Department of Orthopaedic Surgery, National University Health System, Singapore (J.H.T., N.K.); and Department of Radiological Sciences, University of California, Irvine, Orange, Calif (H.Y.)
| | - Yiong Huak Chan
- From the Department of Diagnostic Imaging, National University Hospital, 5 Lower Kent Ridge Rd, Singapore 119074 (J.T.P.D.H., A.M., Y.L.T., S.L., Y.S.C., S.E.E., S.T.Q.); Department of Diagnostic Radiology, Yong Loo Lin School of Medicine, National University of Singapore, Singapore (J.T.P.D.H., A.M., Y.L.T., S.L., Y.S.C., S.E.E., S.T.Q.); NUS Graduate School, Integrative Sciences and Engineering Programme, National University of Singapore, Singapore (L.Z.); Department of Computer Science, School of Computing, National University of Singapore, Singapore (K.Y., B.C.O.); Department of Radiology, Dammam Medical Complex, Dammam, Saudi Arabia (D.A.R.A.); Biostatistics Unit, Yong Loo Lin School of Medicine, Singapore (Q.V.Y., Y.H.C.); University Spine Centre, Department of Orthopaedic Surgery, National University Health System, Singapore (J.H.T., N.K.); and Department of Radiological Sciences, University of California, Irvine, Orange, Calif (H.Y.)
| | - Jiong Hao Tan
- From the Department of Diagnostic Imaging, National University Hospital, 5 Lower Kent Ridge Rd, Singapore 119074 (J.T.P.D.H., A.M., Y.L.T., S.L., Y.S.C., S.E.E., S.T.Q.); Department of Diagnostic Radiology, Yong Loo Lin School of Medicine, National University of Singapore, Singapore (J.T.P.D.H., A.M., Y.L.T., S.L., Y.S.C., S.E.E., S.T.Q.); NUS Graduate School, Integrative Sciences and Engineering Programme, National University of Singapore, Singapore (L.Z.); Department of Computer Science, School of Computing, National University of Singapore, Singapore (K.Y., B.C.O.); Department of Radiology, Dammam Medical Complex, Dammam, Saudi Arabia (D.A.R.A.); Biostatistics Unit, Yong Loo Lin School of Medicine, Singapore (Q.V.Y., Y.H.C.); University Spine Centre, Department of Orthopaedic Surgery, National University Health System, Singapore (J.H.T., N.K.); and Department of Radiological Sciences, University of California, Irvine, Orange, Calif (H.Y.)
| | - Naresh Kumar
- From the Department of Diagnostic Imaging, National University Hospital, 5 Lower Kent Ridge Rd, Singapore 119074 (J.T.P.D.H., A.M., Y.L.T., S.L., Y.S.C., S.E.E., S.T.Q.); Department of Diagnostic Radiology, Yong Loo Lin School of Medicine, National University of Singapore, Singapore (J.T.P.D.H., A.M., Y.L.T., S.L., Y.S.C., S.E.E., S.T.Q.); NUS Graduate School, Integrative Sciences and Engineering Programme, National University of Singapore, Singapore (L.Z.); Department of Computer Science, School of Computing, National University of Singapore, Singapore (K.Y., B.C.O.); Department of Radiology, Dammam Medical Complex, Dammam, Saudi Arabia (D.A.R.A.); Biostatistics Unit, Yong Loo Lin School of Medicine, Singapore (Q.V.Y., Y.H.C.); University Spine Centre, Department of Orthopaedic Surgery, National University Health System, Singapore (J.H.T., N.K.); and Department of Radiological Sciences, University of California, Irvine, Orange, Calif (H.Y.)
| | - Beng Chin Ooi
- From the Department of Diagnostic Imaging, National University Hospital, 5 Lower Kent Ridge Rd, Singapore 119074 (J.T.P.D.H., A.M., Y.L.T., S.L., Y.S.C., S.E.E., S.T.Q.); Department of Diagnostic Radiology, Yong Loo Lin School of Medicine, National University of Singapore, Singapore (J.T.P.D.H., A.M., Y.L.T., S.L., Y.S.C., S.E.E., S.T.Q.); NUS Graduate School, Integrative Sciences and Engineering Programme, National University of Singapore, Singapore (L.Z.); Department of Computer Science, School of Computing, National University of Singapore, Singapore (K.Y., B.C.O.); Department of Radiology, Dammam Medical Complex, Dammam, Saudi Arabia (D.A.R.A.); Biostatistics Unit, Yong Loo Lin School of Medicine, Singapore (Q.V.Y., Y.H.C.); University Spine Centre, Department of Orthopaedic Surgery, National University Health System, Singapore (J.H.T., N.K.); and Department of Radiological Sciences, University of California, Irvine, Orange, Calif (H.Y.)
| | - Hiroshi Yoshioka
- From the Department of Diagnostic Imaging, National University Hospital, 5 Lower Kent Ridge Rd, Singapore 119074 (J.T.P.D.H., A.M., Y.L.T., S.L., Y.S.C., S.E.E., S.T.Q.); Department of Diagnostic Radiology, Yong Loo Lin School of Medicine, National University of Singapore, Singapore (J.T.P.D.H., A.M., Y.L.T., S.L., Y.S.C., S.E.E., S.T.Q.); NUS Graduate School, Integrative Sciences and Engineering Programme, National University of Singapore, Singapore (L.Z.); Department of Computer Science, School of Computing, National University of Singapore, Singapore (K.Y., B.C.O.); Department of Radiology, Dammam Medical Complex, Dammam, Saudi Arabia (D.A.R.A.); Biostatistics Unit, Yong Loo Lin School of Medicine, Singapore (Q.V.Y., Y.H.C.); University Spine Centre, Department of Orthopaedic Surgery, National University Health System, Singapore (J.H.T., N.K.); and Department of Radiological Sciences, University of California, Irvine, Orange, Calif (H.Y.)
| | - Swee Tian Quek
- From the Department of Diagnostic Imaging, National University Hospital, 5 Lower Kent Ridge Rd, Singapore 119074 (J.T.P.D.H., A.M., Y.L.T., S.L., Y.S.C., S.E.E., S.T.Q.); Department of Diagnostic Radiology, Yong Loo Lin School of Medicine, National University of Singapore, Singapore (J.T.P.D.H., A.M., Y.L.T., S.L., Y.S.C., S.E.E., S.T.Q.); NUS Graduate School, Integrative Sciences and Engineering Programme, National University of Singapore, Singapore (L.Z.); Department of Computer Science, School of Computing, National University of Singapore, Singapore (K.Y., B.C.O.); Department of Radiology, Dammam Medical Complex, Dammam, Saudi Arabia (D.A.R.A.); Biostatistics Unit, Yong Loo Lin School of Medicine, Singapore (Q.V.Y., Y.H.C.); University Spine Centre, Department of Orthopaedic Surgery, National University Health System, Singapore (J.H.T., N.K.); and Department of Radiological Sciences, University of California, Irvine, Orange, Calif (H.Y.)
| |
Collapse
|