251
|
Sigaki HYD, Lenzi EK, Zola RS, Perc M, Ribeiro HV. Learning physical properties of liquid crystals with deep convolutional neural networks. Sci Rep 2020; 10:7664. [PMID: 32376993 PMCID: PMC7203147 DOI: 10.1038/s41598-020-63662-9] [Citation(s) in RCA: 18] [Impact Index Per Article: 4.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/10/2019] [Accepted: 04/03/2020] [Indexed: 02/03/2023] Open
Abstract
Machine learning algorithms have been available since the 1990s, but it is much more recently that they have come into use also in the physical sciences. While these algorithms have already proven to be useful in uncovering new properties of materials and in simplifying experimental protocols, their usage in liquid crystals research is still limited. This is surprising because optical imaging techniques are often applied in this line of research, and it is precisely with images that machine learning algorithms have achieved major breakthroughs in recent years. Here we use convolutional neural networks to probe several properties of liquid crystals directly from their optical images and without using manual feature engineering. By optimizing simple architectures, we find that convolutional neural networks can predict physical properties of liquid crystals with exceptional accuracy. We show that these deep neural networks identify liquid crystal phases and predict the order parameter of simulated nematic liquid crystals almost perfectly. We also show that convolutional neural networks identify the pitch length of simulated samples of cholesteric liquid crystals and the sample temperature of an experimental liquid crystal with very high precision.
Collapse
Affiliation(s)
- Higor Y D Sigaki
- Departamento de Física, Universidade Estadual de Maringá, Maringá, PR, 87020-900, Brazil
| | - Ervin K Lenzi
- Departamento de Física, Universidade Estadual de Ponta Grossa, Ponta Grossa, PR, 84030-900, Brazil
| | - Rafael S Zola
- Departamento de Física, Universidade Estadual de Maringá, Maringá, PR, 87020-900, Brazil
- Departamento de Física, Universidade Tecnológica Federal do Paraná, Apucarana, PR, 86812-460, Brazil
| | - Matjaž Perc
- Faculty of Natural Sciences and Mathematics, University of Maribor, Koroška cesta 160, 2000, Maribor, Slovenia
- Department of Medical Research, China Medical University Hospital, China Medical University, Taichung, Taiwan
- Complexity Science Hub Vienna, Josefstädterstraße 39, 1080, Vienna, Austria
| | - Haroldo V Ribeiro
- Departamento de Física, Universidade Estadual de Maringá, Maringá, PR, 87020-900, Brazil.
| |
Collapse
|
252
|
Suojärvi N, Tampio J, Lindfors N, Waris E. Computer-aided 3D analysis of anatomy and radiographic parameters of the distal radius. Clin Anat 2020; 34:574-580. [PMID: 32346905 DOI: 10.1002/ca.23615] [Citation(s) in RCA: 13] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/04/2020] [Revised: 04/21/2020] [Accepted: 04/23/2020] [Indexed: 12/11/2022]
Abstract
INTRODUCTION This study applied mathematical modeling to examine the anatomy of the distal radius; to define the radiographic parameters in a 3D imaging modality; and to report their normal ranges in the uninjured radius. MATERIALS AND METHODS A series of 50 cone-beam computed tomography (CBCT) scans of uninjured radii were analyzed using computer-aided image processing. The radius shape model was used to determine the optimal location for measuring the longitudinal axis. With the axis determined, the volar tilt and radial inclination angles and the areas of the articular facets and their reference points were analyzed. RESULTS The optimal location for determining the longitudinal axis was between 28.8 and 53.3 mm proximally from the articular surface. The mean radial inclination angle was 21.8°. The mean volar tilt angle via the most distal tips of the volar and dorsal rims was 13.0°; along the lunate and scaphoid facets it was 9.1° and 11.2°, respectively. The scaphoid facet was larger than the lunate facet and 25% of it was convex. CONCLUSIONS Computer-aided CBCT image processing offers an advanced tool to record 3D geometry and the radiographic parameters of the osseous structures of the wrist. Analysis of the distal radius' anatomy showed that the longitudinal axis was affected by its measurement location and subsequently also affected the determination of the angular parameters. We also report the variation of the volar tilt along the articular surface and the shapes and sizes of the articular facets.
Collapse
Affiliation(s)
- Nora Suojärvi
- Department of Hand Surgery, Helsinki University Hospital and University of Helsinki, Helsinki, Finland
| | | | - Nina Lindfors
- Department of Hand Surgery, Helsinki University Hospital and University of Helsinki, Helsinki, Finland
| | - Eero Waris
- Department of Hand Surgery, Helsinki University Hospital and University of Helsinki, Helsinki, Finland
| |
Collapse
|
253
|
Deep learning in rare disease. Detection of tubers in tuberous sclerosis complex. PLoS One 2020; 15:e0232376. [PMID: 32348367 PMCID: PMC7190137 DOI: 10.1371/journal.pone.0232376] [Citation(s) in RCA: 16] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/22/2020] [Accepted: 04/13/2020] [Indexed: 01/09/2023] Open
Abstract
Objective To develop and test a deep learning algorithm to automatically detect cortical tubers in magnetic resonance imaging (MRI), to explore the utility of deep learning in rare disorders with limited data, and to generate an open-access deep learning standalone application. Methods T2 and FLAIR axial images with and without tubers were extracted from MRIs of patients with tuberous sclerosis complex (TSC) and controls, respectively. We trained three different convolutional neural network (CNN) architectures on a training dataset and selected the one with the lowest binary cross-entropy loss in the validation dataset, which was evaluated on the testing dataset. We visualized image regions most relevant for classification with gradient-weighted class activation maps (Grad-CAM) and saliency maps. Results 114 patients with TSC and 114 controls were divided into a training set, a validation set, and a testing set. The InceptionV3 CNN architecture performed best in the validation set and was evaluated in the testing set with the following results: sensitivity: 0.95, specificity: 0.95, positive predictive value: 0.94, negative predictive value: 0.95, F1-score: 0.95, accuracy: 0.95, and area under the curve: 0.99. Grad-CAM and saliency maps showed that tubers resided in regions most relevant for image classification within each image. A stand-alone trained deep learning App was able to classify images using local computers with various operating systems. Conclusion This study shows that deep learning algorithms are able to detect tubers in selected MRI images, and deep learning can be prudently applied clinically to manually selected data in a rare neurological disorder.
Collapse
|
254
|
Urban G, Porhemmat S, Stark M, Feeley B, Okada K, Baldi P. Classifying shoulder implants in X-ray images using deep learning. Comput Struct Biotechnol J 2020; 18:967-972. [PMID: 32368331 PMCID: PMC7186366 DOI: 10.1016/j.csbj.2020.04.005] [Citation(s) in RCA: 21] [Impact Index Per Article: 5.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/05/2019] [Revised: 04/04/2020] [Accepted: 04/05/2020] [Indexed: 12/27/2022] Open
Abstract
Total Shoulder Arthroplasty (TSA) is a type of surgery in which the damaged ball of the shoulder is replaced with a prosthesis. Many years later, this prosthesis may be in need of servicing or replacement. In some situations, such as when the patient has changed his country of residence, the model and the manufacturer of the prosthesis may be unknown to the patient and primary doctor. Correct identification of the implant's model prior to surgery is required for selecting the correct equipment and procedure. We present a novel way to automatically classify shoulder implants in X-ray images. We employ deep learning models and compare their performance to alternative classifiers, such as random forests and gradient boosting. We find that deep convolutional neural networks outperform other classifiers significantly if and only if out-of-domain data such as ImageNet is used to pre-train the models. In a data set containing X-ray images of shoulder implants from 4 manufacturers and 16 different models, deep learning is able to identify the correct manufacturer with an accuracy of approximately 80% in 10-fold cross validation, while other classifiers achieve an accuracy of 56% or less. We believe that this approach will be a useful tool in clinical practice, and is likely applicable to other kinds of prostheses.
Collapse
Affiliation(s)
- Gregor Urban
- University of California, Irvine School of Information and Computer Sciences, Irvine, CA, USA
| | - Saman Porhemmat
- University of California, Irvine School of Information and Computer Sciences, Irvine, CA, USA
| | - Maya Stark
- San Francisco State University, Computer Science Department, San Francisco, CA, USA
| | - Brian Feeley
- University of California, San Francisco, Department of Orthopaedic Surgery, San Francisco, CA, USA
| | - Kazunori Okada
- San Francisco State University, Computer Science Department, San Francisco, CA, USA
| | - Pierre Baldi
- University of California, Irvine School of Information and Computer Sciences, Irvine, CA, USA
| |
Collapse
|
255
|
von Schacky CE, Sohn JH, Liu F, Ozhinsky E, Jungmann PM, Nardo L, Posadzy M, Foreman SC, Nevitt MC, Link TM, Pedoia V. Development and Validation of a Multitask Deep Learning Model for Severity Grading of Hip Osteoarthritis Features on Radiographs. Radiology 2020; 295:136-145. [PMID: 32013791 PMCID: PMC7104703 DOI: 10.1148/radiol.2020190925] [Citation(s) in RCA: 50] [Impact Index Per Article: 12.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/26/2019] [Revised: 11/12/2019] [Accepted: 11/22/2019] [Indexed: 11/11/2022]
Abstract
Background A multitask deep learning model might be useful in large epidemiologic studies wherein detailed structural assessment of osteoarthritis still relies on expert radiologists' readings. The potential of such a model in clinical routine should be investigated. Purpose To develop a multitask deep learning model for grading radiographic hip osteoarthritis features on radiographs and compare its performance to that of attending-level radiologists. Materials and Methods This retrospective study analyzed hip joints seen on weight-bearing anterior-posterior pelvic radiographs from participants in the Osteoarthritis Initiative (OAI). Participants were recruited from February 2004 to May 2006 for baseline measurements, and follow-up was performed 48 months later. Femoral osteophytes (FOs), acetabular osteophytes (AOs), and joint-space narrowing (JSN) were graded as absent, mild, moderate, or severe according to the Osteoarthritis Research Society International atlas. Subchondral sclerosis and subchondral cysts were graded as present or absent. The participants were split at 80% (n = 3494), 10% (n = 437), and 10% (n = 437) by using split-sample validation into training, validation, and testing sets, respectively. The multitask neural network was based on DenseNet-161, a shared convolutional features extractor trained with multitask loss function. Model performance was evaluated in the internal test set from the OAI and in an external test set by using temporal and geographic validation consisting of routine clinical radiographs. Results A total of 4368 participants (mean age, 61.0 years ± 9.2 [standard deviation]; 2538 women) were evaluated (15 364 hip joints on 7738 weight-bearing anterior-posterior pelvic radiographs). The accuracy of the model for assessing these five features was 86.7% (1333 of 1538) for FOs, 69.9% (1075 of 1538) for AOs, 81.7% (1257 of 1538) for JSN, 95.8% (1473 of 1538) for subchondral sclerosis, and 97.6% (1501 of 1538) for subchondral cysts in the internal test set, and 82.7% (86 of 104) for FOS, 65.4% (68 of 104) for AOs, 80.8% (84 of 104) for JSN, 88.5% (92 of 104) for subchondral sclerosis, and 91.3% (95 of 104) for subchondral cysts in the external test set. Conclusion A multitask deep learning model is a feasible approach to reliably assess radiographic features of hip osteoarthritis. © RSNA, 2020 Online supplemental material is available for this article.
Collapse
Affiliation(s)
- Claudio E. von Schacky
- From the Department of Radiology and Biomedical Imaging (C.E.v.S., J.H.S., E.O., P.M.J., M.P., S.C.F., T.M.L., V.P.) and Department of Epidemiology and Biostatistics (F.L., M.C.N.), University of California, San Francisco, 185 Berry St, Suite 350, San Francisco, CA 94107; Department of Diagnostic and Interventional Radiology, Technische Universität München, Munich, Germany (C.E.v.S., S.C.F.); Department of Diagnostic and Interventional Radiology, Medical Center–University of Freiburg, Faculty of Medicine, Freiburg, Germany (P.M.J.); and Department of Radiology, University of California Davis Health, Sacramento, Calif (L.N.)
| | - Jae Ho Sohn
- From the Department of Radiology and Biomedical Imaging (C.E.v.S., J.H.S., E.O., P.M.J., M.P., S.C.F., T.M.L., V.P.) and Department of Epidemiology and Biostatistics (F.L., M.C.N.), University of California, San Francisco, 185 Berry St, Suite 350, San Francisco, CA 94107; Department of Diagnostic and Interventional Radiology, Technische Universität München, Munich, Germany (C.E.v.S., S.C.F.); Department of Diagnostic and Interventional Radiology, Medical Center–University of Freiburg, Faculty of Medicine, Freiburg, Germany (P.M.J.); and Department of Radiology, University of California Davis Health, Sacramento, Calif (L.N.)
| | - Felix Liu
- From the Department of Radiology and Biomedical Imaging (C.E.v.S., J.H.S., E.O., P.M.J., M.P., S.C.F., T.M.L., V.P.) and Department of Epidemiology and Biostatistics (F.L., M.C.N.), University of California, San Francisco, 185 Berry St, Suite 350, San Francisco, CA 94107; Department of Diagnostic and Interventional Radiology, Technische Universität München, Munich, Germany (C.E.v.S., S.C.F.); Department of Diagnostic and Interventional Radiology, Medical Center–University of Freiburg, Faculty of Medicine, Freiburg, Germany (P.M.J.); and Department of Radiology, University of California Davis Health, Sacramento, Calif (L.N.)
| | - Eugene Ozhinsky
- From the Department of Radiology and Biomedical Imaging (C.E.v.S., J.H.S., E.O., P.M.J., M.P., S.C.F., T.M.L., V.P.) and Department of Epidemiology and Biostatistics (F.L., M.C.N.), University of California, San Francisco, 185 Berry St, Suite 350, San Francisco, CA 94107; Department of Diagnostic and Interventional Radiology, Technische Universität München, Munich, Germany (C.E.v.S., S.C.F.); Department of Diagnostic and Interventional Radiology, Medical Center–University of Freiburg, Faculty of Medicine, Freiburg, Germany (P.M.J.); and Department of Radiology, University of California Davis Health, Sacramento, Calif (L.N.)
| | - Pia M. Jungmann
- From the Department of Radiology and Biomedical Imaging (C.E.v.S., J.H.S., E.O., P.M.J., M.P., S.C.F., T.M.L., V.P.) and Department of Epidemiology and Biostatistics (F.L., M.C.N.), University of California, San Francisco, 185 Berry St, Suite 350, San Francisco, CA 94107; Department of Diagnostic and Interventional Radiology, Technische Universität München, Munich, Germany (C.E.v.S., S.C.F.); Department of Diagnostic and Interventional Radiology, Medical Center–University of Freiburg, Faculty of Medicine, Freiburg, Germany (P.M.J.); and Department of Radiology, University of California Davis Health, Sacramento, Calif (L.N.)
| | - Lorenzo Nardo
- From the Department of Radiology and Biomedical Imaging (C.E.v.S., J.H.S., E.O., P.M.J., M.P., S.C.F., T.M.L., V.P.) and Department of Epidemiology and Biostatistics (F.L., M.C.N.), University of California, San Francisco, 185 Berry St, Suite 350, San Francisco, CA 94107; Department of Diagnostic and Interventional Radiology, Technische Universität München, Munich, Germany (C.E.v.S., S.C.F.); Department of Diagnostic and Interventional Radiology, Medical Center–University of Freiburg, Faculty of Medicine, Freiburg, Germany (P.M.J.); and Department of Radiology, University of California Davis Health, Sacramento, Calif (L.N.)
| | - Magdalena Posadzy
- From the Department of Radiology and Biomedical Imaging (C.E.v.S., J.H.S., E.O., P.M.J., M.P., S.C.F., T.M.L., V.P.) and Department of Epidemiology and Biostatistics (F.L., M.C.N.), University of California, San Francisco, 185 Berry St, Suite 350, San Francisco, CA 94107; Department of Diagnostic and Interventional Radiology, Technische Universität München, Munich, Germany (C.E.v.S., S.C.F.); Department of Diagnostic and Interventional Radiology, Medical Center–University of Freiburg, Faculty of Medicine, Freiburg, Germany (P.M.J.); and Department of Radiology, University of California Davis Health, Sacramento, Calif (L.N.)
| | - Sarah C. Foreman
- From the Department of Radiology and Biomedical Imaging (C.E.v.S., J.H.S., E.O., P.M.J., M.P., S.C.F., T.M.L., V.P.) and Department of Epidemiology and Biostatistics (F.L., M.C.N.), University of California, San Francisco, 185 Berry St, Suite 350, San Francisco, CA 94107; Department of Diagnostic and Interventional Radiology, Technische Universität München, Munich, Germany (C.E.v.S., S.C.F.); Department of Diagnostic and Interventional Radiology, Medical Center–University of Freiburg, Faculty of Medicine, Freiburg, Germany (P.M.J.); and Department of Radiology, University of California Davis Health, Sacramento, Calif (L.N.)
| | - Michael C. Nevitt
- From the Department of Radiology and Biomedical Imaging (C.E.v.S., J.H.S., E.O., P.M.J., M.P., S.C.F., T.M.L., V.P.) and Department of Epidemiology and Biostatistics (F.L., M.C.N.), University of California, San Francisco, 185 Berry St, Suite 350, San Francisco, CA 94107; Department of Diagnostic and Interventional Radiology, Technische Universität München, Munich, Germany (C.E.v.S., S.C.F.); Department of Diagnostic and Interventional Radiology, Medical Center–University of Freiburg, Faculty of Medicine, Freiburg, Germany (P.M.J.); and Department of Radiology, University of California Davis Health, Sacramento, Calif (L.N.)
| | - Thomas M. Link
- From the Department of Radiology and Biomedical Imaging (C.E.v.S., J.H.S., E.O., P.M.J., M.P., S.C.F., T.M.L., V.P.) and Department of Epidemiology and Biostatistics (F.L., M.C.N.), University of California, San Francisco, 185 Berry St, Suite 350, San Francisco, CA 94107; Department of Diagnostic and Interventional Radiology, Technische Universität München, Munich, Germany (C.E.v.S., S.C.F.); Department of Diagnostic and Interventional Radiology, Medical Center–University of Freiburg, Faculty of Medicine, Freiburg, Germany (P.M.J.); and Department of Radiology, University of California Davis Health, Sacramento, Calif (L.N.)
| | - Valentina Pedoia
- From the Department of Radiology and Biomedical Imaging (C.E.v.S., J.H.S., E.O., P.M.J., M.P., S.C.F., T.M.L., V.P.) and Department of Epidemiology and Biostatistics (F.L., M.C.N.), University of California, San Francisco, 185 Berry St, Suite 350, San Francisco, CA 94107; Department of Diagnostic and Interventional Radiology, Technische Universität München, Munich, Germany (C.E.v.S., S.C.F.); Department of Diagnostic and Interventional Radiology, Medical Center–University of Freiburg, Faculty of Medicine, Freiburg, Germany (P.M.J.); and Department of Radiology, University of California Davis Health, Sacramento, Calif (L.N.)
| |
Collapse
|
256
|
Kalmet PHS, Sanduleanu S, Primakov S, Wu G, Jochems A, Refaee T, Ibrahim A, Hulst LV, Lambin P, Poeze M. Deep learning in fracture detection: a narrative review. Acta Orthop 2020; 91:215-220. [PMID: 31928116 PMCID: PMC7144272 DOI: 10.1080/17453674.2019.1711323] [Citation(s) in RCA: 64] [Impact Index Per Article: 16.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 02/08/2023] Open
Abstract
Artificial intelligence (AI) is a general term that implies the use of a computer to model intelligent behavior with minimal human intervention. AI, particularly deep learning, has recently made substantial strides in perception tasks allowing machines to better represent and interpret complex data. Deep learning is a subset of AI represented by the combination of artificial neuron layers. In the last years, deep learning has gained great momentum. In the field of orthopaedics and traumatology, some studies have been done using deep learning to detect fractures in radiographs. Deep learning studies to detect and classify fractures on computed tomography (CT) scans are even more limited. In this narrative review, we provide a brief overview of deep learning technology: we (1) describe the ways in which deep learning until now has been applied to fracture detection on radiographs and CT examinations; (2) discuss what value deep learning offers to this field; and finally (3) comment on future directions of this technology.
Collapse
Affiliation(s)
| | - Sebastian Sanduleanu
- The D-Lab: Decision Support for Precision Medicine, GROW—School for Oncology and Developmental Biology, Maastricht University Medical Center+, Maastricht
| | - Sergey Primakov
- The D-Lab: Decision Support for Precision Medicine, GROW—School for Oncology and Developmental Biology, Maastricht University Medical Center+, Maastricht
| | - Guangyao Wu
- The D-Lab: Decision Support for Precision Medicine, GROW—School for Oncology and Developmental Biology, Maastricht University Medical Center+, Maastricht
| | - Arthur Jochems
- The D-Lab: Decision Support for Precision Medicine, GROW—School for Oncology and Developmental Biology, Maastricht University Medical Center+, Maastricht
| | - Turkey Refaee
- The D-Lab: Decision Support for Precision Medicine, GROW—School for Oncology and Developmental Biology, Maastricht University Medical Center+, Maastricht
| | - Abdalla Ibrahim
- The D-Lab: Decision Support for Precision Medicine, GROW—School for Oncology and Developmental Biology, Maastricht University Medical Center+, Maastricht
| | - Luca v. Hulst
- Maastricht University Medical Center+, Department of Trauma Surgery, Maastricht
| | - Philippe Lambin
- The D-Lab: Decision Support for Precision Medicine, GROW—School for Oncology and Developmental Biology, Maastricht University Medical Center+, Maastricht
| | - Martijn Poeze
- Maastricht University Medical Center+, Department of Trauma Surgery, Maastricht
- Nutrim School for Nutrition, Toxicology and Metabolism, Maastricht University, Maastricht, The Netherlands
| |
Collapse
|
257
|
Borjali A, Chen AF, Muratoglu OK, Morid MA, Varadarajan KM. Deep Learning in Orthopedics: How Do We Build Trust in the Machine? ACTA ACUST UNITED AC 2020. [DOI: 10.1089/heat.2019.0006] [Citation(s) in RCA: 13] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/18/2022]
Affiliation(s)
- Alireza Borjali
- Department of Orthopaedic Surgery, Harvard Medical School, Boston, Massachusetts
- Harris Orthopaedics Laboratory, Department of Orthopaedic, Massachusetts General Hospital, Boston, Massachusetts
| | - Antonia F. Chen
- Department of Orthopaedic Surgery, Brigham and Women's Hospital, Harvard Medical School, Boston, Massachusetts
| | - Orhun K. Muratoglu
- Department of Orthopaedic Surgery, Harvard Medical School, Boston, Massachusetts
- Harris Orthopaedics Laboratory, Department of Orthopaedic, Massachusetts General Hospital, Boston, Massachusetts
| | - Mohammad A. Morid
- Department of Information Systems and Analytics, Santa Clara University Leavey School of Business, Santa Clara, California
| | - Kartik M. Varadarajan
- Department of Orthopaedic Surgery, Harvard Medical School, Boston, Massachusetts
- Harris Orthopaedics Laboratory, Department of Orthopaedic, Massachusetts General Hospital, Boston, Massachusetts
| |
Collapse
|
258
|
Drozdov I, Forbes D, Szubert B, Hall M, Carlin C, Lowe DJ. Supervised and unsupervised language modelling in Chest X-Ray radiological reports. PLoS One 2020; 15:e0229963. [PMID: 32155219 PMCID: PMC7064166 DOI: 10.1371/journal.pone.0229963] [Citation(s) in RCA: 9] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/23/2019] [Accepted: 02/17/2020] [Indexed: 12/14/2022] Open
Abstract
Chest radiography (CXR) is the most commonly used imaging modality and deep neural network (DNN) algorithms have shown promise in effective triage of normal and abnormal radiograms. Typically, DNNs require large quantities of expertly labelled training exemplars, which in clinical contexts is a major bottleneck to effective modelling, as both considerable clinical skill and time is required to produce high-quality ground truths. In this work we evaluate thirteen supervised classifiers using two large free-text corpora and demonstrate that bi-directional long short-term memory (BiLSTM) networks with attention mechanism effectively identify Normal, Abnormal, and Unclear CXR reports in internal (n = 965 manually-labelled reports, f1-score = 0.94) and external (n = 465 manually-labelled reports, f1-score = 0.90) testing sets using a relatively small number of expert-labelled training observations (n = 3,856 annotated reports). Furthermore, we introduce a general unsupervised approach that accurately distinguishes Normal and Abnormal CXR reports in a large unlabelled corpus. We anticipate that the results presented in this work can be used to automatically extract standardized clinical information from free-text CXR radiological reports, facilitating the training of clinical decision support systems for CXR triage.
Collapse
Affiliation(s)
| | - Daniel Forbes
- Emergency Department, Queen Elizabeth University Hospital, Glasgow, Scotland
| | | | - Mark Hall
- Radiology Department, Queen Elizabeth University Hospital, Glasgow, Scotland
| | - Chris Carlin
- Department of Respiratory Medicine. Queen Elizabeth University Hospital, Glasgow, Scotland
| | - David J. Lowe
- Emergency Department, Queen Elizabeth University Hospital, Glasgow, Scotland
| |
Collapse
|
259
|
Blüthgen C, Becker AS, Vittoria de Martini I, Meier A, Martini K, Frauenfelder T. Detection and localization of distal radius fractures: Deep learning system versus radiologists. Eur J Radiol 2020; 126:108925. [PMID: 32193036 DOI: 10.1016/j.ejrad.2020.108925] [Citation(s) in RCA: 39] [Impact Index Per Article: 9.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/08/2019] [Revised: 02/21/2020] [Accepted: 02/27/2020] [Indexed: 12/29/2022]
Abstract
PURPOSE To evaluate a deep learning based image analysis software for the detection and localization of distal radius fractures. METHOD A deep learning system (DLS) was trained on 524 wrist radiographs (166 showing fractures). Performance was tested on internal (100 radiographs, 42 showing fractures) and external test sets (200 radiographs, 100 showing fractures). Single and combined views of the radiographs were shown to DLS and three readers. Readers were asked to indicate fracture location with regions of interest (ROI). The DLS yielded scores (range 0-1) and a heatmap. Detection performance was expressed as AUC, sensitivity and specificity at the optimal threshold and compared to radiologists' performance. Heatmaps were compared to radiologists' ROIs. RESULTS The DLS showed excellent performance on the internal test set (AUC 0.93 (95% confidence interval (CI) 0.82-0.98) - 0.96 (0.87-1.00), sensitivity 0.81 (0.58-0.95) - 0.90 (0.70-0.99), specificity 0.86 (0.68-0.96) - 1.0 (0.88-1.0)). DLS performance decreased on the external test set (AUC 0.80 (0.71-0.88) - 0.89 (0.81-0.94), sensitivity 0.64 (0.49-0.77) - 0.92 (0.81-0.98), specificity 0.60 (0.45-0.74) - 0.90 (0.78-0.97)). Radiologists' performance was comparable on internal data (sensitivity 0.71 (0.48-0.89) - 0.95 (0.76-1.0), specificity 0.52 (0.32-0.71) - 0.97 (0.82-1.0)) and better on external data (sensitivity 0.88 (0.76-0.96) - 0.98 (0.89-1.0), specificities 0.66 (0.51-0.79) - 1.0 (0.93-1.0), p < 0.05). In over 90%, the areas of peak activation aligned with radiologists' annotations. CONCLUSIONS The DLS was able to detect and localize wrist fractures with a performance comparable to radiologists, using only a small dataset for training.
Collapse
Affiliation(s)
- Christian Blüthgen
- Institute for Diagnostic and Interventional Radiology, University Hospital of Zurich, Switzerland.
| | - Anton S Becker
- Institute for Diagnostic and Interventional Radiology, University Hospital of Zurich, Switzerland
| | | | - Andreas Meier
- Institute for Diagnostic and Interventional Radiology, University Hospital of Zurich, Switzerland
| | - Katharina Martini
- Institute for Diagnostic and Interventional Radiology, University Hospital of Zurich, Switzerland
| | - Thomas Frauenfelder
- Institute for Diagnostic and Interventional Radiology, University Hospital of Zurich, Switzerland
| |
Collapse
|
260
|
Krogue JD, Cheng KV, Hwang KM, Toogood P, Meinberg EG, Geiger EJ, Zaid M, McGill KC, Patel R, Sohn JH, Wright A, Darger BF, Padrez KA, Ozhinsky E, Majumdar S, Pedoia V. Automatic Hip Fracture Identification and Functional Subclassification with Deep Learning. Radiol Artif Intell 2020; 2:e190023. [PMID: 33937815 PMCID: PMC8017394 DOI: 10.1148/ryai.2020190023] [Citation(s) in RCA: 68] [Impact Index Per Article: 17.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/24/2019] [Revised: 11/06/2019] [Accepted: 12/19/2019] [Indexed: 05/01/2023]
Abstract
PURPOSE To investigate the feasibility of automatic identification and classification of hip fractures using deep learning, which may improve outcomes by reducing diagnostic errors and decreasing time to operation. MATERIALS AND METHODS Hip and pelvic radiographs from 1118 studies were reviewed, and 3026 hips were labeled via bounding boxes and classified as normal, displaced femoral neck fracture, nondisplaced femoral neck fracture, intertrochanteric fracture, previous open reduction and internal fixation, or previous arthroplasty. A deep learning-based object detection model was trained to automate the placement of the bounding boxes. A Densely Connected Convolutional Neural Network (or DenseNet) was trained on a subset of the bounding box images, and its performance was evaluated on a held-out test set and by comparison on a 100-image subset with two groups of human observers: fellowship-trained radiologists and orthopedists; senior residents in emergency medicine, radiology, and orthopedics. RESULTS The binary accuracy for detecting a fracture of this model was 93.7% (95% confidence interval [CI]: 90.8%, 96.5%), with a sensitivity of 93.2% (95% CI: 88.9%, 97.1%) and a specificity of 94.2% (95% CI: 89.7%, 98.4%). Multiclass classification accuracy was 90.8% (95% CI: 87.5%, 94.2%). When compared with the accuracy of human observers, the accuracy of the model achieved an expert-level classification, at the very least, under all conditions. Additionally, when the model was used as an aid, human performance improved, with aided resident performance approximating unaided fellowship-trained expert performance in the multiclass classification. CONCLUSION A deep learning model identified and classified hip fractures with expert-level performance, at the very least, and when used as an aid, improved human performance, with aided resident performance approximating that of unaided fellowship-trained attending physicians.Supplemental material is available for this article.© RSNA, 2020.
Collapse
Affiliation(s)
| | | | - Kevin M. Hwang
- From the Departments of Orthopaedic Surgery (J.D.K., K.M.H., P.T., E.G.M., E.J.G., M.Z.), Emergency Medicine (B.F.D., K.A.P.), and Radiology and Biomedical Imaging (K.C.M., R.P., J.H.S., A.W., E.O., S.M., V.P.), University of California, San Francisco, 6945 Geary Blvd, San Francisco, CA 94121; and Department of Electrical Engineering and Computer Sciences, University of California, Berkeley, Berkeley, Calif (K.V.C.)
| | - Paul Toogood
- From the Departments of Orthopaedic Surgery (J.D.K., K.M.H., P.T., E.G.M., E.J.G., M.Z.), Emergency Medicine (B.F.D., K.A.P.), and Radiology and Biomedical Imaging (K.C.M., R.P., J.H.S., A.W., E.O., S.M., V.P.), University of California, San Francisco, 6945 Geary Blvd, San Francisco, CA 94121; and Department of Electrical Engineering and Computer Sciences, University of California, Berkeley, Berkeley, Calif (K.V.C.)
| | - Eric G. Meinberg
- From the Departments of Orthopaedic Surgery (J.D.K., K.M.H., P.T., E.G.M., E.J.G., M.Z.), Emergency Medicine (B.F.D., K.A.P.), and Radiology and Biomedical Imaging (K.C.M., R.P., J.H.S., A.W., E.O., S.M., V.P.), University of California, San Francisco, 6945 Geary Blvd, San Francisco, CA 94121; and Department of Electrical Engineering and Computer Sciences, University of California, Berkeley, Berkeley, Calif (K.V.C.)
| | - Erik J. Geiger
- From the Departments of Orthopaedic Surgery (J.D.K., K.M.H., P.T., E.G.M., E.J.G., M.Z.), Emergency Medicine (B.F.D., K.A.P.), and Radiology and Biomedical Imaging (K.C.M., R.P., J.H.S., A.W., E.O., S.M., V.P.), University of California, San Francisco, 6945 Geary Blvd, San Francisco, CA 94121; and Department of Electrical Engineering and Computer Sciences, University of California, Berkeley, Berkeley, Calif (K.V.C.)
| | - Musa Zaid
- From the Departments of Orthopaedic Surgery (J.D.K., K.M.H., P.T., E.G.M., E.J.G., M.Z.), Emergency Medicine (B.F.D., K.A.P.), and Radiology and Biomedical Imaging (K.C.M., R.P., J.H.S., A.W., E.O., S.M., V.P.), University of California, San Francisco, 6945 Geary Blvd, San Francisco, CA 94121; and Department of Electrical Engineering and Computer Sciences, University of California, Berkeley, Berkeley, Calif (K.V.C.)
| | - Kevin C. McGill
- From the Departments of Orthopaedic Surgery (J.D.K., K.M.H., P.T., E.G.M., E.J.G., M.Z.), Emergency Medicine (B.F.D., K.A.P.), and Radiology and Biomedical Imaging (K.C.M., R.P., J.H.S., A.W., E.O., S.M., V.P.), University of California, San Francisco, 6945 Geary Blvd, San Francisco, CA 94121; and Department of Electrical Engineering and Computer Sciences, University of California, Berkeley, Berkeley, Calif (K.V.C.)
| | - Rina Patel
- From the Departments of Orthopaedic Surgery (J.D.K., K.M.H., P.T., E.G.M., E.J.G., M.Z.), Emergency Medicine (B.F.D., K.A.P.), and Radiology and Biomedical Imaging (K.C.M., R.P., J.H.S., A.W., E.O., S.M., V.P.), University of California, San Francisco, 6945 Geary Blvd, San Francisco, CA 94121; and Department of Electrical Engineering and Computer Sciences, University of California, Berkeley, Berkeley, Calif (K.V.C.)
| | - Jae Ho Sohn
- From the Departments of Orthopaedic Surgery (J.D.K., K.M.H., P.T., E.G.M., E.J.G., M.Z.), Emergency Medicine (B.F.D., K.A.P.), and Radiology and Biomedical Imaging (K.C.M., R.P., J.H.S., A.W., E.O., S.M., V.P.), University of California, San Francisco, 6945 Geary Blvd, San Francisco, CA 94121; and Department of Electrical Engineering and Computer Sciences, University of California, Berkeley, Berkeley, Calif (K.V.C.)
| | - Alexandra Wright
- From the Departments of Orthopaedic Surgery (J.D.K., K.M.H., P.T., E.G.M., E.J.G., M.Z.), Emergency Medicine (B.F.D., K.A.P.), and Radiology and Biomedical Imaging (K.C.M., R.P., J.H.S., A.W., E.O., S.M., V.P.), University of California, San Francisco, 6945 Geary Blvd, San Francisco, CA 94121; and Department of Electrical Engineering and Computer Sciences, University of California, Berkeley, Berkeley, Calif (K.V.C.)
| | - Bryan F. Darger
- From the Departments of Orthopaedic Surgery (J.D.K., K.M.H., P.T., E.G.M., E.J.G., M.Z.), Emergency Medicine (B.F.D., K.A.P.), and Radiology and Biomedical Imaging (K.C.M., R.P., J.H.S., A.W., E.O., S.M., V.P.), University of California, San Francisco, 6945 Geary Blvd, San Francisco, CA 94121; and Department of Electrical Engineering and Computer Sciences, University of California, Berkeley, Berkeley, Calif (K.V.C.)
| | - Kevin A. Padrez
- From the Departments of Orthopaedic Surgery (J.D.K., K.M.H., P.T., E.G.M., E.J.G., M.Z.), Emergency Medicine (B.F.D., K.A.P.), and Radiology and Biomedical Imaging (K.C.M., R.P., J.H.S., A.W., E.O., S.M., V.P.), University of California, San Francisco, 6945 Geary Blvd, San Francisco, CA 94121; and Department of Electrical Engineering and Computer Sciences, University of California, Berkeley, Berkeley, Calif (K.V.C.)
| | - Eugene Ozhinsky
- From the Departments of Orthopaedic Surgery (J.D.K., K.M.H., P.T., E.G.M., E.J.G., M.Z.), Emergency Medicine (B.F.D., K.A.P.), and Radiology and Biomedical Imaging (K.C.M., R.P., J.H.S., A.W., E.O., S.M., V.P.), University of California, San Francisco, 6945 Geary Blvd, San Francisco, CA 94121; and Department of Electrical Engineering and Computer Sciences, University of California, Berkeley, Berkeley, Calif (K.V.C.)
| | - Sharmila Majumdar
- From the Departments of Orthopaedic Surgery (J.D.K., K.M.H., P.T., E.G.M., E.J.G., M.Z.), Emergency Medicine (B.F.D., K.A.P.), and Radiology and Biomedical Imaging (K.C.M., R.P., J.H.S., A.W., E.O., S.M., V.P.), University of California, San Francisco, 6945 Geary Blvd, San Francisco, CA 94121; and Department of Electrical Engineering and Computer Sciences, University of California, Berkeley, Berkeley, Calif (K.V.C.)
| | - Valentina Pedoia
- From the Departments of Orthopaedic Surgery (J.D.K., K.M.H., P.T., E.G.M., E.J.G., M.Z.), Emergency Medicine (B.F.D., K.A.P.), and Radiology and Biomedical Imaging (K.C.M., R.P., J.H.S., A.W., E.O., S.M., V.P.), University of California, San Francisco, 6945 Geary Blvd, San Francisco, CA 94121; and Department of Electrical Engineering and Computer Sciences, University of California, Berkeley, Berkeley, Calif (K.V.C.)
| |
Collapse
|
261
|
Can a machine learning model accurately predict patient resource utilization following lumbar spinal fusion? Spine J 2020; 20:329-336. [PMID: 31654809 DOI: 10.1016/j.spinee.2019.10.007] [Citation(s) in RCA: 24] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 06/30/2019] [Revised: 10/15/2019] [Accepted: 10/16/2019] [Indexed: 02/03/2023]
Abstract
BACKGROUND CONTEXT With the increasing emphasis on value-based healthcare in Centers for Medicare and Medicaid Services reimbursement structures, bundled payment models have been adopted for many orthopedic procedures. Immense variability of patients across hospitals and providers makes these models potentially less viable in spine surgery. Machine-learning models have been shown reliable at predicting patient-specific outcomes following lumbar spine surgery and could, therefore, be applied to developing stratified bundled payment schemes. PURPOSE (1) Can a Naïve Bayes machine-learning model accurately predict inpatient payments, length of stay (LOS), and discharge disposition, following dorsal and lumbar fusion? (2) Can such a model then be used to develop a risk-stratified payment scheme? STUDY DESIGN A Naïve Bayes machine-learning model was constructed using an administrative database. PATIENT SAMPLE Patients undergoing dorsal and lumbar fusion for nondeformity indications from 2009 through 2016 were included. Preoperative inputs included age group, gender, ethnicity, race, type of admission, All Patients Refined (APR) risk of mortality, APR severity of illness, and Clinical Classifications Software diagnosis code. OUTCOME MEASURES Predicted resource utilization outcomes included LOS, discharge disposition, and total inpatient payments. Model validation was addressed via reliability, model output quality, and decision speed, based on application of training and validation sets. Risk-stratified payment models were developed according to APR risk of mortality and severity of illness. RESULTS A Naïve Bayes machine-learning algorithm with adaptive boosting demonstrated high reliability and area under the receiver-operating characteristics curve of 0.880, 0.941, and 0.906 for cost, LOS, and discharge disposition, respectively. Patients with increased risk of mortality or severity of illness incurred costs resulting in greater inpatient payments in a patient-specific tiered bundled payment, reflecting increased risk on institutions caring for these patients. We found that a large range in expected payments due to individuals' preoperative comorbidities indicating an individualized risk-based model is warranted. CONCLUSIONS A Naïve Bayes machine-learning model was shown to have good-to-excellent reliability and responsiveness for cost, LOS, and discharge disposition. Based on APR risk of mortality and APR severity of illness, there was a significant difference in episode costs from lowest to highest risk strata. After using normalized model error to develop a risk-adjusted proposed payment plan, it was found that institutions incur significantly more financial risk in flat bundled payment models for patients with higher rates of comorbidities.
Collapse
|
262
|
X-Ray Bone Fracture Classification Using Deep Learning: A Baseline for Designing a Reliable Approach. APPLIED SCIENCES-BASEL 2020. [DOI: 10.3390/app10041507] [Citation(s) in RCA: 21] [Impact Index Per Article: 5.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/16/2022]
Abstract
In recent years, bone fracture detection and classification has been a widely discussed topic and many researchers have proposed different methods to tackle this problem. Despite this, a universal approach able to classify all the fractures in the human body has not yet been defined. We aim to analyze and evaluate a selection of papers, chosen according to their representative approach, where the authors applied different deep learning techniques to classify bone fractures, in order to select the strengths of each of them and try to delineate a generalized strategy. Each study is summarized and evaluated using a radar graph with six values: area under the curve (AUC), test accuracy, sensitivity, specificity, dataset size and labelling reliability. Plus, we defined the key points which should be taken into account when trying to accomplish this purpose and we compared each study with our baseline. In recent years, deep learning and, in particular, the convolution neural network (CNN), has achieved results comparable to those of humans in bone fracture classification. Adopting a correct generalization, we are reasonably sure that a computer-aided diagnosis (CAD) system, correctly designed to assist doctors, would save a considerable amount of time and would limit the number of wrong diagnoses.
Collapse
|
263
|
Peiffer-Smadja N, Dellière S, Rodriguez C, Birgand G, Lescure FX, Fourati S, Ruppé E. Machine learning in the clinical microbiology laboratory: has the time come for routine practice? Clin Microbiol Infect 2020; 26:1300-1309. [PMID: 32061795 DOI: 10.1016/j.cmi.2020.02.006] [Citation(s) in RCA: 48] [Impact Index Per Article: 12.0] [Reference Citation Analysis] [Abstract] [Key Words] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/31/2019] [Revised: 02/04/2020] [Accepted: 02/06/2020] [Indexed: 12/20/2022]
Abstract
BACKGROUND Machine learning (ML) allows the analysis of complex and large data sets and has the potential to improve health care. The clinical microbiology laboratory, at the interface of clinical practice and diagnostics, is of special interest for the development of ML systems. AIMS This narrative review aims to explore the current use of ML In clinical microbiology. SOURCES References for this review were identified through searches of MEDLINE/PubMed, EMBASE, Google Scholar, biorXiv, arXiV, ACM Digital Library and IEEE Xplore Digital Library up to November 2019. CONTENT We found 97 ML systems aiming to assist clinical microbiologists. Overall, 82 ML systems (85%) targeted bacterial infections, 11 (11%) parasitic infections, nine (9%) viral infections and three (3%) fungal infections. Forty ML systems (41%) focused on microorganism detection, identification and quantification, 36 (37%) evaluated antimicrobial susceptibility, and 21 (22%) targeted the diagnosis, disease classification and prediction of clinical outcomes. The ML systems used very diverse data sources: 21 (22%) used genomic data of microorganisms, 19 (20%) microbiota data obtained by metagenomic sequencing, 19 (20%) analysed microscopic images, 17 (18%) spectroscopy data, eight (8%) targeted gene sequencing, six (6%) volatile organic compounds, four (4%) photographs of bacterial colonies, four (4%) transcriptome data, three (3%) protein structure, and three (3%) clinical data. Most systems used data from high-income countries (n = 71, 73%) but a significant number used data from low- and middle-income countries (n = 36, 37%). Performance measures were reported for the 97 ML systems, but no article described their use in clinical practice or reported impact on processes or clinical outcomes. IMPLICATIONS In clinical microbiology, ML has been used with various data sources and diverse practical applications. The evaluation and implementation processes represent the main gap in existing ML systems, requiring a focus on their interpretability and potential integration into real-world settings.
Collapse
Affiliation(s)
- N Peiffer-Smadja
- National Institute for Health Research Health Protection Research Unit in Healthcare Associated Infections and Antimicrobial Resistance, Imperial College London, London, UK; Université de Paris, IAME, INSERM, F-75018 Paris, France
| | - S Dellière
- Université de Paris, Laboratoire de Parasitologie-Mycologie, Groupe Hospitalier Saint-Louis-Lariboisière-Fernand-Widal, Assistance Publique-Hôpitaux de Paris (AP-HP), Paris, France
| | - C Rodriguez
- Department of Prevention, Diagnosis and Treatment of Infections, Henri-Mondor Hospital, APHP, Université Paris-Est Créteil, IMRB, INSERM U955, Créteil, France
| | - G Birgand
- National Institute for Health Research Health Protection Research Unit in Healthcare Associated Infections and Antimicrobial Resistance, Imperial College London, London, UK
| | - F-X Lescure
- Université de Paris, IAME, INSERM, F-75018 Paris, France
| | - S Fourati
- Department of Prevention, Diagnosis and Treatment of Infections, Henri-Mondor Hospital, APHP, Université Paris-Est Créteil, IMRB, INSERM U955, Créteil, France
| | - E Ruppé
- Université de Paris, IAME, INSERM, F-75018 Paris, France.
| |
Collapse
|
264
|
Graham SA, Lee EE, Jeste DV, Van Patten R, Twamley EW, Nebeker C, Yamada Y, Kim HC, Depp CA. Artificial intelligence approaches to predicting and detecting cognitive decline in older adults: A conceptual review. Psychiatry Res 2020; 284:112732. [PMID: 31978628 PMCID: PMC7081667 DOI: 10.1016/j.psychres.2019.112732] [Citation(s) in RCA: 54] [Impact Index Per Article: 13.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 10/09/2019] [Revised: 12/04/2019] [Accepted: 12/07/2019] [Indexed: 12/13/2022]
Abstract
Preserving cognition and mental capacity is critical to aging with autonomy. Early detection of pathological cognitive decline facilitates the greatest impact of restorative or preventative treatments. Artificial Intelligence (AI) in healthcare is the use of computational algorithms that mimic human cognitive functions to analyze complex medical data. AI technologies like machine learning (ML) support the integration of biological, psychological, and social factors when approaching diagnosis, prognosis, and treatment of disease. This paper serves to acquaint clinicians and other stakeholders with the use, benefits, and limitations of AI for predicting, diagnosing, and classifying mild and major neurocognitive impairments, by providing a conceptual overview of this topic with emphasis on the features explored and AI techniques employed. We present studies that fell into six categories of features used for these purposes: (1) sociodemographics; (2) clinical and psychometric assessments; (3) neuroimaging and neurophysiology; (4) electronic health records and claims; (5) novel assessments (e.g., sensors for digital data); and (6) genomics/other omics. For each category we provide examples of AI approaches, including supervised and unsupervised ML, deep learning, and natural language processing. AI technology, still nascent in healthcare, has great potential to transform the way we diagnose and treat patients with neurocognitive disorders.
Collapse
Affiliation(s)
- Sarah A Graham
- Department of Psychiatry, University of California San Diego, La Jolla, CA, United States; Sam and Rose Stein Institute for Research on Aging, University of California San Diego, La Jolla, CA, United States; IBM-UCSD Artificial Intelligence for Healthy Living Program, La Jolla, CA, United States
| | - Ellen E Lee
- Department of Psychiatry, University of California San Diego, La Jolla, CA, United States; Sam and Rose Stein Institute for Research on Aging, University of California San Diego, La Jolla, CA, United States; IBM-UCSD Artificial Intelligence for Healthy Living Program, La Jolla, CA, United States; VA San Diego Healthcare System, San Diego, CA, United States
| | - Dilip V Jeste
- Department of Psychiatry, University of California San Diego, La Jolla, CA, United States; Sam and Rose Stein Institute for Research on Aging, University of California San Diego, La Jolla, CA, United States; IBM-UCSD Artificial Intelligence for Healthy Living Program, La Jolla, CA, United States; Department of Neurosciences, University of California San Diego, La Jolla, CA, United States.
| | - Ryan Van Patten
- Department of Psychiatry, University of California San Diego, La Jolla, CA, United States; Sam and Rose Stein Institute for Research on Aging, University of California San Diego, La Jolla, CA, United States; IBM-UCSD Artificial Intelligence for Healthy Living Program, La Jolla, CA, United States
| | - Elizabeth W Twamley
- Department of Psychiatry, University of California San Diego, La Jolla, CA, United States; Sam and Rose Stein Institute for Research on Aging, University of California San Diego, La Jolla, CA, United States; IBM-UCSD Artificial Intelligence for Healthy Living Program, La Jolla, CA, United States; VA San Diego Healthcare System, San Diego, CA, United States
| | - Camille Nebeker
- Sam and Rose Stein Institute for Research on Aging, University of California San Diego, La Jolla, CA, United States; IBM-UCSD Artificial Intelligence for Healthy Living Program, La Jolla, CA, United States; Department of Family Medicine and Public Health, University of California San Diego, La Jolla, CA, United States
| | | | - Ho-Cheol Kim
- IBM-UCSD Artificial Intelligence for Healthy Living Program, La Jolla, CA, United States; Scalable Knowledge Intelligence, IBM Research-Almaden, San Jose, CA, United States
| | - Colin A Depp
- Department of Psychiatry, University of California San Diego, La Jolla, CA, United States; Sam and Rose Stein Institute for Research on Aging, University of California San Diego, La Jolla, CA, United States; IBM-UCSD Artificial Intelligence for Healthy Living Program, La Jolla, CA, United States; VA San Diego Healthcare System, San Diego, CA, United States
| |
Collapse
|
265
|
Chea P, Mandell JC. Current applications and future directions of deep learning in musculoskeletal radiology. Skeletal Radiol 2020; 49:183-197. [PMID: 31377836 DOI: 10.1007/s00256-019-03284-z] [Citation(s) in RCA: 55] [Impact Index Per Article: 13.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 05/05/2019] [Revised: 07/11/2019] [Accepted: 07/15/2019] [Indexed: 02/02/2023]
Abstract
Deep learning with convolutional neural networks (CNN) is a rapidly advancing subset of artificial intelligence that is ideally suited to solving image-based problems. There are an increasing number of musculoskeletal applications of deep learning, which can be conceptually divided into the categories of lesion detection, classification, segmentation, and non-interpretive tasks. Numerous examples of deep learning achieving expert-level performance in specific tasks in all four categories have been demonstrated in the past few years, although comprehensive interpretation of imaging examinations has not yet been achieved. It is important for the practicing musculoskeletal radiologist to understand the current scope of deep learning as it relates to musculoskeletal radiology. Interest in deep learning from researchers, radiology leadership, and industry continues to increase, and it is likely that these developments will impact the daily practice of musculoskeletal radiology in the near future.
Collapse
Affiliation(s)
- Pauley Chea
- Division of Musculoskeletal Imaging and Intervention, Department of Radiology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA.
| | - Jacob C Mandell
- Division of Musculoskeletal Imaging and Intervention, Department of Radiology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA
| |
Collapse
|
266
|
Abstract
Artificial intelligence (AI) is a very relevant topic for the medicine of the future. This article focuses on the field of AI in the context of orthopedics and trauma surgery. The main focus is on the potentials of AI in the analysis of symptoms, radiological images, clinical data sets, use in hospitals and operating theaters as well as for training and education. For the orthopedics and trauma surgery of the future AI is much more than pure fiction; however, there is still a long way to go before the potential of an optimized and individualized patient care can be utilized. Interdisciplinary and international approaches, including personnel, economic, legal and ethical aspects will play a decisive role in this respect.
Collapse
|
267
|
|
268
|
Burns JE, Yao J, Summers RM. Artificial Intelligence in Musculoskeletal Imaging: A Paradigm Shift. J Bone Miner Res 2020; 35:28-35. [PMID: 31398274 DOI: 10.1002/jbmr.3849] [Citation(s) in RCA: 14] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 04/11/2019] [Revised: 07/23/2019] [Accepted: 08/05/2019] [Indexed: 01/22/2023]
Abstract
Artificial intelligence is upending many of our assumptions about the ability of computers to detect and diagnose diseases on medical images. Deep learning, a recent innovation in artificial intelligence, has shown the ability to interpret medical images with sensitivities and specificities at or near that of skilled clinicians for some applications. In this review, we summarize the history of artificial intelligence, present some recent research advances, and speculate about the potential revolutionary clinical impact of the latest computer techniques for bone and muscle imaging. © 2019 American Society for Bone and Mineral Research. Published 2019. This article is a U.S. Government work and is in the public domain in the USA.
Collapse
Affiliation(s)
- Joseph E Burns
- Department of Radiological Sciences, University of California-Irvine School of Medicine, Orange, CA, USA
| | - Jianhua Yao
- Imaging Biomarkers and Computer-Aided Diagnosis Laboratory, Radiology and Imaging Sciences Department, Clinical Center, National Institutes of Health, Bethesda, MD, USA
| | - Ronald M Summers
- Imaging Biomarkers and Computer-Aided Diagnosis Laboratory, Radiology and Imaging Sciences Department, Clinical Center, National Institutes of Health, Bethesda, MD, USA
| |
Collapse
|
269
|
Shin H, Lee J, Eo T, Jun Y, Kim S, Hwang D. The Latest Trends in Attention Mechanisms and Their Application in Medical Imaging. JOURNAL OF THE KOREAN SOCIETY OF RADIOLOGY 2020; 81:1305-1333. [PMID: 36237722 PMCID: PMC9431827 DOI: 10.3348/jksr.2020.0150] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 08/13/2020] [Revised: 11/02/2020] [Accepted: 11/07/2020] [Indexed: 11/29/2022]
Abstract
딥러닝 기술은 빅데이터 및 컴퓨팅 파워를 기반으로 최근 영상의학 분야의 연구에서 괄목할만한 성과를 이루어 내고 있다. 하지만 성능 향상을 위해 딥러닝 네트워크가 깊어질수록 그 내부의 계산 과정을 해석하기 어려워졌는데, 이는 환자의 생명과 직결되는 의료분야의 의사 결정 과정에서는 매우 심각한 문제이다. 이를 해결하기 위해 “설명 가능한 인공지능 기술”이 연구되고 있으며, 그중 하나로 개발된 것이 바로 어텐션(attention) 기법이다. 본 종설에서는 이미 학습이 완료된 네트워크를 분석하기 위한 Post-hoc attention과, 네트워크 성능의 추가적인 향상을 위한 Trainable attention 두 종류의 기법에 대해 각각의 방법 및 의료 영상 연구에 적용된 사례, 그리고 향후 전망 등에 대해 자세히 다루고자 한다.
Collapse
Affiliation(s)
- Hyungseob Shin
- Department of Electrical and Electronic Engineering, Yonsei University, Seoul, Korea
| | - Jeongryong Lee
- Department of Electrical and Electronic Engineering, Yonsei University, Seoul, Korea
| | - Taejoon Eo
- Department of Electrical and Electronic Engineering, Yonsei University, Seoul, Korea
| | - Yohan Jun
- Department of Electrical and Electronic Engineering, Yonsei University, Seoul, Korea
| | - Sewon Kim
- Department of Electrical and Electronic Engineering, Yonsei University, Seoul, Korea
| | - Dosik Hwang
- Department of Electrical and Electronic Engineering, Yonsei University, Seoul, Korea
| |
Collapse
|
270
|
Waite S, Farooq Z, Grigorian A, Sistrom C, Kolla S, Mancuso A, Martinez-Conde S, Alexander RG, Kantor A, Macknik SL. A Review of Perceptual Expertise in Radiology-How it develops, How we can test it, and Why humans still matter in the era of Artificial Intelligence. Acad Radiol 2020; 27:26-38. [PMID: 31818384 DOI: 10.1016/j.acra.2019.08.018] [Citation(s) in RCA: 17] [Impact Index Per Article: 4.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/18/2019] [Revised: 08/26/2019] [Accepted: 08/27/2019] [Indexed: 10/25/2022]
Abstract
As the first step in image interpretation is detection, an error in perception can prematurely end the diagnostic process leading to missed diagnoses. Because perceptual errors of this sort-"failure to detect"-are the most common interpretive error (and cause of litigation) in radiology, understanding the nature of perceptual expertise is essential in decreasing radiology's long-standing error rates. In this article, we review what constitutes a perceptual error, the existing models of radiologic image perception, the development of perceptual expertise and how it can be tested, perceptual learning methods in training radiologists, and why understanding perceptual expertise is still relevant in the era of artificial intelligence. Adding targeted interventions, such as perceptual learning, to existing teaching practices, has the potential to enhance expertise and reduce medical error.
Collapse
|
271
|
Kang YJ, Yoo JI, Cha YH, Park CH, Kim JT. Machine learning-based identification of hip arthroplasty designs. J Orthop Translat 2019; 21:13-17. [PMID: 32071870 PMCID: PMC7013104 DOI: 10.1016/j.jot.2019.11.004] [Citation(s) in RCA: 28] [Impact Index Per Article: 5.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 05/17/2019] [Revised: 09/11/2019] [Accepted: 11/21/2019] [Indexed: 11/10/2022] Open
Abstract
Background The purposes of this study were to develop a machine learning–based implant recognition program and to verify its accuracy. Methods Postoperative anteroposterior (AP) X-rays (≥300 dpi) were collected of patients who underwent total hip arthroplasty. X-rays with a wire or plate added and those without a true anteroposterior view were excluded. A total of 170 X-ray images of hip implants from 29 brands were collected from five hospitals and a Google image search. These collected images were manually reorganised to ensure appropriate labelling. Collected images were preprocessed to have grey-scaled pixels with histogram equalisation for efficient training. Images varied by +10/−10°, and 3606 unique images derived from the original 170 images were created for training. Discussion of the validation set being derived 25% of training set. The recognition model structure consisted of two steps: object detection and clustering. Model training was performed with Keras deep learning platform. Results The 170 X-ray images of hip implants were used to build a stem detection model using YOLOv3. Manually labelled images were successfully trained into the stem detection model. Evaluation of 58 newly labelled X-ray images showed highly accurate stem detection (mean average precision > 0.99). Fully connected layers generated 29 class outputs. After training, a receiver operating characteristic curve was generated with a test set containing 25% of all stem-cropped images, yielding an area under the curve of 0.99. Conclusion Femoral stem identification in patients with total hip arthroplasty was very accurate. This technology could be used to collect large-scale implant information. The translational potential of this article This program has the following clinical relevance. First, we can prepare the implants needed for revision surgery by identifying the old types of implants. Second, it can be used to diagnose peripheral osteolysis or periprosthetic fracture by further developing the ability to sensitise implant detection. Third, an automated implant detection system will help organise imaging data systematically and easily for arthroplasty registry construction.
Collapse
Affiliation(s)
- Yang-Jae Kang
- Division of Applied Life Science Department at Gyeongsang National University, PMBBRC, Jinju, Republic of Korea.,Division of Life Science Department at Gyeongsang National University, Jinju, Republic of Korea
| | - Jun-Il Yoo
- Department of Orthopaedic Surgery, Gyeongsang National University Hospital, Jinju, South Korea
| | - Yong-Han Cha
- Department of General Surgery, Eulji University Hospital, Daejeon, South Korea
| | - Chan H Park
- Department of Orthopedic Surgery, Yeungnam University Medical Center, Daegu, South Korea
| | - Jung-Taek Kim
- Department of Orthopedic Surgery, Ajou University School of Medicine, Ajou Medical Center, Suwon, South Korea
| |
Collapse
|
272
|
Deep Convolutional Neural Networks for Automatic Detection of Orbital Blowout Fractures. J Craniofac Surg 2019; 31:400-403. [PMID: 31842071 DOI: 10.1097/scs.0000000000006069] [Citation(s) in RCA: 12] [Impact Index Per Article: 2.4] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/28/2023] Open
Abstract
Orbital blow out fracture is a common disease in emergency department and a delay or failure in diagnosis can lead to permanent visual changes. This study aims to evaluate the ability of an automatic orbital blowout fractures detection system based on computed tomography (CT) data.Orbital CT scans of adult orbital blowout fractures patients and normal cases were obtained from Shanghai Ninth People's Hospital between January and March 2017. The region of fractures was annotated using 3D Slicer. The Inception V3 convolutional neural networks were constructed utilizing the Python programming language with PyTorch as the framework to extract high dimension features from each slice in a CT scan. These extracted features are processed through a XGBoost model to make the final differentiation of fracture cases and nonfracture ones. Accuracy, receiver operating characteristics, and area under the curve were evaluated.This study used 94 CT scans diagnosed with orbital blowout fractures and 94 healthy control cases. The automatic detection system showed accuracy of 92% in single-image classification and 87% in patient level detection. The area under the receiver operating characteristic curve was 0.9574.Using a deep learning-based automatic detection system of orbital blowout fracture can accurately detect and classify orbital blowout fractures from CT scans. The convolutional neural networks model combined with an accurate annotation system could achieve good performance in a small dataset. Further studies with large and multicenter data are required to refine this technology for possible clinical applications.
Collapse
|
273
|
|
274
|
Schneider P, Walters WP, Plowright AT, Sieroka N, Listgarten J, Goodnow RA, Fisher J, Jansen JM, Duca JS, Rush TS, Zentgraf M, Hill JE, Krutoholow E, Kohler M, Blaney J, Funatsu K, Luebkemann C, Schneider G. Rethinking drug design in the artificial intelligence era. Nat Rev Drug Discov 2019. [DOI: 78495111110.1038/s41573-019-0050-3' target='_blank'>'"<>78495111110.1038/s41573-019-0050-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [78495111110.1038/s41573-019-0050-3','', '10.1073/pnas.1806905115')">Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 09/29/2022]
78495111110.1038/s41573-019-0050-3" />
|
275
|
Rethinking drug design in the artificial intelligence era. Nat Rev Drug Discov 2019; 19:353-364. [DOI: 10.1038/s41573-019-0050-3] [Citation(s) in RCA: 222] [Impact Index Per Article: 44.4] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 10/28/2019] [Indexed: 12/17/2022]
|
276
|
Murphy A, Liszewski B. Artificial Intelligence and the Medical Radiation Profession: How Our Advocacy Must Inform Future Practice. J Med Imaging Radiat Sci 2019; 50:S15-S19. [DOI: 10.1016/j.jmir.2019.09.001] [Citation(s) in RCA: 13] [Impact Index Per Article: 2.6] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/10/2019] [Revised: 08/28/2019] [Accepted: 09/03/2019] [Indexed: 02/06/2023]
|
277
|
Hirano T, Nishide M, Nonaka N, Seita J, Ebina K, Sakurada K, Kumanogoh A. Development and validation of a deep-learning model for scoring of radiographic finger joint destruction in rheumatoid arthritis. Rheumatol Adv Pract 2019; 3:rkz047. [PMID: 31872173 PMCID: PMC6921374 DOI: 10.1093/rap/rkz047] [Citation(s) in RCA: 28] [Impact Index Per Article: 5.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/03/2019] [Revised: 11/04/2019] [Indexed: 12/15/2022] Open
Abstract
Objective The purpose of this research was to develop a deep-learning model to assess radiographic finger joint destruction in RA. Methods The model comprises two steps: a joint-detection step and a joint-evaluation step. Among 216 radiographs of 108 patients with RA, 186 radiographs were assigned to the training/validation dataset and 30 to the test dataset. In the training/validation dataset, images of PIP joints, the IP joint of the thumb or MCP joints were manually clipped and scored for joint space narrowing (JSN) and bone erosion by clinicians, and then these images were augmented. As a result, 11 160 images were used to train and validate a deep convolutional neural network for joint evaluation. Three thousand seven hundred and twenty selected images were used to train machine learning for joint detection. These steps were combined as the assessment model for radiographic finger joint destruction. Performance of the model was examined using the test dataset, which was not included in the training/validation process, by comparing the scores assigned by the model and clinicians. Results The model detected PIP joints, the IP joint of the thumb and MCP joints with a sensitivity of 95.3% and assigned scores for JSN and erosion. Accuracy (percentage of exact agreement) reached 49.3–65.4% for JSN and 70.6–74.1% for erosion. The correlation coefficient between scores by the model and clinicians per image was 0.72–0.88 for JSN and 0.54–0.75 for erosion. Conclusion Image processing with the trained convolutional neural network model is promising to assess radiographs in RA.
Collapse
Affiliation(s)
- Toru Hirano
- Department of Respiratory Medicine and Clinical Immunology, Internal Medicine, Graduate School of Medicine, Osaka University, Suita, Osaka
| | - Masayuki Nishide
- Department of Respiratory Medicine and Clinical Immunology, Internal Medicine, Graduate School of Medicine, Osaka University, Suita, Osaka
| | - Naoki Nonaka
- Medical Sciences Innovation Hub Program, RIKEN, Yokohama, Kanagawa
| | - Jun Seita
- Medical Sciences Innovation Hub Program, RIKEN, Yokohama, Kanagawa
| | - Kosuke Ebina
- Department of Musculoskeletal Regenerative Medicine, Graduate School of Medicine, Osaka University, Suita, Osaka, Japan
| | | | - Atsushi Kumanogoh
- Department of Respiratory Medicine and Clinical Immunology, Internal Medicine, Graduate School of Medicine, Osaka University, Suita, Osaka
| |
Collapse
|
278
|
What Are the Applications and Limitations of Artificial Intelligence for Fracture Detection and Classification in Orthopaedic Trauma Imaging? A Systematic Review. Clin Orthop Relat Res 2019; 477:2482-2491. [PMID: 31283727 PMCID: PMC6903838 DOI: 10.1097/corr.0000000000000848] [Citation(s) in RCA: 91] [Impact Index Per Article: 18.2] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 02/07/2023]
Abstract
BACKGROUND Artificial-intelligence algorithms derive rules and patterns from large amounts of data to calculate the probabilities of various outcomes using new sets of similar data. In medicine, artificial intelligence (AI) has been applied primarily to image-recognition diagnostic tasks and evaluating the probabilities of particular outcomes after treatment. However, the performance and limitations of AI in the automated detection and classification of fractures has not been examined comprehensively. QUESTION/PURPOSES In this systematic review, we asked (1) What is the proportion of correctly detected or classified fractures and the area under the receiving operating characteristic (AUC) curve of AI fracture detection and classification models? (2) What is the performance of AI in this setting compared with the performance of human examiners? METHODS The PubMed, Embase, and Cochrane databases were systematically searched from the start of each respective database until September 6, 2018, using terms related to "fracture", "artificial intelligence", and "detection, prediction, or evaluation." Of 1221 identified studies, we retained 10 studies: eight studies involved fracture detection (ankle, hand, hip, spine, wrist, and ulna), one addressed fracture classification (diaphyseal femur), and one addressed both fracture detection and classification (proximal humerus). We registered the review before data collection (PROSPERO: CRD42018110167) and used the Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA). We reported the range of the accuracy and AUC for the performance of the predicted fracture detection and/or classification task. An AUC of 1.0 would indicate perfect prediction, whereas 0.5 would indicate a prediction is no better than a flip-of-a-coin. We conducted quality assessment using a seven-item checklist based on a modified methodologic index for nonrandomized studies instrument (MINORS). RESULTS For fracture detection, the AUC in five studies reflected near perfect prediction (range, 0.95-1.0), and the accuracy in seven studies ranged from 83% to 98%. For fracture classification, the AUC was 0.94 in one study, and the accuracy in two studies ranged from 77% to 90%. In two studies AI outperformed human examiners for detecting and classifying hip and proximal humerus fractures, and one study showed equivalent performance for detecting wrist, hand and ankle fractures. CONCLUSIONS Preliminary experience with fracture detection and classification using AI shows promising performance. AI may enhance processing and communicating probabilistic tasks in medicine, including orthopaedic surgery. At present, inadequate reference standard assignments to train and test AI is the biggest hurdle before integration into clinical workflow. The next step will be to apply AI to more challenging diagnostic and therapeutic scenarios when there is absence of certitude. Future studies should also seek to address legal regulation and better determine feasibility of implementation in clinical practice. LEVEL OF EVIDENCE Level II, diagnostic study.
Collapse
|
279
|
Kelly CJ, Karthikesalingam A, Suleyman M, Corrado G, King D. Key challenges for delivering clinical impact with artificial intelligence. BMC Med 2019; 17:195. [PMID: 31665002 PMCID: PMC6821018 DOI: 10.1186/s12916-019-1426-2] [Citation(s) in RCA: 709] [Impact Index Per Article: 141.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 05/31/2019] [Accepted: 09/16/2019] [Indexed: 02/07/2023] Open
Abstract
BACKGROUND Artificial intelligence (AI) research in healthcare is accelerating rapidly, with potential applications being demonstrated across various domains of medicine. However, there are currently limited examples of such techniques being successfully deployed into clinical practice. This article explores the main challenges and limitations of AI in healthcare, and considers the steps required to translate these potentially transformative technologies from research to clinical practice. MAIN BODY Key challenges for the translation of AI systems in healthcare include those intrinsic to the science of machine learning, logistical difficulties in implementation, and consideration of the barriers to adoption as well as of the necessary sociocultural or pathway changes. Robust peer-reviewed clinical evaluation as part of randomised controlled trials should be viewed as the gold standard for evidence generation, but conducting these in practice may not always be appropriate or feasible. Performance metrics should aim to capture real clinical applicability and be understandable to intended users. Regulation that balances the pace of innovation with the potential for harm, alongside thoughtful post-market surveillance, is required to ensure that patients are not exposed to dangerous interventions nor deprived of access to beneficial innovations. Mechanisms to enable direct comparisons of AI systems must be developed, including the use of independent, local and representative test sets. Developers of AI algorithms must be vigilant to potential dangers, including dataset shift, accidental fitting of confounders, unintended discriminatory bias, the challenges of generalisation to new populations, and the unintended negative consequences of new algorithms on health outcomes. CONCLUSION The safe and timely translation of AI research into clinically validated and appropriately regulated systems that can benefit everyone is challenging. Robust clinical evaluation, using metrics that are intuitive to clinicians and ideally go beyond measures of technical accuracy to include quality of care and patient outcomes, is essential. Further work is required (1) to identify themes of algorithmic bias and unfairness while developing mitigations to address these, (2) to reduce brittleness and improve generalisability, and (3) to develop methods for improved interpretability of machine learning predictions. If these goals can be achieved, the benefits for patients are likely to be transformational.
Collapse
|
280
|
Zhao S, Xie B, Li Y, Zhao X, Kuang Y, Su J, He X, Wu X, Fan W, Huang K, Su J, Peng Y, Navarini AA, Huang W, Chen X. Smart identification of psoriasis by images using convolutional neural networks: a case study in China. J Eur Acad Dermatol Venereol 2019; 34:518-524. [PMID: 31541556 DOI: 10.1111/jdv.15965] [Citation(s) in RCA: 17] [Impact Index Per Article: 3.4] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/11/2019] [Accepted: 08/08/2019] [Indexed: 01/16/2023]
Abstract
BACKGROUND Psoriasis is a chronic inflammatory skin disease, which holds a high incidence in China. However, professional dermatologists who can diagnose psoriasis early and correctly are insufficient in China, especially in the rural areas. A smart approach to identify psoriasis by pictures would be highly adaptable countrywide and could play a useful role in early diagnosis and regular treatment of psoriasis. OBJECTIVES Design and evaluation of a smart psoriasis identification system based on clinical images (without relying on a dermatoscope) that works effectively similar to a dermatologist. METHODS A set of deep learning models using convolutional neural networks (CNNs) was explored and compared in the system for automatic identification of psoriasis. The work was carried out on a standardized dermatological dataset with 8021 clinical images of 9 common disorders including psoriasis along with full electronic medical records of patients built over the last 9 years in China. A two-stage deep neural network was designed and developed to identify psoriasis. In the first stage, a multilabel classifier was trained to learn the visual patterns for each individual skin disease. In the second stage, the output of the first stage was utilized to distinguish psoriasis from other skin diseases. RESULTS The area under the curve (AUC) of the two-stage model reached 0.981 ± 0.015, which outperforms a single-stage model. And, the classifier showed superior performance (missed diagnosis rate: 0.03, misdiagnosis rate: 0.04) than 25 Chinese dermatologists (missed diagnosis rate: 0.19, misdiagnosis rate: 0.10) in the diagnosis of psoriasis on 100 clinical images. CONCLUSIONS Using clinical images to identify psoriasis is feasible and effective based on CNNs, which also builds a solid technical base for smart care of skin diseases especially psoriasis using mobile/tablet applications for teledermatology in China.
Collapse
Affiliation(s)
- S Zhao
- Department of Dermatology, Xiangya Hospital, Central South University, Changsha, China.,Hunan Key Laboratory of Skin Cancer and Psoriasis, Changsha, China.,Hunan Engineering Research Center of Skin Health and Disease, Changsha, China
| | - B Xie
- School of Information Science and Engineering, Central South University, Changsha, China
| | - Y Li
- School of Information Science and Engineering, Central South University, Changsha, China
| | - X Zhao
- School of Information Science and Engineering, Central South University, Changsha, China
| | - Y Kuang
- Department of Dermatology, Xiangya Hospital, Central South University, Changsha, China.,Hunan Key Laboratory of Skin Cancer and Psoriasis, Changsha, China.,Hunan Engineering Research Center of Skin Health and Disease, Changsha, China
| | - J Su
- Department of Dermatology, Xiangya Hospital, Central South University, Changsha, China.,Hunan Key Laboratory of Skin Cancer and Psoriasis, Changsha, China.,Hunan Engineering Research Center of Skin Health and Disease, Changsha, China
| | - X He
- School of Information Science and Engineering, Central South University, Changsha, China
| | - X Wu
- Tencent Medical AI Lab, Beijing, China
| | - W Fan
- Tencent Medical AI Lab, Beijing, China
| | - K Huang
- Department of Dermatology, Xiangya Hospital, Central South University, Changsha, China.,Hunan Key Laboratory of Skin Cancer and Psoriasis, Changsha, China.,Hunan Engineering Research Center of Skin Health and Disease, Changsha, China
| | - J Su
- Faculty of Computer Science, University of Sunderland, Sunderland, UK
| | - Y Peng
- Faculty of Computer Science, University of Sunderland, Sunderland, UK
| | - A A Navarini
- Department of Dermatology, University Hospital of Basel, Basel, Switzerland.,Department of Biomedical Engineering, University Hospital of Basel, Basel, Switzerland
| | - W Huang
- Mobile Health Ministry of Education - China Mobile Joint Laboratory, Xiangya Hospital, Central South University, Changsha, China
| | - X Chen
- Department of Dermatology, Xiangya Hospital, Central South University, Changsha, China.,Hunan Key Laboratory of Skin Cancer and Psoriasis, Changsha, China.,Hunan Engineering Research Center of Skin Health and Disease, Changsha, China
| |
Collapse
|
281
|
Zhang Y, Ouyang M, Ray A, Liu T, Kong J, Bai B, Kim D, Guziak A, Luo Y, Feizi A, Tsai K, Duan Z, Liu X, Kim D, Cheung C, Yalcin S, Ceylan Koydemir H, Garner OB, Di Carlo D, Ozcan A. Computational cytometer based on magnetically modulated coherent imaging and deep learning. LIGHT, SCIENCE & APPLICATIONS 2019; 8:91. [PMID: 31645935 PMCID: PMC6804677 DOI: 10.1038/s41377-019-0203-5] [Citation(s) in RCA: 10] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/26/2019] [Revised: 09/05/2019] [Accepted: 09/12/2019] [Indexed: 05/08/2023]
Abstract
Detecting rare cells within blood has numerous applications in disease diagnostics. Existing rare cell detection techniques are typically hindered by their high cost and low throughput. Here, we present a computational cytometer based on magnetically modulated lensless speckle imaging, which introduces oscillatory motion to the magnetic-bead-conjugated rare cells of interest through a periodic magnetic force and uses lensless time-resolved holographic speckle imaging to rapidly detect the target cells in three dimensions (3D). In addition to using cell-specific antibodies to magnetically label target cells, detection specificity is further enhanced through a deep-learning-based classifier that is based on a densely connected pseudo-3D convolutional neural network (P3D CNN), which automatically detects rare cells of interest based on their spatio-temporal features under a controlled magnetic force. To demonstrate the performance of this technique, we built a high-throughput, compact and cost-effective prototype for detecting MCF7 cancer cells spiked in whole blood samples. Through serial dilution experiments, we quantified the limit of detection (LoD) as 10 cells per millilitre of whole blood, which could be further improved through multiplexing parallel imaging channels within the same instrument. This compact, cost-effective and high-throughput computational cytometer can potentially be used for rare cell detection and quantification in bodily fluids for a variety of biomedical applications.
Collapse
Affiliation(s)
- Yibo Zhang
- Electrical and Computer Engineering Department, University of California, Los Angeles, CA 90095 USA
- Department of Bioengineering, University of California, Los Angeles, CA 90095 USA
- California NanoSystems Institute, University of California, Los Angeles, CA 90095 USA
| | - Mengxing Ouyang
- Department of Bioengineering, University of California, Los Angeles, CA 90095 USA
| | - Aniruddha Ray
- Electrical and Computer Engineering Department, University of California, Los Angeles, CA 90095 USA
- Department of Bioengineering, University of California, Los Angeles, CA 90095 USA
- California NanoSystems Institute, University of California, Los Angeles, CA 90095 USA
- Department of Physics and Astronomy, University of Toledo, Toledo, OH 43606 USA
| | - Tairan Liu
- Electrical and Computer Engineering Department, University of California, Los Angeles, CA 90095 USA
- Department of Bioengineering, University of California, Los Angeles, CA 90095 USA
- California NanoSystems Institute, University of California, Los Angeles, CA 90095 USA
| | - Janay Kong
- Department of Bioengineering, University of California, Los Angeles, CA 90095 USA
| | - Bijie Bai
- Electrical and Computer Engineering Department, University of California, Los Angeles, CA 90095 USA
- Department of Bioengineering, University of California, Los Angeles, CA 90095 USA
- California NanoSystems Institute, University of California, Los Angeles, CA 90095 USA
| | - Donghyuk Kim
- Department of Bioengineering, University of California, Los Angeles, CA 90095 USA
| | - Alexander Guziak
- Department of Physics and Astronomy, University of California, Los Angeles, CA 90095 USA
| | - Yi Luo
- Electrical and Computer Engineering Department, University of California, Los Angeles, CA 90095 USA
- Department of Bioengineering, University of California, Los Angeles, CA 90095 USA
- California NanoSystems Institute, University of California, Los Angeles, CA 90095 USA
| | - Alborz Feizi
- Electrical and Computer Engineering Department, University of California, Los Angeles, CA 90095 USA
- Department of Bioengineering, University of California, Los Angeles, CA 90095 USA
- California NanoSystems Institute, University of California, Los Angeles, CA 90095 USA
- Yale School of Medicine, New Haven, CT 06510 USA
| | - Katherine Tsai
- Department of Biochemistry, University of California, Los Angeles, CA 90095 USA
| | - Zhuoran Duan
- Electrical and Computer Engineering Department, University of California, Los Angeles, CA 90095 USA
| | - Xuewei Liu
- Electrical and Computer Engineering Department, University of California, Los Angeles, CA 90095 USA
| | - Danny Kim
- Department of Bioengineering, University of California, Los Angeles, CA 90095 USA
| | - Chloe Cheung
- Department of Bioengineering, University of California, Los Angeles, CA 90095 USA
| | - Sener Yalcin
- Electrical and Computer Engineering Department, University of California, Los Angeles, CA 90095 USA
| | - Hatice Ceylan Koydemir
- Electrical and Computer Engineering Department, University of California, Los Angeles, CA 90095 USA
- Department of Bioengineering, University of California, Los Angeles, CA 90095 USA
- California NanoSystems Institute, University of California, Los Angeles, CA 90095 USA
| | - Omai B. Garner
- Department of Pathology and Laboratory Medicine, University of California, Los Angeles, CA 90095 USA
| | - Dino Di Carlo
- Department of Bioengineering, University of California, Los Angeles, CA 90095 USA
- California NanoSystems Institute, University of California, Los Angeles, CA 90095 USA
- Department of Mechanical and Aerospace Engineering, University of California, Los Angeles, CA 90095 USA
- Jonsson Comprehensive Cancer Center, University of California, Los Angeles, CA 90095 USA
| | - Aydogan Ozcan
- Electrical and Computer Engineering Department, University of California, Los Angeles, CA 90095 USA
- Department of Bioengineering, University of California, Los Angeles, CA 90095 USA
- California NanoSystems Institute, University of California, Los Angeles, CA 90095 USA
- Department of Surgery, David Geffen School of Medicine, University of California, Los Angeles, CA 90095 USA
| |
Collapse
|
282
|
Liu X, Faes L, Kale AU, Wagner SK, Fu DJ, Bruynseels A, Mahendiran T, Moraes G, Shamdas M, Kern C, Ledsam JR, Schmid MK, Balaskas K, Topol EJ, Bachmann LM, Keane PA, Denniston AK. A comparison of deep learning performance against health-care professionals in detecting diseases from medical imaging: a systematic review and meta-analysis. Lancet Digit Health 2019; 1:e271-e297. [PMID: 33323251 DOI: 10.1016/s2589-7500(19)30123-2] [Citation(s) in RCA: 687] [Impact Index Per Article: 137.4] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/05/2019] [Revised: 08/06/2019] [Accepted: 08/14/2019] [Indexed: 02/06/2023]
Abstract
BACKGROUND Deep learning offers considerable promise for medical diagnostics. We aimed to evaluate the diagnostic accuracy of deep learning algorithms versus health-care professionals in classifying diseases using medical imaging. METHODS In this systematic review and meta-analysis, we searched Ovid-MEDLINE, Embase, Science Citation Index, and Conference Proceedings Citation Index for studies published from Jan 1, 2012, to June 6, 2019. Studies comparing the diagnostic performance of deep learning models and health-care professionals based on medical imaging, for any disease, were included. We excluded studies that used medical waveform data graphics material or investigated the accuracy of image segmentation rather than disease classification. We extracted binary diagnostic accuracy data and constructed contingency tables to derive the outcomes of interest: sensitivity and specificity. Studies undertaking an out-of-sample external validation were included in a meta-analysis, using a unified hierarchical model. This study is registered with PROSPERO, CRD42018091176. FINDINGS Our search identified 31 587 studies, of which 82 (describing 147 patient cohorts) were included. 69 studies provided enough data to construct contingency tables, enabling calculation of test accuracy, with sensitivity ranging from 9·7% to 100·0% (mean 79·1%, SD 0·2) and specificity ranging from 38·9% to 100·0% (mean 88·3%, SD 0·1). An out-of-sample external validation was done in 25 studies, of which 14 made the comparison between deep learning models and health-care professionals in the same sample. Comparison of the performance between health-care professionals in these 14 studies, when restricting the analysis to the contingency table for each study reporting the highest accuracy, found a pooled sensitivity of 87·0% (95% CI 83·0-90·2) for deep learning models and 86·4% (79·9-91·0) for health-care professionals, and a pooled specificity of 92·5% (95% CI 85·1-96·4) for deep learning models and 90·5% (80·6-95·7) for health-care professionals. INTERPRETATION Our review found the diagnostic performance of deep learning models to be equivalent to that of health-care professionals. However, a major finding of the review is that few studies presented externally validated results or compared the performance of deep learning models and health-care professionals using the same sample. Additionally, poor reporting is prevalent in deep learning studies, which limits reliable interpretation of the reported diagnostic accuracy. New reporting standards that address specific challenges of deep learning could improve future studies, enabling greater confidence in the results of future evaluations of this promising technology. FUNDING None.
Collapse
Affiliation(s)
- Xiaoxuan Liu
- Department of Ophthalmology, University Hospitals Birmingham NHS Foundation Trust, Birmingham, UK; Academic Unit of Ophthalmology, Institute of Inflammation & Ageing, College of Medical and Dental Sciences, University of Birmingham, Birmingham, UK; Medical Retina Department, Moorfields Eye Hospital NHS Foundation Trust, London, UK; Health Data Research UK, London, UK
| | - Livia Faes
- Medical Retina Department, Moorfields Eye Hospital NHS Foundation Trust, London, UK; Eye Clinic, Cantonal Hospital of Lucerne, Lucerne, Switzerland
| | - Aditya U Kale
- Department of Ophthalmology, University Hospitals Birmingham NHS Foundation Trust, Birmingham, UK
| | - Siegfried K Wagner
- NIHR Biomedical Research Centre for Ophthalmology, Moorfields Eye Hospital NHS Foundation Trust and UCL Institute of Ophthalmology, London, UK
| | - Dun Jack Fu
- Medical Retina Department, Moorfields Eye Hospital NHS Foundation Trust, London, UK
| | - Alice Bruynseels
- Department of Ophthalmology, University Hospitals Birmingham NHS Foundation Trust, Birmingham, UK
| | - Thushika Mahendiran
- Department of Ophthalmology, University Hospitals Birmingham NHS Foundation Trust, Birmingham, UK
| | - Gabriella Moraes
- Medical Retina Department, Moorfields Eye Hospital NHS Foundation Trust, London, UK
| | - Mohith Shamdas
- Academic Unit of Ophthalmology, Institute of Inflammation & Ageing, College of Medical and Dental Sciences, University of Birmingham, Birmingham, UK
| | - Christoph Kern
- Medical Retina Department, Moorfields Eye Hospital NHS Foundation Trust, London, UK; University Eye Hospital, Ludwig Maximilian University of Munich, Munich, Germany
| | | | - Martin K Schmid
- Eye Clinic, Cantonal Hospital of Lucerne, Lucerne, Switzerland
| | - Konstantinos Balaskas
- Medical Retina Department, Moorfields Eye Hospital NHS Foundation Trust, London, UK; NIHR Biomedical Research Centre for Ophthalmology, Moorfields Eye Hospital NHS Foundation Trust and UCL Institute of Ophthalmology, London, UK
| | - Eric J Topol
- Scripps Research Translational Institute, La Jolla, California
| | | | - Pearse A Keane
- NIHR Biomedical Research Centre for Ophthalmology, Moorfields Eye Hospital NHS Foundation Trust and UCL Institute of Ophthalmology, London, UK; Health Data Research UK, London, UK
| | - Alastair K Denniston
- Department of Ophthalmology, University Hospitals Birmingham NHS Foundation Trust, Birmingham, UK; Academic Unit of Ophthalmology, Institute of Inflammation & Ageing, College of Medical and Dental Sciences, University of Birmingham, Birmingham, UK; Centre for Patient Reported Outcome Research, Institute of Applied Health Research, University of Birmingham, Birmingham, UK; NIHR Biomedical Research Centre for Ophthalmology, Moorfields Eye Hospital NHS Foundation Trust and UCL Institute of Ophthalmology, London, UK; Health Data Research UK, London, UK.
| |
Collapse
|
283
|
Karnuta JM, Navarro SM, Haeberle HS, Helm JM, Kamath AF, Schaffer JL, Krebs VE, Ramkumar PN. Predicting Inpatient Payments Prior to Lower Extremity Arthroplasty Using Deep Learning: Which Model Architecture Is Best? J Arthroplasty 2019; 34:2235-2241.e1. [PMID: 31230954 DOI: 10.1016/j.arth.2019.05.048] [Citation(s) in RCA: 22] [Impact Index Per Article: 4.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 03/28/2019] [Revised: 05/24/2019] [Accepted: 05/28/2019] [Indexed: 02/01/2023] Open
Abstract
BACKGROUND Recent advances in machine learning have given rise to deep learning, which uses hierarchical layers to build models, offering the ability to advance value-based healthcare by better predicting patient outcomes and costs of a given treatment. The purpose of this study is to compare the performance of 2 common deep learning models, traditional multilayer perceptron (MLP), and the newer dense neural network (DenseNet), in predicting outcomes for primary total hip arthroplasty (THA) and total knee arthroplasty (TKA) as a foundation for future musculoskeletal studies seeking to utilize machine learning. METHODS Using 295,605 patients undergoing primary THA and TKA from a New York State inpatient administrative database from 2009 to 2016, 2 neural network designs (MLP vs DenseNet) with different model regularization techniques (dropout, batch normalization, and DeCovLoss) were applied to compare model performance on predicting inpatient procedural cost using the area under the receiver operating characteristic curve (AUC). Models were implemented to identify high-cost surgical cases. RESULTS DenseNet performed similarly to or better than MLP across the different regularization techniques in predicting procedural costs of THA and TKA. Applying regularization to DenseNet resulted in a significantly higher AUC as compared to DenseNet alone (0.813 vs 0.792, P = .011). When regularization methods were applied to MLP, the AUC was significantly lower than without regularization (0.621 vs 0.791, P = 1.1 × 10-15). When the optimal MLP and DenseNet models were compared in a head-to-head fashion, they performed similarly at cost prediction (P > .999). CONCLUSION This study establishes that in predicting costs of lower extremity arthroplasty, DenseNet models improve in performance with regularization, whereas simple neural network models perform significantly worse without regularization. In light of the resource-intensive nature of creating and testing deep learning models for orthopedic surgery, particularly for value-centric procedures such as arthroplasty, this study establishes a set of key technical features that resulted in better prediction of inpatient surgical costs. We demonstrated that regularization is critically important for neural networks in arthroplasty cost prediction and that future studies should utilize these deep learning techniques to predict arthroplasty costs. LEVEL OF EVIDENCE III.
Collapse
Affiliation(s)
- Jaret M Karnuta
- Machine Learning Arthroplasty Laboratory, Cleveland Clinic, Cleveland, OH
| | | | - Heather S Haeberle
- Department of Orthopedic Surgery, Baylor College of Medicine, Houston, TX
| | - J Matthew Helm
- Department of Orthopaedic Surgery, Texas Tech University Health Sciences Center, Lubbock, TX
| | - Atul F Kamath
- Machine Learning Arthroplasty Laboratory, Cleveland Clinic, Cleveland, OH
| | | | - Viktor E Krebs
- Machine Learning Arthroplasty Laboratory, Cleveland Clinic, Cleveland, OH
| | - Prem N Ramkumar
- Machine Learning Arthroplasty Laboratory, Cleveland Clinic, Cleveland, OH
| |
Collapse
|
284
|
Peiffer-Smadja N, Rawson TM, Ahmad R, Buchard A, Georgiou P, Lescure FX, Birgand G, Holmes AH. Machine learning for clinical decision support in infectious diseases: a narrative review of current applications. Clin Microbiol Infect 2019; 26:584-595. [PMID: 31539636 DOI: 10.1016/j.cmi.2019.09.009] [Citation(s) in RCA: 178] [Impact Index Per Article: 35.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/27/2019] [Revised: 08/29/2019] [Accepted: 09/09/2019] [Indexed: 12/22/2022]
Abstract
BACKGROUND Machine learning (ML) is a growing field in medicine. This narrative review describes the current body of literature on ML for clinical decision support in infectious diseases (ID). OBJECTIVES We aim to inform clinicians about the use of ML for diagnosis, classification, outcome prediction and antimicrobial management in ID. SOURCES References for this review were identified through searches of MEDLINE/PubMed, EMBASE, Google Scholar, biorXiv, ACM Digital Library, arXiV and IEEE Xplore Digital Library up to July 2019. CONTENT We found 60 unique ML-clinical decision support systems (ML-CDSS) aiming to assist ID clinicians. Overall, 37 (62%) focused on bacterial infections, 10 (17%) on viral infections, nine (15%) on tuberculosis and four (7%) on any kind of infection. Among them, 20 (33%) addressed the diagnosis of infection, 18 (30%) the prediction, early detection or stratification of sepsis, 13 (22%) the prediction of treatment response, four (7%) the prediction of antibiotic resistance, three (5%) the choice of antibiotic regimen and two (3%) the choice of a combination antiretroviral therapy. The ML-CDSS were developed for intensive care units (n = 24, 40%), ID consultation (n = 15, 25%), medical or surgical wards (n = 13, 20%), emergency department (n = 4, 7%), primary care (n = 3, 5%) and antimicrobial stewardship (n = 1, 2%). Fifty-three ML-CDSS (88%) were developed using data from high-income countries and seven (12%) with data from low- and middle-income countries (LMIC). The evaluation of ML-CDSS was limited to measures of performance (e.g. sensitivity, specificity) for 57 ML-CDSS (95%) and included data in clinical practice for three (5%). IMPLICATIONS Considering comprehensive patient data from socioeconomically diverse healthcare settings, including primary care and LMICs, may improve the ability of ML-CDSS to suggest decisions adapted to various clinical contexts. Currents gaps identified in the evaluation of ML-CDSS must also be addressed in order to know the potential impact of such tools for clinicians and patients.
Collapse
Affiliation(s)
- N Peiffer-Smadja
- National Institute for Health Research Health Protection Research Unit in Healthcare Associated Infections and Antimicrobial Resistance, Imperial College London, London, UK; French Institute for Medical Research (Inserm), Infection Antimicrobials Modelling Evolution (IAME), UMR 1137, University Paris Diderot, Paris, France.
| | - T M Rawson
- National Institute for Health Research Health Protection Research Unit in Healthcare Associated Infections and Antimicrobial Resistance, Imperial College London, London, UK
| | - R Ahmad
- National Institute for Health Research Health Protection Research Unit in Healthcare Associated Infections and Antimicrobial Resistance, Imperial College London, London, UK
| | | | - P Georgiou
- Department of Electrical and Electronic Engineering, Imperial College, London, UK
| | - F-X Lescure
- French Institute for Medical Research (Inserm), Infection Antimicrobials Modelling Evolution (IAME), UMR 1137, University Paris Diderot, Paris, France; Infectious Diseases Department, Bichat-Claude Bernard Hospital, Assistance-Publique Hôpitaux de Paris, Paris, France
| | - G Birgand
- National Institute for Health Research Health Protection Research Unit in Healthcare Associated Infections and Antimicrobial Resistance, Imperial College London, London, UK
| | - A H Holmes
- National Institute for Health Research Health Protection Research Unit in Healthcare Associated Infections and Antimicrobial Resistance, Imperial College London, London, UK
| |
Collapse
|
285
|
Niel O, Bastard P. Artificial Intelligence in Nephrology: Core Concepts, Clinical Applications, and Perspectives. Am J Kidney Dis 2019; 74:803-810. [PMID: 31451330 DOI: 10.1053/j.ajkd.2019.05.020] [Citation(s) in RCA: 70] [Impact Index Per Article: 14.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/24/2018] [Accepted: 05/11/2019] [Indexed: 01/20/2023]
Abstract
Artificial intelligence is playing an increasingly important role in many fields of medicine, assisting physicians in most steps of patient management. In nephrology, artificial intelligence can already be used to improve clinical care, hemodialysis prescriptions, and follow-up of transplant recipients. However, many nephrologists are still unfamiliar with the basic principles of medical artificial intelligence. This review seeks to provide an overview of medical artificial intelligence relevant to the practicing nephrologist, in all fields of nephrology. We define the core concepts of artificial intelligence and machine learning and cover the basics of the functioning of neural networks and deep learning. We also discuss the most recent clinical applications of artificial intelligence in nephrology and medicine; as an example, we describe how artificial intelligence can predict the occurrence of progressive immunoglobulin A nephropathy. Finally, we consider the future of artificial intelligence in clinical nephrology and its impact on medical practice, and conclude with a discussion of the ethical issues that the use of artificial intelligence raises in terms of clinical decision making, physician-patient relationship, patient privacy, and data collection.
Collapse
Affiliation(s)
- Olivier Niel
- Pediatric Nephrology Department, Robert Debré Hospital, Paris, France.
| | - Paul Bastard
- Pediatric Nephrology Department, Robert Debré Hospital, Paris, France
| |
Collapse
|
286
|
Dzobo K, Adotey S, Thomford NE, Dzobo W. Integrating Artificial and Human Intelligence: A Partnership for Responsible Innovation in Biomedical Engineering and Medicine. OMICS-A JOURNAL OF INTEGRATIVE BIOLOGY 2019; 24:247-263. [PMID: 31313972 DOI: 10.1089/omi.2019.0038] [Citation(s) in RCA: 34] [Impact Index Per Article: 6.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 02/06/2023]
Abstract
Historically, the term "artificial intelligence" dates to 1956 when it was first used in a conference at Dartmouth College in the US. Since then, the development of artificial intelligence has in part been shaped by the field of neuroscience. By understanding the human brain, scientists have attempted to build new intelligent machines capable of performing complex tasks akin to humans. Indeed, future research into artificial intelligence will continue to benefit from the study of the human brain. While the development of artificial intelligence algorithms has been fast paced, the actual use of most artificial intelligence (AI) algorithms in biomedical engineering and clinical practice is still markedly below its conceivably broader potentials. This is partly because for any algorithm to be incorporated into existing workflows it has to stand the test of scientific validation, clinical and personal utility, application context, and is equitable as well. In this context, there is much to be gained by combining AI and human intelligence (HI). Harnessing Big Data, computing power and storage capacities, and addressing societal issues emergent from algorithm applications, demand deploying HI in tandem with AI. Very few countries, even economically developed states, lack adequate and critical governance frames to best understand and steer the AI innovation trajectories in health care. Drug discovery and translational pharmaceutical research stand to gain from AI technology provided they are also informed by HI. In this expert review, we analyze the ways in which AI applications are likely to traverse the continuum of life from birth to death, and encompassing not only humans but also all animal, plant, and other living organisms that are increasingly touched by AI. Examples of AI applications include digital health, diagnosis of diseases in newborns, remote monitoring of health by smart devices, real-time Big Data analytics for prompt diagnosis of heart attacks, and facial analysis software with consequences on civil liberties. While we underscore the need for integration of AI and HI, we note that AI technology does not have to replace medical specialists or scientists and rather, is in need of such expert HI. Altogether, AI and HI offer synergy for responsible innovation and veritable prospects for improving health care from prevention to diagnosis to therapeutics while unintended consequences of automation emergent from AI and algorithms should be borne in mind on scientific cultures, work force, and society at large.
Collapse
Affiliation(s)
- Kevin Dzobo
- International Centre for Genetic Engineering and Biotechnology (ICGEB), Cape Town Component, Wernher and Beit Building (South), UCT Medical Campus, Anzio Road, Observatory 7925, Cape Town, South Africa.,Division of Medical Biochemistry and Institute of Infectious Disease and Molecular Medicine, Department of Integrative Biomedical Sciences, Faculty of Health Sciences, University of Cape Town, Cape Town, South Africa
| | - Sampson Adotey
- International Development Innovation Network, D-Lab, Massachusetts Institute of Technology, Cambridge, Massachusetts, USA
| | - Nicholas E Thomford
- Pharmacogenetics Research Group, Division of Human Genetics, Department of Pathology and Institute of Infectious Diseases and Molecular Medicine, Faculty of Health Sciences, University of Cape Town, Observatory 7925, Cape Town, South Africa
| | - Witness Dzobo
- Pathology and Immunology Department, University Hospital Southampton, Mail Point B, Tremona Road, Southampton, UK.,University of Portsmouth, Faculty of Science, St Michael's Building, White Swan Road, Portsmouth, UK
| |
Collapse
|
287
|
Dewey M, Wilkens U. The Bionic Radiologist: avoiding blurry pictures and providing greater insights. NPJ Digit Med 2019; 2:65. [PMID: 31388567 PMCID: PMC6616477 DOI: 10.1038/s41746-019-0142-9] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.4] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/29/2019] [Accepted: 05/28/2019] [Indexed: 12/11/2022] Open
Abstract
Radiology images and reports have long been digitalized. However, the potential of the more than 3.6 billion radiology examinations performed annually worldwide has largely gone unused in the effort to digitally transform health care. The Bionic Radiologist is a concept that combines humanity and digitalization for better health care integration of radiology. At a practical level, this concept will achieve critical goals: (1) testing decisions being made scientifically on the basis of disease probabilities and patient preferences; (2) image analysis done consistently at any time and at any site; and (3) treatment suggestions that are closely linked to imaging results and are seamlessly integrated with other information. The Bionic Radiologist will thus help avoiding missed care opportunities, will provide continuous learning in the work process, and will also allow more time for radiologists' primary roles: interacting with patients and referring physicians. To achieve that potential, one has to cope with many implementation barriers at both the individual and institutional levels. These include: reluctance to delegate decision making, a possible decrease in image interpretation knowledge and the perception that patient safety and trust are at stake. To facilitate implementation of the Bionic Radiologist the following will be helpful: uncertainty quantifications for suggestions, shared decision making, changes in organizational culture and leadership style, maintained expertise through continuous learning systems for training, and role development of the involved experts. With the support of the Bionic Radiologist, disparities are reduced and the delivery of care is provided in a humane and personalized fashion.
Collapse
Affiliation(s)
- Marc Dewey
- Charité—Universitätsmedizin Berlin and Berlin Institute of Health, Berlin, Germany
| | - Uta Wilkens
- Ruhr-University Bochum, Institute of Work Science, Bochum, Germany
| |
Collapse
|
288
|
Esteban-Medina M, Peña-Chilet M, Loucera C, Dopazo J. Exploring the druggable space around the Fanconi anemia pathway using machine learning and mechanistic models. BMC Bioinformatics 2019; 20:370. [PMID: 31266445 PMCID: PMC6604281 DOI: 10.1186/s12859-019-2969-0] [Citation(s) in RCA: 19] [Impact Index Per Article: 3.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/06/2019] [Accepted: 06/25/2019] [Indexed: 02/07/2023] Open
Abstract
BACKGROUND In spite of the abundance of genomic data, predictive models that describe phenotypes as a function of gene expression or mutations are difficult to obtain because they are affected by the curse of dimensionality, given the disbalance between samples and candidate genes. And this is especially dramatic in scenarios in which the availability of samples is difficult, such as the case of rare diseases. RESULTS The application of multi-output regression machine learning methodologies to predict the potential effect of external proteins over the signaling circuits that trigger Fanconi anemia related cell functionalities, inferred with a mechanistic model, allowed us to detect over 20 potential therapeutic targets. CONCLUSIONS The use of artificial intelligence methods for the prediction of potentially causal relationships between proteins of interest and cell activities related with disease-related phenotypes opens promising avenues for the systematic search of new targets in rare diseases.
Collapse
Affiliation(s)
- Marina Esteban-Medina
- Clinical Bioinformatics Area. Fundación Progreso y Salud (FPS). CDCA, Hospital Virgen del Rocio, 41013 Sevilla, Spain
| | - María Peña-Chilet
- Clinical Bioinformatics Area. Fundación Progreso y Salud (FPS). CDCA, Hospital Virgen del Rocio, 41013 Sevilla, Spain
- Bioinformatics in Rare Diseases (BiER). Centro de Investigación Biomédica en Red de Enfermedades Raras (CIBERER), FPS, Hospital Virgen del Rocío, 41013 Sevilla, Spain
| | - Carlos Loucera
- Clinical Bioinformatics Area. Fundación Progreso y Salud (FPS). CDCA, Hospital Virgen del Rocio, 41013 Sevilla, Spain
| | - Joaquín Dopazo
- Clinical Bioinformatics Area. Fundación Progreso y Salud (FPS). CDCA, Hospital Virgen del Rocio, 41013 Sevilla, Spain
- Bioinformatics in Rare Diseases (BiER). Centro de Investigación Biomédica en Red de Enfermedades Raras (CIBERER), FPS, Hospital Virgen del Rocío, 41013 Sevilla, Spain
- INB-ELIXIR-es, FPS, Hospital Virgen del Rocío, 42013 Sevilla, Spain
| |
Collapse
|
289
|
Leopold SS. Editor's Spotlight/Take 5: Can Machine Learning Algorithms Predict Which Patients Will Achieve Minimally Clinically Important Differences From Total Joint Arthroplasty? Clin Orthop Relat Res 2019; 477:1262-1266. [PMID: 31094832 PMCID: PMC6554147 DOI: 10.1097/corr.0000000000000787] [Citation(s) in RCA: 9] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 01/31/2023]
|
290
|
Local-Entropy Based Approach for X-Ray Image Segmentation and Fracture Detection. ENTROPY 2019; 21:e21040338. [PMID: 33267052 PMCID: PMC7514822 DOI: 10.3390/e21040338] [Citation(s) in RCA: 15] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 02/23/2019] [Revised: 03/20/2019] [Accepted: 03/26/2019] [Indexed: 11/17/2022]
Abstract
The paper proposes a segmentation and classification technique for fracture detection in X-ray images. This novel rotation-invariant method introduces the concept of local entropy for de-noising and removing tissue from the analysed X-ray images, followed by an improved procedure for image segmentation and the detection of regions of interest. The proposed local Shannon entropy was calculated for each image pixel using a sliding 2D window. An initial image segmentation was performed on the entropy representation of the original image. Next, a graph theory-based technique was implemented for the purpose of removing false bone contours and improving the edge detection of long bones. Finally, the paper introduces a classification and localisation procedure for fracture detection by tracking the difference between the extracted contour and the estimation of an ideal healthy one. The proposed hybrid method excels at detecting small fractures (which are hard to detect visually by a radiologist) in the ulna and radius bones—common injuries in children. Therefore, it is imperative that a radiologist inspecting the X-ray image receives a warning from the computerised X-ray analysis system, in order to prevent false-negative diagnoses. The proposed method was applied to a data-set containing 860 X-ray images of child radius and ulna bones (642 fracture-free images and 218 images containing fractures). The obtained results showed the efficiency and robustness of the proposed approach, in terms of segmentation quality and classification accuracy and precision (up to 91.16% and 86.22%, respectively).
Collapse
|
291
|
A Human⁻Machine Interface Based on Eye Tracking for Controlling and Monitoring a Smart Home Using the Internet of Things. SENSORS 2019; 19:s19040859. [PMID: 30791414 PMCID: PMC6412435 DOI: 10.3390/s19040859] [Citation(s) in RCA: 20] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 01/02/2019] [Revised: 02/11/2019] [Accepted: 02/12/2019] [Indexed: 11/17/2022]
Abstract
People with severe disabilities may have difficulties when interacting with their home devices due to the limitations inherent to their disability. Simple home activities may even be impossible for this group of people. Although much work has been devoted to proposing new assistive technologies to improve the lives of people with disabilities, some studies have found that the abandonment of such technologies is quite high. This work presents a new assistive system based on eye tracking for controlling and monitoring a smart home, based on the Internet of Things, which was developed following concepts of user-centered design and usability. With this system, a person with severe disabilities was able to control everyday equipment in her residence, such as lamps, television, fan, and radio. In addition, her caregiver was able to monitor remotely, by Internet, her use of the system in real time. Additionally, the user interface developed here has some functionalities that allowed improving the usability of the system as a whole. The experiments were divided into two steps. In the first step, the assistive system was assembled in an actual home where tests were conducted with 29 participants without disabilities. In the second step, the system was tested with online monitoring for seven days by a person with severe disability (end-user), in her own home, not only to increase convenience and comfort, but also so that the system could be tested where it would in fact be used. At the end of both steps, all the participants answered the System Usability Scale (SUS) questionnaire, which showed that both the group of participants without disabilities and the person with severe disabilities evaluated the assistive system with mean scores of 89.9 and 92.5, respectively.
Collapse
|
292
|
Topol EJ. High-performance medicine: the convergence of human and artificial intelligence. Nat Med 2019; 25:44-56. [PMID: 30617339 DOI: 10.1038/s41591-018-0300-7] [Citation(s) in RCA: 2149] [Impact Index Per Article: 429.8] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/16/2018] [Accepted: 11/12/2018] [Indexed: 11/08/2022]
Abstract
The use of artificial intelligence, and the deep-learning subtype in particular, has been enabled by the use of labeled big data, along with markedly enhanced computing power and cloud storage, across all sectors. In medicine, this is beginning to have an impact at three levels: for clinicians, predominantly via rapid, accurate image interpretation; for health systems, by improving workflow and the potential for reducing medical errors; and for patients, by enabling them to process their own data to promote health. The current limitations, including bias, privacy and security, and lack of transparency, along with the future directions of these applications will be discussed in this article. Over time, marked improvements in accuracy, productivity, and workflow will likely be actualized, but whether that will be used to improve the patient-doctor relationship or facilitate its erosion remains to be seen.
Collapse
Affiliation(s)
- Eric J Topol
- Department of Molecular Medicine, Scripps Research, La Jolla, CA, USA.
| |
Collapse
|
293
|
Yang K, Zeng Z, Peng H, Jiang Y. Attitudes Of Chinese Cancer Patients Toward The Clinical Use Of Artificial Intelligence. Patient Prefer Adherence 2019; 13:1867-1875. [PMID: 31802856 PMCID: PMC6830378 DOI: 10.2147/ppa.s225952] [Citation(s) in RCA: 23] [Impact Index Per Article: 4.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 08/04/2019] [Accepted: 10/16/2019] [Indexed: 02/05/2023] Open
Abstract
PURPOSE Artificial intelligence (AI) plays a substantial role in many domains, including medical fields. However, we still lack evidence to support whether or not cancer patients will accept the clinical use of AI. This research aims to assess the attitudes of Chinese cancer patients toward the clinical use of artificial intelligence in medicine (AIM), and to analyze the possible influencing factors. PATIENTS AND METHODS A questionnaire was delivered to 527 participants. Targeted people were Chinese cancer patients who were informed of their cancer diagnosis. RESULTS The effective response rate was 76.3% (402/527). Most cancer patients trusted AIMs in both stages of diagnosis and treatment, and participants who had heard of AIMs were more likely to trust them in the diagnosis phase. When an AIM's diagnosis diverged from a human doctor' s, ethnic minorities, and those who had received traditional Chinese medicine (TCM), had never received chemotherapy, were more likely to choose "AIM", and when an AIM's therapeutic advice diverged from a human doctor's, male participants, and those who had received TCM or surgery, were more likely to choose "AIM". CONCLUSION Most Chinese cancer patients believed in the AIM to some extent. Nevertheless, most still thought that oncology physicians were more trustworthy when their opinions diverged. Participants' gender, race, treatment received, and AIM related knowledge might influence their attitudes toward the AIM. Most participants thought AIM would assist oncology physicians in the future, while little really believed that oncology physicians would completely be replaced.
Collapse
Affiliation(s)
- Keyi Yang
- Department of Head and Neck Oncology, Cancer Center, West China Hospital, Sichuan University, Chengdu, Sichuan, People’s Republic of China
| | - Zhi Zeng
- Department of Head and Neck Oncology, Cancer Center, West China Hospital, Sichuan University, Chengdu, Sichuan, People’s Republic of China
| | - Hu Peng
- Department of Head and Neck Oncology, Cancer Center, West China Hospital, Sichuan University, Chengdu, Sichuan, People’s Republic of China
| | - Yu Jiang
- Department of Head and Neck Oncology, Cancer Center, West China Hospital, Sichuan University, Chengdu, Sichuan, People’s Republic of China
- Correspondence: Yu Jiang Department of Head and Neck Oncology, Cancer Center, West China Hospital, Sichuan University, No. 37, Guo Xue Lane, Chengdu, Sichuan610041, People’s Republic of ChinaTel/fax +86 28 85423278 Email
| |
Collapse
|