1
|
Siew B, Enax J, Meyer F. Case Report on Caries Assessment Using Intraoral Scanner Compared with Bitewing Radiographs. Eur J Dent 2024. [PMID: 38698612 DOI: 10.1055/s-0044-1782192] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 05/05/2024] Open
Abstract
Dental caries remains one of the main reasons patients seeing their dentist. They either show up for preventive measures, diagnostics, or treatment of present caries lesions. To date, diagnostics are performed visually, supported by using bitewing radiographs. While radiographic diagnostics should only be performed on a biannual basis, and some caries process will not be seen with visual diagnostics, there remains a lack in regular checkups. Therefore, different technical applications can be used for regular diagnostics. One of those is the near-infrared imaging (NIRI) technology. In this case report, a patient presented with incipient caries lesions. These lesions were diagnosed visually, radiographically, and using NIRI. After diagnosis of incipient caries lesions, the patient was advised to use a hydroxyapatite toothpaste and a hydroxyapatite gel for the remineralization of the lesions and prevention of caries progression. The patient was followed up for 6 months with regular checkups in between. Visual diagnostics and NIRI were used to detecting the caries progress. After 6 months, bitewing radiographs were used in addition. In this clinical investigation we were able to show for the first time that NIRI and bitewing radiographs are able to detect and follow incipient caries lesions. Additionally, this study highlights that hydroxyapatite-containing oral care products are able to arrest and remineralize the caries process.
Collapse
Affiliation(s)
- Bernard Siew
- Smilefocus, Camden Medical Centre, Singapore, Singapore
| | - Joachim Enax
- Research Department, Dr. Kurt Wolff GmbH & Co. KG, Bielefeld, Germany
| | - Frederic Meyer
- Research Department, Dr. Kurt Wolff GmbH & Co. KG, Bielefeld, Germany
| |
Collapse
|
2
|
Kühnisch J, Aps JK, Splieth C, Lussi A, Jablonski-Momeni A, Mendes FM, Schmalz G, Fontana M, Banerjee A, Ricketts D, Schwendicke F, Douglas G, Campus G, van der Veen M, Opdam N, Doméjean S, Martignon S, Neuhaus KW, Horner K, Huysmans MCD. ORCA-EFCD consensus report on clinical recommendation for caries diagnosis. Paper I: caries lesion detection and depth assessment. Clin Oral Investig 2024; 28:227. [PMID: 38514502 PMCID: PMC10957694 DOI: 10.1007/s00784-024-05597-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/14/2023] [Accepted: 02/29/2024] [Indexed: 03/23/2024]
Abstract
OBJECTIVES The aim of the present consensus paper was to provide recommendations for clinical practice considering the use of visual examination, dental radiography and adjunct methods for primary caries detection. MATERIALS AND METHODS The executive councils of the European Organisation for Caries Research (ORCA) and the European Federation of Conservative Dentistry (EFCD) nominated ten experts each to join the expert panel. The steering committee formed three work groups that were asked to provide recommendations on (1) caries detection and diagnostic methods, (2) caries activity assessment and (3) forming individualised caries diagnoses. The experts responsible for "caries detection and diagnostic methods" searched and evaluated the relevant literature, drafted this manuscript and made provisional consensus recommendations. These recommendations were discussed and refined during the structured process in the whole work group. Finally, the agreement for each recommendation was determined using an anonymous Delphi survey. RESULTS Recommendations (N = 8) were approved and agreed upon by the whole expert panel: visual examination (N = 3), dental radiography (N = 3) and additional diagnostic methods (N = 2). While the quality of evidence was found to be heterogeneous, all recommendations were agreed upon by the expert panel. CONCLUSION Visual examination is recommended as the first-choice method for the detection and assessment of caries lesions on accessible surfaces. Intraoral radiography, preferably bitewing, is recommended as an additional method. Adjunct, non-ionising radiation methods might also be useful in certain clinical situations. CLINICAL RELEVANCE The expert panel merged evidence from the scientific literature with practical considerations and provided recommendations for their use in daily dental practice.
Collapse
Affiliation(s)
- Jan Kühnisch
- Department of Conservative Dentistry and Periodontology, University Hospital, Ludwig-Maximilians Universität München, Poliklinik für Zahnerhaltung und Parodontologie, Goethestraße 70, 80336, München, Germany.
| | | | - Christian Splieth
- Preventive and Pediatric Dentistry, Center for Oral Health, Universitätsmedizin Greifswald, Greifswald, Germany
| | - Adrian Lussi
- University Hospital for Conservative Dentistry and Periodontology, Medical University of Innsbruck, Innsbruck, Austria
- Department of Restorative, Preventive and Pediatric Dentistry, School of Dental Medicine, University of Bern, Bern, Switzerland
| | | | - Fausto M Mendes
- Department of Pediatric Dentistry, School of Dentistry, University of São Paulo, São Paulo, Brazil
| | - Gottfried Schmalz
- Department of Conservative Dentistry and Periodontology, University Hospital Regensburg, Regensburg, Germany
- Department of Periodontology, University of Bern, Bern, Switzerland
| | - Margherita Fontana
- Department of Cariology, Restorative Sciences and Endodontics, University of Michigan School of Dentistry, Ann Arbor, USA
| | - Avijit Banerjee
- Conservative & MI Dentistry, Faculty of Dentistry, Oral & Craniofacial Sciences, King's College London, London, UK
| | - David Ricketts
- Unit of Restorative Dentistry, University of Dundee, Dundee, UK
| | - Falk Schwendicke
- Department of Conservative Dentistry and Periodontology, University Hospital, Ludwig-Maximilians Universität München, Poliklinik für Zahnerhaltung und Parodontologie, Goethestraße 70, 80336, München, Germany
| | - Gail Douglas
- Department of Dental Public Health, University of Leeds Dental School, Leeds, UK
| | - Guglielmo Campus
- Department of Restorative, Preventive and Pediatric Dentistry, School of Dental Medicine, University of Bern, Bern, Switzerland
- Department of Surgery, Microsurgery and Medicine Sciences, School of Dentistry, University of Sassari, Sassari, Italy
| | - Monique van der Veen
- Departments of Preventive Dentistry and Paediatric Dentistry, Academic Centre for Dentistry Amsterdam, University of Amsterdam and VU University, Amsterdam, The Netherlands
- Oral Hygiene School, Inholland University of applied sciences, Amsterdam, The Netherlands
| | - Niek Opdam
- Department of Dentistry, Radboud University Medical Center, Nijmegen, The Netherlands
| | - Sophie Doméjean
- Centre de Recherche en Odontologie Clinique EA 4847, UFR d'Odontologie, Département d'Odontologie Conservatrice, Université Clermont Auvergne, Clermont-Ferrand, France
- Service d'Odontologie, CHU Estaing Clermont-Ferrand, Clermont-Ferrand, France
| | - Stefania Martignon
- UNICA - Caries Research Unit, Research Department, Universidad El Bosque, Bogotá, Colombia
| | - Klaus W Neuhaus
- Department of Pediatric Oral Health, University Center for Dental Medicine Basel (UZB), University of Basel, Basel, Switzerland
- Department of Dermatology, Inselspital, Bern University Hospital, University of Bern, Bern, Switzerland
| | - Keith Horner
- Division of Dentistry, School of Medical Sciences, Faculty of Biology, Medicine and Health, University of Manchester, Manchester Academic Health Science Centre, Manchester, UK
| | | |
Collapse
|
3
|
Veneri F, Filippini T, Consolo U, Vinceti M, Generali L. Ozone Treatment for the Management of Caries in Primary Dentition: A Systematic Review of Clinical Studies. Dent J (Basel) 2024; 12:69. [PMID: 38534293 DOI: 10.3390/dj12030069] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/26/2024] [Revised: 02/24/2024] [Accepted: 03/01/2024] [Indexed: 03/28/2024] Open
Abstract
Dental caries in children is a frequent and debilitating condition, whose management is often challenging. The aim of this systematic review was to investigate the effectiveness of ozone applications for the treatment of caries in primary dentition. According to PRISMA guidelines, a systematic literature search was performed up to 6 January 2024. Clinical studies using ozone to treat caries of deciduous teeth were considered for inclusion. Out of the 215 records retrieved, seven studies were eventually included in the review, all of which used gaseous ozone. Four studies were judged at high risk of bias, two at low risk, and one of some concerns. The great heterogeneity of designs, outcomes, and protocols made it impossible to conduct a meta-analysis. Despite some limitations, the evidence yielded by the included studies suggests that ozone application, regardless of the protocol applied, is comparable to other interventions in terms of clinical outcomes and anti-bacterial activity, with no reported adverse effects and good patient acceptance. Therefore, ozone application may be a non-invasive approach to treat caries in primary dentition, especially in very young and poorly cooperative patients. Further standardized and rigorous studies are, however, needed to identify the best clinical protocols for this specific field.
Collapse
Affiliation(s)
- Federica Veneri
- Unit of Dentistry & Oral-Maxillo-Facial Surgery, Department of Surgery, Medicine, Dentistry and Morphological Sciences with Transplant Surgery, Oncology and Regenerative Medicine Relevance (CHIMOMO), University of Modena and Reggio Emilia, 41124 Modena, Italy
- PhD Program in Clinical and Experimental Medicine, Department of Biomedical, Metabolic and Neural Sciences, University of Modena and Reggio Emilia, 41124 Modena, Italy
| | - Tommaso Filippini
- Environmental, Genetic and Nutritional Epidemiology Research Center (CREAGEN), Section of Public Health, Department of Biomedical, Metabolic and Neural Sciences, University of Modena and Reggio Emilia, 41124 Modena, Italy
- School of Public Health, University of California Berkeley, Berkeley, CA 94704, USA
| | - Ugo Consolo
- Unit of Dentistry & Oral-Maxillo-Facial Surgery, Department of Surgery, Medicine, Dentistry and Morphological Sciences with Transplant Surgery, Oncology and Regenerative Medicine Relevance (CHIMOMO), University of Modena and Reggio Emilia, 41124 Modena, Italy
| | - Marco Vinceti
- Environmental, Genetic and Nutritional Epidemiology Research Center (CREAGEN), Section of Public Health, Department of Biomedical, Metabolic and Neural Sciences, University of Modena and Reggio Emilia, 41124 Modena, Italy
- Department of Epidemiology, Boston University School of Public Health, Boston, MA 02118, USA
| | - Luigi Generali
- Unit of Dentistry & Oral-Maxillo-Facial Surgery, Department of Surgery, Medicine, Dentistry and Morphological Sciences with Transplant Surgery, Oncology and Regenerative Medicine Relevance (CHIMOMO), University of Modena and Reggio Emilia, 41124 Modena, Italy
| |
Collapse
|
4
|
Liu C, Wang YF, Wang P, Guo F, Zhao HY, Wang Q, Shi ZW, Li XF. Predictive value of multiple imaging predictive models for spread through air spaces of lung adenocarcinoma: A systematic review and network meta‑analysis. Oncol Lett 2024; 27:122. [PMID: 38348387 PMCID: PMC10859825 DOI: 10.3892/ol.2024.14255] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/01/2023] [Accepted: 01/03/2024] [Indexed: 02/15/2024] Open
Abstract
Spread Through Air Spaces (STAS) is involved in lung adenocarcinoma (LUAD) recurrence, where cancer cells spread into adjacent lung tissue, impacting surgical planning and prognosis assessment. Radiomics-based models show promise in predicting STAS preoperatively, enhancing surgical precision and prognostic evaluations. The present study performed network meta-analysis to assess the predictive efficacy of imaging models for STAS in LUAD. Data were systematically sourced from PubMed, Embase, Scopus, Wiley and Web of Science, according to the Cochrane Handbook for Systematic Reviews of Interventions) and A Measurement Tool to Assess systematic Reviews 2. Using Stata software v17.0 for meta-analysis, surface under the cumulative ranking area (SUCRA) was applied to identify the most effective diagnostic method. Quality assessments were performed using Cochrane Collaboration's risk-of-bias tool and publication bias was assessed using Deeks' funnel plot. The analysis encompassed 14 articles, involving 3,734 patients, and assessed 17 predictive models for STAS in LUAD. According to comprehensive analysis of SUCRA, the machine learning (ML)_Peri_tumour model had the highest accuracy (56.5), the Features_computed tomography (CT) model had the highest sensitivity (51.9) and the positron emission tomography (pet)_CT model had the highest specificity (53.9). ML_Peri_tumour model had the highest predictive performance. The accuracy was as follows: ML_Peri_tumour vs. Features_CT [relative risk (RR)=1.14; 95% confidence interval (CI), 0.99-1.32]; ML_Peri_tumour vs. ML_Tumour (RR=1.04; 95% CI, 0.83-1.30) and ML_Peri_tumour vs. pet_CT (RR=1.04; 95% CI, 0.84-1.29). Comparative analyses revealed heightened predictive accuracy of the ML_Peri_tumour compared with other models. Nonetheless, the field of radiological feature analysis for STAS prediction remains nascent, necessitating improvements in technical reproducibility and comprehensive model evaluation.
Collapse
Affiliation(s)
- Cong Liu
- Department of Minimally Invasive Oncology, Xuzhou New Health Hospital (The Xuzhou Hospital Affiliated to Jiangsu University), Xuzhou, Jiangsu 221000, P.R. China
| | - Yu-Feng Wang
- Department of Nuclear Medicine, Xuzhou Cancer Hospital (The Xuzhou Hospital Affiliated to Jiangsu University), Xuzhou, Jiangsu 221000, P.R. China
| | - Peng Wang
- Department of Nuclear Medicine, Xuzhou Cancer Hospital (The Xuzhou Hospital Affiliated to Jiangsu University), Xuzhou, Jiangsu 221000, P.R. China
| | - Feng Guo
- Department of Medical Oncology, Xuzhou Cancer Hospital (The Xuzhou Hospital Affiliated to Jiangsu University), Xuzhou, Jiangsu 221000, P.R. China
| | - Hong-Ying Zhao
- Department of Radiotherapy, Xuzhou Cancer Hospital (The Xuzhou Hospital Affiliated to Jiangsu University), Xuzhou, Jiangsu 221000, P.R. China
| | - Qiang Wang
- Department of Radiotherapy, Xuzhou Cancer Hospital (The Xuzhou Hospital Affiliated to Jiangsu University), Xuzhou, Jiangsu 221000, P.R. China
| | - Zhi-Wei Shi
- Department of Radiology, Xuzhou Cancer Hospital (The Xuzhou Hospital Affiliated to Jiangsu University), Xuzhou, Jiangsu 221000, P.R. China
| | - Xiao-Feng Li
- Department of Radiology, Xuzhou Cancer Hospital (The Xuzhou Hospital Affiliated to Jiangsu University), Xuzhou, Jiangsu 221000, P.R. China
| |
Collapse
|
5
|
Schulz-Weidner N, Gruber M, Wöstmann B, Uebereck CF, Krämer N, Schlenz MA. Occlusal Caries Detection with Intraoral Scanners in Pediatric Dentistry: A Comparative Clinical Study. J Clin Med 2024; 13:925. [PMID: 38398239 PMCID: PMC10889243 DOI: 10.3390/jcm13040925] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/30/2023] [Revised: 02/02/2024] [Accepted: 02/04/2024] [Indexed: 02/25/2024] Open
Abstract
Background: The aim of this clinical study was to compare the occlusal caries detection (OCD) performance of the intraoral scanners (IOSs) Trios 4 (TIO, 3Shape) and Emerald S (EME, Planmeca) and the Diagnocam (DIA, KaVo) with the established visual (WHO) examination (VIS, reference method). Methods: Between 08/2022 and 02/2023, 60 children (mean age 9.6 ± 2.5 years) were examined as part of their regular dental checkups. OCD was performed at the tooth level, separately for primary and permanent unrestored teeth. Furthermore, two thresholds were analyzed: sound versus overall caries (pooled data of enamel and dentin caries, TH1) and pooled data of sound and enamel caries versus dentin caries (TH2). Results: The best agreement with the reference method (reliability) in both dentitions was obtained for DIA (ĸ = 0.829/ĸ = 0.846; primary/permanent teeth), followed by EME (ĸ = 0.827/ĸ = 0.837) and TIO (ĸ = 0.714/ĸ = 0.680). Similar results were shown for the diagnostic quality (sensitivity, specificity and area under the curve of the receiver operating characteristic curve), with higher values for TH1 than for TH2. Both IOSs and the DIA showed worse results than the reference method VIS. Conclusions: Currently, IOS should be used as an additional caries detection tool, especially for visualization, and cannot be recommended as a basic tool for diagnosis or invasive/noninvasive therapy decisions in OCD.
Collapse
Affiliation(s)
- Nelly Schulz-Weidner
- Department of Pediatric Dentistry, Dental Clinic, Justus Liebig University, Schlangenzahl 14, 35392 Giessen, Germany; (N.S.-W.)
| | - Marina Gruber
- Department of Prosthodontics, Dental Clinic, Justus Liebig University, Schlangenzahl 14, 35392 Giessen, Germany (B.W.)
| | - Bernd Wöstmann
- Department of Prosthodontics, Dental Clinic, Justus Liebig University, Schlangenzahl 14, 35392 Giessen, Germany (B.W.)
| | - Constanze Friederike Uebereck
- Department of Pediatric Dentistry, Dental Clinic, Justus Liebig University, Schlangenzahl 14, 35392 Giessen, Germany; (N.S.-W.)
| | - Norbert Krämer
- Department of Pediatric Dentistry, Dental Clinic, Justus Liebig University, Schlangenzahl 14, 35392 Giessen, Germany; (N.S.-W.)
| | - Maximiliane Amelie Schlenz
- Department of Prosthodontics, Dental Clinic, Justus Liebig University, Schlangenzahl 14, 35392 Giessen, Germany (B.W.)
| |
Collapse
|
6
|
Carneiro VSM, de Melo EL, de Oliveira Mota CCB, da Silva EJ, da Silva AF, Deama NS, Miranda JM, da Rocha SIS, de Lima Pires C, Gomes ASL, de Martínez Gerbi MEM. Optical clearing agents based on metallic and dielectric nanoparticles for caries diagnostic by optical coherence tomography. Clin Oral Investig 2024; 28:72. [PMID: 38175293 DOI: 10.1007/s00784-023-05473-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/25/2023] [Accepted: 12/20/2023] [Indexed: 01/05/2024]
Abstract
OBJECTIVE This study aimed to assess the effect of titanium dioxide (TiO2) and silver (Ag) nanoparticles dispersed in glycerol or water, serving as optical clearing agents nanocolloids (OCAs-NC), for improving optical coherence tomography (OCT) images and highlighting incipient lesions in ex vivo human teeth. MATERIALS AND METHODS Twelve human teeth with incipient lesions were divided into seven groups according to the OCA-NC; they were subjected to G1 (air), G2 (glycerol), G3 (TiO2 0.1%), G4 (TiO2 0.01%), G5 (TiO2 0.001%), G6 (AgNO3 10%), and G7 (AgNO3 100%). The OCA-NC was applied to the occlusal surface, and two-dimensional images of the specimens were analyzed using OCT (930 nm central wavelength; 100 nm bandwidth; 5 mW output power; axial resolution of 7/5.3 μm in water and air, respectively; lateral resolution of 8 μm; and light penetration depth of 1.6 mm inside the sample). RESULTS The findings demonstrated that the utilization of OCAs-NC containing metallic or dielectric nanoparticles (AgNO3 and TiO2) led to improved differentiation between sound and demineralized enamel on occlusal surfaces. Additionally, it enhanced the depth of image penetration when analyzing this hard tissue with OCT. CLINICAL RELEVANCE In the current context of minimally invasive dentistry, the use of OCAs-NC in conjunction with OCT can provide clinicians with early diagnosis, allowing for the determination of less/more invasive therapies and consequently halting the disease before cavitation of dental tissues occurs.
Collapse
Affiliation(s)
| | | | - Claudia Cristina Brainer de Oliveira Mota
- Dental School of Pernambuco, Universidade de Pernambuco Campus Arcoverde, Arcoverde, PE, Brazil
- Dental School, Centro Universitário Tabosa de Almeida, Caruaru, PE, Brazil
| | - Evair Josino da Silva
- Graduate Program in Dentistry, Universidade Federal de Pernambuco, Recife, PE, Brazil
| | | | - Nathalia Seimi Deama
- Graduate Program in Dentistry, Universidade de Pernambuco (PPGO-UPE), Recife, Brazil
| | | | | | - Caio de Lima Pires
- Graduate Program in Dentistry, Universidade de Pernambuco (PPGO-UPE), Recife, Brazil
| | - Anderson Stevens Leonidas Gomes
- Graduate Program in Dentistry, Universidade Federal de Pernambuco, Recife, PE, Brazil
- Department of Physics, Universidade Federal de Pernambuco, Recife, PE, Brazil
| | | |
Collapse
|
7
|
Spagopoulos D, Michou S, Gizani S, Pappa E, Rahiotis C. Fluorescence and Near-Infrared Light for Detection of Secondary Caries: A Systematic Review. Dent J (Basel) 2023; 11:271. [PMID: 38132409 PMCID: PMC10742255 DOI: 10.3390/dj11120271] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/29/2023] [Revised: 11/16/2023] [Accepted: 11/23/2023] [Indexed: 12/23/2023] Open
Abstract
BACKGROUND Early detection of secondary caries near dental restorations is essential to prevent further complications. This systematic review seeks to evaluate the sensitivity of fluorescence and near-infrared (NIR) imaging techniques for detecting secondary caries and to provide insight into their clinical utility. METHODS A comprehensive search strategy was used to select studies from seven databases, emphasizing diagnostic accuracy studies of secondary caries detection using fluorescence and NIR imaging techniques. The Quality Assessment of Diagnostic Accuracy Studies-2 (QUADAS-2) instrument assessed bias risk and practicality. Two evaluators performed data extraction, screening, and quality assessment independently. RESULTS From 3110 initial recordings, nine studies were selected for full-text analysis. Wide variations in sensitivity (SE) and specificity (SP) values were reported across the studies. These studies exhibited variable SE and SP values, and the findings highlighted the importance of method selection based on clinical context. This systematic review underlines the potential for fluorescence and NIR imaging to detect secondary caries. However, results from different studies vary, indicating the need to consider additional variables such as restoration materials. CONCLUSIONS Although these technologies exhibit potential for detecting caries, our research underscores the complex procedure of identifying secondary caries lesions. It is a continuous necessity for progress in dental diagnostics to promptly identify secondary caries lesions, particularly those in proximity to tooth-colored ones.
Collapse
Affiliation(s)
- Dimitrios Spagopoulos
- Department of Operative Dentistry, National and Kapodistrian University of Athens, 11527 Athens, Greece; (D.S.); (E.P.)
| | - Stavroula Michou
- Department of Odontology, University of Copenhagen, DK-2200 Copenhagen, Denmark;
| | - Sotiria Gizani
- Department of Pediatric Dentistry, National and Kapodistrian University of Athens, 11527 Athens, Greece
| | - Eftychia Pappa
- Department of Operative Dentistry, National and Kapodistrian University of Athens, 11527 Athens, Greece; (D.S.); (E.P.)
| | - Christos Rahiotis
- Department of Operative Dentistry, National and Kapodistrian University of Athens, 11527 Athens, Greece; (D.S.); (E.P.)
| |
Collapse
|
8
|
Felsch M, Meyer O, Schlickenrieder A, Engels P, Schönewolf J, Zöllner F, Heinrich-Weltzien R, Hesenius M, Hickel R, Gruhn V, Kühnisch J. Detection and localization of caries and hypomineralization on dental photographs with a vision transformer model. NPJ Digit Med 2023; 6:198. [PMID: 37880375 PMCID: PMC10600213 DOI: 10.1038/s41746-023-00944-2] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/06/2023] [Accepted: 10/13/2023] [Indexed: 10/27/2023] Open
Abstract
Caries and molar-incisor hypomineralization (MIH) are among the most prevalent diseases worldwide and need to be reliably diagnosed. The use of dental photographs and artificial intelligence (AI) methods may potentially contribute to realizing accurate and automated diagnostic visual examinations in the future. Therefore, the present study aimed to develop an AI-based algorithm that can detect, classify and localize caries and MIH. This study included an image set of 18,179 anonymous photographs. Pixelwise image labeling was achieved by trained and calibrated annotators using the Computer Vision Annotation Tool (CVAT). All annotations were made according to standard methods and were independently checked by an experienced dentist. The entire image set was divided into training (N = 16,679), validation (N = 500) and test sets (N = 1000). The AI-based algorithm was trained and finetuned over 250 epochs by using image augmentation and adapting a vision transformer network (SegFormer-B5). Statistics included the determination of the intersection over union (IoU), average precision (AP) and accuracy (ACC). The overall diagnostic performance in terms of IoU, AP and ACC were 0.959, 0.977 and 0.978 for the finetuned model, respectively. The corresponding data for the most relevant caries classes of non-cavitations (0.630, 0.813 and 0.990) and dentin cavities (0.692, 0.830, and 0.997) were found to be high. MIH-related demarcated opacity (0.672, 0.827, and 0.993) and atypical restoration (0.829, 0.902, and 0.999) showed similar results. Here, we report that the model achieves excellent precision for pixelwise detection and localization of caries and MIH. Nevertheless, the model needs to be further improved and externally validated.
Collapse
Affiliation(s)
- Marco Felsch
- Department of Conservative Dentistry and Periodontology, School of Dentistry, Ludwig-Maximilians University of Munich, Munich, Germany
| | - Ole Meyer
- Institute for Software Engineering, University of Duisburg-Essen, Essen, Germany
| | - Anne Schlickenrieder
- Department of Conservative Dentistry and Periodontology, School of Dentistry, Ludwig-Maximilians University of Munich, Munich, Germany
| | - Paula Engels
- Department of Conservative Dentistry and Periodontology, School of Dentistry, Ludwig-Maximilians University of Munich, Munich, Germany
| | - Jule Schönewolf
- Department of Conservative Dentistry and Periodontology, School of Dentistry, Ludwig-Maximilians University of Munich, Munich, Germany
| | - Felicitas Zöllner
- Department of Conservative Dentistry and Periodontology, School of Dentistry, Ludwig-Maximilians University of Munich, Munich, Germany
| | - Roswitha Heinrich-Weltzien
- Department of Orthodontics, Section of Preventive and Paediatric Dentistry, University Hospital Jena, Jena, Germany
| | - Marc Hesenius
- Institute for Software Engineering, University of Duisburg-Essen, Essen, Germany
| | - Reinhard Hickel
- Department of Conservative Dentistry and Periodontology, School of Dentistry, Ludwig-Maximilians University of Munich, Munich, Germany
| | - Volker Gruhn
- Institute for Software Engineering, University of Duisburg-Essen, Essen, Germany
| | - Jan Kühnisch
- Department of Conservative Dentistry and Periodontology, School of Dentistry, Ludwig-Maximilians University of Munich, Munich, Germany.
| |
Collapse
|
9
|
The Detection of Dental Pathologies on Periapical Radiographs—Results from a Reliability Study. J Clin Med 2023; 12:jcm12062224. [PMID: 36983223 PMCID: PMC10056324 DOI: 10.3390/jcm12062224] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/06/2023] [Revised: 03/05/2023] [Accepted: 03/11/2023] [Indexed: 03/17/2023] Open
Abstract
(1) Background: Caries, periapical lesions, periodontal bone loss (PBL), and endo-perio lesions are common dental findings that require an accurate diagnostic assessment to allow appropriate disease management. The purpose of this reliability study was to compare the inter- and intra-rater reliability for the detection of the above-mentioned pathologies on periapical radiographs. (2) Methods: Fourteen dentists (three with more than two years and eleven with less than two years of work experience) participated in a training workshop prior to data acquisition. A total of 150 radiographs were assessed by all raters in two rounds. Cohen’s Kappa (CK) values and a binary logistic regression were calculated. (3) Results: The reliability was found in a moderate and substantial range of agreement: caries (mean inter-rater CK value/first round 0.704/mean inter-rater CK value/second round 0.659/mean intra-rater CK value 0.778), periapical lesions (0.643/0.611/0.768), PBL (0.454/0.482/0.739) and endo-perio lesion (0.702/0.689/0.840). The regression model revealed a significant influence of the clinical experience, and furthermore, periapical pathologies and PBL were identified less reliably in comparison to caries and endo-perio lesions. (4) Conclusions: The dentist’s ability to detect the chosen pathologies was linked with significant differences. Periapical lesions and PBL were identified less reliably than caries and endo-perio lesions.
Collapse
|
10
|
Göstemeyer G, Preus M, Elhennawy K, Schwendicke F, Paris S, Askar H. Accuracy of different approaches for detecting proximal root caries lesions in vitro. Clin Oral Investig 2023; 27:1143-1151. [PMID: 36112228 PMCID: PMC9985551 DOI: 10.1007/s00784-022-04709-1] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/04/2022] [Accepted: 09/05/2022] [Indexed: 11/25/2022]
Abstract
OBJECTIVES The objective was to evaluate the diagnostic accuracy of radiographic evaluation (XR), visual-tactile assessment (VT), laser-fluorescence (LF) (DIAGNOdent Pen/KaVo), and near-infrared-light transillumination (NILT) (DIAGNOcam/KaVo) on proximal root caries lesions in vitro. METHODS Two-hundred extracted permanent premolars and molars with and without proximal root caries lesions were allocated to 50 diagnostic models simulating the proximal contacts between teeth and mounted in a phantom dummy head. Two independent examiners used the diagnostic approaches to detect any or advanced root caries lesions, with histologic evaluation of the lesions serving as reference. Receiver operating characteristic (ROC) curves were employed, and sensitivity, specificity, and the area under the ROC curve (AUC) are calculated. Significant differences in mean AUCs between approaches were assumed if p < 0.05 (two-sample t-test). RESULTS NILT was not applicable for proximal root caries detection. The sensitivity/specificity to detect any lesions was 0.81/0.63 for XR, 0.76/0.88 for VT and 0.81/0.95 for LF, and the sensitivity/specificity to detect advanced lesions was 0.43/0.94 for XR, 0.66/0.99 for VT, and 0.83/0.78 for LF, respectively. For both, any and advanced root caries lesions, mean AUCs for LF and VT were significantly higher compared to XR (p < 0.05). For any root caries lesions, LF was significantly more accurate than VT (p = 0.01), but not for advanced root caries lesions (p = 0.59). CONCLUSIONS Under the in vitro conditions chosen, LF and VT were more accurate than XR to detect proximal root caries lesions, with LF being particularly useful for initial lesion stages. CLINICAL RELEVANCE LF might be a useful diagnostic aid for proximal root caries diagnosis. Clinical studies are necessary to corroborate the findings.
Collapse
Affiliation(s)
- Gerd Göstemeyer
- Department of Operative, Preventive and Pediatric Dentistry, Charité - Universitätsmedizin Berlin, Assmannshauser Straße 4-6, 14197, Berlin, Germany.
| | - Mareike Preus
- Department of Operative, Preventive and Pediatric Dentistry, Charité - Universitätsmedizin Berlin, Assmannshauser Straße 4-6, 14197, Berlin, Germany
| | - Karim Elhennawy
- Department of Orthodontics, Dentofacial Orthopedics and Pedodontics, Charité - Universitätsmedizin Berlin, Assmannshauser Straße 4-6, 14197, Berlin, Germany
| | - Falk Schwendicke
- Department of Oral Diagnostics, Digital Health and Health Services Research, Charité - Universitätsmedizin Berlin, Assmannshauser Straße 4-6, 14197, Berlin, Germany
| | - Sebastian Paris
- Department of Operative, Preventive and Pediatric Dentistry, Charité - Universitätsmedizin Berlin, Assmannshauser Straße 4-6, 14197, Berlin, Germany
| | - Haitham Askar
- Department of Operative, Preventive and Pediatric Dentistry, Charité - Universitätsmedizin Berlin, Assmannshauser Straße 4-6, 14197, Berlin, Germany
| |
Collapse
|
11
|
Ntovas P, Michou S, Benetti AR, Bakhshandeh A, Ekstrand K, Rahiotis C, Kakaboura A. Occlusal caries detection on 3D models obtained with an intraoral scanner. A validation study. J Dent 2023; 131:104457. [PMID: 36858167 DOI: 10.1016/j.jdent.2023.104457] [Citation(s) in RCA: 5] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/06/2022] [Revised: 02/13/2023] [Accepted: 02/15/2023] [Indexed: 03/02/2023] Open
Abstract
OBJECTIVES To evaluate the diagnostic performance of visual caries assessment on 3D dental models obtained using an intraoral scanner and to compare it with the performance of the clinical visual inspection. METHODS Fifty-three permanent posterior teeth scheduled for extraction were randomly selected and included in this study. One to three independent examination sites on the occlusal surface of each tooth were clinically inspected using International Caries Detection and Assessment System (ICDAS) criteria. Afterwards, the examined teeth were scanned intraorally with a 3D intraoral scanner (TRIOS 4, 3Shape TRIOS A/S, Copenhagen, Denmark) using white and blue-violet light (415 nm wavelength) to capture the colour and fluorescence signal from the tissues. Six months after the clinical examination, the same examiner conducted the on-screen assessment of the obtained 3D digital dental models at the selected examination sites using modified ICDAS criteria. Both tooth colour and fluorescence texture with high resolution were assessed. Lastly, an independent examiner conducted the histological examination of all teeth after extraction. Using histology as the reference test, Sensitivity (SE), Specificity (SP), Accuracy (ACC), area under the Receiver Operating Characteristic (ROC) curve, and Spearman's correlation coefficient were calculated for the clinical and on-screen ICDAS assessments. RESULTS The ACC values of the evaluated methods varied between 0.59-0.79 for initial caries lesions and 0.77-0.99 for moderate-extensive caries lesions. Apart from SE values corresponding to caries in the inner half of enamel, no significant difference was observed between clinical visual inspection and on-screen assessment. In addition, no difference was found in the assessment of 3D models with tooth colour alone or supplemented with fluorescence for all the evaluated diagnostic measures. CONCLUSIONS On-screen visual assessment of 3D digital dental models with tooth colour or fluorescence showed a similar diagnostic performance to the clinical visual inspection when detecting and classifying occlusal caries lesions on permanent teeth. CLINICAL SIGNIFICANCE 3D intraoral scanning can aid the detection and classification of occlusal caries as part of patient screening and can potentially be used in remote caries assessment for clinical and research purposes.
Collapse
Affiliation(s)
- P Ntovas
- Operative Dentistry Department, School of Dentistry, National and Kapodistrian University of Athens, Greece
| | - S Michou
- Department of Odontology, Faculty of Health and Medical Sciences, University of Copenhagen, Denmark; 3Shape TRIOS A/S, Copenhagen, Denmark.
| | - A R Benetti
- Department of Odontology, Faculty of Health and Medical Sciences, University of Copenhagen, Denmark
| | - A Bakhshandeh
- Department of Odontology, Faculty of Health and Medical Sciences, University of Copenhagen, Denmark
| | - K Ekstrand
- Department of Odontology, Faculty of Health and Medical Sciences, University of Copenhagen, Denmark
| | - C Rahiotis
- Operative Dentistry Department, School of Dentistry, National and Kapodistrian University of Athens, Greece
| | - A Kakaboura
- Operative Dentistry Department, School of Dentistry, National and Kapodistrian University of Athens, Greece
| |
Collapse
|
12
|
Kreher D, Ernst BLV, Ziebolz D, Haak R, de Fallois J, Ebert T, Schmalz G. Prevalence of Dental Caries in Patients on Renal Replacement Therapy-A Systematic Review. J Clin Med 2023; 12:jcm12041507. [PMID: 36836050 PMCID: PMC9967680 DOI: 10.3390/jcm12041507] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/29/2022] [Revised: 01/30/2023] [Accepted: 02/10/2023] [Indexed: 02/17/2023] Open
Abstract
Patients under renal replacement therapy (RRT) often show oral problems, including dry mouth, periodontal and dental diseases. This systematic review aimed to evaluate the caries burden in patients on RRT. Therefore, a systematic literature search based on the databases PubMed, Web of Science and Scopus was performed by two independent individuals in August 2022. Search terms were: "caries" AND "dialysis", "caries" AND "renal replacement therapy", "caries" AND "kidney". The systematic process was complemented by manual search. Studies on adult patients (age ≥ 18 years), treated by any form of RRT and explicitly reporting caries prevalence or incidence were checked for their eligibility and subsequently analyzed qualitatively. For all included studies, a quality appraisal was applied. From the systematic search, 653 studies were identified, of which 33 clinical investigations were included in the qualitative analysis. The majority (31 studies) of all included patients underwent hemodialysis (HD), with a sample size between 28 and 512 participants. Eleven studies investigated a healthy control group. Oral examinations were heterogeneous across studies; the caries burden was primarily assessed by decayed-(D-T), missing- and filled-teeth index (DMF-T). The number of decayed teeth ranged between 0.7 and 3.87 across studies. Only six out of these 11 studies found significant differences in caries prevalence/incidence between RRT and controls, whereby only four studies confirmed worse caries burden in RRT individuals. No information was provided on caries stadium (initial caries, advanced caries, invasive treatment need), caries activity or location (e.g., root caries) across studies. Most of the included studies were assessed to be of moderate quality. In conclusion, patients on RRT suffer from a high prevalence of dental caries. Alongside a need for further research in the field, improved, multidisciplinary, patient-centered dental care concepts are required to support dental and overall oral health in individuals on RRT.
Collapse
Affiliation(s)
- Deborah Kreher
- Department of Cariology, Endodontology and Periodontology, University of Leipzig, 04109 Leipzig, Germany
| | - Bero Luke Vincent Ernst
- Department of Cariology, Endodontology and Periodontology, University of Leipzig, 04109 Leipzig, Germany
| | - Dirk Ziebolz
- Department of Cariology, Endodontology and Periodontology, University of Leipzig, 04109 Leipzig, Germany
| | - Rainer Haak
- Department of Cariology, Endodontology and Periodontology, University of Leipzig, 04109 Leipzig, Germany
| | - Jonathan de Fallois
- Medical Department III—Endocrinology, Nephrology, Rheumatology, University of Leipzig, 04109 Leipzig, Germany
| | - Thomas Ebert
- Medical Department III—Endocrinology, Nephrology, Rheumatology, University of Leipzig, 04109 Leipzig, Germany
| | - Gerhard Schmalz
- Department of Cariology, Endodontology and Periodontology, University of Leipzig, 04109 Leipzig, Germany
- Correspondence:
| |
Collapse
|
13
|
Wülk A, Meusburger T, Dujic H, Hickel R, Kessler A, Heck K, Kühnisch J. The Dentist's Ability to Detect Different Restorative Procedures on Periapical Radiographs-Results from a Reliability Study. INTERNATIONAL JOURNAL OF ENVIRONMENTAL RESEARCH AND PUBLIC HEALTH 2023; 20:2619. [PMID: 36767984 PMCID: PMC9915428 DOI: 10.3390/ijerph20032619] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 12/12/2022] [Revised: 01/26/2023] [Accepted: 01/30/2023] [Indexed: 06/18/2023]
Abstract
(1) Background: This in vitro reliability study aimed to determine the inter- and intra-examiner reliability for the detection of direct fillings, indirect crown restorations, root canal fillings and implants on periapical radiographs. (2) Methods: Fourteen dentists (<2 years of clinical experience = 11; >2 years of clinical experience = 3) participated in this diagnostic reliability study in which included a theoretical and practical educational training prior to data collection. The image set of periapical radiographs (N = 150) was examined in two evaluation rounds by all the dentists. Cohen's Kappa (CK) and a binary logistic regression model were computed. (3) Results: The inter- and intra-examiner reliability were found to be in almost perfect agreement: direct fillings (1st round 0.859/2nd round 0.844/intra 0.910), indirect crown restorations (0.932/0.926/0.955), root canal fillings (0.920/0.886/0.941) and dental implants (0.994/0.988/0.987). The binary logistic regression model revealed that the "evaluation round" and "dentist's clinical experience" had no significant influence, but for the "diagnostic category"; small, but statistically significant differences were documented. (4) Conclusions: The reliability for detecting direct and indirect restorations, root canal fillings or implants on periapical radiographs was found to be high in the present reliability study on periapical radiographs.
Collapse
Affiliation(s)
| | | | | | | | | | | | - Jan Kühnisch
- Correspondence: ; Tel.: +49-89-4400-59343 or +49-89-4400-59301
| |
Collapse
|
14
|
Emerging Technologies for Dentin Caries Detection-A Systematic Review and Meta-Analysis. J Clin Med 2022; 11:jcm11030674. [PMID: 35160127 PMCID: PMC8837049 DOI: 10.3390/jcm11030674] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/28/2021] [Revised: 01/19/2022] [Accepted: 01/26/2022] [Indexed: 11/20/2022] Open
Abstract
This systematic review and meta-analysis aimed at assessing the diagnostic accuracy of emerging technologies, such as laser fluorescence (LF), transillumination, light-emitting diode devices, optical coherence tomography (OCT), alternating current impedance spectroscopy, fluorescence cameras (FC), photo-thermal radiometry, and modulated luminescence technology. In vivo and in vitro results of such non-ionizing, non-invasive, and non-destructive methods’ effectiveness in non-cavitated dentin caries detection are sometimes ambiguous. Following the PRISMA guidelines, 34 relevant research articles published between 2011–2021 were selected. The risk of bias was assessed with a tool tailored for caries diagnostic studies, and subsequent quantitative uni- and bi-variate meta-analysis was carried out in separate sub-groups according to the investigated surface (occlusal/proximal) and study setting (in vivo/in vitro). In spite of the high heterogeneity across the review groups, in vitro studies on LF and FC proved a good diagnostic ability for the occlusal surface, with area under the curve (AUC) of 0.803 (11 studies) and 0.845 (five studies), respectively. OCT studies reported an outstanding performance with an overall AUC = 0.945 (four studies). Promising technologies, such as OCT or FC VistaProof, still need well-designed and well-powered studies to accrue experimental and clinical data for conclusive medical evidence, especially for the proximal surface. Registration: INPLASY202210097.
Collapse
|
15
|
STAndard Reporting of CAries Detection and Diagnostic Studies (STARCARDDS). Clin Oral Investig 2021; 26:1947-1955. [PMID: 34623505 PMCID: PMC8816754 DOI: 10.1007/s00784-021-04173-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/07/2020] [Accepted: 08/31/2021] [Indexed: 12/04/2022]
Abstract
Aim The aim of this paper is to present recommendations from an international workshop which evaluated the methodology and reporting of caries diagnostic studies. As a unique feature, this type of studies is focused on caries lesion detection and assessment, and many of them are carried out in vitro, because of the possibility of histological validation of the whole caries spectrum. This feature is not well covered in the existing reporting STARD guideline within the EQUATOR Network. Participants and methods An international working group of 13 cariology researchers was formed. The STARD checklist was reviewed and modified for caries detection and diagnosis purposes, in a three-step process of evaluation, consensual modification, and delivery during three 2-day workshops over 18 months. Special attention was paid to reporting requirements of caries studies that solely focus on reliability. Results The STARD checklist was modified in 14/30 items, with an emphasis on issues of sample selection (tooth selection in in vitro studies), blinding, and detailed reporting of results. Conclusion Following STARCARDDS (STAndard Reporting of CAries Detection and Diagnostic Studies) is expected to result in complete reporting of study design and methodology in future caries diagnosis and detection experiments both in vivo and in vitro, thus allowing for better comparability of studies and higher quality of systematic reviews. Clinical relevance Standardization of caries diagnostic studies leads to a better comparability among future studies, both in vivo and in vitro.
Collapse
|
16
|
Kühnisch J, Meyer O, Hesenius M, Hickel R, Gruhn V. Caries Detection on Intraoral Images Using Artificial Intelligence. J Dent Res 2021; 101:158-165. [PMID: 34416824 PMCID: PMC8808002 DOI: 10.1177/00220345211032524] [Citation(s) in RCA: 40] [Impact Index Per Article: 13.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/04/2023] Open
Abstract
Although visual examination (VE) is the preferred method for caries detection, the analysis of intraoral digital photographs in machine-readable form can be considered equivalent to VE. While photographic images are rarely used in clinical practice for diagnostic purposes, they are the fundamental requirement for automated image analysis when using artificial intelligence (AI) methods. Considering that AI has not been used for automatic caries detection on intraoral images so far, this diagnostic study aimed to develop a deep learning approach with convolutional neural networks (CNNs) for caries detection and categorization (test method) and to compare the diagnostic performance with respect to expert standards. The study material consisted of 2,417 anonymized photographs from permanent teeth with 1,317 occlusal and 1,100 smooth surfaces. All the images were evaluated into the following categories: caries free, noncavitated caries lesion, or caries-related cavitation. Each expert diagnosis served as a reference standard for cyclic training and repeated evaluation of the AI methods. The CNN was trained using image augmentation and transfer learning. Before training, the entire image set was divided into a training and test set. Validation was conducted by selecting 25%, 50%, 75%, and 100% of the available images from the training set. The statistical analysis included calculations of the sensitivity (SE), specificity (SP), and area under the receiver operating characteristic (ROC) curve (AUC). The CNN was able to correctly detect caries in 92.5% of cases when all test images were considered (SE, 89.6; SP, 94.3; AUC, 0.964). If the threshold of caries-related cavitation was chosen, 93.3% of all tooth surfaces were correctly classified (SE, 95.7; SP, 81.5; AUC, 0.955). It can be concluded that it was possible to achieve more than 90% agreement in caries detection using the AI method with standardized, single-tooth photographs. Nevertheless, the current approach needs further improvement.
Collapse
Affiliation(s)
- J Kühnisch
- Department of Conservative Dentistry and Periodontology, University Hospital, School of Dentistry, Ludwig-Maximilians University of Munich, Munich, Germany
| | - O Meyer
- Institute for Software Engineering, University of Duisburg-Essen, Essen, Germany
| | - M Hesenius
- Institute for Software Engineering, University of Duisburg-Essen, Essen, Germany
| | - R Hickel
- Department of Conservative Dentistry and Periodontology, University Hospital, School of Dentistry, Ludwig-Maximilians University of Munich, Munich, Germany
| | - V Gruhn
- Institute for Software Engineering, University of Duisburg-Essen, Essen, Germany
| |
Collapse
|