1
|
Kuo JC, Chan W, Leon-Novelo L, Lairson DR, Brown A, Fujimoto K. Latent classification model for censored longitudinal binary outcome. Stat Med 2024; 43:3943-3957. [PMID: 38951953 DOI: 10.1002/sim.10156] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/08/2023] [Revised: 04/23/2024] [Accepted: 06/10/2024] [Indexed: 07/03/2024]
Abstract
Latent classification model is a class of statistical methods for identifying unobserved class membership among the study samples using some observed data. In this study, we proposed a latent classification model that takes a censored longitudinal binary outcome variable and uses its changing pattern over time to predict individuals' latent class membership. Assuming the time-dependent outcome variables follow a continuous-time Markov chain, the proposed method has two primary goals: (1) estimate the distribution of the latent classes and predict individuals' class membership, and (2) estimate the class-specific transition rates and rate ratios. To assess the model's performance, we conducted a simulation study and verified that our algorithm produces accurate model estimates (ie, small bias) with reasonable confidence intervals (ie, achieving approximately 95% coverage probability). Furthermore, we compared our model to four other existing latent class models and demonstrated that our approach yields higher prediction accuracies for latent classes. We applied our proposed method to analyze the COVID-19 data in Houston, Texas, US collected between January first 2021 and December 31st 2021. Early reports on the COVID-19 pandemic showed that the severity of a SARS-CoV-2 infection tends to vary greatly by cases. We found that while demographic characteristics explain some of the differences in individuals' experience with COVID-19, some unaccounted-for latent variables were associated with the disease.
Collapse
Affiliation(s)
- Jacky C Kuo
- Department of Biostatistics and Data Science, University of Texas Health Science Center at Houston, Houston, Texas, USA
| | - Wenyaw Chan
- Department of Biostatistics and Data Science, University of Texas Health Science Center at Houston, Houston, Texas, USA
| | - Luis Leon-Novelo
- Department of Biostatistics and Data Science, University of Texas Health Science Center at Houston, Houston, Texas, USA
| | - David R Lairson
- Department of Management, Policy and Community Health, University of Texas Health Science Center at Houston, Houston, Texas, USA
| | - Armand Brown
- Bureau of Epidemiology, Houston Health Department, Houston, Texas, USA
| | - Kayo Fujimoto
- Department of Health Promotion and Behaviroal Sciences, University of Texas Health Science Center at Houston, Houston, Texas, USA
| |
Collapse
|
2
|
Allen A, Magee R, Devaney R, Ardis T, McNally C, McCormick C, Presho E, Doyle M, Ranasinghe P, Johnston P, Kirke R, Harwood R, Farrell D, Kenny K, Smith J, Gordon S, Ford T, Thompson S, Wright L, Jones K, Prodohl P, Skuce R. Whole-Genome sequencing in routine Mycobacterium bovis epidemiology - scoping the potential. Microb Genom 2024; 10:001185. [PMID: 38354031 PMCID: PMC10926703 DOI: 10.1099/mgen.0.001185] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/23/2023] [Accepted: 01/09/2024] [Indexed: 02/16/2024] Open
Abstract
Mycobacterium bovis the main agent of bovine tuberculosis (bTB), presents as a series of spatially-localised micro-epidemics across landscapes. Classical molecular typing methods applied to these micro-epidemics, based on genotyping a few variable loci, have significantly improved our understanding of potential epidemiological links between outbreaks. However, they have limited utility owing to low resolution. Conversely, whole-genome sequencing (WGS) provides the highest resolution data available for molecular epidemiology, producing richer outbreak tracing, insights into phylogeography and epidemic evolutionary history. We illustrate these advantages by focusing on a common single lineage of M. bovis (1.140) from Northern Ireland. Specifically, we investigate the spatial sub-structure of 20 years of herd-level multi locus VNTR analysis (MLVA) surveillance data and WGS data from a down sampled subset of isolates of this MLVA type over the same time frame. We mapped 2108 isolate locations of MLVA type 1.140 over the years 2000-2022. We also mapped the locations of 148 contemporary WGS isolates from this lineage, over a similar geographic range, stratifying by single nucleotide polymorphism (SNP) relatedness cut-offs of 15 SNPs. We determined a putative core range for the 1.140 MLVA type and SNP-defined sequence clusters using a 50 % kernel density estimate, using cattle movement data to inform on likely sources of WGS isolates found outside of core ranges. Finally, we applied Bayesian phylogenetic methods to investigate past population history and reproductive number of the 1.140 M. bovis lineage. We demonstrate that WGS SNP-defined clusters exhibit smaller core ranges than the established MLVA type - facilitating superior disease tracing. We also demonstrate the superior functionality of WGS data in determining how this lineage was disseminated across the landscape, likely via cattle movement and to infer how its effective population size and reproductive number has been in flux since its emergence. These initial findings highlight the potential of WGS data for routine monitoring of bTB outbreaks.
Collapse
Affiliation(s)
- Adrian Allen
- Agrifood and Biosciences Institute, Veterinary Sciences Division, Belfast, UK
| | - Ryan Magee
- Queen’s University Belfast, school of Biological Sciences, UK
| | - Ryan Devaney
- Agrifood and Biosciences Institute, Veterinary Sciences Division, Belfast, UK
| | - Tara Ardis
- Agrifood and Biosciences Institute, Veterinary Sciences Division, Belfast, UK
| | - Caitlín McNally
- Agrifood and Biosciences Institute, Veterinary Sciences Division, Belfast, UK
| | - Carl McCormick
- Agrifood and Biosciences Institute, Veterinary Sciences Division, Belfast, UK
| | - Eleanor Presho
- Agrifood and Biosciences Institute, Veterinary Sciences Division, Belfast, UK
| | - Michael Doyle
- Agrifood and Biosciences Institute, Veterinary Sciences Division, Belfast, UK
| | - Purnika Ranasinghe
- Agrifood and Biosciences Institute, Veterinary Sciences Division, Belfast, UK
| | - Philip Johnston
- Department of Agriculture, Environment and Rural Affairs for Northern Ireland, Belfast, UK
| | - Raymond Kirke
- Department of Agriculture, Environment and Rural Affairs for Northern Ireland, Belfast, UK
| | - Roland Harwood
- Department of Agriculture, Environment and Rural Affairs for Northern Ireland, Belfast, UK
| | - Damien Farrell
- Central Veterinary Research Laboratory, Kildare, Ireland
- University College Dublin, Dublin, Ireland
| | - Kevin Kenny
- Central Veterinary Research Laboratory, Kildare, Ireland
| | | | | | - Tom Ford
- Agrifood and Biosciences Institute, Veterinary Sciences Division, Belfast, UK
| | - Suzan Thompson
- Agrifood and Biosciences Institute, Veterinary Sciences Division, Belfast, UK
| | - Lorraine Wright
- Agrifood and Biosciences Institute, Veterinary Sciences Division, Belfast, UK
| | - Kerri Jones
- Agrifood and Biosciences Institute, Veterinary Sciences Division, Belfast, UK
| | - Paulo Prodohl
- Queen’s University Belfast, school of Biological Sciences, UK
| | - Robin Skuce
- Agrifood and Biosciences Institute, Veterinary Sciences Division, Belfast, UK
| |
Collapse
|
3
|
Accuracy of Tests for Diagnosis of Animal Tuberculosis: Moving Away from the Golden Calf (and towards Bayesian Models). Transbound Emerg Dis 2023. [DOI: 10.1155/2023/7615716] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/25/2023]
Abstract
The last decades have seen major efforts to develop new and improved tools to maximize our ability to detect tuberculosis-infected animals and advance towards the objective of disease control and ultimately eradication. Nevertheless, there is still uncertainty regarding test performance due to the wide range of specificity and especially sensitivity estimates published in the scientific literature. Here, we performed a systematic review of the literature on studies that evaluated the performance of tuberculosis diagnostic tests used in animals through Bayesian Latent Class Models (BLCMs), which do not require the application of a (fallible) reference procedure to classify animals as infected with tuberculosis or not. BLCM-based sensitivity and specificity estimates deviated from those obtained using a reference procedure for certain antemortem tests: an overall lower sensitivity of skin tests and serology and a higher sensitivity of interferon-gamma (IFN-γ) assays was reported. In the case of postmortem diagnostic tests, sensitivity estimates from BLCMs were similar to estimates from studies based on other methodologies. For specificity, the range of BLCM-based estimates was narrower than those based on a reference test, reaching values close to 100% (but lower in the case of IFN-γ assays). In conclusion, Bayesian methods have been increasingly applied for the evaluation of tuberculosis diagnostic tests in animals, yielding results that differ (sometimes substantially) from previously reported test performance in the literature, particularly for in vivo tests and sensitivity estimates. Newly developed models that allow adjustment for relevant factors (e.g., age, breed, region, and herd size) can contribute to the generation of more unbiased estimates of test performance. Nevertheless, although BLCMs for tuberculosis do not require the use of an imperfect reference procedure and are therefore not influenced by its limited performance, they require careful implementation, and transparent systematic reporting should be the norm.
Collapse
|
4
|
Marquetoux N, Vignes M, Burroughs A, Sumner E, Sawford K, Jones G. Evaluation of the accuracy of the IDvet serological test for Mycoplasma bovis infection in cattle using latent class analysis of paired serum ELISA and quantitative real-time PCR on tonsillar swabs sampled at slaughter. PLoS One 2023; 18:e0285598. [PMID: 37167206 PMCID: PMC10174590 DOI: 10.1371/journal.pone.0285598] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/20/2022] [Accepted: 04/27/2023] [Indexed: 05/13/2023] Open
Abstract
Mycoplasma bovis (Mbovis) was first detected in cattle in New Zealand (NZ) in July 2017. To prevent further spread, NZ launched a world-first National Eradication Programme in May 2018. Existing diagnostic tests for Mbovis have been applied in countries where Mbovis is endemic, for detecting infection following outbreaks of clinical disease. Diagnostic test evaluation (DTE) under NZ conditions was thus required to inform the Programme. We used Bayesian Latent Class Analysis on paired serum ELISA (ID Screen Mycoplasma bovis Indirect from IDvet) and tonsillar swabs (qPCR) for DTE in the absence of a gold standard. Tested samples were collected at slaughter between June 2018 and November 2019, from infected herds depopulated by the Programme. A first set of models evaluated the detection of active infection, i.e. the presence of Mbovis in the host. At a modified serology positivity threshold of SP%> = 90, estimates of animal-level ELISA sensitivity was 72.8% (95% credible interval 68.5%-77.4%), respectively 97.7% (95% credible interval 97.3%-98.1%) for specificity, while the qPCR sensitivity was 45.2% (95% credible interval 41.0%-49.8%), respectively 99.6% (95% credible interval 99.4%-99.8%) for specificity. In a second set of models, prior information about ELISA specificity was obtained from the National Beef Cattle Surveillance Programme, a population theoretically free-or very low prevalence-of Mbovis. These analyses aimed to evaluate the accuracy of the ELISA test targeting prior exposure to Mbovis, rather than active infection. The specificity of the ELISA for detecting exposure to Mbovis was 99.9% (95% credible interval 99.7%-100.0%), hence near perfect at the threshold SP%=90. This specificity estimate, considerably higher than in the first set of models, was equivalent to the manufacturer's estimate. The corresponding ELISA sensitivity estimate was 66.0% (95% credible interval 62.7%-70.7%). These results confirm that the IDvet ELISA test is an appropriate tool for determining exposure and infection status of herds, both to delimit and confirm the absence of Mbovis.
Collapse
Affiliation(s)
- Nelly Marquetoux
- EpiCentre, School of Veterinary Science, Massey University, Palmerston North, New Zealand
| | - Matthieu Vignes
- School of Mathematical and Computational Sciences, Massey University, Palmerston North, New Zealand
| | - Amy Burroughs
- Ministry for Primary Industries New Zealand, Wellington, New Zealand
| | - Emma Sumner
- Ministry for Primary Industries New Zealand, Wellington, New Zealand
| | - Kate Sawford
- Ministry for Primary Industries New Zealand, Wellington, New Zealand
- Kate Sawford Epidemiol Consulting, Callala Bay, NSW, Australia
| | - Geoff Jones
- School of Mathematical and Computational Sciences, Massey University, Palmerston North, New Zealand
| |
Collapse
|
5
|
Bonestroo J, Fall N, van der Voort M, Klaas IC, Hogeveen H, Emanuelson U. Diagnostic properties of milk diversion and farmer-reported mastitis to indicate clinical mastitis status in dairy cows using Bayesian latent class analysis. Livest Sci 2021. [DOI: 10.1016/j.livsci.2021.104698] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/25/2022]
|
6
|
Gertz M, Krieter J. Effect of sample size and length of observation period on the reliability of apparent pig organ lesion prevalence. Prev Vet Med 2021; 188:105258. [PMID: 33453560 DOI: 10.1016/j.prevetmed.2021.105258] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/08/2020] [Revised: 12/11/2020] [Accepted: 12/30/2020] [Indexed: 11/30/2022]
Abstract
Comprehensive identification of on-farm animal-health issues still requires extensive efforts so that in practice such monitoring is applied only sparsely. An appealing approach to improve on-farm animal health and welfare monitoring is the application of organ lesion scoring data from the abattoir as such is instantly available for every commercial farm in Europe. Unfortunately, it is also well-known that organ lesion scoring is often unreliable because results are altered by several non-health-related factors, diluting the validity of lesion scoring prevalence as a proxy for on-farm animal health. However, it is theoretically possible to improve prevalence reliability a-posteriori by application of time-series smoothing. The aim of this paper was therefore to analyse whether it is practically possible to increase apparent prevalence estimation reliability retrospectively using a running average, and, if so, which window length and smallest sample size should be preferred in such an approach. Because no gold standard for direct evaluation of lesion reliability is available for field-data, apparent prevalence reliability had to be approximated using prevalence agreement over time. Results indicate that by raising the number of lesion scores per prevalence estimate, apparent prevalence agreement over time can in general be considerably increased. Based on findings presented, a reasonable threshold for prevalence estimation is given by at least n = 50 lesions per farm/abattoir/time-series segment. Results further suggest that it is necessary to consider differences in prevalence sample size for future monitoring purposes, because prevalences that are estimated on a continuum of different sample sizes put together in one evaluation may induce substantial error in prevalence estimates.
Collapse
Affiliation(s)
- M Gertz
- Institute of Animal Breeding and Husbandry, Kiel University, 24098 Kiel, Germany.
| | - J Krieter
- Institute of Animal Breeding and Husbandry, Kiel University, 24098 Kiel, Germany
| |
Collapse
|
7
|
Benjamin-Chung J, Pilotte N, Ercumen A, Grant JR, Maasch JRMA, Gonzalez AM, Ester AC, Arnold BF, Rahman M, Haque R, Hubbard AE, Luby SP, Williams SA, Colford JM. Comparison of multi-parallel qPCR and double-slide Kato-Katz for detection of soil-transmitted helminth infection among children in rural Bangladesh. PLoS Negl Trop Dis 2020; 14:e0008087. [PMID: 32330127 PMCID: PMC7202662 DOI: 10.1371/journal.pntd.0008087] [Citation(s) in RCA: 23] [Impact Index Per Article: 5.8] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/01/2019] [Revised: 05/06/2020] [Accepted: 01/24/2020] [Indexed: 01/08/2023] Open
Abstract
There is growing interest in local elimination of soil-transmitted helminth (STH) infection in endemic settings. In such settings, highly sensitive diagnostics are needed to detect STH infection. We compared double-slide Kato-Katz, the most commonly used copromicroscopic detection method, to multi-parallel quantitative polymerase chain reaction (qPCR) in 2,799 stool samples from children aged 2–12 years in a setting in rural Bangladesh with predominantly low STH infection intensity. We estimated the sensitivity and specificity of each diagnostic using Bayesian latent class analysis. Compared to double-slide Kato-Katz, STH prevalence using qPCR was almost 3-fold higher for hookworm species and nearly 2-fold higher for Trichuris trichiura. Ascaris lumbricoides prevalence was lower using qPCR, and 26% of samples classified as A. lumbricoides positive by Kato-Katz were negative by qPCR. Amplicon sequencing of the 18S rDNA from 10 samples confirmed that A. lumbricoides was absent in samples classified as positive by Kato-Katz and negative by qPCR. The sensitivity of Kato-Katz was 49% for A. lumbricoides, 32% for hookworm, and 52% for T. trichiura; the sensitivity of qPCR was 79% for A. lumbricoides, 93% for hookworm, and 90% for T. trichiura. Specificity was ≥ 97% for both tests for all STH except for Kato-Katz for A. lumbricoides (specificity = 68%). There were moderate negative, monotonic correlations between qPCR cycle quantification values and eggs per gram quantified by Kato-Katz. While it is widely assumed that double-slide Kato-Katz has few false positives, our results indicate otherwise and highlight inherent limitations of the Kato-Katz technique. qPCR had higher sensitivity than Kato-Katz in this low intensity infection setting. Soil-transmitted helminth infections (STH) (e.g., Ascaris, hookworm, Trichuris) contribute to a large burden of disease among children in low- and middle-income countries. There is increasing interest in implementing large-scale deworming programs to eliminate STH in certain settings. Efforts to monitor whether local elimination has occurred require sensitive diagnostic tests that will not miss positive cases. Kato-Katz, a microscopy-based diagnostic test, has commonly been used to identify STH eggs in stool, but in settings where infection intensity is low, this method frequently misses positive samples because it requires visual identification of small numbers of eggs, and hookworm eggs may degrade prior to visualization. Quantitative polymerase chain reaction (qPCR) is a molecular diagnostic method for detecting STH. It may detect more low intensity infections than Kato-Katz because it identifies STH DNA in stool; DNA can be detected in very small quantities and is less likely to degrade than STH ova. Thus, qPCR is likely to be more accurate than Kato-Katz. This study compared the performance of double-slide Kato-Katz and qPCR using 2,799 stool samples from children aged 2–12 years in a setting in rural Bangladesh with predominantly low STH infection intensity. qPCR was more sensitive than Kato-Katz for hookworm and Trichuris infections. 26% of samples were classified as Ascaris positive by Kato-Katz and negative by qPCR. DNA sequencing of 10 samples confirmed that Ascaris was absent in samples classified as positive by Kato-Katz and negative by qPCR. We conclude that Kato-Katz likely produced false positive results for Ascaris and that qPCR had a higher sensitivity than double-slide Kato-Katz in this low infection intensity setting.
Collapse
Affiliation(s)
- Jade Benjamin-Chung
- Division of Epidemiology & Biostatistics, University of California, Berkeley, Berkeley, California, United States of America
- * E-mail:
| | - Nils Pilotte
- Department of Biological Sciences, Smith College, Northampton, Massachusetts, United States of America
- Molecular and Cellular Biology Program, University of Massachusetts, Amherst, Massachusetts, United States of America
| | - Ayse Ercumen
- Division of Epidemiology & Biostatistics, University of California, Berkeley, Berkeley, California, United States of America
- Department of Forestry and Environmental Resources, North Carolina State University, Raleigh, North Carolina, United States of America
| | - Jessica R. Grant
- Department of Biological Sciences, Smith College, Northampton, Massachusetts, United States of America
| | | | - Andrew M. Gonzalez
- Department of Biological Sciences, Smith College, Northampton, Massachusetts, United States of America
| | - Ashanta C. Ester
- Department of Biological Sciences, Smith College, Northampton, Massachusetts, United States of America
| | - Benjamin F. Arnold
- Division of Epidemiology & Biostatistics, University of California, Berkeley, Berkeley, California, United States of America
| | - Mahbubur Rahman
- Infectious Disease Division, International Centre for Diarrhoeal Disease Research, Bangladesh (icddr,b), Dhaka, Bangladesh
| | - Rashidul Haque
- Infectious Disease Division, International Centre for Diarrhoeal Disease Research, Bangladesh (icddr,b), Dhaka, Bangladesh
| | - Alan E. Hubbard
- Division of Epidemiology & Biostatistics, University of California, Berkeley, Berkeley, California, United States of America
| | - Stephen P. Luby
- Infectious Diseases & Geographic Medicine, Stanford University, Stanford, California, United States of America
| | - Steven A. Williams
- Department of Biological Sciences, Smith College, Northampton, Massachusetts, United States of America
- Molecular and Cellular Biology Program, University of Massachusetts, Amherst, Massachusetts, United States of America
| | - John M. Colford
- Division of Epidemiology & Biostatistics, University of California, Berkeley, Berkeley, California, United States of America
| |
Collapse
|
8
|
Test characteristics of the tuberculin skin test and post-mortem examination for bovine tuberculosis diagnosis in cattle in Northern Ireland estimated by Bayesian latent class analysis with adjustments for covariates. Epidemiol Infect 2020; 147:e209. [PMID: 31364540 PMCID: PMC6624860 DOI: 10.1017/s0950268819000888] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/30/2022] Open
Abstract
The single intradermal comparative cervical tuberculin (SICCT) test and post-mortem examination are the main diagnostic tools for bovine tuberculosis (bTB) in cattle in the British Isles. Latent class modelling is often used to estimate the bTB test characteristics due to the absence of a gold standard. However, the reported sensitivity of especially the SICCT test has shown a lot of variation. We applied both the Hui–Walter latent class model under the Bayesian framework and the Bayesian model specified at the animal level, including various risk factors as predictors, to estimate the SICCT test and post-mortem test characteristics. Data were collected from all cattle slaughtered in abattoirs in Northern Ireland in 2015. Both models showed comparable posterior median estimation for the sensitivity of the SICCT test (88.61% and 90.56%, respectively) using standard interpretation and for post-mortem examination (53.65% and 53.79%, respectively). Both models showed almost identical posterior median estimates for the specificity (99.99% vs. 99.80% for SICCT test at standard interpretation and 99.66% vs. 99.86% for post-mortem examination). The animal-level model showed slightly narrower posterior 95% credible intervals. Notably, this study was carried out in slaughtered cattle which may not be representative for the general cattle population.
Collapse
|
9
|
Umemneku Chikere CM, Wilson K, Graziadio S, Vale L, Allen AJ. Diagnostic test evaluation methodology: A systematic review of methods employed to evaluate diagnostic tests in the absence of gold standard - An update. PLoS One 2019; 14:e0223832. [PMID: 31603953 PMCID: PMC6788703 DOI: 10.1371/journal.pone.0223832] [Citation(s) in RCA: 97] [Impact Index Per Article: 19.4] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/03/2019] [Accepted: 09/29/2019] [Indexed: 12/29/2022] Open
Abstract
OBJECTIVE To systematically review methods developed and employed to evaluate the diagnostic accuracy of medical test when there is a missing or no gold standard. STUDY DESIGN AND SETTINGS Articles that proposed or applied any methods to evaluate the diagnostic accuracy of medical test(s) in the absence of gold standard were reviewed. The protocol for this review was registered in PROSPERO (CRD42018089349). RESULTS Identified methods were classified into four main groups: methods employed when there is a missing gold standard; correction methods (which make adjustment for an imperfect reference standard with known diagnostic accuracy measures); methods employed to evaluate a medical test using multiple imperfect reference standards; and other methods, like agreement studies, and a mixed group of alternative study designs. Fifty-one statistical methods were identified from the review that were developed to evaluate medical test(s) when the true disease status of some participants is unverified with the gold standard. Seven correction methods were identified and four methods were identified to evaluate medical test(s) using multiple imperfect reference standards. Flow-diagrams were developed to guide the selection of appropriate methods. CONCLUSION Various methods have been proposed to evaluate medical test(s) in the absence of a gold standard for some or all participants in a diagnostic accuracy study. These methods depend on the availability of the gold standard, its' application to the participants in the study and the availability of alternative reference standard(s). The clinical application of some of these methods, especially methods developed when there is missing gold standard is however limited. This may be due to the complexity of these methods and/or a disconnection between the fields of expertise of those who develop (e.g. mathematicians) and those who employ the methods (e.g. clinical researchers). This review aims to help close this gap with our classification and guidance tools.
Collapse
Affiliation(s)
- Chinyereugo M. Umemneku Chikere
- Institute of Health & Society, Faculty of Medical Sciences Newcastle University, Newcastle upon Tyne, England, United Kingdom
| | - Kevin Wilson
- School of Mathematics, Statistics and Physics, Newcastle University, Newcastle upon Tyne, England, United Kingdom
| | - Sara Graziadio
- National Institute for Health Research, Newcastle In Vitro Diagnostics Co-operative, Newcastle upon Tyne Hospitals National Health Services Foundation Trust, Newcastle upon Tyne, England, United Kingdom
| | - Luke Vale
- Institute of Health & Society, Faculty of Medical Sciences Newcastle University, Newcastle upon Tyne, England, United Kingdom
| | - A. Joy Allen
- National Institute for Health Research, Newcastle In Vitro Diagnostics Co-operative, Newcastle University, Newcastle upon Tyne, England, United Kingdom
| |
Collapse
|
10
|
Patel AB, Kurhe K, Prakash A, Bhargav S, Parepalli S, Fogleman EV, Moore JL, Wallace DD, Kulkarni H, Hibberd PL. Early Identification of Preterm Neonates at Birth With a Tablet App for the Simplified Gestational Age Score (T-SGAS) When Ultrasound Gestational Age Dating Is Unavailable: Protocol for a Validation Study. JMIR Res Protoc 2019; 8:e11913. [PMID: 30860484 PMCID: PMC6434403 DOI: 10.2196/11913] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/21/2018] [Revised: 11/19/2018] [Accepted: 11/22/2018] [Indexed: 11/13/2022] Open
Abstract
Background Although rates of preterm birth continue to increase globally, identification of preterm from low birth weight infants remains a challenge. The burden of low birth weight vs preterm is greatest in resource-limited settings, where gestational age (GA) prior to delivery is frequently not known because ultrasound in early pregnancy is not available and estimates of the date of the mother’s last menstrual period (LMP) may not be reliable. An alternative option is to assess GA at birth to optimize referral and care of preterm newborns. We previously developed and pilot-tested a system to measure the simplified gestational age score (SGAS) based on 4 easily observable neonatal characteristics. Objective The objective of this study is to adapt the scoring system as a tablet app (potentially scalable approach) to assess feasibility of use and to validate whether the scoring system accurately predicts prematurity by itself, over and above birth weight in a large sample of newborns. Methods The study is based in Nagpur, India, at the Research Unit of the National Institute of Child Health and Human Development’s Global Network for Women’s and Children’s Health Research. The Android tablet app for the SGAS (T-SGAS) displays de-identified photographs of skin, breasts, and genitalia across a range of GAs and line drawings of infant posture. Each item is associated with a score. The user is trained to choose the photograph or line drawing that most closely matches the newborn being evaluated, and the app determines the neonate’s GA category (preterm or term) from the cumulative score. The validation study will be conducted in 3 second level care facilities (most deliveries in India occur in hospitals, and women known to be at risk of preterm birth are referred to second level care facilities). Within 24 hours of delivery, women and their babies who are stable will be enrolled in the study. Two auxiliary nurse midwives (ANMs) blinded to prior GA assessments will use the T-SGAS to estimate the GA status of the newborn. An independent data collector will abstract the GA from the ultrasound recorded in the hospital chart and record the date of the mother’s LMP. Eligibility for analysis is determined by the ultrasound and LMP data being collected within 1 week of each other to have a rigorous assessment of true GA. Results Publication of the results of the study is anticipated in 2019. Conclusions Until GA dating by ultrasound is universally available and easy to use in resource-limited settings, and where there are restrictions on ultrasound use due to their use for sex determination and abortion of female fetuses, this study will determine whether the T-SGAS app can accurately assess GA in risk categories at birth. Trial Registration ClinicalTrials.gov NCT02408783; https://clinicaltrials.gov/ct2/show/NCT02408783 (Archived by Webcite at http://www.webcitation.org/75S2kmr3T) International Registered Report Identifier (IRRID) DERR1-10.2196/11913
Collapse
Affiliation(s)
- Archana B Patel
- Lata Medical Research Foundation, Nagpur, Maharashtra, India
| | - Kunal Kurhe
- Lata Medical Research Foundation, Nagpur, Maharashtra, India
| | - Amber Prakash
- Lata Medical Research Foundation, Nagpur, Maharashtra, India
| | - Savita Bhargav
- Lata Medical Research Foundation, Nagpur, Maharashtra, India
| | | | | | - Janet L Moore
- RTI International, Research Triangle Park, NC, United States
| | | | - Hemant Kulkarni
- Lata Medical Research Foundation, Nagpur, Maharashtra, India.,M&H Research, LLC, San Antonio, TX, United States
| | - Patricia L Hibberd
- Department of Global Health, Boston University School of Public Health, Boston, MA, United States
| |
Collapse
|
11
|
de la Cruz ML, Branscum AJ, Nacar J, Pages E, Pozo P, Perez A, Grau A, Saez JL, de Juan L, Diaz R, Minguez O, Alvarez J. Evaluation of the Performance of the IDvet IFN-Gamma Test for Diagnosis of Bovine Tuberculosis in Spain. Front Vet Sci 2018; 5:229. [PMID: 30320129 PMCID: PMC6171474 DOI: 10.3389/fvets.2018.00229] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/06/2018] [Accepted: 09/04/2018] [Indexed: 01/03/2023] Open
Abstract
In Spain, the national bovine tuberculosis (bTB) eradication program is based on yearly skin testing of every ≥6 weeks old animal using the single or comparative tuberculin test and parallel use of the interferon-gamma (IFN-γ) assay as an ancillary diagnostic test in infected herds. There are several versions of the latter. Recently, a new commercial IDvet IFN-γ assay has been authorized for use in the program, but there is limited scientific evidence about its performance in different epidemiological settings. Therefore, two studies to evaluate the performance of the IDvet assay were conducted. In study 1, a concordance analysis between the new IDvet and the Bovigam IFN-γ assay in use in Spain for over 10 years was conducted. In study 2, results from the IDvet assay when applied in tandem with a single intradermal tuberculin (SIT) test were used to evaluate the concordance between both tests and to estimate their sensitivity (Se) and specificity (Sp) using a Bayesian latent-class model. Field data from cattle herds located in Madrid and Castilla y Leon (Spain) were collected. For study 1, herd selection was based on a high expected prevalence of reactors to the IFN-γ assay, while herds were selected at random to estimate Se and Sp of the new IDvet assay in study 2. Agreement between the results obtained with both kits for IFN-γ assay was poor (Kappa = 0.20), and a receiver operating characteristic (ROC) analysis indicated a low Se of the new IDvet relative to the Bovigam in a heavily bTB infected population. The Bayesian latent-class analysis estimated the Se of the IDvet assay to be 36.7% [95% probability posterior interval (PPI) 14.7-78.8%] with estimated Sp close to 100% when the cut-off recommended by the manufacturer (35) was applied. At the alternative cut-off values of 16 and 4, the estimated Se of the IDvet assay increased to 49.0% (PPI: 24.8-94.1%) and 56.0% (PPI: 30.8-96.3%), respectively, while maintaining a high specificity. The results suggest that the new IDvet assay may have lower sensitivity than the Bovigam for diagnosis of bTB in cattle herds in Spain, and that adjusting its cut-off might be considered.
Collapse
Affiliation(s)
| | - Adam J. Branscum
- Biostatistics Program, Oregon State University, Corvallis, OR, United States
| | - Jesus Nacar
- Dirección General de Producción Agropecuaria e Infraestructuras Agrarias, Consejería de Agricultura y Ganadería de la Junta de Castilla y León, Valladolid, Spain
| | - Enrique Pages
- Área de Ganadería, Dirección General de Medio Ambiente, Consejería de Medio Ambiente, Vivienda y Ordenación del Territorio de la Comunidad de Madrid, Madrid, Spain
| | - Pilar Pozo
- VISAVET Health Surveillance Centre, Universidad Complutense de Madrid, Madrid, Spain
- MAEVA SERVET, S.L., Madrid, Spain
| | - Andres Perez
- Department of Veterinary Population Medicine, College of Veterinary Medicine, University of Minnesota, Saint Paul, MN, United States
| | - Anna Grau
- Dirección General de Producción Agropecuaria e Infraestructuras Agrarias, Consejería de Agricultura y Ganadería de la Junta de Castilla y León, Valladolid, Spain
| | - Jose Luis Saez
- Ministerio de Agricultura y Pesca, Alimentación y Medio Ambiente, Madrid, Spain
| | - Lucia de Juan
- VISAVET Health Surveillance Centre, Universidad Complutense de Madrid, Madrid, Spain
- Departamento de Sanidad Animal, Facultad de Veterinaria, Universidad Complutense de Madrid, Madrid, Spain
| | - Rosa Diaz
- Área de Ganadería, Dirección General de Medio Ambiente, Consejería de Medio Ambiente, Vivienda y Ordenación del Territorio de la Comunidad de Madrid, Madrid, Spain
| | - Olga Minguez
- Dirección General de Producción Agropecuaria e Infraestructuras Agrarias, Consejería de Agricultura y Ganadería de la Junta de Castilla y León, Valladolid, Spain
| | - Julio Alvarez
- VISAVET Health Surveillance Centre, Universidad Complutense de Madrid, Madrid, Spain
- Department of Veterinary Population Medicine, College of Veterinary Medicine, University of Minnesota, Saint Paul, MN, United States
- Departamento de Sanidad Animal, Facultad de Veterinaria, Universidad Complutense de Madrid, Madrid, Spain
| |
Collapse
|
12
|
Lesko CR, Keil AP, Moore RD, Chander G, Fojo AT, Lau B. Measurement of Current Substance Use in a Cohort of HIV-Infected Persons in Continuity HIV Care, 2007-2015. Am J Epidemiol 2018; 187:1970-1979. [PMID: 29701832 PMCID: PMC6118067 DOI: 10.1093/aje/kwy092] [Citation(s) in RCA: 16] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/23/2017] [Revised: 04/18/2018] [Accepted: 04/18/2018] [Indexed: 01/02/2023] Open
Abstract
Accurate, routine measurement of recent illicit substance use is challenging. The Johns Hopkins Human Immunodeficiency Virus Clinical Cohort (Baltimore, Maryland) collects 2 imperfect but routine measurements of recent substance use: medical record review and self-interview. We used Bayesian latent class modeling to estimate sensitivity and specificity of each measurement as well as prevalence of substance use among 2,064 patients engaged in care during 2007-2015. Sensitivity of medical record review was higher than sensitivity of self-interview for cocaine and heroin use; posterior estimates ranged from 44% to 76% for cocaine use and from 39% to 67% for heroin use, depending on model assumptions and priors. In contrast, sensitivity of self-interview was higher than sensitivity of medical record review for any alcohol use, hazardous alcohol use, and cigarette smoking. Posterior estimates of sensitivity of self-interview were generally above 80%, 85%, and 87% for each substance, respectively. Specificity was high for all measurements. From one model, we estimated prevalence of substance use in the cohort to be 12.5% for cocaine, 9.3% for heroin, 48.5% for alcohol, 21.4% for hazardous alcohol, and 55.4% for cigarettes. Prevalence estimates from other models were generally comparable. Measurement error of substance use is nontrivial and should be accounted for in subsequent analyses.
Collapse
Affiliation(s)
- Catherine R Lesko
- Department of Epidemiology, Bloomberg School of Public Health, Johns Hopkins University, Baltimore, Maryland
| | - Alexander P Keil
- Department of Epidemiology, University of North Carolina, Chapel Hill, Chapel Hill, North Carolina
| | - Richard D Moore
- Department of Epidemiology, Bloomberg School of Public Health, Johns Hopkins University, Baltimore, Maryland
- Department of Medicine, School of Medicine, Johns Hopkins University, Baltimore, Maryland
| | - Geetanjali Chander
- Department of Medicine, School of Medicine, Johns Hopkins University, Baltimore, Maryland
| | - Anthony T Fojo
- Department of Medicine, School of Medicine, Johns Hopkins University, Baltimore, Maryland
| | - Bryan Lau
- Department of Epidemiology, Bloomberg School of Public Health, Johns Hopkins University, Baltimore, Maryland
- Department of Medicine, School of Medicine, Johns Hopkins University, Baltimore, Maryland
| |
Collapse
|
13
|
Bayesian latent class estimation of sensitivity and specificity parameters of diagnostic tests for bovine tuberculosis in chronically infected herds in Northern Ireland. Vet J 2018; 238:15-21. [DOI: 10.1016/j.tvjl.2018.04.019] [Citation(s) in RCA: 40] [Impact Index Per Article: 6.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/28/2017] [Revised: 04/25/2018] [Accepted: 04/26/2018] [Indexed: 01/19/2023]
|
14
|
Mathevon Y, Foucras G, Falguières R, Corbiere F. Estimation of the sensitivity and specificity of two serum ELISAs and one fecal qPCR for diagnosis of paratuberculosis in sub-clinically infected young-adult French sheep using latent class Bayesian modeling. BMC Vet Res 2017; 13:230. [PMID: 28774299 PMCID: PMC5543559 DOI: 10.1186/s12917-017-1145-x] [Citation(s) in RCA: 19] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/14/2017] [Accepted: 07/19/2017] [Indexed: 11/10/2022] Open
Abstract
Background The objective was to evaluate the diagnostic accuracy of two serum ELISAs and one quantitative PCR on feces for the diagnosis of paratuberculosis in sub-clinically infected young-adult sheep. A cross-sectional study was performed to collect 1197 individual blood and fecal samples from 2- to 3-year-old sub-clinically infected ewes in 14 closed meat sheep flocks in France. Fecal excretion was determined using qPCR based on IS900 sequence detection, and serology was performed on serum samples using two commercial ELISAs. Data were analyzed in a 3-test multiple-population Bayesian latent class model accounting for potential dependence between the three tests fitted in OpenBUGS. Separate analyses were performed according to whether doubtful ELISA results were handled as positive or negative and based on two thresholds for fecal qPCR (Ct ≤ 42 or Ct ≤ 40). Results The best fit to the data was provided by accounting for a pairwise dependence between the two ELISAs on sensitivity and pairwise dependence between the three tests on specificity. Under this model, the estimated ELISA sensitivities were 17.4% (95% PCI: 10.6 – 25.9) and 17.9% (95% PCI 11.4 – 25.6), with estimated specificities of 94.8% (95% PCI: 93.1 – 96.3) and 94.0% (95% PCI: 92.2 – 95.7). Fecal qPCR demonstrated significantly higher sensitivity (47.5%; 95% PCI: 29.3 – 69.9) and specificity (99.0%; 95% PCI: 97.9 – 99.9) than the ELISAs. Assumptions regarding doubtful ELISA results and qPCR thresholds had only a slight impact on test accuracy estimates. Models not accounting for pairwise dependence between ELISA and fecal qPCR results yielded higher sensitivity and specificity estimates but always provided a worse fit to the data. Conclusions Although the overall sensitivity of serum ELISAs and fecal qPCR remains low, the higher diagnostic performances of fecal qPCR make it more suitable for paratuberculosis diagnosis in sub-clinically infected sheep. Our results also illustrate that all dependence structures should be investigated when evaluating diagnostic test accuracy and selection based on a rigorous statistical approach. Electronic supplementary material The online version of this article (doi:10.1186/s12917-017-1145-x) contains supplementary material, which is available to authorized users.
Collapse
Affiliation(s)
- Yoann Mathevon
- IHAP, Ecole Nationale Vétérinaire de Toulouse, 23 chemin des Capelles, F-31076, Toulouse Cedex 3, France
| | - Gilles Foucras
- IHAP, Ecole Nationale Vétérinaire de Toulouse, 23 chemin des Capelles, F-31076, Toulouse Cedex 3, France
| | - Rémy Falguières
- CAPEL Ovilot, 237 avenue Pierre Semard, 46000, Cahors, France
| | - Fabien Corbiere
- IHAP, Ecole Nationale Vétérinaire de Toulouse, 23 chemin des Capelles, F-31076, Toulouse Cedex 3, France.
| |
Collapse
|
15
|
Abstract
Germany has been officially free of bovine tuberculosis since 1996. However, in the last years there has been an increase of bovine tuberculosis cases, particularly in the southern part of Germany, in the Allgäu region. As a consequence a one-time tuberculosis surveillance program was revisited with different premortal and postmortal tests. The aim of this paper was to estimate diagnostic sensitivities and specificities of the different tests used within this surveillance program. In the absence of a perfect test with 100% sensitivity and 100% specificity, thus in the absence of a gold standard, a Bayesian latent class approach with two different datasets was performed. The first dataset included 389 animals, tested with single intra-dermal comparative cervical tuberculin (SICCT) test, PCR and pathology; the second dataset contained 175 animals, tested with single intra-dermal cervical tuberculin (SICT) test, Bovigam® assay, pathology and culture. Two-way conditional dependencies were considered within the models. Additionally, inter-laboratory agreement (five officially approved laboratories) of the Bovigam® assay was assessed with Cohen's kappa test (21 blood samples). The results are given in posterior means and 95% credibility intervals. The specificities of the SICT test, SICCT test, PCR and pathology ranged between 75.8% [68.8-82.2%] and 99.0% [96.8-100%]. The Bovigam® assay stood out with a very low specificity (6.9% [3.6-11.1%]), though it had the highest sensitivity (95.7% [91.3-99.2%]). The sensitivities of the SICCT test, PCR, SICT test, pathology and culture varied from 57.8% [48.0-67.6%] to 88.9% [65.5-99.7%]. The prevalences were 19.8% [14.6-26.5%] (three-test dataset) and 7.7% [4.2-12.3%] (four-test dataset). Among all pairwise comparisons the highest agreement was 0.62 [0.15-1]). In conclusion, the specificity of the Bovigam® assay and the inter-laboratory agreement were lower than expected.
Collapse
|
16
|
Kostoulas P, Nielsen SS, Branscum AJ, Johnson WO, Dendukuri N, Dhand NK, Toft N, Gardner IA. STARD-BLCM: Standards for the Reporting of Diagnostic accuracy studies that use Bayesian Latent Class Models. Prev Vet Med 2017; 138:37-47. [PMID: 28237234 DOI: 10.1016/j.prevetmed.2017.01.006] [Citation(s) in RCA: 140] [Impact Index Per Article: 20.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/01/2016] [Revised: 12/21/2016] [Accepted: 01/09/2017] [Indexed: 11/27/2022]
Abstract
The Standards for the Reporting of Diagnostic Accuracy (STARD) statement, which was recently updated to the STARD2015 statement, was developed to encourage complete and transparent reporting of test accuracy studies. Although STARD principles apply broadly, the checklist is limited to studies designed to evaluate the accuracy of tests when the disease status is determined from a perfect reference procedure or an imperfect one with known measures of test accuracy. However, a reference standard does not always exist, especially in the case of infectious diseases with a long latent period. In such cases, a valid alternative to classical test evaluation involves the use of latent class models that do not require a priori knowledge of disease status. Latent class models have been successfully implemented in a Bayesian framework for over 20 years. The objective of this work was to identify the STARD items that require modification and develop a modified version of STARD for studies that use Bayesian latent class analysis to estimate diagnostic test accuracy in the absence of a reference standard. Examples and elaborations for each of the modified items are provided. The new guidelines, termed STARD-BLCM (Standards for Reporting of Diagnostic accuracy studies that use Bayesian Latent Class Models), will facilitate improved quality of reporting on the design, conduct and results of diagnostic accuracy studies that use Bayesian latent class models.
Collapse
Affiliation(s)
- Polychronis Kostoulas
- Laboratory of Epidemiology, Biostatistics and Animal Health Economics, Faculty of Veterinary Medicine, University of Thessaly, Karditsa GR43100, Greece.
| | - Søren S Nielsen
- Department of Veterinary and Animal Sciences, Faculty of Health and Medical Sciences, University of Copenhagen, Grønnegårdsvej 8, DK-1870 Frederiksberg C, Denmark
| | - Adam J Branscum
- Biostatistics Program, Oregon State University, Corvallis, OR, 97331, USA
| | - Wesley O Johnson
- Department of Statistics, University of California, Irvine, CA, 92697, USA
| | - Nandini Dendukuri
- McGill University Health Centre, McGill University, Montréal, QC, Canada
| | - Navneet K Dhand
- Faculty of Veterinary Science, The University of Sydney, 425 Werombi Road, Camden, 2570 NSW, Australia
| | - Nils Toft
- Technical University of Denmark, National Veterinary Institute, Bülowsvej 27, DK-1870 Frederiksberg C, Denmark
| | - Ian A Gardner
- Department of Health Management, Atlantic Veterinary College, University of Prince Edward Island, Charlottetown, Prince Edward Island C1A4P3, Canada
| |
Collapse
|