Diagnostic test accuracy: application and practice using R software

The objective of this paper is to describe general approaches of diagnostic test accuracy (DTA) that are available for the quantitative synthesis of data using R software. We conduct a DTA that summarizes statistics for univariate analysis and bivariate analysis. The package commands of R software were “metaprop” and “metabin” for sensitivity, specificity, and diagnostic odds ratio; forest for forest plot; reitsma of “mada” for a summarized receiver-operating characteristic (ROC) curve; and “metareg” for meta-regression analysis. The estimated total effect sizes, test for heterogeneity and moderator effect, and a summarized ROC curve are reported using R software. In particular, we focus on how to calculate the effect sizes of target studies in DTA. This study focuses on the practical methods of DTA rather than theoretical concepts for researchers whose fields of study were non-statistics related. By performing this study, we hope that many researchers will use R software to determine the DTA more easily, and that there will be greater interest in related research.

Testing likelihood ratios produced from complex DNA profiles

The performance of any model used to analyse DNA profile evidence should be tested using simulation, large scale validation studies based on ground-truth cases, or alignment with trends predicted by theory. We investigate a number of diagnostics to assess the performance of the model using Hd true tests. Of particular focus in this work is the proportion of comparisons to non-contributors that yield a likelihood ratio (LR) higher than or equal to the likelihood ratio of a known contributor (LRPOI), designated as p, and the average LR for Hd true tests. Theory predicts that p should always be less than or equal to 1/LRPOI and hence the observation of this in any particular case is of limited use. A better diagnostic is the average LR for Hd true which should be near to 1. We test the performance of a continuous interpretation model on nine DNA profiles of varying quality and complexity and verify the theoretical expectations.

Isotopic and elemental profiling of ammonium nitrate in forensic explosives investigations

Ammonium nitrate (AN) is frequently encountered in explosives in forensic casework. It is widely available as fertilizer and easy to implement in explosive devices, for example by mixing it with a fuel. Forensic profiling methods to determine whether material found on a crime scene and material retrieved from a suspect arise from the same source are becoming increasingly important. In this work, we have explored the possibility of using isotopic and elemental profiling to discriminate between different batches of AN. Variations within a production batch, between different batches from the same manufacturer, and between batches from different manufacturers were studied using a total of 103 samples from 19 different fertilizer manufacturers. Isotope-ratio mass spectrometry (IRMS) was used to analyze AN samples for their (15)N and (18)O isotopic composition. The trace-elemental composition of these samples was studied using inductively coupled plasma-mass spectrometry (ICP-MS). All samples were analyzed for the occurrence of 66 elements. 32 of these elements were useful for the differentiation of AN samples. These include magnesium (Mg), calcium (Ca), iron (Fe) and strontium (Sr). Samples with a similar elemental profile may be differentiated based on their isotopic composition. Linear discriminant analysis (LDA) was used to calculate likelihood ratios and demonstrated the power of combining elemental and isotopic profiling for discrimination between different sources of AN.

Compbdt: an R program to compare two binary diagnostic tests subject to a paired design

Background

The comparison of the performance of two binary diagnostic tests is an important topic in Clinical Medicine. The most frequent type of sample design to compare two binary diagnostic tests is the paired design. This design consists of applying the two binary diagnostic tests to all of the individuals in a random sample, where the disease status of each individual is known through the application of a gold standard. This article presents an R program to compare parameters of two binary tests subject to a paired design.

Results

The “compbdt” program estimates the sensitivity and the specificity, the likelihood ratios and the predictive values of each diagnostic test applying the confidence intervals with the best asymptotic performance. The program compares the sensitivities and specificities of the two diagnostic tests simultaneously, as well as the likelihood ratios and the predictive values, applying the global hypothesis tests with the best performance in terms of type I error and power. When the global hypothesis test is significant, the causes of the significance are investigated solving the individual hypothesis tests and applying the multiple comparison method of Holm. The most optimal confidence intervals are also calculated for the difference or ratio between the respective parameters. Based on the data observed in the sample, the program also estimates the probability of making a type II error if the null hypothesis is not rejected, or estimates the power if the if the alternative hypothesis is accepted. The “compbdt” program provides all the necessary results so that the researcher can easily interpret them. The estimation of the probability of making a type II error allows the researcher to decide about the reliability of the null hypothesis when this hypothesis is not rejected. The “compbdt” program has been applied to a real example on the diagnosis of coronary artery disease.

Conclusions

The “compbdt” program is one which is easy to use and allows the researcher to compare the most important parameters of two binary tests subject to a paired design. The “compbdt” program is available as supplementary material.

Complex DNA mixture analysis in a forensic context: evaluating the probative value using a likelihood ratio model

The interpretation of mixed DNA profiles obtained from low template DNA samples has proven to be a particularly difficult task in forensic casework. Newly developed likelihood ratio (LR) models that account for PCR-related stochastic effects, such as allelic drop-out, drop-in and stutters, have enabled the analysis of complex cases that would otherwise have been reported as inconclusive. In such samples, there are uncertainties about the number of contributors, and the correct sets of propositions to consider. Using experimental samples, where the genotypes of the donors are known, we evaluated the feasibility and the relevance of the interpretation of high order mixtures, of three, four and five donors. The relative risks of analyzing high order mixtures of three, four, and five donors, were established by comparison of a 'gold standard' LR, to the LR that would be obtained in casework. The 'gold standard' LR is the ideal LR: since the genotypes and number of contributors are known, it follows that the parameters needed to compute the LR can be determined per contributor. The 'casework LR' was calculated as used in standard practice, where unknown donors are assumed; the parameters were estimated from the available data. Both LRs were calculated using the basic standard model, also termed the drop-out/drop-in model, implemented in the LRmix module of the R package Forensim. We show how our results furthered the understanding of the relevance of analyzing high order mixtures in a forensic context. Limitations are highlighted, and it is illustrated how our study serves as a guide to implement likelihood ratio interpretation of complex DNA profiles in forensic casework.

Perception problems of the verbal scale: A reanalysis and application of a membership function approach

Verbal conclusion scales provide a standardized vocabulary that forensic scientists can adopt to describe the amount of support offered by a set of observations with regard to two competing hypotheses. The extent to which these verbal scales can efficiently and accurately communicate strength of support to lay evaluators is, however, an empirical matter of considerable importance. The aim of this paper was to reexamine the results of a recent study measuring lay interpretations of expert verbal phrases (Mullen et al., 2014) and to further improve upon those estimates through the utilization of a membership function approach. Across both the reexamination (n=400) and the new experiment (n=134) 534 participants provided translations of expert verbal conclusion scales used by forensic scientists. Overall, there is compelling evidence that the correspondence between expert intentions and lay interpretations is low, while the potential for miscommunication is high. Consequently, further attention is required to facilitate the development of valid and reliable verbal conclusion scales which clearly communicate expert evaluative opinions.

The Hypothesis-Driven Physical Examination

The physical examination remains a vital part of the clinical encounter. However, physical examination skills have declined in recent years, in part because of decreased time at the bedside. Many clinicians question the relevance of physical examinations in the age of technology. A hypothesis-driven approach to teaching and practicing the physical examination emphasizes the performance of maneuvers that can alter the likelihood of disease. Likelihood ratios are diagnostic weights that allow clinicians to estimate the post-probability of disease. This hypothesis-driven approach to the physical examination increases its value and efficiency, while preserving its cultural role in the patient-physician relationship.

Exact computation of the distribution of likelihood ratios with forensic applications

If complex DNA profiles, conditioned on multiple individuals are evaluated, it may be difficult to assess the strength of the evidence based on the likelihood ratio. A likelihood ratio does not give information about the relative weights that are provided by separate contributors. Alternatively, the observed likelihood ratio can be evaluated with respect to the distribution of the likelihood ratio under the defense hypothesis. We present an efficient algorithm to compute an exact distribution of likelihood ratios that can be applied to any LR-based model. The distribution may have several applications, but is used here to compute a p-value that corresponds to the observed likelihood ratio. The p-value is the probability that a profile under the defense hypothesis, substituted for a questioned contributor e.g. suspect, would attain a likelihood ratio which is at least the same magnitude as that observed. The p-value can be thought of as a scaled version of the likelihood ratio, giving a quantitative measure of the strength of the evidence relative to the specified hypotheses and the model used for the analysis. The algorithm is demonstrated on examples based on real data. R code for the algorithm is freely available in the R package euroMix.

Accurate assessment of the weight of evidence for DNA mixtures by integrating the likelihood ratio

Several methods exist for weight of evidence calculations on DNA mixtures. Especially if dropout is a possibility, it may be difficult to estimate mixture specific parameters needed for the evaluation. For semi-continuous models, the LR for a person to have contributed to a mixture depends on the specified number of contributors and the probability of dropout for each. We show here that, for the semi-continuous model that we consider, the weight of evidence can be accurately obtained by applying the standard statistical technique of integrating the likelihood ratio against the parameter likelihoods obtained from the mixture data. This method takes into account all likelihood ratios belonging to every choice of parameters, but LR's belonging to parameters that provide a better explanation to the mixture data put in more weight into the final result. We therefore avoid having to estimate the number of contributors or their probabilities of dropout, and let the whole evaluation depend on the mixture data and the allele frequencies, which is a practical advantage as well as a gain in objectivity. Using simulated mixtures, we compare the LR obtained in this way with the best informed LR, i.e., the LR using the parameters that were used to generate the data, and show that results obtained by integration of the LR approximate closely these ideal values. We investigate both contributors and non-contributors for mixtures with various numbers of contributors. For contributors we always obtain a result close to the best informed LR whereas non-contributors are excluded more strongly if a smaller dropout probability is imposed for them. The results therefore naturally lead us to reconsider what we mean by a contributor, or by the number of contributors.

Sense and sensibility: on the diagnostic value of control chart rules for detection of shifts in time series data

Background

The aim of this study was to quantify and compare the diagnostic value of The Western Electric (WE) statistical process control (SPC) chart rules and the Anhoej rules for detection of non-random variation in time series data in order to make recommendations for their application in practice.

Methods

SPC charts are point-and-line graphs showing a measure over time and employing statistical tests for identification of non-random variation.

In this study we used simulated time series data with and without non-random variation introduced as shifts in process centre over time. The primary outcome was likelihood ratios of combined tests. Likelihood ratios are useful measures of a test’s ability to discriminate between the true presence or absence of a specific condition.

Results

With short data series (10 data points), the WE rules 1–4 combined and the Anhoej rules alone or combined with WE rule 1 perform well for identifying or excluding persistent shifts in the order of 2 SD. For longer data series, the Anhoej rules alone or in combination with the WE rule 1 seem to perform slightly better than the WE rules combined.

However, the choice of which and how many rules to apply in a given situation should be made deliberately depending on the specific purpose of the SPC analysis and the number of available data points.

Conclusions

Based on these results and our own practical experience, we suggest a stepwise approach to SPC analysis: Start with a run chart using the Anhoej rules and with the median as process centre. If, and only if, the process shows random variation at the desired level, apply the 3-sigma rule in addition to the Anhoej rules using the mean as process centre.

Electronic supplementary material

The online version of this article (10.1186/s12874-018-0564-0) contains supplementary material, which is available to authorized users.

Distinguishing between donors and their relatives in complex DNA mixtures with binary models

While likelihood ratio calculations were until the recent past limited to the evaluation of mixtures in which all alleles of all donors are present in the DNA mixture profile, more recent methods are able to deal with allelic dropout and drop-in. This opens up the possibility to obtain likelihood ratios for mixtures where this was not previously possible, but it also means that a full match between the alleged contributor and the crime stain is no longer necessary. We investigate in this article what the consequences are for relatives of the actual donors, because they typically share more alleles with the true donor than an unrelated individual. We do this with a semi-continuous binary approach, where the likelihood ratios are based on the observed alleles and the dropout probabilities for each donor, but not on the peak heights themselves. These models are widespread in the forensic community. Since in many cases a simple model is used where a uniform dropout probability is assumed for all (or for all unknown) contributors, we explore the extent to which this alters the false positive probabilities for relatives of donors, compared to what would have been obtained with the correct probabilities of dropout for each donor.

Pairwise relatedness testing in the context of inbreeding: expectation and variance of the likelihood ratio

In this paper we investigate various effects of inbreeding on the likelihood ratio (LR) in forensic kinship testing. The basic setup of such testing involves formulating two competing hypotheses, in the form of pedigrees, describing the relationship between the individuals. The likelihood of each hypothesis is computed given the available genetic data, and a conclusion is reached if the ratio of these exceeds some pre-determined threshold. An important aspect of this approach is that the hypotheses are usually not exhaustive: The true relationship may differ from both of the stated pedigrees. It is well known that this may introduce bias in the test results. Previous work has established formulas for the expected value and variance of the LR, given the two competing hypotheses and the true relationship. However, the proposed method only handles cases without inbreeding. In this paper we extend these results to all possible pairwise relationships. The key ingredient is formulating the hypotheses in terms of Jacquard coefficients instead of the more restricted Cotterman coefficients. While the latter describe the relatedness between outbred individuals, the more general Jacquard coefficients allow any level of inbreeding. Our approach also enables scrutiny of another frequently overlooked source of LR bias, namely background inbreeding. This ubiquitous phenomenon is usually ignored in forensic kinship computations, due to lack of adequate methods and software. By leveraging recent work on pedigrees with inbred founders, we show how background inbreeding can be modeled as a continuous variable, providing easy-to-interpret results in specific cases. For example, we show that if true siblings are subjected to a test for parent-offspring, moderate levels of background inbreeding are expected to inflate the LR by more than 50%.

Diagnostic modalities to determine ventriculoperitoneal shunt malfunction: A systematic review and meta-analysis

BACKGROUND

Ventriculoperitoneal (VP) shunt malfunction is an emergency. Timely diagnosis can be challenging because shunt malfunction often presents with symptoms mimicking other common pediatric conditions.

METHODS

We performed a systematic review and meta-analysis to determine which commonly used imaging modalities; Magnetic resonance imaging (MRI), Computed Tomography (CT), X-ray Shunt series or Optic Nerve Sheath Diameter (ONSD) ultrasound, are superior in evaluating shunt malfunction.

INCLUSION CRITERIA

patients less than 21 years old with symptoms of shunt malfunction. We calculated the pooled sensitivity, specificity, Likelihood Ratios (LR+, LR-) using a random-effects model.

RESULTS

Eight studies were included encompassing 1906 patients with a prevalence of VP shunt malfunction of (29.3%). Shunt series: sensitivity (14%-53%), specificity (99%), LR+ (23.2), and LR- (0.47-0.87). CT scan: sensitivity (53%-100%), specificity (27%-98%), LR+ (1.34-22.87), LR- (0.37). MRI: sensitivity (57%), specificity (93%), LR+ (7.66), and LR- (0.49). ONSD: sensitivity (64%), specificity (22%-68%), LR+ (4.4-8.7), LR- (0.93). A positive shunt series, CT scan, MRI, or ONSD has a post-test probability of (23%-84%). A normal shunt series, CT scan, MRI, or ONSD results in a post-test probability of (7%-31%). A positive shunt series results in a post-test probability of 80%, which is equivalent to the post-test probability of CT scan (23-84%) and MRI (83%).

CONCLUSION

Despite the low sensitivity, a positive shunt series obviates the need for further imaging studies. Prompt referral for neurosurgical intervention is recommended. A negative shunt series or any result (positive or negative) from CT, MRI, or ONSD will still require an emergent neurosurgical referral.

Temperature-Dependent DART-MS Analysis of Sexual Lubricants to Increase Accurate Associations

The analysis of lubricant evidence is a recent development in sexual assault investigations and in the absence of any biological evidence may assist in linking an assailant to the victim or crime scene. An ambient ionization technique, high-resolution direct analysis in real-time mass spectrometry (HR-DART-MS), was employed to characterize a sample set of 33 water-based lubricants. As lubricants are complex multicomponent mixtures, this study investigated if different thermal desorption temperatures could elucidate different additives and provide additional information. A low-temperature, high-temperature, and thermal desorption/pyrolysis DART-MS protocol was used to characterize the water-based lubricant sample set. The strength of the methodologies was evaluated using positive and negative likelihood ratios that were calculated from inter- and intra-pairwise comparisons using Pearson correlation coefficients. The low-temperature DART-MS protocol afforded valuable information pertaining to volatile additives (e.g., flavors and fragrances) and provided positive likelihood ratios that would provide strong support for true positive and negatives than the high-temperature protocol when associating between individual samples and samples to their respective sub-groupings. The thermal desorption/pyrolysis DART analytical protocol provided enhanced differentiation between samples due to the precise temperature control using a thermal gradient. Moreover, the total ion spectra obtained from the thermal desorption/pyrolysis protocol, not only had high positive and negative likelihood ratios, this method also provided the most discrimination as determined by empirical cross entropy plots. Graphical Abstract.

The limitations of kinship determinations using STR data in ill-defined populations

The likelihood ratio (LR) method is commonly used to determine kinship in civil, criminal, or forensic cases. For the past 15 years, our research group has also applied LR to ancient STR data and obtained kinship results for collections of graves or necropolises. Although we were able to reconstruct large genealogies, some pairs of individuals showed ambiguous results. Second-degree relationships, half-sibling pairs for example, were often inconsistent with detected first-degree relationships, such as parent/child or brother/sister pairs. We therefore set about providing empirical estimations of the error rates for the LR method in living populations with STR allelic diversities comparable to that of the ancient populations we had previously studied. We collected biological samples in the field in North-Eastern Siberia and West Africa and studied more than 800 pairs of STR profiles from individuals with known relationships. Because commercial STR panels were constructed for specific regions (namely Europe and North America), their allelic makeup showed a significant deficit in diversity when compared to European populations, replicating a situation often faced in ancient DNA studies. We assessed the capacity of the LR method to confirm known relationships (effectiveness) and its capacity to detect those relationships (reliability). Concerns over the effectiveness of LR determinations are mostly an issue in forensic studies, while the reliability of the detection of kinship is an issue for the study of necropolises or other large gatherings of unidentified individuals, such as disaster victims or mass graves. We show that the application of LR to both test populations highlights specific issues (both false positives and false negatives) that prevent the confirmation of second-degree kinship or even full siblingship in small populations. Up to 29% of detected full sibling relationships were either overestimated half-sibling relationships or underestimated parent-offspring relationships. The error rate for detected half-sibling relationships was even higher, reaching 41%. Only parent-offspring pairs were reliably detected or confirmed. This implies that, in populations that are small, ill-defined, or for which the STR loci analyzed are inappropriate, an examiner might not be able to distinguish a pair of full siblings from a pair of half-siblings. Furthermore, half-sibling pairs might be overlooked altogether, an issue that is exacerbated by the common confusion, in many languages and cultures, between half-siblings and full siblings. Consequently, in the study of ancient populations, human remains of unknown origins, or poorly surveyed modern populations, we recommend a conservative approach to kinship determined by LR. Next-generation sequencing data should be used when possible, but the costs and technology involved might be prohibitive. Therefore, in potentially contentious situations or cases lacking sufficient external information, uniparental markers should be analyzed: ideally, complete mitochondrial genomes and Y-chromosome haplotypes (STR, SNP, and/or sequencing).

Translation and validation of an epilepsy-screening questionnaire in three Nigerian languages

OBJECTIVE

We describe the development, translation and validation of epilepsy-screening questionnaires in the three most popular Nigerian languages: Hausa, Igbo and Yoruba.

METHODS

A 9-item epilepsy-screening questionnaire was developed by modifying previously validated English language questionnaires. Separate multilingual experts forward- and back-translated them to the three target languages. Translations were discussed with fieldworkers and community members for ethnolinguistic acceptability and comprehension. We used an unmatched affected-case versus unaffected-control design for the pilot study. Cases were people with epilepsy attending the tertiary hospitals where these languages are spoken. The controls were relatives of cases or people attending for other medical conditions. An affirmative response to any of the nine questions amounted to a positive screen for epilepsy.

RESULTS

We recruited 153 (75 cases and 78 controls) people for the Hausa version, 106 (45 cases and 61 controls) for Igbo and 153 (66 cases and 87 controls) for the Yoruba. The sensitivity and specificity of the questionnaire were: Hausa (97.3% and 88.5%), Igbo (91.1% and 88.5%) and Yoruba (93.9% and 86.7%). The three versions reliably indicated epilepsy with positive predictive values of 85.9% (Hausa), 85.4% (Igbo) and 87.3% (Yoruba) and reliably excluded epilepsy with negative predictive values of 97.1% (Hausa), 93.1% (Igbo) and 95.1% (Yoruba). Positive likelihood ratios were all greater than one.

CONCLUSIONS

Validated epilepsy screening questionnaires are now available for the three languages to be used for community-based epilepsy survey in Nigeria. The translation and validation process are discussed to facilitate usage and development for other languages in sub-Saharan Africa.

The analogy between DNA kinship and DNA mixture evaluation, with applications for the interpretation of likelihood ratios produced by possibly imperfect models

Two main applications of forensic DNA analysis are the investigation of possible relatedness and the investigation whether a person left DNA in a trace. Both of these are usually carried out by the calculation of likelihood ratios. In the kinship case, it is standard to let the likelihood ratio express the support in favour of the investigated relatedness versus no relatedness, and in the investigation of traces, one by default compares the hypothesis that the person of interest contributed DNA, versus that he is unrelated to any of the actual contributors. In both cases however, we can also view the probabilistic procedure as an inference of the profile of the person we look for: in other words, in both cases we carry out probabilistic genotyping. In this article we use this general analogy to develop various more specific analogies between kinship and mixture likelihood ratios. These analogies help to understand the concepts that play a role, and also to understand the importance of the statistical modeling needed for DNA mixtures. In this article, we apply our findings to consider what we can and cannot conclude from a likelihood ratio in favour of contribution to a mixed DNA profile, if that is computed by a model whose specifics are not entirely known to us, or where we do not know whether they provide a good description of the stochastic effects involved in the generation of DNA trace profiles. We show that, if unrelated individuals are adequately modeled, we can give bounds on how often LR's coming from certain types of black box models may arise, both for persons who are actual contributors and who are unrelated. In particular we show that no model, provided it satisfies basic requirements, can overestimate the evidence found for actual contributors both often and strongly.

A simplified method to approximate a ROC curve with a Bézier curve to calculate likelihood ratios of quantitative test results

In order to calculate likeli hood ratios (LR) values for quantitative test results, a distribution-independent algorithm based on Bézier curves is proposed. Receiver operating characteristic (ROC) analysis provides the LR as the slope of the tangent to the ROC curve at the point corresponding to the test result.

• Here, we make use of cubic Bézier curves defined by Bernstein polynomials of degree 3.

• A simplified method to adjust a Bézier curve to a ROC curve is presented

• The crucial advantage of this procedure is that Bézier curves are constructed by tangents to the ROC curve, whose slopes immediately provide the LR of a specific point on the curve.

Concordance of 3 alternative teratogenicity assays with results from corresponding in vivo embryo-fetal development studies: Final report from the International Consortium for Innovation and Quality in Pharmaceutical Development (IQ) DruSafe working group 2

An IQ DruSafe working group evaluated the concordance of 3 alternative teratogenicity assays (rat whole embryo culture, rWEC; zebrafish embryo culture, ZEC; and murine embryonic stem cells, mESC) with findings from rat or rabbit embryo-fetal development (EFD) studies. Data for 90 individual compounds from 9 companies were entered into a database. In vivo findings were deemed positive if malformations or embryo-fetal lethality were reported in either species. Each company used their own criteria for deciding whether the alternative assay predicted the in vivo findings. Standard concordance parameters were calculated, positive and negative predictive values (PPV and NPV) were adjusted for the aggregate portfolio prevalence of positive compounds (established by a survey of participating companies), and positive and negative likelihood ratios (LR+ and iLR-) were calculated. Of the 3 assays, only rWEC data were robustly predictive, particularly for negative predictions (NPVadj = 92%). However, both LR+ (4.92) and iLR- (4.72) were statistically significant for the rWEC assay. When analyzed separately for rats, the NPVadj and iLR-values for the rWEC assay increased to 96% and 9.75, respectively. These data suggest that a negative rWEC outcome could defer or replace a rat EFD study in certain regulatory settings.

The Limits of Bayesian Thinking in Court

We comment on the contributions of Dahlman and of Fenton et al., who both suggested a Bayesian approach to analyze the Simonshaven case. We argue that analyzing a full case with a Bayesian approach is not feasible, and that there are serious problems with assigning actual numbers to probabilities and priors. We also discuss the nature of Bayesian thinking in court, and the nature and interpretation of the likelihood ratio. In particular, we discuss what it could mean that a likelihood ratio is in some sense uncertain.

Determining the number and size of background samples derived from an area adjacent to the target sample that provide the greatest support for a POI in a target sample

When sampling an item or surface for DNA originating from an action of interest, one is likely to collect DNA unrelated to the action of interest (background DNA). While adding to the complexity of a generated DNA profile, background DNA has been shown to aid in resolving the genotypes of contributors in a targeted sample, and where references of donors to the background DNA are not available, strengthen the LR supporting a person of interest contributing to the targeted sample. This is possible thanks to advances in probabilistic genotyping, where forensic labs are able to deconvolute complex DNA profiles to obtain lists of genotypes and their associated weights. Coupled with DBLR™, one can then compare multiple evidentiary profiles to each other to determine the contribution of common, but unknown, contributors. Here, we consider factors associated with taking background samples and whether one should collect multiple background samples that all relate to a single target sample, or if one should collect larger background samples rather than smaller samples. Background samples consisted of DNA accumulated on the items primarily by one or both occupants of a single household, while targeted samples were generated from touch deposits, or saliva deposits that had been left to air dry. Samples were collected from areas of various sizes, consisting of only the background, the target and the background directly beneath it, and the target and additional surrounding background. A broad range of DNA quantities were recovered, with larger background samples (400 cm2) yielding significantly more DNA than smaller background samples (30 cm2). Significant differences in DNA quantities between target samples were not observed. Generated DNA profiles were interpreted using STRmix™ and DBLR™, and where there was support for a common donor between the background and target sample, pairwise comparisons were performed to observe the effect on the LR supporting the target DNA donor contributing to the targeted sample when conditioning on one (or two) common donor between the targeted sample and 1-8 background samples. Multiple background samples gave significantly higher LRs compared to a single background sample, the larger sampled background area resulted in larger LR gains than the smaller areas, and four or more background samples reduced LR variability considerably. Here we provide recommendations for the minimum and ideal number of additional background samples that should be collected, and that several smaller samples may be more beneficial than a single larger sample.

Evidence-based diagnostic accuracy measurement in urine cytology using likelihood ratios

INTRODUCTION

Recent cytology classification systems have become more evidence-based and advocate for the use of risk of malignancy (ROM) as a measure of test performance. From the statistical viewpoint, ROM represents the post-test probability of malignancy, which changes with the test result and also with the prevalence of malignancies (or pre-test probability) in each individual practice setting and individual patient presentation. Evidence-based medicine offers likelihood ratios (LRs) as a measure of diagnostic accuracy for multilevel diagnostic tests, superior to sensitivity and specificity as data binarization and information loss are avoided. LRs are used in clinical medicine and could be successfully applied to the practice of cytopathology. Our aim was to establish LRs to compare diagnostic accuracy of The Paris System for Reporting Urinary Cytology (TPS) and of a historic urine cytology reporting system.

MATERIALS AND METHODS

We analyzed sequential voided urine cytology cases with histologic outcomes: 188 pre-TPS and 167 post-TPS. LRs were calculated as LR = True positive % (per category)/False positive % (per category) [95% confidence interval] and interpreted LRs = 1 nondiagnostic, LR >1 favor, LR >10 strongly favor, LRs <1 favor exclusion, and LR <0.1 strongly favor exclusion of a target condition, respectively. CATmaker open source software and Fagan nomograms were used for calculation and visualization of the corresponding post-test probability (ROM) of high-grade urothelial carcinoma (HGUC) in various scenarios.

RESULTS

Both reporting systems show near-similar performance in terms of LRs, with moderate discriminatory power of negative, suspicious, and positive for HGUC test results. The atypical urothelial cell (AUC) category establishes as indiscriminate LR = 1 in the TPS, whereas in pre-TPS it favored a benign condition. We further demonstrate the utility of LRs to determine individual post-test probability (ROM) in a variety of clinical scenarios in a personalized fashion.

CONCLUSIONS

The LRs allow for a quantitative performance measure in case of urine cytology across different scenarios adding numeric information on diagnostic test accuracy and post-test probability of HGUC. The diagnostic accuracy of pre-TPS and post-TPS remained similar for all but the AUC category. With the TPS, the AUC category has become genuinely diagnostically and statistically indeterminate and requires further patient investigations.

Data on likelihood ratios of two-person DNA mixtures interpreted using semi- and fully continuous systems

In the paper, "Probabilistic approaches to interpreting two-person DNA mixtures from post-coital specimens" [1], we analysed 102 two-person DNA samples from simulated mixtures and male-female and male-male post-coital specimens. We report here data on profile characteristics of these samples and likelihood ratios (LRs) generated using semi- and fully continuous systems. Both log10 LRs from true and non-contributor tests are presented. These data may supplement studies comparing performance of different probabilistic systems for DNA evidence interpretation.

Interpreting Evidence of Torture

The Istanbul Protocol provides a scheme for giving evidence of signs of torture. This scheme does not conform with the principles of logical inference, revolving as it does round the concept of 'consistency'. The shortcomings of the Protocol are explained using the evidence given in the recent case of KV(Sri Lanka) and the logical approach to such evidence explained.

Diagnostic accuracy of antineutrophil cytoplasmic antibodies (ANCA) in predicting relapses of ANCA-associated vasculitis: systematic review and meta-analysis

Relapse in antineutrophil cytoplasmic antibodies (ANCA)-associated vasculitis (AAV) is associated with significant morbidity and mortality. Utility of ANCA for prediction of relapses is still controversial. PubMed/MEDLINE, Scopus, and WebOfScience were searched, screened and confirmed for inclusion [PROSPERO No: CRD42020220308]. Studies measuring serial ANCA by ELISA or indirect immunofluorescence (IF), reporting relapses with sufficient data to calculate sensitivity and specificity were included. Diagnostic odds ratio (OR), sensitivity, specificity and likelihood ratios (LR) were synthesized using a bivariate mixed-effect regression model. Sub-group analysis included a comparison between ELISA and IIF, anti-myeloperoxidase (MPO) and -proteinase 3(PR3), and type of rise in ANCA. For meta-analysis of survival outcomes, hazard ratios were synthesized using a random-effect model. QUADAS-2 was used for assessing quality of studies, I² statistic for heterogeneity Begg's test for publication bias. 2946 abstracts and 43 full-texts were reviewed to identify 26 eligible studies that included 2623 patients with AAV and 848 relapses. Overall heterogeneity was high [I² = 99%] and the overall risk of bias was low to moderate. ANCA positivity by either ELISA or immunofluorescence for predicting relapse of AAV had a sensitivity of 0.70(95% CI 0.58-0.81), specificity of 0.66(0.55-0.76), positive LR of 2.1(1.6-42.7) and negative LR of 0.44(0.30-0.60). ELISA performed marginally better [OR: 5(3-7)] than IIF [OR: 4(2-9)] with similar sensitivity, specificity, PLR and NLR. The area under the curve for PR3 was 0.74(0.7-0.77), while that for MPO was not computed as the number of eligible studies was only three. In the survival analysis, the hazard ratio for relapse was 3.11(1.7-5.65). The meta-analysis shows modest accuracy of ANCA in predicting relapses of ANCA vasculitis and supports the use of serial ANCA monitoring as a biomarker for relapse.