Variation of a test's sensitivity and specificity with disease prevalence

Validation of childhood asthma predictive tools: A systematic review

BACKGROUND

There is uncertainty about the clinical usefulness of currently available asthma predictive tools. Validation of predictive tools in different populations and clinical settings is an essential requirement for the assessment of their predictive performance, reproducibility and generalizability. We aimed to critically appraise asthma predictive tools which have been validated in external studies.

METHODS

We searched MEDLINE and EMBASE (1946-2017) for all available childhood asthma prediction models and focused on externally validated predictive tools alongside the studies in which they were originally developed. We excluded non-English and non-original studies. PROSPERO registration number is CRD42016035727.

RESULTS

From 946 screened papers, eight were included in the review. Statistical approaches for creation of prediction tools included chi-square tests, logistic regression models and the least absolute shrinkage and selection operator. Predictive models were developed and validated in general and high-risk populations. Only three prediction tools were externally validated: the Asthma Predictive Index, the PIAMA and the Leicester asthma prediction tool. A variety of predictors has been tested, but no studies examined the same combination. There was heterogeneity in definition of the primary outcome among development and validation studies, and no objective measurements were used for asthma diagnosis. The performance of tools varied at different ages of outcome assessment. We observed a discrepancy between the development and validation studies in the tools' predictive performance in terms of sensitivity and positive predictive values.

CONCLUSIONS

Validated asthma predictive tools, reviewed in this paper, provided poor predictive accuracy with performance variation in sensitivity and positive predictive value.

Contrast-enhanced endoscopic ultrasound for the differential diagnosis between benign and malignant lymph nodes: a meta-analysis

Background  The differential diagnosis between benign and malignant lymph nodes (LNs) is crucial for patient management and clinical outcome. The use of contrast-enhanced endoscopic ultrasound (EUS) has been evaluated in several studies with diverse results. The aim of this meta-analysis was to evaluate the pooled diagnostic accuracy of contrast-enhanced EUS (CE-EUS) and contrast-enhanced harmonic EUS (CH-EUS) in this setting. Methods  A systematic electronic search was performed, including all original papers dealing with assessment of the nature of the LNs using CE-EUS or CH-EUS. A meta-analysis was performed to obtain pooled sensitivity, specificity, positive and negative likelihood ratio, and diagnostic odds ratio. The Summary Receiver Operating Characteristic (ROC) Curve method was used to calculate the area under the curve. Statistical analysis was carried out using Meta-Disc V.1.4, Stata V.12.0 and Review Manager V.5.2. Results  Among 210 pertinent studies, four (336 patients) were included in the analysis. The pooled sensitivity was 82.1 % (75.1 - 87.7 %) and pooled specificity was 90.7 % (85.9 - 94.3 %) with significant heterogeneity found in sensitivity; the positive-likelihood ratio (LR) was 7.77 (5.09 - 11.85) and the negative-LR was 0.15 (0.05 - 0.46); the pooled diagnostic odds ratio (DOR) was 54 (15 - 190). Subgroup analysis including studies performed using CH-EUS (two studies, 177 LNs) showed a pooled sensitivity of 87.7 % (77.0 - 93.9 %) and a pooled specificity of 91.8 % (84.5 % - 96.4 %) with no significant heterogeneity; the pooled positive-LR was 9.51 (4.95 - 18.28) and the pooled negative-LR was 0.14 (0.06 - 0.35); pooled DOR was 68.42 (15.5 - 301.4). Conclusions  From these data, CE-EUS is not recommended due to inadequate sensitivity. On the other hand, CH-EUS studies showed optimal accuracy (pooled sensitivity 87.7 % and specificity 91.8 %), comparable to elastography and even EUS-guided fine needle aspiration (EUS-FNA), suggesting a role in the diagnostic algorithm.

Ultrasound image analysis using deep learning algorithm for the diagnosis of thyroid nodules

Fine needle aspiration (FNA) is the procedure of choice for evaluating thyroid nodules. It is indicated for nodules >2 cm, even in cases of very low suspicion of malignancy. FNA has associated risks and expenses. In this study, we developed an image analysis model using a deep learning algorithm and evaluated if the algorithm could predict thyroid nodules with benign FNA results.Ultrasonographic images of thyroid nodules with cytologic or histologic results were retrospectively collected. For algorithm training, 1358 (670 benign, 688 malignant) thyroid nodule images were input into the Inception-V3 network model. The model was pretrained to classify nodules as benign or malignant using the ImageNet database. The diagnostic performance of the algorithm was tested with the prospectively collected internal (n = 55) and external test sets (n = 100).For the internal test set, 20 of the 21 FNA malignant nodules were correctly classified as malignant by the algorithm (sensitivity, 95.2%); and of the 22 nodules algorithm classified as benign, 21 were FNA benign (negative predictive value [NPV], 95.5%). For the external test set, 47 of the 50 FNA malignant nodules were correctly classified by the algorithm (sensitivity, 94.0%); and of the 31 nodules the algorithm classified as benign, 28 were FNA benign (NPV, 90.3%).The sensitivity and NPV of the deep learning algorithm shown in this study are promising. Artificial intelligence may assist clinicians to recognize nodules that are likely to be benign and avoid unnecessary FNA.

Urine-NMR metabolomics for screening of advanced colorectal adenoma and early stage colorectal cancer

Although colorectal cancer (CRC) is considered one of the most preventable cancers, no non-invasive, accurate diagnostic tool to screen CRC exists. We explored the potential of urine nuclear magnetic resonance (NMR) metabolomics as a diagnostic tool for early detection of CRC, focusing on advanced adenoma and stage 0 CRC. Urine metabolomics profiles from patients with colorectal neoplasia (CRN; 36 advanced adenomas and 56 CRCs at various stages, n = 92) and healthy controls (normal, n = 156) were analyzed by NMR spectroscopy. Healthy and CRN groups were statistically discriminated using orthogonal projections to latent structure discriminant analysis (OPLS-DA). The class prediction model was validated by three-fold cross-validation. The advanced adenoma and stage 0 CRC were grouped together as pre-invasive CRN. The OPLS-DA score plot showed statistically significant discrimination between pre-invasive CRN as well as advanced CRC and healthy controls with a Q2 value of 0.746. In the prediction validation study, the sensitivity and specificity for diagnosing pre-invasive CRN were 96.2% and 95%, respectively. The grades predicted by the OPLS-DA model showed that the areas under the curve were 0.823 for taurine, 0.783 for alanine, and 0.842 for 3-aminoisobutyrate. In multiple receiver operating characteristics curve analyses, taurine, alanine, and 3-aminoisobutyrate were good discriminators for CRC patients. NMR-based urine metabolomics profiles significantly and accurately discriminate patients with pre-invasive CRN as well as advanced CRC from healthy individuals. Urine-NMR metabolomics has potential as a screening tool for accurate diagnosis of pre-invasive CRN.

Untapped potential of multicenter studies: a review of cardiovascular risk prediction models revealed inappropriate analyses and wide variation in reporting

BACKGROUND

Clinical prediction models are often constructed using multicenter databases. Such a data structure poses additional challenges for statistical analysis (clustered data) but offers opportunities for model generalizability to a broad range of centers. The purpose of this study was to describe properties, analysis, and reporting of multicenter studies in the Tufts PACE Clinical Prediction Model Registry and to illustrate consequences of common design and analyses choices.

METHODS

Fifty randomly selected studies that are included in the Tufts registry as multicenter and published after 2000 underwent full-text screening. Simulated examples illustrate some key concepts relevant to multicenter prediction research.

RESULTS

Multicenter studies differed widely in the number of participating centers (range 2 to 5473). Thirty-nine of 50 studies ignored the multicenter nature of data in the statistical analysis. In the others, clustering was resolved by developing the model on only one center, using mixed effects or stratified regression, or by using center-level characteristics as predictors. Twenty-three of 50 studies did not describe the clinical settings or type of centers from which data was obtained. Four of 50 studies discussed neither generalizability nor external validity of the developed model.

CONCLUSIONS

Regression methods and validation strategies tailored to multicenter studies are underutilized. Reporting on generalizability and potential external validity of the model lacks transparency. Hence, multicenter prediction research has untapped potential.

REGISTRATION

This review was not registered.

Profile of the SeptiCyte™ LAB gene expression assay to diagnose infection in critically ill patients

Sepsis is a severe and frequently occurring clinical syndrome, caused by the inflammatory response to infections. Recent studies on the human transcriptome during sepsis have yielded several gene-expression assays that might assist physicians during clinical assessment of patients suspected of sepsis. SeptiCyte™ LAB (Immunexpress, Seattle, WA) is the first gene expression assay that was cleared by the FDA in the United States to distinguish infectious from non-infectious causes of systemic inflammation in critically ill patients. The test consists of the simultaneous amplification of four RNA transcripts (CEACAM4, LAMP1, PLAC8, and PLA2G7) in whole blood using a quantitative real-time PCR reaction. This review provides an overview of the challenges in the diagnosis of sepsis, the development of gene expression signatures, and a detailed description of available clinical performance studies evaluating SeptiCyte™ LAB.

Visual inspection for diagnosing cutaneous melanoma in adults

BACKGROUND

Melanoma has one of the fastest rising incidence rates of any cancer. It accounts for a small percentage of skin cancer cases but is responsible for the majority of skin cancer deaths. History-taking and visual inspection of a suspicious lesion by a clinician is usually the first in a series of 'tests' to diagnose skin cancer. Establishing the accuracy of visual inspection alone is critical to understating the potential contribution of additional tests to assist in the diagnosis of melanoma.

OBJECTIVES

To determine the diagnostic accuracy of visual inspection for the detection of cutaneous invasive melanoma and atypical intraepidermal melanocytic variants in adults with limited prior testing and in those referred for further evaluation of a suspicious lesion. Studies were separated according to whether the diagnosis was recorded face-to-face (in-person) or based on remote (image-based) assessment.

SEARCH METHODS

We undertook a comprehensive search of the following databases from inception up to August 2016: CENTRAL; CINAHL; CPCI; Zetoc; Science Citation Index; US National Institutes of Health Ongoing Trials Register; NIHR Clinical Research Network Portfolio Database; and the World Health Organization International Clinical Trials Registry Platform. We studied reference lists and published systematic review articles.

SELECTION CRITERIA

Test accuracy studies of any design that evaluated visual inspection in adults with lesions suspicious for melanoma, compared with a reference standard of either histological confirmation or clinical follow-up. We excluded studies reporting data for 'clinical diagnosis' where dermoscopy may or may not have been used.

DATA COLLECTION AND ANALYSIS

Two review authors independently extracted all data using a standardised data extraction and quality assessment form (based on QUADAS-2). We contacted authors of included studies where information related to the target condition or diagnostic threshold were missing. We estimated summary sensitivities and specificities per algorithm and threshold using the bivariate hierarchical model. We investigated the impact of: in-person test interpretation; use of a purposely developed algorithm to assist diagnosis; and observer expertise.

MAIN RESULTS

We included 49 publications reporting on a total of 51 study cohorts with 34,351 lesions (including 2499 cases), providing 134 datasets for visual inspection. Across almost all study quality domains, the majority of study reports provided insufficient information to allow us to judge the risk of bias, while in three of four domains that we assessed we scored concerns regarding applicability of study findings as 'high'. Selective participant recruitment, lack of detail regarding the threshold for deciding on a positive test result, and lack of detail on observer expertise were particularly problematic.Attempts to analyse studies by degree of prior testing were hampered by a lack of relevant information and by the restricted inclusion of lesions selected for biopsy or excision. Accuracy was generally much higher for in-person diagnosis compared to image-based evaluations (relative diagnostic odds ratio of 8.54, 95% CI 2.89 to 25.3, P < 0.001). Meta-analysis of in-person evaluations that could be clearly placed on the clinical pathway showed a general trade-off between sensitivity and specificity, with the highest sensitivity (92.4%, 95% CI 26.2% to 99.8%) and lowest specificity (79.7%, 95% CI 73.7% to 84.7%) observed in participants with limited prior testing (n = 3 datasets). Summary sensitivities were lower for those referred for specialist assessment but with much higher specificities (e.g. sensitivity 76.7%, 95% CI 61.7% to 87.1%) and specificity 95.7%, 95% CI 89.7% to 98.3%) for lesions selected for excision, n = 8 datasets). These differences may be related to differences in the spectrum of included lesions, differences in the definition of a positive test result, or to variations in observer expertise. We did not find clear evidence that accuracy is improved by the use of any algorithm to assist diagnosis in all settings. Attempts to examine the effect of observer expertise in melanoma diagnosis were hindered due to poor reporting.

AUTHORS' CONCLUSIONS

Visual inspection is a fundamental component of the assessment of a suspicious skin lesion; however, the evidence suggests that melanomas will be missed if visual inspection is used on its own. The evidence to support its accuracy in the range of settings in which it is used is flawed and very poorly reported. Although published algorithms do not appear to improve accuracy, there is insufficient evidence to suggest that the 'no algorithm' approach should be preferred in all settings. Despite the volume of research evaluating visual inspection, further prospective evaluation of the potential added value of using established algorithms according to the prior testing or diagnostic difficulty of lesions may be warranted.

Computer-assisted diagnosis techniques (dermoscopy and spectroscopy-based) for diagnosing skin cancer in adults

BACKGROUND

Early accurate detection of all skin cancer types is essential to guide appropriate management and to improve morbidity and survival. Melanoma and cutaneous squamous cell carcinoma (cSCC) are high-risk skin cancers which have the potential to metastasise and ultimately lead to death, whereas basal cell carcinoma (BCC) is usually localised with potential to infiltrate and damage surrounding tissue. Anxiety around missing early curable cases needs to be balanced against inappropriate referral and unnecessary excision of benign lesions. Computer-assisted diagnosis (CAD) systems use artificial intelligence to analyse lesion data and arrive at a diagnosis of skin cancer. When used in unreferred settings ('primary care'), CAD may assist general practitioners (GPs) or other clinicians to more appropriately triage high-risk lesions to secondary care. Used alongside clinical and dermoscopic suspicion of malignancy, CAD may reduce unnecessary excisions without missing melanoma cases.

OBJECTIVES

To determine the accuracy of CAD systems for diagnosing cutaneous invasive melanoma and atypical intraepidermal melanocytic variants, BCC or cSCC in adults, and to compare its accuracy with that of dermoscopy.

SEARCH METHODS

We undertook a comprehensive search of the following databases from inception up to August 2016: Cochrane Central Register of Controlled Trials (CENTRAL); MEDLINE; Embase; CINAHL; CPCI; Zetoc; Science Citation Index; US National Institutes of Health Ongoing Trials Register; NIHR Clinical Research Network Portfolio Database; and the World Health Organization International Clinical Trials Registry Platform. We studied reference lists and published systematic review articles.

SELECTION CRITERIA

Studies of any design that evaluated CAD alone, or in comparison with dermoscopy, in adults with lesions suspicious for melanoma or BCC or cSCC, and compared with a reference standard of either histological confirmation or clinical follow-up.

DATA COLLECTION AND ANALYSIS

Two review authors independently extracted all data using a standardised data extraction and quality assessment form (based on QUADAS-2). We contacted authors of included studies where information related to the target condition or diagnostic threshold were missing. We estimated summary sensitivities and specificities separately by type of CAD system, using the bivariate hierarchical model. We compared CAD with dermoscopy using (a) all available CAD data (indirect comparisons), and (b) studies providing paired data for both tests (direct comparisons). We tested the contribution of human decision-making to the accuracy of CAD diagnoses in a sensitivity analysis by removing studies that gave CAD results to clinicians to guide diagnostic decision-making.

MAIN RESULTS

We included 42 studies, 24 evaluating digital dermoscopy-based CAD systems (Derm-CAD) in 23 study cohorts with 9602 lesions (1220 melanomas, at least 83 BCCs, 9 cSCCs), providing 32 datasets for Derm-CAD and seven for dermoscopy. Eighteen studies evaluated spectroscopy-based CAD (Spectro-CAD) in 16 study cohorts with 6336 lesions (934 melanomas, 163 BCC, 49 cSCCs), providing 32 datasets for Spectro-CAD and six for dermoscopy. These consisted of 15 studies using multispectral imaging (MSI), two studies using electrical impedance spectroscopy (EIS) and one study using diffuse-reflectance spectroscopy. Studies were incompletely reported and at unclear to high risk of bias across all domains. Included studies inadequately address the review question, due to an abundance of low-quality studies, poor reporting, and recruitment of highly selected groups of participants.Across all CAD systems, we found considerable variation in the hardware and software technologies used, the types of classification algorithm employed, methods used to train the algorithms, and which lesion morphological features were extracted and analysed across all CAD systems, and even between studies evaluating CAD systems. Meta-analysis found CAD systems had high sensitivity for correct identification of cutaneous invasive melanoma and atypical intraepidermal melanocytic variants in highly selected populations, but with low and very variable specificity, particularly for Spectro-CAD systems. Pooled data from 22 studies estimated the sensitivity of Derm-CAD for the detection of melanoma as 90.1% (95% confidence interval (CI) 84.0% to 94.0%) and specificity as 74.3% (95% CI 63.6% to 82.7%). Pooled data from eight studies estimated the sensitivity of multispectral imaging CAD (MSI-CAD) as 92.9% (95% CI 83.7% to 97.1%) and specificity as 43.6% (95% CI 24.8% to 64.5%). When applied to a hypothetical population of 1000 lesions at the mean observed melanoma prevalence of 20%, Derm-CAD would miss 20 melanomas and would lead to 206 false-positive results for melanoma. MSI-CAD would miss 14 melanomas and would lead to 451 false diagnoses for melanoma. Preliminary findings suggest CAD systems are at least as sensitive as assessment of dermoscopic images for the diagnosis of invasive melanoma and atypical intraepidermal melanocytic variants. We are unable to make summary statements about the use of CAD in unreferred populations, or its accuracy in detecting keratinocyte cancers, or its use in any setting as a diagnostic aid, because of the paucity of studies.

AUTHORS' CONCLUSIONS

In highly selected patient populations all CAD types demonstrate high sensitivity, and could prove useful as a back-up for specialist diagnosis to assist in minimising the risk of missing melanomas. However, the evidence base is currently too poor to understand whether CAD system outputs translate to different clinical decision-making in practice. Insufficient data are available on the use of CAD in community settings, or for the detection of keratinocyte cancers. The evidence base for individual systems is too limited to draw conclusions on which might be preferred for practice. Prospective comparative studies are required that evaluate the use of already evaluated CAD systems as diagnostic aids, by comparison to face-to-face dermoscopy, and in participant populations that are representative of those in which the test would be used in practice.

Visual inspection and dermoscopy, alone or in combination, for diagnosing keratinocyte skin cancers in adults

BACKGROUND

Early accurate detection of all skin cancer types is important to guide appropriate management, to reduce morbidity and to improve survival. Basal cell carcinoma (BCC) is almost always a localised skin cancer with potential to infiltrate and damage surrounding tissue, whereas a minority of cutaneous squamous cell carcinomas (cSCCs) and invasive melanomas are higher-risk skin cancers with the potential to metastasise and cause death. Dermoscopy has become an important tool to assist specialist clinicians in the diagnosis of melanoma, and is increasingly used in primary-care settings. Dermoscopy is a precision-built handheld illuminated magnifier that allows more detailed examination of the skin down to the level of the superficial dermis. Establishing the value of dermoscopy over and above visual inspection for the diagnosis of BCC or cSCC in primary- and secondary-care settings is critical to understanding its potential contribution to appropriate skin cancer triage, including referral of higher-risk cancers to secondary care, the identification of low-risk skin cancers that might be treated in primary care and to provide reassurance to those with benign skin lesions who can be safely discharged.

OBJECTIVES

To determine the diagnostic accuracy of visual inspection and dermoscopy, alone or in combination, for the detection of (a) BCC and (b) cSCC, in adults. We separated studies according to whether the diagnosis was recorded face-to-face (in person) or based on remote (image-based) assessment.

SEARCH METHODS

We undertook a comprehensive search of the following databases from inception up to August 2016: Cochrane Central Register of Controlled Trials; MEDLINE; Embase; CINAHL; CPCI; Zetoc; Science Citation Index; US National Institutes of Health Ongoing Trials Register; NIHR Clinical Research Network Portfolio Database; and the World Health Organization International Clinical Trials Registry Platform. We studied reference lists and published systematic review articles.

SELECTION CRITERIA

Studies of any design that evaluated visual inspection or dermoscopy or both in adults with lesions suspicious for skin cancer, compared with a reference standard of either histological confirmation or clinical follow-up.

DATA COLLECTION AND ANALYSIS

Two review authors independently extracted all data using a standardised data extraction and quality assessment form (based on QUADAS-2). We contacted authors of included studies where information related to the target condition or diagnostic thresholds were missing. We estimated accuracy using hierarchical summary ROC methods. We undertook analysis of studies allowing direct comparison between tests. To facilitate interpretation of results, we computed values of sensitivity at the point on the SROC curve with 80% fixed specificity and values of specificity with 80% fixed sensitivity. We investigated the impact of in-person test interpretation; use of a purposely-developed algorithm to assist diagnosis; and observer expertise.

MAIN RESULTS

We included 24 publications reporting on 24 study cohorts, providing 27 visual inspection datasets (8805 lesions; 2579 malignancies) and 33 dermoscopy datasets (6855 lesions; 1444 malignancies). The risk of bias was mainly low for the index test (for dermoscopy evaluations) and reference standard domains, particularly for in-person evaluations, and high or unclear for participant selection, application of the index test for visual inspection and for participant flow and timing. We scored concerns about the applicability of study findings as of 'high' or 'unclear' concern for almost all studies across all domains assessed. Selective participant recruitment, lack of reproducibility of diagnostic thresholds and lack of detail on observer expertise were particularly problematic.The detection of BCC was reported in 28 datasets; 15 on an in-person basis and 13 image-based. Analysis of studies by prior testing of participants and according to observer expertise was not possible due to lack of data. Studies were primarily conducted in participants referred for specialist assessment of lesions with available histological classification. We found no clear differences in accuracy between dermoscopy studies undertaken in person and those which evaluated images. The lack of effect observed may be due to other sources of heterogeneity, including variations in the types of skin lesion studied, in dermatoscopes used, or in the use of algorithms and varying thresholds for deciding on a positive test result.Meta-analysis found in-person evaluations of dermoscopy (7 evaluations; 4683 lesions and 363 BCCs) to be more accurate than visual inspection alone for the detection of BCC (8 evaluations; 7017 lesions and 1586 BCCs), with a relative diagnostic odds ratio (RDOR) of 8.2 (95% confidence interval (CI) 3.5 to 19.3; P < 0.001). This corresponds to predicted differences in sensitivity of 14% (93% versus 79%) at a fixed specificity of 80% and predicted differences in specificity of 22% (99% versus 77%) at a fixed sensitivity of 80%. We observed very similar results for the image-based evaluations.When applied to a hypothetical population of 1000 lesions, of which 170 are BCC (based on median BCC prevalence across studies), an increased sensitivity of 14% from dermoscopy would lead to 24 fewer BCCs missed, assuming 166 false positive results from both tests. A 22% increase in specificity from dermoscopy with sensitivity fixed at 80% would result in 183 fewer unnecessary excisions, assuming 34 BCCs missed for both tests. There was not enough evidence to assess the use of algorithms or structured checklists for either visual inspection or dermoscopy.Insufficient data were available to draw conclusions on the accuracy of either test for the detection of cSCCs.

AUTHORS' CONCLUSIONS

Dermoscopy may be a valuable tool for the diagnosis of BCC as an adjunct to visual inspection of a suspicious skin lesion following a thorough history-taking including assessment of risk factors for keratinocyte cancer. The evidence primarily comes from secondary-care (referred) populations and populations with pigmented lesions or mixed lesion types. There is no clear evidence supporting the use of currently-available formal algorithms to assist dermoscopy diagnosis.

Teledermatology for diagnosing skin cancer in adults

BACKGROUND

Early accurate detection of all skin cancer types is essential to guide appropriate management and to improve morbidity and survival. Melanoma and squamous cell carcinoma (SCC) are high-risk skin cancers which have the potential to metastasise and ultimately lead to death, whereas basal cell carcinoma (BCC) is usually localised with potential to infiltrate and damage surrounding tissue. Anxiety around missing early curable cases needs to be balanced against inappropriate referral and unnecessary excision of benign lesions. Teledermatology provides a way for generalist clinicians to access the opinion of a specialist dermatologist for skin lesions that they consider to be suspicious without referring the patients through the normal referral pathway. Teledermatology consultations can be 'store-and-forward' with electronic digital images of a lesion sent to a dermatologist for review at a later time, or can be live and interactive consultations using videoconferencing to connect the patient, referrer and dermatologist in real time.

OBJECTIVES

To determine the diagnostic accuracy of teledermatology for the detection of any skin cancer (melanoma, BCC or cutaneous squamous cell carcinoma (cSCC)) in adults, and to compare its accuracy with that of in-person diagnosis.

SEARCH METHODS

We undertook a comprehensive search of the following databases from inception up to August 2016: Cochrane Central Register of Controlled Trials, MEDLINE, Embase, CINAHL, CPCI, Zetoc, Science Citation Index, US National Institutes of Health Ongoing Trials Register, NIHR Clinical Research Network Portfolio Database and the World Health Organization International Clinical Trials Registry Platform. We studied reference lists and published systematic review articles.

SELECTION CRITERIA

Studies evaluating skin cancer diagnosis for teledermatology alone, or in comparison with face-to-face diagnosis by a specialist clinician, compared with a reference standard of histological confirmation or clinical follow-up and expert opinion. We also included studies evaluating the referral accuracy of teledermatology compared with a reference standard of face-to-face diagnosis by a specialist clinician.

DATA COLLECTION AND ANALYSIS

Two review authors independently extracted all data using a standardised data extraction and quality assessment form (based on QUADAS-2). We contacted authors of included studies where there were information related to the target condition of any skin cancer missing. Data permitting, we estimated summary sensitivities and specificities using the bivariate hierarchical model. Due to the scarcity of data, we undertook no covariate investigations for this review. For illustrative purposes, we plotted estimates of sensitivity and specificity on coupled forest plots for diagnostic threshold and target condition under consideration.

MAIN RESULTS

The review included 22 studies reporting diagnostic accuracy data for 4057 lesions and 879 malignant cases (16 studies) and referral accuracy data for reported data for 1449 lesions and 270 'positive' cases as determined by the reference standard face-to-face decision (six studies). Methodological quality was variable with poor reporting hindering assessment. The overall risk of bias was high or unclear for participant selection, reference standard, and participant flow and timing in at least half of all studies; the majority were at low risk of bias for the index test. The applicability of study findings were of high or unclear concern for most studies in all domains assessed due to the recruitment of participants from secondary care settings or specialist clinics rather than from primary or community-based settings in which teledermatology is more likely to be used and due to the acquisition of lesion images by dermatologists or in specialist imaging units rather than by primary care clinicians.Seven studies provided data for the primary target condition of any skin cancer (1588 lesions and 638 malignancies). For the correct diagnosis of lesions as malignant using photographic images, summary sensitivity was 94.9% (95% confidence interval (CI) 90.1% to 97.4%) and summary specificity was 84.3% (95% CI 48.5% to 96.8%) (from four studies). Individual study estimates using dermoscopic images or a combination of photographic and dermoscopic images generally suggested similarly high sensitivities with highly variable specificities. Limited comparative data suggested similar diagnostic accuracy between teledermatology assessment and in-person diagnosis by a dermatologist; however, data were too scarce to draw firm conclusions. For the detection of invasive melanoma or atypical intraepidermal melanocytic variants both sensitivities and specificities were more variable. Sensitivities ranged from 59% (95% CI 42% to 74%) to 100% (95% CI 48% to 100%) and specificities from 30% (95% CI 22% to 40%) to 100% (95% CI 93% to 100%), with reported diagnostic thresholds including the correct diagnosis of melanoma, classification of lesions as 'atypical' or 'typical, and the decision to refer or to excise a lesion.Referral accuracy data comparing teledermatology against a face-to-face reference standard suggested good agreement for lesions considered to require some positive action by face-to-face assessment (sensitivities of over 90%). For lesions considered of less concern when assessed face-to-face (e.g. for lesions not recommended for excision or referral), agreement was more variable with teledermatology specificities ranging from 57% (95% CI 39% to 73%) to 100% (95% CI 86% to 100%), suggesting that remote assessment is more likely recommend excision, referral or follow-up compared to in-person decisions.

AUTHORS' CONCLUSIONS

Studies were generally small and heterogeneous and methodological quality was difficult to judge due to poor reporting. Bearing in mind concerns regarding the applicability of study participants and of lesion image acquisition in specialist settings, our results suggest that teledermatology can correctly identify the majority of malignant lesions. Using a more widely defined threshold to identify 'possibly' malignant cases or lesions that should be considered for excision is likely to appropriately triage those lesions requiring face-to-face assessment by a specialist. Despite the increasing use of teledermatology on an international level, the evidence base to support its ability to accurately diagnose lesions and to triage lesions from primary to secondary care is lacking and further prospective and pragmatic evaluation is needed.

Dermoscopy, with and without visual inspection, for diagnosing melanoma in adults

BACKGROUND

Melanoma has one of the fastest rising incidence rates of any cancer. It accounts for a small percentage of skin cancer cases but is responsible for the majority of skin cancer deaths. Although history-taking and visual inspection of a suspicious lesion by a clinician are usually the first in a series of 'tests' to diagnose skin cancer, dermoscopy has become an important tool to assist diagnosis by specialist clinicians and is increasingly used in primary care settings. Dermoscopy is a magnification technique using visible light that allows more detailed examination of the skin compared to examination by the naked eye alone. Establishing the additive value of dermoscopy over and above visual inspection alone across a range of observers and settings is critical to understanding its contribution for the diagnosis of melanoma and to future understanding of the potential role of the growing number of other high-resolution image analysis techniques.

OBJECTIVES

To determine the diagnostic accuracy of dermoscopy alone, or when added to visual inspection of a skin lesion, for the detection of cutaneous invasive melanoma and atypical intraepidermal melanocytic variants in adults. We separated studies according to whether the diagnosis was recorded face-to-face (in-person), or based on remote (image-based), assessment.

SEARCH METHODS

We undertook a comprehensive search of the following databases from inception up to August 2016: CENTRAL; MEDLINE; Embase; CINAHL; CPCI; Zetoc; Science Citation Index; US National Institutes of Health Ongoing Trials Register; NIHR Clinical Research Network Portfolio Database; and the World Health Organization International Clinical Trials Registry Platform. We studied reference lists and published systematic review articles.

SELECTION CRITERIA

Studies of any design that evaluated dermoscopy in adults with lesions suspicious for melanoma, compared with a reference standard of either histological confirmation or clinical follow-up. Data on the accuracy of visual inspection, to allow comparisons of tests, was included only if reported in the included studies of dermoscopy.

DATA COLLECTION AND ANALYSIS

Two review authors independently extracted all data using a standardised data extraction and quality assessment form (based on QUADAS-2). We contacted authors of included studies where information related to the target condition or diagnostic threshold were missing. We estimated accuracy using hierarchical summary receiver operating characteristic (SROC),methods. Analysis of studies allowing direct comparison between tests was undertaken. To facilitate interpretation of results, we computed values of sensitivity at the point on the SROC curve with 80% fixed specificity and values of specificity with 80% fixed sensitivity. We investigated the impact of in-person test interpretation; use of a purposely developed algorithm to assist diagnosis; observer expertise; and dermoscopy training.

MAIN RESULTS

We included a total of 104 study publications reporting on 103 study cohorts with 42,788 lesions (including 5700 cases), providing 354 datasets for dermoscopy. The risk of bias was mainly low for the index test and reference standard domains and mainly high or unclear for participant selection and participant flow. Concerns regarding the applicability of study findings were largely scored as 'high' concern in three of four domains assessed. Selective participant recruitment, lack of reproducibility of diagnostic thresholds and lack of detail on observer expertise were particularly problematic.The accuracy of dermoscopy for the detection of invasive melanoma or atypical intraepidermal melanocytic variants was reported in 86 datasets; 26 for evaluations conducted in person (dermoscopy added to visual inspection), and 60 for image-based evaluations (diagnosis based on interpretation of dermoscopic images). Analyses of studies by prior testing revealed no obvious effect on accuracy; analyses were hampered by the lack of studies in primary care, lack of relevant information and the restricted inclusion of lesions selected for biopsy or excision. Accuracy was higher for in-person diagnosis compared to image-based evaluations (relative diagnostic odds ratio (RDOR) 4.6, 95% confidence interval (CI) 2.4 to 9.0; P < 0.001).We compared accuracy for (a), in-person evaluations of dermoscopy (26 evaluations; 23,169 lesions and 1664 melanomas),versus visual inspection alone (13 evaluations; 6740 lesions and 459 melanomas), and for (b), image-based evaluations of dermoscopy (60 evaluations; 13,475 lesions and 2851 melanomas),versus image-based visual inspection (11 evaluations; 1740 lesions and 305 melanomas). For both comparisons, meta-analysis found dermoscopy to be more accurate than visual inspection alone, with RDORs of (a), 4.7 (95% CI 3.0 to 7.5; P < 0.001), and (b), 5.6 (95% CI 3.7 to 8.5; P < 0.001). For a), the predicted difference in sensitivity at a fixed specificity of 80% was 16% (95% CI 8% to 23%; 92% for dermoscopy + visual inspection versus 76% for visual inspection), and predicted difference in specificity at a fixed sensitivity of 80% was 20% (95% CI 7% to 33%; 95% for dermoscopy + visual inspection versus 75% for visual inspection). For b) the predicted differences in sensitivity was 34% (95% CI 24% to 46%; 81% for dermoscopy versus 47% for visual inspection), at a fixed specificity of 80%, and predicted difference in specificity was 40% (95% CI 27% to 57%; 82% for dermoscopy versus 42% for visual inspection), at a fixed sensitivity of 80%.Using the median prevalence of disease in each set of studies ((a), 12% for in-person and (b), 24% for image-based), for a hypothetical population of 1000 lesions, an increase in sensitivity of (a), 16% (in-person), and (b), 34% (image-based), from using dermoscopy at a fixed specificity of 80% equates to a reduction in the number of melanomas missed of (a), 19 and (b), 81 with (a), 176 and (b), 152 false positive results. An increase in specificity of (a), 20% (in-person), and (b), 40% (image-based), at a fixed sensitivity of 80% equates to a reduction in the number of unnecessary excisions from using dermoscopy of (a), 176 and (b), 304 with (a), 24 and (b), 48 melanomas missed.The use of a named or published algorithm to assist dermoscopy interpretation (as opposed to no reported algorithm or reported use of pattern analysis), had no significant impact on accuracy either for in-person (RDOR 1.4, 95% CI 0.34 to 5.6; P = 0.17), or image-based (RDOR 1.4, 95% CI 0.60 to 3.3; P = 0.22), evaluations. This result was supported by subgroup analysis according to algorithm used. We observed higher accuracy for observers reported as having high experience and for those classed as 'expert consultants' in comparison to those considered to have less experience in dermoscopy, particularly for image-based evaluations. Evidence for the effect of dermoscopy training on test accuracy was very limited but suggested associated improvements in sensitivity.

AUTHORS' CONCLUSIONS

Despite the observed limitations in the evidence base, dermoscopy is a valuable tool to support the visual inspection of a suspicious skin lesion for the detection of melanoma and atypical intraepidermal melanocytic variants, particularly in referred populations and in the hands of experienced users. Data to support its use in primary care are limited, however, it may assist in triaging suspicious lesions for urgent referral when employed by suitably trained clinicians. Formal algorithms may be of most use for dermoscopy training purposes and for less expert observers, however reliable data comparing approaches using dermoscopy in person are lacking.

A meta-analysis of the accuracy of embedded performance validity indicators from the repeatable battery for the assessment of neuropsychological status

Objective: Meta-analyze the embedded performance validity tests of the Repeatable Battery for the Assessment of Neuropsychological Status (RBANS), emphasizing two topics: diagnostic accuracy and relationships with demographics. Method: Literature was identified through a review of PsycINFO, PubMed, and Google Scholar. Bivariate linear mixed models were used for diagnostic accuracy analyses. Random-effects analysis of correlations with demographic variables was also used for the Effort Index. Results: Diagnostic accuracy analyses utilized 12 studies of the Effort Index (EI; N = 1469) and 6 studies of the Effort Scale (ES; N = 854). The EI was found to have a sensitivity of .44 (95% CI: .28 - .62), specificity of .87 (95% CI: .78 - .93), and diagnostic odds ratio (DOR) of 5.41 (95% CI: 3.74 - 7.58). The ES produced a sensitivity of .67 (95% CI: .50 - .81), specificity of .72 (95% CI: .53 - .85), and DOR of 5.97 (95% CI: 1.70 - 15.30). Meta-analysis of correlations utilized 14 samples of the EI (N = 3781), but insufficient data were available for the ES. Correlations between the EI and age (r = .10, 95% CI: .05 - .16), education (r = -.11, 95% CI: -.18 to -.04), and the RBANS Total Scale (r = -.45, 95% CI: -.62 to -.24) were significant. These results appeared to be robust to publication bias. Conclusion: Results support use of the EI over the ES; however, the latter is comparatively less studied and did not have similar variability in reported cut-off scores. ES accuracy statistics were significantly predicted by sample variables, which may be related to its relatively fewer studies. Both measures should be used cautiously in the context of genuine cognitive impairment.

A deep neural network learning algorithm outperforms a conventional algorithm for emergency department electrocardiogram interpretation

BACKGROUND

Cardiologs® has developed the first electrocardiogram (ECG) algorithm that uses a deep neural network (DNN) for full 12‑lead ECG analysis, including rhythm, QRS and ST-T-U waves. We compared the accuracy of the first version of Cardiologs® DNN algorithm to the Mortara/Veritas® conventional algorithm in emergency department (ED) ECGs.

METHODS

Individual ECG diagnoses were prospectively mapped to one of 16 pre-specified groups of ECG diagnoses, which were further classified as "major" ECG abnormality or not. Automated interpretations were compared to blinded experts'. The primary outcome was the performance of the algorithms in finding at least one "major" abnormality. The secondary outcome was the proportion of all ECGs for which all groups were identified, with no false negative or false positive groups ("accurate ECG interpretation"). Additionally, we measured sensitivity and positive predictive value (PPV) for any abnormal group.

RESULTS

Cardiologs® vs. Veritas® accuracy for finding a major abnormality was 92.2% vs. 87.2% (p < 0.0001), with comparable sensitivity (88.7% vs. 92.0%, p = 0.086), improved specificity (94.0% vs. 84.7%, p < 0.0001) and improved positive predictive value (PPV 88.2% vs. 75.4%, p < 0.0001). Cardiologs® had accurate ECG interpretation for 72.0% (95% CI: 69.6-74.2) of ECGs vs. 59.8% (57.3-62.3) for Veritas® (P < 0.0001). Sensitivity for any abnormal group for Cardiologs® and Veritas®, respectively, was 69.6% (95CI 66.7-72.3) vs. 68.3% (95CI 65.3-71.1) (NS). Positive Predictive Value was 74.0% (71.1-76.7) for Cardiologs® vs. 56.5% (53.7-59.3) for Veritas® (P < 0.0001).

CONCLUSION

Cardiologs' DNN was more accurate and specific in identifying ECGs with at least one major abnormal group. It had a significantly higher rate of accurate ECG interpretation, with similar sensitivity and higher PPV.

Surgeon-Performed Ultrasound in Diagnosing Acute Cholecystitis and Appendicitis

BACKGROUND

The use of ultrasound (US) outside the radiology department has increased the last decades, but large studies assessing the quality of bedside US are still lacking. This study evaluates surgeon-performed US (SPUS) and radiologist-performed US (RPUS) with respect to biliary disease and appendicitis.

METHODS

Between October 2011 and November 2012, 300 adult patients, with a referral for an abdominal US, were prospectively enrolled in the study and examined by a radiologist as well as a surgeon. The surgeons had undergone a 4-week-long US education. US findings of the surgeon and of the radiologist were compared to final diagnosis, set by an independent external observer going through each patient's chart.

RESULTS

Among 183 patients with suspected biliary disease, 74 had gallstones and 21 had acute cholecystitis. SPUS and RPUS diagnosed gallstones with a sensitivity of 87.1 versus 97.3%. Specificity was 96.0 versus 98.9%, and the accuracy 92.3 versus 98.2%. The sensitivity, specificity and accuracy for acute cholecystitis by SPUS and RPUS were: 60.0 versus 80.0%, 98.6 versus 97.8% and 93.9 versus 95.6%, respectively. Among 58 patients with suspected appendicitis, 15 had the disease. The sensitivity, specificity and accuracy for appendicitis by SPUS and RPUS were: 53.3 versus 73.3%, 89.7 versus 93.3% and 77.3 versus 86.7%, respectively.

CONCLUSION

SPUS is reliable in diagnosing gallstones. Diagnosing cholecystitis and appendicitis with US is more challenging for both surgeons and radiologists.

TRIAL REGISTRATION NUMBER

The study was registered at clinicaltrials.gov. Registration number: NCT02469935.

Spot and Diffuse Signs: Quantitative Markers of Intracranial Hematoma Expansion at Dual-Energy CT

Purpose To compare dual-energy CT with iodine quantification to single-energy CT for evaluation of the spot sign for intracranial hematoma expansion. Materials and Methods In this retrospective study, 42 patients (mean age, 66 years ± 15 [standard deviation]; 19 women) were referred for dual-energy CT assessment of intracranial hemorrhage from October 2014 to January 2017. A machine learning approach (naive Bayes classifier) was used to identify iodine markers of extravasation for risk of hematoma expansion. Specificity and sensitivity of these markers were then independently validated in 65 new patients from February 2017 to February 2018. Results Analysis of dual-energy CT images identified two features of iodine extravasation: total iodine concentration within the hematoma (I_h) and focal iodine concentration in the brightest spot in the hematoma (I_bs) as predictors of expansion. The I2 score derived from these features provided a measure of expansion probability. Optimal classification threshold was an I2 score of 20 (95% confidence interval [CI]: 18, 23), leading to correct identification of 39 of 46 (85%; 95% CI: 71%, 94%) of the hematomas on the training set (sensitivity of 79% [11 of 14; 95% CI: 57%, 100%] and specificity of 88% [28 of 32; 95% CI: 76%, 99%]), and 62 of 70 (89%; 95% CI: 79%, 95%) of the hematomas on the validation set (sensitivity of 71% [10 of 14; 95% CI: 48%, 95%] and specificity of 93% [52 of 56; 95% CI: 86%, 100%]). Sensitivity, specificity, and accuracy of conventional spot sign were, respectively, 57% (eight of 14), 90% (29 of 32), and 80% (37 of 46) on the training set and 57% (eight of 14), 83% (47 of 56), and 75% (53 of 70) on the validation set. Conclusion This study identified two quantitative markers of intracranial hemorrhage expansion at dual-energy CT of the brain. The I2 score derived from these markers highlights the utility of dual-energy CT measurements of iodine content for high sensitivity risk assessment. © RSNA, 2018 Online supplemental material is available for this article.

Optimising sampling regimes and data collection to inform surveillance for trachoma control

It is estimated that 190 million individuals are at risk of blindness from trachoma, and that control by mass drug administration (MDA) is reducing this risk in many populations. Programs are monitored using prevalence of follicular trachoma disease (TF) in children. However, as programs progress to low prevalence there are challenges interpreting this indirect measure of infection. PCR and sero-surveillance are being considered as complementary tools to monitor low-level transmission, but there are questions on how they can be most effectively used. We use a previously-published, mathematical model to explore the dynamic relationship between TF and PCR throughout a control program and a sero-catalytic model to evaluate the utility of two cross-sectional sero-surveys for estimating sero-conversion rates. The simulations show that whilst PCR is more sensitive than TF at detecting infection, the probability of detecting at least one positive individual declines during an MDA program more quickly for PCR than for TF (for the same sample size). Towards the end of a program there is a moderate chance of a random sample showing both low PCR prevalence and higher TF prevalence, which may contribute to the lack of correlation observed in epidemiological studies. We also show that conducting two cross-sectional sero-surveys 10 years apart can provide more precise and accurate estimation of epidemiological parameters than a single survey, supporting previous findings that whilst serology holds great promise, multiple cross-sections from the same community are needed to generate the most valuable information about transmission. These results highlight that the quantitative dynamics of infection and disease should be included alongside the many logistical and practical factors to be considered in designing a monitoring and evaluation strategy at the operational research level, in order to help subsequently inform data collection for individual country programs. Whilst our simulations provide some insight, they also highlight that some level of longitudinal, individual-level data on reinfection and disease may be needed to monitor elimination progress.

Xpert® MTB/RIF assay for extrapulmonary tuberculosis and rifampicin resistance

BACKGROUND

Tuberculosis (TB) is the world's leading infectious cause of death. Extrapulmonary TB accounts for 15% of TB cases, but the proportion is increasing, and over half a million people were newly diagnosed with rifampicin-resistant TB in 2016. Xpert^® MTB/RIF (Xpert) is a World Health Organization (WHO)-recommended, rapid, automated, nucleic acid amplification assay that is used widely for simultaneous detection of Mycobacterium tuberculosis complex and rifampicin resistance in sputum specimens. This Cochrane Review assessed the accuracy of Xpert in extrapulmonary specimens.

OBJECTIVES

To determine the diagnostic accuracy of Xpert a) for extrapulmonary TB by site of disease in people presumed to have extrapulmonary TB; and b) for rifampicin resistance in people presumed to have extrapulmonary TB.

SEARCH METHODS

We searched the Cochrane Infectious Diseases Group Specialized Register, MEDLINE, Embase, Science Citation Index, Web of Science, Latin American Caribbean Health Sciences Literature (LILACS), Scopus, ClinicalTrials.gov, the WHO International Clinical Trials Registry Platform, the International Standard Randomized Controlled Trial Number (ISRCTN) Registry, and ProQuest up to 7 August 2017 without language restriction.

SELECTION CRITERIA

We included diagnostic accuracy studies of Xpert in people presumed to have extrapulmonary TB. We included TB meningitis and pleural, lymph node, bone or joint, genitourinary, peritoneal, pericardial, and disseminated TB. We used culture as the reference standard. For pleural TB, we also included a composite reference standard, which defined a positive result as the presence of granulomatous inflammation or a positive culture result. For rifampicin resistance, we used culture-based drug susceptibility testing or MTBDRplus as the reference standard.

DATA COLLECTION AND ANALYSIS

Two review authors independently extracted data, assessed risk of bias and applicability using the QUADAS-2 tool. We determined pooled predicted sensitivity and specificity for TB, grouped by type of extrapulmonary specimen, and for rifampicin resistance. For TB detection, we used a bivariate random-effects model. Recognizing that use of culture may lead to misclassification of cases of extrapulmonary TB as 'not TB' owing to the paucibacillary nature of the disease, we adjusted accuracy estimates by applying a latent class meta-analysis model. For rifampicin resistance detection, we performed univariate meta-analyses for sensitivity and specificity separately to include studies in which no rifampicin resistance was detected. We used theoretical populations with an assumed prevalence to provide illustrative numbers of patients with false positive and false negative results.

MAIN RESULTS

We included 66 unique studies that evaluated 16,213 specimens for detection of extrapulmonary TB and rifampicin resistance. We identified only one study that evaluated the newest test version, Xpert MTB/RIF Ultra (Ultra), for TB meningitis. Fifty studies (76%) took place in low- or middle-income countries. Risk of bias was low for patient selection, index test, and flow and timing domains and was high or unclear for the reference standard domain (most of these studies decontaminated sterile specimens before culture inoculation). Regarding applicability, in the patient selection domain, we scored high or unclear concern for most studies because either patients were evaluated exclusively as inpatients at tertiary care centres, or we were not sure about the clinical settings.Pooled Xpert sensitivity (defined by culture) varied across different types of specimens (31% in pleural tissue to 97% in bone or joint fluid); Xpert sensitivity was > 80% in urine and bone or joint fluid and tissue. Pooled Xpert specificity (defined by culture) varied less than sensitivity (82% in bone or joint tissue to 99% in pleural fluid and urine). Xpert specificity was ≥ 98% in cerebrospinal fluid, pleural fluid, urine, and peritoneal fluid.Xpert testing in cerebrospinal fluidXpert pooled sensitivity and specificity (95% credible interval (CrI)) against culture were 71.1% (60.9% to 80.4%) and 98.0% (97.0% to 98.8%), respectively (29 studies, 3774 specimens; moderate-certainty evidence).For a population of 1000 people where 100 have TB meningitis on culture, 89 would be Xpert-positive: of these, 18 (20%) would not have TB (false-positives); and 911 would be Xpert-negative: of these, 29 (3%) would have TB (false-negatives).For TB meningitis, ultra sensitivity and specificity against culture (95% confidence interval (CI)) were 90% (55% to 100%) and 90% (83% to 95%), respectively (one study, 129 participants).Xpert testing in pleural fluidXpert pooled sensitivity and specificity (95% CrI) against culture were 50.9% (39.7% to 62.8%) and 99.2% (98.2% to 99.7%), respectively (27 studies, 4006 specimens; low-certainty evidence).For a population of 1000 people where 150 have pleural TB on culture, 83 would be Xpert-positive: of these, seven (8%) would not have TB (false-positives); and 917 would be Xpert-negative: of these, 74 (8%) would have TB (false-negatives).Xpert testing in urineXpert pooled sensitivity and specificity (95% CrI) against culture were 82.7% (69.6% to 91.1%) and 98.7% (94.8% to 99.7%), respectively (13 studies, 1199 specimens; moderate-certainty evidence).For a population of 1000 people where 70 have genitourinary TB on culture, 70 would be Xpert-positive: of these, 12 (17%) would not have TB (false-positives); and 930 would be Xpert-negative: of these, 12 (1%) would have TB (false-negatives).Xpert testing for rifampicin resistanceXpert pooled sensitivity (20 studies, 148 specimens) and specificity (39 studies, 1088 specimens) were 95.0% (89.7% to 97.9%) and 98.7% (97.8% to 99.4%), respectively (high-certainty evidence).For a population of 1000 people where 120 have rifampicin-resistant TB, 125 would be positive for rifampicin-resistant TB: of these, 11 (9%) would not have rifampicin resistance (false-positives); and 875 would be negative for rifampicin-resistant TB: of these, 6 (1%) would have rifampicin resistance (false-negatives).For lymph node TB, the accuracy of culture, the reference standard used, presented a greater concern for bias than in other forms of extrapulmonary TB.

AUTHORS' CONCLUSIONS

In people presumed to have extrapulmonary TB, Xpert may be helpful in confirming the diagnosis. Xpert sensitivity varies across different extrapulmonary specimens, while for most specimens, specificity is high, the test rarely yielding a positive result for people without TB (defined by culture). Xpert is accurate for detection of rifampicin resistance. For people with presumed TB meningitis, treatment should be based on clinical judgement, and not withheld solely on an Xpert result, as is common practice when culture results are negative.

Improving the Specificity of PSA Screening with Serum and Urine Markers

PURPOSE OF REVIEW

Prostate cancer remains a significant public health burden. In order to decrease the morbidity and mortality associated with prostate cancer, serum prostate-specific antigen (PSA) screening has been used since the 1990s. However, there is concern for overdiagnosis of prostate cancer with widespread PSA screening, which could lead to overtreatment and its attendant morbidities.

RECENT FINDINGS

In order to avoid unnecessary biopsy and downstream effects including treatment of insignificant prostate cancer, a number of tests have been proposed to improve upon PSA screening. Increasing the specificity of prostate cancer screening above that of PSA testing should reduce the incidence of unnecessary prostate biopsy. However, an increase in specificity is associated with a decrease in sensitivity, so these tests must be balanced with the concern for missing the diagnosis of potentially significant disease. In this context, we present a review of six common biomarkers proposed to improve the specificity of prostate cancer screening-free PSA, prostate health index, 4Kscore, PCA3, Select MDx, and ExoDx Prostate(Intelliscore).

Hybrid copula mixed models for combining case-control and cohort studies in meta-analysis of diagnostic tests

Copula mixed models for trivariate (or bivariate) meta-analysis of diagnostic test accuracy studies accounting (or not) for disease prevalence have been proposed in the biostatistics literature to synthesize information. However, many systematic reviews often include case-control and cohort studies, so one can either focus on the bivariate meta-analysis of the case-control studies or the trivariate meta-analysis of the cohort studies, as only the latter contains information on disease prevalence. In order to remedy this situation of wasting data we propose a hybrid copula mixed model via a combination of the bivariate and trivariate copula mixed model for the data from the case-control studies and cohort studies, respectively. Hence, this hybrid model can account for study design and also due to its generality can deal with dependence in the joint tails. We apply the proposed hybrid copula mixed model to a review of the performance of contemporary diagnostic imaging modalities for detecting metastases in patients with melanoma.

The Clinical Relevance of Methods for Handling Inconclusive Medical Test Results: Quantification of Uncertainty in Medical Decision-Making and Screening

BACKGROUND

although the existence of inconclusive medical test results or bio-markers is widely recognized, there are indications that this inherent diagnostic uncertainty is sometimes ignored. This paper discusses three methods for defining and determining inconclusive medical test results, which use different definitions and differ in clinical relevance.

METHODS

the TG-ROC (two graphs receiver operating characteristics) method is the easiest to use, while the grey zone method and the uncertain interval method require more extensive calculations.

RESULTS

this paper discusses the technical details of the methods, as well as advantages and disadvantages for their clinical use. TG-ROC and the grey zone method can help in the acquisition of high rates of diagnostic certainty, but can exclude large groups. The uncertain interval method can prevent decisions that are the most uncertain, invalid and unreliable, while excluding smaller groups.

CONCLUSIONS

the identification of uncertain test scores is relevant, because these scores indicate the need to obtain better information or to await further developments. The methods presented help to determine inconclusive test scores and can help to reduce erroneous decisions. However, further research and development is desirable.

Sex-specific performance of pre-imaging diagnostic algorithms for pulmonary embolism

Essentials Decision rules for pulmonary embolism are used indiscriminately despite possible sex-differences. Various pre-imaging diagnostic algorithms have been investigated in several prospective studies. When analysed at an individual patient data level the algorithms perform similarly in both sexes. Estrogen use and male sex were associated with a higher prevalence in suspected pulmonary embolism.

SUMMARY

Background In patients suspected of pulmonary embolism (PE), clinical decision rules are combined with D-dimer testing to rule out PE, avoiding the need for imaging in those at low risk. Despite sex differences in several aspects of the disease, including its diagnosis, these algorithms are used indiscriminately in women and men. Objectives To compare the performance, defined as efficiency and failure rate, of three pre-imaging diagnostic algorithms for PE between women and men: the Wells rule with fixed or with age-adjusted D-dimer cut-off, and a recently validated algorithm (YEARS). A secondary aim was to determine the sex-specific prevalence of PE. Methods Individual patient data were obtained from six studies using the Wells rule (fixed D-dimer, n = 5; age adjusted, n = 1) and from one study using the YEARS algorithm. All studies prospectively enrolled consecutive patients with suspected PE. Main outcomes were efficiency (proportion of patients in which the algorithm ruled out PE without imaging) and failure rate (proportion of patients with PE not detected by the algorithm). Outcomes were estimated using (multilevel) logistic regression models. Results The main outcomes showed no sex differences in any of the separate algorithms. With all three, the prevalence of PE was lower in women (OR, 0.66, 0.68 and 0.74). In women, estrogen use, adjusted for age, was associated with lower efficiency and higher prevalence and D-dimer levels. Conclusions The investigated pre-imaging diagnostic algorithms for patients suspected of PE show no sex differences in performance. Male sex and estrogen use are both associated with a higher probability of having the disease.

Random-effects meta-analysis of the clinical utility of tests and prediction models

The use of data from multiple studies or centers for the validation of a clinical test or a multivariable prediction model allows researchers to investigate the test's/model's performance in multiple settings and populations. Recently, meta-analytic techniques have been proposed to summarize discrimination and calibration across study populations. Here, we rather consider performance in terms of net benefit, which is a measure of clinical utility that weighs the benefits of true positive classifications against the harms of false positives. We posit that it is important to examine clinical utility across multiple settings of interest. This requires a suitable meta-analysis method, and we propose a Bayesian trivariate random-effects meta-analysis of sensitivity, specificity, and prevalence. Across a range of chosen harm-to-benefit ratios, this provides a summary measure of net benefit, a prediction interval, and an estimate of the probability that the test/model is clinically useful in a new setting. In addition, the prediction interval and probability of usefulness can be calculated conditional on the known prevalence in a new setting. The proposed methods are illustrated by 2 case studies: one on the meta-analysis of published studies on ear thermometry to diagnose fever in children and one on the validation of a multivariable clinical risk prediction model for the diagnosis of ovarian cancer in a multicenter dataset. Crucially, in both case studies the clinical utility of the test/model was heterogeneous across settings, limiting its usefulness in practice. This emphasizes that heterogeneity in clinical utility should be assessed before a test/model is routinely implemented.

Faecal immunochemical tests to triage patients with lower abdominal symptoms for suspected colorectal cancer referrals in primary care: a systematic review and cost-effectiveness analysis

BACKGROUND

Colorectal cancer (CRC) is the third most common cancer in the UK. Presenting symptoms that can be associated with CRC usually have another explanation. Faecal immunochemical tests (FITs) detect blood that is not visible to the naked eye and may help to select patients who are likely to benefit from further investigation.

OBJECTIVES

To assess the effectiveness of FITs [OC-Sensor (Eiken Chemical Co./MAST Diagnostics, Tokyo, Japan), HM-JACKarc (Kyowa Medex/Alpha Laboratories Ltd, Tokyo, Japan), FOB Gold (Sentinel/Sysmex, Sentinel Diagnostics, Milan, Italy), RIDASCREEN Hb or RIDASCREEN Hb/Hp complex (R-Biopharm, Darmstadt, Germany)] for primary care triage of people with low-risk symptoms.

METHODS

Twenty-four resources were searched to March 2016. Review methods followed published guidelines. Summary estimates were calculated using a bivariate model or a random-effects logistic regression model. The cost-effectiveness analysis considered long-term costs and quality-adjusted life-years (QALYs) that were associated with different faecal occult blood tests and direct colonoscopy referral. Modelling comprised a diagnostic decision model, a Markov model for long-term costs and QALYs that were associated with CRC treatment and progression, and a Markov model for QALYs that were associated with no CRC.

RESULTS

We included 10 studies. Using a single sample and 10 µg Hb/g faeces threshold, sensitivity estimates for OC-Sensor [92.1%, 95% confidence interval (CI) 86.9% to 95.3%] and HM-JACKarc (100%, 95% CI 71.5% to 100%) indicated that both may be useful to rule out CRC. Specificity estimates were 85.8% (95% CI 78.3% to 91.0%) and 76.6% (95% CI 72.6% to 80.3%). Triage using FITs could rule out CRC and avoid colonoscopy in approximately 75% of symptomatic patients. Data from our systematic review suggest that 22.5-93% of patients with a positive FIT and no CRC have other significant bowel pathologies. The results of the base-case analysis suggested minimal difference in QALYs between all of the strategies; no triage (referral straight to colonoscopy) is the most expensive. Faecal immunochemical testing was cost-effective (cheaper and more, or only slightly less, effective) compared with no triage. Faecal immunochemical testing was more effective and costly than guaiac faecal occult blood testing, but remained cost-effective at a threshold incremental cost-effectiveness ratio of £30,000. The results of scenario analyses did not differ substantively from the base-case. Results were better for faecal immunochemical testing when accuracy of the guaiac faecal occult blood test (gFOBT) was based on studies that were more representative of the correct population.

LIMITATIONS

Only one included study evaluated faecal immunochemical testing in primary care; however, all of the other studies evaluated faecal immunochemical testing at the point of referral. Further, validation data for the Faecal haemoglobin, Age and Sex Test (FAST) score, which includes faecal immunochemical testing, showed no significant difference in performance between primary and secondary care. There were insufficient data to adequately assess FOB Gold, RIDASCREEN Hb or RIDASCREEN Hb/Hp complex. No study compared FIT assays, or FIT assays versus gFOBT; all of the data included in this assessment refer to the clinical effectiveness of individual FIT methods and not their comparative effectiveness.

CONCLUSIONS

Faecal immunochemical testing is likely to be a clinically effective and cost-effective strategy for triaging people who are presenting, in primary care settings, with lower abdominal symptoms and who are at low risk for CRC. Further research is required to confirm the effectiveness of faecal immunochemical testing in primary care practice and to compare the performance of different FIT assays.

STUDY REGISTRATION

This study is registered as PROSPERO CRD42016037723.

FUNDING

The National Institute for Health Research Health Technology Assessment programme.

C-reactive protein for late-onset sepsis diagnosis in very low birth weight infants

BACKGROUND

Late-onset sepsis in very low birth weight (VLBW) infants is a diagnostic challenge. We aimed to evaluate the diagnostic utility of the C-Reactive protein (CRP) and the complete blood count (CBC) for late-onset sepsis in VLBW infants.

METHODS

In a 5-year retrospective cohort of 416 VLBW infants born at less than 1500 g, there were 590 separate late-onset sepsis evaluations. CRP and CBC were drawn at time of initial blood culture (T0), at 16-24 h (T24) and 40-48 h (T48) after. The positive cut-off values for abnormal values were the following: CRP ≥10 mg/L and CBC with at least one anomaly, including white blood cell count < 5000/mm³, immature neutrophil/total neutrophil ratio > 0.10, or platelet count < 100,000/uL. Sensitivity and specificity for predicting late-onset sepsis were calculated for each laboratory test and their combinations. Receiver operating characteristics curves were obtained for each test and for the absolute change from T0 to T24 in the laboratory value of CRP, white blood cell count and immature neutrophil/total neutrophil.

RESULTS

At T0, combining the CBC and the CRP had the highest sensitivity of 66% (95% confidence interval [CI], 58-73) compared to both individual tests for predicting late onset sepsis. At T24, CRP's sensitivity was 84% (95% CI, 78-89) and was statistically higher than the CBC's 59% (95% CI, 51-67). The combination of CBC at T0 and CRP at T24 offered the greatest sensitivity of 88% (95% CI, 82-92) and negative predictive value 93% (95% CI, 89-96), with fewer samples, compared to any other combination of tests. The area under the curve for the change in the white blood cell count from T0 to T24 was 0.82.

CONCLUSION

At initial sepsis evaluation (T0), both CBC and CRP should be performed to increase sensitivity. A highly negative predictive value is reachable with only two tests: a CBC at T0 and a CRP a T24.

Diagnosing occupational asthma

Making an accurate diagnosis of occupational asthma (OA) is, generally, important. The condition has not only significant health consequences for affected workers, but also substantial socio-economic impacts for workers, their employers and wider society. Missing a diagnosis of OA may lead to continued exposure to a causative agent and progressive worsening of disease; conversely, diagnosing OA when it is not present may lead to inappropriate removal from exposure and unnecessary financial and social consequences. While the most accurate investigation is specific inhalation challenge in an experienced centre, this is a scarce resource, and in many cases, reliance is on other tests. This review provides a technical dossier of the diagnostic value of the available methods which include an appropriate clinical history, the use of specific immunology and measurement of inflammatory markers, and various methods of relating functional changes in airway calibre to periods at work. It is recommended that these approaches are used iteratively and in judicious combination, in cognizance of the individual patient's circumstances and requirements. Based on available evidence, a working diagnostic algorithm is proposed that can be adapted to the suspected agent, purpose of diagnosis and available resources. For better or worse, many of the techniques - and their interpretation - are available only in specialized centres and where there is room for doubt, referral to such a centre is probably wise. Accordingly, the implementation or development of such specialized centres with appropriate equipment and expertise should greatly improve the diagnostic evaluation of work-related asthma.

Time is brain: Balancing risk in the treatment of presumed cardioembolic stroke

OBJECTIVE

To assess whether clinical criteria can differentiate between presumed embolic strokes and non-embolic strokes before the full etiologic workup.

METHODS

Between January 1, 2014 to December 30, 2015, patients with a diagnosis of stroke or transient ischemic attack were first classified clinically (without access to a cardiac assessment) as: 1. presumed embolic stroke defined as a combination of definite cardioembolic stroke and likely to be embolic stroke (no evidence of large/small artery atherosclerosis); 2. non-embolic strokes; i.e. small/large artery diseases and stroke due to other causes. Stroke etiology was reassessed after investigations and concordances between the early diagnosis and final classifications were analyzed.

RESULTS

77 patients with early diagnosis of presumed embolic strokes and 45 cases with non-embolic stroke (selected randomly) were enrolled. We were able to differentiate between presumed embolic strokes and non-embolic strokes with a high level of accuracy (sensitivity 81.40%, 95% CI: 71.55%-88.98%; specificity 80.56%, 95% CI: 63.98%-91.81%). A moderate level of agreement between initial and final diagnosis of embolic/non-embolic strokes (kappa 0.58, SE 0.08, p≤0.01) was observed. The results of carotid imaging improved the specificity and positive likelihood ratio of correct differentiation.

CONCLUSIONS

Those at high risk of embolism can be diagnosed clinically even before the completion of tests. This is a practical approach to distinguish patients at risk and help balance early risks of recurrence with those of short-term anticoagulation.

Faecal immunochemical tests (FIT) can help to rule out colorectal cancer in patients presenting in primary care with lower abdominal symptoms: a systematic review conducted to inform new NICE DG30 diagnostic guidance

BACKGROUND

This study has attempted to assess the effectiveness of quantitative faecal immunochemical tests (FIT) for triage of people presenting with lower abdominal symptoms, where a referral to secondary care for investigation of suspected colorectal cancer (CRC) is being considered, particularly when the 2-week criteria are not met.

METHODS

We conducted a systematic review following published guidelines for systematic reviews of diagnostic tests. Twenty-one resources were searched up until March 2016. Summary estimates were calculated using a bivariate model or a random-effects logistic regression model.

RESULTS

Nine studies are included in this review. One additional study, included in our systematic review, was provided as 'academic in confidence' and cannot be described herein. When FIT was based on a single faecal sample and a cut-off of 10 μg Hb/g faeces, sensitivity estimates indicated that a negative result using either the OC-Sensor or HM-JACKarc may be adequate to rule out nearly all CRC; the summary estimate of sensitivity for the OC-Sensor was 92.1% (95% confidence interval, CI 86.9-95.3%), based on four studies (n = 4091 participants, 176 with CRC), and the only study of HM-JACKarc to assess the 10 μg Hb/g faeces cut-off (n = 507 participants, 11 with CRC) reported a sensitivity of 100% (95% CI 71.5-100%). The corresponding specificity estimates were 85.8% (95% CI 78.3-91.0%) and 76.6% (95% CI 72.6-80.3%), respectively. When the diagnostic criterion was changed to include lower grades of neoplasia, i.e. the target condition included higher risk adenoma (HRA) as well as CRC, the rule-out performance of both FIT assays was reduced.

CONCLUSIONS

There is evidence to suggest that triage using FIT at a cut-off around 10 μg Hb/g faeces has the potential to correctly rule out CRC and avoid colonoscopy in 75-80% of symptomatic patients.

SYSTEMATIC REVIEW REGISTRATION

PROSPERO 42016037723.

Recommendations for reporting of systematic reviews and meta-analyses of diagnostic test accuracy: a systematic review

BACKGROUND

This study is to perform a systematic review of existing guidance on quality of reporting and methodology for systematic reviews of diagnostic test accuracy (DTA) in order to compile a list of potential items that might be included in a reporting guideline for such reviews: Preferred Reporting Items for Systematic Reviews and Meta-Analyses of Diagnostic Test Accuracy (PRISMA-DTA).

METHODS

Study protocol published on EQUATOR website. Articles in full text or abstract form that reported on any aspect of reporting systematic reviews of diagnostic test accuracy were eligible for inclusion. We used the Ovid platform to search Ovid MEDLINE®, Ovid MEDLINE® In-Process & Other Non-Indexed Citations and Embase Classic+Embase through May 5, 2016. The Cochrane Methodology Register in the Cochrane Library (Wiley version) was also searched. Title and abstract screening followed by full-text screening of all search results was performed independently by two investigators. Guideline organization websites, published guidance statements, and the Cochrane Handbook for Diagnostic Test Accuracy were also searched. Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) and Standards for Reporting Diagnostic Accuracy (STARD) were assessed independently by two investigators for relevant items.

RESULTS

The literature searched yielded 6967 results; 386 were included after title and abstract screening and 203 after full-text screening. After reviewing the existing literature and guidance documents, a preliminary list of 64 items was compiled into the following categories: title (three items); introduction (two items); methods (35 items); results (13 items); discussion (nine items), and disclosure (two items).

CONCLUSION

Items on the methods and reporting of DTA systematic reviews in the present systematic review will provide a basis for generating a PRISMA extension for DTA systematic reviews.

Latent Class Analysis of Noninvasive Methods and Liver Biopsy in Chronic Hepatitis C: An Approach without a Gold Standard

Aims

To evaluate the applicability of the Latent Class Analysis (LCA) and accuracy of transient elastography (TE), aspartate-to-platelet-ratio-index (APRI), enhanced liver fibrosis (ELF), and liver biopsy (LB) for liver fibrosis assessment in a model without a gold standard.

Methods

Significant fibrosis was defined as TE ≥ 7.1 kPa, APRI ≥ 1.5, ELF ≥ 9.37, or LB METAVIR F ≥ 2. Cirrhosis was defined as TE ≥ 12.5 kPa, APRI ≥ 2.0, ELF ≥ 10.31, or LB as METAVIR F = 4.

Results

117 patients with chronic hepatitis C were included. In the LCA, for significant fibrosis the sensitivities and specificities (95% CI) were 0.92 (0.86–0.98) and 0.79 (0.72–0.86) for TE; 0.47 (0.40–0.54) and 0.99 (0.95–1.00) for APRI; 0.81 (0.74–0.88) and 0.78 (0.71–0.85) for ELF; and 0.86 (0.68–1.00) and 0.91 (0.79–1.00) for LB. For cirrhosis, the sensitivities and specificities were 0.92 (0.76–1.00) and 0.94 (0.91–0.97) for TE; 0.57 (0.37–0.77) and 0.97 (0.93–1.00) for APRI; 0.94 (0.84–1.00) and 0.88 (0.82–0.94) for ELF; and 0.30 (0.12–0.48) and 1.00 for LB.

Conclusion

LCA was useful to evaluate accuracy of methods for liver fibrosis staging. Sensitivities and specificities of noninvasive methods were increased in LCA compared to the use of LB as the gold standard.

Alternatives to Traditional Per-Oral Endoscopy for Screening

Barrett's esophagus (BE) predisposes patients to esophageal adenocarcinoma. 3 to 6% of individuals with gastro-esophageal reflux disease are estimated to have BE but only 20 to 25% of BE patients are currently diagnosed. The current gold standard for diagnosis of BE is per-oral upper GI endoscopy. As this is not suitable for large-scale screening, a number of alternative methods are currently being investigated: transnasal and video capsule endoscopy, endomicroscopy, cell collection devices like the cytosponge and biomarkers. Some of these are promising, however, well powered studies carried out in relevant screening populations are needed.

Systematic reviews of diagnostic tests in endocrinology: an audit of methods, reporting, and performance

BACKGROUND

Systematic reviews provide clinicians and policymakers estimates of diagnostic test accuracy and their usefulness in clinical practice. We identified all available systematic reviews of diagnosis in endocrinology, summarized the diagnostic accuracy of the tests included, and assessed the credibility and clinical usefulness of the methods and reporting.

METHODS

We searched Ovid MEDLINE, EMBASE, and Cochrane CENTRAL from inception to December 2015 for systematic reviews and meta-analyses reporting accuracy measures of diagnostic tests in endocrinology. Experienced reviewers independently screened for eligible studies and collected data. We summarized the results, methods, and reporting of the reviews. We performed subgroup analyses to categorize diagnostic tests as most useful based on their accuracy.

RESULTS

We identified 84 systematic reviews; half of the tests included were classified as helpful when positive, one-fourth as helpful when negative. Most authors adequately reported how studies were identified and selected and how their trustworthiness (risk of bias) was judged. Only one in three reviews, however, reported an overall judgment about trustworthiness and one in five reported using adequate meta-analytic methods. One in four reported contacting authors for further information and about half included only patients with diagnostic uncertainty.

CONCLUSION

Up to half of the diagnostic endocrine tests in which the likelihood ratio was calculated or provided are likely to be helpful in practice when positive as are one-quarter when negative. Most diagnostic systematic reviews in endocrine lack methodological rigor, protection against bias, and offer limited credibility. Substantial efforts, therefore, seem necessary to improve the quality of diagnostic systematic reviews in endocrinology.

Overinterpretation of Research Findings: Evidence of "Spin" in Systematic Reviews of Diagnostic Accuracy Studies

BACKGROUND

We wished to assess the frequency of overinterpretation in systematic reviews of diagnostic accuracy studies.

METHODS

MEDLINE was searched through PubMed from December 2015 to January 2016. Systematic reviews of diagnostic accuracy studies in English were included if they reported one or more metaanalyses of accuracy estimates. We built and piloted a list of 10 items that represent actual overinterpretation in the abstract and/or full-text conclusion, and a list of 9 items that represent potential overinterpretation. Two investigators independently used the items to score each included systematic review, with disagreements resolved by consensus.

RESULTS

We included 112 systematic reviews. The majority had a positive conclusion regarding the accuracy or clinical usefulness of the investigated test in the abstract (n = 83; 74%) and full-text (n = 83; 74%). Of the 112 reviews, 81 (72%) contained at least 1 actual form of overinterpretation in the abstract, and 77 (69%) in the full-text. This was most often a "positive conclusion, not reflecting the reported summary accuracy estimates," in 55 (49%) abstracts and 56 (50%) full-texts and a "positive conclusion, not taking high risk of bias and/or applicability concerns into account," in 47 abstracts (42%) and 26 full-texts (23%). Of these 112 reviews, 107 (96%) contained a form of potential overinterpretation, most frequently "nonrecommended statistical methods for metaanalysis performed" (n = 57; 51%).

CONCLUSIONS

Most recent systematic reviews of diagnostic accuracy studies present positive conclusions and a majority contain a form of overinterpretation. This may lead to unjustified optimism about test performance and erroneous clinical decisions and recommendations.

Cognitive screening tools for identification of dementia in illiterate and low-educated older adults, a systematic review and meta-analysis

BACKGROUND

The majority of older adults with dementia live in low- and middle-income countries (LMICs). Illiteracy and low educational background are common in older LMIC populations, particularly in rural areas, and cognitive screening tools developed for this setting must reflect this. This study aimed to review published validation studies of cognitive screening tools for dementia in low-literacy settings in order to determine the most appropriate tools for use.

METHOD

A systematic search of major databases was conducted according to PRISMA guidelines. Validation studies of brief cognitive screening tests including illiterate participants or those with elementary education were eligible. Studies were quality assessed using the QUADAS-2 tool. Good or fair quality studies were included in a bivariate random-effects meta-analysis and a hierarchical summary receiver operating characteristic (HSROC) curve constructed.

RESULTS

Forty-five eligible studies were quality assessed. A significant proportion utilized a case-control design, resulting in spectrum bias. The area under the ROC (AUROC) curve was 0.937 for community/low prevalence studies, 0.881 for clinic based/higher prevalence studies, and 0.869 for illiterate populations. For the Mini-Mental State Examination (MMSE) (and adaptations), the AUROC curve was 0.853.

CONCLUSION

Numerous tools for assessment of cognitive impairment in low-literacy settings have been developed, and tools developed for use in high-income countries have also been validated in low-literacy settings. Most tools have been inadequately validated, with only MMSE, cognitive abilities screening instrument (CASI), Eurotest, and Fototest having more than one published good or fair quality study in an illiterate or low-literate setting. At present no screening test can be recommended.

Validity of the three apathy items of the Geriatric Depression Scale (GDS-3A) in measuring apathy in older persons

OBJECTIVE

The Geriatric Depression Scale (GDS)-3A, a three-item subset of the GDS-15, is increasingly used as a measure for apathy in research settings to assess factors associating with this neuropsychiatric syndrome. We aimed to assess how accurately the GDS-3A discriminates between presence and absence of apathy in two populations of community-dwelling older persons, using the Apathy Scale as reference standard.

METHODS

Baseline data were used from 427 participants of the Discontinuation of Antihypertensive Treatment in Elderly people (DANTE) Study Leiden and 1118 participants of the PROactive Management Of Depression in the Elderly (PROMODE) Study, all ≥75 years and with available GDS-3A and Apathy Scale measurements. A cut-off score of ≥14 was used for presence of apathy according to the Apathy Scale. Areas under the receiver operating characteristic curve (AUC) were calculated. Based on the likelihood ratios for GDS-3A scores, a cut-off of ≥2 was used for presence of apathy according to the GDS-3A to calculate test characteristics.

RESULTS

The AUC was 0.68 (95% confidence interval 0.62-0.73) in the DANTE Study and 0.72 (0.67-0.77) in the PROMODE Study. In the DANTE Study sensitivity was 29.3% (21.4-38.1) and specificity was 88.5% (84.4-91.8), whereas in the PROMODE Study sensitivity was 32.8% (24.5-41.1) and specificity 92.6% (90.9-94.2). Stratification on population characteristics did not yield more favourable test characteristics.

CONCLUSION

The GDS-3A has low sensitivity and high specificity as a measure of apathy in two populations of older persons. Using the GDS-3A in research might yield estimates biassed towards the null in case of non-differential misclassification. Copyright © 2016 John Wiley & Sons, Ltd.

Accuracy of Daily Lung Ultrasound for the Detection of Pulmonary Edema Following Subarachnoid Hemorrhage

BACKGROUND

Early detection of pulmonary edema is vital to appropriate fluid management following subarachnoid hemorrhage (SAH). Lung ultrasound (LUS) has been shown to accurately identify pulmonary edema in patients with acute respiratory failure (ARF). Our objective was to determine the accuracy of daily screening LUS for the detection of pulmonary edema following SAH.

METHODS

Screening LUS was performed in conjunction with daily transcranial doppler for SAH patients within the delayed cerebral ischemia (DCI) risk period in our neuroICU. We reviewed records of SAH patients admitted 7/2012-5/2014 who underwent bilateral LUS on at least 5 consecutive days. Ultrasound videos were reviewed by an investigator blinded to the final diagnosis. "B+ lines" were defined as ≥3 B-lines on LUS. Two other investigators blinded to ultrasound results determined whether pulmonary edema with ARF (PE-ARF) was present during the period of evaluation on the basis of independent chart review, with a fourth investigator performing adjudication in the event of disagreement. The diagnostic accuracy of B+ lines for the detection of PE-ARF and RPE was determined.

RESULTS

Of 59 patients meeting criteria for inclusion, 21 (36%) had PE-ARF and 26 (44%) had B+ lines. Kappa for inter-rater agreement was 0.821 (p < 0.0001) for clinical diagnosis of PE-ARF between the two investigators. B+ lines demonstrated sensitivity 90% (95% CI 70-99%) and specificity 82% (66-92%), for PE-ARF. Median days from B+ lines onset to PE-ARF was 1 (IQR 0-1).

CONCLUSION

Screening LUS was a sensitive test for the detection of symptomatic pulmonary edema following SAH and may assist with fluid titration during the risk period for DCI.

Texture analysis of the developing human brain using customization of a knowledge-based system

Background: Pattern recognition software originally designed for geospatial and other technical applications could be trained by physicians and used as texture-analysis tools for evidence-based practice, in order to improve diagnostic imaging examination during pregnancy.Methods: Various machine-learning techniques and customized datasets were assessed for training of an integrable knowledge-based system (KBS), to determine a hypothetical methodology for texture classification of closely-related anatomical structures in fetal brain magnetic resonance (MR) images. Samples were manually categorized according to the magnetic field of the MRI scanner (i.e. 1.5-tesla (1.5T), 3-tesla (3T)), rotational planes (i.e. coronal, sagittal and axial), and signal weighting (i.e. spin-lattice, spin-spin, relaxation, proton density). In the machine-learning sessions, the operator manually selected relevant regions of interest (ROI) in 1.5/3T MR images. Semi-automatic procedures in MaZda/B11 were performed to determine optimal parameter sets for ROI classification. Four classes were defined: ventricles, thalamus, grey matter, and white matter. Various textures analysis methods were tested. The KBS performed automatic data pre-processing and semi-automatic classification of ROIs.Results: After testing 3456 ROIs, statistical binary classification revealed that combination of reduction techniques with linear discriminant algorithms (LDA) or nonlinear discriminant algorithms (NDA) yielded the best scoring in terms of sensitivity (both 100%, 95% CI: 99.79-100), specificity (both 100%, 95% CI: 99.79-100) and Fisher coefficient (≈E+4, ≈E+5, respectively). Conclusions: LDA and NDA in MaZda can be useful data mining tools for screening a population of interest subjected to a clinical test.