Evaluation of diagnostic procedures

DxGoals: A Software Tool for Determining and Analyzing Clinically Meaningful Classification Accuracy Goals for Diagnostic Tests

BACKGROUND

To evaluate diagnostic tests for low prevalence conditions, classification accuracy metrics such as sensitivity, specificity, and positive likelihood ratio (PLR) and negative likelihood ratio (NLR) are advantageous because they are prevalence-independent and thus estimable in studies enriched for the condition. However, classification accuracy goals are often chosen without a clear understanding of whether they are clinically meaningful. Pennello (2021) proposed a risk stratification framework for determining classification accuracy goals. A software application is needed to determine the goals and provide data analysis.

METHODS

We introduce DxGoals, a freely available, R-Shiny software application for determining, visualizing, and analyzing classification accuracy goals for diagnostic tests. Given prevalence p for the target condition and specification that a test's positive and negative predictive values PPVand NPV=1-cNPV should satisfy PPV>PPV* and cNPV

RESULTS

We illustrate DxGoals on tests for penicillin allergy, ovarian cancer, and cervical cancer. The inputs cNPV*,p, and PPV* were informed by clinical management guidelines.

CONCLUSIONS

DxGoals facilitates determination, visualization, and analysis of clinically meaningful standalone and comparative classification accuracy goals. It is a potentially useful tool for diagnostic test evaluation.

Collapse

Key Words Collapse

MESH Headings

• Humans
• Software
• Sensitivity and Specificity
• Bayes Theorem
• Diagnostic Tests, Routine/standards
• Diagnostic Tests, Routine/methods
• Predictive Value of Tests

Collapse

Grants Collapse

Affiliation(s)

Ngoc-Ty Nguyen U.S. Food and Drug Administration, Center for Biologics Evaluation and Research, Silver Spring, MD, United States
Gene A Pennello U.S. Food and Drug Administration, Center for Devices and Radiological Health, Silver Spring, MD, United States

Collapse

A review of methods for the analysis of diagnostic tests performed in sequence

BACKGROUND

Many clinical pathways for the diagnosis of disease are based on diagnostic tests that are performed in sequence. The performance of the full diagnostic sequence is dictated by the diagnostic performance of each test in the sequence as well as the conditional dependence between them, given true disease status. Resulting estimates of performance, such as the sensitivity and specificity of the test sequence, are key parameters in health-economic evaluations. We conducted a methodological review of statistical methods for assessing the performance of diagnostic tests performed in sequence, with the aim of guiding data analysts towards classes of methods that may be suitable given the design and objectives of the testing sequence.

METHODS

We searched PubMed, Scopus and Web of Science for relevant papers describing methodology for analysing sequences of diagnostic tests. Papers were classified by the characteristics of the method used, and these were used to group methods into themes. We illustrate some of the methods using data from a cohort study of repeat faecal immunochemical testing for colorectal cancer in symptomatic patients, to highlight the importance of allowing for conditional dependence in test sequences and adjustment for an imperfect reference standard.

RESULTS

Five overall themes were identified, detailing methods for combining multiple tests in sequence, estimating conditional dependence, analysing sequences of diagnostic tests used for risk assessment, analysing test sequences in conjunction with an imperfect or incomplete reference standard, and meta-analysis of test sequences.

CONCLUSIONS

This methodological review can be used to help researchers identify suitable analytic methods for studies that use diagnostic tests performed in sequence.

Diagnostic accuracy of MRI, CT, and [18F]FDG-PET-CT in detecting lymph node metastases in clinically early-stage cervical cancer - a nationwide Dutch cohort study

OBJECTIVES

Imaging is increasingly used to assess lymph node involvement in clinically early-stage cervical cancer. This retrospective study aimed to evaluate the diagnostic accuracy of MRI, CT, and [18F]FDG-PET-CT.

METHODS

Women with International Federation of Gynaecology and Obstetrics (FIGO) 2009 stage IA2-IIA cervical cancer and pretreatment imaging between 2009 and 2017 were selected from the Netherlands Cancer Registry. Patient-based and region-based (i.e. pelvic and common iliac) nodal status was extracted from radiology reports. Pathology results were considered the reference standard for calculating accuracy indices. Multiple imputation was used for missing pathology to limit verification bias risk.

RESULTS

Nodal assessment was performed in 1676 patients with MRI, 926 with CT, and 379 with [18F]FDG-PET-CT, with suspicious nodes detected in 17%, 16%, and 48%, respectively. [18F]FDG-PET-CT was used to confirm MRI/CT results in 95% of patients. Pathology results were imputed for 30% of patients. [18F]FDG-PET-CT outperformed MRI and CT in detecting patient-based nodal metastases with sensitivities of 80%, 48%, and 40%, and AUCs of 0.814, 0.706, and 0.667, respectively, but not in specificity: 79%, 92%, and 92%. Region-based analyses showed similar indices in the pelvic region, but worse performance in the common iliac region with AUCs of 0.575, 0.554, and 0.517, respectively.

CONCLUSIONS

[18F]FDG-PET-CT outperformed MRI and CT in detecting nodal metastases, which may be related to its use as a verification modality. However, MRI and CT had the highest specificity. As MRI is generally performed routinely to assess local and regional spread of cervical cancer, [18F]FDG-PET-CT can be used to confirm suspicious nodes.

CRITICAL RELEVANCE STATEMENT

Accurate assessment of the nodal status in clinically early-stage cervical cancer is essential for tumour staging, treatment decision making and prognosis.

KEY POINTS

• The accuracy of MRI, CT or [18F]FDG-PET-CT for nodal staging in early cervical cancer is a subject of discussion. • Overall, [18F]FDG-PET-CT outperformed MRI, followed by CT, when used as a verification modality. • Staging with MRI and the addition of [18F]FDG-PET-CT to verify high-risk cases seems to be a good approach.

Development of a one-step reverse transcription-quantitative polymerase chain reaction assay for the detection of porcine reproductive and respiratory syndrome virus

Porcine reproductive and respiratory syndrome (PRRS) caused by PRRS virus (PRRSV) is an important disease that severely affects the swine industry and, therefore, warrants rapid and accurate diagnosis for its control. Despite the progress in developing diagnostic tools, including polymerase chain reaction (PCR)-based methods such as reverse transcription quantitative PCR (RT-qPCR) to diagnose PRRSV infection, its diagnosis at the genetic level is challenging because of its high genetic variability. Nevertheless, RT-qPCR is the easiest and fastest method for diagnosing PRRSV. Therefore, this study aimed to develop an RT-qPCR assay for rapid and accurate diagnosis of PRRSV by encompassing all publicly available PRRSV sequences. The developed assay using highly specific primers and probes could detect up to 10 copies of PRRSV-1 and -2 subtypes. Furthermore, a comparison of the performance of the developed assay with those of two commercial kits widely used in South Korea demonstrated the higher efficiency of the developed assay in detecting PRRSV infections in field samples. For PRRSV-1 detection, the developed assay showed a diagnostic agreement of 97.7% with the results of ORF5 sequencing, while for commercial kits, it showed 95.3% and 72.1% agreement. For PRRSV-2, the developed assay showed a diagnostic agreement of 97.7%, whereas the commercial kits showed 93% and 90.7% agreement. In conclusion, we developed an assay with higher accuracy than those of the tested commercial kits, which will contribute markedly to global PRRSV control.

Computational Evidence for Laboratory Diagnostic Pathways: Extracting Predictive Analytes for Myocardial Ischemia from Routine Hospital Data

Background: Laboratory parameters are critical parts of many diagnostic pathways, mortality scores, patient follow-ups, and overall patient care, and should therefore have underlying standardized, evidence-based recommendations. Currently, laboratory parameters and their significance are treated differently depending on expert opinions, clinical environment, and varying hospital guidelines. In our study, we aimed to demonstrate the capability of a set of algorithms to identify predictive analytes for a specific diagnosis. As an illustration of our proposed methodology, we examined the analytes associated with myocardial ischemia; it was a well-researched diagnosis and provides a substrate for comparison. We intend to present a toolset that will boost the evolution of evidence-based laboratory diagnostics and, therefore, improve patient care. Methods: The data we used consisted of preexisting, anonymized recordings from the emergency ward involving all patient cases with a measured value for troponin T. We used multiple imputation technique, orthogonal data augmentation, and Bayesian Model Averaging to create predictive models for myocardial ischemia. Each model incorporated different analytes as cofactors. In examining these models further, we could then conclude the predictive importance of each analyte in question. Results: The used algorithms extracted troponin T as a highly predictive analyte for myocardial ischemia. As this is a known relationship, we saw the predictive importance of troponin T as a proof of concept, suggesting a functioning method. Additionally, we could demonstrate the algorithm's capabilities to extract known risk factors of myocardial ischemia from the data. Conclusion: In this pilot study, we chose an assembly of algorithms to analyze the value of analytes in predicting myocardial ischemia. By providing reliable correlations between the analytes and the diagnosis of myocardial ischemia, we demonstrated the possibilities to create unbiased computational-based guidelines for laboratory diagnostics by using computational power in today's era of digitalization.

Detection of atrial fibrillation in primary care with radial pulse palpation, electronic blood pressure measurement and handheld single-lead electrocardiography: a diagnostic accuracy study

OBJECTIVE

To determine the diagnostic accuracy of three tests-radial pulse palpation, an electronic blood pressure monitor and a handheld single-lead ECG device-for opportunistic screening for unknown atrial fibrillation (AF).

DESIGN

We performed a diagnostic accuracy study in the intention-to-screen arm of a cluster randomised controlled trial aimed at opportunistic screening for AF in general practice. We performed radial pulse palpation, followed by electronic blood pressure measurement (WatchBP Home A) and handheld ECG (MyDiagnostick) in random order. If one or more index tests were positive, we performed a 12-lead ECG at shortest notice. Similarly, to limit verification bias, a random sample of patients with three negative index tests received this reference test. Additionally, we analysed the dataset using multiple imputation. We present pooled diagnostic parameters.

SETTING

47 general practices participated between September 2015 and August 2018.

PARTICIPANTS

In the electronic medical record system of the participating general practices (n=47), we randomly marked 200 patients of ≥65 years without AF. When they visited the practice for any reason, we invited them to participate. Exclusion criteria were terminal illness, inability to give informed consent or visit the practice or having a pacemaker or an implantable cardioverter-defibrillator.

OUTCOMES

Diagnostic accuracy of individual tests and test combinations to detect unknown AF.

RESULTS

We included 4339 patients; 0.8% showed new AF. Sensitivity and specificity were 62.8% (range 43.1%-69.7%) and 91.8% (91.7%-91.8%) for radial pulse palpation, 70.0% (49.0%-80.6%) and 96.5% (96.3%-96.7%) for electronic blood pressure measurement and 90.1% (60.8%-100%) and 97.9% (97.8%-97.9%) for handheld ECG, respectively. Positive predictive values were 5.8% (5.3%-6.1%), 13.8% (12.2%-14.8%) and 25.2% (24.2%-25.8%), respectively. All negative predictive values were ≥99.7%.

CONCLUSION

In detecting AF, electronic blood pressure measurement (WatchBP Home A), but especially handheld ECG (MyDiagnostick) showed better diagnostic accuracy than radial pulse palpation.

TRIAL REGISTRATION NUMBER

Netherlands Trial Register No. NL4776 (old NTR4914).

Validation of a delirium predictive model in patients admitted to surgical intensive care units: a multicentre prospective observational cohort study

OBJECTIVE

To internally and externally validate a delirium predictive model for adult patients admitted to intensive care units (ICUs) following surgery.

DESIGN

A prospective, observational, multicentre study.

SETTING

Three university-affiliated teaching hospitals in Thailand.

PARTICIPANTS

Adults aged over 18 years were enrolled if they were admitted to a surgical ICU (SICU) and had the surgery within 7 days before SICU admission.

MAIN OUTCOME MEASURES

Postoperative delirium was assessed using the Thai version of the Confusion Assessment Method for the ICU. The assessments commenced on the first day after the patient's operation and continued for 7 days, or until either discharge from the ICU or the death of the patient. Validation was performed of the previously developed delirium predictive model: age+(5×SOFA)+(15×benzodiazepine use)+(20×DM)+(20×mechanical ventilation)+(20×modified IQCODE>3.42).

RESULTS

In all, 380 SICU patients were recruited. Internal validation on 150 patients with the mean age of 75±7.5 years resulted in an area under a receiver operating characteristic curve (AUROC) of 0.76 (0.683 to 0.837). External validation on 230 patients with the mean age of 57±17.3 years resulted in an AUROC of 0.85 (0.789 to 0.906). The AUROC of all validation cohorts was 0.83 (0.785 to 0.872). The optimum cut-off value to discriminate between a high and low probability of postoperative delirium in SICU patients was 115. This cut-off offered the highest value for Youden's index (0.50), the best AUROC, and the optimum values for sensitivity (78.9%) and specificity (70.9%).

CONCLUSIONS

The model developed by the previous study was able to predict the occurrence of postoperative delirium in critically ill surgical patients admitted to SICUs.

TRIAL REGISTRATION NUMBER

Thai Clinical Trail Registry (TCTR20180105001).

Rational development and application of biomarkers in the field of autoimmunity: A conceptual framework guiding clinicians and researchers

Clear guidance is needed in the development and implementation of laboratory biomarkers in medicine. So far, no standardized phased approach is established that would pilot researchers and clinicians in this process. This leads to often incompletely validated biomarkers, which can bear the consequence of wrong applications, misinterpretation and inadequate management in the clinical context.

In this conceptual article, we describe a stepwise approach to develop and comprehensively validate laboratory biomarkers. We will delineate basic steps including technical performance, pre-analytical issues, and biological variation, as well as advanced aspects of biomarker utility comprising interpretability, diagnostic and prognostic accuracy, and health-care outcomes. These aspects will be illustrated by using well-known examples from the field of immunology.

The application of this conceptual framework will guide researchers in conducting meaningful projects to develop and evaluate biomarkers for the use in clinical practice. Furthermore, clinicians will be able to adequately interpret pre-clinical and clinical diagnostic literature and rationally apply biomarkers in clinical practice. Improvement in the implementation and application of biomarkers might relevantly change the management and outcomes of our patients for the better.

Decision-making given surrogate outcomes

BACKGROUND

Opinions differ about the extent to which intervention research should and can directly assess the main patient-important health outcomes, what role surrogate endpoints can play, and which requirements should then apply to the scientific underpinning of clinical and policy decisions.

METHOD

In a commentary we elaborate on this and provide guidance for dealing with related dilemmas.

CONCLUSIONS

Ethical, methodological and practical reasons for decision making based on surrogate endpoints can be that (1) reaching the intended patient-important health outcome would take too long to await direct RCT-based evidence, (2) experimental conditions have limited sustainability over time; and (3) the plausibility of an intervention's clinical efficacy, given the already available evidence regarding surrogate endpoints, goes beyond equipoise. Given an expected increase of interventions with a long term patient-important health outcome perspective, dealing with surrogate endpoints will remain an important challenge. Appropriately dealing with a surrogate endpoint includes (1) the assessment of its predictive value for the intended patient-important outcome, where GRADE guidelines for assessing 'indirectness' and 'causal chain analysis' can be helpful; (2) transparency of (absence of) evidence; (3) adequately updating the 'knowledge mosaic'; (4) weighing different perspectives and values, and (5) monitoring whether adjustments need to be made. The remaining level of uncertainty must be balanced against the urgency of clinical or societal decision making and the disadvantages of postponing this. Criteria for using surrogate endpoints are suggested. Patients, citizens and policy makers can be involved in agreeing upon these criteria.

Identifying the mechanisms that contribute to safe and effective electronic test result management systems- a multisite qualitative study

OBJECTIVE

Suboptimal design of health information technology (IT) systems can lead to the introduction of errors in the diagnostic process. We aimed to identify mechanisms that can affect the safety and effectiveness of these systems in hospital settings thus contributing to the building of an explicit and replicable understanding of the variables that can affect the functioning of IT systems.

MATERIALS AND METHODS

This qualitative study drew from observations and semistructured interviews from a purposive sample of 46 participants (26 emergency department and 20 laboratory and medical imaging staff) across 3 Australian hospitals. Iterative, inductive coding of the data led to the development of higher-level themes based on relationships between codes.

RESULTS

Two overarching themes emerged: (1) usability and safety of the electronic test result management system; and (2) system redesign considerations about who is meant to follow up, when and how. The usability and safety of digital systems and the way these systems deal with accountability processes are triggered by mechanisms that are contextually dependent.

DISCUSSION

Our findings highlighted the multitransactional nature of the test result management process involving numerous healthcare professionals across different settings. This communication requires integration of the systems utilized by different departments and transparency of the test result follow-up process to facilitate clear lines of responsibility and accountability.

CONCLUSION

Identifying mechanisms that shape the functionality and sustainability of electronic result management can offer a valuable appreciation of key elements that need to be accounted for, and the circumstances in which they need to operate effectively.

Correlation Between Clinical Diagnosis, MRI, and Arthroscopy in Diagnosing Shoulder Pathology

Introduction

Shoulder disorders are frequently encountered by clinicians and are a common cause of musculoskeletal pain in the general population. Clinical tests specific to each shoulder pathology, MRI, and arthroscopy are the most relied upon modalities of diagnosis used by many clinicians. The aim of this study was to correlate clinical tests and MRI with arthroscopy as the gold standard and whether a negative MRI with a positive clinical test could justify an arthroscopy.

Materials and methods

A total of 120 consecutive patients who had a history of shoulder pain or instability were evaluated by clinical tests and MRI, and underwent arthroscopy. Based on the confirmatory findings of arthroscopy, they were classified as True Positive (TP), True Negative (TN), False Positive (FP) and False Negative (FN) for each modality i.e., clinical tests and MRI.

Results

Clinical assessment of rotator cuff tears in comparison to arthroscopy yielded a sensitivity of 96.88%, specificity of 92.86% and diagnostic accuracy of 95%, whilst MRI had a sensitivity of 90.62%, specificity of 92.86% and diagnostic accuracy of 91.67%. In anterior labral lesions, clinical assessment had a sensitivity of 94.44%, specificity of 97.62 % and diagnostic accuracy of 96.67%, whilst MRI had a sensitivity of 83.33%, specificity of 92.86%, with diagnostic accuracy of 90%. Interestingly, in the clinical assessment of superior labral tear from anterior to posterior (SLAP) lesions, a sensitivity of 90%, specificity of 95%, and diagnostic accuracy of 93.33% were observed while MRI had a sensitivity of 60%, specificity of 92.50%, and diagnostic accuracy of 81.67%.

Conclusion

On the basis of these results, clinical assessment appears to be an effective tool in diagnosing shoulder pathologies, whereas MRI, though reliable in the identification of rotator cuff tears and instability, does not identify patients with SLAP lesions effectively. This study reinforces the importance of a good clinical examination of the shoulder, especially in chronic pain and an uncertain MRI, therefore improving patient management.

Diagnostic Value of Intraoperative Frozen Section in Breast-Conserving Surgery: A Systematic Review and Meta-analysis

Context: According to previous studies, using the frozen section procedure during breast surgery reduces the rate of error and the need for re-surgery. We aimed at performing a comprehensive systematic review and meta-analysis to provide reliable evidence on the diagnostic value of frozen section procedures in breast-conserving surgery (BCS). Data Sources: A thorough search was performed in PubMed, Embase, Cochrane Library, and Web of Science databases for human diagnostic studies that used the frozen section in BCS. Meta-analyses were done to find the sensitivity, specificity, accuracy, positive predictive value (PPV), negative predictive value (NPV), positive likelihood ratio (PLR), and negative likelihood ratio (NLR). Study Selection: Human diagnostic studies used the frozen section in breast-conserving surgery and studies that reported the sensitivity and specificity of the frozen section in BCS or contained data that could be calculated the desired parameters were selected for this meta-analysis. Data Extraction: Assessment of studies quality was done and data was extracted from included papers. Then, the Quality Assessment of Diagnostic Accuracy Studies (QUADAS-2) tool was used to assess the quality of included papers. Results: Thirty-five papers were entered into our study. The meta-analysis indicated the high sensitivity (83.47, 95%CI 79.61 - 87.32) and specificity (99.29, 95%CI 98.89 - 99.68) for the frozen section in BCS, which resulted in an accuracy of 93.77 (95%CI 92.45 - 95.10). We also found a significant PPV (93.26, 95%CI 91.25 - 95.27), NPV (92.17, 95%CI 90.22 - 94.11), PLR (7.99, 95%CI 6.01 - 9.96), and NLR (0.18, 95%CI 0.14 - 0.23). Conclusions: The findings showed that intraoperative frozen section analysis has high sensitivity and specificity for evaluating lumpectomy margins in patients with early-stage breast cancer and significantly reduces the need for re-operation. Accordingly, re-operation costs are not imposed on the patient and reduce the anxiety of the patients.

Why is there variation in test ordering practices for patients presenting to the emergency department with undifferentiated chest pain? A qualitative study

INTRODUCTION

Up to one-third of laboratory tests ordered in the ED for adults presenting with undifferentiated chest pain are generally not indicated by current Australian guidelines. This study set out to undertake a qualitative investigation of clinician perceptions to identify the reasons for variations in pathology requesting.

METHODS

For this study, we draw on data from semistructured interviews (n=38) conducted in the EDs and laboratories across three hospitals as part of a larger study on the test result management process from test request to result follow-up. Thematic analysis was conducted to determine what aspects of the clinical routines and environment might contribute to variations in pathology requesting. Informed by the findings from the analysis, targeted questions were developed and further focus groups (n=5) were held with clinicians, hospital management and electronic medical record (eMR) analysts to investigate in more detail the reasons for requesting outside of guidelines.

RESULTS

Participants cited four main reasons for ordering outside of guidelines. Clinicians requested tests outside of guidelines and the ED scope of practice to facilitate the patient journey along the broader continuum of care, including admission to hospital or transfer to another site. Clinicians were also faced with multiple and inconsistent guidelines regarding appropriate test selection. Limited access to in-house specialty and diagnostic services also influenced ordering patterns in smaller non-referral hospitals. Finally, certain features of the current electronic ordering framework within the eMR facilitated overordering and failed to impose any real restrictions on ordering inappropriately or outside of scope of practice.

CONCLUSION

Beyond the standardisation of pathology requesting advice across electronic decision support, order sets and guidelines, attempts to address issues related to the appropriateness and variation of laboratory test ordering should consider local and systemic factors which also shape the ordering process.

Diagnostic Informatics-The Role of Digital Health in Diagnostic Stewardship and the Achievement of Excellence, Safety, and Value

Diagnostic investigations (pathology laboratory and medical imaging) aim to: increase certainty of the presence or absence of disease by supporting the process of differential diagnosis; support clinical management; and monitor a patient's trajectory (e. g., disease progression or response to treatment). Digital health can be defined as the collection, storage, retrieval, transmission, and utilization of data, information, and knowledge to support healthcare. Digital health has become an essential component of the diagnostic process, helping to facilitate the accuracy and timeliness of information transfer and enhance the effectiveness of decision-making processes. Digital health is also important to diagnostic stewardship, which involves coordinated guidance and interventions to ensure the appropriate utilization of diagnostic tests for therapeutic decision-making. Diagnostic stewardship and informatics are thus important in efforts to establish shared decision-making. This is because they contribute to the establishment of shared information platforms (enabling patients to read, comment on, and share in decisions about their care) based on timely and meaningful communication. This paper will outline key diagnostic informatics and stewardship initiatives across three interrelated fields: (1) diagnostic error and the establishment of outcomes-based diagnostic research; (2) the safety and effectiveness of test result management and follow-up; and (3) digitally enhanced decision support systems.

Diagnostic value of serum HER2 levels in breast cancer: a systematic review and meta-analysis

BACKGROUND

Measurement of serum human epidermal growth factor receptor-2 (HER-2/neu) levels might play an essential role as a diagnostic/screening marker for the early selection of therapeutic approaches and predict prognosis in breast cancer patients. We aimed to undertake a systematic review and meta-analysis focusing on the diagnostic/screening value of serum HER-2 levels in comparison to routine methods.

METHODS

We performed a systematic search via PubMed, Scopus, Cochrane-Library, and Web of Science databases for human diagnostic studies reporting the levels of serum HER-2 in breast cancer patients, which was confirmed using the histopathological examination. Meta-analyses were carried out for sensitivity, specificity, accuracy, area under the ROC curve (AUC), positive predictive value (PPV), negative predictive value (NPV), positive likelihood ratio (PLR), and negative likelihood ratio (NLR).

RESULTS

Fourteen studies entered into this investigation. The meta-analysis indicated the low sensitivity for serum HER2 levels (Sensitivity: 53.05, 95%CI 40.82-65.28), but reasonable specificity of 79.27 (95%CI 73.02-85.51), accuracy of 72.06 (95%CI 67.04-77.08) and AUC of 0.79 (95%CI 0.66-0.92). We also found a significant differences for PPV (PPV: 56.18, 95%CI 44.16-68.20), NPV (NPV: 76.93, 95%CI 69.56-84.31), PLR (PLR: 2.10, 95%CI 1.69-2.50) and NLR (NLR: 0.58, 95%CI 0.44-0.71).

CONCLUSION

Our findings revealed that although serum HER-2 levels showed low se nsitivity for breast cancer diagnosis, its specificity, accuracy and AUC were reasonable. Hence, it seems that the measurement of serum HER-2 levels can play a significant role as a verification test for initial negative screening test results, especially in low-income regions due to its cost-effectiveness and ease of implementation.

Diagnostic accuracy of FeNO [fractional exhaled nitric oxide] and asthma symptoms increased when evaluated with a superior reference standard

OBJECTIVES

The objective of the study is to determine the impact of changing reference standards (RS), namely spirometry vs. whole-body plethysmography (WBP), on estimation of the diagnostic accuracy of fractional exhaled nitric oxide (FeNO) and clinical signs and symptoms (CSS) as index tests regarding asthma diagnosis.

STUDY DESIGN AND SETTING

This was a diagnostic study conducted in 393 patients attending a private practice of pneumologists with complaints suspicious of asthma. First, the index tests were compared with the diagnostic results of spirometry in terms of forced expiratory volume in the first second (FEV₁) responsiveness. Second, the index tests were compared with the results of WBP in terms of specific airway resistance and FEV₁ responsiveness. Areas under the curve (AUC) were compared with a generalized estimating equation approach based on binary logistic regression.

RESULTS

FeNO values and CSS 'wheezing' and 'allergic rhinitis' showed higher specificities (P < 0.001) and sensitivities (not significant) when evaluated with WBP; also, Youden indices increased in these CSS (P < 0.05). AUC of FeNO in combination with 'wheezing' and 'allergic rhinitis' when WBP was used as RS (AUC = 0.724; 95% confidence interval 0.672 to 0.776) was higher compared with spirometry as RS (AUC = 0.654; 95% confidence interval 0.585 to 0.722) (P < 0.001).

CONCLUSION

In case of asthma, superior RS led to more favorable assessment of index tests. FeNO measurement might have been underestimated in some previous studies.

Translating Laboratory Tests into Clinical Practice: A Conceptual Framework

The use of laboratory biomarkers in clinical practice is rapidly increasing. Laboratory tests are, however, rarely evaluated adequately before implementation, and the utility of many tests is essentially unclear. An important reason for this knowledge gap is that a comprehensive and generally accepted methodological framework supporting evaluation studies is essentially lacking. Researchers, clinicians, and decision-makers are often not aware of the methodological tools available and face problems with the appraisal of a test's utility. With the present article, I aim to summarize current concepts and methodological tools and propose a framework for a phased approach that could be used in future evaluation projects. Future research will refine this suggested framework by identifying problems in current evaluation projects, specifying methodological criteria for all phases, as well as developing advanced methodological tools.

Real-world evaluation of a novel technology for quantitative simultaneous antibody detection against multiple SARS-CoV-2 antigens in a cohort of patients presenting with COVID-19 syndrome

An evaluation of a rapid portable gold-nanotechnology measuring SARS-CoV-2 IgM, IgA and IgG antibody concentrations against spike 1 (S1), spike 2 (S) and nucleocapsid (N) was conducted using serum samples from 74 patients tested for SARS-CoV-2 RNA on admission to hospital, and 47 historical control patients from March 2019. 59 patients were RNA(+) and 15 were RNA(-). A serum (±) classification was derived for all three antigens and a quantitative serological profile was obtained. Serum(+) was identified in 30% (95% CI 11-48) of initially RNA(-) patients, in 36% (95% CI 17-54) of RNA(+) patients before 10 days, 77% (95% CI 67-87) between 10 and 20 days and 95% (95% CI 86-100) after 21 days. The patient-level diagnostic accuracy relative to RNA(±) after 10 days displayed 88% sensitivity (95% CI 75-95) and 75% specificity (95% CI 22-99), although specificity compared with historical controls was 100% (95%CI 91-100). This study provides robust support for further evaluation and validation of this novel technology in a clinical setting and highlights challenges inherent in assessment of serological tests for an emerging disease such as COVID-19.

A Virtual 3D Dynamic Model of Caries Lesion Progression as a Learning Object for Caries Detection Training and Teaching: Video Development Study

BACKGROUND

In the last decade, 3D virtual models have been used for educational purposes in the health sciences, specifically for teaching human anatomy and pathology. These models provide an opportunity to didactically visualize key spatial relations that can be poorly understood when taught by traditional educational approaches. Caries lesion detection is a crucial process in dentistry that has been reported to be difficult to learn. One especially difficult aspect is linking clinical characteristics of the different severity stages with their histological features, which is fundamental for treatment decision-making.

OBJECTIVE

This project was designed to develop a virtual 3D digital model of caries lesion formation and progression to aid the detection of lesions at different severity stages as a potential complement to traditional lectures.

METHODS

Pedagogical planning, including identification of objectives, exploration of the degree of difficulty of caries diagnosis-associated topics perceived by dental students and lecturers, review of the literature regarding key concepts, and consultation of experts, was performed prior to constructing the model. An educational script strategy was created based on the topics to be addressed (dental tissues, biofilm stagnation areas, the demineralization process, caries lesion progression on occlusal surfaces, clinical characteristics related to different stages of caries progression, and histological correlations). Virtual 3D models were developed using the Virtual Man Project and refined using multiple 3D software applications. In the next phase, computer graphic modelling and previsualization were executed. After that, the video was revised and edited based on suggestions. Finally, explanatory subtitles were generated, the models were textured and rendered, and voiceovers in 3 languages were implemented.

RESULTS

We developed a 6-minute virtual 3D dynamic video in 3 languages (English, Spanish, and Brazilian Portuguese) intended for dentists and dental students to support teaching and learning of caries lesion detection. The videos were made available on YouTube; to date, they have received more than 100,000 views.

CONCLUSIONS

Complementary pedagogical tools are valuable to support cariology education. This tool will be further tested in terms of utility and usability as well as user satisfaction in achieving the proposed objectives in specific contexts.

The feasibility of measuring calprotectin from a throat swab as a marker of infections caused by group A streptococcus: a case-control feasibility study

Background

Most people with sore throat do not benefit from antibiotic treatment, but nearly three-quarters of those presenting in primary care are prescribed antibiotics. A test that is predictive of bacterial infection could help guide antibiotic prescribing. Calprotectin is a biomarker of neutrophilic inflammation, and may be a useful marker of bacterial throat infections.

Aim

To assess the feasibility of measuring calprotectin from throat swabs, and assess whether individuals with sore throats likely to be caused by streptococcal infections have apparently higher throat calprotectin levels than other individuals with sore throat and healthy volunteers.

Design & setting

A proof of concept case–control study was undertaken, which compared primary care patients with sore throats and healthy volunteers.

Method

Baseline characteristics and throat swabs were collected from 30 primary care patients with suspected streptococcal sore throat, and throat swabs were taken from 10 volunteers without sore throat. Calprotectin level determination and rapid antigen streptococcal testing were conducted on the throat swab eluents. Calprotectin levels in the following groups were compared: volunteers without a sore throat; all patients with a sore throat; patients with a sore throat testing either negative or positive for streptococcal antigen; and those with lower and higher scores on clinical prediction rules for streptococcal sore throat.

Results

Calprotectin was detected in all throat swab samples. Mean calprotectin levels were numerically higher in patients with sore throat compared with healthy volunteers, and sore throat patients who had group A streptococci antigen detected compared with those who did not.

Conclusion

Calprotectin can be measured from throat swab samples and levels are consistent with the hypothesis that streptococcal infection leads to higher throat calprotectin levels. This hypothesis will be tested in a larger study.

Impact of Functional Capacity Evaluation on Patient-Reported Functional Ability: An Exploratory Diagnostic Before-After Study

Introduction Work capacity in patients with orthopedic trauma and long-lasting inactivity is significantly reduced. Functional capacity evaluation (FCE) is a diagnostic approach for developing recommendations for a return to work and further occupational rehabilitation when the ability to carry out previous job demands is uncertain. However, FCE may also have direct effects on the patients' appraisal of their functional ability. Our study therefore evaluated the change in patient-reported functional ability after the performance of an FCE. Methods We performed a diagnostic before-after study in 161 consecutively recruited patients with trauma who were referred for FCE at the end of an interdisciplinary inpatient rehabilitation program in Austria. Patients completed the Spinal Function Sort to assess patient-reported functional ability both prior to the FCE and after completing it. Results Patient-reported functional ability (0-200 points) improved by 14.8 points (95% CI 11.3-18.2). The number of participants who rated their functional ability below their functional capacity as observed by the FCE decreased from 82.6 to 64.6% by about 18 percentage points. Conclusions The performance of the FCE in patients with trauma was associated with an improvement of patient-reported functional ability. The performance of an FCE in trauma rehabilitation may possibly have a direct therapeutic effect on the patient by allowing a more realistic appraisal of the ability to perform relevant work activities.

Diagnostic Test Score Validation With a Fallible Criterion

Criterion-related validation of diagnostic test scores for a construct of interest is complicated by the unavailability of the construct directly. The standard method, Known Group Validation, assumes an infallible reference test in place of the construct, but infallible reference tests are rare. In contrast, Mixed Group Validation allows for a fallible reference test, but has been found to make strong assumptions not appropriate for the majority of diagnostic test validation studies. The Neighborhood model is adapted for the purpose of diagnostic test validation, which makes alternate, but also strong, assumptions. The statistical properties of the Neighborhood model are evaluated and the assumptions are reviewed in the context of diagnostic test validation. Alternatively, strong assumptions may be avoided by estimating only intervals for the validity estimates, instead of point estimates. The Method of Bounds is also adapted for the purpose of diagnostic test validation, and an extension, Method of Bounds-Test Validation, is introduced here for the first time. All three point-estimate methods were found to make strong assumptions concerning the conditional relationships between the tests and the construct of interest, and all three lack robustness to assumption violation. The Method of Bounds-Test Validation was found to perform well across a range of plausible simulated datasets where the point-estimate methods failed. The point-estimate methods are recommended in special cases where the assumptions can be justified, while the interval methods are appropriate more generally.

An analysis reveals differences between pragmatic and explanatory diagnostic accuracy studies

OBJECTIVES

The objective of this study was to clarify a difference between two approaches while evaluating the diagnostic accuracy of medical tests, labeled here as "pragmatic" vs. "explanatory" studies.

METHODS

Using the definitions and characteristics described by Schwartz and Lellouch for randomized trials of interventions, and Schwartz' more general distinction between a pragmatic and an explanatory approach in medical research, we define a similar continuum for diagnostic accuracy studies. Explanatory studies aim to better understand the behavior of a test; pragmatic ones are done to support recommendations or decisions about using the test in clinical practice.

RESULTS

Pragmatic test accuracy studies differ from explanatory test accuracy studies in several ways. The difference in aims has implications for key elements of study design, such as the study eligibility criteria, the recruitment of patients, the reference standard, and the choice of the statistical analysis. Explanatory accuracy studies are often designed to test a hypothesis. They are typically selective in recruitment, may include "healthy controls," with a small sample size, often recruited at a single center. They ignore testing failures in the analysis and more often present their results as ROC curves. By contrast, pragmatic studies are designed to guide decision making. They ideally will recruit a single, large, and representative group of patients at multiple sites and will more often present their results as estimates of sensitivity and specificity or predictive values at a prespecified threshold.

CONCLUSION

Distinguishing between a pragmatic and an explanatory approach can help in the design, analysis, and interpretation of diagnostic accuracy studies. It can clarify debates about the appropriateness of design features to the study purpose and about the validity and applicability of study findings.

Diagnostic accuracy of in-house real-time PCR assay for Mycobacterium tuberculosis: a systematic review and meta-analysis

BACKGROUND

In recent years, studies on the diagnostic accuracy of in-house real-time PCR (hRT-PCR) assay for the detection of Mycobacterium tuberculosis (Mtb) have been reported with unignorable discrepancies. To assess the overall accuracy of the hRT-PCR assay for Mtb diagnosis in different samples for individuals with active pulmonary and extra-pulmonary Mtb infection, a systematic review and meta-analysis were performed.

METHODS

The PUBMED, EMBASE, Web of Science, and Cochrane databases were searched up to June 2017 for eligible studies that estimated diagnostic sensitivity and specificity with the hRT-PCR assay in respiratory and non-respiratory samples in pulmonary and extra-pulmonary Mtb infection patients, with Mtb culture as the reference standard. Bivariate random effect models were used to provide pooled estimation of diagnostic accuracy. Further, subgroup and meta-regression analyses were performed to explore sources of heterogeneity. The risk of bias was assessed by the QUADAS-2 tool.

RESULTS

Of the 3589 candidate studies, 18 eligible studies met our inclusion criteria. Compared to Mtb culture data, the pooled sensitivity and specificity were 0.96 and 0.92, respectively. The diagnostic odds ratio (DOR) was 192.96 (95% CI 68.46, 543.90), and the area under the summary ROC curve (AUC) was 0.9791. There was significant heterogeneity in sensitivity and specificity among the enrolled studies (p < 0.001). The studies with high-quality assessment and application of respiratory specimen were associated with better accuracy.

CONCLUSIONS

In low-income/high-burden settings, our results suggested that the hRT-PCR assay could be a useful test for the diagnosis of TB with high sensitivity and specificity.

Transvaginal/transrectal ultrasound-guided aspiration biopsy for diagnosis of pelvic/pelvic floor tumors in females: A retrospective analysis

Ultrasound guidance for histological diagnosis is in real-time, convenient, and economical. The aim of this study was to determine whether transvaginal ultrasound (TVUS)- and transrectal ultrasound (TRUS)-guided aspiration biopsy allows detection of a malignant pathology of pelvic/pelvic masses. Data of 40 patients with pelvic and pelvic mass lesions by computed tomography or magnetic resonance imaging underwent TVUS- or TRUS-guided biopsy. Tissue samples obtained were assessed on suitability for histopathologic evaluation. The post-biopsy complication was monitored. All the specimens in the pelvic floor, vaginal stump, vaginal fornix, cervix, and posterior wall of the anal canal were adequate for histologic diagnosis. There were no post-biopsy complications. Transvaginal/transrectal ultrasound-guided aspiration biopsy is safe and simple. It can be used for the diagnosis and differential diagnosis of pelvic and pelvic floor lesions in women. Prospective studies are needed to test diagnostic performance across clinical scenarios.

Selecting pregnant or postpartum women with suspected pulmonary embolism for diagnostic imaging: the DiPEP diagnostic study with decision-analysis modelling

BACKGROUND

Pulmonary embolism (PE) is a leading cause of death in pregnancy and post partum, but the symptoms of PE are common in normal pregnancy. Simple diagnostic tests are needed to select women for diagnostic imaging.

OBJECTIVE

To estimate the accuracy, effectiveness and cost-effectiveness of clinical features, decision rules and biomarkers for selecting pregnant or postpartum women with a suspected PE for imaging.

DESIGN

An expert consensus study to develop new clinical decision rules, a case-control study of women with a diagnosed PE or a suspected PE, a biomarker study of women with a suspected PE or diagnosed deep-vein thrombosis (DVT) and decision-analysis modelling.

SETTING

Emergency departments and consultant-led maternity units.

PARTICIPANTS

Pregnant/postpartum women with a diagnosed PE from any hospital reporting to the UK Obstetric Surveillance System research platform and pregnant/postpartum women with a suspected PE or diagnosed DVT at 11 prospectively recruiting sites.

INTERVENTIONS

Clinical features, decision rules and biomarkers.

MAIN OUTCOME MEASURES

Sensitivity, specificity, area under receiver operating characteristic (AUROC) curve, quality-adjusted life-years (QALYs) and health-care costs.

RESULTS

The primary analysis involved 181 women with PE and 259 women without PE in the case-control study and 18 women with DVT, 18 with PE and 247 women without either in the biomarker study. Most clinical features showed no association with PE. The AUROC curves for the clinical decision rules were as follows: primary consensus, 0.626; sensitive consensus, 0.620; specific consensus, 0.589; PE rule-out criteria, 0.621; simplified Geneva score, 0.579; Wells's PE criteria (permissive), 0.577; and Wells's PE criteria (strict), 0.732. The sensitivities and specificities of the D-dimer measurement were 88.4% and 8.8%, respectively, using a standard threshold, and 69.8% and 32.8%, respectively, using a pregnancy-specific threshold. Previous venous thromboembolism, long-haul travel, multiple pregnancy, oxygen saturation, recent surgery, temperature and PE-related chest radiograph abnormality were predictors of PE on multivariable analysis. We were unable to derive a rule through multivariable analysis or recursive partitioning with adequate accuracy. The AUROC curves for the biomarkers were as follows: activated partial thromboplastin time - 0.669, B-type natriuretic peptide - 0.549, C-reactive protein - 0.542, Clauss fibrinogen - 0.589, enzyme-linked immunosorbent assay D-dimer - 0.668, Innovance D-dimer (Siemens Healthcare Diagnostics Products GmbH, distributed by Sysmex UK Ltd, Milton Keynes, UK) - 0.651, mid-regional pro-atrial natriuretic peptide (MRproANP) - 0.524, prothrombin fragment 1 + 2 - 0.562, plasmin-antiplasmin - 0.639, Prothombin time - 0.613, thrombin generation lag time - 0.702, thrombin generation endogenous potential - 0.559, thrombin generation peak - 0.596, thrombin generation time to peak - 0.655, tissue factor - 0.531 and troponin - 0.597. The repeat analysis excluding women who had received anticoagulation was limited by the small number of women with PE (n = 4). The health economic analysis showed that a strategy of scanning all women with a suspected PE accrued more QALYs and incurred fewer costs than any selective strategy based on a clinical decision rule and was therefore the dominant strategy.

LIMITATIONS

The findings apply specifically to the diagnostic assessment of women with a suspected PE in secondary care.

CONCLUSIONS

Clinical features, decision rules and biomarkers do not accurately, effectively or cost-effectively select pregnant or postpartum women with a suspected PE for diagnostic imaging.

FUTURE WORK

New diagnostic technologies need to be developed to detect PE in pregnancy.

TRIAL REGISTRATION

Current Controlled Trials ISRCTN21245595.

FUNDING DETAILS

This project was funded by the National Institute for Health Research (NIHR) Health Technology Assessment programme and will be published in full in Health Technology Assessment; Vol. 22, No. 47. See the NIHR Journals Library website for further project information.

Predictive performance of a multivariable difficult intubation model for obese patients

Background

A predictive model of scores of difficult intubation (DI) may help physicians screen for airway difficulty to reduce morbidity and mortality in obese patients. The present study aimed to set up and evaluate the predictive performance of a newly developed, practical, multivariate DI model for obese patients.

Methods

A prospective multi-center study was undertaken on adults with a body mass index (BMI) of 30 kg/m² or more who were undergoing conventional endotracheal intubation. The BMI and 10 preoperative airway tests (namely, malformation of the teeth in the upper jaw, the modified Mallampati test [MMT], the upper lip bite test, neck mobility testing, the neck circumference [NC], the length of the neck, the interincisor gap, the hyomental distance, the thyromental distance [TM] and the sternomental distance) were examined. A DI was defined as one with an intubation difficulty scale (IDS) score ≥ 5.

Results

The 1,015 patients recruited for the study had a mean BMI of 34.2 (standard deviation: 4.3 kg/m²). The proportions for easy intubation, slight DI and DI were 81%, 15.8% and 3.2%, respectively. Drawing on the results of a multivariate analysis, clinically meaningful variables related to obesity (namely, BMI, MMT, and the ratio of NC to TM) were used to build a predictive model for DI. Nevertheless, the best model only had a fair predictive performance. The area under the receiver operating characteristic curve (AUC) was 0.71 (95% confidence interval 0.68–0.84).

Conclusions

The predictive performance of the selected model showed limited benefit for preoperative screening to predict DI among obese patients.

[GRADE Guidelines: 16. GRADE evidence to decision frameworks for tests in clinical practice and public health]

OBJECTIVE

To describe the Grading of Recommendations Assessment, Development and Evaluation (GRADE) interactive Evidence to Decision (EtD) frameworks for tests and test strategies for clinical, public health or coverage decisions.

STUDY DESIGN AND SETTING

As part of the GRADE Working Group's DECIDE project we conducted workshops, user testing with systematic review authors, guideline developers and other decision makers, and piloted versions of the EtD framework.

RESULTS

EtD frameworks for tests share the structure, explicitness, and transparency of other EtD frameworks. They require specifying the purpose of the test, linked or related management and the key outcomes of concern for different test results and subsequent management. The EtD criteria address test accuracy and assessments of the certainty of the additional evidence necessary for decision-making. When there is no direct evidence of test effects on patient important outcomes, formal or informal modeling is needed to estimate effects. We describe the EtD criteria based on examples developed with GRADEpro (www.gradepro.org), GRADE's software that also provides interactive Summary of Findings Tables.

CONCLUSION

EtD frameworks for developing recommendations and making decisions about tests lay out the sequential steps in reviewing and assessing the different types of evidence that need to be linked.

Accuracy of fundus autofluorescence imaging for the diagnosis and monitoring of retinal conditions: a systematic review

BACKGROUND

Natural fluorescence in the eye may be increased or decreased by diseases that affect the retina. Imaging methods based on confocal scanning laser ophthalmoscopy (cSLO) can detect this 'fundus autofluorescence' (FAF) by illuminating the retina using a specific light 'excitation wavelength'. FAF imaging could assist the diagnosis or monitoring of retinal conditions. However, the accuracy of the method for diagnosis or monitoring is unclear.

OBJECTIVE

To conduct a systematic review to determine the accuracy of FAF imaging using cSLO for the diagnosis or monitoring of retinal conditions, including monitoring of response to therapy.

DATA SOURCES

Electronic bibliographic databases; scrutiny of reference lists of included studies and relevant systematic reviews; and searches of internet pages of relevant organisations, meetings and trial registries. Databases included MEDLINE, EMBASE, The Cochrane Library, Web of Science and the Medion database of diagnostic accuracy studies. Searches covered 1990 to November 2014 and were limited to the English language.

REVIEW METHODS

References were screened for relevance using prespecified inclusion criteria to capture a broad range of retinal conditions. Two reviewers assessed titles and abstracts independently. Full-text versions of relevant records were retrieved and screened by one reviewer and checked by a second. Data were extracted and critically appraised using the Quality Assessment of Diagnostic Accuracy Studies criteria (QUADAS) for assessing risk of bias in test accuracy studies by one reviewer and checked by a second. At all stages any reviewer disagreement was resolved through discussion or arbitration by a third reviewer.

RESULTS

Eight primary research studies have investigated the diagnostic accuracy of FAF imaging in retinal conditions: choroidal neovascularisation (one study), reticular pseudodrusen (three studies), cystoid macular oedema (two studies) and diabetic macular oedema (two studies). Sensitivity of FAF imaging using an excitation wavelength of 488 nm was generally high (range 81-100%), but was lower (55% and 32%) in two studies using longer excitation wavelengths (514 nm and 790 nm, respectively). Specificity ranged from 34% to 100%. However, owing to limitations of the data, none of the studies provide conclusive evidence of the diagnostic accuracy of FAF imaging.

LIMITATIONS

No studies on the accuracy of FAF imaging for monitoring the progression of retinal conditions or response to therapy were identified. Owing to study heterogeneity, pooling of diagnostic outcomes in meta-analysis was not conducted. All included studies had high risk of bias. In most studies the patient spectrum was not reflective of those who would present in clinical practice and no studies adequately reported how FAF images were interpreted.

CONCLUSIONS

Although already in use in clinical practice, it is unclear whether or not FAF imaging is accurate, and whether or not it is applied and interpreted consistently for the diagnosis and/or monitoring of retinal conditions. Well-designed prospective primary research studies, which conform to the paradigm of diagnostic test accuracy assessment, are required to investigate the accuracy of FAF imaging in diagnosis and monitoring of inherited retinal dystrophies, early age-related macular degeneration, geographic atrophy and central serous chorioretinopathy.

STUDY REGISTRATION

This study is registered as PROSPERO CRD42014014997.

FUNDING

The National Institute for Health Research Health Technology Assessment programme.

Development of a clinical prediction rule for tuberculous meningitis in adults in Lima, Peru

OBJECTIVES

Diagnosis of tuberculous meningitis (TM) is a challenge in countries with a high burden of the disease and constrained resources and clinical prediction rules (CPRs) could be of assistance. We aimed at developing a CPR for diagnosis of TM in a Latin American setting with high tuberculosis incidence and a concentrated HIV epidemic.

METHODS

We enrolled adult patients with clinical suspicion of TM attending two hospitals in Lima, Peru. We obtained information on potential anamnestic, clinical and laboratory predictive findings that are easy to collect and promptly available. We independently diagnosed TM according to a composite reference standard that included a series of microbiological tests. We performed bivariate analysis and constructed a logistic regression model to select the predictive findings associated with TM. With the selected predictors included in the model, we developed a score-based CPR. We assessed its internal validity and diagnostic performance.

RESULTS

Of 155 analysed patients, 59 (38%) had TM. The CPR we derived includes three predictors: cough for 14 days or more, 10-500 cells in CSF and adenosine deaminase ≥ 6 U/l in CSF. It classifies patients into high-, moderate- or low-score groups and has an overall area under the ROC curve of 0.87. 59% of patients were assigned to either the high- or the low-score group, permitting prompt decision-making. In patients in the high-score group, it attains a positive likelihood ratio for TM of 10.6 and in patients with low scores, a negative likelihood ratio of 0.10. Bootstrap analysis indicated high internal validity.

CONCLUSION

This CPR could support decision-making in patients with clinical suspicion of TM. External validation and further assessment of its clinical impact are necessary before application in other settings.

A randomised controlled trial, cost-effectiveness and process evaluation of the implementation of self-management for chronic gastrointestinal disorders in primary care, and linked projects on identification and risk assessment

Background

Chronic gastrointestinal disorders are major burdens in primary care. Although there is some evidence that enhancing self-management can improve outcomes, it is not known if such models of care can be implemented at scale in routine NHS settings and whether or not it is possible to develop effective risk assessment procedures to identify patients who are likely to become chronically ill.

Objectives

What is the clinical effectiveness and cost-effectiveness of an intervention to enhance self-management support for patients with chronic conditions when translated from research settings into routine care? What are the barriers and facilitators that affect the implementation of an intervention to enhance self-management support among patients, clinicians and organisations? Is it possible to develop methods to identify patients at risk of long-term problems with functional gastrointestinal disorders in primary care? Data sources included professional and patient interviews, patient self-report measures and data on service utilisation.

Design

A pragmatic, two-arm, practice-level cluster Phase IV randomised controlled trial evaluating outcomes and costs associated with the intervention, with associated process evaluation using interviews and other methods. Four studies around identification and risk assessment: (1) a general practitioner (GP) database study to describe how clinicians in primary care record consultations with patients who experience functional lower gastrointestinal symptoms; (2) a validation of a risk assessment tool; (3) a qualitative study to explore GPs’ views and experiences; and (4) a second GP database study to investigate patient profiles in irritable bowel syndrome, inflammatory bowel disease and abdominal pain.

Setting

Salford, UK.

Participants

People with long-term conditions and professionals in primary care.

Interventions

A practice-level intervention to train practitioners to assess patient self-management capabilities and involve them in a choice of self-management options.

Main outcome measures

Patient self-management, care experience and quality of life, health-care utilisation and costs.

Results

No statistically significant differences were found between patients attending the trained practices and those attending control practices on any of the primary or secondary outcomes. The intervention had little impact on either costs or effects within the time period of the trial. In the practices, self-management tools failed to be normalised in routine care. Full assessment of the predictive tool was not possible because of variable case definitions used in practices. There was a lack of perceived clinical benefit among GPs.

Limitations

The intervention was not implemented fully in practice. Assessment of the risk assessment tool faced barriers in terms of the quality of codting in GP databases and poor recruitment of patients.

Conclusions

The Whole system Informing Self-management Engagement self-management (WISE) model did not add value to existing care for any of the long-term conditions studied.

Future work

The active components required for effective self-management support need further study. The results highlight the challenge of delivering improvements to quality of care for long-term conditions. There is a need to develop interventions that are feasible to deliver at scale, yet demonstrably clinically effective and cost-effective. This may have implications for the piloting of interventions and linking implementation more clearly to local commissioning strategies.

Trial registration

Current Controlled Trial ISRCTN90940049.

Funding

This project was funded by the National Institute for Health Research (NIHR) Programme Grants for Applied Research programme and will be published in full inProgramme Grants for Applied Research; Vol. 6, No. 1. See the NIHR Journals Library website for further project information.

Prediction of post-acute care demand in medical and neurological inpatients: diagnostic assessment of the post-acute discharge score - a prospective cohort study

BACKGROUND

Early identification of patients requiring transfer to post-acute care (PAC) facilities shortens hospital stays. With a focus on interprofessional assessment of biopsychosocial risk, this study's aim was to assess medical and neurological patients' post-acute care discharge (PACD) scores on days 1 and 3 after hospital admission regarding diagnostic accuracy and effectiveness as an early screening tool. The transfer to PAC facilities served as the outcome ("gold standard").

METHODS

In this prospective cohort study, registered at ClinicalTrial.gov (NCT01768494) on January 2013, 1432 medical and 464 neurological patients (total n = 1896) were included consecutively between February and October 2013. PACD scores and other relevant data were extracted from electronic records of patient admissions, hospital stays, and interviews at day 30 post-hospital admission. To gauge the scores' accuracy, we plotted receiver operating characteristic (ROC) curves, calculated area under the curve (AUC), and determined sensitivity and specificity at various cut-off levels.

RESULTS

Medical patients' day 1 and day 3 PACD scores accurately predicted discharge to PAC facilities, with respective discriminating powers (AUC) of 0.77 and 0.82. With a PACD cut-off of ≥8 points, day 1 and 3 sensitivities were respectively 72.6% and 83.6%, with respective specificities of 66.5% and 70.0%. Neurological patients' scores showed lower accuracy both days: using the same cut-off, respective day 1 and day 3 AUCs were 0.68 and 0.78, sensitivities 41.4% and 68.7% and specificities 81.4% and 83.4%.

CONCLUSION

PACD scores at days 1 and 3 accurately predicted transfer to PAC facilities, especially in medical patients on day 3. To confirm and refine these results, PACD scores' value to guide discharge planning interventions and subsequent impact on hospital stay warrants further investigation.

TRIAL REGISTRATION

ClinialTrials.gov Identifier, NCT01768494 .

18F PET with florbetapir for the early diagnosis of Alzheimer's disease dementia and other dementias in people with mild cognitive impairment (MCI)

BACKGROUND

¹⁸F-florbetapir uptake by brain tissue measured by positron emission tomography (PET) is accepted by regulatory agencies like the Food and Drug Administration (FDA) and the European Medicine Agencies (EMA) for assessing amyloid load in people with dementia. Its added value is mainly demonstrated by excluding Alzheimer's pathology in an established dementia diagnosis. However, the National Institute on Aging and Alzheimer's Association (NIA-AA) revised the diagnostic criteria for Alzheimer's disease and confidence in the diagnosis of mild cognitive impairment (MCI) due to Alzheimer's disease may be increased when using amyloid biomarkers tests like ¹⁸F-florbetapir. These tests, added to the MCI core clinical criteria, might increase the diagnostic test accuracy (DTA) of a testing strategy. However, the DTA of ¹⁸F-florbetapir to predict the progression from MCI to Alzheimer's disease dementia (ADD) or other dementias has not yet been systematically evaluated.

OBJECTIVES

To determine the DTA of the ¹⁸F-florbetapir PET scan for detecting people with MCI at time of performing the test who will clinically progress to ADD, other forms of dementia (non-ADD), or any form of dementia at follow-up.

SEARCH METHODS

This review is current to May 2017. We searched MEDLINE (OvidSP), Embase (OvidSP), PsycINFO (OvidSP), BIOSIS Citation Index (Thomson Reuters Web of Science), Web of Science Core Collection, including the Science Citation Index (Thomson Reuters Web of Science) and the Conference Proceedings Citation Index (Thomson Reuters Web of Science), LILACS (BIREME), CINAHL (EBSCOhost), ClinicalTrials.gov (https://clinicaltrials.gov), and the World Health Organization International Clinical Trials Registry Platform (WHO ICTRP) (http://www.who.int/ictrp/search/en/). We also searched ALOIS, the Cochrane Dementia & Cognitive Improvement Group's specialised register of dementia studies (http://www.medicine.ox.ac.uk/alois/). We checked the reference lists of any relevant studies and systematic reviews, and performed citation tracking using the Science Citation Index to identify any additional relevant studies. No language or date restrictions were applied to the electronic searches.

SELECTION CRITERIA

We included studies that had prospectively defined cohorts with any accepted definition of MCI at time of performing the test and the use of ¹⁸F-florbetapir scan to evaluate the DTA of the progression from MCI to ADD or other forms of dementia. In addition, we only selected studies that applied a reference standard for Alzheimer's dementia diagnosis, for example, National Institute of Neurological and Communicative Disorders and Stroke and the Alzheimer's Disease and Related Disorders Association (NINCDS-ADRDA) or Diagnostic and Statistical Manual of Mental Disorders-IV (DSM-IV) criteria.

DATA COLLECTION AND ANALYSIS

We screened all titles and abstracts identified in electronic-database searches. Two review authors independently selected studies for inclusion and extracted data to create two-by-two tables, showing the binary test results cross-classified with the binary reference standard. We used these data to calculate sensitivities, specificities, and their 95% confidence intervals. Two independent assessors performed quality assessment using the QUADAS-2 tool plus some additional items to assess the methodological quality of the included studies.

MAIN RESULTS

We included three studies, two of which evaluated the progression from MCI to ADD, and one evaluated the progression from MCI to any form of dementia.Progression from MCI to ADD was evaluated in 448 participants. The studies reported data on 401 participants with 1.6 years of follow-up and in 47 participants with three years of follow-up. Sixty-one (15.2%) participants converted at 1.6 years follow-up; nine (19.1%) participants converted at three years of follow-up.Progression from MCI to any form of dementia was evaluated in five participants with 1.5 years of follow-up, with three (60%) participants converting to any form of dementia.There were concerns regarding applicability in the reference standard in all three studies. Regarding the domain of flow and timing, two studies were considered at high risk of bias. MCI to ADD;Progression from MCI to ADD in those with a follow-up between two to less than four years had a sensitivity of 67% (95% CI 30 to 93) and a specificity of 71% (95% CI 54 to 85) by visual assessment (n = 47, 1 study).Progression from MCI to ADD in those with a follow-up between one to less than two years had a sensitivity of 89% (95% CI 78 to 95) and a specificity of 58% (95% CI 53 to 64) by visual assessment, and a sensitivity of 87% (95% CI 76 to 94) and a specificity of 51% (95% CI 45 to 56) by quantitative assessment by the standardised uptake value ratio (SUVR)(n = 401, 1 study). MCI to any form of dementia;Progression from MCI to any form of dementia in those with a follow-up between one to less than two years had a sensitivity of 67% (95% CI 9 to 99) and a specificity of 50% (95% CI 1 to 99) by visual assessment (n = 5, 1 study). MCI to any other forms of dementia (non-ADD);There was no information regarding the progression from MCI to any other form of dementia (non-ADD).

AUTHORS' CONCLUSIONS

Although sensitivity was good in one included study, considering the poor specificity and the limited data available in the literature, we cannot recommend routine use of ¹⁸F-florbetapir PET in clinical practice to predict the progression from MCI to ADD.Because of the poor sensitivity and specificity, limited number of included participants, and the limited data available in the literature, we cannot recommend its routine use in clinical practice to predict the progression from MCI to any form of dementia.Because of the high financial costs of ¹⁸F-florbetapir, clearly demonstrating the DTA and standardising the process of this modality are important prior to its wider use.

18F PET with flutemetamol for the early diagnosis of Alzheimer's disease dementia and other dementias in people with mild cognitive impairment (MCI)

BACKGROUND

¹⁸F-flutemetamol uptake by brain tissue, measured by positron emission tomography (PET), is accepted by regulatory agencies like the Food and Drug Administration (FDA) and the European Medicine Agencies (EMA) for assessing amyloid load in people with dementia. Its added value is mainly demonstrated by excluding Alzheimer's pathology in an established dementia diagnosis. However, the National Institute on Aging and Alzheimer's Association (NIA-AA) revised the diagnostic criteria for Alzheimer's disease and the confidence in the diagnosis of mild cognitive impairment (MCI) due to Alzheimer's disease may be increased when using some amyloid biomarkers tests like ¹⁸F-flutemetamol. These tests, added to the MCI core clinical criteria, might increase the diagnostic test accuracy (DTA) of a testing strategy. However, the DTA of ¹⁸F-flutemetamol to predict the progression from MCI to Alzheimer's disease dementia (ADD) or other dementias has not yet been systematically evaluated.

OBJECTIVES

To determine the DTA of the ¹⁸F-flutemetamol PET scan for detecting people with MCI at time of performing the test who will clinically progress to ADD, other forms of dementia (non-ADD) or any form of dementia at follow-up.

SEARCH METHODS

The most recent search for this review was performed in May 2017. We searched MEDLINE (OvidSP), Embase (OvidSP), PsycINFO (OvidSP), BIOSIS Citation Index (Thomson Reuters Web of Science), Web of Science Core Collection, including the Science Citation Index (Thomson Reuters Web of Science) and the Conference Proceedings Citation Index (Thomson Reuters Web of Science), LILACS (BIREME), CINAHL (EBSCOhost), ClinicalTrials.gov (https://clinicaltrials.gov), and the World Health Organization International Clinical Trials Registry Platform (WHO ICTRP) (http://www.who.int/ictrp/search/en/). We also searched ALOIS, the Cochrane Dementia & Cognitive Improvement Group's specialised register of dementia studies (http://www.medicine.ox.ac.uk/alois/). We checked the reference lists of any relevant studies and systematic reviews, and performed citation tracking using the Science Citation Index to identify any additional relevant studies. No language or date restrictions were applied to the electronic searches.

SELECTION CRITERIA

We included studies that had prospectively defined cohorts with any accepted definition of MCI at time of performing the test and the use of ¹⁸F-flutemetamol scan to evaluate the DTA of the progression from MCI to ADD or other forms of dementia. In addition, we only selected studies that applied a reference standard for Alzheimer's dementia diagnosis, for example, National Institute of Neurological and Communicative Disorders and Stroke and the Alzheimer's Disease and Related Disorders Association (NINCDS-ADRDA) or Diagnostic and Statistical Manual of Mental Disorders-IV (DSM-IV) criteria.

DATA COLLECTION AND ANALYSIS

We screened all titles and abstracts identified in electronic-database searches. Two review authors independently selected studies for inclusion and extracted data to create two-by-two tables, showing the binary test results cross-classified with the binary reference standard. We used these data to calculate sensitivities, specificities, and their 95% confidence intervals. Two independent assessors performed quality assessment using the QUADAS-2 tool plus some additional items to assess the methodological quality of the included studies.

MAIN RESULTS

Progression from MCI to ADD was evaluated in 243 participants from two studies. The studies reported data on 19 participants with two years of follow-up and on 224 participants with three years of follow-up. Nine (47.4%) participants converted at two years follow-up and 81 (36.2%) converted at three years of follow-up.There were concerns about participant selection and sampling in both studies. The index test domain in one study was considered unclear and in the second study it was considered at low risk of bias. For the reference standard domain, one study was considered at low risk and the second study was considered to have an unclear risk of bias. Regarding the domains of flow and timing, both studies were considered at high risk of bias. MCI to ADD;Progression from MCI to ADD at two years of follow-up had a sensitivity of 89% (95% CI 52 to 100) and a specificity of 80% (95% CI 44 to 97) by quantitative assessment by SUVR (n = 19, 1 study).Progression from MCI to ADD at three years of follow-up had a sensitivity of 64% (95% CI 53 to 75) and a specificity of 69% (95% CI 60 to 76) by visual assessment (n = 224, 1 study).There was no information regarding the other two objectives in this systematic review (SR): progression from MCI to other forms of dementia and progression to any form of dementia at follow-up.

AUTHORS' CONCLUSIONS

Due to the varying sensitivity and specificity for predicting the progression from MCI to ADD and the limited data available, we cannot recommend routine use of ¹⁸F-flutemetamol in clinical practice. ¹⁸F-flutemetamol has high financial costs; therefore, clearly demonstrating its DTA and standardising the process of the ¹⁸F-flutemetamol modality is important prior to its wider use.

18F PET with florbetaben for the early diagnosis of Alzheimer's disease dementia and other dementias in people with mild cognitive impairment (MCI)

BACKGROUND

18F-florbetaben uptake by brain tissue, measured by positron emission tomography (PET), is accepted by regulatory agencies like the Food and Drug Administration (FDA) and the European Medicine Agencies (EMA) for assessing amyloid load in people with dementia. Its added value is mainly demonstrated by excluding Alzheimer's pathology in an established dementia diagnosis. However, the National Institute on Aging and Alzheimer's Association (NIA-AA) revised the diagnostic criteria for Alzheimer's disease and confidence in the diagnosis of mild cognitive impairment (MCI) due to Alzheimer's disease may be increased when using some amyloid biomarkers tests like 18F-florbetaben. These tests, added to the MCI core clinical criteria, might increase the diagnostic test accuracy (DTA) of a testing strategy. However, the DTA of 18F-florbetaben to predict the progression from MCI to Alzheimer's disease dementia (ADD) or other dementias has not yet been systematically evaluated.

OBJECTIVES

To determine the DTA of the 18F-florbetaben PET scan for detecting people with MCI at time of performing the test who will clinically progress to ADD, other forms of dementia (non-ADD), or any form of dementia at follow-up.

SEARCH METHODS

The most recent search for this review was performed in May 2017. We searched MEDLINE (OvidSP), Embase (OvidSP), PsycINFO (OvidSP), BIOSIS Citation Index (Thomson Reuters Web of Science), Web of Science Core Collection, including the Science Citation Index (Thomson Reuters Web of Science) and the Conference Proceedings Citation Index (Thomson Reuters Web of Science), LILACS (BIREME), CINAHL (EBSCOhost), ClinicalTrials.gov (https://clinicaltrials.gov), and the World Health Organization International Clinical Trials Registry Platform (WHO ICTRP) (http://www.who.int/ictrp/search/en/). We also searched ALOIS, the Cochrane Dementia & Cognitive Improvement Group's specialised register of dementia studies (http://www.medicine.ox.ac.uk/alois/). We checked the reference lists of any relevant studies and systematic reviews, and performed citation tracking using the Science Citation Index to identify any additional relevant studies. No language or date restrictions were applied to electronic searches.

SELECTION CRITERIA

We included studies that had prospectively defined cohorts with any accepted definition of MCI at time of performing the test and the use of 18F-florbetaben scan to evaluate the DTA of the progression from MCI to ADD or other forms of dementia. In addition, we only selected studies that applied a reference standard for Alzheimer's dementia diagnosis, for example, the National Institute of Neurological and Communicative Disorders and Stroke and the Alzheimer's Disease and Related Disorders Association (NINCDS-ADRDA) or Diagnostic and Statistical Manual of Mental Disorders-IV (DSM-IV) criteria.

DATA COLLECTION AND ANALYSIS

We screened all titles and abstracts identified in electronic-database searches. Two review authors independently selected studies for inclusion and extracted data to create two-by-two tables, showing the binary test results cross-classified with the binary reference standard. We used these data to calculate sensitivities, specificities, and their 95% confidence intervals. Two independent assessors performed quality assessment using the QUADAS-2 tool plus some additional items to assess the methodological quality of the included studies.

MAIN RESULTS

Progression from MCI to ADD, any other form of dementia, and any form of dementia was evaluated in one study (Ong 2015). It reported data on 45 participants at four years of follow-up; 21 participants met NINCDS-ADRDA criteria for Alzheimer's disease dementia at four years of follow-up, the proportion converting to ADD was 47% of the 45 participants, and 11% of the 45 participants met criteria for other types of dementias (three cases of FrontoTemporal Dementia (FTD), one of Dementia with Lewy body (DLB), and one of Progressive Supranuclear Palsy (PSP)). We considered the study to be at high risk of bias in the domains of the reference standard, flow, and timing (QUADAS-2). MCI to ADD; 18F-florbetaben PET scan analysed visually: the sensitivity was 100% (95% confidence interval (CI) 84% to 100%) and the specificity was 83% (95% CI 63% to 98%) (n = 45, 1 study). Analysed quantitatively: the sensitivity was 100% (95% CI 84% to 100%) and the specificity was 88% (95% CI 68% to 97%) for the diagnosis of ADD at follow-up (n = 45, 1 study). MCI to any other form of dementia (non-ADD); 18F-florbetaben PET scan analysed visually: the sensitivity was 0% (95% CI 0% to 52%) and the specificity was 38% (95% CI 23% to 54%) (n = 45, 1 study). Analysed quantitatively: the sensitivity was 0% (95% CI 0% to 52%) and the specificity was 40% (95% CI 25% to 57%) for the diagnosis of any other form of dementia at follow-up (n = 45, 1 study). MCI to any form of dementia;18F-florbetaben PET scan analysed visually: the sensitivity was 81% (95% CI 61% to 93%) and the specificity was 79% (95% CI 54% to 94%) (n = 45, 1 study). Analysed quantitatively: the sensitivity was 81% (95% CI 61% to 93%) and the specificity was 84% (95% CI 60% to 97%) for the diagnosis of any form of dementia at follow-up (n = 45, 1 study).

AUTHORS' CONCLUSIONS

Although we were able to calculate one estimation of DTA in, especially, the prediction of progression from MCI to ADD at four years follow-up, the small number of participants implies imprecision of sensitivity and specificity estimates. We cannot make any recommendation regarding the routine use of 18F-florbetaben in clinical practice based on one single study with 45 participants. 18F-florbetaben has high financial costs, therefore, clearly demonstrating its DTA and standardising the process of the 18F-florbetaben modality are important prior to its wider use.

Clinical Benefits of Preoperative Conventional Fluorescence Diagnosis in Surgical Treatment of Extramammary Paget Disease

BACKGROUND

In extramammary Paget disease (EMPD), initial margins of surgical excision are difficult to determine preoperatively. Fluorescence diagnosis (FD) may have utility as a noninvasive method for delineating tumor boundaries.

OBJECTIVE

The relationship between FD-delineated borders and results of multiple scouting biopsies (MSBs) in EMPD was investigated, evaluating the potential clinical role of FD.

MATERIALS AND METHODS

A cohort of 21 patients was studied, looking for consistencies between FD and MSB results. Initial surgical margins were determined by FD or FD plus MSB. All margins were subjected to frozen section analysis.

RESULTS

Paget cells were identified in 88 of 117 (71.8%) biopsies obtained from FD borders. When used preoperatively with MSB in 9 patients, 96.4% of margins proved negative at initial resection stage. Using a 6-mm margin beyond FD borders in another 9 patients, 86.7% of margins were negative at initial stage. The maximum distance of excision beyond FD border was 12 mm.

CONCLUSION

Fluorescence diagnosis borders (identified visually) correlated well with MSB histopathology. Fluorescence diagnosis is a useful method to delineate initial margins for surgical resection of EMPD.

Decision making about healthcare-related tests and diagnostic test strategies. Paper 2: a review of methodological and practical challenges

OBJECTIVES

In this first of a series of five articles, we provide an overview of how and why healthcare-related tests and diagnostic strategies are currently applied. We also describe how our findings can be integrated with existing frameworks for making decisions that guide the use of healthcare-related tests and diagnostic strategies.

STUDY DESIGN AND SETTING

We searched MEDLINE, references of identified articles, chapters in relevant textbooks, and identified articles citing classic literature on this topic.

RESULTS

We provide updated frameworks for the potential roles and applications of tests with suggested definitions and practical examples. We also discuss study designs that are commonly used to assess tests' performance and the effects of tests on people's health. These designs include diagnostic randomized controlled trials and retrospective validation. We describe the utility of these and other currently suggested designs, which questions they can answer and which ones they cannot. In addition, we summarize the challenges unique to decision-making resulting from the use of tests.

CONCLUSION

This overview highlights current challenges in the application of tests in decision-making in healthcare, provides clarifications, and informs the proposed solutions.

Sample size re-estimation in paired comparative diagnostic accuracy studies with a binary response

BACKGROUND

The sample size required to power a study to a nominal level in a paired comparative diagnostic accuracy study, i.e. studies in which the diagnostic accuracy of two testing procedures is compared relative to a gold standard, depends on the conditional dependence between the two tests - the lower the dependence the greater the sample size required. A priori, we usually do not know the dependence between the two tests and thus cannot determine the exact sample size required. One option is to use the implied sample size for the maximal negative dependence, giving the largest possible sample size. However, this is potentially wasteful of resources and unnecessarily burdensome on study participants as the study is likely to be overpowered. A more accurate estimate of the sample size can be determined at a planned interim analysis point where the sample size is re-estimated.

METHODS

This paper discusses a sample size estimation and re-estimation method based on the maximum likelihood estimates, under an implied multinomial model, of the observed values of conditional dependence between the two tests and, if required, prevalence, at a planned interim. The method is illustrated by comparing the accuracy of two procedures for the detection of pancreatic cancer, one procedure using the standard battery of tests, and the other using the standard battery with the addition of a PET/CT scan all relative to the gold standard of a cell biopsy. Simulation of the proposed method illustrates its robustness under various conditions.

RESULTS

The results show that the type I error rate of the overall experiment is stable using our suggested method and that the type II error rate is close to or above nominal. Furthermore, the instances in which the type II error rate is above nominal are in the situations where the lowest sample size is required, meaning a lower impact on the actual number of participants recruited.

CONCLUSION

We recommend multinomial model maximum likelihood estimation of the conditional dependence between paired diagnostic accuracy tests at an interim to reduce the number of participants required to power the study to at least the nominal level.

TRIAL REGISTRATION

ISRCTN ISRCTN73852054 . Registered 9th of January 2015. Retrospectively registered.

CSF tau and the CSF tau/ABeta ratio for the diagnosis of Alzheimer's disease dementia and other dementias in people with mild cognitive impairment (MCI)

BACKGROUND

Research suggests that measurable change in cerebrospinal fluid (CSF) biomarkers occurs years in advance of the onset of clinical symptoms (Beckett 2010). In this review, we aimed to assess the ability of CSF tau biomarkers (t-tau and p-tau) and the CSF tau (t-tau or p-tau)/ABeta ratio to enable the detection of Alzheimer's disease pathology in patients with mild cognitive impairment (MCI). These biomarkers have been proposed as important in new criteria for Alzheimer's disease dementia that incorporate biomarker abnormalities.

OBJECTIVES

To determine the diagnostic accuracy of 1) CSF t-tau, 2) CSF p-tau, 3) the CSF t-tau/ABeta ratio and 4) the CSF p-tau/ABeta ratio index tests for detecting people with MCI at baseline who would clinically convert to Alzheimer's disease dementia or other forms of dementia at follow-up.

SEARCH METHODS

The most recent search for this review was performed in January 2013. We searched MEDLINE (OvidSP), Embase (OvidSP), BIOSIS Previews (Thomson Reuters Web of Science), Web of Science Core Collection, including Conference Proceedings Citation Index (Thomson Reuters Web of Science), PsycINFO (OvidSP), and LILACS (BIREME). We searched specialized sources of diagnostic test accuracy studies and reviews. We checked reference lists of relevant studies and reviews for additional studies. We contacted researchers for possible relevant but unpublished data. We did not apply any language or data restriction to the electronic searches. We did not use any methodological filters as a method to restrict the search overall.

SELECTION CRITERIA

We selected those studies that had prospectively well-defined cohorts with any accepted definition of MCI and with CSF t-tau or p-tau and CSF tau (t-tau or p-tau)/ABeta ratio values, documented at or around the time the MCI diagnosis was made. We also included studies which looked at data from those cohorts retrospectively, and which contained sufficient data to construct two by two tables expressing those biomarker results by disease status. Moreover, studies were only selected if they applied a reference standard for Alzheimer's disease dementia diagnosis, for example, the NINCDS-ADRDA or Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition (DSM-IV) criteria.

DATA COLLECTION AND ANALYSIS

We screened all titles generated by the electronic database searches. Two review authors independently assessed the abstracts of all potentially relevant studies, and the full papers for eligibility. Two independent assessors performed data extraction and quality assessment. Where data allowed, we derived estimates of sensitivity at fixed values of specificity from the model we fitted to produce the summary receiver operating characteristic (ROC) curve.

MAIN RESULTS

In total, 1282 participants with MCI at baseline were identified in the 15 included studies of which 1172 had analysable data; 430 participants converted to Alzheimer's disease dementia and 130 participants to other forms of dementia. Follow-up ranged from less than one year to over four years for some participants, but in the majority of studies was in the range one to three years. Conversion to Alzheimer's disease dementia The accuracy of the CSF t-tau was evaluated in seven studies (291 cases and 418 non-cases).The sensitivity values ranged from 51% to 90% while the specificity values ranged from 48% to 88%. At the median specificity of 72%, the estimated sensitivity was 75% (95% CI 67 to 85), the positive likelihood ratio was 2.72 (95% CI 2.43 to 3.04), and the negative likelihood ratio was 0.32 (95% CI 0.22 to 0.47).Six studies (164 cases and 328 non-cases) evaluated the accuracy of the CSF p-tau. The sensitivities were between 40% and 100% while the specificities were between 22% and 86%. At the median specificity of 47.5%, the estimated sensitivity was 81% (95% CI: 64 to 91), the positive likelihood ratio was 1.55 (CI 1.31 to 1.84), and the negative likelihood ratio was 0.39 (CI: 0.19 to 0.82).Five studies (140 cases and 293 non-cases) evaluated the accuracy of the CSF p-tau/ABeta ratio. The sensitivities were between 80% and 96% while the specificities were between 33% and 95%. We did not conduct a meta-analysis because the studies were few and small. Only one study reported the accuracy of CSF t-tau/ABeta ratio.Our findings are based on studies with poor reporting. A significant number of studies had unclear risk of bias for the reference standard, participant selection and flow and timing domains. According to the assessment of index test domain, eight of 15 studies were of poor methodological quality.The accuracy of these CSF biomarkers for 'other dementias' had not been investigated in the included primary studies. Investigation of heterogeneity The main sources of heterogeneity were thought likely to be reference standards used for the target disorders, sources of recruitment, participant sampling, index test methodology and aspects of study quality (particularly, inadequate blinding).We were not able to formally assess the effect of each potential source of heterogeneity as planned, due to the small number of studies available to be included.

AUTHORS' CONCLUSIONS

The insufficiency and heterogeneity of research to date primarily leads to a state of uncertainty regarding the value of CSF testing of t-tau, p-tau or p-tau/ABeta ratio for the diagnosis of Alzheimer's disease in current clinical practice. Particular attention should be paid to the risk of misdiagnosis and overdiagnosis of dementia (and therefore over-treatment) in clinical practice. These tests, like other biomarker tests which have been subject to Cochrane DTA reviews, appear to have better sensitivity than specificity and therefore might have greater utility in ruling out Alzheimer's disease as the aetiology to the individual's evident cognitive impairment, as opposed to ruling it in. The heterogeneity observed in the few studies awaiting classification suggests our initial summary will remain valid. However, these tests may have limited clinical value until uncertainties have been addressed. Future studies with more uniformed approaches to thresholds, analysis and study conduct may provide a more homogenous estimate than the one that has been available from the included studies we have identified.

Does drug-induced sleep endoscopy change the surgical decision in surgically naïve non-syndromic children with snoring/sleep disordered breathing from the standard adenotonsillectomy? A retrospective cohort study

BACKGROUND

Adenotonsillectomy is the most commonly performed operation for pediatric snoring/sleep disordered breathing (S/SDB). However, 20-40% of patients will fail to improve. Drug-induced sleep endoscopy (DISE) may provide a more individualized surgical plan and limit unsuccessful surgeries. The aim of this study was to assess the impact of DISE on surgical decision-making in surgically naïve children with S/SDB.

METHODS

A retrospective observational cohort study was undertaken at the Stollery Children's Hospital. Patients 3-17 years of age who underwent DISE-directed surgery for S/SDB between January 2009 and December 2015 were eligible. We excluded other indications for tonsillectomy and syndromic children. The primary outcome was the level of agreement between a DISE-based surgical decision and the reference standard based on the American Academy of Pediatrics (AAP) guidelines via un-weighted Cohen's kappa. Secondary outcomes included the frequency and type of alternate surgical targets identified by DISE. The agreement on tonsil size between in-office physical assessment and DISE was also calculated. The effectiveness of DISE-directed surgery on postoperative S/SDB was not investigated.

RESULTS

Five hundred fifty-eight patients were included. DISE changed the surgical plan in 35% of patients. Agreement between DISE-based and AAP clinical practice guidelines-based management was low (κ = 0.354 +/- 0.021 [95% CI 0.312-0.395]). An alternate diagnosis or surgical target was identified by DISE in 54% of patients. There was moderate agreement on tonsil size (κ = 0.44 [0.33-0.55]) between DISE and in-office clinical assessment.

CONCLUSIONS

This is a first phase diagnostic study, which demonstrates that DISE affects decision-making in surgically naïve children with S/SDB in up to 35% of patients. It has utility in individualizing first stage surgical treatments as well as identifying alternate targets for further surgical or medical therapy, while potentially limiting unsuccessful surgeries. Further studies to examine the effect of DISE on surgical outcomes are required.

Stress test of a biological early warning system with zebrafish (Danio rerio)

The aim of this work was to develop a novel methodology to stress test the diagnostic capability of a video tracking system with zebrafish (Danio rerio), against two pre-established disturbances. Eight different treatments were tested varying the presence or absence of a toxicant (NaOCl) and two disturbances: the passing of a shadow (mimicking a predator) and entrapment of the fish. The concentration tested corresponded to a sublethal (1 % 24 h-LC₅₀) and short term exposure (2 h). A total of 56 organisms were tested resulting in 112 diagnoses (before and after the contamination). A statistical model of diagnosis was developed using Self-organizing Map (SOM) and Correspondence Analysis (CA). Sensitivity, specificity, accuracy, false positive and false negative values were calculated to evaluate the diagnostic performance. The disturbances did not negatively affect the capability of the model. In the presence of at least one of these variables, the diagnostic performance was similar or even superior to the baseline results without disturbances. Furthermore, the system produced a large number of correct diagnoses, at an ecologically relevant concentration of exposure, in a non-invasive way.