Defining the clinical pathway in cochrane diagnostic test accuracy reviews

Designing tailored maintenance strategies for systematic reviews and clinical practice guidelines using the Portfolio Maintenance by Test-Treatment (POMBYTT) framework

BACKGROUND

Organizations face diverse contexts and requirements when updating and maintaining their portfolio, or pool, of systematic reviews or clinical practice guidelines they need to manage. We aimed to develop a comprehensive, theoretical framework that might enable the design and tailoring of maintenance strategies for portfolios containing systematic reviews and guidelines.

METHODS

We employed a conceptual approach combined with a literature review. Components of the diagnostic test-treatment pathway used in clinical healthcare were transferred to develop a framework specifically for systematic review and guideline portfolio maintenance strategies.

RESULTS

We developed the Portfolio Maintenance by Test-Treatment (POMBYTT) framework comprising diagnosis, staging, management, and monitoring components. To illustrate the framework's components and their elements, we provided examples from both a clinical healthcare test-treatment pathway and a clinical practice guideline maintenance scenario. Additionally, our literature review provided possible examples for the elements in the framework, such as detection variables, detection tests, and detection thresholds. We furthermore provide three example strategies using the framework, of which one was based on living recommendations strategies.

CONCLUSIONS

The developed framework might support the design of maintenance strategies that could contain multiple options besides updating to manage a portfolio (e.g. withdrawing and archiving), even in the absence of the target condition. By making different choices for variables, tests, test protocols, indications, management options, and monitoring, organizations might tailor their maintenance strategy to suit specific contexts and needs. The framework's elements could potentially aid in the design by being explicit about the operational aspects of maintenance strategies. This might also be helpful for end-users and other stakeholders of systematic reviews and clinical practice guidelines.

Ocular Biomarkers for Alzheimer Disease Dementia: An Umbrella Review of Systematic Reviews and Meta-analyses

Importance

Several ocular biomarkers have been proposed for the early detection of Alzheimer disease (AD) and mild cognitive impairment (MCI), particularly fundus photography, optical coherence tomography (OCT), and OCT angiography (OCTA).

Objective

To perform an umbrella review of systematic reviews to assess the diagnostic accuracy of ocular biomarkers for early diagnosis of Alzheimer disease.

Data Sources

MEDLINE, Embase, and PsycINFO were searched from January 2000 to November 2021. The references of included reviews were also searched.

Study Selection

Systematic reviews investigating the diagnostic accuracy of ocular biomarkers to detect AD and MCI, in secondary care or memory clinics, against established clinical criteria or clinical judgment.

Data Extraction and Synthesis

The Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) reporting guideline checklist was followed and the Risk Of Bias in Systematic reviews tool was used to assess review quality.

Main Outcomes and Measures

The prespecified outcome was the accuracy of ocular biomarkers for diagnosing AD and MCI. The area under the curve (AUC) was derived from standardized mean difference.

Results

From the 591 titles, 14 systematic reviews were included (median [range] number of studies in each review, 14 [5-126]). Only 4 reviews were at low risk of bias on all Risk of Bias in Systematic Reviews domains. The imaging-derived parameters with the most evidence for detecting AD compared with healthy controls were OCT peripapillary retinal nerve fiber layer thickness (38 studies including 1883 patients with AD and 2510 controls; AUC = 0.70; 95% CI, 0.53-0.79); OCTA foveal avascular zone (5 studies including 177 patients with AD and 371 controls; AUC = 0.73; 95% CI, 0.50-0.89); and saccadic eye movements prosaccade latency (30 studies including 651 patients with AD/MCI and 771 controls; AUC = 0.64; 95% CI, 0.58-0.69). Antisaccade error was investigated in fewer studies (12 studies including 424 patients with AD/MCI and 382 controls) and yielded the best accuracy (AUC = 0.79; 95% CI, 0.70-0.88).

Conclusions and Relevance

This umbrella review has highlighted limitations in design and reporting of the existing research on ocular biomarkers for diagnosing AD. Parameters with the best evidence showed poor to moderate diagnostic accuracy in cross-sectional studies. Future longitudinal studies should investigate whether changes in OCT and OCTA measurements over time can yield accurate predictions of AD onset.

Accuracy of optical coherence tomography for diagnosing glaucoma: an overview of systematic reviews

AIMS

To assess the diagnostic accuracy (DTA) of optical coherence tomography (OCT) for detecting glaucoma by systematically searching and appraising systematic reviews (SRs) on this issue.

METHODS

We searched a database of SRs in eyes and vision maintained by the Cochrane Eyes and Vision United States on the DTA of OCT for detecting glaucoma. Two authors working independently screened the records, abstracted data and assessed the risk of bias using the Risk of Bias in Systematic Reviews checklist. We extracted quantitative DTA estimates as well as qualitative statements on their relevance to practice.

RESULTS

We included four SRs published between 2015 and 2018. These SRs included between 17 and 113 studies on OCT for glaucoma diagnosis. Two reviews were at low risk of bias and the other two had two to four domains at high or unclear risk of bias with concerns on applicability. The two reliable SRs reported the accuracy of average retinal nerve fibre layer (RNFL) thickness and found a sensitivity of 0.69 (0.63 to 0.73) and 0.78 (0.74 to 0.83) and a specificity of 0.94 (0.93 to 0.95) and 0.93 (0.92 to 0.95) in 57 and 50 studies, respectively. Only one review included a clear specification of the clinical pathway. Both reviews highlighted the limitations of primary DTA studies on this topic.

CONCLUSIONS

The quality of published DTA reviews on OCT for diagnosing glaucoma was mixed. Two reliable SRs found moderate sensitivity at high specificity for average RNFL thickness in diagnosing manifest glaucoma. Our overview suggests that the methodological quality of both primary and secondary DTA research on glaucoma is in need of improvement.

Towards standardisation of evidence-based clinical care process specifications

There is a strong push towards standardisation of treatment approaches, care processes and documentation of clinical practice. However, confusion persists regarding terminology and description of many clinical care process specifications which this research seeks to resolve by developing a taxonomic characterisation of clinical care process specifications. Literature on clinical care process specifications was analysed, creating the starting point for identifying common characteristics and how each is constructed and used in the clinical setting. A taxonomy for clinical care process specifications is presented. The De Bleser approach to limited clinical care process specifications characterisation was extended and each clinical care process specification is successfully characterised in terms of purpose, core elements and relationship to the other clinical care process specification types. A case study on the diagnosis and treatment of Type 2 Diabetes in the United Kingdom was used to evaluate the taxonomy and demonstrate how the characterisation framework applies. Standardising clinical care process specifications ensures that the format and content are consistent with expectations, can be read more quickly and high-quality information can be recorded about the patient. Standardisation also enables computer interpretability, which is important in integrating Learning Health Systems into the modern clinical environment. The approach presented allows terminologies for clinical care process specifications that were widely used interchangeably to be easily distinguished, thus, eliminating the existing confusion.

Reproducibility protocol for diagnostic procedures in Manual/Musculoskeletal Medicine

The International Academy of Manual/Musculoskeletal Medicine (IAMMM) has completely revised the protocol for reproducibility studies of diagnostic procedures in Manual/Musculoskeletal Medicine (M/M Medicine). The protocol was and is aimed at the practitioners in the field of M/M Medicine. This IAMMM protocol can be used in a very practical way and makes it feasible to perform reproducibility studies equally well in private practices and clinics for M/M Medicine with two or more physicians and as by educational boards of the societies of M/M Medicine. This IAMMM protocol provides practical solutions for sample size calculations in reproducibility studies using kappa statistics. Step by step, many different statistical aspects of reproducibility studies are explained, resulting in a very structured protocol format of how to perform a reproducibility study using overall agreement and the kappa value as statistical outcome.

An Audit of the Diagnostic Accuracy of Rotational Thromboelastometry for the Identification of Hypofibrinogenaemia and Thrombocytopenia during Cardiopulmonary Bypass

We audited the diagnostic accuracy of ROTEM® (TEM Innovations, GmbH, Munich, Germany) measurements of hypofibrinogenaemia (fibrinogen <1.5 g/l) and thrombocytopenia (platelet count <100 × 109/l) in 200 adult non-transplant patients during cardiopulmonary bypass (CPB). Blood samples were obtained for FIBTEM (assay for the fibrin part of the clot), PLTEM (calculated platelet-specific component), and laboratory measurements simultaneously. Our thresholds for FIBTEM and PLTEM were A10 (clot firmness 10 minutes after clotting time) ≤8 mm, and <35 mm respectively. We also calculated the accuracy of smaller thresholds and earlier indices. We found that FIBTEM A10 ≤8 mm had low sensitivity (0.62) for hypofibrinogenaemia. With the 13% hypofibrinogenaemia prevalence in our sample, the positive predictive value (PPV) was 0.47. In contrast, FIBTEM A10 ≤8 mm had higher specificity (0.90) (negative predictive value [NPV] 0.94). Of the other FIBTEM values analysed, only A5 ≤6 mm had similar or superior accuracy. The PLTEM results were less encouraging (sensitivity 0.81, specificity 0.62). With our prevalence of thrombocytopenia (also 13%), the PPV was only 0.24. However, the NPV was high (0.96). Of the other PLTEM values analysed, only A5 <25 mm had similar or superior accuracy. These findings indicate that during CPB FIBTEM A10 ≤8 mm and PLTEM A10 <35 mm have greater accuracy in identifying the absence of hypofibrinogenaemia and thrombocytopenia respectively than their presence. On the basis of these results we would be reassured by FIBTEM A10 values >8 mm and PLTEM A10 values ≥35 mm, but would continue to use laboratory measurements for confirmation. We would not use FIBTEM A10 ≤8 mm or PLTEM A10 <35 mm values alone to guide replacement therapy unless clinical conditions warranted an immediate decision before laboratory measurements were available.

Diagnostic accuracy of the UBC® Rapid Test for bladder cancer: A meta-analysis

Bladder cancer is one of the most common cancer types globally. The UBC^® Rapid Test is a potential novel diagnostic method for bladder cancer, but studies into its accuracy have produced inconsistent results. Thus, the present meta-analysis was conducted in order to determine the overall accuracy of the UBC^® Rapid Test in detecting bladder cancer. A comprehensive literature search was conducted using MEDLINE, Embase, Cochrane Library, Web of Science, Chinese WanFang and the China National Knowledge Infrastructure databases for relevant studies. Quality assessment of diagnostic accuracy studies 2 was used to assess the quality of each included study. The diagnostic accuracy of the UBC^® Rapid Test was evaluated by pooled sensitivity, specificity, positive likelihood ratio (PLR), negative likelihood ratio (NLR), diagnostic odds ratio (DOR) and the area under the curve (AUC). In addition, Deeks' funnel plot was used to evaluate potential publication bias. Eight studies were included in the quantitative meta-analysis. The results were as follows: Sensitivity 0.59 [95% confidence interval (CI), 0.55–0.62], specificity 0.76 (95% CI, 0.72–0.80), PLR 2.55 (95% CI, 1.75–3.70), NLR 0.56 (95% CI, 0.46–0.67), DOR 4.88 (95% CI, 2.82–8.45) and AUC 0.70 (95% CI, 0.67–0.74). According to the present results, the UBC^® rapid test is highly accurate in the diagnosis of bladder cancer, however, further studies with better-designed and larger samples are required in order to support the results of the present study.

Establishing the Value of Diagnostic and Prognostic Tests in Health Technology Assessment

In recent years, Health Technology Assessment (HTA) processes specific to diagnostics and prognostic tests have been created in response to the increased pressure on health systems to decide not only which tests should be used in practice but also the best way to proceed, clinically, from the information they provide. These technologies differ in the way value is accrued to the population of users, depending critically on the value of downstream health care choices. This paper defines an analytical framework for establishing the value of diagnostic and prognostic tests for HTA in a way that is consistent with methods used for the evaluation of other health care technologies. It assumes a linked-evidence approach where modeling is required, and incorporates considerations regarding several different areas of policy, such as personalized medicine. We initially focus on diagnostic technologies with dichotomous results, and then extend the framework by considering diagnostic tests that provide more complex information, such as continuous measures (for example, blood glucose measurements) or multiple categories (such as tumor classification systems). We also consider how the methods of assessment differ for prognostic information or for diagnostics without a reference standard. Throughout, we propose innovative graphical ways of summarizing the results of such complex assessments of value.

STARD 2015 was reproducible in a large set of studies on glaucoma

Aim

To investigate the reproducibility of the updated Standards for the Reporting of Diagnostic Accuracy Studies tool (STARD 2015) in a set of 106 studies included in a Cochrane diagnostic test accuracy (DTA) systematic review of imaging tests for diagnosing manifest glaucoma.

Methods

One senior rater with DTA methodological and clinical expertise used STARD 2015 on all studies, and each of three raters with different training profiles assessed about a third of the studies.

Results

Raw agreement was very good or almost perfect between the senior rater and an ophthalmology resident with DTA methods training, acceptable with a clinical rater with little DTA methods training, and only moderate with a pharmacology researcher with general, but not DTA, systematic review training and no clinical expertise. The relationship between adherence with STARD 2015 and methodological quality with QUADAS 2 was only partial and difficult to investigate, suggesting that raters used substantial context knowledge in risk of bias assessment.

Conclusions

STARD 2015 proved to be reproducible in this specific research field, provided that both clinical and DTA methodological expertise are achieved through training of its users.

Diagnosis in vascular dementia, applying 'Cochrane diagnosis rules' to 'dementia diagnostic tools'

In this issue of Clinical Science, Biesbroek and colleagues describe recent work on magnetic resonance imaging (MRI)-based cerebral lesion location and its association with cognitive decline. The authors conclude that diagnostic neuroimaging in dementia should shift from whole-brain evaluation to focused quantitative analysis of strategic brain areas. This commentary uses the review of lesion location mapping to discuss broader issues around studies of dementia test strategies. We draw upon work completed by the Cochrane Dementia and Cognitive Improvement Group designed to improve design, conduct and reporting of dementia biomarker studies.

Diagnostic accuracy of confocal microscopy imaging vs. punch biopsy for diagnosing and subtyping basal cell carcinoma

Background

In vivo reflectance confocal microscopy (RCM) is a promising non‐invasive skin imaging technique that could facilitate early diagnosis of basal cell carcinoma (BCC) instead of routine punch biopsies. However, the clinical value and utility of RCM vs. a punch biopsy in diagnosing and subtyping BCC is unknown.

Objective

To assess diagnostic accuracy of RCM vs. punch biopsy for diagnosing and subtyping clinically suspected primary BCC.

Methods

A prospective, consecutive cohort of 100 patients with clinically suspected BCC were included at two tertiary hospitals in Amsterdam, the Netherlands, between 3 February 2015 and 2 October 2015. Patients were randomized between two test‐treatment pathways: diagnosing and subtyping using RCM imaging followed by direct surgical excision (RCM one‐stop‐shop) or planned excision based upon the histological diagnosis and subtype of punch biopsy (standard care). The primary outcome was the agreement between the index tests (RCM vs. punch biopsy) and reference standard (excision specimen) in correctly diagnosing BCC. The secondary outcome was the agreement between the index tests and reference standard in correctly identifying the most aggressive BCC subtypes.

Results

Sensitivity to detect BCC was similar for RCM and punch biopsy (100% vs. 93.94%), but a punch biopsy was more specific than RCM (79% vs. 38%). RCM expert evaluation for diagnosing BCC had a sensitivity of 100% and a specificity of 75%. The agreement between RCM and excision specimen in identifying the most aggressive BCC subtype ranged from 50% to 85% vs. 77% by a punch biopsy.

Conclusion

Reflectance confocal microscopy and punch biopsy have comparable diagnostic accuracy to diagnose and subtype BCC depending on RCM experience. Although experienced RCM users could accurately diagnose BCC at a distance, we found an important difference in subtyping BCC. Future RCM studies need to focus on diagnostic accuracy, reliability and specific criteria to improve BCC subtype differentiation.

One-stop-shop with confocal microscopy imaging vs. standard care for surgical treatment of basal cell carcinoma: an open-label, noninferiority, randomized controlled multicentre trial

BACKGROUND

Routine punch biopsies are considered to be standard care for diagnosing and subtyping basal cell carcinoma (BCC) when clinically suspected.

OBJECTIVES

We assessed the efficacy of a one-stop-shop concept using in vivo reflectance confocal microscopy (RCM) imaging as a diagnostic tool vs. standard care for surgical treatment in patients with clinically suspected BCC.

METHODS

In this open-label, parallel-group, noninferiority, randomized controlled multicentre trial we enrolled patients with clinically suspected BCC at two tertiary referral centres in Amsterdam, the Netherlands. Patients were randomly assigned to the RCM one-stop-shop (diagnosing and subtyping using RCM followed by direct surgical excision) or standard care (planned excision based on the histological diagnosis and subtype of a punch biopsy). The primary outcome was the proportion of patients with tumour-free margins after surgical excision of BCC.

RESULTS

Of the 95 patients included, 73 (77%) had a BCC histologically confirmed using a surgical excision specimen. All patients (40 of 40, 100%) in the one-stop-shop group had tumour-free margins. In the standard-care group tumour-free margins were found in all but two patients (31 of 33, 94%). The difference in the proportion of patients with tumour-free margins after BCC excision between the one-stop-shop group and the standard-care group was -0·06 (90% confidence interval -0·17-0·01), establishing noninferiority.

CONCLUSIONS

The proposed new treatment strategy seems suitable in facilitating early diagnosis and direct treatment for patients with BCC, depending on factors such as availability of RCM, size and site of the lesion, patient preference and whether direct surgical excision is feasible.