Analyzing paired diagnostic studies by estimating the expected benefit

Research on point-of-care tests in outpatient care in Germany: A scoping review and definition of relevant endpoints in evaluation studies

BACKGROUND

The fast turnaround time and user-friendliness of point-of-care tests (POCTs) offer a great potential to improve outpatient health care where clinical decisions have to be made during the physician-patient encounter and time resources are limited. The aim of this scoping review is to describe the extent and nature as well as gaps in German research activities on POCT in outpatient care. In addition, we define research endpoints that should be addressed in the comprehensive evaluation of POCTs targeted for outpatient care.

METHODS

We performed a scoping review with a systematic literature search in Medline (via PubMed), Scopus, Web of Science, Cochrane library and Google Scholar for German publications on POCT with relevance to German outpatient care published from January 2005 to November 2020.

RESULTS

Our literature search identified 2,200 unique records. After literature selection 117 articles were included in this scoping review. Just over half of the articles (67/117, 57.3%) were primary research studies with original data, while one third of all the studies (33.3%) were secondary research articles (e.g., review articles). The remaining articles were clinical recommendations / position papers (7/117, 6.0%) and other types of articles (3.4%). The majority of articles focused on POCT use in infectious diseases (44/117, 37.6%), diabetic syndromes (15.4%), cardiac disease (12.0%) or coagulopathies and thrombosis (10.3%), while the remaining articles did not specify the disease (13.7%) or investigated other diseases (11.1%). Similar to international studies, most primary research studies investigated the diagnostic performance of POCT (e.g., sensitivity, specificity). Evidence beyond diagnostic accuracy remains scarce, such as the impact on therapeutic decisions and practice routines, clinical effectiveness, and user perspectives. In line with this, interventional studies (such as RCTs) on the effectiveness of POCT use in German outpatient care are limited. We define six endpoint domains that should be addressed in the evaluation of POCTs targeted for outpatient care: (i) diagnostic performance, (ii) clinical performance, (iii) time and costs, (iv) impact on clinical routines / processes, (v) perspectives of medical professionals and patients, and (vi) broader aspects.

CONCLUSION

There is considerable research activity on POCTs targeted for use in outpatient care in Germany. Data on their potential benefits beyond diagnostic accuracy is often lacking and should be addressed in future POCT research studies.

Randomized test-treatment studies with an outlook on adaptive designs

Background

Diagnostic accuracy studies aim to examine the diagnostic accuracy of a new experimental test, but do not address the actual merit of the resulting diagnostic information to a patient in clinical practice. In order to assess the impact of diagnostic information on subsequent treatment strategies regarding patient-relevant outcomes, randomized test-treatment studies were introduced. Various designs for randomized test-treatment studies, including an evaluation of biomarkers as part of randomized biomarker-guided treatment studies, are suggested in the literature, but the nomenclature is not consistent.

Methods

The aim was to provide a clear description of the different study designs within a pre-specified framework, considering their underlying assumptions, advantages as well as limitations and derivation of effect sizes required for sample size calculations. Furthermore, an outlook on adaptive designs within randomized test-treatment studies is given.

Results

The need to integrate adaptive design procedures in randomized test-treatment studies is apparent. The derivation of effect sizes induces that sample size calculation will always be based on rather vague assumptions resulting in over- or underpowered study results. Therefore, it might be advantageous to conduct a sample size re-estimation based on a nuisance parameter during the ongoing trial.

Conclusions

Due to their increased complexity, compared to common treatment trials, the implementation of randomized test-treatment studies poses practical challenges including a huge uncertainty regarding study parameters like the expected outcome in specific subgroups or disease prevalence which might affect the sample size calculation. Since research on adaptive designs within randomized test-treatment studies is limited so far, further research is recommended.

Supplementary Information

The online version contains supplementary material available at (10.1186/s12874-021-01293-y).

Prevalence of Newly Diagnosed Malignancies in Patients with Polymyalgia Rheumatica and Giant Cell Arteritis, Comparison of 18F-FDG PET/CT Scan with Chest X-ray and Abdominal Ultrasound: Data from a 40 Week Prospective, Exploratory, Single Centre Study

The aim of the study was to identify the prevalence of newly diagnosed malignancies in patients with polymyalgia rheumatica (PMR) and giant cell arteritis (GCA), with the aid of 18F-FDG PET/CT scan compared to conventional imaging techniques: Chest X-ray (CXR) and abdominal ultrasound (US). Secondarily, to examine the relative diagnostic accuracy of these two imaging modalities for the detection of cancer. Eighty consecutive patients with newly diagnosed PMR, GCA, or concomitant PMR and GCA, were included and followed up for 40 weeks. All patients underwent an 18F-FDG PET/CT scan, CXR, and abdominal US at diagnosis. Imaging findings were dichotomously categorized into malignant or benign. Among 80 patients, three patients were diagnosed with seronegative rheumatoid arthritis and were excluded from the analysis. Of the remaining 77, 64 (83.1%) patients were diagnosed with pure PMR, 3 (3.9%) with pure GCA, and 10 (13.0%) with concomitant PMR and GCA. Five types of cancer that were more prevalent than the one-year prevalence of 1.2% among the background population were found in four (5.2%; 95%CI: 1.4-12.8%) patients. CXR/abdominal US could detect the solid cancer in one patient, whereas 18F-FDG PET/CT could identify all four solid cancers. Furthermore, four (5.2%; 95%CI: 1.4-12.8%) cases of monoclonal gammopathy of undetermined significance (MGUS) were found. An increase in C reactive protein (CRP) implicated an increased risk for cancer of 2.4% (OR: 1.024, 95%CI: 1.001-1.047; p = 0.041). 18F-FDG PET/CT can reveal occult cancers at an early stage with a high negative predictive value, and it is specifically beneficial in PMR/GCA patients with nonspecific symptoms.

Upfront PET/CT affects management decisions in patients with recurrent head and neck squamous cell carcinoma

PURPOSE

To compare multidisciplinary team conference (MDTC) decisions regarding treatment intent based on either chest X-ray + MRI of the head and neck (CXR/MRI) or ¹⁸F-FDG-PET/CT (PET/CT) in patients with recurrent head and neck squamous cell carcinoma (HNSCC).

METHODS

Prospective blinded cohort study based on paired data. Consecutive patients with suspected recurrent HNSCC were invited to participate. All included patients underwent CXR/MRI and PET/CT before diagnostic biopsy. An ordinary MDTC using all available imaging data was conducted as per standard practice. After at least three months (to eliminate recall bias in the team), the first project MDTC, based on either CXR/MRI or PET/CT, was conducted, and the tumor board made conclusions regarding treatment. After an additional three months, a second project MDTC was conducted using the complementary imaging strategy. The separate treatment strategies were compared using McNemar's test.

RESULTS

A total of 110 patients (90 males and 20 females, median age 66 years, range 40-87) were included. The initial primary tumor originated from the pharynx in 56 (51%) patients, oral cavity in 17 (15%) patients, and larynx in 37 (34%) patients. Based on CXR/MRI, 87 patients (79%) were recommended curative treatment and 23 (21%) palliative treatment. Based on PET/CT, the MDTC decided that 52 (47%) patients were suitable for curative treatment and 58 (53%) for palliative treatment. The absolute difference of 32% was statistically significant (95% CI: 22-42%, p < 0.001).

CONCLUSIONS

PET/CT affected MDTC decisions in patients with recurrent HNSCC towards less curative and more palliative treatment.

Practice-based evidence for the clinical benefit of PET/CT-results of the first oncologic PET/CT registry in Germany

PURPOSE

The purpose of this study was to evaluate the impact of PET/CT on clinical management of cancer patients based on a prospective data registry. The study was developed to inform consultations with public health insurances on PET/CT coverage.

METHODS

We evaluated a prospective patient cohort having a clinically indicated PET/CT at a single German University Center from April 2013 to August 2016. The registry collected questionnaire data from requesting physicians on intended patient management before and after PET/CT. A total of 4,504 patients with 5,939 PET/CT examinations were enrolled in the registry, resulting in evaluable data from 3,724 patients receiving 4,754 scans. The impact of PET/CT on patient management was assessed across 22 tumor types, for different indications (diagnosis, staging, suspected recurrence) and different categories of management including treatment (curative or palliative) and non-treatment (watchful waiting, additional imaging, invasive tests).

RESULTS

The most frequent PET/CT indication was tumor staging (59.7%). Melanoma, lung cancer, lymphoma, neuroendocrine tumor and prostate cancer accounted for 70% of cases. Overall, the use of PET/CT resulted in a 37.1% change of clinical management (95% CI, 35.7-38.5), most frequently (30.6%) from an intended non-treatment strategy before PET/CT to active treatment after PET/CT. The frequency of changes ranged from 28.3% for head and neck cancers up to 46.0% for melanomas. The impact of PET/CT was greatest in reducing demands for additional imaging which decreased from 66.1% before PET/CT to 6.1% after PET/CT. Pre-PET/CT planned invasive tests could be avoided in 72.7% of cases. The treatment goal changed after PET/CT in 21.7% of cases, in twice as many cases from curative to palliative therapy than vice versa.

CONCLUSIONS

The data of this large prospective registry confirm that physicians often change their intended management on the basis of PET/CT by initiating treatment and reducing additional imaging as well as invasive tests. This applies to various cancer types and indications.

Up-front PET/CT changes treatment intent in patients with head and neck squamous cell carcinoma

PURPOSE

In patients with newly diagnosed head and neck squamous cell carcinoma (HNSCC), we wanted to examine the differences in overall treatment decisions, i.e. curative versus palliative treatment intent, reached by a multidisciplinary team conference (MDTC) based on 18F-fluoro-deoxy-glucose-positron emission tomography/computed tomography (PET/CT) or chest X-ray + MRI of the head and neck (CXR/MRI).

PATIENTS AND METHODS

This was a prospective blinded cohort study based on paired data. Consecutive patients with histologically verified primary HNSCC were invited to participate. All included patients underwent CXR/MRI and PET/CT before diagnostic biopsy. An ordinary MDTC using all available imaging was conducted as per standard practice. After at least 3 months (to eliminate recall bias in the team), the first project MDTC was conducted, based on either CXR/MRI or PET/CT, and the tumor board drew conclusions regarding treatment. After an additional 3 months, a second project MDTC was conducted using the complementary imaging modality.

RESULTS

A total of 307 patients were included. Based on CXR/MRI, 303 patients (99%) were recommended for curative treatment and only four patients (1%) for palliative treatment. Based on PET/CT, the MDTC concluded that 278 (91%) patients were suitable for curative treatment and 29 (9%) patients for palliative treatment. The absolute difference of 8% was statistically significant (95% CI: 4.8%-11.5%, p < 0.001).

CONCLUSIONS

A PET/CT-based imaging strategy significantly changed the decisions regarding treatment intent made by a MDTC for patients diagnosed with HNSCC, when compared with the standard imaging strategy of CXR/MRI.

A PET/CT-Based Strategy Is a Stronger Predictor of Survival Than a Standard Imaging Strategy in Patients with Head and Neck Squamous Cell Carcinoma

Our purpose was to examine whether staging of head and neck squamous cell carcinoma (HNSCC) by upfront ¹⁸F-FDG PET/CT (i.e., on the day of biopsy and before the biopsy) discriminates survival better than the traditional imaging strategies based on chest x-ray plus head and neck MRI (CXR/MRI) or chest CT plus head and neck MRI (CCT/MRI). Methods: We performed a masked prospective cohort study based on paired data. Consecutive patients with histologically verified primary HNSCC were recruited from Odense University Hospital from September 2013 to March 2016. All patients underwent CXR/MRI, CCT/MRI, and PET/CT on the same day. Tumors were categorized as localized (stages I and II), locally advanced (stages III and IVB), or metastatic (stage IVC). Discriminative ability for each imaging modality with respect to HNSCC staging were compared using Kaplan-Meier analysis, Cox proportional hazards regression with the Harrell C-index, and net reclassification improvement. Results: In total, 307 patients with histologically verified HNSCC were included. Use of PET/CT significantly altered the stratification of tumor stage when compared with either CXR/MRI or CCT/MRI (χ², P < 0.001 for both). Cancer stages based on PET/CT, but not CXR/MRI or CCT/MRI, were associated with significant differences in mortality risk on Kaplan-Meier analyses (P ≤ 0.002 for all PET/CT-based comparisons). Furthermore, overall discriminative ability was significantly greater for PET/CT (C-index, 0.712) than for CXR/MRI (C-index, 0.675; P = 0.04) or CCT/MRI (C-index, 0.657; P = 0.02). Finally, PET/CT was significantly associated with a positive net reclassification improvement when compared with CXR/MRI (0.184, P = 0.03) but not CCT/MRI (0.094%, P = 0.31). Conclusion: Tumor stages determined by PET/CT were associated with more distinct prognostic properties in terms of survival than those determined by standard imaging strategies.

Value-based and benefit-based strategies in deciding to bring a test into use should be distinguished

BACKGROUND

Regulatory and health technology assessment agencies have commented differently on the question whether results from enrichment studies can be used to justify to bring a test into use. We try to provide a framework to discuss this issue.

RESULTS

Mathematical definitions for the value and the benefit of a new diagnostic test are given. The possible conclusions about value and benefit from enrichment studies and interaction studies are explored. The terms benefit-based strategy and value-based strategy are introduced. Several potential consequences of using one of the two strategies in deciding to bring a test into use are identified and quantified. Interaction designs allow to assess benefit and value. Enrichment designs allow only to assess benefit. However, it is often probable that interaction studies allow no firm conclusions about the value. The advantage of a benefit-based strategy stems mainly from allowing test-positive patients earlier or even ever to benefit. The main disadvantage is a potential delay in detecting tests of no value.

CONCLUSIONS

Benefit-based strategies are preferable if the risk of off-label use and of delayed decisions on the value of a test can be limited. Otherwise, the superiority depends highly on research practice.

[18F]Fluorodeoxyglucose (FDG)-Positron Emission Tomography (PET)/Computed Tomography (CT) in Suspected Recurrent Breast Cancer: A Prospective Comparative Study of Dual-Time-Point FDG-PET/CT, Contrast-Enhanced CT, and Bone Scintigraphy

PURPOSE

To prospectively investigate the diagnostic accuracy of [(18)F]fluorodeoxyglucose (FDG)-positron emission tomography (PET)/computed tomography (CT) with dual-time-point imaging, contrast-enhanced CT (ceCT), and bone scintigraphy (BS) in patients with suspected breast cancer recurrence.

PATIENTS AND METHODS

One hundred women with suspected recurrence of breast cancer underwent 1-hour and 3-hour FDG-PET/CT, ceCT, and BS within approximately 10 days. The study was powered to estimate the precision of the individual imaging tests. Images were visually interpreted using a four-point assessment scale, and readers were blinded to other test results. The reference standard was biopsy along with treatment decisions and clinical follow-up (median, 17 months).

RESULTS

FDG-PET/CT resulted in no false negatives and fewer false positives than the other imaging techniques. Accuracy of results were similar for 1-hour and 3-hour FDG-PET/CT. For distant recurrence, the area under the receiver operating curve was 0.99 (95% CI, 0.97 to 1) for FDG-PET/CT, 0.84 (95% CI, 0.73 to 0.94) for ceCT, and 0.86 (95% CI, 0.77 to 0.94) for the combined ceCT+BS. Of 100 patients, 22 (22%) were verified with distant recurrence, and 18 of these had bone involvement. Nineteen patients (19%) had local recurrence only. In exploratory analyses, diagnostic accuracy of FDG-PET/CT was better than ceCT alone or ceCT combined with BS in diagnosing distant, bone, and local recurrence, shown by a greater area under the receiver operating curve and higher sensitivity, specificity, and superior likelihood ratios.

CONCLUSION

FDG-PET/CT was accurate in diagnosing recurrence in breast cancer patients. It allowed for distant recurrence to be correctly ruled out and resulted in only a small number of false-positive cases. Exploratory findings suggest that FDG-PET/CT has greater accuracy than conventional imaging technologies in this patient group.