Post hoc subgroup analysis and the truth of a clinical trial

A panel of seven protein tumour markers for effective and affordable multi-cancer early detection by artificial intelligence: a large-scale and multicentre case-control study

Background

Early detection of cancer aims to reduce cancer deaths. Unfortunately, many established cancer screening technologies are not suitable for use in low- and middle-income countries (LMICs) due to cost, complexity, and dependency on extensive medical infrastructure. We aimed to assess the performance and robustness of a protein assay (OncoSeek) for multi-cancer early detection (MCED) that is likely to be more practical in LMICs.

Methods

This observational study comprises a retrospective analysis on the data generated from the routine clinical testings at SeekIn and Sun Yat-sen Memorial Hospital. 7565 participants (954 with cancer and 6611 without) from the two sites were divided into training and independent validation cohort. The second validation cohort (1005 with cancer and 812 without) was from Johns Hopkins University School of Medicine. Patients with cancer prior to therapy were eligible for inclusion in the study. Individuals with no history of cancer were enrolled from the participating sites as the non-cancer group. One tube of peripheral blood was collected from each participant and quantified a panel of seven selected protein tumour markers (PTMs) by a common clinical electrochemiluminescence immunoassay analyser. An algorithm named OncoSeek was established using artificial intelligence (AI) to distinguish patients with cancer from those without cancer by calculating the probability of cancer (POC) index based on the quantification results of the seven PTMs and clinical information including sex and age of the individuals and to predict the possible affected tissue of origin (TOO) for those who have been detected with cancer signals in blood.

Findings

Between November 2012 and May 2022, 7565 participants were enrolled at SeekIn and Sun Yat-sen Memorial Hospital. The conventional clinical method, which relies only on a single threshold for each PTM, would suffer from a high false positive rate that accumulates as the number of markers increased. OncoSeek was empowered by AI technology to significantly reduce the false positive rate, increasing the specificity from 56.9% (95% confidence interval [CI]: 55.8-58.0) to 92.9% (92.3-93.5). In all cancer types, the overall sensitivity of OncoSeek was 51.7% (49.4-53.9), resulting in 84.3% (83.5-85.0) accuracy. The performance was generally consistent in the training and the two validation cohorts. The sensitivities ranged from 37.1% to 77.6% for the detection of the nine common cancer types (breast, colorectum, liver, lung, lymphoma, oesophagus, ovary, pancreas, and stomach), which account for ∼59.2% of global cancer deaths annually. Furthermore, it has shown excellent sensitivity in several high-mortality cancer types for which routine screening tests are lacking in the clinic, such as the sensitivity of pancreatic cancer which was 77.6% (69.3-84.6). The overall accuracy of TOO prediction in the true positives was 66.8%, which could assist the clinical diagnostic workup.

Interpretation

OncoSeek significantly outperforms the conventional clinical method, representing a novel blood-based test for MCED which is non-invasive, easy, efficient, and robust. Moreover, the accuracy of TOO facilitates the follow-up diagnostic workup.

Funding

The National Key Research and Development Programme of China.

Subgroup analyses with special reference to the effect of antiplatelet agents in acute coronary syndromes

Controlled trials estimate treatment effects averaged over the reference population of subjects. However, physicians are interested in whether the treatment effect varies across subgroups (effect heterogeneity) in order to target specific subgroups to maximise the benefit of treatment and minimise harm. Therefore, large clinical trials of antiplatelet agents include subgroup analyses that examine whether treatment effects differ between subgroups of subjects identified by baseline characteristics. Reporting subgroup is pervasive and often accompanied by claims of difference of treatment effects between subgroups with potential important implications for clinical practice. However, subgroup-specific analyses of clinical trial data have inherent limitations that reduce their reliability. These include reduced statistical power, failure to specify the subgroups of interest a priori, failure to account for examining large numbers of subgroups, lack of strong rationale for biological response modification, and performing analyses based on variables measured post randomisation or in trials showing no overall difference between treatments. Rules for interpretation of subgroup findings in subgroups have been suggested but are frequently not applied. In this article we draw attention to the pitfalls of subgroup analyses in the context of recent trials of antiplatelet agents.

Do Work Condition Interventions Affect Quality and Errors in Primary Care? Results from the Healthy Work Place Study

BACKGROUND

While primary care work conditions are associated with adverse clinician outcomes, little is known about the effect of work condition interventions on quality or safety.

DESIGN

A cluster randomized controlled trial of 34 clinics in the upper Midwest and New York City.

PARTICIPANTS

Primary care clinicians and their diabetic and hypertensive patients.

INTERVENTIONS

Quality improvement projects to improve communication between providers, workflow design, and chronic disease management. Intervention clinics received brief summaries of their clinician and patient outcome data at baseline.

MAIN MEASURES

We measured work conditions and clinician and patient outcomes both at baseline and 6-12 months post-intervention. Multilevel regression analyses assessed the impact of work condition changes on outcomes. Subgroup analyses assessed impact by intervention category.

KEY RESULTS

There were no significant differences in error reduction (19 % vs. 11 %, OR of improvement 1.84, 95 % CI 0.70, 4.82, p = 0.21) or quality of care improvement (19 % improved vs. 44 %, OR 0.62, 95 % CI 0.58, 1.21, p = 0.42) between intervention and control clinics. The conceptual model linking work conditions, provider outcomes, and error reduction showed significant relationships between work conditions and provider outcomes (p ≤ 0.001) and a trend toward a reduced error rate in providers with lower burnout (OR 1.44, 95 % CI 0.94, 2.23, p = 0.09).

LIMITATIONS

Few quality metrics, short time span, fewer clinicians recruited than anticipated.

CONCLUSIONS

Work-life interventions improving clinician satisfaction and well-being do not necessarily reduce errors or improve quality. Longer, more focused interventions may be needed to produce meaningful improvements in patient care.

CLINICAL TRIAL REGISTRATION NUMBER

ClinicalTrials.gov # NCT02542995.

Challenges and solutions to pre- and post-randomization subgroup analyses

Subgroup analyses are commonly performed in the clinical trial setting with the purpose of illustrating that the treatment effect was consistent across different patient characteristics or identifying characteristics that should be targeted for treatment. There are statistical issues involved in performing subgroup analyses, however. These have been given considerable attention in the literature for analyses where subgroups are defined by a pre-randomization feature. Although subgroup analyses are often performed with subgroups defined by a post-randomization feature—including analyses that estimate the treatment effect among compliers—discussion of these analyses has been neglected in the clinical literature. Such analyses pose a high risk of presenting biased descriptions of treatment effects. We summarize the challenges of doing all types of subgroup analyses described in the literature. In particular, we emphasize issues with post-randomization subgroup analyses. Finally, we provide guidelines on how to proceed across the spectrum of subgroup analyses.

Is regadenoson an appropriate stressor for MPI in patients with left bundle branch block or pacemakers?

BACKGROUND

Patients with LBBB or ventricular pacemaker undergoing MPI are at risk for false positive MPI results in the setting of an elevated heart rate (HR) with exercise or dobutamine stress. The areas of increased apparent ischemia are typically the LAD and septal territories.

METHODS

In a subanalysis of the ADVANCE MPI 1 and 2 studies, perfusion on an initial adenosine and a second MPI study with regadenoson or adenosine was compared by visual and quantitative analysis. Among 2,015 patients, 64 had LBBB and 93 had pacemakers. The hemodynamic response during the second scan was compared in those with and without LBBB and PM.

RESULTS

Following regadenoson, peak HR in the LBBB group increased by a mean of 25.4 compared to 15.3 bpm following adenosine (P = .0083). In the pacemaker group HR was blunted, 11.8 and 8.1 following regadenoson and adenosine, respectively (P = .1262). However, the visually assessed summed difference score and the quantitatively assessed extent of ischemia for the LAD and septal territories and the entire LV did not differ between the initial adenosine and subsequent regadenoson scans.

CONCLUSIONS

The significant increase in HR observed with regadenoson compared to adenosine did not translate into greater perfusion defects in the LAD or septal territories in patients undergoing regadenoson stress.

The influence of baseline marijuana use on treatment of cocaine dependence: application of an informative-priors bayesian approach

BACKGROUND

Marijuana use is prevalent among patients with cocaine dependence and often non-exclusionary in clinical trials of potential cocaine medications. The dual-focus of this study was to (1) examine the moderating effect of baseline marijuana use on response to treatment with levodopa/carbidopa for cocaine dependence; and (2) apply an informative-priors, Bayesian approach for estimating the probability of a subgroup-by-treatment interaction effect.

METHOD

A secondary data analysis of two previously published, double-blind, randomized controlled trials provided complete data for the historical (Study 1: N = 64 placebo), and current (Study 2: N = 113) data sets. Negative binomial regression evaluated Treatment Effectiveness Scores (TES) as a function of medication condition (levodopa/carbidopa, placebo), baseline marijuana use (days in past 30), and their interaction.

RESULTS

Bayesian analysis indicated that there was a 96% chance that baseline marijuana use predicts differential response to treatment with levodopa/carbidopa. Simple effects indicated that among participants receiving levodopa/carbidopa the probability that baseline marijuana confers harm in terms of reducing TES was 0.981; whereas the probability that marijuana confers harm within the placebo condition was 0.163. For every additional day of marijuana use reported at baseline, participants in the levodopa/carbidopa condition demonstrated a 5.4% decrease in TES; while participants in the placebo condition demonstrated a 4.9% increase in TES.

CONCLUSION

The potential moderating effect of marijuana on cocaine treatment response should be considered in future trial designs. Applying Bayesian subgroup analysis proved informative in characterizing this patient-treatment interaction effect.

Evaluation of heterogeneity in pharmacotherapy trials for drug dependence: a Bayesian approach

AIMS

Difficulty identifying effective pharmacotherapies for cocaine dependence has led to suggestions that subgroup differences may account for some of the heterogeneity in treatment response. Well-attested methodological difficulties associated with these analyses recommend the use of Bayesian statistical reasoning for evaluation of salient interaction effects.

METHODS

A secondary data analysis of a previously published, double-blind, randomized controlled trial examines the interaction of decision-making, as measured by the Iowa Gambling Task, and citalopram in increasing longest sustained abstinence from cocaine use.

RESULTS

Bayesian analysis indicated that there was a 99% chance that improved decision-making enhances response to citalopram. Given the strong positive nature of this finding, a formal, quantitative Bayesian approach to evaluate the result from the perspective of a skeptic was applied.

CONCLUSIONS

Bayesian statistical reasoning provides a formal means of weighing evidence for the presence of an interaction in scenarios where conventional, Frequentist analyses may be less informative. [Supplementary materials are available for this article. Go to the publisher's online edition of The American Journal of Drug and Alcohol Abuse for the following free supplemental resource: Appendix 1].

Subgroup analyses in therapeutic cardiovascular clinical trials: are most of them misleading?

BACKGROUND

Treatment decisions in clinical cardiology are directed by results from randomized clinical trials (RCTs). We studied the appropriateness of the use and interpretation of subgroup analysis in current therapeutic cardiovascular RCTs.

METHODS

We reviewed main reports of phase 3 cardiovascular RCTs with at least 100 patients, published in 2002 and 2004, and from major journals (Circulation, J Am Coll Cardiol, Am Heart J, Am J Cardiol, N Engl J Med, Lancet, JAMA, BMJ, Ann Intern Med). Information on subgroups included prespecification, number, interaction test use, significant subgroups found, and emphasis on findings. We examined appropriateness of reporting and differences according to sample size, overall trial result, and CONSORT adoption.

RESULTS

We selected 63 RCTs, with a median of 496 (range 100-15,245) patients. Thirty-nine RCTs were reported with subgroup analyses and 26 with > 5 subgroups. No trial was specifically powered to detect subgroup effects, and only 14 RCTs were reported with fully prespecified subgroups. Only 11 RCTs were reported with interaction tests. Furthermore, 21 RCTs were reported with claims of significant subgroups and 15 with equal or more emphasis to subgroups than to the overall results. Subgroup analyses in large RCTs (> 500 patients) were reported more often than in small ones (24/30 vs 15/33, P = .005). No differences were found according to overall result (positive/negative) or CONSORT adoption.

CONCLUSIONS

Subgroup analyses in recent cardiovascular RCTs were reported with several shortcomings, including a lack of prespecification and testing of a large number of subgroups without the use of the statistically appropriate test for interaction. Reporting of subgroup analysis needs to be substantially improved because emphasis on these secondary results may mislead treatment decisions.

Effect of a high-intensity weight-bearing exercise program on radiologic damage progression of the large joints in subgroups of patients with rheumatoid arthritis

OBJECTIVE

To investigate whether a high-intensity exercise program accelerates the rate of radiologic damage of the large joints in predefined subgroups of patients with rheumatoid arthritis.

METHODS

The data of 277 participants in a 2-year randomized controlled trial, comparing the effects of high-intensity exercises with usual care, were used. Linear regression analysis was used to test which predefined variables at baseline (age, disease duration, disease activity, physical capacity, functional ability, joint damage) modified the effect of high-intensity exercise on the progression of radiologic damage of the large joints over 24 months.

RESULTS

Baseline radiologic joint damage was the only variable associated with the effect of high-intensity exercise on joint damage progression in large joints. In a subgroup of 218 patients with no or little joint damage (defined as Larsen score < or = 5; 80% of our study population) the proportions of patients with an increase in joint damage were similar for the exercise and usual-care group (35% versus 36%, risk ratio [RR] 1.0 [0.7-1.4]; P = not significant), whereas, in a subgroup of 59 patients who already had extensive damage of large joints (defined as Larsen score >5) the proportion was significantly higher in the exercise group (85% versus 48%, RR 1.8 [1.2-2.6]; P < 0.05).

CONCLUSION

High-intensity weight-bearing exercises appear to accelerate joint damage progression in patients with preexisting extensive damage. Patients with extensive large joint damage should, therefore, be advised to refrain from activities excessively loading the damaged joints.

Differential Treatment Benefit of Platelet Glycoprotein IIb/IIIa Inhibition With Percutaneous Coronary Intervention Versus Medical Therapy for Acute Coronary Syndromes

Background— Although many believe that platelet glycoprotein IIb/IIIa inhibitors should be used only in acute coronary syndrome patients undergoing percutaneous coronary intervention, supporting data from randomized clinical trials are tenuous. The assumption that these agents are useful only in conjunction with percutaneous coronary intervention is based primarily on inappropriate subgroup analyses performed across the glycoprotein IIb/IIIa inhibitor trials.

Methods and Results— We describe the problems with these analytical techniques and demonstrate that different approaches to the question can result in opposing answers.

Conclusions— Clinical-practice decisions and practice guidelines should be based on overall trial results and not analyses of post-randomization subgroups.

A rapid B-type natriuretic peptide assay accurately diagnoses left ventricular dysfunction and heart failure: a multicenter evaluation

BACKGROUND

B-Type natriuretic peptide (BNP), a protein released from the left ventricle in response to volume expansion and pressure overload, has emerged as the first whole blood marker for the identification of individuals with congestive heart failure (CHF).

OBJECTIVE

The purpose of this study was to assess the performance of a point-of-care assay to diagnose and evaluate the severity of CHF on the basis of the New York Heart Association (NYHA) classification system.

METHODS

Through a prospective, multicenter trial, whole blood samples were collected from a total of 1050 inpatients, outpatients, and healthy control patients. Participants were divided into subgroups for BNP analysis: patients without cardiovascular CHF (n = 473), patients with hypertension and no cardiovascular disease (n = 168), NYHA class I CHF (n = 73), class II CHF (n = 135), class III CHF (n = 141), and class IV CHF (n = 60).

RESULTS

Circulating BNP concentrations determined from the bedside assay increased with CHF severity, as determined by the NYHA classification system, but were only statistically significant (P <.001) between individuals with and without CHF. Individuals without CHF had a median BNP concentration of 9.29 pg/mL. Median BNP values, with their corresponding interquartile ranges, for NYHA classification I through IV were 83.1 pg/mL (49.4-137 pg/mL), 235 pg/mL (137-391 pg/mL), 459 pg/mL (200-871 pg/mL), and 1119 pg/mL (728->1300 pg/mL), respectively. With the use of a decision threshold of 100 pg/mL, the assay demonstrated 82% sensitivity and 99% specificity for distinguishing control patients and patients with CHF.

CONCLUSIONS

BNP concentrations obtained from whole blood samples are useful in the diagnosis of CHF and staging the severity of the disease.

Antiresorptive treatment of postmenopausal osteoporosis: comparison of study designs and outcomes in large clinical trials with fracture as an endpoint

Antiresorptive treatments for postmenopausal osteoporosis have been studied extensively, but due to the volume of published data and lack of head-to-head trials, it is difficult to evaluate and compare their fracture reduction efficacy. The objective of this review is to summarize the results from clinical trials that have fracture as an endpoint and to discuss the factors in study design and populations that can affect the interpretation of the results. Although there are numerous observational studies suggesting that estrogen and hormone replacement therapies may reduce the risk of vertebral and nonvertebral fractures, there is no large, prospective, randomized, placebo-controlled, double-blind clinical trial demonstrating fracture efficacy. The effects of raloxifene, alendronate, risedronate, and salmon calcitonin on increasing bone mineral density (BMD) and decreasing fracture risk have been shown in randomized, placebo-controlled, double-blind clinical trials of postmenopausal women with osteoporosis. Although the increases in lumbar spine BMD vary greatly in these trials, the decrease in relative risk of vertebral fractures is similar among therapies. However, nonvertebral fracture efficacy has not been consistently demonstrated. Combined administration of two antiresorptive therapies results in greater BMD increases, but the effects on fracture risk are unknown. Direct comparisons of clinical trial results should be considered carefully, given the differences in study design and populations. Differences in study design that may influence the efficacy of fracture risk reduction include calcium and vitamin D supplementation, primary fracture endpoints, definition of vertebral deformity or fracture, discontinuation rates, and statistical power. Factors in the study population that may influence fracture efficacy include the age of the population and the proportion of subjects with prevalent fractures. The use of surrogate endpoints such as BMD to predict fracture risk should be approached with caution, as the relationship between BMD changes and fracture risk reduction with antiresorptive therapies is uncertain. Consideration of these results from clinical trials can contribute to clinical judgment in selecting the best treatment option for postmenopausal osteoporosis.

B-type natriuretic peptide in the diagnosis and management of congestive heart failure

Finding a simple blood test that would aid in the diagnosis and management of patients with CHF clearly would have a favorable impact on the staggering costs associated with the disease. BNP, which is synthesized in the cardiac ventricles and correlates with LV pressure, amount of dyspnea, and the state of neurohormonal modulation, makes this peptide the first potential "white count" for heart failure. The fact that a point-of-care rapid assay for BNP has been approved by the FDA gives the clinician an opportunity to explore its potential usefulness. The author's data, and data from others, suggest that serial point-of-care testing of BNP will be of immense help in patients presenting to urgent care clinics with dyspnea. Additionally, BNP might serve as a screen for patients referred for echocardiography. A low BNP level makes echocardiographic indices of LV dysfunction (systolic and diastolic) highly unlikely. BNP also might be an effective way to improve the in-hospital management of patients admitted with decompensated CHF. In some instances, BNP levels may obviate the need for invasive hemodynamic monitoring and, in cases where such monitoring is used, may help tailor treatment of the decompensated patient. Finally, the role of BNP in the outpatient cardiac or primary care clinic may be one of critical importance in titration of therapies and in assessment of the state of neurohormonal compensation of the patient.

Practical approaches to treating patients with acute decompensated heart failure

Finding a simple blood test to aid in the diagnosis and treatment of patients with congestive heart failure would have a favorable impact on the costs associated with the disease. B-type natriuretic peptide (BNP) is synthesized in the cardiac ventricles and correlates with left ventricular pressure, amount of dyspnea, and the state of neurohormonal modulation, making this peptide the first potential "white count" for heart failure. Data indicate that serial point-of-care testing of BNP should be helpful in patients presenting to urgent care clinics with dyspnea. BNP might serve as a screen for patients referred for echocardiography. A low BNP level makes echocardiographic indices of left ventricular dysfunction (both systolic and diastolic) highly unlikely. BNP may also be effective in improving in-hospital management of patients admitted with decompensated congestive heart failure. In some cases BNP levels may obviate the need for invasive hemodynamic monitoring and, when such monitoring is used, may help tailor treatment of decompensated patients. Finally, the role of BNP in outpatient cardiac or primary care clinics may be important in the titration of therapies and assessment of the state of neurohormonal compensation of patients.

A rapid test for B-type natriuretic peptide correlates with falling wedge pressures in patients treated for decompensated heart failure: a pilot study

OBJECTIVES

To determine if changes in B-type natriuretic peptide (BNP) levels can accurately reflect acute changes in pulmonary capillary wedge pressure during treatment of decompensated heart failure.

BACKGROUND

Tailored therapy of decompensated congestive heart failure with hemodynamic monitoring is controversial. Other than the expense and complications of Swan-Ganz catheters, its use in titration of drug therapy has no conclusive end point. Because BNP reflects both elevated left ventricular pressure and neurohormonal modulation and has a short half-life, we hypothesized that levels of BNP would decline in association with falling wedge pressures. Final BNP levels would perhaps signify a new set point of neuromodulation.

METHODS AND RESULTS

Twenty patients with decompensated New York Heart Association (NYHA) class III-IV congestive heart failure (CHF) undergoing tailored therapy were studied. BNP levels were drawn every 2 to 4 hours for the first 24 hours (active treatment phase) and then every 4 hours for the next 24 to 48 hours (stabilization period). Hemodynamic data was recorded simultaneously. In 15 patients whose wedge pressure responded to treatment in the first 24 hours, there was a significant drop in BNP levels (55%) versus nonresponders (8%). There was a significant correlation between percent change in wedge pressure from baseline per hour and the percent change of BNP from baseline per hour (r = 0.79, P <.05). When the wedge pressure was kept at a stable, low level during the stabilization phase, BNP levels continued to fall another 37% (937 +/- 140 pg/mL at 24 hours to 605 +/- 128 pg/mL). Patients who died (n = 4) had higher final BNP levels (1,078 +/- 123 pg/mL v 701 +/- 107 pg/mL).

CONCLUSIONS

The data suggest that rapid testing of BNP may be an effective way to improve the in-hospital management of patients admitted with decompensated CHF. Although BNP levels will not obviate the need for invasive hemodynamic monitoring, it may be a useful adjunct in tailoring therapy to these patients.

A rapid bedside test for B-type peptide predicts treatment outcomes in patients admitted for decompensated heart failure: a pilot study

OBJECTIVES

The goal of this study was to determine if B-type natriuretic peptide (BNP) levels predict outcomes of patients admitted with decompensated heart failure.

BACKGROUND

Treatment of decompensated congestive heart failure (CHF) has often been based on titration of drugs to relieve patient's symptoms, a case that could be made for attempting to also treat neurohormonal abnormalities. Because BNP reflects both elevated left ventricular pressure as well as neurohormonal modulation, we hypothesized that BNP might be useful in assessing outcomes in patients admitted with decompensated CHF.

METHODS

We followed 72 patients admitted with decompensated New York Heart Association class III to IV CHF, measuring daily BNP levels. We then determined the association between initial BNP measurement and the predischarge or premoribund BNP measurement and subsequent adverse outcomes (death and 30-day readmission).

RESULTS

Of the 72 patients admitted with decompensated CHF, 22 end points occurred (death: n = 13, readmission: n = 9). In these patients, BNP levels increased during hospitalization (mean increase, 233 pg/ml, p < 0.001). In patients without end points, BNP decreased (mean decrease 215 pg/ml). Univariate analysis revealed that the last measured BNP was strongly associated with the combined end point. In patients surviving hospitalization, BNP discharge concentrations were strong predictors of subsequent readmission (area under the receiver operator curve of 0.73).

CONCLUSIONS

In patients admitted with decompensated CHF, changes in BNP levels during treatment are strong predictors for mortality and early readmission. The results suggest that BNP levels might be used successfully to guide treatment of patients admitted for decompensated CHF.