An audit strategy for progression-free survival

Value of central review of RECIST v1.1 outcomes in the AGITG INTEGRATE randomised phase 2 international trial for advanced oesophago-gastric cancer

PURPOSE

Activity estimates should be accurately evaluated in phase 2 clinical trials to ensure appropriate decisions about proceeding to phase 3 trials. RECIST v1.1. progression-free survival (PFS) is a common endpoint in oncology; however, it can be influenced by assessment criteria and trial design. We assessed the value of central adjudication of investigator-assessed PFS times of participants in a double-blind, randomised phase 2 trial evaluating regorafenib versus placebo in advanced gastro-oesophageal cancer (AGITG INTEGRATE) to inform plans for central review in future trials.

METHODS

We calculated the proportion of participants with a disagreement between the site investigator assessment and blinded independent central review and in whom central review resulted in a change, then evaluated the effect of central review on study conclusions by comparing hazard ratios (HRs) for PFS based on site review versus central review. Post-progression unblinding was assessed with similar methods. Simulation studies explored the effect of differential and non-differential measurement error on treatment effect estimation and study power.

RESULTS

Disagreements between site assessments versus central review occurred in 8/147 (5.4%) participants, 5 resulting in amended date of progression (3.4%). PFS HRs (sites vs central review progression dates) were similar (0.39 vs 0.40). RECIST progression occurred in 82/86 (95%) of cases where post-progression unblinding was requested by the site investigator.

CONCLUSIONS

Blinded independent central review was feasible and supported the reliability of site assessments, trial results, and conclusions. Modelling showed that when treatment effects were large and outcome assessments blinded, central review was unlikely to affect conclusions.

Errors in multiple variables in human immunodeficiency virus (HIV) cohort and electronic health record data: statistical challenges and opportunities

Objectives: Observational data derived from patient electronic health records (EHR) data are increasingly used for human immunodeficiency virus/acquired immunodeficiency syndrome (HIV/AIDS) research. There are challenges to using these data, in particular with regards to data quality; some are recognized, some unrecognized, and some recognized but ignored. There are great opportunities for the statistical community to improve inference by incorporating validation subsampling into analyses of EHR data.Methods: Methods to address measurement error, misclassification, and missing data are relevant, as are sampling designs such as two-phase sampling. However, many of the existing statistical methods for measurement error, for example, only address relatively simple settings, whereas the errors seen in these datasets span multiple variables (both predictors and outcomes), are correlated, and even affect who is included in the study.Results/Conclusion: We will discuss some preliminary methods in this area with a particular focus on time-to-event outcomes and outline areas of future research.

Estimation of discrete survival function for error-prone diagnostic tests

The product limit or Kaplan-Meier (KM) estimator is commonly used to estimate the survival function in the presence of incomplete time to event. Application of this method assumes inherently that the occurrence of an event is known with certainty. However, the clinical diagnosis of an event is often subject to misclassification due to assay error or adjudication error, by which the event is assessed with some uncertainty. In the presence of such errors, the true distribution of the time to first event would not be estimated accurately using the KM method. We develop a method to estimate the true survival distribution by incorporating negative predictive values and positive predictive values, into a KM-like method of estimation. This allows us to quantify the bias in the KM survival estimates due to the presence of misclassified events in the observed data. We present an unbiased estimator of the true survival function and its variance. Asymptotic properties of the proposed estimators are provided, and these properties are examined through simulations. We demonstrate our methods using data from the Viral Resistance to Antiviral Therapy of Hepatitis C study.

Use of administrative data to increase the practicality of clinical trials: Insights from the Women's Health Initiative

BACKGROUND

To reduce research costs in the context of pragmatic trials, consideration is given to using administrative data (Medicare claims) to ascertain clinical outcomes.

METHODS

In the historical context of the Women's Health Initiative, the correspondence between selected cardiovascular events derived from Medicare claims was compared to those documented and adjudicated in this large-scale prevention trial.

RESULTS

Classification performance varies somewhat by type of outcome, but hazard ratios and confidence intervals derived from the two data sources were quite comparable.

CONCLUSION

These encouraging results provided the needed support to launch a new embedded pragmatic trial of physical activity that will rely heavily on Medicare claims to ascertain cardiovascular disease incidence in the majority of those randomized.

Response assessment in lymphoma: Concordance between independent central review and local evaluation in a clinical trial setting

BACKGROUND

Independent central review of clinical imaging remains the standard for oncology clinical trials with registration potential. A limited independent central review strategy has been proposed for solid tumor trials based on concordance between central and local evaluation of response. Concordance between independent central review and local evaluation of response in hematological malignancies is not known.

METHODS

We retrospectively evaluated concordance between prospectively performed central and local assessments of response using the Revised Response Criteria for Malignant Lymphoma across two international, open-label, single-arm, registration studies of brentuximab vedotin in patients with relapsed or refractory Hodgkin lymphoma (N = 102) or systemic anaplastic large-cell lymphoma (N = 58).

RESULTS

Overall objective response rates were similar between assessors for both the trial in Hodgkin lymphoma (75% independent central review, 72% local evaluation) and the trial in anaplastic large-cell lymphoma (86% independent central review, 83% local evaluation). Patient-specific objective response concordance was also substantial (Hodgkin lymphoma: kappa = 0.68; anaplastic large-cell lymphoma: kappa = 0.74). Median progression-free survival was similar between assessors for patients with anaplastic large-cell lymphoma (14.3 months by independent central review (95% confidence interval: 6.9, -); 14.5 months by local evaluation (95% confidence interval: 9.4, -)), but longer by local evaluation in patients with Hodgkin lymphoma (5.8 months by independent central review (95% confidence interval: 5.0, 9.0); 9.0 months by local evaluation (95% confidence interval: 7.1, 12.0)). Median duration of response was longer by local evaluation in both malignancies, which was primarily attributable to earlier computed tomography and positron emission tomography-based scoring of progression by independent central review.

CONCLUSION

A limited independent review audit strategy for clinical trials of some lymphomas appears feasible and practical based on substantial concordance in assessments of overall objective response by central and local evaluation in two international, prospective, registration trials in lymphoma. Some variability between assessors in the time-to-event endpoints was observed, which appeared attributable to earlier assignments of progression by independent central review compared with local evaluation.

Model free audit methodology for bias evaluation of tumour progression in oncology

Many oncology studies incorporate a blinded independent central review (BICR) to make an assessment of the integrity of the primary endpoint, progression free survival. Recently, it has been suggested that, in order to assess the potential for bias amongst investigators, a BICR amongst only a sample of patients could be performed; if evidence of bias is detected, according to a predefined threshold, the BICR is then assessed in all patients, otherwise, it is concluded that the sample was sufficient to rule out meaningful levels of bias. In this paper, we present an approach that adapts a method originally created for defining futility bounds in group sequential designs. The hazard ratio ratio, the ratio of the hazard ratio (HR) for the treatment effect estimated from the BICR to the corresponding HR for the investigator assessments, is used as the metric to define bias. The approach is simple to implement and ensures a high probability that a substantial true bias will be detected. In the absence of bias, there is a high probability of accepting the accuracy of local evaluations based on the sample, in which case an expensive BICR of all patients is avoided. The properties of the approach are demonstrated by retrospective application to a completed Phase III trial in colorectal cancer. The same approach could easily be adapted for other disease settings, and for test statistics other than the hazard ratio.

Efficacy and Safety of Trabectedin or Dacarbazine for Metastatic Liposarcoma or Leiomyosarcoma After Failure of Conventional Chemotherapy: Results of a Phase III Randomized Multicenter Clinical Trial

PURPOSE

This multicenter study, to our knowledge, is the first phase III trial to compare trabectedin versus dacarbazine in patients with advanced liposarcoma or leiomyosarcoma after prior therapy with an anthracycline and at least one additional systemic regimen.

PATIENTS AND METHODS

Patients were randomly assigned in a 2:1 ratio to receive trabectedin or dacarbazine intravenously every 3 weeks. The primary end point was overall survival (OS), secondary end points were disease control-progression-free survival (PFS), time to progression, objective response rate, and duration of response-as well as safety and patient-reported symptom scoring.

RESULTS

A total of 518 patients were enrolled and randomly assigned to either trabectedin (n = 345) or dacarbazine (n = 173). In the final analysis of PFS, trabectedin administration resulted in a 45% reduction in the risk of disease progression or death compared with dacarbazine (median PFS for trabectedin v dacarbazine, 4.2 v 1.5 months; hazard ratio, 0.55; P < .001); benefits were observed across all preplanned subgroup analyses. The interim analysis of OS (64% censored) demonstrated a 13% reduction in risk of death in the trabectedin arm compared with dacarbazine (median OS for trabectedin v dacarbazine, 12.4 v 12.9 months; hazard ratio, 0.87; P = .37). The safety profiles were consistent with the well-characterized toxicities of both agents, and the most common grade 3 to 4 adverse effects were myelosuppression and transient elevation of transaminases in the trabectedin arm.

CONCLUSION

Trabectedin demonstrates superior disease control versus conventional dacarbazine in patients who have advanced liposarcoma and leiomyosarcoma after they experience failure of prior chemotherapy. Because disease control in advanced sarcomas is a clinically relevant end point, this study supports the activity of trabectedin for patients with these malignancies.

Parameter estimation in Cox models with missing failure indicators and the OPPERA study

In a prospective cohort study, examining all participants for incidence of the condition of interest may be prohibitively expensive. For example, the "gold standard" for diagnosing temporomandibular disorder (TMD) is a physical examination by a trained clinician. In large studies, examining all participants in this manner is infeasible. Instead, it is common to use questionnaires to screen for incidence of TMD and perform the "gold standard" examination only on participants who screen positively. Unfortunately, some participants may leave the study before receiving the "gold standard" examination. Within the framework of survival analysis, this results in missing failure indicators. Motivated by the Orofacial Pain: Prospective Evaluation and Risk Assessment (OPPERA) study, a large cohort study of TMD, we propose a method for parameter estimation in survival models with missing failure indicators. We estimate the probability of being an incident case for those lacking a "gold standard" examination using logistic regression. These estimated probabilities are used to generate multiple imputations of case status for each missing examination that are combined with observed data in appropriate regression models. The variance introduced by the procedure is estimated using multiple imputation. The method can be used to estimate both regression coefficients in Cox proportional hazard models as well as incidence rates using Poisson regression. We simulate data with missing failure indicators and show that our method performs as well as or better than competing methods. Finally, we apply the proposed method to data from the OPPERA study.

Assessing temporal agreement between central and local progression-free survival times

In oncology clinical trials, progression-free survival (PFS), generally defined as the time from randomization until disease progression or death, has been a key endpoint to support licensing approval. In the U.S. Food and Drug Administration guidance for industry, May 2007, concerning the PFS as the primary or co-primary clinical trial endpoint, it is recommended to have tumor assessments verified by an independent review committee blinded to study treatments, especially in open-label studies. It is considered reassuring about the lack of reader-evaluation bias if treatment effect estimates from the investigators' and independent review committees' evaluations agree. The agreement between these evaluations may vary for subjects with short or long PFS, while there exist no such statistical quantities that can completely account for this temporal pattern of agreements. Therefore, in this paper, we propose a new method to assess temporal agreement between two time-to-event endpoints, while the two event times are assumed to have a positive probability of being identical. This method measures agreement in terms of the two event times being identical at a given time or both being greater than a given time. Overall scores of agreement over a period of time are also proposed. We propose a maximum likelihood estimation to infer the proposed agreement measures using empirical data, accounting for different censoring mechanisms, including reader's censoring (event from one reader dependently censored by event from the other reader). The proposed method is demonstrated to perform well in small samples via extensive simulation studies and is illustrated through a head and neck cancer trial.

Volume-based response evaluation with consensual lesion selection: a pilot study by using cloud solutions and comparison to RECIST 1.1

RATIONALE AND OBJECTIVES

Lesion volume is considered as a promising alternative to Response Evaluation Criteria in Solid Tumors (RECIST) to make tumor measurements more accurate and consistent, which would enable an earlier detection of temporal changes. In this article, we report the results of a pilot study aiming at evaluating the effects of a consensual lesion selection on volume-based response (VBR) assessments.

MATERIALS AND METHODS

Eleven patients with lung computed tomography scans acquired at three time points were selected from Reference Image Database to Evaluate Response to therapy in lung cancer (RIDER) and proprietary databases. Images were analyzed according to RECIST 1.1 and VBR criteria by three readers working in different geographic locations. Cloud solutions were used to connect readers and carry out a consensus process on the selection of lesions used for computing response. Because there are not currently accepted thresholds for computing VBR, we have applied a set of thresholds based on measurement variability (-35% and +55%). The benefit of this consensus was measured in terms of multiobserver agreement by using Fleiss kappa (κfleiss) and corresponding standard errors (SE).

RESULTS

VBR after consensual selection of target lesions allowed to obtain κfleiss = 0.85 (SE = 0.091), which increases up to 0.95 (SE = 0.092), if an extra consensus on new lesions is added. As a reference, the agreement when applying RECIST without consensus was κfleiss = 0.72 (SE = 0.088). These differences were found to be statistically significant according to a z-test.

CONCLUSIONS

An agreement on the selection of lesions allows reducing the inter-reader variability when computing VBR. Cloud solutions showed to be an interesting and feasible strategy for standardizing response evaluations, reducing variability, and increasing consistency of results in multicenter clinical trials.

Independent radiologic review: bevacizumab in combination with gemcitabine and carboplatin in recurrent ovarian cancer

OBJECTIVE

OCEANS, a randomized, placebo-controlled, phase III trial, found that adding bevacizumab to gemcitabine-carboplatin (GC) significantly improved investigator-determined progression-free survival (PFS) and objective response rate (ORR) in platinum-sensitive, recurrent ovarian cancer. To evaluate the reliability of assessment of progression and objective response per RECIST, radiologic and clinical data were assessed by an independent review committee (IRC).

METHODS

Radiologic images and clinical data were provided prospectively to the IRC for all randomized patients (N=484). Data were reviewed in a blinded fashion per RECIST (modified v1.0). PFS and ORR were analyzed based on the IRC assessment. Concordance between investigator- and IRC-assessed progression and objective response was assessed.

RESULTS

The IRC analysis demonstrated a statistically significant increase in PFS (hazard ratio [HR]=0.451; 95% confidence interval [CI]=0.351 to 0.580, p<0.0001) consistent with the benefit reported by investigators (HR=0.484; 95% CI=0.388 to 0.605, p<0.0001). The concordance rate, defined by agreement on progression status, was 74.2% overall, and comparable between treatment arms (bevacizumab, 75.2% vs. placebo, 73.1%). IRC-assessed ORR was significantly improved with bevacizumab (bevacizumab, 74.8% vs. placebo, 53.7%; p<0.0001), consistent with the investigator-assessed results. The concordance rate for objective response was 79.8% overall, and comparable between treatment arms (bevacizumab, 78.9% vs. placebo, 80.6%).

CONCLUSIONS

IRC-determined results were highly consistent with those determined by investigators, demonstrating that bevacizumab plus GC provides a significant improvement in PFS and ORR. These results suggest that investigators can reliably assess disease progression and objective response in recurrent ovarian cancer using RECIST, without the necessity of a full IRC review.

An audit strategy for time-to-event outcomes measured with error: application to five randomized controlled trials in oncology

BACKGROUND

Measurement error in time-to-event end points complicates interpretation of treatment effects in clinical trials. Non-differential measurement error is unlikely to produce large bias [1]. When error depends on treatment arm, bias is of greater concern. Blinded-independent central review (BICR) of all images from a trial is commonly undertaken to mitigate differential measurement-error bias that may be present in hazard ratios (HRs) based on local evaluations. Similar BICR and local evaluation HRs may provide reassurance about the treatment effect, but BICR adds considerable time and expense to trials.

METHODS

We describe a BICR audit strategy [2] and apply it to five randomized controlled trials to evaluate its use and to provide practical guidelines. The strategy requires BICR on a subset of study subjects, rather than a complete-case BICR, and makes use of an auxiliary-variable estimator.

RESULTS

When the effect size is relatively large, the method provides a substantial reduction in the size of the BICRs. In a trial with 722 participants and a HR of 0.48, an average audit of 28% of the data was needed and always confirmed the treatment effect as assessed by local evaluations. More moderate effect sizes and/or smaller trial sizes required larger proportions of audited images, ranging from 57% to 100% for HRs ranging from 0.55 to 0.77 and sample sizes between 209 and 737.

LIMITATIONS

The method is developed for a simple random sample of study subjects. In studies with low event rates, more efficient estimation may result from sampling individuals with events at a higher rate.

CONCLUSION

The proposed strategy can greatly decrease the costs and time associated with BICR, by reducing the number of images undergoing review. The savings will depend on the underlying treatment effect and trial size, with larger treatment effects and larger trials requiring smaller proportions of audited data.

Independent radiologic review in metastatic colorectal cancer: systematic review and meta-analysis

PURPOSE

To perform a meta-analysis addressing evaluation bias in local radiologic assessment (LRA) of lesions when compared with independent radiologic review (IRR) in randomized controlled trials (RCTs) testing chemotherapy for metastatic colorectal cancer (CRC).

MATERIALS AND METHODS

MEDLINE, EMBASE, ClinicalTrials.gov, the Cochrane Library, and Web sites for major medical meetings were searched for RCTs of chemotherapy for metastatic CRC that reported response evaluation by both LRA and IRR. The risk ratios (RRs) of response in the experimental (RR(exp)) and control (RR(cont)) arms were calculated (response rate in LRA divided by response rate in IRR) for each selected study. The ratio of RR of response was calculated (RR of response of LRA divided by RR of response of IRR). The random-effects model was applied. Meta-regression was used to examine the effect of study characteristics on outcomes.

RESULTS

LRA and IRR results were concordant (13 studies; 7742 patients; ratio of RR of response = 0.97; 95% confidence interval [95% CI]: 0.90, 1.04; P = .35). However, LRA overestimated tumor response independently of therapy allocation (interaction test, P = .81) both in control (RR(cont), 1.163; 95% CI: 1.086, 1.246; P < .001) and experimental (RR(exp), 1.156; 95% CI: 1.093, 1.222; P < .001) therapies. Meta-regression did not show any effect of trial characteristics on effects.

CONCLUSION

LRA yields higher response rates in RCTs testing chemotherapy for metastatic CRC, although there was no sign of bias toward experimental therapy. The need for IRR to control evaluation bias must be reappraised.

The European Medicines Agency: an overview of its mission, responsibilities, and recent initiatives in cancer drug regulation

The European Medicines Agency (EMA) is responsible for the scientific evaluation of medicines developed by pharmaceutical companies for use in the European Union (EU). Since 2005, the agency has become responsible for the approval of all new oncology drugs in the EU. In this article we describe the mission, role, and responsibilities of the EMA, and provide a brief summary of recent initiatives related to cancer drug regulation. The EMA recently published its Road Map to 2015. Over the next 5 years, the agency aims to continue to stimulate drug development in areas of unmet medical needs. Concerning drug safety, one of the priorities over the next few years will be to establish a more proactive approach in ensuring patient safety. This is the result of new EU legislation coming into force in 2012 that will strengthen the way the safety of medicines for human use is monitored in the EU. In terms of its general operation, the agency is committed to increased openness and transparency, and to build on its interactions with stakeholders, including members of academia, health care professionals, patients, and health technology assessment bodies. The agency recently created an oncology working party to expand the current guideline for the development and evaluation of cancer drugs. The guideline focuses on both exploratory and confirmatory studies for different types of agents. The current revision will address a number of topics, including the use of biomarkers as an integrated part of drug development and the use of progression-free survival as a primary endpoint in registration trials.