Considerations for development of an evidence dossier to support the use of mobile sensor technology for clinical outcome assessments in clinical trials

BACKGROUND

Mobile sensors offer enormous potential for the collection of informative clinical endpoints in clinical trials to support regulatory decision making and product labelling. There are currently no specific guidelines on the information needed to enable regulators to review and accept proposed endpoints derived from mobile sensors for use in drug development trials.

OBJECTIVE

The purpose of this working group report is to recommend the structure and content of an evidence dossier intended to support whether a clinical endpoint derived from mobile sensor data is fit-for-purpose for use in regulatory submissions for drug approvals.

EVIDENCE DOSSIER

The structure and content of a dossier to provide evidence supporting the use of a sensor-derived clinical endpoint is described. Sections include clinical endpoint definition and positioning, the concept of interest, the context of use, clinical validation and interpretation, study implementation, and analytical validity with sensor performance verification in support of the selected sensor.

CONCLUSIONS

In the absence of definitive regulatory guidance, this report provides a considered approach to compiling a comprehensive body of evidence to justify acceptance of mobile sensors for support of new drug applications.

Low clinical conversion rate in clinically isolated syndrome patients - diagnostic benefit of McDonald 2010 criteria?

BACKGROUND AND PURPOSE

New diagnostic criteria of multiple sclerosis (MS) increase the number of patients being diagnosed with MS whilst a substantial part might not convert to clinically definite MS (CDMS). The diagnostic accuracy of the McDonald 2005 and 2010 criteria for conversion to CDMS was evaluated in an unselected cohort of patients in whom an MS diagnostic work-up was decided.

METHODS

Clinical, magnetic resonance imaging and cerebrospinal fluid data were analysed for all patients who presented with symptoms suspicious for MS at the university based MS outpatient clinic between 2006 and 2010 (n = 165).

RESULTS

Follow-up was available for 131 patients. During the mean follow-up period of 2 years, 19% of patients developed CDMS whereas 64% of the patients fulfilling McDonald 2010 criteria did not convert to CDMS.

CONCLUSION

The low clinical conversion rate indicates that the new diagnostic criteria may increase the incidence of MS cases with a less active disease course.

The relationship of age with the clinical phenotype in multiple sclerosis

Background:

The multiple sclerosis (MS) clinical course and relapses frequency before progression vary widely.

Objective:

To investigate the influence of age on the MS phenotype.

Methods:

Among 751 primary progressive (PP = 217) and secondary progressive (SP = 534) MS patients from the London Ontario database, we assessed the relationship of age on the relapse frequency and on the progressive phase evolution, and the impact of relapses on the age at onset of progression.

Results:

Age at onset did not influence the early attacks frequency, but patients younger at onset had larger number of total attacks before progression (age = 27.4, 31.0 and 32.8 mean years; ⩾4, 2–3 and 1 relapses, respectively) and longer latency to SP. Although frequent early relapses predicted younger age at SP onset, patients with no attacks (primary progressive multiple sclerosis (PPMS)), or 1, 2–3 and ⩾4 relapses during the relapsing-remitting phase started progressing at similar age (38.6, 41.3, 41.4 and 39.2 mean years, respectively). The age at onset of progressive phase did not affect its evolution.

Conclusions:

Age strongly influences the phenotype before progression. Relapsing-remitting patients younger at onset are more likely to display a predominantly inflammatory course, yet relapses number does not affect the age at onset of progression.

Survival, and time to an advanced disease state or progression, of untreated patients with moderately severe multiple sclerosis in a multicenter observational database: relevance for design of a clinical trial for high dose immunosuppressive therapy with autologous hematopoietic stem cell transplantation

Despite prolonged survival, patients with multiple sclerosis (MS) experience considerable morbidity, which adversely impacts quality of life. To assess the risk-benefit of a clinical trial of high dose immunosuppressive therapy with autologous hematopoietic stem cell transplantation for MS, we sought to determine the natural history of the disease in a comparison group of untreated patients. We identified 285 individuals with 2132 combined observation years (median: 5.6 years; 5th to 95th percentile: 1-21 years), with Expanded Disability Status Scale (EDSS) scores of 3.0-5.5 at baseline observation. Disease-related mortality was zero at five years, 5.4% at 10 years, and 22% at 15 years (40 patients contributing to the data point; 95% confidence interval: 4-32%). Risk for progression to advanced disability, defined as an EDSS score of 8, was very low for the subgroup with a baseline EDSS score of 3-3.5; however, for those with a baseline EDSS score of 4-5.5, 3% had advanced disability after two years, 5% after three years, 6% after four years, 12% after five years, and 40% after 10 years. The estimated probability of disease progression, defined as an increase in EDSS score by ≥ 1.0 sustained for at least 180 days, was 5% after one year, 14% after two years, 22% after three years, 38% after five years, 57% after 10 years, and-80% after 20 years of observation. The relevance of these features to the design of the clinical trial is discussed.

A web-based tool for personalized prediction of long-term disease course in patients with multiple sclerosis

BACKGROUND AND PURPOSE

The Evidence-Based Decision Support Tool in Multiple Sclerosis (EBDiMS) is the first web-based prognostic calculator in multiple sclerosis (MS) capable of delivering individualized estimates of disease progression. It has recently been extended to provide long-term predictions based on the data from a large natural history cohort.

METHODS

We compared the predictive accuracy and consistency of EBDiMS with that of 17 neurologists highly specialized in MS.

RESULTS

We show that whilst the predictive accuracy was similar, neurologists showed a significant intra-rater and inter-rater variability.

CONCLUSIONS

Because EBDiMS was consistent, it is of superior utility in a specialist setting. Further field testing of EBDiMS in non-specialist settings, and investigation of its usefulness for counselling patients in treatment decisions, is warranted.

Age and disability accumulation in multiple sclerosis

OBJECTIVES

We tested the hypothesis that age is a prognostic factor with respect to long-term accumulation of disability in multiple sclerosis (MS).

METHODS

Kaplan-Meier analysis and binary logistic regression models determined the effect of age at disease onset, age at onset of progression, and current age on attainment of severe disability levels (Disability Status Scale [DSS] 6-8-10) from the London, Ontario, database (n = 1,023).

RESULTS

Older age at relapsing-remitting (RR) phase onset was associated with higher risk of reaching advanced DSS scores. This was independent of disease duration and early relapse frequency but secondary to increased risk of conversion to secondary progressive (SP) MS. Onset at age 40 (odds ratio [OR] = 4.22) and at age 50 (OR = 6.04) doubled and tripled risks of developing SP, compared to age 20 (OR = 2.05). Younger age at conversion to SPMS was associated with shorter times to high DSS scores from disease onset. The progressive course, unaffected by age at RR onset, was only modestly affected by age at SP onset. Among primary progressive and RR/SP patients, median ages at attainment of DSS scores were strikingly similar: DSS = 6, 49 vs 48 years; DSS = 8, 58 vs 58 years; and DSS = 10, 78 years for both (p = NS for all comparisons).

CONCLUSIONS

Development of SP is the dominant determinant of long-term prognosis, independent of disease duration and early relapse frequency. Age independently affects disability development primarily by changing probability and latency of SP onset, with little effect on the progressive course.

HAGIL (Hamburg Vigil Study): a randomized placebo-controlled double-blind study with modafinil for treatment of fatigue in patients with multiple sclerosis

Objective: To reassess the effect of modafinil, a wakefulness-promoting artificial psychostimulant, on fatigue and neuropsychological measures in patients with multiple sclerosis.

Methods: Multiple sclerosis (MS) patients with a baseline score of ≥4 on the Fatigue Severity Scale (FSS) and an Expanded Disability Status Scale score <7 were eligible for the 8-week randomized, double-blind, placebo-controlled study. Modafinil was dosed up to 200 mg/day within 1 week. Assessments were performed at baseline and after 4 and 8 weeks. The primary outcome parameter was the mean change of the FSS mean score. Secondary outcome variables were other questionnaires covering fatigue, daytime sleepiness and sleep quality. Cognitive impairment was assessed by the oral version of the Symbol Digit Modalities Test (SDMT) and the Paced Auditory Serial Addition Test (PASAT).

Results: The study included 121 MS patients. Dropout rate was 9%. Both treatment groups showed improvements through time. While mean FSS at 8 weeks showed a trend difference between groups in the intention-to-treat analysis, the primary endpoint was not met. Assessment of cognitive impairment by SDMT and PASAT showed contradictory results. All other secondary endpoints were not met. There was no major safety concern.

Conclusions: In general, the study does not support modafinil as an effective treatment for MS fatigue. However, the study shows the need for new study designs and endpoints in MS fatigue studies.

MRI as an outcome in multiple sclerosis clinical trials

INTRODUCTION

T2-weighted and gadolinium enhanced T1-weighted MRI scans measure plaque burden and breakdown of the blood-brain barrier, respectively, in multiple sclerosis (MS) lesions. These have become widely used outcome measures for monitoring disease activity in clinical trials and clinical practice. However, their use as surrogates or biomarkers for disability and relapses, key clinical outcome measures, has remained incompletely validated.

METHODS

In a clinical trial database comprised of 31 relapsing-remitting and secondary progressive MS trial placebo groups, we assessed relationships between 1) T2 lesion load (TLL) change and disability change and 2) gadolinium enhancement of MS lesions and on-study relapses with univariate and multivariate analyses.

RESULTS

In relapsing-remitting MS, TLL change (n = 223) made no independent contribution to predicting change in disability from baseline to trials' end. Similarly, inclusion of gadolinium enhancing lesions (n = 170) into multivariate models did not independently contribute to the predictive value for on-trial relapses. In secondary progressive MS, a small effect of TLL was found for disability change (n = 355) but in multivariate analysis this accounted for less than 5% of the variance in end-of-trial disability. Results were replicated in independent datasets, more than doubling effective sample sizes.

CONCLUSIONS

MRI measures widely used in trials of relapsing-remitting and progressive multiple sclerosis add little if anything independently to the clinically relevant relapse and disability outcomes. These results reemphasize the importance of validating potential surrogate markers against clinical measures and highlight the need for better MRI markers of disease activity and progression.

CMA: a comprehensive Bioconductor package for supervised classification with high dimensional data

Background

For the last eight years, microarray-based classification has been a major topic in statistics, bioinformatics and biomedicine research. Traditional methods often yield unsatisfactory results or may even be inapplicable in the so-called "p ≫ n" setting where the number of predictors p by far exceeds the number of observations n, hence the term "ill-posed-problem". Careful model selection and evaluation satisfying accepted good-practice standards is a very complex task for statisticians without experience in this area or for scientists with limited statistical background. The multiplicity of available methods for class prediction based on high-dimensional data is an additional practical challenge for inexperienced researchers.

Results

In this article, we introduce a new Bioconductor package called CMA (standing for "Classification for MicroArrays") for automatically performing variable selection, parameter tuning, classifier construction, and unbiased evaluation of the constructed classifiers using a large number of usual methods. Without much time and effort, users are provided with an overview of the unbiased accuracy of most top-performing classifiers. Furthermore, the standardized evaluation framework underlying CMA can also be beneficial in statistical research for comparison purposes, for instance if a new classifier has to be compared to existing approaches.

Conclusion

CMA is a user-friendly comprehensive package for classifier construction and evaluation implementing most usual approaches. It is freely available from the Bioconductor website at .

Disability as an outcome in MS clinical trials

BACKGROUND

Inferences about long-term effects of therapies in multiple sclerosis (MS) have been based on surrogate markers studied in short-term trials. Preventing progressive disability is the key therapeutic goal but there remains no validated definition for its measurement in a trial context. Meanwhile, MS trials continue to shorten and to depend on unvalidated surrogates. Since there have been no treatment claims for improving unremitting disability, worsening of disability in the placebo/control arm must occur for effectiveness on this outcome to be shown.

METHODS

We examined widely-used clinical surrogates of long-term disability progression in individual patients with MS within a unique database from the placebo arms of 31 randomized clinical trials.

RESULTS

Detection of treatment effects in secondary progressive MS trials is undermined by noise in disability measurement. Whereas existing measures can be partially validated in secondary progressive MS, this is not the case in relapsing-remitting MS. Here, examination of widely used definitions of treatment failure demonstrated that disability progression was no more likely than similarly defined improvement. Existing definitions of disease progression in short-term intervention trials in relapsing-remitting patients reflect random variation, measurement error, and remitting relapses.

CONCLUSION

Clinical surrogates of unremitting disability used in trials of relapsing-remitting multiple sclerosis cannot be validated. Trials have been too short or degrees of disability change too small to measure the key outcomes. These analyses highlight the difficulty in determining effectiveness of therapy in chronic diseases.

Evaluating microarray-based classifiers: an overview

For the last eight years, microarray-based class prediction has been the subject of numerous publications in medicine, bioinformatics and statistics journals. However, in many articles, the assessment of classification accuracy is carried out using suboptimal procedures and is not paid much attention. In this paper, we carefully review various statistical aspects of classifier evaluation and validation from a practical point of view. The main topics addressed are accuracy measures, error rate estimation procedures, variable selection, choice of classifiers and validation strategy.

Prognosis of the individual course of disease--steps in developing a decision support tool for Multiple Sclerosis

Background

Multiple sclerosis is a chronic disease of uncertain aetiology. Variations in its disease course make it difficult to impossible to accurately determine the prognosis of individual patients. The Sylvia Lawry Centre for Multiple Sclerosis Research (SLCMSR) developed an "online analytical processing (OLAP)" tool that takes advantage of extant clinical trials data and allows one to model the near term future course of this chronic disease for an individual patient.

Results

For a given patient the most similar patients of the SLCMSR database are intelligently selected by a model-based matching algorithm integrated into an OLAP-tool to enable real time, web-based statistical analyses. The underlying database (last update April 2005) contains 1,059 patients derived from 30 placebo arms of controlled clinical trials. Demographic information on the entire database and the portion selected for comparison are displayed. The result of the statistical comparison is provided as a display of the course of Expanded Disability Status Scale (EDSS) for individuals in the database with regions of probable progression over time, along with their mean relapse rate. Kaplan-Meier curves for time to sustained progression in the EDSS and time to requirement of constant assistance to walk (EDSS 6) are also displayed. The software-application OLAP anticipates the input MS patient's course on the basis of baseline values and the known course of disease for similar patients who have been followed in clinical trials.

Conclusion

This simulation could be useful for physicians, researchers and other professionals who counsel patients on therapeutic options. The application can be modified for studying the natural history of other chronic diseases, if and when similar datasets on which the OLAP operates exist.

Relapses and subsequent worsening of disability in relapsing-remitting multiple sclerosis

OBJECTIVE

To investigate whether relapses contribute to the development of subsequent sustained increase of impairment and disability in patients with multiple sclerosis (MS).

METHODS

In a random sampled subset of 256 relapsing-remitting MS (RRMS) patients from the placebo arms of 20 randomized, controlled clinical trials contained in the Sylvia Lawry Centre for MS Research (SLCMSR) open database (mean follow-up time 2.66 years), the authors tested whether time to an increase of the Expanded Disability Status Scale (EDSS) score (confirmed after 6 months) was related to the occurrence of prior relapses. In the primary analysis, EDSS progressions starting within the period used to calculate the on-study relapse rate (sacrifice period) were not counted. The result obtained was then validated in an independent validation part of the SLCMSR database (n = 320).

RESULTS

Although in the first subset of 256 RRMS patients, occurrence of relapses in the first 4 months on study appeared to be the best predictor for a shorter time to subsequent sustained increase in the EDSS score (hazard ratio [HR] 2.26 [95% CI: 1.36 to 3.75]), this finding was not confirmed in the validation dataset (HR 1.35, one-sided Wald test, lower limit of the 95% CI: 0.90).

CONCLUSION

Although relapses may result into permanent damage and Expanded Disability Status Scale (EDSS) progression, there is no consistent effect of on-study relapses on the subsequent development of sustained EDSS score increase during a typical clinical study observation period.

MRI T2 lesion burden in multiple sclerosis: a plateauing relationship with clinical disability

BACKGROUND

Previous studies have shown only modest correlation between multiple sclerosis (MS) lesions on MRI and clinical disability.

OBJECTIVE

To investigate the relationship between proton density/T2-weighted (T2) burden of disease (BOD) quantitatively measured on MRI scans and clinical determinants including disability.

METHODS

Using the Sylvia Lawry Centre for Multiple Sclerosis Research (SLCMSR) database, the authors studied baseline T2 BOD data from a pooled subsample of 1,312 placebo MS patients from 11 randomized controlled trials. Univariate comparisons guided development of multiple regression models incorporating the most important clinical predictors.

RESULTS

Significant, although weak to moderate, correlations were found between T2 BOD and age at disease onset, disease duration, disease course, disability (as measured by the Expanded Disability Status Scale [EDSS]), relapse rate, certain presenting symptoms, and gadolinium enhancement. An unexpected but key finding that persisted in the multiple regression analyses was a plateauing relationship between T2 BOD and disability for EDSS values above 4.5.

CONCLUSIONS

This study confirmed the limited correlation between clinical manifestations and T2 burden of disease (BOD) but revealed an important plateauing relationship between T2 BOD and disability.

Predicting gadolinium enhancement status in MS patients eligible for randomized clinical trials

BACKGROUND

Gadolinium enhancement is often used in randomized clinical trials to evaluate the efficacy of new drugs in multiple sclerosis (MS). Knowledge about predictors of enhancement status is important for the selection of patients for MRI monitored trials.

METHODS

Data from 17 trials were available in anonymized format through the Sylvia Lawry Centre for MS Research. In an open part containing 1,328 (non primary progressive) patients, two logistic regression analyses were explored, including demographic, clinical, and MRI predictors. The authors examined the area under the curve (AUC) and the increase in positive predictive value (PPV). The final selection of models was validated in a closed part of 848 comparable patients.

RESULTS

Age at onset, disease duration, and disease course (CIS/RR/SP) were important predictors from the multivariate models. Further, a multivariate model including T2 burden of disease was more predictive than one with only clinical predictors (AUC 0.719 vs 0.625, p < 0.001). For the model with T2 burden of disease, the PPV was 66.8%, compared to 58.5% for the model without (a priori chance 46.4%). These findings were unequivocally confirmed in the closed part of the database.

CONCLUSION

Gadolinium status can be predicted by a set of baseline variables, certainly when T2 burden of disease is included. These findings may benefit the design and statistical power of future randomized clinical trials.

MS-Register in Deutschland — Design und erste Ergebnisse der Pilotphase

In the summer of 2001, a nationwide epidemiological multiple sclerosis (MS) register was initiated under the auspices of the German MS Society (DMSG). This project aimed at collecting epidemiological data on the number of patients with MS, course of the disease, and their social situation in Germany. During the 2-year pilot phase, five MS centers with various regional differences and treatment methods participated, leading to a representative selection of patients. In December 2003, standardised data sets of 3,458 MS patients were available for evaluation. After examining the quality of the data, 3,223 sets remained for further analysis. The demographics were similar to those obtained from other epidemiological studies: 72% of the patients were female, mean age was 42.9+/-11.2 years, mean disease duration 12.6+/-8.7 years, and 64% suffered from the relapsing-remitting form of the disease. The median EDSS was 3.0, and 69% of patients had an EDSS </=4.0. The great effect of this disorder was underscored by the fact that one third of the patients had prematurely retired due to MS. After successful completion of the pilot phase, the MS register will provide reliable data and thus serve as an important tool to improve the overall situation of MS patients in Germany.

A randomized, double-blind, placebo-controlled study of oral hydrolytic enzymes in relapsing multiple sclerosis

Oral administration of hydrolytic enzymes (HE), such as bromelain, trypsin and rutosid, may have beneficial effects on the clinical course of neurological symptoms related to multiple sclerosis (MS). This is supported by a complete protection by HE from experimental allergic encephalomyelitis, an animal model related to MS. Three hundred and one patients with relapsing MS were enrolled in a double-blind, placebo-controlled trial. No treatment effect between the placebo and the HE groups was found either for clinical or MRI parameters.