Although not consistently superior, the absolute approach to framing the minimally important difference has advantages over the relative approach

A systematic survey identified methodological issues in studies estimating anchor-based minimal important differences in patient-reported outcomes

OBJECTIVE

To systematically survey the literature addressing the reporting of studies estimating anchor-based minimal important differences (MIDs) and choice of optimal MIDs.

STUDY DESIGN AND SETTING

We searched Medline, Embase and PsycINFO from 1987 to March 2020. Teams of two reviewers independently identified eligible publications and extracted quotations addressing relevant issues for reporting and/or selecting anchor-based MIDs. Using a coding list, we assigned the same code to quotations capturing similar or related issues. For each code, we generated an 'item', i.e. a specific phrase or sentence capturing the underlying concept. When multiple concepts existed under a single code, the team created multiple items for that code. We clustered codes addressing a broader methodological issue into a 'category' and classified items as relevant for reporting, relevant for selecting an anchor-based MID, or both.

RESULTS

We identified 136 eligible publications that provided 6 categories (MID definition, anchors, patient-reported outcome measures, generalizability and statistics) and 24 codes. These codes contained 34 items related to reporting MID studies, of which 29 were also related to selecting MIDs.

CONCLUSION

The systematic survey identified items related to reporting of anchor-based MID studies and selecting optimal MIDs. These provide a conceptual framework to inform the design of studies related to MIDs, and a basis for developing a reporting standard and a selection approach for MIDs.

The relative decision-making algorithm for ranking data

Purpose

Decision-making is always an issue that managers have to deal with. Keenly observing to different preferences of the targets provides useful information for decision-makers who do not require too much information to make decisions. The main purpose is to avoid decision-makers in a dilemma because of too much or opaque information. Based on problem-oriented, this research aims to help decision-makers to develop a macro-vision strategy that fits the needs of different clusters of customers in terms of their favorite restaurants. This research also focuses on providing the rules to rank data sets for decision-makers to make choices for their favorite restaurant.

Design/methodology/approach

When the decision-makers need to rethink a new strategic planning, they have to think about whether they want to retain or rebuild their relationship with the old consumers or continue to care for new customers. Furthermore, many of the lecturers show that the relative concept will be more effective than the absolute one. Therefore, based on rough set theory, this research proposes an algorithm of related concepts and sends questionnaires to verify the efficiency of the algorithm.

Findings

By feeding the relative order of calculating the ranking rules, we find that it will be more efficient to deal with the faced problems.

Originality/value

The algorithm proposed in this research is applied to the ranking data of food. This research proves that the algorithm is practical and has the potential to reveal important patterns in the data set.

Practical help for specifying the target difference in sample size calculations for RCTs: the DELTA2 five-stage study, including a workshop

BACKGROUND

The randomised controlled trial is widely considered to be the gold standard study for comparing the effectiveness of health interventions. Central to its design is a calculation of the number of participants needed (the sample size) for the trial. The sample size is typically calculated by specifying the magnitude of the difference in the primary outcome between the intervention effects for the population of interest. This difference is called the 'target difference' and should be appropriate for the principal estimand of interest and determined by the primary aim of the study. The target difference between treatments should be considered realistic and/or important by one or more key stakeholder groups.

OBJECTIVE

The objective of the report is to provide practical help on the choice of target difference used in the sample size calculation for a randomised controlled trial for researchers and funder representatives.

METHODS

The Difference ELicitation in TriAls2 (DELTA2) recommendations and advice were developed through a five-stage process, which included two literature reviews of existing funder guidance and recent methodological literature; a Delphi process to engage with a wider group of stakeholders; a 2-day workshop; and finalising the core document.

RESULTS

Advice is provided for definitive trials (Phase III/IV studies). Methods for choosing the target difference are reviewed. To aid those new to the topic, and to encourage better practice, 10 recommendations are made regarding choosing the target difference and undertaking a sample size calculation. Recommended reporting items for trial proposal, protocols and results papers under the conventional approach are also provided. Case studies reflecting different trial designs and covering different conditions are provided. Alternative trial designs and methods for choosing the sample size are also briefly considered.

CONCLUSIONS

Choosing an appropriate sample size is crucial if a study is to inform clinical practice. The number of patients recruited into the trial needs to be sufficient to answer the objectives; however, the number should not be higher than necessary to avoid unnecessary burden on patients and wasting precious resources. The choice of the target difference is a key part of this process under the conventional approach to sample size calculations. This document provides advice and recommendations to improve practice and reporting regarding this aspect of trial design. Future work could extend the work to address other less common approaches to the sample size calculations, particularly in terms of appropriate reporting items.

FUNDING

Funded by the Medical Research Council (MRC) UK and the National Institute for Health Research as part of the MRC-National Institute for Health Research Methodology Research programme.

Baseline health status and setting impacted minimal clinically important differences in COPD: an exploratory study

OBJECTIVES

Minimal clinically important differences (MCIDs) are used as fixed numbers in the interpretation of clinical trials. Little is known about its dynamics. This study aims to explore the impact of baseline score, study setting, and patient characteristics on health status MCIDs in chronic obstructive pulmonary disease (COPD).

STUDY DESIGN AND SETTING

Baseline and follow-up data on the COPD Assessment Test (CAT), Clinical COPD Questionnaire (CCQ), and St. George's Respiratory Questionnaire (SGRQ) were retrospectively analyzed from pulmonary rehabilitation (PR) and routine clinical practice (RCP). Anchor- and distribution-based MCID estimates were calculated and tested between settings, gender, age, Global initiative for Obstructive Lung Disease (GOLD) classification, comorbidities, and baseline health status.

RESULTS

In total, 658 patients were included with 2,299 change score measurements. MCID estimates for improvement and deterioration ranged for all subgroups 0.50-6.30 (CAT), 0.10-0.84 (CCQ), and 0.33-12.86 (SGRQ). Larger MCID estimates for improvement and smaller ones for deterioration were noted in patients with worse baseline health status, females, elderly, GOLD I/II patients, and patients with less comorbidities. Estimates from PR were larger.

CONCLUSION

Baseline health status and setting affected MCID estimates of COPD health status questionnaires. Patterns were observed for gender, age, spirometry classification, and comorbidity levels. These outcomes would advocate the need for tailored MCIDs.

Power estimations for non-primary outcomes in randomised clinical trials

OBJECTIVE AND METHODS

It is rare that trialists report power estimations of non-primary outcomes. In the present article, we will describe how to define a valid hierarchy of outcomes in a randomised clinical trial, to limit problems with Type I and Type II errors, using considerations on the clinical relevance of the outcomes and power estimations.

CONCLUSION

Power estimations of non-primary outcomes may guide trialists in classifying non-primary outcomes as secondary or exploratory. The power estimations are simple and if they are used systematically, more appropriate outcome hierarchies can be defined, and trial results will become more interpretable.

A study of target effect sizes in randomised controlled trials published in the Health Technology Assessment journal

BACKGROUND

When designing a randomised controlled trial (RCT), an important consideration is the sample size required. This is calculated from several components; one of which is the target difference. This study aims to review the currently reported methods of elicitation of the target difference as well as to quantify the target differences used in Health Technology Assessment (HTA)-funded trials.

METHODS

Trials were identified from the National Institute of Health Research Health Technology Assessment journal. A total of 177 RCTs published between 2006 and 2016 were assessed for eligibility. Eligibility was established by the design of the trial and the quality of data available. The trial designs were parallel-group, superiority RCTs with a continuous primary endpoint. Data were extracted and the standardised anticipated and observed effect size estimates were calculated. Exclusion criteria was based on trials not providing enough detail in the sample size calculation and results, and trials not being of parallel-group, superiority design.

RESULTS

A total of 107 RCTs were included in the study from 102 reports. The most commonly reported method for effect size derivation was a review of evidence and use of previous research (52.3%). This was common across all clinical areas. The median standardised target effect size was 0.30 (interquartile range: 0.20-0.38), with the median standardised observed effect size 0.11 (IQR 0.05-0.29). The maximum anticipated and observed effect sizes were 0.76 and 1.18, respectively. Only two trials had anticipated target values above 0.60.

CONCLUSION

The most commonly reported method of elicitation of the target effect size is previous published research. The average target effect size was 0.3. A clear distinction between the target difference and the minimum clinically important difference is recommended when designing a trial. Transparent explanation of target difference elicitation is advised, with multiple methods including a review of evidence and opinion-seeking advised as the more optimal methods for effect size quantification.

Minimal Important Difference in Voice Handicap Index-10

Importance

The minimal important difference (MID) on patient-reported outcome measures can indicate how much of a change on that scale is meaningful.

Objective

To use an anchor-based approach to estimate MID in the Voice Handicap Index-10 (VHI-10) total score.

Design, Setting, and Participants

In this cohort study, a volunteer sample of adult patients visiting the voice clinic at the University of Minnesota from April 7, 2013, through July 3, 2016, completed the VHI-10 (range, 0-40, with higher scores indicating greater voice-related handicap) at baseline and 2 weeks later in conjunction with a global rating of change. An anchor-based approach was used to identify an MID. The association between the global change score and change in VHI-10 score was analyzed using Pearson rank correlation. A distribution-based method was used to corroborate the findings.

Main Outcome and Measures

Global rating of change on the VHI-10.

Results

Of the 273 participants, 183 (67.0%) were women and 90 (33.0%) were men (mean [SD] age, 54.3 [15.6] years); 259 (94.9%) were white. Participants had a variety of voice disorders, most commonly muscle tension dysphonia, irritable larynx, benign vocal fold lesions, and motion abnormalities. Among patients reporting no change on the global change score, the mean (SD) change in VHI-10 score was 1 (5). Among those reporting a small change, the mean (SD) change in VHI-10 was also 1 (5). Among those reporting a moderate change in voice symptoms, the mean (SD) change in VHI-10 score was 6 (8). Among those with a large change, the mean (SD) change in VHI-10 score was 9 (13). The correlation between the global change score and the change in VHI-10 score was 0.32 (95% CI, 0.12-0.49). Distribution-based analyses identified effect sizes comparable to those of the anchor-based categories.

Conclusions and Relevance

These findings suggest that a difference of 6 on the VHI-10 may represent an MID. This difference was associated with a moderate change on the global rating scale, and the small-change and no-change categories were indistinguishable. Given the lack of differentiation between small and no change and the modest correlation between the global change score and change in the VHI-10 score, additional studies are needed.

Quality of Life in Palliative Care

Introduction

A main goals of palliative care is to improve the health-related quality of life (QOL) of patients with advanced illnesses. The objective of this narrative review is to provide an updated synopsis on the use of QOL questionnaires in the palliative care setting.

Areas covers

Focusing on the palliative cares setting, we will define QOL, discuss how QOL instruments can be used clinically and in research, review approaches to validate these questionnaires, and how they can be used in utility analyses.

Expert opinion/commentary

Several QOL questionnaires, such as EORTC-QLQ-C30, McGill QOL questionnaire and EQ-5D have been validated in the palliative care setting. However, significant gaps impede their application, including lack of determination of their responsiveness to change and minimal clinically important differences, the need to conduct more psychometric validation on QOL questionnaires among patients at various stages of disease trajectory, and the paucity of studies examining utility and cost-effectiveness. Further research is needed to address these knowledge gaps so QOL questionnaires can be better used to inform clinical practice and research.

Application of minimal important differences in degenerative knee disease outcomes: a systematic review and case study to inform BMJ Rapid Recommendations

OBJECTIVES

To identify the most credible anchor-based minimal important differences (MIDs) for patient important outcomes in patients with degenerative knee disease, and to inform BMJ Rapid Recommendations for arthroscopic surgery versus conservative management DESIGN: Systematic review.

OUTCOME MEASURES

Estimates of anchor-based MIDs, and their credibility, for knee symptoms and health-related quality of life (HRQoL).

DATA SOURCES

MEDLINE, EMBASE and PsycINFO.

ELIGIBILITY CRITERIA

We included original studies documenting the development of anchor-based MIDs for patient-reported outcomes (PROs) reported in randomised controlled trials included in the linked systematic review and meta-analysis and judged by the parallel BMJ Rapid Recommendations panel as critically important for informing their recommendation: measures of pain, function and HRQoL.

RESULTS

13 studies reported 95 empirically estimated anchor-based MIDs for 8 PRO instruments and/or their subdomains that measure knee pain, function or HRQoL. All studies used a transition rating (global rating of change) as the anchor to ascertain the MID. Among PROs with more than 1 estimated MID, we found wide variation in MID values. Many studies suffered from serious methodological limitations. We identified the following most credible MIDs: Western Ontario and McMaster University Osteoarthritis Index (WOMAC; pain: 12, function: 13), Knee injury and Osteoarthritis Outcome Score (KOOS; pain: 12, activities of daily living: 8) and EuroQol five dimensions Questionnaire (EQ-5D; 0.15).

CONCLUSIONS

We were able to distinguish between more and less credible MID estimates and provide best estimates for key instruments that informed evidence presentation in the associated systematic review and judgements made by the Rapid Recommendation panel.

TRIAL REGISTRATION NUMBER

CRD42016047912.

Minimally Important Differences in Patient or Proxy-Reported Outcome Studies Relevant to Children: A Systematic Review

CONTEXT

No study has characterized and appraised all anchor-based minimally important differences (MIDs) associated with patient-reported outcome (PRO) instruments in pediatric studies.

OBJECTIVE

To complete a comprehensive systematic survey and appraisal of published anchor-based MIDs associated with PRO instruments used in children.

DATA SOURCES

Medline, Embase, and PsycINFO (1989 to February 11, 2015).

STUDY SELECTION

Studies reporting empirical ascertainment of anchor-based MIDs among PROs used in pediatric care.

DATA EXTRACTION

All pertinent data items related to the characteristics of PRO instruments, anchors, and MIDs.

RESULTS

Of 4179 unique citations, 30 studies (including 32 cohorts) proved eligible and reported on 28 unique PROs (8 generic, 13 disease-specific, 5 symptoms-specific, 2 function-specific), with 9 (32%) classified as patient-reported, 11 (39%) proxy-reported, and 8 (29%) both patient- and proxy-reported. Of the 30 studies, we rated 14 (44%) as providing highly credible estimates of the MID. Most cohorts (n = 20, 62%) recorded patients' direct response to the target PRO and the use of an independent standard of comparison (n = 25, 78%). Most, however, failed to effectively report measurement properties of the anchor (n = 24, 75%).

LIMITATIONS

We have not yet addressed the measurement properties of instrument to measure credibility; our search was restricted to 3 electronic sources, and we used a single data abstractor.

CONCLUSIONS

Our study found 28 PROs that have been developed for children, with fewer than half providing credible estimates. Clinicians, clinical trialists, systematic reviewers, and guideline developers seeking to effectively summarize and interpret results of studies addressing PROs in child health are likely to find our comprehensive compendium of MIDs of use, both in providing best estimates of MIDs and identifying credible estimates.

Meta-analysis on continuous outcomes in minimal important difference units: an application with appropriate variance calculations

OBJECTIVE

To compare results from meta-analyses for mean differences in minimal important difference (MID) units (MD_MID), when MID is treated as a random variable vs. a constant.

STUDY DESIGN AND SETTING

Meta-analyses of published data. We calculated the variance of MD_MID as a random variable using the delta method and as a constant. We assessed performance under different assumptions. We compare meta-analysis results from data originally used to present the MD_MID and data from osteoarthritis studies using different domain instruments.

RESULTS

Depending on the data set and depending on the values of rho and coefficient of variation of the MID (CoV_MID), estimates of treatment effect and P-values between an approach considering the MID as a constant vs. as a random variable may differ appreciably. Using our data sets, we provide examples of the potential magnitude. When rho = 0.5 and CoV_MID = 0.8, considering MID as a constant overestimated the treatment effect by 33-110% and decreased the P-value for heterogeneity from above 0.95 to below 0.08. When rho = 0.8 and CoV_MID = 0.5, the magnitude of the effects was similar.

CONCLUSIONS

Considering MID as a random variable avoids unrealistic assumptions and provides more appropriate treatment effect estimates.

Methylprednisolone for the Treatment of Patients with Acute Spinal Cord Injuries: A Systematic Review and Meta-Analysis

Previous meta-analyses of methylprednisolone (MPS) for patients with acute traumatic spinal cord injuries (TSCIs) have not addressed confidence in the quality of evidence used for pooled effect estimates, and new primary studies have been recently published. We aimed to determine whether MPS improves motor recovery and is associated with increased risks for adverse events. We searched MEDLINE, EMBASE, and The Cochrane Library, and two reviewers independently screened articles, extracted data, and evaluated risk of bias. We pooled outcomes from randomized, controlled trials (RCTs) and controlled observational studies separately and used the Grades of Recommendation, Assessment, Development, and Evaluation approach to evaluate confidence. We included four RCTs and 17 observational studies. MPS was not associated with an increase in long-term motor score recovery (two RCTs: 335 participants; mean difference [MD], −1.11; 95% confidence interval [CI], −4.75 to 2.53; p = 0.55, low confidence; two observational studies: 528 participants; MD, 1.37; 95% CI, −3.08 to 5.83; p = 0.55, very low confidence) or improvement by at least one motor grade (three observational studies: 383 participants; risk ratio [RR], 0.84; 95% CI, 0.53–1.33; p = 0.46, very low confidence). Evidence from two RCTs demonstrated superior short-term motor score improvement if MPS was administered within 8 h of injury (two RCTs: 250 participants; MD, 4.46; 95% CI, 0.97–7.94; p = 0.01, low confidence), but risk of bias and imprecision limit confidence in these findings. Observational studies demonstrated a significantly increased risk for gastrointestinal bleeding (nine studies: 2857 participants; RR, 2.18; 95% CI, 1.13–4.19; p = 0.02, very low confidence), but RCTs did not. Pooled evidence does not demonstrate a significant long-term benefit for MPS in patients with acute TSCIs and suggests it may be associated with increased gastrointestinal bleeding. These findings support current guidelines against routine use, but strong recommendations are not warranted because confidence in the effect estimates is limited.