Commentary: External validity of results of randomized trials: disentangling a complex concept

Real-world effectiveness of first- and second-line anti-angiogenesis therapy in RCC: analysis of a UK-based population

Aim: Renal cell carcinoma (RCC) is the seventh commonest cancer in the UK, where first-line (1L) sunitinib and second-line (2L) axitinib are treatment options.Methods: Retrospective, non-interventional data from the Christie NHS Foundation Trust (Manchester, UK). The primary end point was median progression-free survival (mPFS).Results: For 1L sunitinib (n = 622) and 2L axitinib (n = 121), mPFS (95% CI) was 8.4 (7.6, 9.9) and 6.2 (4.9, 9.3) months, respectively. In 1L, Karnofsky performance status, lactate dehydrogenase (LDH), neutrophils, hemoglobin, time from diagnosis to treatment and age were predictors (p < 0.05) of PFS. In 2L, LDH and platelets were predictors of PFS (p < 0.05).Conclusion: Sunitinib and axitinib were effective treatments for RCC. PFS predictors varied between 1L and 2L; LDH was a predictor for both.Clinical Trial Registration: NCT04033991 (ClinicalTrials.gov).

Real-world experience of second-line axitinib in metastatic renal cell carcinoma: analysis of the Swedish population

Aim: Assess the time-to-treatment discontinuation (TTD) and overall survival (OS) in a Swedish metastatic renal cell carcinoma (mRCC) nationwide cohort who received second-line axitinib.Methods: Retrospective analysis of 110 patients with mRCC treated with second-line axitinib in Sweden (2012-2019). Patients included in the study received axitinib after mainly first-line sunitinib or pazopanib.Results: The median (95% CI) TTD of patients who received second-line axitinib was 5.2 (3.7-6.1) months with 6 (5.5%) patients still receiving treatment at the time of analysis. Median (95% CI) OS was 12.2 (7.7-14.2) months.Conclusion: The results are consistent with previous findings in mRCC and add to the evidence demonstrating efficacy of second-line axitinib, after failure of a prior anti-angiogenic therapy in a real-world setting.Clinical Trial Registration: NCT04669366 (ClinicalTrials.gov).

Model-based standardization using multiple imputation

BACKGROUND

When studying the association between treatment and a clinical outcome, a parametric multivariable model of the conditional outcome expectation is often used to adjust for covariates. The treatment coefficient of the outcome model targets a conditional treatment effect. Model-based standardization is typically applied to average the model predictions over the target covariate distribution, and generate a covariate-adjusted estimate of the marginal treatment effect.

METHODS

The standard approach to model-based standardization involves maximum-likelihood estimation and use of the non-parametric bootstrap. We introduce a novel, general-purpose, model-based standardization method based on multiple imputation that is easily applicable when the outcome model is a generalized linear model. We term our proposed approach multiple imputation marginalization (MIM). MIM consists of two main stages: the generation of synthetic datasets and their analysis. MIM accommodates a Bayesian statistical framework, which naturally allows for the principled propagation of uncertainty, integrates the analysis into a probabilistic framework, and allows for the incorporation of prior evidence.

RESULTS

We conduct a simulation study to benchmark the finite-sample performance of MIM in conjunction with a parametric outcome model. The simulations provide proof-of-principle in scenarios with binary outcomes, continuous-valued covariates, a logistic outcome model and the marginal log odds ratio as the target effect measure. When parametric modeling assumptions hold, MIM yields unbiased estimation in the target covariate distribution, valid coverage rates, and similar precision and efficiency than the standard approach to model-based standardization.

CONCLUSION

We demonstrate that multiple imputation can be used to marginalize over a target covariate distribution, providing appropriate inference with a correctly specified parametric outcome model and offering statistical performance comparable to that of the standard approach to model-based standardization.

Deficiencies in reporting inclusion/exclusion criteria and characteristics of patients in randomized controlled trials of therapeutic interventions in pressure injuries: a systematic methodological review

Wound care is a complex procedure and the related research may include many variables. Deficiencies in the sample inclusion and exclusion criteria may limit the generalizability of randomized controlled trials (RCTs) for wound patients in the real world. This study aimed to evaluate deficiencies in reporting the inclusion and exclusion criteria and the characteristics of patients in RCTs of pressure injuries (PI) therapeutic interventions. We conducted a systematic methodological review in which 40 full text RCTs of PI treatment interventions published in English, from 2008 to 2020, were identified. Data on the general characteristics of the included RCTs and data about inclusion/exclusion criteria and characteristics of patients were collected. The inclusion/exclusion criteria were categorized into five domains (definition of disease, precision, safety, ethical/legal and administrative). Study duration (in weeks) was 8.0 (quartile 1: 2.0; quartile 3: 48.0); only 5.0% of the trials mentioned race, skin colour or ethnicity, and 37.5% reported the duration of the wound. Only 9 (22.5%) studies reported the drugs that the included patients were using and 10 (25.0%) RCTs reported adverse events. The presence of the five domains was observed only in 12.5% of RCTs and only 12 (30.0%) had the precision domain. Much more research is required in systematic assessments of the external validity of trials because there is substantial disparity between the information that is provided by RCTs and the information that is required by clinicians. We concluded that there are deficiencies in reporting of data related to inclusion/exclusion criteria and characteristics of patients of RCTs assessing PI therapeutic interventions.

Efficacy and safety of pembrolizumab and axitinib as first-line treatment for patients with advanced renal cell carcinoma: Real-world experience in Japan

OBJECTIVES

The objective of this study was to assess the clinical outcomes following combined treatment with pembrolizumab and axitinib as first-line therapy for patients with advanced RCC.

METHODS

This study retrospectively included 47 consecutive Japanese patients who were diagnosed with advanced RCC and subsequently received pembrolizumab and axitinib between February 2020 and January 2022. Efficacy and safety of this combined therapy in these patients were comprehensively investigated.

RESULTS

The 47 included patients were classified into the following 3 groups by the IMDC system: favorable, 7 (14.9%); intermediate, 24 (51.1%) and poor, 16 (34.0%). Responses to this combined therapy in the 47 patients were as follows: CR, 8 (17.0%); PR, 20 (42.6%); SD, 16 (34.0%) and PD, 3 (6.4%); thus, the ORR was 59.6%. During the observation period, disease progression and death occurred in 19 (40.4%) and 9 (19.1%) patients, respectively, and the median PFS and OS were 18 months and not reached, respectively. Univariate analyses identified the following significant predictors for poor prognostic outcomes: lack of nephrectomy, liver metastasis, bone metastasis, elevated CRP and IMDC poor risk for PFS; and lack of nephrectomy, non-CCC and elevated CRP for OS. AEs and those corresponding to grade ≥ 3 occurred in all (100%) and 30 (63.8%) patients, respectively.

CONCLUSIONS

To our knowledge, this is the first study focusing on real-world outcomes following pembrolizumab and axitinib for treatment-naïve advanced Japanese RCC patients, which showed the efficacy and safety of this combined therapy being similar or even superior to those in clinical trial.

Research objectives and general considerations for pragmatic clinical trials of pain treatments: IMMPACT statement

ABSTRACT

Many questions regarding the clinical management of people experiencing pain and related health policy decision-making may best be answered by pragmatic controlled trials. To generate clinically relevant and widely applicable findings, such trials aim to reproduce elements of routine clinical care or are embedded within clinical workflows. In contrast with traditional efficacy trials, pragmatic trials are intended to address a broader set of external validity questions critical for stakeholders (clinicians, healthcare leaders, policymakers, insurers, and patients) in considering the adoption and use of evidence-based treatments in daily clinical care. This article summarizes methodological considerations for pragmatic trials, mainly concerning methods of fundamental importance to the internal validity of trials. The relationship between these methods and common pragmatic trials methods and goals is considered, recognizing that the resulting trial designs are highly dependent on the specific research question under investigation. The basis of this statement was an Initiative on Methods, Measurement, and Pain Assessment in Clinical Trials (IMMPACT) systematic review of methods and a consensus meeting. The meeting was organized by the Analgesic, Anesthetic, and Addiction Clinical Trial Translations, Innovations, Opportunities, and Networks (ACTTION) public-private partnership. The consensus process was informed by expert presentations, panel and consensus discussions, and a preparatory systematic review. In the context of pragmatic trials of pain treatments, we present fundamental considerations for the planning phase of pragmatic trials, including the specification of trial objectives, the selection of adequate designs, and methods to enhance internal validity while maintaining the ability to answer pragmatic research questions.

Is fatty infiltration in paraspinal muscles reversible with exercise in people with low back pain? A systematic review

PURPOSE

Increased fatty infiltration in paraspinal muscles has been recognized as a feature of muscle quality loss in people with Low Back Pain (LBP) and is highly associated with the severity of LBP and dysfunction. Reducing fatty infiltration has been recognized as a rehabilitation aim. An earlier systematic review published in 2014 revealed conflicting evidence for the reversibility of paraspinal muscle quality by means of exercise and no updates have been published since. A new systematic literature search is warranted.

METHOD

Pubmed, CINAHL and Embase were searched from inception to July 2022. Randomized, non-randomized controlled trials (RCT and non-RCT) and single-arm trials were included if they reported the effect of exercise on paraspinal fatty infiltration in people with LBP. Effect sizes and statistical power were calculated for (1) exercise versus control, and (2) pre-post exercise changes. Available data from the RCTs were pooled via meta-analysis when appropriate. Otherwise, data were synthesized qualitatively.

RESULTS

Two RCTs, one non-RCT and three single-arm trials met the selection criteria. Data were not pooled due to substantial clinical heterogeneity. Effect sizes from the RCTs revealed no significant difference for exercise versus control. One single-arm trial with high risk of bias demonstrated a significant pre-post difference with moderate effect size, but only at one (T12-L1) of the investigated levels.

CONCLUSION

Moderate quality evidence is available that paraspinal fatty infiltration is not reversible with exercise in people with LBP. More larger RCT's are needed to draw firmer conclusions.

Real-World Therapy Management and Outcomes of First-Line Axitinib Plus Pembrolizumab in Patients With Advanced Renal Cell Carcinoma in the United States

Background

Combination axitinib plus pembrolizumab is a standard of care in the first-line treatment of patients with advanced clear cell renal cell carcinoma (RCC). This analysis describes the clinical characteristics, treatment management and outcomes of patients receiving first-line (1L) axitinib plus pembrolizumab in a real-world US setting.

Methods

Electronic health record (EHR)-derived data from the Flatiron Health Database, which includes ~280 cancer clinics across 800 sites in the US, were used. Patients had confirmed Stage IV or metastatic RCC and initiated 1L axitinib plus pembrolizumab on or after 1/1/2018 to 3/31/2021. Outcomes were best overall response rate; real-world progression-free survival (rwPFS) and overall survival (OS) at landmark time periods (3, 6, 9, and 12 months). Therapy management (TM) included dose hold, dose change and discontinuation. Data are reported as medians (IQR) unless otherwise noted.

Results

355 patients received 1L axitinib plus pembrolizumab, with median follow-up of 9.7 (0.1–24.3) months. IMDC Risk Score was favorable, intermediate, and poor in 27 (7.6%), 126 (35.5%), and 76 (21.4%) patients, respectively (23.4% intermediate/poor, 12.1% unknown). 270 patients (76.1%) received only 1L axitinib plus pembrolizumab and 85 patients (24.3%) received ≥1 subsequent line of treatment; cabozantinib was the most frequent subsequent line of treatment (47.9%). rwPFS at 3 months and 1 year was 77.2% and 39.3%, respectively. OS ranged from 90.8% at 3 months to 73.5% at 1 year. Best overall response rate was 47.9%. Toxicity was the most common reason for first TM events of dose hold, change and discontinuation at, 58.6%, 58.5%, and 45.8%, respectively. Over 80% of patients with TM were able to continue with 1L axitinib plus pembrolizumab.

Conclusions

In a real-world setting, axitinib plus pembrolizumab was effective as a 1L treatment for patients with advanced RCC. Dose holds, changes and discontinuation were driven by treatment-related toxicity. Dose holds may represent an effective TM strategy to toxicity.

External Validity of Somatostatin Analogs Trials in Advanced Neuroendocrine Neoplasms: The GETNE-TRASGU Study

INTRODUCTION

Somatostatin analogs (SSA) prolong progression-free survival (PFS) in patients with well-differentiated gastroenteropancreatic neuroendocrine neoplasms (GEP-NENs). However, the eligibility criteria in randomized clinical trials (RCTs) have been restricted, which contrasts with the vast heterogeneity found in NENs.

METHODS

We identified patients with well-differentiated (Ki-67% ≤20%), metastatic GEP-NENs treated in first line with SSA monotherapy from the Spanish R-GETNE registry. The therapeutic effect was evaluated using a Bayesian Cox model. The objective was to compare survival-based outcomes from real-world clinical practice versus RCTs.

RESULTS

The dataset contained 535 patients with a median age of 62 years (range: 26-89). The median Ki-67% was 4 (range: 0-20). The most common primary tumor sites were as follows: midgut, 46%; pancreas, 34%; unknown primary, 10%; and colorectal, 10%. Half of the patients received octreotide LAR (n = 266) and half, lanreotide autogel (n = 269). The median PFS was 28.0 months (95% CI: 22.1-32.0) for octreotide versus 30.1 months (95% CI: 23.1-38.0) for lanreotide. The overall hazard ratio for lanreotide versus octreotide was 0.90 (95% credible interval: 0.71-1.12). The probability of effect sizes >30% with lanreotide versus octreotide was 2 and 6% for midgut and foregut NENs, respectively.

CONCLUSION

Our study evaluated the external validity of RCTs examining SSAs in the real world, as well as the main effect-modifying factors (progression status, symptoms, tumor site, specific metastases, and analytical data). Our results indicate that both octreotide LAR and lanreotide autogel had a similar effect on PFS. Consequently, both represent valid alternatives in patients with well-differentiated, metastatic GEP-NENs.

Generalizability and Transportability of the National Lung Screening Trial Data: Extending Trial Results to Different Populations

BACKGROUND

Randomized controlled trials (RCT) play a central role in evidence-based healthcare. However, the clinical and policy implications of implementing RCTs in clinical practice are difficult to predict as the studied population is often different from the target population where results are being applied. This study illustrates the concepts of generalizability and transportability, demonstrating their utility in interpreting results from the National Lung Screening Trial (NLST).

METHODS

Using inverse-odds weighting, we demonstrate how generalizability and transportability techniques can be used to extrapolate treatment effect from (i) a subset of NLST to the entire NLST population and from (ii) the entire NLST to different target populations.

RESULTS

Our generalizability analysis revealed that lung cancer mortality reduction by LDCT screening across the entire NLST [16% (95% confidence interval [CI]: 4-24)] could have been estimated using a smaller subset of NLST participants. Using transportability analysis, we showed that populations with a higher prevalence of females and current smokers had a greater reduction in lung cancer mortality with LDCT screening [e.g., 27% (95% CI, 11-37) for the population with 80% females and 80% current smokers] than those with lower prevalence of females and current smokers.

CONCLUSIONS

This article illustrates how generalizability and transportability methods extend estimation of RCTs' utility beyond trial participants, to external populations of interest, including those that more closely mirror real-world populations.

IMPACT

Generalizability and transportability approaches can be used to quantify treatment effects for populations of interest, which may be used to design future trials or adjust lung cancer screening eligibility criteria.

Target validity: Bringing treatment of external validity in line with internal validity

Purpose of Review

"Target bias" is the difference between an estimate of association from a study sample and the causal effect in the target population of interest. It is the sum of internal and external bias. Given the extensive literature on internal validity, here, we review threats and methods to improve external validity.

Recent findings

External bias may arise when the distribution of modifiers of the effect of treatment differs between the study sample and the target population. Methods including those based on modeling the outcome, modeling sample membership, and doubly robust methods are available, assuming data on the target population is available.

Summary

The relevance of information for making policy decisions is dependent on both the actions that were studied and the sample in which they were evaluated. Combining methods for addressing internal and external validity can improve the policy relevance of study results.

Assessing the uncertainty of treatment outcomes in a previous systematic review of venous leg ulcer randomized controlled trials: Additional secondary analysis

In this secondary analysis of a previous systematic review, we assessed randomized controlled trials evaluating treatments of venous leg ulcers in terms of factors that affect risk of bias at the study level and thus uncertainty of outcomes obtained from the interventions. Articles that assessed the wound bed condition in venous leg ulcers and that were published in English between 1998 and May 22, 2018 were previously searched in PubMed, Embase, CINAHL, CENTRAL, Scopus, Science Direct, and Web of Science. Duplicates and retracted articles were excluded. The following data were extracted to assess the risk of bias: treatment groups; primary and secondary endpoints that were statistically tested between groups, including their results and p values; whether blinding of patients and assessors was done; whether allocation concealment was adequate; whether an intention‐to‐treat analysis was conducted; whether an appropriate power calculation was correctly done; and whether an appropriate multiplicity adjustment was made, as necessary. Pre‐ and post‐study power calculations were made. The step‐up Hochberg procedure adjusted for multiplicity. Results were analysed for all studies, pre‐2013 studies, and 2013/post‐2013 studies. We included 142 randomized controlled trials that evaluated 14,141 patients. Most studies lacked blinding (72.5–77.5%) and allocation concealment (88.7%). Only 49.3% of trials provided a power calculation, with 27.5% having an appropriate calculation correctly done. Adequate statistical power of the primary endpoint was found in 27.2% of trials. The lack of multiplicity adjustment in 98.6% of studies affected the uncertainty of outcomes in 20% of studies, with the majority of the secondary endpoints (67.7%) in those studies becoming non‐significant after multiplicity adjustment. Recent studies tended to weakly demonstrate improved certainty of outcomes. Venous leg ulcer randomized controlled trials have a high degree of uncertainty associated with treatment outcomes. Greater attention to trial design and conduct is needed to improve the evidence base.

External validity of docetaxel triplet trials in advanced gastric cancer: are there patients who still benefit?

BACKGROUND

The purpose of our study was to develop an online calculator to estimate the effect of docetaxel triplets (DPF) in first line of advanced gastric cancer (AGC), and to assess the external validity of docetaxel trials in individual patients.

METHODS

The study includes patients with HER2(-) AGC treated with platin and fluoropyrimidine (PF) or with DPF in first line. Treatment effect and interactions were assessed using Bayesian accelerated failure time models.

RESULT

The series comprises 1376 patients; 238 treated with DPF and 1138 with PF between 2008 and 2019. DPF was associated with increased progression-free survival (PFS) and overall survival (OS) with time ratio (TR) 1.27 (95% credible interval [CrI], 1.15-1.40), and TR 1.19 (95% CrI, 1.09-1.27), respectively. Serious adverse events were more common with DPF, particularly hematological effects (32% vs 22%). Younger participants received greater DPF dose density without achieving greater disease control, while severe toxicity was likewise higher. DPF yielded superior OS in Lauren intestinal (TR 1.27, 95% CrI, 1.08-1.11) vs diffuse subtype (TR 1.17, 95% CrI, 1.09-1.24) and the probability of increasing OS > 15% was 90% vs 67% in each subtype, respectively. The effect dwindles over time, which can be attributed to pathological changes and clinical practice changes.

CONCLUSION

Our study confirms the effect of DPF is highly dependent on several clinical-pathological variables, with discreet and gradually declining benefit over platinum doublets in later years, at the expense of increased toxicity. These results may help to underpin the idea that external validity of AGC trials should be revised regularly.

Natural Language Processing for Mimicking Clinical Trial Recruitment in Critical Care: A Semi-Automated Simulation Based on the LeoPARDS Trial

Clinical trials often fail to recruit an adequate number of appropriate patients. Identifying eligible trial participants is resource-intensive when relying on manual review of clinical notes, particularly in critical care settings where the time window is short. Automated review of electronic health records (EHR) may help, but much of the information is in free text rather than a computable form. We applied natural language processing (NLP) to free text EHR data using the CogStack platform to simulate recruitment into the LeoPARDS study, a clinical trial aiming to reduce organ dysfunction in septic shock. We applied an algorithm to identify eligible patients using a moving 1-hour time window, and compared patients identified by our approach with those actually screened and recruited for the trial, for the time period that data were available. We manually reviewed records of a random sample of patients identified by the algorithm but not screened in the original trial. Our method identified 376 patients, including 34 patients with EHR data available who were actually recruited to LeoPARDS in our centre. The sensitivity of CogStack for identifying patients screened was 90% (95% CI 85%, 93%). Of the 203 patients identified by both manual screening and CogStack, the index date matched in 95 (47%) and CogStack was earlier in 94 (47%). In conclusion, analysis of EHR data using NLP could effectively replicate recruitment in a critical care trial, and identify some eligible patients at an earlier stage, potentially improving trial recruitment if implemented in real time.

Sunitinib for Metastatic Renal Cell Carcinoma: A Systematic Review and Meta-Analysis of Real-World and Clinical Trials Data

Background

Randomized controlled trials (RCTs) have stringent inclusion criteria and may not fully represent patients seen in everyday clinical practice. Real-world data (RWD) can provide supportive evidence for the effectiveness of medical interventions in more heterogeneous populations than RCTs. Sunitinib is a widely used first-line treatment for patients with metastatic renal cell carcinoma (mRCC).

Objective

This is the first comprehensive meta-analysis to evaluate the efficacy of sunitinib using the novel approach of combining RCTs and RWD.

Methods

RCTs and RWD studies published between 2000 and 2017 were identified from PubMed, Ovid, MEDLINE, and EMBASE. Eligible studies contained a cohort of ≥ 50 adult patients with mRCC receiving first-line sunitinib treatment. The meta-analysis combined RWD and RCT treatment groups, adjusting for data type (RCT or RWD). Recorded outcomes were median progression-free survival (mPFS), median overall survival (mOS), and objective response rate (ORR). Publication bias was assessed via review of funnel plots for each outcome measure. A random effects model to account for study heterogeneity was applied to each endpoint. Sensitivity analyses evaluated the robustness of the overall estimates.

Results

Of the 3611 studies identified through medical database searches, 22 (15 RWD studies, 7 RCTs) met eligibility criteria and were analyzed. mPFS (18 studies), mOS (19 studies), and ORR (15 studies) were reported for aggregate measures based on 4815, 5321, and 4183 patients, respectively. Reported mPFS (RWD, 7.5–11.0 months; RCTs, 5.6–15.1 months) and ORR data (RWD, 14.0–34.6%; RCTs, 18.8–46.9%) were consistent with the overall confidence estimates (95% confidence interval [CI]) of 9.3 (8.6–10.2) months and 27.9% (24.2–32.0), respectively. Reported mOS showed greater variation in RWD (6.8–33.2 months) compared with RCTs (21.8–31.5 months), with an overall confidence estimate (95% CI) of 23.0 (19.2–27.6) months. Inspection of funnel plots and sensitivity analyses indicated that there was no publication bias for any efficacy endpoint. Sensitivity analyses showed no evidence of lack of robustness for mPFS, mOS, or ORR. Interpretation of these results is limited by differences in trial design, cohort characteristics, and missing data.

Conclusions

This novel, comprehensive meta-analysis validates sunitinib as an effective first-line treatment for patients with mRCC in both RCTs and everyday clinical practice. The methodology provides a framework for future analyses combining data from RCTs and RWD.

Electronic supplementary material

The online version of this article (10.1007/s11523-019-00653-5) contains supplementary material, which is available to authorized users.

Completeness of the description of manipulation and mobilisation techniques in randomized controlled trials in neck pain; A review using the TiDieR checklist

STUDY DESIGN

A secondary analysis of a systematic review.

BACKGROUND

Manipulations or mobilizations are commonly used interventions in patients with mechanical neck pain. The treatment effects have often been studied in randomized controlled trials (RCT) which are generally considered the gold standard in evaluating the treatment effects, mainly due to its high internal validity. External validity is defined as the extent to which the effects can be generalised to clinical practice. An important prerequisite for this is that interventions used in clinical trials can be replicated in clinical practice. It can be questioned if interventions utilized in randomized controlled trials can be translated into clinical practice.

OBJECTIVES

The overall aim of this study is to examine whether the quality of the description of manipulation and mobilization interventions is sufficient for to replication of these interventions in clinical practice.

METHODS

A comprehensive literature search was performed. Two independent researchers used the Template for Intervention Description and Replication (TIDieR) which is a 12-item checklist for describing the completeness of the interventions.

RESULTS

Sixty-seven articles were included that used manipulation and/or mobilization interventions for patients with mechanical neck pain. None of the articles describe the intervention e.g. all the items on the TIDieR list. Considering item 8 (a-f) of the TIDieR checklist only one article described the used techniques completely.

CONCLUSION

Manipulation or a mobilization interventions are poorly reported in RCTs, which jeopardize the external validity of RCTs, making it difficult for clinicians and researchers to replicate these interventions.

Augmenting the randomized controlled trial with real-world data to aid clinical decision making in metastatic renal cell carcinoma: a systematic review and meta-analysis

Aim: To evaluate how efficacy outcomes from real-world data (RWD) can support those from randomized controlled trials (RCTs), in the context of first-line tyrosine kinase inhibitor treatment of metastatic renal cell carcinoma. Patients & methods: PubMed, Ovid, MEDLINE and EMBASE were searched for RCTs and RWD studies with ≥50 adult patients per arm published in 2000–2017. Outcome measures were median progression-free survival, median overall survival and objective response rate. Results: A total of 13 RCTs and 22 RWD studies met eligibility criteria; 31, 28 and 25 studies, respectively, reported median progression-free survival, median overall survival and objective response rate. Summary outcome measures were similar in RWD and RCTs. Conclusion: RWD validates efficacy-based outcomes from RCTs and may provide supportive evidence to inform clinical decisions.

No differences were found between effect estimates from conventional and registry-based randomized controlled trials

OBJECTIVES

The study aims to assess whether the results from registry-based randomized controlled trials (RRCTs) systematically differ from the results of conventional randomized controlled trials (CRCTs).

STUDY DESIGN

The study was meta-epidemiological study. We identified RRCTs (February 2016) and subsequent systematic reviews (SRs) that included one RRCT (04/2017). We calculated pooled odds ratios for RRCTs and CRCTs for mortality and other incidence measures (e.g., cardiovascular events). We assessed the agreement between RRCTs and CRCTs using various measures with descriptive statistics and the odds ratio of pooled odds ratios (OROR) for RRCT(s) vs. CRCTs. An OROR of >1 indicates that the effect estimates from RRCTs were larger than the effects estimates from CRCTs.

RESULTS

Overall, we compared 15 and 14 effect estimates for mortality and incidence measures, respectively. The 95% confidence interval (CI) overlap was 100% for any outcome. Conflicting effect directions were observed in 27% (4/15) of mortality and 7% (1/14) of incidence measures. The ORORs for RRCT(s) vs. CRCTs was 1.03 (95% CI 0.97-1.09) for mortality and 1.05 (95% CI 0.98-1.12) for other incidence measures.

CONCLUSIONS

Our analysis indicates that for objective outcomes, there is no systematic difference between effect estimates from RRCTs and CRCTs.

Generalizing Study Results: A Potential Outcomes Perspective

Great care is taken in epidemiologic studies to ensure the internal validity of causal effect estimates; however, external validity has received considerably less attention. When the study sample is not a random sample of the target population, the sample average treatment effect, even if internally valid, cannot usually be expected to equal the average treatment effect in the target population. The utility of an effect estimate for planning purposes and decision making will depend on the degree of departure from the true causal effect in the target population due to problems with both internal and external validity. Herein, we review concepts from recent literature on generalizability, one facet of external validity, using the potential outcomes framework. Identification conditions sufficient for external validity closely parallel identification conditions for internal validity, namely conditional exchangeability; positivity; the same distributions of the versions of treatment; no interference; and no measurement error. We also require correct model specification. Under these conditions, we discuss how a version of direct standardization (the g-formula, adjustment formula, or transport formula) or inverse probability weighting can be used to generalize a causal effect from a study sample to a well-defined target population, and demonstrate their application in an illustrative example.

A prospective cohort and extended comprehensive-cohort design provided insights about the generalizability of a pragmatic trial: the ProtecT prostate cancer trial

OBJECTIVES

Randomized controlled trials (RCTs) deliver robust internally valid evidence but generalizability is often neglected. Design features built into the Prostate testing for cancer and Treatment (ProtecT) RCT of treatments for localized prostate cancer (PCa) provided insights into its generalizability.

STUDY DESIGN AND SETTING

Population-based cluster randomization created a prospective study of prostate-specific antigen (PSA) testing and a comprehensive-cohort study including groups choosing treatment or excluded from the RCT, as well as those randomized. Baseline information assessed selection and response during RCT conduct.

RESULTS

The prospective study (82,430 PSA-tested men) represented healthy men likely to respond to a screening invitation. The extended comprehensive cohort comprised 1,643 randomized, 997 choosing treatment, and 557 excluded with advanced cancer/comorbidities. Men choosing treatment were very similar to randomized men except for having more professional/managerial occupations. Excluded men were similar to the randomized socio-demographically but different clinically, representing less healthy men with more advanced PCa.

CONCLUSION

The design features of the ProtecT RCT provided data to assess the representativeness of the prospective cohort and generalizability of the findings of the RCT. Greater attention to collecting data at the design stage of pragmatic trials would better support later judgments by clinicians/policy-makers about the generalizability of RCT findings in clinical practice.

Ledipasvir/Sofosbuvir Effectively Treats Hepatitis C Virus Infections in an Underserved Population

BACKGROUND

Underserved populations have an unequal burden of HCV infections and poor outcomes with interferon-based treatments. Direct-acting antivirals have the potential to reduce these inequalities.

AIMS

We aimed to estimate sustained virologic response (SVR) following treatment with sofosbuvir-based regimens for HCV infections among underserved individuals and summarize the frequency of SVR across published studies of underserved populations.

METHODS

We used data from a clinical cohort of patients aged ≥ 18 years who initiated sofosbuvir-based regimens for HCV infection between February 2014 and June 2016 at an urban public hospital network that serves as the healthcare safety-net for Tarrant County, Texas. We estimated SVR with corresponding 95% confidence limits (CL). In addition, we systematically reviewed the evidence to identify other studies of direct-acting antivirals among underserved populations.

RESULTS

Our study population comprised 435 patients. The majority of patients were aged ≥ 50 years (76%), male (52%), non-Hispanic White (54%), HCV genotype 1 (79%) and treated with ledipasvir/sofosbuvir (69%). Overall SVR was 89% (95% CL 86, 92%) and highest for ledipasvir/sofosbuvir (SVR = 95%, 95% CL 92, 97%). The reported SVR following direct-acting antivirals among 837 underserved patients from three other studies ranged between 90 and 99%.

CONCLUSIONS

Our results suggest that direct-acting antivirals, particularly ledipasvir/sofosbuvir, are generally effective for achieving SVR among underserved patients with HCV infections and may help reduce inequalities in HCV prevalence and outcomes for this vulnerable population.

Advancing 'real-world' trials that take account of social context and human volition

Background

The recent paper in Trials by Porter and colleagues highlights the utility of applying a critical realism approach in randomised trials, an approach central to the Medical Research Council’s (MRC) Framework for the Development and Evaluation of Complex Healthcare Interventions. The MRC framework offers a pragmatic step towards a more open systems approach that bridges randomised evaluation with social context and human agency in an effort to improve the generalisability of trial outcomes.

Main body

The MRC framework has contributed to the proliferation of a more open systems approach in health research; however, the broader acceptance of the realist approach to health research does not seem to be emulated by norms in research fund allocation, which largely prioritises laboratory-based research.

Conclusion

This commentary is simply a plea, to those who make the strategic decisions regarding allocation of research funding, to support all phases of health intervention research in complex systems that contribute to the development of effective, translational and sustainable interventions in the promotion of health.

A bias-adjusted evidence synthesis of RCT and observational data: the case of total hip replacement

Evaluation of clinical effectiveness of medical devices differs in some aspects from the evaluation of pharmaceuticals. One of the main challenges identified is lack of robust evidence and a will to make use of experimental and observational studies (OSs) in quantitative evidence synthesis accounting for internal and external biases. Using a case study of total hip replacement to compare the risk of revision of cemented and uncemented implant fixation modalities, we pooled treatment effect estimates from OS and RCTs, and simplified existing methods for bias-adjusted evidence synthesis to enhance practical application. We performed an elicitation exercise using methodological and clinical experts to determine the strength of beliefs about the magnitude of internal and external bias affecting estimates of treatment effect. We incorporated the bias-adjusted treatment effects into a generalized evidence synthesis, calculating both frequentist and Bayesian statistical models. We estimated relative risks as summary effect estimates with 95% confidence/credibility intervals to capture uncertainty. When we compared alternative approaches to synthesizing evidence, we found that the pooled effect size strongly depended on the inclusion of observational data as well as on the use bias-adjusted estimates. We demonstrated the feasibility of using observational studies in meta-analyses to complement RCTs and incorporate evidence from a wider spectrum of clinically relevant studies and healthcare settings. To ensure internal validity, OS data require sufficient correction for confounding and selection bias, either through study design and primary analysis, or by applying post-hoc bias adjustments to the results. © 2017 The Authors. Health Economics published by John Wiley & Sons, Ltd.

Randomised trials comparing different healthcare settings: an exploratory review of the impact of pre-trial preferences on participation, and discussion of other methodological challenges

BACKGROUND

We recently published a systematic review of different healthcare settings (such as outpatient, community or home) for administering intravenous chemotherapy, and concluded that performing conventionally designed randomised trials was difficult. The main problems were achieving adequate trial accrual rates and recruiting a study population which adequately represented the target population of interest. These issues stemmed from the fact that potential participants may have had pre-trial perceptions about the trial settings they may be allocated; such preferences will sometimes be strong enough for patients to decline an invitation to participate in a trial. A patient preference trial design (in which patients can choose, or be randomised to, an intervention) may have obviated these recruitment issues, although none of the trials used such a design.

METHODS

In order to gain a better understanding of the broader prevalence and extent of these preference issues (and any other methodological challenges), we undertook an exploratory review of settings trials in any area of healthcare treatment research. We searched The Cochrane Library and Google Scholar and used snowballing methods to identify trials comparing different healthcare settings.

RESULTS

Trial accrual was affected by patient preferences for a setting in 15 of the 16 identified studies; birth setting trials were the most markedly affected, with between 68 % and 85 % of eligible women declining to participate specifically because of preference for a particular healthcare setting. Recruitment into substance abuse and chemotherapy setting studies was also notably affected by preferences. Only four trials used a preference design: the proportion of eligible patients choosing to participate via a preference group ranged from between 33 % and 67 %.

CONCLUSIONS

In trials of healthcare settings, accrual may be seriously affected by patient preferences. The use of trial designs which incorporate a preference component should therefore strongly be considered. When designing such trials, investigators should consider settings to be complex interventions, which are likely to have linked components which may be difficult to control for. Careful thought is also needed regarding the choice of comparator settings and the most appropriate outcome measures to be used.

Systematic review and meta-analysis of randomised controlled trials on the effects of potassium supplements on serum potassium and creatinine

OBJECTIVES

High potassium intake could prevent stroke, but supplementation is considered hazardous. We assessed the effect of oral potassium supplementation on serum or plasma potassium levels and renal function.

SETTING

We updated a systematic review of the effects of potassium supplementation in randomised clinical trials carried out worldwide, published in 2013, extending it to July 2015. We followed the PRISMA guidelines.

PARTICIPANTS

Any individual taking part in a potassium supplementation randomised clinical trial. Studies included met the following criteria: randomised clinical trials, potassium supplement given and circulating potassium levels reported.

INTERVENTION

Oral potassium supplementation.

PRIMARY OUTCOME MEASURES

Serum or plasma potassium and serum or plasma creatinine.

RESULTS

A total of 20 trials (21 independent groups) were included (1216 participants from 12 different countries). All but 2 were controlled (placebo n=16, control n=2). Of these trials, 15 were crossover, 4 had a parallel group and 1 was sequential. The duration of supplementation varied from 2 to 24 weeks and the amount of potassium given from 22 to 140 mmol/day. In the pooled analysis, potassium supplementation caused a small but significant increase in circulating potassium levels (weighted mean difference (WMD) 0.14 mmol/L, 95% CI 0.09 to 0.19, p<1×10(-5)), not associated with dose or duration of treatment. The average increase in urinary potassium excretion was 45.75 mmol/24 hours, 95% CI 38.81 to 53.69, p<1×10(-5). Potassium supplementation did not cause any change in circulating creatinine levels (WMD 0.30 µmol/L, 95% CI -1.19 to 1.78, p=0.70).

CONCLUSIONS

In short-term studies of relatively healthy persons, a moderate oral potassium supplement resulted in a small increase in circulating potassium levels and no change in renal function.

The Importance of Computer Science for Public Health Training: An Opportunity and Call to Action

A century ago, the Welch-Rose Report established a public health education system in the United States. Since then, the system has evolved to address emerging health needs and integrate new technologies. Today, personalized health technologies generate large amounts of data. Emerging computer science techniques, such as machine learning, present an opportunity to extract insights from these data that could help identify high-risk individuals and tailor health interventions and recommendations. As these technologies play a larger role in health promotion, collaboration between the public health and technology communities will become the norm. Offering public health trainees coursework in computer science alongside traditional public health disciplines will facilitate this evolution, improving public health's capacity to harness these technologies to improve population health.

Toward phase 4 trials in heart failure: A social and corporate responsibility of the medical profession

Congestive heart failure (CHF) is a chronic condition, requiring polypharmacy, allied health supports and regular monitoring. All these factors are needed to ensure compliance and to deliver the positive outcomes demonstrated from randomized controlled trials. Unfortunately many centers around the world are unable to match trial level support. The outcomes for many communities are thus unclear. Research design factors in post-marketing surveillance to address this issue. Phase 4 studies is the name given to trials designed to obtain such community level data and thus address issues of external validity. CHF phase 4 studies are relatively underutilized. We feel the onus for this research lies with the health profession. In this commentary we provide arguments as to why phase 4 studies should be viewed as a social and corporate responsibility of health professional that care for clients with CHF.

Cause of death in patients with atrial fibrillation admitted to French hospitals in 2012: a nationwide database study

Objective

Most patients with atrial fibrillation (AF) have risk factors and coexisting conditions that increase their mortality risk. We performed a cause-of-death analysis to identify predictors of mortality in hospitalised patients with AF in France.

Methods and results

In this retrospective, population-based cross-sectional study, the Programme de médicalisation des systèmes d'information was used to identify 533 044 adults with a diagnosis of AF or atrial flutter hospitalised for any reason in France from January through December 2012. Stepwise multivariable analyses were performed to identify determinants of mortality. The mean age was 78.0±11.4 years, 47.1% were women, and the mean CHA₂DS₂-VASc score was 4.0±1.8. During hospitalisation, 9.4% (n=50 165) of the patients died, 34% due to a cardiovascular event, most often heart failure (16.6%), stroke/transient ischaemic attack/systemic embolism (9.8%) or vascular or ischaemic disease (4.0%). The strongest predictors of overall death were age ≥75 years (OR 2.57, 95% CI 2.47 to 2.68), renal failure (OR 1.85, 95% CI 1.81 to 1.89), cancer (OR 1.81, 95% CI 1.78 to 1.85) and lung disease (OR 1.58, 95% CI 1.55 to 1.62).

Conclusions

Cardiovascular events were the most common cause of death, occurring in one-third of patients, in this comprehensive study of hospitalised patients with AF. Despite the high risk of stroke in this population, only 10% died from stroke/transient ischaemic attack/systemic embolism. The strongest predictors of overall death were non-cardiovascular. Physicians should be encouraged to focus on preventable serious and disabling cardiovascular events (such as stroke) as well as on potentially fatal non-cardiovascular comorbidities.

Randomised trials in context: practical problems and social aspects of evidence-based medicine and policy

Randomised trials can provide excellent evidence of treatment benefit in medicine. Over the last 50 years, they have been cemented in the regulatory requirements for the approval of new treatments. Randomised trials make up a large and seemingly high-quality proportion of the medical evidence-base. However, it has also been acknowledged that a distorted evidence-base places a severe limitation on the practice of evidence-based medicine (EBM). We describe four important ways in which the evidence from randomised trials is limited or partial: the problem of applying results, the problem of bias in the conduct of randomised trials, the problem of conducting the wrong trials and the problem of conducting the right trials the wrong way. These problems are not intrinsic to the method of randomised trials or the EBM philosophy of evidence; nevertheless, they are genuine problems that undermine the evidence that randomised trials provide for decision-making and therefore undermine EBM in practice. Finally, we discuss the social dimensions of these problems and how they highlight the indispensable role of judgement when generating and using evidence for medicine. This is the paradox of randomised trial evidence: the trials open up expert judgment to scrutiny, but this scrutiny in turn requires further expertise.

Exploring threats to generalisability in a large international rehabilitation trial (AVERT)

OBJECTIVE

The purpose of this paper is to examine potential threats to generalisability of the results of a multicentre randomised controlled trial using data from A Very Early Rehabilitation Trial (AVERT).

DESIGN

AVERT is a prospective, parallel group, assessor-blinded randomised clinical trial. This paper presents data assessing the generalisability of AVERT.

SETTING

Acute stroke units at 44 hospitals in 8 countries.

PARTICIPANTS

The first 20,000 patients screened for AVERT, of whom 1158 were recruited and randomised.

MODEL

We use the Proximal Similarity Model, which considers the person, place, and setting and practice, as a framework for considering generalisability. As well as comparing the recruited patients with the target population, we also performed an exploratory analysis of the demographic, clinical, site and process factors associated with recruitment.

RESULTS

The demographics and stroke characteristics of the included patients in the trial were broadly similar to population-based norms, with the exception that AVERT had a greater proportion of men. The most common reason for non-recruitment was late arrival to hospital (ie, >24 h). Overall, being older and female reduced the odds of recruitment to the trial. More women than men were excluded for most of the reasons, including refusal. The odds of exclusion due to early deterioration were particularly high for those with severe stroke (OR=10.4, p<0.001, 95% CI 9.27 to 11.65).

CONCLUSIONS

A model which explores person, place, and setting and practice factors can provide important information about the external validity of a trial, and could be applied to other clinical trials.

TRIAL REGISTRATION NUMBER

Australian New Zealand Clinical Trials Registry (ACTRN12606000185561) and Clinicaltrials.gov (NCT01846247).

Lymphoma incidence, survival and prevalence 2004-2014: sub-type analyses from the UK's Haematological Malignancy Research Network

BACKGROUND

Population-based information about cancer occurrence and survival are required to inform clinical practice and research; but for most lymphomas data are lacking.

METHODS

Set within a socio-demographically representative UK population of nearly 4 million, lymphoma data (N=5796) are from an established patient cohort.

RESULTS

Incidence, survival (overall and relative) and prevalence estimates for >20 subtypes are presented. With few exceptions, males tended to be diagnosed at younger ages and have significantly (P<0.05) higher incidence rates. Differences were greatest at younger ages: the <15 year male/female rate ratio for all subtypes combined being 2.2 (95% CI 1.3-3.4). These gender differences impacted on prevalence; most subtype estimates being significantly (P<0.05) higher in males than females. Outcome varied widely by subtype; survival of patients with nodular lymphocyte predominant Hodgkin lymphoma approached that of the general population, whereas less than a third of those with other B-cell (e.g., mantle cell) or T-cell (e.g., peripheral-T) lymphomas survived for ≥5 years. No males/female survival differences were detected.

CONCLUSIONS

Major strengths of our study include completeness of ascertainment, world-class diagnostics and generalisability. The marked variations demonstrated confirm the requirement for 'real-world' data to inform aetiological hypotheses, health-care planning and the future monitoring of therapeutic changes.

Economic evaluations in bipolar disorder: a systematic review and critical appraisal

BACKGROUND

Bipolar disorder (BD) is a chronic illness and is one of the worldwide leading causes of disability. It is often a lifelong illness and incurs a substantial economic burden on the health care system, the patients, and society as a whole. However, there are few studies evaluating the economic impact of alternative strategies in the management of BD.

OBJECTIVES

We reviewed and critically appraised the available published economic evidence on BD management. In addition, we explored advantages and disadvantages of different methods used in the economic evaluation of the management of BD.

METHODS

A systematic literature search was undertaken using seven electronic databases to identify all English language articles published between January 1980 and March 2012 that provided data on complete economic evaluations for any treatment strategy for BD. The quality of included studies was appraised according to recommendations from the Cochrane Collaboration.

RESULTS

A total of 7,284 citations were obtained. After initial screening, 20 eligible studies were identified, five of which were trial-based, and 15 of which were model-based economic evaluations. Given the variability in methods and the quality of the identified studies, no conclusive recommendation for the most cost-effective therapy for BD could be provided.

CONCLUSIONS

The cost-effectiveness of different treatment strategies varied between settings, and transferability of these results across settings remains questionable. Although additional research using a longer time horizon is required to validate the findings for trial-based economic evaluations, discrete event simulation appears to be the most natural and plausible technique for modeling the cost-effectiveness of alternative BD treatment strategies.

ASSIST applicability scoring of surgical trials. an investigator-reported assessment tool

Context

We aimed to develop a new tool for assessing and depicting the applicability of the results of surgical randomized controlled trials (RCTs) from the trial investigators' perspective.

Methods

We identified all items related to applicability by a systematic methodological review, and then a sample of surgeons used these items in a web-based survey to evaluate the applicability of their own trial results. For each applicability item, participants had to indicate on a numerical scale that was simplified as a three-item scale: 1) items essential to consider, 2) items requiring attention, and 3) items inconsequential to the applicability of the results of their own RCT to clinical practice. For the final tool, we selected only items that were rated as being essential or requiring attention for at least 25% of the trials evaluated. We propose a specific process to construct the tool and to depict applicability in a graph. We identified all investigators of published and registered ongoing RCTs assessing surgery and invited them to participate in the web-based survey.

Results

148 surgeons assessed applicability for their own trial and participated in the process of item selection. The final tool contains 22 items (4 dedicated to patients, 5 to centers, 5 to surgeons and 8 to the intervention). We proposed a straightforward process of constructing the graphical tool: 1) a multidisciplinary team of investigators or other care providers participating in the trial could independently assess each item, 2) a consensus method could be used, and 3) the investigators could depict their assessment of the applicability of the trial results in 4 graphs related to patients, centers, surgeons and the intervention.

Conclusions

This investigator-reported assessment tool could help readers define under what conditions they could reasonably apply the results of a surgical RCT to their clinical practice.

Results from a blind and a non-blind randomised trial run in parallel: experience from the Estonian Postmenopausal Hormone Therapy (EPHT) Trial

BACKGROUND

The Estonian Postmenopausal Hormone Therapy (EPHT) Trial assigned 4170 potential participants prior to recruitment to blind or non-blind hormone therapy (HT), with placebo or non-treatment the respective alternatives. Before having to decide on participation, women were told whether they had been randomised to the blind or non-blind trial. Eligible women who were still willing to join the trial were recruited. After recruitment participants in the non-blind trial (N = 1001) received open-label HT or no treatment, participants in the blind trial (N = 777) remained blinded until the end of the trial. The aim of this paper is to analyse the effect of blinding on internal and external validity of trial outcomes.

METHODS

Effect of blinding was calculated as the hazard ratio of selected chronic diseases, total mortality and all outcomes. For analysing the effect of blinding on external validity, the hazard ratios from women recruited to the placebo arm and to the non-treatment arm were compared with those not recruited; for analysing the effect of blinding on internal validity, the hazard ratios from the blind trial were compared with those from the non-blind trial.

RESULTS

The women recruited to the placebo arm had less cerebrovascular disease events (HR 0.43; 95% CI: 0.26-0.71) and all outcomes combined (HR 0.76; 95% CI: 0.63-0.91) than those who were not recruited. Among women recruited or not recruited to the non-treatment arm, no differences were observed for any of the outcomes studied.Among women recruited to the trial, the risk for coronary heart disease events (HR 0.77; 95% CI: 0.64-0.93), cerebrovascular disease events (HR 0.66; 95%CI: 0.47-0.92), and all outcomes combined (HR 0.82; 95% CI: 0.72-0.94) was smaller among participants in the blind trial than in the non-blind trial. There was no difference between the blind and the non-blind trial for total cancer (HR 0.95; 95% CI: 0.64-1.42), bone fractures (0.93; 95% CI: 0.74-1.16), and total mortality (HR 1.03; 95% CI: 0.53-1.98).

CONCLUSIONS

The results from blind and non-blind trials may differ, even if the target population is the same. Blinding may influence both internal and external validity. The effect of blinding may vary for different outcome events.

TRIAL REGISTRATION

[ISRCTN35338757].

Generalizability of systematic reviews of the effectiveness of health care interventions to primary health care: concepts, methods and future research

BACKGROUND

There is a push to make clinical decision making more evidence based and patient oriented. However, current systematic reviews are limited by the lack of consideration of generalizability.

OBJECTIVE

To develop a guide for investigators (systematic reviewers) on how to adapt the methodology of a systematic review to facilitate the exploration of the results to primary care.

METHODS

We reviewed the method guidelines of five organizations and working groups. Then, we conducted a search of literature in PubMed for articles on external validity of systematic. Based on the literature, we developed general steps to incorporate generalizability in systematic reviews and then integrated main primary care themes as outlined in WONCA definition of primary care and the definition of person-centred medicine across those steps.

RESULTS

We identified a wide variety of approaches to address the issue of generalizability in systematic reviews. We extracted four major primary care themes from the two definitions of primary care and person-centred medicine that we mapped across the methods of generalizability. We suggested a two-step approach: Step 1 planning the scope of a review using four-layer schematic model and Step 2 to follow four steps to incorporate the concept of generalizability to primary care in the review.

CONCLUSIONS

Systematic reviewers can take several steps to improve the generalizability of their review to primary care. However, more research is needed to determine which approach provides better results than others.

Bringing values back into evidence-based nursing: the role of patients in resisting empiricism

We examine problems resulting from the narrow empirical focus associated with evidence-based nursing, including the deleterious influence of vested interests, disattention to patients' experiences, underestimation of the importance of social processes, lack of an individualized research perspective, marginalization of other forms of knowledge, and the undermining of patients' autonomy. Addressing each problem in turn, we argue that inclusion of patients at all stages of evidence-based practice can counter or ameliorate these problems. While we concede that patient involvement is not a complete solution to the problem of empiricism, it is the most effective means available to defend nursing values.