Treatment effects in randomised trials using routinely collected data for outcome assessment versus traditional trials: meta-research study

Effect of utilizing either a self-reported questionnaire or administrative data alone or in combination on the findings of a randomized controlled trial of the long-term effects of antenatal corticosteroids

INTRODUCTION

A combination of self-reported questionnaire and administrative data could potentially enhance ascertainment of outcomes and alleviate the limitations of both in follow up studies. However, it is uncertain how access to only one of these data sources to assess outcomes impact study findings. Therefore, this study aimed to determine whether the study findings would be altered if the outcomes were assessed by different data sources alone or in combination.

METHODS

At 50-year follow-up of participants in a randomized trial, we assessed the effect of antenatal betamethasone exposure on the diagnosis of diabetes, pre-diabetes, hyperlipidemia, hypertension, mental health disorders, and asthma using a self-reported questionnaire, administrative data, a combination of both, or any data source, with or without adjudication by an expert panel of five clinicians. Differences between relative risks derived from each data source were calculated using the Bland-Altman approach.

RESULTS

There were 424 participants (46% of those eligible, aged 49 years, SD 1, 50% male). There were no differences in study outcomes between participants exposed to betamethasone and those exposed to placebo when the outcomes were assessed using different data sources. When compared to the study findings determined using adjudicated outcomes, the mean difference (limits of agreement) in relative risks derived from other data sources were: self-reported questionnaires 0.02 (-0.35 to 0.40), administrative data 0.06 (-0.32 to 0.44), both questionnaire and administrative data 0.01 (-0.41 to 0.43), and any data source, 0.01 (-0.08 to 0.10).

CONCLUSION

Utilizing a self-reported questionnaire, administrative data, both questionnaire and administrative data, or any of these sources for assessing study outcomes had no impact on the study findings compared with when study outcomes were assessed using adjudicated outcomes.

Evidence base for yearly respiratory virus vaccines: Current status and proposed improved strategies

Annual vaccination is widely recommended for influenza and SARS-CoV-2. In this essay, we analyse and question the prevailing policymaking approach to these respiratory virus vaccines, especially in the United States. Every year, licensed influenza vaccines are reformulated to include specific strains expected to dominate in the season ahead. Updated vaccines are rapidly manufactured and approved without further regulatory requirement of clinical data. Novel vaccines (i.e. new products) typically undergo clinical trials, though generally powered for clinically unimportant outcomes (e.g. lab-confirmed infections, regardless of symptomatology or antibody levels). Eventually, the current and future efficacy of influenza and COVID-19 vaccines against hospitalization or death carries considerable uncertainty. The emergence of highly transmissible SARS-CoV-2 variants and waning vaccine-induced immunity led to plummeting vaccine effectiveness, at least against symptomatic infection, and booster doses have since been widely recommended. No further randomized trials were performed for clinically important outcomes for licensed updated boosters. In both cases, annual vaccine effectiveness estimates are generated by observational research, but observational studies are particularly susceptible to confounding and bias. Well-conducted experimental studies, particularly randomized trials, are necessary to address persistent uncertainties about influenza and COVID-19 vaccines. We propose a new research framework which would render results relevant to the current or future respiratory viral seasons. We demonstrate that experimental studies are feasible by adopting a more pragmatic approach and provide strategies on how to do so. When it comes to implementing policies that seriously impact people's lives, require substantial public resources and/or rely on widespread public acceptance, high evidence standards are desirable.

Comparison of outcomes of the 50-year follow-up of a randomized trial assessed by study questionnaire and by data linkage: The CONCUR study

BACKGROUND/AIMS

Self-reported questionnaires on health status after randomized trials can be time-consuming, costly, and potentially unreliable. Administrative data sets may provide cost-effective, less biased information, but it is uncertain how administrative and self-reported data compare to identify chronic conditions in a New Zealand cohort. This study aimed to determine whether record linkage could replace self-reported questionnaires to identify chronic conditions that were the outcomes of interest for trial follow-up.

METHODS

Participants in 50-year follow-up of a randomized trial were asked to complete a questionnaire and to consent to accessing administrative data. The proportion of participants with diabetes, pre-diabetes, hyperlipidaemia, hypertension, mental health disorders, and asthma was calculated using each data source and agreement between data sources assessed.

RESULTS

Participants were aged 49 years (SD = 1, n = 424, 50% male). Agreement between questionnaire and administrative data was slight for pre-diabetes (kappa = 0.10), fair for hyperlipidaemia (kappa = 0.27), substantial for diabetes (kappa = 0.65), and moderate for other conditions (all kappa >0.42). Administrative data alone identified two to three times more cases than the questionnaire for all outcomes except hypertension and mental health disorders, where the questionnaire alone identified one to two times more cases than administrative data. Combining all sources increased case detection for all outcomes.

CONCLUSIONS

A combination of questionnaire, pharmaceutical, and laboratory data with expert panel review were required to identify participants with chronic conditions of interest in this follow-up of a clinical trial.

Validation and impact of algorithms for identifying variables in observational studies of routinely collected data

BACKGROUND AND OBJECTIVES

Among observational studies of routinely collected health data (RCD) for exploring treatment effects, algorithms are used to identify study variables. However, the extent to which algorithms are reliable and impact the credibility of effect estimates is far from clear. This study aimed to investigate the validation of algorithms for identifying study variables from RCD, and examine the impact of alternative algorithms on treatment effects.

METHODS

We searched PubMed for observational studies published in 2018 that used RCD to explore drug treatment effects. Information regarding the reporting, validation, and interpretation of algorithms was extracted. We summarized the reporting and methodological characteristics of algorithms and validation. We also assessed the divergence in effect estimates given alternative algorithms by calculating the ratio of estimates of the primary vs. alternative analyses.

RESULTS

A total of 222 studies were included, of which 93 (41.9%) provided a complete list of algorithms for identifying participants, 36 (16.2%) for exposure, and 132 (59.5%) for outcomes, and 15 (6.8%) for all study variables including population, exposure, and outcomes. Fifty-nine (26.6%) studies stated that the algorithms were validated, and 54 (24.3%) studies reported methodological characteristics of 66 validations, among which 61 validations in 49 studies were from the cross-referenced validation studies. Of those 66 validations, 22 (33.3%) reported sensitivity and 16 (24.2%) reported specificity. A total of 63.6% of studies reporting sensitivity and 56.3% reporting specificity used test-result-based sampling, an approach that potentially biases effect estimates. Twenty-eight (12.6%) studies used alternative algorithms to identify study variables, and 24 reported the effects estimated by primary analyses and sensitivity analyses. Of these, 20% had differential effect estimates when using alternative algorithms for identifying population, 18.2% for identifying exposure, and 45.5% for classifying outcomes. Only 32 (14.4%) studies discussed how the algorithms may affect treatment estimates.

CONCLUSION

In observational studies of RCD, the algorithms for variable identification were not regularly validated, and-even if validated-the methodological approach and performance of the validation were often poor. More seriously, different algorithms may yield differential treatment effects, but their impact is often ignored by researchers. Strong efforts, including recommendations, are warranted to improve good practice.

Harnessing the potential of data-driven strategies to optimise transfusion practice

No one doubts the significant variation in the practice of transfusion medicine. Common examples are the variability in transfusion thresholds and the use of tranexamic acid for surgery with likely high blood loss despite evidence-based standards. There is a long history of applying different strategies to address this variation, including education, clinical guidelines, audit and feedback, but the effectiveness and cost-effectiveness of these initiatives remains unclear. Advances in computerised decision support systems and the application of novel electronic capabilities offer alternative approaches to improving transfusion practice. In England, the National Institute for Health and Care Research funded a Blood and Transplant Research Unit (BTRU) programme focussing on 'A data-enabled programme of research to improve transfusion practices'. The overarching aim of the BTRU is to accelerate the development of data-driven methods to optimise the use of blood and transfusion alternatives, and to integrate them within routine practice to improve patient outcomes. One particular area of focus is implementation science to address variation in practice.

Reporting, handling, and interpretation of time-varying drug treatments in observational studies using routinely collected healthcare data

BACKGROUND

Time-varying drug treatments are common in studies using routinely collected health data (RCD) for assessing treatment effects. This study aimed to examine how these studies reported, handled, and interpreted time-varying drug treatments.

METHODS

A systematic search was conducted on PubMed from 2018 to 2020. Eligible studies were those used RCD to explore drug treatment effects. We summarized the reporting characteristics and methods employed for handling time-varying treatments. Logistic regressions were performed to investigate the association between study characteristics and the reporting of time-varying treatments.

RESULTS

Two hundred and fifty-six studies were included, and 225 (87.9%) studies involved time-varying treatments. Of these, 24 (10.7%) reported the proportion of time-varying treatments and 105 (46.7%) reported methods used to handle time-varying treatments. Multivariable logistic regression showed that medical studies, prespecified protocol, and involvement of methodologists were associated with a higher likelihood of reporting the methods applied to handle time-varying treatments. Among the 105 studies that reported methods, as-treated analyses were the most commonly used analysis sets, which were employed in 73.9%, 75.3% and 88.2% of studies that reported approaches for treatment discontinuation, treatment switching and treatment add-on. Among the 225 studies involved time-varying treatments, 27 (12.0%) acknowledged the potential bias introduced by treatment change, of which 14 (51.9%) suggested that potential biases may impact acceptance or rejection of the null hypothesis.

CONCLUSIONS

Among observational studies using RCD, the underreporting about the presence and methods for handling time-varying treatments was largely common. The potential biases due to time-varying treatments have frequently been disregarded. Collaborative endeavors are strongly needed to enhance the prevailing practices.

Using novel data linkage of congenital heart disease biobank data with administrative health data to identify cardiovascular outcomes to inform genomic analysis

Introduction

Contemporary care of congenital heart disease (CHD) is largely standardised, however there is heterogeneity in post-surgical outcomes that may be explained by genetic variation. Data linkage between a CHD biobank and routinely collected administrative datasets is a novel method to identify outcomes to explore the impact of genetic variation.

Objective

Use data linkage to identify and validate patient outcomes following surgical treatment for CHD.

Methods

Data linkage between clinical and biobank data of children born from 2001-2014 that had a procedure for CHD in New South Wales, Australia, with hospital discharge data, education and death data. The children were grouped according to CHD lesion type and age at first cardiac surgery. Children in each 'lesion/age at surgery group' were classified into 'favourable' and 'unfavourable' cardiovascular outcome groups based on variables identified in linked administrative data including; total time in intensive care, total length of stay in hospital, and mechanical ventilation time up to 5 years following the date of the first cardiac surgery. A blind medical record audit of 200 randomly chosen children from 'favourable' and 'unfavourable' outcome groups was performed to validate the outcome groups.

Results

Of the 1872 children in the dataset that linked to hospital or death data, 483 were identified with a 'favourable' cardiovascular outcome and 484 were identified as having a 'unfavourable' cardiovascular outcome. The medical record audit found concordant outcome groups for 182/192 records (95%) compared to the outcome groups categorized using the linked data.

Conclusions

The linkage of a curated biobank dataset with routinely collected administrative data is a reliable method to identify outcomes to facilitate a large-scale study to examine genetic variance. These genetic hallmarks could be used to identify patients who are at risk of unfavourable cardiovascular outcomes, to inform strategies for prevention and changes in clinical care.

Data source profile reporting by studies that use routinely collected health data to explore the effects of drug treatment

BACKGROUND

Routinely collected health data (RCD) are important resource for exploring drug treatment effects. Adequate reporting of data source profiles may increase the credibility of evidence generated from these data. This study conducted a systematic literature review to evaluate the reporting characteristics of databases used by RCD studies to explore the effects of drug treatment.

METHODS

Observational studies published in 2018 that used RCD to explore the effects of drug treatment were identified by searching PubMed. We categorized eligible reports into two groups by journal impact factor (IF), including the top 5 general medical journals (NEJM, Lancet, JAMA, BMJ and JAMA Internal Medicine) and the other journals. The reporting characteristics of the databases used were described and compared between the two groups and between studies citing and not citing database references.

RESULTS

A total of 222 studies were included, of which 53 (23.9%) reported that they applied data linkage, 202 (91.0%) reported the type of database, and 211 (95.0%) reported the coverage of the data source. Only 81 (36.5%) studies reported the timeframe of the database. Studies in high-impact journals were more likely to report that they applied data linkage (65.1% vs. 20.2%) and used electronic medical records (EMR) (73.7% vs. 30.0%) and national data sources (77.8% vs. 51.3%) than those published in other medical journals. There were 137/222 (61.7%) cited database references. Studies with database-specific citations had better reporting of the data sources and were more likely to publish in high-impact journals than those without (mean IF, 6.08 vs. 4.09).

CONCLUSIONS

Some deficits were found in the reporting quality of databases in studies that used RCD to explore the effects of drug treatment. Studies citing database-specific references may provide detailed information regarding data source characteristics. The adoption of reporting guidelines and education on their use is urgently needed to promote transparency by research groups.

High variability of COVID-19 case fatality rate in Germany

BACKGROUND

During the first wave of the COVID-19 pandemic a high case fatality rate (CFR) was noticed worldwide including also Germany where the first European cases have been observed. The WHO recommended immediate intubation for patients with dyspnoea which has since been revised after reviewing the initial clinical outcome. The objective of this study is to analyze CFR and assess if there is an advantage of a more conservative management of COVID-19 induced hypoxemia.

METHODS

PCR confirmed COVID-19 infections and death counts were obtained for all German districts from 27 Jan 2020 until 15 Feb 2021 using official reports of Robert Koch Institute Berlin, Germany. A moving average CFRt was constructed by dividing disease related deaths two weeks after a given index day by the number of infections two weeks prior to that date. In addition to a local comparison also mortality outcomes in other German speaking countries were compared.

RESULTS

The mean CFR is estimated to be 2.92% based on 71.965 fatalities and 2.465.407 cases. There was a large regional scattering of CFRs across the German districts. Differences of the mortality pattern were observed also at state level and preserved across different sex and age groups while being largely independent of case numbers. Although Munich city had higher infection rates, more patients died during the first wave in Hamburg (OR 1.6, 95% CI 1.3-1.9) which was mirrored also by higher death risk at Hamburg intensive care units (OR 2.0, 95% CI 1.3-3.1). While the majority of Munich hospitals favoured a conservative management of hypoxemia including high flow nasal cannula (HFNC), Hamburg hospitals followed a more aggressive scheme of early mechanical ventilation (MV). Austria and Switzerland experienced higher CFRs than Germany during the first wave but after changing their treatment guidelines, both countries experienced lower CFRs during the second wave.

CONCLUSION

Using retrospective public health data, different case fatality rates have been observed across Germany. A more conservative management of COVID-19 induced Adult Respiratory Distress Syndrome (ARDS) is justified also by epidemiological data.

Impact of design characteristics among studies comparing coronary computed tomography angiography to noninvasive functional testing in chronic coronary syndromes

BACKGROUND

Coronary computed tomography angiography (CCTA) is widely adopted to detect obstructive coronary artery disease (CAD) in patients with chronic coronary syndromes (CCS). However, it is unknown to which extent study-specific characteristics yield different conclusions.

METHODS

We summarized non-randomized and randomized studies comparing CCTA and noninvasive functional testing for CCS with information on the outcome of myocardial infarction (MI). We evaluated the differential effect according to study characteristics using random-effect meta-analysis with Hartung-Knapp-Sidik-Jonkman adjustments.

RESULTS

Fifteen studies (8 non-randomized, 7 randomized) were included. CCTA was associated with decrease in relative (odds ratio (OR) 0.54, 95%CI 0.47 to 0.62, P < .001) and absolute MI risk (risk difference (RD) -0.4%, 95%CI -0.6 to -0.1, P = .005). The results remained consistent among the non-randomized (RD -0.4%, 95%CI -0.7 to -0.1, P=.029), but not among the randomized trials where there was no difference in the observed risk (RD 0.2%, 95%CI -0.6 to 0.1, P = .158). CCTA was not associated with MI reduction in studies with clinical outcome definition (OR 0.77, 95%CI 0.41 to 1.44, P = .212), research driven follow-up (OR 0.54, 95%CI 0.24 to 1.21, P = .090), central event assessment (OR 0.63, 95%CI 0.21 to 1.86, P = .207), outcome adjudication (OR 0.74, 95%CI 0.24 to 2.23, P = .178), or at low-risk of bias (OR 0.74, 95%CI 0.24 to 2.23, P = .178).

CONCLUSIONS

Among studies of any design, CCTA was associated with lower risk of MI in CCS compared to noninvasive functional testing. This benefit was diminished among studies with clinical outcome definition, central outcome assessment/adjudication or at low-risk of bias.

Randomised controlled trials in hand surgery: a scoping review

OBJECTIVES

To identify the evidence gaps that exist regarding the efficacy or effectiveness of hand surgery.

SETTING

A scoping review. We systematically searched MEDLINE, Embase and CENTRAL databases to identify all hand surgical randomised controlled trials from inception to 7 November 2020.

RESULTS

Of the 220 identified randomised controlled trials, none were fundamental efficacy trials, that is, compared surgery with placebo surgery. 172 (78%) trials compared the outcomes of different surgical techniques, and 143 (65%) trials were trauma related. We identified only 47 (21%) trials comparing surgery with non-operative care or injection.

CONCLUSION

The evidence supporting use of surgery especially for chronic hand conditions is scarce. To determine optimal care for people with hand conditions, more resources should be aimed at placebo-controlled trials and pragmatic effectiveness trials comparing hand surgery with non-operative care.

PROSPERO REGISTRATION NUMBER

CRD42019122710.

Impact of systematic medication review in emergency department on patients’ post-discharge outcomes—A randomized controlled clinical trial

Introduction

The main objective of this study was to investigate whether systematic medication review conducted by clinical pharmacists can impact clinical outcomes and post-discharge outcomes for patients admitted to the emergency department.

Method

This parallel group, non-blinded, randomized controlled trial was conducted in the emergency department, Diakonhjemmet Hospital, Oslo, Norway. The study was registered in ClinicalTrials.gov, Identifier: NCT03123640 in April 2017. From April 2017 to May 2018, patients ≥18 years were included and randomized (1:1) to intervention- or control group. The control group received standard care from emergency department physicians and nurses. In addition to standard care, the intervention group received systematic medication review including medication reconciliation conducted by pharmacists, during the emergency department stay. The primary outcome was proportion of patients with an unplanned contact with hospital within 12 months from inclusion stay discharge.

Results

In total, 807 patients were included and randomized, 1:1, to intervention or control group. After excluding 8 patients dying during hospital stay and 10 patients lacking Norwegian personal identification number, the primary analysis comprised 789 patients: 394 intervention group patients and 395 control group patients. Regarding the primary outcome, there was no significant difference in proportion of patients with an unplanned contact with hospital within 12 months after inclusion stay discharge between groups (51.0% of intervention group patients vs. 53.2% of control group patients, p = 0.546).

Conclusion

As currently designed, emergency department pharmacist-led medication review did not significantly influence clinical- or post-discharge outcomes. This study did, however pinpoint important practical implementations, which can be used to design tailored pharmacist-led interventions and workflow regarding drug-related issues in the emergency department setting.

Systematic approach to outcome assessment from coded electronic healthcare records in the DaRe2THINK NHS-embedded randomized trial. EUROPEAN HEART JOURNAL. DIGITAL HEALTH 2022;3:426-436. [PMID: 36712153 PMCID: PMC9708037 DOI: 10.1093/ehjdh/ztac046]

Aims

Improving the efficiency of clinical trials is key to their continued importance in directing evidence-based patient care. Digital innovations, in particular the use of electronic healthcare records (EHRs), allow for large-scale screening and follow up of participants. However, it is critical these developments are accompanied by robust and transparent methods that can support high-quality and high clinical value research.

Methods and results

The DaRe2THINK trial includes a series of novel processes, including nationwide pseudonymized pre screening of the primary-care EHR across England, digital enrolment, remote e-consent, and 'no-visit' follow up by linking all primary- and secondary-care health data with patient-reported outcomes. DaRe2THINK is a pragmatic, healthcare-embedded randomized trial testing whether earlier use of direct oral anticoagulants in patients with prior or current atrial fibrillation can prevent thromboembolic events and cognitive decline (www.birmingham.ac.uk/dare2think). This study outlines the systematic approach and methodology employed to define patient information and outcome events. This includes transparency on all medical code lists and phenotypes used in the trial across a variety of national data sources, including Clinical Practice Research Datalink Aurum (primary care), Hospital Episode Statistics (secondary care), and the Office for National Statistics (mortality).

Conclusion

Co-designed by a patient and public involvement team, DaRe2THINK presents an opportunity to transform the approach to randomized trials in the setting of routine healthcare, providing high-quality evidence generation in populations representative of the community at risk.

The Helpfulness of Web-Based Mental Health and Well-being Forums for Providing Peer Support for Young People: Cross-sectional Exploration

Background

Young people are increasingly seeking out web-based support for their mental health and well-being. Peer support forums are popular with this age group, with young individuals valuing the fact that the forums are available 24/7, providing a safe and anonymous space for exploration. Currently, little systematic evaluation of the helpfulness of such forums in providing support has been conducted.

Objective

This study examined the helpfulness of the support offered within web-based mental health and well-being peer support forums for young people. It specifically investigated the self-reported user ratings of helpfulness reported through the completion of a developing experience measure. The ratings will be used to consider further development of the measure and reflect upon the overall helpfulness of the forums as indicated by the reported scores.

Methods

The study used routinely collected practice-based outcome data from web-based mental health forums for young people. These forums are hosted by the UK-based web-based therapy and support service, Kooth. A cross-sectional design was used to explore—using a range of inferential statistical measures—the outcomes reported by those accessing the forums using a Peer Online Community Experience Measure (POCEM). To consider the helpfulness in general, 23,443 POCEMs completed in 2020 were used. A second data set of 17,137 completed POCEMs from the same year was used to consider whether various engagement indicators had an impact upon the helpfulness rating.

Results

Female users aged between 11 and 16 years predominantly completed the POCEM. This is in keeping with the majority of those using the service. In total, 74.6% (8240/11,045) of the scores on the POCEM indicated that the individuals found the posts helpful. An ANOVA indicated that male users were more likely to report obtaining intrapersonal support, whereas female users obtained interpersonal support. Furthermore, the POCEM scores reflected the internal consistency of the measure and provided an insight into the way that young people made use of the peer support resource; for instance, posts that were rated more helpful were correlated with spending longer time reading them, and the topics discussed varied throughout the day with more mental health issues being discussed later at night.

Conclusions

The results seem to demonstrate that, overall, the young people involved in this study found web-based peer support helpful. They indicate that peer support can provide an important strand of care within a supportive mental health ecosystem, particularly during time periods when in-person support is typically closed. However, limitations were noted, suggesting that caution is needed when interpreting the results of this study. Although such services are incredibly well used, they have received little research attention to date. As such, further investigation into what constitutes helpful and unhelpful peer support is needed.

Generalizability and reach of a randomized controlled trial to improve oral health among home care recipients: comparing participants and nonparticipants at baseline and during follow-up

Background

The generalizability of randomized controlled trials (RCTs) with a low response can be limited by systematic differences between participants and nonparticipants. This participation bias, however, is rarely investigated because data on nonparticipants is usually not available. The purpose of this article is to compare all participants and nonparticipants of a RCT to improve oral health among home care recipients at baseline and during follow-up using claims data.

Methods

Seven German statutory health and long-term care insurance funds invited 9656 home care recipients to participate in the RCT MundPflege. Claims data for all participants (n = 527, 5.5% response) and nonparticipants (n = 9129) were analyzed. Associations between trial participation and sex, age, care dependency, number of Elixhauser diseases, and dementia, as well as nursing, medical, and dental care utilization at baseline, were investigated using multivariable logistic regression. Associations between trial participation and the probability of (a) moving into a nursing home, (b) being hospitalized, and (c) death during 1 year of follow-up were examined via Cox proportional hazards regressions, controlling for baseline variables.

Results

At baseline, trial participation was positively associated with male sex (odds ratio 1.29 [95% confidence interval 1.08–1.54]), high (vs. low 1.46 [1.15–1.86]) care dependency, receiving occasional in-kind benefits to relieve caring relatives (1.45 [1.15–1.84]), having a referral by a general practitioner to a medical specialist (1.62 [1.21–2.18]), and dental care utilization (2.02 [1.67–2.45]). It was negatively associated with being 75–84 (vs. < 60 0.67 [0.50–0.90]) and 85 + (0.50 [0.37–0.69]) years old. For morbidity, hospitalizations, and formal, respite, short-term, and day or night care, no associations were found. During follow-up, participants were less likely to move into a nursing home than nonparticipants (hazard ratio 0.50 [0.32–0.79]). For hospitalizations and mortality, no associations were found.

Conclusions

For half of the comparisons, differences between participants and nonparticipants were observed. The RCT’s generalizability is limited, but to a smaller extent than one would expect because of the low response. Routine data provide a valuable source for investigating potential differences between trial participants and nonparticipants, which might be used by future RCTs to evaluate the generalizability of their findings.

Trial registration

German Clinical Trials Register DRKS00013517. Retrospectively registered on June 11, 2018.

Supplementary Information

The online version contains supplementary material available at 10.1186/s13063-022-06470-y.

The need for increased pragmatism in cardiovascular clinical trials

The majority of cardiovascular randomized controlled trials (RCTs) test interventions in selected patient populations under explicitly protocol-defined settings. Although these ‘explanatory’ trial designs optimize conditions to test the efficacy and safety of an intervention, they limit the generalizability of trial findings in broader clinical settings. The concept of ‘pragmatism’ in RCTs addresses this concern by providing counterbalance to the more idealized situation underpinning explanatory RCTs and optimizing effectiveness over efficacy. The central tenets of pragmatism in RCTs are to test interventions in routine clinical settings, with patients who are representative of broad clinical practice, and to reduce the burden on investigators and participants by minimizing the number of trial visits and the intensity of trial-based testing. Pragmatic evaluation of interventions is particularly important in cardiovascular diseases, where the risk of death among patients has remained fairly stable over the past few decades despite the development of new therapeutic interventions. Pragmatic RCTs can help to reveal the ‘real-world’ effectiveness of therapeutic interventions and elucidate barriers to their implementation. In this Review, we discuss the attributes of pragmatism in RCT design, conduct and interpretation as well as the general need for increased pragmatism in cardiovascular RCTs. We also summarize current challenges and potential solutions to the implementation of pragmatism in RCTs and highlight selected ongoing and completed cardiovascular RCTs with pragmatic trial designs.

In this Review, Khan and colleagues discuss the benefits and challenges of including pragmatism in the design, conduct and interpretation of randomized controlled trials (RCTs) for cardiovascular disease and highlight selected ongoing and completed cardiovascular RCTs that incorporate a pragmatic design.

Most cardiovascular randomized controlled trials (RCTs) conducted to date have been ‘explanatory’, that is, designed to study the intervention in optimized conditions with selected patient populations and frequent protocolized assessments.

Although explanatory RCT designs increase validity, they limit the generalizability of trial findings, whereas a ‘pragmatic’ approach to RCTs yields findings more relevant to real-world practice.

In pragmatic RCTs, interventions are tested in patients who are broadly representative of the condition being studied, and the study is aligned with routine clinical care to reduce costs and organizational burden.

Although pragmatic RCTs tend to attenuate estimates of treatment effects, they do provide a more realistic understanding of population-level effectiveness and costs than explanatory trials.

Pragmatic trials can highlight barriers to the implementation of therapies and are better suited than explanatory RCTs to assessing the effects of implementation strategies and health-care policies at the population level.

Widespread implementation of pragmatic trials would require the development of technological infrastructure to collect and share data as well as regulatory guidelines amenable to findings derived from routinely collected data.

A Scoping Review on Use of Drugs Targeting the JAK/STAT Pathway in Psoriasis

Introduction

The Janus kinase–signal transducer and activator of transcription (JAK/STAT) pathway are known to be involved in inflammatory immune-mediated skin diseases, including psoriasis. The development of drugs targeting the JAK/STAT signaling pathway presents new treatment opportunities for psoriasis. However, the application of JAK inhibitors for the treatment of dermatological disorders is still in its early stages of development. This review summarizes available evidence in an attempt to identify knowledge gaps for conducting further research studies and improving clinical decision-making.

Objective

The objective of this study is to conduct a scoping review of the use of drugs targeting the JAK/STAT pathway in the treatment of psoriasis.

Methods

A priori protocol for scoping review was published in 2019. The Joanna Briggs Institute Reviewer's Manual and the PRISMA Extension for Scoping Review were used for the review. MEDLINE, EMBASE, CINAHL, Scopus, and Web of Science databases and ClinicalTrials registry were referred to in April 2019 and March 2021, respectively. References in English involving evidence on the use of drugs targeting the JAK/STAT pathway in patients with psoriasis were included. Data charting was performed by two authors using tables and figures.

Results

The evidence found on the efficacy and safety of drugs targeting the JAK/STAT pathway in patients with psoriasis comes from 118 articles reporting the results of 34 randomized clinical trials (RCTs). Nine different drugs administered through various routes were identified (systemic: peficitinib, baricitinib, solcitinib, itacitinib, abrocitinib, deucravacitinib, and brepocitinib; topical: ruxolitinib; and both: tofacitinib). Knowledge articles are mainly created and published by pharmaceutical companies and authors through their own funding or by those related to them. Only tofacitinib and deucravacitinib have undergone phase III clinical trials, being the only ones tested with active comparators etanercept and apremilast, respectively. Proportions of Psoriasis Area and Severity Index (PASI) and Physician's Global Assessment (PGA) were the efficacy variables most frequently studied in systemic treatments. Only two RCTs declared the safety data collected by systematic assessment; the only systemic drug with phase III data was tofacitinib. Tofacitinib 5 mg two times daily (BID)/10 mg BID efficacy was compared with etanercept 50 mg/week and a placebo. At 12–16 weeks, PASI 75/PGA 01 ranges were as follows: 38.07–80%/37.16–67.4% for tofacitinib 5 mg BID; 54.79–100%/50–75.6% for tofacitinib 10 mg BID; 58.8/66.8% for etanercept, date from one only study; and 0–33.3%/9.04–33.3% for the placebo group. Other drugs in earlier stages of development showed values within these ranges. The most frequent adverse events (AEs) were nasopharyngitis and upper respiratory tract infections in all treatment groups.

Conclusion

There is increasing evidence on the use of drugs targeting the JAK/STAT pathway as a treatment for psoriasis, although they are in the early phases of development. The trials conducted to date have been financed directly or indirectly by the pharmaceutical industry, which must be taken into account when interpreting the results of the trials. Psoriasis treatment is currently symptomatic and could potentially present a significant risk of toxicity. Therefore, the design of principal efficacy outcome measures considering the impact of the outcome on quality of life and a drug assessment methodology aimed at improving safety would probably strengthen the evidence and decision-making process.

COVID-19: A Catalyst for Transforming Randomized Trials

The coronavirus disease 2019 (COVID-19) pandemic incited a global clinical trial research agenda of unprecedented speed and high volume. This expedited research activity in a time of crisis produced both successes and failures that offer valuable learning opportunities for the scientific community to consider. Successes include the implementation of large adaptive and pragmatic trials as well as burgeoning efforts toward rapid data synthesis and open science principles. Conversely, notable failures include: (1) inadequate study design and execution; (2) data reversal, fraud, and retraction; and (3) research duplication and waste. Other challenges that became highlighted were the need to find unbiased designs for investigating complex, nonpharmaceutical interventions and the use of routinely collected data for outcomes assessment. This article discusses these issues juxtaposing the COVID-19 trials experience against trials in anesthesiology and other fields. These lessons may serve as a positive catalyst for transforming future clinical trial research.

Remote, proactive, telephone based management of toxicity in outpatients during adjuvant or neoadjuvant chemotherapy for early stage breast cancer: pragmatic, cluster randomised trial

OBJECTIVE

To evaluate the effectiveness of remote proactive management of toxicities during chemotherapy for early stage breast cancer.

DESIGN

Pragmatic, cluster randomised trial.

SETTING

20 cancer centres in Ontario, Canada, allocated by covariate constrained randomisation to remote management of toxicities or routine care.

PARTICIPANTS

All patients starting adjuvant or neoadjuvant chemotherapy for early stage breast cancer at each centre. 25 patients from each centre completed patient reported outcome questionnaires.

INTERVENTIONS

Proactive, standardised, nurse led telephone management of common toxicities at two time points after each chemotherapy cycle.

MAIN OUTCOME MEASURES

The primary outcome, cluster level mean number of visits to the emergency department or admissions to hospital per patient during the whole course of chemotherapy treatment, was evaluated with routinely available administrative healthcare data. Secondary patient reported outcomes included toxicity, self-efficacy, and quality of life.

RESULTS

Baseline characteristics of participants were similar in the intervention (n=944) and control arms (n=1214); 22% were older than 65 years. Penetration (that is, the percentage of patients who received the intervention at each centre) was 50-86%. Mean number of visits to the emergency department or admissions to hospital per patient was 0.91 (standard deviation 0.28) in the intervention arm and 0.94 (0.40) in the control arm (P=0.94); 47% (1014 of 2158 patients) had at least one visit to the emergency department or a hospital admission during chemotherapy. Among 580 participants who completed the patient reported outcome questionnaires, at least one grade 3 toxicity was reported by 48% (134 of 278 patients) in the intervention arm and by 58% (163 of 283) in the control arm. No differences in self-efficacy, anxiety, or depression were found. Compared with baseline, the functional assessment of cancer therapy trial outcome index decreased by 6.1 and 9.0 points in the intervention and control participants, respectively.

CONCLUSIONS

Proactive, telephone based management of toxicities during chemotherapy did not result in fewer visits to the emergency department or hospital admissions. With the rapid rise in remote care because of the covid-19 pandemic, identifying scalable strategies for remote management of patients during cancer treatment is particularly relevant.

TRIAL REGISTRATION

ClinicalTrials.gov NCT02485678.

Comparative effectiveness and safety of pharmaceuticals assessed in observational studies compared with randomized controlled trials

BACKGROUND

There have been ongoing efforts to understand when and how data from observational studies can be applied to clinical and regulatory decision making. The objective of this review was to assess the comparability of relative treatment effects of pharmaceuticals from observational studies and randomized controlled trials (RCTs).

METHODS

We searched PubMed and Embase for systematic literature reviews published between January 1, 1990, and January 31, 2020, that reported relative treatment effects of pharmaceuticals from both observational studies and RCTs. We extracted pooled relative effect estimates from observational studies and RCTs for each outcome, intervention-comparator, or indication assessed in the reviews. We calculated the ratio of the relative effect estimate from observational studies over that from RCTs, along with the corresponding 95% confidence interval (CI) for each pair of pooled RCT and observational study estimates, and we evaluated the consistency in relative treatment effects.

RESULTS

Thirty systematic reviews across 7 therapeutic areas were identified from the literature. We analyzed 74 pairs of pooled relative effect estimates from RCTs and observational studies from 29 reviews. There was no statistically significant difference (based on the 95% CI) in relative effect estimates between RCTs and observational studies in 79.7% of pairs. There was an extreme difference (ratio < 0.7 or > 1.43) in 43.2% of pairs, and, in 17.6% of pairs, there was a significant difference and the estimates pointed in opposite directions.

CONCLUSIONS

Overall, our review shows that while there is no significant difference in the relative risk ratios between the majority of RCTs and observational studies compared, there is significant variation in about 20% of comparisons. The source of this variation should be the subject of further inquiry to elucidate how much of the variation is due to differences in patient populations versus biased estimates arising from issues with study design or analytical/statistical methods.

Digital health, big data and smart technologies for the care of patients with systemic autoimmune diseases: Where do we stand?

The past decade has seen tremendous development in digital health, including in innovative new technologies such as Electronic Health Records, telemedicine, virtual visits, wearable technology and sophisticated analytical tools such as artificial intelligence (AI) and machine learning for the deep-integration of big data. In the field of rare connective tissue diseases (rCTDs), these opportunities include increased access to scarce and remote expertise, improved patient monitoring, increased participation and therapeutic adherence, better patient outcomes and patient empowerment. In this review, we discuss opportunities and key-barriers to improve application of digital health technologies in the field of autoimmune diseases. We also describe what could be the fully digital pathway of rCTD patients. Smart technologies can be used to provide real-world evidence about the natural history of rCTDs, to determine real-life drug utilization, advanced efficacy and safety data for rare diseases and highlight significant unmet needs. Yet, digitalization remains one of the most challenging issues faced by rCTD patients, their physicians and healthcare systems. Digital health technologies offer enormous potential to improve autoimmune rCTD care but this potential has so far been largely unrealized due to those significant obstacles. The need for robust assessments of the efficacy, affordability and scalability of AI in the context of digital health is crucial to improve the care of patients with rare autoimmune diseases.

Randomized COVID-19 vaccination rollout can offer direct real-world evidence

Vaccines are vital to control the Coronavirus disease 2019 (COVID-19) pandemic, but the pressure to quickly move from research to implementation in the context of a pandemic crisis raises concerns about benefits and harms, vaccine acceptance and fair access. Here we present a strategy for the COVID-19 vaccination rollout which can be rapidly embedded and would offer direct real-world evidence of vaccines on a large scale to generate otherwise unobtainable knowledge on the safety and perhaps efficacy of COVID-19 vaccines. Such strategic rollouts leveraging randomization can provide important evidence, for COVID-19 and in future occasions, for vaccines and beyond.

Application of Janus Kinase Inhibitors in Atopic Dermatitis: An Updated Systematic Review and Meta-Analysis of Clinical Trials

Janus kinase (JAK) inhibitors are promising treatments for atopic dermatitis (AD). The aim of this study was to assess the efficacy and safety of JAK inhibitors for AD treatment via the “Grading of Recommendations Assessment, Development, and Evaluation” approach. We identified 15 randomized controlled trials comparing oral or topical JAK inhibitors against placebo to treat AD. A random-effects meta-analysis was performed, and the numbers-needed-to-treat (NNTs)/numbers-needed-to-harm (NNHs) were calculated. Patients treated with JAK inhibitors were associated with higher rates of achieving eczema area and severity index-75 (rate ratio (RR): 2.84; 95% confidence interval (CI): 2.20–3.67; I²: 38.9%; NNT = 3.97), Investigator’s Global Assessment response (RR: 2.99; 95% CI: 2.26–3.95; I²: 0%; NNT = 5.72), and pruritus numerical rating scale response (RR: 2.52; 95% CI: 1.90–3.35; I²: 39.4%; NNT = 4.91) than those treated with placebo. Moreover, patients treated with JAK inhibitors had a higher risk of treatment-emergent adverse events (RR: 1.14; 95% CI: 1.02–1.28; I²: 52%; NNH = 14.80) but not adverse events leading to drug discontinuation. According to the evidence-based results, JAK inhibitors are potentially effective strategies (certainty of evidence: “moderate”) for treating AD with tolerable side effects (certainty of evidence: “low”). Nevertheless, long-term follow-up is required.