Clinical drug trials in general practice: how well are external validity issues reported?

A Comparison of Individuals with Diabetes and EMPA-REG Trial Participants: Exploring Aspects of External Validity

BACKGROUND

There is increasing use of sodium glucose co-transporter 2 (SGLT2) inhibitors to treat diabetes. Since trials apply specific entry and exclusion criteria to ensure internal validity, comparisons of trial populations with nationally representative samples can inform the applicability of study findings to practice.

OBJECTIVE

To compare individuals with diabetes from a nationally representative sample to patients who underwent randomization in the EMPA-REG trial. A secondary aim was to characterize what proportion of individuals prescribed an SGLT2 inhibitor in a nationally representative sample would have been included in the EMPA-REG trial.

DESIGN

Retrospective cross-sectional study.

PARTICIPANTS

Adults with diabetes who took part in the National Health and Nutrition Examination Survey (NHANES) between 2011-2014 (primary analysis corresponding to EMPA-REG enrollment) and 2015-2018 (secondary analysis corresponding to contemporary sample).

MAIN MEASURES

The primary outcome was a comparison of demographic (age, sex, ethnicity, and pregnancy status), clinical (comorbidities and medication use), examination (weight, body mass index, and systolic and diastolic blood pressure), and laboratory (hgba1c, low- and high-density lipoprotein cholesterol, triglycerides, and estimated glomerular filtration rate) characteristics of NHANES respondents versus EMPA-REG trial participants. The secondary outcome was the proportion of NHANES respondents who had been prescribed an SGLT2 inhibitor that would have met inclusion criteria for the EMPA-REG trial.

KEY RESULTS

There were 655 and 48 respondents, representing a weighted sample of 21,849,775 and 1,062,573 individuals, included in the primary and secondary analyses, respectively. Overall, 7.6% (95% CI 4.8-10.6%) of 2011-2014 NHANES respondents would have met all EMPA-REG trial inclusion criteria. NHANES respondents and EMPA-REG participants differed across demographic, clinical, examination, and laboratory domains. Of NHANES respondents from 2015 to 2018 who were prescribed an SGLT2 inhibitor, 10.6% (95% CI <1-24.7%) would have met all inclusion criteria for the EMPA-REG trial.

CONCLUSIONS

The EMPA-REG population differed from a nationally representative sample, which could affect generalizability.

Precision Reimbursement for Precision Medicine: Using Real-World Evidence to Evolve From Trial-and-Project to Track-and-Pay to Learn-and-Predict

Basic scientists and drug developers are accelerating innovations toward the goal of precision medicine. Regulators create pathways for timely patient access to precision medicines, including individualized therapies. Healthcare payors acknowledge the need for change but downstream innovation for coverage and reimbursement is only haltingly occurring. Performance uncertainty, high price‐tags, payment timing, and actuarial risk issues associated with precision medicines present novel financial challenges for payors. With traditional drug reimbursement frameworks, payment is based on an assumed randomized controlled trial (RCT) projection of real‐world effectiveness, a “trial‐and‐project” strategy; the clinical benefit realized for patients is not usually ascertained ex post by collection of real‐world data (RWD). To mitigate financial risks resulting from clinical performance uncertainty, manufacturers and payors devised “track‐and‐pay” frameworks (i.e., the tracking of a pre‐agreed treatment outcome which is linked to financial consequences). Whereas some track‐and‐pay arrangements have been successful, inherent weaknesses include the potential for misalignment of incentives, the risk of channeling of patients, and a failure to use the RWD generated to enable continuous learning about treatments. “Precision reimbursement” (PR) intends to overcome inherent weaknesses of simple track‐and‐pay schemes. In combining the collection of RWD with advanced analytics (e.g., artificial intelligence and machine learning) to generate actionable real‐world evidence, with prospective alignment of incentives across all stakeholders (including providers and patients), and with pre‐agreed use and dissemination of information generated, PR becomes a “learn‐and‐predict” model of payment for performance. We here describe in detail the concept of PR and lay out the next steps to make it a reality.

Recruitment challenges to the I CARE study: a randomised trial on general practitioner-led colon cancer survivorship care

OBJECTIVES

The I CARE study (Improving Care After colon canceR treatment in the Netherlands) aims to compare surgeon-led to general practitioner (GP)-led colon cancer survivorship care. Recruitment to the trial took longer than expected. In this descriptive study, recruitment is critically reviewed.

SETTING

Patients were recruited from eight Dutch medical centres.

PARTICIPANTS

Patients treated with curative intent for stages I-III colon cancer. Target patient sample size was calculated at 300.

INTERVENTIONS

Patients were randomised to surgeon-led (usual) versus GP-led care, with or without access to an eHealth application (Oncokompas).

OUTCOME MEASURES

Baseline characteristics of (non-)participants, reasons for non-participation and strategies to improve recruitment were reviewed.

RESULTS

Out of 1238 eligible patients, 353 patients were included. Of these, 50 patients dropped out shortly after randomisation and before start of the intervention, resulting in a participation rate of 25%. Participants were on average slightly younger (68.1 years vs 69.3 years) and more often male (67% vs 50%) in comparison to non-participants. A total of 806 patients declined participation for reasons most often relating to research (57%), including the wish to remain in specialist care (31%) and too much effort to participate (12%). Some patients mentioned health (9%) and confrontation with the disease (5%) as a reason. In 43 cases, GPs declined participation, often related to the study objective, need for financial compensation and time restraints. The generally low participation rate led to concerns about reaching the target sample size. Methods to overcome recruitment challenges included changes to the original recruitment procedure and the addition of new study centres.

CONCLUSIONS

Challenges were faced in the recruitment to a randomised trial on GP-led colon cancer survivorship care. Research on the transition of care requires sufficient time, funding and support base among patients and healthcare professionals. These findings will help inform researchers and policy-makers on the development of future practices.

TRIAL REGISTRATION NUMBER

NTR4860.

Mapping the literature on primary care research reporting: a scoping review

BACKGROUND

Despite broad efforts to improve the reporting of biomedical research, no reporting guideline exists for primary care (PC) research. Little is known about current reporting practices or how well reports meet the needs of varied users in PC.

OBJECTIVE

To map the published literature on PC research reporting: quality, strengths and weaknesses, recommendations and efforts to improve reporting.

METHODS

Scoping review of literature across seven major databases and search engines to identify all articles on PC research reporting published in English, 2000-20. An additional secondary search of references of these 25 articles and consideration of expert panel suggestions. Structured data extraction by multiple reviewers using a predetermined form.

RESULTS

Search yielded 2847 unique titles, of which 126 underwent full-text review and 25 met inclusion criteria. Publications included opinion pieces (9), systematic reviews (5), methods articles (2), literature reviews (4), qualitative studies (4) and surveys (1). Studies focussed on a variety of topics and research methods. All publications identified the need for improved reporting and recommended items to include in reports. Most commonly, publications cited the need for more detailed reporting on the context of study interventions, clinical settings and health care systems. Most publications endorsed the use of reporting guidelines and recognized the unique needs of PC research reporting.

CONCLUSIONS

Published research and opinion identify unique needs for PC research reports and support new guidance to improve the validity, generalizability and application of study findings.

External validity of phase III trials on vaccines against SARS-CoV-2 to a middle-aged and elderly Western European population

Initial results from various phase-III trials on vaccines against SARS-CoV-2 are promising. For proper translation of these results to clinical guidelines, it is essential to determine how well the general population is reflected in the study populations of these trials. This study was conducted among 7162 participants (age-range: 51-106 years; 58% women) from the Rotterdam Study. We quantified the proportion of participants that would be eligible for the nine ongoing phase-III trials. We further quantified the eligibility among participants at high risk to develop severe COVID-19. Since many trials were not explicit in their exclusion criterion with respect to 'acute' or 'unstable preexisting' diseases, we performed two analyses. First, we included all participants irrespective of this criterion. Second, we excluded persons with acute or 'unstable preexisting' diseases. 97% of 7162 participants was eligible for any trial with eligibility for separate trials ranging between 11-97%. For high-risk individuals the corresponding numbers were 96% for any trial with separate trials ranging from 5-96%. Importantly, considering persons ineligible due to 'acute' or 'unstable pre-existing' disease drastically dropped the eligibilities for all trials below 43% for the total population and below 36% for high-risk individuals. The eligibility for ongoing vaccine trials against SARS-CoV-2 can reduce by half depending on interpretation and application of a single unspecified exclusion criterion. This exclusion criterion in our study would especially affect the elderly and those with pre-existing morbidities. These findings thus indicate the difficulty as well as importance of developing clinical recommendations for vaccination and applying these to the appropriate target populations. This becomes especially paramount considering the fact that many countries worldwide have initiated their vaccination programs by first targeting the elderly and most vulnerable persons.

Field trials of chemical suppression of embryonic cane toads (Rhinella marina) by older conspecifics

Laboratory experiments have shown that the viability of embryos of the invasive cane toad (Rhinella marina) can be reduced by exposure to chemical cues from older conspecific larvae. These effects (very strong in laboratory trials) may offer an exciting new approach to controlling this problematic invasive species in Australia. However, the degree to which the method works in natural environments has yet to be assessed.Our experiments in the laboratory and in seminatural outdoor waterbodies show that chemical cues from tadpoles do indeed suppress the growth, development, and survival of conspecific larvae that are exposed as embryos and do so in a dose-dependent manner; higher tadpole densities cause greater suppression of embryos.In seminatural outdoor waterbodies, suppressor-exposed tadpoles were less than half as likely to survive to metamorphosis as were controls, and were much smaller when they did so and hence, less likely to survive the metamorph stage. Additionally, female cane toads were less likely to oviposit in a waterbody containing free-ranging (but not cage-enclosed) tadpoles, suggesting that the presence of tadpoles (rather than the chemical cues they produce) may discourage oviposition.Broadly, our results suggest that the suppression effect documented in laboratory studies does indeed occur in the field also, and hence that we may be able to translate that approach to develop new and more effective ways to reduce rates of recruitment of peri-urban populations of cane toads in their invasive range.

The onset and accumulation of physical multimorbidity in severe and common mental disorders

PURPOSE OF REVIEW

Despite of the heightened risks and burdens of physical comorbidities across the entire spectrum of mental disorders, relatively little is known about physical multimorbidity in this population. The aim of this narrative review is to present recent data regarding the onset and accumulation of physical multimorbidity and to assess its impact on the onset, course, treatment, and outcomes of mental disorders.

RECENT FINDINGS

A substantial body of literature shows increased risk of physical multimorbidity among people with mental disorders. The disparity in physical multimorbidity occurs even before the diagnosis of mental disorder, and the younger age group appears to be at particular risk. Numerous patterns of association between mental disorders and medical disorders involving multiple organ systems have been identified. Physical multimorbidity affects people with mental disorders across their life spans, is associated with a wide range of unfavorable outcomes and presents significant clinical and public health concerns.

SUMMARY

To address physical health inequalities among people with mental disorders compared with the general population, we must focus on the physical health from the very first point of contact with a mental health service. Treatment of mental disorders must be customized to meet the needs of patients with different physical multimorbidity patterns. Future work is needed to clarify how physical multimorbidity influences mental disorder treatment outcomes.

Nursing Personnel in the Era of Personalized Healthcare in Clinical Practice

Personalized, stratified, or precision medicine (PM) introduces a new era in healthcare that tries to identify and predict optimum treatment outcomes for a patient or a cohort. It also introduces new scientific terminologies regarding therapeutic approaches and the need of their adoption from healthcare providers. Till today, evidence-based practice (EBP) was focusing on population averages and their variances among cohorts for clinical values that are essential for optimizing healthcare outcome. It can be stated that EBP and PM are complementary approaches for a modern healthcare system. Healthcare providers through EBP often see the forest (population averages) but miss the trees (individual patients), whereas utilization of PM may not see the forest for the trees. Nursing personnel (NP) play an important role in modern healthcare since they are consulting, educating, and providing care to patients whose needs often needs to be individualized (personalized nursing care, PNC). Based on the clinical issues earlier addressed from clinical pharmacology, EBP, and now encompassed in PM, this review tries to describe the challenges that NP have to face in order to meet the requisites of the new era in healthcare. It presents the demands that should be met for upgrading the provided education and expertise of NP toward an updated role in a modern healthcare system.

Articles provided insufficient information to conduct an appropriate retrospective assessment of the pragmatic/explanatory features of medicine trials with the PRECIS-2 tool

PURPOSE

To assess whether, in the retrospective assessment of the pragmatic/explanatory features of pragmatic randomized controlled trials (pRCTs), the nine PRECIS-2 domain scores using the information provided in articles were modified after using the information reported in other publicly available sources.

METHODS

This is a cross-sectional study of participant-level pRCTs published in July 2018 to December 2019 in the four highest-impact general medicine journals. The articles described the main results of pRCTs assessing medicines in one or more arms that were not in the pre-licensing phases. The information reported in trial full protocols, published protocols, and other publications, registries, and trial websites were assessed and scored, and compared with that previously obtained after reviewing the information reported in the articles.

RESULTS

Out of 76 articles on pRCTs, 13 (17%) were included in the analysis. All were two-arm trials, assessing medicines only (n = 7), medicine vs device (n = 2), medicine vs surgery (n = 1), or medicine vs placebo (n = 3). Seven were open-label trials, and six had any type of masking. All except one had the full protocol available and/or published protocol; seven had other types of publication available. The assessment of the nine PRECIS-2 domains with the information reported in the 13 articles was changed in all trials after using the information included in other additional available sources. Between one (n = 1 article) and six (n = 2) domains were modified in each pRCT. The domains that most commonly changed were "organization" (n = 12), "recruitment" (n = 11), and "follow-up" (n = 8). "Primary outcome" and "primary analysis" were not modified in any trial. Eight percent of all domains could not be assessed due to inadequate or lack of information in seven articles; those were "recruitment" (n = 3), "organization" (n = 3), "setting" (n = 2), and "flexibility:adherence" (n = 1).

CONCLUSION

Articles describing the trial main results are usually insufficient for the appropriate retrospective assessment of the pragmatic/explanatory features of a pRCT by authors not involved in the conduct of the trial. To address this issue, editors should require the submission of the original full protocol and final full protocol with the history of amendments to be published as supplementary material to the article.

Understanding the nature and scope of clinical research commentaries in PubMed

Scientific commentaries are expected to play an important role in evidence appraisal, but it is unknown whether this expectation has been fulfilled. This study aims to better understand the role of scientific commentary in evidence appraisal. We queried PubMed for all clinical research articles with accompanying comments and extracted corresponding metadata. Five percent of clinical research studies (N = 130 629) received postpublication comments (N = 171 556), resulting in 178 882 comment-article pairings, with 90% published in the same journal. We obtained 5197 full-text comments for topic modeling and exploratory sentiment analysis. Topics were generally disease specific with only a few topics relevant to the appraisal of studies, which were highly prevalent in letters. Of a random sample of 518 full-text comments, 67% had a supportive tone. Based on our results, published commentary, with the exception of letters, most often highlight or endorse previous publications rather than serve as a prominent mechanism for critical appraisal.

Factors Associated With High-Quality Guidelines for the Pharmacologic Management of Chronic Diseases in Primary Care: A Systematic Review

IMPORTANCE

As the rate of publication of new and sometimes conflicting medical research increases, clinicians rely heavily on clinical practice guidelines (CPGs) to inform practice. However, CPGs are of widely variable quality, and there are no existing objective measures to rate the quality of CPGs.

OBJECTIVE

To systematically assess 421 CPGs for the management of common noncommunicable diseases in primary care using the validated Appraisal of Guidelines for Research and Evaluation Instrument, version II (AGREE-II) tool and elucidate the factors associated with quality of CPGs.

EVIDENCE REVIEW

MEDLINE, Embase, the Cochrane Library, and 12 websites for CPGs were searched for CPGs for the management of common noncommunicable diseases in primary care published between January 1, 2011, and August 30, 2017. The assessment of the quality of CPGs was performed by 3 appraisers using the 6 domains of the AGREE-II instrument. A multiple logistic regression was performed to identify factors associated with quality of CPGs.

FINDINGS

Of the 421 CPGs reviewed, 23.5% (99) were classified as high quality. Among included guidelines, clarity of presentation (70%) and scope and purpose (61%) had the highest median AGREE-II scores. The domains with the lowest median scores were applicability (22%) and rigor of development (33%). Factors associated with high-quality CPGs included having more than 20 authors (odds ratio, 9.08; 95% CI, 3.35-24.62), development at governmental institutions (odds ratio, 10.38; 95% CI, 2.72-39.60), and reporting funding (odds ratio, 10.34; 95% CI, 4.77-22.39). Year of publication, region, guideline version, and scope were not associated with quality among included CPGs.

CONCLUSIONS AND RELEVANCE

Primary care professionals and policymakers should be aware that CPGs in primary care are of widely variable quality, with less than 25% of included CPGs rated as high quality. High-quality CPGs were associated with a higher number of authors, governmental institutions, and the report of funding. Region of origin was not associated with quality of CPGs, which suggests that the improvement of the quality of CPGs should be an international concern.