Research methods must find ways of accommodating clinical reality, not ignoring it: the need for pragmatic trials

Use of real-world evidence in postmarketing medicines regulation in the European Union: a systematic assessment of European Medicines Agency referrals 2013-2017

OBJECTIVES

To assess the use, and evaluate the usefulness, of non-interventional studies and routinely collected healthcare data in postmarketing assessments conducted by the European Medicines Agency (EMA).

DESIGN

We reviewed and systematically assessed all referrals to the EMA made due to safety or efficacy concerns that were evaluated between 1 January 2013 and 30 June 2017. We extracted information from the assessment report and the referral notification. Two reviewers independently assessed the contribution of non-interventional evidence to decision-making.

RESULTS

The preliminary evidence leading to the assessment in 52 eligible referrals was mostly from spontaneous reports (cited in 26 of 52 referrals) and randomised trials (22/52). In contrast, many evidence types were used for the full assessment. Non-interventional studies were frequently used in the full assessment for the evaluation of product safety (31/52) and product efficacy (18/52). In particular, non-interventional studies were relied on for the evaluation of safety and efficacy in subgroups, the evaluation of safety relating to a rare adverse event, understanding product usage and misuse and for evaluation of the effectiveness of risk minimisation measures. The most common recommendations were changes to product information (43/52) and marketing authorisation withdrawal or suspension (12/52). In the majority of referrals, non-interventional evidence was judged to contribute to the decision made (30/52) and in three referrals it was the primary source of evidence.

CONCLUSIONS

European regulatory decision-making relies on multiple evidence types, particularly randomised trials, spontaneous reports and non-interventional studies. Non-interventional studies had an important role particularly for the characterisation and quantification of adverse events, the evaluation of product usage and for evaluating the effectiveness of regulatory action to minimise risk.

Point-of-Care Clinical Trials in Sports Medicine Research: Identifying Effective Treatment Interventions Through Comparative Effectiveness Research

CONTEXT

Recently, calls to conduct comparative effectiveness research (CER) in athletic training to better support patient care decisions have been circulated. Traditional research methods (eg, randomized controlled trials [RCTs], observational studies) may be ill suited for CER. Thus, innovative research methods are needed to support CER efforts.

OBJECTIVES

To discuss the limitations of traditional research designs in CER studies, describe a novel methodologic approach called the point-of-care clinical trial (POC-CT), and highlight components of the POC-CT (eg, incorporation of an electronic medical record [EMR], Bayesian adaptive feature) that allow investigators to conduct scientifically rigorous studies at the point of care.

DESCRIPTION

Practical concerns (eg, high costs and limited generalizability of RCTs, the inability to control for bias in observational studies) may stall CER efforts in athletic training. In short, the aim of the POC-CT is to embed a randomized pragmatic trial into routine care; thus, patients are randomized to minimize potential bias, but the study is conducted at the point of care to limit cost and improve the generalizability of the findings. Furthermore, the POC-CT uses an EMR to replace much of the infrastructure associated with a traditional RCT (eg, research team, patient and clinician reminders) and a Bayesian adaptive feature to help limit the number of patients needed for the study. Together, the EMR and Bayesian adaptive feature can improve the overall feasibility of the study and preserve the typical clinical experiences of the patient and clinician.

CLINICAL ADVANTAGES

The POC-CT includes the basic tenets of practice-based research because studies are conducted at the point of care, in real-life settings, and during routine clinical practice. If implemented effectively, the POC-CT can be seamlessly integrated into daily clinical practice, allowing investigators to establish patient-reported evidence that may be quickly applied to patient care decisions. This design appears to be a promising approach for CER investigations and may help establish a "learning health care system" in the sports medicine community.

An analysis reveals differences between pragmatic and explanatory diagnostic accuracy studies

OBJECTIVES

The objective of this study was to clarify a difference between two approaches while evaluating the diagnostic accuracy of medical tests, labeled here as "pragmatic" vs. "explanatory" studies.

METHODS

Using the definitions and characteristics described by Schwartz and Lellouch for randomized trials of interventions, and Schwartz' more general distinction between a pragmatic and an explanatory approach in medical research, we define a similar continuum for diagnostic accuracy studies. Explanatory studies aim to better understand the behavior of a test; pragmatic ones are done to support recommendations or decisions about using the test in clinical practice.

RESULTS

Pragmatic test accuracy studies differ from explanatory test accuracy studies in several ways. The difference in aims has implications for key elements of study design, such as the study eligibility criteria, the recruitment of patients, the reference standard, and the choice of the statistical analysis. Explanatory accuracy studies are often designed to test a hypothesis. They are typically selective in recruitment, may include "healthy controls," with a small sample size, often recruited at a single center. They ignore testing failures in the analysis and more often present their results as ROC curves. By contrast, pragmatic studies are designed to guide decision making. They ideally will recruit a single, large, and representative group of patients at multiple sites and will more often present their results as estimates of sensitivity and specificity or predictive values at a prespecified threshold.

CONCLUSION

Distinguishing between a pragmatic and an explanatory approach can help in the design, analysis, and interpretation of diagnostic accuracy studies. It can clarify debates about the appropriateness of design features to the study purpose and about the validity and applicability of study findings.

Prevalence of postherpetic neuralgia after a first episode of herpes zoster: prospective study with long term follow up

OBJECTIVE

To estimate the frequency, duration, and clinical importance of postherpetic neuralgia after a single episode of herpes zoster.

DESIGN

Prospective cohort study with long term follow up.

SETTING

Primary health care in Iceland.

PARTICIPANTS

421 patients with a single episode of herpes zoster.

MAIN OUTCOME MEASURES

Age and sex distribution of patients with herpes zoster, point prevalence of postherpetic neuralgia, and severity of pain at 1, 3, 6, and 12 months and up to 7.6 years after the outbreak of zoster.

RESULTS

Among patients younger than 60 years, the risk of postherpetic neuralgia three months after the start of the zoster rash was 1.8% (95% confidence interval 0.59% to 4.18%) and pain was mild in all cases. In patients 60 years and older, the risk of postherpetic neuralgia increased but the pain was usually mild or moderate. After three months severe pain was recorded in two patients older than 60 years (1.7%, 2.14% to 6.15%). After 12 months no patient reported severe pain and 14 patients (3.3%) had mild or moderate pain. Seven of these became pain free within two to seven years, and five reported mild pain and one moderate pain after 7.6 years of follow up. Sex was not a predictor of postherpetic neuralgia. Possible immunomodulating comorbidity (such as malignancy, systemic steroid use, diabetes) was present in 17 patients.

CONCLUSIONS

The probability of longstanding pain of clinical importance after herpes zoster is low in an unselected population of primary care patients essentially untreated with antiviral drugs.

A prospective randomised pilot study of sedation regimens in a general ICU population: a reality-based medicine study

BACKGROUND: For logistical reasons sedation studies are often carried out in elective surgical patients and the results extrapolated to the general intensive care unit (ICU) population. We question the validity of this approach. We compared the two sedation regimens used in our general ICU in a trial structured to mimic clinical practice as closely as possible. RESULTS: Forty patients were randomised to intermittent diazepam or continuous midazolam and sedation monitored with hourly sedation scores; 31 patients completed the study. Scores indicating undersedation were more common with diazepam (P <0.01); overall adequate sedation midazolam 64.7%, diazepam 35.7% (P =0.21). No patient exhibited inappropriately prolonged sedation. Cost was: midazolam AUS$1.98/h; diazepam AUS$0.06/h. CONCLUSION: Both regimens produced rapid onset of acceptable sedation but undersedation appeared more common with the cheaper diazepam regimen. At least 140 patients should be studied to provide evidence applicable to the general ICU population. Used alone, a sedation score may be an inappropriate outcome measure for a sedation trial.