Common misunderstandings of evidence-based medicine

Currently, most evidence assessments in guidelines or health technology assessments (HTAs) rely on the assumption that a randomized controlled trial (RCT) is always the best source of evidence. However, if the outcome in a control group is certain, e.g. death within a short time with an almost 100% chance, or if an event can only occur in the treatment group, there is no need for a randomized control group; the evidence cannot be improved by a control group, nor by an RCT design. If a cause-effect relationship is certain ("primary or direct evidence"), a therapeutic effect can be diluted in the population of an RCT by cross-over, etc. This can lead to serious misinterpretations of the effect. While experts such as the GRADE group or Cochrane institutes recommend using all available evidence, the leading approach in many guidelines and HTAs is assessing "the best available trials", i.e. RCTs. But since RCTs only deliver probabilities of cause-effect relationships, it is not appropriate to demand RCTs for certain effects. A control group can only diminish the net value of a treatment since the outcome in the control group is subtracted from the outcome in the treatment group. Therefore, under identical circumstances, an RCT will always show lower effect rates compared to a single arm study of the same quality, for desired as well as for adverse effects. Considering these inconsistencies in evidence-based medicine interpretation, the evidence pyramid with RCTs at the top is not always a reliable indicator for the best quality of evidence.

[Eradication Rates of 10-day Sequential Therapy for Helicobacter pylori: Results of an 8-year Prospective Study Conducted at a Tertiary Korean Hospital]

Background/Aims

The Helicobacter pylori (H. pylori) eradication rate of standard triple therapy is unsatisfactory in Korea, and sequential therapy (SQT) has been suggested to be a practical first-line alternative regimen. The aim of this prospective study was to document changes in annual eradication rates of SQT.

Methods

A total of 983 H. pylori-positive subjects were enrolled from 2010 to 2018 and their data were subjected to intention-to-treat (ITT) and per-protocol (PP) analysis. All subjects received 10-day sequential therapy consisting of 40 mg esomeprazole and 1 g amoxicillin b.i.d for 5 days followed by 40 mg esomeprazole b.i.d, 500 mg clarithromycin b.i.d and 500 mg metronidazole t.i.d for 5 days. The ¹³C-urea breath test, rapid urease test (CLO test^®), and histology were used to confirm eradication. Compliance and side effects were also investigated.

Results

ITT and PP eradication rates of SQT were 69.9% (687 of 983) and 87.1% (657 of 754), respectively. The annual eradication rate of ITT remained consistent over the 8-year study period (p for trend=0.167), whereas PP analysis showed the eradication rate increased (p for trend=0.042). The overall adverse event rate for SQT was 41.7% (410 subjects).

Conclusions

Despite high antibiotic resistance rates in Korea, the eradication rate of SQT did not decrease over the 8-year study period.

Compliance effects in a randomised controlled trial of yoga for chronic low back pain: a methodological study

STUDY DESIGN

Methodological study nested within a multicentre randomised controlled trial (RCT) of yoga plus usual general practitioner (GP) care vs usual GP care for chronic low back pain.

OBJECTIVE

To explore the treatment effects of non-compliance using three approaches in an RCT evaluating yoga for low back pain.

SUMMARY OF BACKGROUND DATA

A large multicentre RCT using intention-to-treat (ITT) analysis found that participants with chronic low back pain who were offered a 12-week progressive programme of yoga plus usual GP care had better back function than those offered usual GP care alone. However, ITT analysis can underestimate the effect of treatment in those who comply with treatment. As such, the data were analysed using other approaches to assess the problem of non-compliance. The main outcome measure was the self-reported Roland Morris Disability Questionnaire (RMDQ).

METHODS

Complier average causal effect (CACE) analysis, per-protocol analysis and on-treatment analysis were conducted on the data of participants who were fully compliant, predefined as attendance of at least three of the first six sessions and at least three other sessions. The analysis was repeated for participants who had attended at least one yoga session (i.e. any compliance), which included participants who were fully compliant. Each approach was described, including strengths and weaknesses, and the results of the different approaches were compared with those of the ITT analysis.

RESULTS

For the participants who were fully compliant (n=93, 60%), a larger beneficial treatment effect was seen using CACE analysis compared with per-protocol, on-treatment and ITT analyses at 3 and 12 months. The difference in mean change in RMDQ score between randomised groups was -3.30 [95% confidence interval (CI) -4.90 to -1.70, P<0.001] at 3 months and -2.23 (95% CI -3.93 to -0.53, P=0.010) at 12 months for CACE analysis, -3.12 (95% CI -4.26 to -1.98, P<0.001) at 3 months and -2.11 (95% CI -3.33 to -0.89, P=0.001) at 12 months for per-protocol analysis, and -2.91 (95% CI -4.06 to -1.76, P<0.001) at 3 months and -2.10 (95% CI -3.31 to -0.89, P=0.001) at 12 months for on-treatment analysis. For the participants who demonstrated any compliance (n=133, 85%), the results were generally consistent with the fully compliant group at 3 months, but the treatment effect was smaller. The difference in mean change in RMDQ score between randomised groups was -2.45 (95% CI -3.67 to -1.24) for CACE analysis, -2.30 (95% CI -3.43 to 1.17) for per-protocol analysis and -2.15 (95% CI -3.25 to -1.06) for on-treatment analysis, which was slightly less than that for ITT analysis. In contrast, at 12 months, per-protocol and on-treatment analyses showed a larger treatment effect compared with CACE and ITT analyses: per protocol analysis -1.86 (95% CI -3.02 to -0.71), on-treatment analysis -1.99 (95% CI -3.13 to -0.86) and CACE analysis -1.67 (95% CI -2.95 to -0.40).

CONCLUSION

ITT analysis estimated a slightly smaller treatment effect in participants who complied with treatment. When examining compliance, CACE analysis is more rigorous than per-protocol and on-treatment analyses. Using CACE analysis, the treatment effect was larger in participants who complied with treatment compared with participants who were allocated to treatment, and the difference between ITT and CACE analyses for the fully compliant group at 3 months was small but clinically important. Per-protocol and on-treatment analyses may produce unreliable estimates when the effect of treatment is small.

INTERNATIONAL STANDARD RANDOMISED TRIAL NUMBER REGISTER

ISRCTN 81079604.