Evaluating the Generalisability of Trial Results: Introducing a Centre- and Trial-Level Generalisability Index

Timing of dialysis in acute kidney injury using routinely collected data and dynamic treatment regimes

Background and objectives

Defining the optimal moment to start renal replacement therapy (RRT) in acute kidney injury (AKI) remains challenging. Multiple randomized controlled trials (RCTs) addressed this question whilst using absolute criteria such as pH or serum potassium. However, there is a need for identification of the most optimal cut-offs of these criteria. We conducted a causal analysis on routinely collected data (RCD) to compare the impact of different pre-specified dynamic treatment regimes (DTRs) for RRT initiation based on time-updated levels of potassium, pH, and urinary output on 30-day ICU mortality.

Design, setting, participants, and measurements

Patients in the ICU of Ghent University Hospital were included at the time they met KDIGO-AKI-stage ≥ 2. We applied inverse-probability-of-censoring-weighted Aalen–Johansen estimators to evaluate 30-day survival under 81 DTRs prescribing RRT initiation under different thresholds of potassium, pH, or persisting oliguria.

Results

Out of 13,403 eligible patients (60.8 ± 16.8 years, SOFA 7.0 ± 4.1), 5622 (63.4 ± 15.3 years, SOFA 8.2 ± 4.2) met KDIGO-AKI-stage ≥ 2. The DTR that delayed RRT until potassium ≥ 7 mmol/l, persisting oliguria for 24–36 h, and/or pH < 7.0 (non-oliguric) or < 7.2 (oliguric) despite maximal conservative treatment resulted in a reduced 30-day ICU mortality (from 12.7% [95% CI 11.9–13.6%] under current standard of care to 10.5% [95% CI 9.5–11.7%]; risk difference 2.2% [95% CI 1.3–3.8%]) with no increase in patients starting RRT (from 471 [95% CI 430–511] to 475 [95% CI 342–572]). The fivefold cross-validation benchmark for the optimal DTR resulted in 30-day ICU mortality of 10.7%.

Conclusions

Our causal analysis of RCD to compare RRT initiation at different thresholds of refractory low pH, high potassium, and persisting oliguria identified a DTR that resulted in a decrease in 30-day ICU mortality without increase in number of RRTs. Our results suggest that the current criteria to start RRT as implemented in most RCTs may be suboptimal. However, as our analysis is hypothesis generating, this optimal DTR should ideally be validated in a multicentric RCT.

Efficacy of a 3% Kānuka oil cream for the treatment of moderate-to-severe eczema: A single blind randomised vehicle-controlled trial

BACKGROUND

Māori, the indigenous people of New Zealand, have traditionally used the kānuka tree as part of their healing system, Rongoā Māori, and the oil from the kānuka tree has demonstratable anti-inflammatory and anti-bacterial properties. This trial investigated the efficacy and safety of a 3% kānuka oil (KO) cream compared to vehicle control (VC) for the topical treatment of eczema. The trial was conducted through a nationwide community pharmacy research network.

METHODS

This single-blind, parallel-group, randomised, vehicle-controlled trial was undertaken in 11 research trained community pharmacies across New Zealand. Eighty adult participants with self-reported moderate-to-severe eczema, assessed by Patient Orientated Eczema Measure (POEM) were randomised by blinded investigators to apply 3% KO cream or VC topically, twice daily, for six weeks. Randomisation was stratified by site and eczema severity, moderate versus severe. Primary outcome was difference in POEM scores at week six between groups by intention to treat. The study is registered on the Australian New Zealand Clinical Trial Registry (ANZCTR) reference number, ACTRN12618001754235.

FINDINGS

Eighty participants were recruited between 17 May 2019 and 10 May 2021 (41 KO group, 39 VC group). Mean POEM score (standard deviation) improved between baseline and week six for KO group, 18·4 (4·4) to 6·8 (5·5), and VC group, 18·7 (4·5) to 9·8 (6·5); mean difference between groups (95% confidence interval) was -3·1 (-6·0 to -0·2), p = 0·036. There were three adverse events reported in the KO group related to the intervention and two in the control group.

INTERPRETATION

The KO group had a significant improvement in POEM score compared to VC. Rates of adverse events and withdrawals were similar between groups with no serious adverse events reported. Treatment acceptability was high for both groups across all domains. Our results suggest that in adults with moderate-to-severe eczema, the addition of KO to a daily emollient regimen led to a reduction in POEM score compared to VC. KO may represent an effective, safe, and well tolerated treatment for moderate-to-severe eczema in adults.

FUNDING

Hikurangi Bioactives (Ruatoria, New Zealand) and HoneyLab (Tauranga, New Zealand), supported by a grant from Callaghan Innovation.

Identification of tools used to assess the external validity of randomized controlled trials in reviews: a systematic review of measurement properties

Background

Internal and external validity are the most relevant components when critically appraising randomized controlled trials (RCTs) for systematic reviews. However, there is no gold standard to assess external validity. This might be related to the heterogeneity of the terminology as well as to unclear evidence of the measurement properties of available tools. The aim of this review was to identify tools to assess the external validity of RCTs. It was further, to evaluate the quality of identified tools and to recommend the use of individual tools to assess the external validity of RCTs in future systematic reviews.

Methods

A two-phase systematic literature search was performed in four databases: PubMed, Scopus, PsycINFO via OVID, and CINAHL via EBSCO. First, tools to assess the external validity of RCTs were identified. Second, studies investigating the measurement properties of these tools were selected. The measurement properties of each included tool were appraised using an adapted version of the COnsensus based Standards for the selection of health Measurement INstruments (COSMIN) guidelines.

Results

38 publications reporting on the development or validation of 28 included tools were included. For 61% (17/28) of the included tools, there was no evidence for measurement properties. For the remaining tools, reliability was the most frequently assessed property. Reliability was judged as “sufficient” for three tools (very low certainty of evidence). Content validity was rated as “sufficient” for one tool (moderate certainty of evidence).

Conclusions

Based on these results, no available tool can be fully recommended to assess the external validity of RCTs in systematic reviews. Several steps are required to overcome the identified difficulties to either adapt and validate available tools or to develop a better suitable tool.

Trial registration

Prospective registration at Open Science Framework (OSF): 10.17605/OSF.IO/PTG4D.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12874-022-01561-5.

The concept of justifiable healthcare and how big data can help us to achieve it

Over the last decades, the face of health care has changed dramatically, with big improvements in what is technically feasible. However, there are indicators that the current approach to evaluating evidence in health care is not holistic and hence in the long run, health care will not be sustainable. New conceptual and normative frameworks for the evaluation of health care need to be developed and investigated. The current paper presents a novel framework of justifiable health care and explores how the use of artificial intelligence and big data can contribute to achieving the goals of this framework.

Kanuka honey versus aciclovir for the topical treatment of herpes simplex labialis: a randomised controlled trial

OBJECTIVE

To compare New Zealand medical grade kanuka honey with topical aciclovir for the treatment of herpes simplex labialis.

DESIGN

Prospective parallel randomised controlled open-label superiority trial.

SETTING

76 community pharmacies across New Zealand between 10 September 2015 and 13 December 2017.

PARTICIPANTS

952 adults randomised within the first 72 hours of a herpes simplex labialis episode.

INTERVENTIONS

Random assignment 1:1 to either 5% aciclovir cream or medical grade kanuka honey (90%)/glycerine (10%) cream, both applied five times daily.

OUTCOME MEASURES

The primary outcome was time from randomisation to return to normal skin (stage 7). Secondary outcomes included time from randomisation to stage 4 (open wound), time from stage 4 to 7, maximal pain, time to pain resolution and treatment acceptability.

RESULTS

Primary outcome variable: Kaplan-Meier-based estimates (95% CI) for the median time in days for return to normal skin were 8 (8 to 9) days for aciclovir and 9 (8 to 9) for honey; HR (95% CI) 1.06 (0.92 to 1.22), p=0.56. There were no statistically significant differences between treatments for all secondary outcome variables. No related serious adverse events were reported.

CONCLUSION

There was no evidence of a difference in efficacy between topical medical grade kanuka honey and 5% aciclovir in the pharmacy-based treatment of herpes simplex labialis.

TRIAL REGISTRATION NUMBER

ACTRN12615000648527;Post-results.

Design, implementation and reporting strategies to reduce the instance and impact of missing patient-reported outcome (PRO) data: a systematic review

OBJECTIVES

Patient-reported outcomes (PROs) provide important information about the impact of treatment from the patients' perspective. However, missing PRO data may compromise the interpretability and value of the findings. We aimed to report: (1) a non-technical summary of problems caused by missing PRO data; and (2) a systematic review by collating strategies to: (A) minimise rates of missing PRO data, and (B) facilitate transparent interpretation and reporting of missing PRO data in clinical research. Our systematic review does not address statistical handling of missing PRO data.

DATA SOURCES

MEDLINE and Cumulative Index to Nursing and Allied Health Literature (CINAHL) databases (inception to 31 March 2015), and citing articles and reference lists from relevant sources.

ELIGIBILITY CRITERIA

English articles providing recommendations for reducing missing PRO data rates, or strategies to facilitate transparent interpretation and reporting of missing PRO data were included.

METHODS

2 reviewers independently screened articles against eligibility criteria. Discrepancies were resolved with the research team. Recommendations were extracted and coded according to framework synthesis.

RESULTS

117 sources (55% discussion papers, 26% original research) met the eligibility criteria. Design and methodological strategies for reducing rates of missing PRO data included: incorporating PRO-specific information into the protocol; carefully designing PRO assessment schedules and defining termination rules; minimising patient burden; appointing a PRO coordinator; PRO-specific training for staff; ensuring PRO studies are adequately resourced; and continuous quality assurance. Strategies for transparent interpretation and reporting of missing PRO data include utilising auxiliary data to inform analysis; transparently reporting baseline PRO scores, rates and reasons for missing data; and methods for handling missing PRO data.

CONCLUSIONS

The instance of missing PRO data and its potential to bias clinical research can be minimised by implementing thoughtful design, rigorous methodology and transparent reporting strategies. All members of the research team have a responsibility in implementing such strategies.

Recruitment, screening, and baseline participant characteristics in the WALK 2.0 study: A randomized controlled trial using web 2.0 applications to promote physical activity

OBJECTIVE

To describe in detail the recruitment methods and enrollment rates, the screening methods, and the baseline characteristics of a sample of adults participating in the Walk 2.0 Study, an 18 month, 3-arm randomized controlled trial of a Web 2.0 based physical activity intervention.

METHODS

A two-fold recruitment plan was developed and implemented, including a direct mail-out to an extract from the Australian Electoral Commission electoral roll, and other supplementary methods including email and telephone. Physical activity screening involved two steps: a validated single-item self-report instrument and the follow-up Active Australia Questionnaire. Readiness for physical activity participation was also based on a two-step process of administering the Physical Activity Readiness Questionnaire and, where needed, further clearance from a medical practitioner.

RESULTS

Across all recruitment methods, a total of 1244 participants expressed interest in participating, of which 656 were deemed eligible. Of these, 504 were later enrolled in the Walk 2.0 trial (77% enrollment rate) and randomized to the Walk 1.0 group (n = 165), the Walk 2.0 group (n = 168), or the Logbook group (n = 171). Mean age of the total sample was 50.8 years, with 65.2% female and 79.1% born in Australia.

CONCLUSION

The results of this recruitment process demonstrate the successful use of multiple strategies to obtain a diverse sample of adults eligible to take part in a web-based physical activity promotion intervention. The use of dual screening processes ensured safe participation in the intervention. This approach to recruitment and physical activity screening can be used as a model for further trials in this area.