Linking quality indicators to clinical trials: an automated approach

Selection of indicators reporting response rate in pharmaceutical trials for systemic lupus erythematosus: preference and relative sensitivity

OBJECTIVE

SLE is a common multisystem autoimmune disease with chronic inflammation. Many efficacy evaluation indicators of randomised clinical trials (RCTs) for SLE have been proposed but the comparability remains unknown. We aim to explore the preference and comparability of indicators reporting response rate and provide basis for primary outcome selection when evaluating the efficacy of SLE pharmaceutical treatment.

METHODS

We systematically searched three databases and three registries to identify pharmacological intervention-controlled SLE RCTs. Relative discriminations between indicators were assessed by the Bayesian hierarchical linear mixed model.

RESULTS

33 RCTs met our inclusion criteria and we compared eight of the most commonly used indicators reporting response rate. SLE Disease Activity Index 4 (SLEDAI-4) and SLE Responder Index 4 were considered the best recommended indicators reporting response rate to discriminate the pharmacological efficacy. Indicator preference was altered by disease severity, classification of drugs and outcome of trials, but SLEDAI-4 had robust efficacy in discriminating ability for most interventions. Of note, BILAG Index-based Combined Lupus Assessment showed efficacy in trials covering all-severity patients, as well as non-biologics RCTs. The British Isles Lupus Assessment Group response and Physician's Global Assessment response were more cautious in evaluating disease changes. Serious adverse event was often applied to evaluate the safety and tolerability of treatments rather than efficacy.

CONCLUSIONS

The impressionable efficacy discrimination ability of indicators highlights the importance of flexibility and comprehensiveness when choosing primary outcome(s). As for trials that are only evaluated by SLEDAI-4, attention should be paid to outcome interpretation to avoid the exaggeration of treatment efficacy. Further subgroup analyses are limited by the number of included RCTs.

PROSPERO REGISTRATION NUMBER

CRD42022334517.

Does health informatics have a replication crisis?

Objective

Many research fields, including psychology and basic medical sciences, struggle with poor reproducibility of reported studies. Biomedical and health informatics is unlikely to be immune to these challenges. This paper explores replication in informatics and the unique challenges the discipline faces.

Methods

Narrative review of recent literature on research replication challenges.

Results

While there is growing interest in re-analysis of existing data, experimental replication studies appear uncommon in informatics. Context effects are a particular challenge as they make ensuring replication fidelity difficult, and the same intervention will never quite reproduce the same result in different settings. Replication studies take many forms, trading-off testing validity of past findings against testing generalizability. Exact and partial replication designs emphasize testing validity while quasi and conceptual studies test generalizability of an underlying model or hypothesis with different methods or in a different setting.

Conclusions

The cost of poor replication is a weakening in the quality of published research and the evidence-based foundation of health informatics. The benefits of replication include increased rigor in research, and the development of evaluation methods that distinguish the impact of context and the nonreproducibility of research. Taking replication seriously is essential if biomedical and health informatics is to be an evidence-based discipline.

Clinical Trial Assessment Principles of National Class III Medical Devices in China

Class III medical devices are defined as those which are implanted inside the human body and applied to maintain normal life and retain original tissue or organic functions. Because these devices are associated with high risk, their effectiveness and safety should be strictly monitored and clinically investigated. The aim of clinical investigation of these medical devices is to ensure the acceptability of their effectiveness and safety levels. On designing the clinical trial, the investigator should determine the indices to assess the effectiveness and safety of medical devices, select reasonable data‐analyzing methods, and pay attention to several other issues. Although some guidelines on specific class III medical devices have illustrated those aspects in detail, there is still no comprehensive report that details all those principles and methodologies. This article aims to summarize the common features among the instruction principles and provide technological support for the clinical study of class III medical devices.

Clinical indicators for common paediatric conditions: Processes, provenance and products of the CareTrack Kids study

BACKGROUND

In order to determine the extent to which care delivered to children is appropriate (in line with evidence-based care and/or clinical practice guidelines (CPGs)) in Australia, we developed a set of clinical indicators for 21 common paediatric medical conditions for use across a range of primary, secondary and tertiary healthcare practice facilities.

METHODS

Clinical indicators were extracted from recommendations found through systematic searches of national and international guidelines, and formatted with explicit criteria for inclusion, exclusion, time frame and setting. Experts reviewed the indicators using a multi-round modified Delphi process and collaborative online wiki to develop consensus on what constituted appropriate care.

RESULTS

From 121 clinical practice guidelines, 1098 recommendations were used to draft 451 proposed appropriateness indicators. In total, 61 experts (n = 24 internal reviewers, n = 37 external reviewers) reviewed these indicators over 40 weeks. A final set of 234 indicators resulted, from which 597 indicator items were derived suitable for medical record audit. Most indicator items were geared towards capturing information about under-use in healthcare (n = 551, 92%) across emergency department (n = 457, 77%), hospital (n = 450, 75%) and general practice (n = 434, 73%) healthcare facilities, and based on consensus level recommendations (n = 451, 76%). The main reason for rejecting indicators was 'feasibility' (likely to be able to be used for determining compliance with 'appropriate care' from medical record audit).

CONCLUSION

A set of indicators was developed for the appropriateness of care for 21 paediatric conditions. We describe the processes (methods), provenance (origins and evolution of indicators) and products (indicator characteristics) of creating clinical indicators within the context of Australian healthcare settings. Developing consensus on clinical appropriateness indicators using a Delphi approach and collaborative online wiki has methodological utility. The final indicator set can be used by clinicians and organisations to measure and reflect on their own practice.