A comparison of approaches for adjudicating outcomes in clinical trials

Efficacy and Safety of P2Y12 monotherapy vs standard DAPT in patients undergoing percutaneous coronary intervention: meta-analysis of randomized trials

BACKGROUND

Debates persist regarding the optimal duration of dual antiplatelet therapy (DAPT) after percutaneous coronary intervention (PCI) in coronary artery disease (CAD). Recent trials have introduced a novel approach involving P2Y12 inhibitor monotherapy with ticagrelor or clopidogrel, after a short DAPT. However, the effectiveness and safety of this strategy remains to be established. We aimed to perform a meta-analysis comparing monotherapy with P2Y12 inhibitors versus standard DAPT in patients undergoing PCI at 12 months.

METHODS

Multiple databases were searched. Six RCTs with a total of 24877 patients were included. The primary endpoint was all-cause mortality at 12 months of follow-up. The secondary endpoints were cardiovascular mortality, myocardial infarction, probable or definite stent thrombosis, stroke events, and major bleeding. The study is registered with PROSPERO (CRD42024499529).

RESULTS

Monotherapy with P2Y12 inhibitor ticagrelor significantly reduced both allcause mortality (HR 0.71, 95 CI [0.55-0.91], P = 0.007) and cardiovascular mortality (HR 0.66, 95% CI [0.49-0.89], P = 0.006) compared to standard DAPT. In contrast, clopidogrel monotherapy did not demonstrate a similar reduction. The decrease in mortality associated with ticagrelor was primarily due to a lower risk of major bleeding (HR 0.56, 95% CI [0.43-0.72], P < 0.001), while the risk of myocardial infarction (MI) remained unchanged (HR 0.90, 95% CI [0.73-1.11], P = 0.32). The risk of stroke was found to be similar across treatments.

CONCLUSIONS

In comparison to standard DAPT, P2Y12 inhibitor monotherapy with ticagrelor may lead to a reduced mortality. The clinical benefits are driven by a reduction of bleeding risk without ischemic risk trade-off.

Adjudication of a primary trial outcome: Results of a calibration exercise and protocol for a large international trial

Background

Ascertainment of the severity of the primary outcome of upper gastrointestinal (GI) bleeding is integral to stress ulcer prophylaxis trials. This protocol outlines the adjudication process for GI bleeding events in an international trial comparing pantoprazole to placebo in critically ill patients (REVISE: Re-Evaluating the Inhibition of Stress Erosions). The primary objective of the adjudication process is to assess episodes submitted by participating sites to determine which fulfil the definition of the primary efficacy outcome of clinically important upper GI bleeding. Secondary objectives are to categorize the bleeding severity if deemed not clinically important, and adjudicate the bleeding site, timing, investigations, and treatments.

Methods

Research coordinators follow patients daily for any suspected clinically important upper GI bleeding, and submit case report forms, doctors' and nurses' notes, laboratory, imaging, and procedural reports to the methods center. An international central adjudication committee reflecting diverse specialty backgrounds conducted an initial calibration exercise to delineate the scope of the adjudication process, review components of the definition, and agree on how each criterion will be considered fulfilled. Henceforth, bleeding events will be stratified by study drug, and randomly assigned to adjudicator pairs (blinded to treatment allocation, and study center).

Results

Crude agreement, chance-corrected agreement, or chance-independent agreement if data have a skewed distribution will be calculated.

Conclusions

Focusing on consistency and accuracy, central independent blinded duplicate adjudication of suspected clinically important upper GI bleeding events will determine which events fulfil the definition of the primary efficacy outcome for this stress ulcer prophylaxis trial.

Registration

NCT03374800 (REVISE: Re-Evaluating the Inhibition of Stress Erosions).

Improvement of assessment in surrogate endpoint and safety outcome of single-arm trials for anticancer drugs

INTRODUCTION

Single-arm trials (SATs) and surrogate endpoints were adopted as pivotal evidence for accelerated approval of anticancer drugs for more than 30 years. However, concerns regarding clinical evidence quality in trials, particularly in the SATs of anticancer drugs have increasingly been raised. SAT may not always provide strong evidence due to the lack of control and endpoint of overall survival that is typically present in randomized controlled trials.

AREAS COVERED

Clinical trial endpoint adjudication is a crucial factor in surrogate outcome measurement to ensure the data quality of the clinical trial of anticancer drugs. In this review, we systematically discuss the characteristics of adjudications in assessments in surrogate endpoint and safety outcome respectively, which are essential for ensuring reliable and transparent outcomes. Endpoint adjudication effectively reduces potential bias and mitigates variance that may be introduced by investigators when analyzing the medical records for the surrogate endpoints. We analyze the advantages and disadvantages of each type of adjudicator and provide a summary of the roles of adjudicators.

EXPERT OPINION

By suggestion of improving data reliability and transparency in pivotal trials, this review aims to supply a strategy for better clinical investigation for anticancer drugs, ultimately leading to better patient outcomes.

Massive transfusion protocol reactivation as a novel marker of physician team under-triage after injury

BACKGROUND

Large trauma centers have protocols for the assessment of injury and triaging of care with attempts to over-triage to ensure adequate care for all patients. We noted that a significant number of patients undergo a second massive transfusion protocol (MTP) activation in the first 24 h of care and conducted a retrospective cohort study of patients involved over a 3-year period.

METHODS

Transfusion service records of MTP activations 2019-2021 were linked to Trauma Registry records and divided into cohorts receiving a single versus a reactivation of the MTP. Time of activation and amounts of blood products issued were linked to demographic, injury severity, and outcome data. Categorical and continuous data were compared between cohorts with chi-squared, Fisher's, and Wilcoxan tests as appropriate, and multivariable regression models were used to seek interactions (p < .05).

RESULTS

MTP activation was recorded for 1884 acute trauma patients over our 3-year study period, 142 of whom (7.5%) had reactivation. Factors associated with reactivation included older age (46 vs. 40 years), higher injury severity score (ISS, 27 vs. 22), leg injuries, and presentation during morning shift change (5-7 a.m., 3.3% vs. 7.7%). Patients undergoing MTP reactivation used more RBCs (5 U vs. 2 U) and had more ICU days (3 vs. 2).

CONCLUSIONS

Older patients and those presenting during shift change are at risk for failure to recognize their complex injury patterns and under-triage for trauma care. The fidelity and granularity of transfusion service records can provide unique opportunities for quality assessment and improvement in trauma care.

Can we predict discordant RECIST 1.1 evaluations in double read clinical trials?

Background

In lung clinical trials with imaging, blinded independent central review with double reads is recommended to reduce evaluation bias and the Response Evaluation Criteria In Solid Tumor (RECIST) is still widely used. We retrospectively analyzed the inter-reader discrepancies rate over time, the risk factors for discrepancies related to baseline evaluations, and the potential of machine learning to predict inter-reader discrepancies.

Materials and methods

We retrospectively analyzed five BICR clinical trials for patients on immunotherapy or targeted therapy for lung cancer. Double reads of 1724 patients involving 17 radiologists were performed using RECIST 1.1. We evaluated the rate of discrepancies over time according to four endpoints: progressive disease declared (PDD), date of progressive disease (DOPD), best overall response (BOR), and date of the first response (DOFR). Risk factors associated with discrepancies were analyzed, two predictive models were evaluated.

Results

At the end of trials, the discrepancy rates between trials were not different. On average, the discrepancy rates were 21.0%, 41.0%, 28.8%, and 48.8% for PDD, DOPD, BOR, and DOFR, respectively. Over time, the discrepancy rate was higher for DOFR than DOPD, and the rates increased as the trial progressed, even after accrual was completed. It was rare for readers to not find any disease, for less than 7% of patients, at least one reader selected non-measurable disease only (NTL). Often the readers selected some of their target lesions (TLs) and NTLs in different organs, with ranges of 36.0-57.9% and 60.5-73.5% of patients, respectively. Rarely (4-8.1%) two readers selected all their TLs in different locations. Significant risk factors were different depending on the endpoint and the trial being considered. Prediction had a poor performance but the positive predictive value was higher than 80%. The best classification was obtained with BOR.

Conclusion

Predicting discordance rates necessitates having knowledge of patient accrual, patient survival, and the probability of discordances over time. In lung cancer trials, although risk factors for inter-reader discrepancies are known, they are weakly significant, the ability to predict discrepancies from baseline data is limited. To boost prediction accuracy, it would be necessary to enhance baseline-derived features or create new ones, considering other risk factors and looking into optimal reader associations.

An Adjudication Protocol for Severe Pneumonia

Background

Clinical end points that constitute successful treatment in severe pneumonia are difficult to ascertain and vulnerable to bias. The utility of a protocolized adjudication procedure to determine meaningful end points in severe pneumonia has not been well described.

Methods

This was a single-center prospective cohort study of patients with severe pneumonia admitted to the medical intensive care unit. The objective was to develop an adjudication protocol for severe bacterial and/or viral pneumonia. Each episode of pneumonia was independently reviewed by 2 pulmonary and critical care physicians. If a discrepancy occurred between the 2 adjudicators, a third adjudicator reviewed the case. If a discrepancy remained after all 3 adjudications, consensus was achieved through committee review.

Results

Evaluation of 784 pneumonia episodes during 593 hospitalizations achieved only 48.1% interobserver agreement between the first 2 adjudicators and 78.8% when agreement was defined as concordance between 2 of 3 adjudicators. Multiple episodes of pneumonia and presence of bacterial/viral coinfection in the initial pneumonia episode were associated with lower interobserver agreement. For an initial episode of bacterial pneumonia, patients with an adjudicated day 7-8 clinical impression of cure (compared with alternative impressions) were more likely to be discharged alive (odds ratio, 6.3; 95% CI, 3.5-11.6).

Conclusions

A comprehensive adjudication protocol to identify clinical end points in severe pneumonia resulted in only moderate interobserver agreement. An adjudicated end point of clinical cure by day 7-8 was associated with more favorable hospital discharge dispositions, suggesting that clinical cure by day 7-8 may be a valid end point to use in adjudication protocols.

Should we adjudicate outcomes in stroke trials? A systematic review

BACKGROUND

Central adjudication of outcomes is common in randomized clinical trials in stroke. The rationale for adjudication is clear; centrally adjudicated outcomes should have less random and systematic errors than outcomes assessed locally by site investigators. However, adjudication brings added complexities to a clinical trial and can be costly.

AIM

To assess the evidence for outcome adjudication in stroke trials.

SUMMARY OF REVIEW

We identified 12 studies evaluating central adjudication in stroke trials. The majority of these were secondary analyses of trials, and the results of all of these would have remained unchanged had central adjudication not taken place, even for trials without sufficient blinding. The largest differences between site-assessed and adjudicator-assessed outcomes were between the most subjective outcomes, such as causality of serious adverse events. We found that the cost of adjudication could be upward of £100,000 for medium to large prevention trials. These findings suggest that the cost of central adjudication may outweigh the advantages it brings in many cases. However, through simulation, we found that only a small amount of bias is required in site investigators' outcome assessments before adjudication becomes important.

CONCLUSION

Central adjudication may not be necessary in stroke trials with blinded outcome assessment. However, for open-label studies, central adjudication may be more important.

Endocrine and multiple sclerosis outcomes in patients with autoimmune thyroid events in the alemtuzumab CARE-MS studies

Background

Alemtuzumab is an effective therapy for relapsing multiple sclerosis. Autoimmune thyroid events are a common adverse event.

Objective

Describe endocrine and multiple sclerosis outcomes over 6 years for alemtuzumab-treated relapsing multiple sclerosis patients in the phase 3 CARE-MS I, II, and extension studies who experienced adverse thyroid events.

Methods

Endocrine and multiple sclerosis outcomes were evaluated over 6 years. Thyroid event cases, excluding those pre-existing or occurring after Year 6, were adjudicated retrospectively by expert endocrinologists independently of the sponsor and investigators.

Results

Thyroid events were reported for 378/811 (46.6%) alemtuzumab-treated patients. Following adjudication, endocrinologists reached consensus on 286 cases (75.7%). Of these, 39.5% were adjudicated to Graves' disease, 2.5% Hashimoto's disease switching to hyperthyroidism, 15.4% Hashimoto's disease, 4.9% Graves' disease switching to hypothyroidism, 10.1% transient thyroiditis, and 27.6% with uncertain diagnosis; inclusion of anti-thyroid antibody status reduced the number of uncertain diagnoses. Multiple sclerosis outcomes of those with and without thyroid events were similar.

Conclusion

Adjudicated thyroid events occurring over 6 years for alemtuzumab-treated relapsing multiple sclerosis patients were primarily autoimmune. Thyroid events were considered manageable and did not affect disease course. Thyroid autoimmunity is a common but manageable adverse event in alemtuzumab-treated relapsing multiple sclerosis patients.ClinicalTrials.gov Registration Numbers: CARE-MS I (NCT00530348); CARE-MS II (NCT00548405); CARE-MS Extension (NCT00930553).

Comparison of Investigator-Reported and Centrally Adjudicated Heart Failure Outcomes in the EMPEROR-Reduced Trial

BACKGROUND

There is limited published information on outcome adjudication in heart failure (HF).

OBJECTIVE

The authors sought to compare investigator reports (IRs) to a Clinical Events Committee (CEC) and the impact of Standardized Clinical Trial Initiative definitions (SCTI).

METHODS

In EMPEROR-Reduced, the authors compared IR to CEC for concordance; treatment effect on primary composite outcome events; and the components first event hospitalization primarily for HF (HHF) or cardiovascular mortality (CVM), prognosis after HHF, total HHFs, and trial duration with and without SCTI.

RESULTS

The CEC confirmed 76.3% of IR events for the primary outcome (CVM: 89.1%; HHF: 73.7%). The HR for treatment effect did not differ between adjudication methods for the primary outcome (IR: 0.75 [95% CI: 0.66-0.85]; CEC: 0.75 [95% CI: 0.65-0.86]), its components, or total HHFs. The prognosis after first HHF for all-cause mortality and CVM also did not differ between IR or CEC. Interestingly, IR primary HHF with different CEC primary cause had the highest subsequent fatal event rate. Full SCTI criteria were present in 90% of CEC HHFs-with a similar treatment effect to non-SCTI. The IR primary event reached the protocol target number (841) 3 months earlier than CEC (4 months with full SCTI criteria).

CONCLUSIONS

Investigator adjudication is an alternative to a CEC with similar accuracy and faster event accumulation. The use of granular (SCTI) criteria did not improve trial performance. Finally, our data suggest that consideration be given to broadening the HHF definition to include "for or with" worsening disease. (Empagliflozin Outcome Trial in Patients With Chronic Heart Failure With Reduced Ejection Fraction [EMPEROR-Reduced]; NCT03057977).

Heterogeneity in pragmatic randomised trials: sources and management

BACKGROUND

Pragmatic trials aim to generate evidence to directly inform patient, caregiver and health-system manager policies and decisions. Heterogeneity in patient characteristics contributes to heterogeneity in their response to the intervention. However, there are many other sources of heterogeneity in outcomes. Based on the expertise and judgements of the authors, we identify different sources of clinical and methodological heterogeneity, which translate into heterogeneity in patient responses-some we consider as desirable and some as undesirable. For each of them, we discuss and, using real-world trial examples, illustrate how heterogeneity should be managed over the whole course of the trial.

MAIN TEXT

Heterogeneity in centres and patients should be welcomed rather than limited. Interventions can be flexible or tailored and control interventions are expected to reflect usual care, avoiding use of a placebo. Co-interventions should be allowed; adherence should not be enforced. All these elements introduce heterogeneity in interventions (experimental or control), which has to be welcomed because it mimics reality. Outcomes should be objective and possibly routinely collected; standardised assessment, blinding and adjudication should be avoided as much as possible because this is not how assessment would be done outside a trial setting. The statistical analysis strategy must be guided by the objective to inform decision-making, thus favouring the intention-to-treat principle. Pragmatic trials should consider including process analyses to inform an understanding of the trial results. Needed data to conduct these analyses should be collected unobtrusively. Finally, ethical principles must be respected, even though this may seem to conflict with goals of pragmatism; consent procedures could be incorporated in the flow of care.

Frequency of Asymptomatic Optic Nerve Enhancement in a Large Retrospective Cohort of Patients With Aquaporin-4+ NMOSD

BACKGROUND AND OBJECTIVES

Asymptomatic or persistent optic nerve enhancement in aquaporin-4 (AQP4)-immunoglobulin G (IgG)-positive neuromyelitis optica spectrum disorder (NMOSD) is thought to be rare. Improved understanding may have important implications for assessment of treatment efficacy in clinical trials and in clinical practice. Our objective was to characterize NMOSD interattack optic nerve enhancement.

METHODS

This was a retrospective cohort study performed between 2000 and 2019 (median follow-up 5.5 [range 1-35] years) of patients with AQP4-IgG-positive optic neuritis (ON) evaluated at Mayo Clinic. MRI orbits were reviewed by a neuroradiologist, neuro-ophthalmologist, and neuroimmunologist blinded to the clinical history. Interattack optic nerve enhancement (>30 days after attack) was measured. The correlation between interattack enhancement and Snellen visual acuity (VA), converted to logarithm of the minimum angle of resolution (logMAR), at attack and at follow-up were assessed.

RESULTS

A total of 198 MRI scans in 100 patients with AQP4-IgG+ NMOSD were identified, with 107 interattack MRIs from 78 unique patients reviewed. Seven scans were performed before any ON (median 61 days before attack [range 21-271 days]) and 100 after ON (median 400 days after attack [33-4,623 days]). Optic nerve enhancement was present on 18/107 (16.8%) interattack scans (median 192.5 days from attack [33-2,943]) of patients with preceding ON. On 15 scans, enhancement occurred at the site of prior attacks; the lesion location was unchanged, but the lesion length was shorter. Two scans (1.8%) demonstrated new asymptomatic lesions (prior scan demonstrated no enhancement). In a third patient with subjective blurry vision, MRI showed enhancement preceding detectable eye abnormalities on examination noted 15 days later. There was no difference in VA at preceding attack nadir (logMAR VA 1.7 vs 2.1; p = 0.79) or long-term VA (logMAR VA 0.4 vs 0.2, p = 0.56) between those with and without interattack optic nerve enhancement.

DISCUSSION

Asymptomatic optic nerve enhancement occurred in 17% of patients with NMOSD predominantly at the site of prior ON attacks and may represent intermittent blood-brain barrier breakdown or subclinical ON. New asymptomatic enhancement was seen only in 2% of patients. Therapeutic clinical trials for NMOSD require blinded relapse adjudication when assessing treatment efficacy, and it is important to recognize that asymptomatic optic nerve enhancement can occur in patients with ON.

Comparing control room operators' and experts' assessment of team performance using structured task-specific observation protocols and scenario replay

Operators' self-assessment has received limited interest within process control or human-system evaluation. Research on self-assessment has been criticised for poor assessment methodology, and consequently, its status is unclear. This study hypothesised that, given adequate assessment methods (such as task-specific assessment items and scenario replay), we could observe relatively accurate self-assessment results. Eighteen licensed operators and two experts assessed team performance in six nuclear control room scenarios. The results reveal an overall agreement between operators and experts, measured by the intraclass correlation coefficient, ranging from 0.60 to 0.70, which lies close to the intraclass correlation coefficient of 0.75 for the experts. This demonstrates potential for achievement of relatively accurate operator self-assessment for complex work. The agreement varied in a similar manner for both expert agreement and operator-expert agreement across eight performance dimensions. In addition, the operators' self-assessment provided additional information beyond observer assessment in identifying non-acceptable performance items.

Comparison of Clinically Adjudicated Versus Flow-Based Adjudication of Revascularization Events in Randomized Controlled Trials

BACKGROUND

In clinical trials, the optimal method of adjudicating revascularization events as clinically or nonclinically indicated (CI) is to use an independent Clinical Events Committee (CEC). However, the Academic Research Consortium-2 currently recommends using physiological assessment. The level of agreement between these methods of adjudication remains unknown.

METHODS

Data for all CEC adjudicated revascularization events among the 3457 patients followed-up for 2-years in the TALENT trial, and 3-years in the DESSOLVE III, PIONEER, and SYNTAX II trial were collected and readjudicated according to a quantitative flow ratio (QFR) analysis of the revascularized vessels, by an independent core lab blinded to the results of the conventional CEC adjudication. The κ statistic was used to assess the level of agreement between the 2 methods.

RESULTS

In total, 351 CEC-adjudicated repeat revascularization events occurred, with retrospective QFR analysis successfully performed in 212 (60.4%). According to QFR analysis, 104 events (QFR ≤0.80) were adjudicated as CI revascularizations and 108 (QFR >0.80) were not. The agreement between CEC and QFR based adjudication was just fair (κ=0.335). Between the 2 methods of adjudication, there was a disagreement of 26.4% and 7.1% in CI and non-CI revascularization, respectively. Overall, the concordance and discordance rates were 66.5% and 33.5%, respectively.

CONCLUSIONS

In this event-level analysis, QFR based adjudication had a relatively low agreement with CEC adjudication with respect to whether revascularization events were CI or not. CEC adjudication appears to overestimate CI revascularization as compared with QFR adjudication. Direct comparison between these 2 strategies in terms of revascularization adjudication is warranted in future trials. Registration: URL: https://www.clinicaltrials.gov; Unique identifier: TALENT trial: NCT02870140, DESSOLVE III trial: NCT02385279, SYNTAX II: NCT02015832, and PIONEER trial: NCT02236975.

Cancer Clinical Trials: What Every Radiologist Wants to Know but Is Afraid to Ask

OBJECTIVE. The purpose of this article is to provide radiologists with a guide to the fundamental principles of oncology clinical trials. The review summarizes the evolution and structure of modern clinical trials with an emphasis on the relevance of clinical trials in the field of oncologic imaging. CONCLUSION. Understanding the structure and clinical relevance of modern clinical trials is beneficial for radiologists in the field of oncologic imaging.

Completeness and accuracy of national cancer and death registration for outcome ascertainment in trials-an ovarian cancer exemplar

BACKGROUND

There is a trend to increasing use of routinely collected health data to ascertain outcome measures in trials. We report on the completeness and accuracy of national ovarian cancer and death registration in the United Kingdom Collaborative Trial of Ovarian Cancer Screening (UKCTOCS).

METHODS

Of the 202,638 participants, 202,632 were successfully linked and followed through national cancer and death registries of Northern Ireland, Wales and England. Women with registrations of any of 19 pre-defined ICD-10 codes suggestive of tubo-ovarian cancer or notification of ovarian/tubal/peritoneal cancer from hospital episode statistics or trial sites were identified. Copies of hospital and primary care notes were retrieved and reviewed by an independent outcomes review committee. National registration of site and cause of death as ovarian/tubal/peritoneal cancer (C56/C57/C48) obtained up to 3 months after trial censorship was compared to that assigned by outcomes review (reference standard).

RESULTS

Outcome review was undertaken in 3110 women on whom notification was received between 2001 and 2014. Ovarian cancer was confirmed in 1324 of whom 1125 had a relevant cancer registration. Sensitivity and specificity of ovarian/tubal/peritoneal cancer registration were 85.0% (1125/1324; 95% CI 83.7-86.2%) and 94.0% (1679/1786; 95% CI 93.2-94.8%), respectively. Of 2041 death registrations reviewed, 681 were confirmed to have a tubo-ovarian cancer of whom 605 had relevant death registration. Sensitivity and specificity were 88.8% (605/681; 95% CI 86.4-91.2%) and 96.7% (1482/1533, 95% CI 95.8-97.6%), respectively. When multiple electronic health record sources were considered, sensitivity for cancer site increased to 91.1% (1206/1324, 95% CI 89.4-92.5%) and for cause of death 94.0% (640/681, 95% CI 91.9-95.5%). Of 1232 with cancer registration, 8.7% (107/1232) were wrongly designated as ovarian/tubal/peritoneal cancers by the registry and 4.0% (47/1172) of confirmed tubo-ovarian cancers were mis-registered. In 656 with death registrations, 7.8% (51/656) were wrongly assigned as due to ovarian/tubal/peritoneal cancers while 6.2% (40/645) of confirmed tubo-ovarian cancer deaths were mis-registered.

CONCLUSION

Follow-up of trial participants for tubo-ovarian cancer using national registry data will result in incomplete ascertainment, particularly of the site due in part to the latency of registration. This can be reduced by using other routinely collected data such as hospital episode statistics. Central adjudication by experts though resource intensive adds value by improving the accuracy of diagnoses.

TRIAL REGISTRATION

ISRCTN: ISRCTN22488978 . Registered on 6 April 2000.

Diagnostic Errors Are Common in Acute Pediatric Respiratory Disease: A Prospective, Single-Blinded Multicenter Diagnostic Accuracy Study in Australian Emergency Departments

Background: Diagnostic errors are a global health priority and a common cause of preventable harm. There is limited data available for the prevalence of misdiagnosis in pediatric acute-care settings. Respiratory illnesses, which are particularly challenging to diagnose, are the most frequent reason for presentation to pediatric emergency departments. Objective: To evaluate the diagnostic accuracy of emergency department clinicians in diagnosing acute childhood respiratory diseases, as compared with expert panel consensus (reference standard). Methods: Prospective, multicenter, single-blinded, diagnostic accuracy study in two well-resourced pediatric emergency departments in a large Australian city. Between September 2016 and August 2018, a convenience sample of children aged 29 days to 12 years who presented with respiratory symptoms was enrolled. The emergency department discharge diagnoses were reported by clinicians based upon standard clinical diagnostic definitions. These diagnoses were compared against consensus diagnoses given by an expert panel of pediatric specialists using standardized disease definitions after they reviewed all medical records. Results: For 620 participants, the sensitivity and specificity (%, [95% CI]) of the emergency department compared with the expert panel diagnoses were generally poor: isolated upper respiratory tract disease (64.9 [54.6, 74.4], 91.0 [88.2, 93.3]), croup (76.8 [66.2, 85.4], 97.9 [96.2, 98.9]), lower respiratory tract disease (86.6 [83.1, 89.6], 92.9 [87.6, 96.4]), bronchiolitis (66.9 [58.6, 74.5], 94.3 [80.8, 99.3]), asthma/reactive airway disease (91.0 [85.8, 94.8], 93.0 [90.1, 95.3]), clinical pneumonia (63·9 [50.6, 75·8], 95·0 [92·8, 96·7]), focal (consolidative) pneumonia (54·8 [38·7, 70·2], 86.2 [79.3, 91.5]). Only 59% of chest x-rays with consolidation were correctly identified. Between 6.9 and 14.5% of children were inappropriately prescribed based on their eventual diagnosis. Conclusion: In well-resourced emergency departments, we have identified a previously unrecognized high diagnostic error rate for acute childhood respiratory disorders, particularly in pneumonia and bronchiolitis. These errors lead to the potential of avoidable harm and the administration of inappropriate treatment.

Congo red test for identification of preeclampsia: Results of a prospective diagnostic case-control study in Bangladesh and Mexico

BACKGROUND

Misfolded proteins in the urine of women with preeclampsia bind to Congo Red dye (urine congophilia). We evaluated a beta prototype of a point-of-care test for the identification of urine congophilia in preeclamptic women.

METHODS

Prospective diagnostic case-control study conducted in 409 pregnant women (n = 204 preeclampsia; n = 205 uncomplicated pregnancies) presenting for delivery in two tertiary level hospitals located in Bangladesh and Mexico. The GV-005, a beta prototype of a point-of-care test for detecting congophilia, was performed on fresh and refrigerated urine samples. The primary outcome was the prevalence of urine congophilia in each of the two groups. Secondary outcome was the likelihood of the GV-005 (index test) to confirm and rule-out preeclampsia based on an adjudicated diagnosis (reference standard).

FINDINGS

The GV-005 was positive in 85% of clinical cases (83/98) and negative in 81% of clinical controls (79/98) in the Bangladesh cohort. In the Mexico cohort, the GV-005 test was positive in 48% of clinical cases (51/106) and negative in 77% of clinical controls (82/107). Adjudication confirmed preeclampsia in 92% of Bangladesh clinical cases (90/98) and 61% of Mexico clinical cases (65/106). The odds ratio of a urine congophilia in adjudicated cases versus controls in the Bangladesh cohort was 34^.5 (14^.7 - 81^.1) (p<0^.001) compared to 4^.2 (2^.1 - 8^.4; p<0^.001) in the Mexico cohort.

INTERPRETATION

The GV-005, a beta prototype of a point-of-care test for detection of urine congophilia, is a promising tool for rapid identification of preeclampsia.

FUNDING

Saving Lives at Birth.

Quality evaluation of case series describing four-factor prothrombin complex concentrate in oral factor Xa inhibitor-associated bleeding: a systematic review

INTRODUCTION

As oral factor Xa (oFXa) inhibitor use has increased, so has publication of case series describing related bleeding managed with four-factor prothrombin complex concentrate (4F-PCC).

OBJECTIVE

This review aimed to identify case series describing 4F-PCC management of oFXa inhibitor-related bleeding and appraise their methodological and reporting quality.

DESIGN

We searched Medline and EMBASE (1 January 2011 to 31 May 2020) to identify series of ≥10 patients with oFXa inhibitor-related major bleeding given off-label 4F-PCC. Case series were evaluated using a validated tool adapted for this topic. The tool addressed patient selection, bleed/outcome ascertainment, causal/temporal association and reporting.

RESULTS

We identified 14 case series. None had ≥100 patients (range=13-84), three were prospective, two detailed appropriate inclusion criteria and four noted consecutive inclusion. While 12 series provided clear/appropriate methods for diagnosis of intracranial haemorrhage (ICH); none did so for extracranial bleeds and it was not clear whether bleeding was adjudicated in any. Haemostatic effectiveness, thrombosis and mortality were together evaluated in 12 series, but only seven used validated methods to evaluate/diagnosis haemostasis in ICH, six in gastrointestinal bleeds, five in other bleeds and three in thrombosis. Independent adjudication of haemostasis (n=1) and thrombosis (n=2) was infrequent. Thirty-day follow-up for mortality and thrombosis was noted in five and seven series. Anticoagulation measurement/levels in at least some patients were conveyed in three series. Few series provided data on anticoagulant agent/dose (n=4), time from anticoagulant (n=4), time-to-reversal (n=7), baseline (n=7) or change (n=0) in neurologic function.

CONCLUSIONS

Although many case series describe off-label use of 4F-PCC for oFXa inhibitor-related bleeding, methodological flaws and/or poor reporting necessitates caution in interpretation.

Cost-benefit of outcome adjudication in nine randomised stroke trials

BACKGROUND

Central adjudication of outcomes is common for randomised trials and should control for differential misclassification. However, few studies have estimated the cost of the adjudication process.

METHODS

We estimated the cost of adjudicating the primary outcome in nine randomised stroke trials (25,436 participants). The costs included adjudicators' time, direct payments to adjudicators, and co-ordinating centre costs (e.g. uploading cranial scans and general set-up costs). The number of events corrected after adjudication was our measure of benefit. We calculated cost per corrected event for each trial and in total.

RESULTS

The primary outcome in all nine trials was either stroke or a composite that included stroke. In total, the adjudication process associated with this primary outcome cost in excess of £100,000 for a third of the trials (3/9). Mean cost per event corrected by adjudication was £2295.10 (SD: £1482.42).

CONCLUSIONS

Central adjudication is a time-consuming and potentially costly process. These costs need to be considered when designing a trial and should be evaluated alongside the potential benefits adjudication brings to determine whether they outweigh this expense.

Comparison of Outcome Adjudication by Investigators and by a Central End Point Committee in Heart Failure Trials

Background:

The usefulness of adjudication by central end point committees (CECs) is poorly assessed in heart failure (HF) trials. We aimed to assess its impact on the outcome of the SHIFT trial (Systolic HF Treatment With the If Inhibitor Ivabradine Trial).

Methods:

SHIFT was a randomized placebo-controlled trial investigating the effect of ivabradine in 6505 HF patients with reduced ejection fraction. Prespecified end points, reported by investigators (all cardiologists) using specific case report form pages, included all-cause and specific causes of deaths and hospitalizations. The primary end point was a composite of cardiovascular deaths or hospitalizations for worsening HF. We compared the adjudication of prespecified end points made by investigators and by the CEC.

Results:

Investigators identified 7529 prespecified end points, 6793 of which were confirmed by the CEC: 98.1% of cardiovascular deaths, 88.6% of all hospitalizations, and 84.4% of hospitalizations for worsening HF. These differences had no meaningful impact on the study results; hazard ratio for the primary composite end point: investigators, 0.83 (95% CI, 0.76–0.91) versus CEC, 0.82 (95% CI, 0.75–0.90), with similar results for each component of the primary end point (hazard ratio of 0.92 versus 0.91 for cardiovascular death and 0.78 versus 0.74 for hospitalization for worsening HF).

Conclusions:

Central adjudication by a CEC in the SHIFT study confirmed most of cardiovascular deaths and worsening HF hospitalizations assessed by cardiologists and did not result in a significant change of the final result as compared to investigator judgment. In this context, the benefits of CEC in blinded HF trials should be reconsidered.

Registration:

URL: https://www.clinicaltrials.gov ; Unique identifier: NCT02441218. URL: http://www.isrctn.com/ISRCTN70429960 ; Unique identifier: ISRCTN70429960.

A Large Cross-Sectional Survey Study of the Prevalence of Alopecia Areata in the United States

Purpose

Alopecia areata (AA) is an autoimmune disease characterized by the development of non-scarring alopecia. The prevalence is not well known, and estimates vary considerably with no recent estimates in the United States (US). The objective of this study was to define the current AA point prevalence estimate among the general population in the US overall and by severity.

Patients and Methods

We administered an online, cross-sectional survey to a representative sample of the US population. Participants self-screening as positive for AA using the Alopecia Assessment Tool (ALTO) also completed the Severity of Alopecia Tool (SALT) to measure the severity of disease as a percent of scalp hair loss. Self-reported AA participants were invited to upload photographs for adjudication of AA by 3 clinicians.

Results

The average age of participants was 43 years. Approximately half of the participants (49.2%) were male, and the majority were white (77.1%) and not of Hispanic origin (93.2%). Among the 511 self-reported AA participants, 104 (20.4%) uploaded photographs for clinician evaluation. Clinician-adjudicated point prevalence of AA was 0.21% (95% CI: 0.17%, 0.25%) overall, 0.12% (95% CI: 0.09%, 0.15%) for “mild” disease (≤50% SALT score), and 0.09% (95% CI: 0.06%, 0.11%) for “moderate to severe” disease (>50% SALT score) with 0.04% (95% CI: 0.02%, 0.06%) for the alopecia totalis/alopecia universalis (100% SALT score) “moderate to severe” subgroup. The average SALT score was 44.4% overall, 8.8% for “mild”, and 93.4% for “moderate to severe”.

Conclusion

This study suggests that the current AA prevalence in the US is similar to the upper estimates from the 1970s at approximately 0.21% (700,000 persons) with the current prevalence of “moderate to severe” disease at approximately 0.09% (300,000 persons). Given this prevalence and the substantial impact of AA on quality of life, the burden of AA within the US is considerable.

Results of the 2017 inspection campaign of French phase I/II research sites in Île-de-France following the BIA 10-2474 accident: Medical vs. regulatory relevance

AIMS

Following the serious adverse events that occurred in January 2016 during the BIA 10-2474 First-in-Human study, the French Ministry of Health asked the Regional Health Agencies to inspect operations at all research sites conducting phase I/II clinical trials of experimental drugs. The aim of this study was to assess the medical relevance of the inspections made in Île-de-France (Paris region) in 2017.

METHODS

All 30 sites of Île-de-France region fully authorized to perform phase I/II trials were inspected by a public health physician and a public health pharmacist. Their reported list of observations was submitted to three physicians with longstanding experience of early pharmacology studies performed in academic or private research facilities. These physicians were asked to adjudicate each observation according to their perceived medical importance regarding safety. Adjudications were first performed separately and disagreements were later settled during a final adjudication meeting.

RESULTS

At least one disagreement occurred initially among the 3 adjudicators for 84 of the 120 observations (70%) reported by the inspectors. Following reconciliation, the 3 physicians agreed that 20% of the observations were likely to have potentially serious medical consequences. These observations mainly concerned the management of emergencies and of serious adverse events and the continuity of care.

CONCLUSIONS

Maintenance of on-site inspections periodically carried out by regulatory authorities granting authorisations to perform phase I/II trials are justified. However, the medical relevance of these inspections can be improved with more emphasis on factors affecting the safety of research participants than on administrative or purely regulatory issues.

Outcome Assessment by Central Adjudicators Versus Site Investigators in Stroke Trials: A Systematic Review and Meta-Analysis

Background and Purpose- In randomized stroke trials, central adjudication of a trial's primary outcome is regularly implemented. However, recent evidence questions the importance of central adjudication in randomized trials. The aim of this review was to compare outcomes assessed by central adjudicators with outcomes assessed by site investigators. Methods- We included randomized stroke trials where the primary outcome had undergone an assessment by site investigators and central adjudicators. We searched MEDLINE, EMBASE, CENTRAL (Cochrane Central Register of Controlled Trials), Web of Science, PsycINFO, and Google Scholar for eligible studies. We extracted information about the adjudication process as well as the treatment effect for the primary outcome, assessed both by central adjudicators and by site investigators. We calculated the ratio of these treatment effects so that a ratio of these treatment effects >1 indicated that central adjudication resulted in a more beneficial treatment effect than assessment by the site investigator. A random-effects meta-analysis model was fitted to estimate a pooled effect. Results- Fifteen trials, comprising 69 560 participants, were included. The primary outcomes included were stroke (8/15, 53%), a composite event including stroke (6/15, 40%) and functional outcome after stroke measured on the modified Rankin Scale (1/15, 7%). The majority of site investigators were blind to treatment allocation (9/15, 60%). On average, there was no difference in treatment effect estimates based on data from central adjudicators and site investigators (pooled ratio of these treatment effects=1.02; 95% CI, [0.95-1.09]). Conclusions- We found no evidence that central adjudication of the primary outcome in stroke trials had any impact on trial conclusions. This suggests that potential advantages of central adjudication may not outweigh cost and time disadvantages in stroke studies if the primary purpose of adjudication is to ensure validity of trial findings.

A prospective multicentre study testing the diagnostic accuracy of an automated cough sound centred analytic system for the identification of common respiratory disorders in children

Background

The differential diagnosis of paediatric respiratory conditions is difficult and suboptimal. Existing diagnostic algorithms are associated with significant error rates, resulting in misdiagnoses, inappropriate use of antibiotics and unacceptable morbidity and mortality. Recent advances in acoustic engineering and artificial intelligence have shown promise in the identification of respiratory conditions based on sound analysis, reducing dependence on diagnostic support services and clinical expertise. We present the results of a diagnostic accuracy study for paediatric respiratory disease using an automated cough-sound analyser.

Methods

We recorded cough sounds in typical clinical environments and the first five coughs were used in analyses. Analyses were performed using cough data and up to five-symptom input derived from patient/parent-reported history. Comparison was made between the automated cough analyser diagnoses and consensus clinical diagnoses reached by a panel of paediatricians after review of hospital charts and all available investigations.

Results

A total of 585 subjects aged 29 days to 12 years were included for analysis. The Positive Percent and Negative Percent Agreement values between the automated analyser and the clinical reference were as follows: asthma (97, 91%); pneumonia (87, 85%); lower respiratory tract disease (83, 82%); croup (85, 82%); bronchiolitis (84, 81%). Conclusion: The results indicate that this technology has a role as a high-level diagnostic aid in the assessment of common childhood respiratory disorders.

Trial registration

Australian and New Zealand Clinical Trial Registry (retrospective) - ACTRN12618001521213: 11.09.2018.

Assessment of the Impact of an Endpoint Committee in the Ocular Hypertension Treatment Study

PURPOSE

To assess the impact of a masked Endpoint Committee on estimates of the incidence of primary open-angle glaucoma (POAG) treatment efficacy and statistical power of the Ocular Hypertension Treatment Study-Phase 1, 1994-2002 (OHTS-1).

DESIGN

Retrospective interrater reliability analysis of endpoint attribution by the Endpoint Committee.

METHODS

After study closeout, we recalculated estimates of endpoint incidence, treatment efficacy, and statistical power using all-cause endpoints and POAG endpoints. To avoid bias, only the first endpoint per participant is included in this report.

RESULTS

The Endpoint Committee reviewed 267 first endpoints from 1636 participants. The Endpoint Committee attributed 58% (155 of 267) of the endpoints to POAG. The incidence of all-cause endpoints vs POAG endpoints was 19.5% and 13.2%, respectively, in the observation group and 13.1% and 5.8%, respectively, in the medication group. Treatment effect for all-cause endpoints was a 33% reduction in risk (relative risk = 0.67, 95% confidence interval [CI] of 0.54-0.84) and a 56% reduction in risk for POAG endpoints (relative risk = 0.44, 95% CI of 0.31-0.61). Post hoc statistical power for detecting treatment effect was 0.94 for all-cause endpoints and 0.99 for POAG endpoints.

CONCLUSION

Endpoint Committee adjudication of endpoints improved POAG incidence estimates, increased statistical power, and increased calculated treatment effect by 23%. An Endpoint Committee should be considered in therapeutic trials when common ocular and systemic comorbidities, other than the target condition, could compromise study results.

Congo Red Dot Paper Test for Antenatal Triage and Rapid Identification of Preeclampsia

BACKGROUND

Proteins in the urine of women with preeclampsia (PE) bind Congo Red dye (urine congophilia). We sought to determine the diagnostic performance of a paper-based point-of-care test detecting urine congophilia for rapid triage and diagnosis of PE.

METHODS

Prospective cohort study conducted in 346 consecutive pregnant women evaluated for PE in the Labour and Delivery triage unit at our institution. The Congo Red Dot (CRD) Paper Test (index test) was performed on fresh urine samples. The CRD Paper Test results were compared to an expert adjudicated diagnosis in each case. The accuracy of the CRD Paper Test was also compared to urine and serum analytes (placental growth factor and soluble fms-like tyrosine kinase-1) previously proposed as diagnostic aids for PE.

FINDINGS

During the first triage visit, 32% (112/346) of women received a clinical diagnosis of PE. Yet, 63% (217/346) were admitted for in-patient diagnostic work-up or delivery. The CRD Paper Test was positive in 25% (86/346) of the cases. Adjudication confirmed PE in 28% (96/346) of all cases. The CRD Paper Test outperformed measured serum and urine markers (80·2% sensitivity, 89·2% specificity, 92·1% negative predictive value, 86·7% accuracy). The pre-test, positive and negative post-test probabilities were 27·7%, 74·0%, and 8·0%, respectively. Of women who were discharged undelivered, 38% (133/346) had at least one additional triage visit and the interval between the last negative and first positive CRD Paper Test was 12 (interquartile range, [5-34]) days.

INTERPRETATION

The CRD Paper Test is a simple, non-invasive, "sample-in/answer-out" point-of-care clinical tool for rapid identification of PE.

FUNDING

Saving Lives at Birth Program and NICHD.

#Fake news: a systematic review of mechanical thrombectomy results among neurointerventional stroke surgeons on Twitter

Objective

Twitter is a popular social media platform among physicians. Neurointerventionalists frequently document their lifesaving mechanical thrombectomy cases on Twitter with very favorable results. We fear that there may be some social media publication bias to tweeted mechanical thrombectomy cases with neurointerventionalists being more likely to tweet cases with favorable outcomes. We used these publicly documented cases to analyze post-intervention Twitter-reported outcomes and compared these outcomes with the data provided in the gold standard literature.

Methods

Two reviewers performed a search of Twitter for tweeted cases of acute ischemic strokes treated with mechanical thrombectomy. Data were abstracted from each tweet regarding baseline characteristics and outcomes. Twitter-reported outcomes were compared with the Highly Effective Reperfusion Evaluated in Multiple Endovascular Stroke (HERMES) trial individual patient meta-analysis.

Results

When comparing the tweeted results to HERMES, tweeted cases had a higher post-intervention rate of modified Thrombolysis In Cerebral Infarction (mTICI) scale score of 2c/3 (94% vs 71%, respectively; p<0.0001) and rate of National Institutes of Health Stroke Scale (NIHSS) score ≤2 (81% vs 21%, respectively; p<0.0001). There were no reported complications; thus, tweeted cases also had significantly lower rates of complications, including symptomatic intracerebral hemorrhage (0% vs 4.4%, respectively; p<0.0001), type 2 parenchymal hemorrhage (0% vs 5.1%, respectively; p<0.0001), and mortality (0% vs 15.3%, respectively; p<0.0001).

Conclusions

There is a significant difference between social media and reality even within the ‘MedTwitter’ sphere, which is likely due to a strong publication bias in Twitter-reported cases. Content on ‘MedTwitter’, as with most social media, should be accepted cautiously.