Heavy tailed distributions of effect sizes in systematic reviews of complex interventions

Delivery of Lung-protective Ventilation for Acute Respiratory Distress Syndrome: A Hybrid Implementation-Effectiveness Trial

Rationale: Lung-protective ventilation (LPV) improves outcomes for patients with acute respiratory distress syndrome (ARDS), but adherence remains inadequate. Objectives: To measure the process and clinical impacts of implementation of a science-based intervention to improve LPV adherence for patients with ARDS, in part by increased use of clinical decision support (CDS). Methods: We conducted a type III hybrid implementation/effectiveness pilot trial enrolling adult patients with ARDS admitted to three hospitals before and after the launch of a multimodal implementation intervention to increase the use of mechanical ventilation CDS and improve LPV adherence. The primary outcome was patients' percentage of time adherent to low tidal volume (⩽6.5 ml/kg predicted body weight) ventilation (LTVV). Secondary outcomes included adherence to prescribed oxygenation settings, the use of the CDS tool's independent oxygenation and ventilation components, ventilator-free days, and mortality. Analyses employed multivariable regression to compare adjusted pre- versus postintervention outcomes after the exclusion of a postintervention wash-in period. A sensitivity analysis measured process outcomes' level and trend change postintervention using segmented regression. Results: The 446 included patients had a mean age of 60 years, and 43% were female. Demographic and clinical characteristics were similar pre- versus postintervention. The adjusted proportion of adherent time increased postintervention for LTVV (9.2%; 95% confidence interval [CI], 3.8-14.5%) and prescribed oxygenation settings (11.9%; 95% CI, 7.2-16.5%), as did the probability patients spent ⩾90% of ventilated time on LTVV (adjusted odds ratio [aOR] 2.58; 95% CI, 1.64-4.10) and use of ventilation CDS (aOR, 41.3%; 95% CI, 35.9-46.7%) and oxygenation CDS (aOR, 54.3%; 95% CI, 50.9-57.7%). Ventilator-free days (aOR, 1.15; 95% CI, 0.81-1.62) and 28-day mortality (aOR, 0.78; 95% CI, 0.50-1.20) did not change significantly after intervention. Segmented regression analysis supported a causal relationship between the intervention and improved CDS usage but suggested trends before intervention rather than the studied intervention could explain increased LPV adherence after the intervention. Conclusions: In this pilot trial, a multimodal implementation intervention was associated with increased use of ventilator management CDS for patients with ARDS but was not associated with differences in clinical outcomes and may not have independently caused the observed postintervention improvements in LPV adherence. Clinical trial registered with www.clinicaltrials.gov (NCT03984175).

Frequent attendance at the emergency department shows typical features of complex systems: analysis of multicentre linked data

OBJECTIVE

Frequent attendance at the ED is a worldwide problem. We hypothesised that frequent attendance could be understood as a feature of a complex system comprising patients, healthcare and society. Complex systems have characteristic statistical properties, with stable patterns at the level of the system emerging from unstable patterns at the level of individuals who make up the system.

METHODS

Analysis of a linked dataset of routinely collected health records from all 13 hospital trusts providing ED care in the Yorkshire and Humber region of the UK (population 5.5 million). We analysed the distribution of attendances per person in each of 3 years and measured the transition of individual patients between frequent, infrequent and non-attendance. We fitted data to power law distributions typically seen in complex systems using maximum likelihood estimation.

RESULTS

The data included 3.6 million attendances at EDs in 13 hospital trusts. 29/39 (74.3%) analyses showed a statistical fit to a power law; 2 (5.1%) fitted an alternative distribution. All trusts' data fitted a power law in at least 1 year. Differences over time and between hospital trusts were small and partly explained by demographics. In contrast, individual patients' frequent attendance was unstable between years.

CONCLUSIONS

ED attendance patterns are stable at the level of the system, but unstable at the level of individual frequent attenders. Attendances follow a power law distribution typical of complex systems. Interventions to address ED frequent attendance need to consider the whole system and not just the individual frequent attenders.

Parametric evaluation of impedance curve in radiofrequency ablation: A quantitative description of the asymmetry and dynamic variation of impedance in bovine ex vivo model

Radiofrequency ablation (RFA) is a treatment for liver tumors with advantages over the traditional treatment of surgical resection. This procedure has the shortest recovery time in early stage tumors. The objective of this study is to parameterize the impedance curve of the RFA procedure in an ex vivo model by defining seven parameters (t_1/2, t_minimum, t_end, Z_initial, Z_1/2, Z_minimum and Z_end). Based on these parameters, three performance indices are defined: one to identify the magnitude of impedance curve asymmetry (δ), one Drop ratio (DR) describing the percentage of impedance decrease until the minimum impedance point is reached, and Ascent Ratio (AR) describing the magnitude of increase in impedance from the minimum impedance point to its maximum point. Fifty ablations were performed in a bovine ex vivo model to measure and evaluate the proposed parameters and performance index. The results show that the groups had an average δ of 29.02%, DR of 22.41%, and AR of 545.33% for RFA without the use of saline or deionized solutions. The saline solution and deionized water-cooled groups indicated the correlation of performance indices δ, DR, and AR with the obtained final ablation volume. Therefore, by controlling these parameters and indices, lower recurrence is achieved.

Rapid Implementation of a Complex, Multimodal Technology Response to COVID-19 at an Integrated Community-Based Health Care System

BACKGROUND

The rapid spread of severe acute respiratory syndrome coronavirus-2 or SARS-CoV-2 necessitated a scaled treatment response to the novel coronavirus disease 2019 (COVID-19).

OBJECTIVE

This study aimed to characterize the design and rapid implementation of a complex, multimodal, technology response to COVID-19 led by the Intermountain Healthcare's (Intermountain's) Care Transformation Information Systems (CTIS) organization to build pandemic surge capacity.

METHODS

Intermountain has active community-spread cases of COVID-19 that are increasing. We used the Centers for Disease Control and Prevention Pandemic Intervals Framework (the Framework) to characterize CTIS leadership's multimodal technology response to COVID-19 at Intermountain. We provide results on implementation feasibility and sustainability of health information technology (HIT) interventions as of June 30, 2020, characterize lessons learned and identify persistent barriers to sustained deployment.

RESULTS

We characterize the CTIS organization's multimodal technology response to COVID-19 in five relevant areas of the Framework enabling (1) incident management, (2) surveillance, (3) laboratory testing, (4) community mitigation, and (5) medical care and countermeasures. We are seeing increased use of traditionally slow-to-adopt technologies that create additional surge capacity while sustaining patient safety and care quality. CTIS leadership recognized early that a multimodal technology intervention could enable additional surge capacity for health care delivery systems with a broad geographic and service scope. A statewide central tracking system to coordinate capacity planning and management response is needed. Order interoperability between health care systems remains a barrier to an integrated response.

CONCLUSION

The rate of future pandemics is estimated to increase. The pandemic response of health care systems, like Intermountain, offers a blueprint for the leadership role that HIT organizations can play in mainstream care delivery, enabling a nimbler, virtual health care delivery system that is more responsive to current and future needs.

The Moderating Role of the School Context on the Effects of the Healthy Primary School of the Future

Background: The current study investigated the moderating role of the school context on the effects of a Dutch health promoting school initiative on children's health and health behaviors. Methods: The study used a mixed-methods design. The school context (n = 4) was assessed by the characteristics of the school population, teacher's health-promoting (HP) practices, implementers' perceived barriers, school's HP elements, and dominating organizational issues. Outcomes included objectively assessed BMI z-scores and physical activity (PA), and parent and child-reported dietary intake. Analyses included linear mixed models (four intervention schools versus four control schools), and qualitative comparisons between intervention schools with similar HP changes. Results: Effects on outcomes varied considerably across schools (e.g., range in effect size on light PA of 0.01-0.26). Potentially moderating contextual aspects were the child's socioeconomic background and baseline health behaviors; practices and perceived barriers of employees; and organizational issues at a school level. Conclusions: Similar HP changes lead to different outcomes across schools due to differences in the school context. The adoption of a complex adaptive systems perspective contributes to a better understanding of the variation in effects and it can provide insight on which contextual aspects to focus on or intervene in to optimize the effects of HP initiatives.

Do healthcare services behave as complex systems? Analysis of patterns of attendance and implications for service delivery

BACKGROUND

The science of complex systems has been proposed as a way of understanding health services and the demand for them, but there is little quantitative evidence to support this. We analysed patterns of healthcare use in different urgent care settings to see if they showed two characteristic statistical features of complex systems: heavy-tailed distributions (including the inverse power law) and generative burst patterns.

METHODS

We conducted three linked studies. In study 1 we analysed the distribution of number of contacts per patient with an urgent care service in two settings: emergency department (ED) and primary care out-of-hours (PCOOH) services. We hypothesised that these distributions should be heavy-tailed (inverse power law or log-normal) in keeping with typical complex systems. In study 2 we analysed the distribution of bursts of contact with urgent care services by individuals: correlated bursts of activity occur in complex systems and represent a mechanism by which overall heavy-tailed distributions arise. In study 3 we replicated the approach of study 1 using data systematically identified from published sources.

RESULTS

Study 1 involved data from a PCOOH service in Scotland (725,000) adults, 1.1 million contacts) and an ED in New Zealand (60,000 adults, 98,000 contacts). The total number of contacts per individual in each dataset was statistically indistinguishable from an inverse power law (p > 0.05) above 4 contacts for the PCOOH data and 3 contacts for the ED data. Study 2 found the distribution of contact bursts closely followed a heavy-tailed distribution (p < 0.008), indicating the presence of correlated bursts. Study 3 identified data from 17 studies across 8 countries and found distributions similar to study 1 in all of them.

CONCLUSIONS

Urgent healthcare use displays characteristic statistical features of large complex systems. These studies provide strong quantitative evidence that healthcare services behave as complex systems and have important implications for urgent care. Interventions to manage demand must address drivers for consultation across the whole system: focusing on only the highest users (in the tail of the distribution) will have limited impact on efficiency. Bursts of attendance - and ways to shorten them - represent promising targets for managing demand.

Not Normal: the uncertainties of scientific measurements

Judging the significance and reproducibility of quantitative research requires a good understanding of relevant uncertainties, but it is often unclear how well these have been evaluated and what they imply. Reported scientific uncertainties were studied by analysing 41 000 measurements of 3200 quantities from medicine, nuclear and particle physics, and interlaboratory comparisons ranging from chemistry to toxicology. Outliers are common, with 5σ disagreements up to five orders of magnitude more frequent than naively expected. Uncertainty-normalized differences between multiple measurements of the same quantity are consistent with heavy-tailed Student's t-distributions that are often almost Cauchy, far from a Gaussian Normal bell curve. Medical research uncertainties are generally as well evaluated as those in physics, but physics uncertainty improves more rapidly, making feasible simple significance criteria such as the 5σ discovery convention in particle physics. Contributions to measurement uncertainty from mistakes and unknown problems are not completely unpredictable. Such errors appear to have power-law distributions consistent with how designed complex systems fail, and how unknown systematic errors are constrained by researchers. This better understanding may help improve analysis and meta-analysis of data, and help scientists and the public have more realistic expectations of what scientific results imply.

Using qualitative comparative analysis in a systematic review of a complex intervention

BACKGROUND

Systematic reviews evaluating complex interventions often encounter substantial clinical heterogeneity in intervention components and implementation features making synthesis challenging. Qualitative comparative analysis (QCA) is a non-probabilistic method that uses mathematical set theory to study complex phenomena; it has been proposed as a potential method to complement traditional evidence synthesis in reviews of complex interventions to identify key intervention components or implementation features that might explain effectiveness or ineffectiveness. The objective of this study was to describe our approach in detail and examine the suitability of using QCA within the context of a systematic review.

METHODS

We used data from a completed systematic review of behavioral interventions to improve medication adherence to conduct two substantive analyses using QCA. The first analysis sought to identify combinations of nine behavior change techniques/components (BCTs) found among effective interventions, and the second analysis sought to identify combinations of five implementation features (e.g., agent, target, mode, time span, exposure) found among effective interventions. For each substantive analysis, we reframed the review's research questions to be designed for use with QCA, calibrated sets (i.e., transformed raw data into data used in analysis), and identified the necessary and/or sufficient combinations of BCTs and implementation features found in effective interventions.

RESULTS

Our application of QCA for each substantive analysis is described in detail. We extended the original review findings by identifying seven combinations of BCTs and four combinations of implementation features that were sufficient for improving adherence. We found reasonable alignment between several systematic review steps and processes used in QCA except that typical approaches to study abstraction for some intervention components and features did not support a robust calibration for QCA.

CONCLUSIONS

QCA was suitable for use within a systematic review of medication adherence interventions and offered insights beyond the single dimension stratifications used in the original completed review. Future prospective use of QCA during a review is needed to determine the optimal way to efficiently integrate QCA into existing approaches to evidence synthesis of complex interventions.

Observational Evidence of For-Profit Delivery and Inferior Nursing Home Care: When Is There Enough Evidence for Policy Change?

Margaret McGregor and colleagues consider Bradford Hill's framework for examining causation in observational research for the association between nursing home care quality and for-profit ownership.

Lessons from complex interventions to improve health

Complexity-resulting from interactions among many component parts-is a property of both the intervention and the context (or system) into which it is placed. Complexity increases the unpredictability of effects. Complexity invites new approaches to logic modeling, definitions of integrity and means of standardization, and evaluation. New metaphors and terminology are needed to capture the recognition that knowledge generation comes from the hands of practitioners/implementers as much as it comes from those usually playing the role of intervention researcher. Failure to acknowledge this may blind us to the very mechanisms we seek to understand. Researchers in clinical settings are documenting health improvement gains made as a consequence of complex systems thinking. Improvement science in clinical settings has much to offer researchers in population health.