Rapid systematic review on risks and outcomes of sepsis: the influence of risk factors associated with health inequalities

BACKGROUND AND AIMS

Sepsis is a serious and life-threatening condition caused by a dysregulated immune response to an infection. Recent guidance issued in the UK gave recommendations around recognition and antibiotic treatment of sepsis, but did not consider factors relating to health inequalities. The aim of this study was to summarise the literature investigating associations between health inequalities and sepsis.

METHODS

Searches were conducted in Embase for peer-reviewed articles published since 2010 that included sepsis in combination with one of the following five areas: socioeconomic status, race/ethnicity, community factors, medical needs and pregnancy/maternity.

RESULTS

Five searches identified 1,402 studies, with 50 unique studies included in the review after screening (13 sociodemographic, 14 race/ethnicity, 3 community, 3 care/medical needs and 20 pregnancy/maternity; 3 papers examined multiple health inequalities). Most of the studies were conducted in the USA (31/50), with only four studies using UK data (all pregnancy related). Socioeconomic factors associated with increased sepsis incidence included lower socioeconomic status, unemployment and lower education level, although findings were not consistent across studies. For ethnicity, mixed results were reported. Living in a medically underserved area or being resident in a nursing home increased risk of sepsis. Mortality rates after sepsis were found to be higher in people living in rural areas or in those discharged to skilled nursing facilities while associations with ethnicity were mixed. Complications during delivery, caesarean-section delivery, increased deprivation and black and other ethnic minority race were associated with post-partum sepsis.

CONCLUSION

There are clear correlations between sepsis morbidity and mortality and the presence of factors associated with health inequalities. To inform local guidance and drive public health measures, there is a need for studies conducted across more diverse setting and countries.

A scoping review finds a growing trend in studies validating multimorbidity patterns and identifies five broad types of validation methods

OBJECTIVES

Multimorbidity, the presence of two or more long-term conditions, is a growing public health concern. Many studies use analytical methods to discover multimorbidity patterns from data. We aimed to review approaches used in published literature to validate these patterns.

STUDY DESIGN AND SETTING

We systematically searched PubMed and Web of Science for studies published between July 2017 and July 2023 that used analytical methods to discover multimorbidity patterns.

RESULTS

Out of 31,617 studies returned by the searches, 172 were included. Of these, 111 studies (64%) conducted validation, the number of studies with validation increased from 53.13% (17 out of 32 studies) to 71.25% (57 out of 80 studies) in 2017-2019 to 2022-2023, respectively. Five types of validation were identified: assessing the association of multimorbidity patterns with clinical outcomes (n = 79), stability across subsamples (n = 26), clinical plausibility (n = 22), stability across methods (n = 7) and exploring common determinants (n = 2). Some studies used multiple types of validation.

CONCLUSION

The number of studies conducting a validation of multimorbidity patterns is clearly increasing. The most popular validation approach is assessing the association of multimorbidity patterns with clinical outcomes. Methodological guidance on the validation of multimorbidity patterns is needed.

Clinical prediction models and the multiverse of madness

BACKGROUND

Each year, thousands of clinical prediction models are developed to make predictions (e.g. estimated risk) to inform individual diagnosis and prognosis in healthcare. However, most are not reliable for use in clinical practice.

MAIN BODY

We discuss how the creation of a prediction model (e.g. using regression or machine learning methods) is dependent on the sample and size of data used to develop it-were a different sample of the same size used from the same overarching population, the developed model could be very different even when the same model development methods are used. In other words, for each model created, there exists a multiverse of other potential models for that sample size and, crucially, an individual's predicted value (e.g. estimated risk) may vary greatly across this multiverse. The more an individual's prediction varies across the multiverse, the greater the instability. We show how small development datasets lead to more different models in the multiverse, often with vastly unstable individual predictions, and explain how this can be exposed by using bootstrapping and presenting instability plots. We recommend healthcare researchers seek to use large model development datasets to reduce instability concerns. This is especially important to ensure reliability across subgroups and improve model fairness in practice.

CONCLUSIONS

Instability is concerning as an individual's predicted value is used to guide their counselling, resource prioritisation, and clinical decision making. If different samples lead to different models with very different predictions for the same individual, then this should cast doubt into using a particular model for that individual. Therefore, visualising, quantifying and reporting the instability in individual-level predictions is essential when proposing a new model.

Clinical and health inequality risk factors for non-COVID-related sepsis during the global COVID-19 pandemic: a national case-control and cohort study

Background

Sepsis, characterised by significant morbidity and mortality, is intricately linked to socioeconomic disparities and pre-admission clinical histories. This study aspires to elucidate the association between non-COVID-19 related sepsis and health inequality risk factors amidst the pandemic in England, with a secondary focus on their association with 30-day sepsis mortality.

Methods

With the approval of NHS England, we harnessed the OpenSAFELY platform to execute a cohort study and a 1:6 matched case-control study. A sepsis diagnosis was identified from the incident hospital admissions record using ICD-10 codes. This encompassed 248,767 cases with non-COVID-19 sepsis from a cohort of 22.0 million individuals spanning January 1, 2019, to June 31, 2022. Socioeconomic deprivation was gauged using the Index of Multiple Deprivation score, reflecting indicators like income, employment, and education. Hospitalisation-related sepsis diagnoses were categorised as community-acquired or hospital-acquired. Cases were matched to controls who had no recorded diagnosis of sepsis, based on age (stepwise), sex, and calendar month. The eligibility criteria for controls were established primarily on the absence of a recorded sepsis diagnosis. Associations between potential predictors and odds of developing non-COVID-19 sepsis underwent assessment through conditional logistic regression models, with multivariable regression determining odds ratios (ORs) for 30-day mortality.

Findings

The study included 224,361 (10.2%) cases with non-COVID-19 sepsis and 1,346,166 matched controls. The most socioeconomic deprived quintile was associated with higher odds of developing non-COVID-19 sepsis than the least deprived quintile (crude OR 1.80 [95% CI 1.77-1.83]). Other risk factors (after adjusting comorbidities) such as learning disability (adjusted OR 3.53 [3.35-3.73]), chronic liver disease (adjusted OR 3.08 [2.97-3.19]), chronic kidney disease (stage 4: adjusted OR 2.62 [2.55-2.70], stage 5: adjusted OR 6.23 [5.81-6.69]), cancer, neurological disease, immunosuppressive conditions were also associated with developing non-COVID-19 sepsis. The incidence rate of non-COVID-19 sepsis decreased during the COVID-19 pandemic and rebounded to pre-pandemic levels (April 2021) after national lockdowns had been lifted. The 30-day mortality risk in cases with non-COVID-19 sepsis was higher for the most deprived quintile across all periods.

Interpretation

Socioeconomic deprivation, comorbidity and learning disabilities were associated with an increased odds of developing non-COVID-19 related sepsis and 30-day mortality in England. This study highlights the need to improve the prevention of sepsis, including more precise targeting of antimicrobials to higher-risk patients.

Funding

The UK Health Security Agency, Health Data Research UK, and National Institute for Health Research.

The impact of COVID-19 on antibiotic prescribing in primary care in England: Evaluation and risk prediction of appropriateness of type and repeat prescribing

BACKGROUND

This study aimed to predict risks of potentially inappropriate antibiotic type and repeat prescribing and assess changes during COVID-19.

METHODS

With the approval of NHS England, we used OpenSAFELY platform to access the TPP SystmOne electronic health record (EHR) system and selected patients prescribed antibiotics from 2019 to 2021. Multinomial logistic regression models predicted patient's probability of receiving inappropriate antibiotic type or repeat antibiotic course for each common infection.

RESULTS

The population included 9.1 million patients with 29.2 million antibiotic prescriptions. 29.1% of prescriptions were identified as repeat prescribing. Those with same day incident infection coded in the EHR had considerably lower rates of repeat prescribing (18.0%) and 8.6% had potentially inappropriate type. No major changes in the rates of repeat antibiotic prescribing during COVID-19 were found. In the 10 risk prediction models, good levels of calibration and moderate levels of discrimination were found.

CONCLUSIONS

Our study found no evidence of changes in level of inappropriate or repeat antibiotic prescribing after the start of COVID-19. Repeat antibiotic prescribing was frequent and varied according to regional and patient characteristics. There is a need for treatment guidelines to be developed around antibiotic failure and clinicians provided with individualised patient information.

Developing prediction models to estimate the risk of two survival outcomes both occurring: A comparison of techniques

INTRODUCTION

This study considers the prediction of the time until two survival outcomes have both occurred. We compared a variety of analytical methods motivated by a typical clinical problem of multimorbidity prognosis.

METHODS

We considered five methods: product (multiply marginal risks), dual-outcome (directly model the time until both events occur), multistate models (msm), and a range of copula and frailty models. We assessed calibration and discrimination under a variety of simulated data scenarios, varying outcome prevalence, and the amount of residual correlation. The simulation focused on model misspecification and statistical power. Using data from the Clinical Practice Research Datalink, we compared model performance when predicting the risk of cardiovascular disease and type 2 diabetes both occurring.

RESULTS

Discrimination was similar for all methods. The product method was poorly calibrated in the presence of residual correlation. The msm and dual-outcome models were the most robust to model misspecification but suffered a drop in performance at small sample sizes due to overfitting, which the copula and frailty model were less susceptible to. The copula and frailty model's performance were highly dependent on the underlying data structure. In the clinical example, the product method was poorly calibrated when adjusting for 8 major cardiovascular risk factors.

DISCUSSION

We recommend the dual-outcome method for predicting the risk of two survival outcomes both occurring. It was the most robust to model misspecification, although was also the most prone to overfitting. The clinical example motivates the use of the methods considered in this study.

Impact of COVID-19 on broad-spectrum antibiotic prescribing for common infections in primary care in England: a time-series analyses using OpenSAFELY and effects of predictors including deprivation

Background

The COVID-19 pandemic impacted the healthcare systems, adding extra pressure to reduce antimicrobial resistance. Therefore, we aimed to evaluate changes in antibiotic prescription patterns after COVID-19 started.

Methods

With the approval of NHS England, we used the OpenSAFELY platform to access the TPP SystmOne electronic health record (EHR) system in primary care and selected patients prescribed antibiotics from 2019 to 2021. To evaluate the impact of COVID-19 on broad-spectrum antibiotic prescribing, we evaluated prescribing rates and its predictors and used interrupted time series analysis by fitting binomial logistic regression models.

Findings

Over 32 million antibiotic prescriptions were extracted over the study period; 8.7% were broad-spectrum. The study showed increases in broad-spectrum antibiotic prescribing (odds ratio [OR] 1.37; 95% confidence interval [CI] 1.36-1.38) as an immediate impact of the pandemic, followed by a gradual recovery with a 1.1-1.2% decrease in odds of broad-spectrum prescription per month. The same pattern was found within subgroups defined by age, sex, region, ethnicity, and socioeconomic deprivation quintiles. More deprived patients were more likely to receive broad-spectrum antibiotics, which differences remained stable over time. The most significant increase in broad-spectrum prescribing was observed for lower respiratory tract infection (OR 2.33; 95% CI 2.1-2.50) and otitis media (OR 1.96; 95% CI 1.80-2.13).

Interpretation

An immediate reduction in antibiotic prescribing and an increase in the proportion of broad-spectrum antibiotic prescribing in primary care was observed. The trends recovered to pre-pandemic levels, but the consequence of the COVID-19 pandemic on AMR needs further investigation.

Funding

This work was supported by Health Data Research UK and by National Institute for Health Research.

Minimum sample size for developing a multivariable prediction model using multinomial logistic regression

AIMS

Multinomial logistic regression models allow one to predict the risk of a categorical outcome with > 2 categories. When developing such a model, researchers should ensure the number of participants (n ) is appropriate relative to the number of events (E k ) and the number of predictor parameters (p k ) for each category k. We propose three criteria to determine the minimum n required in light of existing criteria developed for binary outcomes.

PROPOSED CRITERIA

The first criterion aims to minimise the model overfitting. The second aims to minimise the difference between the observed and adjusted R 2 Nagelkerke. The third criterion aims to ensure the overall risk is estimated precisely. For criterion (i), we show the sample size must be based on the anticipated Cox-snell R 2 of distinct 'one-to-one' logistic regression models corresponding to the sub-models of the multinomial logistic regression, rather than on the overall Cox-snell R 2 of the multinomial logistic regression.

EVALUATION OF CRITERIA

We tested the performance of the proposed criteria (i) through a simulation study and found that it resulted in the desired level of overfitting. Criterion (ii) and (iii) were natural extensions from previously proposed criteria for binary outcomes and did not require evaluation through simulation.

SUMMARY

We illustrated how to implement the sample size criteria through a worked example considering the development of a multinomial risk prediction model for tumour type when presented with an ovarian mass. Code is provided for the simulation and worked example. We will embed our proposed criteria within the pmsampsize R library and Stata modules.

Analysis of disease clusters and patient outcomes in people with multiple long term conditions using hypergraphs

ObjectivesHaving multiple long term health conditions (MLTCs), also known as multimorbidity, is becoming increasingly common as populations age. Understanding how clusters of diseases are likely to lead to other diseases and the effect of multimorbidity on healthcare resource use (HRU) will be of great importance as this trend continues. ApproachGraph-based approaches, also called network analysis in the literature, have been used previously to study multimorbidity. The use of hypergraphs, which are generalisations of graphs where edges can connect to any number of nodes, and their application to the problem of understanding multimorbidity will be discussed. Analysis using hypergraphs was carried out using a population-scale cohort of people in the Secure Anonymised Information Linkage (SAIL) Databank to find the diseases and disease sets which are most important based on a measure of prevalence and measures of healthcare resource utilisation in secondary care. ResultsThe most important sets of diseases based on the centrality of a hypergraph weighted by a measure of prevalence featured hypertension, and the most important was hypertension and diabetes. The most important sets of diseases based on the centrality of a hypergraph weighted by a measure of unplanned inpatient HRU were arrhythmia, heart failure and hypertension while for a measure of outpatient HRU the most important set of diseases was diabetes and hypertension. ConclusionHypergraphs are very flexible and general mathematical objects and there is still a great deal of development that can be done to make them more useful in epidemiological settings and beyond.

An assessment of the potential miscalibration of cardiovascular disease risk predictions caused by a secular trend in cardiovascular disease in England

Background

A downwards secular trend in the incidence of cardiovascular disease (CVD) in England was identified through previous work and the literature. Risk prediction models for primary prevention of CVD do not model this secular trend, this could result in over prediction of risk for individuals in the present day. We evaluate the effects of modelling this secular trend, and also assess whether it is driven by an increase in statin use during follow up.

Methods

We derived a cohort of patients (1998–2015) eligible for cardiovascular risk prediction from the Clinical Practice Research Datalink with linked hospitalisation and mortality records (N = 3,855,660). Patients were split into development and validation cohort based on their cohort entry date (before/after 2010). The calibration of a CVD risk prediction model developed in the development cohort was tested in the validation cohort. The calibration was also assessed after modelling the secular trend. Finally, the presence of the secular trend was evaluated under a marginal structural model framework, where the effect of statin treatment during follow up is adjusted for.

Results

Substantial over prediction of risks in the validation cohort was found when not modelling the secular trend. This miscalibration could be minimised if one was to explicitly model the secular trend. The reduction in risk in the validation cohort when introducing the secular trend was 35.68 and 33.24% in the female and male cohorts respectively. Under the marginal structural model framework, the reductions were 33.31 and 32.67% respectively, indicating increasing statin use during follow up is not the only the cause of the secular trend.

Conclusions

Inclusion of the secular trend into the model substantially changed the CVD risk predictions. Models that are being used in clinical practice in the UK do not model secular trend and may thus overestimate the risks, possibly leading to patients being treated unnecessarily. Wider discussion around the modelling of secular trends in a risk prediction framework is needed.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12874-020-01173-x.

Toward a framework for the design, implementation, and reporting of methodology scoping reviews

BACKGROUND AND OBJECTIVE

In view of the growth of published articles, there is an increasing need for studies that summarize scientific research. An increasingly common review is a "methodology scoping review," which provides a summary of existing analytical methods, techniques and software that have been proposed or applied in research articles to address an analytical problem or further an analytical approach. However, guidelines for their design, implementation, and reporting are limited.

METHODS

Drawing on the experiences of the authors, which were consolidated through a series of face-to-face workshops, we summarize the challenges inherent in conducting a methodology scoping review and offer suggestions of best practice to promote future guideline development.

RESULTS

We identified three challenges of conducting a methodology scoping review. First, identification of search terms; one cannot usually define the search terms a priori, and the language used for a particular method can vary across the literature. Second, the scope of the review requires careful consideration because new methodology is often not described (in full) within abstracts. Third, many new methods are motivated by a specific clinical question, where the methodology may only be documented in supplementary materials. We formulated several recommendations that build upon existing review guidelines. These recommendations ranged from an iterative approach to defining search terms through to screening and data extraction processes.

CONCLUSION

Although methodology scoping reviews are an important aspect of research, there is currently a lack of guidelines to standardize their design, implementation, and reporting. We recommend a wider discussion on this topic.

The impact of statin discontinuation and restarting rates on the optimal time to initiate statins and on the number of cardiovascular events prevented

INTRODUCTION

A patient is eligible for statins in England if they have a 10-year risk of cardiovascular disease >10%. We hypothesize that if statin discontinuation rates are high it may be better to delay statin initiation until patients are at a higher risk, to maximize the benefit of the drug.

METHODS

A four-state health state transition model was used to assess the optimal time to initiate statins after a risk assessment, in order to prevent the highest number of cardiovascular events, for a given risk profile (age, gender, risk) and adherence rate. A Clinical Practice Research Datalink dataset linked to Hospital Episodes Statistics and Office for National Statistics was used to inform the transition probabilities in this model, taking into account observed statin discontinuation and re-continuation patterns.

RESULTS

Our results suggest, if statins are initiated in a cohort of 50-year old men with a 10% 10-year risk, we prevent 4.78 events per 100 individuals. If we wait 10 years to prescribe, at which point 10-year risk scores are at 20%, we prevent 5.45 events per 100 individuals. If the observed discontinuation rate was reduced by a sixth, third or half in the same cohort, we would prevent 7.29, 9.01 or 10.22 events per 100 individuals.

CONCLUSIONS

In certain scenarios, extra cardiovascular disease events could be prevented by delaying statin initiation beyond a risk of 10% until reaching a age (59 for men, 63 for women), based on statin discontinuation rates in England. The optimal time to initiate statins was driven by age, not by cardiovascular risk.

Impact of sample size on the stability of risk scores from clinical prediction models: a case study in cardiovascular disease

BACKGROUND

Stability of risk estimates from prediction models may be highly dependent on the sample size of the dataset available for model derivation. In this paper, we evaluate the stability of cardiovascular disease risk scores for individual patients when using different sample sizes for model derivation; such sample sizes include those similar to models recommended in the national guidelines, and those based on recently published sample size formula for prediction models.

METHODS

We mimicked the process of sampling N patients from a population to develop a risk prediction model by sampling patients from the Clinical Practice Research Datalink. A cardiovascular disease risk prediction model was developed on this sample and used to generate risk scores for an independent cohort of patients. This process was repeated 1000 times, giving a distribution of risks for each patient. N = 100,000, 50,000, 10,000, N _min (derived from sample size formula) and N _epv10 (meets 10 events per predictor rule) were considered. The 5-95th percentile range of risks across these models was used to evaluate instability. Patients were grouped by a risk derived from a model developed on the entire population (population-derived risk) to summarise results.

RESULTS

For a sample size of 100,000, the median 5-95th percentile range of risks for patients across the 1000 models was 0.77%, 1.60%, 2.42% and 3.22% for patients with population-derived risks of 4-5%, 9-10%, 14-15% and 19-20% respectively; for N = 10,000, it was 2.49%, 5.23%, 7.92% and 10.59%, and for N using the formula-derived sample size, it was 6.79%, 14.41%, 21.89% and 29.21%. Restricting this analysis to models with high discrimination, good calibration or small mean absolute prediction error reduced the percentile range, but high levels of instability remained.

CONCLUSIONS

Widely used cardiovascular disease risk prediction models suffer from high levels of instability induced by sampling variation. Many models will also suffer from overfitting (a closely linked concept), but at acceptable levels of overfitting, there may still be high levels of instability in individual risk. Stability of risk estimates should be a criterion when determining the minimum sample size to develop models.

Correction to: The uncertainty with using risk prediction models for individual decision making: an exemplar cohort study examining the prediction of cardiovascular disease in English primary care

The original article [1] contained an error in the abstract. The mentioned cohort size now correctly states 'N = 3,855,660'.

Chronic obstructive pulmonary disease exacerbation episodes derived from electronic health record data validated using clinical trial data

Purpose

To validate an algorithm for acute exacerbations of chronic obstructive pulmonary disease (AECOPD) episodes derived in an electronic health record (EHR) database, against AECOPD episodes collected in a randomized clinical trial using an electronic case report form (eCRF).

Methods

We analyzed two data sources from the Salford Lung Study in COPD: trial eCRF and the Salford Integrated Record, a linked primary‐secondary routine care EHR database of all patients in Salford. For trial participants, AECOPD episodes reported in eCRF were compared with algorithmically derived moderate/severe AECOPD episodes identified in EHR. Episode characteristics (frequency, duration), sensitivity, and positive predictive value (PPV) were calculated. A match between eCRF and EHR episodes was defined as at least 1‐day overlap.

Results

In the primary effectiveness analysis population (n = 2269), 3791 EHR episodes (mean [SD] length: 15.1 [3.59] days; range: 14‐54) and 4403 moderate/severe AECOPD eCRF episodes (mean length: 13.8 [16.20] days; range: 1‐372) were identified. eCRF episodes exceeding 28 days were usually broken up into shorter episodes in the EHR. Sensitivity was 63.6% and PPV 71.1%, where concordance was defined as at least 1‐day overlap.

Conclusions

The EHR algorithm performance was acceptable, indicating that EHR‐derived AECOPD episodes may provide an efficient, valid method of data collection. Comparing EHR‐derived AECOPD episodes with those collected by eCRF resulted in slightly fewer episodes, and eCRF episodes of extreme lengths were poorly captured in EHR. Analysis of routinely collected EHR data may be reasonable when relative, rather than absolute, rates of AECOPD are relevant for stakeholders' decision making.

The uncertainty with using risk prediction models for individual decision making: an exemplar cohort study examining the prediction of cardiovascular disease in English primary care

BACKGROUND

Risk prediction models are commonly used in practice to inform decisions on patients' treatment. Uncertainty around risk scores beyond the confidence interval is rarely explored. We conducted an uncertainty analysis of the QRISK prediction tool to evaluate the robustness of individual risk predictions with varying modelling decisions.

METHODS

We derived a cohort of patients eligible for cardiovascular risk prediction from the Clinical Practice Research Datalink (CPRD) with linked hospitalisation and mortality records (N = 3,792,474). Risk prediction models were developed using the methods reported for QRISK2 and 3, before adjusting for additional risk factors, a secular trend, geographical variation in risk and the method for imputing missing data when generating a risk score (model A-model F). Ten-year risk scores were compared across the different models alongside model performance metrics.

RESULTS

We found substantial variation in risk on the individual level across the models. The 95 percentile range of risks in model F for patients with risks between 9 and 10% according to model A was 4.4-16.3% and 4.6-15.8% for females and males respectively. Despite this, the models were difficult to distinguish using common performance metrics (Harrell's C ranged from 0.86 to 0.87). The largest contributing factor to variation in risk was adjusting for a secular trend (HR per calendar year, 0.96 [0.95-0.96] and 0.96 [0.96-0.96]). When extrapolating to the UK population, we found that 3.8 million patients may be reclassified as eligible for statin prescription depending on the model used. A key limitation of this study was that we could not assess the variation in risk that may be caused by risk factors missing from the database (such as diet or physical activity).

CONCLUSIONS

Risk prediction models that use routinely collected data provide estimates strongly dependent on modelling decisions. Despite this large variability in patient risk, the models appear to perform similarly according to standard performance metrics. Decision-making should be supplemented with clinical judgement and evidence of additional risk factors. The largest source of variability, a secular trend in CVD incidence, can be accounted for and should be explored in more detail.

Using marginal structural models to adjust for treatment drop-in when developing clinical prediction models

Clinical prediction models (CPMs) can inform decision making about treatment initiation, which requires predicted risks assuming no treatment is given. However, this is challenging since CPMs are usually derived using data sets where patients received treatment, often initiated postbaseline as "treatment drop-ins." This study proposes the use of marginal structural models (MSMs) to adjust for treatment drop-in. We illustrate the use of MSMs in the CPM framework through simulation studies that represent randomized controlled trials and real-world observational data and the example of statin initiation for cardiovascular disease prevention. The simulations include a binary treatment and a covariate, each recorded at two timepoints and having a prognostic effect on a binary outcome. The bias in predicted risk was examined in a model ignoring treatment, a model fitted on treatment-naïve patients (at baseline), a model including baseline treatment, and the MSM. In all simulation scenarios, all models except the MSM underestimated the risk of outcome given absence of treatment. These results were supported in the statin initiation example, which showed that ignoring statin initiation postbaseline resulted in models that significantly underestimated the risk of a cardiovascular disease event occurring within 10 years. Consequently, CPMs that do not acknowledge treatment drop-in can lead to underallocation of treatment. In conclusion, when developing CPMs to predict treatment-naïve risk, researchers should consider using MSMs to adjust for treatment drop-in, and also seek to exploit the ability of MSMs to allow estimation of individual treatment effects.

Study investigating the generalisability of a COPD trial based in primary care (Salford Lung Study) and the presence of a Hawthorne effect

Introduction

Traditional phase IIIb randomised trials may not reflect routine clinical practice. The Salford Lung Study in chronic obstructive pulmonary disease (SLS COPD) allowed broad inclusion criteria and followed patients in routine practice. We assessed whether SLS COPD approximated the England COPD population and evidence for a Hawthorne effect.

Methods

This observational cohort study compared patients with COPD in the usual care arm of SLS COPD (2012–2014) with matched non-trial patients with COPD in England from the Clinical Practice Research Datalink database. Generalisability was explored with baseline demographics, clinical and treatment variables; outcomes included COPD exacerbations in adjusted models and pretrial versus peritrial comparisons.

Results

Trial participants were younger (mean, 66.7 vs 71.1 years), more deprived (most deprived quintile, 51.5% vs 21.4%), more current smokers (47.5% vs 32.1%), with more severe Global initiative for chronic Obstructive Lung Disease stages but less comorbidity than non-trial patients. There were no material differences in other characteristics. Acute COPD exacerbation rates were high in the trial population (98.37th percentile).

Conclusion

The trial population was similar to the non-trial COPD population. We observed some evidence of a Hawthorne effect, with more exacerbations recorded in trial patients; however, the largest effect was observed through behavioural changes in patients and general practitioner coding practices.

The injured heart: early cardiac effects of hematopoietic stem cell transplantation in children and young adults

We hypothesized that subclinical cardiac injury in the peri-transplant period is more frequent than currently appreciated in children and young adults. We performed echocardiographic screening on 227 consecutive patients prior to hematopoietic stem cell transplantation (HSCT), and 7, 30 and 100 days after transplant. We measured cardiac biomarkers cardiac troponin-I (cTn-I), and soluble suppressor of tumorigenicity 2 (sST2) prior to transplant, during conditioning, and days +7, +14, +28 and +49 in 26 patients. We subsequently analyzed levels of cTn-I every 48-72 h in 15 consecutive children during conditioning. Thirty-two percent (73/227) of patients had a new abnormality on echocardiogram. New left ventricular systolic dysfunction (LVSD) occurred in 6.2% of subjects and new pericardial effusion in 27.3%. Eight of 227 (3.5%) patients underwent pericardial drain placement, and 5 (2.2%) received medical therapy for clinically occult LVSD. cTn-I was elevated in 53.0% of all samples and sST2 in 38.2%. At least one sample had a detectable cTn-I in 84.6% of patients and an elevated sST2 in 76.9%. Thirteen of fifteen patients monitored frequently during condition had elevation of cTn-I. Echocardiographic and biochemical abnormalities are frequent in the peri-HSCT period. Echocardiogram does not detect all subclinical cardiac injuries that may become clinically relevant over longer periods.

Evaluation of biases present in the cohort multiple randomised controlled trial design: a simulation study

Background

The cohort multiple randomised controlled trial (cmRCT) design provides an opportunity to incorporate the benefits of randomisation within clinical practice; thus reducing costs, integrating electronic healthcare records, and improving external validity. This study aims to address a key concern of the cmRCT design: refusal to treatment is only present in the intervention arm, and this may lead to bias and reduce statistical power.

Methods

We used simulation studies to assess the effect of this refusal, both random and related to event risk, on bias of the effect estimator and statistical power. A series of simulations were undertaken that represent a cmRCT trial with time-to-event endpoint. Intention-to-treat (ITT), per protocol (PP), and instrumental variable (IV) analysis methods, two stage predictor substitution and two stage residual inclusion, were compared for various refusal scenarios.

Results

We found the IV methods provide a less biased estimator for the causal effect when refusal is present in the intervention arm, with the two stage residual inclusion method performing best with regards to minimum bias and sufficient power. We demonstrate that sample sizes should be adapted based on expected and actual refusal rates in order to be sufficiently powered for IV analysis.

Conclusion

We recommend running both an IV and ITT analyses in an individually randomised cmRCT as it is expected that the effect size of interest, or the effect we would observe in clinical practice, would lie somewhere between that estimated with ITT and IV analyses. The optimum (in terms of bias and power) instrumental variable method was the two stage residual inclusion method. We recommend using adaptive power calculations, updating them as refusal rates are collected in the trial recruitment phase in order to be sufficiently powered for IV analysis.

Cohort Multiple Randomised Controlled Trials (cmRCT) design: efficient but biased? A simulation study to evaluate the feasibility of the Cluster cmRCT design

BACKGROUND

The Cohort Multiple Randomised Controlled Trial (cmRCT) is a newly proposed pragmatic trial design; recently several cmRCT have been initiated. This study tests the unresolved question of whether differential refusal in the intervention arm leads to bias or loss of statistical power and how to deal with this.

METHODS

We conduct simulations evaluating a hypothetical cluster cmRCT in patients at risk of cardiovascular disease (CVD). To deal with refusal, we compare the analysis methods intention to treat (ITT), per protocol (PP) and two instrumental variable (IV) methods: two stage predictor substitution (2SPS) and two stage residual inclusion (2SRI) with respect to their bias and power. We vary the correlation between treatment refusal probability and the probability of experiencing the outcome to create different scenarios.

RESULTS

We found ITT to be biased in all scenarios, PP the most biased when correlation is strong and 2SRI the least biased on average. Trials suffer a drop in power unless the refusal rate is factored into the power calculation.

CONCLUSIONS

The ITT effect in routine practice is likely to lie somewhere between the ITT and IV estimates from the trial which differ significantly depending on refusal rates. More research is needed on how refusal rates of experimental interventions correlate with refusal rates in routine practice to help answer the question of which analysis more relevant. We also recommend updating the required sample size during the trial as more information about the refusal rate is gained.

Effects of perinatal thyroid hormone deprivation on the growth and behaviour of newborn mice

Experiments were designed to distinguish between neonatal effects due to maternal thyroxine (T4) deprivation and those due to autonomous (fetus/pup) T4 deprivation, employing mice heterozygous for the bTG-tk transgene TG66,19 which specifically directs high-level expression of herpes virus type I thymidine kinase to the thyrocytes. Heterozygous TG66.19 females were either untreated or Ganciclovir was administered to destroy their thyrocytes and so render them T4-deficient. When mated to normal males these heterozygous females are expected to produce on average 50% normal and 50% heterozygous transgenic conceptuses. Ganciclovir was administered to the dams (both untreated and Ganciclovir-pretreated) during days 14-18 of gestation. At optimum levels of in utero Ganciclovir administration the non-transgenic pups showed no discernible effect while the transgenic pups were rendered athyrocytic and completely T4-deficient. The dams pretreated with Ganciclovir are hypothyroid throughout gestation, while the dams to which Ganciclovir was administered for the first time during gestation are not expected to become hypothyroid until about the time of parturition. In this way four sets of pups were generated for purposes of comparison: hypothyroid (transgenic) and euthyroid (non-transgenic) pups born to euthyroid dams and hypothyroid and euthyroid pups born to hypothyroid (Ganciclovir-pretreated) dams. Normal growth during days 1-10 after birth was dominated by the T4 status of the dam during gestation. Growth during days 11-21 and the correct timing of eye opening and ear elevation were dominated by the autonomous T4 status of the fetus/pup. The timely development of the surface-righting reflex (relative to weight gain) was shown to require both maternal and fetus/pup T4. The development of the cliff-avoidance reflex was independent of the T4 status of both pup and dam and of pup weight. The size of the pups at birth depended primarily on a normal T4 status in the dam but surprisingly T4 deficiency in fetuses/pups partly compensated for maternal T4 deficiency. The results presented here clearly demonstrate the utility of the HSV-tk-transgene-Ganciclovir-administration protocol in studying the interplay of maternal and fetal T4 deprivation in rodents.