Aleatoric Uncertainty for Errors-in-Variables Models in Deep Regression

A Bayesian treatment of deep learning allows for the computation of uncertainties associated with the predictions of deep neural networks. We show how the concept of Errors-in-Variables can be used in Bayesian deep regression to also account for the uncertainty associated with the input of the employed neural network. The presented approach thereby exploits a relevant, but generally overlooked, source of uncertainty and yields a decomposition of the predictive uncertainty into an aleatoric and epistemic part that is more complete and, in many cases, more consistent from a statistical perspective. We discuss the approach along various simulated and real examples and observe that using an Errors-in-Variables model leads to an increase in the uncertainty while preserving the prediction performance of models without Errors-in-Variables. For examples with known regression function we observe that this ground truth is substantially better covered by the Errors-in-Variables model, indicating that the presented approach leads to a more reliable uncertainty estimation.

OpenFL: the open federated learning library

Objective.Federated learning (FL) is a computational paradigm that enables organizations to collaborate on machine learning (ML) and deep learning (DL) projects without sharing sensitive data, such as patient records, financial data, or classified secrets.Approach.Open federated learning (OpenFL) framework is an open-source python-based tool for training ML/DL algorithms using the data-private collaborative learning paradigm of FL, irrespective of the use case. OpenFL works with training pipelines built with both TensorFlow and PyTorch, and can be easily extended to other ML and DL frameworks.Main results.In this manuscript, we present OpenFL and summarize its motivation and development characteristics, with the intention of facilitating its application to existing ML/DL model training in a production environment. We further provide recommendations to secure a federation using trusted execution environments to ensure explicit model security and integrity, as well as maintain data confidentiality. Finally, we describe the first real-world healthcare federations that use the OpenFL library, and highlight how it can be applied to other non-healthcare use cases.Significance.The OpenFL library is designed for real world scalability, trusted execution, and also prioritizes easy migration of centralized ML models into a federated training pipeline. Although OpenFL's initial use case was in healthcare, it is applicable beyond this domain and is now reaching wider adoption both in research and production settings. The tool is open-sourced atgithub.com/intel/openfl.

The federated tumor segmentation (FeTS) tool: an open-source solution to further solid tumor research

Objective.De-centralized data analysis becomes an increasingly preferred option in the healthcare domain, as it alleviates the need for sharing primary patient data across collaborating institutions. This highlights the need for consistent harmonized data curation, pre-processing, and identification of regions of interest based on uniform criteria.Approach.Towards this end, this manuscript describes theFederatedTumorSegmentation (FeTS) tool, in terms of software architecture and functionality.Main results.The primary aim of the FeTS tool is to facilitate this harmonized processing and the generation of gold standard reference labels for tumor sub-compartments on brain magnetic resonance imaging, and further enable federated training of a tumor sub-compartment delineation model across numerous sites distributed across the globe, without the need to share patient data.Significance.Building upon existing open-source tools such as the Insight Toolkit and Qt, the FeTS tool is designed to enable training deep learning models targeting tumor delineation in either centralized or federated settings. The target audience of the FeTS tool is primarily the computational researcher interested in developing federated learning models, and interested in joining a global federation towards this effort. The tool is open sourced athttps://github.com/FETS-AI/Front-End.

Clinical characteristics with inflammation profiling of long COVID and association with 1-year recovery following hospitalisation in the UK: a prospective observational study

BACKGROUND

No effective pharmacological or non-pharmacological interventions exist for patients with long COVID. We aimed to describe recovery 1 year after hospital discharge for COVID-19, identify factors associated with patient-perceived recovery, and identify potential therapeutic targets by describing the underlying inflammatory profiles of the previously described recovery clusters at 5 months after hospital discharge.

METHODS

The Post-hospitalisation COVID-19 study (PHOSP-COVID) is a prospective, longitudinal cohort study recruiting adults (aged ≥18 years) discharged from hospital with COVID-19 across the UK. Recovery was assessed using patient-reported outcome measures, physical performance, and organ function at 5 months and 1 year after hospital discharge, and stratified by both patient-perceived recovery and recovery cluster. Hierarchical logistic regression modelling was performed for patient-perceived recovery at 1 year. Cluster analysis was done using the clustering large applications k-medoids approach using clinical outcomes at 5 months. Inflammatory protein profiling was analysed from plasma at the 5-month visit. This study is registered on the ISRCTN Registry, ISRCTN10980107, and recruitment is ongoing.

FINDINGS

2320 participants discharged from hospital between March 7, 2020, and April 18, 2021, were assessed at 5 months after discharge and 807 (32·7%) participants completed both the 5-month and 1-year visits. 279 (35·6%) of these 807 patients were women and 505 (64·4%) were men, with a mean age of 58·7 (SD 12·5) years, and 224 (27·8%) had received invasive mechanical ventilation (WHO class 7-9). The proportion of patients reporting full recovery was unchanged between 5 months (501 [25·5%] of 1965) and 1 year (232 [28·9%] of 804). Factors associated with being less likely to report full recovery at 1 year were female sex (odds ratio 0·68 [95% CI 0·46-0·99]), obesity (0·50 [0·34-0·74]) and invasive mechanical ventilation (0·42 [0·23-0·76]). Cluster analysis (n=1636) corroborated the previously reported four clusters: very severe, severe, moderate with cognitive impairment, and mild, relating to the severity of physical health, mental health, and cognitive impairment at 5 months. We found increased inflammatory mediators of tissue damage and repair in both the very severe and the moderate with cognitive impairment clusters compared with the mild cluster, including IL-6 concentration, which was increased in both comparisons (n=626 participants). We found a substantial deficit in median EQ-5D-5L utility index from before COVID-19 (retrospective assessment; 0·88 [IQR 0·74-1·00]), at 5 months (0·74 [0·64-0·88]) to 1 year (0·75 [0·62-0·88]), with minimal improvements across all outcome measures at 1 year after discharge in the whole cohort and within each of the four clusters.

INTERPRETATION

The sequelae of a hospital admission with COVID-19 were substantial 1 year after discharge across a range of health domains, with the minority in our cohort feeling fully recovered. Patient-perceived health-related quality of life was reduced at 1 year compared with before hospital admission. Systematic inflammation and obesity are potential treatable traits that warrant further investigation in clinical trials.

FUNDING

UK Research and Innovation and National Institute for Health Research.

Explainability for deep learning in mammography image quality assessment

The application of deep learning has recently been proposed for the assessment of image quality in mammography. It was demonstrated in a proof-of-principle study that the proposed approach can be more efficient than currently applied automated conventional methods. However, in contrast to conventional methods, the deep learning approach has a black-box nature and, before it can be recommended for the routine use, it must be understood more thoroughly. For this purpose, we propose and apply a new explainability method: the oriented, modified integrated gradients (OMIG) method. The design of this method is inspired by the integrated gradientsmethod but adapted considerably to the use case at hand. To further enhance this method, an upsampling technique is developed that produces high-resolution explainability maps for the downsampled data used by the deep learning approach. Comparison with established explainability methods demonstrates that the proposed approach yields substantially more expressive and informative results for our specific use case. Application of the proposed explainability approach generally confirms the validity of the considered deep learning-based mammography image quality assessment (IQA) method. Specifically, it is demonstrated that the predicted image quality is based on a meaningful mapping that makes successful use of certain geometric structures of the images. In addition, the novel explainability method helps us to identify the parts of the employed phantom that have the largest impact on the predicted image quality, and to shed some light on cases in which the trained neural networks fail to work as expected. While tailored to assess a specific approach from deep learning for mammography IQA, the proposed explainability method could also become relevant in other, similar deep learning applications based on high-dimensional images.

Evaluation of prognostic risk models for postoperative pulmonary complications in adult patients undergoing major abdominal surgery: a systematic review and international external validation cohort study

BACKGROUND

Stratifying risk of postoperative pulmonary complications after major abdominal surgery allows clinicians to modify risk through targeted interventions and enhanced monitoring. In this study, we aimed to identify and validate prognostic models against a new consensus definition of postoperative pulmonary complications.

METHODS

We did a systematic review and international external validation cohort study. The systematic review was done in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines. We searched MEDLINE and Embase on March 1, 2020, for articles published in English that reported on risk prediction models for postoperative pulmonary complications following abdominal surgery. External validation of existing models was done within a prospective international cohort study of adult patients (≥18 years) undergoing major abdominal surgery. Data were collected between Jan 1, 2019, and April 30, 2019, in the UK, Ireland, and Australia. Discriminative ability and prognostic accuracy summary statistics were compared between models for the 30-day postoperative pulmonary complication rate as defined by the Standardised Endpoints in Perioperative Medicine Core Outcome Measures in Perioperative and Anaesthetic Care (StEP-COMPAC). Model performance was compared using the area under the receiver operating characteristic curve (AUROCC).

FINDINGS

In total, we identified 2903 records from our literature search; of which, 2514 (86·6%) unique records were screened, 121 (4·8%) of 2514 full texts were assessed for eligibility, and 29 unique prognostic models were identified. Nine (31·0%) of 29 models had score development reported only, 19 (65·5%) had undergone internal validation, and only four (13·8%) had been externally validated. Data to validate six eligible models were collected in the international external validation cohort study. Data from 11 591 patients were available, with an overall postoperative pulmonary complication rate of 7·8% (n=903). None of the six models showed good discrimination (defined as AUROCC ≥0·70) for identifying postoperative pulmonary complications, with the Assess Respiratory Risk in Surgical Patients in Catalonia score showing the best discrimination (AUROCC 0·700 [95% CI 0·683-0·717]).

INTERPRETATION

In the pre-COVID-19 pandemic data, variability in the risk of pulmonary complications (StEP-COMPAC definition) following major abdominal surgery was poorly described by existing prognostication tools. To improve surgical safety during the COVID-19 pandemic recovery and beyond, novel risk stratification tools are required.

FUNDING

British Journal of Surgery Society.

"You can't just eat 16 teaspoons of sugar so why would you drink 16 teaspoons' worth of sugar?": a qualitative study of young adults' reactions to sugary drink warning labels

Background

Several jurisdictions have introduced nutrient warning front of pack (FoP) labels in an effort to curb consumption of ultra-processed foods and beverages high in free sugars (sugars added to foods and beverages, and sugars naturally present in honey, syrups, fruit juices and fruit juice concentrates). This study aimed to explore consumer understanding and perceptions of FoP warning labels that convey different nutritional and health information messages regarding the consumption of sugary drinks.

Methods

Sixteen focus groups were held with 4–8 young adults per group (aged 18–24; n = 105 participants in total) stratified by education level, location (rural centres, large cities) and gender (males, females) to ensure diversity. Labels shown to participants during group discussions included text warning labels of health effects, exercise equivalents, calorie/kilojoule information and sugar content as a “high in” label and as teaspoons (text and pictograms). Thematic analysis was undertaken.

Results

Four themes were identified related to participants’ perceived effectiveness of labels: the extent to which labels were perceived to be useful, relevant and credible; the extent to which a label elicited shock or disgust (perceived aversiveness); the extent to which the label message was resistant to self-exemption; and participants’ perceived potential of the label to reduce purchasing and consumption behaviour. Across all four themes, labels communicating the number of teaspoons of sugar in a sugary drink (whether by text or pictogram) were perceived as the most impactful, resistant to self-exemption and to have the greatest potential to reduce consumption, with enhanced reactions to the pictogram label. Labels depicting health effects, exercise equivalents, calorie/kilojoule information or a general ‘high in sugar’ warning were perceived by consumers to be less effective in one or more themes.

Conclusions

Labels conveying the amount of sugar in a beverage in teaspoons were perceived as highly factual, relatable and interpretable, and as having the greatest potential to impact consumption attitudes and intentions. Further quantitative studies are required to compare the potential effectiveness of the teaspoons of sugar labels in reducing purchasing and consumption behaviour than other alternative warning labels, such as health effects or “high in” sugar labels.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12889-022-13648-1.

Martin Library Leadership survey: development, reliability and validation

Purpose

An effective measurement of library leadership is crucial to understanding the current state of library leadership and to developing library leaders. This study sought to validate and measure the reliability of the Martin Library Leadership survey.

Design/methodology/approach

This survey is based on the Martin Library Leadership Definition, an evidence-based definition of library leadership. The first version of the survey consisted of 28 questions plus questions on respondent and library leader demographics. Each question measured one of the three components of the definition. This version of the survey was distributed to multiple ALA listservs and after analysis 16 items were removed. The resulting 12 question version of the survey was sent to the same ALA listservs and completed by 291 librarians and library staff from various library types and library work areas. The responses were analyzed using SPSS.

Findings

Exploratory factor analysis found three factors that align with the three components of the Martin Library Leadership Definition, and questions loaded in their expected factors at least 0.7. Cronbach's alpha was used to determine internal consistency. The alpha for the entire survey was 0.956. The Martin Library Leadership survey was validated and found to be reliable.

Originality/value

The results of this study provide strong and consistent evidence the Martin Library Leadership survey is valid and can be used in further library leadership research and professional development.

799 FRAILTY MATTERS PROJECT

Introduction

Preventing and managing frailty is a new area for many community practitioners yet frailty specific education remains limited. We aimed to understand and strengthen the capability of District Nurses (DNs) in leading personalised care for older people with frailty.

Methods

We conducted a participatory action research (PAR) study with DNs in one Scottish NHS Board area. Phase 1 involved three focus groups (n = 17); one one-to-one interviews; and collection of baseline team dynamics questionnaires (n = 10). Evidence from phase 1 informed co-design of an educational framework, delivered in Phase 2 as a combined coaching and educational programme through small group learning, web based coaching and bite sized online education. Interactive sessions were supported by a person-centred coach and 2 older citizen ‘co-coaches’ to bring the experience of people affected by frailty. In Phase 3 we analysed participant feedback and assessed transferability to other disciplines and to health and care settings across the UK.

Results

At baseline, DNs did not perceive frailty as a long term condition. They identified a need for help to understand the concept of frailty and to build skills and confidence in delivering community interventions. Participants embraced the coaching and educational intervention and valued the opportunity for dialogue with peers and citizen co-coaches about what really matters to patients, families and professionals. Our survey of other disciplines and teams highlighted this educational programme is both relevant and transferable.

Conclusion

Through co-design we developed a contextually sensitive programme that makes sense of frailty in the reality of both community professionals and people living with frailty. Combining technical knowledge and relational skills-building with peer support and coaching helps prepare DNs to lead interprofessional teams caring for people living with frailty. The educational framework and combined coaching and educational package are highly applicable to interdisciplinary teams in other community settings.

Surgical site infection after wound closure with staples versus sutures in elective knee and hip arthroplasty: a systematic review and meta-analysis

Purpose

This systematic review and meta-analysis aimed to study surgical site infection of wound closure using staples versus sutures in elective knee and hip arthroplasties.

Methods

A systematic literature review was performed to search for randomized controlled trials that compared surgical site infection after wound closure using staples versus sutures in elective knee and hip arthroplasties. The primary outcome was surgical site infection. The risk of bias was assessed with the Cochrane risk of bias assessment tool. The relative risk and 95% confidence interval with a random-effects model were assessed.

Results

Eight studies were included in this study, including 2 studies with a low risk of bias, 4 studies having ‘some concerns’, and 2 studies with high risk of bias. Significant difference was not found in the risk of SSI for patients with staples (n = 557) versus sutures (n = 573) (RR: 1.70, 95% CI: 0.94–3.08, I² = 16%). The results were similar after excluding the studies with a high risk of bias (RR: 1.67, 95% CI: 0.91–3.07, I² = 32%). Analysis of studies with low risk of bias revealed a significantly higher risk of surgical site infection in patients with staples (n = 331) compared to sutures (n = 331) (RR: 2.56, 95% CI: 1.20–5.44, I² = 0%). There was no difference between continuous and interrupted sutures (P > 0.05). In hip arthroplasty, stapling carried a significantly higher risk of surgical site infection than suturing (RR: 2.51, 95% CI: 1.15–5.50, I² = 0%), but there was no significant difference in knee arthroplasty (RR: 0.87, 95% CI: 0.33–2.25, I² = 22%; P > 0.05).

Conclusions

Stapling might carry a higher risk of surgical site infection than suturing in elective knee and hip arthroplasties, especially in hip arthroplasty.

Supplementary Information

The online version contains supplementary material available at 10.1186/s42836-021-00110-7.

Racial and ethnic disparities in COVID-19 disease incidence independent of comorbidities, among people with HIV in the US

OBJECTIVES

To define the incidence of clinically-detected COVID-19 in people with HIV (PWH) in the US and evaluate how racial and ethnic disparities, comorbidities, and HIV-related factors contribute to risk of COVID-19.

DESIGN

Observational study within the CFAR Network of Integrated Clinical Systems cohort in 7 cities during 2020.

METHODS

We calculated cumulative incidence rates of COVID-19 diagnosis among PWH in routine care by key characteristics including race/ethnicity, current and lowest CD4 count, and geographic area. We evaluated risk factors for COVID-19 among PWH using relative risk regression models adjusted with disease risk scores.

RESULTS

Among 16,056 PWH in care, of whom 44.5% were Black, 12.5% were Hispanic, with a median age of 52 years (IQR 40-59), 18% had a current CD4 count < 350, including 7% < 200; 95.5% were on antiretroviral therapy, and 85.6% were virologically suppressed. Overall in 2020, 649 PWH were diagnosed with COVID-19 for a rate of 4.94 cases per 100 person-years. The cumulative incidence of COVID-19 was 2.4-fold and 1.7-fold higher in Hispanic and Black PWH respectively, than non-Hispanic White PWH. In adjusted analyses, factors associated with COVID-19 included female sex, Hispanic or Black identity, lowest historical CD4 count <350 (proxy for CD4 nadir), current low CD4/CD8 ratio, diabetes, and obesity.

CONCLUSIONS

Our results suggest that the presence of structural racial inequities above and beyond medical comorbidities increased the risk of COVID-19 among PWHPWH with immune exhaustion as evidenced by lowest historical CD4 or current low CD4:CD8 ratio had greater risk of COVID-19.

NIMG-32. THE FEDERATED TUMOR SEGMENTATION (FETS) INITIATIVE: THE FIRST REAL-WORLD LARGE-SCALE DATA-PRIVATE COLLABORATION FOCUSING ON NEURO-ONCOLOGY

PURPOSE

Robustness and generalizability of artificial intelligent (AI) methods is reliant on the training data size and diversity, which are currently hindered in multi-institutional healthcare collaborations by data ownership and legal concerns. To address these, we introduce the Federated Tumor Segmentation (FeTS) Initiative, as an international consortium using federated learning (FL) for data-private multi-institutional collaborations, where AI models leverage data at participating institutions, without sharing data between them. The initial FeTS use-case focused on detecting brain tumor boundaries in MRI.

METHODS

The FeTS tool incorporates: 1) MRI pre-processing, including image registration and brain extraction; 2) automatic delineation of tumor sub-regions, by label fusion of pretrained top-performing BraTS methods; 3) tools for manual delineation refinements; 4) model training. 55 international institutions identified local retrospective cohorts of glioblastoma patients. Ground truth was generated using the first 3 FeTS functionality modes as mentioned earlier. Finally, the FL training mode comprises of i) an AI model trained on local data, ii) local model updates shared with an aggregator, which iii) combines updates from all collaborators to generate a consensus model, and iv) circulates the consensus model back to all collaborators for iterative performance improvements.

RESULTS

The first FeTS consensus model, from 23 institutions with data of 2,200 patients, showed an average improvement of 11.1% in the performance of the model on each collaborator’s validation data, when compared to a model trained on the publicly available BraTS data (n=231).

CONCLUSION

Our findings support that data increase alone would lead to AI performance improvements without any algorithmic development, hence indicating that the model performance would improve further when trained with all 55 collaborating institutions. FL enables AI model training with knowledge from data of geographically-distinct collaborators, without ever having to share any data, hence overcoming hurdles relating to legal, ownership, and technical concerns of data sharing.

Interim data monitoring in cluster randomised trials: Practical issues and a case study

BACKGROUND

There is an abundance of guidance for the interim monitoring of individually randomised trials. While methodological literature exists on how to extend these methods to cluster randomised trials, there is little guidance on practical implementation. Cluster trials have many features which make their monitoring needs different. We outline the methodological and practical challenges of interim monitoring of cluster trials; and apply these considerations to a case study.

CASE STUDY

The E-MOTIVE study is an 80-cluster randomised trial of a bundle of interventions to treat postpartum haemorrhage. The proposed data monitoring plan includes (1) monitor sample size assumptions, (2) monitor for evidence of selection bias, and (3) an interim assessment of the primary outcome, as well as monitoring data completeness. The timing of the sample size monitoring is chosen with both consideration of statistical precision and to allow time to recruit more clusters. Monitoring for selection bias involves comparing individual-level characteristics and numbers recruited between study arms to identify any post-randomisation participant identification bias. An interim analysis of outcomes presented with 99.9% confidence intervals using the Haybittle-Peto approach should mitigate any concern regarding the inflation of type-I error. The pragmatic nature of the trial means monitoring for adherence is not relevant, as it is built into a process evaluation.

CONCLUSIONS

The interim analyses of cluster trials have a number of important differences to monitoring individually randomised trials. In cluster trials, there will often be a greater need to monitor nuisance parameters, yet there will often be considerable uncertainty in their estimation. This means the utility of sample size re-estimation can be questionable particularly when there are practical or funding difficulties associated with making any changes to planned sample sizes. Perhaps most importantly interim monitoring has the potential to identify selection bias, particularly in trials with post-randomisation identification or recruitment. Finally, the pragmatic nature of cluster trials might mean that the utility of methods to allow for interim monitoring of outcomes based on statistical testing, or monitoring for adherence to study interventions, are less relevant. Our intention is to facilitate the planning of future cluster randomised trials and to promote discussion and debate to improve monitoring of these studies.

BSHI/BTS guidance on crossmatching before deceased donor kidney transplantation

All UK H&I laboratories and transplant units operate under a single national kidney offering policy, but there have been variations in approach regarding when to undertake the pre‐transplant crossmatch test. In order to minimize cold ischaemia times for deceased donor kidney transplantation we sought to find ways to be able to report a crossmatch result as early as possible in the donation process. A panel of experts in transplant surgery, nephrology, specialist nursing in organ donation and H&I (all relevant UK laboratories represented) assessed evidence and opinion concerning five factors that relate to the effectiveness of the crossmatch process, as follows: when the result should be ready for reporting; what level of donor HLA typing is needed; crossmatch sample type and availability; fairness and equity; risks and patient safety. Guidelines aimed at improving practice based on these issues are presented, and we expect that following these will allow H&I laboratories to contribute to reducing CIT in deceased donor kidney transplantation.

Increasing the value of quality management systems

Purpose

Over one million organisations have a quality management system (QMS) certified to the ISO 9001 standard; however, the system requires a lot of resources and its value has been questioned. This critique also leads to a questioning of the strategic relevance of quality management. The purpose of this paper is to explore how different types of uses of QMS correlate with management perceptions of quality management in terms of respect, cost and strategic importance.

Design/methodology/approach

The paper is based on a mixed method data collection strategy, quantitative data being collected from a survey in 8 organisations (n = 108) and qualitative data being collected from 12 interviews with quality managers in 12 different organisations.

Findings

The paper shows that a compliance-oriented QMS usage will more likely lead to a view of quality management as costly and of little respect, than a business or improvement-oriented QMS usage. Moreover, it nuances the view on compliance-oriented usage, showing that it is mainly documentation that negatively influences how management views quality management, whereas standardisation that is part of the compliance-oriented use is perceived as more value-adding.

Originality/value

This paper suggests three types of QMS use, namely, business management, improvement, and compliance-oriented use, and that a wise selection of how to use the QMS will affect the respect, strategic importance and cost that management associates with quality management.