CogStack - experiences of deploying integrated information retrieval and extraction services in a large National Health Service Foundation Trust hospital

Early evaluation of a natural language processing tool to improve access to educational resources for surgical patients

PURPOSE

Accessible patient information sources are vital in educating patients about the benefits and risks of spinal surgery, which is crucial for obtaining informed consent. We aim to assess the effectiveness of a natural language processing (NLP) pipeline in recognizing surgical procedures from clinic letters and linking this with educational resources.

METHODS

Retrospective examination of letters from patients seeking surgery for degenerative spinal disease at a single neurosurgical center. We utilized MedCAT, a named entity recognition and linking NLP, integrated into the electronic health record (EHR), which extracts concepts and links them to systematized nomenclature of medicine-clinical terms (SNOMED-CT). Investigators reviewed clinic letters, identifying words or phrases that described or identified operations and recording the SNOMED-CT terms as ground truth. This was compared to SNOMED-CT terms identified by the model, untrained on our dataset. A pipeline linking clinic letters to patient-specific educational resources was established, and precision, recall, and F1 scores were calculated.

RESULTS

Across 199 letters the model identified 582 surgical procedures, and the overall pipeline after adding rules a total of 784 procedures (precision = 0.94, recall = 0.86, F1 = 0.91). Across 187 letters with identified SNOMED-CT terms the integrated pipeline linking education resources directly to the EHR was successful in 157 (78%) patients (precision = 0.99, recall = 0.87, F1 = 0.92).

CONCLUSIONS

NLP accurately identifies surgical procedures in pre-operative clinic letters within an untrained subspecialty. Performance varies among letter authors and depends on the language used by clinicians. The identified procedures can be linked to patient education resources, potentially improving patients' understanding of surgical procedures.

Potential application of artificial intelligence in cancer therapy

PURPOSE OF REVIEW

This review underscores the critical role and challenges associated with the widespread adoption of artificial intelligence in cancer care to enhance disease management, streamline clinical processes, optimize data retrieval of health information, and generate and synthesize evidence.

RECENT FINDINGS

Advancements in artificial intelligence models and the development of digital biomarkers and diagnostics are applicable across the cancer continuum from early detection to survivorship care. Additionally, generative artificial intelligence has promised to streamline clinical documentation and patient communications, generate structured data for clinical trial matching, automate cancer registries, and facilitate advanced clinical decision support. Widespread adoption of artificial intelligence has been slow because of concerns about data diversity and data shift, model reliability and algorithm bias, legal oversight, and high information technology and infrastructure costs.

SUMMARY

Artificial intelligence models have significant potential to transform cancer care. Efforts are underway to deploy artificial intelligence models in the cancer practice, evaluate their clinical impact, and enhance their fairness and explainability. Standardized guidelines for the ethical integration of artificial intelligence models in cancer care pathways and clinical operations are needed. Clear governance and oversight will be necessary to gain trust in artificial intelligence-assisted cancer care by clinicians, scientists, and patients.

Diagnostic accuracy, clinical characteristics, and prognostic differences of patients with acute myocarditis according to inclusion criteria

INTRODUCTION

The diagnosis of acute myocarditis (AM) is complex due to its heterogeneity and typically is defined by either Electronic Healthcare Records (EHRs) or advanced imaging and endomyocardial biopsy, but there is no consensus. We aimed to investigate the diagnostic accuracy of these approaches for AM.

METHODS

Data on ICD 10th Revision(ICD-10) codes corresponding to AM were collected from two hospitals and compared to cardiac magnetic resonance (CMR)-confirmed or clinically suspected (CS)-AM cases with respect to diagnostic accuracy, clinical characteristics, and all-cause mortality. Next, we performed a review of published AM studies according to inclusion criteria.

RESULTS

We identified 291 unique admissions with ICD-10 codes corresponding to AM in the first three diagnostic positions. The positive predictive value of ICD-10 codes for CMR-confirmed or CS-AM was 36%, and patients with CMR-confirmed or CS-AM had a lower all-cause mortality than those with a refuted diagnosis (P = 0.019). Using an unstructured approach, patients with CMR-confirmed and CS-AM had similar demographics, comorbidity profiles and survival over a median follow-up of 52 months (P = 0.72). Our review of the literature confirmed our findings. Outcomes for patients included in studies using CMR-confirmed criteria were favourable compared to studies with endomyocardial biopsy-confirmed AM cases.

CONCLUSION

ICD-10 codes have poor accuracy in identification of AM cases and should be used with caution in clinical research. There are important differences in management and outcomes of patients according to the selection criteria used to diagnose AM. Potential selection biases must be considered when interpreting AM cohorts and requires standardization of inclusion criteria for AM studies.

Clinical Information Retrieval: A Literature Review

Clinical information retrieval (IR) plays a vital role in modern healthcare by facilitating efficient access and analysis of medical literature for clinicians and researchers. This scoping review aims to offer a comprehensive overview of the current state of clinical IR research and identify gaps and potential opportunities for future studies in this field. The main objective was to assess and analyze the existing literature on clinical IR, focusing on the methods, techniques, and tools employed for effective retrieval and analysis of medical information. Adhering to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines, we conducted an extensive search across databases such as Ovid Embase, Ovid Medline, Scopus, ACM Digital Library, IEEE Xplore, and Web of Science, covering publications from January 1, 2010, to January 4, 2023. The rigorous screening process led to the inclusion of 184 papers in our review. Our findings provide a detailed analysis of the clinical IR research landscape, covering aspects like publication trends, data sources, methodologies, evaluation metrics, and applications. The review identifies key research gaps in clinical IR methods such as indexing, ranking, and query expansion, offering insights and opportunities for future studies in clinical IR, thus serving as a guiding framework for upcoming research efforts in this rapidly evolving field. The study also underscores an imperative for innovative research on advanced clinical IR systems capable of fast semantic vector search and adoption of neural IR techniques for effective retrieval of information from unstructured electronic health records (EHRs).

Supplementary Information

The online version contains supplementary material available at 10.1007/s41666-024-00159-4.

Foresight-a generative pretrained transformer for modelling of patient timelines using electronic health records: a retrospective modelling study

BACKGROUND

An electronic health record (EHR) holds detailed longitudinal information about a patient's health status and general clinical history, a large portion of which is stored as unstructured, free text. Existing approaches to model a patient's trajectory focus mostly on structured data and a subset of single-domain outcomes. This study aims to evaluate the effectiveness of Foresight, a generative transformer in temporal modelling of patient data, integrating both free text and structured formats, to predict a diverse array of future medical outcomes, such as disorders, substances (eg, to do with medicines, allergies, or poisonings), procedures, and findings (eg, relating to observations, judgements, or assessments).

METHODS

Foresight is a novel transformer-based pipeline that uses named entity recognition and linking tools to convert EHR document text into structured, coded concepts, followed by providing probabilistic forecasts for future medical events, such as disorders, substances, procedures, and findings. The Foresight pipeline has four main components: (1) CogStack (data retrieval and preprocessing); (2) the Medical Concept Annotation Toolkit (structuring of the free-text information from EHRs); (3) Foresight Core (deep-learning model for biomedical concept modelling); and (4) the Foresight web application. We processed the entire free-text portion from three different hospital datasets (King's College Hospital [KCH], South London and Maudsley [SLaM], and the US Medical Information Mart for Intensive Care III [MIMIC-III]), resulting in information from 811 336 patients and covering both physical and mental health institutions. We measured the performance of models using custom metrics derived from precision and recall.

FINDINGS

Foresight achieved a precision@10 (ie, of 10 forecasted candidates, at least one is correct) of 0·68 (SD 0·0027) for the KCH dataset, 0·76 (0·0032) for the SLaM dataset, and 0·88 (0·0018) for the MIMIC-III dataset, for forecasting the next new disorder in a patient timeline. Foresight also achieved a precision@10 value of 0·80 (0·0013) for the KCH dataset, 0·81 (0·0026) for the SLaM dataset, and 0·91 (0·0011) for the MIMIC-III dataset, for forecasting the next new biomedical concept. In addition, Foresight was validated on 34 synthetic patient timelines by five clinicians and achieved a relevancy of 33 (97% [95% CI 91-100]) of 34 for the top forecasted candidate disorder. As a generative model, Foresight can forecast follow-on biomedical concepts for as many steps as required.

INTERPRETATION

Foresight is a general-purpose model for biomedical concept modelling that can be used for real-world risk forecasting, virtual trials, and clinical research to study the progression of disorders, to simulate interventions and counterfactuals, and for educational purposes.

FUNDING

National Health Service Artificial Intelligence Laboratory, National Institute for Health and Care Research Biomedical Research Centre, and Health Data Research UK.

Diagnostic yield and limitations of in-hospital documentation in patients with epilepsy

OBJECTIVE

To determine the diagnostic yield of in-hospital video-electroencephalography (EEG) monitoring to document seizures in patients with epilepsy.

METHODS

Retrospective analysis of electronic seizure documentation at the University Hospital Freiburg (UKF) and at King's College London (KCL). Statistical assessment of the role of the duration of monitoring, and subanalyses on presurgical patient groups and patients undergoing reduction of antiseizure medication.

RESULTS

Of more than 4800 patients with epilepsy undergoing in-hospital recordings at the two institutions since 2005, seizures with documented for 43% (KCL) and 73% (UKF).. Duration of monitoring was highly significantly associated with seizure recordings (p < .0001), and presurgical patients as well as patients with drug reduction had a significantly higher diagnostic yield (p < .0001). Recordings with a duration of >5 days lead to additional new seizure documentation in only less than 10% of patients.

SIGNIFICANCE

There is a need for the development of new ambulatory monitoring strategies to document seizures for diagnostic and monitoring purposes for a relevant subgroup of patients with epilepsy in whom in-hospital monitoring fails to document seizures.

Trustworthy Data and AI Environments for Clinical Prediction: Application to Crisis-Risk in People With Depression

Depression is a common mental health condition that often occurs in association with other chronic illnesses, and varies considerably in severity. Electronic Health Records (EHRs) contain rich information about a patient's medical history and can be used to train, test and maintain predictive models to support and improve patient care. This work evaluated the feasibility of implementing an environment for predicting mental health crisis among people living with depression based on both structured and unstructured EHRs. A large EHR from a mental health provider, Mersey Care, was pseudonymised and ingested into the Natural Language Processing (NLP) platform CogStack, allowing text content in binary clinical notes to be extracted. All unstructured clinical notes and summaries were semantically annotated by MedCAT and BioYODIE NLP services. Cases of crisis in patients with depression were then identified. Random forest models, gradient boosting trees, and Long Short-Term Memory (LSTM) networks, with varying feature arrangement, were trained to predict the occurrence of crisis. The results showed that all the prediction models can use a combination of structured and unstructured EHR information to predict crisis in patients with depression with good and useful accuracy. The LSTM network that was trained on a modified dataset with only 1000 most-important features from the random forest model with temporality showed the best performance with a mean AUC of 0.901 and a standard deviation of 0.006 using a training dataset and a mean AUC of 0.810 and 0.01 using a hold-out test dataset. Comparing the results from the technical evaluation with the views of psychiatrists shows that there are now opportunities to refine and integrate such prediction models into pragmatic point-of-care clinical decision support tools for supporting mental healthcare delivery.

Current and Future Advances in Surgical Therapy for Pituitary Adenoma

The vital physiological role of the pituitary gland, alongside its proximity to critical neurovascular structures, means that pituitary adenomas can cause significant morbidity or mortality. While enormous advancements have been made in the surgical care of pituitary adenomas, numerous challenges remain, such as treatment failure and recurrence. To meet these clinical challenges, there has been an enormous expansion of novel medical technologies (eg, endoscopy, advanced imaging, artificial intelligence). These innovations have the potential to benefit each step of the patient's journey, and ultimately, drive improved outcomes. Earlier and more accurate diagnosis addresses this in part. Analysis of novel patient data sets, such as automated facial analysis or natural language processing of medical records holds potential in achieving an earlier diagnosis. After diagnosis, treatment decision-making and planning will benefit from radiomics and multimodal machine learning models. Surgical safety and effectiveness will be transformed by smart simulation methods for trainees. Next-generation imaging techniques and augmented reality will enhance surgical planning and intraoperative navigation. Similarly, surgical abilities will be augmented by the future operative armamentarium, including advanced optical devices, smart instruments, and surgical robotics. Intraoperative support to surgical team members will benefit from a data science approach, utilizing machine learning analysis of operative videos to improve patient safety and orientate team members to a common workflow. Postoperatively, neural networks leveraging multimodal datasets will allow early detection of individuals at risk of complications and assist in the prediction of treatment failure, thus supporting patient-specific discharge and monitoring protocols. While these advancements in pituitary surgery hold promise to enhance the quality of care, clinicians must be the gatekeepers of the translation of such technologies, ensuring systematic assessment of risk and benefit prior to clinical implementation. In doing so, the synergy between these innovations can be leveraged to drive improved outcomes for patients of the future.

Streams, rivers and data lakes: an introduction to understanding modern electronic healthcare records

As foundation doctors, we have often found ourselves informing patients that a certain aspect of their medical information cannot be immediately found, either because it is on an electronic system we cannot access, or it is in a hospital that is unlinked to our own. Unsurprisingly, this frequently leaves patients flabbergasted and confused. We started to wonder: if patients' data are entered onto an electronic system: where do those data go? If medical data are searched for, where do those data come from? Why are there so many hidden sources of information that clinicians cannot access? In an ever-increasing digital sphere, electronic data will be the future of holistic health and social care planning, impacting every clinician's day-to-day role. From electronic healthcare records to the use of artificial intelligence solutions, this article will serve as an introduction to how data flows in modern healthcare systems.

Developing a linked electronic health record derived data platform to support research into healthy ageing

Introduction

Digitalisation of Electronic Health Record (EHR) data has created unique opportunities for research. However, these data are routinely collected for operational purposes and so are not curated to the standard required for research. Harnessing such routine data at large scale allows efficient and long-term epidemiological and health services research.

Objectives

To describe the establishment a linked EHR derived data platform in the National Centre for Healthy Ageing, Melbourne, Australia, aimed at enabling research targeting national health priority areas in ageing.

Methods

Our approach incorporated: data validation, curation and warehousing to ensure quality and completeness; end-user engagement and consensus on the platform content; implementation of an artificial intelligence (AI) pipeline for extraction of text-based data items; early consumer involvement; and implementation of routine collection of patient reported outcome measures, in a multisite public health service.

Results

Data for a cohort of >800,000 patients collected over a 10-year period have been curated within the platform's research data warehouse. So far 117 items have been identified as suitable for inclusion, from 11 research relevant datasets held within the health service EHR systems. Data access, extraction and release processes, guided by the Five Safes Framework, are being tested through project use-cases. A natural language processing (NLP) pipeline has been implemented and a framework for the routine collection and incorporation of patient reported outcome measures developed.

Conclusions

We highlight the importance of establishing comprehensive processes for the foundations of a data platform utilising routine data not collected for research purposes. These robust foundations will facilitate future expansion through linkages to other datasets for the efficient and cost-effective study of health related to ageing at a large scale.

Hospital-wide natural language processing summarising the health data of 1 million patients

Electronic health records (EHRs) represent a major repository of real world clinical trajectories, interventions and outcomes. While modern enterprise EHR's try to capture data in structured standardised formats, a significant bulk of the available information captured in the EHR is still recorded only in unstructured text format and can only be transformed into structured codes by manual processes. Recently, Natural Language Processing (NLP) algorithms have reached a level of performance suitable for large scale and accurate information extraction from clinical text. Here we describe the application of open-source named-entity-recognition and linkage (NER+L) methods (CogStack, MedCAT) to the entire text content of a large UK hospital trust (King's College Hospital, London). The resulting dataset contains 157M SNOMED concepts generated from 9.5M documents for 1.07M patients over a period of 9 years. We present a summary of prevalence and disease onset as well as a patient embedding that captures major comorbidity patterns at scale. NLP has the potential to transform the health data lifecycle, through large-scale automation of a traditionally manual task.

Discharge summary hospital course summarisation of in patient Electronic Health Record text with clinical concept guided deep pre-trained Transformer models

Brief Hospital Course (BHC) summaries are succinct summaries of an entire hospital encounter, embedded within discharge summaries, written by senior clinicians responsible for the overall care of a patient. Methods to automatically produce summaries from inpatient documentation would be invaluable in reducing clinician manual burden of summarising documents under high time-pressure to admit and discharge patients. Automatically producing these summaries from the inpatient course, is a complex, multi-document summarisation task, as source notes are written from various perspectives (e.g. nursing, doctor, radiology), during the course of the hospitalisation. We demonstrate a range of methods for BHC summarisation demonstrating the performance of deep learning summarisation models across extractive and abstractive summarisation scenarios. We also test a novel ensemble extractive and abstractive summarisation model that incorporates a medical concept ontology (SNOMED) as a clinical guidance signal and shows superior performance in 2 real-world clinical data sets.

Diagnostic signature for heart failure with preserved ejection fraction (HFpEF): a machine learning approach using multi-modality electronic health record data

BACKGROUND

Heart failure with preserved ejection fraction (HFpEF) is thought to be highly prevalent yet remains underdiagnosed. Evidence-based treatments are available that increase quality of life and decrease hospitalization. We sought to develop a data-driven diagnostic model to predict from electronic health records (EHR) the likelihood of HFpEF among patients with unexplained dyspnea and preserved left ventricular EF.

METHODS AND RESULTS

The derivation cohort comprised patients with dyspnea and echocardiography results. Structured and unstructured data were extracted using an automated informatics pipeline. Patients were retrospectively diagnosed as HFpEF (cases), non-HF (control cohort I), or HF with reduced EF (HFrEF; control cohort II). The ability of clinical parameters and investigations to discriminate cases from controls was evaluated by extreme gradient boosting. A likelihood scoring system was developed and validated in a separate test cohort. The derivation cohort included 1585 consecutive patients: 133 cases of HFpEF (9%), 194 non-HF cases (Control cohort I) and 1258 HFrEF cases (Control cohort II). Two HFpEF diagnostic signatures were derived, comprising symptoms, diagnoses and investigation results. A final prediction model was generated based on the averaged likelihood scores from these two models. In a validation cohort consisting of 269 consecutive patients [with 66 HFpEF cases (24.5%)], the diagnostic power of detecting HFpEF had an AUROC of 90% (P < 0.001) and average precision of 74%.

CONCLUSION

This diagnostic signature enables discrimination of HFpEF from non-cardiac dyspnea or HFrEF from EHR and can assist in the diagnostic evaluation in patients with unexplained dyspnea. This approach will enable identification of HFpEF patients who may then benefit from new evidence-based therapies.

A survey on clinical natural language processing in the United Kingdom from 2007 to 2022

Much of the knowledge and information needed for enabling high-quality clinical research is stored in free-text format. Natural language processing (NLP) has been used to extract information from these sources at scale for several decades. This paper aims to present a comprehensive review of clinical NLP for the past 15 years in the UK to identify the community, depict its evolution, analyse methodologies and applications, and identify the main barriers. We collect a dataset of clinical NLP projects (n = 94; £ = 41.97 m) funded by UK funders or the European Union's funding programmes. Additionally, we extract details on 9 funders, 137 organisations, 139 persons and 431 research papers. Networks are created from timestamped data interlinking all entities, and network analysis is subsequently applied to generate insights. 431 publications are identified as part of a literature review, of which 107 are eligible for final analysis. Results show, not surprisingly, clinical NLP in the UK has increased substantially in the last 15 years: the total budget in the period of 2019-2022 was 80 times that of 2007-2010. However, the effort is required to deepen areas such as disease (sub-)phenotyping and broaden application domains. There is also a need to improve links between academia and industry and enable deployments in real-world settings for the realisation of clinical NLP's great potential in care delivery. The major barriers include research and development access to hospital data, lack of capable computational resources in the right places, the scarcity of labelled data and barriers to sharing of pretrained models.

Natural Language Processing in Spine Surgery: A Systematic Review of Applications, Bias, and Reporting Transparency

BACKGROUND

Natural language processing (NLP) is a discipline of machine learning concerned with the analysis of language and text. Although NLP has been applied to various forms of clinical text, the applications and utility of NLP in spine surgery remain poorly characterized. Here, we systematically reviewed studies that use NLP for spine surgery applications, and analyzed applications, bias, and reporting transparency of the studies.

METHODS

We performed a literature search using the PubMed, Scopus, and Embase databases. Data extraction was performed after appropriate screening. The risk of bias and reporting quality were assessed using the PROBAST and TRIPOD tools.

RESULTS

A total of 12 full-text articles were included. The most common diseases represented include spondylolisthesis (25%), scoliosis (17%), and lumbar disk herniation (17%). The most common procedures included spinal fusion (42%), imaging (e.g. magnetic resonance, X-ray) (25%), and scoliosis correction (17%). Reported outcomes were diverse and included incidental durotomy, venous thromboembolism, and the tone of social media posts regarding scoliosis surgery. Common sources of bias identified included the use of older methods that do not capture the nuance of a text, and not using a prespecified or standard outcome measure when evaluating NLP methods.

CONCLUSIONS

Although the application of NLP to spine surgery is expanding, current studies face limitations and none are indicated as ready for clinical use. Thus, for future studies we recommend an emphasis on transparent reporting and collaboration with NLP experts to incorporate the latest developments to improve models and contribute to further innovation.

Prognostic relevance of demographic factors in cardiac magnetic resonance-proven acute myocarditis: A cohort study

Aim

Acute myocarditis (AM) is a heterogeneous condition with variable estimates of survival. Contemporary criteria for the diagnosis of clinically suspected AM enable non-invasive assessment, resulting in greater sensitivity and more representative cohorts. We aimed to describe the demographic characteristics and long-term outcomes of patients with AM diagnosed using non-invasive criteria.

Methods and results

A total of 199 patients with cardiac magnetic resonance (CMR)-confirmed AM were included. The majority (n = 130, 65%) were male, and the average age was 39 ± 16 years. Half of the patients were White (n = 99, 52%), with the remainder from Black and Minority Ethnic (BAME) groups. The most common clinical presentation was chest pain (n = 156, 78%), with smaller numbers presenting with breathlessness (n = 25, 13%) and arrhythmias (n = 18, 9%). Patients admitted with breathlessness were sicker and more often required inotropes, steroids, and renal replacement therapy (p < 0.001, p < 0.001, and p = 0.01, respectively). Over a median follow-up of 53 (IQR 34-76) months, 11 patients (6%) experienced an adverse outcome, defined as a composite of all-cause mortality, resuscitated cardiac arrest, and appropriate implantable cardioverter defibrillator (ICD) therapy. Patients in the arrhythmia group had a worse prognosis, with a nearly sevenfold risk of adverse events [hazard ratio (HR) 6.97; 95% confidence interval (CI) 1.87-26.00, p = 0.004]. Sex and ethnicity were not significantly associated with the outcome.

Conclusion

AM is highly heterogeneous with an overall favourable prognosis. Three-quarters of patients with AM present with chest pain, which is associated with a benign prognosis. AM presenting with life-threatening arrhythmias is associated with a higher risk of adverse events.

Deployment of a Free-Text Analytics Platform at a UK National Health Service Research Hospital: CogStack at University College London Hospitals

BACKGROUND

As more health care organizations transition to using electronic health record (EHR) systems, it is important for these organizations to maximize the secondary use of their data to support service improvement and clinical research. These organizations will find it challenging to have systems capable of harnessing the unstructured data fields in the record (clinical notes, letters, etc) and more practically have such systems interact with all of the hospital data systems (legacy and current).

OBJECTIVE

We describe the deployment of the EHR interfacing information extraction and retrieval platform CogStack at University College London Hospitals (UCLH).

METHODS

At UCLH, we have deployed the CogStack platform, an information retrieval platform with natural language processing capabilities. The platform addresses the problem of data ingestion and harmonization from multiple data sources using the Apache NiFi module for managing complex data flows. The platform also facilitates the extraction of structured data from free-text records through use of the MedCAT natural language processing library. Finally, data science tools are made available to support data scientists and the development of downstream applications dependent upon data ingested and analyzed by CogStack.

RESULTS

The platform has been deployed at the hospital, and in particular, it has facilitated a number of research and service evaluation projects. To date, we have processed over 30 million records, and the insights produced from CogStack have informed a number of clinical research use cases at the hospital.

CONCLUSIONS

The CogStack platform can be configured to handle the data ingestion and harmonization challenges faced by a hospital. More importantly, the platform enables the hospital to unlock important clinical information from the unstructured portion of the record using natural language processing technology.

A critical care outreach team under strain - Evaluation of the service provided to patients with haematological malignancy during the Covid-19 pandemic

Purpose

Critical Care Outreach Teams (CCOTs) have been associated with improved outcomes in patients with haematological malignancy (HM). This study aims to describe CCOT activation by patients with HM before and during the Covid-19 pandemic, assess amny association with worse outcomes, and examine the psychological impact on the CCOT.

Materials and methods

A retrospective, mixed-methods analysis was performed in HM patients reviewed by the CCOT over a two-year period, 01 July 2019 to 31 May 2021.

Results

The CCOT increased in size during the surge period and reviewed 238 HM patients, less than in the pre- and post-surge periods. ICU admission in the baseline, surge and the non-surge periods were 41.7%, 10.4% and 47.9% respectively. ICU mortality was 22.5%, 0% and 21.7% for the same times. Time to review was significantly decreased (p = 0.012). Semi-structured interviews revealed four themes of psychological distress: 1) time-critical work; 2) non-evidence based therapies; 3) feelings of guilt; 4) increased decision-making responsibility.

Conclusions

Despite the increase in total hospital referrals, the number of patients with HM that were reviewed during the surge periods decreased, as did their ICU admission rate and mortality. The quality of care provided was not impaired, as reflected by the number of patients receiving bedside reviews and the shorter-than-pre-pandemic response time.

A framework for de-identification of free-text data in electronic medical records enabling secondary use

Clinical free-text data represent a vast, untapped source of rich information. If more accessible for research it would supplement information captured in structured fields. Data need to be de-identified prior to being reused for research. However, a lack of transparency with existing de-identification software tools makes it difficult for data custodians to assess potential risks associated with the release of de-identified clinical free-text data. This case study describes the development of a framework for releasing de-identified clinical free-text data in two local health districts in NSW, Australia. A sample of clinical documents (n = 14 768 965), including progress notes, nursing and medical assessments and discharge summaries, were used for development. An algorithm was designed to identify and mask patient names without damaging data utility. For each note, the algorithm output the (i) note length before and after de-identification, (ii) the number of patient names and (iii) the number of common words. These outputs were used to iteratively refine the algorithm performance. This was followed by manual review of a random subset of records by a health information manager. Notes that were not correctly de-identified were fixed, and performance was reassessed until resolution. All notes in this sample were suitably de-identified using this method. Developing a transparent method for de-identifying clinical free-text data enables informed-decision making by data custodians and the safe re-use of clinical free-text data for research and public benefit.

An implementation framework and a feasibility evaluation of a clinical decision support system for diabetes management in secondary mental healthcare using CogStack

BACKGROUND

Improvements to the primary prevention of physical health illnesses like diabetes in the general population have not been mirrored to the same extent in people with serious mental illness (SMI). This work evaluates the technical feasibility of implementing an electronic clinical decision support system (eCDSS) for supporting the management of dysglycaemia and diabetes in patients with serious mental illness in a secondary mental healthcare setting.

METHODS

A stepwise approach was taken as an overarching and guiding framework for this work. Participatory methods were employed to design and deploy a monitoring and alerting eCDSS. The eCDSS was evaluated for its technical feasibility. The initial part of the feasibility evaluation was conducted in an outpatient community mental health team. Thereafter, the evaluation of the eCDSS progressed to a more in-depth in silico validation.

RESULTS

A digital health intervention that enables monitoring and alerting of at-risk patients based on an approved diabetes management guideline was developed. The eCDSS generated alerts according to expected standards and in line with clinical guideline recommendations.

CONCLUSIONS

It is feasible to design and deploy a functional monitoring and alerting eCDSS in secondary mental healthcare. Further work is required in order to fully evaluate the integration of the eCDSS into routine clinical workflows. By describing and sharing the steps that were and will be taken from concept to clinical testing, useful insights could be provided to teams that are interested in building similar digital health interventions.

A National Network of Safe Havens: Scottish Perspective

For over a decade, Scotland has implemented and operationalized a system of Safe Havens, which provides secure analytics platforms for researchers to access linked, deidentified electronic health records (EHRs) while managing the risk of unauthorized reidentification. In this paper, a perspective is provided on the state-of-the-art Scottish Safe Haven network, including its evolution, to define the key activities required to scale the Scottish Safe Haven network's capability to facilitate research and health care improvement initiatives. A set of processes related to EHR data and their delivery in Scotland have been discussed. An interview with each Safe Haven was conducted to understand their services in detail, as well as their commonalities. The results show how Safe Havens in Scotland have protected privacy while facilitating the reuse of the EHR data. This study provides a common definition of a Safe Haven and promotes a consistent understanding among the Scottish Safe Haven network and the clinical and academic research community. We conclude by identifying areas where efficiencies across the network can be made to meet the needs of population-level studies at scale.

Impact of direct-access IBD physician delivered endoscopy on clinical outcomes: a pre-implementation and post-implementation study

INTRODUCTION

Patients with suspected inflammatory bowel disease (IBD) referred from primary care often face diagnostic and treatment delays. This study aimed to compare a novel direct-access IBD endoscopy pathway with the traditional care model.

METHOD

Single centre real-world study analysing primary care referrals with suspected IBD. Group A: patients triaged to direct-access IBD endoscopy. Group B: patients undergoing traditional outpatient appointments before the availability of direct-access IBD endoscopy. Demographics, fecal calprotectin (FCP), C-reactive protein (CRP), disease activity score, endoscopy findings, treatment and follow-up were collected and statistically analysed. Ranked semantic analysis of IBD symptoms contained within referral letters was performed.

RESULTS

Referral letters did not differ significantly in Groups A and B. Demographic data, FCP and CRP values were similar. Referral to treatment time (RTT) at the time of IBD endoscopy was reduced from 177 days (Group B) to 24 days (Group A) (p<0.0001). Diagnostic yield of IBD was 35.6% (Group B) versus 62.0% (Group A) (p=0.0003). 89.2% of patients underwent colonoscopy in Group B versus 46.4% in Group A. DNA rates were similar in both groups. The direct to IBD endoscopy pathway saved 100% of initial IBD consultant clinics with a 2.5-fold increase in IBD nurse-led follow-up.

CONCLUSION

Our novel pathway resulted in an 86% reduction in RTT with associated increased diagnostic yield while saving 100% of initial IBD consultant outpatient appointments. Replication in other trusts may improve patient experience and accelerate time to diagnosis/treatment while optimising the use of healthcare resources.

Best practices in the real-world data life cycle

With increasing digitization of healthcare, real-world data (RWD) are available in greater quantity and scope than ever before. Since the 2016 United States 21st Century Cures Act, innovations in the RWD life cycle have taken tremendous strides forward, largely driven by demand for regulatory-grade real-world evidence from the biopharmaceutical sector. However, use cases for RWD continue to grow in number, moving beyond drug development, to population health and direct clinical applications pertinent to payors, providers, and health systems. Effective RWD utilization requires disparate data sources to be turned into high-quality datasets. To harness the potential of RWD for emerging use cases, providers and organizations must accelerate life cycle improvements that support this process. We build on examples obtained from the academic literature and author experience of data curation practices across a diverse range of sectors to describe a standardized RWD life cycle containing key steps in production of useful data for analysis and insights. We delineate best practices that will add value to current data pipelines. Seven themes are highlighted that ensure sustainability and scalability for RWD life cycles: data standards adherence, tailored quality assurance, data entry incentivization, deploying natural language processing, data platform solutions, RWD governance, and ensuring equity and representation in data.

Estimating redundancy in clinical text

The current mode of use of Electronic Health Records (EHR) elicits text redundancy. Clinicians often populate new documents by duplicating existing notes, then updating accordingly. Data duplication can lead to propagation of errors, inconsistencies and misreporting of care. Therefore, measures to quantify information redundancy play an essential role in evaluating innovations that operate on clinical narratives. This work is a quantitative examination of information redundancy in EHR notes. We present and evaluate two methods to measure redundancy: an information-theoretic approach and a lexicosyntactic and semantic model. Our first measure trains large Transformer-based language models using clinical text from a large openly available US-based ICU dataset and a large multi-site UK based Hospital. By comparing the information-theoretic efficient encoding of clinical text against open-domain corpora, we find that clinical text is ∼1.5× to ∼3× less efficient than open-domain corpora at conveying information. Our second measure, evaluates automated summarisation metrics Rouge and BERTScore to evaluate successive note pairs demonstrating lexicosyntactic and semantic redundancy, with averages from ∼43 to ∼65%.

Natural language word embeddings as a glimpse into healthcare language and associated mortality surrounding end of life

OBJECTIVES

To clarify real-world linguistic nuances around dying in hospital as well as inaccuracy in individual-level prognostication to support advance care planning and personalised discussions on limitation of life sustaining treatment (LST).

DESIGN

Retrospective cross-sectional study of real-world clinical data.

SETTING

Secondary care, urban and suburban teaching hospitals.

PARTICIPANTS

All inpatients in 12-month period from 1 October 2018 to 30 September 2019.

METHODS

Using unsupervised natural language processing, word embedding in latent space was used to generate phrase clusters with most similar semantic embeddings to 'Ceiling of Treatment' and their prognostication value.

RESULTS

Word embeddings with most similarity to 'Ceiling of Treatment' clustered around phrases describing end-of-life care, ceiling of care and LST discussions. The phrases have differing prognostic profile with the highest 7-day mortality in the phrases most explicitly referring to end of life-'Withdrawal of care' (56.7%), 'terminal care/end of life care' (57.5%) and 'un-survivable' (57.6%).

CONCLUSION

Vocabulary used at end-of-life discussions are diverse and has a range of associations to 7-day mortality. This highlights the importance of correct application of terminology during LST and end-of-life discussions.

Predicting fracture outcomes from clinical registry data using artificial intelligence supplemented models for evidence-informed treatment (PRAISE) study protocol

Background

Distal radius (wrist) fractures are the second most common fracture admitted to hospital. The anatomical pattern of these types of injuries is diverse, with variation in clinical management, guidelines for management remain inconclusive, and the uptake of findings from clinical trials into routine practice limited. Robust predictive modelling, which considers both the characteristics of the fracture and patient, provides the best opportunity to reduce variation in care and improve patient outcomes. This type of data is housed in unstructured data sources with no particular format or schema. The “Predicting fracture outcomes from clinical Registry data using Artificial Intelligence (AI) Supplemented models for Evidence-informed treatment (PRAISE)” study aims to use AI methods on unstructured data to describe the fracture characteristics and test if using this information improves identification of key fracture characteristics and prediction of patient-reported outcome measures and clinical outcomes following wrist fractures compared to prediction models based on standard registry data.

Methods and design

Adult (16+ years) patients presenting to the emergency department, treated in a short stay unit, or admitted to hospital for >24h for management of a wrist fracture in four Victorian hospitals will be included in this study. The study will use routine registry data from the Victorian Orthopaedic Trauma Outcomes Registry (VOTOR), and electronic medical record (EMR) information (e.g. X-rays, surgical reports, radiology reports, images). A multimodal deep learning fracture reasoning system (DLFRS) will be developed that reasons on EMR information. Machine learning prediction models will test the performance with/without output from the DLFRS.

Discussion

The PRAISE study will establish the use of AI techniques to provide enhanced information about fracture characteristics in people with wrist fractures. Prediction models using AI derived characteristics are expected to provide better prediction of clinical and patient-reported outcomes following distal radius fracture.

Countries' experiences in reforming hospital administration structure based on the Parker and Harding model: A systematic review study

In recent years, many reforms have been made on the structure of hospital administration, most of which are proposed by Parker-Harding models. Therefore, the purpose of this study is to systematically review global relevant experiences in reforming the hospital governance structure with emphasis on the Parker-Harding model. Required information was collected using keywords autonomization, corporatization, privatization, decentralization, reform, hospital autonomy, governance model, and structural reform in databases such as EMBASE, PubMed, Scopus, SID, MagIran, and other resources. Information on the subjects under study was collected from 1990 to 2020. The content extraction method was used for data extraction and data analysis. Thirty-nine sources were included in the study. Results of searching for relevant evidence on a variety of hospital governance models (government, board, corporate, and private) based on the Parker-Harding model in four categories including strengths (31), weaknesses (30), outcomes (26), and interventions (21) are outlined. In this study, strengths, weaknesses, outcomes, and corrective interventions were presented for different models of hospital administration that could be used by healthcare policymakers. Also, According to the results of this study, governmental model less recommended.

Regional performance variation in external validation of four prediction models for severity of COVID-19 at hospital admission: An observational multi-centre cohort study

BACKGROUND

Prediction models should be externally validated to assess their performance before implementation. Several prediction models for coronavirus disease-19 (COVID-19) have been published. This observational cohort study aimed to validate published models of severity for hospitalized patients with COVID-19 using clinical and laboratory predictors.

METHODS

Prediction models fitting relevant inclusion criteria were chosen for validation. The outcome was either mortality or a composite outcome of mortality and ICU admission (severe disease). 1295 patients admitted with symptoms of COVID-19 at Kings Cross Hospital (KCH) in London, United Kingdom, and 307 patients at Oslo University Hospital (OUH) in Oslo, Norway were included. The performance of the models was assessed in terms of discrimination and calibration.

RESULTS

We identified two models for prediction of mortality (referred to as Xie and Zhang1) and two models for prediction of severe disease (Allenbach and Zhang2). The performance of the models was variable. For prediction of mortality Xie had good discrimination at OUH with an area under the receiver-operating characteristic (AUROC) 0.87 [95% confidence interval (CI) 0.79-0.95] and acceptable discrimination at KCH, AUROC 0.79 [0.76-0.82]. In prediction of severe disease, Allenbach had acceptable discrimination (OUH AUROC 0.81 [0.74-0.88] and KCH AUROC 0.72 [0.68-0.75]). The Zhang models had moderate to poor discrimination. Initial calibration was poor for all models but improved with recalibration.

CONCLUSIONS

The performance of the four prediction models was variable. The Xie model had the best discrimination for mortality, while the Allenbach model had acceptable results for prediction of severe disease.

Pre-existing cardiovascular disease rather than cardiovascular risk factors drives mortality in COVID-19

BACKGROUND

The relative association between cardiovascular (CV) risk factors, such as diabetes and hypertension, established CV disease (CVD), and susceptibility to CV complications or mortality in COVID-19 remains unclear.

METHODS

We conducted a cohort study of consecutive adults hospitalised for severe COVID-19 between 1st March and 30th June 2020. Pre-existing CVD, CV risk factors and associations with mortality and CV complications were ascertained.

RESULTS

Among 1721 patients (median age 71 years, 57% male), 349 (20.3%) had pre-existing CVD (CVD), 888 (51.6%) had CV risk factors without CVD (RF-CVD), 484 (28.1%) had neither. Patients with CVD were older with a higher burden of non-CV comorbidities. During follow-up, 438 (25.5%) patients died: 37% with CVD, 25.7% with RF-CVD and 16.5% with neither. CVD was independently associated with in-hospital mortality among patients < 70 years of age (adjusted HR 2.43 [95% CI 1.16-5.07]), but not in those ≥ 70 years (aHR 1.14 [95% CI 0.77-1.69]). RF-CVD were not independently associated with mortality in either age group (< 70 y aHR 1.21 [95% CI 0.72-2.01], ≥ 70 y aHR 1.07 [95% CI 0.76-1.52]). Most CV complications occurred in patients with CVD (66%) versus RF-CVD (17%) or neither (11%; p < 0.001). 213 [12.4%] patients developed venous thromboembolism (VTE). CVD was not an independent predictor of VTE.

CONCLUSIONS

In patients hospitalised with COVID-19, pre-existing established CVD appears to be a more important contributor to mortality than CV risk factors in the absence of CVD. CVD-related hazard may be mediated, in part, by new CV complications. Optimal care and vigilance for destabilised CVD are essential in this patient group. Trial registration n/a.

Multi-domain clinical natural language processing with MedCAT: The Medical Concept Annotation Toolkit

Electronic health records (EHR) contain large volumes of unstructured text, requiring the application of information extraction (IE) technologies to enable clinical analysis. We present the open source Medical Concept Annotation Toolkit (MedCAT) that provides: (a) a novel self-supervised machine learning algorithm for extracting concepts using any concept vocabulary including UMLS/SNOMED-CT; (b) a feature-rich annotation interface for customizing and training IE models; and (c) integrations to the broader CogStack ecosystem for vendor-agnostic health system deployment. We show improved performance in extracting UMLS concepts from open datasets (F1:0.448-0.738 vs 0.429-0.650). Further real-world validation demonstrates SNOMED-CT extraction at 3 large London hospitals with self-supervised training over ∼8.8B words from ∼17M clinical records and further fine-tuning with ∼6K clinician annotated examples. We show strong transferability (F1 > 0.94) between hospitals, datasets and concept types indicating cross-domain EHR-agnostic utility for accelerated clinical and research use cases.

Developing automated methods for disease subtyping in UK Biobank: an exemplar study on stroke

BACKGROUND

Better phenotyping of routinely collected coded data would be useful for research and health improvement. For example, the precision of coded data for hemorrhagic stroke (intracerebral hemorrhage [ICH] and subarachnoid hemorrhage [SAH]) may be as poor as < 50%. This work aimed to investigate the feasibility and added value of automated methods applied to clinical radiology reports to improve stroke subtyping.

METHODS

From a sub-population of 17,249 Scottish UK Biobank participants, we ascertained those with an incident stroke code in hospital, death record or primary care administrative data by September 2015, and ≥ 1 clinical brain scan report. We used a combination of natural language processing and clinical knowledge inference on brain scan reports to assign a stroke subtype (ischemic vs ICH vs SAH) for each participant and assessed performance by precision and recall at entity and patient levels.

RESULTS

Of 225 participants with an incident stroke code, 207 had a relevant brain scan report and were included in this study. Entity level precision and recall ranged from 78 to 100%. Automated methods showed precision and recall at patient level that were very good for ICH (both 89%), good for SAH (both 82%), but, as expected, lower for ischemic stroke (73%, and 64%, respectively), suggesting coded data remains the preferred method for identifying the latter stroke subtype.

CONCLUSIONS

Our automated method applied to radiology reports provides a feasible, scalable and accurate solution to improve disease subtyping when used in conjunction with administrative coded health data. Future research should validate these findings in a different population setting.

Biological responses to COVID-19: Insights from physiological and blood biomarker profiles

BACKGROUND

Understanding the spectrum and course of biological responses to coronavirus disease 2019 (COVID-19) may have important therapeutic implications. We sought to characterise biological responses among patients hospitalised with severe COVID-19 based on serial, routinely collected, physiological and blood biomarker values.

METHODS AND FINDINGS

We performed a retrospective cohort study of 1335 patients hospitalised with laboratory-confirmed COVID-19 (median age 70 years, 56 % male), between 1st March and 30th April 2020. Latent profile analysis was performed on serial physiological and blood biomarkers. Patient characteristics, comorbidities and rates of death and admission to intensive care, were compared between the latent classes. A five class solution provided the best fit. Class 1 "Typical response" exhibited a moderately elevated and rising C-reactive protein (CRP), stable lymphopaenia, and the lowest rates of 14-day adverse outcomes. Class 2 "Rapid hyperinflammatory response" comprised older patients, with higher admission white cell and neutrophil counts, which declined over time, accompanied by a very high and rising CRP and platelet count, and exibited the highest mortality risk. Class 3 "Progressive inflammatory response" was similar to the typical response except for a higher and rising CRP, though similar mortality rate. Class 4 "Inflammatory response with kidney injury" had prominent lymphopaenia, moderately elevated (and rising) CRP, and severe renal failure. Class 5 "Hyperinflammatory response with kidney injury" comprised older patients, with a very high and rising CRP, and severe renal failure that attenuated over time. Physiological measures did not substantially vary between classes at baseline or early admission.

CONCLUSIONS AND RELEVANCE

Our identification of five distinct classes of biomarker profiles provides empirical evidence for heterogeneous biological responses to COVID-19. Early hyperinflammatory responses and kidney injury may signify unique pathophysiology that requires targeted therapy.

Business intelligence tools to optimize the appropriateness of the diagnostic process for clinical and epidemiologic purposes in a multicenter veterinary pathology service

Laboratory tests provide essential support to the veterinary practitioner, and their use has grown exponentially. This growth is the result of several factors, such as the eradication of historical diseases, the occurrence of multifactorial diseases, and the obligation to control endemic and epidemic diseases. However, the introduction of novel techniques is counterbalanced by economic constraints, and the establishment of evidence- and consensus-based guidelines is essential to support the pathologist. Therefore, we developed standardized protocols, categorized by species, type of production, age, and syndrome at the Istituto Zooprofilattico Sperimentale delle Venezie (IZSVe), a multicenter institution for animal health and food safety. We have 72 protocols in use for livestock, poultry, and pets, categorized as, for example, "bovine enteric calf", "rabbit respiratory", "broiler articular". Each protocol consists of a panel of tests, divided into 'mandatory' and 'ancillary', to be selected by the pathologist in order to reach the final diagnosis. After autopsy, the case is categorized into a specific syndrome, subsequently referred to as a syndrome-specific panel of analyses. The activity of the laboratories is monitored through a web-based dynamic reporting system developed using a business intelligence product (QlikView) connected to the laboratory information management system (IZILAB). On a daily basis, reports become available at general, laboratory, and case levels, and are updated as needed. The reporting system highlights epidemiologic variations in the field and allows verification of compliance with the protocols within the organization. The diagnostic protocols are revised annually to increase system efficiency and to address stakeholder requests.

A fast, accurate, and generalisable heuristic-based negation detection algorithm for clinical text

Negation detection is an important task in biomedical text mining. Particularly in clinical settings, it is of critical importance to determine whether findings mentioned in text are present or absent. Rule-based negation detection algorithms are a common approach to the task, and more recent investigations have resulted in the development of rule-based systems utilising the rich grammatical information afforded by typed dependency graphs. However, interacting with these complex representations inevitably necessitates complex rules, which are time-consuming to develop and do not generalise well. We hypothesise that a heuristic approach to determining negation via dependency graphs could offer a powerful alternative. We describe and implement an algorithm for negation detection based on grammatical distance from a negatory construct in a typed dependency graph. To evaluate the algorithm, we develop two testing corpora comprised of sentences of clinical text extracted from the MIMIC-III database and documents related to hypertrophic cardiomyopathy patients routinely collected at University Hospitals Birmingham NHS trust. Gold-standard validation datasets were built by a combination of human annotation and examination of algorithm error. Finally, we compare the performance of our approach with four other rule-based algorithms on both gold-standard corpora. The presented algorithm exhibits the best performance by f-measure over the MIMIC-III dataset, and a similar performance to the syntactic negation detection systems over the HCM dataset. It is also the fastest of the dependency-based negation systems explored in this study. Our results show that while a single heuristic approach to dependency-based negation detection is ignorant to certain advanced cases, it nevertheless forms a powerful and stable method, requiring minimal training and adaptation between datasets. As such, it could present a drop-in replacement or augmentation for many-rule negation approaches in clinical text-mining pipelines, particularly for cases where adaptation and rule development is not required or possible.

Real-time clinician text feeds from electronic health records

Analyses of search engine and social media feeds have been attempted for infectious disease outbreaks, but have been found to be susceptible to artefactual distortions from health scares or keyword spamming in social media or the public internet. We describe an approach using real-time aggregation of keywords and phrases of freetext from real-time clinician-generated documentation in electronic health records to produce a customisable real-time viral pneumonia signal providing up to 4 days warning for secondary care capacity planning. This low-cost approach is open-source, is locally customisable, is not dependent on any specific electronic health record system and can provide an ensemble of signals if deployed at multiple organisational scales.

Evaluation and improvement of the National Early Warning Score (NEWS2) for COVID-19: a multi-hospital study

BACKGROUND

The National Early Warning Score (NEWS2) is currently recommended in the UK for the risk stratification of COVID-19 patients, but little is known about its ability to detect severe cases. We aimed to evaluate NEWS2 for the prediction of severe COVID-19 outcome and identify and validate a set of blood and physiological parameters routinely collected at hospital admission to improve upon the use of NEWS2 alone for medium-term risk stratification.

METHODS

Training cohorts comprised 1276 patients admitted to King's College Hospital National Health Service (NHS) Foundation Trust with COVID-19 disease from 1 March to 30 April 2020. External validation cohorts included 6237 patients from five UK NHS Trusts (Guy's and St Thomas' Hospitals, University Hospitals Southampton, University Hospitals Bristol and Weston NHS Foundation Trust, University College London Hospitals, University Hospitals Birmingham), one hospital in Norway (Oslo University Hospital), and two hospitals in Wuhan, China (Wuhan Sixth Hospital and Taikang Tongji Hospital). The outcome was severe COVID-19 disease (transfer to intensive care unit (ICU) or death) at 14 days after hospital admission. Age, physiological measures, blood biomarkers, sex, ethnicity, and comorbidities (hypertension, diabetes, cardiovascular, respiratory and kidney diseases) measured at hospital admission were considered in the models.

RESULTS

A baseline model of 'NEWS2 + age' had poor-to-moderate discrimination for severe COVID-19 infection at 14 days (area under receiver operating characteristic curve (AUC) in training cohort = 0.700, 95% confidence interval (CI) 0.680, 0.722; Brier score = 0.192, 95% CI 0.186, 0.197). A supplemented model adding eight routinely collected blood and physiological parameters (supplemental oxygen flow rate, urea, age, oxygen saturation, C-reactive protein, estimated glomerular filtration rate, neutrophil count, neutrophil/lymphocyte ratio) improved discrimination (AUC = 0.735; 95% CI 0.715, 0.757), and these improvements were replicated across seven UK and non-UK sites. However, there was evidence of miscalibration with the model tending to underestimate risks in most sites.

CONCLUSIONS

NEWS2 score had poor-to-moderate discrimination for medium-term COVID-19 outcome which raises questions about its use as a screening tool at hospital admission. Risk stratification was improved by including readily available blood and physiological parameters measured at hospital admission, but there was evidence of miscalibration in external sites. This highlights the need for a better understanding of the use of early warning scores for COVID.

A case-control and cohort study to determine the relationship between ethnic background and severe COVID-19

BACKGROUND

People of minority ethnic backgrounds may be disproportionately affected by severe COVID-19. Whether this relates to increased infection risk, more severe disease progression, or worse in-hospital survival is unknown. The contribution of comorbidities or socioeconomic deprivation to ethnic patterning of outcomes is also unclear.

METHODS

We conducted a case-control and a cohort study in an inner city primary and secondary care setting to examine whether ethnic background affects the risk of hospital admission with severe COVID-19 and/or in-hospital mortality. Inner city adult residents admitted to hospital with confirmed COVID-19 (n = 872 cases) were compared with 3,488 matched controls randomly sampled from a primary healthcare database comprising 344,083 people residing in the same region. For the cohort study, we studied 1827 adults consecutively admitted with COVID-19. The primary exposure variable was self-defined ethnicity. Analyses were adjusted for socio-demographic and clinical variables.

FINDINGS

The 872 cases comprised 48.1% Black, 33.7% White, 12.6% Mixed/Other and 5.6% Asian patients. In conditional logistic regression analyses, Black and Mixed/Other ethnicity were associated with higher admission risk than white (OR 3.12 [95% CI 2.63-3.71] and 2.97 [2.30-3.85] respectively). Adjustment for comorbidities and deprivation modestly attenuated the association (OR 2.24 [1.83-2.74] for Black, 2.70 [2.03-3.59] for Mixed/Other). Asian ethnicity was not associated with higher admission risk (adjusted OR 1.01 [0.70-1.46]). In the cohort study of 1827 patients, 455 (28.9%) died over a median (IQR) of 8 (4-16) days. Age and male sex, but not Black (adjusted HR 1.06 [0.82-1.37]) or Mixed/Other ethnicity (adjusted HR 0.72 [0.47-1.10]), were associated with in-hospital mortality. Asian ethnicity was associated with higher in-hospital mortality but with a large confidence interval (adjusted HR 1.71 [1.15-2.56]).

INTERPRETATION

Black and Mixed ethnicity are independently associated with greater admission risk with COVID-19 and may be risk factors for development of severe disease, but do not affect in-hospital mortality risk. Comorbidities and socioeconomic factors only partly account for this and additional ethnicity-related factors may play a large role. The impact of COVID-19 may be different in Asians.

FUNDING

British Heart Foundation; the National Institute for Health Research; Health Data Research UK.

Natural Language Processing for Mimicking Clinical Trial Recruitment in Critical Care: A Semi-Automated Simulation Based on the LeoPARDS Trial

Clinical trials often fail to recruit an adequate number of appropriate patients. Identifying eligible trial participants is resource-intensive when relying on manual review of clinical notes, particularly in critical care settings where the time window is short. Automated review of electronic health records (EHR) may help, but much of the information is in free text rather than a computable form. We applied natural language processing (NLP) to free text EHR data using the CogStack platform to simulate recruitment into the LeoPARDS study, a clinical trial aiming to reduce organ dysfunction in septic shock. We applied an algorithm to identify eligible patients using a moving 1-hour time window, and compared patients identified by our approach with those actually screened and recruited for the trial, for the time period that data were available. We manually reviewed records of a random sample of patients identified by the algorithm but not screened in the original trial. Our method identified 376 patients, including 34 patients with EHR data available who were actually recruited to LeoPARDS in our centre. The sensitivity of CogStack for identifying patients screened was 90% (95% CI 85%, 93%). Of the 203 patients identified by both manual screening and CogStack, the index date matched in 95 (47%) and CogStack was earlier in 94 (47%). In conclusion, analysis of EHR data using NLP could effectively replicate recruitment in a critical care trial, and identify some eligible patients at an earlier stage, potentially improving trial recruitment if implemented in real time.

Toward the Development of Data Governance Standards for Using Clinical Free-Text Data in Health Research: Position Paper

BACKGROUND

Clinical free-text data (eg, outpatient letters or nursing notes) represent a vast, untapped source of rich information that, if more accessible for research, would clarify and supplement information coded in structured data fields. Data usually need to be deidentified or anonymized before they can be reused for research, but there is a lack of established guidelines to govern effective deidentification and use of free-text information and avoid damaging data utility as a by-product.

OBJECTIVE

This study aimed to develop recommendations for the creation of data governance standards to integrate with existing frameworks for personal data use, to enable free-text data to be used safely for research for patient and public benefit.

METHODS

We outlined data protection legislation and regulations relating to the United Kingdom for context and conducted a rapid literature review and UK-based case studies to explore data governance models used in working with free-text data. We also engaged with stakeholders, including text-mining researchers and the general public, to explore perceived barriers and solutions in working with clinical free-text.

RESULTS

We proposed a set of recommendations, including the need for authoritative guidance on data governance for the reuse of free-text data, to ensure public transparency in data flows and uses, to treat deidentified free-text data as potentially identifiable with use limited to accredited data safe havens, and to commit to a culture of continuous improvement to understand the relationships between the efficacy of deidentification and reidentification risks, so this can be communicated to all stakeholders.

CONCLUSIONS

By drawing together the findings of a combination of activities, we present a position paper to contribute to the development of data governance standards for the reuse of clinical free-text data for secondary purposes. While working in accordance with existing data governance frameworks, there is a need for further work to take forward the recommendations we have proposed, with commitment and investment, to assure and expand the safe reuse of clinical free-text data for public benefit.

Generating high-quality data abstractions from scanned clinical records: text-mining-assisted extraction of endometrial carcinoma pathology features as proof of principle

OBJECTIVE

Medical research studies often rely on the manual collection of data from scanned typewritten clinical records, which can be laborious, time consuming and error prone because of the need to review individual clinical records. We aimed to use text mining to assist with the extraction of clinical features from complex text-based scanned pathology records for medical research studies.

DESIGN

Text mining performance was measured by extracting and annotating three distinct pathological features from scanned photocopies of endometrial carcinoma clinical pathology reports, and comparing results to manually abstracted terms. Inclusion and exclusion keyword trigger terms to capture leiomyomas, endometriosis and adenomyosis were provided based on expert knowledge. Terms were expanded with character variations based on common optical character recognition (OCR) error patterns as well as negation phrases found in sample reports. The approach was evaluated on an unseen test set of 1293 scanned pathology reports originating from laboratories across Australia.

SETTING

Scanned typewritten pathology reports for women aged 18-79 years with newly diagnosed endometrial cancer (2005-2007) in Australia.

RESULTS

High concordance with final abstracted codes was observed for identifying the presence of three pathology features (94%-98% F-measure). The approach was more consistent and reliable than manual abstractions, identifying 3%-14% additional feature instances.

CONCLUSION

Keyword trigger-based automation with OCR error correction and negation handling proved not only to be rapid and convenient, but also providing consistent and reliable data abstractions from scanned clinical records. In conjunction with manual review, it can assist in the generation of high-quality data abstractions for medical research studies.

Multidisciplinary research priorities for the COVID-19 pandemic: a call for action for mental health science

The coronavirus disease 2019 (COVID-19) pandemic is having a profound effect on all aspects of society, including mental health and physical health. We explore the psychological, social, and neuroscientific effects of COVID-19 and set out the immediate priorities and longer-term strategies for mental health science research. These priorities were informed by surveys of the public and an expert panel convened by the UK Academy of Medical Sciences and the mental health research charity, MQ: Transforming Mental Health, in the first weeks of the pandemic in the UK in March, 2020. We urge UK research funding agencies to work with researchers, people with lived experience, and others to establish a high level coordination group to ensure that these research priorities are addressed, and to allow new ones to be identified over time. The need to maintain high-quality research standards is imperative. International collaboration and a global perspective will be beneficial. An immediate priority is collecting high-quality data on the mental health effects of the COVID-19 pandemic across the whole population and vulnerable groups, and on brain function, cognition, and mental health of patients with COVID-19. There is an urgent need for research to address how mental health consequences for vulnerable groups can be mitigated under pandemic conditions, and on the impact of repeated media consumption and health messaging around COVID-19. Discovery, evaluation, and refinement of mechanistically driven interventions to address the psychological, social, and neuroscientific aspects of the pandemic are required. Rising to this challenge will require integration across disciplines and sectors, and should be done together with people with lived experience. New funding will be required to meet these priorities, and it can be efficiently leveraged by the UK's world-leading infrastructure. This Position Paper provides a strategy that may be both adapted for, and integrated with, research efforts in other countries.

Angiotensin-converting enzyme inhibitors and angiotensin II receptor blockers are not associated with severe COVID-19 infection in a multi-site UK acute hospital trust

AIMS

The SARS-CoV-2 virus binds to the angiotensin-converting enzyme 2 (ACE2) receptor for cell entry. It has been suggested that angiotensin-converting enzyme inhibitors (ACEi) and angiotensin II receptor blockers (ARB), which are commonly used in patients with hypertension or diabetes and may raise tissue ACE2 levels, could increase the risk of severe COVID-19 infection.

METHODS AND RESULTS

We evaluated this hypothesis in a consecutive cohort of 1200 acute inpatients with COVID-19 at two hospitals with a multi-ethnic catchment population in London (UK). The mean age was 68 ± 17 years (57% male) and 74% of patients had at least one comorbidity. Overall, 415 patients (34.6%) reached the primary endpoint of death or transfer to a critical care unit for organ support within 21 days of symptom onset. A total of 399 patients (33.3%) were taking ACEi or ARB. Patients on ACEi/ARB were significantly older and had more comorbidities. The odds ratio for the primary endpoint in patients on ACEi and ARB, after adjustment for age, sex and co-morbidities, was 0.63 (95% confidence interval 0.47-0.84, P < 0.01).

CONCLUSIONS

There was no evidence for increased severity of COVID-19 in hospitalised patients on chronic treatment with ACEi or ARB. A trend towards a beneficial effect of ACEi/ARB requires further evaluation in larger meta-analyses and randomised clinical trials.

Should free-text data in electronic medical records be shared for research? A citizens' jury study in the UK

BACKGROUND

Use of routinely collected patient data for research and service planning is an explicit policy of the UK National Health Service and UK government. Much clinical information is recorded in free-text letters, reports and notes. These text data are generally lost to research, due to the increased privacy risk compared with structured data. We conducted a citizens' jury which asked members of the public whether their medical free-text data should be shared for research for public benefit, to inform an ethical policy.

METHODS

Eighteen citizens took part over 3 days. Jurors heard a range of expert presentations as well as arguments for and against sharing free text, and then questioned presenters and deliberated together. They answered a questionnaire on whether and how free text should be shared for research, gave reasons for and against sharing and suggestions for alleviating their concerns.

RESULTS

Jurors were in favour of sharing medical data and agreed this would benefit health research, but were more cautious about sharing free-text than structured data. They preferred processing of free text where a computer extracted information at scale. Their concerns were lack of transparency in uses of data, and privacy risks. They suggested keeping patients informed about uses of their data, and giving clear pathways to opt out of data sharing.

CONCLUSIONS

Informed citizens suggested a transparent culture of research for the public benefit, and continuous improvement of technology to protect patient privacy, to mitigate their concerns regarding privacy risks of using patient text data.

Multidisciplinary research priorities for the COVID-19 pandemic: a call for action for mental health science

The coronavirus disease 2019 (COVID-19) pandemic is having a profound effect on all aspects of society, including mental health and physical health. We explore the psychological, social, and neuroscientific effects of COVID-19 and set out the immediate priorities and longer-term strategies for mental health science research. These priorities were informed by surveys of the public and an expert panel convened by the UK Academy of Medical Sciences and the mental health research charity, MQ: Transforming Mental Health, in the first weeks of the pandemic in the UK in March, 2020. We urge UK research funding agencies to work with researchers, people with lived experience, and others to establish a high level coordination group to ensure that these research priorities are addressed, and to allow new ones to be identified over time. The need to maintain high-quality research standards is imperative. International collaboration and a global perspective will be beneficial. An immediate priority is collecting high-quality data on the mental health effects of the COVID-19 pandemic across the whole population and vulnerable groups, and on brain function, cognition, and mental health of patients with COVID-19. There is an urgent need for research to address how mental health consequences for vulnerable groups can be mitigated under pandemic conditions, and on the impact of repeated media consumption and health messaging around COVID-19. Discovery, evaluation, and refinement of mechanistically driven interventions to address the psychological, social, and neuroscientific aspects of the pandemic are required. Rising to this challenge will require integration across disciplines and sectors, and should be done together with people with lived experience. New funding will be required to meet these priorities, and it can be efficiently leveraged by the UK's world-leading infrastructure. This Position Paper provides a strategy that may be both adapted for, and integrated with, research efforts in other countries.

Implementation of a Real-Time Psychosis Risk Detection and Alerting System Based on Electronic Health Records using CogStack

Recent studies have shown that an automated, lifespan-inclusive, transdiagnostic, and clinically based, individualized risk calculator provides a powerful system for supporting the early detection of individuals at-risk of psychosis at a large scale, by leveraging electronic health records (EHRs). This risk calculator has been externally validated twice and is undergoing feasibility testing for clinical implementation. Integration of this risk calculator in clinical routine should be facilitated by prospective feasibility studies, which are required to address pragmatic challenges, such as missing data, and the usability of this risk calculator in a real-world and routine clinical setting. Here, we present an approach for a prospective implementation of a real-time psychosis risk detection and alerting service in a real-world EHR system. This method leverages the CogStack platform, which is an open-source, lightweight, and distributed information retrieval and text extraction system. The CogStack platform incorporates a set of services that allow for full-text search of clinical data, lifespan-inclusive, real-time calculation of psychosis risk, early risk-alerting to clinicians, and the visual monitoring of patients over time. Our method includes: 1) ingestion and synchronization of data from multiple sources into the CogStack platform, 2) implementation of a risk calculator, whose algorithm was previously developed and validated, for timely computation of a patient's risk of psychosis, 3) creation of interactive visualizations and dashboards to monitor patients' health status over time, and 4) building automated alerting systems to ensure that clinicians are notified of patients at-risk, so that appropriate actions can be pursued. This is the first ever study that has developed and implemented a similar detection and alerting system in clinical routine for early detection of psychosis.

A national initiative in data science for health: an evaluation of the UK Farr Institute

OBJECTIVE

To evaluate the extent to which the inter-institutional, inter-disciplinary mobilisation of data and skills in the Farr Institute contributed to establishing the emerging field of data science for health in the UK.

DESIGN AND OUTCOME MEASURES

We evaluated evidence of six domains characterising a new field of science:defining central scientific challenges,demonstrating how the central challenges might be solved,creating novel interactions among groups of scientists,training new types of experts,re-organising universities,demonstrating impacts in society.We carried out citation, network and time trend analyses of publications, and a narrative review of infrastructure, methods and tools.

SETTING

Four UK centres in London, North England, Scotland and Wales (23 university partners), 2013-2018.

RESULTS

1. The Farr Institute helped define a central scientific challenge publishing a research corpus, demonstrating insights from electronic health record (EHR) and administrative data at each stage of the translational cycle in 593 papers with at least one Farr Institute author affiliation on PubMed. 2. The Farr Institute offered some demonstrations of how these scientific challenges might be solved: it established the first four ISO27001 certified trusted research environments in the UK, and approved more than 1000 research users, published on 102 unique EHR and administrative data sources, although there was no clear evidence of an increase in novel, sustained record linkages. The Farr Institute established open platforms for the EHR phenotyping algorithms and validations (>70 diseases, CALIBER). Sample sizes showed some evidence of increase but remained less than 10% of the UK population in primary care-hospital care linked studies. 3.The Farr Institute created novel interactions among researchers: the co-author publication network expanded from 944 unique co-authors (based on 67 publications in the first 30 months) to 3839 unique co-authors (545 papers in the final 30 months). 4. Training expanded substantially with 3 new masters courses, training >400 people at masters, short-course and leadership level and 48 PhD students. 5. Universities reorganised with 4/5 Centres established 27 new faculty (tenured) positions, 3 new university institutes. 6. Emerging evidence of impacts included: > 3200 citations for the 10 most cited papers and Farr research informed eight practice-changing clinical guidelines and policies relevant to the health of millions of UK citizens.

CONCLUSION

The Farr Institute played a major role in establishing and growing the field of data science for health in the UK, with some initial evidence of benefits for health and healthcare. The Farr Institute has now expanded into Health Data Research (HDR) UK but key challenges remain including, how to network such activities internationally.