Continuous time recurrent neural networks: Overview and benchmarking at forecasting blood glucose in the intensive care unit

OBJECTIVE

Blood glucose measurements in the intensive care unit (ICU) are typically made at irregular intervals. This presents a challenge in choice of forecasting model. This article gives an overview of continuous time autoregressive recurrent neural networks (CTRNNs) and evaluates how they compare to autoregressive gradient boosted trees (GBT) in forecasting blood glucose in the ICU.

METHODS

Continuous time autoregressive recurrent neural networks (CTRNNs) are a deep learning model that account for irregular observations through incorporating continuous evolution of the hidden states between observations. This is achieved using a neural ordinary differential equation (ODE) or neural flow layer. In this manuscript, we give an overview of these models, including the varying architectures that have been proposed to account for issues such as ongoing medical interventions. Further, we demonstrate the application of these models to probabilistic forecasting of blood glucose in a critical care setting using electronic medical record and simulated data and compare with GBT and linear models.

RESULTS

The experiments confirm that addition of a neural ODE or neural flow layer generally improves the performance of autoregressive recurrent neural networks in the irregular measurement setting. However, several CTRNN architecture are outperformed by a GBT model (Catboost), with only a long short-term memory (LSTM) and neural ODE based architecture (ODE-LSTM) achieving comparable performance on probabilistic forecasting metrics such as the continuous ranked probability score (ODE-LSTM: 0.118 ± 0.001; Catboost: 0.118 ± 0.001), ignorance score (0.152 ± 0.008; 0.149 ± 0.002) and interval score (175 ± 1; 176 ± 1).

CONCLUSION

The application of deep learning methods for forecasting in situations with irregularly measured time series such as blood glucose shows promise. However, appropriate benchmarking by methods such as GBT approaches (plus feature transformation) are key in highlighting whether novel methodologies are truly state of the art in tabular data settings.

Pathling: analytics on FHIR

Background

Health data analytics is an area that is facing rapid change due to the acceleration of digitization of the health sector, and the changing landscape of health data and clinical terminology standards. Our research has identified a need for improved tooling to support analytics users in the task of analyzing Fast Healthcare Interoperability Resources (FHIR^®) data and associated clinical terminology.

Results

A server implementation was developed, featuring a FHIR API with new operations designed to support exploratory data analysis (EDA), advanced patient cohort selection and data preparation tasks. Integration with a FHIR Terminology Service is also supported, allowing users to incorporate knowledge from rich terminologies such as SNOMED CT within their queries. A prototype user interface for EDA was developed, along with visualizations in support of a health data analysis project.

Conclusions

Experience with applying this technology within research projects and towards the development of analytics-enabled applications provides a preliminary indication that the FHIR Analytics API pattern implemented by Pathling is a valuable abstraction for data scientists and software developers within the health care domain. Pathling contributes towards the value proposition for the use of FHIR within health data analytics, and assists with the use of complex clinical terminologies in that context.

Supplementary Information

The online version contains supplementary material available at 10.1186/s13326-022-00277-1.

Re-analysis of genomic data: An overview of the mechanisms and complexities of clinical adoption

Re-analyzing genomic information from a patient suspected of having an underlying genetic condition can improve the diagnostic yield of sequencing tests, potentially providing significant benefits to the patient and to the health care system. Although a significant number of studies have shown the clinical potential of re-analysis, less work has been performed to characterize the mechanisms responsible for driving the increases in diagnostic yield. Complexities surrounding re-analysis have also emerged. The terminology itself represents a challenge because "re-analysis" can refer to a range of different concepts. Other challenges include the increased workload that re-analysis demands of curators, adequate reimbursement pathways for clinical and diagnostic services, and the development of systems to handle large volumes of data. Re-analysis also raises ethical implications for patients and families, most notably when re-classification of a variant alters diagnosis, treatment, and prognosis. This review highlights the possibilities and complexities associated with the re-analysis of existing clinical genomic data. We propose a terminology that builds on the foundation presented in a recent statement from the American College of Medical Genetics and Genomics and describes each re-analysis process. We identify mechanisms for increasing diagnostic yield and provide perspectives on the range of challenges that must be addressed by health care systems and individual patients.

Semantic Search for Large Scale Clinical Ontologies

Finding concepts in large clinical ontologies can be challenging when queries use different vocabularies. A search algorithm that overcomes this problem is useful in applications such as concept normalisation and ontology matching, where concepts can be referred to in different ways, using different synonyms. In this paper, we present a deep learning based approach to build a semantic search system for large clinical ontologies. We propose a Triplet-BERT model and a method that generates training data directly from the ontologies. The model is evaluated using five real benchmark data sets and the results show that our approach achieves high results on both free text to concept and concept to concept searching tasks, and outperforms all baseline methods.

GA4GH: International policies and standards for data sharing across genomic research and healthcare

The Global Alliance for Genomics and Health (GA4GH) aims to accelerate biomedical advances by enabling the responsible sharing of clinical and genomic data through both harmonized data aggregation and federated approaches. The decreasing cost of genomic sequencing (along with other genome-wide molecular assays) and increasing evidence of its clinical utility will soon drive the generation of sequence data from tens of millions of humans, with increasing levels of diversity. In this perspective, we present the GA4GH strategies for addressing the major challenges of this data revolution. We describe the GA4GH organization, which is fueled by the development efforts of eight Work Streams and informed by the needs of 24 Driver Projects and other key stakeholders. We present the GA4GH suite of secure, interoperable technical standards and policy frameworks and review the current status of standards, their relevance to key domains of research and clinical care, and future plans of GA4GH. Broad international participation in building, adopting, and deploying GA4GH standards and frameworks will catalyze an unprecedented effort in data sharing that will be critical to advancing genomic medicine and ensuring that all populations can access its benefits.

Incorporating real-world evidence into the development of patient blood glucose prediction algorithms for the ICU

Objective

Glycemic control is an important component of critical care. We present a data-driven method for predicting intensive care unit (ICU) patient response to glycemic control protocols while accounting for patient heterogeneity and variations in care.

Materials and Methods

Using electronic medical records (EMRs) of 18 961 ICU admissions from the MIMIC-III dataset, including 318 574 blood glucose measurements, we train and validate a gradient boosted tree machine learning (ML) algorithm to forecast patient blood glucose and a 95% prediction interval at 2-hour intervals. The model uses as inputs irregular multivariate time series data relating to recent in-patient medical history and glycemic control, including previous blood glucose, nutrition, and insulin dosing.

Results

Our forecasting model using routinely collected EMRs achieves performance comparable to previous models developed in planned research studies using continuous blood glucose monitoring. Model error, expressed as mean absolute percentage error is 16.5%–16.8%, with Clarke error grid analysis demonstrating that 97% of predictions would be clinically acceptable. The 95% prediction intervals achieve near intended coverage at 93%–94%.

Discussion

ML algorithms built on observational data sources, such as EMRs, present a promising approach for personalization and automation of glycemic control in critical care. Future research may benefit from applying a combination of methodologies and data sources to develop robust methodologies that account for the variations seen in ICU patients and difficultly in detecting the extremes of observed blood glucose values.

Conclusion

We demonstrate that EMRs can be used to train ML algorithms that may be suitable for incorporation into ICU decision support systems.

Investigation of current models of care for genetic heart disease in Australia: A national clinical audit

BACKGROUND

This sub-study of the Australian Genomics Cardiovascular Genetic Disorders Flagship sought to conduct the first nation-wide audit in Australia to establish the current practices across cardiac genetics clinics.

METHOD

An audit of records of patients with a suspected genetic heart disease (cardiomyopathy, primary arrhythmia, autosomal dominant congenital heart disease) who had a cardiac genetics consultation between 1st January 2016 and 31 July 2018 and were offered a diagnostic genetic test.

RESULTS

This audit included 536 records at multidisciplinary cardiac genetics clinics from 11 public tertiary hospitals across five Australian states. Most genetic consultations occurred in a clinic setting (90%), followed by inpatient (6%) and Telehealth (4%). Queensland had the highest proportion of Telehealth consultations (9% of state total). Sixty-six percent of patients had a clinical diagnosis of a cardiomyopathy, 28% a primary arrhythmia, and 0.7% congenital heart disease. The reason for diagnosis was most commonly as a result of investigations of symptoms (73%). Most patients were referred by a cardiologist (85%), followed by a general practitioner (9%) and most genetic tests were funded by the state Genetic Health Service (73%). Nationally, 29% of genetic tests identified a pathogenic or likely pathogenic gene variant; 32% of cardiomyopathies, 26% of primary arrhythmia syndromes, and 25% of congenital heart disease.

CONCLUSION

We provide important information describing the current models of care for genetic heart diseases throughout Australia. These baseline data will inform the implementation and impact of whole genome sequencing in the Australian healthcare landscape.

Interoperable medical data: The missing link for understanding COVID-19

Being able to link clinical outcomes to SARS‐CoV‐2 virus strains is a critical component of understanding COVID‐19. Here, we discuss how current processes hamper sustainable data collection to enable meaningful analysis and insights. Following the ‘Fast Healthcare Interoperable Resource’ (FHIR) implementation guide, we introduce an ontology‐based standard questionnaire to overcome these shortcomings and describe patient 'journeys' in coordination with the World Health Organization's recommendations. We identify steps in the clinical health data acquisition cycle and workflows that likely have the biggest impact in the data‐driven understanding of this virus. Specifically, we recommend detailed symptoms and medical history using the FHIR standards. We have taken the first steps towards this by making patient status mandatory in GISAID (‘Global Initiative on Sharing All Influenza Data’), immediately resulting in a measurable increase in the fraction of cases with useful patient information. The main remaining limitation is the lack of controlled vocabulary or a medical ontology.

FHIR OWL: Transforming OWL ontologies into FHIR terminology resources

The FHIR specification provides a mechanism to access clinical terminologies using a standard API, and many existing terminologies, such as SNOMED CT, are well supported. However, in areas such as genomics, terminologies from other domains are starting to be used in clinical settings. Many of these are authored or distributed in Web Ontology Language (OWL) format. In this paper we describe a transformation between OWL ontologies and FHIR terminology resources. The results show that there are several challenges in implementing the transformation, with the major one being the lack of a modularisation mechanism in the FHIR code system resource that resembles the import mecha nism available in OWL. A workaround with minimal drawbacks was successfully implemented in this solution. The availability of this transformation is significant because it enables a broad range of terminologies that are currently available in OWL to be available using the FHIR API.

A SMART on FHIR Prototype for Genomic Test Ordering

Queensland Genomics recently undertook a number of Clinical Demonstration Projects (CDPs) to demonstrate the benefits of genomics in clinical practice. Integration of this testing requires the health system to provide the infrastructure for the appropriate ordering of these tests. Ordering of genomics tests will likely require greater exchange of information between the ordering clinician and the lab that is producing a clinical test report. The clinical demonstration projects were used to understand the information flow and the use of genomic, phenotypic and other information through the test ordering, analysis and reporting stages. This information was used to inform a set of requirements for a genomics test ordering and reporting system. A prototype of this system was developed as a SMART on FHIR application. This prototype will inform a future production system with FHIR Resources, software interfaces and interoperability requirements.

Can Wikipedia Be Used to Derive an Open Clinical Terminology?

Clinical terminologies play an essential role in enabling semantic interoperability between medical records. However, existing terminologies have several issues that impact data quality, such as content gaps and slow updates. In this study we explore the suitability of existing, community-driven resources, specifically Wikipedia, as a potential source to bootstrap an open clinical terminology, in terms of content coverage. In order to establish the extent of the coverage, a team of expert clinical terminologists manually mapped a clinically-relevant subset of SNOMED CT to Wikipedia articles. The results show that approximately 80% of the concepts are covered by Wikipedia. Most concepts that do not have a direct match in Wikipedia are composable from multiple articles. These findings are encouraging and suggest that it should be possible to bootstrap an open clinical terminology from Wikipedia.

FHIRCap: Transforming REDCap forms into FHIR resources

Clinical trials and studies are increasingly using systems, such as REDCap, to capture data in electronic form. However these tools are not designed to mimic the capture of clinical information for regular clinical care and lack support for sharing the data effectively. In this paper we describe the implementation of a transformation engine, FHIRCap, that allows defining rules to map REDCap forms into FHIR resources. To assess the feasibility of the system, a case study with one of the Australian Genomics clinical demonstration projects was done. The case study showed that the transformation language is flexible enough to handle most of the data being captured in REDCap for a typical clinical trial. A number of design issues in the forms were identified and a series of recommendations were provided to enable a more accurate transformation. These results show that it is possible to transform most data in existing REDCap projects to FHIR resources without having to modify the forms. This is significant because it demonstrates that most data in existing clinical trials and studies can be made available in a standardised manner.

Ontoserver: a syndicated terminology server

Background

Even though several high-quality clinical terminologies, such as SNOMED CT and LOINC, are readily available, uptake in clinical systems has been slow and many continue to capture information in plain text or using custom terminologies. This paper discusses some of the challenges behind this slow uptake and describes a clinical terminology server implementation that aims to overcome these obstacles and contribute to the widespread adoption of standardised clinical terminologies.

Results

Ontoserver is a clinical terminology server based on the Fast Health Interoperability Resources (FHIR) standard. Some of its key features include: out-of-the-box support for SNOMED CT, LOINC and OWL ontologies, such as the Human Phenotype Ontology (HPO); a fast, prefix-based search algorithm to ensure users can easily find content and are not discouraged from entering coded data; a syndication mechanism to facilitate keeping terminologies up to date; and a full implementation of SNOMED CT’s Expression Constraint Language (ECL), which enables sophisticated data analytics.

Conclusions

Ontoserver has been designed to overcome some of the challenges that have hindered adoption of standardised clinical terminologies and is used in several organisations throughout Australia. Increasing adoption is an important goal because it will help improve the quality of clinical data, which can lead to better clinical decision support and ultimately to better patient outcomes.

Towards achieving semantic interoperability of clinical study data with FHIR

BACKGROUND

Observational clinical studies play a pivotal role in advancing medical knowledge and patient healthcare. To lessen the prohibitive costs of conducting these studies and support evidence-based medicine, results emanating from these studies need to be shared and compared to one another. Current approaches for clinical study management have limitations that prohibit the effective sharing of clinical research data.

METHODS

The objective of this paper is to present a proposal for a clinical study architecture to not only facilitate the communication of clinical study data but also its context so that the data that is being communicated can be unambiguously understood at the receiving end. Our approach is two-fold. First we outline our methodology to map clinical data from Clinical Data Interchange Standards Consortium Operational Data Model (ODM) to the Fast Healthcare Interoperable Resource (FHIR) and outline the strengths and weaknesses of this approach. Next, we propose two FHIR-based models, to capture the metadata and data from the clinical study, that not only facilitate the syntactic but also semantic interoperability of clinical study data.

CONCLUSIONS

This work shows that our proposed FHIR resources provide a good fit to semantically enrich the ODM data. By exploiting the rich information model in FHIR, we can organise clinical data in a manner that preserves its organisation but captures its context. Our implementations demonstrate that FHIR can natively manage clinical data. Furthermore, by providing links at several levels, it improves the traversal and querying of the data. The intended benefits of this approach is more efficient and effective data exchange that ultimately will allow clinicians to switch their focus back to decision-making and evidence-based medicines.