Assessing the Impact of Imputation on the Interpretations of Prediction Models: A Case Study on Mortality Prediction for Patients with Acute Myocardial Infarction

Acute myocardial infarction poses significant health risks and financial burden on healthcare and families. Prediction of mortality risk among AM! patients using rich electronic health record (EHR) data can potentially save lives and healthcare costs. Nevertheless, EHR-based prediction models usually use a missing data imputation method without considering its impact on the performance and interpretability of the model, hampering its real-world applicability in the healthcare setting. This study examines the impact of different methods for imputing missing values in EHR data on both the performance and the interpretations of predictive models. Our results showed that a small standard deviation in root mean squared error across different runs of an imputation method does not necessarily imply a small standard deviation in the prediction models' performance and interpretation. We also showed that the level of missingness and the imputation method used can have a significant impact on the interpretation of the models.

Exploring the feasibility of using real-world data from a large clinical data research network to simulate clinical trials of Alzheimer's disease

In this study, we explored the feasibility of using real-world data (RWD) from a large clinical research network to simulate real-world clinical trials of Alzheimer’s disease (AD). The target trial (i.e., NCT00478205) is a Phase III double-blind, parallel-group trial that compared the 23 mg donepezil sustained release with the 10 mg donepezil immediate release formulation in patients with moderate to severe AD. We followed the target trial’s study protocol to identify the study population, treatment regimen assignments and outcome assessments, and to set up a number of different simulation scenarios and parameters. We considered two main scenarios: (1) a one-arm simulation: simulating a standard-of-care (SOC) arm that can serve as an external control arm; and (2) a two-arm simulation: simulating both intervention and control arms with proper patient matching algorithms for comparative effectiveness analysis. In the two-arm simulation scenario, we used propensity score matching controlling for baseline characteristics to simulate the randomization process. In the two-arm simulation, higher serious adverse event (SAE) rates were observed in the simulated trials than the rates reported in original trial, and a higher SAE rate was observed in the 23 mg arm than in the 10 mg SOC arm. In the one-arm simulation scenario, similar estimates of SAE rates were observed when proportional sampling was used to control demographic variables. In conclusion, trial simulation using RWD is feasible in this example of AD trial in terms of safety evaluation. Trial simulation using RWD could be a valuable tool for post-market comparative effectiveness studies and for informing future trials’ design. Nevertheless, such an approach may be limited, for example, by the availability of RWD that matches the target trials of interest, and further investigations are warranted.

Directional and Fast Photoluminescence from CsPbI3 Nanocrystals Coupled to Dielectric Circular Bragg Gratings

Lead halide perovskite nanocrystals (NCs), especially the all-inorganic perovskite NCs, have drawn substantial attention for both fundamental research and device applications in recent years due to their unique optoelectronic properties. To build high-performance nanophotonic devices based on perovskite NCs, it is highly desirable to couple the NCs to photonic nanostructures for enhancing the radiative emission rate and improving the emission directionality of the NCs. In this work, we synthesized high-quality CsPbI₃ NCs and further coupled them to dielectric circular Bragg gratings (CBGs). The efficient couplings between the perovskite NCs and the CBGs resulted in a 45.9-fold enhancement of the photoluminescence (PL) intensity and 3.2-fold acceleration of the radiative emission rate. Our work serves as an important step for building high-performance nanophotonic light emitting devices by integrating perovskite NCs with photonic nanostructures.

How the clinical research community responded to the COVID-19 pandemic: an analysis of the COVID-19 clinical studies in ClinicalTrials.gov

OBJECTIVE

In the past few months, a large number of clinical studies on the novel coronavirus disease (COVID-19) have been initiated worldwide to find effective therapeutics, vaccines, and preventive strategies for COVID-19. In this study, we aim to understand the landscape of COVID-19 clinical research and identify the issues that may cause recruitment difficulty or reduce study generalizability.

METHODS

We analyzed 3765 COVID-19 studies registered in the largest public registry-ClinicalTrials.gov, leveraging natural language processing (NLP) and using descriptive, association, and clustering analyses. We first characterized COVID-19 studies by study features such as phase and tested intervention. We then took a deep dive and analyzed their eligibility criteria to understand whether these studies: (1) considered the reported underlying health conditions that may lead to severe illnesses, and (2) excluded older adults, either explicitly or implicitly, which may reduce the generalizability of these studies to the older adults population.

RESULTS

Our analysis included 2295 interventional studies and 1470 observational studies. Most trials did not explicitly exclude older adults with common chronic conditions. However, known risk factors such as diabetes and hypertension were considered by less than 5% of trials based on their trial description. Pregnant women were excluded by 34.9% of the studies.

CONCLUSIONS

Most COVID-19 clinical studies included both genders and older adults. However, risk factors such as diabetes, hypertension, and pregnancy were under-represented, likely skewing the population that was sampled. A careful examination of existing COVID-19 studies can inform future COVID-19 trial design towards balanced internal validity and generalizability.

Indoor heat exposure in Baltimore: does outdoor temperature matter?

Heat exposure of a population is often estimated by applying temperatures from outdoor monitoring stations. However, this can lead to exposure misclassification if residents do not live close to the monitoring station and temperature varies over small spatial scales due to land use/built environment variability, or if residents generally spend more time indoors than outdoors. Here, we compare summertime temperatures measured inside 145 homes in low-income households in Baltimore city with temperatures from the National Weather Service weather station in Baltimore. There is a large variation in indoor temperatures, with daily-mean indoor temperatures varying from 10 °C lower to 10 °C higher than outdoor temperatures. Furthermore, there is only a weak association between the indoor and outdoor temperatures across all houses, indicating that the outdoor temperature is not a good predictor of the indoor temperature for the residences sampled. It is shown that much of the variation is due to differences in the availability of air conditioning (AC). Houses with central AC are generally cooler than outdoors (median difference of - 3.4 °C) while those with no AC are generally warmer (median difference of 1.4 °C). For the collection of houses with central or room AC, there is essentially no relationship between indoor and outdoor temperatures, but for the subset of houses with no AC, there is a weak relationship (correlation coefficient of 0.36). The results presented here suggest future epidemiological studies of indoor exposure to heat would benefit from information on the availability of AC within the population.

Attention Mechanism with BERT for Content Annotation and Categorization of Pregnancy-Related Questions on a Community Q&A Site

In recent years, the social web has been increasingly used for health information seeking, sharing, and subsequent health-related research. Women often use the Internet or social networking sites to seek information related to pregnancy in different stages. They may ask questions about birth control, trying to conceive, labor, or taking care of a newborn or baby. Classifying different types of questions about pregnancy information (e.g., before, during, and after pregnancy) can inform the design of social media and professional websites for pregnancy education and support. This research aims to investigate the attention mechanism built-in or added on top of the BERT model in classifying and annotating the pregnancy-related questions posted on a community Q&A site. We evaluated two BERT-based models and compared them against the traditional machine learning models for question classification. Most importantly, we investigated two attention mechanisms: the built-in self-attention mechanism of BERT and the additional attention layer on top of BERT for relevant term annotation. The classification performance showed that the BERT-based models worked better than the traditional models, and BERT with an additional attention layer can achieve higher overall precision than the basic BERT model. The results also showed that both attention mechanisms work differently on annotating relevant content, and they could serve as feature selection methods for text mining in general.

Pregnancy-Related Information Seeking in Online Health Communities: A Qualitative Study

Pregnancy often imposes risks on women's health. Consumers are increasingly turning to online resources (e.g., online health communities) to look for pregnancy-related information for better care management. To inform design opportunities for online support interventions, it is critical to thoroughly understand consumers' information needs throughout the entire course of pregnancy including three main stages: pre-pregnancy, during-pregnancy, and postpartum. In this study, we present a content analysis of pregnancy-related question posts on Yahoo! Answers to examine how they formulated their inquiries, and the types of replies that information seekers received. This analysis revealed 14 main types of information needs, most of which were "stage-based". We also found that peers from online health communities provided a variety of support, including affirmation of pregnancy, opinions or suggestions, health information, personal experience, and reference to health providers' service. Insights derived from the findings are drawn to discuss design opportunities for tailoring informatics interventions to support consumers' information needs at different pregnancy stages.

Absence of heterozygosity detected by single-nucleotide polymorphism array in prenatal diagnosis

OBJECTIVES

To investigate the general occurrence and clinical significance of absence of heterozygosity (AOH), detected by single-nucleotide polymorphism (SNP) array on prenatal diagnosis.

METHODS

We recruited pregnancies undergoing invasive prenatal diagnosis at our fetal medicine center over a 6-year period. All fetuses underwent SNP array using the Affymetrix CytoScan HD array platform. AOH was defined as a chromosomal homozygosity segment with neutral copy number. Cases with AOH over 10 Mb in size or with suspected pathogenicity were further analyzed, and the clinical features and outcome were reviewed.

RESULTS

Of 10 294 recruited fetuses, 100 (0.97%) with AOH were identified; in 81 (81.0%) of these, AOH occurred in a single chromosome, while 19 (19.0%) patients had multiple AOHs in different chromosomes. AOH was observed in all chromosomes, chromosomes X, 2 and 16 being the most frequently involved. The length of AOH ranged from partial chromosome (9.002-80.222 Mb) to the entire chromosome. Similar AOH regions displayed varied clinical manifestations. In total, 55 patients presented with concomitant ultrasound abnormalities, the most common being multiple abnormalities (14/55 (25.5%)), genitourinary malformations (8/55 (14.5%)), skeletal malformations (5/55 (9.1%)) and small-for-gestational age (5/55 (9.1%)). Notably, the rate of adverse perinatal outcome (including termination of pregnancy, neonatal death, fetal death, selective reduction and miscarriage) in fetuses with AOH and ultrasound abnormalities (30/48 (62.5%)) was higher than in those without ultrasound abnormalities (6/40 (15.0%)) (P < 0.001). Further non-invasive prenatal testing using cell-free fetal DNA from maternal blood indicated chromosomal copy number abnormalities in 11 patients; however, they were confirmed as AOH by SNP array of the amniotic fluid.

CONCLUSIONS

Genetic counseling regarding a prenatal diagnosis of AOH remains challenging. To evaluate comprehensively its significance, we propose a management strategy involving further serial ultrasound examinations, parental verification, whole-exome sequencing, placental study and effective follow-up. Copyright © 2019 ISUOG. Published by John Wiley & Sons Ltd.

Large few-layer hexagonal boron nitride flakes for nonlinear optics

Hexagonal boron nitride (hBN) is a layered dielectric material with a wide range of applications in optics and photonics. In this work, we demonstrate a fabrication method for few-layer hBN flakes with areas up to 5000µm². We show that hBN in this form can be integrated with photonic microstructures: as an example, we use a circular Bragg grating (CBG). The layer quality of the exfoliated hBN flake on and off a CBG is confirmed by Raman spectroscopy and second-harmonic generation (SHG) microscopy. We show that the SHG signal is uniform across the hBN sample outside the CBG and is amplified in the center of the CBG.

Retrieving Lab Test Related Questions from Social Q&A Sites by Combining Shallow Features and Deep Representations

Patients face challenges in accurately interpreting their lab test results. To fulfill their knowledge gap, patients often turn to online resources, such as Community Question-Answering (CQA) sites, to seek meaningful information and support from their peers. Retrieving the most relevant information to patients' queries is important to help patients understand lab test results. However, few studies investigated the retrieval of lab test-related questions on CQA platforms. To address this research gap, we build and evaluate a system that automatically ranks questions about lab tests based on their similarity to a given question. The system is tested using diabetes-related questions collected from Yahoo! Answers' health section. Experimental results show that the regression-weighted combination of deep representations and shallow features was most effective in the Yahoo! Answers dataset. The proposed system can be extended to medical question retrieval, where questions contain a variety of lab tests.

Using Real-World Data to Rationalize Clinical Trials Eligibility Criteria Design: A Case Study of Alzheimer's Disease Trials

Low trial generalizability is a concern. The Food and Drug Administration had guidance on broadening trial eligibility criteria to enroll underrepresented populations. However, investigators are hesitant to do so because of concerns over patient safety. There is a lack of methods to rationalize criteria design. In this study, we used data from a large research network to assess how adjustments of eligibility criteria can jointly affect generalizability and patient safety (i.e the number of serious adverse events [SAEs]). We first built a model to predict the number of SAEs. Then, leveraging an a priori generalizability assessment algorithm, we assessed the changes in the number of predicted SAEs and the generalizability score, simulating the process of dropping exclusion criteria and increasing the upper limit of continuous eligibility criteria. We argued that broadening of eligibility criteria should balance between potential increases of SAEs and generalizability using donepezil trials as a case study.

Deep Learning Approach to Parse Eligibility Criteria in Dietary Supplements Clinical Trials Following OMOP Common Data Model

Dietary supplements (DSs) have been widely used in the U.S. and evaluated in clinical trials as potential interventions for various diseases. However, many clinical trials face challenges in recruiting enough eligible patients in a timely fashion, causing delays or even early termination. Using electronic health records to find eligible patients who meet clinical trial eligibility criteria has been shown as a promising way to assess recruitment feasibility and accelerate the recruitment process. In this study, we analyzed the eligibility criteria of 100 randomly selected DS clinical trials and identified both computable and non-computable criteria. We mapped annotated entities to OMOP Common Data Model (CDM) with novel entities (e.g., DS). We also evaluated a deep learning model (Bi-LSTM-CRF) for extracting these entities on CLAMP platform, with an average F1 measure of 0.601. This study shows the feasibility of automatic parsing of the eligibility criteria following OMOP CDM for future cohort identification.

Machine learning-based prediction of health outcomes in pediatric organ transplantation recipients

OBJECTIVES

Prediction of post-transplant health outcomes and identification of key factors remain important issues for pediatric transplant teams and researchers. Outcomes research has generally relied on general linear modeling or similar techniques offering limited predictive validity. Thus far, data-driven modeling and machine learning (ML) approaches have had limited application and success in pediatric transplant outcomes research. The purpose of the current study was to examine ML models predicting post-transplant hospitalization in a sample of pediatric kidney, liver, and heart transplant recipients from a large solid organ transplant program.

MATERIALS AND METHODS

Various logistic regression, naive Bayes, support vector machine, and deep learning (DL) methods were used to predict 1-, 3-, and 5-year post-transplant hospitalization using patient and administrative data from a large pediatric organ transplant center.

RESULTS

DL models generally outperformed traditional ML models across organtypes and prediction windows with area under the receiver operating characteristic curve values ranging from 0.750 to 0.851. Shapley additive explanations (SHAP) were used to increase the interpretability of DL model results. Various medical, patient, and social variables were identified as salient predictors across organ types.

DISCUSSION

Results demonstrate the utility of DL modeling for health outcome prediction with pediatric patients, and its use represents an important development in the prediction of post-transplant outcomes in pediatric transplantation compared to prior research.

CONCLUSION

Results point to DL models as potentially useful tools in decision-support systems assisting physicians and transplant teams in identifying patients at a greater risk for poor post-transplant outcomes.

An Introduction to the Adherence Promotion With Person-Centered Technology Project

The massive potential of cognitive training and longitudinal cognitive assessment to detect and prevent age-related cognitive decline and dementia will not be realized unless individuals are willing and able to engage with these protocols for an extended period of time. Unfortunately, similar to other health behaviors, adherence to home-based assessment and training is frequently poor. Addressing the gap between potential and realized benefits is an urgent goal as the population ages. APPT investigates these and related issues within samples of older adults with and without cognitive impairment. Ultimately, two randomized controlled trials will test whether an adaptive, tailored, and integrated technology-based adherence support system can boost adherence, with the ultimate goal being the early detection and treatment of age-related cognitive decline and dementia. Initial algorithm development and application to existing datasets will be presented that will inform the design of a smart reminder system that will later be assessed.

How the clinical research community responded to the COVID-19 pandemic: An analysis of the COVID-19 clinical studies in ClinicalTrials.gov

OBJECTIVE

The novel coronavirus disease (COVID-19), broke out in December 2019, and is now a global pandemic. In the past few months, a large number of clinical studies have been initiated worldwide to find effective therapeutics, vaccines, and preventive strategies for COVID-19. In this study, we aim to understand the landscape of COVID-19 clinical research and identify the gaps such as the lack of population representativeness and issues that may cause recruitment difficulty.

MATERIALS AND METHODS

We analyzed 3,765 COVID-19 studies registered in the largest public registry - ClinicalTrials.gov, leveraging natural language processing and using descriptive, association, and clustering analyses. We first characterized COVID-19 studies by study features such as phase and tested intervention. We then took a deep dive and analyzed their eligibility criteria to understand whether these studies: (1) considered the reported underlying health conditions that may lead to severe illnesses, and (2) excluded older adults, either explicitly or implicitly, which may reduce the generalizability of these studies to the older adults population.

RESULTS

Most trials did not have an upper age limit and did not exclude patients with common chronic conditions such as hypertension and diabetes that are more prevalent in older adults. However, known risk factors that may lead to severe illnesses have not been adequately considered.

CONCLUSIONS

A careful examination of existing COVID-19 studies can inform future COVID-19 trial design towards balanced internal validity and generalizability.

Selected articles from the Fourth International Workshop on Semantics-Powered Data Mining and Analytics (SEPDA 2019)

In this introduction, we first summarize the Fourth International Workshop on Semantics-Powered Data Mining and Analytics (SEPDA 2019) held on October 26, 2019 in conjunction with the 18th International Semantic Web Conference (ISWC 2019) in Auckland, New Zealand, and then briefly introduce seven research articles included in this supplement issue, covering the topics on Knowledge Graph, Ontology-Powered Analytics, and Deep Learning.

A deep learning approach for identifying cancer survivors living with post-traumatic stress disorder on Twitter

Background

Emotions after surviving cancer can be complicated. The survivors may have gained new strength to continue life, but some of them may begin to deal with complicated feelings and emotional stress due to trauma and fear of cancer recurrence. The widespread use of Twitter for socializing has been the alternative medium for data collection compared to traditional studies of mental health, which primarily depend on information taken from medical staff with their consent. These social media data, to a certain extent, reflect the users’ psychological state. However, Twitter also contains a mix of noisy and genuine tweets. The process of manually identifying genuine tweets is expensive and time-consuming.

Methods

We stream the data using cancer as a keyword to filter the tweets with cancer-free and use post-traumatic stress disorder (PTSD) related keywords to reduce the time spent on the annotation task. Convolutional Neural Network (CNN) learns the representations of the input to identify cancer survivors with PTSD.

Results

The results present that the proposed CNN can effectively identify cancer survivors with PTSD. The experiments on real-world datasets show that our model outperforms the baselines and correctly classifies the new tweets.

Conclusions

PTSD is one of the severe anxiety disorders that could affect individuals who are exposed to traumatic events, including cancer. Cancer survivors are at risk of short-term or long-term effects on physical and psycho-social well-being. Therefore, the evaluation and treatment of PTSD are essential parts of cancer survivorship care. It will act as an alarming system by detecting the PTSD presence based on users’ postings on Twitter.

Patient Challenges and Needs in Comprehending Laboratory Test Results: Mixed Methods Study

Background

Patients are increasingly able to access their laboratory test results via patient portals. However, merely providing access does not guarantee comprehension. Patients could experience confusion when reviewing their test results.

Objective

The aim of this study is to examine the challenges and needs of patients when comprehending laboratory test results.

Methods

We conducted a web-based survey with 203 participants and a set of semistructured interviews with 13 participants. We assessed patients’ perceived challenges and needs (both informational and technological needs) when they attempted to comprehend test results, factors associated with patients’ perceptions, and strategies for improving the design of patient portals to communicate laboratory test results more effectively. Descriptive and correlation analysis and thematic analysis were used to analyze the survey and interview data, respectively.

Results

Patients face a variety of challenges and confusion when reviewing laboratory test results. To better comprehend laboratory results, patients need different types of information, which are grouped into 2 categories—generic information (eg, reference range) and personalized or contextual information (eg, treatment options, prognosis, what to do or ask next). We also found that several intrinsic factors (eg, laboratory result normality, health literacy, and technology proficiency) significantly impact people’s perceptions of using portals to view and interpret laboratory results. The desired enhancements of patient portals include providing timely explanations and educational resources (eg, a health encyclopedia), increasing usability and accessibility, and incorporating artificial intelligence–based technology to provide personalized recommendations.

Conclusions

Patients face significant challenges in interpreting the meaning of laboratory test results. Designers and developers of patient portals should employ user-centered approaches to improve the design of patient portals to present information in a more meaningful way.