A data-driven approach for quality assessment of radiologic interpretations

A model of integrating convolution and BiGRU dual-channel mechanism for Chinese medical text classifications

Recently, a lot of Chinese patients consult treatment plans through social networking platforms, but the Chinese medical text contains rich information, including a large number of medical nomenclatures and symptom descriptions. How to build an intelligence model to automatically classify the text information consulted by patients and recommend the correct department for patients is very important. In order to address the problem of insufficient feature extraction from Chinese medical text and low accuracy, this paper proposes a dual channel Chinese medical text classification model. The model extracts feature of Chinese medical text at different granularity, comprehensively and accurately obtains effective feature information, and finally recommends departments for patients according to text classification. One channel of the model focuses on medical nomenclatures, symptoms and other words related to hospital departments, gives different weights, calculates corresponding feature vectors with convolution kernels of different sizes, and then obtains local text representation. The other channel uses the BiGRU network and attention mechanism to obtain text representation, highlighting the important information of the whole sentence, that is, global text representation. Finally, the model uses full connection layer to combine the representation vectors of the two channels, and uses Softmax classifier for classification. The experimental results show that the accuracy, recall and F1-score of the model are improved by 10.65%, 8.94% and 11.62% respectively compared with the baseline models in average, which proves that our model has better performance and robustness.

Structured reporting of computed tomography in the staging of colon cancer: a Delphi consensus proposal

BACKGROUND

Structured reporting (SR) in radiology is becoming increasingly necessary and has been recognized recently by major scientific societies. This study aims to build structured CT-based reports in colon cancer during the staging phase in order to improve communication between the radiologist, members of multidisciplinary teams and patients.

MATERIALS AND METHODS

A panel of expert radiologists, members of the Italian Society of Medical and Interventional Radiology, was established. A modified Delphi process was used to develop the SR and to assess a level of agreement for all report sections. Cronbach's alpha (Cα) correlation coefficient was used to assess internal consistency for each section and to measure quality analysis according to the average inter-item correlation.

RESULTS

The final SR version was built by including n = 18 items in the "Patient Clinical Data" section, n = 7 items in the "Clinical Evaluation" section, n = 9 items in the "Imaging Protocol" section and n = 29 items in the "Report" section. Overall, 63 items were included in the final version of the SR. Both in the first and second round, all sections received a higher than good rating: a mean value of 4.6 and range 3.6-4.9 in the first round; a mean value of 5.0 and range 4.9-5 in the second round. In the first round, Cronbach's alpha (Cα) correlation coefficient was a questionable 0.61. In the first round, the overall mean score of the experts and the sum of scores for the structured report were 4.6 (range 1-5) and 1111 (mean value 74.07, STD 4.85), respectively. In the second round, Cronbach's alpha (Cα) correlation coefficient was an acceptable 0.70. In the second round, the overall mean score of the experts and the sum of score for structured report were 4.9 (range 4-5) and 1108 (mean value 79.14, STD 1.83), respectively. The overall mean score obtained by the experts in the second round was higher than the overall mean score of the first round, with a lower standard deviation value to underline greater agreement among the experts for the structured report reached in this round.

CONCLUSIONS

A wide implementation of SR is of critical importance in order to offer referring physicians and patients optimum quality of service and to provide researchers with the best quality data in the context of big data exploitation of available clinical data. Implementation is a complex procedure, requiring mature technology to successfully address the multiple challenges of user-friendliness, organization and interoperability.

Collecting specialty-related medical terms: Development and evaluation of a resource for Spanish

BACKGROUND

Controlled vocabularies are fundamental resources for information extraction from clinical texts using natural language processing (NLP). Standard language resources available in the healthcare domain such as the UMLS metathesaurus or SNOMED CT are widely used for this purpose, but with limitations such as lexical ambiguity of clinical terms. However, most of them are unambiguous within text limited to a given clinical specialty. This is one rationale besides others to classify clinical text by the clinical specialty to which they belong.

RESULTS

This paper addresses this limitation by proposing and applying a method that automatically extracts Spanish medical terms classified and weighted per sub-domain, using Spanish MEDLINE titles and abstracts as input. The hypothesis is biomedical NLP tasks benefit from collections of domain terms that are specific to clinical subdomains. We use PubMed queries that generate sub-domain specific corpora from Spanish titles and abstracts, from which token n-grams are collected and metrics of relevance, discriminatory power, and broadness per sub-domain are computed. The generated term set, called Spanish core vocabulary about clinical specialties (SCOVACLIS), was made available to the scientific community and used in a text classification problem obtaining improvements of 6 percentage points in the F-measure compared to the baseline using Multilayer Perceptron, thus demonstrating the hypothesis that a specialized term set improves NLP tasks.

CONCLUSION

The creation and validation of SCOVACLIS support the hypothesis that specific term sets reduce the level of ambiguity when compared to a specialty-independent and broad-scope vocabulary.

Artificial intelligence in sepsis early prediction and diagnosis using unstructured data in healthcare

Sepsis is a leading cause of death in hospitals. Early prediction and diagnosis of sepsis, which is critical in reducing mortality, is challenging as many of its signs and symptoms are similar to other less critical conditions. We develop an artificial intelligence algorithm, SERA algorithm, which uses both structured data and unstructured clinical notes to predict and diagnose sepsis. We test this algorithm with independent, clinical notes and achieve high predictive accuracy 12 hours before the onset of sepsis (AUC 0.94, sensitivity 0.87 and specificity 0.87). We compare the SERA algorithm against physician predictions and show the algorithm's potential to increase the early detection of sepsis by up to 32% and reduce false positives by up to 17%. Mining unstructured clinical notes is shown to improve the algorithm's accuracy compared to using only clinical measures for early warning 12 to 48 hours before the onset of sepsis.

Structured Reporting of Computed Tomography in the Staging of Neuroendocrine Neoplasms: A Delphi Consensus Proposal

BACKGROUND

Structured reporting (SR) in radiology is becoming increasingly necessary and has been recognized recently by major scientific societies. This study aims to build structured CT-based reports in Neuroendocrine Neoplasms during the staging phase in order to improve communication between the radiologist and members of multidisciplinary teams.

MATERIALS AND METHODS

A panel of expert radiologists, members of the Italian Society of Medical and Interventional Radiology, was established. A Modified Delphi process was used to develop the SR and to assess a level of agreement for all report sections. Cronbach's alpha (Cα) correlation coefficient was used to assess internal consistency for each section and to measure quality analysis according to the average inter-item correlation.

RESULTS

The final SR version was built by including n=16 items in the "Patient Clinical Data" section, n=13 items in the "Clinical Evaluation" section, n=8 items in the "Imaging Protocol" section, and n=17 items in the "Report" section. Overall, 54 items were included in the final version of the SR. Both in the first and second round, all sections received more than a good rating: a mean value of 4.7 and range of 4.2-5.0 in the first round and a mean value 4.9 and range of 4.9-5 in the second round. In the first round, the Cα correlation coefficient was a poor 0.57: the overall mean score of the experts and the sum of scores for the structured report were 4.7 (range 1-5) and 728 (mean value 52.00 and standard deviation 2.83), respectively. In the second round, the Cα correlation coefficient was a good 0.82: the overall mean score of the experts and the sum of scores for the structured report were 4.9 (range 4-5) and 760 (mean value 54.29 and standard deviation 1.64), respectively.

CONCLUSIONS

The present SR, based on a multi-round consensus-building Delphi exercise following in-depth discussion between expert radiologists in gastro-enteric and oncological imaging, derived from a multidisciplinary agreement between a radiologist, medical oncologist and surgeon in order to obtain the most appropriate communication tool for referring physicians.

Current approaches to identify sections within clinical narratives from electronic health records: a systematic review

BACKGROUND

The identification of sections in narrative content of Electronic Health Records (EHR) has demonstrated to improve the performance of clinical extraction tasks; however, there is not yet a shared understanding of the concept and its existing methods. The objective is to report the results of a systematic review concerning approaches aimed at identifying sections in narrative content of EHR, using both automatic or semi-automatic methods.

METHODS

This review includes articles from the databases: SCOPUS, Web of Science and PubMed (from January 1994 to September 2018). The selection of studies was done using predefined eligibility criteria and applying the PRISMA recommendations. Search criteria were elaborated by using an iterative and collaborative keyword enrichment.

RESULTS

Following the eligibility criteria, 39 studies were selected for analysis. The section identification approaches proposed by these studies vary greatly depending on the kind of narrative, the type of section, and the application. We observed that 57% of them proposed formal methods for identifying sections and 43% adapted a previously created method. Seventy-eight percent were intended for English texts and 41% for discharge summaries. Studies that are able to identify explicit (with headings) and implicit sections correspond to 46%. Regarding the level of granularity, 54% of the studies are able to identify sections, but not subsections. From the technical point of view, the methods can be classified into rule-based methods (59%), machine learning methods (22%) and a combination of both (19%). Hybrid methods showed better results than those relying on pure machine learning approaches, but lower than rule-based methods; however, their scope was more ambitious than the latter ones. Despite all the promising performance results, very few studies reported tests under a formal setup. Almost all the studies relied on custom dictionaries; however, they used them in conjunction with a controlled terminology, most commonly the UMLSⓇ metathesaurus.

CONCLUSIONS

Identification of sections in EHR narratives is gaining popularity for improving clinical extraction projects. This study enabled the community working on clinical NLP to gain a formal analysis of this task, including the most successful ways to perform it.

A clinical text classification paradigm using weak supervision and deep representation

BACKGROUND

Automatic clinical text classification is a natural language processing (NLP) technology that unlocks information embedded in clinical narratives. Machine learning approaches have been shown to be effective for clinical text classification tasks. However, a successful machine learning model usually requires extensive human efforts to create labeled training data and conduct feature engineering. In this study, we propose a clinical text classification paradigm using weak supervision and deep representation to reduce these human efforts.

METHODS

We develop a rule-based NLP algorithm to automatically generate labels for the training data, and then use the pre-trained word embeddings as deep representation features for training machine learning models. Since machine learning is trained on labels generated by the automatic NLP algorithm, this training process is called weak supervision. We evaluat the paradigm effectiveness on two institutional case studies at Mayo Clinic: smoking status classification and proximal femur (hip) fracture classification, and one case study using a public dataset: the i2b2 2006 smoking status classification shared task. We test four widely used machine learning models, namely, Support Vector Machine (SVM), Random Forest (RF), Multilayer Perceptron Neural Networks (MLPNN), and Convolutional Neural Networks (CNN), using this paradigm. Precision, recall, and F1 score are used as metrics to evaluate performance.

RESULTS

CNN achieves the best performance in both institutional tasks (F1 score: 0.92 for Mayo Clinic smoking status classification and 0.97 for fracture classification). We show that word embeddings significantly outperform tf-idf and topic modeling features in the paradigm, and that CNN captures additional patterns from the weak supervision compared to the rule-based NLP algorithms. We also observe two drawbacks of the proposed paradigm that CNN is more sensitive to the size of training data, and that the proposed paradigm might not be effective for complex multiclass classification tasks.

CONCLUSION

The proposed clinical text classification paradigm could reduce human efforts of labeled training data creation and feature engineering for applying machine learning to clinical text classification by leveraging weak supervision and deep representation. The experimental experiments have validated the effectiveness of paradigm by two institutional and one shared clinical text classification tasks.

Structured Reporting in Radiology

Radiology reports are vital for patient care as referring physicians depend upon them for deciding appropriate patient management. Traditional narrative reports are associated with excessive variability in the language, length, and style, which can minimize report clarity and make it difficult for referring clinicians to identify key information needed for patient care. Structured reporting has been advocated as a potential solution for improving the quality of radiology reports. The Association of University Radiologists-Radiology Research Alliance Structured Reporting Task Force convened to explore the current and future role of structured reporting in radiology and summarized its finding in this article. We review the advantages and disadvantages of structured radiology reports and discuss the current prevailing sentiments among radiologists regarding structured reports. We also discuss the obstacles to the use of structured reports and highlight ways to overcome some of those challenges. We also discuss the future directions in radiology reporting in the era of personalized medicine.

Clinical information extraction applications: A literature review

BACKGROUND

With the rapid adoption of electronic health records (EHRs), it is desirable to harvest information and knowledge from EHRs to support automated systems at the point of care and to enable secondary use of EHRs for clinical and translational research. One critical component used to facilitate the secondary use of EHR data is the information extraction (IE) task, which automatically extracts and encodes clinical information from text.

OBJECTIVES

In this literature review, we present a review of recent published research on clinical information extraction (IE) applications.

METHODS

A literature search was conducted for articles published from January 2009 to September 2016 based on Ovid MEDLINE In-Process & Other Non-Indexed Citations, Ovid MEDLINE, Ovid EMBASE, Scopus, Web of Science, and ACM Digital Library.

RESULTS

A total of 1917 publications were identified for title and abstract screening. Of these publications, 263 articles were selected and discussed in this review in terms of publication venues and data sources, clinical IE tools, methods, and applications in the areas of disease- and drug-related studies, and clinical workflow optimizations.

CONCLUSIONS

Clinical IE has been used for a wide range of applications, however, there is a considerable gap between clinical studies using EHR data and studies using clinical IE. This study enabled us to gain a more concrete understanding of the gap and to provide potential solutions to bridge this gap.

Imaging approaches to optimize molecular therapies

Imaging, including its use for innovative tissue sampling, is slowly being recognized as playing a pivotal role in drug development, clinical trial design, and more effective delivery and monitoring of molecular therapies. The challenge is that, while a considerable number of new imaging technologies and new targeted tracers have been developed for cancer imaging in recent years, the technologies are neither evenly distributed nor evenly implemented. Furthermore, many imaging innovations are not validated and are not ready for widespread use in drug development or in clinical trial designs. Inconsistent and often erroneous use of terminology related to quantitative imaging biomarkers has also played a role in slowing their development and implementation. We examine opportunities for, and challenges of, the use of imaging biomarkers to facilitate development of molecular therapies and to accelerate progress in clinical trial design. In the future, in vivo molecular imaging, image-guided tissue sampling for mutational analyses ("high-content biopsies"), and noninvasive in vitro tests ("liquid biopsies") will likely be used in various combinations to provide the best possible monitoring and individualized treatment plans for cancer patients.