Visual saliency models for summarization of diagnostic hysteroscopy videos in healthcare systems

Patient Information Summarization in Clinical Settings: Scoping Review

BACKGROUND

Information overflow, a common problem in the present clinical environment, can be mitigated by summarizing clinical data. Although there are several solutions for clinical summarization, there is a lack of a complete overview of the research relevant to this field.

OBJECTIVE

This study aims to identify state-of-the-art solutions for clinical summarization, to analyze their capabilities, and to identify their properties.

METHODS

A scoping review of articles published between 2005 and 2022 was conducted. With a clinical focus, PubMed and Web of Science were queried to find an initial set of reports, later extended by articles found through a chain of citations. The included reports were analyzed to answer the questions of where, what, and how medical information is summarized; whether summarization conserves temporality, uncertainty, and medical pertinence; and how the propositions are evaluated and deployed. To answer how information is summarized, methods were compared through a new framework "collect-synthesize-communicate" referring to information gathering from data, its synthesis, and communication to the end user.

RESULTS

Overall, 128 articles were included, representing various medical fields. Exclusively structured data were used as input in 46.1% (59/128) of papers, text in 41.4% (53/128) of articles, and both in 10.2% (13/128) of papers. Using the proposed framework, 42.2% (54/128) of the records contributed to information collection, 27.3% (35/128) contributed to information synthesis, and 46.1% (59/128) presented solutions for summary communication. Numerous summarization approaches have been presented, including extractive (n=13) and abstractive summarization (n=19); topic modeling (n=5); summary specification (n=11); concept and relation extraction (n=30); visual design considerations (n=59); and complete pipelines (n=7) using information extraction, synthesis, and communication. Graphical displays (n=53), short texts (n=41), static reports (n=7), and problem-oriented views (n=7) were the most common types in terms of summary communication. Although temporality and uncertainty information were usually not conserved in most studies (74/128, 57.8% and 113/128, 88.3%, respectively), some studies presented solutions to treat this information. Overall, 115 (89.8%) articles showed results of an evaluation, and methods included evaluations with human participants (median 15, IQR 24 participants): measurements in experiments with human participants (n=31), real situations (n=8), and usability studies (n=28). Methods without human involvement included intrinsic evaluation (n=24), performance on a proxy (n=10), or domain-specific tasks (n=11). Overall, 11 (8.6%) reports described a system deployed in clinical settings.

CONCLUSIONS

The scientific literature contains many propositions for summarizing patient information but reports very few comparisons of these proposals. This work proposes to compare these algorithms through how they conserve essential aspects of clinical information and through the "collect-synthesize-communicate" framework. We found that current propositions usually address these 3 steps only partially. Moreover, they conserve and use temporality, uncertainty, and pertinent medical aspects to varying extents, and solutions are often preliminary.

A Security Management Framework for Big Data in Smart Healthcare

Big Data analytics in the medical sector can assist medical professionals to facilitate improvement in healthcare. With the help of data analysis, clinical images of patients can be used to detect certain medical conditions. In the COVID-19 pandemic, many integrated technologies are being used to remodel the healthcare systems. The management of an integrated healthcare solution necessitates the need for security of the medical data. In this paper, we propose a security framework based on the Logistic equation, Hyperchaotic equation, and Deoxyribonucleic Acid (DNA) encoding. Subsequently, a Lossless Computational Secret Image Sharing (CSIS) method is used to convert the encrypted secret image into shares for distributed storage in cloud-based servers. Hyperchaotic and DNA encryption is performed to improve the overall security of the system. Furthermore, Pseudorandom Numbers (PRN) generated by the logistic equation are XORed with the image sequence in two phases by changing the parameters slightly. Finally, the application of Secret Sharing (SS) generates completely noise-like cipher images that enhance the security of the cloud-based cryptosystem. The generated shares are small in size and require fewer resources like storage capacity and transmission bandwidth which is highly desirable in IoT-based systems. It is verified that the cryptosystem is highly secure against attacks as well as interferences and has a very strong key-sensitivity.

Bayesian Update with Information Quality under the Framework of Evidence Theory

Bayesian update is widely used in data fusion. However, the information quality is not taken into consideration in classical Bayesian update method. In this paper, a new Bayesian update with information quality under the framework of evidence theory is proposed. First, the discounting coefficient is determined by information quality. Second, the prior probability distribution is discounted as basic probability assignment. Third, the basic probability assignments from different sources can be combined with Dempster's combination rule to obtain the fusion result. Finally, with the aid of pignistic probability transformation, the combination result is converted to posterior probability distribution. A numerical example and a real application in target recognition show the efficiency of the proposed method. The proposed method can be seen as the generalized Bayesian update. If the information quality is not considered, the proposed method degenerates to the classical Bayesian update.

Keyframe extraction from laparoscopic videos based on visual saliency detection

BACKGROUND AND OBJECTIVE

Laparoscopic surgery offers the potential for video recording of the operation, which is important for technique evaluation, cognitive training, patient briefing and documentation. An effective way for video content representation is to extract a limited number of keyframes with semantic information. In this paper we present a novel method for keyframe extraction from individual shots of the operational video.

METHODS

The laparoscopic video was first segmented into video shots using an objectness model, which was trained to capture significant changes in the endoscope field of view. Each frame of a shot was then decomposed into three saliency maps in order to model the preference of human vision to regions with higher differentiation with respect to color, motion and texture. The accumulated responses from each map provided a 3D time series of saliency variation across the shot. The time series was modeled as a multivariate autoregressive process with hidden Markov states (HMMAR model). This approach allowed the temporal segmentation of the shot into a predefined number of states. A representative keyframe was extracted from each state based on the highest state-conditional probability of the corresponding saliency vector.

RESULTS

Our method was tested on 168 video shots extracted from various laparoscopic cholecystectomy operations from the publicly available Cholec80 dataset. Four state-of-the-art methodologies were used for comparison. The evaluation was based on two assessment metrics: Color Consistency Score (CCS), which measures the color distance between the ground truth (GT) and the closest keyframe, and Temporal Consistency Score (TCS), which considers the temporal proximity between GT and extracted keyframes. About 81% of the extracted keyframes matched the color content of the GT keyframes, compared to 77% yielded by the second-best method. The TCS of the proposed and the second-best method was close to 1.9 and 1.4 respectively.

CONCLUSIONS

Our results demonstrated that the proposed method yields superior performance in terms of content and temporal consistency to the ground truth. The extracted keyframes provided highly semantic information that may be used for various applications related to surgical video content representation, such as workflow analysis, video summarization and retrieval.