Vision-Language Model for Generating Textual Descriptions From Clinical Images: Model Development and Validation Study

BACKGROUND

The automatic generation of radiology reports, which seeks to create a free-text description from a clinical radiograph, is emerging as a pivotal intersection between clinical medicine and artificial intelligence. Leveraging natural language processing technologies can accelerate report creation, enhancing health care quality and standardization. However, most existing studies have not yet fully tapped into the combined potential of advanced language and vision models.

OBJECTIVE

The purpose of this study was to explore the integration of pretrained vision-language models into radiology report generation. This would enable the vision-language model to automatically convert clinical images into high-quality textual reports.

METHODS

In our research, we introduced a radiology report generation model named ClinicalBLIP, building upon the foundational InstructBLIP model and refining it using clinical image-to-text data sets. A multistage fine-tuning approach via low-rank adaptation was proposed to deepen the semantic comprehension of the visual encoder and the large language model for clinical imagery. Furthermore, prior knowledge was integrated through prompt learning to enhance the precision of the reports generated. Experiments were conducted on both the IU X-RAY and MIMIC-CXR data sets, with ClinicalBLIP compared to several leading methods.

RESULTS

Experimental results revealed that ClinicalBLIP obtained superior scores of 0.570/0.365 and 0.534/0.313 on the IU X-RAY/MIMIC-CXR test sets for the Metric for Evaluation of Translation with Explicit Ordering (METEOR) and the Recall-Oriented Understudy for Gisting Evaluation (ROUGE) evaluations, respectively. This performance notably surpasses that of existing state-of-the-art methods. Further evaluations confirmed the effectiveness of the multistage fine-tuning and the integration of prior information, leading to substantial improvements.

CONCLUSIONS

The proposed ClinicalBLIP model demonstrated robustness and effectiveness in enhancing clinical radiology report generation, suggesting significant promise for real-world clinical applications.

Mediastinal hematoma due to traumatic injury to the azygous vein

Rupture of the azygous vein may result from abrupt deceleration applied to the mobile azygous arch, which can initiate shearing forces within the thorax.

A systematic review and clinical atlas on mucocutaneous presentations of the current monkeypox outbreak: With a comprehensive approach to all dermatologic and nondermatologic aspects of the new and previous monkeypox outbreaks

Monkeypox is a zoonotic disease, endemic in central and west African regions, and has re-emerged, currently causing an outbreak as of May 2022. In this systematic review, we aimed to characterize the current face of the disease, with a detailed categorization of mucocutaneous, as well as systemic symptoms of the disease. We searched four main online databases with the keywords "monkeypox" and "Orthopoxvirus". A total of 46 articles were included, with a cumulative number of 1984 confirmed cases. Patients were predominantly men who have sex with men, who were mostly in their 30s, with a history of unprotected sexual contact or international travel. Among mucocutaneous manifestations, anogenital lesions were the most commonly observed, followed by lesions on the limbs, face, trunk, and palms or soles. Among lesion types, vesiculopustular, pustular or pseudo-pustular, vesicular-umbilicated and papular lesions were the most common, mainly presenting asynchronously, with less than 10 lesions on each patient. Almost all patients also reported systemic manifestations, namely fever, lymphadenopathy, fatigue, myalgia, headaches, pharyngitis, and proctitis. Sexual contact is the main pathway of transmission in the current outbreak, with viral shedding in bodily fluids playing a key role. We've compared these idiosyncratic findings of the new outbreak with previous outbreaks. We've also gathered and categorized images from our included studies to make a "clinical atlas" for this "new" face of monkeypox, which can be of utmost importance for clinicians to be familiarized with, and have a clear picture of monkeypox for their differential diagnoses.

Factors important in the correct evaluation of oral high-risk lesions during the telehealth era

BACKGROUND

Considering that early detection of squamous cell carcinoma (SCC) improves prognosis and clinical examination is the primary detection method, we identified factors related to the clinical evaluation of oral mucosal lesions. Due to the growing role of telehealth, our study was based on clinical image evaluation.

SUBJECTS AND METHODS

Oral medicine specialists and dental students evaluated six images of benign, potentially malignant, or SCC lesions (18 images in total). We analyzed the role of personal factors of the examiners and the visual pathological features of the lesion upon which the participants based their evaluation.

RESULTS

One hundred thirty-three subjects participated. Half of the benign images were correctly evaluated. On average 1.2 (±SD1.3) cancer pictures were recognized correctly and 3.66 (±SD1.42) images were considered potentially malignant. Potentially malignant lesions were correctly evaluated at an average of 4.08 (±SD1.48) images. For cancer and potentially malignant lesion images, there were significantly better results among clinicians with the worst results from the fourth-year students. Student results correlated significantly with years of study, number of weeks spent in the oral medicine clinic, and interest in oral pathology. Consideration of lesion irregularity yielded a correct diagnosis, whereas wrong answers were based on color changes. Lesion size and margins were considered equally important.

CONCLUSIONS

Using clinical images as part of the diagnostic process provides good results, though increased clinical experience for graduates and undergraduates may be necessary to improve accuracy. Therefore, emphasizing the important visual parameters of malignancy may be valuable in the current telehealth era.

Diverticular Bleeding: A Clinical Image

A man in his 40’s was admitted to the general surgery ward with multiple episodes of large amounts of painless bright red per rectal (PR) bleeding and passage of clots. Urgent outpatient colonoscopy revealed a diverticulum which was associated with a wide diameter blood vessel originating from its base which was not actively bleeding. The clinical picture presented by the colonoscopy is one of the first to clearly identify large caliber blood vessels emerging from a colonic diverticulum.

Eagle syndrome: a rare pathology presented in a 3-Dimensional reconstructed view

A 42-year-old male presented with 3-month history of constant right-sided frontal headaches, severe right-sided intermittent sharp jaw pain, odynophagia, globus pharyngis, and worsening episodes of blurry vision in his right eye. Cervicocerebral angiography demonstrated a prominent, 4 cm right sided styloid process with close proximity to the right internal carotid artery (ICA). The patient was referred to otorhinolaryngology for styloidectomy and continued care.

Dicephalus parapagus conjoined twins

Dicephalus parapagus are conjoined twins with two separate heads. Proper identification helps in patient counseling and timely referral to specialist centers.

An Elusive Case of Mycosis Fungoides: Case Report and Review of the Literature

Erythroderma refers to a spectrum of skin diseases resulting in diffuse erythema and scaling encompassing ≥ 90% of the body surface area. The differential diagnosis ranges from primary dermatologic diseases such as atopic dermatitis and psoriasis to potentially deadly causes such as staphylococcal toxic shock syndrome, toxic epidermal necrolysis, and malignancy. Cutaneous T cell lymphoma (CTCL) is an uncommon but highly morbid cause of erythroderma. This non-Hodgkin lymphoma remains a diagnostic challenge due to its variable clinical presentation and varied histologic features. Mycosis fungoides (MF) is the most common form of CTCL. Making a timely diagnosis is challenging as it may mimic inflammatory diseases of the skin including eczema, psoriasis, lichen planus, and cutaneous lupus. We present a case of a 58-year-old man who presented with 5 years of cutaneous symptoms and several months of fevers and night sweats, ultimately diagnosed as MF. Owing to diffuse CD30 positivity, he was a candidate for brentuximab vedotin, an antibody-drug conjugate medication that selectively targets the CD30 antigen. This resulted in an excellent therapeutic response.

The Possibility of Deep Learning-Based, Computer-Aided Skin Tumor Classifiers

The incidence of skin tumors has steadily increased. Although most are benign and do not affect survival, some of the more malignant skin tumors present a lethal threat if a delay in diagnosis permits them to become advanced. Ideally, an inspection by an expert dermatologist would accurately detect malignant skin tumors in the early stage; however, it is not practical for every single patient to receive intensive screening by dermatologists. To overcome this issue, many studies are ongoing to develop dermatologist-level, computer-aided diagnostics. Whereas, many systems that can classify dermoscopic images at this dermatologist-equivalent level have been reported, a much fewer number of systems that can classify conventional clinical images have been reported thus far. Recently, the introduction of deep-learning technology, a method that automatically extracts a set of representative features for further classification has dramatically improved classification efficacy. This new technology has the potential to improve the computer classification accuracy of conventional clinical images to the level of skilled dermatologists. In this review, this new technology and present development of computer-aided skin tumor classifiers will be summarized.

[Evaluation of Entrance Surface Dose in Chest Radiography Using Clinical Images: Estimation of Subject Thickness Using Two-direction Radiography]

The International Commission on Radiological Protection recommends adaptation of the diagnostic reference levels as an indicator of optimization of protection, and diagnostic reference levels of 2015 were also published in Japan in 2015 (Japan DRLs 2015). The entrance surface dose (ESD) is evaluated to the published standard subject thickness in Japan DRLs 2015. However, the standard radiographic settings of each facility may not be a radiographic condition of the standard subject thickness of Japan DRLs 2015. We measure and record the thickness of the subject in every examination, and it can solve this problem, but it is difficult to carry out it in the actual clinical scene. In this study, we aimed to estimate the subject thickness by using chest clinical images and to calculate ESD for each radiography. We evaluated and compared with Japan DRLs 2015 using these data. The subject thickness was estimated from 200 cases of digital imaging and communications in medicine (DICOM) image obtained by both the frontal and lateral views of the chest radiography. Also, at the same time, the radiographic settings were acquired from the information of the DICOM tag. The subject thickness was 23.60 cm on the average, and the median of the ESD was 0.104 mGy. Also, the median of the ESD at the standard subject thickness of 20 cm in Japan DRLs 2015 was 0.075 mGy. The ESD can be calculated without measuring the body thickness of the patient of every examination by using the method of this study.

Erythema nodosum - review of the literature

Erythema nodosum is the most common form of panniculitis. It may have many aetiological factors. Erythema nodosum occurs three to five times more often in female patients. It appears as erythematous painful rounded nodules, located most often on the anterior surface of the lower extremities, and may be accompanied by systemic symptoms such as fever, malaise and arthralgia. During diagnosis, oncological vigilance should be maintained, because erythema nodosum may be a paraneoplastic symptom. It requires an interdisciplinary approach and exclusion of many underlying causes.

Cardiac tamponade - presentation of type A aortic dissection

Acute aortic dissection usually presents with severe chest and/or back pain but may have a varied presentation ranging from syncope, stroke, and heart failure to shock or tamponade. We present classic chest computed tomography images of a case of type A aortic dissection presenting with cardiac tamponade.