BioBERTurk: Exploring Turkish Biomedical Language Model Development Strategies in Low-Resource Setting

Pretrained language models augmented with in-domain corpora show impressive results in biomedicine and clinical Natural Language Processing (NLP) tasks in English. However, there has been minimal work in low-resource languages. Although some pioneering works have shown promising results, many scenarios still need to be explored to engineer effective pretrained language models in biomedicine for low-resource settings. This study introduces the BioBERTurk family and four pretrained models in Turkish for biomedicine. To evaluate the models, we also introduced a labeled dataset to classify radiology reports of head CT examinations. Two parts of the reports, impressions and findings, are evaluated separately to observe the performance of models on longer and less informative text. We compared the models with the Turkish BERT (BERTurk) pretrained with general domain text, multilingual BERT (mBERT), and LSTM+attention-based baseline models. The first model initialized from BERTurk and then further pretrained with biomedical corpus performs statistically better than BERTurk, multilingual BERT, and baseline for both datasets. The second model continues to pretrain the BERTurk model by using only radiology Ph.D. theses to test the effect of task-related text. This model slightly outperformed all models on the impression dataset and showed that using only radiology-related data for continual pre-training could be effective. The third model continues to pretrain by adding radiology theses to the biomedical corpus but does not show a statistically meaningful difference for both datasets. The final model combines radiology and biomedicine corpora with the corpus of BERTurk and pretrains a BERT model from scratch. This model is the worst-performing model of the BioBERT family, even worse than BERTurk and multilingual BERT.

Prediction of malignancy upgrade rate in high-risk breast lesions using an artificial intelligence model: a retrospective study

PURPOSE

High-risk breast lesions (HRLs) are associated with future risk of breast cancer. Considering the pathological subtypes, malignancy upgrade rate differs according to each subtype and depends on various factors such as clinical and radiological features and biopsy method. Using artificial intelligence and machine learning models in breast imaging, evaluations can be made in terms of risk estimation in different research areas. This study aimed to develop a machine learning model to distinguish HRL cases requiring surgical excision from lesions with a low risk of accompanying malignancy.

METHODS

A total of 94 patients who were diagnosed with HRL by image-guided biopsy between January 2008 and March 2020 were included in the study. A structured database was created with clinical and radiological characteristics and histopathological results. A machine learning prediction model was created to make binary classifications of lesions as malignant or benign. Random forest, decision tree, K-nearest neighbors, logistic regression, support vector machine (SVM), and multilayer perceptron machine learning algorithms were used. Among these algorithms, SVM was the most successful. The estimations of malignancy for each case detected by artificial intelligence were combined and statistical analyses were performed.

RESULTS

Considering all cases, the malignancy upgrade rate was 24.5%. A significant association was observed between malignancy upgrade rate and lesion size (P = 0.004), presence of mammography findings (P = 0.022), and breast imaging-reporting and data system category (P = 0.001). A statistically significant association was also found between the artificial intelligence prediction model and malignancy upgrade rate (P < 0.001). With the SVM model, an 84% accuracy and 0.786 area-underthe- curve score were obtained in classifying the data as benign or malignant.

CONCLUSION

Our artificial intelligence model (SVM) can predict HRLs that can be followed up with a lower risk of accompanying malignancy. Unnecessary surgeries can be reduced, or second line vacuum excisions can be performed in HRLs, which are mostly benign, by evaluating on a case-by-case basis, in line with radiology-pathology compatibility and by using an artificial intelligence model.

Diagnostic performance rates of the ACR-TIRADS and EU-TIRADS based on histopathological evidence

PURPOSE

In this study, we aimed to assess the effectiveness of malignancy stratification algorithms of the American College of Radiology (ACR) and European Thyroid Association (ETA) in the delineation of thyroid nodules using a database of nodules that were unequivocally diagnosed by means of histopathological examination and meticulously matched with the imaged nodules.

METHODS

A total of 165 patients having 251 thyroid nodules with histopathologically proven definitive diagnoses during a 5-year period were included in this study. All patients had preoperatively undergone ultrasonography (US) examination, and US characteristics of the thyroid nodules were retrospectively analyzed and assigned in compliance with the thyroid imaging reporting and data system categories recommended by the ACR (ACR-TIRADS) and ETA (EU-TIRADS). The diagnostic effectiveness in the delineation of thyroid nodules and unnecessary fine-needle aspiration (FNAB) rates were evaluated.

RESULTS

Overall, 189 nodules (75.30%) were diagnosed as benign, while 62 nodules (24.70%) were reported to be malignant based on histopathological assessment. Sensitivity and specificity rates were 71% and 75% for ACR-TIRADS and 73% and 80% for EU-TIRADS. The area under the curve values were 0.78 and 0.80 for ACR-TIRADS and EU-TIRADS, respectively. The unnecessary FNAB rates were 61% for ACR-TIRADS and 64% for EU-TIRADS as per the recommended criteria of each algorithm.

CONCLUSION

The diagnostic performance of both malignancy stratification systems was signified to be moderate and sufficient in a cohort of nodules with definite histopathological diagnosis. In light of our results, we demonstrated the strengths and weaknesses of the ACR- and EU-TIRADS for physicians who should be familiar with them for optimal management of thyroid nodules.

Impact of the HEAD-US Scoring System for Observing the Protective Effect of Prophylaxis in Hemophilia Patients: A Prospective, Multicenter, Observational Study

Objective:

This study aimed to observe the preventive effect of prophylactic treatment on joint health in people with hemophilia (PwH) and to investigate the importance of integration of ultrasonographic examination into clinical and radiological evaluation of the joints.

Materials and Methods:

This national, multicenter, prospective, observational study included male patients aged ≥6 years with the diagnosis of moderate or severe hemophilia A or B from 8 centers across Turkey between January 2017 and March 2019. Patients were followed for 1 year with 5 visits (baseline and 3^rd, 6^th, 9^th, and 12^th month visits). The Hemophilia Joint Health Score (HJHS) was used for physical examination of joints, the Pettersson scoring system was used for radiological assessment, point-of-care (POC) ultrasonography was used for bilateral examinations of joints, and the Hemophilia Early Arthropathy Detection with Ultrasound (HEAD-US) score was used for evaluation of ultrasonography results.

Results:

Seventy-three PwH, of whom 62 had hemophilia A and 11 had hemophilia B, were included and 24.7% had target joints at baseline. The HJHS and HEAD-US scores were significantly increased at the 12^th month in all patients. These scores were also higher in the hemophilia A subgroup than the hemophilia B subgroup. However, in the childhood group, the increment of scores was not significant. The HEAD-US total score was significantly correlated with both the HJHS total score and Pettersson total score at baseline and at the 12^th month.

Conclusion:

The HEAD-US and HJHS scoring systems are valuable tools during follow-up examinations of PwH and they complement each other. We suggest that POC ultrasonographic evaluation and the HEAD-US scoring system may be integrated into differential diagnosis of bleeding and long-term monitoring for joint health as a routine procedure.

COVID-19 radiology CT personnel management

A new coronavirus outbreak called COVID-19 started in December 2019. In Turkey, the first case was reported on 10 March 2020. In this article, information will be given about the patient and staff management and organization that we have implemented in the Radiology Department of our hospital during the COVID-19 pandemic. The rules we followed were: 1- Performing the examinations of COVID-19 patients and suspects with a CT device isolated from other patients; 2- Reducing the unnecessary workload in imaging modalities other than CT, emergency radiography, and emergency ultrasonography; 3- Directing and managing patients and their relatives in accordance with the mask and distancing rules; 4- Disinfecting the device with an appropriate disinfectant after each patient in order to prevent cross-contamination; 5- Protecting the entire technician team from infection by employing one week work, two weeks off shifts of fixed teams; 6- Ensuring adequate ventilation of the gantry room. Adhering to the above rules, no infection spread was reported from the Radiology department and especially the COVID-19 CT unit.

Hepatic and splenic sonographic and sonoelastographic findings in pulmonary arterial hypertension

Purpose

The aim of this study was to evaluate the associations of sonographic and sonoelastographic parameters with clinical cardiac parameters, as well as to assess their value in predicting survival in patients with pulmonary arterial hypertension (PAH).

Methods

Thirty-six patients with PAH and normal liver function were prospectively enrolled in this prospective study along with 26 healthy controls, all of whom underwent ultrasound and point shear wave elastography examinations. Additionally, the portal vein pulsatility index (PVPI), inferior vena cava collapsibility index, and clinical cardiac variables were obtained in PAH patients. The values of hepatic (LV_s) and splenic shear wave velocity (SV_s) were compared between PAH patients and controls. The relationships between all sonographic and clinical parameters in the PAH patients were analyzed. Furthermore, their prognostic value in predicting survival was investigated.

Results

LV_s values in PAH patients (median, 1.62 m/s) were significantly higher than in controls (median, 0.99 m/s), while no significant difference was observed in SV_s values. Patients with higher grades of tricuspid regurgitation (TR) had significantly different values of PVPI (P=0.010) and sonoelastographic parameters (P<0.001 for LV_s and P=0.004 for SV_s) compared to those with less severe TR. Tricuspid annular plane systolic excursion values were the only investigated parameter found to be associated with survival (hazard ratio, 0.814; 95% confidence interval, 0.694 to 0.954; P=0.011).

Conclusion

Our results demonstrated a direct association between cardiac congestion (i.e., the severity of TR) and liver stiffness, which should be kept in mind during the assessment of fibrosis in patients with PAH.

[The Effects of Lithium on Calcium and Parathormone Levels: A Cross-sectional Study with Healthy Controls]

OBJECTIVE

Despite lithium associated hyperparathyroidism (LAH) can lead to many complications, little notice has been paid to this sideeffect. The aim of this study was to investigate the effects of lithium on calcium and parathyroid hormone levels and the relation between lithium use and thyroid diseases.

METHOD

This cross-sectional study was carried out with 87 lithiumtreated patients and 65 volunteers who had a similar age and gender distribution with the lithium group. Serum levels of corrected calcium, intact parathormone, phosphorus, magnesium, alkaline phosphatase, free thyroxine, thyroid stimulating hormone, thyroid autoantibodies and creatinine were assessed, and also, thyroid and parathyroid ultrasonography was conducted. Further detailed investigations were made depending on the elevation of the initially measured calcium and/ or parathormone levels.

RESULTS

Median values of serum levels of the corrected calcium and the intact parathormone were significantly higher in the lithium group. Calcium levels had a mild correlation with the duration of lithium treatment. In the first assessment, while all control individuals had values within the normal reference range, 11 lithium-treated patients had corrected calcium and/or intact parathormone levels above the normal reference levels. All of the five patients, who were diagnosed with LAH after further investigation, were also diagnosed with a thyroid disorder.

CONCLUSION

These results demonstrate that lithium treatment has a relationship with calcium and parathormone levels. The 5.7% prevalence of LAH and potential life-threatening conditions associated with LAH necessitates the use of available low-cost METHODS to monitor blood calcium levels of lithium-treated patients for early diagnosis.

Thyroid Function and Ultrasonography Abnormalities in Lithium-Treated Bipolar Patients: A Cross-sectional Study with Healthy Controls

INTRODUCTION

Lithium has many effects on thyroid physiology. Although these side effects have been known for a long time, large sample studies of lithium-treated patients using ultrasonography are lacking. The aim of this study is to investigate the detailed thyroid morphologies, hormone levels, and antibodies of lithium-treated patients compared with healthy controls.

METHODS

This cross-sectional study involved 84 lithium-treated patients with bipolar disorder and 65 gender and age similar controls who had never been exposed to lithium. Subjects between 18 and 65 years of age were eligible for the study. Venous blood samples were acquired to determine the levels of free thyroxine (fT4), thyroid stimulating hormone (TSH), and thyroid antibodies; also, ultrasonographic examinations of the patients' thyroid glands were performed.

RESULTS

There were no statistically significant differences in smoking habits, known thyroid disease, thyroid medication use, familial thyroid disease, fT4 level, autoimmunity, thyroid nodule presence, or Hashimoto's thyroiditis between the lithium and control groups. The median TSH level and thyroid volume were significantly higher in the lithium group. In the lithium group, 14 cases (16.7%) of hypothyroidism, seven cases (8.3%) of subclinical hypothyroidism, and one case (1.2%) of subclinical hyperthyroidism were defined; in the control group, seven cases (10.8%) of hypothyroidism and two cases (3.1%) of subclinical hyperthyroidism were defined. Thyroid dysfunction, goiter, parenchymal abnormality, ultrasonographically defined thyroid abnormality, and thyroid disorder were found to be more prevalent in the lithium group. 90% of patients with goiter and 74.3% of patients with ultrasonographic pathologies were euthyroid.

CONCLUSION

It is important to note that 90% of the patients with goiter were euthyroid. This indicates that monitoring by blood test alone is insufficient. The prevalence rates of 47.6% for goiter and 83.3% for ultrasonographic pathology demonstrate that ultasonographic follow-up may be useful in lithium-treated patients. To determine whether routine ultrasonographic examination is necessary, large sample prospective studies are necessary due to the limitations of this study.