How generative artificial intelligence portrays science: Interviewing ChatGPT from the perspective of different audience segments

Generative artificial intelligence in general and ChatGPT in particular have risen in importance. ChatGPT is widely known and used increasingly as an information source for different topics, including science. It is therefore relevant to examine how ChatGPT portrays science and science-related issues. Research on this question is lacking, however. Hence, we simulate "interviews" with ChatGPT and reconstruct how it presents science, science communication, scientific misbehavior, and controversial scientific issues. Combining qualitative and quantitative content analysis, we find that, generally, ChatGPT portrays science largely as the STEM disciplines, in a positivist-empiricist way and a positive light. When comparing ChatGPT's responses to different simulated user profiles and responses from the GPT-3.5 and GPT-4 versions, we find similarities in that the scientific consensus on questions such as climate change, COVID-19 vaccinations, or astrology is consistently conveyed across them. Beyond these similarities in substance, however, pronounced differences are found in the personalization of responses to different user profiles and between GPT-3.5 and GPT-4.

A Bilingual Readability Assessment of Online Breast Cancer Screening and Treatment Information

INTRODUCTION

Presenting health information at a sixth-grade reading level is advised to accommodate the general public's abilities. Breast cancer (BC) is the second-most common malignancy in women, but the readability of online BC information in English and Spanish, the two most commonly spoken languages in the United States, is uncertain.

METHODS

Three search engines were queried using: "how to do a breast examination," "when do I need a mammogram," and "what are the treatment options for breast cancer" in English and Spanish. Sixty websites in each language were studied and classified by source type and origin. Three readability frameworks in each language were applied: Flesch Kincaid Reading Ease, Flesch Kincaid Grade Level, and Simple Measure of Gobbledygook (SMOG) for English, and Fernández-Huerta, Spaulding, and Spanish adaptation of SMOG for Spanish. Median readability scores were calculated, and corresponding grade level determined. The percentage of websites requiring reading abilities >sixth grade level was calculated.

RESULTS

English-language websites were predominantly hospital-affiliated (43.3%), while Spanish websites predominantly originated from foundation/advocacy sources (43.3%). Reading difficulty varied across languages: English websites ranged from 5th-12th grade (Flesch Kincaid Grade Level/Flesch Kincaid Reading Ease: 78.3%/98.3% above sixth grade), while Spanish websites spanned 4th-10th grade (Spaulding/Fernández-Huerta: 95%/100% above sixth grade). SMOG/Spanish adaptation of SMOG scores showed lower reading difficulty for Spanish, with few websites exceeding sixth grade (1.7% and 0% for English and Spanish, respectively).

CONCLUSIONS

Online BC resources have reading difficulty levels that exceed the recommended sixth grade, although these results vary depending on readability framework. Efforts should be made to establish readability standards that can be translated into Spanish to enhance accessibility for this patient population.

Psycholinguistic and emotion analysis of cryptocurrency discourse on X platform

This paper provides an extensive examination of a sizable dataset of English tweets focusing on nine widely recognized cryptocurrencies, specifically Cardano, Binance, Bitcoin, Dogecoin, Ethereum, Fantom, Matic, Shiba, and Ripple. Our goal was to conduct a psycholinguistic and emotional analysis of social media content associated with these cryptocurrencies. Such analysis can enable researchers and experts dealing with cryptocurrencies to make more informed decisions. Our work involved comparing linguistic characteristics across the diverse digital coins, shedding light on the distinctive linguistic patterns emerging in each coin's community. To achieve this, we utilized advanced text analysis techniques. Additionally, this work unveiled an understanding of the interplay between these digital assets. By examining which coin pairs are mentioned together most frequently in the dataset, we established co-mentions among different cryptocurrencies. To ensure the reliability of our findings, we initially gathered a total of 832,559 tweets from X. These tweets underwent a rigorous preprocessing stage, resulting in a refined dataset of 115,899 tweets that were used for our analysis. Overall, our research offers valuable perception into the linguistic nuances of various digital coins' online communities and provides a deeper understanding of their interactions in the cryptocurrency space.

Readability of Individuals With Disabilities Education Act Part B Procedural Safeguards Notices Written in Spanish

PURPOSE

The purpose of this study was to analyze the readability of the Individuals with Disabilities Education Act (IDEA) Part B procedural safeguard notices written in Spanish, as distributed by each of the 50 states in the United States and the District of Columbia. Results were compared with the 6th-8th grade document readability guideline recommended by experts in health communication.

METHOD

A commercially available readability software program was used to assess document readability using three readability indices: the Gilliam-Peña-Mountain Graph, SOL, and the Fry Readability Adaptation for Spanish Evaluation.

RESULTS

All procedural safeguard notices were written at a reading grade level greater than established guidelines. This finding agrees with research that has examined the readability of English-language versions of IDEA Part B procedural safeguard notices.

CONCLUSIONS

Literacy-related skills and demands may influence the ability of the parent to advocate on behalf of his or her child with a disability within the school setting. Educators and specialists should be sensitive to parent reading preferences and proficiency when reviewing IDEA procedural safeguards notices. Authors of procedural safeguard notices should account for the native language and culture of the intended reader as revisions to, and translations are made of, IDEA procedural safeguards notices.

Textual complexity adjustments to the English reading comprehension test for undergraduate EFL students

It can be challenging for teachers to prepare students for a reading comprehension test. While most research on reading comprehension tests focuses on the interaction between the text complexity and test-taker's ability, this study investigates the interaction between the text complexity and degree of difficulty of the tasks, following each text in an adapted reading comprehension test. The experiment examined the plausibility of adapting a reading comprehension test for university students through textual complexity management. It involved undergraduate English-as-a-foreign-language (EFL) participants (N = 1000) with English proficiency levels ranging from A1 to C2. A 38-item reading comprehension test with textual complexity adjustment was adapted. Item fit was assessed using the Rasch model analysis. ANOVA was performed to determine which reading comprehension subsamples differed significantly, whether the difficulty level of the test confirmed that of the texts, and what cognitive process contributed the most to test difficulty. The findings generated 32 qualified items, which fit the Rasch model. Participants significantly differed in reading comprehension, indicating the test's ability to differentiate the participants based on their classification. The study carried an implication that the difficulty of reading comprehension test is not solely contingent on textual complexity but also relies on task difficulty. Therefore, teachers must pay attention to both when preparing students for a reading comprehension test.

Readability of Commonly Used Quality of Life Outcome Measures for Youth Self-Report

Self-report measures are central in capturing young people’s perspectives on mental health concerns and treatment outcomes. For children and adolescents to complete such measures meaningfully and independently, the reading difficulty must match their reading ability. Prior research suggests a frequent mismatch for mental health symptom measures. Similar analyses are lacking for measures of Quality of Life (QoL). We analysed the readability of 13 commonly used QoL self-report measures for children and adolescents aged 6 to 18 years by computing five readability formulas and a mean reading age across formulas. Across measures, the mean reading age for item sets was 10.7 years (SD = 1.2). For almost two-thirds of the questionnaires, the required reading age exceeded the minimum age of the target group by at least one year, with an average discrepancy of 3.0 years (SD = 1.2). Questionnaires with matching reading ages primarily targeted adolescents. Our study suggests a frequent mismatch between the reading difficulty of QoL self-report measures for pre-adolescent children and this group’s expected reading ability. Such discrepancies risk undermining the validity of measurement, especially where children also have learning or attention difficulties. Readability should be critically considered in measure development, as one aspect of the content validity of self-report measures for youth.

Vocabulary: Common or Basic?

Neither linguistics nor psychology offers a single, unified notion of simplicity, and therefore the simplest “core” layer of vocabulary is hard to define in theory and hard to pinpoint in practice. In section 1 we briefly survey the main approaches, and distinguish two that are highly relevant to lexicography: we will call these common and basic. In sections 2 and 3 we compare these approaches, and in section 4 we point the reader to Kolmogorov complexity, unfamiliar as it may be to most working psychologists, lexicographers, and educators, as the best formal means to deal with core vocabulary.

Developing and Establishing Content Validity of Vignettes for Health Care Education and Research

The use of vignettes in research and education is well documented. Documentation of how vignettes are developed and content validity is established, however, are rare. The purpose of this paper is to describe a method for development and establishing content validity of vignettes that can be used to assess student performance and research participant outcomes. This three-phase method was designed to validate seven vignettes for a future research project. Content validity index survey and expert panel interviews were used to inform improvements in validity. Findings showed that clinical vignettes were improved with respect to content relevance (from 0.93 to 0.96) and importance (from 0.68 to 0.93). Developing validated vignettes is essential when used to measure outcomes in education and in research. This three-phase method of vignette development and validation is feasible and effective in improving content validity of vignettes and can be used in other education and research projects.

Enhancing Software Comments Readability Using Flesch Reading Ease Score

Comments are used to explain the meaning of code and ease communications between programmers themselves, quality assurance auditors, and code reviewers. A tool has been developed to help programmers write readable comments and measure their readability level. It is used to enhance software readability by providing alternatives to both keywords and comment statements from a local database and an online dictionary. It is also a word-finding query engine for developers. Readability level is measured using three different formulas: the fog index, the Flesch reading ease score, and Flesch–Kincaid grade levels. A questionnaire has been distributed to 42 programmers and 35 students to compare the readability aspect between both new comments written by the tool and the original comments written by previous programmers and developers. Programmers stated that the comments from the proposed tool had fewer complex words and took less time to read and understand. Nevertheless, this did not significantly affect the understandability of the text, as programmers normally have quite a high level of English. However, the results from students show that the tool affects the understandability of text and the time taken to read it, while text complexity results show that the tool makes new comment text that is more readable by changing the three studied variables.

Readability of online patient education materials for parents after a failed newborn hearing screen

OBJECTIVE

A cross-sectional study design was utilized to evaluate the readability of patient education materials on the newborn hearing screen from Google and major institutions.

METHODS

The top 55 websites from the Google search "failed newborn hearing screen" and websites from major institutions (the U.S. News & World Report ranked top 10 children's hospitals, the top 5 pediatric otolaryngology fellowships as ranked by Doximity Residency Navigator, the Centers for Disease and Control, the American Academy of Pediatrics and the American Academy of Otolaryngology-Head and Neck Surgery) were compiled. Text from each website was edited to remove extraneous text. Readability grade was calculated using the Flesch-Kincaid Grade Level (FKGL), Flesch Reading Ease Score, Gunning-Fog Index (GFI), Simple Measure of Gobbledygook (SMOG), Coleman-Liau Index, and Automated Readability Index. Intra- and inter-observer reliability were assessed.

RESULTS

26 websites from Google and 29 websites from major institutions were evaluated. From Google, provider-oriented websites (n = 2) were more difficult to read than patient-oriented websites (n = 24) with statistical significance for FKGL (p < 0.001), GFI (p < 0.013) and SMOG (p < 0.001). From the major institutions, more than half were at a reading level that exceeded the average American adult with an average FKGL 9.71 ± 2.69. Intra- and inter-observer reliability were both excellent with an intra-class correlation coefficient for each readability tool ≥0.950 (p < 0.001).

CONCLUSION

Online patient education materials about the newborn hearing screen may be too difficult for the average reader. Revisions to these materials and redirection to more readable online resources may be necessary to benefit a more inclusive patient population.

CSMDSE-Cuckoo Search Based Multi Document Summary Extractor

In the current scenario, managing of a useful web of information has become a challenging issue due to a large amount of information related to many fields is online. The summarization of text is considered as one of the solutions to extract pertinent text from vast documents. Hence, a novel Cuckoo Search-based multi document summary extractor (CSMDSE) is presented to handle the multi-document summarization (MDS) problem. The proposed CSMDSE is assimilating with few other swarm-based summary extractors, such as Cat Swarm Optimization based Extractor (CSOE), Particle Swarm Optimization based Extractor (PSOE), Improved Particle Swarm Optimization based Extractor (IPSOE) and Ant Colony Optimization based Extractor (ACOE). Finally, a simulation of CSMDSE is compared with other techniques with respect to the traditional benchmark datasets for summarization problem. The experimental analysis clearly indicates CSMDSE has good performance than the other summary extractors discussed in this study.

Health Literacy Demands of Patient-Reported Evaluation Tools in Orthopedics: A Mixed-Methods Case Study

BACKGROUND

In response to an assessment of organizational health literacy practices at a major academic health center, this case study evaluated the health literacy demands of patient-reported outcome measures commonly used in orthopedic surgery practices to identify areas for improvement.

METHODS

A mixed-methods approach was used to analyze the readability and patient feedback of orthopedic patient-reported outcome materials. Qualitative results were derived from focus group notes, observations, recordings, and consensus documents. Results were combined to formulate recommendations for quality improvement.

RESULTS

Readability results indicated that narrative portions of sample patient outcome tools were written within or below the recommended eighth-grade reading level (= 5.9). However, document literacy results were higher than the recommended reading level (= 9.8). Focus group results revealed that participants had consensus on 8 of 12 plain language best practices, including use of bullet lists and jargon or technical words in both instruments.

CONCLUSIONS

Although the typical readability of both instruments was not exceedingly high, appropriate readability formula and assessment methods gave a more comprehensive assessment of true readability. In addition, participant feedback revealed the need to reduce jargon and improve formatting to lessen the health literacy demands on patients. As clinicians turn more toward patient-reported measures to assess health care quality, it is important to consider the health literacy demands that are inherent in the instruments they are given in our health systems.

The readability of scientific texts is decreasing over time

Clarity and accuracy of reporting are fundamental to the scientific process. Readability formulas can estimate how difficult a text is to read. Here, in a corpus consisting of 709,577 abstracts published between 1881 and 2015 from 123 scientific journals, we show that the readability of science is steadily decreasing. Our analyses show that this trend is indicative of a growing use of general scientific jargon. These results are concerning for scientists and for the wider public, as they impact both the reproducibility and accessibility of research findings.

Readability and Comprehensibility of Patient Information Leaflets for Antidiabetic Medications in Qatar

Background:The readability and comprehensibility of the patient information leaflets (PILs) provided with antidiabetic medications are of questionable standards; this issue negatively affects adherence to drug therapy, especially in patients with limited literacy skills. Objective: To evaluate the readability and comprehensibility of PILs supplied with medications used for the treatment of type 2 diabetes mellitus in Qatar. Methods: All PILs of the antidiabetic medications in Qatar were evaluated using the Flesch Reading Ease (FRE) score for readability. The Flesch-Kincaid Grade Level, Gunning-Fog Index, and SMOG Grading were used to estimate the comprehensibility of PILs in terms of school grade levels. Results: A total of 45 PILs were evaluated: 32 (71.1%) PILs of brand-name products and 13 (28.9%) for generics. Nine (20%) of the PILs were in English only; 8 (17.8%) were in English, Arabic, and French; and 28 (62.2%) were in English and Arabic. The mean FRE score was 37.71 (±15.85), and the most readable PIL had FRE score of 62. The mean scores for the comprehensibility evaluations were 10.96 (±2.67), 15.02 (±2.52), and 11.41 (±1.6) for the Flesch-Kincaid Grade Level, Gunning-Fog Index, and SMOG Grading, respectively. The most commonly used antidiabetic medication was metformin with 1372.9 (±552.9) as PILs' mean number of words. Conclusion: Only 2.2% of PILs had acceptable readability scores. All PILs could be comprehended by at least an 11th grade student, which exceeds the recommended grade level for health-related materials. Approximately 20% of these PILs were in English only and were not readable by most patients.

The Stoplight Method: A Qualitative Approach for Health Literacy Research

The Stoplight Feedback Toolkit was developed to engage patients with low health literacy in qualitative research, to overcome known barriers to participation, and to field test written health materials. Three focus groups of patients with low health literacy were conducted using the Stoplight methods. Participants with inadequate health literacy were identified using data from electronic health records at an academic medical center, using a validated screening question. Participants were recruited by phone using a script that was written in plain language. Three focus groups were conducted with a total of 10 unique patients. In each focus group, a facilitator read the health materials aloud to the participants and then walked them through a guided process of stoplight coding of the materials, consensus discussion, and user feedback. Color-coded materials, verbal comments and feedback, and behavioral observations were collected as qualitative data. Data were analyzed, sorted into themes, and mapped onto health literacy and plain language best practices. The Stoplight methods were successful in engaging patients with low health literacy to overcome barriers to participation, as well as in soliciting qualitative results that guided recommendations for improvement of the materials. [Health Literacy Research and Practice. 2017;1(2):e18-e22.].