Situating Wikipedia as a health information resource in various contexts: A scoping review

Chat Generative Pretraining Transformer Answers Patient-focused Questions in Cervical Spine Surgery

STUDY DESIGN

Review of Chat Generative Pretraining Transformer (ChatGPT) outputs to select patient-focused questions.

OBJECTIVE

We aimed to examine the quality of ChatGPT responses to cervical spine questions.

BACKGROUND

Artificial intelligence and its utilization to improve patient experience across medicine is seeing remarkable growth. One such usage is patient education. For the first time on a large scale, patients can ask targeted questions and receive similarly targeted answers. Although patients may use these resources to assist in decision-making, there still exists little data regarding their accuracy, especially within orthopedic surgery and more specifically spine surgery.

METHODS

We compiled 9 frequently asked questions cervical spine surgeons receive in the clinic to test ChatGPT's version 3.5 ability to answer a nuanced topic. Responses were reviewed by 2 independent reviewers on a Likert Scale for the accuracy of information presented (0-5 points), appropriateness in giving a specific answer (0-3 points), and readability for a layperson (0-2 points). Readability was assessed through the Flesh-Kincaid grade level analysis for the original prompt and for a second prompt asking for rephrasing at the sixth-grade reading level.

RESULTS

On average, ChatGPT's responses scored a 7.1/10. Accuracy was rated on average a 4.1/5. Appropriateness was 1.8/3. Readability was a 1.2/2. Readability was determined to be at the 13.5 grade level originally and at the 11.2 grade level after prompting.

CONCLUSIONS

ChatGPT has the capacity to be a powerful means for patients to gain important and specific information regarding their pathologies and surgical options. These responses are limited in their accuracy, and we, in addition, noted readability is not optimal for the average patient. Despite these limitations in ChatGPT's capability to answer these nuanced questions, the technology is impressive, and surgeons should be aware patients will likely increasingly rely on it.

Quality of ChatGPT Responses to Frequently Asked Questions in Carpal Tunnel Release Surgery

Background

Although demonstrating remarkable promise in other fields, the impact of artificial intelligence (including ChatGPT in hand surgery and medical practice) remains largely undetermined. In this study, we asked ChatGPT frequently asked patient-focused questions surgeons may receive in clinic from patients who have carpel tunnel syndrome (CTS) and evaluated the quality of its output.

Methods

Using ChatGPT, we asked 10 frequently asked questions that hand surgeons may receive in the clinic before carpel tunnel release (CTR) surgery. Included questions were generated from the authors' own experiences regarding conservative and operative treatment of CTS.

Results

Responses from the following 10 questions were included: (1) What is CTS and what are its signs and symptoms? (2) What are the nonsurgical options for CTS? (3) Should I get surgery for CTS? (4) What is a CTR and how is it preformed? (5) What are the differences between open and endoscopic CTR? (6) What are the risks associated with CTR and how frequently do they occur? (7) Does CTR cure CTS? (8) How much improvement in my symptoms can I expect after CTR? (9) How long is the recovery after CTR? (10) Can CTS recur after surgery?

Conclusions

Overall, the chatbot provided accurate and comprehensive information in response to most common and nuanced questions regarding CTS and CTR surgery, all in a way that would be easily understood by many patients. Importantly, the chatbot did not provide patient-specific advice and consistently advocated for consultation with a healthcare provider.

The performance of artificial intelligence large language model-linked chatbots in surgical decision-making for gastroesophageal reflux disease

BACKGROUND

Large language model (LLM)-linked chatbots may be an efficient source of clinical recommendations for healthcare providers and patients. This study evaluated the performance of LLM-linked chatbots in providing recommendations for the surgical management of gastroesophageal reflux disease (GERD).

METHODS

Nine patient cases were created based on key questions addressed by the Society of American Gastrointestinal and Endoscopic Surgeons (SAGES) guidelines for the surgical treatment of GERD. ChatGPT-3.5, ChatGPT-4, Copilot, Google Bard, and Perplexity AI were queried on November 16th, 2023, for recommendations regarding the surgical management of GERD. Accurate chatbot performance was defined as the number of responses aligning with SAGES guideline recommendations. Outcomes were reported with counts and percentages.

RESULTS

Surgeons were given accurate recommendations for the surgical management of GERD in an adult patient for 5/7 (71.4%) KQs by ChatGPT-4, 3/7 (42.9%) KQs by Copilot, 6/7 (85.7%) KQs by Google Bard, and 3/7 (42.9%) KQs by Perplexity according to the SAGES guidelines. Patients were given accurate recommendations for 3/5 (60.0%) KQs by ChatGPT-4, 2/5 (40.0%) KQs by Copilot, 4/5 (80.0%) KQs by Google Bard, and 1/5 (20.0%) KQs by Perplexity, respectively. In a pediatric patient, surgeons were given accurate recommendations for 2/3 (66.7%) KQs by ChatGPT-4, 3/3 (100.0%) KQs by Copilot, 3/3 (100.0%) KQs by Google Bard, and 2/3 (66.7%) KQs by Perplexity. Patients were given appropriate guidance for 2/2 (100.0%) KQs by ChatGPT-4, 2/2 (100.0%) KQs by Copilot, 1/2 (50.0%) KQs by Google Bard, and 1/2 (50.0%) KQs by Perplexity.

CONCLUSIONS

Gastrointestinal surgeons, gastroenterologists, and patients should recognize both the promise and pitfalls of LLM's when utilized for advice on surgical management of GERD. Additional training of LLM's using evidence-based health information is needed.

Wikipedia as an academic service-learning tool in science and technology: higher education case from Siberia

Wikipedia, the open crowdsourced encyclopedia that anyone can edit, ranks among the top ten most-visited websites globally. Its integration into university curriculum as an innovative educational tool is a slowly growing trend; however, many higher education institutions have yet to fully grasp its potential. In response, a specific optional module for Wikipedia editing, designed for the selected undergraduate science courses at the School of Advanced Studies, Russia, was implemented as an optional extra credit service-learning activity, a teaching methodology combining meaningful service to the community with curriculum-based learning. Students who chose to participate and those who preferred not to participate in the activity were invited to participate in a research project to explore their perspectives and experiences. In total, five sessions of focus group discussions were conducted with participants (12 females and 2 males) in one set and non-participants (5 females and 4 males) in another to identify students' perspectives on themes such as their interest in science, reasons for their choices, and their expectations before the activity while post-experience focus group discussions were used to identify the perspectives of participant students on themes, encompassing contribution of the service-learning activity, acquisition of new skills, and the development of prosocial behaviors. Students' opinions on integrating social responsibility topics into the curriculum were also explored. The results extracted from these focus group discussions, analyzed through consensual coding, revealed factors promoting student participation, like interest in the subject, novelty of the activity, and grade improvement opportunities, as well as factors deterring participation, such as concerns about academic benefits, workload, and time constraints. Furthermore, the results demonstrated that Wikipedia editing serves as a novel teaching methodology, promoting student learning and development in digital literacy and information literacy, which are among the twenty-first-century skills. Interestingly, at the same time, not all students could address the value of contributing to open, crowdsourced knowledge for public service or interpret this activity as an academic service-learning. These suggest that Wikipedia editing is an innovative teaching approach, fostering students' learning and development while also indicating its potential to enhance students' understanding of responsible citizenship and public service in the digital age.

Who are the "Heroes of CRISPR"? Public science communication on Wikipedia and the challenge of micro-notability

Wikipedia's influence in shaping public perceptions of science underscores the significance of scientists being recognized on the platform, as it can impact their careers. Although Wikipedia offers guidelines for determining when a scientist qualifies for their own article, it currently lacks guidance regarding whether a scientist should be acknowledged in articles related to the innovation processes to which they have contributed. To explore how Wikipedia addresses this issue of scientific "micro-notability," we introduce a digital method called Name Edit Analysis, enabling us to quantitatively and qualitatively trace mentions of scientists within Wikipedia's articles. We study two CRISPR-related Wikipedia articles and find dynamic negotiations of micro-notability as well as a surprising tension between Wikipedia's principle of safeguarding against self-promotion and the scholarly norm of "due credit." To reconcile this tension, we propose that Wikipedians and scientists collaborate to establish specific micro-notability guidelines that acknowledge scientific contributions while preventing excessive self-promotion.

Leveraging Wikipedia in undergraduate health sciences education: a key tool for information literacy and knowledge translation

Background

Academic institutions and libraries are familiar with Wikipedia. There is growing momentum in higher education for using Wikipedia as a learning tool in various contexts. These include, but are not limited to, the use of Wikipedia-based assignments to teach information literacy, science communication, evidence-based practice, and more. Although there is growing acceptance of Wikipedia's value in the classroom, there are limited exemplars available for how it is applied in undergraduate health sciences education.

Description

This program description describes a librarian instructed course in the Bachelor of Health Sciences Program at McMaster University in which students dedicate one academic term to learning about Wikipedia content production and making contributions to a health-related Wikipedia article of their choice.

Outcomes

In the five iterations of this course that have been offered, undergraduate health sciences students have made significant contributions to 25 health-related articles in Wikipedia. They have added more than 120,000 words and over 2,000 references to high-quality literature. In class, conversations emerged about the meaningfulness of the editing Wikipedia, information literacy, and knowledge translation.

Online platforms' framing around vaping

In this paper, we provide a descriptive overview of how vaping is framed differently between various online platforms (Wikipedia, Quora, Medium, Reddit, Stack Exchange, wikiHow, Facebook, and online news media). We provide an overview of >1 million posts and news articles about vaping to study the differences in framing between online platforms. Findings indicate an inconsistent framing around vaping across platforms. Stakeholders may utilize our findings to intervene around the framing of vaping and may design communications campaigns that improve the way society sees vaping, possibly aiding smoking cessation and reducing youth vaping.

Wikipedia as a tool for contemporary history of science: A case study on CRISPR

Rapid developments and methodological divides hinder the study of how scientific knowledge accumulates, consolidates and transfers to the public sphere. Our work proposes using Wikipedia, the online encyclopedia, as a historiographical source for contemporary science. We chose the high-profile field of gene editing as our test case, performing a historical analysis of the English-language Wikipedia articles on CRISPR. Using a mixed-method approach, we qualitatively and quantitatively analyzed the CRISPR article's text, sections and references, alongside 50 affiliated articles. These, we found, documented the CRISPR field's maturation from a fundamental scientific discovery to a biotechnological revolution with vast social and cultural implications. We developed automated tools to support such research and demonstrated its applicability to two other scientific fields-coronavirus and circadian clocks. Our method utilizes Wikipedia as a digital and free archive, showing it can document the incremental growth of knowledge and the manner scientific research accumulates and translates into public discourse. Using Wikipedia in this manner compliments and overcomes some issues with contemporary histories and can also augment existing bibliometric research.

Wikipedia page views for health research: a review

Wikipedia is an open-source online encyclopedia and one of the most-read sources of online health information. Likewise, Wikipedia page views have also been analyzed to inform public health services and policies. The present review analyzed 29 studies utilizing Wikipedia page views for health research. Most reviewed studies were published in recent years and emanated from high-income countries. Together with Wikipedia page views, most studies also used data from other internet sources, such as Google, Twitter, YouTube, and Reddit. The reviewed studies also explored various non-communicable diseases, infectious diseases, and health interventions to describe changes in the utilization of online health information from Wikipedia, to examine the effect of public events on public interest and information usage about health-related Wikipedia pages, to estimate and predict the incidence and prevalence of diseases, to predict data from other internet data sources, to evaluate the effectiveness of health education activities, and to explore the evolution of a health topic. Given some of the limitations in replicating some of the reviewed studies, future research can specify the specific Wikipedia page or pages analyzed, the language of the Wikipedia pages examined, dates of data collection, dates explored, type of data, and whether page views were limited to Internet users and whether web crawlers and redirects to the Wikipedia page were included. Future research can also explore public interest in other commonly read health topics available in Wikipedia, develop Wikipedia-based models that can be used to predict disease incidence and improve Wikipedia-based health education activities.

Can pharmacy students use Wikipedia as a learning resource? Critical assessment of articles on chemotherapeutic drugs

Pharmacy students tend to use Wikipedia as a quick resource of knowledge. This study aimed to evaluate the accuracy of content and readability level of Wikipedia articles on chemotherapeutics, using quality and readability tools. Using the British National Formulary (BNF-2018) and ClinicalTrials.gov, we identified 188 chemotherapeutic drugs. We randomly selected 100 drugs with an Excel randomization program. The English Wikipedia was searched for the selected 100 drugs, and prints of the identified articles were obtained. Readability was calculated with an online instrument (http://www.readabilityformulas.com/). Articles were independently scored by two researchers using the modified DISCERN tool for content assessment. The modified DISCERN scores had a median value of 24 [interquartile range (IQR) = 7.5]. Two articles (2%) had good quality (DISCERN score 36-40), thirty-eight (38%) were moderate (DISCERN 26-35), and sixty (60%) were poor in score (DISCERN ≤25). The articles covered drug indications and most side effects. However, the majority lacked information on the routes of administration, contraindications, pharmacokinetics, and mechanisms of action. We found a correlation between DISCERN scores and number of edits (P value = 0.00033, R² = 0.1238). The number of references varied from 2 to 150 (median= 17, IQR = 17). Several problems were identified in the lists of references and citations. Most articles lacked tables and figures. The readability of the articles was 14.35 ± 3.13, consistent with the readability level of university students. In conclusion, the Wikipedia articles on chemotherapeutic drugs were not written for professional pharmacy students. Although they matched the expected readability level of university students, most were incomplete and lacked essential information.NEW & NOTEWORTHY Pharmacy students use Wikipedia as a quick resource of knowledge. However, Wikipedia articles are not written for professional pharmacy students. The study shows that although Wikipedia articles on chemotherapeutic drugs matched the expected readability level of university students, most needed to be completed and lacked essential information.

The Promotion of Hearing Health through Wikipedia Campaigns: Article Quality and Reach Assessment

This case study examined the feasibility, reach, and potential impact of using Wikipedia as a tool for hearing health promotion. Activities involved editing existing Portuguese-language Wikipedia hearing health articles, as well as translating English-language hearing health articles to Portuguese during the Wiki4WorldHearingDay2019 and Wiki4YearOfSound2020 online campaigns. The Wikipedia efforts that took place in Brazil were carried out by 10 volunteer undergraduate students in Speech-Language Pathology and Audiology at the Federal University of Santa Catarina, in Brazil. Among new and existing Wikipedia articles, the group edited 37 articles, which attracted more than 220,000 views during the set tracking period. Students were responsible for 60% of the Portuguese-language edits during the Wiki4WorldHearingDay2019 campaign and more than 90% of the Portuguese-language edits during the first half of the Wiki4YearOfSound2020 campaign. Moreover, the quality indexes for pages either created or edited were improved in all situations by registering an increase rate ranging from 33% to 100%. Wikipedia-centered activities expanded the availability of quality scientific content, written in plain language, to the public. Students worked together in order to select topics, assess existing information, validate it, create new content, and share information-steps that contributed to the mission of health promotion and knowledge dissemination for the benefit of society.

Enculturating a Community of Action: Health Professions Educators' Perspectives on Teaching With Wikipedia

PURPOSE

Health professions educators are increasingly called on to engage learners in more meaningful instruction. Many have used Wikipedia to offer an applied approach to engage learners, particularly learning related to evidence-based medicine (EBM). However, little is known about the benefits and challenges of using Wikipedia as a pedagogic tool from the collective experience of educators who have sought to improve their instructional practice with it. This study aims to synthesize the perspectives of health professions education (HPE) instructors on the incorporation of Wikipedia editing into their HPE courses.

METHOD

Applying a constructivist approach, the authors conducted semistructured interviews from July to December 2020, with 17 participating HPE instructors who had substantively integrated Wikipedia into their curriculum at 13 institutions. Participants were interviewed about their experiences of integrating Wikipedia editing into their courses. Thematic analysis was conducted on resulting transcripts.

RESULTS

The authors observed 2 broad themes among participants' expressed benefits of teaching with Wikipedia. First, Wikipedia provides a meaningful instructional alternative that also helps society and develops learners' information literacy and EBM skills. Second, Wikipedia supports learners' careers and professional identity formation. Identified challenges included high effort and time, restrictive Wikipedia sourcing guidelines, and difficult interactions with stakeholders.

CONCLUSIONS

Findings build on known benefits, such as providing a real-world collaborative project that contextualizes students' learning experiences. They also echo known challenges, such as the resource-intensive nature of teaching with Wikipedia. The findings of this study reveal the potential of Wikipedia to enculturate HPE students within a situated learning context. They also present implications for HPE programs that are considering implementing Wikipedia and faculty development needed to help instructors harness crowd-sourced information tools' pedagogic opportunities as well as anticipate their challenges.

A systematic review of the effectiveness of patient education through patient portals

Objective

The objective of this study was to systematically review all literature studying the effect of patient education on patient engagement through patient portals.

Introduction

Patient portals provide patients access to health records, lab results, medication refills, educational materials, secure messaging, appointment scheduling, and telehealth visits, allowing patients to take a more active role in their health care decisions and management. A debate remains around whether these additional aids actually improve patient engagement and increase their ability to manage their own health conditions. This systematic review looks specifically at the effect of educational materials included in patient portals.

Materials and Methods

In accordance with PRISMA guidelines, the literature search was mapped across 5 databases (PubMed, CINAHL, Scopus, PsychINFO, Embase), and implemented on June 2, 2020.

Results

Fifty-two studies were included in the review. Forty-six (88.5%) reported rates of patient utilization of educational resources in the patient portal. Thirty (57.9%) shared patients' perceptions of the usefulness of the education materials. Twenty-one (40.4%) reported changes in health outcomes following educational interventions through the patient portal. This review found that efforts are indeed being made to raise awareness of educational resources in patient portals, that patients are increasingly utilizing these resources, that patients are finding them useful, and that they are improving health outcomes.

Conclusion

It seems that patient portals are becoming a powerful tool for patient education and engagement, and show promise as a means of achieving the quadruple aim of healthcare. Moving forward, research should establish more uniform methods of measurement in order to strengthen the literature surrounding the effectiveness of patient education through patient portals.

Accuracy and Utility of Internet Image Search as a Learning Tool for Retinal Pathology

Purpose  Ophthalmology residency training heavily relies on visual and pattern recognition-based learning. In parallel with traditional reference texts, online internet search via Google Image Search (GIS) is commonly used and offers an accessible fund of reference images for ophthalmology trainees seeking rapid exposure to images of retinal pathology. However, the accuracy and quality of this tool within this context is unknown. We aim to evaluate the accuracy and quality of GIS images of selected retinal pathologies. Methods  A cross-sectional study was performed of GIS of 15 common and 15 rare retinal diseases drawn from the American Academy of Ophthalmology residency textbook series. A total of 300 evaluable image results were assessed for accuracy of images and image source accountability in consultation with a vitreoretinal surgeon. Results  A total of 377 images were reviewed with 77 excluded prior to final analysis. A total of 288 (96%) search results accurately portrayed the retinal disease being searched, whereas 12 (4%) were of an erroneous diagnosis. More images of common retinal diseases were from patient education Web sites than were images of rare diseases ( p  < 0.01). Significantly more images of rare retinal diseases were found in peer-reviewed sources ( p  = 0.01). Conclusions  GIS search results yielded a modest level of accuracy for the purposes of ophthalmic education. Despite the ease and rapidity of accessing multimodal retinal imaging examples, this tool may best be suited as a supplementary resource for learning among residents due to limited accuracy, lack of sufficient supporting information, and the source Web site's focus on patient education.

What's hot and what's not in lay psychology: Wikipedia's most-viewed articles

We studied views of articles about psychology on 10 language editions of Wikipedia from July 1, 2015, to January 6, 2021. We were most interested in what psychology topics Wikipedia users wanted to read, and how the frequency of views changed during the COVID-19 pandemic and lockdowns. Our results show that the topics of interest to people seeking psychological knowledge changed during the pandemic. In addition, the interests differ noticeably among the languages. We made two important observations. The first was that during the pandemic, people in most countries looked for new ways to manage their stress without resorting to external help. This is understandable, given the increased stress of lockdown and the limited amount of professional help available. We also found that academic topics, typically covered in university classes, experienced a substantial drop in traffic, which could be indicative of issues with remote teaching.

Supplementary Information

The online version contains supplementary material available at 10.1007/s12144-022-03826-0.

[Contraception in the German-language Wikipedia: a content and quality analysis]

BACKGROUND

Adolescents and adults today often obtain information about contraception from the Internet, especially from the online encyclopedia Wikipedia. This is because Google searches usually return Wikipedia entries as top hits.

RESEARCH AIM

Against this background, the aim of the current study is to systematically analyze for the first time the content and quality of Wikipedia articles on contraceptive methods. Five central quality dimensions are examined: the degree of correctness (research question RQ1), completeness (RQ2), neutrality (RQ3), comprehensibility (RQ4), and currency (RQ5) of the contraceptive information - and on this basis the overall quality of the articles (RQ6).

MATERIALS AND METHODS

A sample of all German-language Wikipedia articles on all contraceptive methods was formed (N = 25). These articles were analyzed by three independent, trained coders using a codebook that was developed based on the current state of the research and tested for reliability. Data analysis was performed using SPSS. The study is preregistered and all data, materials, and analysis scripts are publicly available.

RESULTS

The 25 Wikipedia articles on contraceptive methods were found to vary widely in content quality. While they showed good quality on average in terms of correctness (RQ1) and neutrality (RQ3), they scored mediocre in terms of completeness (RQ2), comprehensibility (RQ4), and currency (RQ5), resulting in moderate overall quality (RQ6).

DISCUSSION

More research as well as practice measures are needed to further assess and improve the quality of contraceptive information on Wikipedia and in other social media in a more targeted way.

Citation needed? Wikipedia bibliometrics during the first wave of the COVID-19 pandemic

Background

With the COVID-19 pandemic’s outbreak, millions flocked to Wikipedia for updated information. Amid growing concerns regarding an “infodemic,” ensuring the quality of information is a crucial vector of public health. Investigating whether and how Wikipedia remained up to date and in line with science is key to formulating strategies to counter misinformation. Using citation analyses, we asked which sources informed Wikipedia’s COVID-19–related articles before and during the pandemic’s first wave (January–May 2020).

Results

We found that coronavirus-related articles referenced trusted media outlets and high-quality academic sources. Regarding academic sources, Wikipedia was found to be highly selective in terms of what science was cited. Moreover, despite a surge in COVID-19 preprints, Wikipedia had a clear preference for open-access studies published in respected journals and made little use of preprints. Building a timeline of English-language COVID-19 articles from 2001–2020 revealed a nuanced trade-off between quality and timeliness. It further showed how pre-existing articles on key topics related to the virus created a framework for integrating new knowledge. Supported by a rigid sourcing policy, this “scientific infrastructure” facilitated contextualization and regulated the influx of new information. Last, we constructed a network of DOI-Wikipedia articles, which showed the landscape of pandemic-related knowledge on Wikipedia and how academic citations create a web of shared knowledge supporting topics like COVID-19 drug development.

Conclusions

Understanding how scientific research interacts with the digital knowledge-sphere during the pandemic provides insight into how Wikipedia can facilitate access to science. It also reveals how, aided by what we term its “citizen encyclopedists,” it successfully fended off COVID-19 disinformation and how this unique model may be deployed in other contexts.

Do We Trust the Crowd? Effects of Crowdsourcing on Perceived Credibility of Online Health Information

Crowdsourcing websites such as Wikipedia have become go-to places for health information. To what extent do we trust such health content that is generated by other Internet users? Will it make a difference if such entries are curated by medical professionals? Does the affordance of crowdsourcing make users feel like they themselves could be contributors, and does that influence their credibility judgments? We explored these questions with a 2 (Crowdsourcing: absence vs. presence) × 2 (Professional source: absence vs. presence) × 2 (Message: sunscreen vs. milk) between-subjects experiment (N = 189). Two indirect paths for crowdsourcing effects were found. The crowd-as-source path suggests that crowdsourcing negatively affects content credibility through decreased source trustworthiness and information completeness. In contrast, the self-as-source path indicates that crowdsourcing elevates source trustworthiness via heightened interactivity and sense of control. Although the additional professional source raises perceived gatekeeping on the site, it does not have substantial influence on credibility judgments. Theoretical and practical implications of the findings are discussed.

Reliability of Commercial Voice Assistants' Responses to Health-Related Questions in Noncommunicable Disease Management: Factorial Experiment Assessing Response Rate and Source of Information

BACKGROUND

Noncommunicable diseases (NCDs) constitute a burden on public health. These are best controlled through self-management practices, such as self-information. Fostering patients' access to health-related information through efficient and accessible channels, such as commercial voice assistants (VAs), may support the patients' ability to make health-related decisions and manage their chronic conditions.

OBJECTIVE

This study aims to evaluate the reliability of the most common VAs (ie, Amazon Alexa, Apple Siri, and Google Assistant) in responding to questions about management of the main NCD.

METHODS

We generated health-related questions based on frequently asked questions from health organization, government, medical nonprofit, and other recognized health-related websites about conditions associated with Alzheimer's disease (AD), lung cancer (LCA), chronic obstructive pulmonary disease, diabetes mellitus (DM), cardiovascular disease, chronic kidney disease (CKD), and cerebrovascular accident (CVA). We then validated them with practicing medical specialists, selecting the 10 most frequent ones. Given the low average frequency of the AD-related questions, we excluded such questions. This resulted in a pool of 60 questions. We submitted the selected questions to VAs in a 3×3×6 fractional factorial design experiment with 3 developers (ie, Amazon, Apple, and Google), 3 modalities (ie, voice only, voice and display, display only), and 6 diseases. We assessed the rate of error-free voice responses and classified the web sources based on previous research (ie, expert, commercial, crowdsourced, or not stated).

RESULTS

Google showed the highest total response rate, followed by Amazon and Apple. Moreover, although Amazon and Apple showed a comparable response rate in both voice-and-display and voice-only modalities, Google showed a slightly higher response rate in voice only. The same pattern was observed for the rate of expert sources. When considering the response and expert source rate across diseases, we observed that although Google remained comparable, with a slight advantage for LCA and CKD, both Amazon and Apple showed the highest response rate for LCA. However, both Google and Apple showed most often expert sources for CVA, while Amazon did so for DM.

CONCLUSIONS

Google showed the highest response rate and the highest rate of expert sources, leading to the conclusion that Google Assistant would be the most reliable tool in responding to questions about NCD management. However, the rate of expert sources differed across diseases. We urge health organizations to collaborate with Google, Amazon, and Apple to allow their VAs to consistently provide reliable answers to health-related questions on NCD management across the different diseases.

Intercessory Rote Prayer, Life Longevity and the Mortality of Roman Catholic Bishops: An Exploratory Study

Based on a computational analysis of a large dataset, this study explores if there is a significant longevity effect of intercessory prayer for a named individual's well-being, if he receives a very high number of prayers per annum for an extended period. We relied on an observational cohort study, based on data from 1988 to 2018, including 857 Roman Catholic bishops, 500 Catholic priests, and 3038 male academics from six countries. We measured the covariance of the mean length of life, controlled for nationality. It was found that there is a main effect for occupation F(2, 4391) = 4.07, p = 0.017, η_p² = 0.002, with pairwise comparisons indicating significant differences between the mean life duration of bishops (M = 30,489) and of priests (M = 29,894), but none between the academic teachers (M = 30,147) and either of the other groups. A comparison analysis between bishops from the largest and the smallest dioceses showed no significant difference t(67.31) = 1.61, p = 0.11. The first analysis proved that bishops live longer than priests, but due to a marginal effect size this result should be treated with caution. No difference was found between the mean length of life of bishops from the largest and the smallest dioceses.

Wikipedia in Health Professional Schools: from an Opponent to an Ally

As an online encyclopedia, Wikipedia is the world's largest reference Web site, with 1.7 billion visits per month. Given how easy it is to access and read, students use Wikipedia globally, despite most faculty members' admonitions. Since 2013, health professional schools worldwide have incorporated Wiki-editing into their formal curricula. These courses impact students by (1) strengthening their ability to evaluate evidence-based content and (2) multiplying their contributions to society through improvements to Wikipedia articles accessed by millions. We showcase several models of incorporating Wikipedia-editing assignments into health professions education worldwide. These successful initiatives can be replicated everywhere.

Wikipedia, friend or foe regarding information on diabetic retinopathy? A content analysis in the world's leading 19 languages

Objective

To compare the completeness and quality of information about diabetic retinopathy on Wikipedia in the world’s leading spoken languages in 2020.

Design and methods

An observational, descriptive, cross-sectional study. The information on diabetic retinopathy obtained from the free encyclopedia Wikipedia^® was assessed in languages with one hundred million or more total speakers. The term "diabetic retinopathy" was accessed in the corresponding Wikipedia entry in English, while the "more languages" function gives access to other languages. The information on the sites was collected by three ophthalmologist observers. A database was created with the most important subtopics for the education of patients with diabetic retinopathy in any of its classifications, based on a 25-question survey. The results were stratified on a scale from 0 to 4. A confirming correlation was found in the statistical analysis among the observers.

Results

No language achieved the label “excellent”; 2 languages were rated as “fair “; 4 languages qualified as “substandard”; and 7 languages were scored as “poor.” No information could be found in five languages.

Conclusions

As would be expected, the quality of content is variable across different languages. However, if anyone can edit Wikipedia, health professionals can do so as well to improve the quality and quantity of information for patients.

Profile of the Users and the Most Visited Topics of a Pediatric eHealth Website

Parents need information about their children's health, and the Internet has become an essential repository for this information. However, there is almost no information about which topics are the most searched, consulted, or shared, or about the main characteristics of pediatric website users. The main aim of this research was to describe the profile of the users of a pediatric website, which topics they searched for, which topics were the most consulted, and which were the most shared. Users' demographic data were analyzed regarding their use of the Internet to search for information about health. A pediatric website for parents was analyzed. A 26-item questionnaire collected demographical features of the users of the website. Descriptive and analytical analyses were performed. Most participants used general search engines for their health searches, and the most searched keywords were prurigo, barking cough, and laryngitis. The most visited topics were unexplained fever, snots, and laryngitis. The most shared were snots, lipotomy, and dizziness. The users were mainly women (67.8%), with an average age of 38.8 years, and one or two children (89%) with a mean age of 4.6 years. The users who mainly used the Internet for health purposes were women of younger age, and with a higher educational level.

Expanding Reach of Occupational Health Knowledge: Contributing Subject-Matter Expertise to Wikipedia as a Class Assignment

The National Institute for Occupational Safety and Health (NIOSH) and several university programs have collaborated on a large effort to expand and improve occupational safety and health content in Wikipedia using a platform developed by Wiki Education. This article describes the initiative, student contributions, and evaluations of this effort by instructors from two universities between 2016 and 2020. The Wiki Education platform allowed instructors to set timelines and track students’ progress throughout the semester while students accessed training to best expand health content in Wikipedia. Students chose topics in occupational health based on their interests and by a set of topics deemed as a priority by the “WikiProject Occupational Safety and Health.” Students’ contributions were peer-reviewed by instructors, NIOSH Wikipedians-in-Residence, and traditional Wikipedians. Students presented their projects in class at the end of the semester. Students from both schools expanded 55 articles, created 8 new articles, and translated 2 articles to Spanish, adding 1270 references; these articles were viewed over 8 million times by May 2020. Feedback received from the implementation suggested that students learned about science communication and digital literacy—providing valuable content on occupational health while reducing misinformation in the public domain. The process of identifying and addressing gaps in occupational health in Wikipedia requires participation and engagement toward improving access to information that otherwise would be restricted to the scientific literature, often behind a paywall. The Wikipedia assignment proved to be an engaging approach for instruction and information literacy. It helped students improve their science communication skills and digital literacy, tools that are likely to be critical for successful communication of science in their future careers.

The accuracy and completeness of drug information in Google snippet blocks

Introduction:

Consumers commonly use the Internet for immediate drug information. In 2014, Google introduced the snippet block to programmatically search available websites to answer a question entered into the search engine without the need for the user to enter any websites. This study compared the accuracy and completeness of drug information found in Google snippet blocks to US Food and Drug Administration (FDA) medication guides.

Methods:

Ten outpatient drugs were selected from the 2018 Clinical Drugstats Database Medical Expenditure Panel Survey. Six questions in the medication guide for each drug were entered into the Google search engine to find the snippet block. The accuracy and completeness of drug information in the Google snippet block were quantified by two different pharmacists using a scoring system of 1 (less than 25% accurate/complete information) to 5 (100% accurate/complete information). Descriptive statistics were used to summarize the scores.

Results:

For five out of the six questions, the information in the Google snippets had less than 50% accuracy and completeness compared to the medication guides. The average accuracy and completeness scores of the Google snippets were highest for “What are the ingredients of [the drug]?” with scores of 3.38 (51–75%) and 3.00 (51–75%), respectively. The question on “How to take [drug]?” had the lowest score with averages of 1.00 (<25%) for both accuracy and completeness.

Conclusion:

Google snippets provide inaccurate and incomplete drug information when compared to FDA-approved drug medication guides. This aspect may cause patient harm; therefore, it is imperative for health care and health information professionals to provide reliable drug resources to patients and consumers if written information may be needed.

Wikipedia: an unexplored resource for understanding consumer health information behaviour in library and information science scholarship

Purpose

To date, health information behaviour (HIB) models have not been applied to an exploration of Wikipedia as a consumer health information resource. Wikipedia has been situated and is well established as a valuable resource for the general layperson wishing to learn more about their health or the health of a loved one. This paper aims to identify an approach to exploring the role of Wikipedia in consumer health information behaviour (CHIB) that is grounded in a conceptual framework from the library and information science (LIS) discipline.

Design/methodology/approach

The author draws on current HIB models and relevant theories from existing LIS literature and applies them to propose a new definition of CHIB. The author uses this definition to frame Wikipedia as an unexplored consumer health information resource in the LIS scholarship and suggests future directions for placing such investigations within a conceptual framework from LIS.

Findings

The paper finds that Longo's expanded conceptual model of health information-seeking behaviour (ECMHISB) could be valuable and useful for the exploration of CHIB in relation to Wikipedia's health and medical content. Due to Wikipedia's online nature, research framed by these models must acknowledge and take under consideration the digital divide phenomenon and various factors that influence an individual's place within it.

Research limitations/implications

This work builds a foundation upon which future research into the role of Wikipedia's health and medical content in CHIB can be grounded. Using Longo's model, future research might provide insight into who Wikipedia is helping and who it has left behind. LIS scholars, practicing health librarians and perhaps health workers stand to gain a deeper understanding of the potential influence of Wikipedia's health information on its consumers.

Originality/value

For LIS scholars, this paper is novel in the fact that a HIB model has not yet been applied to the study of Wikipedia's health content. This paper provides a foundation for this research.

Wikipedia for multilingual COVID-19 vaccine education at scale

We present the design of a project to develop Wikipedia content on general vaccine safety and the COVID-19 vaccines, specifically. This proposal describes what a team would need to distribute public health information in Wikipedia in multiple languages in response to a disaster or crisis, and to measure and report the communication impact of the same. Researchers at the School of Data Science at the University of Virginia made this proposal in response to a February 2021 call from a sponsor which was seeking to share public health information to respond globally to vaccine hesitancy related to the COVID-19 vaccines. This proposal was not selected for funding, and now the research team is sharing the proposal here with an open copyright license for anyone to reuse and remix. Most of the text here is from the original proposal, but there are modifications to remove the names of the funder, named partners, and for other details to make this text more reusable. The budget in this proposal has been converted from a dollar amount to equivalent descriptions in terms of labor hours, and the timeline was adapted from absolute to relative months.

Developing a scalable framework for partnerships between health agencies and the Wikimedia ecosystem

In this era of information overload and misinformation, it is a challenge to rapidly translate evidence-based health information to the public. Wikipedia is a prominent global source of health information with high traffic, multilingual coverage, and acceptable quality control practices. Viewership data following the Ebola crisis and during the COVID-19 pandemic reveals that a significant number of web users located health guidance through Wikipedia and related projects, including its media repository Wikimedia Commons and structured data complement, Wikidata.

The basic idea discussed in this paper is to increase and expedite health institutions' global reach to the general public, by developing a specific strategy to maximize the availability of focused content into Wikimedia’s public digital knowledge archives. It was conceptualized from the experiences of leading health organizations such as Cochrane, the World Health Organization (WHO) and other United Nations Organizations, Cancer Research UK, National Network of Libraries of Medicine, and Centers for Disease Control and Prevention (CDC)'s National Institute for Occupational Safety and Health (NIOSH). Each has customized strategies to integrate content in Wikipedia and evaluate responses.

We propose the development of an interactive guide on the Wikipedia and Wikidata platforms to support health agencies, health professionals and communicators in quickly distributing key messages during crisis situations. The guide aims to cover basic features of Wikipedia, including adding key health messages to Wikipedia articles, citing expert sources to facilitate fact-checking, staging text for translation into multiple languages; automating metrics reporting; sharing non-text media; anticipating offline reuse of Wikipedia content in apps or virtual assistants; structuring data for querying and reuse through Wikidata, and profiling other flagship projects from major health organizations.

In the first phase, we propose the development of a curriculum for the guide using information from prior case studies. In the second phase, the guide would be tested on select health-related topics as new case studies. In its third phase, the guide would be finalized and disseminated.

COVID-19 and the social organization of knowledge in Wikipedia: a study of social representations

Purpose

The paper aims to present the development of conceptualization of coronavirus disease 2019 (COVID-19) based on associations with other articles on English edition of Wikipedia. The main goal of the paper is to study the social organization of knowledge about COVID-19 within the Wikipedia community of practice.

Design/methodology/approach

The methodological approach taken in this study was based on the application of Moscovici's theory of social representations to Wikipedia's knowledge organization system (KOS). Internal links in the Wikipedia article about COVID-19 were considered anchors in its social representations. Each link in the introductory part of the article was considered an indicator of the semantic relationship between COVID-19 and other concepts from Wikipedia's knowledge base. The subject of this study was links extracted from all revisions of the COVID-19 article between February and September 2020. Qualitative and quantitative analyses were performed on these conceptual structures using both synchronic and diachronic approaches.

Findings

It was found that the evolution of anchors in the Wikipedia article on COVID-19 was in line with the mechanism of symbolic coping related to infectious disease. It went through stages of divergence, convergence and normalization. It shows that this mechanism governs the social organization of knowledge related to COVID-19 on Wikipedia.

Originality/value

No studies have been devoted to the image of COVID-19 as presented by the evolution of links in Wikipedia and its implications for knowledge organization (KO).

Interest on the internet for systemic lupus erythematosus: Analysis of Google Trends in Latin American countries

BACKGROUND/OBJECTIVE

Information available on the internet about Systemic Lupus Erythematosus (SLE) can influence the doctor-patient relationship. Therefore, the aim of this study was to identify the terms used for SLE on the internet.

METHODS

We analyzed the data downloaded from Google Trends, considering the term "Lupus" in a six-year web-based research. The frequency of the terms for each Pan-American country was obtained automatically from Google Trends, which reports relative search volumes or RSV (on a scale from 0 to 100) across regions.

RESULTS

We obtained a total of 67 registered terms in 18 countries. The terms were distributed into five categories. The categories with interest in all countries were "definition" and "symptoms".

CONCLUSIONS

Google Trends allows us to find useful information about SLE on the internet; once the accuracy of this information is validated, it can be used by patients, health institutions, rheumatologists and other health professionals.

Wikipedia citations: A comprehensive data set of citations with identifiers extracted from English Wikipedia

Wikipedia’s content is based on reliable and published sources. To this date, relatively little is known about what sources Wikipedia relies on, in part because extracting citations and identifying cited sources is challenging. To close this gap, we release Wikipedia Citations, a comprehensive data set of citations extracted from Wikipedia. We extracted29.3 million citations from 6.1 million English Wikipedia articles as of May 2020, and classified as being books, journal articles, or Web content. We were thus able to extract 4.0 million citations to scholarly publications with known identifiers—including DOI, PMC, PMID, and ISBN—and further equip an extra 261 thousand citations with DOIs from Crossref. As a result, we find that 6.7% of Wikipedia articles cite at least one journal article with an associated DOI, and that Wikipedia cites just 2% of all articles with a DOI currently indexed in the Web of Science. We release our code to allow the community to extend upon our work and update the data set in the future.

Correlation between flu and Wikipedia's pages visualization

Introduction:

This study aimed to assess if the frequency of the Italian general public searches for influenza, using the Wikipedia web-page, are aligned with Istituto Superiore di Sanità (ISS) influenza cases.

Materials and Methods:

The reported cases of flu were selected from October 2015 to May 2019. Wikipedia Trends was used to assess how many times a specific page was read by users; data were extracted as daily data and aggregated on a weekly basis. The following data were extracted: number of weekly views by users from the October 2015 to May 2019 of the pages: Influenza, Febbre and Tosse (Flu, Fever and Cough, in English). Cross-correlation results are obtained as product-moment correlations between the two times series.

Results:

Regarding the database with weekly data, temporal correlation was observed between the bulletin of ISS and Wikipedia search trends. The strongest correlation was at a lag of 0 for number of cases and Flu (r=0.7571), Fever and Cough (r=0.7501). The strongest correlation was at a lag of -1 for Fever and Cough (r=0.7501). The strongest correlation was at a lag of 1 for number of cases and Flu (r=0.7559), Fever and Cough (r=0.7501).

Conclusions:

A possible future application for programming and management interventions of Public Health is proposed.

Public Interest and Behavior Change in the United States Regarding Colorectal Cancer Following the Death of Chadwick Boseman: Infodemiology Investigation of Internet Search Trends Nationally and in At-Risk Areas

Background

Colorectal cancer (CRC) has the third highest cancer mortality rate in the United States. Enhanced screening has reduced mortality rates; however, certain populations remain at high risk, notably African Americans. Raising awareness among at-risk populations may lead to improved CRC outcomes. The influence of celebrity death and illness is an important driver of public awareness. As such, the death of actor Chadwick Boseman from CRC may have influenced CRC awareness.

Objective

We sought to assess the influence of Chadwick Boseman's death on public interest in CRC in the United States, evidenced by internet searches, website traffic, and donations to prominent cancer organizations.

Methods

We used an auto-regressive integrated moving average model to forecast Google searching trends for the topic "Colorectal cancer" in the United States. We performed bivariate and multivariable regressions on state-wise CRC incidence rate and percent Black population. We obtained data from the American Cancer Society (ACS) and the Colon Cancer Foundation (CCF) for information regarding changes in website traffic and donations.

Results

The expected national relative search volume (RSV) for colorectal cancer was 2.71 (95% CI 1.76-3.66), reflecting a 3590% (95% CI 2632%-5582%) increase compared to the expected values. With multivariable regression, the statewise RSV increased for each percent Black population by 1.09 (SE 0.18, P<.001), with 42% of the variance explained (P<.001). The American Cancer Society reported a 58,000% increase in CRC-related website traffic the weekend following Chadwick Boseman's death compared to the weekend before. The Colon Cancer Foundation reported a 331% increase in donations and a 144% increase in revenue in the month following Boseman's death compared to the month prior.

Conclusions

Our results suggest that Chadwick Boseman's death was associated with substantial increases in awareness of CRC. Increased awareness of CRC may support earlier detection and better prognoses.

Integrating Wikipedia editing into health professions education: a curricular inventory and review of the literature

INTRODUCTION

Wikipedia is an online encyclopedia read by millions seeking medical information. To provide health professions students with skills to critically assess, edit, and improve Wikipedia's medical content, a skillset aligned with evidence-based medicine (EBM), Wikipedia courses have been integrated into health professions schools' curriculum. This literature review and curricular inventory of Wikipedia educational initiatives provides an overview of current approaches and identifies directions for future initiatives and research.

METHODS

Five databases were searched for articles describing educational interventions to train health professional students to edit Wikipedia. Course dashboards, maintained by Wiki Education (Wiki Edu), were searched for curricular materials. From these sources, key details were extracted and synthesized, including student and instructor type, course content, educational methods, and student outcomes.

RESULTS

Six articles and 27 dashboards reported courses offered between 2015 and 2019. Courses were predominantly offered to medical and nursing students. Instructors delivered content via videos, live lectures, and online interactive modules. Course content included logistics of Wikipedia editing, EBM skills, and health literacy. All courses included assignments requiring students to edit Wikipedia independently or in groups. Limited details on assessment of student learning were available.

DISCUSSION

A small but growing number of schools are training health professions education students to improve Wikipedia's medical content. Course details are available on Wiki Edu dashboards and, to a lesser extent, in peer-reviewed publications. While more needs to be done in conducting and sharing assessment of student learning, integrating Wikipedia into health professions education has potential to facilitate learning of EBM and communication skills, improve Wikipedia's online content, and engage students with an autonomous environment while learning. Future considerations should include a thorough assessment of student learning and practices, a final review of student edits to ensure they follow Wikipedia's guidelines and are written in clear language, and improved sharing of teaching resources by instructors.

COVID-19 research in Wikipedia

Wikipedia is one of the main sources of free knowledge on the Web. During the first few months of the pandemic, over 5,200 new Wikipedia pages on COVID-19 were created, accumulating over 400 million page views by mid-June 2020. 1 At the same time, an unprecedented amount of scientific articles on COVID-19 and the ongoing pandemic have been published online. Wikipedia’s content is based on reliable sources, such as scientific literature. Given its public function, it is crucial for Wikipedia to rely on representative and reliable scientific results, especially in a time of crisis. We assess the coverage of COVID-19-related research in Wikipedia via citations to a corpus of over 160,000 articles. We find that Wikipedia editors are integrating new research at a fast pace, and have cited close to 2% of the COVID-19 literature under consideration. While doing so, they are able to provide a representative coverage of COVID-19-related research. We show that all the main topics discussed in this literature are proportionally represented from Wikipedia, after accounting for article-level effects. We further use regression analyses to model citations from Wikipedia and show that Wikipedia editors on average rely on literature that is highly cited, widely shared on social media, and peer-reviewed.

Developing an automated mechanism to identify medical articles from wikipedia for knowledge extraction

Wikipedia contains rich biomedical information that can support medical informatics studies and applications. Identifying the subset of medical articles of Wikipedia has many benefits, such as facilitating medical knowledge extraction, serving as a corpus for language modeling, or simply making the size of data easy to work with. However, due to the extremely low prevalence of medical articles in the entire Wikipedia, articles identified by generic text classifiers would be bloated by irrelevant pages. To control the false discovery rate while maintaining a high recall, we developed a mechanism that leverages the rich page elements and the connected nature of Wikipedia and uses a crawling classification strategy to achieve accurate classification. Structured assertional knowledge in Infoboxes and Wikidata items associated with the identified medical articles were also extracted. This automatic mechanism is aimed to run periodically to update the results and share them with the informatics community.

Scientometric trends for coronaviruses and other emerging viral infections

BACKGROUND

COVID-19 is the most rapidly expanding coronavirus outbreak in the past 2 decades. To provide a swift response to a novel outbreak, prior knowledge from similar outbreaks is essential.

RESULTS

Here, we study the volume of research conducted on previous coronavirus outbreaks, specifically SARS and MERS, relative to other infectious diseases by analyzing >35 million articles from the past 20 years. Our results demonstrate that previous coronavirus outbreaks have been understudied compared with other viruses. We also show that the research volume of emerging infectious diseases is very high after an outbreak and decreases drastically upon the containment of the disease. This can yield inadequate research and limited investment in gaining a full understanding of novel coronavirus management and prevention.

CONCLUSIONS

Independent of the outcome of the current COVID-19 outbreak, we believe that measures should be taken to encourage sustained research in the field.

Wikipedia: A Medical Student Educational Project to Edit Wikipedia in Preparation for Practicing Evidence-Based Pain Medicine

OBJECTIVE

Wikipedia is commonly used to acquire information about various medical conditions such as chronic pain. Ideally, better online pain management content could reduce the burden of opioid use disorders. Our goal was to improve the quality of the content available on Wikipedia to make it more accurate and applicable to medical students and the general public while training medical students to practice evidence-based medicine and critically assess their sources of information.

METHODS

An elective class in Neuroscience, Pain, and Opioids composed of 10 medical students met biweekly to discuss landmark and practice-changing research articles in the fields of acute pain, chronic pain, and opioid management. The professor chose Wikipedia articles relevant to this course. Three independent viewers analyzed the quality of citations, anecdotal medical content, and content value for both patients and medical professionals. As part of their coursework, students then edited the Wikipedia articles.

RESULTS

Although some of the Wikipedia pain topic content (6.7% ± 2.0) was anecdotal, financially biased, or inconsistent with Western Medical Practice content, overall articles included primarily high-quality citations (85.6% ± 3.1). On a 0-5 Likert scale, students felt content would be moderately helpful for both medical students/professionals (3.4 ± 0.2) and laypersons (3.5 ± 0.2). Editing and adding citations was feasible, but novel material was often reverted.

CONCLUSION

A significant amount of pain medicine content was relevant and amenable to student editing. Therefore, future use of this tactic could provide a unique opportunity to integrate evidence-based medicine into the medical curriculum and have a direct impact on the widely available medical information. Future refinement in the editorial process may also further improve online information.