Information on the Internet Regarding Orthognathic Surgery in Turkey: Is It an Adequate Guide for Potential Patients?

A comparative analysis of AI-based chatbots: Assessing data quality in orthognathic surgery related patient information

INTRODUCTION

The aim of the current study is to evaluate the quality, reliability, readability, and similarity of data provided by different AI-based chatbots in the field of orthognathic surgery.

MATERIALS AND METHODS

Guidelines on orthognathic surgery were reviewed, and a list of questions for patients to ask chatbots was produced by two reasearchers. The questions were categorized into 'General Information and Procedure' and 'Results and Recovery', with 30 questions in each category. Five different scoring criteria were used to evaluate the chatbot responses to 60 questions: Ensuring Quality Information for Patients (EQIP) tool, Reliability Scoring System (adapted from DISCERN), Global Quality Scale (GQS), Simple Measure of Gobbledygook (SMOG) and Similarity Index.

RESULTS

The highest mean values were observed in OpenEvidence for EQIP tool, SMOG, and Similarity Index, while for Reliability and GQS assessment criteria, MediSearch showed the highest values. When evaluated in terms of reliability and quality, all three AI-based chatbots demonstrated high reliability and good quality; however, they required at least a college-level education for readability based on the SMOG index. Additionally, upon assessment of similarity, ChatGPT-4 displayed high originality, while OpenEvidence exhibited a high degree of similarity.

CONCLUSION

AI-based chatbots with a variety of features have usually provided answers with high quality, reliability, and difficult readability to questions. Although the medical information in the field of orthognathic surgery provided through chatbots is of higher quality, it is recommended that individuals consult their healthcare professionals on this issue.

Quality assessment of available Internet information on early orthodontic treatment

BACKGROUND

This study aimed to evaluate the content, reliability, quality and readability of information on Internet websites about early orthodontic treatment.

METHODS

The "early orthodontic treatment" search term was individually entered into four web search engines. The content quality and reliability were reviewed with DISCERN, Journal of American Medical Association (JAMA), and Health on the Net code (HONcode) tools using the contents of websites meeting predetermined criteria. The readability of websites was evaluated with Flesch Reading Facilitate Score (FRES) and Flesch-Kincaid Grade Level (FKGL).

RESULTS

Eighty-six websites were suitable for inclusion and scoring of the 200 websites. 80.2% of websites belonged to orthodontists, 15.1% to multidisciplinary dental clinics and 4.7% to professional organizations. The mean DISCERN score of all websites (parts 1 and 2) was 27.98/75, ranging between 19 and 67. Professional organization websites had the highest scores for DISCERN criteria. Moreover, 45.3% of websites were compatible with JAMA's disclosure criterion, 7% with the currency criterion, 5.8% with the authorship criterion and 5.8% with the attribution criterion. Only three websites met all JAMA criteria, and these websites belonged to professional organizations. None of the websites had the HONcode logo. Mean FRES and FKGL were 47.6 and 11.6, respectively.

CONCLUSIONS

The quality of web-based information about early orthodontic treatment is poor, and readability is insufficient. More accurate and higher quality Internet sources are required on the web.

Quality assessment of online information on orthodontic Web sites in the United States

INTRODUCTION

The objective of this study was to assess the quality of online information on orthodontic treatment provided by orthodontic Web sites in the United States and investigate their claims.

METHODS

Three hundred and one American orthodontic Web sites were identified after an advanced Google search. Data collection included: the location of the clinic, treatment options offered, quality-of-information assessment using the DISCERN tool, and finally, claims when promoting 1 treatment option against another, as well as the presence of information on relapse risk and retention needs.

RESULTS

All Web sites belonged to private clinics, with more than half (60.5%) in a single location. Invisalign (Align Technology, Santa Clara, Calif) was the most commonly promoted treatment option (94%), followed by the full fixed appliance (FFA) (92%). The mean DISCERN total score was poor (36.78 out of 80.00), whereas the mean reliability (questions 1-8) and quality-of-information (questions 9-15) scores were 17.06 out of 40.00 and 16.85 out of 35.00, respectively. Almost one-third (28%) of the Web sites compared aligners to FFA, whereas 25% claimed that aligners are less painful than FFA, faster than FFA (14%), or give better results than FFA (1%). Almost half of the Web sites (47%) failed to display information on relapse risk and retention needs after orthodontic treatment (41%). The Web sites that displayed such info had higher DISCERN total scores (P <0.001).

CONCLUSIONS

Invisalign seems to be the treatment modality most commonly mentioned online. According to DISCERN, U.S. orthodontic Web sites display poor or fair quality information. On many Web sites, aligner treatment was compared with FFA, with some stating that aligners cause less pain than FFA or are more efficient/faster than FFA. Moreover, almost half of the American orthodontic Web sites failed to display information on the relapse risk or retention need. Display of such information can be an indicator of better-quality Web sites. There is ample room for improvement in the online information American orthodontists provide to potential patients.

Reliability of Educational Content Videos in YouTubeTM about Stainless Steel Crowns

The purpose of the present study is to evaluate the quality and educational information provided on YouTube^TM about stainless steel crowns (SSC). Videos were searched for on YouTube ^TM using keywords related to stainless steel crowns in pediatric dentistry in the Google Trends application. A total of 52 videos were available. After exclusion criteria application, 22 videos were selected for the study for analysis. To classify the video content as high or low quality, a scoring system formed of seven parameters was used. For a global evaluation of the video quality, the video information and quality index were applied. Data obtained were analyzed statistically. Only a few videos explained the importance of SSCs. Most of the videos uploaded with a high number of likes were of low quality. Very few videos were of high quality. The content quality and educational quality of videos is poor and incomplete. None of the videos explained the need for the SSC and its benefits in pediatric dentistry.

Quality and reliability of web-based information regarding restorative treatment in pediatric patients

Purpose:

The aim of the present study is to assess the quality and reliability of web-based information about restorative treatment in pediatric patients on the internet using different scales.

Materials and methods:

Websites obtained by using keywords about restorative treatment in pediatric patients on Google and Yandex were included in the study. The study was conducted in English on a total of 440 websites. Websites were evaluated using the quality criteria for consumer health information (DISCERN toolkit), Journal of American Medical Association (JAMA) benchmarks, and Health on the Net Code of Conduct Certification (HONCode).

Results:

The mean DISCERN points of the websites were moderate. Among the evaluated websites, the quality of the knowledge in 20% of the websites was low. The rate of websites with a score below 40 was 37.5%. None of them has reached an excellent score. No websites met all JAMA criteria. There was no HONCode Certificate on any website.

Conclusion:

This study showed that the quality of the web-based information about restorative treatment in pediatric patients was generally inadequate and scientifically imperfect.

Help or hinder? An assessment of the accessibility, usability, reliability and readability of disability funding website information for Australian mental health consumers

This study aimed to assess the accessibility, usability, reliability and readability of those websites most likely encountered by Australian mental health consumers when using the internet to find information regarding the National Disability Insurance Scheme (NDIS). Websites were systematically identified with 127 deemed relevant for assessment in 2018. The LIDA instrument, the Simple Measure of Gobbledygook and Flesch-Kincaid Grade Level were used to evaluate the quality of information provided to mental health consumers. The study identified mediocre results for the accessibility, usability, reliability and readability of websites as they pertain to the needs of mental health consumers. Furthermore, it was identified that mental health support websites did not provide a more appropriate online experience for mental health consumers than general information websites, despite their focus on this demographic. These findings suggest a lack of understanding regarding the needs of mental health consumers and their experience of the online environment, which may in turn affect their access to information, agency and, ultimately, their uptake of the NDIS. The establishment of guidelines around enhancements to the online environment for mental health consumers would provide an experience that instils confidence, returns dignity and aids this group in realising their personal recovery journey.

Orthodontic temporary anchorage devices: A qualitative evaluation of Internet information available to the general public

INTRODUCTION

The evaluation of online information regarding orthodontic temporary anchorage devices (TADs) is lacking despite the increase in their use by orthodontists. This cross-sectional study aimed to investigate the quality of information regarding TADs available on the Internet to the general public.

METHODS

Two search terms ("orthodontic temporary anchorage device" and "orthodontic miniscrew") were entered separately into a total of 5 search engines. The DISCERN instrument, Journal of the American Medical Association (JAMA) benchmarks, and Health on the Net Foundation Code of Conduct were used to evaluate the quality of information contained within Web sites that satisfied the inclusion and/or exclusion criteria. Web site readability was assessed via the Simple Measure of Gobbledygook and Flesch Reading Ease Score tools. Descriptive statistical analyses and Cohen's kappa intrarater reliability tests were performed.

RESULTS

Thirty-one Web sites were evaluated. Most were authored by orthodontists (77.4%) and originated from the U.S. (38.7%). The mean (standard deviation [SD]) DISCERN score was 41.87 (8.45) out of 80, with a range of 27-57. Intrarater reliability testing for DISCERN scores was excellent (0.84). Four Web sites achieved all 4 JAMA benchmarks, and 2 achieved none. Referencing of content sources throughout the Web sites scored least via DISCERN (mean 1.49 out of 5 per Web site [SD, 0.77]) and JAMA (19.35% of Web sites). One Web site contained the Health on the Net Foundation Code of Conduct seal. The mean (SD) Simple Measure of Gobbledygook score was 8.75 (1.25), with a range of 6.5-11.3. The mean (SD) Flesch Reading Ease Score was 59.81 (7.17), with a range of 47.6-73.8.

CONCLUSIONS

The quality of information related to TADs on the Internet is moderate. The usefulness of the information may be further reduced because it was beyond the readability of the average member of the general public. Web site authors should consider the use of additional expertise, quality of information tools, and readability formulas to ensure high-quality and easily readable content.

Quality of information on YouTube about artificial intelligence in dental radiology

OBJECTIVES

This study was designed to investigate Artificial Intelligence in Dental Radiology (AIDR) videos on YouTube in terms of popularity, content, reliability, and educational quality.

METHODS

Two researchers systematically searched about AIDR on YouTube on January 27, 2020, by using the terms "artificial intelligence in dental radiology," "machine learning in dental radiology," and "deep learning in dental radiology." The search was performed in English, and 60 videos for each keyword were assessed. Video source, content type, time since upload, duration, and number of views, likes, and dislikes were recorded. Video popularity was reported using Video Power Index (VPI). The accuracy and reliability of the source of information were measured using the adapted DISCERN score. The quality of the videos was measured using JAMAS and modified Global Quality Score (mGQS) and content via Total Concent Evaluation (TCE).

RESULTS

There was high interobserver agreement for DISCERN (intraclass correlation coefficient [ICC]: 0.975; 95% confidence interval [CI]: 0.957-0.985; P: 0.000; P < 0.05) and mGQS (ICC: 0.904; 95% CI: 0.841-0.943; P: 0.000; P < 0.05). Academic source videos had higher DISCERN, GQS, and TCE, revealing both reliability and quality. Also, positive relationship of VPI with mGQS (30.1%) (P: 0.035) and DISCERN (38.1%) (P: 0.007) is detected. The scores revealed 51.9% relationship between mGQS and DISCERN (P: 0.001); and educational quality predictor scores revealed 62.5% relationship between TCE and GQS (P: 0.000).

CONCLUSION

Despite the limited number of relevant videos, YouTube involves reliable and quality videos that can be used by dentists about learning AIDR.

Web-based information on orthodontic clear aligners: a qualitative and readability assessment

BACKGROUND

Data regarding the quality of information (QOI) related to orthodontic clear aligners (OCAs) on the internet are lacking. The aim of the present investigation was to assess the quality and readability of websites providing information regarding OCAs to prospective patients.

METHODS

Three search terms were entered separately into three search engines. Four validated QOI instruments were used to determine the QOI and readability of the content contained within the websites that satisfied predetermined criteria. Descriptive statistical analyses and intrarater testing for DISCERN scores were calculated.

RESULTS

Forty-nine websites satisfied inclusion/exclusion criteria. General dentists (44.9%) and specialist orthodontists (26.5%) authored the majority of websites. One website contained the Health on the Net (HON) code seal. Cohen's kappa for DISCERN scores was good (0.87). The overall mean (SD) DISCERN score was 38.55 (8.00)/80, range 26-60. The websites authored by 'Direct-to-consumer' OCA providers recorded the lowest DISCERN mean score (33/80). One website contained all four Journal of the American Medical Association (JAMA) benchmarks. The mean (SD) Flesch Reading Ease score was 53.92 (9.08), range 36-82.5.

CONCLUSIONS

The QOI of the website content related to OCAs is poor and is 'fairly difficult' to read. Website authors should consider using QOI instruments and readability tools to enable the delivery of evidence-based and easily readable information regarding OCAs to prospective patients.

Evaluation of quality and reliability of websites about orthognathic surgery using Google Trends™ application

Objectives:

Orthognathic surgery is the only treatment option for some cases in orthodontics. Two main goals were aimed in this study. The first goal was to determine the top three keywords that Google searchers look for “orthognathic surgery” using the Google Trends™ application. The second was to assess the quality and reliability of the webpages yielded by searching those four keywords on Google.

Materials and Methods:

Google Trends was searched using the keyword “Orthognathic Surgery.” Then, Google was searched using keywords: “Orthognathic surgery” and the other first three keywords. An oral surgeon and an orthodontist analyzed the quality and reliability of the websites using the DISCERN assessment tool.

Results:

The correlation between evaluators was low. Forty-six of the 70 websites were the websites of different oral and maxillofacial clinics and orthodontic clinics. The percentage of websites was 11% for personal doctor websites, 10% for academic journals, 7% for institutes, 6% for orthodontic clinics, and 1.4% for dental clinics. Overall rating scores of the websites for researchers were 2.41±1.35 for oral surgeon and 1.99±1.49 for orthodontist.

Conclusions:

The reliability and quality of the information in all of the websites was very low. The data on the websites did not have any scientific review processes such as review-peer review.

Assessment of Reliability of YouTube Videos on Orthodontics

Objective

In addition to being an entertainment channel, YouTube is also one of the most popular visual information sources today. People search YouTube to consult also on orthodontics, as well as on many other topics. The objective of the present study was to analyze the quality and reliability of information of the videos on YouTube about orthodontics.

Methods

YouTube was searched systematically by two researchers on orthodontics by using the keywords "Orthodontics," "Orthodontist," and "Orthodontic Treatment." Videos on the first three pages (60 videos) for each keyword were assessed. Researchers evaluated the reliability of the videos by using the Reliability Score (adapted from DISCERN) and the quality of the videos by using the Global Quality Score (GQS).

Results

The mean GQS results were 2.6±1.3 for videos in the "Orthodontist" group, 3.2±1.3 for videos in the "Orthodontics" group, and 2.3±1.2 for videos in the "Orthodontic Treatment" group on a 5-point scale. The Reliability Score results were 2, 2, and 1.5 for videos in the "Orthodontist," "Orthodontics," and "Orthodontic Treatment" groups, respectively, on a 5-point scale. The Intraclass Correlation Coefficient results presented a positive relationship between the researchers.

Conclusion

Owing to the lack of peer-review process and pre-upload scientific evaluation process, videos on YouTube can lead the public to misinformation.

The Accuracy of Information about Orthodontics Available on the Internet

Objective

The aim of the present study was to evaluate the content of the informative websites related to orthodontic terms in Turkish and in English.

Methods

Five different orthodontic terms ("orthodontic appliances (ortodontik aygıtlar)," "orthodontic braces (ortodontik braketler)," "orthodontic elastics (ortodontik elastikler)," "orthognathic surgery (ortognatik cerrahi)," and "orthodontic treatment (ortodontik tedavi)") both in Turkish and in English were searched using Google. There were 25 websites evaluated for each term. A total of 137 different websites were evaluated with three measurement tools: DISCERN (questionnaire, University of Oxford, 1999) (quality of information), LIDA (v1.2 Minervation, 2007) (accessibility, usability, and reliability), and AChecker (v0.1 ATutor, 2011) (accessibility).

Results

The mean overall score of the quality of information was "good" for terms in Turkish and in English. The LIDA score was classified as "moderate" for terms in Turkish and in English. More accessibility errors were found on the Turkish websites than on the English counterparts. Most of the statistical evaluations between Turkish and English terms were insignificant. However, intragroup evaluation of the terms mostly showed significant differences.

Conclusion

Accessibility, usability, and reliability; quality of information; and scores of access errors showed variations among Turkish and English sites. The collaboration of website designers and clinicians to increase the quality level of the websites is recommended.