Influence of predicting the diagnosis from history on the accuracy of physical examination

Importance of Patient History in Artificial Intelligence-Assisted Medical Diagnosis: Comparison Study

Background

Medical history contributes approximately 80% to a diagnosis, although physical examinations and laboratory investigations increase a physician's confidence in the medical diagnosis. The concept of artificial intelligence (AI) was first proposed more than 70 years ago. Recently, its role in various fields of medicine has grown remarkably. However, no studies have evaluated the importance of patient history in AI-assisted medical diagnosis.

Objective

This study explored the contribution of patient history to AI-assisted medical diagnoses and assessed the accuracy of ChatGPT in reaching a clinical diagnosis based on the medical history provided.

Methods

Using clinical vignettes of 30 cases identified in The BMJ, we evaluated the accuracy of diagnoses generated by ChatGPT. We compared the diagnoses made by ChatGPT based solely on medical history with the correct diagnoses. We also compared the diagnoses made by ChatGPT after incorporating additional physical examination findings and laboratory data alongside history with the correct diagnoses.

Results

ChatGPT accurately diagnosed 76.6% (23/30) of the cases with only the medical history, consistent with previous research targeting physicians. We also found that this rate was 93.3% (28/30) when additional information was included.

Conclusions

Although adding additional information improves diagnostic accuracy, patient history remains a significant factor in AI-assisted medical diagnosis. Thus, when using AI in medical diagnosis, it is crucial to include pertinent and correct patient histories for an accurate diagnosis. Our findings emphasize the continued significance of patient history in clinical diagnoses in this age and highlight the need for its integration into AI-assisted medical diagnosis systems.

The effect of the auditory signal on videolaryngostroboscopy ratings and interpretation

OBJECTIVE

The primary aim of this study was to examine the effect of the videolaryngostroboscopic auditory signal on videolaryngostroboscopy (VLS) ratings and interpretation in normophonic and dysphonic speakers.

STUDY DESIGN

Prospective repeated measures design METHOD: Eight speech-language pathologists evaluated rigid VLS exams obtained from 12 dysphonic speakers with vocal fold pathology and 4 normophonic speakers with normal VLS exams. VLS exams were evaluated with the auditory signal present and absent with a washout period between rating sessions. VLS measures were obtained using the Voice-vibratory Assessment of Laryngeal Imaging (VALI) and a 100mm visual analog scale (VAS). The effects of the auditory signal and its interaction with voice quality severity on 9 VLS ratings, diagnostic billing codes, and treatment recommendations were examined.

RESULTS

There was no effect of auditory information on VLS measures or overall severity of laryngeal function evaluated using the VAS (ps > 0.05). There was a main effect of auditory information and a significant interaction with voice quality severity for only one VLS measure (non-vibrating portion-left) evaluated using the VALI (P = 0.05). Post-hoc analysis for this rating showed significant increases (t-test adjusted P < 0.05) when voice quality severity was moderate-severe (M = 4.8%; SD = 1.65%) and auditory information was present. Agreement in individual clinician's selection of diagnostic codes (73%) and treatment recommendations (65.6%) when auditory cues were present and absent was moderate to high.

CONCLUSION

The presence of the videolaryngostroboscopic auditory signal had a minimal effect on VLS ratings, treatment recommendations, or diagnostic billing codes.

Improving decision accuracy using a clinical decision support system for medical students during history-taking: a randomized clinical trial

BACKGROUND

A clinical diagnostic support system (CDSS) can support medical students and physicians in providing evidence-based care. In this study, we investigate diagnostic accuracy based on the history of present illness between groups of medical students using a CDSS, Google, and neither (control). Further, the degree of diagnostic accuracy of medical students using a CDSS is compared with that of residents using neither a CDSS nor Google.

METHODS

This study is a randomized educational trial. The participants comprised 64 medical students and 13 residents who rotated in the Department of General Medicine at Chiba University Hospital from May to December 2020. The medical students were randomly divided into the CDSS group (n = 22), Google group (n = 22), and control group (n = 20). Participants were asked to provide the three most likely diagnoses for 20 cases, mainly a history of a present illness (10 common and 10 emergent diseases). Each correct diagnosis was awarded 1 point (maximum 20 points). The mean scores of the three medical student groups were compared using a one-way analysis of variance. Furthermore, the mean scores of the CDSS, Google, and residents' (without CDSS or Google) groups were compared.

RESULTS

The mean scores of the CDSS (12.0 ± 1.3) and Google (11.9 ± 1.1) groups were significantly higher than those of the control group (9.5 ± 1.7; p = 0.02 and p = 0.03, respectively). The residents' group's mean score (14.7 ± 1.4) was higher than the mean scores of the CDSS and Google groups (p = 0.01). Regarding common disease cases, the mean scores were 7.4 ± 0.7, 7.1 ± 0.7, and 8.2 ± 0.7 for the CDSS, Google, and residents' groups, respectively. There were no significant differences in mean scores (p = 0.1).

CONCLUSIONS

Medical students who used the CDSS and Google were able to list differential diagnoses more accurately than those using neither. Furthermore, they could make the same level of differential diagnoses as residents in the context of common diseases.

TRIAL REGISTRATION

This study was retrospectively registered with the University Hospital Medical Information Network Clinical Trials Registry on 24/12/2020 (unique trial number: UMIN000042831).

The effects of simple graphical and mental visualization of lung sounds in teaching lung auscultation during clinical clerkship: A preliminary study

INTRODUCTION

The study aimed to evaluate visualization-based training's effects on lung auscultation during clinical clerkship (CC) in the Department of Respiratory Medicine on student skills and confidence.

METHODS

The study period was December 2020-November 2021. Overall, 65 students attended a lecture on lung auscultation featuring a simulator (Mr. Lung™). Among them, 35 (visualization group) received additional training wherein they were asked to mentally visualize lung sounds using a graphical visualized lung sounds diagram as an example. All students answered questions on their self-efficacy regarding lung auscultation before and after four weeks of CC. They also took a lung auscultation test with the simulator at the beginning of CC (pre-test) and on the last day of the third week (post-test) (maximum score: 25). We compared the answers in the questionnaire and the test scores between the visualization group and students who only attended the lecture (control group, n = 30). The Wilcoxon signed-rank test and analysis of covariance were used to compare the answers to the questionnaire about confidence in lung auscultation and the scores of the lung auscultation tests before and after the training.

RESULTS

Confidence in auscultation of lung sounds significantly increased in both groups (five-point Likert scale, visualization group: pre-questionnaire median 1 [Interquartile range 1] to post-questionnaire 3 [1], p<0.001; control group: 2 [1] to 3 [1], p<0.001) and was significantly higher in the visualization than in the control group. Test scores increased in both groups (visualization group: pre-test 11 [2] to post-test 15 [4], p<0.001; control group: 11 [5] to 14 [4], p<0.001). However, there were no differences between both groups' pre and post-tests scores (p = 0.623).

CONCLUSION

Visualizing lung sounds may increase medical students' confidence in their lung auscultation skills; this may reduce their resistance to lung auscultation and encourage the repeated auscultation necessary to further improve their long-term auscultation abilities.

The Effect of Clinical History on Diagnostic Imaging Interpretation - A Systematic Review

RATIONALE AND OBJECTIVES

To provide updated information on the effect of clinical history on diagnostic image interpretation and to provide study methodology and design recommendations for future studies assessing the effect of clinical history on diagnostic image performance.

MATERIALS AND METHODS

A literature search of Medline, Embase, Scopus, Web of Science, and the Cochrane Central Register of Controlled Trials (CENTRAL) databases was conducted from database inception to July 21, 2020. Studies comparing diagnostic imaging performance with and without clinical history, using observers reading images under both conditions that used an independent reference standard were included.

RESULTS

Twenty-two studies met the inclusion criteria, with 15 showing clinical history improved diagnostic performance. One study reported a decrease in diagnostic performance with clinical history and the remaining six studies found no significant change in performance. Two studies used the free response paradigm with both reporting clinical history increased location sensitivity, decreased specificity and had no overall change in diagnostic performance. The disease spectrum of included cases was largely unreported and a balanced reading design was not used in 19 studies.

CONCLUSION

Most published studies found that clinical history improved diagnostic performance. More recent studies accounting for abnormality location and multiple abnormalities showed an increase in false positives and no significant change in overall diagnostic performance with clinical history.

Correlation of student performance on clerkship with quality of medical chart documentation in a simulation setting

BACKGROUND

Medical chart documentation is an essential skill acquired in a clinical clerkship (CC). However, the utility of medical chart writing simulations as a component of the objective structured clinical examination (OSCE) has not been sufficiently evaluated. In this study, medical chart documentation in several clinical simulation settings was performed as part of the OSCE, and its correlation with CC performance was evaluated.

METHODS

We created a clinical situation video and images involving the acquisition of informed consent, cardiopulmonary resuscitation, and diagnostic imaging in the emergency department, and assessed medical chart documentation performance by medical students as part of the OSCE. Evaluations were conducted utilizing original checklist (0-10 point). We also analyzed the correlation between medical chart documentation OSCE scores and CC performance of 120 medical students who performed their CC in 2019 as 5th year students and took the Post-CC OSCE in 2020 as 6th year students.

RESULTS

Of the OSCE components, scores for the acquisition of informed consent and resuscitation showed significant correlations with CC performance (P<0.001 for each). In contrast, scores for diagnostic imaging showed a slightly positive, but non-significant, correlation with CC performance (P = 0.107). Overall scores for OSCE showed a significant correlation with CC performance (P<0.001).

CONCLUSION

We conducted a correlation analysis of CC performance and the quality of medical chart documentation in a simulation setting. Our results suggest that medical chart documentation can be one possible alternative component in the OSCE.

Development of video otoscopy quiz using a smartphone adaptable otoscope

Background

Otoscopy examination can be challenging. Traditional teaching uses still image illustrations. Newer attempts use video samples to simulate the otoscopy exam which is a dynamic process.

Aims/Objective: To assess whether recorded otoscopy videos from a smartphone adaptable otoscope can be used to develop a video-based otoscopy quiz which may be used for instructing and familiarizing participants to normal anatomy and pathologic ear conditions. To use this quiz to assess current pediatric residents’ competency of common otoscopy diagnosis.

Method and materials

This study was conducted in 2018. Video samples of ear pathology were collected at the Albany Medical Center using a smartphone adaptable otoscope- Cellscope. The videos were used to create a video otoscopy quiz (VOQ) without clinical vignettes. 45 pediatric residents from 3 academic institutions were evaluated with the quiz.

Results

The weighted mean for the VOQ was 66.90% (95%CI 58.89%–68.42%). The breakdown by questions are: myringosclerosis 72.88%, retraction pocket 80.65%, cholesteatoma 42.22%, hemotympanum 75.04%, tympanic membrane perforation 79.62%, cerumen impaction 95.46%, otitis externa 52.54%, otitis media with effusion 63.30%, acute otitis media 75.55%, normal ear 36.39%.

Conclusion

We found that videos of otoscopy exams can be obtained with a smartphone adaptable otoscope and validated to develop a video-based quiz, which may be used to supplement otoscopic instruction. Following our testing process, we found pediatric residents are relatively well equipped to identify ear pathology on VOQ.

Relationships between objective structured clinical examination, computer-based testing, and clinical clerkship performance in Japanese medical students

Background

It is unclear how comprehensive evaluations conducted prior to clinical clerkships (CC), such as the objective structured clinical examination (OSCE) and computer-based testing (CBT), reflect the performance of medical students in CC. Here we retrospectively analyzed correlations between OSCE and CBT scores and CC performance.

Methods

Ethical approval was obtained from our institutional review board. We analyzed correlations between OSCE and CBT scores and CC performance in 94 medical students who took the OSCE and CBT in 2017 when they were 4th year students, and who participated in the basic CC in 2018 when they were 5th year students.

Results

Total scores for OSCE and CBT were significantly correlated with CC performance (P<0.001, each). More specifically, medical interview and chest examination components of the OSCE were significantly correlated with CC performance (P = 0.001, each), while the remaining five components of the OSCE were not.

Conclusion

Our findings suggest that the OSCE and CBT play important roles in predicting CC performance in Japanese medical education context. Among OSCE components, medical interview and chest examination were suggested to be important for predicting CC performance.

Physical examination competence as a predictor of diagnostic reasoning among new advanced practice nurse students: a cross-sectional study

Aim: To investigate levels of diagnostic reasoning among new APN students as well as students' characteristic affecting such reasoning levels.Design: a descriptive correlational design.Methods: One hundred fourteen APN students in an advanced health assessment course self-assessed their competene in performing examination. The students' self-reported Diagnostic Reasoning Assessment scores were used to categorize them into pre-novice, novice, advanced beginner, and competent levels. Ordinal logistic regression analysis was used.Results: Only physical examination competence significantly predicted the students' diagnostic reasoning levels. Being in a higher diagnostic reasoning level relative to being in a pre-novice, novice, or advanced beginner level was 16.1, 2.6, and 1.9 times greater in the physical examination competence, respectively.Conclusions: Promoting physical examination competence can be an effective teaching strategy for enhancing the diagnostic reasoning of newly enrolled APN students, especially for students who have poor diagnostic reasoning performance.