ChatGPT vs. web search for patient questions: what does ChatGPT do better?

PURPOSE

Chat generative pretrained transformer (ChatGPT) has the potential to significantly impact how patients acquire medical information online. Here, we characterize the readability and appropriateness of ChatGPT responses to a range of patient questions compared to results from traditional web searches.

METHODS

Patient questions related to the published Clinical Practice Guidelines by the American Academy of Otolaryngology-Head and Neck Surgery were sourced from existing online posts. Questions were categorized using a modified Rothwell classification system into (1) fact, (2) policy, and (3) diagnosis and recommendations. These were queried using ChatGPT and traditional web search. All results were evaluated on readability (Flesch Reading Ease and Flesch-Kinkaid Grade Level) and understandability (Patient Education Materials Assessment Tool). Accuracy was assessed by two blinded clinical evaluators using a three-point ordinal scale.

RESULTS

54 questions were organized into fact (37.0%), policy (37.0%), and diagnosis (25.8%). The average readability for ChatGPT responses was lower than traditional web search (FRE: 42.3 ± 13.1 vs. 55.6 ± 10.5, p < 0.001), while the PEMAT understandability was equivalent (93.8% vs. 93.5%, p = 0.17). ChatGPT scored higher than web search for questions the 'Diagnosis' category (p < 0.01); there was no difference in questions categorized as 'Fact' (p = 0.15) or 'Policy' (p = 0.22). Additional prompting improved ChatGPT response readability (FRE 55.6 ± 13.6, p < 0.01).

CONCLUSIONS

ChatGPT outperforms web search in answering patient questions related to symptom-based diagnoses and is equivalent in providing medical facts and established policy. Appropriate prompting can further improve readability while maintaining accuracy. Further patient education is needed to relay the benefits and limitations of this technology as a source of medial information.

Medial prefrontal cortex neuropeptide Y modulates binge-like ethanol consumption in C57BL/6J mice

Neuropeptide Y (NPY) signaling via limbic NPY1 and 2 receptors (NPY1R and NPY2R, respectively) is known to modulate binge-like ethanol consumption in rodents. However, the role of NPY signaling in the medial prefrontal cortex (mPFC), which provides top-down modulation of the limbic system, is unknown. Here, we used "drinking-in-the-dark" (DID) procedures in C57BL/6J mice to address this gap in the literature. First, the impact of DID on NPY immunoreactivity (IR) was assessed in the mPFC. Next, the role of NPY1R and NPY2R signaling in the mPFC on ethanol consumption was evaluated through site-directed pharmacology. Chemogenetic inhibition of NPY1R+ neurons in the mPFC was performed to further evaluate the role of this population. To determine the potential role of NPY1R+ neurons projecting from the mPFC to the basolateral amygdala (BLA) this efferent population was selectively silenced. Three, 4-day cycles of DID reduced NPY IR in the mPFC. Intra-mPFC activation of NPY1R and antagonism of NPY2R resulted in decreased binge-like ethanol intake. Silencing of mPFC NPY1R+ neurons overall, and specifically NPY1R+ neurons projecting to the BLA, significantly reduced binge-like ethanol intake. We provide novel evidence that (1) binge-like ethanol intake reduces NPY levels in the mPFC; (2) activation of NPY1R or blockade of NPY2R reduces binge-like ethanol intake; and (3) chemogenetic inhibition of NPY1R+ neurons in the mPFC and NPY1R+ mPFC neurons projecting to the BLA blunts binge-like drinking. These observations provide the first direct evidence that NPY signaling in the mPFC modulates binge-like ethanol consumption.