Natural Language Processing of Patient-Experience Comments After Primary Total Knee Arthroplasty

Can ChatGPT make surgical decisions with confidence similar to experienced knee surgeons?

BACKGROUND

Unicompartmental knee replacements (UKRs) have become an increasingly attractive option for end-stage single-compartment knee osteoarthritis (OA). However, there remains controversy in patient selection. Natural language processing (NLP) is a form of artificial intelligence (AI). We aimed to determine whether general-purpose open-source natural language programs can make decisions regarding a patient's suitability for a total knee replacement (TKR) or a UKR and how confident AI NLP programs are in surgical decision making.

METHODS

We conducted a case-based cohort study using data from a separate study, where participants (73 surgeons and AI NLP programs) were presented with 32 fictitious clinical case scenarios that simulated patients with predominantly medial knee OA who would require surgery. Using the overall UKR/TKR judgments of the 73 experienced knee surgeons as the gold standard reference, we calculated the sensitivity, specificity, and positive predictive value of AI NLP programs to identify whether a patient should undergo UKR.

RESULTS

There was disagreement between the surgeons and ChatGPT in only five scenarios (15.6%). With the 73 surgeons' decision as the gold standard, the sensitivity of ChatGPT in determining whether a patient should undergo UKR was 0.91 (95% confidence interval (CI): 0.71 to 0.98). The positive predictive value for ChatGPT was 0.87 (95% CI: 0.72 to 0.94). ChatGPT was more confident in its UKR decision making (surgeon mean confidence = 1.7, ChatGPT mean confidence = 2.4).

CONCLUSIONS

It has been demonstrated that ChatGPT can make surgical decisions, and exceeded the confidence of experienced knee surgeons with substantial inter-rater agreement when deciding whether a patient was most appropriate for a UKR.

What is measured matters: A scoping review of analysis methods used for qualitative patient reported experience measure data

INTRODUCTION

Hospitals are increasingly turning to patients for valuable feedback regarding their care experience. A common method to collect this information is patient reported experience measures (PREMs) surveys. Health care workers report qualitative PREMs as more interesting, relevant, and informative than quantitative survey responses. However, a major barrier to utilising qualitative PREMs data to drive quality improvements is a lack of resources to analyse the data. This scoping review aimed to review the methods used to analyse qualitative PREMs survey data from routine hospital care.

METHODS

We utilised the JBI scoping review methodology, and searched four databases for articles from 2013 to 2023 which analysed qualitative PREMs survey data from routine care in hospitals. Study characteristics were extracted, as well as the analysis method - specifically, whether the study used traditional manual analysis methods in which the researcher reads the text and categorise the data, or automated methods utilising computers and algorithms to read and categorise the data.

RESULTS

From 960 unique articles, 123 went through full-text review and 54 were deemed eligible. 75.9 % used only manual content analysis methods to analyse the qualitative responses, 16.7 % of studies used a combination of manual and automated methods, and only 7.4 % used exclusively automated methods. Automated methods were used in 27.5 % of studies published 2019-2023, compared to 14.3 % of studies published 2013-2018. All bar one study using automated methods focused on investigating the validity of the automated methodology or used it to complement manual content analysis.

CONCLUSION

The studies included in this review show a transition from traditional time-consuming manual analyses to computerised methods enabling analysis at a larger scale. As the volume of PREMs data collected grows, efficient and effective ways to analyse qualitative PREMs data at scale are required to enable health services to capture the patient voice and drive consumer-centred improvements in care.

Large Language Model Prompting Techniques for Advancement in Clinical Medicine

Large Language Models (LLMs have the potential to revolutionize clinical medicine by enhancing healthcare access, diagnosis, surgical planning, and education. However, their utilization requires careful, prompt engineering to mitigate challenges like hallucinations and biases. Proper utilization of LLMs involves understanding foundational concepts such as tokenization, embeddings, and attention mechanisms, alongside strategic prompting techniques to ensure accurate outputs. For innovative healthcare solutions, it is essential to maintain ongoing collaboration between AI technology and medical professionals. Ethical considerations, including data security and bias mitigation, are critical to their application. By leveraging LLMs as supplementary resources in research and education, we can enhance learning and support knowledge-based inquiries, ultimately advancing the quality and accessibility of medical care. Continued research and development are necessary to fully realize the potential of LLMs in transforming healthcare.

Artificial intelligence in total and unicompartmental knee arthroplasty

The application of Artificial intelligence (AI) and machine learning (ML) tools in total (TKA) and unicompartmental knee arthroplasty (UKA) emerges with the potential to improve patient-centered decision-making and outcome prediction in orthopedics, as ML algorithms can generate patient-specific risk models. This review aims to evaluate the potential of the application of AI/ML models in the prediction of TKA outcomes and the identification of populations at risk.An extensive search in the following databases: MEDLINE, Scopus, Cinahl, Google Scholar, and EMBASE was conducted using the PIOS approach to formulate the research question. The PRISMA guideline was used for reporting the evidence of the data extracted. A modified eight-item MINORS checklist was employed for the quality assessment. The databases were screened from the inception to June 2022.Forty-four out of the 542 initially selected articles were eligible for the data analysis; 5 further articles were identified and added to the review from the PUBMED database, for a total of 49 articles included. A total of 2,595,780 patients were identified, with an overall average age of the patients of 70.2 years ± 7.9 years old. The five most common AI/ML models identified in the selected articles were: RF, in 38.77% of studies; GBM, in 36.73% of studies; ANN in 34.7% of articles; LR, in 32.65%; SVM in 26.53% of articles.This systematic review evaluated the possible uses of AI/ML models in TKA, highlighting their potential to lead to more accurate predictions, less time-consuming data processing, and improved decision-making, all while minimizing user input bias to provide risk-based patient-specific care.

Applications of Natural Language Processing for Automated Clinical Data Analysis in Orthopaedics

Natural language processing is an exciting and emerging field in health care that can transform the field of orthopaedics. It can aid in the process of automated clinical data analysis, changing the way we extract data for various purposes including research and registry formation, diagnosis, and medical billing. This scoping review will look at the various applications of NLP in orthopaedics. Specific examples of NLP applications include identification of essential data elements from surgical and imaging reports, patient feedback analysis, and use of AI conversational agents for patient engagement. We will demonstrate how NLP has proven itself to be a powerful and valuable tool. Despite these potential advantages, there are drawbacks we must consider. Concerns with data quality, bias, privacy, and accessibility may stand as barriers in the way of widespread implementation of NLP technology. As natural language processing technology continues to develop, it has the potential to revolutionize orthopaedic research and clinical practices and enhance patient outcomes.

Revealing patient-reported experiences in healthcare from social media using thedesign-acquire-process-model-analyse-visualise framework

Understanding patient experience in healthcare is increasingly important and desired by medical professionals in a patient-centred care approach. Healthcare discourse on social media presents an opportunity to gain a unique perspective on patient-reported experiences, complementing traditional survey data. These social media reports often appear as first-hand accounts of patients' journeys through the healthcare system, whose details extend beyond the confines of structured surveys and at a far larger scale than focus groups. However, in contrast with the vast presence of patient-experience data on social media and the potential benefits the data offers, it attracts comparatively little research attention due to the technical proficiency required for text analysis. In this article, we introduce the design-acquire-process-model-analyse-visualise framework to provide an overview of techniques and an approach to capture patient-reported experiences from social media data. We apply this framework in a case study on prostate cancer data from /r/ProstateCancer, demonstrate the framework's value in capturing specific aspects of patient concern (such as sexual dysfunction), provide an overview of the discourse, and show narrative and emotional progression through these stories. We anticipate this framework to apply to a wide variety of areas in healthcare, including capturing and differentiating experiences across minority groups, geographic boundaries, and types of illnesses.

Lai E, Bradford L, Bass AR, Amen TB, Russell LA, Mehta B, Parks M, Figgie M, Goodman S. Racial Differences in Patient Satisfaction With the Hospital Experience Undergoing Primary Unilateral Hip and Knee Arthroplasty: A Retrospective Study

Background

Press Ganey (PG) inpatient survey is widely used to track patient satisfaction with the hospital experience. Our aim was to use the PG survey to determine if there are racial differences in overall hospital experience and perception of nurses and surgeons following hip and knee arthroplasty.

Methods

We retrospectively analyzed Black and White patients from hip and knee arthroplasty registries from a single institution between July 2010 and February 2012. The overall assessment score for the hospital experience and perception of the nurse and surgeon questions from the PG inpatient survey were dichotomized as "not completely satisfied" or "completely satisfied". Multivariable logistic regression models were developed to determine the impact of race on the likelihood of being 'completely satisfied' in the hip and knee cohorts.

Results

There were 2517 hip and 2114 knee patients who underwent surgery and completed the PG survey, of whom 3.9% were Black and 96.0% were White. Black patients were less likely to be completely satisfied with their hospital experience compared to White patients in the hip (odds ratio 0.62, confidence interval 0.39-1.00, P = .049) and knee (odds ratio 0.52, confidence interval 0.33-0.82, P = .005) cohorts. Black patients were also less likely to be completely satisfied with multiple aspects of care they received from the nurse and surgeon in both cohorts.

Conclusions

We found that the PG Survey shows Black patients were less likely to be completely satisfied than White patients with the hospital experience, including their interactions with nurses and surgeons. More work is needed to understand this difference.

Demystifying the Role of Natural Language Processing (NLP) in Smart City Applications: Background, Motivation, Recent Advances, and Future Research Directions

Smart cities provide an efficient infrastructure for the enhancement of the quality of life of the people by aiding in fast urbanization and resource management through sustainable and scalable innovative solutions. The penetration of Information and Communication Technology (ICT) in smart cities has been a major contributor to keeping up with the agility and pace of their development. In this paper, we have explored Natural Language Processing (NLP) which is one such technical discipline that has great potential in optimizing ICT processes and has so far been kept away from the limelight. Through this study, we have established the various roles that NLP plays in building smart cities after thoroughly analyzing its architecture, background, and scope. Subsequently, we present a detailed description of NLP's recent applications in the domain of smart healthcare, smart business, and industry, smart community, smart media, smart research, and development as well as smart education accompanied by NLP's open challenges at the very end. This work aims to throw light on the potential of NLP as one of the pillars in assisting the technical advancement and realization of smart cities.

Impact of the COVID-19 Pandemic on Patient Satisfaction After Total Joint Arthroplasty

The coronavirus disease 2019 (COVID-19) pandemic impacted the inpatient experience before and after total joint arthroplasty (TJA). This study aimed to examine how these changes affected patient satisfaction following TJA as recorded by Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS) postdischarge surveys and comments at 2 large academic institutions. A retrospective review identified patients who completed HCAHPS surveys following primary and revision TJA at 2 academic institutions: 1 in a predominately rural southern state (Institution A) and 1 in a northeastern metropolitan city (Institution B). Patients were grouped by discharge date: pre-COVID-19 (April 1, 2019, to October 31, 2019) or COVID-19 affected (April 1, 2020, to October 31, 2020). Differences in demographics, survey responses, and comment sentiments and themes were collected and evaluated. The number of HCAHPS surveys completed increased between periods at Institution A but decreased at Institution B (Institution A, 61 vs 103; Institution B, 524 vs 296). Rates of top-box survey responses remained the same across the 2 periods. The number of comments decreased at Institution B (1977 vs 1012) but increased at Institution A (55 vs 88). During the COVID-19-affected period, there was a significant increase in the negative comment rate from Institution B (11.6% vs 14.8%, P=.013) and a significant decrease in the positive comment rate from Institution A (70.9% vs 44.3%, P<.001). There was an increase in negative patient sentiments following TJA during the COVID-19 pandemic as seen in qualitative comments but not quantitative responses. This suggests that certain aspects of the TJA patient experience were impacted by COVID-19. [Orthopedics. 2023;46(2):e105-e110.].

Global research landscape on artificial intelligence in arthroplasty: A bibliometric analysis

Background

Artificial intelligence (AI) has promising applications in arthroplasty. In response to the knowledge explosion resulting from the rapid growth of publications, we applied bibliometric analysis to explore the research profile and topical trends in this field.

Methods

The articles and reviews related to AI in arthroplasty were retrieved from 2000 to 2021. The Java-based Citespace, VOSviewer, R software-based Bibiometrix, and an online platform systematically evaluated publications by countries, institutions, authors, journals, references, and keywords.

Results

A total of 867 publications were included. Over the past 22 years, the number of AI-related publications in the field of arthroplasty has grown exponentially. The United States was the most productive and academically influential country. The Cleveland Clinic was the most prolific institution. Most publications were published in high academic impact journals. However, collaborative networks revealed a lack and imbalance of inter-regional, inter-institutional, and inter-author cooperation. Two emerging research areas represented the development trends: major AI subfields such as machine learning and deep learning, and the other is research related to clinical outcomes.

Conclusion

AI in arthroplasty is evolving rapidly. Collaboration between different regions and institutions should be strengthened to deepen our understanding further and exert critical implications for decision-making. Predicting clinical outcomes of arthroplasty using novel AI strategies may be a promising application in this field.

Concerns surrounding application of artificial intelligence in hip and knee arthroplasty

Literature surrounding artificial intelligence (AI)-related applications for hip and knee arthroplasty has proliferated. However, meaningful advances that fundamentally transform the practice and delivery of joint arthroplasty are yet to be realized, despite the broad range of applications as we continue to search for meaningful and appropriate use of AI. AI literature in hip and knee arthroplasty between 2018 and 2021 regarding image-based analyses, value-based care, remote patient monitoring, and augmented reality was reviewed. Concerns surrounding meaningful use and appropriate methodological approaches of AI in joint arthroplasty research are summarized. Of the 233 AI-related orthopaedics articles published, 178 (76%) constituted original research, while the rest consisted of editorials or reviews. A total of 52% of original AI-related research concerns hip and knee arthroplasty (n = 92), and a narrative review is described. Three studies were externally validated. Pitfalls surrounding present-day research include conflating vernacular (“AI/machine learning”), repackaging limited registry data, prematurely releasing internally validated prediction models, appraising model architecture instead of inputted data, withholding code, and evaluating studies using antiquated regression-based guidelines. While AI has been applied to a variety of hip and knee arthroplasty applications with limited clinical impact, the future remains promising if the question is meaningful, the methodology is rigorous and transparent, the data are rich, and the model is externally validated. Simple checkpoints for meaningful AI adoption include ensuring applications focus on: administrative support over clinical evaluation and management; necessity of the advanced model; and the novelty of the question being answered.

Cite this article: Bone Joint J 2022;104-B(12):1292–1303.

Efficacy and Applications of Artificial Intelligence and Machine Learning Analyses in Total Joint Arthroplasty: A Call for Improved Reporting

BACKGROUND

There has been a considerable increase in total joint arthroplasty (TJA) research using machine learning (ML). Therefore, the purposes of this study were to synthesize the applications and efficacies of ML reported in the TJA literature, and to assess the methodological quality of these studies.

METHODS

PubMed, OVID/MEDLINE, and Cochrane libraries were queried in January 2021 for articles regarding the use of ML in TJA. Study demographics, topic, primary and secondary outcomes, ML model development and testing, and model presentation and validation were recorded. The TRIPOD (Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis) guidelines were used to assess the methodological quality.

RESULTS

Fifty-five studies were identified: 31 investigated clinical outcomes and resource utilization; 11, activity and motion surveillance; 10, imaging detection; and 3, natural language processing. For studies reporting the area under the receiver operating characteristic curve (AUC), the median AUC (and range) was 0.80 (0.60 to 0.97) among 26 clinical outcome studies, 0.99 (0.83 to 1.00) among 6 imaging-based studies, and 0.88 (0.76 to 0.98) among 3 activity and motion surveillance studies. Twelve studies compared ML to logistic regression, with 9 (75%) reporting that ML was superior. The average number of TRIPOD guidelines met was 11.5 (range: 5 to 18), with 38 (69%) meeting greater than half of the criteria. Presentation and explanation of the full model for individual predictions and assessments of model calibration were poorly reported (<30%).

CONCLUSIONS

The performance of ML models was good to excellent when applied to a wide variety of clinically relevant outcomes in TJA. However, reporting of certain key methodological and model presentation criteria was inadequate. Despite the recent surge in TJA literature utilizing ML, the lack of consistent adherence to reporting guidelines needs to be addressed to bridge the gap between model development and clinical implementation.

Artificial intelligence in knee arthroplasty: current concept of the available clinical applications

Background

Artificial intelligence (AI) is defined as the study of algorithms that allow machines to reason and perform cognitive functions such as problem-solving, objects, images, word recognition, and decision-making. This study aimed to review the published articles and the comprehensive clinical relevance of AI-based tools used before, during, and after knee arthroplasty.

Methods

The search was conducted through PubMed, EMBASE, and MEDLINE databases from 2000 to 2021 using the 2009 Preferred Reporting Items for Systematic Reviews and Meta-Analyses Protocol (PRISMA).

Results

A total of 731 potential articles were reviewed, and 132 were included based on the inclusion criteria and exclusion criteria. Some steps of the knee arthroplasty procedure were assisted and improved by using AI-based tools. Before surgery, machine learning was used to aid surgeons in optimizing decision-making. During surgery, the robotic-assisted systems improved the accuracy of knee alignment, implant positioning, and ligamentous balance. After surgery, remote patient monitoring platforms helped to capture patients’ functional data.

Conclusion

In knee arthroplasty, the AI-based tools improve the decision-making process, surgical planning, accuracy, and repeatability of surgical procedures.

Natural Language Processing and Its Use in Orthopaedic Research

PURPOSE OF REVIEW

This review aims to demonstrate how natural language processing is used in orthopaedic research.

RECENT FINDINGS

Natural language processing is a form of artificial intelligence that involves encoding human-generated text or speech into a form which can be interpreted by computers to perform a variety of tasks. Natural language processing gathers, processes, and organizes large amounts of free-text data more efficiently than humans. In orthopaedics, it has been utilized for retrospective chart review, automated reporting of electronic health record data, analyzing operative notes and radiology reports, and patient reviews of physicians and practices. Although still in its infancy, natural language processing promises to be a valuable tool in the future of orthopaedic research. It will not eliminate the need for the essential human component of questioning involved in research, but natural language processing can improve the quality, efficiency, and thoroughness of research, thus improving patient care.