Nomograms in PCa: where do we stand

Quality of information and appropriateness of ChatGPT outputs for urology patients

BACKGROUND

The proportion of health-related searches on the internet is continuously growing. ChatGPT, a natural language processing (NLP) tool created by OpenAI, has been gaining increasing user attention and can potentially be used as a source for obtaining information related to health concerns. This study aims to analyze the quality and appropriateness of ChatGPT's responses to Urology case studies compared to those of a urologist.

METHODS

Data from 100 patient case studies, comprising patient demographics, medical history, and urologic complaints, were sequentially inputted into ChatGPT, one by one. A question was posed to determine the most likely diagnosis, suggested examinations, and treatment options. The responses generated by ChatGPT were then compared to those provided by a board-certified urologist who was blinded to ChatGPT's responses and graded on a 5-point Likert scale based on accuracy, comprehensiveness, and clarity as criterias for appropriateness. The quality of information was graded based on the section 2 of the DISCERN tool and readability assessments were performed using the Flesch Reading Ease (FRE) and Flesch-Kincaid Reading Grade Level (FKGL) formulas.

RESULTS

52% of all responses were deemed appropriate. ChatGPT provided more appropriate responses for non-oncology conditions (58.5%) compared to oncology (52.6%) and emergency urology cases (11.1%) (p = 0.03). The median score of the DISCERN tool was 15 (IQR = 5.3) corresponding to a quality score of poor. The ChatGPT responses demonstrated a college graduate reading level, as indicated by the median FRE score of 18 (IQR = 21) and the median FKGL score of 15.8 (IQR = 3).

CONCLUSIONS

ChatGPT serves as an interactive tool for providing medical information online, offering the possibility of enhancing health outcomes and patient satisfaction. Nevertheless, the insufficient appropriateness and poor quality of the responses on Urology cases emphasizes the importance of thorough evaluation and use of NLP-generated outputs when addressing health-related concerns.

Quality of information and appropriateness of Open AI outputs for prostate cancer

Chat-GPT, a natural language processing (NLP) tool created by Open-AI, can potentially be used as a quick source for obtaining information related to prostate cancer. This study aims to analyze the quality and appropriateness of Chat-GPT's responses to inquiries related to prostate cancer compared to those of the European Urology Association's (EAU) 2023 prostate cancer guidelines. Overall, 195 questions were prepared according to the recommendations gathered in the prostate cancer section of the EAU 2023 Guideline. All questions were systematically presented to Chat-GPT's August 3 Version, and two expert urologists independently assessed and assigned scores ranging from 1 to 4 to each response (1: completely correct, 2: correct but inadequate, 3: a mix of correct and misleading information, and 4: completely incorrect). Sub-analysis per chapter and per grade of recommendation were performed. Overall, 195 recommendations were evaluated. Overall, 50/195 (26%) were completely correct, 51/195 (26%) correct but inadequate, 47/195 (24%) a mix of correct and misleading and 47/195 (24%) incorrect. When looking at different chapters Open AI was particularly accurate in answering questions on follow-up and QoL. Worst performance was recorded for the diagnosis and treatment chapters with respectively 19% and 30% of the answers completely incorrect. When looking at the strength of recommendation, no differences in terms of accuracy were recorded when comparing weak and strong recommendations (p > 0,05). Chat-GPT has a poor accuracy when answering questions on the PCa EAU guidelines recommendations. Future studies should assess its performance after adequate training.

External validation of an algorithm to personalize nerve sparing approaches during robot-assisted radical prostatectomy in men with unilateral high-risk prostate cancer

Limited evidence exists about preserving neurovascular bundles during radical prostatectomy (RP) for high-risk prostate cancer (HRPCa) patients. Hence, we validated an existing algorithm predicting contralateral extraprostatic extension (cEPE) risk in unilateral high-risk cases. This algorithm aims to assist in determining the suitability of unilateral nerve-sparing RP. Among 264 patients, 48 (18%) had cEPE. The risk of cECE varied: 8%, 17.2%, and 30.8% for the low, intermediate, and high-risk groups, respectively. Despite a higher risk of cECE among individuals classified as low-risk in the development group compared to the validation group, our algorithm's superiority over always/never nerve-sparing RP was reaffirmed by decision curve analysis. Therefore, we conclude that bilateral excision may not always be justified in men with unilateral HRPCa. Instead, decisions can be based on our suggested nomogram.

Development and internal validation of a nomogram predicting significant prostate cancer: Is it clinically applicable in low prevalent prostate cancer countries? A multicenter study

BACKGROUND

Accurately identifying aggressive prostate tumors and studying them as a separate outcome are urgently needed. Nomogram is a predictive tool using an algorithm, it has been widely applied in clinical practice to predict prognosis. We aimed to develop and internally validate a nomogram predicting clinically significant prostate cancer (csPCa).

METHODS

Data were retrieved from the records of the two main hospitals in Riyadh, during the period 2019-2022. Significant variables associated with csPCa cases were used to develop and internally validate a novel nomogram, utilizing the C index, and calibration curves. Decision curve analysis (DCA) was used to assess its clinical utility.

RESULTS

Prostate imaging reporting and data system (PI-RADS), smaller prostate volume, and prostate-specific antigen (PSA) > 10 ng/mL were significantly associated with the risk csPCa, respectively. The model developed by the nomogram showed an excellent accuracy for csPCa discrimination, as indicated by area under the curve (0.83), and calibration curves. DCA showed that our model was superior and surpassed all other models with a larger net benefit for various threshold probabilities. Based on our model, at a probability threshold of 30%, biopsying patients is the equivalent of a strategy that led to an absolute 5% reduction in the number of biopsies without missing any csPCa.

CONCLUSION

The developed nomogram consisting of PI-RAD, total PSA, and prostate volume showed a robust predictive capacity for csPCa before prostate biopsy that may be valuable for clinical judgment to prevent needless biopsy. Yet, the small percentage (5%) of yielded unnecessary biopsies that could be saved by using such a model, cast an important question on its merit and clinical applicability.

Benefit and harm of lymphadenectomy in intermediate risk prostate cancer: comparison of five nomograms

BACKGROUND

Pelvic lymph node dissection (PLND) is recommended method for detecting prostate cancer (PCa) nodal metastases although associated with serious complications. In this study, we aimed to assess benefit/harm of routine PLND in intermediate risk PCa patients and to compare diagnostic yield of five different nomograms in predicting lymph node invasion (LNI).

METHODS

Retrospective analysis of consecutive PCa patients with intermediate risk of biochemical recurrence who underwent open radical prostatectomy (RP) with bilateral PLND between January 2017 and December 2019 at our institution. Partin, 2012-Briganti, 2018-Briganti, Cagiannos and Memorial Sloan Kettering Cancer Center (MSKCC) values were calculated. To compare accuracy, sensitivity, specificity, and area under receiver-operating curve (AUC) were calculated and then optimal cutoff values were estimated, analyses repeated and compared. To assess benefit and harm of PLND, relative risk (RR) and number need to treat (NNT) with LNI and complications set as outcome were calculated.

RESULTS

Total 309 subjects. Average age 62.2 years, average PSA 7.2 ng/mL; 18 (5.8%) had LNI; 88 (28.5%) suffered Clavien-Dindo grade 3-5 complication. AUC for predicting LNI: 0.729 for 2012-Briganti, 0.660 for MSKCC, 0.521 for 2018-Briganti, 0.486 for Cagiannos, and 0.424 for Partin. None of pairwise AUC comparisons based on default and newly established cutoff values were statistically significant. Lowest NNT was for Partin and Cagiannos with default cutoff (≥ 5%). Risks of serious complications between higher/lower than cutoff values were non-significant across nomograms.

CONCLUSIONS

2012-Briganti nomogram outperforms, although not significantly, MSKCC, 2018-Briganti, Cagiannos, and Partin nomograms in classifying LNI in intermediate risk PCa patients. Routine PLND in these patients should be avoided, due to high rate and severity of complications.

The predictive value of machine learning and nomograms for lymph node metastasis of prostate cancer: a systematic review and meta-analysis

BACKGROUND

In clinical practice, there are currently a variety of nomograms for predicting lymph node metastasis (LNM) of prostate cancer. At the same time, some scholars have introduced machine learning (ML) into the prediction of LNM of prostate cancer. However, the predictive value of nomograms and ML remains controversial. Based on this situation, this systematic review and meta-analysis was performed to explore the predictive value of various nomograms currently recommended and newly-developed ML models for LNM in prostate cancer patients.

EVIDENCE ACQUISITION

Cochrane, PubMed, Embase, and Web of Science were searched up to November 1, 2022. The risk of bias in the included studies was evaluated using the Prediction model Risk of Bias Assessment Tool (PROBAST). The concordance index (C-index), sensitivity, and specificity were adopted to evaluate the predictive accuracy of the models.

RESULTS

Thirty-one studies (18,803 patients) were included. Seven kinds of nomograms currently recommended, dominated by Briganti nomogram or MSKCC nomogram, were covered in the included studies. For newly-developed ML models, the C-index for LNM prediction in the training set and validation set was 0.846 [95%CI (0.818, 0.873)] and 0.862 [95%CI (0.819-0.905)] respectively. Most ML models in the training set were based on Logistic Regression (LR), which had a sensitivity of 0.78 [95%CI (0.70, 0.85)] and a specificity of 0.85 [95%CI (0.77, 0.90)] in the training set, and a sensitivity of 0.81 [95%CI (0.67, 0.89)] and a specificity of 0.82 [95%CI (0.75, 0.88)] in the validation set. For the recommended nomograms, the C-index in the validation set was 0.745 [95%CI (0.701, 0.790)] for the Briganti nomogram and 0.714 [95%CI (0.662, 0.765)] for the MSKCC nomogram.

CONCLUSION

The predictive accuracy of ML is superior to existing clinically recommended nomograms, and appropriate updates can be conducted to existing nomograms according to special situations.