An early analysis of the cost-effectiveness of a diagnostic classifier for risk stratification of haematuria patients (DCRSHP) compared to flexible cystoscopy in the diagnosis of bladder cancer

Stratifying risk of disease in haematuria patients using machine learning techniques to improve diagnostics

Background

Detailed and invasive clinical investigations are required to identify the causes of haematuria. Highly unbalanced patient population (predominantly male) and a wide range of potential causes make the ability to correctly classify patients and identify patient-specific biomarkers a major challenge. Studies have shown that it is possible to improve the diagnosis using multi-marker analysis, even in unbalanced datasets, by applying advanced analytical methods. Here, we applied several machine learning algorithms to classify patients from the haematuria patient cohort (HaBio) by analysing multiple biomarkers and to identify the most relevant ones.

Materials and methods

We applied several classification and feature selection methods (k-means clustering, decision trees, random forest with LIME explainer and CACTUS algorithm) to stratify patients into two groups: healthy (with no clear cause of haematuria) or sick (with an identified cause of haematuria e.g., bladder cancer, or infection). The classification performance of the models was compared. Biomarkers identified as important by the algorithms were also analysed in relation to their involvement in the pathological processes.

Results

Results showed that a high unbalance in the datasets significantly affected the classification by random forest and decision trees, leading to the overestimation of the sick class and low model performance. CACTUS algorithm was more robust to the unbalance in the dataset. CACTUS obtained a balanced accuracy of 0.747 for both genders, 0.718 for females and 0.803 for males. The analysis showed that in the classification process for the whole dataset: microalbumin, male gender, and tPSA emerged as the most informative biomarkers. For males: age, microalbumin, tPSA, cystatin C, BTA, HAD and S100A4 were the most significant biomarkers while for females microalbumin, IL-8, pERK, and CXCL16.

Conclusions

CACTUS algorithm demonstrated improved performance compared with other methods such as decision trees and random forest. Additionally, we identified the most relevant biomarkers for the specific patient group, which could be considered in the future as novel biomarkers for diagnosis. Our results have the potential to inform future research and provide new personalised diagnostic approaches tailored directly to the needs of the individuals.

The role of URO17® in diagnosis and follow up of bladder cancer patients

OBJECTIVE

to evaluate the role of urinary URO17® biomarker in the detection of urothelial tumors in haematuria patients and the detection of recurrence in non-muscle invasive bladder urothelial tumors.

MATERIALS AND METHODS

Our study was formed of two cohorts of patients, group I represents patients presenting with haematuria (n = 98), while group II represents patients with known non-muscle invasive bladder cancers on their scheduled follow up cystoscopic investigation (n = 51). For both groups, patients were asked to provide urine samples before cystoscopy, either primary as part of the haematuria investigation or as a scheduled follow-up. Urine samples were sent anonymously for standard urine cytology and URO17® biomarker immunostaining. Results were compared to cystoscopic findings using Chi-square analysis and Fisher's exact test (P < 0.05).

RESULTS

Group I was formed of 98 patients, with an average age of 60 years. URO17® showed 100% sensitivity and 96.15% specificity with a negative predictive value (NPV) of 100 and a positive predictive value (PPV) of 95.83. The results showed statistical significance with P value < 0.001. Group II was formed of 51 patients, with an average age of 75 years. URO17® was shown to have a sensitivity of 85.71% and NPV of 95.45. Eleven patients of group II were on scheduled BacillusCalmette-Guerin (BCG) and another 5 received Mitomycin C (MMC). The overall results of both groups combined (n = 149) showed statistical significance between flexible cystoscopy results and the results of urinary URO17® and urine cytology.

CONCLUSION

URO17® has a potential to be a reliable test for diagnosis and follow up of urothelial cancer patients and a screening tool adjunct to flexible cystoscopy.

TRIAL REGISTRATION

Not applicable as the current study is not a clinical trial, as per according to the National Institutes of Health, "studies that involve a comparison of methods and that do not evaluate the effect of the interventions on the participant do not meet the NIH clinical trial definition."

Critical Appraisal of Decision Models Used for the Economic Evaluation of Bladder Cancer Screening and Diagnosis: A Systematic Review

BACKGROUND AND OBJECTIVE

Bladder cancer is common among current and former smokers. High bladder cancer mortality may be decreased through early diagnosis and screening. The aim of this study was to appraise decision models used for the economic evaluation of bladder cancer screening and diagnosis, and to summarise the main outcomes of these models.

METHODS

MEDLINE via PubMed, Embase, EconLit and Web of Science databases was systematically searched from January 2006 to May 2022 for modelling studies that assessed the cost effectiveness of bladder cancer screening and diagnostic interventions. Articles were appraised according to Patient, Intervention, Comparator and Outcome (PICO) characteristics, modelling methods, model structures and data sources. The quality of the studies was also appraised using the Philips checklist by two independent reviewers.

RESULTS

Searches identified 3082 potentially relevant studies, which resulted in 18 articles that met our inclusion criteria. Four of these articles were on bladder cancer screening, and the remaining 14 were diagnostic or surveillance interventions. Two of the four screening models were individual-level simulations. All screening models (n = 4, with three on a high-risk population and one on a general population) concluded that screening is either cost saving or cost effective with cost-effectiveness ratios lower than $53,000/life-years saved. Disease prevalence was a strong determinant of cost effectiveness. Diagnostic models (n = 14) assessed multiple interventions; white light cystoscopy was the most common intervention and was considered cost effective in all studies (n = 4). Screening models relied largely on published evidence generalised from other countries and did not report the validation of their predictions to external data. Almost all diagnostic models (n = 13 out of 14) had a time horizon of 5 years or less and most of the models (n = 11) did not incorporate health-related utilities. In both screening and diagnostic models, epidemiological inputs were based on expert elicitation, assumptions or international evidence of uncertain generalisability. In modelling disease, seven models did not use a standard classification system to define cancer states, others used risk-based, numerical or a Tumour, Node, Metastasis classification. Despite including certain components of disease onset or progression, no models included a complete and coherent model of the natural history of bladder cancer (i.e. simulating the progression of asymptomatic primary bladder cancer from cancer onset, i.e. in the absence of treatment).

CONCLUSIONS

The variation in natural history model structures and the lack of data for model parameterisation suggest that research in bladder cancer early detection and screening is at an early stage of development. Appropriate characterisation and analysis of uncertainty in bladder cancer models should be considered a priority.

Cost-Effectiveness of Nadofaragene Firadenovec and Pembrolizumab in Bacillus Calmette-Guérin Immunotherapy Unresponsive Non-Muscle Invasive Bladder Cancer

OBJECTIVES

Nadofaragene firadenovec is a gene therapy for bacillus Calmette-Guérin (BCG)-unresponsive non-muscle-invasive bladder cancer (NMIBC) undergoing Food and Drug Administration review. Pembrolizumab is approved for treating patients with BCG-unresponsive NMIBC with carcinoma in situ (CIS). We evaluated the cost-effectiveness of these treatments compared with a hypothetical therapeutic alternative, at a willingness-to-pay threshold of $150 000 per quality-adjusted life-year (QALY) gained, in CIS and non-CIS BCG-unresponsive NMIBC populations.

METHODS

We developed a Markov cohort simulation model with a 3-month cycle length and lifetime horizon to estimate the total costs, QALYs, and cost per additional QALY from the health sector perspective. Clinical inputs were informed by results of single-arm clinical trials evaluating the treatments, and systematic literature reviews were conducted to obtain other model inputs. Sensitivity analyses were conducted to assess uncertainty in model results.

RESULTS

Nadofaragene firadenovec, at a placeholder price 10% higher than the price of pembrolizumab, had an incremental cost-effectiveness ratio of $263 000 and $145 000 per QALY gained in CIS and non-CIS populations, respectively. Pembrolizumab had an incremental cost-effectiveness ratio of $168 000 per QALY gained for CIS. A 5.4% reduction in pembrolizumab's price would make it cost-effective. The model was sensitive to many inputs, especially to the probabilities of disease progression, initial treatment response and durability, and drug price.

CONCLUSIONS

The cost-effectiveness of nadofaragene firadenovec will depend upon its price. Pembrolizumab, although not cost-effective in our base-case analysis, is an important alternative in this population with an unmet medical need. Comparative trials of these treatments are warranted to better estimate cost-effectiveness.

Cost-Utility Analysis of Major System Change in Specialist Cancer Surgery in London, England, Using Linked Patient-Level Electronic Health Records and Difference-in-Differences Analysis

BACKGROUND

Studies have shown that centralising surgical treatment for some cancers can improve patient outcomes, but there is limited evidence of the impact on costs or health-related quality of life.

OBJECTIVES

We report the results of a cost-utility analysis of the RESPECT-21 study using difference-in-differences, which investigated the reconfiguration of specialist surgery services for four cancers in an area of London, compared to the Rest of England (ROE).

METHODS

Electronic health records data were obtained from the National Cancer Registration and Analysis Service for patients diagnosed with one of the four cancers of interest between 2012 and 2017. The analysis for each tumour type used a short-term decision tree followed by a 10-year Markov model with 6-monthly cycles. Costs were calculated by applying National Health Service (NHS) Reference Costs to patient-level hospital resource use and supplemented with published data. Cancer-specific preference-based health-related quality-of-life values were obtained from the literature to calculate quality-adjusted life-years (QALYs). Total costs and QALYs were calculated before and after the reconfiguration, in the London Cancer (LC) area and in ROE, and probabilistic sensitivity analysis was performed to illustrate the uncertainty in the results.

RESULTS

At a threshold of £30,000/QALY gained, LC reconfiguration of prostate cancer surgery services had a 79% probability of having been cost-effective compared to non-reconfigured services using difference-in-differences. The oesophago-gastric, bladder and renal reconfigurations had probabilities of 62%, 49% and 12%, respectively, of being cost-effective at the same threshold. Costs and QALYs per surgical patient increased over time for all cancers across both regions to varying degrees. Bladder cancer surgery had the smallest patient numbers and changes in costs, and QALYs were not significant. The largest improvement in outcomes was in renal cancer surgery in ROE, making the relative renal improvements in LC appear modest, and the probability of the LC reconfiguration having been cost-effective low.

CONCLUSIONS

Prostate cancer reconfigurations had the highest probability of being cost-effective. It is not clear, however, whether the prostate results can be considered in isolation, given the reconfigurations occurred simultaneously with other system changes, and healthcare delivery in the NHS is highly networked and collaborative. Routine collection of quality-of-life measures such as the EQ-5D-5L would have improved the analysis.

Cost-utility analysis of robotic-assisted radical cystectomy for bladder cancer compared to open radical cystectomy in the United Kingdom

Background

Bladder cancer is the tenth most common cancer in the United Kingdom. Currently, open radical cystectomy (ORC) is the gold standard. Due to the risk of complications and a 2.3-8% mortality rate1, there is growing interest in the use of robot-assisted radical cystectomy (RARC). The aim of this study is to perform a cost-utility analysis, comparing RARC to ORC for bladder cancer patients from the perspective of the National Health Service England.

Methods

A three-stage decision tree: surgery, post-surgery transfusions and complications, in a 90-day time horizon, was produced to simulate possible pathways of patients. The incremental cost-effectiveness ratio (ICER) was calculated based on data derived from current literature. Multiple univariate sensitivity analysis was carried out to evaluate influences of varying costs of RARC and ORC on the ICER.

Results

The ICER for RARC compared to ORC resulted in £25,536/QALY. At the lower threshold of £20,000/QALY, RARC resulted in a negative NMB (£-4,843.32) and at the upper threshold of £30,000/QALY, a positive NMB (£624.61) compared to ORC. Threshold analysis showed that the intervention costs of £13,497 and £14,403 are met at the lower and upper threshold respectively. The univariate sensitivity analysis showed that the intervention costs of RARC or ORC, and the probabilities of complications, had the greatest impact on the ICER.

Conclusion

As the resultant ICER did not fall below the £20,000/QALY threshold, our study did not provide a definitive recommendation for RARC for bladder cancer. Negative values for the NMB at the lower threshold indicated the intervention was not feasible from a cost perspective. At the upper threshold of £30,000/QALY, this situation was reversed. The intervention became cost-effective. Therefore, further research is needed to justify the intervention.

Cost-effectiveness of mass screening for dipstick hematuria in Japan

BACKGROUND

Dipstick urine tests are a simple and inexpensive method for detecting kidney and urological diseases, such as IgA nephropathy and bladder cancer. The nationwide mass screening program, Specific Health Checkup (SHC), started in Japan in 2008 and targeted all adults between 40 and 74 years of age. Dipstick urine tests for proteinuria and glucosuria are mandatory as part of the SHC, but dipstick urine tests for hematuria are not. However, the dipstick hematuria test is often administered simultaneously with these mandatory tests by some health insurers. Hematuria is common in Japanese general screening participants, particularly elderly women, and the necessity of mass screening using the dipstick hematuria test has been discussed. This study aimed to evaluate the cost-effectiveness of mass screening for dipstick hematuria tests in addition to the SHC.

METHODS

Using a decision tree and Markov modeling, we conducted a cost-effectiveness analysis from a Japanese societal perspective.

RESULTS

Compared with the current SHC, mass screening for dipstick hematuria tests, in addition to the SHC, costs less and gains more, which means cost-saving. Similar findings were observed in the sex-specific analysis.

CONCLUSION

Our results suggest that mandating the dipstick hematuria test could be justifiable as an efficient use of finite healthcare resources. The results have implications for mass screening programs not only in Japan but worldwide.

Urine TERT promoter mutations-based tumor DNA detection in patients with bladder cancer: A pilot study

Telomerase reverse transcriptase (TERT) promoter mutations are the most frequent genetic events in bladder cancer (BC). The aim of the present pilot study was to evaluate the diagnostic potential of urine TERT promoter mutations-based liquid biopsy in patients with an ongoing oncological process, as well as in post-resection patients at risk of BC recurrence. A total of 60 patients were enrolled, of whom 27 patients had histologically proven BC; 23 had no signs of BC (control group); and 10 patients underwent transurethral malignancy resection 3-6 months prior to urine donation ('second look' group). Urine TERT promoter mutations were detected using Droplet Digital PCR. Receiver operating characteristic curve analysis revealed significant diagnostic power of the present approach (area under the curve: -0.768). At the cut-off value of tumor DNA fraction 0.34%, the sensitivity and specificity were 55.56 and 100%, respectively. In the positive samples, tumor DNA fraction varied significantly from 0.59 to 48.77%. In the 'second look' group, tumor DNA was detected in 4/10 patients, highlighting the possibility of BC recurrence with its fraction ranging only from 0.90 to 6.61%. Therefore, urine TERT promoter mutations-based liquid biopsy appears to be a promising tool for BC diagnosis and surveillance. The main study will include recruitment of additional patients, extension of the mutation panel, prolonged follow-up of the post-resection patients, as well as screening of industrial workers exposed to specific carcinogens.

Integrating Early Economic Evaluation into Target Product Profile development for medical tests: advantages and potential applications

Target Product Profiles (TPPs) outline the characteristics that new health technologies require to address an unmet clinical need. To date, published TPPs for medical tests have focused on infectious diseases, mostly in the context of low- and middle-income countries. Recently, there have been calls for a broader use of TPPs as a mechanism to ensure that diagnostic innovation is aligned with clinical needs, yet the methodology underpinning TPP development remains suboptimal. Here, we propose that early economic evaluation (EEE) should be integrated within the TPP methodology to create a more rigorous framework for the development of "fit-for-purpose" tests. We discuss the potential benefits that EEE could bring to the core activities underpinning TPP development-scoping, drafting, consensus building, and updating-and argue that using EEE to help inform TPPs provides a more objective, evidence-based, and transparent approach to defining test specifications.

Early economic evaluation of an intervention to improve uptake of the NHS England Diabetes Prevention Programme

BACKGROUND

Despite reported increases in referral numbers, a large proportion of those with prediabetes still decline participation in the NHS England Diabetes Prevention Programme (NDPP). The aim of this study was to explore whether investment in interventions to improve uptake of the programme has the potential to be cost-effective.

METHODS

An early cost-utility analysis was conducted using a Markov model parameterized based on secondary data sources. We explored different uptake scenarios and the impact that this had on the maximum allowable intervention price based on cost-effectiveness at the UK NICE willingness to pay threshold of £20,000 (US$ 25,913). Value of information analyses were conducted to explore the potential value of further research to resolve uncertainty at each uptake level.

RESULTS

As uptake levels increase, the maximum allowable intervention price and overall expected value of removing decision uncertainty increases. For 5 percentage and 50 percentage points increase in uptake levels, the maximum allowable intervention price is £41.86 (US$ 54.23) and £418.59 (US$ 542.34) per person, and the overall expected value of removing decision uncertainty are £361,818,839 (US$ 468,786,625) and £1,468,712,316 (US$ 1,902,921,063) respectively.

CONCLUSION

There is headroom for investment in interventions that improve uptake to the NDPP, thereby allowing the programme itself to be delivered in a manner that remains cost-effective.

The role of URO17™ biomarker to enhance diagnosis of urothelial cancer in new hematuria patients—First European Data

Introduction and objectives

Novel biomarker research is vital for the progression of safe and thorough diagnostic medicine. There is now a need to improve the diagnosis of bladder cancer via a noninvasive urine test while balancing the risks of harm from investigational procedures, such as cystoscopy and radiological tests, against the likelihood of malignancy. We evaluate the diagnostic accuracy and sensitivity of Uro17™ urinary biomarker for the detection of urothelial cancer in hematuria patients in a prospective blinded validation study. Uro17™ is an immunobiomarker which binds to the oncoprotein Keratin 17, which is involved in the replication cycle of malignant cells. This study compared cystoscopic and histological investigations against Uro17™ results in patients being investigated for symptoms of urothelial cancer.

Materials and methods

After receiving both local and national ethics/protocol approval, 71 patients were consented and recruited into the study. All patients were scheduled to undergo cystoscopic investigation, and following recruitment, a urine sample was collected. Urine samples were anonymized and processed as per standard cytology protocols and stained using Uro17™ immunobiomarker. The pathologists assessing the results were blinded to the patient and background history, and the results were compared to the biopsy histology.

Results

The full cohort of enrolled patients consisted of 71 participants included. There were 55 males and 16 females, with an average age of 70. Thirteen were current smokers, 42 ex‐smokers, and 16 nonsmokers. The malignancies detected included both muscle‐invasive (n = 6) and non‐muscle‐invasive tumors (n = 38), and tumors of all grades and carcinoma in situ. Uro17™ was shown to have an overall sensitivity of 100% and a specificity of 92.6%, with a positive predictive value of 0.957 and negative predictive value of 1. Uro17™ investigation was positive in every case of urothelial malignancy.

Conclusions

Our current data indicates Uro17™ is a highly sensitive noninvasive bladder cancer urine detection test that can improve the diagnosis of Bladder cancer. This can further improve diagnostic capabilities in primary care, reduce the number of referrals to Urology department, and reduce the number of unnecessary invasive procedures for new patients with a suspected urinary bladder cancer.

Early Economic Evaluation of Diagnostic Technologies: Experiences of the NIHR Diagnostic Evidence Co-operatives

Diagnostic tests are expensive and time-consuming to develop. Early economic evaluation using decision modeling can reduce commercial risk by providing early evidence on cost-effectiveness. The National Institute for Health Research Diagnostic Evidence Co-operatives (DECs) was established to catalyze evidence generation for diagnostic tests by collaborating with commercial developers; DEC researchers have consequently made extensive use of early modeling. The aim of this article is to summarize the experiences of the DECs using early modeling for diagnostics. We draw on 8 case studies to illustrate the methods, highlight methodological strengths and weaknesses particular to diagnostics, and provide advice. The case studies covered diagnosis, screening, and treatment stratification. Treatment effectiveness was a crucial determinant of cost-effectiveness in all cases, but robust evidence to inform this parameter was sparse. This risked limiting the usability of the results, although characterization of this uncertainty in turn highlighted the value of further evidence generation. Researchers evaluating early models must be aware of the importance of treatment effect evidence when reviewing the cost-effectiveness of diagnostics. Researchers planning to develop an early model of a test should also 1) consult widely with clinicians to ensure the model reflects real-world patient care; 2) develop comprehensive models that can be updated as the technology develops, rather than taking a “quick and dirty” approach that may risk producing misleading results; and 3) use flexible methods of reviewing evidence and evaluating model results, to fit the needs of multiple decision makers. Decision models can provide vital information for developers at an early stage, although limited evidence mean researchers should proceed with caution.