A new logistic regression approach for the evaluation of diagnostic test results

Combination prediction method of students' performance based on ant colony algorithm

Students' performance is an important factor for the evaluation of teaching quality in colleges. The prediction and analysis of students' performance can guide students' learning in time. Aiming at the low accuracy problem of single model in students' performance prediction, a combination prediction method is put forward based on ant colony algorithm. First, considering the characteristics of students' learning behavior and the characteristics of the models, decision tree (DT), support vector regression (SVR) and BP neural network (BP) are selected to establish three prediction models. Then, an ant colony algorithm (ACO) is proposed to calculate the weight of each model of the combination prediction model. The combination prediction method was compared with the single Machine learning (ML) models and other methods in terms of accuracy and running time. The combination prediction model with mean square error (MSE) of 0.0089 has higher performance than DT with MSE of 0.0326, SVR with MSE of 0.0229 and BP with MSE of 0.0148. To investigate the efficacy of the combination prediction model, other prediction models are used for a comparative study. The combination prediction model with MSE of 0.0089 has higher performance than GS-XGBoost with MSE of 0.0131, PSO-SVR with MSE of 0.0117 and IDA-SVR with MSE of 0.0092. Meanwhile, the running speed of the combination prediction model is also faster than the above three methods.

Comparing diagnostic performance of Cantonese-Chinese version of Rome IV criteria and a short Reference Standard for functional dyspepsia in China

INTRODUCTION

Functional dyspepsia (FD) is diagnosed based on self-reported symptoms and negative upper gastrointestinal endoscopic findings. The Rome criteria were not adopted as a diagnostic instrument in clinical guidelines due to their complexity. Different guidelines used relatively simple symptom assessment schemes with contents that vary significantly. A previously evaluated short Reference Standard may serve as a more standardised tool for guidelines. We evaluated its diagnostic accuracy against the Rome IV criteria in a cross-sectional study in Hong Kong.

METHODS

A total of 220 dyspeptic patients sampled consecutively from a tertiary hospital and the community completed the Rome IV diagnostic questionnaire, which was translated into Cantonese-Chinese, and the Reference Standard. Sensitivity, specificity, positive and negative likelihood ratios (LRs), and area under the receiver operating characteristics curve (AUC), with 95% confidence intervals (CIs), were calculated.

RESULTS

Among the participants, 160 (72.7%) fulfilled the Reference Standard with negative upper gastrointestinal endoscopic results. The Reference Standard identified patients with Rome IV-defined FD with 91.1% (95% CI 82.6%-96.4%) sensitivity and 37.6% (95% CI 29.6%-46.1%) specificity. The positive and negative LRs were 1.46 (95% CI 1.26-1.69) and 0.24 (95% CI 0.11-0.49), respectively. The AUC value was 0.64 (95% CI 0.59-0.69).

CONCLUSIONS

The Reference Standard can rule out patients without Rome IV-defined FD. It may be used as an initial screening tool for FD in settings where the use of the Rome IV criteria is impractical. It may also provide a uniform definition and diagnostic rule for future updates of clinical guidelines.

Development and internal validation of simplified predictive scoring (ICU-SEPSA score) for mortality in patients with multidrug resistant infection

Background: Mortality from multidrug-resistant (MDR) pathogens is an urgent healthcare crisis worldwide. At present we do not have any simplified screening tools to predict the risk of mortality associated with MDR infections. The aim of this study was to develop a screening tool to predict mortality in patients with multidrug-resistant organisms.

Methods: A retrospective cohort study to evaluate mortality risks in patients with MDR infections was conducted at Phrae Hospital. Univariable and multivariable analyses were used to classify possible risk factors. The model performance was internally validated utilizing the mean of three measures of discrimination corrected by the optimism using a 1000-bootstrap procedure. The coefficients were transformed into item scores by dividing each coefficient with the lowest coefficient and then rounding to the most adjacent number. The area under the receiver operating characteristic curve (AuROC) was used to determine the performance of the model.

Results: Between 1 October 2018 and 30 September 2020, a total of 504 patients with MDR infections were enrolled. The ICU-SEPSA score composed of eight clinical risk factors: 1) immunocompromised host, 2) chronic obstructive pulmonary disease, 3) urinary tract infection, 4) sepsis, 5) placement of endotracheal tube, 6) pneumonia, 7) septic shock, and 8) use of antibiotics within the past 3 months. The model showed good calibration (Hosmer-Lemeshow χ2 = 19.27; p-value = 0.50) and good discrimination after optimism correction (AuROC 84.6%, 95% confidence interval [Cl]: 81.0%–88.0%). The positive likelihood ratio of low risk (score ≤ 5) and high risk (score ≥ 8) were 2.07 (95% CI: 1.74–2.46) and 12.35 (95% CI: 4.90–31.13), respectively.

Conclusion: A simplified predictive scoring tool wad developed to predict mortality in patients with MDR infections. Due to a single-study design of this study, external validation of the results before applying in other clinical practice settings is warranted.

Novel clinical risk scoring model for predicting mortality in patients with necrotizing fasciitis: The MNF scoring system

Necrotizing fasciitis (NF) is a life-threatening soft tissue infection that rapidly progresses and requires urgent surgery and medical therapy. If treatment is delayed, the likelihood of an unfavorable outcome, including death, is significantly increased. The goal of this study was to develop and validate a novel scoring model for predicting mortality in patients with NF. The proposed system is hereafter referred to as the Mortality in Necrotizing Fasciitis (MNF) scoring system. A total of 1503 patients with NF were recruited from 3 provincial hospitals in Thailand during January 2009 to December 2012. Patients were randomly allocated into either the derivation cohort (n = 1192) or the validation cohort (n = 311). Clinical risk factors used to develop the MNF scoring system were determined by logistic regression. Regression coefficients were transformed into item scores, the sum of which reflected the total MNF score. The following 6 clinical predictors were included: female gender; age > 60 years; white blood cell (WBC) ≤5000/mm3; WBC ≥ 35,000/mm3; creatinine ≥ 1.6 mg/dL, and pulse rate > 130/min. Area under the receiver operating characteristic curve (AuROC) analysis showed the MNF scoring system to have moderate power for predicting mortality in patients with NF (AuROC: 76.18%) with good calibration (Hosmer-Lemeshow χ2: 1.01; P = .798). The positive likelihood ratios of mortality in patients with low-risk scores (≤2.5) and high-risk scores (≥7) were 11.30 (95% confidence interval [CI]: 6.16-20.71) and 14.71 (95%CI: 7.39-29.28), sequentially. When used to the validation cohort, the MNF scoring system presented good performance with an AuROC of 74.25%. The proposed MNF scoring system, which includes 6 commonly available and easy-to-use parameters, was shown to be an effective tool for predicting mortality in patients with NF. This validated instrument will help clinicians identify at-risk patients so that early investigations and interventions can be performed that will reduce the mortality rate among patients with NF.

Novel Clinical Risk Scoring Model for Predicting Amputation in Patients With Necrotizing Fasciitis: The ANF Risk Scoring System

Background: Necrotizing fasciitis (NF) is a life-threatening infection of the skin and soft tissue that spreads quickly and requires immediate surgery and medical treatment. Amputation or radical debridement of necrotic tissue is generally always required. The risks and benefits of both the surgical options are weighed before deciding whether to amputate or debride. This study set forth to create an easy-to-use risk scoring system for predicting the risk scoring system for amputation in patients with NF (ANF).

Methods: This retrospective study included 1,506 patients diagnosed with surgically confirmed NF at three general hospitals in Thailand from January 2009 to December 2012. All diagnoses were made by surgeons who strictly observed the guidelines for skin and soft tissue infections produced by the Infectious Diseases Society of America. Patients were randomly allocated to either the derivation (n = 1,193) or validation (n = 313) cohort. Clinical risk factors assessed at the time of recruitment were used to create the risk score, which was then developed using logistic regression. The regression coefficients were converted into item scores, and the total score was calculated.

Results: The following four clinical predictors were used to create the model: female gender, diabetes mellitus, wound appearance stage 3 (skin necrosis and gangrene), and creatinine ≥1.6 mg/dL. Using the area under the receiver operating characteristic curve (AuROC), the ANF system showed moderate power (78.68%) to predict amputation in patients with NF with excellent calibration (Hosmer-Lemeshow χ² = 2.59; p = 0.8586). The positive likelihood ratio of amputation in low-risk (score ≤ 4) and high-risk (score ≥ 7) patients was 2.17 (95%CI: 1.66–2.82) and 6.18 (95%CI: 4.08–9.36), respectively. The ANF system showed good performance (AuROC 76.82%) when applied in the validation cohort.

Conclusion: The developed ANF risk scoring system, which includes four easy to obtain predictors, provides physicians with prediction indices for amputation in patients with NF. This model will assist clinicians with surgical decision-making in this time-sensitive clinical setting.

Association of Age With the Diagnostic Value of Coronary Artery Calcium Score for Ruling Out Coronary Stenosis in Symptomatic Patients

Importance

The diagnostic value is unclear of a 0 coronary artery calcium (CAC) score to rule out obstructive coronary artery disease (CAD) and near-term clinical events across different age groups.

Objective

To assess the diagnostic value of a CAC score of 0 for reducing the likelihood of obstructive CAD and to assess the implications of such a CAC score and obstructive CAD across different age groups.

Design, Setting, and Participants

This cohort study obtained data from the Western Denmark Heart Registry and had a median follow-up time of 4.3 years. Included patients were aged 18 years or older who underwent computed tomography angiography (CTA) between January 1, 2008, and December 31, 2017, because of symptoms that were suggestive of CAD. Data analysis was performed from April 5 to July 7, 2021.

Exposures

Obstructive CAD, which was defined as 50% or more luminal stenosis.

Main Outcomes and Measures

Proportion of individuals with obstructive CAD who had a CAC score of 0. Risk-adjusted diagnostic likelihood ratios were used to assess the diagnostic value of a CAC score of 0 for reducing the likelihood of obstructive CAD beyond clinical variables. Risk factors associated with myocardial infarction and death were estimated.

Results

A total of 23 759 symptomatic patients, of whom 12 771 (54%) had a CAC score of 0, were included. This cohort had a median (IQR) age of 58 (49-65) years and was primarily composed of women (13 160 [55%]). Overall, the prevalence of obstructive CAD was relatively low across all age groups, ranging from 3% (39 of 1278 patients) in those who were younger than 40 years to 8% (52 of 619) among those who were 70 years or older. In patients with obstructive CAD, 14% (725 of 5043) had a CAC score of 0, and the prevalence varied across age groups from 58% (39 of 68) among those who were younger than 40 years, 34% (192 of 562) among those aged 40 to 49 years, 18% (268 of 1486) among those aged 50 to 59 years, 9% (174 of 1963) among those aged 60 to 69 years, to 5% (52 of 964) among those who were 70 years or older. The added diagnostic value of a CAC score of 0 decreased at a younger age, with a risk factor-adjusted diagnostic likelihood ratio of a CAC score of 0 ranging from 0.68 (approximately 32% lower likelihood of obstructive CAD than expected) in those who were younger than 40 years to 0.18 (approximately 82% lower likelihood than expected) in those who were 70 years or older. The presence of obstructive vs nonobstructive CAD among those with a CAC score of 0 was associated with a multivariable adjusted hazard ratio of 1.51 (95% CI, 0.98-2.33) for myocardial infarction and all-cause death; however, this hazard ratio varied from 1.80 (95% CI, 1.02-3.19) in those who were younger than 60 years to 1.24 (95% CI, 0.64-2.39) in those who were 60 years or older.

Conclusions and Relevance

This cohort study found that the diagnostic value of a CAC score of 0 to rule out obstructive CAD beyond clinical variables was dependent on age, with the added diagnostic value being smaller for younger patients. In symptomatic patients who were younger than 60 years, a sizable proportion of obstructive CAD occurred among those without CAC and was associated with an increased risk of myocardial infarction and all-cause death.

Predictive Value of Odor Identification for Incident Dementia: The Shanghai Aging Study

OBJECTIVE

This study aimed to evaluate the value of odors in the olfactory identification (OI) test and other known risk factors for predicting incident dementia in the prospective Shanghai Aging Study.

METHODS

At baseline, OI was assessed using the Sniffin' Sticks Screening Test 12, which contains 12 different odors. Cognition assessment and consensus diagnosis were conducted at both baseline and follow-up to identify incident dementia. Four different multivariable logistic regression (MLR) models were used for predicting incident dementia. In the no-odor model, only demographics, lifestyle, and medical history variables were included. In the single-odor model, we further added one single odor to the first model. In the full model, all 12 odors were included. In the stepwise model, the variables were selected using a bidirectional stepwise selection method. The predictive abilities of these models were evaluated by the area under the receiver operating characteristic curve (AUC). The permutation importance method was used to evaluate the relative importance of different odors and other known risk factors.

RESULTS

Seventy-five (8%) incident dementia cases were diagnosed during 4.9 years of follow-up among 947 participants. The full and the stepwise MLR model (AUC = 0.916 and 0.914, respectively) have better predictive abilities compared with those of the no- or single-odor models. The five most important variables are Mini-Mental State Examination (MMSE) score, age, peppermint detection, coronary artery disease, and height in the full model, and MMSE, age, peppermint detection, stroke, and education in the stepwise model. The combination of only the top five variables in the stepwise model (AUC = 0.901 and sensitivity = 0.880) has as a good a predictive ability as other models.

CONCLUSION

The ability to smell peppermint might be one of the useful indicators for predicting dementia. Combining peppermint detection with MMSE, age, education, and history of stroke may have sensitive and robust predictive value for dementia in older adults.

Negative Risk Markers for Cardiovascular Events in the Elderly

BACKGROUND

Cardiovascular risk increases dramatically with age, leading to nearly universal risk-based statin eligibility in the elderly population. To limit overtreatment, elderly individuals at truly low risk need to be identified.

OBJECTIVES

Discovering "negative" risk markers able to identify elderly individuals at low short-term risk for coronary heart disease and cardiovascular disease.

METHODS

In 5,805 BioImage participants (mean age 69 years; median follow-up 2.7 years), the authors evaluated 13 candidate markers: coronary artery calcium (CAC) = 0, CAC ≤10, no carotid plaque, no family history, normal ankle-brachial index, test result <25th percentile (carotid intima-media thickness, apolipoprotein B, galectin-3, high-sensitivity C-reactive protein, lipoprotein(a), N-terminal pro-B-type natriuretic peptide, and transferrin), and apolipoprotein A1 >75th percentile. Negative risk marker performance was compared using patient-specific diagnostic likelihood ratio (DLR) and binary net reclassification index (NRI).

RESULTS

CAC = 0 and CAC ≤10 were the strongest negative risk markers with mean DLRs of 0.20 and 0.20 for coronary heart disease (i.e., ≈80% lower risk than expected from traditional risk factor assessment) and 0.41 and 0.48 for cardiovascular disease, respectively, followed by galectin-3 <25th percentile (DLR 0.44 and 0.43, respectively) and absence of carotid plaque (DLR 0.39 and 0.65, respectively). Results obtained by other candidate markers were less impressive. Accurate downward risk reclassification across the Class I statin-eligibility threshold defined by the American College of Cardiology/American Heart Association was largest for CAC = 0 (NRI 0.23) and CAC ≤10 (NRI 0.28), followed by galectin-3 <25th percentile (NRI 0.14) and absence of carotid plaque (NRI 0.08).

CONCLUSIONS

Elderly individuals with CAC = 0, CAC ≤10, low galectin-3, or no carotid plaque had remarkable low cardiovascular risk, calling into question the appropriateness of a treat-all approach in the elderly population.

The statistical importance of P-POSSUM scores for predicting mortality after emergency laparotomy in geriatric patients

Background

Geriatric patients frequently undergo emergency general surgery and accrue a greater risk of postoperative complications and fatal outcomes than the general population. It is highly relevant to develop the most appropriate care measures and to guide patient-centered decision-making around end-of-life care.

Portsmouth - Physiological and Operative Severity Score for the enumeration of Mortality and morbidity (P-POSSUM) has been used to predict mortality in patients undergoing different types of surgery. In the present study, we aimed to evaluate the relative importance of the P-POSSUM score for predicting 90-day mortality in the elderly subjected to emergency laparotomy from statistical aspects.

Methods

One hundred and fifty-seven geriatric patients aged ≥65 years undergoing emergency laparotomy between January 1st, 2015 and December 31st, 2016 were included in the study. Mortality and 27 other patient characteristics were retrieved from the computerized records of Örebro University Hospital in Örebro, Sweden. Two supervised classification machine methods (logistic regression and random forest) were used to predict the 90-day mortality risk. Three scalers (Standard scaler, Robust scaler and Min-Max scaler) were used for variable engineering. The performance of the models was evaluated using accuracy, sensitivity, specificity and area under the receiver operating characteristic curve (AUC). Importance of the predictors were evaluated using permutation variable importance and Gini importance.

Results

The mean age of the included patients was 75.4 years (standard deviation =7.3 years) and the 90-day mortality rate was 29.3%. The most common indication for surgery was bowel obstruction occurring in 92 (58.6%) patients. Types of post-operative complications ranged between 7.0–36.9% with infection being the most common type. Both the logistic regression and random forest models showed satisfactory performance for predicting 90-day mortality risk in geriatric patients after emergency laparotomy, with AUCs of 0.88 and 0.93, respectively. Both models had an accuracy > 0.8 and a specificity ≥0.9. P-POSSUM had the greatest relative importance for predicting 90-day mortality in the logistic regression model and was the fifth important predictor in the random forest model. No notable change was found in sensitivity analysis using different variable engineering methods with P-POSSUM being among the five most accurate variables for mortality prediction.

Conclusion

P-POSSUM is important for predicting 90-day mortality after emergency laparotomy in geriatric patients. The logistic regression model and random forest model may have an accuracy of > 0.8 and an AUC around 0.9 for predicting 90-day mortality. Further validation of the variables’ importance and the models’ robustness is needed by use of larger dataset.

Development of a hyponatremia screening tool (ABCDF-S score) for patients with hypertension using thiazide diuretic agents

WHAT IS KNOWN AND OBJECTIVE

Hyponatremia is a common side effect of thiazide diuretics that can lead to increased mortality and hospitalization. A rapid and accurate screening tool is needed for rapid and appropriate management. In this study, we report on the development of a simple clinical screening tool for hyponatremia using thiazide diuretics.

METHODS

This nested case-control study was performed by collecting data from 1 January 2015 to 30 June 2017. Univariable and multivariable logistic regressions were used to identify potential risk factors. The regression coefficients were converted into item scores by dividing each regression coefficient with the minimum coefficient in the model and rounding to the nearest integer. This value was then summed to the total score. The prediction power of the model was determined by the area under the receiver operating characteristic curve (AuROC).

RESULTS AND DISCUSSION

Six clinical risk factors, namely age ≥65 years, benzodiazepine use, history of a cerebrovascular accident, dose of hydrochlorothiazide ≥25 mg, female sex and statin use, were included in our ABCDF-S score. The model showed good power of prediction (AuROC 81.53%, 95% confidence interval [CI]: 78%-84%) and good calibration (Hosmer-Lemeshow X²  = 23.20; P = .39). The positive likelihood ratios of hyponatremia in patients with low risk (score ≤ 6) and high risk (score ≥ 8) were 0.26 (95% CI: 0.21-0.32) and 3.89 (95% CI: 3.11-4.86), respectively.

WHAT IS NEW AND CONCLUSION

The screening tool with six risk predictors provided a useful prediction index for thiazide-associated hyponatremia. However, further validation of the tool is warranted prior to its utilization in routine clinical practice.

Establishment and verification of a surgical prognostic model for cervical spinal cord injury without radiological abnormality

Some studies have suggested that early surgical treatment can effectively improve the prognosis of cervical spinal cord injury without radiological abnormality, but no research has focused on the development of a prognostic model of cervical spinal cord injury without radiological abnormality. This retrospective analysis included 43 patients with cervical spinal cord injury without radiological abnormality. Seven potential factors were assessed: age, sex, external force strength causing damage, duration of disease, degree of cervical spinal stenosis, Japanese Orthopaedic Association score, and physiological cervical curvature. A model was established using multiple binary logistic regression analysis. The model was evaluated by concordant profiling and the area under the receiver operating characteristic curve. Bootstrapping was used for internal validation. The prognostic model was as follows: logit(P) = -25.4545 + 21.2576VALUE + 1.2160SCORE - 3.4224TIME, where VALUE refers to the Pavlov ratio indicating the extent of cervical spinal stenosis, SCORE refers to the Japanese Orthopaedic Association score (0-17) after the operation, and TIME refers to the disease duration (from injury to operation). The area under the receiver operating characteristic curve for all patients was 0.8941 (95% confidence interval, 0.7930-0.9952). Three factors assessed in the predictive model were associated with patient outcomes: a great extent of cervical stenosis, a poor preoperative neurological status, and a long disease duration. These three factors could worsen patient outcomes. Moreover, the disease prognosis was considered good when logit(P) ≥ -2.5105. Overall, the model displayed a certain clinical value. This study was approved by the Biomedical Ethics Committee of the Second Affiliated Hospital of Xi'an Jiaotong University, China (approval number: 2018063) on May 8, 2018.

Spectrum effect and spectrum bias in the oscillometric ankle brachial index to diagnose peripheral arterial disease: Clinical implications

BACKGROUND AND AIMS

The diagnostic performance of the oscillometric ankle brachial index (ABI) to detect peripheral arterial disease (PAD) varies among populations, suggesting a spectrum effect. When this heterogeneity modifies post-test probabilities, a spectrum bias arises. This study evaluates the presence and influence of spectrum effect and spectrum bias on test performance and clinical decisions.

METHODS

Oscillometric and Doppler ABI were compared in two settings: Primary-Care (333 legs) and Vascular-Service (41 legs). Spectrum effect was assessed using stratification and logistic regression, while spectrum bias was assessed through graphical and statistical tests based on predictive values and likelihood ratios, respectively.

RESULTS

Across subgroups, sensitivity ranged from 61.5% to 90.9%, and specificity from 81.8% to 99.1%. Logistic regression confirmed a spectrum effect in setting, diabetes, smoking status and age (univariate), and setting and diabetes (multivariate model). The positive likelihood ratio ranged from 5.0 to 89.1 in subgroups, leading to a spectrum bias in diabetic, smoking (both subgroups) and age (both subgroups). Therefore, a positive test ruled in differently the disease across subgroups, with a high rate of false positives in diabetic, smoking and >75-year-old patients. The negative likelihood ratio ranged from 0.09 to 0.39 in subgroups, with significant spectrum bias in Primary-Care patients, non-diabetics and smokers. Thus, in these subgroups, a negative test ruled out the disease with less certainty.

CONCLUSIONS

Spectrum effect and spectrum bias were found in oscillometric ABI to detect PAD, potentially affecting clinical decisions, especially for positive tests. Information about spectrum variables and the application of specific subgroups indicators are necessary.

Defining Diarrhea: A Population-Based Validation Study of Caregiver-Reported Stool Consistency in the Amhara Region of Ethiopia

Diarrhea is a leading cause of death among children aged less than five years globally. Most studies of pediatric diarrhea rely on caregiver-reported stool consistency and frequency to define the disease. Research on the validity of caregiver-reported diarrhea is sparse. We collected stool samples from 2,398 children participating in two clinical trials in the Amhara region of Ethiopia. The consistency of each stool sample was graded by the child's caregiver and two trained laboratory technicians according to an illustrated stool consistency scale. We assessed the reliability of graded stool consistency among the technicians, and then compared the caregiver's grade with the technician's grade. We also tested if the illustrated stool consistency scale could improve the validity of caregiver's report. The weighted kappa measuring the agreement between the two laboratory technicians reached 0.90 after 500 stool samples were graded. The sensitivity of caregiver-reported loose or watery stool was 15.5% (95% confidence interval [CI]: 9.7, 24.2) and the specificity was 98.4% (95% CI 97.1, 99.1). With the illustrated scale, the sensitivity was 68.5% (95% CI: 58.5, 77.1) and the specificity was 86.1% (95% CI: 79.3, 90.9). The results indicate that caregiver-reported stool consistency using the terms "loose or watery" does not accurately describe stool consistency as graded by trained laboratory technicians. Given the predominance of using caregiver-reported stool consistency to define diarrheal disease, the low sensitivity identified in this study suggests that the burden of diarrheal disease may be underestimated and intervention effects could be biased. The illustrated scale is a potential low-lost tool to improve the validity of caregiver-reported stool consistency.

Hearing loss screening tool (COBRA score) for newborns in primary care setting

Purpose

To develop and evaluate a simple screening tool to assess hearing loss in newborns. A derived score was compared with the standard clinical practice tool.

Methods

This cohort study was designed to screen the hearing of newborns using transiently evoked otoacoustic emission and auditory brain stem response, and to determine the risk factors associated with hearing loss of newborns in 3 tertiary hospitals in Northern Thailand. Data were prospectively collected from November 1, 2010 to May 31, 2012. To develop the risk score, clinical-risk indicators were measured by Poisson risk regression. The regression coefficients were transformed into item scores dividing each regression-coefficient with the smallest coefficient in the model, rounding the number to its nearest integer, and adding up to a total score.

Results

Five clinical risk factors (Craniofacial anomaly, Ototoxicity, Birth weight, family history [Relative] of congenital sensorineural hearing loss, and Apgar score) were included in our COBRA score. The screening tool detected, by area under the receiver operating characteristic curve, more than 80% of existing hearing loss. The positive-likelihood ratio of hearing loss in patients with scores of 4, 6, and 8 were 25.21 (95% confidence interval [CI], 14.69–43.26), 58.52 (95% CI, 36.26–94.44), and 51.56 (95% CI, 33.74–78.82), respectively. This result was similar to the standard tool (The Joint Committee on Infant Hearing) of 26.72 (95% CI, 20.59–34.66).

Conclusion

A simple screening tool of five predictors provides good prediction indices for newborn hearing loss, which may motivate parents to bring children for further appropriate testing and investigations.

Clinical risk scoring for predicting non-alcoholic fatty liver disease in metabolic syndrome patients (NAFLD-MS score)

BACKGROUND & AIMS

Non-alcoholic fatty liver disease (NAFLD) can progress from simple steatosis to hepatocellular carcinoma. None of tools have been developed specifically for high-risk patients. This study aimed to develop a simple risk scoring to predict NAFLD in patients with metabolic syndrome (MetS).

METHODS

A total of 509 patients with MetS were recruited. All were diagnosed by clinicians with ultrasonography-confirmed whether they were patients with NAFLD. Patients were randomly divided into derivation (n=400) and validation (n=109) cohort. To develop the risk score, clinical risk indicators measured at the time of recruitment were built by logistic regression. Regression coefficients were transformed into item scores and added up to a total score. A risk scoring scheme was developed from clinical predictors: BMI ≥25, AST/ALT ≥1, ALT ≥40, type 2 diabetes mellitus and central obesity. The scoring scheme was applied in validation cohort to test the performance.

RESULTS

The scheme explained, by area under the receiver operating characteristic curve (AuROC), 76.8% of being NAFLD with good calibration (Hosmer-Lemeshow χ² =4.35; P=.629). The positive likelihood ratio of NAFLD in patients with low risk (scores below 3) and high risk (scores 5 and over) were 2.32 (95% CI: 1.90-2.82) and 7.77 (95% CI: 2.47-24.47) respectively. When applied in validation cohort, the score showed good performance with AuROC 76.7%, and illustrated 84%, and 100% certainty in low- and high-risk groups respectively.

CONCLUSIONS

A simple and non-invasive scoring scheme of five predictors provides good prediction indices for NAFLD in MetS patients. This scheme may help clinicians in order to take further appropriate action.

Comparison of approaches for incorporating new information into existing risk prediction models

We compare the calibration and variability of risk prediction models that were estimated using various approaches for combining information on new predictors, termed 'markers', with parameter information available for other variables from an earlier model, which was estimated from a large data source. We assess the performance of risk prediction models updated based on likelihood ratio (LR) approaches that incorporate dependence between new and old risk factors as well as approaches that assume independence ('naive Bayes' methods). We study the impact of estimating the LR by (i) fitting a single model to cases and non-cases when the distribution of the new markers is in the exponential family or (ii) fitting separate models to cases and non-cases. We also evaluate a new constrained maximum likelihood method. We study updating the risk prediction model when the new data arise from a cohort and extend available methods to accommodate updating when the new data source is a case-control study. To create realistic correlations between predictors, we also based simulations on real data on response to antiviral therapy for hepatitis C. From these studies, we recommend the LR method fit using a single model or constrained maximum likelihood. Copyright © 2016 John Wiley & Sons, Ltd.

Efficacy according to blind independent central review: Post-hoc analyses from the phase III, randomized, multicenter, IPASS study of first-line gefitinib versus carboplatin/paclitaxel in Asian patients with EGFR mutation-positive advanced NSCLC

OBJECTIVE

The Phase III, randomized, open-label IPASS study (NCT00322452) of first-line epidermal growth factor receptor tyrosine kinase inhibitor (EGFR TKI) gefitinib versus carboplatin/paclitaxel for Asian patients with advanced non-small-cell lung cancer (NSCLC) showed that investigator-assessed progression-free survival (PFS) and objective response rate (ORR) were significantly prolonged in patients with EGFR mutation-positive NSCLC who received gefitinib versus patients with EGFR mutation-negative NSCLC. We report post-hoc analyses of IPASS data by blind independent central review (BICR), performed at the request of the US FDA, in a subset of patients with EGFR mutation-positive NSCLC.

PATIENTS AND METHODS

Eligible patients (aged ≥18 years; histologically/cytologically confirmed Stage IIB/IV adenocarcinoma NSCLC; non- or former light-smokers; treatment-naïve) were randomly assigned 1:1 to gefitinib (250mg/day) or carboplatin (dose calculated to produce an area under the curve of 5 or 6 mg/mL/minute)/paclitaxel (200mg/m²). Primary endpoint: PFS. BICR analyses included PFS, ORR, and duration of response (DoR).

RESULTS

Scans from 186 IPASS patients (gefitinib n=88, carboplatin/paclitaxel n=98) with EGFR mutation-positive NSCLC were available for BICR. Consistent with investigator-assessed results, in patients with EGFR mutation-positive NSCLC: PFS (hazard ratio 0.54; 95% confidence interval [CI] 0.38, 0.79; p=0.0012) and ORR (odds ratio 3.00; 95% CI 1.63, 5.54; p=0.0004) were significantly longer with gefitinib versus carboplatin/paclitaxel. The median DoR by BICR was 9.6 months with gefitinib and 5.5 months with carboplatin/paclitaxel.

CONCLUSION

BICR analysis of IPASS data support the original, investigator-assessed results. EGFR mutation-positive status remains a significant predictor of response to first-line TKI therapy.

Role of Coronary Artery Calcium Score of Zero and Other Negative Risk Markers for Cardiovascular Disease: The Multi-Ethnic Study of Atherosclerosis (MESA)

BACKGROUND

Limited attention has been paid to negative cardiovascular disease (CVD) risk markers despite their potential to improve medical decision making. We compared 13 negative risk markers using diagnostic likelihood ratios (DLRs), which model the change in risk for an individual after the result of an additional test.

METHODS AND RESULTS

We examined 6814 participants from the Multi-Ethnic Study of Atherosclerosis. Coronary artery calcium score of 0, carotid intima-media thickness <25th percentile, absence of carotid plaque, brachial flow-mediated dilation >5% change, ankle-brachial index >0.9 and <1.3, high-sensitivity C-reactive protein <2 mg/L, homocysteine <10 µmol/L, N-terminal pro-brain natriuretic peptide <100 pg/mL, no microalbuminuria, no family history of coronary heart disease (any/premature), absence of metabolic syndrome, and healthy lifestyle were compared for all and hard coronary heart disease and all CVD events over the 10-year follow-up. Models were adjusted for traditional CVD risk factors. Among all negative risk markers, coronary artery calcium score of 0 was the strongest, with an adjusted mean DLR of 0.41 (SD, 0.12) for all coronary heart disease and 0.54 (SD, 0.12) for CVD, followed by carotid intima-media thickness <25th percentile (DLR, 0.65 [SD, 0.04] and 0.75 [SD, 0.04], respectively). High-sensitivity C-reactive protein <2 mg/L and normal ankle-brachial index had DLRs >0.80. Among clinical features, absence of any family history of coronary heart disease was the strongest (DLRs, 0.76 [SD, 0.07] and 0.81 [SD, 0.06], respectively). Net reclassification improvement analyses yielded similar findings, with coronary artery calcium score of 0 resulting in the largest, most accurate downward risk reclassification.

CONCLUSIONS

Negative results of atherosclerosis-imaging tests, particularly coronary artery calcium score of 0, resulted in the greatest downward shift in estimated CVD risk. These results may help guide discussions on the identification of individuals less likely to receive net benefit from lifelong preventive pharmacotherapy.

Personalized Evaluation of Biomarker Value: A Cost-Benefit Perspective

For a patient who is facing a treatment decision, the added value of information provided by a biomarker depends on the individual patient's expected response to treatment with and without the biomarker, as well as his/her tolerance of disease and treatment harm. However, individualized estimators of the value of a biomarker are lacking. We propose a new graphical tool named the subject-specific expected benefit curve for quantifying the personalized value of a biomarker in aiding a treatment decision. We develop semiparametric estimators for two general settings: (i) when biomarker data are available from a randomized trial; and (ii) when biomarker data are available from a cohort or a cross-sectional study, together with external information about a multiplicative treatment effect. We also develop adaptive bootstrap confidence intervals for consistent inference in the presence of nonregularity. The proposed method is used to evaluate the individualized value of the serum creatinine marker in informing treatment decisions for the prevention of renal artery stenosis.

Diagnostic performance of computed tomography coronary angiography (from the Prospective National Multicenter Multivendor EVASCAN Study)

Computed tomographic coronary angiography (CTCA) has been proposed as a noninvasive test for significant coronary artery disease (CAD), but only limited data are available from prospective multicenter trials. The goal of this study was to establish the diagnostic accuracy of CTCA compared to coronary angiography (CA) in a large population of symptomatic patients with clinical indications for coronary imaging. This national, multicenter study was designed to prospectively evaluate stable patients able to undergo CTCA followed by conventional CA. Data from CTCA and CA were analyzed in a blinded fashion at central core laboratories. The main outcome was the evaluation of patient-, vessel-, and segment-based diagnostic performance of CTCA to detect or rule out significant CAD (≥50% luminal diameter reduction). Of 757 patients enrolled, 746 (mean age 61 ± 12 years, 71% men) were analyzed. They underwent CTCA followed by CA 1.7 ± 0.8 days later using a 64-detector scanner. The prevalence of significant CAD in native coronary vessels by CA was 54%. The rate of nonassessable segments by CTCA was 6%. In a patient-based analysis, sensitivity, specificity, positive and negative predictive values, and positive and negative likelihood ratios of CTCA were 91%, 50%, 68%, 83%, 1.82, and 0.18, respectively. The strongest predictors of false-negative results on CTCA were high estimated pretest probability of CAD (odds ratio [OR] 1.97, p <0.001), male gender (OR 1.5, p <0.002), diabetes (OR 1.5, p <0.0001), and age (OR 1.2, p <0.0001). In conclusion, in this large multicenter study, CTCA identified significant CAD with high sensitivity. However, in routine clinical practice, each patient should be individually evaluated, and the pretest probability of obstructive CAD should be taken into account when deciding which method, CTCA or CA, to use to diagnose its presence and severity.

Testing for improvement in prediction model performance

Authors have proposed new methodology in recent years for evaluating the improvement in prediction performance gained by adding a new predictor, Y, to a risk model containing a set of baseline predictors, X, for a binary outcome D. We prove theoretically that null hypotheses concerning no improvement in performance are equivalent to the simple null hypothesis that Y is not a risk factor when controlling for X, H0 : P(D = 1 | X,Y ) = P(D = 1 | X). Therefore, testing for improvement in prediction performance is redundant if Y has already been shown to be a risk factor. We also investigate properties of tests through simulation studies, focusing on the change in the area under the ROC curve (AUC). An unexpected finding is that standard testing procedures that do not adjust for variability in estimated regression coefficients are extremely conservative. This may explain why the AUC is widely considered insensitive to improvements in prediction performance and suggests that the problem of insensitivity has to do with use of invalid procedures for inference rather than with the measure itself. To avoid redundant testing and use of potentially problematic methods for inference, we recommend that hypothesis testing for no improvement be limited to evaluation of Y as a risk factor, for which methods are well developed and widely available. Analyses of measures of prediction performance should focus on estimation rather than on testing for no improvement in performance.

Validity of biomarkers predicting onset or progression of nephropathy in patients with Type 2 diabetes: a systematic review

Novel biomarkers predicting onset or progression of nephropathy in patients with Type 2 diabetes have been recently identified. We performed a systematic review to assess the validity of biomarkers predicting onset or progression of nephropathy in patients with Type 2 diabetes in longitudinal studies. The methodological quality of the studies was scored using Standards for Reporting of Diagnostic Accuracy (STARD) criteria and the independent predictive value of the biomarkers beyond conventional risk factors was scored according to the adjustment for these risk factors. Validity of the biomarkers was determined by summarizing the methodological quality and the adjustment score. We identified 15 studies describing 27 biomarkers. Six studies had sufficient methodological quality. These studies identified 13 valid and significant markers for nephropathy in diabetes: serum interleukin 18, plasma asymmetric dimethylarginine; and urinary ceruloplasmin, immunoglobulin G and transferrin were considered valid markers predicting onset of nephropathy. Plasma asymmetric dimethylarginine, vascular cell adhesion molecule 1, interleukin 6, von Willebrand factor and intercellular cell adhesion molecule 1 were considered valid biomarkers predicting progression of nephropathy. Plasma high-sensitivity C-reactive protein, E-selectin, tissue-type plasminogen activator, von Willebrand factor and triglycerides were considered valid markers predicting onset and progression of nephropathy. Several novel biomarkers for prediction of nephropathy in diabetes have been published, which can potentially be applied in clinical practice and research in future. Because of the heterogeneous quality of biomarker studies in this field, a more rigorous evaluation of these biomarkers and validation in larger trials are advocated.

Joint modeling, covariate adjustment, and interaction: contrasting notions in risk prediction models and risk prediction performance

Epidemiologic methods are well established for investigating the association of a predictor of interest and disease status in the presence of covariates also associated with disease. There is less consensus on how to handle covariates when the goal is to evaluate the increment in prediction performance gained by a new marker when a set of predictors already exists. We distinguish between adjusting for covariates and joint modeling of disease risk in this context. We show that adjustment and joint modeling are distinct concepts, and we describe the specific conditions where they are the same. We also discuss the concept of interaction among variables and describe a notion of interaction that is relevant to prediction performance. We conclude with a discussion of the most appropriate methods for evaluating new biomarkers in the presence of existing predictors.

Updating risk prediction tools: a case study in prostate cancer

Online risk prediction tools for common cancers are now easily accessible and widely used by patients and doctors for informed decision-making concerning screening and diagnosis. A practical problem is as cancer research moves forward and new biomarkers and risk factors are discovered, there is a need to update the risk algorithms to include them. Typically, the new markers and risk factors cannot be retrospectively measured on the same study participants used to develop the original prediction tool, necessitating the merging of a separate study of different participants, which may be much smaller in sample size and of a different design. Validation of the updated tool on a third independent data set is warranted before the updated tool can go online. This article reports on the application of Bayes rule for updating risk prediction tools to include a set of biomarkers measured in an external study to the original study used to develop the risk prediction tool. The procedure is illustrated in the context of updating the online Prostate Cancer Prevention Trial Risk Calculator to incorporate the new markers %freePSA and [-2]proPSA measured on an external case-control study performed in Texas, U.S.. Recent state-of-the art methods in validation of risk prediction tools and evaluation of the improvement of updated to original tools are implemented using an external validation set provided by the U.S. Early Detection Research Network.

Value of EUS in Determining Curative Resectability in Reference to CT and FDG-PET: The Optimal Sequence in Preoperative Staging of Esophageal Cancer?

BACKGROUND

F-fluorodeoxyglucose positron emission tomography (FDG-PET) in the optimal sequence in staging esophageal cancer has not been investigated adequately.

METHODS

The staging records of 216 consecutive operable patients with esophageal cancer were reviewed blindly. Different staging strategies were analyzed, and the likelihood ratio (LR) of each module was calculated conditionally on individual patient characteristics. A logistic regression approach was used to determine the most favorable staging strategy.

RESULTS

Initial EUS results were not significantly related to the LRs of initial CT and FDG-PET results. The positive LR (LR+) of EUS-fine-needle aspiration (FNA) was 4, irrespective of CT and FDG-PET outcomes. The LR+ of FDG-PET varied from 13 (negative CT) to 6 (positive CT). The LR+ of CT ranged from 3-4 (negative FDG-PET) to 2-3 (positive FDG-PET). Age, histology, and tumor length had no significant impact on the LRs of the three diagnostic tests.

CONCLUSIONS

This study argues in favor of PET/CT rather than EUS as a predictor of curative resectability in esophageal cancer. EUS does not correspond with either CT or FDG-PET. LRs of FDG-PET were substantially different between subgroups of negative and positive CT results and vice versa.

Clinical prediction rules combining signs, symptoms and epidemiological context to distinguish influenza from influenza-like illnesses in primary care: a cross sectional study

Background

During an influenza epidemic prompt diagnosis of influenza is important. This diagnosis however is still essentially based on the interpretation of symptoms and signs by general practitioners. No single symptom is specific enough to be useful in differentiating influenza from other respiratory infections. Our objective is to formulate prediction rules for the diagnosis of influenza with the best diagnostic performance, combining symptoms, signs and context among patients with influenza-like illness.

Methods

During five consecutive winter periods (2002-2007) 138 sentinel general practitioners sampled (naso- and oropharyngeal swabs) 4597 patients with an influenza-like illness (ILI) and registered their symptoms and signs, general characteristics and contextual information. The samples were analysed by a DirectigenFlu-A&B and RT-PCR tests. 4584 records were useful for further analysis.

Starting from the most relevant variables in a Generalized Estimating Equations (GEE) model, we calculated the area under the Receiver Operating Characteristic curve (ROC AUC), sensitivity, specificity and likelihood ratios for positive (LR+) and negative test results (LR-) of single and combined signs, symptoms and context taking into account pre-test and post-test odds.

Results

In total 52.6% (2409/4584) of the samples were positive for influenza virus: 64% (2066/3212) during and 25% (343/1372) pre/post an influenza epidemic. During and pre/post an influenza epidemic the LR+ of 'previous flu-like contacts', 'coughing', 'expectoration on the first day of illness' and 'body temperature above 37.8°C' is 3.35 (95%CI 2.67-4.03) and 1.34 (95%CI 0.97-1.72), respectively. During and pre/post an influenza epidemic the LR- of 'coughing' and 'a body temperature above 37.8°C' is 0.34 (95%CI 0.27-0.41) and 0.07 (95%CI 0.05-0.08), respectively.

Conclusions

Ruling out influenza using clinical and contextual information is easier than ruling it in. Outside an influenza epidemic the absence of cough and fever (> 37,8°C) makes influenza 14 times less likely in ILI patients. During an epidemic the presence of 'previous flu-like contacts', cough, 'expectoration on the first day of illness' and fever (>37,8°C) increases the likelihood for influenza threefold. The additional diagnostic value of rapid point of care tests especially for confirming influenza still has to be established.

Estimating the diagnostic likelihood ratio of a continuous marker

The diagnostic likelihood ratio function, DLR, is a statistical measure used to evaluate risk prediction markers. The goal of this paper is to develop new methods to estimate the DLR function. Furthermore, we show how risk prediction markers can be compared using rank-invariant DLR functions. Various estimators are proposed that accommodate cohort or case-control study designs. Performances of the estimators are compared using simulation studies. The methods are illustrated by comparing a lung function measure and a nutritional status measure for predicting subsequent onset of major pulmonary infection in children suffering from cystic fibrosis. For continuous markers, the DLR function is mathematically related to the slope of the receiver operating characteristic (ROC) curve, an entity used to evaluate diagnostic markers. We show that our methodology can be used to estimate the slope of the ROC curve and illustrate use of the estimated ROC derivative in variance and sample size calculations for a diagnostic biomarker study.

Semiparametric methods for evaluating the covariate-specific predictiveness of continuous markers in matched case-control studies

To assess the value of a continuous marker in predicting the risk of a disease, a graphical tool called the predictiveness curve has been proposed. It characterizes the marker's predictiveness, or capacity to risk stratify the population by displaying the distribution of risk endowed by the marker. Methods for making inference about the curve and for comparing curves in a general population have been developed. However, knowledge about a marker's performance in the general population only is not enough. Since a marker's effect on the risk model and its distribution can both differ across subpopulations, its predictiveness may vary when applied to different subpopulations. Moreover, information about the predictiveness of a marker conditional on baseline covariates is valuable for individual decision making about having the marker measured or not. Therefore, to fully realize the usefulness of a risk prediction marker, it is important to study its performance conditional on covariates. In this article, we propose semiparametric methods for estimating covariate-specific predictiveness curves for a continuous marker. Unmatched and matched case-control study designs are accommodated. We illustrate application of the methodology by evaluating serum creatinine as a predictor of risk of renal artery stenosis.

In utero and at birth diagnosis of congenital toxoplasmosis: use of likelihood ratios for clinical management

BACKGROUND

The results of the ante- and neonatal diagnostic tests for congenital toxoplasmosis influence the decision to treat the newborn immediately after birth. Here, we estimate the positive and negative likelihood ratios (LRs) and the probabilities of congenital infection according to PCR and IgM-IgA tests results.

METHODS

The study concerned 767 children born between 1996 and 2002 and followed-up for 1-year at Croix-Rousse hospital, Lyon, France. The LRs and the post-test probabilities were estimated conditionally on gestational age at maternal infection using a logistic regression approach.

RESULTS

For the PCR and the IgM-IgA tests, the positive LRs were high. In children with one positive test when only one test was done, the probability of infection reached 90% when the maternal infection occurred at 18-weeks gestation or later. This probability was close to 100% when the 2 tests were positive, whatever the gestational age. The negative LRs of the 2 tests moved closer to 0 at later gestational ages. However, when the tests were negative, the probability of infection remained greater or equal to 10%, except in early maternal infection. When the 2 tests were discordant, the probability of infection was about 50% in early maternal infection.

CONCLUSION

Providing reliable probabilities of congenital infection, the PCR and IgM-IgA tests guide clinical management and counseling of parents.

Estimating the capacity for improvement in risk prediction with a marker

Consider a set of baseline predictors X to predict a binary outcome D and let Y be a novel marker or predictor. This paper is concerned with evaluating the performance of the augmented risk model P(D = 1|Y,X) compared with the baseline model P(D = 1|X). The diagnostic likelihood ratio, DLR(X)(y), quantifies the change in risk obtained with knowledge of Y = y for a subject with baseline risk factors X. The notion is commonly used in clinical medicine to quantify the increment in risk prediction due to Y. It is contrasted here with the notion of covariate-adjusted effect of Y in the augmented risk model. We also propose methods for making inference about DLR(X)(y). Case-control study designs are accommodated. The methods provide a mechanism to investigate if the predictive information in Y varies with baseline covariates. In addition, we show that when combined with a baseline risk model and information about the population distribution of Y given X, covariate-specific predictiveness curves can be estimated. These curves are useful to an individual in deciding if ascertainment of Y is likely to be informative or not for him. We illustrate with data from 2 studies: one is a study of the performance of hearing screening tests for infants, and the other concerns the value of serum creatinine in diagnosing renal artery stenosis.

A methodological framework to distinguish spectrum effects from spectrum biases and to assess diagnostic and screening test accuracy for patient populations: application to the Papanicolaou cervical cancer smear test

BACKGROUND

A spectrum effect was defined as differences in the sensitivity or specificity of a diagnostic test according to the patient's characteristics or disease features. A spectrum effect can lead to a spectrum bias when subgroup variations in sensitivity or specificity also affect the likelihood ratios and thus post-test probabilities. We propose and illustrate a methodological framework to distinguish spectrum effects from spectrum biases.

METHODS

Data were collected for 1781 women having had a cervical smear test and colposcopy followed by biopsy if abnormalities were detected (the reference standard). Logistic models were constructed to evaluate both the sensitivity and specificity, and the likelihood ratios, of the test and to identify factors independently affecting the test's characteristics.

RESULTS

For both tests, human papillomavirus test, study setting and age affected sensitivity or specificity of the smear test (spectrum effect), but only human papillomavirus test and study setting modified the likelihood ratios (spectrum bias) for clinical reading, whereas only human papillomavirus test and age modified the likelihood ratios (spectrum bias) for "optimized" interpretation.

CONCLUSION

Fitting sensitivity, specificity and likelihood ratios simultaneously allows the identification of covariates that independently affect diagnostic or screening test results and distinguishes spectrum effect from spectrum bias. We recommend this approach for the development of new tests, and for reporting test accuracy for different patient populations.