Diagnostic and prognostic prediction models

Risk prediction models of depression in older adults with chronic diseases

BACKGROUND

Detecting potential depression and identifying the critical predictors of depression among older adults with chronic diseases are essential for timely intervention and management of depression. Therefore, risk prediction models (RPMs) of depression in elderly people should be further explored.

METHODS

A total of 3959 respondents aged 60 years or over from the wave four survey of the China Health and Retired Longitudinal Study (CHARLS) were included in this study. We used five machine learning (ML) algorithms and three data balancing techniques to construct RPMs of depression and calculated feature importance scores to determine which features are essential to depression.

RESULTS

The prevalence of depression was 19.2 % among older Chinese adults with chronic diseases in the wave four survey. The random forest (RF) model was more accurate than the other models after balancing the data using the Synthetic Minority Oversampling Technique (SMOTE) algorithm, with an area under the receiver operating characteristic curve (AUROC) and area under the precision-recall curve (AUPRC) of 0.957 and 0.920, respectively, a balanced accuracy of 0.891 and a sensitivity of 0.875. Furthermore, we further identified several important predictors between male and female patients via constructed sex-stratified models.

LIMITATIONS

Further research on the clinical impact studies of our models and external validation are needed.

CONCLUSIONS

After several techniques were used to address class imbalance issues, most RPMs achieved satisfactory accuracy in predicting depression among elderly people with chronic diseases. RPMs may thus become valuable screening tools for both older individuals and healthcare practitioners to assess the risk of depression.

A literature review of quality assessment and applicability to HTA of risk prediction models of coronary heart disease in patients with diabetes

This literature review had two objectives: to identify models for predicting the risk of coronary heart diseases in patients with diabetes (DM); and to assess model quality in terms of risk of bias (RoB) and applicability for the purpose of health technology assessment (HTA). We undertook a targeted review of journal articles published in English, Dutch, Chinese, or Spanish in 5 databases from 1st January 2016 to 18th December 2022, and searched three systematic reviews for the models published after 2012. We used PROBAST (Prediction model Risk Of Bias Assessment Tool) to assess RoB, and used findings from Betts et al. 2019, which summarized recommendations and criticisms of HTA agencies on cardiovascular risk prediction models, to assess model applicability for the purpose of HTA. As a result, 71 % and 67 % models reporting C-index showed good discrimination abilities (C-index >= 0.7). Of the 26 model studies and 30 models identified, only one model study showed low RoB in all domains, and no model was fully applicable for HTA. Since the major cause of high RoB is inappropriate use of analysis method, we advise clinicians to carefully examine the model performance declared by model developers, and to trust a model if all PROBAST domains except analysis show low RoB and at least one validation study conducted in the same setting (e.g. country) is available. Moreover, since general model applicability is not informative for HTA, novel adapted tools may need to be developed.

Development and external validation of a clinical prediction model for predicting quality of recovery up to 1 week after surgery

The Quality of Recovery Score-40 (QoR-40) has been increasingly used for assessing recovery after patients undergoing surgery. However, a prediction model estimating quality of recovery is lacking. The aim of the present study was to develop and externally validate a clinical prediction model that predicts quality of recovery up to one week after surgery. The modelling procedure consisted of two models of increasing complexity (basic and full model). To assess the internal validity of the developed model, bootstrapping (1000 times) was applied. At external validation, the model performance was evaluated according to measures for overall model performance (explained variance (R2)) and calibration (calibration plot and slope). The full model consisted of age, sex, previous surgery, BMI, ASA classification, duration of surgery, HADS and preoperative QoR-40 score. At model development, the R2 of the full model was 0.24. At external validation the R2 dropped as expected. The calibration analysis showed that the QoR-40 predictions provided by the developed prediction models are reliable. The presented models can be used as a starting point for future updating in prediction studies. When the predictive performance is improved it could be implemented clinically in the future.

The Vienna Prediction Model for identifying patients at low risk of recurrent venous thromboembolism: a prospective cohort study

BACKGROUND AND AIMS

Patients with unprovoked venous thromboembolism (VTE) have a high recurrence risk, and guidelines suggest extended-phase anticoagulation. Many patients never experience recurrence but are exposed to bleeding. The aim of this study was to assess the performance of the Vienna Prediction Model (VPM) and to evaluate if the VPM accurately identifies these patients.

METHODS

In patients with unprovoked VTE, the VPM was performed 3 weeks after anticoagulation withdrawal. Those with a predicted 1-year recurrence risk of ≤5.5% were prospectively followed. Study endpoint was recurrent VTE over 2 years.

RESULTS

A total of 818 patients received anticoagulation for a median of 3.9 months. 520 patients (65%) had a predicted annual recurrence risk of ≤5.5%. During a median time of 23.9 months, 52 patients had non-fatal recurrence. The recurrence risk was 5.2% [95% confidence interval (CI) 3.2-7.2] at 1 year and 11.2% (95% CI 8.3-14) at 2 years. Model calibration was adequate after 1 year. The VPM underestimated the recurrence risk of patients with a 2-year recurrence rate of >5%. In a post-hoc analysis, the VPM's baseline hazard was recalibrated. Bootstrap validation confirmed an ideal ratio of observed and expected recurrence events. The recurrence risk was highest in men with proximal deep-vein thrombosis or pulmonary embolism and lower in women regardless of the site of incident VTE.

CONCLUSIONS

In this prospective evaluation of the performance of the VPM, the 1-year rate of recurrence in patients with unprovoked VTE was 5.2%. Recalibration improved identification of patients at low recurrence risk and stratification into distinct low-risk categories.

Diagnostic clinical prediction rules for categorising low back pain: A systematic review

BACKGROUND

Low back pain (LBP) is a common complex condition, where specific diagnoses are hard to identify. Diagnostic clinical prediction rules (CPRs) are known to improve clinical decision-making. A review of LBP diagnostic-CPRs by Haskins et al. (2015) identified six diagnostic-CPRs in derivation phases of development, with one tool ready for implementation. Recent progress on these tools is unknown. Therefore, this review aimed to investigate developments in LBP diagnostic-CPRs and evaluate their readiness for implementation.

METHODS

A systematic review was performed on five databases (Medline, Amed, Cochrane Library, PsycInfo, and CINAHL) combined with hand-searching and citation-tracking to identify eligible studies. Study and tool quality were appraised for risk of bias (Quality Assessment of Diagnostic Accuracy Studies-2), methodological quality (checklist using accepted CPR methodological standards), and CPR tool appraisal (GRade and ASsess Predictive).

RESULTS

Of 5021 studies screened, 11 diagnostic-CPRs were identified. Of the six previously known, three have been externally validated but not yet undergone impact analysis. Five new tools have been identified since Haskin et al. (2015); all are still in derivation stages. The most validated diagnostic-CPRs include the Lumbar-Spinal-Stenosis-Self-Administered-Self-Reported-History-Questionnaire and Diagnosis-Support-Tool-to-Identify-Lumbar-Spinal-Stenosis, and the StEP-tool which differentiates radicular from axial-LBP.

CONCLUSIONS

This updated review of LBP diagnostic CPRs found five new tools, all in the early stages of development. Three previously known tools have now been externally validated but should be used with caution until impact evaluation studies are undertaken. Future funding should focus on externally validating and assessing the impact of existing CPRs on clinical decision-making.

Prediction models in prehospital and emergency medicine research: How to derive and internally validate a clinical prediction model

Clinical prediction models are created to help clinicians with medical decision making, aid in risk stratification, and improve diagnosis and/or prognosis. With growing availability of both prehospital and in-hospital observational registries and electronic health records, there is an opportunity to develop, validate, and incorporate prediction models into clinical practice. However, many prediction models have high risk of bias due to poor methodology. Given that there are no methodological standards aimed at developing prediction models specifically in the prehospital setting, the objective of this paper is to describe the appropriate methodology for the derivation and validation of clinical prediction models in this setting. What follows can also be applied to the emergency medicine (EM) setting. There are eight steps that should be followed when developing and internally validating a prediction model: (1) problem definition, (2) coding of predictors, (3) addressing missing data, (4) ensuring adequate sample size, (5) variable selection, (6) evaluating model performance, (7) internal validation, and (8) model presentation. Subsequent steps include external validation, assessment of impact, and cost-effectiveness. By following these steps, researchers can develop a prediction model with the methodological rigor and quality required for prehospital and EM research.

Risk assessment and clinical prediction model of planned transfer to the ICU after hip arthroplasty in elderly individuals

BACKGROUND

With the development of hip arthroplasty technology and rapid rehabilitation theory, the number of hip arthroplasties in elderly individuals is gradually increasing, and their satisfaction with surgery is also gradually improving. However, for elderly individuals, many basic diseases, poor nutritional status, the probability of surgery, anaesthesia and postoperative complications cannot be ignored. How to reduce the incidence of postoperative complications, optimize medical examination for elderly patients, and reasonably allocate medical resources. This study focuses on the construction of a clinical prediction model for planned transfer to the ICU after hip arthroplasty in elderly individuals.

METHODS

We retrospectively analysed 325 elderly patients who underwent hip arthroplasty. The general data and preoperative laboratory test results of the patients were collected. Univariate and multivariate logistic regression analyses were performed to screen independent influencing factors. The backwards LR method was used to establish the prediction model. Then, we assessed and verified the degree of discrimination, calibration and clinical usefulness of the model. Finally, the prediction model was rendered in the form of a nomogram.

RESULTS

Age, blood glucose, direct bilirubin, glutamic-pyruvic transaminase, serum albumin, prothrombin time and haemoglobin were independent influencing factors of planned transfer to the ICU after hip arthroplasty. The area under the curve (AUC) of discrimination and the 500 bootstrap internal validation AUC of this prediction model was 0.793. The calibration curve fluctuated around the ideal curve and had no obvious deviation from the ideal curve. When the prediction probability was 12%-80%, the clinical decision curve was above two extreme lines. The discrimination, calibration and clinical applicability of this prediction model were good. The clinical prediction model was compared with the seven factors in the model for discrimination and clinical use. The discrimination and clinical practicability of this prediction model were superior to those of the internal factors.

CONCLUSION

The prediction model has good clinical prediction ability and clinical practicability. The model is presented in the form of a linear graph, which provides an effective reference for the individual risk assessment of patients.

Prediction of very early major bleeding risk in acute pulmonary embolism: an independent external validation of the Pulmonary Embolism-Syncope, Anemia, and Renal Dysfunction (PE-SARD) bleeding score

BACKGROUND

The Pulmonary Embolism-Syncope, Anemia, and Renal Dysfunction (PE-SARD) bleeding score was derived to predict very early major bleeding (MB) in patients with acute pulmonary embolism (PE). Before adoption into practice, the score requires external validation in different populations.

OBJECTIVES

We independently validated the PE-SARD score in a prospective multicenter Swiss cohort of 687 patients aged ≥65 years with acute PE.

METHODS

The PE-SARD score uses 3 variables (syncope, anemia, and renal dysfunction) to classify patients into 3 categories of increasing bleeding risk. The outcomes were very early MB at 7 days (primary) and MB at later time points (secondary). We calculated the PE-SARD score for each patient and classified the proportion of patients as being at low, intermediate, and high risk. To assess discrimination and calibration, we calculated the area under the receiver operating characteristic curve and the Hosmer-Lemeshow goodness-of-fit test, respectively.

RESULTS

The prevalence of MB was 2.0% (14/687) at 7 days and 14.0% (96/687) after a median follow-up of 30 months. The PE-SARD score classified 40.2%, 42.2%, and 17.6% of patients as low, intermediate, and high risk for MB, respectively. The frequency of observed very early MB at 7 days was 1.8% in low-, 2.1% in intermediate-, and 2.5% in high-risk patients. The area under the receiver operating characteristic curve was 0.52 (95% CI, 0.48-0.56) at 7 days and increased to 0.60 (95% CI, 0.56-0.64) at the end of follow-up. Score calibration was adequate (p > .05) over the entire follow-up.

CONCLUSION

In our independent validation, the PE-SARD score did not accurately predict very early MB and may not be transportable to older patients with PE.

Development of a Model for Preliminary Diagnosis of Human Granulocytic Anaplasmosis

Background: Human granulocytic anaplasmosis (HGA) is a vector-borne natural focal disease that is not officially registered in Ukraine. The first 13 cases of HGA in adults in Ukraine were identified in 2007. The purpose of our study was to develop a predictive model of HGA based on clinical and laboratory characteristics to develop a three-level standard case definition of HGA. Materials and Methods: Researchers examined 498 patients with suspected tick-borne infections and carried out a retrospective clinical and epidemiological analysis of 60 cases recruited from Lviv regional infectious disease hospitals. Logistic regression was used to create a model of the probability of the diagnosis of HGA depending on the presence of certain clinical and laboratory factors that, when examined, together may help to confirm a case of HGA. For logistic regression, eight clinical and laboratory factors were selected: history of tick bite, hyperthermia, signs of pharyngitis, changes in chest X-ray picture (enhancement of the pulmonary pattern and enlargement of the lung root boundaries), increased bilirubin (˃21 μmol/L), increased alanine aminotransferase (ALT ˃36 U/L), erythema migrans, and detected Lyme disease. Results: In the presence of all eight factors, the probability of HGA is 95.7%. When the five main signs are absent-signs of pharyngitis, changes in chest X-ray picture, increased bilirubin and ALT, and a history of tick bite-the probability of HGA in the patient dramatically decreases to 6.8%, meaning that HGA might be excluded. Conclusions: Based on the analysis of epidemiological, clinical, and laboratory signs, criteria for establishing a suspected, probable, and confirmed diagnosis of HGA have been developed to improve diagnosis.

Prediction of in-hospital bleeding in acutely ill medical patients: External validation of the IMPROVE bleeding risk score

INTRODUCTION

Pharmacological thromboprophylaxis slightly increases bleeding risk. The only risk assessment model to predict bleeding in medical inpatients, the IMPROVE bleeding risk score, has never been validated using prospectively collected outcome data.

METHODS

We validated the IMPROVE bleeding risk score in a prospective multicenter cohort of medical inpatients. Primary outcome was in-hospital clinically relevant bleeding (CRB) within 14 days of admission, a secondary outcome was major bleeding (MB). We classified patients according to the score in high or low bleeding risk. We assessed the score's predictive performance by calculating subhazard ratios (sHRs) adjusted for thromboprophylaxis use, positive and negative predictive values (PPV, NPV), and the area under the receiver operating characteristic curves (AUC).

RESULTS

Of 1155 patients, 8 % were classified as high bleeding risk. CRB and MB within 14 days occurred in 0.94 % and 0.47 % of low-risk and in 5.6 % and 3.4 % of high-risk patients, respectively. Adjusted for thromboprophylaxis, classification in the high-risk group was associated with an increased risk of 14-day CRB (sHR 4.7, 95 % confidence interval [CI] 1.5-14.5) and MB (sHR 4.9, 95%CI 1.0-23.4). PPV was 5.6 % and 3.4 %, while NPV was 99.1 % and 99.5 % for CRB and MB, respectively. The AUC was 0.68 (95%CI 0.66-0.71) for CRB and 0.73 (95%CI 0.71-0.76) for MB.

CONCLUSION

The IMPROVE bleeding risk score showed moderate to good discriminatory power to predict bleeding in medical inpatients. The score may help identify patients at high risk of in-hospital bleeding, in whom careful assessment of the risk-benefit ratio of pharmacological thromboprophylaxis is warranted.

Semantic Visualization in Functional Recovery Prediction of Intravenous Thrombolysis following Acute Ischemic Stroke in Patients by Using Biostatistics: An Exploratory Study

(1) Background: Intravenous thrombolysis following acute ischemic stroke (AIS) can reduce disability and increase the survival rate. We designed a functional recovery analysis by using semantic visualization to predict the recovery probability in AIS patients receiving intravenous thrombolysis; (2) Methods: We enrolled 131 AIS patients undergoing intravenous thrombolysis from 2011 to 2015 at the Medical Center in northern Taiwan. An additional 54 AIS patients were enrolled from another community hospital. A modified Rankin Score ≤2 after 3 months of follow-up was defined as favorable recovery. We used multivariable logistic regression with forward selection to construct a nomogram; (3) Results: The model included age and the National Institutes of Health Stroke Scale (NIHSS) score as immediate pretreatment parameters. A 5.23% increase in the functional recovery probability occurred for every 1-year reduction in age, and a 13.57% increase in the functional recovery probability occurred for every NIHSS score reduction. The sensitivity, specificity, and accuracy of the model in the validation dataset were 71.79%, 86.67%, and 75.93%, respectively, and the area under the receiver operating characteristic curve (AUC) was 0.867; (4) Conclusions: Semantic visualization-based functional recovery prediction models may help physicians assess the recovery probability before patients undergo emergency intravenous thrombolysis.

Risk of thrombosis in essential thrombocythemia according to three prediction models: an external validation study

BACKGROUND

Thrombosis is a major complication of essential thrombocythemia (ET). There are three well-known prediction models for thrombotic risk in ET patients. However, only few external validation studies for the performance of these models in Asian populations have been conducted. Thus, we aimed to evaluate the performance of these models for predicting the risk of thrombosis in Thai patients with ET.

METHODS

We retrospectively evaluated the clinical characteristics and thrombotic risk of 149 Thai ET patients in a university hospital in Southern Thailand between 2002 and 2019. Thrombotic risk variables were evaluated using Cox proportional hazard regression. The Brier score, calibration plot, and Harrel concordance index (C-index) were used to evaluate the performance of the three models.

RESULTS

With a median follow-up of 5.2 years, there were a total of 28 thrombotic events in 26 patients. Age > 60 years was a significant prognostic factor for thrombosis in the multivariate Cox regression analysis. The Brier scores were 0.251, 0.273, and 0.276 in the conventional, IPSET-thrombosis, and revised IPSET-thrombosis models, respectively. The conventional model had optimal calibration and good discrimination (C-index, 0.67; 95%CI:0.55-0.79). The IPSET thrombosis (C-index 0.33; 95%CI:0.20-0.49) and revised IPSET thrombosis (C-index 0.31; 95%CI:0.18-0.44) models showed poor discrimination.

CONCLUSION

The conventional model, which is based on age and history of thrombosis, is the best model to predict thrombotic risk in Thai ET patients. Further studies with a larger number of patients with thrombotic events are needed to validate the IPSET-thrombosis and revised IPSET-thrombosis models.

Prediction of intrapartum fever using continuously monitored vital signs and heart rate variability

Objectives

Neonatal early onset sepsis (EOS), bacterial infection during the first seven days of life, is difficult to diagnose because presenting signs are non-specific, but early diagnosis before birth can direct life-saving treatment for mother and baby. Specifically, maternal fever during labor from placental infection is the strongest predictor of EOS. Alterations in maternal heart rate variability (HRV) may precede development of intrapartum fever, enabling incipient EOS detection. The objective of this work was to build a predictive model for intrapartum fever.

Methods

Continuously measured temperature, heart rate, and beat-to-beat RR intervals were obtained from wireless sensors on women (n = 141) in labor; traditional manual vital signs were taken every 3-6 hours. Validated measures of HRV were calculated in moving 5-minute windows of RR intervals: standard deviation of normal-to-normal intervals (SDNN) and root mean square of successive differences (RMSSD) between normal heartbeats.

Results

Fever (>38.0 °C) was detected by manual or continuous measurements in 48 women. Compared to afebrile mothers, average SDNN and RMSSD in febrile mothers decreased significantly (p < 0.001) at 2 and 3 hours before fever onset, respectively. This observed HRV divergence and raw recorded vitals were applied to a logistic regression model at various time horizons, up to 4-5 hours before fever onset. Model performance increased with decreasing time horizons, and a model built using continuous vital signs as input variables consistently outperformed a model built from episodic vital signs.

Conclusions

HRV-based predictive models could identify mothers at risk for fever and infants at risk for EOS, guiding maternal antibiotic prophylaxis and neonatal monitoring.

Types and associated methodologies of overviews of reviews in health care: a methodological study with published examples

OBJECTIVES

To provide a descriptive insight into the different types of research questions/objectives and associated methodologies of overviews of reviews, supplemented by representative examples from the health care literature.

STUDY DESIGN AND SETTING

We searched in methodological articles for information on types and methodologies used in overviews and we explored the typology of reviews to identify similar types in literature of overviews. We categorized the types of overviews based on the research question/objective and the methodological approach used. Indicative examples for each category were selected from a sample of 2,121 overviews that were retrieved between 2000 and 2022 from MEDLINE, Scopus, and Cochrane Database of Systematic Reviews.

RESULTS

Based on type of research question, overviews were classified as overviews of reviews of interventions, associations, prediction, diagnostic accuracy, prevalence/incidence, experiences/views, economic evaluation, and measurement properties. Based on the methodological approach, we identified a variety of methods (systematic, living, rapid, scoping, evidence mapping, framework, and methodological) used in overviews.

CONCLUSION

The proposed classification and examples provide an essential starting point for future theory-building research on typologies and study designs of overviews of reviews. It is important for methodologists to make vigorous effort to create consensus-based methodological and reporting guidelines to cover these diverse types and key methodological challenges.

Classification of breast cancer recurrence based on imputed data: a simulation study

Several studies have been conducted to classify various real life events but few are in medical fields; particularly about breast recurrence under statistical techniques. To our knowledge, there is no reported comparison of statistical classification accuracy and classifiers' discriminative ability on breast cancer recurrence in presence of imputed missing data. Therefore, this article aims to fill this analysis gap by comparing the performance of binary classifiers (logistic regression, linear and quadratic discriminant analysis) using several datasets resulted from imputation process using various simulation conditions. Our study aids the knowledge about how classifiers' accuracy and discriminative ability in classifying a binary outcome variable are affected by the presence of imputed numerical missing data. We simulated incomplete datasets with 15, 30, 45 and 60% of missingness under Missing At Random (MAR) and Missing Completely At Random (MCAR) mechanisms. Mean imputation, hot deck, k-nearest neighbour, multiple imputations via chained equation, expected-maximisation, and predictive mean matching were used to impute incomplete datasets. For each classifier, correct classification accuracy and area under the Receiver Operating Characteristic (ROC) curves under MAR and MCAR mechanisms were compared. The linear discriminant classifier attained the highest classification accuracy (73.9%) based on mean-imputed data at 45% of missing data under MCAR mechanism. As a classifier, the logistic regression based on predictive mean matching imputed-data yields the greatest areas under ROC curves (0.6418) at 30% missingness while k-nearest neighbour tops the value (0.6428) at 60% of missing data under MCAR mechanism.

Prospective cohort study of psoriatic arthritis risk in patients with psoriasis in a real-world psoriasis registry

BACKGROUND

The characteristics that predict the onset of psoriatic arthritis (PsA) among patients with psoriasis (PsO) may inform diagnosis and treatment.

OBJECTIVE

To develop a model to predict the 2-year risk of developing PsA among patients with PsO.

METHODS

This was a prospective cohort study of patients in the CorEvitas Psoriasis Registry without PsA at enrollment and with 24-month follow-up. Unregularized and regularized logistic regression models were developed and tested using descriptive variables to predict dermatologist-identified PsA at 24 months. Model performance was compared using the area under the receiver operating characteristic curve (AUC), sensitivity, and specificity.

RESULTS

A total of 1489 patients were included. Nine unique predictive models were developed and tested. The optimal model, including Psoriasis Epidemiology Screening Tool (PEST), body mass index (BMI), modified Rheumatic Disease Comorbidity Index, work status, alcohol use, and patient-reported fatigue, predicted the onset of PsA within 24 months (AUC = 68.9%, sensitivity = 82.9%, specificity = 48.8%). A parsimonious model including PEST and BMI had similar performance (AUC = 68.8%; sensitivity = 92.7%, specificity = 36.5%).

LIMITATIONS

PsA misclassification bias by dermatologists.

CONCLUSION

PEST and BMI were important factors in predicting the development of PsA in patients with PsO over 2 years and thereby foundational for future PsA risk model development.

Development and internal validation of a clinical prediction model for the diagnosis of immune thrombocytopenia

BACKGROUND

Immune thrombocytopenia (ITP) is a diagnosis of exclusion that can resemble other thrombocytopenic disorders.

OBJECTIVES

To develop a clinical prediction model (CPM) for the diagnosis of ITP to aid hematogists in investigating patients presenting with undifferentiated thrombocytopenia.

METHODS

We designed a CPM for ITP diagnosis at the time of the initial hematology consultation using penalized logistic regression based on data from patients with thrombocytopenia enrolled in the McMaster ITP registry (n = 523) called the Predict-ITP Tool. The case definition for ITP was a platelet count less than 100 × 109 /L and a platelet count response after high-dose corticosteroids or intravenous immune globulin, defined as the achievement of a platelet count above 50 × 109 /L and at least a doubling of baseline. Internal validation was done using bootstrap resampling. Model discrimination was assessed by the c-statistic, and calibration was assessed by the calibration slope, calibration-in-the-large, and calibration plot.

RESULTS

The final model included the following variables: (1) platelet count variability (based on three or more platelet count values), (2) lowest platelet count value, (3) maximum mean platelet volume, and (4) history of major bleeding (defined by the ITP bleeding scale). The optimism-corrected c-statistic was 0.83, the calibration slope was 0.88, and calibration-in-the-large for all performance measures was <0.001 with standard error <0.001, indicating good discrimination and excellent calibration.

CONCLUSIONS

The Predict-ITP Tool can estimate the likelihood of ITP for a given patient with thrombocytopenia at the time of the initial hematology consultation. The tool had high predictive accuracy for the diagnosis of ITP.

Suicidal Ideation in the Australian Construction Industry: Prevalence and the Associations of Psychosocial Job Adversity and Adherence to Traditional Masculine Norms

BACKGROUND

Suicide in the Australian Construction Industry (ACI) is a significant issue, however minimal understanding of suicidal ideation prevalence, as well as the potential role psychosocial job adversity and increased adherence to traditional masculine norms may play in its presence, is apparent.

METHOD

A representative sample of Australian men (n = 11,132) were used to create initial understandings of prevalence of suicidal ideation (past two weeks), psychosocial job adversities and level of adherence to traditional masculine norms for the ACI (n = 1721) in comparison to a general population comprised of the remaining employed males from Other Industries (n = 9411). Additionally, due to their reported increased suicide vulnerability investigation of associations between suicidal ideation, psychosocial job adversities and adherence to traditional masculine norms for the ACI were undertaken.

RESULTS

No difference in suicidal ideation prevalence was reported between the ACI and those employed in Other Industries (p > 0.05), however, increased prevalence of psychosocial job adversities (p ≤ 0.001) and adherence to traditional masculine norms (p ≤ 0.001) for the ACI was seen. Significant multivariate associations between suicidal ideation, psychosocial job adversities (OR = 1.79, 95%CI [1.12-2.85]) and two domains of traditional masculine norms, self-reliance (OR = 1.29, 95%CI [1.09-1.51]) and risk-taking (OR = 1.20, 95%CI [1.01-1.41]), were reported.

CONCLUSION

Results suggest need for increased understanding of later stage suicidal trajectory drivers in the ACI. Findings indicate need for prevention group/industry concentration on mitigation of psychosocial job adversities, as well as a more nuanced and increased discussion of the negative role of self-reliance and risk-taking domains of traditional masculine norms may play in ACI suicidal ideation, as opposed to the construct as a whole.

A framework for evaluating predictive models

Predictive models provide estimates on an individual's probability of having a disease or developing a disease/disease outcome. Clinicians often use them to support clinical decision-making. Many prediction models are published annually; online versions of models (such as MDCalc and QxMD) facilitate their use at the point of care. However, before using a model, the clinician should first establish that the model has undergone external validation demonstrating satisfactory predictive performance. Ideally, the model should also demonstrate improved outcomes from an impact analysis. This article summarizes the basic steps of predictive model evaluation, and is followed by an application example.

Prognostic Modelling Studies of Coronary Heart Disease—A Systematic Review of Conventional and Genetic Risk Factor Studies

This study aims to provide an overview of multivariable prognostic modelling studies developed for coronary heart disease (CHD) in the general population and to explore the optimal prognostic model by comparing the models’ performance. A systematic review was performed using Embase, PubMed, Cochrane, Web of Science, and Scopus databases until 30 November 2019. In this work, only prognostic studies describing conventional risk factors alone or a combination of conventional and genomic risk factors, being developmental and/or validation prognostic studies of a multivariable model, were included. A total of 4021 records were screened by titles and abstracts, and 72 articles were eligible. All the relevant studies were checked by comparing the discrimination, reclassification, and calibration measures. Most of the models were developed in the United States and Canada and targeted the general population. The models included a set of similar predictors, such as age, sex, smoking, cholesterol level, blood pressure, BMI, and diabetes mellitus. In this study, many articles were identified and screened for consistency and reliability using CHARM and GRIPS statements. However, the usefulness of most prognostic models was not demonstrated; only a limited number of these models supported clinical evidence. Unfortunately, substantial heterogeneity was recognized in the definition and outcome of CHD events. The inclusion of genetic risk scores in addition to conventional risk factors might help in predicting the incidence of CHDs; however, the generalizability of the existing prognostic models remains open. Validation studies for the existing developmental models are needed to ensure generalizability, improve the research quality, and increase the transparency of the study.

A Comorbidity Knowledge-Aware Model for Disease Prognostic Prediction

Prognostic prediction is the task of estimating a patient's risk of disease development based on various predictors. Such prediction is important for healthcare practitioners and patients because it reduces preventable harm and costs. As such, a prognostic prediction model is preferred if: 1) it exhibits encouraging performance and 2) it can generate intelligible rules, which enable experts to understand the logic of the model's decision process. However, current studies usually concentrated on only one of the two features. Toward filling this gap, in the present study, we develop a novel knowledge-aware Bayesian model taking into consideration accuracy and transparency simultaneously. Real-world case studies based on four years' territory-wide electronic health records are conducted to test the model. The results show that the proposed model surpasses state-of-the-art prognostic prediction models in accuracy and c-statistic. In addition, the proposed model can generate explainable rules.

Prediction models of diabetes complications: a scoping review

BACKGROUND

Diabetes often places a large burden on people with diabetes (hereafter 'patients') and the society, that is, in part attributable to its complications. However, evidence from models predicting diabetes complications in patients remains unclear. With the collaboration of patient partners, we aimed to describe existing prediction models of physical and mental health complications of diabetes.

METHODS

Building on existing frameworks, we systematically searched for studies in Ovid-Medline and Embase. We included studies describing prognostic prediction models that used data from patients with pre-diabetes or any type of diabetes, published between 2000 and 2020. Independent reviewers screened articles, extracted data and narratively synthesised findings using established reporting standards.

RESULTS

Overall, 78 studies reported 260 risk prediction models of cardiovascular complications (n=42 studies), mortality (n=16), kidney complications (n=14), eye complications (n=10), hypoglycaemia (n=8), nerve complications (n=3), cancer (n=2), fracture (n=2) and dementia (n=1). Prevalent complications deemed important by patients such as amputation and mental health were poorly or not at all represented. Studies primarily analysed data from older people with type 2 diabetes (n=54), with little focus on pre-diabetes (n=0), type 1 diabetes (n=8), younger (n=1) and racialised people (n=10). Per complication, predictors vary substantially between models. Studies with details of calibration and discrimination mostly exhibited good model performance.

CONCLUSION

This rigorous knowledge synthesis provides evidence of gaps in the landscape of diabetes complication prediction models. Future studies should address unmet needs for analyses of complications n> and among patient groups currently under-represented in the literature and should consistently report relevant statistics. SCOPING REVIEW REGISTRATION: https://osf.io/fjubt/.

Machine learning for medical imaging: methodological failures and recommendations for the future

Research in computer analysis of medical images bears many promises to improve patients' health. However, a number of systematic challenges are slowing down the progress of the field, from limitations of the data, such as biases, to research incentives, such as optimizing for publication. In this paper we review roadblocks to developing and assessing methods. Building our analysis on evidence from the literature and data challenges, we show that at every step, potential biases can creep in. On a positive note, we also discuss on-going efforts to counteract these problems. Finally we provide recommendations on how to further address these problems in the future.

A risk prediction model for acute kidney injury in patients with pulmonary tuberculosis during anti-tuberculosis treatment

Background

Acute kidney injury (AKI) is not a rare complication during anti-tuberculosis treatment in some patients with pulmonary tuberculosis (PTB). We aimed to develop a risk prediction model for early recognition of patients with PTB at high risk for AKI during anti-TB treatment.

Methods

This retrospective cohort study assessed the clinical baseline, and laboratory test data of 315 inpatients with active PTB who were screened for predictive factors from January 2019 to June 2020. The elements were analyzed by logistic regression analysis. A nomogram was established by the results of the logistic regression analysis. The prediction model discrimination and calibration were evaluated by the concordance index (C-index), ROC curve, and Hosmer-Lemeshow analysis.

Results

A total of 315 patients with PTB were enrolled (67 patients with AKI and 248 patients without AKI). Seven factors, including microalbuminuria, hematuria, cystatin-C (CYS-C), albumin (ALB), creatinine-based estimated glomerular filtration rates (eGFRs), body mass index (BMI), and CA-125 were acquired to develop the predictive model. According to the logistic regression, microalbuminuria (OR = 3.038, 95%CI 1.168–7.904), hematuria (OR = 3.656, 95%CI 1.325–10.083), CYS-C (OR = 4.416, 95%CI 2.296–8.491), and CA-125 (OR = 3.93, 95%CI 1.436–10.756) were risk parameter, while ALB (OR = 0.741, 95%CI 0.650–0.844) was protective parameter. The nomogram demonstrated good prediction in estimating AKI (C-index= 0.967, AUC = 0.967, 95%CI (0.941–0.984), sensitivity = 91.04%, specificity = 93.95%, Hosmer-Lemeshow analysis SD = 0.00054, and quantile of absolute error = 0.049).

Conclusions

Microalbuminuria, hematuria, ALB reduction, elevated CYS-C, and CA-125 are predictive factors for the development of AKI in patients with PTB during anti-TB treatments. The predictive nomogram based on five predictive factors is achieved good risk prediction for AKI during anti-TB treatments.

A new horizon of precision medicine: combination of the microbiome and extracellular vesicles

Over several decades, the disease pattern of intractable disease has changed from acute infection to chronic disease accompanied by immune and metabolic dysfunction. In addition, scientific evidence has shown that humans are holobionts; of the DNA in humans, 1% is derived from the human genome, and 99% is derived from microbial genomes (the microbiome). Extracellular vesicles (EVs) are lipid bilayer-delimited nanoparticles and key messengers in cell-to-cell communication. Many publications indicate that microbial EVs are both positively and negatively involved in the pathogenesis of various intractable diseases, including inflammatory diseases, metabolic disorders, and cancers. Microbial EVs in feces, blood, and urine show significant differences in their profiles between patients with a particular disease and healthy subjects, demonstrating the potential of microbial EVs as biomarkers for disease diagnosis, especially for assessing disease risk. Furthermore, microbial EV therapy offers a variety of advantages over live biotherapeutics and human cell EV (or exosome) therapy for the treatment of intractable diseases. In summary, microbial EVs are a new tool in medicine, and microbial EV technology might provide us with innovative diagnostic and therapeutic solutions in precision medicine.

The tiny membrane-bound vesicles containing various biomolecules that the organisms comprising our microbiome release could offer a powerful tool for precision medicine. Our bodies are home to trillions of microbes, which interact closely with our tissues to maintain a healthy physiological environment. Yoon-Keun Kim of the Institute of MD Healthcare, Seoul, South Korea, and colleagues have reviewed current research into the extracellular vesicles that these microbes use to communicate with other microbes and their human hosts. The authors note that these vesicles affect tissues throughout the body, and their activities have been linked to various disorders including asthma, Crohn’s disease and cancer. A deeper understanding of how these vesicles prevent or accelerate various conditions in different individuals could yield useful new diagnostic biomarkers and provide the foundation for interventions that are optimized for each patient.

Composite Outcomes in Clinical Prediction Modeling: Are We Trying to Predict Apples and Oranges?

Composite outcomes are widely used in clinical research. Existing literature has considered the pros and cons of composite outcomes in clinical trials, but their extensive use in clinical prediction has received much less attention. Clinical prediction assists decision-making by directing patients with higher risks of adverse outcomes toward interventions that provide the greatest benefits to those at the greatest risk. In this article, we summarize our existing understanding of the advantages and disadvantages of composite outcomes, consider how these relate to clinical prediction, and highlight the problem of key predictors having markedly different associations with individual components of the composite outcome. We suggest that a "composite outcome fallacy" may occur when a clinical prediction model is based on strong associations between key predictors and one component of a composite outcome (such as mortality) and used to direct patients toward intervention when these predictors actually have an inverse association with a more relevant component of the composite outcome (such as the use of a lifesaving intervention). We propose that clinical prediction scores using composite outcomes should report their accuracy for key components of the composite outcome and examine for inconsistencies among predictor variables.

Prediction models for venous thromboembolism in ambulatory adults with pancreatic and gastro-oesophageal cancer: protocol for systematic review and meta-analysis

INTRODUCTION

Venous thromboembolism (VTE) is a common complication of cancer. Pancreatic and gastro-oesophageal cancers are among malignancies that have the highest rates of VTE occurrence. VTE can increase cancer-related morbidity and mortality and disrupt cancer treatment. The risk of VTE can be managed with measures such as using anticoagulant drugs, although the risk of bleeding may be an impeding factor. Therefore, a VTE risk assessment should be performed before the start of anticoagulation in individual patients. Several prediction models have been published, but most of them have low sensitivity and unknown clinical applicability in pancreatic or gastro-oesphageal cancers. We intend to do this systematic review to identify all applicable published predictive models and compare their performance in those types of cancer.

METHODS AND ANALYSIS

All studies in which a prediction model for VTE have been developed, validated or compared using adult ambulatory patients with pancreatic or gastro-oesphageal cancers will be identified and the reported predictive performance indicators will be extracted. Full text peer-reviewed journal articles of observational or experimental studies published in English will be included. Five databases (Medline, EMBASE, Web of Science, CINAHL and Cochrane) will be searched. Two reviewers will independently undertake each of the phases of screening, data extraction and risk of bias assessment. The quality of the selected studies will be assessed using Prediction model Risk Of Bias Assessment Tool. The results from the review will be used for a narrative information synthesis, and if the same models have been validated in multiple studies, meta-analyses will be done to pool the predictive performance measures.

ETHICS AND DISSEMINATION

There is no need for ethics approval because the review will use previously peer-reviewed articles. The results will be published.

PROSPERO REGISTRATION NUMBER

CRD42021253887.

Rational development and application of biomarkers in the field of autoimmunity: A conceptual framework guiding clinicians and researchers

Clear guidance is needed in the development and implementation of laboratory biomarkers in medicine. So far, no standardized phased approach is established that would pilot researchers and clinicians in this process. This leads to often incompletely validated biomarkers, which can bear the consequence of wrong applications, misinterpretation and inadequate management in the clinical context.

In this conceptual article, we describe a stepwise approach to develop and comprehensively validate laboratory biomarkers. We will delineate basic steps including technical performance, pre-analytical issues, and biological variation, as well as advanced aspects of biomarker utility comprising interpretability, diagnostic and prognostic accuracy, and health-care outcomes. These aspects will be illustrated by using well-known examples from the field of immunology.

The application of this conceptual framework will guide researchers in conducting meaningful projects to develop and evaluate biomarkers for the use in clinical practice. Furthermore, clinicians will be able to adequately interpret pre-clinical and clinical diagnostic literature and rationally apply biomarkers in clinical practice. Improvement in the implementation and application of biomarkers might relevantly change the management and outcomes of our patients for the better.

A prediction model for early systemic recurrence in breast cancer using a molecular diagnostic analysis of sentinel lymph nodes: A large-scale, multicenter cohort study

BACKGROUND

The one-step nucleic acid amplification (OSNA) assay can quantify the cytokeratin 19 messenger RNA copy number as a proxy for sentinel lymph node (SN) metastasis in breast cancer. A large-scale, multicenter cohort study was performed to determine the prognostic value of the SN tumor burden based on a molecular readout and to establish a model for the prediction of early systemic recurrence in patients using the OSNA assay.

METHODS

SN biopsies from 4757 patients with breast cancer were analyzed with the OSNA assay. The patients were randomly assigned to the training or validation cohort at a ratio of 2:1. On the basis of the training cohort, the threshold SN tumor burden value for stratifying distant recurrence was determined with Youden's index; predictors of distant recurrence were investigated via multivariable analyses. Based on the selected predictors, a model for estimating 5-year distant recurrence-free survival was constructed, and predictive performance was measured with the validation cohort.

RESULTS

The prognostic cutoff value for the SN tumor burden was 1100 copies/μL. The following variables were significantly associated with distant recurrence and were used to construct the prediction model: SN tumor burden, age, pT classification, grade, progesterone receptor, adjuvant cytotoxic chemotherapy, and adjuvant anti-human epidermal growth factor receptor 2 therapy. The values for the area under the curve, sensitivity, specificity, and accuracy of the prediction model were 0.83, 63.4%, 81.7%, and 81.1%, respectively.

CONCLUSIONS

Using the OSNA assay, the molecular readout-based SN tumor burden is an independent prognostic factor for early breast cancer. This model accurately predicts early systemic recurrence and may facilitate decision-making related to treatment.

A Surgeon's Guide to Understanding Artificial Intelligence and Machine Learning Studies in Orthopaedic Surgery

PURPOSE OF REVIEW

In recent years, machine learning techniques have been increasingly utilized across medicine, impacting the practice and delivery of healthcare. The data-driven nature of orthopaedic surgery presents many targets for improvement through the use of artificial intelligence, which is reflected in the increasing number of publications in the medical literature. However, the unique methodologies utilized in AI studies can present a barrier to its widespread acceptance and use in orthopaedics. The purpose of our review is to provide a tool that can be used by practitioners to better understand and ultimately leverage AI studies.

RECENT FINDINGS

The increasing interest in machine learning across medicine is reflected in a greater utilization of AI in recent medical literature. The process of designing machine learning studies includes study design, model choice, data collection/handling, model development, training, testing, and interpretation. Recent studies leveraging ML in orthopaedics provide useful examples for future research endeavors. This manuscript intends to create a guide discussing the use of machine learning and artificial intelligence in orthopaedic surgery research. Our review outlines the process of creating a machine learning algorithm and discusses the different model types, utilizing examples from recent orthopaedic literature to illustrate the techniques involved.

Endothelial Cell Activation and Thrombin Generation Assessment for the Risk of Severe Early Onset Preeclampsia. the ROADMAP-EOP Study

Background

The ROADMAP-EOP study aimed to identify clinically relevant biomarkers of hypercoagulability for the identification of pregnant women at risk of early onset preeclampsia worsening.

Methods

The ROADMAP-EOP observational single center retrospective case–control study was conducted in Greece (Centre for Human Reproduction, Genesis Athens Clinic, Athens, Greece) from July 2020 to July and enrolled pregnant women diagnosed with EOP stratified in mild EOP group (n = 34) and severe EOP group (n = 15) as well as women with uncomplicated pregnancy (control group; n = 35). All women were assessed with thromboelastometry (ROTEM®), Calibrated Automated Thrombogram®, tissue factor activity (TFa), procoagulant phospholipid dependentclotting time (Procoag-PPL®), Proteins S (PS), TFPI, D-dimer, antithrombin (AT), thrombomodulin (TM), fibrinogen, prothrombin time (PT) and activated partial thromboplastin time (aPTT). The primary study end-point was severe earlyonset preeclampsia. Principal component analysis (PCA) was performed.

Results

The PCA analysis showed that a score composed of the lag-time, ttPeak and Procoag-PPL accurately predicted severe EOP (sensitivity 71.4%, specificity 61.8%, and AUC of the ROC analysis 0.953).

Conclusion

The pilot ROADMAP-EOP shows that activation of endothelial cells and blood hypercoagulability are driven events in the worsening of EOP. Among a large panel of biomarkers and coagulation assays, thrombingeneration test and procoagulant phospholipid dependent clotting time emerged as clinically relevant for the evaluation of the risk of severe EOP. This methodology for the development of a new clinic-biological risk assessment model for prompt identification of pregnant women at risk of severe EOP must be validated in a large multi-centerprospective study.

Risk assessment and prediction model of renal damage in childhood immunoglobulin A vasculitis

OBJECTIVES

To explore the risk factors for renal damage in childhood immunoglobulin A vasculitis (IgAV) within 6 months and construct a clinical model for individual risk prediction.

METHODS

We retrospectively analyzed the clinical data of 1,007 children in our hospital and 287 children in other hospitals who were diagnosed with IgAV. Approximately 70% of the cases in our hospital were randomly selected using statistical product service soltions (SPSS) software for modeling. The remaining 30% of the cases were selected for internal verification, and the other hospital's cases were reviewed for external verification. A clinical prediction model for renal damage in children with IgAV was constructed by analyzing the modeling data through single-factor and multiple-factor logistic regression analyses. Then, we assessed and verified the degree of discrimination, calibration and clinical usefulness of the model. Finally, the prediction model was rendered in the form of a nomogram.

RESULTS

Age, persistent cutaneous purpura, erythrocyte distribution width, complement C₃, immunoglobulin G and triglycerides were independent influencing factors of renal damage in IgAV. Based on these factors, the area under the curve (AUC) for the prediction model was 0.772; the calibration curve did not significantly deviate from the ideal curve; and the clinical decision curve was higher than two extreme lines when the prediction probability was ~15-82%. When the internal and external verification datasets were applied to the prediction model, the AUC was 0.729 and 0.750, respectively, and the Z test was compared with the modeling AUC, P > 0.05. The calibration curves fluctuated around the ideal curve, and the clinical decision curve was higher than two extreme lines when the prediction probability was 25~84% and 14~73%, respectively.

CONCLUSION

The prediction model has a good degree of discrimination, calibration and clinical usefulness. Either the internal or external verification has better clinical efficacy, indicating that the model has repeatability and portability.

CLINICAL TRIAL REGISTRATION

www.chictr.org.cn, identifier ChiCTR2000033435.

Comparing LASSO and random forest models for predicting neurological dysfunction among fluoroquinolone users

BACKGROUND

Fluoroquinolones are associated with central (CNS) and peripheral (PNS) nervous system symptoms, and predicting the risk of these outcomes may have important clinical implications. Both LASSO and random forest are appealing modeling methods, yet it is not clear which method performs better for clinical risk prediction.

PURPOSE

To compare models developed using LASSO versus random forest for predicting neurological dysfunction among fluoroquinolone users.

METHODS

We developed and validated risk prediction models using claims data from a commercially insured population. The study cohort included adults dispensed an oral fluoroquinolone, and outcomes were CNS and PNS dysfunction. Model predictors included demographic variables, comorbidities and medications known to be associated with neurological symptoms, and several healthcare utilization predictors. We assessed the accuracy and calibration of these models using measures including AUC, calibration curves, and Brier scores.

RESULTS

The underlying cohort contained 16 533 (1.18%) individuals with CNS dysfunction and 46 995 (3.34%) individuals with PNS dysfunction during 120 days of follow-up. For CNS dysfunction, LASSO had an AUC of 0.81 (95% CI: 0.80, 0.82), while random forest had an AUC of 0.80 (95% CI: 0.80, 0.81). For PNS dysfunction, LASSO had an AUC of 0.75 (95% CI: 0.74, 0.76) versus an AUC of 0.73 (95% CI: 0.73, 0.74) for random forest. Both LASSO models had better calibration, with Brier scores 0.17 (LASSO) versus 0.20 (random forest) for CNS dysfunction and 0.20 (LASSO) versus 0.25 (random forest) for PNS dysfunction.

CONCLUSIONS

LASSO outperformed random forest in predicting CNS and PNS dysfunction among fluoroquinolone users, and should be considered for modeling when the cohort is modest in size, when the number of model predictors is modest, and when predictors are primarily binary.

Optimal surveillance strategies for patients with stage 1 cutaneous melanoma post primary tumour excision: three systematic reviews and an economic model

BACKGROUND

Malignant melanoma is the fifth most common cancer in the UK, with rates continuing to rise, resulting in considerable burden to patients and the NHS.

OBJECTIVES

The objectives were to evaluate the effectiveness and cost-effectiveness of current and alternative follow-up strategies for stage IA and IB melanoma.

REVIEW METHODS

Three systematic reviews were conducted. (1) The effectiveness of surveillance strategies. Outcomes were detection of new primaries, recurrences, metastases and survival. Risk of bias was assessed using the Cochrane Collaboration's Risk-of-Bias 2.0 tool. (2) Prediction models to stratify by risk of recurrence, metastases and survival. Model performance was assessed by study-reported measures of discrimination (e.g. D-statistic, Harrel's c-statistic), calibration (e.g. the Hosmer-Lemeshow 'goodness-of-fit' test) or overall performance (e.g. Brier score, R ²). Risk of bias was assessed using the Prediction model Risk Of Bias ASsessment Tool (PROBAST). (3) Diagnostic test accuracy of fine-needle biopsy and ultrasonography. Outcomes were detection of new primaries, recurrences, metastases and overall survival. Risk of bias was assessed using the Quality Assessment of Diagnostic Accuracy Studies-2 (QUADAS-2) tool. Review data and data from elsewhere were used to model the cost-effectiveness of alternative surveillance strategies and the value of further research.

RESULTS

(1) The surveillance review included one randomised controlled trial. There was no evidence of a difference in new primary or recurrence detected (risk ratio 0.75, 95% confidence interval 0.43 to 1.31). Risk of bias was considered to be of some concern. Certainty of the evidence was low. (2) Eleven risk prediction models were identified. Discrimination measures were reported for six models, with the area under the operating curve ranging from 0.59 to 0.88. Three models reported calibration measures, with coefficients of ≥ 0.88. Overall performance was reported by two models. In one, the Brier score was slightly better than the American Joint Committee on Cancer scheme score. The other reported an R ² of 0.47 (95% confidence interval 0.45 to 0.49). All studies were judged to have a high risk of bias. (3) The diagnostic test accuracy review identified two studies. One study considered fine-needle biopsy and the other considered ultrasonography. The sensitivity and specificity for fine-needle biopsy were 0.94 (95% confidence interval 0.90 to 0.97) and 0.95 (95% confidence interval 0.90 to 0.97), respectively. For ultrasonography, sensitivity and specificity were 1.00 (95% confidence interval 0.03 to 1.00) and 0.99 (95% confidence interval 0.96 to 0.99), respectively. For the reference standards and flow and timing domains, the risk of bias was rated as being high for both studies. The cost-effectiveness results suggest that, over a lifetime, less intensive surveillance than recommended by the National Institute for Health and Care Excellence might be worthwhile. There was considerable uncertainty. Improving the diagnostic performance of cancer nurse specialists and introducing a risk prediction tool could be promising. Further research on transition probabilities between different stages of melanoma and on improving diagnostic accuracy would be of most value.

LIMITATIONS

Overall, few data of limited quality were available, and these related to earlier versions of the American Joint Committee on Cancer staging. Consequently, there was considerable uncertainty in the economic evaluation.

CONCLUSIONS

Despite adoption of rigorous methods, too few data are available to justify changes to the National Institute for Health and Care Excellence recommendations on surveillance. However, alternative strategies warrant further research, specifically on improving estimates of incidence, progression of recurrent disease; diagnostic accuracy and health-related quality of life; developing and evaluating risk stratification tools; and understanding patient preferences.

STUDY REGISTRATION

This study is registered as PROSPERO CRD42018086784.

FUNDING

This project was funded by the National Institute for Health Research Health Technology Assessment programme and will be published in full in Health Technology Assessment; Vol 25, No. 64. See the NIHR Journals Library website for further project information.

Statistical models to preoperatively predict operative difficulty in laparoscopic cholecystectomy: A systematic review

BACKGROUND

Laparoscopic cholecystectomy operative difficulty is highly variable and influences outcomes. This systematic review analyzes the performance and clinical value of statistical models to preoperatively predict laparoscopic cholecystectomy operative difficulty.

METHODS

PRISMA guidelines were followed. PubMed, Embase, and the Cochrane Library were searched until June 2020. Primary studies developing or validating preoperative models predicting laparoscopic cholecystectomy operative difficulty in cohorts of >100 patients were included. Studies not reporting performance metrics or enough information for clinical implementation were excluded. Data were extracted according to CHARMS, and study quality was assessed using the PROBAST tool.

RESULTS

In total, 2,654 articles were identified, and 22 met eligibility criteria. Eighteen were model development, whereas 4 were validation studies. Eighteen studies were at high risk of bias. However, 11 studies showed low concern for applicability. Identified models predict 9 definitions of laparoscopic cholecystectomy operative difficulty, the most common being conversion to open surgery and operating time. The most validated models predict an intraoperative difficulty scale and procedures >90 minutes with an area under the curve of >0.70 and >0.76, respectively. Commonly used predictors include demographic variables such as age and gender (9/18 models) and ultrasound findings such as gallbladder wall thickness (11/18). Clinical implementation was never studied.

CONCLUSION

There is a longstanding interest in estimating laparoscopic cholecystectomy operative difficulty. Models to preoperatively predict laparoscopic cholecystectomy operative difficulty have generally good performance and seem applicable. However, an unambiguous definition of operative difficulty, validations, and clinical studies are needed to implement patients' stratification in laparoscopic cholecystectomy.

Early-life childhood obesity risk prediction: A Danish register-based cohort study exploring the predictive value of infancy weight gain

BACKGROUND

Information on postnatal weight gain is important for predicting later overweight and obesity, but it is unclear whether inclusion of this postnatal predictor improves the predictive performance of a comprehensive model based on prenatal and birth-related predictors.

OBJECTIVES

To compare performance of prediction models based on predictors available at birth, with and without information on infancy weight gain during the first year when predicting childhood obesity risk.

METHODS

A Danish register-based cohort study including 55.041 term children born between January 2004 and July 2011 with birthweight >2500 g registered in The Children's Database was used to compare model discrimination, reclassification, sensitivity and specificity of two models predicting risk of childhood obesity at school age. Each model consisted of eight predictors available at birth, one additionally including information on weight gain during the first 12 months of life.

RESULTS

The area under the receiving operating characteristic curve increased from 0.785 (95% confidence interval (CI) [0.773-0.798]) to 0.812 (95% CI [0.801-0.824]) after adding weight gain information when predicting childhood obesity. Adding this information correctly classified 30% more children without obesity and 21% with obesity and improved sensitivity from 0.42 to 0.48. Specificity remained unchanged at 0.91.

CONCLUSION

Adding infancy weight gain information improves discrimination, reclassification and sensitivity of a comprehensive prediction model based on predictors available at birth.

Development of a risk score to guide targeted hepatitis C testing among human immunodeficiency virus patients in Cambodia

BACKGROUND

The World Health Organization recommends testing all human immunodeficiency virus (HIV) patients for hepatitis C virus (HCV). In resource-constrained contexts with low-to-intermediate HCV prevalence among HIV patients, as in Cambodia, targeted testing is, in the short-term, potentially more feasible and cost-effective.

AIM

To develop a clinical prediction score (CPS) to risk-stratify HIV patients for HCV coinfection (HCV RNA detected), and derive a decision rule to guide prioritization of HCV testing in settings where ‘testing all’ is not feasible or unaffordable in the short term.

METHODS

We used data of a cross-sectional HCV diagnostic study in the HIV cohort of Sihanouk Hospital Center of Hope in Phnom Penh. Key populations were very rare in this cohort. Score development relied on the Spiegelhalter and Knill-Jones method. Predictors with an adjusted likelihood ratio ≥ 1.5 or ≤ 0.67 were retained, transformed to natural logarithms, and rounded to integers as score items. CPS performance was evaluated by the area-under-the-ROC curve (AUROC) with 95% confidence intervals (CI), and diagnostic accuracy at the different cut-offs. For the decision rule, HCV coinfection probability ≥1% was agreed as test-threshold.

RESULTS

Among the 3045 enrolled HIV patients, 106 had an HCV coinfection. Of the 11 candidate predictors (from history-taking, laboratory testing), seven had an adjusted likelihood ratio ≥ 1.5 or ≤ 0.67: ≥ 50 years (+1 point), diabetes mellitus (+1), partner/household member with liver disease (+1), generalized pruritus (+1), platelets < 200 × 10⁹/L (+1), aspartate transaminase (AST) < 30 IU/L (-1), AST-to-platelet ratio index (APRI) ≥ 0.45 (+1), and APRI < 0.45 (-1). The AUROC was 0.84 (95%CI: 0.80-0.89), indicating good discrimination of HCV/HIV coinfection and HIV mono-infection. The CPS result ≥0 best fits the test-threshold (negative predictive value: 99.2%, 95%CI: 98.8-99.6). Applying this threshold, 30% (n = 926) would be tested. Sixteen coinfections (15%) would have been missed, none with advanced fibrosis.

CONCLUSION

The CPS performed well in the derivation cohort, and bears potential for other contexts of low-to-intermediate prevalence and little onward risk of transmission(i.e. cohorts without major risk factors as injecting drug use, men having sex with men), and where available resources do not allow to test all HIV patients as recommended by WHO. However, the score requires external validation in other patient cohorts before any wider use can be considered.

Cancer diagnostic tools to aid decision-making in primary care: mixed-methods systematic reviews and cost-effectiveness analysis

BACKGROUND

Tools based on diagnostic prediction models are available to help general practitioners diagnose cancer. It is unclear whether or not tools expedite diagnosis or affect patient quality of life and/or survival.

OBJECTIVES

The objectives were to evaluate the evidence on the validation, clinical effectiveness, cost-effectiveness, and availability and use of cancer diagnostic tools in primary care.

METHODS

Two systematic reviews were conducted to examine the clinical effectiveness (review 1) and the development, validation and accuracy (review 2) of diagnostic prediction models for aiding general practitioners in cancer diagnosis. Bibliographic searches were conducted on MEDLINE, MEDLINE In-Process, EMBASE, Cochrane Library and Web of Science) in May 2017, with updated searches conducted in November 2018. A decision-analytic model explored the tools' clinical effectiveness and cost-effectiveness in colorectal cancer. The model compared patient outcomes and costs between strategies that included the use of the tools and those that did not, using the NHS perspective. We surveyed 4600 general practitioners in randomly selected UK practices to determine the proportions of general practices and general practitioners with access to, and using, cancer decision support tools. Association between access to these tools and practice-level cancer diagnostic indicators was explored.

RESULTS

Systematic review 1 - five studies, of different design and quality, reporting on three diagnostic tools, were included. We found no evidence that using the tools was associated with better outcomes. Systematic review 2 - 43 studies were included, reporting on prediction models, in various stages of development, for 14 cancer sites (including multiple cancers). Most studies relate to QCancer^® (ClinRisk Ltd, Leeds, UK) and risk assessment tools.

DECISION MODEL

In the absence of studies reporting their clinical outcomes, QCancer and risk assessment tools were evaluated against faecal immunochemical testing. A linked data approach was used, which translates diagnostic accuracy into time to diagnosis and treatment, and stage at diagnosis. Given the current lack of evidence, the model showed that the cost-effectiveness of diagnostic tools in colorectal cancer relies on demonstrating patient survival benefits. Sensitivity of faecal immunochemical testing and specificity of QCancer and risk assessment tools in a low-risk population were the key uncertain parameters.

SURVEY

Practitioner- and practice-level response rates were 10.3% (476/4600) and 23.3% (227/975), respectively. Cancer decision support tools were available in 83 out of 227 practices (36.6%, 95% confidence interval 30.3% to 43.1%), and were likely to be used in 38 out of 227 practices (16.7%, 95% confidence interval 12.1% to 22.2%). The mean 2-week-wait referral rate did not differ between practices that do and practices that do not have access to QCancer or risk assessment tools (mean difference of 1.8 referrals per 100,000 referrals, 95% confidence interval -6.7 to 10.3 referrals per 100,000 referrals).

LIMITATIONS

There is little good-quality evidence on the clinical effectiveness and cost-effectiveness of diagnostic tools. Many diagnostic prediction models are limited by a lack of external validation. There are limited data on current UK practice and clinical outcomes of diagnostic strategies, and there is no evidence on the quality-of-life outcomes of diagnostic results. The survey was limited by low response rates.

CONCLUSION

The evidence base on the tools is limited. Research on how general practitioners interact with the tools may help to identify barriers to implementation and uptake, and the potential for clinical effectiveness.

FUTURE WORK

Continued model validation is recommended, especially for risk assessment tools. Assessment of the tools' impact on time to diagnosis and treatment, stage at diagnosis, and health outcomes is also recommended, as is further work to understand how tools are used in general practitioner consultations.

STUDY REGISTRATION

This study is registered as PROSPERO CRD42017068373 and CRD42017068375.

FUNDING

This project was funded by the National Institute for Health Research (NIHR) Health Technology programme and will be published in full in Health Technology Assessment; Vol. 24, No. 66. See the NIHR Journals Library website for further project information.

Clinical outcome and comparison of burn injury scoring systems in burn patient in Indonesia

Introduction

The purpose of this study was to explore and compare the performance of four burn injury scoring systems in Indonesia. In a retrospective study, data of all burn patients admitted to the emergency centre (EC) were collected. The following clinical outcome and four burn injury scoring systems were used to assess each patient: Abbreviated Burn Severity Index (ABSI), Belgian Outcome in Burn Injury (BOBI), the Ryan model, and revised Baux Score.

Methods

From April 2017 to April 2018, clinical outcome and burn injury score for every admitted patient were calculated to evaluate burn prognosis. Demographic information, ABSI score, full-thickness total body surface area (TBSA), overall TBSA, hospital stay, and inhalation injury were noted for analysis. Discriminative ability and goodness-of-fit of the prediction models were determined by receiver operating characteristic curve analysis and Hosmer–Lemeshow tests.

Results

We included 72 patients (mean age: 40.79 ± 16.30 years, average TBSA: 23.59% ± 24.84). Only 1 (1.4%) of them was diagnosed with inhalation injury. Mortality rate was 25%. Deceased patients had significantly higher mean age, %TBSA, and number of inhalation injuries. The ABSI model with sensitivity was 81.6, specificity was 92.5, accuracy was 87.3 and under the Receiver Operator Characteristics curve (AUC) was 0.93 (SE = 0.03).

Conclusions

The best estimation of predicted mortality was obtained with the ABSI model.

Diagnosing hospital bacteraemia in the framework of predictive, preventive and personalised medicine using electronic health records and machine learning classifiers

Background

The bacteraemia prediction is relevant because sepsis is one of the most important causes of morbidity and mortality. Bacteraemia prognosis primarily depends on a rapid diagnosis. The bacteraemia prediction would shorten up to 6 days the diagnosis, and, in conjunction with individual patient variables, should be considered to start the early administration of personalised antibiotic treatment and medical services, the election of specific diagnostic techniques and the determination of additional treatments, such as surgery, that would prevent subsequent complications. Machine learning techniques could help physicians make these informed decisions by predicting bacteraemia using the data already available in electronic hospital records.

Objective

This study presents the application of machine learning techniques to these records to predict the blood culture's outcome, which would reduce the lag in starting a personalised antibiotic treatment and the medical costs associated with erroneous treatments due to conservative assumptions about blood culture outcomes.

Methods

Six supervised classifiers were created using three machine learning techniques, Support Vector Machine, Random Forest and K-Nearest Neighbours, on the electronic health records of hospital patients. The best approach to handle missing data was chosen and, for each machine learning technique, two classification models were created: the first uses the features known at the time of blood extraction, whereas the second uses four extra features revealed during the blood culture.

Results

The six classifiers were trained and tested using a dataset of 4357 patients with 117 features per patient. The models obtain predictions that, for the best case, are up to a state-of-the-art accuracy of 85.9%, a sensitivity of 87.4% and an AUC of 0.93.

Conclusions

Our results provide cutting-edge metrics of interest in predictive medical models with values that exceed the medical practice threshold and previous results in the literature using classical modelling techniques in specific types of bacteraemia. Additionally, the consistency of results is reasserted because the three classifiers' importance ranking shows similar features that coincide with those that physicians use in their manual heuristics. Therefore, the efficacy of these machine learning techniques confirms their viability to assist in the aims of predictive and personalised medicine once the disease presents bacteraemia-compatible symptoms and to assist in improving the healthcare economy.

Relative contribution of vitamin D deficiency to subclinical atherosclerosis in Indian context: Preliminary findings

Asian Indians have a genetic predisposition to atherothrombotic risk. common carotid intima-media thickness (CCIMT) measured by ultrasound is a quantitative marker for atherosclerotic burden and a derived variable, that is, "CCIMT statistical Z-score (Z-score)" is useful for better quantification. The association between vitamin D deficiency and atherosclerosis is inconclusive. Since, vitamin D deficiency is highly prevalent in India, there is a need to study its relative contribution to subclinical atherosclerotic burden.This prospective cross-sectional study (n = 117) in apparently healthy individuals aged 20 to 60 years sought to identify the determinants of CCIMT Z score with CCIMT measured by "echo-tracking" method. A multivariable linear regression analysis was done with CCIMT Z score as dependent variable and the following as independent variables: age, body mass index, waist-to-height ratio, total cholesterol to HDL ratio (TC-HDL ratio), serum vitamin D3 levels (ng/mL), sex, diabetes mellitus, current cigarette smoking status. A diagnostic prediction model was also developed with a threshold value of 1.96 for CCIMT Z score.The mean (SD) for calendar age (y) was 40 (8). There were 26 (22.22%) individuals in sample with CCIMT Z score ≥1.96 (advanced stage) of whom 14 (23.33%) were <40 y (n = 60). The mean score was 1.28 (90th percentile) in the entire sample. Vitamin D3 deficiency with a mean (SD) blood level (ng/mL) of 14.3 (6.4) was noted and prevalence of deficiency was 81%. The final model wasCCIMT Z-score = 0.80 +  (0.841 × current smoking = 1) + (0.156 × TC-HDL ratio) - (0.0263 × vitamin D3 blood level in ng/mL).The decreasing order of association is smoking, TC-HDL ratio, and vitamin D3. With the model, likelihood ratio (95% CIs) was better for positive test 3.5 (1.23-9.94) than that for a negative test 0.83 (0.66-1.02).Internal validation with Bootstrap resampling revealed stability of baseline diagnostic variables.There is substantial subclinical atherosclerotic burden in Indian setting with independent contribution by vitamin D deficiency. The model is valuable in "ruling-in" of the underlying advanced atherosclerosis. The study is limited by convenient sampling and lack of external validation of the model.

Heparin-induced thrombocytopenia: Construction of a pretest diagnostic score derived from the analysis of a prospective multinational database, with internal validation

BACKGROUND

Diagnosis of heparin-induced thrombocytopenia (HIT) requires pretest probability assessment and dedicated laboratory assays.

OBJECTIVE

To develop a pretest score for HIT.

DESIGN

Observational; analysis of prospectively collected data of hospitalized patients suspected with HIT (ClinicalTrials.gov NCT00748839).

SETTING

Thirty-one tertiary hospitals in France, Switzerland, and Belgium.

PATIENTS

Patients tested for HIT antibodies (2280 evaluable), randomly allocated to derivation and validation cohorts.

MEASUREMENTS

Independent adjudicators diagnosed HIT based on the prospectively collected data and serotonin release assay results.

RESULTS

Heparin-induced thrombocytopenia was diagnosed in 234 (14.7%) and 99 (14.5%) patients in the two cohorts. Eight features were associated with HIT (in brackets, points assigned for score calculation of the score): unfractionated heparin (1); therapeutic-dose heparin (1); cardiopulmonary bypass (cardiac surgery) (2); major trauma (3); 5- to 21-day interval from anticoagulation initiation to suspicion of HIT (4); ≥40% decrease in platelet count over ≤6 days (3); thrombotic event, arterial (3) or venous (3). The C-statistic was 0.79 (95% CI, 0.76-0.82). In the validation cohort, the area under the receiver operating characteristic curve was 0.77 (95% CI, 0.74-0.80). Three groups of scores were defined; HIT prevalence reached almost 30% in the high-probability group.

LIMITATION

The performance of the score may depend on settings and practices.

CONCLUSION

The objective, easy-to-collect, clinical features of HIT we evidenced were incorporated into a pretest score, which may guide clinical decisions regarding diagnostic testing and anticoagulation.

Patient similarity analytics for explainable clinical risk prediction

BACKGROUND

Clinical risk prediction models (CRPMs) use patient characteristics to estimate the probability of having or developing a particular disease and/or outcome. While CRPMs are gaining in popularity, they have yet to be widely adopted in clinical practice. The lack of explainability and interpretability has limited their utility. Explainability is the extent of which a model's prediction process can be described. Interpretability is the degree to which a user can understand the predictions made by a model.

METHODS

The study aimed to demonstrate utility of patient similarity analytics in developing an explainable and interpretable CRPM. Data was extracted from the electronic medical records of patients with type-2 diabetes mellitus, hypertension and dyslipidaemia in a Singapore public primary care clinic. We used modified K-nearest neighbour which incorporated expert input, to develop a patient similarity model on this real-world training dataset (n = 7,041) and validated it on a testing dataset (n = 3,018). The results were compared using logistic regression, random forest (RF) and support vector machine (SVM) models from the same dataset. The patient similarity model was then implemented in a prototype system to demonstrate the identification, explainability and interpretability of similar patients and the prediction process.

RESULTS

The patient similarity model (AUROC = 0.718) was comparable to the logistic regression (AUROC = 0.695), RF (AUROC = 0.764) and SVM models (AUROC = 0.766). We packaged the patient similarity model in a prototype web application. A proof of concept demonstrated how the application provided both quantitative and qualitative information, in the form of patient narratives. This information was used to better inform and influence clinical decision-making, such as getting a patient to agree to start insulin therapy.

CONCLUSIONS

Patient similarity analytics is a feasible approach to develop an explainable and interpretable CRPM. While the approach is generalizable, it can be used to develop locally relevant information, based on the database it searches. Ultimately, such an approach can generate a more informative CRPMs which can be deployed as part of clinical decision support tools to better facilitate shared decision-making in clinical practice.

Differentiating PSP from MSA using MR planimetric measurements: a systematic review and meta-analysis

Differential diagnosis of parkinsonian syndromes is considered one of the most challenging in neurology. Quantitative MR planimetric measurements were reported to discriminate between progressive supranuclear palsy (PSP) and non-PSP-parkinsonism. Several studies have used midbrain to pons ratio (M/P) and the Magnetic Resonance Parkinsonism Index (MRPI) in distinguishing PSP patients from those with Parkinson's disease. The current meta-analysis aimed to compare the performance of these measures in discriminating PSP from multiple system atrophy (MSA). A systematic MEDLINE review identified 59 out of 2984 studies allowing a calculation of sensitivity and specificity using the MRPI or M/P. Meta-analyses of results were carried out using random effects modelling. To assess study quality and risk of bias, the QUADAS-2 tool was used. Eight studies were suitable for analysis. The meta-analysis showed a pooled sensitivity and specificity for the MRPI of PSP versus MSA of 79.2% (95% CI 72.7-84.4%) and 91.2% (95% CI 79.5-96.5%), and 84.1% (95% CI 77.2-89.2%) and 89.2% (95% CI 81.8-93.8%), respectively, for the M/P. The QUADAS-2 toolbox revealed a high risk of bias regarding the methodological quality of patient selection and index test, as all patients were seen in a specialized outpatient department without avoiding case control design and no predefined threshold was given regarding MRPI or M/P cut-offs. Planimetric brainstem measurements, in special the MRPI and M/P, yield high diagnostic accuracy for the discrimination of PSP from MSA. However, there is an urgent need for well-designed, prospective validation studies to ameliorate the concerns regarding the risk of bias.

A simplified primary aldosteronism surgical outcome score is a useful prediction model when target organ damage is unknown - Retrospective cohort study

Background

Cure of hypertension after adrenalectomy for primary aldosteronism is no certainty and therefore preoperative patient counseling is essential. The Primary Aldosteronism Surgical Outcome (PASO) Score is a useful prediction model with an area under the curve (AUC) of 0.839. The PASO Score includes ‘Target Organ Damage’ (TOD) (i.e., left ventricular hypertrophy and/or microalbuminuria), which is often unavailable during preoperative counseling and might therefore limit its use in clinical practice. We hypothesized that the PASO score would still be useful if TOD is unknown at time of counseling. Therefore, we aimed to examine the predictive performance of the simplified PASO Score, without taking TOD into account.

Materials and methods

In this retrospective cohort study, patients who underwent unilateral adrenalectomy between 2010 and 2016 in 16 medical centers from North America, Europe and Australia were included. TOD was unknown in our database and therefore assigned as absent. Patients were classified as complete, partial or absent clinical success using the PASO consensus criteria.

Results

A total of 380 (73.9%) patients were eligible for analysis. Complete, partial and absent clinical success were observed in 29.5%, 55.8% and 14.7% of patients, respectively. The simplified PASO Score had an AUC of 0.730 (95% confidence interval 0.674–0.785) in our total cohort.

Conclusion

Without taking TOD into account, the simplified PASO Score had a lower predictive value as compared to the original derivation cohort. Ideally, the complete PASO Score should be used, but when data on TOD are not readily available, the simplified PASO Score is a useful and reasonable alternative.

•

We aimed to examine the predictive performance of the PASO Score, without taking ‘target organ damage’ (TOD) into account.

•

This simplified PASO Score had a lower predictive value as compared to the PASO Score in the original derivation cohort.

•

The simplified PASO Score increases the applicability of the model and is reasonable for clinicians to use in daily practice.

•

Ideally, the complete PASO Score should be used, but the simplified PASO Score is a useful and reasonable alternative.

Negligible Effects of the Survey Modes for Patient-Reported Outcomes: A Report From the Childhood Cancer Survivor Study

PURPOSE

This study compared the measurement properties for multiple modes of survey administration, including postal mail, telephone interview, and Web-based completion of patient-reported outcomes (PROs) among survivors of childhood cancer.

METHODS

The population included 6,974 adult survivors of childhood cancer in the Childhood Cancer Survivor Study who completed the Brief Symptom Inventory-18 (BSI-18), which measured anxiety, depression, and somatization symptoms. Scale reliability, construct validity, and known-groups validity related to health status were tested for each mode of completion. The multiple indicators and multiple causes technique was used to identify differential item functioning (DIF) for the BSI-18 items that responded through a specific survey mode. The impact of the administration mode was tested by comparing differences in BSI-18 scores between the modes accounting for DIF effects.

RESULTS

Of the respondents, 58%, 27%, and 15% completed postal mail, Web-based, and telephone surveys, respectively. Survivors who were male; had lower education, lower household income, or poorer health status; or were treated with cranial radiotherapy were more likely to complete a telephone-based survey compared with either a postal mail or Web-based survey (all P < .05). Scale reliability and validity were equivalent across the 3 survey options. One, 2, and 5 items from the anxiety, depression, and somatization domains, respectively, were identified as having significant DIF among survivors who responded by telephone (P < .05). However, estimated BSI-18 domain scores, especially depression and anxiety, between modes did not differ after accounting for DIF effects.

CONCLUSION

Certain survivor characteristics were associated with choosing a specific mode for PRO survey completion. However, measurement properties among these modes were equivalent, and the impact of using a specific mode on scores was minimal.

Design, methods, and reporting of impact studies of cardiovascular clinical prediction rules are suboptimal: a systematic review

OBJECTIVES

To evaluate design, methods, and reporting of impact studies of cardiovascular clinical prediction rules (CPRs).

STUDY DESIGN AND SETTING

We conducted a systematic review. Impact studies of cardiovascular CPRs were identified by forward citation and electronic database searches. We categorized the design of impact studies as appropriate for randomized and nonrandomized experiments, excluding uncontrolled before-after study. For impact studies with appropriate study design, we assessed the quality of methods and reporting. We compared the quality of methods and reporting between impact and matched control studies.

RESULTS

We found 110 impact studies of cardiovascular CPRs. Of these, 65 (59.1%) used inappropriate designs. Of 45 impact studies with appropriate design, 31 (68.9%) had substantial risk of bias. Mean number of reporting domains that impact studies with appropriate study design adhered to was 10.2 of 21 domains (95% confidence interval, 9.3 and 11.1). The quality of methods and reporting was not clearly different between impact and matched control studies.

CONCLUSION

We found most impact studies either used inappropriate study design, had substantial risk of bias, or poorly complied with reporting guidelines. This appears to be a common feature of complex interventions. Users of CPRs should critically evaluate evidence showing the effectiveness of CPRs.

Understanding and avoiding late presentation for HIV diagnosis - study protocol of a trial using mixed methods (FindHIV)

Many patients infected with HIV are diagnosed at an advanced stage of illness. These late presenters are individuals with a CD4 cell count of less than 350 cells/µL and/or an AIDS defining disease at initial HIV diagnosis. Purpose of FindHIV is to develop and distribute a questionnaire/scoring system aimed at a reduction in late presentation. FindHIV uses a mixed methods approach. In a first step, primary data of patients were collected. Inclusion criteria were: age ≥ 18 years, cognitive ability and language skills to participate in the study, initial HIV diagnosis within the past 6 months, and patient informed consent. Descriptive methods and regression models are used to identify: (1) patient characteristics associated with late presentation and (2) contacts to the healthcare system with indicator diseases that did not lead to HIV testing. Secondly, a questionnaire/scoring system is created by an expert panel. Afterwards the questionnaire/scoring system is to be disseminated. The greatest challenge was in reaching an adequate sample size. Another risk may be a recall bias. Nevertheless, FindHIV is devised as an in-depth study of the phenomenon of late presentation with potential to significantly improve HIV detection.