Clinical calculators in hospital medicine: Availability, classification, and needs

Local anaesthetics risks perception: A web-based survey

Background

The use of local anaesthetics (LAs) is usually associated with few adverse effects, but local anaesthetic systemic toxicity (LAST) can result in serious harm and even death. However, practitioner awareness regarding this risk has been little studied.

Methods

This was a closed, web-based study carried out at two Swiss university hospitals using a fully automated questionnaire. The main objective was to evaluate LAST awareness and LA use among various medical practitioners. The secondary objective was to determine whether these physicians felt that a tool designed to compute maximum safe LA doses should be developed.

Results

The overall participation rate was 40.2 % and was higher among anaesthesiologists (154/249, 61.8 % vs 159/530, 30.0 %; P < .001). Anaesthesiologists identified the risk of LAST and the systems involved more frequently than non-anaesthesiologists (85.1 % vs 43.4 %, P < .001). After adjusting for years of clinical experience, age, country of diploma, frequency of LA use, clinical position and being an anaesthesiologist, the only significant associations were this latter factor (P < .001) and clinical position (P = .016 for fellows and P = .046 for consultants, respectively). Most respondents supported the development of a tool designed to compute maximum safe LA doses (251/313, 80.2 %) and particularly of a mobile app (190/251, 75.7 %).

Conclusions

LAST awareness is limited among practitioners who use LAs on a regular basis. Educational interventions should be created, and tools designed to help calculate maximum safe LA doses developed. The actual frequency of unsafe LA doses administration would also deserve further study.

Clinical Algorithms with Race: An Online Database

Some clinical algorithms incorporate a person's race, ethnicity, or both as an input variable or predictor in determining diagnoses, prognoses, treatment plans, or risk assessments. Inappropriate use of race and ethnicity in clinical algorithms at the point of care may exacerbate health disparities and promote harmful practices of race-based medicine. This article describes a comprehensive search of online resources, the scientific literature, and the FDA Drug Label Information that uncovered 39 race-based risk calculators, six laboratory test results with race-based reference ranges, one race-based therapy recommendation, and 15 medications with race-based recommendations. These clinical algorithms based on race are freely accessible through an online database. This resource aims to raise awareness about the use of race-based clinical algorithms and track the progress made toward eradicating the inappropriate use of race. The database will be actively updated to include clinical algorithms based on race that were previously omitted, along with additional characteristics of these algorithms.

Clinical risk calculators informing the decision to admit: A methodologic evaluation and assessment of applicability

INTRODUCTION

Clinical prediction and decision tools that generate outcome-based risk stratification and/or intervention recommendations are prevalent. Appropriate use and validity of these tools, especially those that inform complex clinical decisions, remains unclear. The objective of this study was to assess the methodologic quality and applicability of clinical risk scoring tools used to guide hospitalization decision-making.

METHODS

In February 2021, a comprehensive search was performed of a clinical calculator online database (mdcalc.com) that is publicly available and well-known to clinicians. The primary reference for any calculator tool informing outpatient versus inpatient disposition was considered for inclusion. Studies were restricted to the adult, acute care population. Those focused on obstetrics/gynecology or critical care admission were excluded. The Wasson-Laupacis framework of methodologic standards for clinical prediction rules was applied to each study.

RESULTS

A total of 22 calculators provided hospital admission recommendations for 9 discrete medical conditions using adverse events (14/22), mortality (6/22), or confirmatory diagnosis (2/22) as outcomes of interest. The most commonly met methodologic standards included mathematical technique description (22/22) and clinical sensibility (22/22) and least commonly met included reproducibility of the rule (1/22) and measurement of effect on clinical use (1/22). Description of the studied population was often lacking, especially patient race/ethnicity (2/22) and mental or behavioral health (0/22). Only one study reported any item related to social determinants of health.

CONCLUSION

Studies commonly do not meet rigorous methodologic standards and often fail to report pertinent details that would guide applicability. These clinical tools focus primarily on specific disease entities and clinical variables, missing the breadth of information necessary to make a disposition determination and raise significant validation and generalizability concerns.

Validation of a simple risk stratification tool for COVID-19 mortality

Risk prediction is an essential part of clinical care, in order to allocate resources and provide care appropriately. During the COVID-19 pandemic risk prediction became a matter of political and public debate as a major clinical need to guide medical and organizational decisions. We previously presented a simplified risk stratification score based on a nomogram developed in Wuhan, China in the early phase of the pandemic. Here we aimed to validate this simplified risk stratification score in a larger patient cohort from one city in Austria. Age, oxygen saturation, C-reactive protein levels and creatinine levels were used to estimate the in-hospital mortality risk for COVID-19 patients in a point based score: 1 point per age decade, 4 points for oxygen saturation <92%, 8 points for CRP > 10 mg/l and 4 points for creatinine > 84 μmol/l. Between June 2020 and March 2021, during the “second wave” of the pandemic, 1,472 patients with SARS-CoV-2 infection were admitted to two hospitals in Graz, Austria. In 961 patients the necessary dataset to calculate the simplified risk stratification score was available. In this cohort, as in the cohort that was used to develop the score, a score above 22 was associated with a significantly higher mortality (p < 0.001). Cox regression confirmed that an increase of one point in the risk stratification score increases the 28-day-mortality risk approximately 1.2-fold. Patients who were categorized as high risk (≥22 points) showed a 3–4 fold increased mortality risk. Our simplified risk stratification score performed well in a separate, larger validation cohort. We therefore propose that our risk stratification score, that contains only two routine laboratory parameter, age and oxygen saturation as variables can be a useful and easy to implement tool for COVID-19 risk stratification and beyond. The clinical usefulness of a risk prediction/stratification tool needs to be assessed prospectively (https://www.cbmed.at/covid-19-risk-calculator/).

Accuracy of freely available online GFR calculators using the CKD-EPI equation

Purpose

Estimated glomerular filtration rate (eGFR) as calculated by the Chronic Kidney Disease Epidemiology Collaboration (CKD-EPI) equation is used for detection of chronic kidney disease and drug dose adjustment. The purpose of the present study was to investigate the accuracy of freely available eGFR online calculators.

Methods

All identified CKD-EPI online calculators were run with five reference cases differing in age, sex, serum creatinine, and ethnicity. Conversion from eGFR_indexed (unit ml/min per 1.73 m²) to eGFR_non-indexed (unit ml/min) and creatinine unit from milligramme/decilitre to micromole/litre was checked, if available.

Results

Only 36 of 47 calculators (76.6%) produced accurate eGFR results for all reference cases. Eight of 47 (17.0%) calculators were considered as faulty because of errors relating to ethnicity (4 calculators), to conversion of the eGFR unit (2 calculators), to erroneous eGFR values without obvious explanation (2 calculators), to conversion of the creatinine unit (1 calculator), and to an error in the eGFR unit displayed (1 calculator). Overall, 28 errors were found (range 59 to 147% of the correct eGFR value), the majority concerning calculation of eGFR_indexed and the conversion to eGFR_non-indexed. Only 7 of 47 (14.9%) calculators offered conversion of the eGFR unit.

Conclusions

Erroneous calculations that might lead to inappropriate clinical decision-making were found in 8 of 47 calculators. Thus, online calculators should be evaluated more thoroughly after implementation. Conversion of eGFR units that might be needed for drug dose adjustments should be implemented more often.

Electronic supplementary material

The online version of this article (10.1007/s00228-020-02932-x) contains supplementary material, which is available to authorized users.

Novel Calculators for Risk Stratification in Hepatocellular Carcinoma: A Concise Review

Among clinicians, the users of medical calculators have expanded in recent years to an unprecedented number. The usefulness of some of these calculators is sometimes debatable, and experienced professionals may at times be right in avoiding their use; however, many may simply be unaware of the very existence of medical calculators applicable to their field of interest. The authors felt that this latter scenario might possibly apply to hepatocellular carcinoma (HCC). Hence, the authors concisely reviewed 10 free online medical calculators proposed in the last 8 years, categorizing them on the basis of the purpose for which they were developed (risk of harboring or developing HCC, N=4; prognostication in established HCC, N=6). In addition, the authors tried to establish the success each calculator has had so far in the medical community, by 2 criteria: having been included in the more popular app of medical calculators and being highly cited in the scientific literature.

World competitive contest-based artificial neural network: A new class-specific method for classification of clinical and biological datasets

Many data mining methods have been proposed to generate computer-aided diagnostic systems, which may determine diseases in their early stages by categorizing the data into some proper classes. Considering the importance of the existence of a suitable classifier, the present study aims to introduce an efficient approach based on the World Competitive Contests (WCC) algorithm as well as a multi-layer perceptron artificial neural network (ANN). Unlike the previously introduced methods, which each has developed a universal model for all different kinds of data classes, our proposed approach generates a single specific model for each individual class of data. The experimental results show that the proposed method (ANNWCC), which can be applied to both the balanced and unbalanced datasets, yields more than 76% (without applying feature selection methods) and 90% (with applying feature selection methods) of the average five-fold cross-validation accuracy on the 13 clinical and biological datasets. The findings also indicate that under different conditions, our proposed method can produce better results in comparison to some state-of-art meta-heuristic algorithms and methods in terms of various statistical and classification measurements. To classify the clinical and biological data, a multi-layer ANN and the WCC algorithm were combined. It was shown that developing a specific model for each individual class of data may yield better results compared with creating a universal model for all of the existing data classes. Besides, some efficient algorithms proved to be essential to generate acceptable biological results, and the methods' performance was found to be enhanced by fuzzifying or normalizing the biological data.

•

We combined multi-layer artificial neural networks and world competitive contests algorithms to classify biological datasets

•

The proposed method has been investigated on 13 clinical datasets with different properties

•

Efficient models may yield better classification models and health diagnostic systems

•

Feature selection methods can improve the performance of a model in separating case and control samples

Machine Learning and Prediction of All-Cause Mortality in COPD

BACKGROUND

COPD is a leading cause of mortality.

RESEARCH QUESTION

We hypothesized that applying machine learning to clinical and quantitative CT imaging features would improve mortality prediction in COPD.

STUDY DESIGN AND METHODS

We selected 30 clinical, spirometric, and imaging features as inputs for a random survival forest. We used top features in a Cox regression to create a machine learning mortality prediction (MLMP) in COPD model and also assessed the performance of other statistical and machine learning models. We trained the models in subjects with moderate to severe COPD from a subset of subjects in Genetic Epidemiology of COPD (COPDGene) and tested prediction performance in the remainder of individuals with moderate to severe COPD in COPDGene and Evaluation of COPD Longitudinally to Identify Predictive Surrogate Endpoints (ECLIPSE). We compared our model with the BMI, airflow obstruction, dyspnea, exercise capacity (BODE) index; BODE modifications; and the age, dyspnea, and airflow obstruction index.

RESULTS

We included 2,632 participants from COPDGene and 1,268 participants from ECLIPSE. The top predictors of mortality were 6-min walk distance, FEV1 % predicted, and age. The top imaging predictor was pulmonary artery-to-aorta ratio. The MLMP-COPD model resulted in a C index ≥ 0.7 in both COPDGene and ECLIPSE (6.4- and 7.2-year median follow-ups, respectively), significantly better than all tested mortality indexes (P < .05). The MLMP-COPD model had fewer predictors but similar performance to that of other models. The group with the highest BODE scores (7-10) had 64% mortality, whereas the highest mortality group defined by the MLMP-COPD model had 77% mortality (P = .012).

INTERPRETATION

An MLMP-COPD model outperformed four existing models for predicting all-cause mortality across two COPD cohorts. Performance of machine learning was similar to that of traditional statistical methods. The model is available online at: https://cdnm.shinyapps.io/cgmortalityapp/.

Medical calculators: Prevalence, and barriers to use

BACKGROUND AND OBJECTIVES

Medical calculators synthesize measurable evidence and help introduce new medical guidelines and standards. Some medical calculators can fulfill the role of CDS for Meaningful Use purposes. However, there are barriers for clinicians to use medical calculators in practice. Objectives of this study were to determine whether lack of EHR integration would be a barrier to use of medical calculators, and understand factors that may limit use and perceived usefulness of calculators METHODS: A survey about medical calculators as they relate to clinical efficiency, perceived usefulness, and barriers to effective use was conducted at a medium-sized academic medical center. 819 physicians were invited to participate in an online survey with a 13% response rate. Results were statistically analyzed to highlight factors related to use or non-use of medical calculators.

RESULTS

We found a negative correlation between use of medical calculators and years of experience (p < 0.001), with decreasing calculator use as experience goes up. Barriers to using medical calculators by non-users and users of medical calculators show that necessity and integration are significantly different with p < 0.001 and p = 0.037, respectively. 46.7% of non-users reported necessity as a barrier compared to 7.7% of users. Integration was reported as a barrier for 43.6% of users, but only 13.3% of non-users. 61% of users indicated that calculators made them more efficient, and 70% reported that unavailability of normally used calculators make them less efficient. 60% of users indicated that they are somewhat or very likely to use newly published medical calculators.

CONCLUSION

The results highlight that medical calculators are important for care delivery by both users and non-users. For non-users, they are seen as having a potentially positive impact on patient care, but unnecessary as part of clinical practice. For medical calculator users, calculators are an important part of regular workflow for efficiency improvement. Clinicians with fewer years of experience show an eagerness to consume newly published calculators, making these kinds of CDS a potentially useful way to disseminate new medical evidence. The survey results suggest that when medical calculators can be automated and integrated into the EHR as part of everyday workflow then efficiency and adoption may increase.

Electronic health record with computerized decision support tools for the purposes of a pediatric cardiovascular heart disease screening program in Crete

BACKGROUND AND OBJECTIVE

Early detection of cardiovascular (CV) disease or associated risk factors during childhood is of paramount importance, allowing for early treatment or lifestyle modifications, respectively. The objective of this study was to describe the development of an electronic health record (EHR), with integrated computerized decision support system (CDSS), specifically designed for supporting the needs of a pilot pediatric CV disease screening program applied on primary school students of a Mediterranean island.

METHODS

Evidence-based knowledge, national and international practice guidelines regarding sport preparticipation CV screening of children and young athletes has been used for the design of the designated EHR. A CDSS, capable for providing alerts for further cardiology evaluation need, has been incorporated into the EHR, based on normative anthropometric and electrocardiographic data as well as predefined positive history responses.

RESULTS

We developed a designated EHR with integrated CDSS supporting pediatric CV disease screening, capable for documenting CV-related personal and family history responses, physical evaluation data (weight, height, blood pressure), allowing for entering electrocardiogam (ECG) measurements and for uploading of multimedia files (including ECG images and digital phonocardiogram audio files). The EHR incorporates clinical calculators and referral alerts for the presence (and degree) of adiposity, hypertension, ECG abnormalities and positive history responses indicative of high CV disease risk. In a preliminary EHR validation, performed by entering data from 53 previously available paper-based health records, the EHR was proven to be fully functional.

CONCLUSIONS

The pediatric cardiology EHR with CDSS features which we developed might serve as a model for EHR for primary health care purposes, capable to document and early detect CV disease and associated risk factors in pediatric populations.

Development and Validation of a Novel Pediatric Appendicitis Risk Calculator (pARC)

OBJECTIVES

We sought to develop and validate a clinical calculator that can be used to quantify risk for appendicitis on a continuous scale for patients with acute abdominal pain.

METHODS

The pediatric appendicitis risk calculator (pARC) was developed and validated through secondary analyses of 3 distinct cohorts. The derivation sample included visits to 9 pediatric emergency departments between March 2009 and April 2010. The validation sample included visits to a single pediatric emergency department from 2003 to 2004 and 2013 to 2015. Variables evaluated were as follows: age, sex, temperature, nausea and/or vomiting, pain duration, pain location, pain with walking, pain migration, guarding, white blood cell count, and absolute neutrophil count. We used stepwise regression to develop and select the best model. Test performance of the pARC was compared with the Pediatric Appendicitis Score (PAS).

RESULTS

The derivation sample included 2423 children, 40% of whom had appendicitis. The validation sample included 1426 children, 35% of whom had appendicitis. The final pARC model included the following variables: sex, age, duration of pain, guarding, pain migration, maximal tenderness in the right-lower quadrant, and absolute neutrophil count. In the validation sample, the pARC exhibited near perfect calibration and a high degree of discrimination (area under the curve: 0.85; 95% confidence interval: 0.83 to 0.87) and outperformed the PAS (area under the curve: 0.77; 95% confidence interval: 0.75 to 0.80). By using the pARC, almost half of patients in the validation cohort could be accurately classified as at <15% risk or ≥85% risk for appendicitis, whereas only 23% would be identified as having a comparable PAS of <3 or >8.

CONCLUSIONS

In our validation cohort of patients with acute abdominal pain, the pARC accurately quantified risk for appendicitis.

Defining the Intrinsic Cardiac Risks of Operations to Improve Preoperative Cardiac Risk Assessments

Background

Current preoperative cardiac risk stratification practices group operations into broad categories, which might inadequately consider the intrinsic cardiac risks of individual operations. We sought to define the intrinsic cardiac risks of individual operations and to demonstrate how grouping operations might lead to imprecise estimates of perioperative cardiac risk.

Methods

Elective operations (based on Common Procedural Terminology codes) performed from January 1, 2010 to December 31, 2015 at hospitals participating in the American College of Surgeons National Surgical Quality Improvement Program were studied. A composite measure of perioperative adverse cardiac events was defined as either cardiac arrest requiring cardiopulmonary resuscitation or acute myocardial infarction. Operations’ intrinsic cardiac risks were derived from mixed-effects models while controlling for patient mix. Resultant risks were sorted into low-, intermediate-, and high-risk categories, and the most commonly performed operations within each category were identified. Intrinsic operative risks were also examined using a representative grouping of operations to portray within-group variation.

Results

Sixty-six low, 30 intermediate, and 106 high intrinsic cardiac risk operations were identified. Excisional breast biopsy had the lowest intrinsic cardiac risk (overall rate, 0.01%; odds ratio, 0.11; 95% CI, 0.02 to 0.25) relative to the average, whereas aorto-bifemoral bypass grafting had the highest (overall rate, 4.1%; odds ratio, 6.61; 95% CI, 5.54 to 7.90). There was wide variation in the intrinsic cardiac risks of operations within the representative grouping (median odds ratio, 1.40; interquartile range, 0.88 to 2.17).

Conclusions

A continuum of intrinsic cardiac risk exists among operations. Grouping operations into broad categories inadequately accounts for the intrinsic cardiac risk of individual operations.

Automating Clinical Score Calculation within the Electronic Health Record. A Feasibility Assessment

OBJECTIVES

Evidence-based clinical scores are used frequently in clinical practice, but data collection and data entry can be time consuming and hinder their use. We investigated the programmability of 168 common clinical calculators for automation within electronic health records.

METHODS

We manually reviewed and categorized variables from 168 clinical calculators as being extractable from structured data, unstructured data, or both. Advanced data retrieval methods from unstructured data sources were tabulated for diagnoses, non-laboratory test results, clinical history, and examination findings.

RESULTS

We identified 534 unique variables, of which 203/534 (37.8%) were extractable from structured data and 269/534 (50.4.7%) were potentially extractable using advanced techniques. Nearly half (265/534, 49.6%) of all variables were not retrievable. Only 26/168 (15.5%) of scores were completely programmable using only structured data and 43/168 (25.6%) could potentially be programmable using widely available advanced information retrieval techniques. Scores relying on clinical examination findings or clinical judgments were most often not completely programmable.

CONCLUSION

Complete automation is not possible for most clinical scores because of the high prevalence of clinical examination findings or clinical judgments - partial automation is the most that can be achieved. The effect of fully or partially automated score calculation on clinical efficiency and clinical guideline adherence requires further study.

Towards automated calculation of evidence-based clinical scores

AIM

To determine clinical scores important for automated calculation in the inpatient setting.

METHODS

A modified Delphi methodology was used to create consensus of important clinical scores for inpatient practice. A list of 176 externally validated clinical scores were identified from freely available internet-based services frequently used by clinicians. Scores were categorized based on pertinent specialty and a customized survey was created for each clinician specialty group. Clinicians were asked to rank each score based on importance of automated calculation to their clinical practice in three categories - “not important”, “nice to have”, or “very important”. Surveys were solicited via specialty-group listserv over a 3-mo interval. Respondents must have been practicing physicians with more than 20% clinical time spent in the inpatient setting. Within each specialty, consensus was established for any clinical score with greater than 70% of responses in a single category and a minimum of 10 responses. Logistic regression was performed to determine predictors of automation importance.

RESULTS

Seventy-nine divided by one hundred and forty-four (54.9%) surveys were completed and 72/144 (50%) surveys were completed by eligible respondents. Only the critical care and internal medicine specialties surpassed the 10-respondent threshold (14 respondents each). For internists, 2/110 (1.8%) of scores were “very important” and 73/110 (66.4%) were “nice to have”. For intensivists, no scores were “very important” and 26/76 (34.2%) were “nice to have”. Only the number of medical history (OR = 2.34; 95%CI: 1.26-4.67; P < 0.05) and vital sign (OR = 1.88; 95%CI: 1.03-3.68; P < 0.05) variables for clinical scores used by internists was predictive of desire for automation.

CONCLUSION

Few clinical scores were deemed “very important” for automated calculation. Future efforts towards score calculator automation should focus on technically feasible “nice to have” scores.