Correlates of diagnostic accuracy in patients with nonspecific complaints

Implementation of a care-pathway at the emergency department for older people presenting with nonspecific complaints; a protocol for a multicenter parallel cohort study

BACKGROUND

Older adults frequently attend the Emergency Department (ED) with poorly defined symptoms, often called nonspecific complaints (NSC). NSC such as 'weakness' and 'not feeling well', often lead to an extensive differential diagnosis. Patients with NSC experience a prolonged length of stay at the ED and are prone to adverse outcomes. Currently, a care pathway for patients with NSC does not exist. A special structured care pathway for patients with NSC was designed to improve the efficiency and quality of care at the ED.

METHOD

A multicenter parallel cohort study, organized in different hospitals in the Noord-Brabant area, the Netherlands, in which general practitioners (GP), elderly care physicians (ECP), Emergency Physicians (EP), geriatricians and internists will collaborate. Patients ≥ 70 years presenting with NSC and in need of ED admission as indicated by their own GP or ECP are eligible for inclusion. Before implementation each hospital will retrospectively include their own control-group. After implementation, patients will prospectively be included. The care-pathway exists of risk stratification by the APOP-screener, in-depth history taking, i.e. limited comprehensive geriatric assessment (CGA) and a standard set of diagnostics, and a dedicated ED-nurse (if possible) present to ensure the care-pathway is followed. The primary outcome is length of stay at the ED (LOS-ED) and perceived quality of care. Secondary outcomes are hospital length of stay, revisits, readmissions and mortality at 30- and 90-day follow-up.

DISCUSSION

This study proposes a structured care pathway for older patients presenting at the ED with NSCs and considering effectiveness and perceived quality this may improve acute care for these patients.

TRIAL REGISTRATION

Dutch Trial register, number NL8960.

Emergency department patients with weakness or fatigue: Can physicians predict their outcomes at the front door? A prospective observational study

BACKGROUND

Generalized weakness and fatigue are underexplored symptoms in emergency medicine. Triage tools often underestimate patients presenting to the emergency department (ED) with these nonspecific symptoms (Nemec et al., 2010). At the same time, physicians' disease severity rating (DSR) on a scale from 0 (not sick at all) to 10 (extremely sick) predicts key outcomes in ED patients (Beglinger et al., 2015; Rohacek et al., 2015). Our goals were (1) to characterize ED patients with weakness and/or fatigue (W|F); to explore (2) to what extent physicians' DSR at triage can predict five key outcomes in ED patients with W|F; (3) how well DSR performs relative to two commonly used benchmark methods, the Emergency Severity Index (ESI) and the Charlson Comorbidity Index (CCI); (4) to what extent DSR provides predictive information beyond ESI, CCI, or their linear combination, i.e., whether ESI and CCI should be used alone or in combination with DSR; and (5) to what extent ESI, CCI, or their linear combination provide predictive information beyond DSR alone, i.e., whether DSR should be used alone or in combination with ESI and / or CCI.

METHODS

Prospective observational study between 2013-2015 (analysis in 2018-2020, study team blinded to hypothesis) conducted at a single center. We study an all-comer cohort of 3,960 patients (48% female patients, median age = 51 years, 94% completed 1-year follow-up). We looked at two primary outcomes (acute morbidity (Bingisser et al., 2017; Weigel et al., 2017) and all-cause 1- year mortality) and three secondary outcomes (in-hospital mortality, hospitalization and transfer to ICU). We assessed the predictive power (i.e., resolution, measured as the Area under the ROC Curve, AUC) of the scores and, using logistic regression, their linear combinations.

FINDINGS

Compared to patients without W|F (n = 3,227), patients with W|F (n = 733) showed higher prevalences for all five outcomes, reported more symptoms across both genders, and received higher DSRs (median = 4; interquartile range (IQR) = 3-6 vs. median = 3; IQR = 2-5). DSR predicted all five outcomes well above chance (i.e., AUCs > ~0.70), similarly well for both patients with and without W|F, and as good as or better than ESI and CCI in patients with and without W|F (except for 1-year mortality where CCI performs better). For acute morbidity, hospitalization, and transfer to ICU there is clear evidence that adding DSR to ESI and/or CCI improves predictions for both patient groups; for 1-year mortality and in-hospital mortality this holds for most, but not all comparisons. Adding ESI and/or CCI to DSR generally did not improve performance or even decreased it.

CONCLUSIONS

The use of physicians' disease severity rating has never been investigated in patients with generalized weakness and fatigue. We show that physicians' prediction of acute morbidity, mortality, hospitalization, and transfer to ICU through their DSR is also accurate in these patients. Across all patients, DSR is less predictive of acute morbidity for female than male patients, however. Future research should investigate how emergency physicians judge their patients' clinical state at triage and how this can be improved and used in simple decision aids.

The GI Simulated Clinic: A Clinical Reasoning Exercise Supporting Medical Students' Basic and Clinical Science Integration

INTRODUCTION

Cognitive integration is required to perform clinical decision-making tasks, even in the preclinical curriculum of medical school. Simulation supports students' cognitive integration by providing practical application of basic science knowledge in a relevant clinical context. To address the need for integrative activities in our curriculum, we implemented a simulated clinic exercise with cases representing gastrointestinal diseases for first-year medical students.

METHODS

Basic science and clinical skills course directors collaborated to design this simulated clinic event, during which student small groups rotated through a series of standardized patient encounters. During each encounter, one student performed the history and physical exam, following which the small group collaboratively developed a prioritized differential diagnosis. Afterwards, the gastroenterology course director debriefed students to highlight key learning points. We collected learner evaluation data following the event.

RESULTS

Two hundred eighty first-year medical students participated in the simulated clinic in 2018 and 2019. Students rated these events as effective for learning about clinical features of the diseases presented and for reinforcing skills learned in the clinical skills course. Students agreed that the small-group format, pace, and duration were appropriate and that the problem-solving aspect was intellectually stimulating. The most effective aspects were opportunities to solidify illness scripts, apply knowledge to solve a problem, and encounter diseases in a realistic clinical context.

DISCUSSION

This simulated clinic model effectively supported preclinical students' basic and clinical science integration to complete diagnostic reasoning tasks for gastrointestinal gastrointestinal conditions and was evaluated favorably by learners.

Measurement precision at the cut score in medical multiple choice exams: Theory matters

INTRODUCTION

In high-stakes assessment, the measurement precision of pass-fail decisions is of great importance. A concept for analyzing the measurement precision at the cut score is conditional reliability, which describes measurement precision for every score achieved in an exam. We compared conditional reliabilities in Classical Test Theory (CTT) and Item Response Theory (IRT) with a special focus on the cut score and potential factors influencing conditional reliability at the cut score.

METHODS

We analyzed 32 multiple-choice exams from three Swiss medical schools comparing conditional reliability at the cut score in IRT and CCT. Additionally, we analyzed potential influencing factors such as the range of examinees' performance, year of study, and number of items using multiple regression.

RESULTS

In CTT, conditional reliability was highest for very low and very high scores, whereas examinees with medium scores showed low conditional reliabilities. In IRT, the maximum conditional reliability was in the middle of the scale. Therefore, conditional reliability at the cut score was significantly higher in IRT compared with CTT. It was influenced by the range of examinees' performance and number of items. This influence was more pronounced in CTT.

DISCUSSION

We found that conditional reliability shows inverse distributions and conclusions regarding the measurement precision at the cut score depending on the theory used. As the use of IRT seems to be more appropriate for criterion-oriented standard setting in the framework of competency-based medical education, our findings might have practical implications for the design and quality assurance of medical education assessments.

Nonspecific complaints in the emergency department - a systematic review

BACKGROUND

Nonspecific complaint (NSC) is a common presenting complaint in the emergency setting, especially in the elderly population. Individual studies have shown that it is associated with significant morbidity and mortality. This prognostic systematic review draws a synthesis of reported outcomes for patients presenting with NSC and compares them with outcomes for patients presenting with a specific complaint.

METHODS

We conducted a literature search for publications, abstracts and conference presentations from Ovid, Scopus and Web of Science for the past 20 years. Studies were included which treated adult patients presenting to the Emergency Medical Services or Emergency Department with NSC. 2599 studies were screened for eligibility and quality was assessed using the SIGN assessment for bias tool. We excluded any low-quality studies, resulting in nine studies for quantitative analysis. We analysed the included studies for in-hospital mortality, triage category, emergency department length of stay, admission rate, hospital length of stay, intensive care admissions and re-visitation rate and compared outcomes to patients presenting with specific complaints (SC), where data were available. We grouped discharge diagnoses by ICD-10 category.

RESULTS

We found that patients presenting with NSC were mostly older adults. Mortality for patients with NSC was significantly increased compared to patients presenting with SC [OR 2.50 (95% CI 1.40-4.47)]. They were triaged as urgent less often than SC patients [OR 2.12 (95% CI 1.08-4.16)]. Emergency department length of stay was increased in two out of three studies. Hospital length of stay was increased by 1-3 days. Admission rates were high in most studies, 55 to 84%, and increased in comparison to patients with SC [OR 3.86 (95% CI 1.76-8.47)]. These patients seemed to require more resources than patients with SC. The number for intensive care admissions did not seem to be increased. Data were insufficient to make conclusions regarding re-visitation rates. Discharge diagnoses were spread throughout the ICD-10 main chapters, infections being the most prevalent.

CONCLUSIONS

Patients with NSC have a high risk of mortality and their care in the Emergency Department requires more time and resources than for patients with SC. We suggest that NSC should be considered a major emergency presentation.

Disability and morbidity among older patients in the emergency department: a Danish population-based cohort study

OBJECTIVES

The objective was to describe the prevalence of geriatric conditions among older medical patients in the emergency department (ED) and the association with admission, mortality, reattendance and loss of independency.

DESIGN

Population-based prospective cohort study.

SETTING

ED of a large university hospital.

PARTICIPANTS

All medical patients ≥65 years of age from a single municipality with a first attendance to the ED during a 1-year period (November 2013 to November 2014).

PRIMARY AND SECONDARY OUTCOME MEASURES

Based on information from healthcare registers, we defined geriatric conditions as disability, recently increased disability, polypharmacy and comorbidity. Outcomes were admission, length of admission, 30 days postdischarge mortality, 30 days hospital reattendance and home care dependency 0-360 days following ED contact.

RESULTS

Totally, 3775 patients (55% women) were included, age 78 (71-85) years (median (IQR)). No patients were lost to follow-up. The prevalence of 0-4 geriatric conditions was 14.9%, 27.3%, 25.2%, 22.3% and 10.3%, respectively. The number of conditions was significantly associated with hospital admission, length of admission, 30 days postdischarge mortality and 30 days hospital reattendance. Among patients with no geriatric conditions, 70% lived independent all 360 days after discharge, whereas all patients with ≥3 conditions had some dependency or were dead within 360 days following discharge.

CONCLUSION

Among older medical patients in the ED, 50% had two or more geriatric conditions which were associated with poor health outcomes. This highlights the need for studies of the effect of geriatric awareness and competences in the ED.

Why assessment in medical education needs a solid foundation in modern test theory

Despite the frequent use of state-of-the-art psychometric models in the field of medical education, there is a growing body of literature that questions their usefulness in the assessment of medical competence. Essentially, a number of authors raised doubt about the appropriateness of psychometric models as a guiding framework to secure and refine current approaches to the assessment of medical competence. In addition, an intriguing phenomenon known as case specificity is specific to the controversy on the use of psychometric models for the assessment of medical competence. Broadly speaking, case specificity is the finding of instability of performances across clinical cases, tasks, or problems. As stability of performances is, generally speaking, a central assumption in psychometric models, case specificity may limit their applicability. This has probably fueled critiques of the field of psychometrics with a substantial amount of potential empirical evidence. This article aimed to explain the fundamental ideas employed in psychometric theory, and how they might be problematic in the context of assessing medical competence. We further aimed to show why and how some critiques do not hold for the field of psychometrics as a whole, but rather only for specific psychometric approaches. Hence, we highlight approaches that, from our perspective, seem to offer promising possibilities when applied in the assessment of medical competence. In conclusion, we advocate for a more differentiated view on psychometric models and their usage.

Elderly emergency patients presenting with non-specific complaints: Characteristics and outcomes

Background

Non-specific complaints (NSC) are common at the emergency department, but only a few studies have shown evidence that these complaints are associated with a poor prognosis in elderly emergency patients.

Objective

To describe patient characteristics and outcomes in a cohort of elderly emergency patients presenting with NSC. Outcomes were: patient characteristics, hospitalization, 90-day ED-return visits, and 30-day mortality.

Method

A retrospective cohort study was conducted amongst elderly patients present to the Internal Medicine Emergency Department (ED) between 01-09-2010 and 31-08-2011. NSC were defined as indefinable complaints that lack a pre-differential diagnosis needed to initiate of a standardized patient evaluation. Cox regression was performed to calculate Hazard Ratios (HR) and corrected for confounders such as comorbidity.

Results

In total, 1784 patients were enrolled; 244 (13.7%) presented with NSC. Compared to those with SC, comorbidity was higher in the NSC-group (Charlson comorbidity index 3.0 vs. 2.4, p<0.001). The triage level did not differ, but ED-length of stay was longer in the NSC-group (188 vs. 178 minutes, p = 0.004). Hospitalization was more frequent (84.0 vs. 71.1%, p<0.001) and the length of hospital stay (9 vs. 6 days, p<0.001 was longer in the NSC- than in the SC-group. The number of ED-return visits were comparable between both groups (HR 0.8, 95%CI 0.6–1.1). Mortality within 30-days was higher in the NSC- (20.1%) than in the SC-group (11.0%, HR 1.7 95%CI 1.2–2.4).

Conclusion

Elderly patients present with NSC at the ED regularly. These patients are more often hospitalized and have a substantially higher 30-day mortality than patients with SC.

Early diagnostic suggestions improve accuracy of GPs: a randomised controlled trial using computer-simulated patients

Background

Designers of computerised diagnostic support systems (CDSSs) expect physicians to notice when they need advice and enter into the CDSS all information that they have gathered about the patient. The poor use of CDSSs and the tendency not to follow advice once a leading diagnosis emerges would question this expectation.

Aim

To determine whether providing GPs with diagnoses to consider before they start testing hypotheses improves accuracy.

Design and setting

Mixed factorial design, where 297 GPs diagnosed nine patient cases, differing in difficulty, in one of three experimental conditions: control, early support, or late support.

Method

Data were collected over the internet. After reading some initial information about the patient and the reason for encounter, GPs requested further information for diagnosis and management. Those receiving early support were shown a list of possible diagnoses before gathering further information. In late support, GPs first gave a diagnosis and were then shown which other diagnoses they could still not discount.

Results

Early support significantly improved diagnostic accuracy over control (odds ratio [OR] 1.31; 95% confidence interval [95%CI] = 1.03 to 1.66, P = 0.027), while late support did not (OR 1.10; 95% CI = 0.88 to 1.37). An absolute improvement of 6% with early support was obtained. There was no significant interaction with case difficulty and no effect of GP experience on accuracy. No differences in information search were detected between experimental conditions.

Conclusion

Reminding GPs of diagnoses to consider before they start testing hypotheses can improve diagnostic accuracy irrespective of case difficulty, without lengthening information search.

Are Mortality and Acute Morbidity in Patients Presenting With Nonspecific Complaints Predictable Using Routine Variables?

OBJECTIVES

Patients presenting to the emergency department (ED) with nonspecific complaints are difficult to accurately triage, risk stratify, and diagnose. This can delay appropriate treatment. The extent to which key medical outcomes are at all predictable in these patients, and which (if any) predictors are useful, has previously been unclear. To investigate these questions, we tested an array of statistical and machine learning models in a large group of patients and estimated the predictability of mortality (which occurred in 6.6% of our sample of patients), acute morbidity (58%), and presence of acute infectious disease (28.2%).

METHODS

To investigate whether the best available tools can predict the three key outcomes, we fed data from a sample of 1,278 ED patients with nonspecific complaints into 17 state-of-the-art statistical and machine learning models. The patient sample stems from a diagnostic multicenter study with prospective 30-day follow-up conducted in Switzerland. Predictability of the three key medical outcomes was quantified by computing the area under the receiver operating characteristic curve (AUC) for each model.

RESULTS

The models performed at different levels but, on average, the predictability of the target outcomes ranged between 0.71 and 0.82. The better models clearly outperformed physicians' intuitive judgments of how ill patients looked (AUC = 0.67 for mortality, 0.65 for morbidity, and 0.60 for infectious disease).

CONCLUSIONS

Modeling techniques can be used to derive formalized models that, on average, predict the outcomes of mortality, acute morbidity, and acute infectious disease in patients with nonspecific complaints with a level of accuracy far beyond chance. The models also predicted these outcomes more accurately than did physicians' intuitive judgments of how ill the patients look; however, the latter was among the small set of best predictors for mortality and acute morbidity. These results lay the groundwork for further refining triage and risk stratification tools for patients with nonspecific complaints. More research, informed by whether the goal of a model is high sensitivity or high specificity, is needed to develop readily applicable clinical decision support tools (e.g., decision trees) that could be supported by electronic health records.

Emergency Presentations With Nonspecific Complaints-the Burden of Morbidity and the Spectrum of Underlying Disease: Nonspecific Complaints and Underlying Disease

The prevalence of diagnoses, morbidity, and mortality of patients with nonspecific complaints (NSC) presenting to the emergency department (ED) is unknown.To determine the prevalence of diagnoses, acute morbidity, and mortality of patients with NSC.Prospective observational study with a 30-day follow-up. Patients presenting to 2 EDs were enrolled by a study team and diagnosed according to the World Health Organization ICD-10 System.Of 217,699 presentations to the ED from May 2007 through to February 2011, a total of 1300 patients were enrolled. After exclusion of 90 patients who fulfilled exclusion criteria, 1210 patients were analyzed. No patient was lost to follow-up. In patients with NSC, the underlying diseases were spread throughout 18 chapters of the ICD-10. A total of 58.7% of the patients were diagnosed with acute morbidity. Thirty-day mortality was 6.4% overall. Patients with acute morbidity and suffering from heart failure and pneumonia had mortalities >15%; patients lacking acute morbidity, but suffering from functional impairment or depression/anxiety had mortalities of 0%. Although the history did not allow any prediction, age and sex were predictive of morbidity and mortality.The differential diagnoses in patients presenting with NSC is broad. Acute morbidity and mortality were high in the presented cohort, the predictors of morbidity and mortality being age and sex rather than the nature of the complaints. Urgently needed management strategies could be based on these results.ClinicalTrials.gov (#NCT00920491).

Physician's first clinical impression of emergency department patients with nonspecific complaints is associated with morbidity and mortality

The association between the physician's first clinical impression of a patient with nonspecific complaints and morbidity and mortality is unknown. The aim was to evaluate the association of the physician's first clinical impression with acute morbidity and mortality. We conducted a prospective observational study with a 30-day follow-up. This study was performed at the emergency departments (EDs) of 1 secondary and 1 tertiary care hospital, from May 2007 to February 2011. The first clinical impression ("looking ill"), expressed on a numerical rating scale from 0 to 100, age, sex, and the Charlson Comorbidity Index (CCI) were evaluated. The association was determined between these variables and acute morbidity and mortality, together with receiver operating characteristics, and validity. Of 217,699 presentations to the ED, a total of 1278 adult nontrauma patients with nonspecific complaints were enrolled by a study team. No patient was lost to follow-up. A total of 84 (6.6%) patients died during follow-up, and 742 (58.0%) patients were classified as suffering from acute morbidity. The variable "looking ill" was significantly associated with mortality and morbidity (per 10 point increase, odds ratio 1.23, 95% confidence interval [CI] 1.12-1.34, P < 0.001, and odds ratio 1.19, 95% CI 1.14-1.24, P < 0.001, respectively). The combination of the variables "looking ill," "age," "male sex," and "CCI" resulted in the best prediction of these outcomes (mortality: area under the curve [AUC] 0.77, 95% CI 0.72-0.82; morbidity: AUC 0.68, 95% CI 0.65-0.71). The physician's first impression, with or without additional variables such as age, male sex, and CCI, was associated with morbidity and mortality. This might help in the decision to perform further diagnostic tests and to hospitalize ED patients.