Misdiagnosis-related harm quantification through mixture models and harm measures

Investigating and monitoring misdiagnosis-related harm is crucial for improving health care. However, this effort has traditionally focused on the chart review process, which is labor intensive, potentially unstable, and does not scale well. To monitor medical institutes' diagnostic performance and identify areas for improvement in a timely fashion, researchers proposed to leverage the relationship between symptoms and diseases based on electronic health records or claim data. Specifically, the elevated disease risk following a false-negative diagnosis can be used to signal potential harm. However, off-the-shelf statistical methods do not fully accommodate the data structure of a well-hypothesized risk pattern and thus fail to address the unique challenges adequately. To fill these gaps, we proposed a mixture regression model and its associated goodness-of-fit testing. We further proposed harm measures and profiling analysis procedures to quantify, evaluate, and compare misdiagnosis-related harm across institutes with potentially different patient population compositions. We studied the performance of the proposed methods through simulation studies. We then illustrated the methods through data analyses on stroke occurrence data from the Taiwan Longitudinal Health Insurance Database. From the analyses, we quantitatively evaluated risk factors for being harmed due to misdiagnosis, which unveiled some insights for health care quality research. We also compared general and special care hospitals in Taiwan and observed better diagnostic performance in special care hospitals using various new evaluation measures.

Opportunities to improve diagnosis in emergency transfers to the pediatric intensive care unit

BACKGROUND

Late recognition of in-hospital deterioration is a source of preventable harm. Emergency transfers (ET), when hospitalized patients require intensive care unit (ICU) interventions within 1 h of ICU transfer, are a proximal measure of late recognition associated with increased mortality and length of stay (LOS).

OBJECTIVE

To apply diagnostic process improvement frameworks to identify missed opportunities for improvement in diagnosis (MOID) in ETs and evaluate their association with outcomes.

DESIGN, SETTINGS, AND PARTICIPANTS

A single-center retrospective cohort study of ETs, January 2015 to June 2019. ET criteria include intubation, vasopressor initiation, or≥ $\ge \phantom{\rule{}{0ex}}$ 60 mL/kg fluid resuscitation 1 h before to 1 h after ICU transfer. The primary exposure was the presence of MOID, determined using SaferDx. Cases were screened by an ICU and non-ICU physician. Final determinations were made by an interdisciplinary group. Diagnostic process improvement opportunities were identified.

MAIN OUTCOME AND MEASURES

Primary outcomes were in-hospital mortality and posttransfer LOS, analyzed by multivariable regression adjusting for age, service, deterioration category, and pretransfer LOS.

RESULTS

MOID was identified in 37 of 129 ETs (29%, 95% confidence interval [CI] 21%-37%). Cases with MOID differed in originating service, but not demographically. Recognizing the urgency of an identified condition was the most common diagnostic process opportunity. ET cases with MOID had higher odds of mortality (odds ratio 5.5; 95% CI 1.5-20.6; p = .01) and longer posttransfer LOS (rate ratio 1.7; 95% CI 1.1-2.6; p = .02).

CONCLUSION

MOID are common in ETs and are associated with increased mortality risk and posttransfer LOS. Diagnostic improvement strategies should be leveraged to support earlier recognition of clinical deterioration.

How does work environment relate to diagnostic quality? A prospective, mixed methods study in primary care

OBJECTIVES

The quest to measure and improve diagnosis has proven challenging; new approaches are needed to better understand and measure key elements of the diagnostic process in clinical encounters. The aim of this study was to develop a tool assessing key elements of the diagnostic assessment process and apply it to a series of diagnostic encounters examining clinical notes and encounters' recorded transcripts. Additionally, we aimed to correlate and contextualise these findings with measures of encounter time and physician burnout.

DESIGN

We audio-recorded encounters, reviewed their transcripts and associated them with their clinical notes and findings were correlated with concurrent Mini Z Worklife measures and physician burnout.

SETTING

Three primary urgent-care settings.

PARTICIPANTS

We conducted in-depth evaluations of 28 clinical encounters delivered by seven physicians.

RESULTS

Comparing encounter transcripts with clinical notes, in 24 of 28 (86%) there was high note/transcript concordance for the diagnostic elements on our tool. Reliably included elements were red flags (92% of notes/encounters), aetiologies (88%), likelihood/uncertainties (71%) and follow-up contingencies (71%), whereas psychosocial/contextual information (35%) and mentioning common pitfalls (7%) were often missing. In 22% of encounters, follow-up contingencies were in the note, but absent from the recorded encounter. There was a trend for higher burnout scores being associated with physicians less likely to address key diagnosis items, such as psychosocial history/context.

CONCLUSIONS

A new tool shows promise as a means of assessing key elements of diagnostic quality in clinical encounters. Work conditions and physician reactions appear to correlate with diagnostic behaviours. Future research should continue to assess relationships between time pressure and diagnostic quality.

Association between COVID-19 and Male Fertility: Systematic Review and Meta-Analysis of Observational Studies

PURPOSE

Whether COVID-19 reduces male fertility remains requires further investigation. This meta-analysis and systematic review evaluated the impact of COVID-19 on male fertility.

MATERIALS AND METHODS

The literature in PubMed, Embase, MEDLINE, Web of Science, and Cochrane Library up to January 01, 2022 was systematically searched, and a meta-analysis was conducted to investigate the effect of COVID-19 on male fertility. Totally 17 studies with a total of 1,627 patients and 1,535 control subjects were included in our meta-analysis.

RESULTS

Regarding sperm quality, COVID-19 decreased the total sperm count (p=0.012), sperm concentration (p=0.001), total motility (p=0.001), progressive sperm motility (p=0.048), and viability (p=0.031). Subgroup analyses showed that different control group populations did not change the results. It was found that during the illness stage of COVID-19, semen volume decreased, and during the recovery stage of COVID-19, sperm concentration and total motility decreased <90 days. We found that sperm concentration and total motility decreased during recovery for ≥90 days. Fever because of COVID-19 significantly reduced sperm concentration and progressive sperm motility, and COVID-19 without fever ≥90 days, the sperm total motility and progressive sperm motility decreased. Regarding disease severity, the moderate type of COVID-19 significantly reduced sperm total motility, but not the mild type. Regarding sex hormones, COVID-19 increased prolactin and estradiol. Subgroup analyses showed that during the illness stage, COVID-19 decreased testosterone (T) levels and increased luteinizing hormone levels. A potential publication bias may have existed in our meta-analysis.

CONCLUSIONS

COVID-19 in men significantly reduced sperm quality and caused sex hormone disruption. COVID-19 had long-term effects on sperm quality, especially on sperm concentration and total motility. It is critical to conduct larger multicenter studies to determine the consequences of COVID-19 on male fertility.

Diagnostic quality model (DQM): an integrated framework for the assessment of diagnostic quality when using AI/ML

BACKGROUND

Laboratory medicine has reached the era where promises of artificial intelligence and machine learning (AI/ML) seem palpable. Currently, the primary responsibility for risk-benefit assessment in clinical practice resides with the medical director. Unfortunately, there is no tool or concept that enables diagnostic quality assessment for the various potential AI/ML applications. Specifically, we noted that an operational definition of laboratory diagnostic quality - for the specific purpose of assessing AI/ML improvements - is currently missing.

METHODS

A session at the 3rd Strategic Conference of the European Federation of Laboratory Medicine in 2022 on "AI in the Laboratory of the Future" prompted an expert roundtable discussion. Here we present a conceptual diagnostic quality framework for the specific purpose of assessing AI/ML implementations.

RESULTS

The presented framework is termed diagnostic quality model (DQM) and distinguishes AI/ML improvements at the test, procedure, laboratory, or healthcare ecosystem level. The operational definition illustrates the nested relationship among these levels. The model can help to define relevant objectives for implementation and how levels come together to form coherent diagnostics. The affected levels are referred to as scope and we provide a rubric to quantify AI/ML improvements while complying with existing, mandated regulatory standards. We present 4 relevant clinical scenarios including multi-modal diagnostics and compare the model to existing quality management systems.

CONCLUSIONS

A diagnostic quality model is essential to navigate the complexities of clinical AI/ML implementations. The presented diagnostic quality framework can help to specify and communicate the key implications of AI/ML solutions in laboratory diagnostics.

The presence of diabetic retinopathy closely associated with the progression of non-alcoholic fatty liver disease: A meta-analysis of observational studies

Background and Objective: Although growing evidence indicates that non-alcoholic fatty liver disease is related to diabetic retinopathy (DR), research results significantly vary. Therefore, we conducted a meta-analysis to assess the association between the progression of non-alcoholic fatty liver disease and the onset of DR. Methods: PubMed, Embase, and Cochrane databases were searched until 7 November 2021. Combined odds ratios (ORs) and 95% confidence intervals (CIs) were used to assess the association. Results: We identified 18 studies involving 12,757 patients. The pooled effect assessment showed that liver fibrosis was positively correlated with DR (OR = 1.69, 95%CI 1.30-2.20; p < 0.0001); non-alcoholic fatty liver disease was not associated with the risk of DR (OR = 1.15, 95%CI 0.75-1.76; p = 0.51); non-alcoholic fatty liver disease was positively correlated with DR in patients with type 1 diabetes (OR = 2.96, 95%CI 1.48-5.94; p = 0.002). In patients with type 2 diabetes, there was no association between non-alcoholic fatty liver disease and DR (OR = 0.92, 95%CI 0.59-1.43; p = 0.70). Subgroup analysis showed no correlation in both Asian and Caucasian races. Conclusion: There is a significant correlation between liver fibrosis and DR. This suggests that the ocular examination of DR could be helpful in predicting whether patients with non-alcoholic fatty liver disease would progress to liver fibrosis.

Advancing Diagnostic Safety Research: Results of a Systematic Research Priority Setting Exercise

BACKGROUND

Diagnostic errors are a major source of preventable harm but the science of reducing them remains underdeveloped.

OBJECTIVE

To identify and prioritize research questions to advance the field of diagnostic safety in the next 5 years.

PARTICIPANTS

Ninety-seven researchers and 42 stakeholders were involved in the identification of the research priorities.

DESIGN

We used systematic prioritization methods based on the Child Health and Nutrition Research Initiative (CHNRI) methodology. We first invited a large international group of expert researchers in various disciplines to submit research questions while considering five prioritization criteria: (1) usefulness, (2) answerability, (3) effectiveness, (4) potential for translation, and (5) maximal potential for effect on diagnostic safety. After consolidation, these questions were prioritized at an in-person expert meeting in April 2019. Top-ranked questions were subsequently reprioritized through scoring on the five prioritization criteria using an online questionnaire. We also invited non-research stakeholders to assign weights to the five criteria and then used these weights to adjust the final prioritization score for each question.

KEY RESULTS

Of the 207 invited researchers, 97 researchers responded and 78 submitted 333 research questions which were then consolidated. Expert meeting participants (n = 21) discussed questions in different breakout sessions and prioritized 50, which were subsequently reduced to the top 20 using the online questionnaire. The top 20 questions addressed mostly system factors (e.g., implementation and evaluation of information technologies), teamwork factors (e.g., role of nurses and other health professionals in the diagnostic process), and strategies to engage patients in the diagnostic process.

CONCLUSIONS

Top research priorities for advancing diagnostic safety in the short-term include strengthening systems and teams and engaging patients to support diagnosis. High-priority areas identified using these systematic methods can inform an actionable research agenda for reducing preventable diagnostic harm.

How patients describe their diagnosis compared to clinical documentation

OBJECTIVES

To explore how patients describe their diagnoses following Emergency Department (ED) discharge, and how this compares to electronic medical record (EMR) documentation.

METHODS

We conducted a cohort study of patients discharged from three EDs. Patients completed questionnaires regarding their understanding of their diagnosis. Inclusion criteria: adult ED patients aged 18 and older seen within the last seven days. We independently compared patient-reported new diagnoses following discharge to EMR-documented diagnoses regarding diagnostic content (identical, insignificantly different, different, not enough detail) and the level of technical language in diagnostic description (technical, semi-technical, lay).

RESULTS

The majority of participants (n=95 out of 137) reported receiving a diagnosis and stated the given diagnosis. Of those who reported their diagnosis, 66%, were females (n=62), the average age was 43 (SD 16), and a fourth (n=24) were Black and 66% (n=63) were white. The majority (84%) described either the same or an insignificantly different diagnosis. For 11% the patient-reported diagnosis differed from the one documented. More than half reported their diagnosis using semi-technical (34%) or technical language (26%), and over a third (40%) described their diagnosis in lay language.

CONCLUSIONS

Patient-reported diagnoses following ED discharge had moderate agreement with EMR-documented diagnoses. Findings suggest that patients might reproduce verbatim semi-technical or technical diagnoses they received from clinicians, but not fully understood what the diagnosis means for them.

Association between human papillomavirus infection and sperm quality: A systematic review and a meta-analysis

Human papillomavirus (HPV) has a high incidence rate in both males and females. HPV infection in women has been shown to affect fertility and lead to foetal death and pregnancy loss. However, research on HPV infection in men is limited. The aim of this study was to study the effect of HPV infection in semen on sperm quality and present the findings of previous studies through a meta-analysis. Databases including PubMed, MEDLINE, EMBASE, Web of Science, Cochrane Library, WanFang data and China National Knowledge Infrastructure were searched for relevant studies. A systematic review and meta-analysis were performed, and 17 studies were included for analyses based on a set criterion. Meta-analyses indicated that HPV infection in semen significantly reduced sperm concentration (SMD = -0.12, 95% CI: -0.21 to -0.03, p = .009), sperm motility (SMD = -0.55, 95% CI: -0.780 to -0.33, p = .000), sperm viability (SMD = -0.55, 95% CI: -0.780 to -0.33, p = .000) and sperm morphology (SMD = -0.34, 95% CI: -0.61 to -0.07, p = .015). The high-risk HPV (HrHPV) infection could significantly reduce sperm count (SMD = -0.65, 95% CI: -1.11 to -0.18, p = .007) compared with high-risk HPV (LrHPV) infection. In conclusion, HPV infection in semen significantly reduced sperm quality, and the HrHPV infection could significantly reduce sperm count compared with LrHPV.

Operational measurement of diagnostic safety: state of the science

Reducing the incidence of diagnostic errors is increasingly a priority for government, professional, and philanthropic organizations. Several obstacles to measurement of diagnostic safety have hampered progress toward this goal. Although a coordinated national strategy to measure diagnostic safety remains an aspirational goal, recent research has yielded practical guidance for healthcare organizations to start using measurement to enhance diagnostic safety. This paper, concurrently published as an Issue Brief by the Agency for Healthcare Research and Quality, issues a "call to action" for healthcare organizations to begin measurement efforts using data sources currently available to them. Our aims are to outline the state of the science and provide practical recommendations for organizations to start identifying and learning from diagnostic errors. Whether by strategically leveraging current resources or building additional capacity for data gathering, nearly all organizations can begin their journeys to measure and reduce preventable diagnostic harm.

Symptom-Disease Pair Analysis of Diagnostic Error (SPADE): a conceptual framework and methodological approach for unearthing misdiagnosis-related harms using big data

BACKGROUND

The public health burden associated with diagnostic errors is likely enormous, with some estimates suggesting millions of individuals are harmed each year in the USA, and presumably many more worldwide. According to the US National Academy of Medicine, improving diagnosis in healthcare is now considered 'a moral, professional, and public health imperative.' Unfortunately, well-established, valid and readily available operational measures of diagnostic performance and misdiagnosis-related harms are lacking, hampering progress. Existing methods often rely on judging errors through labour-intensive human reviews of medical records that are constrained by poor clinical documentation, low reliability and hindsight bias.

METHODS

Key gaps in operational measurement might be filled via thoughtful statistical analysis of existing large clinical, billing, administrative claims or similar data sets. In this manuscript, we describe a method to quantify and monitor diagnostic errors using an approach we call 'Symptom-Disease Pair Analysis of Diagnostic Error' (SPADE).

RESULTS

We first offer a conceptual framework for establishing valid symptom-disease pairs illustrated using the well-known diagnostic error dyad of dizziness-stroke. We then describe analytical methods for both look-back (case-control) and look-forward (cohort) measures of diagnostic error and misdiagnosis-related harms using 'big data'. After discussing the strengths and limitations of the SPADE approach by comparing it to other strategies for detecting diagnostic errors, we identify the sources of validity and reliability that undergird our approach.

CONCLUSION

SPADE-derived metrics could eventually be used for operational diagnostic performance dashboards and national benchmarking. This approach has the potential to transform diagnostic quality and safety across a broad range of clinical problems and settings.