Interobserver variation in classifying lymphomas among hematopathologists

Impact of a deep learning assistant on the histopathologic classification of liver cancer

Artificial intelligence (AI) algorithms continue to rival human performance on a variety of clinical tasks, while their actual impact on human diagnosticians, when incorporated into clinical workflows, remains relatively unexplored. In this study, we developed a deep learning-based assistant to help pathologists differentiate between two subtypes of primary liver cancer, hepatocellular carcinoma and cholangiocarcinoma, on hematoxylin and eosin-stained whole-slide images (WSI), and evaluated its effect on the diagnostic performance of 11 pathologists with varying levels of expertise. Our model achieved accuracies of 0.885 on a validation set of 26 WSI, and 0.842 on an independent test set of 80 WSI. Although use of the assistant did not change the mean accuracy of the 11 pathologists (p = 0.184, OR = 1.281), it significantly improved the accuracy (p = 0.045, OR = 1.499) of a subset of nine pathologists who fell within well-defined experience levels (GI subspecialists, non-GI subspecialists, and trainees). In the assisted state, model accuracy significantly impacted the diagnostic decisions of all 11 pathologists. As expected, when the model's prediction was correct, assistance significantly improved accuracy (p = 0.000, OR = 4.289), whereas when the model's prediction was incorrect, assistance significantly decreased accuracy (p = 0.000, OR = 0.253), with both effects holding across all pathologist experience levels and case difficulty levels. Our results highlight the challenges of translating AI models into the clinical setting, and emphasize the importance of taking into account potential unintended negative consequences of model assistance when designing and testing medical AI-assistance tools.

Diagnostic errors in fatal medical malpractice cases in Shanghai, China: 1990-2015

Background

Medical disputes remain unabated in China. Previous studies have shown the changes of diagnostic discrepancy over time in developed countries, but diagnostic discrepancy remains understudied in China, especially in the setting of medical disputes. We sought to describe the year-based changes of diagnostic discrepancies in medical disputes, and to identify factors associated with classes of diagnostic discrepancy.

Methods

We conducted a retrospective cohort study of all medically disputed cases from 1990 through 2015 in Shanghai, China, with use of necropsy as the gold standard for diagnosis. Cases were grouped based on national legislative eras. Diagnostic discrepancy was classified as major errors (class I and II), minor errors (class III and IV), no discrepancy (class V) and undetermined (class VI) based on discrepancy severity.

Results

There were 482 medical disputes. Cases were predominantly males (male: female = 1.6:1) and concentrated in patients less than 10 years old or between 50 and 70 years. Major and minor discrepancy accounted for 51.7 and 34.8%, respectively. Fifty-five cases (11.2%) were non-discrepant (Class V). The dispute rate remained high before the first round of legislation (mean 0.31 per 1 million patients) but declined dramatically afterwards (R² = − 0.82, p < 0.001 for time trends). Over the national legislative eras, the annual number of cases with diagnostic errors declined steadily. Incidence rates of discrepancy decreased significantly for class I (R² = − 0.73, p = 0.024), II (R² = − 0.48, p = 0.013), III (R² = − 0.69, p < 0.0001), IV (R² = − 0.69, p < 0.0001) and V discrepancy (R² = − 0.58, p = 0.0018). Diseases from the respiratory system had significantly lower risks of any diagnostic errors (OR = 0.48, 95% 0.24–0.95, p = 0.036). A neoplasm carrier increased by 92% the risk of any diagnostic error (OR = 1.92; 95%CI 1.18–3.14; p = 0.009) and hypertension reduced by 78% the risk of minor errors (OR = 0.22, 95%CI 0.06–0.91, p = 0.036). Severity of discrepancy relieved over years and associated with ageing in patients with cardiovascular diseases (p = 0.01).

Conclusions

The rate of fatal medical disputes and diagnostic discrepancy declined after stepwise legislations in China. Respiratory diseases, neoplasm carrier and hypertension could be independent predictors for assessing diagnostic errors.