Diagnostic accuracy of second-opinion diagnoses based on still images

[From symptom to diagnosis-symptom checkers re-evaluated : Are symptom checkers finally sufficient and accurate to use? An update from the ENT perspective]

BACKGROUND

Every seventh diagnosis is a misdiagnosis. Each year, 1.5 million lives could be saved worldwide with the correct diagnosis. Physicians have to consider over 20,000 diseases. A study from Harvard University published in 2015 tested 19 symptom checkers and found them to be insufficient, with only 29-71% accuracy in diagnosis.

OBJECTIVE

The current study investigates the diagnostic accuracy of new symptom checkers from an ENT perspective.

MATERIALS AND METHODS

The authors update the abovenamed diagnostic accuracy comparison by (1) including the five new symptom checkers Symptoma, Ada, FindZebra, Mediktor, and Babylon; and (2) normalizing results of the previously tested symptom checkers as to reflect each diagnostic accuracy based on the same set of patient vignettes. The winner is then compared to the two symptom checkers with the most scientific evidence, namely Isabel and FindZebra, on the basis of an ENT-specific test with patient vignettes sourced from the British Medical Journal.

RESULTS

Most of the new symptom checkers demonstrated diagnostic accuracy rates within the previously established range, with the exception of Symptoma, which scored the right diagnosis in 82.2% of cases at the top of the list (+38% points), and in 100% of cases in the top 3 (+29% points) and the top 10 (+16% points), thus raising the bar in this field. The cross-validation with ENT cases resulted in a diagnostic accuracy of 64.3 vs. 21.4 vs. 26.2% (top 1), 92.9 vs. 40.5 vs. 42.9% (top 3), and 100 vs. 61.9 vs. 54.8% (top 10) for Symptoma vs. Isabel vs. FindZebra, respectively.

CONCLUSIONS

Symptoma is the first and only viable solution in this market. Large-scale studies should be conducted to further validate these results as well as to assess the actual practical performance of the symptom checkers and their ability to diagnose rare diseases.

The history of pathology informatics: A global perspective

Pathology informatics has evolved to varying levels around the world. The history of pathology informatics in different countries is a tale with many dimensions. At first glance, it is the familiar story of individuals solving problems that arise in their clinical practice to enhance efficiency, better manage (e.g., digitize) laboratory information, as well as exploit emerging information technologies. Under the surface, however, lie powerful resource, regulatory, and societal forces that helped shape our discipline into what it is today. In this monograph, for the first time in the history of our discipline, we collectively perform a global review of the field of pathology informatics. In doing so, we illustrate how general far-reaching trends such as the advent of computers, the Internet and digital imaging have affected pathology informatics in the world at large. Major drivers in the field included the need for pathologists to comply with national standards for health information technology and telepathology applications to meet the scarcity of pathology services and trained people in certain countries. Following trials by a multitude of investigators, not all of them successful, it is apparent that innovation alone did not assure the success of many informatics tools and solutions. Common, ongoing barriers to the widespread adoption of informatics devices include poor information technology infrastructure in undeveloped areas, the cost of technology, and regulatory issues. This review offers a deeper understanding of how pathology informatics historically developed and provides insights into what the promising future might hold.

A study assessing the feasibility and diagnostic accuracy of real-time teledermatopathology

Dermatopathology represents the gold standard for the diagnosis of skin diseases and neoplasms that cannot be diagnosed on clinical grounds alone. The aim of this study was to test the feasibility and to assess the accuracy of an Internet-based real-time (live) teledermatopathology consultation. Twenty teaching cases and 10 randomly selected routine cases were presented to four expert dermatopathologists, first by real-time teledermatopathology and, subsequently, in a blinded fashion, using light microscopy. Throughout the study the overall diagnostic accuracy did not differ for the two methods. However, the mean level of confidence and the mean observation times differed significantly between real-time teledermatopathology and light microscopy (92.6±0.24% versus 99.5±0.02%, and 96.31±11.55 sec versus 25.47±3.85 sec, respectively). Assessment of routine cases did not produce significant diagnostic differences between the two methods. These results prove that real-time teledermatopathology offers an affordable and technically simple technology that lends itself to training as well as to diagnosis of lesions from routine practice by experts situated at remote sites.

Infection, rejection, and hospitalizations in transplant recipients using telehealth

CONTEXT

Telehealth technology serves individuals who live in geographical areas that prohibit easy access to specialized health care and can provide transplant recipients with access to transplant center personnel for adjunctive follow-up care.

OBJECTIVE

To compare infection, rejection, and hospitalization events in subjects randomized to telehealth or to standard posttransplant care. STUDY DESIGN, STUDY PARTICIPANTS, SETTING AND RESEARCH PROCEDURE: This longitudinal prospective study compared transplant outcomes (infections, rejections, and hospitalizations) of 106 subjects who were randomized to either the telehealth (n = 53) or standard care (n = 53) group and met the 6-month study end point. Sex, race, and transplant type were evenly distributed within the 2 groups. Subjects received primary follow-up care from nurse practitioners. The telehealth visits were conducted via live interactive sessions with digitized equipment used to perform physical examinations.

MAIN OUTCOMES

Infections, rejections, and hospitalizations were summarized for each of the groups. Subgroup analyses were performed by sex, transplant type, and time since transplant.

RESULTS

No differences were found between the telehealth and standard care groups for infections, rejections, or hospitalizations at the 6-month data end point. Overall, females had twice as many infections as males (P = .01). In this analysis, group assignment did not affect study outcomes.

CONCLUSIONS

The rates of infection, rejection, and hospitalization in a sample of primarily long-term transplant patients did not differ between patients who received telehealth follow-up and patients who received standard care, indicating that this delivery system can be used to provide follow-up care after transplant.

Infection, Rejection, and Hospitalizations in Transplant Recipients Using Telehealth

Context

Telehealth technology serves individuals who live in geographical areas that prohibit easy access to specialized health care and can provide transplant recipients with access to transplant center personnel for adjunctive follow-up care.

Objective

To compare infection, rejection, and hospitalization events in subjects randomized to telehealth or to standard posttransplant care.

Study Design, Study Participants, Setting and Research Procedure

This longitudinal prospective study compared transplant outcomes (infections, rejections, and hospitalizations) of 106 subjects who were randomized to either the telehealth (n=53) or standard care (n=53) group and met the 6-month study end point. Sex, race, and transplant type were evenly distributed within the 2 groups. Subjects received primary follow-up care from nurse practitioners. The telehealth visits were conducted via live interactive sessions with digitized equipment used to perform physical examinations.

Main Outcomes

Infections, rejections, and hospitalizations were summarized for each of the groups. Subgroup analyses were performed by sex, transplant type, and time since transplant.

Results

No differences were found between the telehealth and standard care groups for infections, rejections, or hospitalizations at the 6-month data end point. Overall, females had twice as many infections as males ( P = .01). In this analysis, group assignment did not affect study outcomes.

Conclusions

The rates of infection, rejection, and hospitalization in a sample of primarily long-term transplant patients did not differ between patients who received telehealth follow-up and patients who received standard care, indicating that this delivery system can be used to provide follow-up care after transplant.

A randomized controlled trial of the diagnostic accuracy of internet-based telepathology compared with conventional microscopy

AIMS

To compare the diagnostic accuracy of internet-based virtual microscopy with conventional light microscopy, in the context of renal biopsies assessed by participants in the UK National Renal Pathology External Quality Assessment (EQA) Scheme.

METHODS AND RESULTS

'Virtual slides' of current EQA cases were made available over the internet to participants in the scheme. The approach permitted what may reasonably be described as a randomized controlled trial of the diagnostic accuracy of this mode of telepathology, in the context of renal pathology. No significant difference in diagnostic accuracy could be detected between the diagnoses proffered on the basis of virtual slides and conventional slides; but using virtual slides took pathologists considerably longer.

CONCLUSIONS

This result provides some encouragement for the implementation of such virtual slide-based telepathology systems. However, in failing to detect a difference, equivalence of diagnostic accuracy has not been proved; nor has the relevance of this result to other fields of histopathology been demonstrated. A decision to implement a diagnostic telepathology system requires consideration of numerous factors beyond diagnostic accuracy, including financial, legal, professional and ethical issues.

Teledermatopathology: A Controlled Study About Diagnostic Validity and Technical Requirements for Digital Transmission

Telepathology is the practice of diagnostic histopathology performed on digital pictures. In this study, we focused on the technical requirements for achievement of a correct diagnosis on digital histopathologic images. A collection of 560 melanocytic lesions was selected from the files of the Department of Dermatology, Medical University of Graz, Austria. From each lesion one histologic slide was completely digitally scanned with a robotic microscope. Digital pictures were reviewed by 4 dermatopathologists using a presentation program, which recorded the number of image calls, applied magnifications, overall time needed, and amount of transmitted bits during the digital sign-out. One month later, the 4 microscopists had to review the corresponding slides and render a direct diagnosis on each case. Telepathologic diagnoses corresponded with the original diagnoses in a range from 90.4% to 96.4% of cases (kappa 0.80 to 0.93; P < 0.001). The median time needed for achievement of a diagnosis was 22 seconds and was significantly higher for melanomas compared with nevi. The median transmission effort for each diagnosis was 510 kilobytes after JPEG compression. Using an ISDN line with a transmission capacity of 64 kilobits/ second, this correlates to a transmission time of about 1 minute. Our results demonstrate that correct reporting on digital histopathologic images is possible with only a little time exposure. For an adequately fast transmission ISDN lines are suffcient after JPEG compression.

A feasibility study of email communication between the patient's family and the specialist burns team

We investigated whether the parents of burns patients could capture suitable clinical images with a digital camera and add the necessary text information to enable the paediatric burns team to provide follow-up care via email. Four families were involved in the study, each of whom sent regular email consultations for six months. The results were very encouraging. The burns team felt confident that the clinical information in 30 of the 32 email messages (94%) they received was accurate, although in 11 of these 30 cases (37%) they stated that there was room for improvement (the quality was nonetheless adequate for clinical decision making). The study also showed that low-resolution images (average size 37 kByte) were satisfactory for diagnosis. Families were able to participate in the service without intensive training and support. The user survey showed that all four families found it easy and convenient to take the digital photographs and to participate in the study. The results suggest that the technique has potential as a low-cost telemedicine service in burns follow-up, and that it requires only modest investment in equipment, training and support.

An array microscope for ultrarapid virtual slide processing and telepathology. Design, fabrication, and validation study

This paper describes the design and fabrication of a novel array microscope for the first ultrarapid virtual slide processor (DMetrix DX-40 digital slide scanner). The array microscope optics consists of a stack of three 80-element 10 x 8-lenslet arrays, constituting a "lenslet array ensemble." The lenslet array ensemble is positioned over a glass slide. Uniquely shaped lenses in each of the lenslet arrays, arranged perpendicular to the glass slide constitute a single "miniaturized microscope." A high-pixel-density image sensor is attached to the top of the lenslet array ensemble. In operation, the lenslet array ensemble is transported by a motorized mechanism relative to the long axis of a glass slide. Each of the 80 miniaturized microscopes has a lateral field of view of 250 microns. The microscopes of each row of the array are offset from the microscopes in other rows. Scanning a glass slide with the array microscope produces seamless two-dimensional image data of the entire slide, that is, a virtual slide. The optical system has a numerical aperture of N.A.= 0.65, scans slides at a rate of 3 mm per second, and accrues up to 3,000 images per second from each of the 80 miniaturized microscopes. In the ultrarapid virtual slide processing cycle, the time for image acquisition takes 58 seconds for a 2.25 cm2 tissue section. An automatic slide loader enables the scanner to process up to 40 slides per hour without operator intervention. Slide scanning and image processing are done concurrently so that post-scan processing is eliminated. A virtual slide can be viewed over the Internet immediately after the scanning is complete. A validation study compared the diagnostic accuracy of pathologist case readers using array microscopy (with images viewed as virtual slides) and conventional light microscopy. Four senior pathologists diagnosed 30 breast surgical pathology cases each using both imaging modes, but on separate occasions. Of 120 case reads by array microscopy, there were 3 incorrect diagnoses, all of which were made on difficult cases with equivocal diagnoses by light microscopy. There was a strong correlation between array microscopy vs. "truth" diagnoses based on surgical pathology reports. The kappa statistic for the array microscopy vs. truth was 0.96, which is highly significant (z=10.33, p <0.001). There was no statistically significant difference between rates of agreement with truth between array microscopy and light microscopy (z=0.134, p >0.05). Array microscopy and light microscopy did not differ significantly with respect to the number/percent of correct decisions rendered (t=0.552, p=0.6376) or equivocal decisions rendered (t=2.449, p=0.0917). Pathologists rated 95.8% of array microscopy virtual slide images as good or excellent. None were rated as poor. The mean viewing time for a DMetrix virtual slide was 1.16 minutes. The DMetrix virtual slide processor has been found to reduce the virtual slide processing cycle more than 10 fold, as compared with other virtual slide systems reported to date. The virtual slide images are of high quality and suitable for diagnostic pathology, second opinions, expert opinions, clinical trials, education, and research.