Increasing cytotechnologist workload above 100 slides per day using the ThinPrep imaging system leads to significant reductions in screening accuracy

Cervical cancer: Automation of Pap test screening

BACKGROUND

Cervical cancer progresses slowly, increasing the chance of early detection of pre-neoplastic lesions via Pap exam test and subsequently preventing deaths. However, the exam presents both false-negatives and false-positives results. Therefore, automatic methods (AMs) of reading the Pap test have been used to improve the quality control of the exam. We performed a literature review to evaluate the feasibility of implementing AMs in laboratories.

METHODS

This work reviewed scientific publications regarding automated cytology from the last 15 years. The terms used were "Papanicolaou test" and "Automated cytology screening" in Portuguese, English, and Spanish, in the three scientific databases (SCIELO, PUBMED, MEDLINE).

RESULTS

Of the resulting 787 articles, 34 were selected for a complete review, including three AMs: ThinPrep Imaging System, FocalPoint GS Imaging System and CytoProcessor. In total, 1 317 148 cytopathological slides were evaluated automatically, with 1 308 028 (99.3%) liquid-based cytology slides and 9120 (0.7%) conventional cytology smears. The AM diagnostic performances were statistically equal to or better than those of the manual method. AM use increased the detection of cellular abnormalities and reduced false-negatives. The average sample rejection rate was ≤3.5%.

CONCLUSION

AMs are relevant in quality control during the analytical phase of cervical cancer screening. This technology eliminates slide-handling steps and reduces the sample space, allowing professionals to focus on diagnostic interpretation while maintaining high-level care, which can reduce false-negatives. Further studies with conventional cytology are needed. The use of AM is still not so widespread in cytopathology laboratories.

Computational Cytology: Lessons Learned from Pap Test Computer-Assisted Screening

BACKGROUND

In the face of rapid technological advances in computational cytology including artificial intelligence (AI), optimization of its application to clinical practice would benefit from reflection on the lessons learned from the decades-long journey in the development of computer-assisted Pap test screening.

SUMMARY

The initial driving force for automated screening in cytology was the overwhelming number of Pap tests requiring manual screening, leading to workflow backlogs and incorrect diagnoses. Several companies invested resources to address these concerns utilizing different specimen processing techniques and imaging systems. However, not all companies were commercially prosperous. Successful implementation of this new technology required viable use cases, improved clinical outcomes, and an acceptable means of integration into the daily workflow of cytopathology laboratories. Several factors including supply and demand, Food and Drug Administration (FDA) oversight, reimbursement, overcoming learning curves and workflow changes associated with the adoption of new technology, and cytologist apprehension, played a significant role in either promoting or preventing the widespread adoption of automated screening technologies. Key Messages: Any change in health care, particularly those involving new technology that impacts clinical workflow, is bound to have its successes and failures. However, perseverance through learning curves, optimizing workflow processes, improvements in diagnostic accuracy, and regulatory and financial approval can facilitate widespread adoption of these technologies. Given their history with successfully implementing automated Pap test screening, cytologists are uniquely positioned to not only help with the development of AI technology for other areas of pathology, but also to guide how they are utilized, regulated, and managed.

The impact of speed and bias on the cognitive processes of experts and novices in medical image decision-making

Training individuals to make accurate decisions from medical images is a critical component of education in diagnostic pathology. We describe a joint experimental and computational modeling approach to examine the similarities and differences in the cognitive processes of novice participants and experienced participants (pathology residents and pathology faculty) in cancer cell image identification. For this study we collected a bank of hundreds of digital images that were identified by cell type and classified by difficulty by a panel of expert hematopathologists. The key manipulations in our study included examining the speed-accuracy tradeoff as well as the impact of prior expectations on decisions. In addition, our study examined individual differences in decision-making by comparing task performance to domain general visual ability (as measured using the Novel Object Memory Test (NOMT) (Richler et al. Cognition 166:42–55, 2017). Using signal detection theory and the diffusion decision model (DDM), we found many similarities between experts and novices in our task. While experts tended to have better discriminability, the two groups responded similarly to time pressure (i.e., reduced caution under speed instructions in the DDM) and to the introduction of a probabilistic cue (i.e., increased response bias in the DDM). These results have important implications for training in this area as well as using novice participants in research on medical image perception and decision-making.

Factors related to inter-observer reproducibility of conventional Pap smear cytology: a multilevel analysis of smear and laboratory characteristics

OBJECTIVE

To identify factors that influence the inter-observer reproducibility of the routine, conventional Pap smear cytology (Pap smear test) in a network of certificated laboratories in a middle-income Latin American country.

METHODS

Twenty-six laboratories provided each an average of 26 negative for malignancy (NILM) and high-grade squamous intraepithelial lesion (HSIL) Pap smears. An external panel reviewed the slides. The kappa index and multilevel logistic regression were used to estimate the reproducibility and odds ratios (OR) of a false result with 95% confidence intervals (95% CI), respectively. Results are presented for laboratories that collect (collector laboratories) and do not collect (non-collector laboratories) samples.

RESULTS

The agreements ranged widely (median kappa 0.51, range 0.16-0.70). The overall false-positive (FP) and false-negative (FN) rates were 31% (95% CI 27-35) and 11% (95% CI 7-17). Among collector laboratories (N = 14), a bigger sample collection volume decreased the probability of a FP (OR-adjusted 0.05, 95% CI 0.02-0.1) whereas the number of quality defects (OR-adjusted 1.67, 95% CI 1.25-2.24), high workload (OR-adjusted 5.52, 95% CI 3.85-7.92) and collection by cytotechnologists (OR-adjusted 1.28, 95% CI 1.15-1.42) or health professionals (OR-adjusted 2.26, 95% CI 2.04-2.49) instead of nursing assistants increased it. Among non-collector laboratories (N = 9), the FP rate increased with the number of quality defects (OR-adjusted 1.86, 95% CI 1.06-3.26) but decreased if the samples were collected by health professionals instead of nursing assistants (OR-adjusted 0.37, 95%CI 0.17-0.80). No significant associations were observed for FN.

CONCLUSIONS

Staff in charge of cervical sampling significantly determined the reproducibility of the Pap smear test, but this depended on whether the laboratory collects samples or read samples collected elsewhere.

Time consumed by microscopic and nonmicroscopic tasks in image-assisted gynecologic screening: Implications for workload assessment

BACKGROUND

Gynecologic screening cytology is a complex task that includes microscopic activities and nonmicroscopic activities. The authors sought to determine the amount and percentage of time that cytotechnologists spend on those activities using the ThinPrep imaging system.

METHODS

In arm 1, a total of 550 consecutive unselected slides were reviewed by 11 cytotechnologists, and the time used for individual subtasks of the screening process was recorded. In arm 2, a total of 20 unselected slides were each screened by 10 different cytotechnologists (200 slides in total) and total screening times and full manual review (FMR) times were recorded.

RESULTS

In arm 1, cases with and without FMR required an average of 5.6 minutes and 3.0 minutes, respectively, to screen. Overall, review of fields of view (FOVs) took 95 seconds. FMR took an average of 2.6 minutes. The average screening times for FOV-only cases was significantly longer than the US Food and Drug Administration/Centers for Medicare and Medicaid Services (FDA/CMS) workload limit of 2.4 minutes (P = .005). However, in arm 2, the time needed to screen a case increased by an average of 1 minute compared with arm 1, including 1.1 minute for FOV-only cases and >2 minutes for FMR plus FOV cases. Approximately 100% of cases screened as FOV only exceeded the FDA/CMS workload limit of 2.4 minutes.

CONCLUSIONS

The FDA/CMS workload limits for FOV-only cases appears to significantly underestimate the time needed to screen those cases, but seems to be appropriate for the majority of FMR plus FOV cases. Approximately 60% and 30% of the time designated to screening slides was spent on nonmicroscopic activities for FOV-only cases and FMR cases, respectively. Cancer Cytopathol 2016;124:501-7. © 2016 American Cancer Society.

Comparative analysis of cervical cytology screening methods and staining protocols for detection rate and accurate interpretation of ASC-H: Data from a high-volume laboratory in Turkey

BACKGROUND

This study evaluated the effectiveness of the ThinPrep(®) Imaging System (TIS) and ThinPrep(®) Pap Stain (TPPS). A comparative analysis was conducted to determine the detection rates of atypical squamous cells, cannot exclude high-grade squamous intraepithelial lesion (ASC-H), the ASC:squamous intraepithelial lesion (SIL) ratio, biopsy follow-up for ASC-H in terms of the screening method used (manual screening [MS] vs. TIS screening [TISS]) and the staining protocol (regular Pap stain [RPS] vs. TPPS).

METHODS

This study was performed over two periods. The RPS period included manually screened slides, whereas the TPPS period included TIS + manually screened slides. All data from the study periods were compared using statistical analysis.

RESULTS

The detection rate of ASC-H was significantly higher during the TPPS period than during the RPS period (0.49% vs. 0.23%); this finding is in contrast to the insignificant difference between the screening method periods. The positive predictive value (PPV) of ASC-H cytodiagnosis for cervical intraepithelial neoplasia of grade 2 or more severe histologies was significantly different between manually screened and TIS slides (22.10% vs. 38.55%), in contrast to an insignificant difference between RPS and TPPS periods (37.14% vs. 29.77%).

CONCLUSION

Implementation of the TIS did not change the ASC-H detection rates appreciably. However, the new technology improved PPV for ASC-H cytodiagnosis and enabled the detection of true disease. Our laboratory statistics indicate that the TPPS is not a superior staining protocol and did not increase our diagnostic accuracy for ASC-H compared with RPS.

Trends in Cervical Cytology Screening and Reporting Practices: Results From the College of American Pathologists 2011 PAP Education Supplemental Questionnaire

Context

The College of American Pathologists periodically surveys laboratories to determine changes in cytopathology practices. We report the results of a 2011 gynecologic cytology survey.

Objective

To provide a cross-sectional survey of gynecologic cytology practices in 2010.

Design

In 2011, a survey was sent to 1604 laboratories participating in the College of American Pathologists gynecologic cytology interlaboratory comparison education program and proficiency testing programs requesting data from 2010 on the following topics: terminology/reporting, cytotechnologist workload, quality assurance, reagents, and ancillary testing.

Results

Six hundred and twenty-five laboratories (39%) replied to the survey. The nonstandard use of “low-grade squamous intraepithelial lesion cannot exclude high-grade squamous intraepithelial lesion” is used by most laboratories to report the presence of low-grade squamous intraepithelial lesion with possibility of high-grade squamous intraepithelial lesion. Most laboratories also report the presence or absence of cells from the transformation zone. Most respondents do not limit cytotechnologist screening workload during the work shift. Only about one-third of laboratories (188 of 582; 32%) use image-assisted screening devices. Rapid prescreening as a quality assurance measure is used by only 3.5% (21 of 594) of the laboratories. When used for screening, most laboratories use the imager for retrospective review of slides to detect human locator and interpretive errors. Most laboratories receive both liquid-based cytology samples (mainly ThinPrep, Hologic, Marlborough, Massachusetts) and conventional Papanicolaou tests. Expiration dates of liquid-based cytology test vials are not usually recorded.

Conclusions

The field of gynecologic cytology is evolving rapidly. These survey results offer a snapshot of national gynecologic cytology practices in 2010.

Health technology assessment of computer-assisted pap test screening in Italy

OBJECTIVE

To assess the introduction of computer-assisted Pap test screening in cervical cancer screening. Various scenarios are considered: conventional and liquid-based cytology (LBC) slides, fully automatic instrumentation (Becton Dickinson FocalPoint™ Slide Profiler and Hologic ThinPrep® Imaging System), and semiautomatic scanner (Hologic Integrated Imager I-Squared).

METHODS

A working group was formed that included researchers from the largest centers already using instrumentation. A questionnaire on laboratory management and on social/ethical issues and annual workload was proposed. Prices for the technology were obtained directly from the producers; costs were calculated from observed and literature data. The scope of the report and final draft were submitted to a consulting committee of stakeholders.

RESULTS

The break-even point was found to be 49,000 cases/year, if conventional slides were used, while it was near the theoretical maximum capacity, 70,000 cases/year, with LBC slides. Efficiency increased with the volume of slides. Screening time decreased by two thirds for conventional slides and by less than half for LBC slides. Acceptance of the instrumentation by the users was good.

CONCLUSIONS

Computer-assisted screening may increase productivity even if in most situations it will mean additional costs. Furthermore, primary screening with human papillomavirus tests will drastically reduce the need for Pap test reading.

Assessment of manual workload limits in gynecologic cytology: reconciling data from 3 major prospective trials of automated screening devices

Previous prospective studies have shown different results when comparing automated and manual screening of gynecologic cytology. The results of 3 large prospective studies were reviewed and relative sensitivity used as a gold standard. No significant differences could be shown in relative sensitivity between the ThinPrep Imaging System and the FocalPoint GS Imaging System (P > .05). When manual screening was restricted to less than 6 hours per day, 50 or fewer slides per day, and at least 6 minutes per slide (<10 slides/h), the relative sensitivity for automation was significantly lower for atypical squamous cells of undetermined significance and above (ASC+) (0.81; 95% confidence interval [CI], 0.79-0.83) than when manual screening was not restricted (1.07; 95% CI, 1.03-1.10). All 3 sites that screened 10 or more slides per hour manually had a relative sensitivity for automation that was significantly higher for high-grade squamous intraepithelial lesions and above (HSIL+) than for the remaining groups who screened less than 10 slides per hour (1.40 [95% CI, 1.22-1.60] vs 0.97 [95% CI, 0.95-1.00]). These results suggest that location finding of abnormalities (ASC+) may be more strongly associated with time spent screening per day, whereas classification/interpretation skills (HSIL+) may depend on time spent on an individual case. There is no evidence that automated screening devices are more sensitive than manual screening performed at lower well-defined workloads. More restricted workloads (≤41 slides/d, ≤4.5 h/d) for manual screening may perform significantly better than automated screening devices as measured by histologic cervical intraepithelial neoplasia 2 and above.

Increasing cytotechnologist workload above 100 slides per day using the BD FocalPoint GS imaging system negatively affects screening performance

Studies examining the effects of increased workload on the performance of individual cytotechnologists are limited. Using FocalPoint GS, the performance of 3 cytotechnologists was evaluated. The study consisted of 3 phases. In phase I, cytotechnologists were asked to screen at their usual pace. In phase II, cytotechnologists were asked to screen as fast as possible without feeling that the quality of their work was diminished. In phase III, cytotechnologists were asked to screen at least 15% more than their daily workload from phase II. Productivity was increased by decreasing the percentage of cases that underwent full manual review (from 38% to 19%) and by decreasing the time spent on each slide (from 5.5 min to 3.7 min). Overall, the total abnormal rate decreased by 31.9% from phase I to phase III of the study. In addition, the false-negative fraction increased significantly, from 1% to 6.9%. Our results indicated a negative association between increased cytotechnologist daily workload with FocalPoint GS and CT screening performance. Workloads were increased by decreasing the time spent reviewing 10 fields of view and the percentage of cases that underwent full manual review.

Quality Improvement in Cytology: Where Do We Go From Here?

Context.—Cytology is a success because of the many quality controls used to ensure the accuracy of its results. Nevertheless, additional information is becoming available to the cytologist, often from untraditional sources, and the best way to use that information to improve the quality of cytology is not yet known.

Objective.—To review ways to use new information to improve the quality of cytology.

Data Sources.—Review of relevant literature.

Results.—Information contained in many sources can be used in new ways to improve the quality of cytology. These include the timing of cytologic and histologic correlation, electronic medical records, workload information, prior aspirations, and molecular tests.

Conclusions.—To maintain their high standard of excellence, cytologists should seek to define the most appropriate way to incorporate this new information into their interpretation of individual cases.

Cytology workforce study: a report of current practices and trends in New York State

A survey was conducted among 130 New York State (NYS) registered cytology laboratories to better understand current and future changes in the practice of cytology, changes in the cytotechnologist (CT) scope of practice, and the future need for CTs. A 51.5% (67/130) response rate was obtained. Trends for gynecologic case volume varied across facility types. Nongynecologic volume is growing primarily in hospitals and large medical center laboratories and private laboratories; the fine-needle aspiration volume is growing in hospital and large medical center laboratories. One third of responding laboratories anticipate a continued demand for CTs within the next 3 years owing to impending retirements. Few laboratories also report the gradual adoption of molecular testing with CTs directly involved. Because 60% (3/5) of NYS CT training programs have closed since 2008, the 2 remaining programs are a valuable key staffing resource for CTs. Continued viability of these programs is essential to provide the necessary training and staffing of NYS laboratories for cytopathology practice.

Manual, semi-automated, and automated delineation of chronic brain lesions: a comparison of methods

The exact delineation of chronic brain lesions is a crucial step when investigating the relationship between brain structure and (dys-)function. For this, manual tracing, although very time-consuming, is still the gold standard. In order to assess the possible contributions from other methods, we compared manual tracing of lesion boundaries with a newly developed semi-automated and a fully automated approach for lesion definition in a sample of chronic stroke patients (n=11, 5m, median age 12, range 10-30years). Manual tracing requires substantially more human input (4.8-9.6h/subject) than semi-automated (24.9min/subject) and automated processing (1min/subject). When compared with manual tracing as the gold standard, both the semi-automated (tested with 4 different smoothing filters) and the automated approach towards lesion definition performed on an acceptable level, with an average Dice's similarity index of .53-.60 (semi-automated) and .49 (automated processing). In all semi-automated and automated approaches, larger lesions were identified with a significantly higher performance than smaller lesions, as were central versus peripheral voxels, indicating that the surface-to-volume ratio explains this trend. The automated approach failed to identify two lesions. In several cases, indirect lesion effects (such as enlarged ventricles) were detected using the semi-automated or the automated approach. We conclude that manual tracing remains the gold standard for exact lesion delineation, but that semi-automated and automated approaches may be alternatives for larger lesions and/or larger studies. The detection of indirect lesion effects may be another application of such approaches in the future.