The potential for failure in gynecologic regulatory proficiency testing with current slide validation criteria: results from the College of American Pathologists Interlaboratory Comparison in Gynecologic Cytology Program

Gynecologic Cytology Proficiency Testing Failures: What Have We Learned?: Observations From the College of American Pathologists Gynecologic Cytology Proficiency Testing Program

Context.—In 2006, the first gynecologic cytology proficiency tests were offered by the College of American Pathologists. Four years of data are now available using field-validated slides, including conventional and liquid-based Papanicolaou tests.

Objective.—To characterize the pattern of error types that resulted in initial proficiency-test failure for cytotechnologists, primary screening pathologists, and secondary pathologists (those whose slides are prescreened by cytotechnologists).

Design.—The results of 37 029 initial College of American Pathologists Papanicolaou proficiency tests were reviewed from 4 slide-set modules: conventional, ThinPrep, SurePath, or a module containing all 3 slide types.

Results.—During this 4-year period, cytotechnologists were least likely to fail the initial test (3.4%; 614 of 18 264), followed by secondary pathologists (ie, those reviewing slides already screened by a cytotechnologist) with a failure rate of 4.2% (728 of 17 346), and primary pathologists (ie, those screening their own slides) having the highest level of failure (13.7%; 194 of 1419). Failure rates have fallen for all 3 groups over time. Pathologists are graded more stringently on proficiency tests, and more primary pathologists would have passed if they had been graded as cytotechnologists. There were no significant differences among performances using different types of slide sets. False-positive errors were common for both primary (63.9%; 124 of 194 errors) and secondary (55.6%; 405 of 728 errors) pathologists, whereas automatic failures were most common for cytotechnologists (75.7%; 465 of 614 errors).

Conclusions.—The failure rate is decreasing for all participants. The failures for primary pathologist screeners are due to false-positive responses. Primary screening cytotechnologists and secondary pathologists have automatic failures more often than do primary screening pathologists.

Automatic failure in gynecologic cytology proficiency testing. Results from the College of American Pathologists proficiency testing program

CONTEXT

Automatic failure in gynecologic cytology proficiency testing occurs when a high-grade lesion or carcinoma (HSIL+, Category D) is misinterpreted as negative for intraepithelial lesion or malignancy (Category B).

OBJECTIVES

To document the automatic failure rate in 2006 and 2007 from the College of American Pathologists proficiency testing program (PAP PT) and compare them to projected values from 2004.

DESIGN

Identify automatic failures from PAP PT in 2006 and 2007 and compare the rates of failure regarding participant and preparation type to validated slides in the College of American Pathologists Interlaboratory Comparison Program in 2004.

RESULTS

There were 65 264 participant responses for HSIL+ slides included in this analysis from 2006 and 2007. Overall, 1% (666 of 65 264) of the HSIL+ responses were classified as negative, resulting in automatic failure for the participant. There were significantly fewer automatic failures in 2007 as compared with either 2006 or projected from 2004 data (P < .001). Conventional preparations had a lower automatic failure rate than liquid-based preparations but only for 2006. Both pathologists and cytotechnologists interpreting liquid-based preparations faired better than projected from 2004 data.

CONCLUSIONS

The automatic failure rate in PAP PT is lower than expected based on 2004 data from the College of American Pathologists Interlaboratory Comparison Program. Automatic failures are a relatively small component (1% or less) of proficiency testing failures. The rate of automatic failure decreased from 2006 to 2007 and may be due to loss of poor performers in the testing pool, the test-taking environment, or removal of less robust slides from the program.

Summary of the 2006 College of American Pathologists Gynecologic Cytology Proficiency Testing Program

CONTEXT

Creating a tool that assesses professional proficiency in gynecologic cytology is challenging. A valid proficiency test (PT) must reflect practice conditions, evaluate locator and interpretive skills, and discriminate between those practitioners who are competent and those who need more education. The College of American Pathologists Gynecologic Cytology Proficiency Testing Program (PAPPT) was approved to enroll participants in a nationwide PT program in 2006.

OBJECTIVE

Report results from the 2006 PAPPT program.

DESIGN

Summarize PT results by pass/fail rate, participant type, and slide-set modules.

RESULTS

Nine thousand sixty-nine participants showed initial PT failure rates of 5%, 16%, and 6% for cytotechnologists, primary screening pathologists, and secondary screening pathologist, respectively. The overall initial test failure rate was 6%. After 3 retests, 9029 (99.6%) of the participants were able to achieve compliance with the PT requirement. No participant "tested out"; however, 40 individuals "dropped out" of the testing sequence (8 cytotechnologists, 9 primary screening pathologists, 23 secondary screening pathologists). Initial failure rates by slide-set modules were 6% conventional, 6% ThinPrep, 6% SurePath, and 5% mixture of all 3 slide types.

CONCLUSIONS

A total of 99.6% of individuals enrolled in the 2006 PAPPT program achieved satisfactory results. The data confirm that cytotechnologists have higher initial pass rates than pathologists and pathologists who are secondary screeners perform better than those who are primary screeners. There was no difference identified in overall pass rates between the slide-set modules. Further analysis of data should help define the results and ongoing challenges of providing a nationwide federally mandated proficiency testing program in gynecologic cytology.