What's the "secret sauce"? How implementation variation affects the success of colorectal cancer screening outreach

BACKGROUND

Mailed fecal immunochemical testing (FIT) programs can improve colorectal cancer (CRC) screening rates, but health systems vary how they implement (i.e., adapt) these programs for their organizations. A health insurance plan implemented a mailed FIT program (named BeneFIT), and participating health systems could adapt the program. This multi-method study explored which program adaptations might have resulted in higher screening rates.

METHODS

First, we conducted a descriptive analysis of CRC screening rates by key health system characteristics and program adaptations. Second, we generated an overall model by fitting a weighted regression line to our data. Third, we applied Configurational Comparative Methods (CCMs) to determine how combinations of conditions were linked to higher screening rates. The main outcome measure was CRC screening rates.

RESULTS

Seventeen health systems took part in at least 1 year of BeneFIT. The overall screening completion rate was 20% (4-28%) in year 1 and 25% (12-35%) in year 2 of the program. Health systems that used two or more adaptations had higher screening rates, and no single adaptation clearly led to higher screening rates. In year 1, small systems, with just one clinic, that used phone reminders (n = 2) met the implementation success threshold (≥ 19% screening rate) while systems with > 1 clinic were successful when offering a patient incentive (n = 4), scrubbing mailing lists (n = 4), or allowing mailed FIT returns with no other adaptations (n = 1). In year 2, larger systems with 2-4 clinics were successful with a phone reminder (n = 4) or a patient incentive (n = 3). Of the 10 systems that implemented BeneFIT in both years, seven improved their CRC screening rates in year 2.

CONCLUSIONS

Health systems can choose among many adaptations and successfully implement a health plan's mailed FIT program. Different combinations of adaptations led to success with health system size emerging as an important contextual factor.

Verifying performance characteristics of quantitative analytical systems: calibration verification, linearity, and analytical measurement range

CONTEXT

Both the regulations in the Clinical Laboratory Improvement Amendments of 1988 (CLIA) and the checklists of the College of American Pathologists (CAP) Laboratory Accreditation Program require clinical laboratories to verify performance characteristics of quantitative test systems. Laboratories must verify performance claims when introducing an unmodified, US Food and Drug Administration-cleared or approved test system, and they must comply with requirements for periodic calibration and calibration verification for existing test systems. They must also periodically verify the analytical measurement range of many quantitative test systems.

OBJECTIVE

To provide definitions for many of the terms used in these regulations, to describe a set of basic analyses that laboratories may adapt to demonstrate compliance with both CLIA and the CAP Laboratory Accreditation Program checklists for performing calibration verification and for verifying the analytical measurement range of test systems, to review some of the recommended procedures for establishing performance goals, and to provide data illustrating the performance goals used in some of the CAP's calibration verification and linearity surveys.

DATA SOURCES

The CAP's calibration verification and linearity survey programs, the CLIA regulations, the Laboratory Accreditation Program requirements, and published literature were used to meet these objectives.

CONCLUSIONS

Calibration verification and linearity and analytical measurement range verification should be performed using suitable materials with assessment of results using well-defined evaluation protocols. We describe the CAP's calibration verification and linearity programs that may be used for these purposes.

Recent trends in performance and current state of creatinine assays

CONTEXT

Measured plasma or serum creatinine concentration is a primary component of equations used to calculate estimated glomerular filtration rate (eGFR). In recent years, most assay manufacturers have adopted creatinine calibration procedures that are traceable to the National Institute of Standards and Technology's Standard Reference Material 967.

OBJECTIVES

To examine the current performance of creatinine assays, to compare changes in assay performance since 2003, and to examine the reliability of laboratory eGFR calculations.

DESIGN

Serum samples spiked with different concentrations of creatinine were analyzed by participating laboratories in the College of American Pathologists' LN24 survey. Participants' reported values were compared against values measured by liquid chromatography-isotope dilution mass spectrometry. Participants were asked to calculate the eGFR for certain samples, and results were compared with those obtained from the 4-parameter Modification of Diet in Renal Disease equation.

RESULTS

Biases among current creatinine methods are in the range of -5% to 10%, compared with -7% to 34% seen in a 2003 study. This degree of bias in eGFR calculations is of clinical significance only for concentrations near the cut points used to stage chronic kidney disease. Approximately 20% of laboratories report eGFR values that exceed ±1 mL/min per 1.73 m(2) from the expected eGFR using the 4-parameter Modification of Diet in Renal Disease equation.

CONCLUSIONS

Since 2003, there have been improvements in the performance of creatinine assays, which appear to be related to the adoption of standard reference materials for calibration. The effect of the observed method biases in clinical practice now appears minimal. Laboratories should continue to monitor the accuracy of eGFR calculations.

Papanicolaou Tests With Mixed High-Grade and Low-Grade Squamous Intraepithelial Lesion Features: Distinct Performance in the College of American Pathologists Interlaboratory Comparison Program in Cervicovaginal Cytopathology

Context.—Previous studies have shown that in gynecologic cytology, cases of low-grade squamous intraepithelial lesion (LSIL) and high-grade squamous intraepithelial lesion (HSIL) perform differently on interpretive review. The performance of cases with mixed LSIL and HSIL features is unknown.

Objective.—To compare the performance of gynecologic cytology cases of “pure” LSIL and HSIL with cases showing mixed LSIL and HSIL features.

Design.—We compiled performance data from the College of American Pathologists Interlaboratory Comparison Program in Cervicovaginal Cytopathology from the years 2003 and 2004, and compared the performance of slides showing relatively pure LSIL and HSIL (≤10% misclassification as HSIL and LSIL, respectively) with slides showing mixed LSIL or HSIL features (cases misclassified as LSIL or HSIL >10% of the time).

Results.—Interpretations from a total of 4508 cases (2452 HSIL and 2056 LSIL) were analyzed. Overall, the sensitivity of participants on slides with a reference diagnosis of HSIL was 97.3%, and of LSIL was 95.9%. Performance trends for pure versus mixed cases varied by slide type and reference diagnosis. For conventional slides, participant sensitivity on pure HSIL cases was greatest (98.0%) and on pure LSIL cases was least (95.2%), while participant performance on cases with mixed features was intermediate (97.0% for mixed HSIL and 96.7% for mixed LSIL). In contrast, participant performance on ThinPrep slides showed the greatest sensitivity for mixed LSIL slides (97.9%), while performance on mixed HSIL slides showed the lowest sensitivity (95.7%); slides with pure features had intermediate sensitivity levels (96.3% for both HSIL and LSIL). Further evaluation demonstrated that conventional pure HSIL slides performed significantly better than mixed HSIL slides (P = .006), whereas mixed LSIL slides performed better than pure LSIL slides (P = .01). For ThinPrep slides, pure HSIL cases performed similarly to mixed HSIL cases (P = .43), while mixed LSIL cases performed better than pure LSIL cases (P = .04).

Conclusion.—Slides with mixed LSIL and HSIL features have measurably distinct performance characteristics in comparison to slides with pure LSIL or HSIL features. Participant performance on conventional mixed cases is distinctly different from performance on ThinPrep mixed cases.

Atypical epithelial cells and specimen adequacy: current laboratory practices of participants in the college of American pathologists interlaboratory comparison program in cervicovaginal cytology

CONTEXT

The Bethesda System for reporting cervical/vaginal cytologic diagnoses introduced terminology for atypical squamous and glandular cells and categories for specimen adequacy.

OBJECTIVES

To analyze current laboratory reporting practices and compare trends to previous surveys.

DESIGN

Questionnaire surveys were mailed to 2000 laboratories in 1996 and 1997.

PARTICIPANTS

Laboratories enrolled in the College of American Pathologists Interlaboratory Comparison Program in Cervicovaginal Cytology.

MAIN OUTCOME MEASURES

Laboratory policies, criteria, and reporting rates for Bethesda System categories.

RESULTS

The 1996 specimen adequacy survey had 1166 respondents, and 768 laboratories returned the 1997 questionnaire focusing on atypical squamous cells of undetermined significance (ASCUS) and glandular cells of undetermined significance (AGUS). Nearly all laboratories (92%) routinely reported specimen adequacy, an increase from the 66% rate in 1991. The median rate for unsatisfactory specimens was 0.5% (mean 0.95%), and the median rate for the satisfactory but limited category was 5.8% (mean 9.3%). The Bethesda criteria for designating a specimen unsatisfactory were used by more than 90% of laboratories. Nearly all laboratories (97%) used the term ASCUS in 1997, and more than 80% of laboratories used the Bethesda criteria for this category. Median reporting rates for epithelial abnormalities were as follows: ASCUS, 4.5%; AGUS, 0.3%; low-grade squamous intraepithelial lesion (SIL), 1.6%; and high-grade SIL, 0.5%. The median ASCUS/SIL ratio was 2.0, with 80% of laboratories reporting ratios between 0.64 and 4.23. The median ASCUS rate and ASCUS/SIL ratio were higher than 1993 survey results. Nearly all laboratories attempted follow-up studies on patients with abnormal cytology results, and midsized laboratories achieved the highest rates of follow-up. Median rates of abnormalities following an ASCUS or AGUS diagnosis were 20% and 15%, respectively. Laboratory respondents commonly used written recommendations in ASCUS/AGUS reports.

CONCLUSIONS

Most laboratories that responded to the surveys had adopted Bethesda terminology and criteria for specimen adequacy and ASCUS/AGUS. Reporting rates for SIL and adequacy categories have remained stable, but median ASCUS rates and ASCUS/SIL ratios are higher than in 1993. The AGUS category is reported infrequently, but can be associated with significant pathology.

Effect of outdoor airborne particulate matter on daily death counts

To investigate the possible relationship between airborne particulate matter and mortality, we developed regression models of daily mortality counts using meteorological covariates and measures of outdoor PM10. Our analyses included data from Cook County, Illinois, and Salt Lake County, Utah. We found no evidence that particulate matter < or = 10 microns (PM10) contributes to excess mortality in Salt Lake County, Utah. In Cook County, Illinois, we found evidence of a positive PM10 effect in spring and autumn, but not in winter and summer. We conclude that the reported effects of particulates on mortality are unconfirmed.