Behavioral interventions to improve population health outreach for hepatitis C screening: randomized clinical trial

OBJECTIVE

To evaluate whether opt out framing, messaging incorporating behavioral science concepts, or electronic communication increases the uptake of hepatitis C virus (HCV) screening in patients born between 1945 and 1965.

DESIGN

Pragmatic randomized controlled trial.

SETTING

43 primary care practices from one academic health system (Philadelphia, PA, USA) between April 2019 and May 2020.

PARTICIPANTS

Patients born between 1945 and 1965 with no history of screening and at least two primary care visits in the two years before the enrollment period.

INTERVENTIONS

This multilevel trial was divided into two studies. Substudy A included 1656 eligible patients of 17 primary care clinicians who were randomized in a 1:1 ratio to a mailed letter about HCV screening (letter only), or a similar letter with a laboratory order for HCV screening (letter+order). Substudy B included the remaining 19 837 eligible patients followed by 417 clinicians. Active electronic patient portal users were randomized 1:5 to receive a mailed letter about HCV screening (letter), or an electronic patient portal message with similar content (patient portal); inactive patient portal users were mailed a letter. In a factorial design, patients in substudy B were also randomized 1:1 to receive standard content (usual care), or content based on principles of social norming, anticipated regret, reciprocity, and commitment (behavioral content).

MAIN OUTCOME MEASURES

Proportion of patients who completed HCV testing within four months.

RESULTS

21 303 patients were included in the intention-to-treat analysis. Among the 1642 patients in substudy A, 19.2% (95% confidence interval 16.5% to 21.9%) completed screening in the letter only arm and 43.1% (39.7% to 46.4%) in the letter+order arm (P<0.001). Among the 19 661 patients in substudy B, 14.6% (13.9% to 15.3%) completed screening with usual care content and 13.6% (13.0% to 14.3%) with behavioral science content (P=0.06). Among active patient portal users, 17.8% (16.0% to 19.5%) completed screening after receiving a letter and 13.8% (13.1% to 14.5%) after receiving a patient portal message (P<0.001).

CONCLUSIONS

Opt out framing and effort reduction by including a signed laboratory order with outreach increased screening for HCV. Behavioral science messaging content did not increase uptake, and mailed letters achieved a greater response rate than patient portal messages.

TRIAL REGISTRATION

ClinicalTrials.gov NCT03712553.

Phenotyping physician practice patterns and associations with response to a nudge in the electronic health record for influenza vaccination: A quasi-experimental study

BACKGROUND

Health systems routinely implement changes to the design of electronic health records (EHRs). Physician behavior may vary in response and methods to identify this variation could help to inform future interventions. The objective of this study was to phenotype primary care physician practice patterns and evaluate associations with response to an EHR nudge for influenza vaccination.

METHODS AND FINDINGS

During the 2016-2017 influenza season, 3 primary care practices at Penn Medicine implemented an active choice intervention in the EHR that prompted medical assistants to template influenza vaccination orders for physicians to review during the visit. We used latent class analysis to identify physician phenotypes based on 9 demographic, training, and practice pattern variables, which were obtained from the EHR and publicly available sources. A quasi-experimental approach was used to evaluate response to the intervention relative to control practices over time in each of the physician phenotype groups. For each physician latent class, a generalized linear model with logit link was fit to the binary outcome of influenza vaccination at the patient visit level. The sample comprised 45,410 patients with a mean (SD) age of 58.7 (16.3) years, 67.1% were white, and 22.1% were black. The sample comprised 56 physicians with mean (SD) of 24.6 (10.2) years of experience and 53.6% were male. The model segmented physicians into groups that had higher (n = 41) and lower (n = 15) clinical workloads. Physicians in the higher clinical workload group had a mean (SD) of 818.8 (429.1) patient encounters, 11.6 (4.7) patient appointments per day, and 4.0 (1.1) days per week in clinic. Physicians in the lower clinical workload group had a mean (SD) of 343.7 (129.0) patient encounters, 8.0 (2.8) patient appointments per day, and 3.1 (1.2) days per week in clinic. Among the higher clinical workload group, the EHR nudge was associated with a significant increase in influenza vaccination (adjusted difference-in-difference in percentage points, 7.9; 95% CI, 0.4-9.0; P = .01). Among the lower clinical workload group, the EHR nudge was not associated with a significant difference in influenza vaccination rates (adjusted difference-in-difference in percentage points, -1.0; 95% CI, -5.3-5.8; P = .90).

CONCLUSIONS

A model-based approach categorized physician practice patterns into higher and lower clinical workload groups. The higher clinical workload group was associated with a significant response to an EHR nudge for influenza vaccination.

Association of an Active Choice Intervention in the Electronic Health Record Directed to Medical Assistants With Clinician Ordering and Patient Completion of Breast and Colorectal Cancer Screening Tests

IMPORTANCE

Early cancer detection can lead to improved outcomes, but cancer screening tests are often underused.

OBJECTIVE

To evaluate the association of an active choice intervention in the electronic health record directed to medical assistants with changes in clinician ordering and patient completion of breast and colorectal cancer screening tests.

DESIGN, SETTING, AND PARTICIPANTS

A retrospective quality improvement study was conducted among 69 916 patients eligible for breast or colorectal cancer screening at 25 primary care practices at the University of Pennsylvania Health System between September 1, 2014, and August 31, 2017. Data analysis was conducted from January 21 to July 8, 2019.

INTERVENTIONS

From 2016 to 2017, 3 primary care practices at the University of Pennsylvania Health System implemented an active choice intervention in the electronic health record that prompted medical assistants to inform patients about cancer screening during check-in and template orders for clinicians to review during the visit.

MAIN OUTCOMES AND MEASURES

The primary outcome was clinician ordering of cancer screening tests. The secondary outcome was patient completion of cancer screening tests within 1 year of the primary care visit.

RESULTS

The sample eligible for breast cancer screening comprised 26 269 women with a mean (SD) age of 60.4 (6.9) years; 15 873 (60.4%) were white and 7715 (29.4%) were black. The sample eligible for colorectal cancer screening comprised 43 647 patients with a mean (SD) age of 59.4 (7.5) years; 24 416 (55.9%) were women, 19 231 (44.1%) were men, 29 029 (66.5%) were white, and 9589 (22.0%) were black. For breast cancer screening, the intervention was associated with a significant increase in clinician ordering of tests (22.2 percentage points; 95% CI, 17.2-27.6 percentage points; P < .001) but no change in patient completion (0.1 percentage points; 95% CI, -4.0 to 4.3 percentage points; P = .45). For colorectal cancer screening, the intervention was associated with a significant increase in clinician ordering of tests (13.7 percentage points; 95% CI, 8.0-18.9 percentage points; P < .001) but no change in patient completion (1.0 percentage points; 95% CI, -3.2 to 4.6 percentage points; P = .36).

CONCLUSIONS AND RELEVANCE

An active choice intervention in the electronic health record directed to medical assistants was associated with a significant increase in clinician ordering of breast and colorectal cancer screening tests. However, it was not associated with a significant change in patient completion of either cancer screening test during a 1-year follow-up.

Seeking Consensus on the Terminology of Value-Based Transformation Through use of a Delphi Process

Collaboration among diverse stakeholders involved in the value transformation of health care requires consistent use of terminology. The objective of this study was to reach consensus definitions for the terms value-based care, value-based payment, and population health. A modified Delphi process was conducted from February 2017 to July 2017. An in-person panel meeting was followed by 3 rounds of surveys. Panelists anonymously rated individual components of definitions and full definitions on a 9-point Likert scale. Definitions were modified in an iterative process based on results of each survey round. Participants were a panel of 18 national leaders representing population health, health care delivery, academic medicine, payers, patient advocacy, and health care foundations. Main measures were survey ratings of definition components and definitions. At the conclusion of round 3, consensus was reached on the following definition for value-based payment, with 13 of 18 panelists (72.2%) assigning a high rating (7– 9) and 1 of 18 (5.6%) assigning a low rating (1–3): “Value-based payment aligns reimbursement with achievement of value-based care (health outcomes/cost) in a defined population with providers held accountable for achieving financial goals and health outcomes. Value-based payment encourages optimal care delivery, including coordination across healthcare disciplines and between the health care system and community resources, to improve health outcomes, for both individuals and populations.” The iterative process elucidated specific areas of agreement and disagreement for value-based care and population health but did not reach consensus. Policy makers cannot assume uniform interpretation of other concepts underlying health care reform efforts.

Association of Primary Care Clinic Appointment Time With Clinician Ordering and Patient Completion of Breast and Colorectal Cancer Screening

IMPORTANCE

As the clinic day progresses, clinicians may fall behind schedule and experience decision fatigue. However, the association of time of day with cancer screening rates is unknown.

OBJECTIVE

To evaluate the association of primary care clinic appointment time with clinician ordering and patient completion of breast and colorectal cancer screening.

DESIGN, SETTING, AND PARTICIPANTS

Retrospective, quality improvement study of 33 primary care practices in Pennsylvania and New Jersey from September 1, 2014, to August 31, 2016. Participants included adults eligible for breast or colorectal cancer screening. Data analysis was conducted from April 24, 2018, to November 8, 2018.

EXPOSURES

Clinic appointment time during each patient's first primary care physician visit in the study period.

MAIN OUTCOMES AND MEASURES

Primary outcome was clinician ordering of the screening test during the visit. Secondary outcome was patient completion of the tests within 1 year of the visit.

RESULTS

Among the 19 254 patients eligible for breast cancer screening, the mean (SD) age was 60.2 (6.9) years; 19 254 (100%) were female, 11 682 (60.7%) were white, and 5495 (28.5%) were black. Screening test order rates were highest at 8 am at 63.7%, decreased throughout the morning to 48.7% at 11 am, increased to 56.2% at noon, and then decreased to 47.8% at 5 pm (adjusted odds ratio [OR] for overall trend, 0.94; 95% CI, 0.93-0.96; P < .001). Trends in screening test completion rates were similar beginning at 33.2% at 8 am and decreasing to 17.8% at 5 pm (adjusted OR, 0.95; 95% CI, 0.94-0.97; P < .001). Among the 33 468 patients eligible for colorectal cancer screening, the mean (SD) age was 59.6 (7.4) years; 18 672 (55.8%) were female, 22 157 (66.2%) were white, and 7296 (21.8%) were black. Screening test order rates were 36.5% at 8 am, decreased to 31.3% by 11 am, increased at noon to 34.4%, and then decreased to 23.4% at 5 pm (adjusted OR, 0.94; 95% CI, 0.93-0.95; P < .001). Trends in screening test completion rates were similar beginning at 28.0% at 8 am and decreasing to 17.8% at 5 pm (adjusted OR, 0.97; 95% CI, 0.96-0.98; P < .001).

CONCLUSIONS AND RELEVANCE

Clinician ordering of cancer screening tests significantly decreased as the clinic day progressed. Patient completion of cancer screening tests within 1 year of the visit was also lower as the primary care appointment time was later in the day. Future interventions targeting improvements in cancer screening should consider how time of day may influence these behaviors.

Social Incentives and Gamification to Promote Weight Loss: The LOSE IT Randomized, Controlled Trial

BACKGROUND

Social networks influence obesity patterns, but interventions to leverage social incentives to promote weight loss have not been well evaluated.

OBJECTIVE

To test the effectiveness of gamification interventions designed using insights from behavioral economics to enhance social incentives to promote weight loss.

DESIGN

The Leveraging Our Social Experiences and Incentives Trial (LOSE IT) was a 36-week randomized, controlled trial with a 24-week intervention and 12-week follow-up.

PARTICIPANTS

One hundred and ninety-six obese adults (body mass index ≥ 30) comprising 98 two-person teams.

INTERVENTIONS

All participants received a wireless weight scale, used smartphones to track daily step counts, formed two-person teams with a family member or friend, and selected a weight loss goal. Teams were randomly assigned to control or one of two gamification interventions for 36 weeks that used points and levels to enhance collaborative social incentives. One of the gamification arms also had weight and step data shared regularly with each participant's primary care physician (PCP).

MAIN OUTCOME MEASURES

The primary outcome was weight loss at 24 weeks. Secondary outcomes included weight loss at 36 weeks.

KEY RESULTS

At 24 weeks, participants lost significant weight from baseline in the control arm (mean: - 3.9 lbs; 95% CI: - 6.1 to - 1.7; P < 0.001), the gamification arm (mean: - 6.6 lbs; 95% CI: - 9.4 to - 3.9; P < 0.001), and the gamification arm with PCP data sharing (mean: - 4.8 lbs; 95% CI: - 7.4 to - 2.3; P < 0.001). At 36 weeks, weight loss from baseline remained significant in the control arm (mean: - 3.5 lbs; 95% CI: - 6.1 to - 0.8; P = 0.01), the gamification arm (mean: - 6.3 lbs; 95% CI: - 9.2 to - 3.3; P < 0.001), and the gamification arm with PCP data sharing (mean: - 5.2 lbs; 95% CI: - 8.5 to - 2.0; P < 0.01). However, in the main adjusted model, there were no significant differences in weight loss between each of the intervention arms and control at either 12, 24, or 36 weeks.

CONCLUSIONS

Using digital health devices to track behavior with a partner led to significant weight loss through 36 weeks, but the gamification interventions were not effective at promoting weight loss when compared to control.

TRIAL REGISTRATION

clinicaltrials.gov Identifier: 02564445.

Variations in Influenza Vaccination by Clinic Appointment Time and an Active Choice Intervention in the Electronic Health Record to Increase Influenza Vaccination

IMPORTANCE

Influenza vaccination rates in the United States are suboptimal near 40%, but little is known about variations in care based on clinic appointment time.

OBJECTIVES

To compare differences in influenza vaccination rates by clinic appointment time and to evaluate the association of an active choice intervention in the electronic health record with changes in vaccination rates.

DESIGN, SETTING, AND PARTICIPANTS

Retrospective, quality improvement study of 11 primary care practices at the University of Pennsylvania Health System from September 1, 2014, to March 31, 2017. Participants included adults eligible for influenza vaccination. Data analysis was conducted from October 20, 2017, to March 9, 2018.

INTERVENTIONS

During the 2016 to 2017 influenza season, 3 primary care practices at the University of Pennsylvania Health System implemented an active choice intervention in the electronic health record that prompted medical assistants to ask patients about influenza vaccination during check-in and template vaccination orders for clinicians to review during the visit.

MAIN OUTCOMES AND MEASURES

Influenza vaccination rates.

RESULTS

The sample comprised 96 291 patients with a mean (SD) age of 56.2 (17.0) years; 41 865 (43.5%) were men, 61 813 (64.2%) were white, and 23 802 (24.7%) were black. Among all practices across all 3 years, vaccination rates were approximately 44% from 8 am to 10 am, declined to 41.2% by 11 am and 38.3% at noon, increased to 40.2% at 1 pm, and then declined to 34.3% at 3 pm and 32.0% at 4 pm (P < .001 for adjusted linear trend). For the 3 years, vaccination rates were 46.9%, 47.2%, and 45.6% at control practices and 49.7%, 52.2%, and 59.3% at intervention practices, respectively. In adjusted analyses, compared with control practices over time, the active choice intervention was associated with a significant 9.5-percentage point increase in vaccination rates (95% CI, 4.1-14.3; P < .001). Vaccination rates increased similarly across times of the day.

CONCLUSIONS AND RELEVANCE

Influenza vaccination rates significantly declined as the clinic day progressed. The active choice intervention was associated with a significant increase in influenza vaccination rates that were similar in magnitude throughout the day.

Patient and physician predictors of hyperlipidemia screening and statin prescription

OBJECTIVES

Appropriate lipid management has been demonstrated to reduce cardiovascular events, but rates of hyperlipidemia screening and statin therapy are suboptimal. We aimed to evaluate patient and physician predictors of guideline-concordant hyperlipidemia screening and statin prescription.

STUDY DESIGN

Retrospective study of patients with primary care provider (PCP) visits from 2014 to 2016 at the University of Pennsylvania Health System.

METHODS

Data on patients, screening orders, and prescriptions were obtained from the electronic health record. Multivariate logistic regression models were fit to binary outcomes of lipid screening and statin prescription.

RESULTS

Among 97,189 eligible patients, 79.9% had an order for hyperlipidemia screening. In adjusted models, significant patient predictors of greater odds of having screening ordered included a history of diabetes (odds ratio [OR], 1.19; 95% CI, 1.10-1.29; P <.001) or hypertension (OR, 1.16; 95% CI, 1.10-1.23; P <.001). Significant provider predictors of lower odds of having screening ordered were being a resident PCP (OR, 0.63; 95% CI, 0.43-0.93; P = .021) or being trained in family medicine (OR, 0.37; 95% CI, 0.30-0.47; P <.001). Among 40,845 eligible patients, 56.1% were prescribed a statin. In adjusted models, significant patient predictors of greater odds of being prescribed a statin were if they had a history of diabetes (OR, 2.70; 95% CI, 2.32-3.13; P <.001) or clinical cardiovascular disease (OR, 2.26; 95% CI, 1.85-2.76; P <.001). Significant provider predictors of lower odds of being prescribed a statin were being a physician assistant (OR, 0.65; 95% CI, 0.52-0.81; P <.001) or female (OR, 0.82; 95% CI, 0.70-0.96; P = .01).

CONCLUSIONS

Both patient and provider factors significantly predicted guideline-concordant care for hyperlipidemia screening and statin therapy.

Effect of an Automated Patient Dashboard Using Active Choice and Peer Comparison Performance Feedback to Physicians on Statin Prescribing: The PRESCRIBE Cluster Randomized Clinical Trial

IMPORTANCE

Statins are not prescribed to approximately 50% of patients who could benefit from them.

OBJECTIVE

To evaluate the effectiveness of an automated patient dashboard using active choice framing with and without peer comparison feedback on performance to nudge primary care physicians (PCPs) to increase guideline-concordant statin prescribing.

DESIGN, SETTING, AND PARTICIPANTS

This 3-arm cluster randomized clinical trial was conducted from February 21, 2017, to April 21, 2017, at 32 practice sites in Pennsylvania and New Jersey. Participants included 96 PCPs and 4774 patients not previously receiving statin therapy. Data were analyzed from April 25, 2017, to June 16, 2017.

INTERVENTIONS

Primary care physicians in the 2 intervention arms were emailed a link to an automated online dashboard listing their patients who met national guidelines for statin therapy but had not been prescribed this medication. The dashboard included relevant patient information, and for each patient, PCPs were asked to make an active choice to prescribe atorvastatin, 20 mg, once daily, atorvastatin at another dose, or another statin or not prescribe a statin and select a reason. The dashboard was available for 2 months. In 1 intervention arm, the email to PCPs also included feedback on their statin prescribing rate compared with their peers. Primary care physicians in the usual care group received no interventions.

MAIN OUTCOMES AND MEASURES

Statin prescription rates.

RESULTS

Patients had a mean (SD) age of 62.4 (8.3) years and a mean (SD) 10-year atherosclerotic cardiovascular disease risk score of 13.6 (8.2); 2625 (55.0%) were male, 3040 (63.7%) were white, and 1318 (27.6%) were black. In the active choice arm, 16 of 32 PCPs (50.0%) accessed the patient dashboard, but only 2 of 32 (6.3%) signed statin prescription orders. In the active choice with peer comparison arm, 12 of 32 PCPs (37.5%) accessed the patient dashboard and 8 of 32 (25.0%) signed statin prescription orders. Statins were prescribed in 40 of 1566 patients (2.6%) in the usual care arm, 116 of 1743 (6.7%) in the active choice arm, and 117 of 1465 (8.0%) in the active choice with peer comparison arm. In the main adjusted model, compared with usual care, there was a significant increase in statin prescribing in the active choice with peer comparison arm (adjusted difference in percentage points, 5.8; 95% CI, 0.9-13.5; P = .008), but not in the active choice arm (adjusted difference in percentage points, 4.1; 95% CI, -0.8 to 13.1; P = .11).

CONCLUSIONS AND RELEVANCE

An automated patient dashboard using both active choice framing and peer comparison feedback led to a modest but significant increase in guideline-concordant statin prescribing rates.

TRIAL REGISTRATION

ClinicalTrials.gov Identifier: NCT03021759.

The Use of Aggregate Scoring for a Recertifying Examination

Aggregate scoring derives weights for the responses to test questions that are the proportion of a criterion group of experts; examinees' test scores are simply the sum of the weights of the responses they choose. This study applied aggregate scoring to a recertifying examination where it is particularly useful. It is an efficient means of generating an answer key, it ensures that the answer key reflects differences in practice, and examinees may find it reassuring to be judged against the performance of their peers. Results indicated considerable agreement between the traditional answer key and the aggregate answer key. Although the scores produced by the two answer keys were similar, aggregate scoring slightly favored individuals out of training longer. Generalizability analyses (Brennan, 1983) produced the expected results. The use of several experts in aggregate scoring made a sizeable contribution to reduction in measurement error. The choice of either completely crossed designs or nested designs for collecting the responses of experts depends on the resources available.

Obtaining patient feedback at point of service using electronic kiosks

BACKGROUND

Engaging patients in their healthcare is a goal of healthcare reform. Obtaining sufficient, reliable patient feedback about their experiences in an office encounter has been a challenge.

OBJECTIVE

To determine the feasibility of collecting feedback from patients regarding their office encounter at the point of care using touch screen kiosk technology in an urban primary care clinic.

METHODS

We analyzed response rate, ease of use, provider data, and condition-specific data. The study was conducted over a 45-day period at 1 internal medicine academic teaching practice. Providers, staff, and a sponsor-supported monitor directed patients to use the kiosk after an office visit.

RESULTS

A total of 1923 surveys were completed from 3850 office visits (50%). There was no appreciable impact on office flow in terms of wait time, checkout procedures, or visit with provider. Characteristics of patients completing the surveys were similar to practice demographics of patients with an office visit during the study period in terms of sex, but differed by age and race. Small but statistically significant differences were seen among patient ratings of resident versus attending physicians. Patients with depression were less likely than patients with diabetes, chronic low back pain, or asthma to report that they had set personal goals to manage their condition.

CONCLUSION

This technology represents an important advance in our ability to capture the patient's opinion regarding quality and practice improvement initiatives, and has the potential for directly engaging patients in their care.

A primary care intervention for weight loss: results of a randomized controlled pilot study

Most primary care providers (PCPs), constrained by time and resources, cannot provide intensive behavioral counseling for obesity. This study evaluated the effect of using medical assistants (MAs) as weight loss counselors. The study was a randomized controlled trial conducted in two primary care offices at an academic medical center. Patients (n = 50) had a BMI of 27-50 kg/m(2) and no contraindications to weight loss. They were randomized to quarterly PCP visits and weight loss materials (Control group) or to the same approach combined with eight visits with a MA over 6 months (Brief Counseling). Outcomes included change in weight and cardiovascular risk factors (glucose, lipids, blood pressure, and waist circumference). Patients in the Brief Counseling and Control groups lost 4.4 +/- 0.6 kg (5.1 +/- 0.7% of initial weight) and 0.9 +/- 0.6 kg (1.0 +/- 0.7%), respectively, at month 6 (P < 0.001). There were no significant differences between groups for changes in cardiovascular risk factors. Brief Counseling patients regained weight between month 6 and month 12, when MA visits were discontinued. Attrition was 10% after 6 months and 6% after 12 months. Brief Counseling by MAs induced significant weight loss during 6 months. Office-based obesity treatment should be tested in larger trials and should include weight loss maintenance counseling.

Internal medicine-pediatrics combined residency graduates: what are they doing now? Results of a survey

BACKGROUND

While the number of internal medicine-pediatrics (med/peds) residency training programs has increased considerably in the past decade, questions continue to be raised about career paths of the graduates of these programs. It is uncertain whether med/peds graduates follow a generalist career path and whether they continue to practice both specialties.

OBJECTIVE

To determine the career outcomes of graduates of med/peds residency programs.

DESIGN

A survey questionnaire of graduates of med/peds residency programs.

METHODS

The computer databases of the American Board of Pediatrics and the American Board of Internal Medicine were used to identify 1482 individuals who had completed training in combined med/peds residency programs between 1986 and 1995 and who had applied to either board for certification. The survey questionnaire was mailed to all graduates identified.

MAIN OUTCOME MEASURES

Time spent in professional activity (patient care, teaching, administration, and research), site of principal clinical activity, ages of the patient population, types of hospital privileges, practice organization, subspecialty activity, night and weekend coverage arrangements, community size of practice, involvement in teaching, and membership in professional organizations.

RESULTS

Of the total group of 1482 graduates, 87.3% are certified by the American Board of Internal Medicine, 91.3% by the American Board of Pediatrics, and 81.6% by both boards. The survey was completed by 1005 graduates (67.8%). The principal activity of almost 70% of the graduates was direct patient care. Most graduates cared for patients of all ages. More than half of all respondents noted that their principal clinical site is a community office practice. Eighty-five percent managed patients who require hospitalization. Approximately 50% of respondents had a medical school appointment.

CONCLUSIONS

This study, the largest survey to date of med/peds graduates, provides strong evidence that most med/peds graduates are practicing generalists who care for adults and children. In addition, the fact that 80% of graduates achieve dual board certification suggests that these physicians are well qualified to care for the spectrum of health care needs of children and adults. Because the changing US health care system mandates a strong primary care base, these physicians will play a small but important role in providing high-quality generalist care.

Guidelines for interpretation of some common indicators of residency program performance

OBJECTIVE

To provide national norms for indicators of residency-training program quality and information on their reproducibility.

PARTICIPANTS

The 364 residency-training programs that had 4 or more candidates take the 1989 to 1991 certifying examination in internal medicine for the first time.

DESIGN

Within each residency, program directors' ratings of medical knowledge, certifying examination scores, and certification status (pass or fail) were available for each candidate from 1989 to 1991. Means of these data were calculated for each program for each year of the study. To provide a way of comparing an individual program with all other programs, percentiles are reported for each year. To assess the precision of the measures, generalizability theory was applied and confidence intervals for all data are reported for programs of various size (1 to 25 residents taking the examination) and over the years (1 to 3).

RESULTS

Over the 3 years of the study, knowledge ratings, certification rates, and composite scores declined slightly. The correlations between program ratings of medical knowledge and the composite scores ranged from .47 to .60 and certification rates ranged from .44 to .55. The confidence intervals around all of the program performance indicators are relatively large and are affected most by the number of residents in the program. There is little variability across the years.

CONCLUSIONS

In smaller programs the precision of the performance indicators is poor; in programs with only a few residents they are virtually meaningless. On the positive side, programs are relatively stable and aggregating indicators over years is a reasonable way to increase their precision in assessing program performance. If the goal of program evaluation is to identify programs at the extremes, especially those at the low end, then such data may help guide program directors and educators.

Certification in internal medicine: 1989-1992

OBJECTIVE

To determine whether changes in the demographic/educational mix of those entering internal medicine from 1986 to 1989 were associated with differences among them at the time of certification.

PARTICIPANTS

Included in the study were all candidates for the 1989 to 1992 American Board of Internal Medicine certifying examinations in internal medicine.

MEASUREMENTS

Demographic information and medical school, residency training, and examination experience were available for each candidate. Data defining quality, size, and number of subspecialties were available for internal medicine training programs.

RESULTS

From 1990 to 1992, the total number of men and women candidates increased as did the numbers of foreign-citizen non-U.S. medical school graduates and osteopathic medical school graduates; the number of U.S. medical school graduates remained nearly constant and the number of U.S.-citizen graduates of non-U.S. medical schools declined. The pass rates for all groups of first-time examination takers decreased, while the ratings of program directors remained relatively constant. Program quality, size, and number of subspecialty programs had modest positive relationships with examination performance.

CONCLUSIONS

Changes in the characteristics of those entering internal medicine from 1986 to 1989 were associated with declines in performance at the time of certification. These declines occurred in all content areas of the test and were apparent regardless of program quality. These data identify some of the challenges internal medicine faces in the years ahead.

Methods of preparing for the Certifying Examination in internal medicine and their efficacy

Candidates for the 1991 Certifying Examination were asked how they prepared for the examination. There were 2,780 respondents (32% of the eligible candidates). The responding candidates used a mean of 5.2 study methods and gave higher educational value ratings to methods used most frequently. Regression analyses showed no independent contribution of study method or effort to explaining the variance in score for first-time takers, and a 2% contribution for repeat takers. Program director ratings were the most important predictors of score for first-time takers and previous examination score for repeat takers. Intensive study is likely to produce at most a small improvement in performance.

A core component of the certification examination in internal medicine

OBJECTIVE

To develop and test the psychometric characteristics of an examination of core content in internal medicine.

DESIGN

A cross-sectional pilot test comparing the core examination with the 1988 certifying examination and two pretest examinations.

SETTING

The 1988 certifying examination of the American Board of Internal Medicine.

PARTICIPANTS

A random sample of 2,975 candidates from 8,968 candidates who took the 1988 certifying examination were given the core examination; similarly drawn samples were each given one of two pretests of traditional questions.

INTERVENTIONS

A framework for developing an examination of core internal medicine questions was designed and used to develop a 92-question core test with an absolute pass/fail standard.

RESULTS

Candidates answered 74% of core internal medicine questions, compared with 64%, 52%, and 53% of traditional questions on the 1988 certifying examination and the two pretests. The discriminating ability of the core internal medicine examination was lower than that of the certifying examination (r-values were 0.28 and 0.34, respectively). The pass rate was 83% for the core internal medicine examination and 57% for the certifying examination; 27% passed the core examination and failed the certifying examination; 1% passed the certifying examination and failed the core examination.

CONCLUSION

Core internal medicine questions were easier than but almost as discriminating as traditional questions of the certifying examination. A small percentage of candidates passed the certifying examination but failed the core examination.

The relevance to clinical practice of the certifying examination in internal medicine

OBJECTIVE

To determine the relevance of the initial certifying examination to the practice of internal medicine and the suitability of items used in initial certification for recertification.

DESIGN

Using a matrix-sampling approach, items from the 1991 Certifying Examination were assigned to two sets of judges: directors of the American Board of Internal Medicine (ABIM) and practicing general internists. Each judge rated the relevance of items on a five-point scale.

PARTICIPANTS

54 current or former directors of the ABIM and 72 practicing general internists; practitioners were nominated by directors and their ratings were included if they spent > 80% of their time in direct patient care.

RESULTS

The directors' mean rating of all 576 items was 3.98 (SD = 0.62); the practitioners' mean rating was 4.11 (SD = 0.82). The directors assigned to 27 items ratings of less than 3 and the practitioners assigned to 42 items ratings of less than 3; seven of these items received low ratings from both groups. There were differences in the two groups' ratings of the relevance of various medical content categories, but the mean rating of core items was higher than that of noncore items and the mean rating of items testing clinical judgment was higher than that of items testing knowledge or synthesis.

CONCLUSIONS

These findings suggest that the initial certifying examination is relevant to clinical practice and that many of the examination items are suitable for use in recertification. Differences in perception appear to exist between practitioners and directors, and the use of practitioner ratings is likely to be a routine part of judging the suitability of items for Board examinations in the future.

A longitudinal description of patterns of certification in internal medicine and the subspecialties

OBJECTIVE

To document the timings, frequencies, and outcomes of attempts at certification in internal medicine and the internal medicine subspecialties in the years following residency training for two cohorts of residents.

DESIGN

Residents who had completed residency training and had been admitted to an American Board of Internal Medicine (ABIM) certifying examination in 1982 or 1983 were tracked through the ABIM database for five years.

PARTICIPANTS

A total of 10,568 residents were studied. Of the cohort, 79% were men, 21% were women, 79% were graduates of U.S./Canadian medical schools (USMGs), and 21% were graduates of foreign medical schools (FMGs).

MAIN RESULTS

Ultimately, 85% of the residents achieved certification in internal medicine. Cumulative pass rates were 87% for men, 81% for women, 92% for USMGs, and 60% for FMGs; rates increased minimally after the second attempt. Most (87%) residents first attempted the internal medicine examination in the year in which training had been completed. Delaying the first examination was associated with lower pass rates. Half of the candidates who had passed the internal medicine examination attempted subspecialty certification. Over all nine subspecialty examinations, the two-cycle cumulative pass rate was 87%. Higher percentages of FMGs than of any other subgroup attempted subspecialty certification.

CONCLUSIONS

The detailed description extends the body of knowledge about certification in internal medicine and the nine internal medicine subspecialties. Questions are raised, such as why some candidates delay the first internal medicine examination and why some residents never seek certification. Future research could explore these issues as well as explanations for the observed differences in pass rates.

Long-term evaluation of cardiac function in obese patients treated with a very-low-calorie diet: a controlled clinical study of patients without underlying cardiac disease

Twenty obese women were randomly assigned to consume (for 16 wk) either a 420-kcal/d liquid diet (n = 12) or a 1200-kcal/d balanced diet (n = 8). Thereafter, patients in both conditions were prescribed a 1200-kcal/d diet for the remainder of treatment (week 45). Six obese nondieters served as control subjects. Ambulatory electrocardiographic (Holter) monitor readings were obtained on all patients at baseline and weeks 3, 9, 13, 17, 19, and 45 of the study and were analyzed for ventricular premature depolarizations (VPDs) per hour, paired forms, and runs of ventricular tachycardia. There were no statistically significant changes in VPDs in any condition during treatment. Similarly, there were no significant changes in the PR, QRS, and corrected QT intervals. The results indicate that under appropriate medical supervision, very-low-calorie diets can be used safely for up to 16 wk by significantly obese patients free of pre-existing cardiac disease.

Residents' perception of evaluation procedures used by their training program

OBJECTIVE

To determine the methods of evaluation used routinely by training programs and to obtain information concerning the frequencies with which various evaluation methods were used.

DESIGN

Survey of residents who had recently completed internal medicine training.

PARTICIPANTS

5,693 respondents who completed residencies in 1987 and 1988 and were registered as first-time takers for the 1988 Certifying Examination in Internal Medicine. This constituted a 76% response rate.

MAIN RESULTS

Virtually all residents were aware that routine evaluations were submitted on inpatient rotations, but were more uncertain about the evaluation process in the outpatient setting and the methods used to assess their humanistic qualities. Most residents had undergone a Clinical Evaluation Exercise (CEX); residents' clinical skills were less likely to be evaluated by direct observation of history or physical examination skills. Resident responses were aggregated within training programs to determine the pattern of evaluation across programs. The majority of programs used Advanced Cardiac Life Support (ACLS) certification, medical record audit, and the national In-Training Examination to assess most of their residents. Performance-based tests were used selectively by a third or more of the programs. Breast and pelvic examination skills and ability to perform sigmoidoscopy were thought not to be adequately assessed by the majority of residents in almost half of the programs.

CONCLUSIONS

While most residents are receiving routine evaluation, including a CEX, increased efforts to educate residents about their evaluation system, to strengthen evaluation in the outpatient setting, and to evaluate certain procedural skills are recommended.

A controlled trial to improve delivery of preventive care: physician or patient reminders?

OBJECTIVE

To improve the delivery of preventive care in a medical clinic, a controlled trial was conducted of two interventions that were expected to influence delivery of preventive services differently, depending on level of initiative required of the physician or patient to complete a service.

DESIGN

A prospective, controlled trial of five-months' duration.

SETTING

A university hospital-based, general medical clinic.

PARTICIPANTS

Thirty-nine junior and senior medical residents who saw patients in stable clinic teams throughout the study.

INTERVENTION

A computerized reminder system for physicians and a patient questionnaire and educational hand-out on preventive care.

MEASUREMENTS AND MAIN RESULTS

Delivery of five of six audited preventive services improved significantly after the interventions were introduced. The computerized reminder alone increased completion rates of services that relied primarily on physician initiative; the questionnaire alone increased completion rate of the service that depended more on patient compliance as well as on some physician-dependent services. Both interventions used together were slightly less effective in improving performance of physician-dependent services than the computerized reminder used alone.

CONCLUSIONS

These interventions can improve the delivery of preventive care but they differ in their impacts on physician and patient behaviors. Overall, the computer reminder was the more effective intervention.

Gender differences in the clinical competence of residents in internal medicine

OBJECTIVE

To study the differences between cognitive and noncognitive skills of men and those of women entering internal medicine.

DESIGN

Comparison of program directors' ratings of overall clinical competence and its specific components and pass rates for men and women taking the Certifying Examinations in Internal Medicine in 1984-1987.

PARTICIPANTS

14,340 U.S. and Canadian graduates taking the Certifying Examinations of the American Board of Internal Medicine for the first time in 1984-1987.

MEASUREMENTS/RESULTS

Average program directors' ratings of overall competence were 6.70-6.78 for men and 6.60-6.71 for women. The greatest differences in ratings of specific components of competence were in the areas of medical knowledge and procedural skills, where men were rated higher than women, and humanistic qualities, where women were rated higher than men. Pass rates were stable over the four years of the study, and ranged from 85 to 86% for men and from 79 to 81% for women. Men consistently performed slightly better than women regardless of the type of residency or quality of medical school attended.

CONCLUSIONS

Small but consistent differences were found in the performances of men and those of women completing training in Internal Medicine as measured by program directors' ratings and ABIM Certifying Examination performances.

Long-term survival after transient loss of consciousness

To determine the factors that influenced the long-term outcome of 198 patients who presented to the emergency ward with transient loss of consciousness, the authors followed them for a median of 83 months. Forty-one patients (21%) died, including nine patients who had out-of-hospital sudden cardiac arrest. Compared with age- and sex-adjusted mortality rates for the United States, long-term mortality was not increased among patients with benign causes of syncope. Multivariate analysis revealed that the long-term mortality rate was significantly increased in patients with a prior history of coronary or cerbrovascular disease (RR = 6.7), those who had cancer (RR = 7.3), and those who had drug or metabolic (RR = 5.9), central nervous system (RR = 5.7) or cardiac (RR = 9.2) causes of transient loss of consiousness. Among patients who experience transient losses of consciousness, the cause of the episode is significantly correlated with mortality for at least the next seven years.

Less food, less hunger: reports of appetite and symptoms in a controlled study of a protein-sparing modified fast

This study compared reports of appetite and symptoms in 28 obese subjects randomly assigned to either a 500 calorie protein-sparing modified fast (PSMF) or a 1200-kcal balanced diet. During the first comparison month, subjects consuming the PSMF lost significantly more weight and reported significantly less hunger than did subjects consuming the 1200 kcal diet. Similar results were obtained for the second month, but differences in hunger were not statistically significant. There were no significant differences between conditions in subjects' ratings of their preoccupation with eating or in their ratings of the acceptability or disruptiveness of their diets. PSMF subjects reported significantly greater problems with cold intolerance, constipation, dizziness, dry skin, and fatigue. These symptoms remitted completely, however, when PSMF subjects consumed a 1200-kcal balanced diet. There were no significant differences between conditions in subjects' reports of psychological functioning. Results are discussed in terms of the need for further research to identify the characteristics of PSMF which confer anorexia.

A comparison of two very-low-calorie diets: protein-sparing-modified fast versus protein-formula-liquid diet

This study investigated the acceptability of two very-low-calorie diets in 16 moderately overweight persons participating in a weight reduction program. Subjects were prescribed a 1000-1200 kcal balanced diet the first month and asked to complete appetite and mood scales on a weekly basis. They were then randomly assigned to either a protein-sparing-modified fast (PSMF) or a protein-formula-liquid diet, each of which provided about 400 kcal daily. Analysis of the appetite data showed that PSMF subjects reported significantly less hunger and preoccupation with eating than did liquid diet subjects during 2 of the 4 weeks on a very-low-calorie diet. Subjects in both conditions reported significant reductions in anxiety. Results are discussed in terms of possible advantages of PSMF.

Evaluation and outcome of emergency room patients with transient loss of consciousness

We identified 198 patients who presented to our emergency room with transient loss of consciousness. Seizures (29 percent of patients) and vasovagal/psychogenic episodes (40 percent of patients) were the most common presumptive causes of loss of consciousness, but the cause of loss of consciousness remained uncertain even at follow-up in 11 +/- 6 months in 13 percent of the patients. The history and physical examinations were sufficient for diagnosis in 85 percent of the patients in whom a diagnosis could be established. These data guided inpatient and outpatient with potentially dangerous causes of loss of consciousness except for one patient who had pulmonary embolism. In selected patient, diagnostic tests such as blood chemistries (three patients), electrocardiograms (four patients) electroencephalograms (three patients), and Holter monitoring (four patients) provided crucial information, and CT scans identified new brain tumors in four patients with focal neurologic presentations. At the time of follow-up, 7.5 percent of patients had suffered either major morbidity or death related to the cause of the index episode of loss of consciousness. Patients with cardiac causes represented a high risk (33 percent) group for such poor outcome, whereas patients who were under age 30, or who were under age 70 and had loss of consciousness on a vasovagal/psychogenic or unknown basis, constituted a low risk (1 percent) subgroup.