The international medical graduate pipeline

This paper presents information from the Educational Commission for Foreign Medical Graduates (ECFMG) on the certification and practice of international medical graduates (IMGs). IMGs constitute 25 percent of the U.S. physician workforce, and there is a vigorous pipeline of highly qualified candidates seeking graduate training in the United States. When considering how to address the potential physician shortfall described by Richard Cooper, policymakers will need to consider U.S. health care needs in the context of the intense international debate regarding global physician migration and its implications for the developing world.

Evaluating the spoken English proficiency of international medical graduates: detecting threats to the validity of standardised patient ratings

PURPOSE

To investigate potential threats to the validity of the spoken English proficiency ratings provided by standardised patients (SPs) in high-stakes clinical skills examinations.

METHOD

Spoken English ratings from 43 327 patient encounters were studied. These involved over 5000 candidates, 40% of whom were female and 33% of whom self-reported English to be their native language. Over 100 SPs were involved in the study, 51% of whom were female and 90% of whom were native English speakers. Possible performance differences in English ratings were studied as a function of candidate and SP gender, and as a function of candidate and SP native language (English versus all other languages).

RESULTS

No significant candidate by SP gender effect was detected. There were no meaningful differences in mean English ratings as a function of SP or candidate gender. Likewise, English ratings did not vary as a function of either candidate or SP native language. While candidate mean English ratings were not associated with the native language of the SP, native English-speaking candidates did achieve significantly higher ratings.

DISCUSSION

The lack of significant interaction between candidate and SP gender, and candidate and SP native language, suggests that the SPs provided unbiased English ratings. These results, combined with the expected higher English ratings given to candidates with English-speaking backgrounds, provides additional evidence to support the validity and fairness of spoken English proficiency ratings provided by standardised patients.

Recruits from overseas need better assessment and support

The decision of David Nunn, a consultant orthopaedic surgeon, to stop a patient’s operation half way through because some of the nurses could not follow his instructions has raised much debate about the ability of foreign healthcare professionals to communicate effectively and the consequences this might have on patient care. This is not a new problem as the NHS has periodically recruited internationally to meet the demand for doctors and nurses. However, the problem has been magnified in recent years because of massive increases in targeted overseas recruitment.

A pre-employment programme for overseas-trained doctors entering the Australian workforce, 1997-99

OBJECTIVES

Overseas-trained doctors (OTDs) have limited access and formal interaction with the Australian health care system prior to joining the Australian medical workforce. A pre-employment programme was designed to familiarize OTDs with the Australian health care system.

METHOD

All OTDs who had passed their Australian Medical Council (AMC) exams and were applying for a pre-registration year in New South Wales were invited to participate in the voluntary, free programme. A 4-week full-time programme was developed consisting of core group teaching and a hospital attachment. The curriculum included communication, health and workplace skills; and sessions on culture shock and the role of junior doctors. A pilot programme was run in 1997. The programme was repeated in 1998 and 1999. The OTDs' confidence regarding the general duties of internship, and attitudes towards hospital workplace skills were examined.

RESULTS

The 66 OTDs reported greater understanding of staff and communication issues and familiarization with the hospital environment. They reported a more realistic understanding of the role of a junior doctor, the need for separation of workplace and personal responsibilities and knowledge of pathways for future professional development. The course structure, with a focus on hospital attachments, establishment of a peer network, and workplace familiarization facilitated entry into the hospital workforce.

CONCLUSION

The pre-employment programme enabled the OTDs to have a more equitable entry into the public hospital system, resulting in a more integrated, confident and functional workforce.

Exclusion of international medical graduates from federal health-care programs

BACKGROUND

The professional standards of international medical graduates have been the subject of controversy, but empirical research on this topic has been limited.

OBJECTIVES

This report considers whether international medical graduates are at greater risk than US medical graduates for exclusion by the federal government from federally funded programs, such as Medicare and Medicaid.

RESEARCH DESIGN

The list of excluded physicians was merged with data regarding 87,729 family and general practice physicians from the American Medical Association Physician Masterfile, 555 of whom were currently excluded. Logistic regression was used to estimate the effect of international medical graduate status on the probability of exclusion, controlling for board-certification status and other physician characteristics. International medical graduates from high-income Organization for Economic Cooperation and Development (OECD) countries are distinguished from other international medical graduates.

RESULTS

The adjusted exclusion rates of international medical graduates from OECD countries were similar to that of US medical graduates. Among board-certified physicians, the relative risk of exclusion of non-OECD international medical graduates was 2.19 (P <0.001) compared with US medical graduates. Board certification had an even stronger association: US medical graduates who had never been board certified had a relative risk of 4.12 (P <0.001) compared with board-certified US medical graduates. The never board-certified relative risk was 1.72 (P <0.001) among non-OECD international medical graduates compared with board-certified graduates. Among physicians who had never been board certified, rates of US and international medical graduates did not differ substantially.

CONCLUSIONS

Further investigation is needed regarding the causal determinants of exclusion disparities. It is unclear to what extent these disparities may reflect differences in ethical conduct, quality of care, or prejudicial enforcement practices, and the extent to which board certification can causally reduce actions leading to exclusion.

Assessing the comparability of standardized patient and physician evaluations of clinical skills

Accumulating evidence to defend decisions based on scores from evaluations is an ongoing process. The purpose of this investigation was to gather additional data to support the validity of inferences made from scores on the Educational Commission for Foreign Medical Graduates' Clinical Skills Assessment (CSA). This was accomplished by contrasting CSA candidate scores, and pass/fail decisions, with those obtained from the American Board of Internal Medicine's Mini-CEX (Clinical Evaluation Exercise). Data gathering performance based on the number of unweighted history taking and physical examination checklist items adequately predicted the global ratings provided by physician observers. CSA ratings of doctor-patient communication skills correlated with mini-CEX ratings of like constructs, indicating that physician observers, using mini-CEX rating scales, are able to make realistic assessments of interpersonal skills. These results provide evidence of the convergent validity of CSA scores.

Early medical registration in Australia. Part 2

The lure of land and gold, the hope for better health and an overcrowded profession in Britain led to an increasing number of doctors migrating to Australia. This migration was even more marked after the goldrush, especially in Victoria. The fifth article in this series looks at the way in which medical practice was controlled by excluding many of those doctors who were from non-British origins.

Workforce planning. Of one mind

The NHS is heavily reliant on locum doctors, many of whom trained overseas. It is important they are appropriately trained so they can be integrated into the workforce. A pilot nine-session course for consultant locum psychiatrists was well received by participants. Medical directors have expressed willingness to pay for locums to attend such courses.

Experience with training in a foreign country

Everyone is interested in going to a foreign country during or after trainingship. But going abroad will have an enormous impact on personal and professional life. We will reveal some tips and tricks about the USA, UK, France and The Netherlands.

Validation of the doctor-patient communication component of the Educational Commission for Foreign Medical Graduates Clinical Skills Assessment

BACKGROUND

The pivotal role of doctor-patient communication in effective health care delivery led the Educational Commission for Foreign Medical Graduates (ECFMG) to incorporate the assessment of interpersonal skills and spoken English proficiency into its Clinical Skills Assessment (CSA). Furthermore, it was decided that to pass the CSA, a candidate would need to meet or surpass defined performance standards for doctor-patient communication as a discrete component. This requirement, among others, is designed to ensure the readiness of graduates of foreign medical schools (FMGs) to enter postgraduate medical education programmes in the United States.

OBJECTIVE

The primary focus of this study was to determine the extent to which performance in a simulated testing environment is related to performance in the clinical setting.

METHOD

Nurses were trained to rate the communication skills of residents from the patient's perspective. A total of 43 first-year residents were evaluated. The survey ratings (n=225) were compared with the residents' CSA communication scores.

RESULTS

Corrected correlations between CSA ratings and those obtained from nurses ranged from 0.61 to 0.73.

CONCLUSION

This study provides evidence for the validity of the communication ratings provided by standardized patients. The reasonably strong associations between ratings obtained during testing and those obtained through observation of 'real' patient interactions suggest that external observers can provide accurate evaluations of doctor-patient communication.

Evaluating the spoken English proficiency of graduates of foreign medical schools

PURPOSE

The purpose of this study was to gather additional evidence for the validity and reliability of spoken English proficiency ratings provided by trained standardized patients (SPs) in high-stakes clinical skills examination.

METHOD

Over 2500 candidates who took the Educational Commission for Foreign Medical Graduates' (ECFMG) Clinical Skills Assessment (CSA) were studied. The CSA consists of 10 or 11 timed clinical encounters. Standardized patients evaluate spoken English proficiency and interpersonal skills in every encounter. Generalizability theory was used to estimate the consistency of spoken English ratings. Validity coefficients were calculated by correlating summary English ratings with CSA scores and other external criterion measures. Mean spoken English ratings were also compared by various candidate background variables.

RESULTS

The reliability of the spoken English ratings, based on 10 independent evaluations, was high. The magnitudes of the associated variance components indicated that the evaluation of a candidate's spoken English proficiency is unlikely to be affected by the choice of cases or SPs used in a given assessment. Proficiency in spoken English was related to native language (English versus other) and scores from the Test of English as a Foreign Language (TOEFL).

DISCUSSION

The pattern of the relationships, both within assessment components and with external criterion measures, suggests that valid measures of spoken English proficiency are obtained. This result, combined with the high reproducibility of the ratings over encounters and SPs, supports the use of trained SPs to measure spoken English skills in a simulated medical environment.

Assessment of English proficiency in international medical graduates by physician examiners and standardized patients

CONTEXT

Since 1986, the Ontario Ministry of Health has provided a medical licensure preparation programme for international medical graduates. Because of the diversity in candidates' oral English proficiency, this competency has been viewed as a particularly important selection criterion.

OBJECTIVES

To assess and compare the quality of ratings of oral English proficiency of international medical graduates provided by physician examiners and by standardized patients (SPs). PARTICIPANTS AND MATERIALS: The study samples consisted of 73 candidates for the Ontario International Medical Graduate (IMG) Program, and physician examiners and SPs in five 10-minute encounter objective structured clinical examination (OSCE) stations. Materials used were a seven-item speaking performance rating instrument prepared for the Ontario IMG Program.

METHODS

Rating sheets were scanned and the results analysed using SPSS 9.0 for Windows.

RESULTS

Correlations between the physician and SP ratings on the seven items ranged from 0.52 to 0.70. The SPs provided more lenient ratings. Mean alpha reliability for the physicians' ratings on the seven items was 0.59, and for the SPs' 0.64. There was poor agreement between the two sets of raters in identifying problematic candidates.

CONCLUSIONS

Notwithstanding the sizable correlations between the ratings provided by the two rater groups, the results demonstrated that there was little agreement between the two groups in identifying the potentially problematic candidates. The physicians were less prone than the SPs to rate candidates as problematic. SPs may be better placed than the physician examiners to directly assess IMG candidates' oral English proficiency.

General Medical Council. A clear case of

A review of how the GMC handles complaints against doctors found shortcomings in consistency and transparency of decision-making. The lack of a common definition of what constitutes serious professional misconduct is a major problem. The review found no evidence of discrimination or racial bias in complaints handling. Protocols should be developed for dealing with complaints alleging poor treatment, which account for some 70 per cent of complaints received by the GMC. The whole process of complaints handling should be subject to continual audit, and written records should be kept of all decisions taken.

The management of patient encounter time in a high-stakes assessment using standardized patients

OBJECTIVES

The purpose of this study was to gather information regarding the appropriateness of the length of time allotted for candidates to complete the history taking and physical examination tasks in a high-stakes standardized patient (SP) assessment.

DESIGN

Data were collected on actual time used by 1548 examinees for each of their 10 standardized patient encounters, for which a maximum of 15 minutes was allotted, but not required.

SETTING

The Clinical Skills Assessment Center of the Educational Commission for Foreign Medical Graduates (ECFMG), Philadelphia, Pennsylvania, USA.

SUBJECTS

Graduates of foreign medical schools who are seeking ECFMG certification.

RESULTS

The average time spent with the standardized patient was 13.3 minutes, suggesting that the 15-minute time limit was sufficient. A positive correlation was found between data-gathering scores and patient interview times. Candidates did tend to spend more time with SPs presenting with cases involving complex histories, as well as with cases of chronic conditions.

CONCLUSIONS

Candidate time use varied as a function of type of clinical encounter, providing additional evidence of the content validity of the Clinical Skills Assessment.

[Assessment of clinical competence of medical school graduates in Italy with standardized patients. The opinion of the examinees]

An international collaborative project for the evaluation of clinical competence at the end of the Medical School curriculum using the ECFMG-CSA (Educational Commission for Foreign Medical Graduates--Clinical Skills Assessment) prototype was started in Italy in April 1996. Faculty representatives from Italian Medical Schools and experts from the ECFMG in Philadelphia participated in the Project. The CSA consists of integrated clinical encounters with 10 standardized patients during which the examinee is asked to obtain a focused history, perform a relevant physical examination and communicate initial diagnoses and management plan to the Standardized Patient (SP). The SP then completes checklists that are scored by Faculty members. The project was concluded in Spring 1998 and a total of 173 new graduates were examined. The data elaborated by the primary site in Chieti University will be available in the Fall 1998 by the ECFMG in Philadelphia. This preliminary communication reports the opinions of the examinees on the ECFMG-CSA, contained in the questionnaires administered after the test. Most of the examinees considered this new methodology as a valid tool for the assessment of clinical competence, especially history-taking and interpersonal skills and stated that the SP simulations were realistic. The 72% of examinees indicated that the Medical School curriculum does not adequately prepare for the CSA examination. Lastly, 68% was in favour of including the SP in the Medical Licensing Examination.

The quality of care of international medical graduates: how does it compare to that of U.S. medical graduates?

This article reviews the literature on the quality of care provided by foreign-trained physicians (international medical graduates or IMGs) compared with that of U.S. medical graduates (USMGs). As concerns are raised about IMGs in the U.S. physician workforce, there are suggestions that IMGs do not deliver care equal in quality to that of USMGs. The review of process and outcome studies finds little support for this claim. However, lower IMG levels of performance on structural measures of quality like credentialing examinations exist and may indicate quality differences. Because no consistent evidence exists that there is a connection between IMG test scores and process or outcome measures of quality of care, whether test scores matter in clinical practice and its outcome is uncertain. Until research shows the contrary, one should be cautious in accepting IMG-USMG quality arguments to support policy to reduce the size of the IMG component of the physician workforce.

Storm of protest greets motion to restrict specialty exams

A motion by the Royal College of Physicians and Surgeons of Canada to restrict specialty examinations to Canadian- and US-trained physicians has met a storm of protest from provinces facing acute physician shortages in remote areas. The motion was called a "recipe for disaster" for provinces such as Saskatchewan, Manitoba and Newfoundland that traditionally have recruited specialists from Commonwealth countries to fill positions. The protests prompted the college to postpone action on its motion until June.

Senior house officer training: are overseas graduates treated differently?

In a survey of senior house officers' (SHOs') satisfaction with training, overseas graduates rated their posts at least as highly as British graduates. Overseas graduates were older, more experienced and more likely to be training in district general hospitals.

Another go at the experiment

A number of states have experimented with legislation that would allow unlicensed international medical graduates to become physician assistants. These attempts have failed. The authors conducted a pilot evaluation study in California in response to legislative efforts. They examined the medical knowledge of a group of unlicensed international medical graduates and compared their clinical skills with those of a control group of recent graduates from a physician assistant training program. The unlicensed international medical graduates were below standard in medical knowledge and clinical skills and failed to make critical diagnoses in the tests with standardized patients. The authors conclude that unlicensed international medical graduates need additional training to be incorporated into the U.S. health care system as physician assistants.