The objective structured public health examination: a study of reliability using multi-level analysis

BACKGROUND

Introduced in 2006, the objective structured public health examination (OSPHE) is a part of the examination for the UK Membership of the Faculty of Public Health. Designed to simulate real work in public health, the same five generic competencies were assessed simultaneously in each of six stations in 1 h 48 min. This study estimates the examination reliability of the OSPHE.

METHODS

Data from all 198 candidates, 54 questions, 39 examiners over 9 test days in 2006 were analysed for reliability and sources of error using multilevel cross-classified analysis and generalisability theory.

RESULTS

A reliability co-efficient of 0.85 was estimated (95% confidence interval 0.82-0.88). Little variance was accounted for by questions or examiners. Twelve questions would increase reliability to 0.92. More than one examiner per question has negligible impact on reliability.

CONCLUSION

Multilevel modelling estimates examination reliability which is more rigorous than Cronbach's alpha. Traditional generalisability methods based on the analysis of variance (ANOVA) are unable to do this. The reliability of this examination compares favourably with the literature despite having just six test stations and under 2 h per candidate. Other specialties could consider the use of parallel testing of the same key competencies at all stations within simulated workplace events.

Sustained continuing education: how a masters programme makes a difference to primary care dentists

Courses that are part of a longer-term planned programme of continuing education are more likely to impact on the practice of dentists than short course interventions.This study assessed the impact of a part-time, five-year Master of Science (MSc) programme at the School of Dentistry, University of Birmingham, designed specifically for general dental practitioners. Perceptions were sought from all 22 dentists registered on the MSc at the time of the study. The study provides evidence that a well-targeted masters programme for primary care dentists can make a difference to participants' knowledge and confidence and, importantly, generate change in practice.

CLINICAL RELEVANCE

Sustained continuing education programmes, such as the University of Birmingham Masters Degree targeting primary care dentists, can provide valued contact with other dentists, and enhance practitioners' clinical knowledge.

360 degree assessment (multisource feedback) of UK trainee doctors: field testing of team assessment of behaviours (TAB)

BACKGROUND

This study was to see if the team assessment of behaviours (TAB) 360 degree assessment tool was able to identify interpersonal behaviour problems in doctors in training, to see if feedback was useful, to gauge the value of the process by those involved, and to learn lessons about implementing the process for the future.

METHODS

TAB was administered to assess interpersonal behaviours of senior house officers in four hospitals in the West Midlands, UK. In addition, questionnaires were sent to all participants, some were interviewed about the whole process, and records kept of the time involved.

RESULTS

One hundred and seventy-one SHO volunteers received 1378 assessments. The median number of ratings per SHO was 8 (mode 9). Sixty-four percent of SHOs received 'no concern' ratings in all four behaviours (domains) assessed. Twenty-one percent received one 'some concern' rating. Fifteen percent received more than one 'concern' rating.

CONCLUSION

Assessors and trainees found the process practical, valuable and fair. Educational supervisors found it valuable, although only 23% learned something new about their trainees. Clinical tutors valued the system. Administrative staff found it time consuming. The TAB four-domain rating form with its single pass category identified specific concern about volunteer trainees' professional behaviour. Not all trainees received skilled feedback.

A literature review of multi-source feedback systems within and without health services, leading to 10 tips for their successful design

Multi-source feedback (MSF) has become the accepted mechanism of ensuring the appropriate professional behaviour of doctors. It is part of the mandatory assessment of doctors in training and is to be utilized as part of the revalidation of trained doctors. There is significant variation in the models of MSF currently used within the National Health Service and new models of MSF are being designed by various specialties. No single model has been recognized as the 'gold standard'. However, there is a large published literature concerning MSF, both in the context of health systems and, more extensively, within industry. This published literature is reviewed, drawing attention to aspects of MSF systems in which there is consensus on effective approaches as well as other aspects in which there is doubt about the optimum approach. In the light of the review 10 principles key in the development of effective MSF models have been produced.

'Team observation': a six-year study of the development and use of multi-source feedback (360-degree assessment) in obstetrics and gynaecology training in the UK

Multi-source feedback, or 360-degree assessment, is an important part of the assessment of people in the workplace, in both health and industry. Almost all published work concentrates on content validity and generalizability. However, an assessment system needs construct validity, and has to have practicability and acceptability, without sacrificing fitness for purpose, content validity or inter-rater reliability. This was a six-year study of the first UK-wide hospital-based multi-source feedback system, in the specialty of obstetrics and gynaecology. This paper describes the development of the assessment tool, its use and the analyses of the results in several areas. These are picking up poor performance, congratulating good behaviour, construct validity, the number of domains to be measured, and the minimum number of raters. The study demonstrated that the Team Observation system in reality only measured a very limited number of attributes, and that the main construct under scrutiny is interpersonal behaviour. The system can identify those who may have a problem, using less than 10 raters, and yet the process can be a positive experience for the large majority of people who have been assessed.

Evaluation of a pilot careers advice service for junior doctors

This study evaluates a pilot careers service for junior doctors provided by a non-medical careers adviser. Evaluation forms and interviews indicate high satisfaction with this service. The junior doctors valued help with CVs and interviews, information regarding career routes and meetings with senior doctors arranged by the adviser. A longitudinal survey suggests that these doctors became clearer about their career pathways compared with controls, although they did not view the adviser as influencing their career choices. The employment of non-medical careers advisers is proposed. The evaluation was principally conducted by the Centre for Research in Medical and Dental Education, School of Education, University of Birmingham, UK.

Tadalafil is efficacious in Black American and Hispanic men with erectile dysfunction: results from multiple observations in men with erectile dysfunction in national tadalafil study in the US (MOMENTUS)

To show that tadalafil is efficacious in Black American and Hispanic men with erectile dysfunction (ED) and efficacy is noninferior to that in Caucasian men. Multiple observations in men with ED in national tadalafil study in the US, a multicenter, open-label study, assessed the efficacy of tadalafil 20 mg taken as needed (maximum one dose/day before sexual activity) for 12 weeks by patients with ED in various populations. This analysis focuses on three groups: Caucasian (Reference group), Black American, and Hispanic men. Primary measurement of efficacy was change from baseline in erectile function (EF) domain of the International Index of Erectile Function (IIEF) and the primary analysis was whether efficacy in Black American and Hispanic groups was noninferior to efficacy in the Caucasian group. Secondary efficacy measures included sexual encounter profile (SEP), IIEF intercourse satisfaction (IS) and overall satisfaction (OS) domains, Global Assessment Question (GAQ), and Psychological and Interpersonal Relationship Scale (PAIRS). Safety was assessed from adverse events (AEs) reported by all enrolled patients. The increase in IIEF EF domain score (>or=9.5) from baseline for each group was statistically significant (P<0.001) and clinically relevant. Efficacy of tadalafil in Black American and Hispanic patients was noninferior to the Caucasian group. IS and OS domains of IIEF had a statistically significant increase from baseline (P<0.001). Change from baseline in positive responses to SEP questions for each group was significant (P<0.001). At least 77% of intercourse attempts were successful over various time intervals up to 36 h postdose. At least 88% of patients in the various groups had a positive response to GAQ1. Improvement from baseline in Sexual Self-Confidence and Spontaneity domains of PAIRS was statistically significant (P<0.001). A low number of AEs and a low AE-related discontinuation rate (2.3%) were reported in all groups. Tadalafil 20 mg was as efficacious in Black American and Hispanic men with ED as in Caucasian patients and was well tolerated.

A preliminary investigation into factors influencing limiting long-standing illness among UK university graduates: a retrospective cohort study

Two competing hypotheses underpinned an investigation into limiting long-standing illness (LLI) among UK graduates. Hypothesis 1 proposed childhood social class (CSC) influences LLI independently of educational attainment and adult income. Hypothesis 2 proposed typical middle-class graduates would have lower LLI prevalence than typical and atypical working-class and atypical middle-class graduates. Working/ middleclass refers to childhood circumstances. Atypical/typical refers to full-time employment duration before attending university. Graduates (1985; N= 5093 and 1990; N= 8147) were followed up in 1996. Logistic regression was used to examine LLI in 1996 by CSC only and CSC, atypical graduate status and their interaction, adjusting for age and adult income. Hypothesis 1 was not confirmed. Hypothesis 2 was partially confirmed. Typical middle-class graduates had a lower LLI prevalence than typical working-class and atypical middle-class graduates. These results support the idea that opportunities for good human functioning are culturally determined and affect health.

The influence of school culture on smoking among pupils

School factors and not solely pupil composition probably cause variation in smoking prevalence amongst schools, but there are no theoretical models to explain why. In this paper we propose a hypothesis to explain schools' influence on pupils' smoking and test this using an existing cross-sectional survey of 23,282 pupils from 166 secondary schools in the West Midlands, UK. We hypothesise that school-level educational achievement scores would not be associated with smoking prevalence, but schools providing value-added education given the social background of pupils (authoritative schools) would provide effective support and control, have a relatively strong influence on pupils' lives and be associated with lower than average smoking prevalence. Schools providing value-denuded education (laissez-faire schools) would have a relatively weak influence on pupils' lives and be associated with higher than average smoking prevalence. The school achievement measures were the proportion of pupils achieving 5A-C General Certificates of Secondary Education (5A-Cs) grades and the proportion of half days lost to truancy. Value-added/denuded terms were created by regressing 5A-Cs and truancy on five markers of the social profile of pupils at the school. Authoritative schools achieved better than expected rates on both measures. Laissez-faire schools achieved worse than expected rates on both measures. All other schools were classed as indeterminate. Multilevel logistic regression was used to relate the risk of regular smoking to school culture in both achievement and authoritative/laissez-faire terms, both with and without adjustment for pupil-level risk factors for smoking. As predicted, schools' achievement measures were unrelated to pupils' smoking. The odds ratios (95% confidence intervals) for smoking in authoritative and laissez-faire schools relative to indeterminate schools were 0.80 (0.70-0.91) and 1.16 (1.07-1.27), respectively. Adjustment for pupil-level smoking risk factors had little effect. School culture is an independent risk factor for adolescent smoking. Schools providing effective support and control might protect pupils from smoking.

Curriculum and assessment in higher specialist training

Higher Specialist Training (HST) has been radically reformed in the last decade, and specialties have been urged to conduct further reform. This paper reports on an explorative investigation into curriculum provision and assessment within four specialties in the West Midlands. Methods include analysis of curriculum documents; observation of the RITA process; and interviews with consultants (4), specialist registrars (5) and members of the regional higher specialist training management team (3). Three major themes identified within the interviews are discussed: the content and use of the curriculum; the impact of service-based learning; and issues around assessment.

Participation of UK dentists in continuing professional development

INTRODUCTION

This paper reports participation of dentists in continuing professional development (CPD) and factors affecting participation.

METHOD

All general dental practitioners (GDPs) in three deaneries in England were surveyed. The overall response rate was 54% (n = 2082); by deanery it was 68% West Midlands, 45% South West and 44% Anglia. Findings across deaneries were remarkably similar. Comparisons with national data show no notable bias in the sample for gender, owners/partners and age/experience.

RESULTS

Most frequent forms of CPD were journal reading and courses in which almost all engaged. A score based on individual participation in CPD over the 12-month period was calculated. The mean score (hours) for participation in verifiable CPD was 31 (median 25) and for general, 29 (median 29). In terms of the GDC's Lifelong Learning Scheme, 57% were already undertaking 50 hours. Net of other effects, those less likely to be doing 50 hours are those with more years in practice and single-handed practitioners. Greater access to courses and media-based CPD is desired.

CONCLUSION

Certain groups of dentists will need support to meet the requirements of the GDC's Lifelong Learning Scheme. Statutory peer review or clinical audit will significantly alter the CPD profile of most dentists. This has implications for facilitators.

Do the Royal college curriculum statements guide the assessment and learning for specialist registrars?

The Royal College curriculum statements should be a vital part of the teaching and learning of specialist registrars, and of their assessments. This does not appear to be the case in the four specialties studied.

Are Polish and Swedish dental graduates adequately prepared for dental practice in the UK? A discussion of the transferability of general dental practitioners in Europe

In the context of European Union harmonization, this article compares the systems of training for general dental practice in three European countries, the UK, Sweden and Poland. A UK perspective is adopted and the question as to whether dentists who have qualified in Poland or Sweden are adequately prepared for practice in the UK is explored. The paper is a result of discussion between providers of dental training in the three study countries. Key similarities, strengths and weaknesses are identified and issues pertinent to the transferability of general dental practitioners within Europe are raised.

Measuring effectiveness for best evidence medical education: a discussion

This paper considers how educational interventions should be evaluated for their effectiveness. Five levels of effectiveness are clarified and illustrated: outcomes, behaviour, learning, reaction and participation. These levels are then discussed, within the context of research evidence for education within the medical profession. Methodological and practical research conclusions are then drawn. From an analysis of over 300 abstracts the evidence shows that only limited research on healthcare outcomes has been undertaken, and there is considerable ambiguity over what constitutes an effectiveness measure. The benefits of a consistent set of terms are then considered. Finally, five key messages are distilled from the discussion and summarized.

A river network based hydrological model for predicting natural and influenced flow statistics at ungauged sites: Micro LOW FLOWS

Information on the magnitude and variability of flow regimes at the river reach scale is a central component of most aspects of water resource and water quality management. Within the UK the number of river reaches that are gauged represent less than 1% of the total number of river stretches mapped at a scale of 1:50,000. The paper is the first of two papers that describe the development and application of procedures within Micro LOW FLOWS for estimating the variability and magnitude of flow regimes at ungauged sites as represented by flow duration and mean flow statistics. This paper describes the development of the models whilst Young et al. (this volume) describes a pragmatic approach for applying the model to a river basin with complex water use patterns.

Getting tough with home health accounts receivable

Home health organizations seeking to improve the performance of their accounts receivable should concentrate first on establishing practices that promote the greatest percentage of clean claims and reviewing payments received to be sure rates paid are optimal. They also may find it worthwhile to survey their business office operations for potential improvements to key areas.

Comparison of anchors used in anti-incontinence surgery

OBJECTIVES

To compare the resistance to caudally directed force at different fixation points used in female anti-incontinence surgery.

METHODS

Sutures were placed in Cooper's ligament, rectus fascia, and in bone anchors on the pubic symphysis in 6 fresh, unembalmed cadavers and transposed vaginally with a ligature carrier. Force was applied in a caudal direction to each suture. Displacement of the suture with increasing force and the force required to overcome the cranial fixation point were measured with a highly precise force gauge. Two measurements were made for each anchor point and the measurements were averaged.

RESULTS

With caudally directed force, sutures fixed to Cooper's ligament were displaced to an equal extent as sutures attached to a bone anchor. Sutures anchored to the rectus fascia were displaced the most with increasing force. The maximal force supported by Cooper's ligament fixation and bone anchors was similar. The event limiting each test was suture breakage, except when one suture anchored in the rectus fascia tore out.

CONCLUSIONS

In a cadaver model, bone anchors placed in the pubic symphysis offer no structural advantage over Cooper's ligament fixation.

Single lead VDD pacing in children with complete heart block

OBJECTIVE

To evaluate clinically transvenous single lead VDD pacing in children with complete heart block.

DESIGN

Case series.

SETTING

Tertiary care unit of a pediatric teaching hospital.

PATIENTS

Consecutive sample of children with complete heart block requiring pacing and meeting the criteria for VDD pacing.

INTERVENTIONS

VDD single lead pacing systems (Thera VDD 8948 generator, Capsure VDD 5032 single pass lead) were implanted in nine children (mean age 10.7 years, 95% CI 8.9 to 12.3). All patients were reviewed six weeks and six months post-implantation with clinical review, detailed pacemaker interrogation, optimal device programming, treadmill exercise testing, 24 h Holter monitoring and echocardiography.

RESULTS

Mean procedure time was 62.5 mins (CI 52 to 73), mean fluoroscopy time 3.8 mins (CI 2.4 to 5.1) and mean P wave sensing amplitude 2.75 mV (CI 2.5 to 3), without significant decrease during Valsalva manoeuvre (mean 2.4 mV, CI 2.2 to 2.8). Low pacing thresholds of mean amplitude 0.39 V (CI 0.38 to 0.4) were established at pulse width 0.5 ms. Two patients required early revision--one lead dislodged and one patient had a loose set screw. Follow-up showed satisfactory pacemaker function, and mean percentage atrioventricular (AV) synchronous pacing 96.8% (CI 93 to 99) at six weeks and 97% (CI 95 to 99) at six months. P wave sensing failures were less than 1%. AV synchronous pacing was maintained for more than 99% of the period of treadmill exercise testing. There was a significant mean increase in cardiac output of 24% (CI 8 to 40, P < 0.05) at rest in VDD mode compared with VVIR mode.

CONCLUSIONS

Single lead VDD pacing is an effective and reliable method of 'physiological' rate responsive AV synchronous pacing in children with complete heart block and should be considered a viable alternative to dual lead pacing systems.

Determining normative standards for functional independence measure transitions in rehabilitation

We present a method for determination of normative standards for Functional Independence Measure (FIM) transitions in rehabilitation. Data from 230 consecutive brain-injured patients treated before 1991 were used to characterize transitions in patient FIM values between admission and discharge. The pre-1991 average and standard deviation FIM transitions, computed as a function of admission values, are used as standards ("norms") for comparing rehabilitation transitions among institutions or in one institution over time (say, yearly) and for identifying patients with striking transitions, believed worthy of audit. The evaluation method requires the computation of two statistics, z and W, which compare the actual transitions for patients of one time period (in this instance the 1991 patients) to the expected transitions as computed from the pre-1991 norms. The z and W values indicated that 1991 transitions were neither statistically nor clinically different from pre-1991 ones. Also introduced in the paper are the concepts of Mean Gain, Ideal Gain, and the ratio Mean Gain/Ideal Gain. Ideal Gain is the greatest possible "aggregated" transitions score for a study patient set and the ratio Mean Gain/Ideal Gain may be interpreted as "the degree of ideal rehabilitation transitions achieved."

High dose nebulized steroid in the treatment of chronic steroid-dependent asthma

Despite the dramatic success of inhaled steroids in controlling asthma symptoms there remains a small number of patients in whom asthma can only be treated with continuous oral steroids. Eighteen such patients, aged 19-62 years (seven males, 11 females) were followed in an open trial of nebulized budesonide over 12-18 months. All had required at least 7.5 mg or more daily prednisolone to control their symptoms over the preceeding 2 or more years and were taking 1200 micrograms beclomethasone dipropionate or 1600 micrograms budesonide daily. With a daily dose ranging between 4 and 8 mg nebulized budesonide, 14 patients successfully stopped oral steroids while in three the dose was reduced; only one patient failed to benefit. There was an increase in the mean FEV1 from 1.9 (+/- 0.9) to 2.2 (+/- 0.9) l, and in the mean morning PEFR, from 238 (+/- 119) to 286 (+/- 130) l min-1. There was also a significant decrease in the mean number of hospital admissions for acute severe asthma, from 1.5 (+/- 1.8) to 0.9 (+/- 1.1) per year. These findings should encourage a careful and controlled evaluation of nebulized steroids as a substitute for oral steroids in this difficult group of asthmatics.

Renal autotransplantation in the management of a ureteroarterial fistula: a case report and review of the literature

A 57-year-old woman who had been treated with an indwelling ureteral stent for over a year was admitted with massive gross hematuria caused by a fistula between the left ureter and hypogastric artery. Despite intensive radiographic evaluation, the definitive diagnosis was made only at the time of surgical exploration. Because of radiation-induced retroperitoneal fibrosis, midureteral obstruction, and prior pelvic and abdominal surgery, primary ureteral repair was not possible, and renal autotransplantation was performed. This case illustrates the need to consider the diagnosis of ureteroarterial fistula in patients with massive hematuria who have chronic indwelling stents, and the feasibility of autotransplantation when primary ureteral repair is not feasible.

Mu opiate receptors are selectively labelled by [3H]carfentanil in human and rat brain

[11C]Carfentanil is a potent opioid agonist currently in use as a specific PET (position emission tomography) scan radioligand for brain mu opioid receptors. In order to investigate the receptor interactions of carfentanil in detail [3H]carfentanil was used as a radioligand for labelling receptors in rat and human brain tissue homogenates. [3H]Carfentanil was found to bind saturably and with high affinity (KD = 0.08 +/- 0.01 nM) to membranes prepared from human cortical (Bmax = 42 +/- 3 fmol/mg) and thalamic (Bmax = 84 +/- 3 fmol/mg) tissues and rat cortex (Bmax = 82 +/- 4 fmol/mg) and diencephalon (Bmax = 105 +/- 5 fmol/mg). Association (1.23 +/- 0.19 X 10(10) Mol-1 X min-1 and dissociation rate (0.19 +/- 0.03 min-1) constants were determined in human cortical tissues; results from studies in rat cortical, and rat diencephalon tissue homogenates produced similar kinetic rate constants. Competition studies with a variety of drugs indicated that [3H]carfentanil interacts primarily with mu opioid receptors in the four tissues studied; the affinities of a series of non-radioactive opioid ligands were essentially identical in the four tissues (correlation coefficients = 0.88-0.93). Naloxone, morphine, DAGO [( D-Ala2-MePhe4-Gly-ol5]enkephalin), DADL [( D-Ala2-D-Leu5]enkephalin) and EKC (ehtylketazocine) potently displaced specific [3H]carfentanil binding with nM potency while the kappa agonist U-69593, the sigma agonists (+)-SKF 10047, (+)-3-PPP [3-hydroxyphenyl)-N-propylpiperidine) and haloperidol and PCP (phencyclidine) were less potent displacing agents. The higher affinities of DAGO and morphine versus DADL for the [3H]carfentanil binding sites indicates that delta opioid receptors are not being labelled.(ABSTRACT TRUNCATED AT 250 WORDS)

Communication and the nonverbal, multihandicapped child

A pupil at the Kennedy Memorial Hospital Day School, Boston, was trained by an occupational therapist to use the Auto-Com, a device that enables those with severe motor impairment, such as cerebral palsy, to communicate effectively with others. The device was developed by the Cerebral Palsy Communication Group of the University of Wisconsin, Madison. The child continues to use the device and has become a more involved member of the class.