Evaluation of web-based learning tools: lessons learned from the thoracic surgery directors association curriculum project three-year experience

BACKGROUND

In 2001 the Thoracic Surgery Directors Association implemented a web-based curriculum for prematriculated TS residents containing 75 learning modules organized into 13 sections Each module contains an audio/video presentation, a summary, frequently asked questions, references, a self-assessment, and a critique. Curriculum content was provided as a randomized prospective trial one year prior to matriculation to residents matching into a TS program from 2001 to 2004.

METHODS

Over 3 years the utilization and impact of this curriculum was evaluated by 8 different methods including resident surveys, faculty surveys, web tracking, examination scores, and individual online learning module critiques. Each completed critique evaluated the resident's perception of the relevance, content quality and presentation quality of the module on a scale corresponding to low, average and high.

RESULTS

A total of 11,117 learning modules were used through the end of 2004 with complete critiques available on 1458 (13%). Utilization and measured quality continue to increase over the 3-year interval. The average rating scores for relevance, content and presentation tracked together and were overall rated as good or excellent (92%). There were large variations in utilization and perceived quality by section subject. As the average module rating score improved, the utilization increased and the variability of the rating decreased substantially.

CONCLUSIONS

Module specific learner evaluation of educational content is one of several important feedback tools for monitoring ongoing curricular development and refinement. These studies, based upon web tracking, demonstrate increasing resident use and strong resident satisfaction. This educational format will hopefully lend itself to many important improvements in web-based Thoracic Surgery education.

Competencies, essential training, and resources viewed by designated institutional officials as important to the position in graduate medical education

PURPOSE

In 1998, the Accreditation Council for Graduate Medical Education (ACGME) added the stipulation that each institution providing graduate medical education (GME) have a Designated Institutional Official (DIO). The authors conducted this study via a cross-sectional survey designed to provide descriptive data on the beliefs held by DIOs regarding required competencies, training and experience, and desired resources for the position.

METHOD

The authors collected data between January 2004 and May 2004 using a multistep process that included a prenotice letter; a survey, cover letter, and stamped return envelope; a thank you/reminder postcard; and a replacement survey with new cover letter and stamped return envelope. Data were summarized using descriptive statistics.

RESULTS

Completed surveys were received from 243 of 363 DIOs (66.9%). Eighty-two percent indicated that DIOs should have specified minimum experience or training requirements. Ten competency items were viewed as essential by greater than 50% of respondents, with five items endorsed by over 95% of respondents: professionalism, verbal communication skills, interpersonal skills, leadership skills, and written communications skills. The percentage of responding DIOs who indicated they would be very likely to use resources were as follows: templates for GME policies, contracts, and affiliation agreements (83.1%); DIO-specific training (58.0%); data on DIO demographics (53.9%); DIO job description templates (46.9%); a clear description of DIO versus program director responsibilities (46.1%); and a DIO certification program (32.1%).

CONCLUSIONS

Designated Institutional Officials supported the idea that there should be minimum experience and requirements and demonstrated remarkable consistency in endorsing essential competencies for the position. DIOs, their respective institutions, the ACGME, and other GME organizations and associations may use the data from this study to develop the role further.

Mentoring the mentors: Aligning mentor and mentee expectations

The Radiology Alliance for Health Services Research sponsored a symposium at the 2005 Annual Meeting of the Association of University Radiologists, which focused on the issue of aligning mentor and mentee expectations to foster successful mentoring relationships. This article presents a summary of the informal discussion of the panelists' individual experiences, common themes, and insights gained from the panel participants.

Doctor‐managers as decision makers in hospitals and health centres

PURPOSE

This paper describes factors influencing doctor-managers' decision making in specialised health care, health centres and at different levels of management.

DESIGN/METHODOLOGY/APPROACH

Data were collected as part of a survey on physicians graduating in 1977-1991 as drawn from the register of the Finnish Medical Association. The study sample was formed by selecting all physicians born on odd days (n=4144) from the baseline group (n=8232). The category of doctor-managers comprised physicians reporting as their main occupation: principal or assistant principal physician of hospital, medical director or principal physician of health centre, senior ward physician of hospital, and health centre physician in charge of a population area.

FINDINGS

Independent of gender, all doctor-managers responding to the survey reported that the most important base for decision making was personal professional experience. Position in organisation (first-line manager, principal physician) had no impact on the base of decision making. Doctor-managers in primary health care utilised knowledge on norms and knowledge available from their organisation in support of their decision making to a greater degree compared with doctor-managers in specialised health care.

RESEARCH IMPLICATIONS

Evolution discourse from public administration is not yet receiving much response in Finnish doctor-managers' activities, instead, they still act as clinicians.

ORIGINALITY/VALUE

Facing the growing challenges of the future, the paper shows that doctor-managers should reconstruct their orientation and to act more like managers.

Management training of physician executives, their leadership style, and care management performance: an empirical study

OBJECTIVE

To examine associations between management training of physician executives and their leadership styles, as well as effectiveness in achieving disease management goals.

STUDY DESIGN

Cross-sectional national survey.

METHODS

Executive directors of community health centers (269 respondents; response rate = 40.9%) were surveyed regarding their perceptions of the medical director's leadership, and for quantitative information on the center's achievement of clinical (mostly disease management) goals. The dependent variables were the medical director's scores (as perceived by the executive director) on transformational, transactional, and laissez-faire leadership, effectiveness, satisfaction with the leader, and subordinate extra effort, using an adapted Multifactor Leadership Questionnaire (43 items; 5-point Likert scale). The independent variable was the medical director's management training status.

RESULTS

Compared with medical directors with < 30 days of inservice training, medical directors with an MHA, MPH, or MBA, or > or =30 days of in-service training, had 0.32, 0.35, 0.30, 0.36, and 0.37 higher scores on transformational leadership, transactional leadership, rated effectiveness, satisfaction, and subordinate extra effort, respectively, and 0.31 lower score on laissez-faire leadership (all P < .001). Medical directors without management degrees but with > or =30 days of in-service training had 0.34, 0.36, 0.50, and 0.47 higher scores on transformational leadership, transactional leadership, rated effectiveness, and satisfaction with the leader (all P < .02). Our data previously had demonstrated that medical directors' transformational leadership significantly influences achievement of disease management goals.

CONCLUSION

Training may enable physician executives to develop leadership styles that are effective in influencing clinical providers' adoption of disease management guidelines under managed care.

Needs assessment for health care management education in Russia

INTRODUCTION

For more than 70 years, health care management in the Soviet Union reflected a centralized directive style familiar to the Soviet political system. Market-oriented reform in post-Soviet Russia is pushing practicing physicians and physician-executives to acquire new information and skills regarding health care management. To assist with health care management in Russia, we analyzed health care providers' educational needs, as they perceive them.

METHODS

A total of 4, 367 questionnaires were sent to practicing physicians and physician-executives in 3 regions of the Russian Federation. The questionnaires included 14 items designed to determine respondents' current levels of satisfaction and preferences for health care management education in Russia.

RESULTS

There were 847 questionnaires returned, for a total response rate of 19.4%. Physicians were dissatisfied with the current status of health care management education, but they were interested in learning the skills of health care management. The health care legal system and computer systems and data analysis were the most popular courses requested; health care management and marketing were the least popular. Learning interests of executives differed from those of practicing physicians, and regions differed in their combined interests for courses. Medical school and continuing education departments were seen as the best choices for delivering health care management education.

DISCUSSION

As health care policy shifts in new directions, physicians in leadership positions expect Russian medical schools and continuing education departments to provide courses that enable improvements in health care.

Lessons learned in implementing evidence-based practices: implications for psychiatric administrators

Factors related to the dissemination and implementation of evidence-based practices (EBPs) are discussed. Extensive effort is required to successfully implement and sustain EBPs that improve clinical outcomes. There is a rapid rate of discovery of new EBPs. Examples of large-scale implementations of EBPs in mental health are described with emphasis on the factors thought critical for success. The need for designing systems which can cost-effectively implement new EBPs is highlighted. Finally, the implications for psychiatric administrators are discussed.

Challenges in implementing a successful newborn cystic fibrosis screening program

OBJECTIVE

To identify necessary components of a successful cystic fibrosis (CF) newborn screening (NBS) program.

STUDY DESIGN

The approach to CF NBS used by the Massachusetts NBS program was examined.

RESULTS

Several key components were identified that should be addressed when a state has made the decision to screen, and well in advance of actual implementation. These components include (1) inclusion of CF center directors in the development process; (2) logistics of choosing a screening algorithm relative to practices in place and community wishes; (3) projections of medical service needs from specific algorithms; (4) identification of critical reporting components; (5) identification of critical follow-up components; and (6) recognition of educational needs.

CONCLUSIONS

Careful examination of a wide variety of issues is needed to ensure optimal implementation of NBS for CF.