Performance-based assessment of moral reasoning and ethical judgment among medical students

Evaluating interventions to improve ethical decision making in clinical practice: a review of the literature and reflections on the challenges posed

Since the 1980s, there has been an increasing acknowledgement of the importance of recognising the ethical dimension of clinical decision-making. Medical professional regulatory authorities in some countries now include ethical knowledge and practice in their required competencies for undergraduate and post graduate medical training. Educational interventions and clinical ethics support services have been developed to support and improve ethical decision making in clinical practice, but research evaluating the effectiveness of these interventions has been limited. We undertook a systematic review of the published literature on measures or models of evaluation used to assess the impact of interventions to improve ethical decision making in clinical care. We identified a range of measures to evaluate educational interventions, and one tool used to evaluate a clinical ethics support intervention. Most measures did not evaluate the key impact of interest, that is the quality of ethical decision making in real-world clinical practice. We describe the results of our review and reflect on the challenges of assessing ethical decision making in clinical practice that face both developers of educational and support interventions and the regulatory organisations that set and assess competency standards.

A systematic scoping review of undergraduate medical ethics education programs from 1990 to 2020

INTRODUCTION

Ensuring medical students are equipped with essential knowledge and portable skills to face complex ethical issues underlines the need for ethics education in medical school. Yet such training remains variable amidst evolving contextual, sociocultural, legal and financial considerations that inform training across different healthcare systems. This review aims to map how undergraduate medical schools teach and assess ethics.

METHODS

Guided by the Systematic Evidence-Based Approach (SEBA), two concurrent systematic scoping reviews were carried out, one on ethics teaching and another on their assessment. Searches were conducted on PubMed, Embase, PsycINFO and ERIC between 1 January 1990 and 31 December 2020. Data was independently analysed using thematic and content analysis.

RESULTS

Upon scrutinising the two sets of full-text articles, we identified 141 articles on ethics teaching and 102 articles on their assessments. 83 overlapped resulting in 160 distinct articles. Similar themes and categories were identified, these include teaching modalities, curriculum content, enablers and barriers to teaching, assessment methods, and their pros and cons.

CONCLUSION

This review reveals the importance of adopting an interactive, multimodal and interdisciplinary team-teaching approach to ethics education, involving community resource partners and faculty trained in ethics, law, communication, professionalism, and other intertwining healthcare professions. Conscientious effort should also be put into vertically and horizontally integrating ethics into formal medical curricula to ensure contextualisation and application of ethics knowledge, skills and attitudes, as well as protected time and adequate resources. A stage-based multimodal assessment approach should be used to appropriately evaluate knowledge acquisition, application and reflection across various practice settings. To scaffold personalised development plans and remediation efforts, multisource evaluations may be stored in a centralised portfolio. Whilst standardisation of curricula content ensures cross-speciality ethical proficiency, deliberative curriculum inquiry performed by faculty members using a Delphi approach may help to facilitate the narrowing of relevant topics.

Integrating patient management, reflective practice, and ethical decision-making in an emergency medicine intern boot camp

BACKGROUND

Integration of clinical skills, ethical decision-making, and reflection skills have emerged as cornerstones of clinical teaching in medical schools. This study aimed to detect whether a multimodal learning environment approach consisting of lectures, a drill, post-drill video debriefing, and written reflection in an emergency medicine rotation boot camp improves interns' patient management skills, ethical decision-making, and reflection skills.

METHODS

A multimodal learning environment was created by the collaboration of emergency medicine, ethics, and medical education specialists. Multiple educational techniques involving lectures, case discussions, and role-playing a crisis scenario were applied. Pre-test and post-test, debriefing on performances on video records, video-recorded performance assessment, and reflective essays about their own and group's performances were used to assess various aspects of the student performances. Additionally, a meeting was organized with the presence of the authors to create qualitative data obtained through the program evaluation meeting conducted on three themes: influences of teaching methods, students' performances, and common achievements and mistakes of students.

RESULTS

133 students participated. Post-test multiple-choice question (MCQ) test scores were slightly higher than pre-test. A low and medium correlation was detected among pre-test and post-test patient management problem (PMP) and reflection scores, which was more prominent for female students. Multiple linear regression showed that pre-test and post-test PMP scores significantly contributed to reflection scores. These results might support that better patient management predicts more robust reflective practice. Teachers observed that students appreciated being inspired by well-performing peers, particularly noting the empathic needs of patients, companions, and other health professionals. However, students overlooked summoning forensic or social services and were inhibited by the pressure of the contextual traits of the drill.

CONCLUSION

The multimodal learning environment created by multidisciplinary collaboration contributed to the improvement of components of situational awareness of the interns: patient management skills, ethical decision-making, and reflective practice. During this research, we created a toolbox better to capture the richness and diversity of student interactions. Considering the scarcity of context-specific assessment methods and widespread use of MCQs or generic scales for higher-order thinking skills in medicine, this study might be regarded as a step forward in that context.

Developing an intervention around referral and admissions to intensive care: a mixed-methods study

Background

Intensive care treatment can be life-saving, but it is invasive and distressing for patients receiving it and it is not always successful. Deciding whether or not a patient will benefit from intensive care is a difficult clinical and ethical challenge.

Objectives

To explore the decision-making process for referral and admission to the intensive care unit and to develop and test an intervention to improve it.

Methods

A mixed-methods study comprising (1) two systematic reviews investigating the factors associated with decisions to admit patients to the intensive care unit and the experiences of clinicians, patients and families; (2) observation of decisions and interviews with intensive care unit doctors, referring doctors, and patients and families in six NHS trusts in the Midlands, UK; (3) a choice experiment survey distributed to UK intensive care unit consultants and critical care outreach nurses, eliciting their preferences for factors used in decision-making for intensive care unit admission; (4) development of a decision-support intervention informed by the previous work streams, including an ethical framework for decision-making and supporting referral and decision-support forms and patient and family information leaflets. Implementation feasibility was tested in three NHS trusts; (5) development and testing of a tool to evaluate the ethical quality of decision-making related to intensive care unit admission, based on the assessment of patient records. The tool was tested for inter-rater and intersite reliability in 120 patient records.

Results

Influences on decision-making identified in the systematic review and ethnographic study included age, presence of chronic illness, functional status, presence of a do not attempt cardiopulmonary resuscitation order, referring specialty, referrer seniority and intensive care unit bed availability. Intensive care unit doctors used a gestalt assessment of the patient when making decisions. The choice experiment showed that age was the most important factor in consultants’ and critical care outreach nurses’ preferences for admission. The ethnographic study illuminated the complexity of the decision-making process, and the importance of interprofessional relationships and good communication between teams and with patients and families. Doctors found it difficult to articulate and balance the benefits and burdens of intensive care unit treatment for a patient. There was low uptake of the decision-support intervention, although doctors who used it noted that it improved articulation of reasons for decisions and communication with patients.

Limitations

Limitations existed in each of the component studies; for example, we had difficulty recruiting patients and families in our qualitative work. However, the project benefited from a mixed-method approach that mitigated the potential limitations of the component studies.

Conclusions

Decision-making surrounding referral and admission to the intensive care unit is complex. This study has provided evidence and resources to help clinicians and organisations aiming to improve the decision-making for and, ultimately, the care of critically ill patients.

Future work

Further research is needed into decision-making practices, particularly in how best to engage with patients and families during the decision process. The development and evaluation of training for clinicians involved in these decisions should be a priority for future work.

Study registration

The systematic reviews of this study are registered as PROSPERO CRD42016039054, CRD42015019711 and CRD42015019714.

Funding

The National Institute for Health Research Health Services and Delivery Research programme. The University of Aberdeen and the Chief Scientist Office of the Scottish Government Health and Social Care Directorates fund the Health Economics Research Unit.

Twelve tips for the construction of ethical dilemma case-based assessment

Ethical dilemma case-based examination (ethics Script Concordance Test, eSCT) is a written examination that can be delivered to a large group of examinees for the purpose of measuring high-level thinking. As it accommodates for diverse responses from experts, ethics SCT allows partial credits. The framework of ethics SCT includes a vignette with an ethical dilemma and a leading question, which asks the examinee to "agree" or "disagree", plus the shifts of prior decision by adding new information. In this article, the following tips for constructing this type of examination are provided: use "true" dilemmas, select an appropriate ethical issue, target high-level cognitive tasks, list key components, keep a single central theme, device quality scoring system, be important and plausible, be clear, select quality experts, validate, know the limitation, and be familiar with test materials. The use of eSCT to measure ethical reasoning ability appears to be both viable and desirable.

The Effectiveness of Using Simulated Patients versus Videotapes of Simulated Patients to Teach Clinical Skills to Occupational and Physical Therapy Students

Seventy-three occupational and physical therapy students participated in two seminars. In one seminar, students interacted directly with a simulated patient; in the other, students viewed a videotape of a simulated patient interacting with a clinician. Each student submitted the following for each seminar: a list of problems and treatment plans, a rating form indicating satisfaction with group participation, and a rating form indicating satisfaction with topics covered. Within-group comparisons showed that identification of patient problems by the students tended to agree with expert clinicians in the simulated patient condition when compared with the videotape condition although this difference failed to reach significance ( P=0.056). More of the student treatment plans agreed with expert clinicians in the videotape condition when compared with the simulated patient condition ( P<0.01). Both group process and content ratings in the simulated patient condition exceeded those in the videotape condition.

Measuring 'virtue' in medicine

Virtue-approaches to medical ethics are becoming ever more influential. Virtue theorists advocate redefining right or good action in medicine in terms of the character of the doctor performing the action (rather than adherence to rules or principles). In medical education, too, calls are growing to reconceive medical education as a form of character formation (rather than instruction in rules or principles). Empirical studies of doctors' ethics from a virtue-perspective, however, are few and far between. In this respect, theoretical and empirical study of medical ethics are out of alignment. In this paper, we survey the empirical study of medical ethics and find that most studies of doctors' ethics are rules- or principles-based and not virtue-based. We outline the challenges that exist for studying medical ethics empirically from a virtue-based perspective and canvas the runners and riders in the effort to find virtue-based assessments of medical ethics.

Cultural similarities and differences in medical professionalism: a multi-region study

CONTEXT

Over the last two decades, many medical educators have sought to define professionalism. Initial attempts to do so were focused on defining professionalism in a manner that allowed for universal agreement. This quest was later transformed into an effort to 'understand professionalism' as many researchers realised that professionalism is a social construct and is culture-sensitive. The determination of cultural differences in the understanding of professionalism, however, has been subject to very little research, possibly because of the practical difficulties of doing so. In this multi-region study, we illustrate the universal and culture-specific aspects of medical professionalism as it is perceived by medical practitioners.

METHODS

Forty-six professional attributes were identified by reviewing the literature. A total of 584 medical practitioners, representing the UK, Europe, North America and Asia, participated in a survey in which they indicated the importance of each of these attributes. We determined the 'essentialness' of each attribute in different geographic regions using the content validity index, supplemented with kappa statistics.

RESULTS

With acceptable levels of consensus, all regional groups identified 29 attributes as 'essential', thereby indicating the universality of these professional attributes, and six attributes as non-essential. The essentialness of the rest varied by regional group.

CONCLUSIONS

This study has helped to identify regional similarities and dissimilarities in understandings of professionalism, most of which can be explained by cultural differences in line with the theories of cultural dimensions and cultural value. However, certain dissonances among regions may well be attributable to socio-economic factors. Some of the responses appear to be counter-cultural and demonstrate practitioners' keenness to overcome cultural barriers in order to provide better patient care.

Medical professionalism: what does the public think?

In the context of professionalism being viewed increasingly as a social contract, a survey was conducted to investigate the importance placed by the general public on doctors' professional attributes. A quota sample of 953 responded to a 55-item online inventory of professional attributes. The quotas closely represented the national census. The majority of the highly important attributes focused on the relationship with patients. Statistically, the responses emerged as a three-facet model (clinicianship, workmanship and citizenship) of medical professionalism. The general public did not equate professionalism with social standing, wealth production, physique or appearance. They recognised doctors as professionals by their good behaviour, high values and positive attitudes as clinicians, workmen and citizens. Although, their preference of professional attributes varied with the setting, eg patient consultation, working with others and behaving in society, they expected doctors to be confident, reliable, dependable, composed, accountable and dedicated across all settings.

The New York City Rheumatology Objective Structured Clinical Examination: five-year data demonstrates its validity, usefulness as a unique rating tool, objectivity, and sensitivity to change

OBJECTIVE

Traditional means of testing rheumatology fellows do not adequately assess some skills that are required to practice medicine well, such as humanistic qualities, communication skills, or professionalism. Institution of the New York City Rheumatology Objective Structured Clinical Examination (ROSCE) and our sequential 5 years of experience have provided us with a unique opportunity to assess its usefulness and objectivity as a rheumatology assessment tool.

METHODS

Prior to taking the examination, all of the fellows were rated by their program directors. Fellows from the participating institutions then underwent a multistation patient-interactive examination observed and rated by patient actors and faculty raters. Assessments were recorded by all of the participants using separate but overlapping sets of instruments testing the Accreditation Council of Graduate Medical Education (ACGME) core competencies of patient care, interpersonal and communication skills, professionalism, and overall medical knowledge.

RESULTS

Although the program directors tended to rate their fellows more highly than the ROSCE raters, typically there was agreement between the program directors and the ROSCE faculty in distinguishing between the highest- and lowest- performing fellows. The ROSCE faculty and patient actor assessments of individual trainees were notable for a high degree of concordance, both quantitatively and qualitatively.

CONCLUSION

The ROSCE provides a unique opportunity to obtain a patient-centered assessment of fellows' ACGME-mandated competencies that traditional knowledge-based examinations, such as the rheumatology in-service examination, cannot measure. The ability of the ROSCE to provide a well-rounded and objective assessment suggests that it should be considered an important component of the rheumatology training director's toolbox.

Evaluating medical students' skills in obtaining informed consent for HIV testing

OBJECTIVE

To evaluate fourth-year medical students' abilities to obtain informed consent or refusal for HIV testing through a performance-based evaluation method.

DESIGN

Student competence was assessed in a standardized patient interaction in which the student obtained informed consent or refusal for HIV testing. A previously validated 16-item checklist was completed by the standardized patient. A subset was independently reviewed and scored by a faculty member to calculate interrater reliability for this report. Student feedback on the assessment was elicited.

SETTING

School of Medicine at the University of New Mexico.

PATIENTS/PARTICIPANTS

All senior medical students in the class of 2000 were included.

INTERVENTIONS

A 10-minute standardized patient interaction was administered within the context of a formal comprehensive performance assessment.

MEASUREMENTS AND MAIN RESULTS

Seventy-nine students participated, and most (96%) demonstrated competence on the station. For the 15 specific items, the mean score was 25.5 out of 30 possible points (range, 13 to 30; SD, 3.5) on the checklist. A strong positive correlation (rs =.79) was found between the total score on the 15 Likert-scaled items and the score in response to the global item, "I would return to this clinician" (mean, 3.5; SD, 1.0). Scores given by the standardized patients and the faculty rater were well correlated. The station was generally well received by students, many of whom were stimulated to pursue further learning.

CONCLUSIONS

This method of assessing medical students' abilities to obtain informed consent or refusal for HIV testing can be translated to a variety of clinical settings. Such efforts may help in demonstrating competence in performing key ethics skills and may help ensure ethically sound clinical care for people at risk for HIV infection.

Assessing students' performances in a competency-based curriculum

Brown Medical School embarked on planning a competency-based curriculum in 1989. The curriculum was fully implemented in 1996, effective for the MD class of 2000. To graduate, a student must (1) demonstrate mastery of the medical knowledge base, (2) achieve beginning and intermediate levels of proficiency in nine key abilities, and (3) attain an advanced level in the ability called "problem solving" and three other abilities that the student chooses based on his or her interests. The nine abilities are effective communication; basic clinical skills; using basic science in the practice of medicine; diagnosis, management, and prevention; lifelong learning; self-awareness, self-care, and personal growth; the social and community contexts of health care; moral reasoning and clinical ethics; and problem solving. As a major educational innovation, the new competency-based curriculum has been successfully woven into the fabric of the medical school learning environment. In this article, the authors describe how faculty go about assessing students' performances relative to the nine abilities.

Assessing professional behavior: yesterday, today, and tomorrow

PURPOSE

The author interprets the state of the art of assessing professional behavior. She defines the concept of professionalism, reviews the psychometric properties of key approaches to assessing professionalism, conveys major findings that these approaches produced, and discusses recommendations to improve the assessment of professionalism.

METHOD

The author reviewed professionalism literature from the last 30 years that had been identified through database searches; included in conference proceedings, bibliographies, and reference lists; and suggested by experts. The cited literature largely came from peer-reviewed journals, represented themes or novel approaches, reported qualitative or quantitative data about measurement instruments, or described pragmatic or theoretical approaches to assessing professionalism.

RESULTS

A circumscribed concept of professionalism is available to serve as a foundation for next steps in assessing professional behavior. The current array of assessment tools is rich. However, their measurement properties should be strengthened. Accordingly, future research should explore rigorous qualitative techniques; refine quantitative assessments of competence, for example, through OSCEs; and evaluate separate elements of professionalism. It should test the hypothesis that assessment tools will be better if they define professionalism as behaviors expressive of value conflicts, investigate the resolution of these conflicts, and recognize the contextual nature of professional behaviors. Whether measurement tools should be tailored to the stage of a medical career and how the environment can support or sabotage the assessment of professional behavior are central issues. FINAL THOUGHT: Without solid assessment tools, questions about the efficacy of approaches to educating learners about professional behavior will not be effectively answered.

Uncertainty and opposition of medical students toward assisted death practices

To explore medical students' views of assisted death practices in patient cases that describe different degrees and types of physical and mental suffering, an anonymous survey was administered to all students at one medical school. Respondents were asked about the acceptability of assisted death activities in five patient vignettes and withdrawal of life support in a sixth vignette. In the vignettes, actions were performed by four possible agents: the medical student personally; a referral physician; physicians in general; or non-physicians. Of 306 medical students, 166 (54%) participated. Respondents expressed opposition or uncertainty about assisted death practices in the five patient cases that illustrated severe forms of suffering which were secondary to amyotrophic lateral sclerosis, treatment-resistant depressive and somatoform disorders, antisocial and sexually violent behavior, or AIDS. Students supported the withdrawal of life support in the sixth vignette depicting exceptional futility secondary to AIDS. Students were especially opposed to their own involvement and to the participation of non-physicians in assisted death activities. Differences in views related to sex, religious beliefs, and personal philosophy were found. Medical students do not embrace assisted death practices, although they exhibit tolerance regarding the choices of medical colleagues. How these attributes of medical students will translate into future behaviors toward patients and peers remains uncertain. Medical educators must strive to understand the perspectives of physicians-in-training. Expanded, empirically informed education that is attuned to the attitudes of medical students may be helpful in fulfilling the responsibility of imparting optimal clinical care skills.

Review of ethics curricula in undergraduate medical education

Medical ethics education, it has been said, has 'come of age' in recent years in terms of its formal inclusion in undergraduate medical curricula. This review article examines the background to its inclusion in undergraduate curricula and goes on to examine the consensus that has arisen on the design of ethics curricula, using Harden's curriculum and S.P.I.C.E.S models as templates. While there is consensus on content for undergraduate medical ethics education, there is still significant debate on learning and teaching methods. Despite the broad agreement on the need to apply adult education principles to ethics teaching, there would appear to be some tension between balancing the need for experiential learning and achieving the 'core curriculum'. There are also as yet unresolved difficulties with regards to resources for delivery, academic expertise, curriculum integration and consolidation of learning. Assessment methods also remain contentious. Although there is consensus that the ultimate goal of medical ethics, and indeed of medical education as a whole, is to create 'good doctors', the influence of the 'hidden curriculum' on students' development is only beginning to be recognized, and strategies to counteract its effects are in their infancy. The need for proper evaluation studies is recognized. It is suggested that the areas of debate appearing in the literature could be used as a starting point for evaluation studies, which would form the empirical basis of future curriculum development.

Qualitative outcome assessment of a medical ethics program for clinical clerkships: a pilot study

This study assessed the usefulness of an open-ended case analysis test instrument for evaluating the effects of a 1-year ethics course on medical students' decision-making skills. Through case-oriented seminars in gynecology, internal medicine, obstetrics, pediatrics, psychiatry, and surgery, third-year medical students were taught a structured analytic framework for analyzing clinical ethical problems stressing the interactive relationships among medical indications, patient preferences, quality of life, and contextual (social, legal, economic) matters. At precourse, the students were given a test case and asked to provide a line of reasoning for their clinical decisions. At postcourse, the students were given the same case. Content analysis of pre- and postcourse responses of a random student sample revealed increases in student awareness in the following areas: 1) consideration of informed consent, 2) professional liability, 3) physician-assisted suicide, and 4) resource utilization. With some modifications, open-ended case analysis holds promise for evaluating medical ethics courses. The authors make recommendations for future research in evaluating the true impact of clinical ethics courses in medical education.

Teaching medical ethics to orthopaedic surgery residents

Orthopaedic surgery residents will be faced with a variety of ethical issues when they enter clinical practice. A previous survey suggested that they lack knowledge about how to approach several types of medical ethics dilemmas. We developed a medical ethics curriculum for orthopaedic surgery residents and presented it over a one-year period to the residents in one training program. The effect of the educational intervention on the residents' knowledge of medical ethics and their ability to handle hypothetical situations was measured by comparing their responses to a questionnaire, administered before and after the intervention, with those of residents in a training program in which the intervention was not provided. The twenty-five residents at the site of the educational intervention had a mean improvement of 0.10 in the overall score, from a mean score of 0.71 on the baseline survey to a mean score of 0.81 on the follow-up survey. This improvement was significantly greater than the mean improvement of 0.02 for the thirty residents at the control site, who had a mean score of 0.72 on the baseline survey and a mean score of 0.74 on the follow-up survey (p = 0.002). Six residents who participated in the medical ethics curriculum rated it as very useful; seventeen, as somewhat useful; one, as slightly useful; and one, as not at all useful. A medical ethics curriculum can increase orthopaedic residents' knowledge of medical ethics. Whether this curriculum also will lead to behavioral changes requires additional evaluation.