Ottawa 2020 consensus statements for programmatic assessment - 2. Implementation and practice

INTRODUCTION

Programmatic assessment is a longitudinal, developmental approach that fosters and harnesses the learning function of assessment. Yet the implementation, a critical step to translate theory into practice, can be challenging. As part of the Ottawa 2020 consensus statement on programmatic assessment, we sought to provide descriptions of the implementation of the 12 principles of programmatic assessment and to gain insight into enablers and barriers across different institutions and contexts.

METHODS

After the 2020 Ottawa conference, we surveyed 15 Health Profession Education programmes from six different countries about the implementation of the 12 principles of programmatic assessment. Survey responses were analysed using a deductive thematic analysis.

RESULTS AND DISCUSSION

A wide range of implementations were reported although the principles remained, for the most part, faithful to the original enunciation and rationale. Enablers included strong leadership support, ongoing faculty development, providing students with clear expectations about assessment, simultaneous curriculum renewal and organisational commitment to change. Most barriers were related to the need for a paradigm shift in the culture of assessment. Descriptions of implementations in relation to the theoretical principles, across multiple educational contexts, coupled with explanations of enablers and barriers, provided new insights and a clearer understanding of the strategic and operational considerations in the implementation of programmatic assessment. Future research is needed to further explore how contextual and cultural factors affect implementation.

Ottawa 2020 consensus statement for programmatic assessment - 1. Agreement on the principles

INTRODUCTION

In the Ottawa 2018 Consensus framework for good assessment, a set of criteria was presented for systems of assessment. Currently, programmatic assessment is being established in an increasing number of programmes. In this Ottawa 2020 consensus statement for programmatic assessment insights from practice and research are used to define the principles of programmatic assessment.

METHODS

For fifteen programmes in health professions education affiliated with members of an expert group (n = 20), an inventory was completed for the perceived components, rationale, and importance of a programmatic assessment design. Input from attendees of a programmatic assessment workshop and symposium at the 2020 Ottawa conference was included. The outcome is discussed in concurrence with current theory and research.

RESULTS AND DISCUSSION

Twelve principles are presented that are considered as important and recognisable facets of programmatic assessment. Overall these principles were used in the curriculum and assessment design, albeit with a range of approaches and rigor, suggesting that programmatic assessment is an achievable education and assessment model, embedded both in practice and research. Knowledge on and sharing how programmatic assessment is being operationalized may help support educators charting their own implementation journey of programmatic assessment in their respective programmes.

The alignment imperative in curriculum renewal

RATIONALE

There are perennial calls for MD curricula to reform in order to meet the changing needs of students, patients, and society. And yet, efforts at renewal have also been suggested to have minimal impact on the pedagogy and outcomes of medical education. One reason may be misalignment between the components of the curriculum during design and implementation. The University of Toronto MD program recently renewed its undergraduate preclinical Foundations curriculum. Mindful of the pitfalls of misalignment, the renewal process focused deliberately on alignment between the various components of the curriculum: instructional methods, student assessment, faculty development, and the larger purpose of serving students and society.

INNOVATION

Educational evidence was used to drive the alignment process which resulted in three major changes. First, we created a spiral curriculum centered on 72 virtual patient cases designed to integrate content and prepare students for clinical learning. Second, we introduced a novel medical psychiatry component to address a core societal need in mental health. This exposed students early to experiences of complexity, ambiguity, and integrated patient care. Lastly, a shift to assessment for learning and programmatic assessment was designed and implemented concurrently to reinforce the pedagogy of the curriculum. Synchronous faculty development was developed for the new roles required of faculty.

CONCLUSIONS

Early program evaluation shows alignment of these curricular components requires ongoing attention and resources in order to be successful. The potential benefits of this alignment are well prepared students who can meet the needs of their patients and society in an increasingly complex health system.

Pulling together and pulling apart: influences of convergence and divergence on distributed healthcare teams

Effective healthcare requires both competent individuals and competent teams. With this recognition, health professions education is grappling with how to factor team competence into training and assessment strategies. These efforts are impeded, however, by the absence of a sophisticated understanding of the the relationship between competent individuals and competent teams . Using data from a constructivist grounded theory study of team-based healthcare for patients with advanced heart failure, this paper explores the relationship between individual team members' perceived goals, understandings, values and routines and the collective competence of the team. Individual interviews with index patients and their healthcare team members formed Team Sampling Units (TSUs). Thirty-seven TSUs consisting of 183 interviews were iteratively analysed for patterns of convergence and divergence in an inductive process informed by complex adaptive systems theory. Convergence and divergence were identifiable on all teams, regularly co-occurred on the same team, and involved recurring themes. Convergence and divergence had nonlinear relationships to the team's collective functioning. Convergence could foster either shared action or collective paralysis; divergence could foster problematic incoherence or productive disruption. These findings advance our understanding of the complex relationship between the individual and the collective on a healthcare team, and they challenge conventional narratives of healthcare teamwork which derive largely from acute care settings and emphasize the importance of common goals and shared mental models. Complex adaptive systems theory helps us to understand the implications of these insights for healthcare teams' delivery of care for the complex, chronically ill.

'Who is on your health-care team?' Asking individuals with heart failure about care team membership and roles

BACKGROUND

Complex, chronically ill patients require interprofessional teams to address their multiple health needs; heart failure (HF) is an iconic example of this growing problem. While patients are the common denominator in interprofessional care teams, patients have not explicitly informed our understanding of team composition and function. Their perspectives are crucial for improving quality, patient-centred care.

OBJECTIVES

To explore how individuals with HF conceptualize their care team, and perceive team members' roles.

SETTING AND PARTICIPANTS

Individuals with advanced HF were recruited from five cities in three Canadian provinces.

DESIGN

Individuals were asked to identify their HF care team during semi-structured interviews. Team members' titles and roles, quotes pertaining to team composition and function, and frailty criteria were extracted and analysed using descriptive statistics and content analysis.

RESULTS

A total of 62 individuals with HF identified 2-19 team members. Caregivers, nurses, family physicians and cardiologists were frequently identified; teams also included dentists, foot care specialists, drivers, housekeepers and spiritual advisors. Most individuals met frailty criteria and described participating in self-management.

DISCUSSION

Individuals with HF perceived being active participants, not passive recipients, of care. They identified teams that were larger and more diverse than traditional biomedical conceptualizations. However, the nature and importance of team members' roles varied according to needs, relationships and context. Patients' degree of agency was negotiated within this context, causing multiple, sometimes conflicting, responses.

CONCLUSION

Ignoring the patient's role on the care team may contribute to fragmented care. However, understanding the team through the patient's lens - and collaborating meaningfully among identified team members - may improve health-care delivery.

Adaptive practices in heart failure care teams: implications for patient-centered care in the context of complexity

BACKGROUND

Heart failure (HF), one of the three leading causes of death, is a chronic, progressive, incurable disease. There is growing support for integration of palliative care's holistic approach to suffering, but insufficient understanding of how this would happen in the complex team context of HF care. This study examined how HF care teams, as defined by patients, work together to provide care to patients with advanced disease.

METHODS

Team members were identified by each participating patient, generating team sampling units (TSUs) for each patient. Drawn from five study sites in three Canadian provinces, our dataset consists of 209 interviews from 50 TSUs. Drawing on a theoretical framing of HF teams as complex adaptive systems (CAS), interviews were analyzed using the constant comparative method associated with constructivist grounded theory.

RESULTS

This paper centers on the dominant theme of system practices, how HF care delivery is reported to work organizationally, socially, and practically, and describes two subthemes: "the way things work around here", which were commonplace, routine ways of doing things, and "the way we make things work around here", which were more conscious, effortful adaptations to usual practice in response to emergent needs. An adaptive practice, often a small alteration to routine, could have amplified effects beyond those intended by the innovating team member and could extend to other settings.

CONCLUSION

Adaptive practices emerged unpredictably and were variably experienced by team members. Our study offers an empirically grounded explanation of how HF care teams self-organize and how adaptive practices emerge from nonlinear interdependencies among diverse agents. We use these insights to reframe the question of palliative care integration, to ask how best to foster palliative care-aligned adaptive practices in HF care. This work has implications for health care's growing challenge of providing care to those with chronic medical illness in complex, team-based settings.

Residents learning from a narrative experience with dying patients: a qualitative study

For patients at the end of life, it is crucial to address the psychological, existential, and spiritual distress of patients. Medical education research suggests trainees feel unprepared to provide the whole person, humanistic care held as the ideal. This study used an empirically based narrative intervention, the dignity interview, as an educational intervention with first year residents. The interview helps patients tell and make meaning of their life story. The intervention was aimed at addressing trainee perceived gaps in the non-physical aspects of end-of-life care. It was also intended to stimulate broader reflection on lessons learned in medical education about the value of narrative as part of humanistic care. Twelve first year residents administered a 1 h interview to dying patients. The resident returned to read the transcribed story back to the patient. Semi-structured interviews of the residents were transcribed and analyzed using the constant comparative method to identify emergent themes. This experience was seen as distinct from the 'traditional" medical interview. Residents reflected on lessons learned from patients and on their own professional and personal lives. Residents felt conversations with dying patients, and more broadly the art of soliciting a patient's story are poorly taught and modeled. More concerning, the hidden curriculum seems to be sending messages that learning a patient's story is not the domain of a physician and that it is not valued like the curing and technical imperatives. These findings have implications for medical education's ongoing attempts to better produce humanistic physicians.

Understanding palliative care on the heart failure care team: an innovative research methodology

CONTEXT

There is a growing call to integrate palliative care for patients with advanced heart failure (HF). However, the knowledge to inform integration efforts comes largely from interview and survey research with individual patients and providers. This work has been critically important in raising awareness of the need for integration, but it is insufficient to inform solutions that must be enacted not by isolated individuals but by complex care teams. Research methods are urgently required to support systematic exploration of the experiences of patients with HF, family caregivers, and health care providers as they interact as a care team.

OBJECTIVES

To design a research methodology that can support systematic exploration of the experiences of patients with HF, caregivers, and health care providers as they interact as a care team.

METHODS

This article describes in detail a methodology that we have piloted and are currently using in a multisite study of HF care teams.

RESULTS

We describe three aspects of the methodology: the theoretical framework, an innovative sampling strategy, and an iterative system of data collection and analysis that incorporates four data sources and four analytical steps.

CONCLUSION

We anticipate that this innovative methodology will support groundbreaking research in both HF care and other team settings in which palliative integration efforts are emerging for patients with advanced nonmalignant disease.

End-of-life care education for psychiatric residents: attitudes, preparedness, and conceptualizations of dignity

OBJECTIVE

The authors examined psychiatric residents' attitudes, perceived preparedness, experiences, and needs in end-of-life care education. They also examined how residents conceptualized good end-of-life care and dignity.

METHODS

The authors conducted an electronic survey of 116 psychiatric residents at the University of Toronto. The survey had a mix of qualitative and quantitative questions.

RESULTS

Eighty-two of 116 invited psychiatric residents participated for a response rate of 71%. With favorable attitudes, residents felt least prepared in existential, spiritual, cultural, and some psychological aspects of caring for dying patients. Trainees conceptualized dignity at the end of life in a way very similar to that of patients, including concerns of the mind, body, soul, relationships, and autonomy. Residents desired more longitudinal, contextualized training, particularly in the psychosocial, existential, and spiritual aspects of care.

CONCLUSION

This is the first study to examine the end-of-life educational experience of psychiatric residents. Despite conceptualizing quality care and the construct of dignity similarly to dying patients, psychiatric residents feel poorly prepared to deliver such care, particularly the nonphysical aspects of caring for the dying. These results will inform curriculum development in end-of-life care for psychiatric residents, a complex area now considered a core competency.

The research innovation and scholarship in education program: an innovative way to nurture education

OBJECTIVE

Education is becoming a recognized career path in psychiatry. Yet, there are few published accounts of how to create sustainable structures within departments to support this academic focus. The authors document the creation and 5-year progress of the Research Innovation and Scholarship in Education (RISE) program at the largest psychiatry department in Canada.

METHODS

The authors analyzed the RISE archive of early proposals for enhancing scholarship in the department, the 5-year plan, annual reports, and curricular vitae of members and also gathered testimonials from inaugural residents and fellows of the program. Materials were analyzed using Boyer's framework of scholarship.

RESULTS

Organizationally, RISE has embodied all four tenets of Boyer's model of scholarship. The program has allowed education research, teaching, and creative professional development to flourish in the department, and there are considerably fewer barriers to pursuing an education career path. However, as the program expands, more work needs to be done to increase funding and protected time so that even more residents, fellows, and faculty can engage in educational scholarship.

CONCLUSION

Enhancing medical education scholarship through a model that actively integrates research with teaching, creative professional development, and mentorship can help the trajectories of faculty and students wishing to make education a priority in their careers.

Five year review of the Rotaglide total knee arthroplasty

In this study we present the outcome for patients with the Rotaglide mobile meniscal knee prosthesis implanted for osteoarthritis. All patients reviewed had this prosthesis implanted as a primary total knee Arthroplasty in Crosshouse Hospital. The minimum follow-up period was 5 years (range 5-8.2). Patients were assessed clinically by the junior author (C.W.) and results were standardised using the Hospital for Specialist Surgery (HSS) knee score. Standard radiographs were taken in antero-posterior and lateral planes to assess for loosening using the Knee Society roentgenographic system. Case notes were then examined for evidence of peri and post-operative problems. Sixty-seven patients (71 knees) were reviewed. Ninety-four percent of patients had an excellent clinical outcome with HSS scores of 85 or more. Two knees were revised, one for meniscal fracture and one for meniscal dislocation. Both of these failures were early in our series and in a total of 312 knees to date there have been no other meniscal failures. No knees were revised for aseptic loosening and there have been no deep infections. We feel this prosthesis offers a safe and effective treatment for osteoarthritis with a good clinical outcome at 5 years with a low level of complications.

Pentagastrin-induced release of free fatty acids in healthy volunteers and patients with panic disorder: effect of pretreatment with ethinyl estradiol

OBJECTIVE

The primary objective of this study was to assess whether pentagastrin-induced panic symptoms are associated with release of free fatty acids (FFAs) in a manner that could explain the mechanism of correlations observed between serum cholesterol levels and frequency and severity of panic attacks in patients with panic disorder (PD). A secondary objective was to assess whether pretreatment with ethinyl estradiol (EE) attenuates pentagastrin-induced release of FFAs.

METHODS

A double-blind, crossover, placebo-controlled study was conducted in which patients with PD and healthy volunteers received 2 injections of pentagastrin, 7-10 days apart, with randomization of the order of pretreatment with placebo and EE.

RESULTS

We found a statistically significant, time-dependent release of FFAs in response to pentagastrin challenge. However, this release of FFAs was not attenuated by pretreatment with EE.

CONCLUSIONS

These results support the hypothesis that release of FFAs in association with panic attacks occurs in a manner similar to the stress-induced lipolysis model. This suggests a possible mechanism for the elevated serum cholesterol levels observed in patients with PD. However, the occurrence of a delayed increase in low-density lipoprotein (LDL) cholesterol following induction of a panic attack remains to be tested in studiesincorporating a placebo injection visit and timed measurements of LDL cholesterol.

Neuroactive steroid changes in response to challenge with the panicogenic agent pentagastrin

BACKGROUND

Female hormones and female hormone derivatives, including neuroactive steroids (NASs) have been suspected to play a role in the pathophysiology of panic disorder (PD). The panicogenic agent CO(2) has been shown to induce a delayed release of NASs in both brain and plasma of rats. In the present study, we measured NASs plasma levels in response to challenge with another panicogenic agent, pentagastrin, and assessed the effect of ethynil estradiol (EE) pretreatment.

METHODS

A double-blind cross-over placebo-controlled design with randomization of the order of a three day pretreatment of EE (50 microg/day) or placebo was used to assess the effect of a 30 microg iv bolus injection of pentagastrin on the release of allopregnanolone (ALLO) and dehydroepiandrosterone (DHEA) into plasma in 15 male PD patients and 10 male healthy volunteers (HV).

RESULTS

After pentagastrin challenge there was a significant release of DHEA and a trend for the release of ALLO. EE pretreatment did not affect the pentagastrin-induced panic response or NAS release.

CONCLUSIONS

Pentagastrin induced release of NASs into plasma, the purpose of which remains to be determined.

Utilisation of bone allograft by orthopaedic surgeons in Scotland

The use of bone allograft in orthopaedic surgery has been predicted to increase, particularly in joint revision surgery. This has led to a potential problem with supply. Questionnaires were distributed to all 146 Consultant Orthopaedic surgeons working in Scotland in 2000. They were asked to indicate their current usage of bone and tissue allograft, any problems encountered with supply and if alternatives to allograft, such as processed bone, might be used. The questions asked were very similar to those asked in a previous study in 1995 to enable comparisons to be made. Replies were received from 125 Consultants (87%) of whom 93 reported using bone allograft. Forty-one consultants (46%) predicted an increase in their requirement for bone allograft, and 23 (26%) felt they could currently use more bone if this was available. Sixty percent of surgeons would consider using processed bone as an alternative. In comparison with figures from 1995, an increasing number of surgeons are prepared to use processed bone as an alternative to fresh frozen allograft.

Effect of ethinyl estradiol on the panic response to the panicogenic agent pentagastrin

BACKGROUND

Panic disorder (PD) symptomatology has been reported to be altered by hormonal events or treatments which affect estrogen levels. Coryell et al. [Arch. Gen. Psychiatry, 39 (1982) 701-703; Am. J. Psychiatry, 143 (1986) 508-510] have suggested that the increased cardiovascular risk associated with PD is significantly greater in males, alluding to a potential cardioprotective effect of female hormones in the context of panic attacks. In the present study, we were, therefore, interested in elucidating the role of estrogen in modulating the behavioural and cardiovascular responses induced by the panicogenic agent pentagastrin, a cholecystokinin-B (CCK(B)) receptor agonist.

METHODS

A double-blind cross-over placebo-controlled design with randomization of the order of a 3-day pretreatment of ethinyl estradiol (EE) (50 microg/day) or placebo was used to assess the effect of a 30-microg i.v. bolus injection of pentagastrin on panic symptom intensity and on increases in heart rate (DeltaHR), systolic (DeltaSBP) and diastolic (DeltaDBP) blood pressure following each pretreatment. Subjects were 9 male healthy controls and 11 male PD patients.

RESULTS

EE pretreatment did not significantly reduce the pentagastrin-induced panic symptom scale (PSS) scores and had no effect on DeltaDBP or DeltaSBP. EE did, however, attenuate the pentagastrin-induced increase in HR in both PD patients and healthy controls.

LIMITATIONS

Only male subjects were included in the present study; however, we are currently investigating the influence of female gonadal hormones on the panic response to pentagastrin in female PD patients and healthy controls.

CONCLUSION

Our results suggest that estrogens may display cardioprotective effects in the context of panic attacks.

Effect of medroxyprogesterone pretreatment on pentagastrin-induced panic symptoms in females with panic disorder

Clinical observation, as well as epidemiological and research data, suggest that female gonadal hormones influence the course of panic disorder (PD). Panicogenic agents such as pentagastrin are useful tools with which to study the pathophysiology of panic attacks. Nine women with PD were randomly assigned to receive, in a crossover design, a 3-day pretreatment with medroxyprogesterone acetate (MP) prior to an injection of pentagastrin, and a 3-day pretreatment with a placebo prior to another injection of pentagastrin. The panic response and the anxiety response to pentagastrin were decreased after MP pretreatment. These preliminary results support the use of laboratory models for investigations of the interactions between progestins and anxiety.

Caveat surgeon: do orthopaedic surgeons take adequate precautions against blood-borne viral infections, in particular the human immunodeficiency virus (HIV)?

One year ago the British Orthopaedic Association issued guidelines for the prevention of cross-infection with special reference to HIV and the hepatitis viruses. We were interested to establish whether the guidelines were being widely applied and whether they had changed general orthopaedic practice. We distributed a questionnaire to Scottish Orthopaedic Fellows, Associates of the BOA, and orthopaedic trainees. With a 70 per cent return rate, it would appear that the recommendations are not adhered to in full. Of respondents, 84 per cent were immunized or undergoing immunization against hepatitis B. In all, 30 per cent were operating on high-risk patients on a monthly basis, 60 per cent thought that their current practice was low risk, and only 15 per cent thought that their future practice would be high risk; 81 per cent were concerned and yet only 60 per cent had altered their practice. It is of some concern that orthopaedic surgeons may not take the threat of HIV cross-infection seriously enough and do not consider precautions mandatory. Further pressure and support from the BOA may be necessary to encourage a change in orthopaedic practice as the threat of HIV is increasing.

Unnecessary arthroscopies

A review of the results of arthroscopies on patients taken from an 'open access' service waiting list has been carried out. One-third of all arthroscopies proved to be negative. Accuracy of clinical diagnosis was not related to the clinical grade of the examining doctor. The excess annual cost to the Northern Ireland Orthopaedic Service of unnecessary arthroscopies may have been of the order of 89,000 pounds.

Delayed diagnosis of injuries from the M1 aircraft accident. The Nottingham, Leicester, Derby, Belfast Study Group

The incidence of a delay in the diagnosis of injuries from the 83 survivors of the M1 aircraft accident is described. A total of 32 injuries in 25 patients were not initially diagnosed, nine each in the upper and lower limbs and 14 in the spine. This was equivalent to 9.6 per cent of all the major injuries suffered by the survivors. Five patients required surgery for a late diagnosed injury. The incidence of a delayed diagnosis injury (DDI) was not related to the overall severity of injury. Delay in diagnosis of injuries after such a major accident was attributable to failure of clinical examination, failure to radiograph symptomatic areas and failure of radiographic interpretation.

Survivors of the M1 aircrash. Outcome of injuries after 1 year. The Nottingham, Leicester, Derby and Belfast Study Group

The 79 survivors of the M1 aircrash have been assessed at an average of great britain 1 year after the accident. Return to work, continued hospital care and recovery from injuries has been found to be related to the severity of injury. The 188 pelvic and limb fractures have been reviewed and the incidence of complications from these injuries has been low. Spinal and lower limb injuries account for the majority of the residual disability.

Initial management of open fractures sustained in the M1 aircraft disaster. Nottingham, Leicester, Derby and Belfast (NLDB) Study Group

There were 83 survivors of the M1 aircrash admitted to hospital; of these, 28 patients sustained a total of 40 open fractures. These open fractures were classified as 19 grade 3, 16 grade 2 and five grade 1. There were six upper limb fractures and 34 lower limb fractures. A high proportion of grade 3 open fractures were identified in this series (47 per cent) and one-half of these occurred at or below the ankle. This reflected the high energy and direction of the forces involved in the aircrash. The early results in terms of the incidence of wound infection (15 per cent), delayed wound healing (7.5 per cent) and skin flap necrosis (7.5 per cent) were similar to other series. The controversy over whether to leave grade 3 wounds open or closed was not clarified by this study, but the closure of grade 3 wounds after internal fixation is to be avoided. This study emphasizes the need for provision of adequate specialist manpower, equipment and resources to manage the unexpected major disaster successfully.

Simultaneous bilateral fractures of the femoral neck

An 80-year-old woman suffered bilateral simultaneous fractures of her hips. This is the first reported case in the elderly. Both hips were replaced by Thompson hemi-arthroplasties at the same operation, and the patient was discharged home.

Isolated dislocation of the radial head: a report of two cases

Two cases of isolated dislocation of the radial head in children are presented. The clinical presentation, radiographic appearances, management and prognosis are discussed, and a brief literature review is presented.