Generalizability of the Ottawa Surgical Competency Operating Room Evaluation (O-SCORE) Scale to Assess Medical Student Performance on Core EPAs in the Workplace: Findings From One Institution

PURPOSE

Assessment of the Core Entrustable Professional Activities for Entering Residency (Core EPAs) requires direct observation of learners in the workplace to support entrustment decisions. The purpose of this study was to examine the internal structure validity evidence of the Ottawa Surgical Competency Operating Room Evaluation (O-SCORE) scale when used to assess medical student performance in the Core EPAs across clinical clerkships.

METHOD

During the 2018-2019 academic year, the Virginia Commonwealth University School of Medicine implemented a mobile-friendly, student-initiated workplace-based assessment (WBA) system to provide formative feedback for the Core EPAs across all clinical clerkships. Students were required to request a specified number of Core EPA assessments in each clerkship. A modified O-SCORE scale (1 = "I had to do" to 4 = "I needed to be in room just in case") was used to rate learner performance. Generalizability theory was applied to assess the generalizability (or reliability) of the assessments. Decision studies were then conducted to determine the number of assessments needed to achieve a reasonable reliability.

RESULTS

A total of 10,680 WBAs were completed on 220 medical students. The majority of ratings were completed on EPA 1 (history and physical) (n = 3,129; 29%) and EPA 6 (oral presentation) (n = 2,830; 26%). Mean scores were similar (3.5-3.6 out of 4) across EPAs. Variance due to the student ranged from 3.5% to 8%, with the majority of the variation due to the rater (29.6%-50.3%) and other unexplained factors. A range of 25 to 63 assessments were required to achieve reasonable reliability (Phi > 0.70).

CONCLUSIONS

The O-SCORE demonstrated modest reliability when used across clerkships. These findings highlight specific challenges for implementing WBAs for the Core EPAs including the process for requesting WBAs, rater training, and application of the O-SCORE scale in medical student assessment.

Improving Passage Rate on USMLE Step 2 Clinical Skills: Results from a Pilot Program

In the United States (US), successful passage of United States Medical Licensing Examination (USMLE) Step 2 Clinical Skills (Step 2 CS) is required to enter into residency training. In 2017, the USMLE announced an increase in performance standards for Step 2 CS. As a consequence, it is anticipated that the passage rate for the examination will decrease significantly for both US and international students. While many US institutions offer a cumulative clinical skills examination, their effect on Step 2 CS passage rates has not been studied. The authors developed a six-case, standardized patient (SP)-based examination to mirror Step 2 CS and measured impact on subsequent Step 2 CS passage rates. Students were provided structured quantitative and qualitative feedback and were given a final designation of "pass" or "fail" for the practice examination. A total of 173 out of 184 (94.5%) students participated in the examination. Twenty SPs and $26,000 in direct costs were required. The local failure rate for Step 2 CS declined from 4.5% in the year proceeding the intervention to 2.1% following the intervention. In the same timeframe, the US failure rate for Step 2 CS increased from 3.8 to 5.1%, though the difference between local and national groups was not significantly different (P = .07). Based on the initial success of the intervention, educational leaders may consider developing a similar innovation to optimize passage rates at their institutions.

Are Scores From NBME Subject Examinations Valid Measures of Knowledge Acquired During Clinical Clerkships?

PURPOSE

The National Board of Medical Examiners' Clinical Science Subject Examinations are a component used by most U.S. medical schools to determine clerkship grades. The purpose of this study was to examine the validity of this practice.

METHOD

This was a retrospective cohort study of medical students at the Virginia Commonwealth University School of Medicine who completed clerkships in 2012 through 2014. Linear regression was used to determine how well United States Medical Licensing Examination Step 1 scores predicted Subject Examination scores in seven clerkships. The authors then substituted each student's Subject Examination standard scores with his or her Step 1 standard score. Clerkship grades based on the Step 1 substitution were compared with actual grades with the Wilcoxon rank test.

RESULTS

A total of 2,777 Subject Examination scores from 432 students were included in the analysis. Step 1 scores significantly predicted between 23% and 44% of the variance in Subject Examination scores, P < .001 for all clerkship regression equations. Mean differences between expected and actual Subject Examination scores were small (≤ 0.2 points). There was a match between 73% of Step 1 substituted final clerkship grades and actual final clerkship grades.

CONCLUSIONS

The results of this study suggest that performance on Step 1 can be used to identify and counsel students at risk for poor performance on the Subject Examinations. In addition, these findings call into the question the validity of using scores from Subject Examinations as a high-stakes assessment of learning in individual clerkships.

Doing Duo - a case study of entrainment in William Forsythe's choreography "Duo"

Entrainment theory focuses on processes in which interacting (i.e., coupled) rhythmic systems stabilize, producing synchronization in the ideal sense, and forms of phase related rhythmic coordination in complex cases. In human action, entrainment involves spatiotemporal and social aspects, characterizing the meaningful activities of music, dance, and communication. How can the phenomenon of human entrainment be meaningfully studied in complex situations such as dance? We present an in-progress case study of entrainment in William Forsythe's choreography Duo, a duet in which coordinated rhythmic activity is achieved without an external musical beat and without touch-based interaction. Using concepts of entrainment from different disciplines as well as insight from Duo performer Riley Watts, we question definitions of entrainment in the context of dance. The functions of chorusing, turn-taking, complementary action, cues, and alignments are discussed and linked to supporting annotated video material. While Duo challenges the definition of entrainment in dance as coordinated response to an external musical or rhythmic signal, it supports the definition of entrainment as coordinated interplay of motion and sound production by active agents (i.e., dancers) in the field. Agreeing that human entrainment should be studied on multiple levels, we suggest that entrainment between the dancers in Duo is elastic in time and propose how to test this hypothesis empirically. We do not claim that our proposed model of elasticity is applicable to all forms of human entrainment nor to all examples of entrainment in dance. Rather, we suggest studying higher order phase correction (the stabilizing tendency of entrainment) as a potential aspect to be incorporated into other models.

De novo generalized periodic discharges related to anesthetic withdrawal resolve spontaneously

Pentobarbital and propofol are used for the treatment of refractory status epilepticus or elevated intracranial pressure, typically with continuous EEG monitoring. We report a series of patients who developed generalized periodic discharges related to anesthetic withdrawal (GRAWs), different from previous seizure activity. At times, this pattern was misinterpreted as recurrent seizure activity, leading to reinstitution of drug-induced coma, but resolved spontaneously without additional treatment.We identified five patients who developed GRAWs during pentobarbital or propofol withdrawal. Two patients received pentobarbital for increased intracranial pressure. One patient received pentobarbital and propofol for encephalopathy accompanied by a rhythmic EEG pattern erroneously thought to be ictal. Two patients received pentobarbital for refractory partial status epilepticus. In all cases, anesthetic agents were withdrawn after 24 to 48 hours of burst suppression on EEG. We analyzed the course of GRAWs on EEG and the associated clinical outcomes.All five patients developed GRAWs, consisting of periodic 1 to 4 Hz generalized periodic discharge, not previously seen on EEG. In all cases, the pattern eventually resolved spontaneously, over 12 to 120 hours. However, in three cases, the pattern was initially thought to represent ictal activity, and drug-induced coma was reinitiated. The pattern recurred during repeated anesthetic withdrawal, was then recognized as nonictal, and then resolved without further treatment. In all cases but one, the patients exhibited improvement to near-baseline mentation.Generalized periodic discharges related to anesthetic withdrawal may occur de novo after pentobarbital or propofol withdrawal. They should resolve spontaneously without treatment and without recurrence of clinical seizure activity. However, GRAWs are not likely to represent status epilepticus and should not prompt resumption of drug-induced coma, unless there is reappearance of original electrographic seizure activity.

Status epilepticus

Status epilepticus (SE) is one of the most commonly occurring neurologic emergencies. About 40% of SE cases occur in people with epilepsy. Convulsive SE is easily recognized, but nonconvulsive SE is not and requires both a high index of suspicion and EEG confirmation. SE has a high mortality risk and requires rapid effective treatment for optimal response to therapy and outcome. The goal of treatment is to stop all clinical and electrographic seizures while maintaining vital functions. If seizures continue after initial treatment with a benzodiazepine, additional antiepileptic therapy should be administered. When SE is refractory to these treatments, continuous IV infusion with midazolam, propofol, or a barbiturate suppresses seizure activity. Standard treatment protocols are useful in promoting rapid intervention with appropriate medications.

Aberrant drug-taking behaviors and headache: patient versus physician report

OBJECTIVE

To explore prevalence of aberrant medication-taking behaviors (AMTB) among headache patients and treating physician's awareness of such behaviors.

METHODS

Fifty patientphysician dyads were surveyed on patients' AMTB.

RESULTS

The most frequently endorsed behaviors by patients and physicians, respectively, were going to the ER for pain medication (n = 19) and continuing to take pain medication despite minimal relief (n = 23). For the majority of AMTB, phi coefficients indicating level of patient-physician agreement were equal to chance.

CONCLUSIONS

Headache patients perform a wide range of AMTB. Low rates of patient-physician agreement indicate that physicians possess limited knowledge of patients' AMTB.

Seizures in the elderly: Nuances in presentation and treatment

Acute symptomatic seizures and epilepsy are two of the most common neurologic complaints in the elderly. Stroke is the leading underlying etiology for both. Because clinical seizure manifestations in the elderly often differ from those in younger adults, they may be difficult to recognize or may be misdiagnosed. Interpretation of diagnostic tests in elderly patients with seizures is often complicated by comorbidities, and treatment decisions require careful consideration in the context of age-related physiologic changes, comorbidities, and the use of concomitant medications. Treatment of an acute seizure with a clear precipitating cause involves correcting the underlying etiology; antiepileptic drug (AED) therapy is generally reserved for patients with epilepsy (recurrent unprovoked seizures). The prognosis for elderly epilepsy patients treated with AEDs is generally good. Both older and newer AEDs are efficacious but have respective advantages and disadvantages; no ideal AED yet exists. Status epilepticus is a neurologic emergency that is particularly frequent in the elderly and associated with high mortality, although treatment can be effective.

Management of status epilepticus

Status epilepticus is an increasingly recognized public health problem in the United States. Status epilepticus is associated with a high mortality rate that is largely contingent on the duration of the condition before initial treatment, the etiology of the condition, and the age of the patient. Treatment is evolving as new medications become available. Three new preparations--fosphenytoin, rectal diazepam, and parenteral valproate--have implications for the management of status epilepticus. However, randomized controlled trials show that benzodiazepines (in particular, diazepam and lorazepam) should be the initial drug therapy in patients with status epilepticus. Despite the paucity of clinical trials comparing medication regimens for acute seizures, there is broad consensus that immediate diagnosis and treatment are necessary to reduce the morbidity and mortality of this condition. Moreover, investigators have reported that status epilepticus often is not considered in patients with altered consciousness in the intensive care setting. In patients with persistent alteration of consciousness for which there is no clear etiology, physicians should be more quickly prepared to obtain electroencephalography to identify status epilepticus. Physicians should rely on a standardized protocol for management of status epilepticus to improve care for this neurologic emergency.

New horizons in ambulatory electroencephalography

Since its inception 30 years ago, AEEG has continued to evolve--from four-channel tape recorders to 32-channel digital recorders with sophisticated automatic spike and seizure detection algorithms. AEEG remains an important tool in epilepsy evaluation. In the near future, smaller, faster, and more sophisticated AEEGs will be developed. Seizure detection/anticipation systems will allow the wearer to be forewarned of a seizure so that appropriate safety measures can be taken. With further refinement in our understanding of nonlinear dynamic analysis to define the pre-ictal state, AEEG will be coupled with an accurate seizure anticipation device in a closed-loop system, providing a time window during which therapeutic intervention can occur, to prevent a seizure. The therapeutic intervention will most likely involve vagus nerve or deep brain stimulation. An alternative is that the patient may learn to recognize early symptoms of the pre-ictal state and use behavioral biofeedback interventions to avoid a clinical seizure. In order to achieve convenient ambulatory recording and seizure detection that could realistically improve the lives of patients with refractory epilepsy, the process of miniaturization of such a device to a convenient size must be accomplished. One of the aspects of epilepsy that patients find most frustrating, and that most limits activities, is the vulnerability to sudden unexpected incapacitation due to the occurrence of a seizure. With miniaturization of AEEG and seizure anticipation technology, and advancements in our ability to identify the transition from pre-ictal to ictal state, there is realistic hope that patients with refractory epilepsy may gain control over their seizures and enjoy significantly improved quality of life.

Predictors of Early Seizures after Stroke

Prevalence and Predictors of Early Seizure and Status Epilepticus after First Stroke

Labovitz DL, Allen Hauser W, Sacco RL

Neurology 2001;57:200–206

Background

Early seizure (ES) has been reported in 2 to 6% of strokes and is a predictor of recurrent seizures. Acute stroke has been reported to cause 22% of all cases of status epilepticus (SE) in adults. The determinants of ES and SE after stroke, however, are not well understood.

Methods

An incidence study was conducted to identify all cases of first stroke in adult residents of northern Manhattan. Cases of ES and SE within 7 days of stroke were identified through medical record review. Statistical analyses were performed by using univariate and multivariate logistic regression models.

Results

The cohort consisted of 904 patients; ES occurred in 37 (4.1%). The frequency of ES by stroke subtype and location was deep infarct in two (0.6%) of 356, lobar infarct in 20 (5.9%) of 341, deep intracerebral hemorrhage (ICH) in four (4.0%) of 101, lobar ICH in seven (14.3%) of 49, and subarachnoid hemorrhage in four (8.0%) of 50. SE occurred in 10 (1.1%) patients, representing 27.0% of patients with ES. Diabetes, hypertension, current smoking, alcohol use, age, gender, and race/ethnicity were not significant determinants of ES. In a subgroup of patients who had a National Institutes of Health (NIH) stroke scale (NIHSS) score recorded, NIHSS score was not an independent predictor of ES in multivariate analysis. After accounting for stroke severity, ES was not a predictor of 30-day case fatality.

Conclusions

Lesion location and stroke subtype are strong determinants of ES risk, even after adjusting for stroke severity. ES does not predict 30-day mortality. SE occurs in more than one fourth of patients with ES.

Status Epilepticus in Acute Settings

Status Epilepticus Arising De Novo in Hospitalized Patients: An Analysis of 41 Patients

Delanty N, French JA, Labar DR, Pedley TA, Rowan AJ

Seizure 2001;10: 116–119

Most of the information on predisposing factors and mortality in status epilepticus (SE) arises from data obtained from patients presenting to the casualty department. However, another population which is frequently seen by consultative neurologists are medically ill patients who develop SE while in hospital. These patients are often notoriously difficult to treat once SE arises. We sought to characterize patients at risk for SE arising when they are hospitalized for other reasons. By doing this, risk factors for developing SE and prognostic indicators might be determined. We retrospectively reviewed records from three urban hospitals in the United States to identify hospitalized patients developing SE over a 1 year period. SE was defined as a clinical seizure lasting 30 minutes or longer, or repeated seizures without recovery. Patients who were admitted in SE or for an epilepsy-related problem, or who were less than 1 year old were excluded from the study. Forty-one patients with in-hospital SE were identified. There were 28 males and 13 females with an age range from 1 to 91 years (mean: 60 years, median: 65 years). The mean interval from hospital admission to the onset of status epilepticus was 26 days. Nineteen (46%) patients had a prior history of either epilepsy or symptomatic seizures, and of these, 10 were inadequately treated as judged by serum anticonvulsant levels at the time SE developed. Focal brain abnormality was present in 26 (63%) patients, the most common of which was stroke (17 patients). Major metabolic derangements including hypoxia, electrolyte imbalance, hepatic encephalopathy, and sepsis were present in 23 (56%) patients. Eleven (27%) patients were being treated with theophylline preparations at the time SE developed. Mortality in this group of patients with in-hospital SE was 61% (25 deaths), with about one-third dying while in status, and two-thirds dying subsequently in hospital. In this retrospective study, there was no clear relationship between mortality and the duration of SE in this group of patients. In-hospital development of SE is usually related to underlying focal brain abnormality, especially stroke, in combination with systemic metabolic derangement. Prognosis is poor, and appears to be more related to underlying conditions rather than to status duration. More accurate prospective studies are warranted.

Status Epilepticus After Stroke

Velioglu SK, Ozmenoglu M, Boz C, Alioglu Z

Stroke 2001;32:1169–1172

Background and Purpose

Objective of our study was to determine the risk and predictive factors of status epilepticus (SE) after stroke.

Methods

From 1988 to 2000, 1174 patients were admitted to the Department of Neurology at the Karadeniz Technical University Farabi Hospital with first-time strokes. Of these, 180 patients had poststroke first-time seizures (PFSs). We followed these 180 PFS patients for an average of 3.7 years or until death to determine the occurrence rate of SE. By comparing these data with those of PFS patients without SE, we investigated whether there were significant differences.

Results

A total of 17 of the 180 PFS patients (9%) had SE. There was no relationship between the occurrence of SE and stroke risk factors, stroke type (ischemic or hemorrhagic stroke), stroke topography and cause, cortical involvement, size of lesion, seizure type, or electroencephalographic findings. SE occurred more frequently among patients with a higher disability rating (Rankin scale >3; odds ratio, 4.36). Recurrent SE was identified in 5 of 17 patients with SE. In all 5 of these patients, the first episode of SE occurred within the first 7 days after stroke (early-onset SE). Statistical analysis demonstrated that early-onset SE was associated with a higher risk for SE recurrence (P=0.003) and a higher mortality rate (P=0.04).

Conclusions

SE was not associated with a higher mortality rate but with higher functional disability. We also found that early-onset SE (within the first 7 days after stroke) was associated with a higher risk for SE recurrence and a higher mortality rate than late-onset SE (after 7 days after stroke).

I-123 iofetamine SPECT scan in systemic lupus erythematosus patients with cognitive and other minor neuropsychiatric symptoms: a pilot study

Accurate diagnosis of central nervous system (CNS) lupus remains difficult, especially when the manifestations are of subtle cognitive and affective changes. This pilot study reports on the use of I-123 iofetamine single photon emission computerized tomography (SPECT) scans in 18 such patients with documented systemic lupus erythematosus. Eight of the 18 scans were abnormal (44%), four in a diffuse bi-temporo-parietal pattern previously noted only in Alzheimer's disease, and four with large focal deficits. Neither the existence of the abnormal scan nor the particular pattern of abnormality correlated with the results of other diagnostic tests. These preliminary results raise the possibility that SPECT scans may offer an additional valuable diagnostic instrument in CNS lupus, although further studies are necessary to delineate their precise role.