1
|
Bobola MS, Chen L, Ezeokeke CK, Olmstead TA, Nguyen C, Sahota A, Williams RG, Mourad PD. Transcranial focused ultrasound, pulsed at 40 Hz, activates microglia acutely and reduces Aβ load chronically, as demonstrated in vivo. Brain Stimul 2020; 13:1014-1023. [PMID: 32388044 DOI: 10.1016/j.brs.2020.03.016] [Citation(s) in RCA: 36] [Impact Index Per Article: 9.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/21/2019] [Revised: 02/18/2020] [Accepted: 03/26/2020] [Indexed: 02/07/2023] Open
Abstract
OBJECTIVE Iaccarino et al. (2016) [1] exposed 1 h of light flickering at 40 Hz to awake 5XFAD Alzheimer's Disease (AD) mouse models, generating action potentials at 40 Hz, activating ∼54% of microglia to colocalize with Aβ plaque, acutely, and clearing ∼ 50% of Aβ plaque after seven days, but only in the visual cortex. HYPOTHESIS Transcranially delivered, focused ultrasound (tFUS) can replicate the results of Iaccarino et al. (2016) [1] but throughout its area of application. METHODS We exposed sedated 5XFAD mice to tFUS (2.0 MHz carrier frequency, 40 Hz pulse repetition frequency, 400 μs-long pulses, spatial peak pulse average value of 190 W/cm2). Acute studies targeted tFUS into one hemisphere of brain centered on its hippocampus for 1 h. Chronic studies targeted comparable brain in each hemisphere for 1 h/day for five days. RESULTS Acute application of tFUS activated more microglia that colocalized with Aβ plaque relative to sham ultrasound (36.0 ± 4.6% versus 14.2 ± 2.6% [mean ± standard error], z = 2.45, p < 0.014) and relative to the contralateral hemisphere of treated brain (36.0 ± 4.6% versus 14.3 ± 4.0%, z = 2.61, p < 0.009). Chronic application over five days reduced their Aβ plaque burden by nearly half relative to paired sham animals (47.4 ± 5.8%, z = - 2.79, p < 0.005). CONCLUSION Our results compare to those of Iaccarino et al. (2016) [1] but throughout the area of ultrasound-exposed brain. Our results also compare to those achieved by medications that target Aβ, but over a substantially shorter period of time. The proximity of our ultrasound protocol to those shown safe for non-human primates and humans may motivate its rapid translation to human studies.
Collapse
Affiliation(s)
- M S Bobola
- Department of Neurological Surgery, University of Washington, Seattle, WA, USA
| | - L Chen
- Department of Neurological Surgery, University of Washington, Seattle, WA, USA
| | - C K Ezeokeke
- Department of Neurological Surgery, University of Washington, Seattle, WA, USA
| | - T A Olmstead
- Department of Neurological Surgery, University of Washington, Seattle, WA, USA
| | - C Nguyen
- Department of Neurological Surgery, University of Washington, Seattle, WA, USA
| | - A Sahota
- Department of Neurological Surgery, University of Washington, Seattle, WA, USA
| | - R G Williams
- Department of Neurological Surgery, University of Washington, Seattle, WA, USA
| | - P D Mourad
- Department of Neurological Surgery, University of Washington, Seattle, WA, USA; Division of Engineering and Mathematics, University of Washington, Bothell, WA, USA.
| |
Collapse
|
2
|
Ahle SL, Schuller M, Clark MJ, Williams RG, Wnuk G, Fryer JP, George BC. Do End-of-Rotation Evaluations Adequately Assess Readiness to Operate? Acad Med 2019; 94:1946-1952. [PMID: 31397708 DOI: 10.1097/acm.0000000000002936] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/10/2023]
Abstract
PURPOSE Medical educators have developed no standard way to assess the operative performance of surgical residents. Most residency programs use end-of-rotation (EOR) evaluations for this purpose. Recently, some programs have implemented workplace-based "microassessment" tools that faculty use to immediately rate observed operative performance. The authors sought to determine (1) the degree to which EOR evaluations correspond to workplace-based microassessments and (2) which factors most influence EOR evaluations and directly observed workplace-based performance ratings and how the influence of those factors differs for each assessment method. METHOD In 2017, the authors retrospectively analyzed EOR evaluations and immediate postoperative assessment ratings of surgical trainees from a university-based training program from the 2015-2016 academic year. A Bayesian multivariate mixed model was constructed to predict operative performance ratings for each type of assessment. RESULTS Ratings of operative performance from EOR evaluations vs workplace-based microassessment ratings had a Pearson correlation of 0.55. Postgraduate year (PGY) of training was the most important predictor of operative performance ratings on EOR evaluations: Model estimates ranged from 0.62 to 1.75 and increased with PGY. For workplace-based assessment, operative autonomy rating was the most important predictor of operative performance (coefficient = 0.74). CONCLUSIONS EOR evaluations are perhaps most useful in assessing the ability of a resident to become a surgeon compared with other trainees in the same PGY of training. Workplace-based microassessments may be better for assessing a trainee's ability to perform specific procedures autonomously, thus perhaps providing more insight into a trainee's true readiness for operative independence.
Collapse
Affiliation(s)
- Samantha L Ahle
- S.L. Ahle is a general surgery resident, Yale School of Medicine, New Haven, Connecticut. M. Schuller is manager, Surgical Education, Department of Surgery, Northwestern University, Chicago, Illinois. M.J. Clark is lead statistician, Consulting for Statistics, Computing and Analytics Research, University of Michigan, Ann Arbor, Michigan. R.G. Williams was, at the time of the research reported here, adjunct research professor, Indiana University School of Medicine, Indianapolis, Indiana. G. Wnuk is program manager, Center for Surgical Training and Research, University of Michigan, Ann Arbor, Michigan. J.P. Fryer is vice chair of education, Department of Surgery, Northwestern University, Chicago, Illinois. B.C. George is assistant professor and director, Center for Surgical Training and Research, Department of Surgery, University of Michigan, Ann Arbor, Michigan; ORCID: https://orcid.org/0000-0002-9404-5255
| | | | | | | | | | | | | |
Collapse
|
3
|
Meyerson SL, Odell DD, Zwischenberger JB, Schuller M, Williams RG, Bohnen JD, Dunnington GL, Torbeck L, Mullen JT, Mandell SP, Choti MA, Foley E, Are C, Auyang E, Chipman J, Choi J, Meier AH, Smink DS, Terhune KP, Wise PE, Soper N, Lillemoe K, Fryer JP, George BC. The effect of gender on operative autonomy in general surgery residents. Surgery 2019; 166:738-743. [PMID: 31326184 PMCID: PMC7382913 DOI: 10.1016/j.surg.2019.06.006] [Citation(s) in RCA: 78] [Impact Index Per Article: 15.6] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/18/2019] [Revised: 05/02/2019] [Accepted: 06/04/2019] [Indexed: 10/26/2022]
Abstract
BACKGROUND Despite an increasing number of women in the field of surgery, bias regarding cognitive or technical ability may continue to affect the experience of female trainees differently than their male counterparts. This study examines the differences in the degree of operative autonomy given to female compared with male general surgery trainees. METHODS A smartphone app was used to collect evaluations of operative autonomy measured using the 4-point Zwisch scale, which describes defined steps in the progression from novice ("show and tell") to autonomous surgeon ("supervision only"). Differences in autonomy between male and female residents were compared using hierarchical logistic regression analysis. RESULTS A total of 412 residents and 524 faculty from 14 general surgery training programs evaluated 8,900 cases over a 9-month period. Female residents received less autonomy from faculty than did male residents overall (P < .001). Resident level of training and case complexity were the strongest predictors of autonomy. Even after controlling for potential confounding factors, including level of training, intrinsic procedural difficulty, patient-related case complexity, faculty sex, and training program environment, female residents still received less operative autonomy than their male counterparts. The greatest discrepancy was in the fourth year of training. CONCLUSION There is a sex-based difference in the autonomy granted to general surgery trainees. This gender gap may affect female residents' experience in training and possibly their preparation for practice. Strategies need to be developed to help faculty and residents work together to overcome this gender gap.
Collapse
Affiliation(s)
| | - David D Odell
- Department of Surgery, Northwestern University, Chicago, IL
| | | | - Mary Schuller
- Department of Surgery, Northwestern University, Chicago, IL
| | | | - Jordan D Bohnen
- Department of Surgery, Massachusetts General Hospital, Boston
| | | | - Laura Torbeck
- Department of Surgery, Indiana University, Indianapolis
| | - John T Mullen
- Department of Surgery, Massachusetts General Hospital, Boston
| | | | - Michael A Choti
- Department of Surgery, Banner MD Anderson Cancer Center, Gilbert, AZ
| | - Eugene Foley
- Department of Surgery, University of Wisconsin, Madison, WI
| | | | - Edward Auyang
- Department of Surgery, University of New Mexico, Albuquerque
| | | | - Jennifer Choi
- Department of Surgery, Indiana University, Indianapolis
| | - Andreas H Meier
- Department of Surgery, State University of New York Upstate Medical University, Syracuse
| | - Douglas S Smink
- Department of Surgery, Brigham and Women's Hospital, Boston, MA
| | - Kyla P Terhune
- Department of Surgery, Vanderbilt University, Nashville, TN
| | - Paul E Wise
- Department of Surgery, Washington University School of Medicine, St. Louis, MO
| | | | - Keith Lillemoe
- Department of Surgery, Massachusetts General Hospital, Boston
| | | | - Brian C George
- Department of Surgery, University of Michigan, Ann Arbor
| |
Collapse
|
4
|
Eaton M, Scully R, Schuller M, Yang A, Smink D, Williams RG, Bohnen JD, George BC, Fryer JP, Meyerson SL. Value and Barriers to Use of the SIMPL Tool for Resident Feedback. J Surg Educ 2019; 76:620-627. [PMID: 30770304 DOI: 10.1016/j.jsurg.2019.01.012] [Citation(s) in RCA: 27] [Impact Index Per Article: 5.4] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/03/2018] [Revised: 01/14/2019] [Accepted: 01/19/2019] [Indexed: 06/09/2023]
Abstract
OBJECTIVE The System for Improving and Measuring Procedural Learning (SIMPL) is a smart-phone application used to provide residents with an evaluation of operative autonomy and feedback. This study investigated the perceived benefits and barriers to app use. DESIGN A database of previously performed SIMPL evaluations was analyzed to identify high, low, and never users. Potential predisposing factors to use were explored. A survey investigating key areas of value and barriers to use for the SIMPL application was sent to resident and faculty users. Respondents were asked to self-identify how often they used the app. The perceived benefits and barriers were correlated with the level of usage. Qualitative analysis of free text responses was used to determine strategies to increase usage. SETTING General surgery training programs who are members of the Procedural Learning and Safety Collaborative. PARTICIPANTS Surgical residents and faculty. RESULTS At least 1 SIMPL evaluation was created for 411 residents and 524 faculty. Thirty percent of both faculty and residents were high-frequency users. Thirty percent of faculty were never users. One hundred eighty-eight residents and 207 faculty (response rate 46%) completed the survey. High-frequency resident users were more likely to perceive a benefit for both numerical evaluations (76% vs 30%) and dictated feedback (92% vs 30%). Faculty and residents commonly blamed each other for not creating and completing evaluations regularly (87% of residents, 81% of faculty). Suggested strategies to increase usage included reminders and integration with existing data systems. CONTRIBUTIONS Frequent users perceive value from the application, particularly from dictated feedback and see a positive impact on feedback in their programs. Faculty engagement represents a major barrier to adoption. Mechanisms which automatically remind residents to initiate an evaluation will help improve utilization but programs must work to enhance faculty willingness to respond and dictate feedback.
Collapse
Affiliation(s)
- Megan Eaton
- University of Texas Health San Antonio, San Antonio, Texas
| | - Rebecca Scully
- Massachusetts General Hospital, Department of Surgery, Boston, Massachusetts
| | - Mary Schuller
- Northwestern University, Department of Surgery, Chicago, Illinois
| | - Amy Yang
- Northwestern University, Department of Surgery, Chicago, Illinois
| | - Douglas Smink
- Massachusetts General Hospital, Department of Surgery, Boston, Massachusetts
| | - Reed G Williams
- Indiana University, Department of Surgery, Bloomington, Indiana
| | - Jordan D Bohnen
- Massachusetts General Hospital, Department of Surgery, Boston, Massachusetts
| | - Brian C George
- University of Michigan, Department of Surgery, Ann Arbor, Michigan
| | - Jonathan P Fryer
- Northwestern University, Department of Surgery, Chicago, Illinois
| | - Shari L Meyerson
- University of Kentucky, Department of Surgery, Lexington, Kentucky.
| |
Collapse
|
5
|
Lozier MS, Li F, Bacon S, Bahr F, Bower AS, Cunningham SA, de Jong MF, de Steur L, deYoung B, Fischer J, Gary SF, Greenan BJW, Holliday NP, Houk A, Houpert L, Inall ME, Johns WE, Johnson HL, Johnson C, Karstensen J, Koman G, Le Bras IA, Lin X, Mackay N, Marshall DP, Mercier H, Oltmanns M, Pickart RS, Ramsey AL, Rayner D, Straneo F, Thierry V, Torres DJ, Williams RG, Wilson C, Yang J, Yashayaev I, Zhao J. A sea change in our view of overturning in the subpolar North Atlantic. Science 2019; 363:516-521. [PMID: 30705189 DOI: 10.1126/science.aau6592] [Citation(s) in RCA: 37] [Impact Index Per Article: 7.4] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/05/2018] [Accepted: 12/18/2018] [Indexed: 11/03/2022]
Abstract
To provide an observational basis for the Intergovernmental Panel on Climate Change projections of a slowing Atlantic meridional overturning circulation (MOC) in the 21st century, the Overturning in the Subpolar North Atlantic Program (OSNAP) observing system was launched in the summer of 2014. The first 21-month record reveals a highly variable overturning circulation responsible for the majority of the heat and freshwater transport across the OSNAP line. In a departure from the prevailing view that changes in deep water formation in the Labrador Sea dominate MOC variability, these results suggest that the conversion of warm, salty, shallow Atlantic waters into colder, fresher, deep waters that move southward in the Irminger and Iceland basins is largely responsible for overturning and its variability in the subpolar basin.
Collapse
Affiliation(s)
- M S Lozier
- Division of Earth and Ocean Sciences, Duke University, Durham, NC, USA.
| | - F Li
- Division of Earth and Ocean Sciences, Duke University, Durham, NC, USA.
| | - S Bacon
- National Oceanography Centre, Southampton, UK
| | - F Bahr
- Woods Hole Oceanographic Institution, Woods Hole, MA, USA
| | - A S Bower
- Woods Hole Oceanographic Institution, Woods Hole, MA, USA
| | | | - M F de Jong
- Royal Netherlands Institute for Sea Research and Utrecht University, Texel, Netherlands
| | - L de Steur
- Royal Netherlands Institute for Sea Research and Utrecht University, Texel, Netherlands
| | - B deYoung
- Department of Physics and Physical Oceanography, Memorial University, St. John'?s, Newfoundland, Canada
| | - J Fischer
- GEOMAR Helmholtz Centre for Ocean Research Kiel, Kiel, Germany
| | - S F Gary
- Scottish Association for Marine Science, Oban, UK
| | - B J W Greenan
- Bedford Institute of Oceanography, Dartmouth, Nova Scotia, Canada
| | | | - A Houk
- Department of Ocean Sciences, University of Miami, Miami, FL, USA
| | - L Houpert
- Scottish Association for Marine Science, Oban, UK
| | - M E Inall
- Scottish Association for Marine Science, Oban, UK.,School of Geosciences, University of Edinburgh, Edinburgh, UK
| | - W E Johns
- Department of Ocean Sciences, University of Miami, Miami, FL, USA
| | - H L Johnson
- Department of Earth Sciences, University of Oxford, Oxford, UK
| | - C Johnson
- Scottish Association for Marine Science, Oban, UK
| | - J Karstensen
- GEOMAR Helmholtz Centre for Ocean Research Kiel, Kiel, Germany
| | - G Koman
- Department of Ocean Sciences, University of Miami, Miami, FL, USA
| | - I A Le Bras
- Scripps Institution of Oceanography, UCSD, La Jolla, CA, USA
| | - X Lin
- Physical Oceanography Laboratory and Institute for Advanced Ocean Studies, Ocean University of China and Qingdao National Laboratory for Marine Science and Technology, Qingdao, China
| | - N Mackay
- National Oceanography Centre, Liverpool, UK
| | - D P Marshall
- Department of Physics, Oxford University, Oxford, UK
| | - H Mercier
- CNRS, Laboratoire d'?Océanographie Physique et Spatiale, Plouzané, France
| | - M Oltmanns
- GEOMAR Helmholtz Centre for Ocean Research Kiel, Kiel, Germany
| | - R S Pickart
- Woods Hole Oceanographic Institution, Woods Hole, MA, USA
| | - A L Ramsey
- Woods Hole Oceanographic Institution, Woods Hole, MA, USA
| | - D Rayner
- National Oceanography Centre, Southampton, UK
| | - F Straneo
- Scripps Institution of Oceanography, UCSD, La Jolla, CA, USA
| | - V Thierry
- IFREMER, Laboratoire d'Océanographie Physique et Spatiale, Plouzané, France
| | - D J Torres
- Woods Hole Oceanographic Institution, Woods Hole, MA, USA
| | - R G Williams
- Department of Earth, Ocean and Ecological Sciences, University of Liverpool, Liverpool, UK
| | - C Wilson
- National Oceanography Centre, Liverpool, UK
| | - J Yang
- Woods Hole Oceanographic Institution, Woods Hole, MA, USA
| | - I Yashayaev
- Bedford Institute of Oceanography, Dartmouth, Nova Scotia, Canada
| | - J Zhao
- Woods Hole Oceanographic Institution, Woods Hole, MA, USA
| |
Collapse
|
6
|
Williams RG, George BC, Bohnen JD, Meyerson SL, Schuller MC, Meier AH, Torbeck L, Mandell SP, Mullen JT, Smink DS, Chipman JG, Auyang ED, Terhune KP, Wise PE, Choi J, Foley EF, Choti MA, Are C, Soper N, Zwischenberger JB, Dunnington GL, Lillemoe KD, Fryer JP. Is the operative autonomy granted to a resident consistent with operative performance quality. Surgery 2018; 164:566-570. [DOI: 10.1016/j.surg.2018.04.034] [Citation(s) in RCA: 11] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [What about the content of this article? (0)] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/16/2018] [Revised: 03/20/2018] [Accepted: 04/27/2018] [Indexed: 10/28/2022]
|
7
|
Williams RG, George BC, Meyerson SL, Bohnen JD, Dunnington GL, Schuller MC, Torbeck L, Mullen JT, Auyang E, Chipman JG, Choi J, Choti M, Endean E, Foley EF, Mandell S, Meier A, Smink DS, Terhune KP, Wise P, DaRosa D, Soper N, Zwischenberger JB, Lillemoe KD, Fryer JP. What factors influence attending surgeon decisions about resident autonomy in the operating room? Surgery 2017; 162:1314-1319. [PMID: 28950992 DOI: 10.1016/j.surg.2017.07.028] [Citation(s) in RCA: 41] [Impact Index Per Article: 5.9] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/26/2017] [Revised: 07/10/2017] [Accepted: 07/29/2017] [Indexed: 11/28/2022]
Abstract
BACKGROUND Educating residents in the operating room requires balancing patient safety, operating room efficiency demands, and resident learning needs. This study explores 4 factors that influence the amount of autonomy supervising surgeons afford to residents. METHODS We evaluated 7,297 operations performed by 487 general surgery residents and evaluated by 424 supervising surgeons from 14 training programs. The primary outcome measure was supervising surgeon autonomy granted to the resident during the operative procedure. Predictor variables included resident performance on that case, supervising surgeon history with granting autonomy, resident training level, and case difficulty. RESULTS Resident performance was the strongest predictor of autonomy granted. Typical autonomy by supervising surgeon was the second most important predictor. Each additional factor led to a smaller but still significant improvement in ability to predict the supervising surgeon's autonomy decision. The 4 factors together accounted for 54% of decision variance (r = 0.74). CONCLUSION Residents' operative performance in each case was the strongest predictor of how much autonomy was allowed in that case. Typical autonomy granted by the supervising surgeon, the second most important predictor, is unrelated to resident proficiency and warrants efforts to ensure that residents perform each procedure with many different supervisors.
Collapse
Affiliation(s)
- Reed G Williams
- Department of Surgery, Indiana University, Indianapolis, IN.
| | - Brian C George
- Department of Surgery, University of Michigan, Ann Arbor, MI
| | | | - Jordan D Bohnen
- Department of Surgery, Massachusetts General Hospital, Boston, MA
| | | | | | - Laura Torbeck
- Department of Surgery, Indiana University, Indianapolis, IN
| | - John T Mullen
- Department of Surgery, Massachusetts General Hospital, Boston, MA
| | - Edward Auyang
- Department of Surgery, University of New Mexico, Albuquerque, NM
| | | | - Jennifer Choi
- Department of Surgery, Indiana University, Indianapolis, IN
| | - Michael Choti
- Department of Surgery, University of Texas Southwestern, Surgery, Dallas, TX
| | - Eric Endean
- Department of Surgery, University of Kentucky, Lexington, KY
| | - Eugene F Foley
- Department of Surgery, University Of Wisconsin, Madison, WI
| | - Samuel Mandell
- Department of Surgery, University of Washington, Surgery, Seattle, WA
| | - Andreas Meier
- Department of Surgery, State University of New York, Surgery, Syracuse, NY
| | - Douglas S Smink
- Department of Surgery, Brigham and Women's Hospital, Surgery, Boston, MA
| | - Kyla P Terhune
- Department of Surgery, Vanderbilt University Medical Center, Surgery, Nashville, TN
| | - Paul Wise
- Department of Surgery, Washington University, Surgery, St. Louis, MO
| | - Debra DaRosa
- Department of Surgery, Northwestern University, Chicago, IL
| | | | | | - Keith D Lillemoe
- Department of Surgery, Massachusetts General Hospital, Boston, MA
| | | | | |
Collapse
|
8
|
Mellinger JD, Williams RG, Sanfey H, Fryer JP, DaRosa D, George BC, Bohnen JD, Schuller MC, Sandhu G, Minter RM, Gardner AK, Scott DJ. In Brief. Curr Probl Surg 2017. [DOI: 10.1067/j.cpsurg.2016.11.008] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/22/2022]
|
9
|
Mellinger JD, Williams RG, Sanfey H, Fryer JP, DaRosa D, George BC, Bohnen JD, Schuller MC, Sandhu G, Minter RM, Gardner AK, Scott DJ. Teaching and assessing operative skills: From theory to practice. Curr Probl Surg 2016; 54:44-81. [PMID: 28212782 DOI: 10.1067/j.cpsurg.2016.11.007] [Citation(s) in RCA: 17] [Impact Index Per Article: 2.1] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/28/2016] [Accepted: 11/22/2016] [Indexed: 11/22/2022]
Affiliation(s)
- John D Mellinger
- Department of Surgery, Southern Illinois University School of Medicine, Springfield, IL.
| | - Reed G Williams
- Department of Surgery, Southern Illinois University School of Medicine, Springfield, IL; Department of Surgery, Indiana University School of Medicine, Indianapolis, IN
| | - Hilary Sanfey
- Department of Surgery, Southern Illinois University School of Medicine, Springfield, IL; American College of Surgeons, Chicago, IL
| | - Jonathan P Fryer
- Department of Surgery, Feinberg School of Medicine, Northwestern University, Chicago, IL
| | - Debra DaRosa
- Department of Surgery, Feinberg School of Medicine, Northwestern University, Chicago, IL
| | - Brian C George
- Department of Surgery, University of Michigan, Ann Arbor, MI
| | - Jordan D Bohnen
- Department of General Surgery, Massachussetts General Hospital and Harvard University, Boston, MA
| | - Mary C Schuller
- Department of Surgery, Feinberg School of Medicine, Northwestern University, Chicago, IL
| | - Gurjit Sandhu
- Department of Surgery, University of Michigan, Ann Arbor, MI; Department of Learning Health Sciences, University of Michigan, Ann Arbor, MI
| | - Rebecca M Minter
- Department of Surgery, University of Texas Southwestern Medical Center, Dallas, TX
| | - Aimee K Gardner
- Department of Surgery, University of Texas Southwestern Medical Center, Dallas, TX; UT Southwestern Simulation Center, University of Texas Southwestern Medical Center, Dallas, TX
| | - Daniel J Scott
- Department of Surgery, University of Texas Southwestern Medical Center, Dallas, TX; UT Southwestern Simulation Center, University of Texas Southwestern Medical Center, Dallas, TX
| |
Collapse
|
10
|
|
11
|
Bohnen JD, George BC, Williams RG, Schuller MC, DaRosa DA, Torbeck L, Mullen JT, Meyerson SL, Auyang ED, Chipman JG, Choi JN, Choti MA, Endean ED, Foley EF, Mandell SP, Meier AH, Smink DS, Terhune KP, Wise PE, Soper NJ, Zwischenberger JB, Lillemoe KD, Dunnington GL, Fryer JP. The Feasibility of Real-Time Intraoperative Performance Assessment With SIMPL (System for Improving and Measuring Procedural Learning): Early Experience From a Multi-institutional Trial. J Surg Educ 2016; 73:e118-e130. [PMID: 27886971 DOI: 10.1016/j.jsurg.2016.08.010] [Citation(s) in RCA: 133] [Impact Index Per Article: 16.6] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/02/2016] [Revised: 07/12/2016] [Accepted: 08/18/2016] [Indexed: 06/06/2023]
Abstract
PURPOSE Intraoperative performance assessment of residents is of growing interest to trainees, faculty, and accreditors. Current approaches to collect such assessments are limited by low participation rates and long delays between procedure and evaluation. We deployed an innovative, smartphone-based tool, SIMPL (System for Improving and Measuring Procedural Learning), to make real-time intraoperative performance assessment feasible for every case in which surgical trainees participate, and hypothesized that SIMPL could be feasibly integrated into surgical training programs. METHODS Between September 1, 2015 and February 29, 2016, 15 U.S. general surgery residency programs were enrolled in an institutional review board-approved trial. SIMPL was made available after 70% of faculty and residents completed a 1-hour training session. Descriptive and univariate statistics analyzed multiple dimensions of feasibility, including training rates, volume of assessments, response rates/times, and dictation rates. The 20 most active residents and attendings were evaluated in greater detail. RESULTS A total of 90% of eligible users (1267/1412) completed training. Further, 13/15 programs began using SIMPL. Totally, 6024 assessments were completed by 254 categorical general surgery residents (n = 3555 assessments) and 259 attendings (n = 2469 assessments), and 3762 unique operations were assessed. There was significant heterogeneity in participation within and between programs. Mean percentage (range) of users who completed ≥1, 5, and 20 assessments were 62% (21%-96%), 34% (5%-75%), and 10% (0%-32%) across all programs, and 96%, 75%, and 32% in the most active program. Overall, response rate was 70%, dictation rate was 24%, and mean response time was 12 hours. Assessments increased from 357 (September 2015) to 1146 (February 2016). The 20 most active residents each received mean 46 assessments by 10 attendings for 20 different procedures. CONCLUSIONS SIMPL can be feasibly integrated into surgical training programs to enhance the frequency and timeliness of intraoperative performance assessment. We believe SIMPL could help facilitate a national competency-based surgical training system, although local and systemic challenges still need to be addressed.
Collapse
Affiliation(s)
- Jordan D Bohnen
- Department of Surgery, Harvard Medical School, Massachusetts General Hospital, Boston, Massachusetts.
| | - Brian C George
- Department of Surgery, Harborview Medical Center, University of Washington, Seattle, Washington
| | - Reed G Williams
- Department of Surgery, Indiana University School of Medicine, Indianapolis, Indiana
| | - Mary C Schuller
- Department of Surgery, Northwestern University Feinberg School of Medicine, Chicago, Illinois
| | - Debra A DaRosa
- Department of Surgery, Northwestern University Feinberg School of Medicine, Chicago, Illinois
| | - Laura Torbeck
- Department of Surgery, Indiana University School of Medicine, Indianapolis, Indiana
| | - John T Mullen
- Department of Surgery, Harvard Medical School, Massachusetts General Hospital, Boston, Massachusetts
| | - Shari L Meyerson
- Department of Surgery, Northwestern University Feinberg School of Medicine, Chicago, Illinois
| | - Edward D Auyang
- Department of Surgery, University of New Mexico School of Medicine, Albuquerque, New Mexico
| | - Jeffrey G Chipman
- Department of Surgery, University of Minnesota Medical School, Minneapolis, Minnesota
| | - Jennifer N Choi
- Department of Surgery, Indiana University School of Medicine, Indianapolis, Indiana
| | - Michael A Choti
- Department of Surgery, University of Texas Southwestern Medical Center, Dallas, Texas
| | - Eric D Endean
- Department of Surgery, University of Kentucky College of Medicine, Lexington, Kentucky
| | - Eugene F Foley
- Department of Surgery, University of Wisconsin School of Medicine and Public Health, Madison, Wisconsin
| | - Samuel P Mandell
- Department of Surgery, Harborview Medical Center, University of Washington, Seattle, Washington
| | - Andreas H Meier
- Department of Surgery, State University of New York Upstate Medical University, Syracuse, New York
| | - Douglas S Smink
- Department of Surgery, Brigham and Women׳s Hospital, Boston, Massachusetts
| | - Kyla P Terhune
- Department of Surgery, Vanderbilt University Medical Center, Nashville, Tennessee
| | - Paul E Wise
- Department of Surgery, Washington University School of Medicine, St. Louis, Missouri
| | - Nathaniel J Soper
- Department of Surgery, Northwestern University Feinberg School of Medicine, Chicago, Illinois
| | | | - Keith D Lillemoe
- Department of Surgery, Harvard Medical School, Massachusetts General Hospital, Boston, Massachusetts
| | - Gary L Dunnington
- Department of Surgery, Indiana University School of Medicine, Indianapolis, Indiana
| | - Jonathan P Fryer
- Department of Surgery, Northwestern University Feinberg School of Medicine, Chicago, Illinois
| |
Collapse
|
12
|
Klamen DL, Williams RG, Roberts N, Cianciolo AT. Competencies, milestones, and EPAs - Are those who ignore the past condemned to repeat it? Med Teach 2016; 38:904-910. [PMID: 26805785 DOI: 10.3109/0142159x.2015.1132831] [Citation(s) in RCA: 16] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/05/2023]
Abstract
BACKGROUND The idea of competency-based education sounds great on paper. Who wouldn't argue for a standardized set of performance-based assessments to assure competency in graduating students and residents? Even so, conceptual concerns have already been raised about this new system and there is yet no evidence to refute their veracity. AIMS We argue that practical concerns deserve equal consideration, and present evidence strongly suggesting these concerns should be taken seriously. METHOD Specifically, we share two historical examples that illustrate what happened in two disparate contexts (K-12 education and the Department of Defense [DOD]) when competency (or outcomes-based) assessment frameworks were implemented. We then examine how observation and assessment of clinical performance stands currently in medical schools and residencies, since these methodologies will be challenged to a greater degree by expansive lists of competencies and milestones. RESULTS/CONCLUSIONS We conclude with suggestions as to a way forward, because clearly the assessment of competency and the ability to guarantee that graduates are ready for medical careers is of utmost importance. Hopefully the headlong rush to competencies, milestones, and core entrustable professional activities can be tempered before even more time, effort, frustration and resources are invested in an endeavor which history suggests will collapse under its own weight.
Collapse
|
13
|
Williams RG, Mellinger JD, Dunnington GL. A problem-oriented approach to resident performance ratings. Surgery 2016; 160:936-945. [PMID: 27460933 DOI: 10.1016/j.surg.2016.04.040] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/20/2016] [Revised: 02/20/2016] [Accepted: 04/13/2016] [Indexed: 10/21/2022]
Abstract
BACKGROUND Global, end-of-rotation evaluations are often difficult to interpret due to their high level of abstraction (eg, excellent, good, poor) and the bias toward high ratings. This study documents the utility of and measurement characteristics of serious problem items, an alternative item format. METHODS This report is based on 4,234 faculty performance ratings for 105 general surgery residents. Faculty members reported whether each resident had a serious problem for each of 8 areas of clinical performance and 6 areas of professional behavior. RESULTS A total of 263 serious problems were reported. The performance category with the most total serious problems noted was knowledge and that with the least problems noted was relations with patients and family members. Seven residents accounted for 86.9% of all serious problem reports. Each resident had serious problems in multiple performance areas. Problems were reported most frequently in knowledge, management, technical/procedural skills, ability to assume responsibility within level of competence, and problem identification. Citations of these serious problems were most common in year 3. Traditional ratings of global performance were not an adequate means for identifying residents with serious performance problems. CONCLUSION Serious problem ratings can communicate faculty concerns about residents more directly and can be used as a complement to conventional global rating scales without substantially increasing faculty workload.
Collapse
Affiliation(s)
- Reed G Williams
- Department of Surgery, Indiana University School of Medicine, Indianapolis, IN.
| | - John D Mellinger
- Department of Surgery, Southern Illinois University School of Medicine, Springfield, IL
| | - Gary L Dunnington
- Department of Surgery, Indiana University School of Medicine, Indianapolis, IN
| |
Collapse
|
14
|
Abstract
We performed a review of the economic literature to identify what is known about the relationship between Expanded Disability Status Scale (EDSS) categories and cost of multiple sclerosis (MS). We sought cohort studies of patients with multiple sclerosis that described the costs attributed to each EDSS score and utilized specific inclusion criteria for the selection of 10 studies. We found that both direct and indirect costs rise continuously with increasing EDSS category, and this rise is qualitatively exponential. The rise in indirect costs appears at lower EDSS scores. The cost of a relapse occurring in any given EDSS category exceeds that associated with that particular EDSS category. Few studies comprehensively assessed the entire spectrum of the costs, and much of the literature is based on EDSS categories in coarse groupings. In spite of several variations between studies, one important conclusion that we can draw is that rise in cost is positively correlated to scores on the EDSS categories, and therefore agents with a capacity to prevent or arrest the rate of MS progression may affect the overall cost of MS.
Collapse
Affiliation(s)
- M B Patwardhan
- Duke Center for Clinical Health Policy Research, Duke University Medical Center, Durham, NC 27705, USA.
| | | | | | | | | | | |
Collapse
|
15
|
Abstract
The purpose of this study was to compare the continuous method of scoring a performance test composed of standardized patients with a derivative method that assigned each examinee a dichotomous score and to explore the use of Ang off 's method at the case level with these two scoring methods. Both scoring systems produced reasonable means and distributions of scores. The continuous scores were somewhat more reproducible than the dichotomous scores although neither was very relay?ducible. Pass rates for both scoring systems were appropriately very high and thus the reproducibility of the pass/fail decisions was also high. Regardless of the scoring system, the application of Angoff's method reported in this article has the advantages of efficiency and enhanced credibility to the experts.
Collapse
Affiliation(s)
| | | | | | | | | | | | - Miriam Friedman
- Educational Commission for Foreign, Medical Graduates, Philadelphia
| |
Collapse
|
16
|
Abstract
INTRODUCTION It is generally accepted that paediatric intranasal foreign bodies should be removed in the emergency setting. In the case of a difficult to access dissolvable foreign body in an uncooperative child, the question must be raised regarding whether or not a watch and wait strategy is more appropriate. We ask: How long does it take for popular sweets (candy) to dissolve in the human nose? METHODS Five popular UK sweets were placed in the right nasal cavity of a 29-year-old male (the author) with no sino-nasal disease. Time taken to dissolve was recorded. RESULTS All five sweets were completely dissolved in under one hour. DISCUSSION A watch and wait strategy in favour of examination under anaesthetic may be a viable option in some cases. Limitations of the study include the age of the participant and size of the sweets. It is also important in practice that the clinician is able to elicit an accurate history regarding the exact nature of the foreign body. CONCLUSION It remains prudent to perform an examination under anaesthetic of an uncooperative child with a solid or unknown nasal foreign body. However, if the clinician can be certain the foreign body is a small sugar or chocolate based sweet only, a watch and wait strategy may be a reasonable choice.
Collapse
|
17
|
Chen XP, Williams RG, Smink DS. Dissecting Attending Surgeons' Operating Room Guidance: Factors That Affect Guidance Decision Making. J Surg Educ 2015; 72:e137-e144. [PMID: 26153113 DOI: 10.1016/j.jsurg.2015.06.003] [Citation(s) in RCA: 17] [Impact Index Per Article: 1.9] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/31/2015] [Revised: 05/29/2015] [Accepted: 06/04/2015] [Indexed: 06/04/2023]
Abstract
PURPOSE The amount of guidance provided by the attending surgeon in the operating room (OR) is a key element in developing residents' autonomy. The purpose of this study is to explore factors that affect attending surgeons' decision making regarding OR guidance provided to the resident. METHODS We used video-stimulated recall interviews (VSRI) throughout this 2-phase study. In Phase 1, 3 attending surgeons were invited to review separately 30 to 45 minute video segments of their prerecorded surgical operations to explore factors that influenced their OR guidance decision making. In Phase 2, 3 attending surgeons were observed and documented in the OR (4 operations, 341min). Each operating surgeon reviewed their videotaped surgical performance within 5 days of the operation to reflect on factors that affected their decision making during the targeted guidance events. All VSRI were recorded. Thematic analysis and manual coding were used to synthesize and analyze data from VSRI transcripts, OR observation documents, and field notes. RESULTS A total of 255 minutes of VSRI involving 6 surgeons and 7 surgical operations from 5 different procedures were conducted. A total of 13 guidance decision-making influence factors from 4 categories were identified (Cohen's κ = 0.674): Setting (case schedule and patient morbidity), content (procedure attributes and case progress), resident (current competency level, trustworthiness, self-confidence, and personal traits), and attending surgeon (level of experience, level of comfort, preferred surgical technique, OR training philosophy, and responsibility as surgeon). A total of 5 factors (case schedule, patient morbidity, procedure attributes, resident current competency level, and trustworthiness) influenced attending surgeons' pre-OR guidance plans. "OR training philosophy" and "responsibility as surgeon" were anchor factors that affected attending surgeons' OR guidance decision-making patterns. CONCLUSIONS Surgeons' OR guidance decision making is a dynamic process that is influenced by 13 situational factors. These factors can be used by residency programs to tailor strategies designed to increase resident autonomy in the OR.
Collapse
Affiliation(s)
| | - Reed G Williams
- Department of Surgery, Indiana University, Indianapolis, Indiana
| | - Douglas S Smink
- Department of Surgery, Brigham and Women's Hospital, Boston, Massachusetts
| |
Collapse
|
18
|
Williams RG, Verhulst S, Mellinger JD, Dunnington GL. Is a Single-Item Operative Performance Rating Sufficient? J Surg Educ 2015; 72:e212-e217. [PMID: 26610357 DOI: 10.1016/j.jsurg.2015.05.002] [Citation(s) in RCA: 14] [Impact Index Per Article: 1.6] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/16/2015] [Accepted: 05/04/2015] [Indexed: 06/05/2023]
Abstract
OBJECTIVE A valid measure of resident operative performance ability requires direct observation and accurate rating of multiple resident performances under the normal range of operating conditions. The challenge is to create an operative performance rating (OPR) system that: is easy to use, encourages completion of many ratings immediately after performances and minimally disrupts supervising surgeons' work days. The purpose of this study was to determine whether a score based on a single-item overall OPR provides a valid and stable appraisal of resident operative performances. DESIGN A retrospective comparison of a single-item OPR with a gold-standard rating based on multiple procedure-specific and general OPR items. SETTING Data were collected in the general surgery residency program at Southern Illinois University from 2001 through 2012. PARTICIPANTS Assessments of 1033 operative performances (3 common procedures, 2 laparoscopic, and 1 open) by general surgery residents were collected. OPRs based on single-item overall performance scale scores were compared with gold-standard ratings for the same performances. RESULTS Differences in performance scores using the 2 scales averaged 0.02 points (5-point scale). Correlations of the single-item and gold-standard scale scores averaged 0.95. Based on generalizability analyses of laparoscopic cholecystectomy ratings, each instrument required 5 observations to achieve reliabilities of 0.80 and 11 observations to achieve reliabilities of 0.90. Only 4.4% of single-item ratings misclassified the performance when compared with the gold-standard rating and all misclassifications were near misses. For 80% of misclassified ratings, single-item ratings were lower. CONCLUSIONS Single-item operative performance measures produced ratings that were virtually identical to gold-standard scale ratings. Misclassifications occurred infrequently and were minor in magnitude. Ratings using the single-item scale: take less time to complete, should increase the sample of procedures rated, and encourage attending surgeons to complete ratings immediately after observing performances. Face-to-face and written comments and suggestions should continue to be used to provide the granular feedback residents need to improve subsequent performances.
Collapse
Affiliation(s)
- Reed G Williams
- Department of Surgery, Indiana University School of Medicine, Indianapolis, Indiana.
| | - Steven Verhulst
- Department of Medical Education, Southern Illinois University School of Medicine, Springfield, Illinois
| | - John D Mellinger
- Department of Surgery, Southern Illinois University School of Medicine, Springfield, Illinois
| | - Gary L Dunnington
- Department of Surgery, Indiana University School of Medicine, Indianapolis, Indiana
| |
Collapse
|
19
|
Williams RG, Klamen DL. Twenty Questions game performance on medical school entrance predicts clinical performance. Med Educ 2015; 49:920-927. [PMID: 26296408 DOI: 10.1111/medu.12758] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.1] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/30/2014] [Revised: 03/12/2015] [Accepted: 03/30/2015] [Indexed: 06/04/2023]
Abstract
CONTEXT This study is based on the premise that the game of 'Twenty Questions' (TQ) tests the knowledge people acquire through their lives and how well they organise and store it so that they can effectively retrieve, combine and use it to address new life challenges. Therefore, performance on TQ may predict how effectively medical school applicants will organise and store knowledge they acquire during medical training to support their work as doctors. OBJECTIVES This study was designed to determine whether TQ game performance on medical school entrance predicts performance on a clinical performance examination near graduation. METHODS This prospective, longitudinal, observational study involved each medical student in one class playing a game of TQ on a non-medical topic during the first week of medical school. Near graduation, these students completed a 14-case clinical performance examination. Performance on the TQ task was compared with performance on the clinical performance examination. RESULTS The 24 students who exhibited a logical approach to the TQ task performed better on all senior clinical performance examination measures than did the 26 students who exhibited a random approach. Approach to the task was a better predictor of senior examination diagnosis justification performance than was the Medical College Admission Test (MCAT) Biological Science Test score and accounts for a substantial amount of score variation not attributable to a co-relationship with MCAT Biological Science Test performance. CONCLUSIONS Approach to the TQ task appears to be one reasonable indicator of how students process and store knowledge acquired in their everyday lives and may be a useful predictor of how they will process the knowledge acquired during medical training. The TQ task can be fitted into one slot of a mini medical interview.
Collapse
Affiliation(s)
- Reed G Williams
- Department of Surgery, Indiana University School of Medicine, Indianapolis, IN, USA
| | - Debra L Klamen
- Department of Medical Education, Southern Illinois University, Springfield, IL, USA
| |
Collapse
|
20
|
Williams RG, Dunnington GL, Mellinger JD, Klamen DL. Placing constraints on the use of the ACGME milestones: a commentary on the limitations of global performance ratings. Acad Med 2015; 90:404-407. [PMID: 25295965 DOI: 10.1097/acm.0000000000000507] [Citation(s) in RCA: 22] [Impact Index Per Article: 2.4] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/03/2023]
Abstract
As part of the outcomes-based accreditation process, the Accreditation Council for Graduate Medical Education (ACGME) now requires that medical specialties formulate and use educational milestones to assess residents' performance. These milestones are specialty-specific achievements that residents are expected to demonstrate at established intervals in their training. In this Commentary, the authors argue that the pressure to efficiently use program directors' and faculty members' time, particularly in the increasingly clinical-revenue-dependent model of the academic medical center, will lead program directors to meet these new accreditation expectations solely by adding items that assess these competencies to global end-of-rotation rating forms. This approach will increase the workload of faculty but will not provide new and useful information about residents' competence. These same concerns could apply if assessment committees attempt to measure these new performance dimensions without using direct observation to evaluate residents' performance. In these circumstances, the milestones movement will fall short of its intention and potential. In this Commentary, the authors outline and provide evidence from the literature for their concerns. They discuss the role that human judges play in measuring performance, the measurement characteristics of global performance ratings, and the problems associated with simply adding items to existing global rating forms.
Collapse
Affiliation(s)
- Reed G Williams
- Dr. Williams is adjunct professor of surgery, Indiana University School of Medicine, Indianapolis, Indiana, and J. Roland Folse, MD, Professor of Surgical Education Research and Development Emeritus, Southern Illinois University School of Medicine, Springfield, Illinois. He served as a member of the General Surgery Milestones Development Committee. Dr. Dunnington is chairman, Department of Surgery, and Jay L. Grosfeld Professor of Surgery, Indiana University School of Medicine, Indianapolis, Indiana. He served as a member of the committee that developed the original ACGME competencies. Dr. Mellinger is J. Roland Folse, MD, Chair and professor, Division of General Surgery, and program director, General Surgery Residency Program, Southern Illinois University School of Medicine, Springfield, Illinois. Dr. Klamen is associate dean for education and curriculum and chair, Department of Medical Education, Southern Illinois University School of Medicine, Springfield, Illinois
| | | | | | | |
Collapse
|
21
|
Wilson AB, Choi JN, Torbeck LJ, Mellinger JD, Dunnington GL, Williams RG. Clinical Assessment and Management Examination--Outpatient (CAMEO): its validity and use in a surgical milestones paradigm. J Surg Educ 2015; 72:33-40. [PMID: 25088367 DOI: 10.1016/j.jsurg.2014.06.010] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/18/2014] [Revised: 05/13/2014] [Accepted: 06/17/2014] [Indexed: 06/03/2023]
Abstract
OBJECTIVES Clinical Assessment and Management Examination--Outpatient (CAMEO) is a metric for evaluating the clinical performance of surgery residents. The aim of this study was to investigate the measurement characteristics of CAMEO and propose how it might be used as an evaluation tool within the general surgery milestones project. DESIGN A total of 117 CAMEO evaluations were gathered and used for analysis. Internal consistency reliability was estimated, and item characteristics were explored. A Kruskal-Wallis procedure was performed to discern how well the instrument discriminated between training levels. An exploratory factor analysis was also conducted to understand the dimensionality of the evaluation. SETTING CAMEO evaluations were collected from 2 departments of surgery geographically located in the Midwestern United States. Combined, the participating academic institutions graduate approximately 18 general surgery residents per year. PARTICIPANTS In this retrospective data analysis, the number of evaluations per resident ranged from 1 to 7, and evaluations were collected from 2006 to 2013. For the purpose of data analysis, residents were classified as interns (postgraduate year 1 [PGY1]), juniors (PGY2-3), or seniors (PGY4-5). RESULTS CAMEO scores were found to have high internal consistency (Cronbach's α = 0.96), and all items were highly correlated (≥ 0.86) to composite CAMEO scores. Scores discriminated between senior residents (PGY4-5) and lower level residents (PGY1-3). Per an exploratory factor analysis, CAMEO was revealed to measure a single dimension of "clinical competence." CONCLUSIONS The findings of this research aligned with related literature and verified that CAMEO scores have desirable measurement properties, making CAMEO an attractive resource for evaluating the clinical performance of surgery residents.
Collapse
Affiliation(s)
- Adam B Wilson
- Department of Surgery, Indiana University School of Medicine, Indianapolis, Indiana.
| | - Jennifer N Choi
- Department of Surgery, Indiana University School of Medicine, Indianapolis, Indiana
| | - Laura J Torbeck
- Department of Surgery, Indiana University School of Medicine, Indianapolis, Indiana
| | - John D Mellinger
- Division of General Surgery, Southern Illinois University School of Medicine, Springfield, Illinois
| | - Gary L Dunnington
- Department of Surgery, Indiana University School of Medicine, Indianapolis, Indiana
| | - Reed G Williams
- Department of Surgery, Indiana University School of Medicine, Indianapolis, Indiana
| |
Collapse
|
22
|
Chen XP, Williams RG, Sanfey HA, Smink DS. A taxonomy of surgeons' guiding behaviors in the operating room. Am J Surg 2014; 209:15-20. [PMID: 25454960 DOI: 10.1016/j.amjsurg.2014.07.018] [Citation(s) in RCA: 21] [Impact Index Per Article: 2.1] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/04/2014] [Revised: 06/20/2014] [Accepted: 07/09/2014] [Indexed: 10/24/2022]
Abstract
BACKGROUND This study explores the nature and the intention of attending surgeons' guiding behaviors performed in the operating room (OR) in order to build taxonomy of OR guiding behavior. METHODS Nine attending surgeons and 8 surgical residents were invited to observe 8 prerecorded surgical cases from 4 common procedures and completed semistructured interviews. All video-based observations were videotaped. Thematic analysis was applied to identify surgeons' OR guiding behavior. RESULTS Seven hundred eighty minutes of video-based observations with interviews were conducted. Sixteen types of OR guiding behaviors in 3 intention-based categories were identified: 3 of the 16 was "teaching" (18.75%), 8 of the 16 was "directing" (50%), and 5 of the 16 was "assisting" (31.25%). CONCLUSIONS Surgeons' OR guiding behaviors were grounded in 3 behavioral intentions: teaching, directing, and assisting. This taxonomy of OR guiding behavior can be used as a basis for developing OR guiding strategy to improve residents' intraoperative competency, autonomy, and independence.
Collapse
Affiliation(s)
- Xiaodong Phoenix Chen
- Department of Surgery, Brigham and Women's Hospital, 75 Francis Street, Boston, MA 02115, USA.
| | - Reed G Williams
- Department of Surgery, Indiana University, 545 Barnhill Drive, Indianapolis, IN 46202
| | - Hilary A Sanfey
- Department of Surgery, Southern Illinois University, 747 N Rutledge Street, Springfield, IL 62794
| | - Douglas S Smink
- Department of Surgery, Brigham and Women's Hospital, 75 Francis Street, Boston, MA 02115, USA
| |
Collapse
|
23
|
Chen XP, Williams RG, Smink DS. Do residents receive the same OR guidance as surgeons report? Difference between residents' and surgeons' perceptions of OR guidance. J Surg Educ 2014; 71:e79-e82. [PMID: 24931416 DOI: 10.1016/j.jsurg.2014.04.010] [Citation(s) in RCA: 12] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/26/2014] [Revised: 04/25/2014] [Accepted: 04/27/2014] [Indexed: 06/03/2023]
Abstract
PURPOSE Operating room (OR) guidance is important for surgical residents' performance and, ultimately, for the development of independence and autonomy. This study explores the differences in surgical residents' and attending surgeons' perceptions of OR guidance in prerecorded surgical cases. METHODS A total of 9 attending surgeons and 8 surgical residents observed 8 prerecorded surgical cases and were asked to identify both the presence and the type of attending surgeons' OR guidance. Each recorded case was observed by 2 attending surgeons and 1 resident. A previously developed taxonomy for types of OR guidance was applied to analyze the data to explore the difference. Agreement by both attending surgeons on the presence and the type of OR guidance served as the concordant guidance behaviors to which the responses of the residents were compared. RESULTS Overall, 116 OR guidance events were identified. Attending surgeons agreed on the presence of guidance in 80 of 116 (69.8%) events and consistently identified the type of OR guidance in 91.4% (73/80, Cohen κ = 0.874) of them. However, surgical residents only agreed with attending surgeons on the presence of guidance in 61.25% (49/80) of the events. In addition, there was significant disagreement (Cohen κ = 0.319) between surgical residents and attending surgeons in the type of OR guidance; the residents only identified 54.8% (40/73) of concordant guidance behaviors in the same guidance category as both the surgeons. Among the types of OR guidance, residents and attending surgeons were most likely to agree on the teaching guidance (66.67%) and least likely to agree on the assisting guidance (36.84%). CONCLUSIONS Surgical residents and attending surgeons have different perceptions of both the presence and the type of OR guidance. This difference in perception of OR guidance has important implications for the efficiency of training surgical residents in the OR, and, ultimately on residents' development of independence and autonomy.
Collapse
Affiliation(s)
| | - Reed G Williams
- Department of Surgery, School of Medicine, Indiana University, Indianapolis, Indiana
| | - Douglas S Smink
- Department of Surgery, Brigham and Women's Hospital, Boston, Massachusetts
| |
Collapse
|
24
|
Williams RG, Chen XP, Sanfey H, Markwell SJ, Mellinger JD, Dunnington GL. The measured effect of delay in completing operative performance ratings on clarity and detail of ratings assigned. J Surg Educ 2014; 71:e132-e138. [PMID: 25088368 DOI: 10.1016/j.jsurg.2014.06.015] [Citation(s) in RCA: 36] [Impact Index Per Article: 3.6] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/13/2014] [Revised: 05/19/2014] [Accepted: 06/23/2014] [Indexed: 06/03/2023]
Abstract
PURPOSE Operative performance ratings (OPRs) need adequate clarity and detail to support self-directed learning and valid progress decisions. This study was designed to determine (1) the elapsed time between observing operative performances and completing performance ratings under field conditions and (2) the effect of increased elapsed time on rating clarity and detail. METHODS Overall, 895 OPRs by 19 faculty members for 37 general surgery residents were the focus of this study. The elapsed time between observing the performance and completing the evaluation was recorded. No-delay comparison data included 45 additional ratings of 8 performances collected under controlled conditions immediately following the performance by 17 surgeons whose sole responsibility was to observe and rate the performances. Item-to-item OPR variation and the presence and nature of comments were indicators of evaluation clarity, detail, and quality. RESULTS Elapsed time between observing and evaluating performances under field conditions were as follows: 1 day or less, 116 performances (13%); 2 to 3 days, 178 performances (20%); 4 to 14 days, 377 performances (42%); and more than 14 days, 224 performances (25%). Overall, 87% of performances rated more than 14 days after observation had no item-to-item ratings variation compared with 62% rated with a delay of 4 to 14 days, 41% rated with a delay of 2 to 3 days, 42% rated within 1 day, and 2% rated immediately. In addition, 70% of ratings completed more than 14 days after observation had no written comments, compared with 49% for those completed with a delay of 4 to 14 days, 45% for those completed in 2 to 3 days, and 46% for those completed within 1 day. Moreover, 47% of comments submitted after more than 14 days were exclusively global comments (less instructionally useful) compared with 7% for those completed with a delay of 4 to 14 days and 5% for those completed in 1 to 3 days. CONCLUSIONS The elapsed time between observation and rating of operative performances should be recorded. Immediate ratings should be encouraged. Ratings completed more than 3 days after observation should be discouraged and discounted, as they lack clarity and detail about the performance.
Collapse
Affiliation(s)
- Reed G Williams
- Department of Surgery, Indiana University School of Medicine, Indianapolis, Indiana.
| | | | - Hilary Sanfey
- Department of Surgery, Southern Illinois University School of Medicine, Springfield, Illinois
| | - Stephen J Markwell
- Department of Surgery, Southern Illinois University School of Medicine, Springfield, Illinois
| | - John D Mellinger
- Department of Surgery, Southern Illinois University School of Medicine, Springfield, Illinois
| | - Gary L Dunnington
- Department of Surgery, Indiana University School of Medicine, Indianapolis, Indiana
| |
Collapse
|
25
|
Torbeck L, Williams RG, Choi J, Schmitz CC, Chipman JG, Dunnington GL. How much guidance is given in the operating room? Factors influencing faculty self-reports, resident perceptions, and faculty/resident agreement. Surgery 2014; 156:797-803. [DOI: 10.1016/j.surg.2014.06.069] [Citation(s) in RCA: 32] [Impact Index Per Article: 3.2] [Reference Citation Analysis] [What about the content of this article? (0)] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/13/2013] [Accepted: 06/26/2014] [Indexed: 10/24/2022]
|
26
|
Williams RG, Klamen DL, Markwell SJ, Cianciolo AT, Colliver JA, Verhulst SJ. Variations in senior medical student diagnostic justification ability. Acad Med 2014; 89:790-798. [PMID: 24667511 DOI: 10.1097/acm.0000000000000215] [Citation(s) in RCA: 20] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/03/2023]
Abstract
PURPOSE To determine the diagnostic justification proficiency of senior medical students across a broad spectrum of cases with common chief complaints and diagnoses. METHOD The authors gathered diagnostic justification exercise data from the Senior Clinical Comprehensive Examination taken by Southern Illinois University School of Medicine's students from the classes of 2011 (n = 67), 2012 (n = 66), and 2013 (n = 79). After interviewing and examining standardized patients, students listed their key findings and diagnostic possibilities considered, and provided a written explanation of how they used key findings to move from their initial differential diagnoses to their final diagnosis. Two physician judges blindly rated responses. RESULTS Student diagnostic justification performance was highly variable from case to case and often rated below expectations. Of the students in the classes of 2011, 2012, and 2013, 57% (38/67), 23% (15/66), and 33% (26/79) were judged borderline or poor on diagnostic justification performance for more than 50% of the cases on the examination. CONCLUSIONS Student diagnostic justification performance was inconsistent across the range of cases, common chief complaints, and underlying diagnoses used in this study. More than 20% of students exhibited borderline or poor diagnostic justification performance on more than 50% of the cases. If these results are confirmed in other medical schools, attention needs to be directed to investigating new curricular methods that ensure deliberate practice of these competencies across the spectrum of common chief complaints and diagnoses and do not depend on the available mix of patients.
Collapse
Affiliation(s)
- Reed G Williams
- Dr. Williams is adjunct professor, Department of Surgery, Indiana University School of Medicine, Indianapolis, Indiana. He is emeritus professor of medical education and J. Roland Folse Professor of Surgical Education Research and Development Emeritus, Southern Illinois University School of Medicine, Springfield, Illinois. Dr. Klamen is associate dean for education and curriculum and professor and chair, Department of Medical Education, Southern Illinois University School of Medicine, Springfield, Illinois. Mr. Markwell is director of statistics and research consulting, Department of Surgery, Southern Illinois University School of Medicine, Springfield, Illinois. Dr. Cianciolo is assistant professor of medical education, Southern Illinois University School of Medicine, Springfield, Illinois. Dr. Colliver is professor of medical education emeritus and past director of statistics and research consulting, Southern Illinois University School of Medicine, Springfield, Illinois. Dr. Verhulst is professor and director of statistics and research informatics, Center for Clinical Research, Southern Illinois University School of Medicine, Springfield, Illinois
| | | | | | | | | | | |
Collapse
|
27
|
Copeland G, Lee EPF, Williams RG, Archibald AT, Shallcross DE, Dyke JM. Determination of the photolysis rate coefficient of monochlorodimethyl sulfide (MClDMS) in the atmosphere and its implications for the enhancement of SO2 production from the DMS + Cl2 reaction. Environ Sci Technol 2014; 48:1557-1565. [PMID: 24280000 DOI: 10.1021/es402956r] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/02/2023]
Abstract
In this work, the photolysis rate coefficient of CH3SCH2Cl (MClDMS) in the lower atmosphere has been determined and has been used in a marine boundary layer (MBL) box model to determine the enhancement of SO2 production arising from the reaction DMS + Cl2. Absorption cross sections measured in the 28000-34000 cm(-1) region have been used to determine photolysis rate coefficients of MClDMS in the troposphere at 10 solar zenith angles (SZAs). These have been used to determine the lifetimes of MClDMS in the troposphere. At 0° SZA, a photolysis lifetime of 3-4 h has been obtained. The results show that the photolysis lifetime of MClDMS is significantly smaller than the lifetimes with respect to reaction with OH (≈ 4.6 days) and with Cl atoms (≈ 1.2 days). It has also been shown, using experimentally derived dissociation energies with supporting quantum-chemical calculations, that the dominant photodissocation route of MClDMS is dissociation of the C-S bond to give CH3S and CH2Cl. MBL box modeling calculations show that buildup of MClDMS at night from the Cl2 + DMS reaction leads to enhanced SO2 production during the day. The extra SO2 arises from photolysis of MClDMS to give CH3S and CH2Cl, followed by subsequent oxidation of CH3S.
Collapse
Affiliation(s)
- G Copeland
- School of Chemistry, University of Southampton , Highfield, Southampton SO17 1BJ, U.K
| | | | | | | | | | | |
Collapse
|
28
|
Roberts NK, Williams RG, Schwind CJ, Sutyak JA, McDowell C, Griffen D, Wall J, Sanfey H, Chestnut A, Meier AH, Wohltmann C, Clark TR, Wetter N. The impact of brief team communication, leadership and team behavior training on ad hoc team performance in trauma care settings. Am J Surg 2013; 207:170-8. [PMID: 24468024 DOI: 10.1016/j.amjsurg.2013.06.016] [Citation(s) in RCA: 78] [Impact Index Per Article: 7.1] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/23/2013] [Revised: 06/11/2013] [Accepted: 06/11/2013] [Indexed: 11/29/2022]
Abstract
BACKGROUND Communication breakdowns and care coordination problems often cause preventable adverse patient care events, which can be especially acute in the trauma setting, in which ad hoc teams have little time for advanced planning. Existing teamwork curricula do not address the particular issues associated with ad hoc emergency teams providing trauma care. METHODS Ad hoc trauma teams completed a preinstruction simulated trauma encounter and were provided with instruction on appropriate team behaviors and team communication. Teams completed a postinstruction simulated trauma encounter immediately afterward and 3 weeks later, then completed a questionnaire. Blinded raters rated videotapes of the simulations. RESULTS Participants expressed high levels of satisfaction and intent to change practice after the intervention. Participants changed teamwork and communication behavior on the posttest, and changes were sustained after a 3-week interval, though there was some loss of retention. CONCLUSIONS Brief training exercises can change teamwork and communication behaviors on ad hoc trauma teams.
Collapse
Affiliation(s)
- Nicole K Roberts
- Department of Medical Education, Southern Illinois University School of Medicine, PO Box 19681, Springfield, IL 62794-9681, USA.
| | - Reed G Williams
- Department of Surgery, Southern Illinois University School of Medicine, Springfield, IL, USA
| | - Cathy J Schwind
- Department of Surgery, Southern Illinois University School of Medicine, Springfield, IL, USA
| | - John A Sutyak
- Department of Surgery, Southern Illinois University School of Medicine, Springfield, IL, USA
| | - Christopher McDowell
- Department of Surgery, Southern Illinois University School of Medicine, Springfield, IL, USA
| | - David Griffen
- Department of Surgery, Southern Illinois University School of Medicine, Springfield, IL, USA
| | - Jarrod Wall
- Department of Surgery, Southern Illinois University School of Medicine, Springfield, IL, USA
| | - Hilary Sanfey
- Department of Surgery, Southern Illinois University School of Medicine, Springfield, IL, USA
| | | | - Andreas H Meier
- Department of Surgery, Southern Illinois University School of Medicine, Springfield, IL, USA
| | - Christopher Wohltmann
- Department of Surgery, Southern Illinois University School of Medicine, Springfield, IL, USA
| | - Ted R Clark
- Department of Surgery, Southern Illinois University School of Medicine, Springfield, IL, USA
| | - Nathan Wetter
- Department of Surgery, Southern Illinois University School of Medicine, Springfield, IL, USA
| |
Collapse
|
29
|
Paquette LB, Wisnowski JL, Ceschin R, Pruetz JD, Detterich JA, Del Castillo S, Nagasunder AC, Kim R, Painter MJ, Gilles FH, Nelson MD, Williams RG, Blüml S, Panigrahy A. Abnormal cerebral microstructure in premature neonates with congenital heart disease. AJNR Am J Neuroradiol 2013; 34:2026-33. [PMID: 23703146 DOI: 10.3174/ajnr.a3528] [Citation(s) in RCA: 28] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/07/2022]
Abstract
BACKGROUND AND PURPOSE Abnormal cerebral microstructure has been documented in term neonates with congenital heart disease, portending risk for injury and poor neurodevelopmental outcome. Our hypothesis was that preterm neonates with congenital heart disease would demonstrate diffuse cerebral microstructural abnormalities when compared with critically ill neonates without congenital heart disease. A secondary aim was to identify any association between microstructural abnormalities, white matter injury (eg, punctate white matter lesions), and other clinical variables, including heart lesions. MATERIALS AND METHODS With the use of tract-based spatial statistics, an unbiased, voxelwise method for analyzing diffusion tensor imaging data, we compared 21 preterm neonates with congenital heart disease with 2 cohorts of neonates without congenital heart disease: 28 term and 27 preterm neonates, identified from the same neonatal intensive care unit. RESULTS Compared with term neonates without congenital heart disease, preterm neonates with congenital heart disease had microstructural abnormalities in widespread regions of the central white matter. However, 42% of the preterm neonates with congenital heart disease had punctate white matter lesions. When neonates with punctate white matter lesions were excluded, microstructural abnormalities remained only in the splenium. Preterm neonates with congenital heart disease had similar microstructure to preterm neonates without congenital heart disease. CONCLUSIONS Diffuse microstructural abnormalities were observed in preterm neonates with congenital heart disease, strongly associated with punctate white matter lesions. Independently, regional vulnerability of the splenium, a structure associated with visual spatial function, was observed in all preterm neonates with congenital heart disease.
Collapse
|
30
|
Cianciolo AT, Williams RG, Klamen DL, Roberts NK. Biomedical knowledge, clinical cognition and diagnostic justification: a structural equation model. Med Educ 2013; 47:309-16. [PMID: 23398017 DOI: 10.1111/medu.12096] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/18/2023]
Abstract
CONTEXT The process whereby medical students employ integrated analytic and non-analytic diagnostic strategies is not fully understood. Analysing academic performance data could provide a perspective complementary to that of laboratory experiments when investigating the nature of diagnostic strategy. This study examined the performance data of medical students in an integrated curriculum to determine the relative contributions of biomedical knowledge and clinical pattern recognition to diagnostic strategy. METHODS Structural equation modelling was used to examine the relationship between biomedical knowledge and clinical cognition (clinical information gathering and interpretation) assessed in Years 1 and 2 of medical school and their relative contributions to diagnostic justification assessed at the beginning of Year 4. Modelling was applied to the academic performance data of 133 medical students who received their md degrees in 2011 and 2012. RESULTS The model satisfactorily fit the data. The correlation between biomedical knowledge and clinical cognition was low-moderate (0.26). The paths between these two constructs and diagnostic justification were moderate and slightly favoured biomedical knowledge (0.47 and 0.40 for biomedical knowledge and clinical cognition, respectively). CONCLUSIONS The findings suggest that within the first 2 years of medical school, students possessed separate, but complementary, cognitive tools, comprising biomedical knowledge and clinical pattern recognition, which contributed to an integrated diagnostic strategy at the beginning of Year 4. Assessing diagnostic justification, which requires students to make their thinking explicit, may promote the integration of analytic and non-analytic processing into diagnostic strategy.
Collapse
Affiliation(s)
- Anna T Cianciolo
- Department of Medical Education, Southern Illinois University School of Medicine, Springfield, Illinois 62794-9681, USA.
| | | | | | | |
Collapse
|
31
|
Williams RG, Verhulst S, Colliver JA, Sanfey H, Chen X, Dunnington GL. A template for reliable assessment of resident operative performance: Assessment intervals, numbers of cases and raters. Surgery 2012; 152:517-24; discussion 524-7. [DOI: 10.1016/j.surg.2012.07.004] [Citation(s) in RCA: 42] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [What about the content of this article? (0)] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/13/2011] [Accepted: 07/05/2012] [Indexed: 11/24/2022]
|
32
|
Abstract
PURPOSE Fostering ability to organize and use medical knowledge to guide data collection, make diagnostic decisions, and defend those decisions is at the heart of medical training. However, these abilities are not systematically examined prior to graduation. This study examined diagnostic justification (DXJ) ability of medical students shortly before graduation. METHOD All senior medical students in the Classes of 2011 (n = 67) and 2012 (n = 70) at Southern Illinois University were required to take and pass a 14-case, standardized patient examination prior to graduation. For nine cases, students were required to write a free-text response indicating how they used patient data to move from their differential to their final diagnosis. Two physicians graded each DXJ response. DXJ scores were compared with traditional standardized patient examination (SCCX) scores. RESULTS The average intraclass correlation between raters' rankings of DXJ responses was 0.75 and 0.64 for the Classes of 2011 and 2012, respectively. Student DXJ scores were consistent across the nine cases. Using SCCX and DXJ scores led to the same pass-fail decision in a majority of cases. However, there were many cases where discrepancies occurred. In a majority of those cases, students would fail using the DXJ score but pass using the SCCX score. Common DXJ errors are described. CONCLUSIONS Commonly used standardized patient examination component scores (history/physical examination checklist score, findings, differential diagnosis, diagnosis) are not direct, comprehensive measures of DXJ ability. Critical deficiencies in DXJ abilities may thus go undiscovered.
Collapse
Affiliation(s)
- Reed G Williams
- Department of Surgery, Southern Illinois University School of Medicine, Springfield, IL 62794-9638, USA.
| | | |
Collapse
|
33
|
Sanfey H, DaRosa DA, Hickson GB, Williams B, Sudan R, Boehler ML, Klingensmith ME, Klamen D, Mellinger JD, Hebert JC, Richard KM, Roberts NK, Schwind CJ, Williams RG, Sachdeva AK, Dunnington GL. Pursuing Professional Accountability. ACTA ACUST UNITED AC 2012; 147:642-7. [DOI: 10.1001/archsurg.2012.832] [Citation(s) in RCA: 35] [Impact Index Per Article: 2.9] [Reference Citation Analysis] [What about the content of this article? (0)] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/14/2022]
|
34
|
Chen X(P, Williams RG, Sanfey HA, Dunnington GL. How do supervising surgeons evaluate guidance provided in the operating room? Am J Surg 2012; 203:44-8. [DOI: 10.1016/j.amjsurg.2011.09.003] [Citation(s) in RCA: 48] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/18/2011] [Revised: 09/27/2011] [Accepted: 09/27/2011] [Indexed: 10/15/2022]
|
35
|
Roberts NK, Williams RG, Klingensmith M, Sullivan M, Boehler M, Hickson G, Kim MJ, Klamen DL, Leblang T, Schwind C, Titchenal K, Dunnington GL. The case of the entitled resident: a composite case study of a resident performance problem syndrome with interdisciplinary commentary. Med Teach 2012; 34:1024-32. [PMID: 22957508 DOI: 10.3109/0142159x.2012.719654] [Citation(s) in RCA: 6] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/17/2023]
Abstract
BACKGROUND Residents with performance problems create substantial burden on programs and institutions. Understanding the nature and quality of performance problems can help in learning to address performance problems. AIM We sought to illuminate the effects of resident performance problems and the potential solutions for those problems from the perspectives of people with various roles in health care. METHODS We created a composite portrait from several residents who demonstrated a cluster of common performance characteristics and whose chronic or serious maladaptive behavior and response to situations created problems for themselves, for their clinical colleagues, and for faculty of their residency program. The composite was derived from in-depth interviews of program directors and review of resident records. We solicited practitioners from multiple fields to respond to the portrait by answering a series of questions about severity, prognosis, and how and whether one could reliably remediate a person with these performance characteristics. We present their perspectives in a manner borrowed from the New England Journal of Medicine's "Case Records of the Massachusetts General Hospital." RESULTS We created a composite portrait of a resident whose behavior suggested he felt entitled to benefits his peers were not entitled to. Experts reflecting on his behavior varied in their opinion about the effect the resident would have on the health care system. They suggested approaches to remediation that required substantial time and effort from the faculty. CONCLUSION Programs must balance the needs of individual residents to adjust their behaviors with the needs of the health care system and other people within it.
Collapse
Affiliation(s)
- Nicole K Roberts
- Department of Medical Education, Southern Illinois University School of Medicine, P.O. Box 19681, Springfield, IL 62794-9681, USA.
| | | | | | | | | | | | | | | | | | | | | | | |
Collapse
|
36
|
Sanfey H, Williams RG, Chen X, Dunnington GL. Evaluating resident operative performance: a qualitative analysis of expert opinions. Surgery 2011; 150:759-70. [PMID: 22000189 DOI: 10.1016/j.surg.2011.07.058] [Citation(s) in RCA: 30] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/11/2011] [Accepted: 07/12/2011] [Indexed: 11/30/2022]
Abstract
PURPOSE To elicit and analyze the strengths and weaknesses of resident operative performance (ROP) as identified by independent expert ratings. METHODS Four groups of expert surgeons, blinded to resident training level, evaluated ROP on 2 each of laparoscopic cholecystectomy, thyroidectomy, open inguinal, and laparoscopic ventral hernia repair audio videos, and listed strengths and weaknesses. Comments were coded as technical skills, forward planning, self-direction, situation awareness, and patient safety, and as either procedure specific or general skills. RESULTS Eighteen experts independently entered 1087 comments on 8 ROP strengths and weakness. In the post independent rating discussion, consensus was reached on 85 (28%) of 300 post rating comments with majority agreement on another 83 (28%). Overall, the dominant focus was on forward planning. With the exception of the laparoscopic cholecystectomy cases, raters focused more on general than on procedure-specific skills (P < .05). CONCLUSION Fewer than 30% of expert rater comments focused on technical skills when considering ROP strengths and weaknesses. Although there was some variation in individual comments, majority agreement was reached on 56% of comments during the post independent rating discussion. These findings will improve rater training and further the implementation of a national assessment process to evaluate end of training surgical competence and operative proficiency.
Collapse
Affiliation(s)
- Hilary Sanfey
- Southern Illinois University School of Medicine, Department of Surgery, Springfield, IL 62794, USA.
| | | | | | | |
Collapse
|
37
|
Williams RG, Klamen DL, White CB, Petrusa E, Fincher RME, Whitfield CF, Shatzer JH, McCarty T, Miller BM. Tracking development of clinical reasoning ability across five medical schools using a progress test. Acad Med 2011; 86:1148-1154. [PMID: 21785314 DOI: 10.1097/acm.0b013e31822631b3] [Citation(s) in RCA: 33] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/27/2023]
Abstract
PURPOSE Little is known about the acquisition of clinical reasoning skills in medical school, the development of clinical reasoning over the medical curriculum as a whole, and the impact of various curricular methodologies on these skills. This study investigated (1) whether there are differences in clinical reasoning skills between learners at different years of medical school, and (2) whether there are differences in performance between students at schools with various curricular methodologies. METHOD Students (n = 2,394) who had completed zero to three years of medical school at five U.S. medical schools participated in a cross-sectional study in 2008. Students took the same diagnostic pattern recognition (DPR) and clinical data interpretation (CDI) tests. Percent correct scores were used to determine performance differences. Data from all schools and students at all levels were aggregated for further analysis. RESULTS Student performance increased substantially as a result of each year of training. Gains in DPR and CDI performance during the third year of medical school were not as great as in previous years across the five schools. CDI performance and performance gains were lower than DPR performance and gains. Performance gains attributable to training at each of the participating medical schools were more similar than different. CONCLUSIONS Years of training accounted for most of the variation in DPR and CDI performance. As a rule, students at higher training levels performed better on both tests, though the expected larger gains during the third year of medical school did not materialize.
Collapse
Affiliation(s)
- Reed G Williams
- Department of Surgery, Southern Illinois University School of Medicine, Springfield, 62794-9622, USA
| | | | | | | | | | | | | | | | | |
Collapse
|
38
|
|
39
|
Schwind CJ, Boehler ML, Markwell SJ, Williams RG, Brenner MJ. Use of simulated pages to prepare medical students for internship and improve patient safety. Acad Med 2011; 86:77-84. [PMID: 21099392 DOI: 10.1097/acm.0b013e3181ff9893] [Citation(s) in RCA: 17] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/23/2023]
Abstract
PURPOSE During the transition from medical school to internship, trainees experience high levels of stress related to pages on the inpatient wards. The steep learning curve during this period may also affect patient safety. The authors piloted the use of simulated pages to improve medical student preparedness, decrease stress related to pages, and familiarize medical students with common patient problems. METHOD A multidisciplinary team at Southern Illinois University School of Medicine developed simulated pages that were tested among senior medical students. Sixteen medical students were presented with 11 common patient scenarios. Data on assessment, management, and global performance were collected. Mean confidence levels were evaluated pre- and postintervention. Students were also surveyed on how the simulated pages program influenced their perceived comfort in managing patient care needs and the usefulness of the exercise in preparing them to handle inpatient pages. RESULTS Mean scores on the assessment and management portions of the scenarios varied widely depending on the scenario (range -15.6 ± 41.6 to 95.7 ± 9.5). Pass rates based on global performance ranged from 12% to 93%. Interrater agreement was high (mean kappa = 0.88). Students' confidence ratings on a six-point scale increased from 1.87 preintervention to 3.53 postintervention (P < .0001). CONCLUSIONS Simulated pages engage medical students and may foster medical student preparedness for internship. Students valued the opportunity to simulate "on call" responsibilities, and exposure to simulated pages significantly increased their confidence levels. Further studies are needed to determine effects on patient safety outcomes.
Collapse
Affiliation(s)
- Cathy J Schwind
- Department of Surgery, Southern Illinois University School of Medicine, Springfield, Illinois 62794-9649, USA
| | | | | | | | | |
Collapse
|
40
|
Abstract
BACKGROUND Current remediation strategies for students failing standardized patient examinations represent poorly targeted approaches since the specific nature of clinical performance weaknesses has not been defined. PURPOSE The purpose is to determine the impact of a specifically targeted clinical performance course required of students who failed a clinical performance examination. METHODS A month-long clinical performance course, targeted to treat specific types of clinical performance deficiencies, was designed to remediate students failing standardized patient examinations in 2007 (n=8) and 2008 (n=5). Participating students were assessed on pre- and postperformance measures, including multiple-choice tests that measured diagnostic pattern recognition and clinical data interpretation and clinical performance measures using standardized clinical encounters. Comparisons between average pre- and postintervention performance scores were computed using paired sample t tests. Results were adjusted for regression toward the mean. RESULTS In both 2007 and 2008, the mean preintervention clinical data interpretation and standardized patient examination scores were below the criterion referenced passing standard set for the clinical competency exam. In both years the mean postintervention scores for the participants were above the passing standard for these two examinations. Pre- and postintervention differences were statistically significant in both cases. CONCLUSIONS This study provides insight into the reasons that students fail clinical performance examinations and elucidates one method by which such students may be successfully remediated.
Collapse
Affiliation(s)
- Debra L Klamen
- Department of Medical Education, Southern Illinois University School of Medicine, Springfield, Illinois 62794-9622, USA.
| | | |
Collapse
|
41
|
Kim MJ, Boehler ML, Ketchum JK, Bueno R, Williams RG, Dunnington GL. Skills coaches as part of the educational team: A randomized controlled trial of teaching of a basic surgical skill in the laboratory setting. Am J Surg 2010; 199:94-8. [PMID: 20103072 DOI: 10.1016/j.amjsurg.2009.08.016] [Citation(s) in RCA: 37] [Impact Index Per Article: 2.6] [Reference Citation Analysis] [What about the content of this article? (0)] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/08/2009] [Revised: 08/25/2009] [Accepted: 08/25/2009] [Indexed: 10/20/2022]
|
42
|
Abstract
BACKGROUND The Outcome Project requires high-quality assessment approaches to provide reliable and valid judgments of the attainment of competencies deemed important for physician practice. INTERVENTION The Accreditation Council for Graduate Medical Education (ACGME) convened the Advisory Committee on Educational Outcome Assessment in 2007-2008 to identify high-quality assessment methods. The assessments selected by this body would form a core set that could be used by all programs in a specialty to assess resident performance and enable initial steps toward establishing national specialty databases of program performance. The committee identified a small set of methods for provisional use and further evaluation. It also developed frameworks and processes to support the ongoing evaluation of methods and the longer-term enhancement of assessment in graduate medical education. OUTCOME The committee constructed a set of standards, a methodology for applying the standards, and grading rules for their review of assessment method quality. It developed a simple report card for displaying grades on each standard and an overall grade for each method reviewed. It also described an assessment system of factors that influence assessment quality. The committee proposed a coordinated, national-level infrastructure to support enhancements to assessment, including method development and assessor training. It recommended the establishment of a new assessment review group to continue its work of evaluating assessment methods. The committee delivered a report summarizing its activities and 5 related recommendations for implementation to the ACGME Board in September 2008.
Collapse
Affiliation(s)
- Susan R. Swing
- Corresponding author: Susan Swing, PhD, Accreditation Council for Graduate Medical Education (ACGME), 515 North State Street, Suite 2000, Chicago, IL 60654, 312.755.7447,
| | | | | | | |
Collapse
|
43
|
Williams RG, Roberts NK, Schwind CJ, Dunnington GL. The nature of general surgery resident performance problems. Surgery 2009; 145:651-8. [DOI: 10.1016/j.surg.2009.01.019] [Citation(s) in RCA: 49] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [What about the content of this article? (0)] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/25/2008] [Accepted: 01/12/2009] [Indexed: 10/20/2022]
|
44
|
DaRosa D, Rogers DA, Williams RG, Hauge LS, Sherman H, Murayama K, Nagle A, Dunnington GL. Impact of a structured skills laboratory curriculum on surgery residents' intraoperative decision-making and technical skills. Acad Med 2008; 83:S68-S71. [PMID: 18820505 DOI: 10.1097/acm.0b013e318183cdb1] [Citation(s) in RCA: 13] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/26/2023]
Abstract
BACKGROUND This project sought to study the effectiveness of a curriculum to enhance the intraoperative clinical judgment and procedural skill of surgical residents. METHOD A multiinstitutional, prospective, randomized study was performed. A cognitive task analysis of laparoscopic cholecystectomy (LC) was conducted on which instructional activities and measurement instruments were designed. Residents were randomly assigned to a control or intervention group. Subjects took written pre- and posttests examining procedure-related judgment and knowledge. The intervention group participated in a three-session curriculum emphasizing LC critical decisions and error prevention. All subjects were evaluated performing the procedure on a cadaveric model. Scores from written and practical exams were compared using independent-sample and paired Student t tests. RESULTS Written examination scores increased for both groups. The intervention group scored significantly higher (P < .05) on the written posttest than the control group. There were no differences between groups on the practical examination. Reliability coefficients for the written examination ranged from .65 to .75. Reliability coefficients for the oral exam, technical skill, and error items on the porcine practical exam were .83, .90, and .53. CONCLUSIONS The curriculum resulted in enhanced performance on a written exam designed to assess intraoperative judgment, but no differences in technical skills, showing important implications for future skills lab curriculum formats.
Collapse
Affiliation(s)
- Debra DaRosa
- Department of Surgery, 251 E. Huron, Galter 3-150, Chicago, IL 60611, USA.
| | | | | | | | | | | | | | | |
Collapse
|
45
|
Abstract
BACKGROUND Working knowledge of physicians manifests as a combination of diagnostic pattern recognition and clinical data interpretation (analytic fact checking). PURPOSE The purpose was to study medical student acquisition of these abilities as a function of years of medical training/experience. METHODS A cross-sectional study involving students who had completed 0, 1, 2, and 3 years of medical school. All students at all levels of training took the same tests of diagnostic pattern recognition and clinical data interpretation. Percent correct scores were calculated and used to estimate learning curves. A cohort of family physicians also took the test to provide a benchmark. RESULTS Student diagnostic pattern recognition and clinical data interpretation ability demonstrated a steady upward growth curve but leveled off in Year 3. Diagnostic pattern recognition performance was consistently higher than clinical data interpretation performance. The rate of diagnostic performance gain with training and experience was also higher. CONCLUSIONS Medical students acquired diagnostic pattern recognition ability and all years of medical training contributed. The rate of clinical data interpretation performance improvement was slower, and the absolute performance level was lower. What was surprising was the lower rate of improvement in diagnostic pattern recognition and clinical data interpretation performance for students during their 1st year of clinical training. Students' understanding of findings and their relationships to disease processes may be affected by their limited patient experience.
Collapse
Affiliation(s)
- Reed G Williams
- Department of Surgery, Southern Illinois University School of Medicine, Springfield, Illinois 62794-9638, USA.
| | | | | |
Collapse
|
46
|
Williams RG, Klamen DL, Mayer D, Valaski M, Roberts NK. A sampling strategy for promoting and assessing medical student retention of physical examination skills. Acad Med 2007; 82:S22-5. [PMID: 17895682 DOI: 10.1097/acm.0b013e318141f5ca] [Citation(s) in RCA: 6] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/17/2023]
Abstract
BACKGROUND Skill acquisition and maintenance requires spaced deliberate practice. Assessing medical students' physical examination performance ability is resource intensive. The authors assessed the nature and size of physical examination performance samples necessary to accurately estimate total physical examination skill. METHOD Physical examination assessment data were analyzed from second year students at the University of Illinois College of Medicine at Chicago in 2002, 2003, and 2004 (N = 548). Scores on subgroups of physical exam maneuvers were compared with scores on the total physical exam, to identify sound predictors of total test performance. RESULTS Five exam subcomponents were sufficiently correlated to overall test performance and provided adequate sensitivity and specificity to serve as a means to prompt continued student review and rehearsal of physical examination technical skills. CONCLUSIONS Selection and administration of samples of the total physical exam provide a resource-saving approach for promoting and estimating overall physical examination skills retention.
Collapse
Affiliation(s)
- Reed G Williams
- Southern Illinois University School of Medicine, Department of Surgery, 800 North Rutledge Street, PO Box 19638, Springfield, IL 62794-9638, USA.
| | | | | | | | | |
Collapse
|
47
|
Boehler ML, Schwind CJ, Rogers DA, Ketchum J, O'Sullivan E, Mayforth R, Quin J, Wohltman C, Johnson C, Williams RG, Dunnington G. A Theory-Based Curriculum for Enhancing Surgical Skillfulness. J Am Coll Surg 2007; 205:492-7. [PMID: 17765166 DOI: 10.1016/j.jamcollsurg.2007.04.018] [Citation(s) in RCA: 23] [Impact Index Per Article: 1.4] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/06/2006] [Revised: 04/19/2007] [Accepted: 04/19/2007] [Indexed: 10/23/2022]
Abstract
BACKGROUND Curricula for surgical technical skills laboratories have traditionally been designed to accommodate the clinical activities of residents, so they typically consist of individual, episodic training sessions. We believe that the skills laboratory offers an opportunity to design a surgical skills curriculum based on the fundamental elements known to be important for motor skill instruction. We hypothesized that training novices with such a curriculum for a 1-month period would yield skills performance levels equivalent to those of second year surgery residents who had trained in a traditional program. STUDY DESIGN Fourth-year medical students served as study subjects (novice group) during a 4-week senior elective. They were taught each skill during a 1-week period. Subjects received instruction by a content expert followed by a 1-week period of deliberate practice with feedback. The novice performances were videotaped both before and after the intervention, and each videotape was evaluated in a blinded fashion by experts using a validated evaluation instrument. These results were compared with skill performance ratings of first- and second-year surgery residents that had been accumulated over the previous 3 years. RESULTS Average performance ratings for the novices substantially improved for all four skills after training. There was no marked difference between average performance ratings of postintervention novice scores when compared with the average scores in the resident group. Inter-rater agreement in scoring for the videotaped novice performances exceeded 0.87 (intraclass correlation) for all ratings of pre- and posttraining. CONCLUSIONS These results demonstrate the effectiveness of a laboratory-based training program that includes fundamentals of motor skills acquisition.
Collapse
Affiliation(s)
- Margaret L Boehler
- Department of Surgery, Southern Illinois University School of Medicine, Springfield, IL 62794-9655, USA
| | | | | | | | | | | | | | | | | | | | | |
Collapse
|
48
|
Baath J, Ells AL, Kherani A, Williams RG. Severe retinal injuries from paintball projectiles. Can J Ophthalmol 2007; 42:620-3. [PMID: 17641709 DOI: 10.3129/can] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 05/16/2023]
Abstract
BACKGROUND To determine the outcomes and circumstances of retinal injuries caused by blunt trauma from paintball pellet projectiles. METHODS Retrospective case series of all patients who presented with retinal injuries due to paintball-related trauma to 2 retina specialists in a clinical setting from 2004 to 2005. Patients were followed for a mean of 7.3 months, and retinal trauma was documented with retinal photographs and ocular coherence tomography as needed. Best corrected visual acuity (BCVA) was the main outcome measure. RESULTS Three eyes of 3 patients suffered severe retinal injuries after blunt trauma from a paintball pellet. Together, the 3 eyes demonstrated extensive retinal findings, including commotio retinae, choroidal rupture, and macular hole. BCVA at last follow-up ranged from 20/80 to hand motions. INTERPRETATION Our small case series indicates that retinal trauma from paintball injuries is not uncommon and results in severe long-term visual morbidity. The sale of paintball guns and pellets should be strictly prohibited for minors, and adults should be educated about the need for appropriate ocular protection and the potentially serious consequences of the use of these guns outside of commercial settings.
Collapse
Affiliation(s)
- J Baath
- Undergraduate Medicine Department, University of Toronto, Ont
| | | | | | | |
Collapse
|
49
|
Abstract
BACKGROUND There is currently a widespread use of intravitreal triamcinolone acetonide (IVTA) for age-related macular degeneration, diabetic macular edema, cystoid macular edema secondary to retinal vein occlusions, and uveitis. The aim of this investigation was to assess the rates of various complications associated with this treatment and to determine which factors are associated with the development of these complications. METHODS A retrospective interventional case series of all patients from one retina specialist undergoing IVTA was conducted in a clinical setting from 2002 to 2005. All disease entities were included. Patients were followed for a mean of 9.5 months after receiving 4 mg (0.1 mL) of nonfiltered triamcinolone acetonide (TA). All complications associated with the injection procedure or with the TA were noted. RESULTS Two hundred and twenty-three (223) eyes of 192 patients received a total of 336 IVTA injections between 2002 and 2005. The mean age was 73.3 years and mean follow-up was 9.5 months. A single injection was performed in 144 eyes (64.6%); 2 IVTAs in 55 eyes (24.7%); 3 IVTAs in 16 eyes (7.2%), and 3.6% of eyes had more than 3 injections at a minimal interval of 3 months. The only immediate complication was a single injection (0.3%) associated with a temporary occlusion of the central retinal artery, which opened immediately following anterior paracentesis. Late complications included endophthalmitis in 1 of 336 (0.3%) injections and a steroid response requiring glaucoma medication in 60 of 192 patients (31.3%). In patients with preexisting glaucoma, 58.8% required additional glaucoma medication. Glaucoma-filtering surgery was required in 2 of 192 patients (1.0%). CONCLUSIONS In the study center, the IVTA is extremely safe in patients without a history of glaucoma. However, patients with preexisting glaucoma with progressive optic neuropathy must be treated with great caution.
Collapse
Affiliation(s)
- J Baath
- Undergraduate Medicine, University of Toronto, Toronto, Ontario, Canada
| | | | | | | | | |
Collapse
|
50
|
Williams RG, Silverman R, Schwind C, Fortune JB, Sutyak J, Horvath KD, Van Eaton EG, Azzie G, Potts JR, Boehler M, Dunnington GL. Surgeon information transfer and communication: factors affecting quality and efficiency of inpatient care. Ann Surg 2007; 245:159-69. [PMID: 17245166 PMCID: PMC1877003 DOI: 10.1097/01.sla.0000242709.28760.56] [Citation(s) in RCA: 131] [Impact Index Per Article: 7.7] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/26/2022]
Abstract
OBJECTIVE To determine the nature of surgeon information transfer and communication (ITC) errors that lead to adverse events and near misses. To recommend strategies for minimizing or preventing these errors. SUMMARY BACKGROUND DATA Surgical hospital practice is changing from a single provider to a team-based approach. This has put a premium on effective ITC. The Information Transfer and Communication Practices (ITCP) Project is a multi-institutional effort to: 1) better understand surgeon ITCP and their patient care consequences, 2) determine what has been done to improve ITCP in other professions, and 3) recommend ways to improve these practices among surgeons. METHODS Separate, semi-structured focus group sessions were conducted with surgical residents (n = 59), general surgery attending physicians (n = 36), and surgical nurses (n = 42) at 5 medical centers. Case descriptions and general comments were classified by the nature of ITC lapses and their effects on patients and medical care. Information learned was combined with a review of ITC strategies in other professions to develop principles and guidelines for re-engineering surgeon ITCP. RESULTS : A total of 328 case descriptions and general comments were obtained and classified. Incidents fell into 4 areas: blurred boundaries of responsibility (87 reports), decreased surgeon familiarity with patients (123 reports), diversion of surgeon attention (31 reports), and distorted or inhibited communication (67 reports). Results were subdivided into 30 contributing factors (eg, shift change, location change, number of providers). Consequences of ITC lapses included delays in patient care (77% of cases), wasted surgeon/staff time (48%), and serious adverse patient consequences (31%). Twelve principles and 5 institutional habit changes are recommended to guide ITCP re-engineering. CONCLUSIONS Surgeon communication lapses are significant contributors to adverse patient consequences, and provider inefficiency. Re-engineering ITCP will require significant cultural changes.
Collapse
Affiliation(s)
- Reed G Williams
- Southern Illinois University School of Medicine, Springfield, IL, USA.
| | | | | | | | | | | | | | | | | | | | | |
Collapse
|