1
|
Dexter F, Hindman BJ, Bayman EO, Mueller RN. Patient and Operational Factors Do Not Substantively Affect the Annual Departmental Quality of Anesthesiologists' Clinical Supervision and Nurse Anesthetists' Work Habits. Cureus 2024; 16:e55346. [PMID: 38559506 PMCID: PMC10981928 DOI: 10.7759/cureus.55346] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 02/28/2024] [Indexed: 04/04/2024] Open
Abstract
INTRODUCTION Although safety climate, teamwork, and other non-technical skills in operating rooms probably influence clinical outcomes, direct associations have not been shown, at least partially due to sample size considerations. We report data from a retrospective cohort of anesthesia evaluations that can simplify the design of prospective observational studies in this area. Associations between non-technical skills in anesthesia, specifically anesthesiologists' quality of clinical supervision and nurse anesthetists' work habits, and patient and operational factors were examined. METHODS Eight fiscal years of evaluations and surgical cases from one hospital were included. Clinical supervision by anesthesiologists was evaluated daily using a nine-item scale. Work habits of nurse anesthetists were evaluated daily using a six-item scale. The dependent variables for both groups of staff were binary, whether all items were given the maximum score or not. Associations were tested with patient and operational variables for the entire day. RESULTS There were 40,718 evaluations of faculty anesthesiologists by trainees, 53,772 evaluations of nurse anesthetists by anesthesiologists, and 296,449 cases that raters and ratees started together. Cohen's d values were small (≤0.10) for all independent variables, suggesting a lack of any clinically meaningful association between patient and operational factors and evaluations given the maximum scores. For supervision quality, the day's count of orthopedic cases was a significant predictor of scores (P = 0.0011). However, the resulting absolute marginal change in the percentage of supervision scores equal to the maximum was only 0.8% (99% confidence interval: 0.2% to 1.4%), i.e., too small to be of clinical or managerial importance. Neurosurgical cases may have been a significant predictor of work habits (P = 0.0054). However, the resulting marginal change in the percentage of work habits scores equal to the maximum, an increase of 0.8% (99% confidence interval: 0.1% to 1.6%), which was again too small to be important. CONCLUSIONS When evaluating the effect of assigning anesthesiologists and nurse anesthetists with different clinical performance quality on clinical outcomes, supervision quality and work habits scores may be included as independent variables without concern that their effects are confounded by association with the patient or case characteristics. Clinical supervision and work habits are measures of non-technical skills. Hence, these findings suggest that non-technical performance can be judged by observing the typical small sample size of cases. Then, associations can be tested with administrative data for a far greater number of patients because there is unlikely to be a confounding association between patient and case characteristics and the clinicians' non-technical performance.
Collapse
Affiliation(s)
| | | | - Emine O Bayman
- Biostatistics/Anesthesia, University of Iowa, Iowa City, USA
| | | |
Collapse
|
2
|
Dexter F, Hindman BJ, Epstein RH. Overall anesthesia department quality of clinical supervision of trainees over a year evaluated using mixed effects models. J Clin Anesth 2023; 87:111114. [PMID: 37004458 DOI: 10.1016/j.jclinane.2023.111114] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/02/2023] [Revised: 03/05/2023] [Accepted: 03/19/2023] [Indexed: 04/03/2023]
Abstract
BACKGROUND Earlier studies of supervision in anesthesiology focused on how to evaluate the quality of individual anesthesiologist's clinical supervision of trainees. What is unknown is how to evaluate clinical supervision collectively, as provided by the department's faculty anesthesiologists. This information can be a metric that departments report annually or use to evaluate the effect of programs on the quality of clinical supervision over time. METHODS This retrospective cohort study used all 48,788 evaluations of the 115 faculty anesthesiologists using the De Oliveira Filho supervision scale completed by 202 residents and fellows over nine academic years at one department. RESULTS The distributions of mean scores among raters had marked negative skewness and were inconsistent with normal distributions. Consequently, accurate confidence intervals were impracticably wide, and their interpretation suggested lack of validity. In contrast, the logits of the proportions of scores equaling the maximum possible value, calculated for each rater, followed distributions sufficiently close to normal for statistically reliable use in random effects modeling. Parameters and confidence intervals were estimated using the intercept only random effects models, and then inverses computed to convert results from the logit scale to proportions. That approach is analogous to random effect meta-analysis of proportional incidence (or prevalence). Departments that chose to use semi-annual or annual surveys of trainees regarding supervision quality, and report those raw counts, will have far lower estimates of supervision quality versus when calculated accurately using daily evaluations of individual anesthesiologists. CONCLUSIONS Random effects meta-analysis of percentage incidences of maximum scores is a suitable statistical approach to analyze the daily supervision scores of individual anesthesiologists to evaluate the overall quality of clinical supervision provided to the trainees by the department over a year.
Collapse
|
3
|
Hadler RA, Dexter F, Hindman BJ. Effect of Insufficient Interaction on the Evaluation of Anesthesiologists’ Quality of Clinical Supervision by Anesthesiology Residents and Fellows. Cureus 2022; 14:e23500. [PMID: 35494980 PMCID: PMC9036497 DOI: 10.7759/cureus.23500] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 03/25/2022] [Indexed: 11/16/2022] Open
Abstract
Introduction In this study, we tested whether raters’ (residents and fellows) decisions to evaluate (or not) critical care anesthesiologists were significantly associated with clinical interactions documented from electronic health record progress notes and whether that influenced the reliability of supervision scores. We used the de Oliveira Filho clinical supervision scale for the evaluation of faculty anesthesiologists. Email requests were sent to raters who worked one hour or longer with the anesthesiologist the preceding day in an operating room. In contrast, potential raters were requested to evaluate all critical care anesthesiologists scheduled in intensive care units during the preceding week. Methods Over 7.6 years, raters (N=172) received a total of 7764 requests to evaluate 21 critical care anesthesiologists. Each rater received a median/mode of three evaluation requests, one per anesthesiologist on service that week. In this retrospective cohort study, we related responses (2970 selections of “insufficient interaction” to evaluate the faculty, and 3127 completed supervision scores) to progress notes (N=25,469) electronically co-signed by the rater and anesthesiologist combination during that week. Results Raters with few jointly signed notes were more likely to select insufficient interaction for evaluation (P < 0.0001): 62% when no joint notes versus 1% with at least 20 joint notes during the week. Still, rater-anesthesiologist combinations with no co-authored notes accounted not only for most (78%) of the evaluation requests but also most (56%) of the completed evaluations (both P < 0.0001). Among rater and anesthesiologist combinations with each anesthesiologist receiving evaluations from multiple (at least nine) raters and each rater evaluating multiple anesthesiologists, most (72%) rater-anesthesiologist combinations were among raters who had no co-authored notes with the anesthesiologist (P < 0.0001). Conclusions Regular use of the supervision scale should be practiced with raters who were selected not only from their scheduled clinical site but also using electronic health record data verifying joint workload with the anesthesiologist.
Collapse
|
4
|
Fang YL, Huang HH, Jiang S, Li J, Cheng BW, Tsao CH, Ho ACY. Nurse anesthetist training Center on IFNA standards in Mainland China. NURSE EDUCATION TODAY 2021; 99:104775. [PMID: 33549958 DOI: 10.1016/j.nedt.2021.104775] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/16/2020] [Revised: 12/15/2020] [Accepted: 01/10/2021] [Indexed: 06/12/2023]
Abstract
AIM The first training center for nurse anesthetists in Jiangsu Province, China was established. The aim of this study is to understand the status of and effectiveness of a training program based on International Federation of Nurse Anesthetists (IFNA) standards for nurse anesthetists in China. BACKGROUND China recently issued a clear policy to improve the quality of anesthesia care and to ensure the safety and comfort of patients. As a result, the roles of nurse anesthetists have received more attention, highlighting the importance of the development and implementation of relevant training. INTRODUCTION IFNA standards were applied to the development of a specialized training program for nurse anesthetists, which is expected to serve as a reference for various medical institutions and to lead to increased uniformity and improvement in the quality and efficiency of care. METHODS These standards are divided into two indicators: curriculum and core competencies. Through questionnaires, course evaluations, and satisfaction surveys before and after the implementation of the training program, the effectiveness of the training program was analyzed. RESULTS Theoretical assessment of the program was higher after its completion than before and the core competencies of nurse anesthetists significantly improved. Overall, the curriculum was highly satisfactory and important. DISCUSSION Curriculum design was based on IFNA standards. Nurse anesthetists who are trained in a systematic way can work better with other professionals in providing anesthesia care. The objectives are to improve anesthesia care, reduce errors, improve physician satisfaction, and allow nurse anesthetists to realize their value. Overall, quality of anesthesia care improved after implementation of IFNA-standard training.
Collapse
Affiliation(s)
- Yu-Lan Fang
- Department of Anesthesiology, BenQ Medical Center, The Affiliated BenQ Hospital of Nanjing Medical University, Nanjing, Jiangsu Province, China; Institue for Hospital Management of Tsinghua University, Beijing, China
| | - Hsin-Hsin Huang
- Institute of Medicine, Chung Shan Medical University, Taichung, Taiwan
| | - Shunshun Jiang
- Department of Anesthesiology, BenQ Medical Center, The Affiliated BenQ Hospital of Nanjing Medical University, Nanjing, Jiangsu Province, China
| | - Jinxin Li
- Department of Anesthesiology, BenQ Medical Center, The Affiliated BenQ Hospital of Nanjing Medical University, Nanjing, Jiangsu Province, China
| | - Bor-Wen Cheng
- Department of Industrial Engineering, National Yunlin University of Science and Technology, Yunlin, Taiwan
| | - Chien-Han Tsao
- Department of Otolaryngology, Chung Shan Medical University Hospital, Taichung, Taiwan; School of Medicine, Chung Shan Medical University, Taichung, Taiwan
| | - Angie Chi-Yueh Ho
- Department of Anesthesiology, BenQ Medical Center, The Affiliated BenQ Hospital of Nanjing Medical University, Nanjing, Jiangsu Province, China.
| |
Collapse
|
5
|
Association between leniency of anesthesiologists when evaluating certified registered nurse anesthetists and when evaluating didactic lectures. Health Care Manag Sci 2020; 23:640-648. [PMID: 32946045 DOI: 10.1007/s10729-020-09518-0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/02/2019] [Accepted: 08/03/2020] [Indexed: 10/23/2022]
Abstract
Daily evaluations of certified registered nurse anesthetists' (CRNAs') work habits by anesthesiologists should be adjusted for rater leniency. The current study tested the hypothesis that there is a pairwise association by rater between leniencies of evaluations of CRNAs' daily work habits and of didactic lectures. The historical cohorts were anesthesiologists' evaluations over 53 months of CRNAs' daily work habits and 65 months of didactic lectures by visiting professors and faculty. The binary endpoints were the Likert scale scores for all 6 and 10 items, respectively, equaling the maximums of 5 for all items, or not. Mixed effects logistic regression estimated the odds of each ratee performing above or below average adjusted for rater leniency. Bivariate errors in variables least squares linear regression estimated the association between the leniency of the anesthesiologists' evaluations of work habits and didactic lectures. There were 29/107 (27%) raters who were more severe in their evaluations of CRNAs' work habits than other anesthesiologists (two-sided P < 0.01); 34/107 (32%) raters were more lenient. When evaluating lectures, 3/81 (4%) raters were more severe and 8/81 (10%) more lenient. Among the 67 anesthesiologists rating both, leniency (or severity) for work habits was not associated with that for lectures (P = 0.90, unitless slope between logits 0.02, 95% confidence interval -0.34 to 0.30). Rater leniency is of large magnitude when making daily clinical evaluations, even when using a valid and psychometrically reliable instrument. Rater leniency was context dependent, not solely a reflection of raters' personality or rating style.
Collapse
|
6
|
Reliability of ranking anesthesiologists and nurse anesthetists using leniency-adjusted clinical supervision and work habits scores. J Clin Anesth 2020; 61:109639. [DOI: 10.1016/j.jclinane.2019.109639] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/30/2019] [Revised: 08/22/2019] [Accepted: 10/07/2019] [Indexed: 11/18/2022]
|
7
|
Dexter F, Ledolter J, Wong CA, O'Brien MK, Hindman BJ. Nurse anesthetists' preferences for anesthesiologists' participation in patient care at a large teaching hospital. J Clin Anesth 2019; 57:131-138. [DOI: 10.1016/j.jclinane.2019.04.018] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/16/2019] [Revised: 03/29/2019] [Accepted: 04/10/2019] [Indexed: 10/27/2022]
|
8
|
Epstein RH, Dexter F, O'Neill L. Development and Validation of an Algorithm to Classify as Equivalent the Procedures in ICD-10-PCS That Differ Only by Laterality. Anesth Analg 2019; 128:1138-1144. [PMID: 31094780 DOI: 10.1213/ane.0000000000003340] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/05/2022]
Abstract
BACKGROUND The switch from International Classification of Diseases, Ninth Revision, Clinical Modification to International Classification of Diseases, Tenth Revision, Procedure Coding System (ICD-10-PCS) for coding of inpatient procedures in the United States increased the number of procedural codes more than 19-fold, in large part due to the addition of laterality. We examined ICD-10-PCS codes for pairs of mirror-image procedures that are surgically equivalent. METHODS We developed an algorithm in structured query language (SQL) to identify ICD-10-PCS codes differing only by laterality. We quantified the impact of laterality on the number of commonly performed major therapeutic procedures (ie, surgical diversity) using 2 quarters of discharge abstracts from Texas. RESULTS Of the 75,789 ICD-10-PCS codes from federal fiscal year 2017, 16,839 (22.3%) pairs differed only by laterality (with each pair contributing 2 codes). With the combining of equivalent codes, diversity in the state of Texas decreased from 78.2 to 74.1 operative procedures (95% confidence interval, 5.1 to -3.1; P < .001). CONCLUSIONS Our algorithm identifies ICD-10-PCS codes that differ only by laterality. However, laterality had a small effect on surgical diversity among major therapeutic procedures. Our SQL code and the lookup table will be useful for all US inpatient analyses of ICD-10-PCS surgical data, because combining procedures differing only by laterality will often be desired.
Collapse
Affiliation(s)
- Richard H Epstein
- From the Department of Anesthesiology, Pain Management and Perioperative Medicine, University of Miami, Miami, Florida
| | - Franklin Dexter
- Division of Management Consulting, Department of Anesthesia, University of Iowa, Iowa City, Iowa
| | - Liam O'Neill
- Department of Health Behavior and Health Systems, School of Public Health University of North Texas-Health Science Center, Fort Worth, Texas
| |
Collapse
|
9
|
O'Brien MK, Dexter F, Kreiter CD, Slater-Scott C, Hindman BJ. Nurse anesthetists' evaluations of anesthesiologists' operating room performance are sensitive to anesthesiologists' years of postgraduate practice. J Clin Anesth 2019; 54:102-110. [DOI: 10.1016/j.jclinane.2018.10.027] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/19/2018] [Revised: 10/03/2018] [Accepted: 10/28/2018] [Indexed: 10/27/2022]
|
10
|
Danhakl V, Miltiades A, Ing C, Chang B, Edmondson D, Landau R, Gallos G. Observational study evaluating obstetric anesthesiologist residents' well-being, anxiety and stress in a North American academic program. Int J Obstet Anesth 2018; 38:75-82. [PMID: 30509676 DOI: 10.1016/j.ijoa.2018.10.011] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 04/04/2018] [Revised: 07/23/2018] [Accepted: 10/26/2018] [Indexed: 10/28/2022]
Abstract
BACKGROUND The obstetric work environment has a unique set of stressors that may be associated with burnout. We investigated how well-being during the obstetric anesthesia (ObA) rotation compared to other rotations; which workplace environment characteristics precipitated the greatest stress; and whether anxiety and stress levels changed in trainees before and after an ObA rotation. METHODS Using a survey, anesthesia residents (n=36) ranked their well-being on each anesthesia rotation and answered questions about their work environment. A separate survey measured anxiety and stress before and after an ObA rotation. Friedman's test was used to compare ranking data and Likert responses. T-tests were used to compare stress and anxiety scores. RESULTS Residents' ranking of well-being on ObA was higher than that on another high demand rotation (cardiothoracic anesthesia, P=0.007). Work environment stress scores were significantly higher among community and fairness domains than for workload (P=0.002 and P=0.0001, respectively). While stress and anxiety scores did not significantly differ before and after the ObA rotation, they were higher than the reference population scores. CONCLUSIONS We provide the first example of tools for assessing work environment stressors in ObA. Our study illustrates that beyond excessive workload, lack of fairness and community values are areas that impact physician well-being. Use of these tools can guide initiatives to address work environment concerns, and presents a need for a validated well-being instrument to gauge physician well-being, in order to create a cultural shift from burnout to one of well-being.
Collapse
Affiliation(s)
- V Danhakl
- Department of Anesthesiology, Columbia University Medical Center, United States of America.
| | - A Miltiades
- Department of Anesthesiology, Columbia University Medical Center, United States of America
| | - C Ing
- Department of Anesthesiology, Columbia University Medical Center, United States of America
| | - B Chang
- Department of Emergency Medicine, Columbia University Medical Center, United States of America
| | - D Edmondson
- Center for Behavioral Cardiovascular Health, Columbia University Medical Center, United States of America
| | - R Landau
- Department of Anesthesiology, Columbia University Medical Center, United States of America
| | - G Gallos
- Department of Anesthesiology, Columbia University Medical Center, United States of America
| |
Collapse
|
11
|
Dexter F, Jarvie C, Epstein RH. Heterogeneity among hospitals statewide in percentage shares of the annual growth of surgical caseloads of inpatient and outpatient major therapeutic procedures. J Clin Anesth 2018; 49:126-130. [DOI: 10.1016/j.jclinane.2018.04.003] [Citation(s) in RCA: 14] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/20/2018] [Revised: 03/27/2018] [Accepted: 04/07/2018] [Indexed: 11/27/2022]
|
12
|
Hospital physicians can’t get no long-term satisfaction – an indicator for fairness in preference fulfillment on duty schedules. Health Care Manag Sci 2018; 22:691-708. [DOI: 10.1007/s10729-018-9452-8] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/28/2018] [Accepted: 07/11/2018] [Indexed: 10/28/2022]
|
13
|
Dexter F, Epstein RH, Thenuwara K, Lubarsky DA. Large Variability in the Diversity of Physiologically Complex Surgical Procedures Exists Nationwide Among All Hospitals Including Among Large Teaching Hospitals. Anesth Analg 2018; 127:190-197. [DOI: 10.1213/ane.0000000000002634] [Citation(s) in RCA: 10] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/05/2022]
|
14
|
Years Versus Days Between Successive Surgeries, After an Initial Outpatient Procedure, for the Median Patient Versus the Median Surgeon in the State of Iowa. Anesth Analg 2018; 126:787-793. [DOI: 10.1213/ane.0000000000002774] [Citation(s) in RCA: 5] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/05/2022]
|
15
|
Anesthesiologists' perceptions of minimum acceptable work habits of nurse anesthetists. J Clin Anesth 2017; 38:107-110. [DOI: 10.1016/j.jclinane.2017.01.031] [Citation(s) in RCA: 9] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/27/2016] [Revised: 01/17/2017] [Accepted: 01/21/2017] [Indexed: 11/24/2022]
|