1
|
Andersen BR, Ammitzbøll I, Hinrich J, Lehmann S, Ringsted CV, Løkkegaard ECL, Tolsgaard MG. Using machine learning to identify quality-of-care predictors for emergency caesarean sections: a retrospective cohort study. BMJ Open 2022; 12:e049046. [PMID: 35256439 PMCID: PMC8905885 DOI: 10.1136/bmjopen-2021-049046] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 11/06/2022] Open
Abstract
OBJECTIVES Emergency caesarean sections (ECS) are time-sensitive procedures. Multiple factors may affect team efficiency but their relative importance remains unknown. This study aimed to identify the most important predictors contributing to quality of care during ECS in terms of the arrival-to-delivery interval. DESIGN A retrospective cohort study. ECS were classified by urgency using emergency categories one/two and three (delivery within 30 and 60 min). In total, 92 predictor variables were included in the analysis and grouped as follows: 'Maternal objective', 'Maternal psychological', 'Fetal factors', 'ECS Indication', 'Emergency category', 'Type of anaesthesia', 'Team member qualifications and experience' and 'Procedural'. Data was analysed with a linear regression model using elastic net regularisation and jackknife technique to improve generalisability. The relative influence of the predictors, percentage significant predictor weight (PSPW) was calculated for each predictor to visualise the main determinants of arrival-to-delivery interval. SETTING AND PARTICIPANTS Patient records for mothers undergoing ECS between 2010 and 2017, Nordsjællands Hospital, Capital Region of Denmark. PRIMARY OUTCOME MEASURES Arrival-to-delivery interval during ECS. RESULTS Data was obtained from 2409 patient records for women undergoing ECS. The group of predictors representing 'Team member qualifications and experience' was the most important predictor of arrival-to-delivery interval in all ECS emergency categories (PSPW 25.9% for ECS category one/two; PSPW 35.5% for ECS category three). In ECS category one/two the 'Indication for ECS' was the second most important predictor group (PSPW 24.9%). In ECS category three, the second most important predictor group was 'Maternal objective predictors' (PSPW 24.2%). CONCLUSION This study provides empirical evidence for the importance of team member qualifications and experience relative to other predictors of arrival-to-delivery during ECS. Machine learning provides a promising method for expanding our current knowledge about the relative importance of different factors in predicting outcomes of complex obstetric events.
Collapse
Affiliation(s)
- Betina Ristorp Andersen
- Department of Gynecology and Obstetrics, Nordsjællands Hospital & Department of Clinical Medicine, University of Copenhagen, Hillerod, Capital Region, Denmark
| | - Ida Ammitzbøll
- Department of Gynecology and Obstetrics, Nordsjællands Hospital & Department of Clinical Medicine, University of Copenhagen, Hillerod, Capital Region, Denmark
| | - Jesper Hinrich
- Cognitive Systems, Department of Applied Mathematics and Computer Science, Technical University of Denmark, Lyngby, Denmark
| | - Sune Lehmann
- Cognitive Systems, Department of Applied Mathematics and Computer Science, Technical University of Denmark, Lyngby, Denmark
| | | | - Ellen Christine Leth Løkkegaard
- Department of Gynecology and Obstetrics, Nordsjællands Hospital & Department of Clinical Medicine, University of Copenhagen, Hillerod, Capital Region, Denmark
| | - Martin G Tolsgaard
- Copenhagen Academy of Medical Education and Simulation, Rigshospitalet, Kobenhavn, Capital Region, Denmark
| |
Collapse
|
2
|
Williams JC, Ireland T, Warman S, Cake MA, Dymock D, Fowler E, Baillie S. Instruments to measure the ability to self-reflect: A systematic review of evidence from workplace and educational settings including health care. EUROPEAN JOURNAL OF DENTAL EDUCATION : OFFICIAL JOURNAL OF THE ASSOCIATION FOR DENTAL EDUCATION IN EUROPE 2019; 23:389-404. [PMID: 31108006 DOI: 10.1111/eje.12445] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/26/2018] [Accepted: 05/16/2019] [Indexed: 05/17/2023]
Abstract
INTRODUCTION Self-reflection has become recognised as a core skill in dental education, although the ability to self-reflect is valued and measured within several professions. This review appraises the evidence for instruments available to measure the self-reflective ability of adults studying or working within any setting, not just health care. MATERIALS AND METHODS A systematic review was conducted of 20 electronic databases (including Medline, ERIC, CINAHL and Business Source Complete) from 1975 to 2017, supplemented by citation searches. Data were extracted from each study and the studies graded against quality indicators by at least two independent reviewers, using a coding sheet. Reviewers completed a utility analysis of the assessment instruments described within included studies, appraising their reported reliability, validity, educational impact, acceptability and cost. RESULTS A total of 131 studies met the inclusion criteria. Eighteen were judged to provide higher quality evidence for the review and three broad types of instrument were identified, namely: rubrics (or scoring guides), self-reported scales and observed behaviour. CONCLUSIONS Three types of instrument were identified to assess the ability to self-reflect. It was not possible to recommend a single most effective instrument due to under reporting of the criteria necessary for a full utility analysis of each. The use of more than one instrument may therefore be appropriate dependent on the acceptability to the faculty, assessor, student and cost. Future research should report on the utility of assessment instruments and provide guidance on what constitutes thresholds of acceptable or unacceptable ability to self-reflect, and how this should be managed.
Collapse
Affiliation(s)
- Julie C Williams
- Bristol Dental School, Faculty of Health Sciences, University of Bristol, Bristol, UK
| | - Tony Ireland
- Bristol Dental School, Faculty of Health Sciences, University of Bristol, Bristol, UK
| | - Sheena Warman
- Bristol Veterinary School, Faculty of Health Sciences, University of Bristol, Bristol, UK
| | - Martin A Cake
- School of Veterinary and Biomedical Sciences, Murdoch University, Perth, Western Australia, Australia
| | - David Dymock
- Bristol Dental School, Faculty of Health Sciences, University of Bristol, Bristol, UK
| | - Ellayne Fowler
- Centre for Medical Education, University of Bristol, Bristol, UK
| | - Sarah Baillie
- Bristol Veterinary School, Faculty of Health Sciences, University of Bristol, Bristol, UK
| |
Collapse
|
3
|
Lemos JD, Hernandez AM, Soto-Romero G. An Instrumented Glove to Assess Manual Dexterity in Simulation-Based Neurosurgical Education. SENSORS 2017; 17:s17050988. [PMID: 28468268 PMCID: PMC5469341 DOI: 10.3390/s17050988] [Citation(s) in RCA: 12] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 10/31/2016] [Revised: 04/01/2017] [Accepted: 04/07/2017] [Indexed: 12/03/2022]
Abstract
The traditional neurosurgical apprenticeship scheme includes the assessment of trainee’s manual skills carried out by experienced surgeons. However, the introduction of surgical simulation technology presents a new paradigm where residents can refine surgical techniques on a simulator before putting them into practice in real patients. Unfortunately, in this new scheme, an experienced surgeon will not always be available to evaluate trainee’s performance. For this reason, it is necessary to develop automatic mechanisms to estimate metrics for assessing manual dexterity in a quantitative way. Authors have proposed some hardware-software approaches to evaluate manual dexterity on surgical simulators. This paper presents IGlove, a wearable device that uses inertial sensors embedded on an elastic glove to capture hand movements. Metrics to assess manual dexterity are estimated from sensors signals using data processing and information analysis algorithms. It has been designed to be used with a neurosurgical simulator called Daubara NS Trainer, but can be easily adapted to another benchtop- and manikin-based medical simulators. The system was tested with a sample of 14 volunteers who performed a test that was designed to simultaneously evaluate their fine motor skills and the IGlove’s functionalities. Metrics obtained by each of the participants are presented as results in this work; it is also shown how these metrics are used to automatically evaluate the level of manual dexterity of each volunteer.
Collapse
Affiliation(s)
- Juan Diego Lemos
- Bioinstrumentation and Clinical Engineering Research Group-GIBIC, Bioengineering Department, Engineering Faculty, Universidad de Antioquia UdeA, Calle 70 No. 52-21, Medellín 050010, Colombia.
| | - Alher Mauricio Hernandez
- Bioinstrumentation and Clinical Engineering Research Group-GIBIC, Bioengineering Department, Engineering Faculty, Universidad de Antioquia UdeA, Calle 70 No. 52-21, Medellín 050010, Colombia.
| | - Georges Soto-Romero
- LAAS-CNRS, Université de Toulouse, CNRS, Toulouse 31400, France.
- ISIFC, Université de Franche-Comté, Besançon 25000, France.
| |
Collapse
|
4
|
Shalhoub J, Marshall DC, Ippolito K. Perspectives on procedure-based assessments: a thematic analysis of semistructured interviews with 10 UK surgical trainees. BMJ Open 2017; 7:e013417. [PMID: 28341687 PMCID: PMC5372077 DOI: 10.1136/bmjopen-2016-013417] [Citation(s) in RCA: 26] [Impact Index Per Article: 3.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 11/04/2022] Open
Abstract
OBJECTIVES The introduction of competency-based training has necessitated development and implementation of accompanying mechanisms for assessment. Procedure-based assessments (PBAs) are an example of workplace-based assessments that are used to examine focal competencies in the workplace. The primary objective was to understand surgical trainees' perspective on the value of PBA. DESIGN Semistructured interviews with 10 surgical trainees individually interviewed to explore their views. Interviews were audio-recorded and transcribed; following this, they were open and axial coded. Thematic analysis was then performed. RESULTS Semistructured interviews yielded several topical and recurring themes. In trainees' experience, the use of PBAs as a summative tool limits their educational value. Trainees reported a lack of support from seniors and variation in the usefulness of the tool based on stage of training. Concerns related to the validity of PBAs for evaluating trainees' performance with reports of 'gaming' the system and trainees completing their own assessments. Trainees did identify the significant value of PBAs when used correctly. Benefits included the identification of additional learning opportunities, standardisation of assessment and their role in providing a measure of progress. CONCLUSIONS The UK surgical trainees interviewed identified both limitations and benefits to PBAs; however, we would argue based on their responses and our experience that their use as a summative tool limits their formative use as an educational opportunity. PBAs should either be used exclusively to support learning or solely as a summative tool; if so, further work is needed to audit, validate and standardise them for this purpose.
Collapse
Affiliation(s)
- Joseph Shalhoub
- Faculty of Medicine, Department of Surgery & Cancer, Imperial College London, London, UK
| | | | - Kate Ippolito
- Educational Development Unit, School of Professional Development, Imperial College London, London, UK
| |
Collapse
|
5
|
Shalhoub J, Vesey AT, Fitzgerald JEF. What evidence is there for the use of workplace-based assessment in surgical training? JOURNAL OF SURGICAL EDUCATION 2014; 71:906-915. [PMID: 24831439 DOI: 10.1016/j.jsurg.2014.03.013] [Citation(s) in RCA: 12] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/07/2014] [Revised: 02/24/2014] [Accepted: 03/25/2014] [Indexed: 06/03/2023]
Abstract
BACKGROUND Recent years have seen broad changes in postgraduate training with a move toward structured formative and summative evaluation of trainees' competencies using workplace-based assessment (WBA) tools. The fitness for purpose of these instruments in surgery has been much debated. The aim of this study is to explore the evidence underlying the introduction and ongoing use of WBAs in surgical training. METHODS A critical literature review was conducted to identify studies evaluating the use of WBAs in postgraduate surgical training. The search was conducted using the electronic databases PubMed for full-text articles in English. Additional critical evaluations of the curriculum relating to WBAs were included. The articles were synthesized in a narrative review. RESULTS The implementation of WBA requirements in surgical training has occurred despite a relative dearth of direct evidence of their efficacy and benefit. Studies and critical reviews are being regularly undertaken to ensure that supporting evidence is accrued and the drive for improvement and refinement is maintained. It is emerging that WBAs are (contrary to their current nomenclature) formative tools for feedback and hence learning. They can facilitate the progression toward expert practice at the center of the zone of proximal development and the higher levels of Miller's pyramid, but fall short--owing to their focus on competence--of guiding surgical trainees to the higher levels of Maslow's hierarchy. CONCLUSIONS Limited evidence has potentially undermined the introduction of WBAs in surgical training to date. There are misunderstandings regarding their use as either summative or formative educational tools. These shortcomings are an opportunity for further work in examining WBAs in their current or modified form.
Collapse
Affiliation(s)
- Joseph Shalhoub
- Imperial College London, United Kingdom; Association of Surgeons in Training, London, United Kingdom.
| | - Alex Thomas Vesey
- Association of Surgeons in Training, London, United Kingdom; University of Edinburgh, United Kingdom
| | | |
Collapse
|
6
|
Arah OA, Hoekstra JBL, Bos AP, Lombarts KMJMH. New tools for systematic evaluation of teaching qualities of medical faculty: results of an ongoing multi-center survey. PLoS One 2011; 6:e25983. [PMID: 22022486 PMCID: PMC3193529 DOI: 10.1371/journal.pone.0025983] [Citation(s) in RCA: 53] [Impact Index Per Article: 4.1] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/17/2011] [Accepted: 09/14/2011] [Indexed: 11/19/2022] Open
Abstract
Background Tools for the evaluation, improvement and promotion of the teaching excellence of faculty remain elusive in residency settings. This study investigates (i) the reliability and validity of the data yielded by using two new instruments for evaluating the teaching qualities of medical faculty, (ii) the instruments' potential for differentiating between faculty, and (iii) the number of residents' evaluations needed per faculty to reliably use the instruments. Methods and Materials Multicenter cross-sectional survey among 546 residents and 629 medical faculty representing 29 medical (non-surgical) specialty training programs in the Netherlands. Two instruments—one completed by residents and one by faculty—for measuring teaching qualities of faculty were developed. Statistical analyses included factor analysis, reliability and validity exploration using standard psychometric methods, calculation of the numbers of residents' evaluations needed per faculty to achieve reliable assessments and variance components and threshold analyses. Results A total of 403 (73.8%) residents completed 3575 evaluations of 570 medical faculty while 494 (78.5%) faculty self-evaluated. In both instruments five composite-scales of faculty teaching qualities were detected with high internal consistency and reliability: learning climate (Cronbach's alpha of 0.85 for residents' instrument, 0.71 for self-evaluation instrument, professional attitude and behavior (0.84/0.75), communication of goals (0.90/0.84), evaluation of residents (0.91/0.81), and feedback (0.91/0.85). Faculty tended to evaluate themselves higher than did the residents. Up to a third of the total variance in various teaching qualities can be attributed to between-faculty differences. Some seven residents' evaluations per faculty are needed for assessments to attain a reliability level of 0.90. Conclusions The instruments for evaluating teaching qualities of medical faculty appear to yield reliable and valid data. They are feasible for use in medical residencies, can detect between-faculty differences and supply potentially useful information for improving graduate medical education.
Collapse
Affiliation(s)
- Onyebuchi A. Arah
- Department of Epidemiology, UCLA School of Public Health, University of California Los Angeles, Los Angeles, California, United States of America
- UCLA Center for Health Policy Research, Los Angeles, California, United States of America
- Department of Quality Management and Process Innovation, Academic Medical Center, University of Amsterdam, Amsterdam, The Netherlands
| | - Joost B. L. Hoekstra
- Department of Internal Medicine, Academic Medical Center, University of Amsterdam, Amsterdam, The Netherlands
| | - Albert P. Bos
- Department of Pediatrics, Academic Medical Center, University of Amsterdam, Amsterdam, The Netherlands
| | - Kiki M. J. M. H. Lombarts
- Department of Quality Management and Process Innovation, Academic Medical Center, University of Amsterdam, Amsterdam, The Netherlands
- * E-mail:
| |
Collapse
|