Mathematical model of the risk of drug error during anaesthesia: the influence of drug choices, injection routes, operation duration and fatigue

Clinical Decision Support as a Prevention Tool for Medication Errors in the Operating Room: A Retrospective Cross-Sectional Study

BACKGROUND

Medication errors in the operating room have high potential for patient harm. While electronic clinical decision support (CDS) software has been effective in preventing medication errors in many nonoperating room patient care areas, it is not yet widely used in operating rooms. The purpose of this study was to determine the percentage of self-reported intraoperative medication errors that could be prevented by CDS algorithms.

METHODS

In this retrospective cross-sectional study, we obtained safety reports involving medication errors documented by anesthesia clinicians between August 2020 and August 2022 at a 1046-bed tertiary care academic medical center. Reviewers classified each medication error by its stage in the medication use process, error type, presence of an adverse medication event, and its associated severity and preventability by CDS. Informational gaps were corroborated by retrospective chart review and disagreements between reviewers were resolved by consensus. The primary outcome was the percentage of errors that were preventable by CDS. Secondary outcomes were preventability by CDS stratified by medication error type and severity.

RESULTS

We received 127 safety reports involving 80 medication errors, and 76/80 (95%) of the errors were classified as preventable by CDS. Certain error types were more likely to be preventable by CDS than others (P < .001). The most likely error types to be preventable by CDS were wrong medication (N = 36, 100% rated as preventable), wrong dose (N = 30, 100% rated as preventable), and documentation errors (N = 3, 100% rated as preventable). The least likely error type to be preventable by CDS was inadvertent bolus (N = 3, none rated as preventable).

CONCLUSIONS

Ninety-five percent of self-reported medication errors in the operating room were classified as preventable by CDS. Future research should include a randomized controlled trial to assess medication error rates and types with and without the use of CDS.

Surgery duration: Optimized prediction and causality analysis

Accurate estimation of duration of surgery (DOS) can lead to cost-effective utilization of surgical staff and operating rooms and decrease patients’ waiting time. In this study, we present a supervised DOS nonlinear regression prediction model whose accuracy outperforms earlier results. In addition, unlike previous studies, we identify the features that influence DOS prediction. Further, in difference from others, we study the causal relationship between the feature set and DOS. The feature sets used in prior studies included a subset of the features presented in this study. This study aimed to derive influential effectors of duration of surgery via optimized prediction and causality analysis. We implemented an array of machine learning algorithms and trained them on datasets comprising surgery-related data, to derive DOS prediction models. The datasets we acquired contain patient, surgical staff, and surgery features. The datasets comprised 23,293 surgery records of eight surgery types performed over a 10-year period in a public hospital. We have introduced new, unstudied features and combined them with features adopted from previous studies to generate a comprehensive feature set. We utilized feature importance methods to identify the influential features, and causal inference methods to identify the causal features. Our model demonstrates superior performance in comparison to DOS prediction models in the art. The performance of our DOS model in terms of the mean absolute error (MAE) was 14.9 minutes. The algorithm that derived the model with the best performance was the gradient boosted trees (GBT). We identified the 10 most influential features and the 10 most causal features. In addition, we showed that 40% of the influential features have a significant (p-value = 0.05) causal relationship with DOS. We developed a DOS prediction model whose accuracy is higher than that of prior models. This improvement is achieved via the introduction of a novel feature set on which the model was trained. Utilizing our prediction model, hospitals can improve the efficiency of surgery schedules, and by exploiting the identified causal relationship, can influence the DOS. Further, the feature importance methods we used can help explain the model’s predictions.

Continuous real-time prediction of surgical case duration using a modular artificial neural network

BACKGROUND

Real-time prediction of surgical duration can inform perioperative decisions and reduce surgical costs. We developed a machine learning approach that continuously incorporates preoperative and intraoperative information for forecasting surgical duration.

METHODS

Preoperative (e.g. procedure name) and intraoperative (e.g. medications and vital signs) variables were retrieved from anaesthetic records of surgeries performed between March 1, 2019 and October 31, 2019. A modular artificial neural network was developed and compared with a Bayesian approach and the scheduled surgical duration. Continuous ranked probability score (CRPS) was used as a measure of time error to assess model accuracy. For evaluating clinical performance, accuracy for each approach was assessed in identifying cases that ran beyond 15:00 (commonly scheduled end of shift), thus identifying opportunities to avoid overtime labour costs.

RESULTS

The analysis included 70 826 cases performed at eight hospitals. The modular artificial neural network had the lowest time error (CRPS: mean=13.8; standard deviation=35.4 min), which was significantly better (mean difference=6.4 min [95% confidence interval: 6.3-6.5]; P<0.001) than the Bayesian approach. The modular artificial neural network also had the highest accuracy in identifying operating theatres that would overrun 15:00 (accuracy at 1 h prior=89%) compared with the Bayesian approach (80%) and a naïve approach using the scheduled duration (78%).

CONCLUSIONS

A real-time neural network model using preoperative and intraoperative data had significantly better performance than a Bayesian approach or scheduled duration, offering opportunities to avoid overtime labour costs and reduce the cost of surgery by providing superior real-time information for perioperative decision support.

An integrative review of method types used in the study of medication error during anaesthesia: implications for estimating incidence

To meet the WHO vision of reducing medication errors by 50%, it is essential to know the current error rate. We undertook an integrative review of the literature, using a systematic search strategy. We included studies that provided an estimate of error rate (i.e. both numerator and denominator data), regardless of type of study (e.g. RCT or observational study). Under each method type, we categorised the error rate by type, by classification used by the primary studies (e.g. wrong drug, wrong dose, wrong time), and then pooled numerator and denominator data across studies to obtain an aggregate error rate for each method type. We included a total of 30 studies in this review. Of these, two studies were national audit projects containing relevant data, and for 28 studies we identified five discrete method types: retrospective recall (6), self-reporting (7), observational (5), large databases (7), and observing for drug calculation errors (3). Of these 28 studies we included 22 for a numerical analysis and used six to inform a narrative review. Drug error is recalled by ~1 in 5 anaesthetists as something that happened over their career; in self-reports there is an admitted rate of ~1 in 200 anaesthetics. In observed practice, error is seen in almost every anaesthetic. In large databases, drug error constitutes ~10% of anaesthesia incidents reported. Wrong drug or dose form the most common type of error across all five study method types (especially dosing error in paediatric studies). We conclude that medication error is common in anaesthetic practice, although we were uncertain of the precise frequency or extent of harm. Studies concerning medication error are very heterogenous, and we recommend consideration of standardised reporting as in other research domains.

Probabilistic forecasting of surgical case duration using machine learning: model development and validation

OBJECTIVE

Accurate estimations of surgical case durations can lead to the cost-effective utilization of operating rooms. We developed a novel machine learning approach, using both structured and unstructured features as input, to predict a continuous probability distribution of surgical case durations.

MATERIALS AND METHODS

The data set consisted of 53 783 surgical cases performed over 4 years at a tertiary-care pediatric hospital. Features extracted included categorical (American Society of Anesthesiologists [ASA] Physical Status, inpatient status, day of week), continuous (scheduled surgery duration, patient age), and unstructured text (procedure name, surgical diagnosis) variables. A mixture density network (MDN) was trained and compared to multiple tree-based methods and a Bayesian statistical method. A continuous ranked probability score (CRPS), a generalized extension of mean absolute error, was the primary performance measure. Pinball loss (PL) was calculated to assess accuracy at specific quantiles. Performance measures were additionally evaluated on common and rare surgical procedures. Permutation feature importance was measured for the best performing model.

RESULTS

MDN had the best performance, with a CRPS of 18.1 minutes, compared to tree-based methods (19.5-22.1 minutes) and the Bayesian method (21.2 minutes). MDN had the best PL at all quantiles, and the best CRPS and PL for both common and rare procedures. Scheduled duration and procedure name were the most important features in the MDN.

CONCLUSIONS

Using natural language processing of surgical descriptors, we demonstrated the use of ML approaches to predict the continuous probability distribution of surgical case durations. The more discerning forecast of the ML-based MDN approach affords opportunities for guiding intelligent schedule design and day-of-surgery operational decisions.

Rational performance metrics for operating theatres, principles of efficiency, and how to achieve it

BACKGROUND

Several performance metrics are commonly used by National Health Service (NHS) organizations to measure the efficiency and productivity of operating lists. These include: start time, utilization, cancellations, number of operations and gap time between operations. The authors describe reasons why these metrics are flawed, and use clinical evidence and mathematics to define a rational, balanced efficiency metric.

METHODS

A narrative review of literature on the efficiency and productivity of elective NHS operating lists was undertaken. The aim was to rationalize how best to define and measure the efficiency of an operating list, and describe strategies to achieve it.

RESULTS

There is now a wealth of literature on how optimally to measure the performance of elective surgical lists. Efficiency may be defined as the completion of all scheduled operations within the allocated time with no over- or under-runs.

CONCLUSION

Achieving efficiency requires appropriate scheduling using specific procedure mean (or median) times and their associated variance (standard deviation or interquartile range) to calculate the probability they can be completed on time. The case mix may be adjusted to yield better time management. This review outlines common misconceptions applied to managing scheduled operating theatre lists and the challenges of measuring unscheduled operations in emergency settings.

Rational planning of operating lists: a prospective comparison of 'booking to the mean' vs. 'probabilistic case scheduling' in urology

The efficient use of operating theatres requires accurate case scheduling. One common method is 'booking to the mean'. Here, the mean times for individual operations are summed to approximate the time allocated to the list. An alternative approach is 'probabilistic scheduling'. Here, the means and standard deviation of the individual case times are combined to estimate the probability that the planned list will finish on time. This study assessed how probabilistic booking would have changed list utilisation, over-running and case cancellations in 60 urology lists during eight months that had been 'booked to the mean'. Booking to the mean resulted in 53/60 (88%) lists over-running and correctly predicted the finish times in just 13% of lists. Out of 264 patients, 36 (14%) were cancelled on the day due to over-runs in 24/60 (40%) lists. In contrast, probabilistic scheduling correctly predicted an over-run or under-run in 77% of lists, which would have allowed the case mix to be adjusted to prevent cancellation and optimise utilisation.

A national survey of out-of-hours working and fatigue in consultants in anaesthesia and paediatric intensive care in the UK and Ireland

The tragic death of an anaesthetic trainee driving home after a series of night shifts prompted a national survey of fatigue in trainee anaesthetists. This indicated that fatigue was widespread, with significant impact on trainees' health and well-being. Consultants deliver an increasing proportion of patient care resulting in long periods of continuous daytime duty and overnight on-call work, so we wished to investigate their experience of out-of-hours working and the causes and impact of work-related fatigue. We conducted a national survey of consultant anaesthetists and paediatric intensivists in the UK and Ireland between 25 June and 6 August 2018. The response rate was 46% (94% of hospitals were represented): 84% of respondents (95%CI 83.1-84.9%) contribute to a night on-call rota with 32% (30.9-33.1%) working 1:8 or more frequently. Sleep disturbance on-call is common: 47% (45.6-48.4%) typically receive two to three phone calls overnight, and 48% (46.6-49.4%) take 30 min or more to fall back to sleep. Only 15% (14.0-16.0%) reported always achieving 11 h of rest between their on-call and their next clinical duty, as stipulated by the European Working Time Directive. Moreover, 24% (22.8-25.2%) stated that there is no departmental arrangement for covering scheduled clinical duties following a night on-call if they have been in the hospital overnight. Overall, 91% (90.3-91.7%) reported work-related fatigue with over half reporting a moderate or significantly negative impact on health, well-being and home life. We discuss potential explanations for these results and ways to mitigate the effects of fatigue among consultants.