Modeling and closed-loop control of hypnosis by means of bispectral index (BIS) with isoflurane

Model-Informed Reinforcement Learning for Enabling Precision Dosing Via Adaptive Dosing

Precision dosing, the tailoring of drug doses to optimize therapeutic benefits and minimize risks in each patient, is essential for drugs with a narrow therapeutic window and severe adverse effects. Adaptive dosing strategies extend the precision dosing concept to time-varying treatments which require sequential dose adjustments based on evolving patient conditions. Reinforcement learning (RL) naturally fits this paradigm: it perfectly mimics the sequential decision-making process where clinicians adapt dose administration based on patient response and evolution monitoring. This paper aims to investigate the potentiality of coupling RL with population PK/PD models to develop precision dosing algorithms, reviewing the most relevant works in the field. Case studies in which PK/PD models were integrated within RL algorithms as simulation engine to predict consequences of any dosing action have been considered and discussed. They mainly concern propofol-induced anesthesia, anticoagulant therapy with warfarin and a variety of anticancer treatments differing for administered agents and/or monitored biomarkers. The resulted picture highlights a certain heterogeneity in terms of precision dosing approaches, applied methodologies, and degree of adherence to the clinical domain. In addition, a tutorial on how a precision dosing problem should be formulated in terms of the key elements composing the RL framework (i.e., system state, agent actions and reward function), and on how PK/PD models could enhance RL approaches is proposed for readers interested in delving in this field. Overall, the integration of PK/PD models into a RL-framework holds great promise for precision dosing, but further investigations and advancements are still needed to address current limitations and extend the applicability of this methodology to drugs requiring adaptive dosing strategies.

Feasibility of intelligent drug control in the maintenance phase of general anesthesia based on convolutional neural network

Background

The growth and aging process of the human population has accelerated the increase in surgical procedures. Yet, the demand for increasing operations can be hardly met since the training of anesthesiologists is usually a long-term process. Closed-loop artificial intelligence (AI) model provides the possibility to solve intelligent decision-making for anesthesia auxiliary control and, as such, has allowed breakthroughs in closed-loop control of clinical practices in intensive care units (ICUs). However, applying an open-loop artificial intelligence algorithm to build up personalized medication for anesthesia still needs to be further explored. Currently, anesthesiologists have selected doses of intravenously pumped anesthetic drugs mainly based on the blood pressure and bispectral index (BIS), which can express the depth of anesthesia. Unfortunately, BIS cannot be monitored at some medical centers or operational procedures and only be regulated by blood pressure. As a result, here we aim to inaugurally explore the feasibility of a basic intelligent control system applied to drug delivery in the maintenance phase of general anesthesia, based on a convolutional neural network model with open-loop design, according to AI learning of existing anesthesia protocols.

Methods

A convolutional neural network, combined with both sliding window sampling method and residual learning module, was utilized to establish an "AI anesthesiologist" model for intraoperative dosing of personalized anesthetic drugs (propofol and remifentanil). The fitting degree and difference in pumping dose decision, between the AI anesthesiologist and the clinical anesthesiologist, for these personalized anesthetic drugs were examined during the maintenance phase of anesthesia.

Results

The medication level established by the "AI anesthesiologist" was comparable to that obtained by the clinical anesthesiologist during the maintenance phase of anesthesia.

Conclusion

The application of an open-loop decision-making plan by convolutional neural network showed that intelligent anesthesia control is consistent with the actual anesthesia control, thus providing possibility for further evolution and optimization of auxiliary intelligent control of depth of anesthesia.

Continuous action deep reinforcement learning for propofol dosing during general anesthesia

PURPOSE

Anesthesiologists simultaneously manage several aspects of patient care during general anesthesia. Automating administration of hypnotic agents could enable more precise control of a patient's level of unconsciousness and enable anesthesiologists to focus on the most critical aspects of patient care. Reinforcement learning (RL) algorithms can be used to fit a mapping from patient state to a medication regimen. These algorithms can learn complex control policies that, when paired with modern techniques for promoting model interpretability, offer a promising approach for developing a clinically viable system for automated anesthestic drug delivery.

METHODS

We expand on our prior work applying deep RL to automated anesthetic dosing by now using a continuous-action model based on the actor-critic RL paradigm. The proposed RL agent is composed of a policy network that maps observed anesthetic states to a continuous probability density over propofol-infusion rates and a value network that estimates the favorability of observed states. We train and test three versions of the RL agent using varied reward functions. The agent is trained using simulated pharmacokinetic/pharmacodynamic models with randomized parameters to ensure robustness to patient variability. The model is tested on simulations and retrospectively on nine general anesthesia cases collected in the operating room. We utilize Shapley additive explanations to gain an understanding of the factors with the greatest influence over the agent's decision-making.

RESULTS

The deep RL agent significantly outperformed a proportional-integral-derivative controller (median episode median absolute performance error 1.9% ± 1.8 and 3.1% ± 1.1). The model that was rewarded for minimizing total doses performed the best across simulated patient demographics (median episode median performance error 1.1% ± 0.5). When run on real-world clinical datasets, the agent recommended doses that were consistent with those administered by the anesthesiologist.

CONCLUSIONS

The proposed approach marks the first fully continuous deep RL algorithm for automating anesthestic drug dosing. The reward function used by the RL training algorithm can be flexibly designed for desirable practices (e.g. use less anesthetic) and bolstered performances. Through careful analysis of the learned policies, techniques for interpreting dosing decisions, and testing on clinical data, we confirm that the agent's anesthetic dosing is consistent with our understanding of best-practices in anesthesia care.

The Drug Titration Paradox: Correlation of More Drug With Less Effect in Clinical Data

While analyzing clinical data where an anesthetic was titrated based on an objective measure of drug effect, we observed paradoxically that greater effect was associated with lesser dose. With this study we sought to find a mathematical explanation for this negative correlation between dose and effect, to confirm its existence with additional clinical data, and to explore it further with Monte Carlo simulations. Automatically recorded dosing and effect data from more than 9,000 patients was available for the analysis. The anesthetics propofol and sevoflurane and the catecholamine norepinephrine were titrated to defined effect targets, i.e., the processed electroencephalogram (Bispectral Index, BIS) and the blood pressure. A proportional control titration algorithm was developed for the simulations. We prove by deduction that the average dose–effect relationship during titration to the targeted effect will associate lower doses with greater effects. The finding of negative correlations between propofol and BIS, sevoflurane and BIS, and norepinephrine and mean arterial pressure confirmed the titration paradox. Monte Carlo simulations revealed two additional factors that contribute to the paradox. During stepwise titration toward a target effect, the slope of the dose–effect data for the population will be “reversed,” i.e., the correlation between dose and effect will not be positive, but will be negative, and will be “horizontal” when the titration is “perfect.” The titration paradox must be considered whenever data from clinical titration (flexible dose) studies are interpreted. Such data should not be used naively for the development of dosing guidelines.

Apprenticeship Learning for a Predictive State Representation of Anesthesia

OBJECTIVE

In this paper, we present an original decision support algorithm to assist the anesthesiologists delivery of drugs to maintain the optimal Depth of Anesthesia (DoA).

METHODS

Derived from a Transform Predictive State Representation algorithm (TPSR), our model learned by observing anesthesiologists in practice. This framework, known as apprenticeship learning, is particularly useful in the medical field as it is not based on an exploratory process - a prohibitive behavior in healthcare. The model only relied on the four commonly monitored variables: Heart Rate (HR), the Mean Blood Pressure (MBP), the Respiratory Rate (RR) and the concentration of anesthetic drug (AAFi).

RESULTS

Thirty-one patients have been included. The performances of the model is analyzed with metrics derived from the Hamming distance and cross entropy. They demonstrated that low rank dynamical system had the best performances on both predictions and simulations. Then, a confrontation of our agent to a panel of six real anesthesiologists demonstrated that 95.7% of the actions were valid.

CONCLUSION

These results strongly support the hypothesis that TPSR based models convincingly embed the behavior of anesthesiologists including only four variables that are commonly assessed to predict the DoA.

SIGNIFICANCE

The proposed novel approach could be of great help for clinicians by improving the fine tuning of the DoA. Furthermore, the possibility to predict the evolutions of the variables would help preventing side effects such as low blood pressure. A tool that could autonomously help the anesthesiologist would thus improve safety-level in the surgical room.

Effects of neuromuscular blockade reversal on bispectral index and frontal electromyogram during steady-state desflurane anesthesia: a randomized trial

The degree of neuromuscular blockade reversal may affect bispectral index (BIS) value. One possible reason is that the reverse of neuromuscular blockade affects electromyographic (EMG) signals of fascial muscle. Another reason is, the afferentation theory, the reverse of neuromuscular blockade relieves block signals generated in muscle stretch receptors from accessing the brain through afferent nerve pathways and induces arousal. Inaccurate BIS value may lead to overdose of drugs or the risk of intraoperative awareness. We compared changes in BIS and EMG values according to neuromuscular blockade reversal agents under steady-state desflurane anesthesia. A total of 65 patients were randomly allocated to receive either neostigmine 0.05 mg/kg, sugammadex 4 mg/kg, or pyridostigmine 0.25 mg/kg for neuromuscular blockade reversal under stable desflurane anesthesia, and 57 patients completed the study. The primary outcome was change in BIS and EMG values before and after administration of neuromuscular blockade reversal agents (between train-of-four [TOF] count 1-2 and TOF ratio 0.9). The change in BIS and EMG values before and after administration of neuromuscular blockade reversal agents were statistically different in each group (BIS: Neostigmine group, P < 0.001; Sugammadex group, P < 0.001; Pyridostigmine group, P = 0.001; EMG: Neostigmine group, P = 0.001; Sugammadex group, P < 0.001; Pyridostigmine group, P = 0.001; respectively). The BIS and EMG values had a positive correlation (P < 0.001). Our results demonstrate that the EMG and BIS values have increased after neuromuscular blockade reversal under desflurane anesthesia regardless of the type of neuromuscular blockade reversal agent. BIS should be applied carefully to measure of depth of anesthesia after neuromuscular blockade reversal.

Credibility Evidence for Computational Patient Models Used in the Development of Physiological Closed-Loop Controlled Devices for Critical Care Medicine

Physiological closed-loop controlled medical devices automatically adjust therapy delivered to a patient to adjust a measured physiological variable. In critical care scenarios, these types of devices could automate, for example, fluid resuscitation, drug delivery, mechanical ventilation, and/or anesthesia and sedation. Evidence from simulations using computational models of physiological systems can play a crucial role in the development of physiological closed-loop controlled devices; but the utility of this evidence will depend on the credibility of the computational model used. Computational models of physiological systems can be complex with numerous non-linearities, time-varying properties, and unknown parameters, which leads to challenges in model assessment. Given the wide range of potential uses of computational patient models in the design and evaluation of physiological closed-loop controlled systems, and the varying risks associated with the diverse uses, the specific model as well as the necessary evidence to make a model credible for a use case may vary. In this review, we examine the various uses of computational patient models in the design and evaluation of critical care physiological closed-loop controlled systems (e.g., hemodynamic stability, mechanical ventilation, anesthetic delivery) as well as the types of evidence (e.g., verification, validation, and uncertainty quantification activities) presented to support the model for that use. We then examine and discuss how a credibility assessment framework (American Society of Mechanical Engineers Verification and Validation Subcommittee, V&V 40 Verification and Validation in Computational Modeling of Medical Devices) for medical devices can be applied to computational patient models used to test physiological closed-loop controlled systems.

Optimizing Robust PID Control of Propofol Anesthesia for Children: Design and Clinical Evaluation

OBJECTIVE

The goal of this paper was to optimize robust PID control for propofol anesthesia in children aged 5-10 years to improve performance, particularly to decrease the time of induction of anesthesia while maintaining robustness.

METHODS

We analyzed results of a previous study conducted by our group to identify opportunities for system improvement. Allometric scaling was introduced to reduce the interpatient variability and a new robust PID controller was designed using an optimization-based method. We evaluated this optimized design in a clinical study involving 16 new cases.

RESULTS

The optimized controller design achieved the performance predicted in simulation studies in the design stage. Time of induction of anesthesia was median [Q1, Q3] 3.7 [2.3, 4.1] min and the achieved global score was 13.4 [9.9, 16.8].

CONCLUSION

Allometric scaling reduces the interpatient variability in this age group and allows for improved closed-loop performance. The uncertainty described by the model set, the predicted closed-loop responses, and the predicted robustness margins are realistic. The system meets the design objectives of improved speed of induction of anesthesia while maintaining robustness and improving clinically relevant system behavior.

SIGNIFICANCE

Control system optimization and ongoing system improvements are essential to the development of a clinically relevant commercial device. This paper demonstrates the validity of our approach, including system modeling, controller optimization, and pre-clinical testing in simulation.

Closed-loop Administration of General Anaesthesia: From Sensor to Medical Device

Closed-loop administration devices for general anaesthesia have become a common subject of clinical research over the last decade and appear more and more acceptable in clinical practice. They encompass various therapeutic needs of the anesthetized patient, e. g. fluid administration, hypnotic and analgesic drug administration, myorelaxation. Multiple clinical trials involving closed-loop devices have underscored their safety, but data concerning their clinical benefit to the patient are still lacking. As the marketing of various devices increases, clinicians need to understand how comparisons between these devices can be made: the measure of performance error and wobble are technical but have also a clinical meaning, to which clinical outcomes can be added, such as drug consumption and maintenance of hemodynamic parameters (e. g. heart rate and blood pressure) within predefined ranges. Clinicians using closed-loop devices need especially to understand how various physiological signals lead to specific drug adaptations, which means that they switch from decision making to supervision of general anaesthesia.

A Review of Modern Control Strategies for Clinical Evaluation of Propofol Anesthesia Administration Employing Hypnosis Level Regulation

Regulating the depth of hypnosis during surgery is one of the major objectives of an anesthesia infusion system. Continuous administration of Propofol infusion during surgical procedures is essential but it unduly increases the load of an anesthetist working in a multitasking scenario in the operation theatre. Manual and target controlled infusion systems are not appropriate to handle instabilities like blood pressure and heart rate changes arising due to interpatient and intrapatient variability. Patient safety, large interindividual variability, and less postoperative effects are the main factors motivating automation in anesthesia administration. The idea of automated system for Propofol infusion excites control engineers to come up with more sophisticated systems that can handle optimum delivery of anesthetic drugs during surgery and avoid postoperative effects. A linear control technique is applied initially using three compartmental pharmacokinetic and pharmacodynamic models. Later on, sliding mode control and model predicative control achieve considerable results with nonlinear sigmoid model. Chattering and uncertainties are further improved by employing adaptive fuzzy control and H∞ control. The proposed sliding mode control scheme can easily handle the nonlinearities and achieve an optimum hypnosis level as compared to linear control schemes, hence preventing mishaps such as underdosing and overdosing of anesthesia.

A Comparison of Multiscale Permutation Entropy Measures in On-Line Depth of Anesthesia Monitoring

OBJECTIVE

Multiscale permutation entropy (MSPE) is becoming an interesting tool to explore neurophysiological mechanisms in recent years. In this study, six MSPE measures were proposed for on-line depth of anesthesia (DoA) monitoring to quantify the anesthetic effect on the real-time EEG recordings. The performance of these measures in describing the transient characters of simulated neural populations and clinical anesthesia EEG were evaluated and compared.

METHODS

Six MSPE algorithms-derived from Shannon permutation entropy (SPE), Renyi permutation entropy (RPE) and Tsallis permutation entropy (TPE) combined with the decomposition procedures of coarse-graining (CG) method and moving average (MA) analysis-were studied. A thalamo-cortical neural mass model (TCNMM) was used to generate noise-free EEG under anesthesia to quantitatively assess the robustness of each MSPE measure against noise. Then, the clinical anesthesia EEG recordings from 20 patients were analyzed with these measures. To validate their effectiveness, the ability of six measures were compared in terms of tracking the dynamical changes in EEG data and the performance in state discrimination. The Pearson correlation coefficient (R) was used to assess the relationship among MSPE measures.

RESULTS

CG-based MSPEs failed in on-line DoA monitoring at multiscale analysis. In on-line EEG analysis, the MA-based MSPE measures at 5 decomposed scales could track the transient changes of EEG recordings and statistically distinguish the awake state, unconsciousness and recovery of consciousness (RoC) state significantly. Compared to single-scale SPE and RPE, MSPEs had better anti-noise ability and MA-RPE at scale 5 performed best in this aspect. MA-TPE outperformed other measures with faster tracking speed of the loss of unconsciousness.

CONCLUSIONS

MA-based multiscale permutation entropies have the potential for on-line anesthesia EEG analysis with its simple computation and sensitivity to drug effect changes. CG-based multiscale permutation entropies may fail to describe the characteristics of EEG at high decomposition scales.

Robust control of burst suppression for medical coma

OBJECTIVE

Medical coma is an anesthetic-induced state of brain inactivation, manifest in the electroencephalogram by burst suppression. Feedback control can be used to regulate burst suppression, however, previous designs have not been robust. Robust control design is critical under real-world operating conditions, subject to substantial pharmacokinetic and pharmacodynamic parameter uncertainty and unpredictable external disturbances. We sought to develop a robust closed-loop anesthesia delivery (CLAD) system to control medical coma.

APPROACH

We developed a robust CLAD system to control the burst suppression probability (BSP). We developed a novel BSP tracking algorithm based on realistic models of propofol pharmacokinetics and pharmacodynamics. We also developed a practical method for estimating patient-specific pharmacodynamics parameters. Finally, we synthesized a robust proportional integral controller. Using a factorial design spanning patient age, mass, height, and gender, we tested whether the system performed within clinically acceptable limits. Throughout all experiments we subjected the system to disturbances, simulating treatment of refractory status epilepticus in a real-world intensive care unit environment.

MAIN RESULTS

In 5400 simulations, CLAD behavior remained within specifications. Transient behavior after a step in target BSP from 0.2 to 0.8 exhibited a rise time (the median (min, max)) of 1.4 [1.1, 1.9] min; settling time, 7.8 [4.2, 9.0] min; and percent overshoot of 9.6 [2.3, 10.8]%. Under steady state conditions the CLAD system exhibited a median error of 0.1 [-0.5, 0.9]%; inaccuracy of 1.8 [0.9, 3.4]%; oscillation index of 1.8 [0.9, 3.4]%; and maximum instantaneous propofol dose of 4.3 [2.1, 10.5] mg kg(-1). The maximum hourly propofol dose was 4.3 [2.1, 10.3] mg kg(-1) h(-1). Performance fell within clinically acceptable limits for all measures.

SIGNIFICANCE

A CLAD system designed using robust control theory achieves clinically acceptable performance in the presence of realistic unmodeled disturbances and in spite of realistic model uncertainty, while maintaining infusion rates within acceptable safety limits.

Hipnose, imobilidade tônica e eletroencefalograma

OBJETIVO: Apresentar uma revisão sobre as características da atividade elétrica cerebral que acompanha a hipnose animal, estado induzido em laboratório em mamíferos por manipulações experimentais, bem como sobre as alterações encontradas no EEG durante o estado de hipnose, visando à discussão dos resultados encontrados na busca de evidências dos fundamentos filogenéticos que possam conduzir ao entendimento dos rudimentos neurais da hipnose humana. MÉTODO: Livros e bases eletrônicas de dados foram consultados. Critério de inclusão: artigos originais publicados entre 1966-2012. Critério de exclusão: artigos que se afastavam da visão eletroneurofisiológica da hipnose. RESULTADOS: Foram encontradas 662 referências, tendo sido selecionados os artigos e livros referenciados. Além desses artigos, foi incluído no estudo o artigo de Hoagland, publicado em 1928, que é um clássico na área de imobilidade tônica em vertebrados. CONCLUSÕES: O estado de hipnose humano resulta de processamentos em inúmeros circuitos paralelos distribuídos em uma complexa rede neuronal, envolvendo, dessa forma, uma ampla área do encéfalo. Na trajetória evolutiva, a grande ampliação dos recursos corticais pode ter tornado as respostas de imobilidade tônica passíveis de modulação consciente, respostas essas ainda presentes nos humanos e que se manifestam involuntariamente em situações de grande ameaça. Vários estudos têm evidenciado mecanismos neurofisiológicos capazes de reforçar a visão da hipnose não só como um eficiente recurso para procedimentos médicos e odontológicos, funcionando como auxiliar na analgesia e sedação, mas também como excelente ferramenta psicoterapêutica.

An enriched simulation environment for evaluation of closed-loop anesthesia

To simulate and evaluate the administration of anesthetic agents in the clinical setting, many pharmacology models have been proposed and validated, which play important roles for in silico testing of closed-loop control methods. However, to the authors' best knowledge, there is no anesthesia simulator incorporating closed-loop feedback control of anesthetic agent administration freely available and accessible to the public. Consequently, many necessary but time consuming procedures, such as selecting models from the available literatures and establishing new simulator algorithms, will be repeated by different researchers who intend to explore a novel control algorithm for closed-loop anesthesia. To address this issue, an enriched anesthesia simulator was devised in our laboratory and made freely available to the anesthesia community. This simulator was built by using MATLAB(®) (The MathWorks, Natick, MA). The GUI technology embedded in MATLAB was chosen as the tool to develop a human-machine interface. This simulator includes four types of anesthetic models, and all have been wildly used in closed-loop anesthesia studies. For each type of model, 24 virtual patients were created with significant diversity. In addition, the platform also provides a model identification module and a control method library. For the model identification module, the least square method and particle swarm optimization were presented. In the control method library, a proportional-integral-derivative control and a model predictive control were provided. Both the model identification module and the control method library are extensive and readily accessible for users to add user-defined functions. This simulator could be a benchmark-testing platform for closed-loop control of anesthesia, which is of great value and has significant development potential. For convenience, this simulator is termed as Wang's Simulator, which can be downloaded from http://www.AutomMed.org .

Clinical evaluation of a simultaneous closed-loop anaesthesia control system for depth of anaesthesia and neuromuscular blockade*

We developed a closed-loop system to control the depth of anaesthesia and neuromuscular blockade using the bispectral index and the electromyogram simultaneously and evaluated the clinical performance of this combined system for general anaesthesia. Twenty-two adult patients were included in this study. Anaesthesia was induced by a continuous infusion of remifentanil at 0.4 μg.kg(-1) .min(-1) (induction dose) and then 0.25 μg.kg(-1) .min(-1) (maintenance dose) and propofol at 2 mg.kg(-1) 3 min later. The combined automatic control was started 2 min after tracheal intubation. The depth of anaesthesia was recorded using bispectral index monitoring using a target value of 40. The target value of neuromuscular blockade, using mivacurium, was a T1/T1(0) twitch height of 10%. The precision of the system was calculated using internationally defined performance parameters. Twenty patients were included in the data analysis. The mean (SD) duration of simultaneous control was 129 (69) min. No human intervention was necessary during the computer-controlled administration of propofol and mivacurium. All patients assessed the quality of anaesthesia as 'good' to 'very good'; there were no episodes of awareness. The mean (SD) median performance error, median absolute performance error and wobble for the control of depth of anaesthesia and for neuromuscular blockade were -0.31 (1.78), 6.76 (3.45), 6.32 (2.93) and -0.38 (1.68), 3.75 (4.83), 3.63 (4.69), respectively. The simultaneous closed-loop system using propofol and mivacurium was able to maintain the target values with a high level of precision in a clinical setting.

Robust closed-loop control of propofol administration using WAVCNS index as the controlled variable

This paper presents a robust closed-loop strategy for control of depth of hypnosis. The proposed method regulates the electroencephalogram (EEG)-derived WAVCNS index as a hypnosis measure by manipulating intravenous propofol administration. In contrast to many existing closed-loop methods, the control design presented in this paper produces stability and robustness against uncertainty by explicitly accounting for the pharmacokinetic (PK) and pharmacodynamic (PD) variability between different individuals, as well as unpredictable surgical stimuli that the closed-loop control is required to tolerate. This closed-loop control was evaluated using simulated surgical procedures in 44 patient models whose PK and PD were identified from real clinical data. The controller can deliver consistent and acceptable closed-loop induction and maintenance phase responses for patients with wide-ranging PK and PD differences.

Reinforcement learning versus proportional-integral-derivative control of hypnosis in a simulated intraoperative patient

BACKGROUND

Research has demonstrated the efficacy of closed-loop control of anesthesia using bispectral index (BIS) as the controlled variable. Model-based and proportional-integral-derivative (PID) controllers outperform manual control. We investigated the application of reinforcement learning (RL), an intelligent systems control method, to closed-loop BIS-guided, propofol-induced hypnosis in simulated intraoperative patients. We also compared the performance of the RL agent against that of a conventional PID controller.

METHODS

The RL and PID controllers were evaluated during propofol induction and maintenance of hypnosis. The patient-hypnotic episodes were designed to challenge both controllers with varying degrees of interindividual variation and noxious surgical stimulation. Each controller was tested in 1000 simulated patients, and control performance was assessed by calculating the median performance error (MDPE), median absolute performance error (MDAPE), Wobble, and Divergence for each controller group. A separate analysis was performed for the induction and maintenance phases of hypnosis.

RESULTS

During maintenance, RL control demonstrated an MDPE of -1% and an MDAPE of 3.75%, with 80% of the time at BIS(target) ± 5. The PID controller yielded a MDPE of -8.5% and an MDAPE of 8.6%, with 57% of the time at BIS(target) ± 5. In comparison, the MDAPE in the worst-controlled patient of the RL group was observed to be almost half that of the worst-controlled patient in the PID group.

CONCLUSIONS

When compared with the PID controller, RL control resulted in slower induction but less overshoot and faster attainment of steady state. No difference in interindividual patient variation and noxious destabilizing challenge on control performance was observed between the 2 patient groups.

Decision-oriented multi-outcome modeling for anesthesia patients

Anesthesia drugs have impact on multiple outcomes of an anesthesia patient. Most typical outcomes include anesthesia depth, blood pressures, heart rates, etc. Traditional diagnosis and control in anesthesia focus on a one-drug-one-outcome scenario. This paper studies the problem of real-time modeling for monitoring, diagnosing, and predicting multiple outcomes of anesthesia patients. It is shown that consideration of multiple outcomes is necessary and beneficial for anesthesia managements. Due to limited real-time data, real-time modeling in multi-outcome modeling requires low-complexity model strucrtures. This paper introduces a method of decision-oriented modeling that significantly reduces the complexity of the problem. The method employs simplified and combined model functions in a Wiener structure to contain model complexity. The ideas of drug impact prediction and reachable sets are introduced for utility of the models in diagnosis, outcome prediction, and decision assistance. Clinical data are used to evaluate the effectiveness of the method.

Closed-loop control for intensive care unit sedation

The potential clinical applications of active control for pharmacology in general, and anesthesia and critical care unit medicine in particular, are clearly apparent. Specifically, monitoring and controlling the depth of anesthesia in surgery and the intensive care unit is of particular importance. Nonnegative and compartmental models provide a broad framework for biological and physiological systems, including clinical pharmacology, and are well suited for developing models for closed-loop control for drug administration. These models are derived from mass and energy balance considerations that involve dynamic states whose values are nonnegative and are characterized by conservation laws (e.g., mass, energy, fluid, etc.) capturing the exchange of material between kinetically homogenous entities called compartments. Compartmental models have been particularly important for understanding pharmacokinetics and pharmacodynamics. One of the basic motivations for pharmacokinetic/pharmacodynamic research is to improve drug delivery. In critical care medicine it is current clinical practice to administer potent drugs that profoundly influence levels of consciousness, respiratory, and cardiovascular function by manual control based on the clinician's experience and intuition. Open-loop control (manual control) by clinical personnel can be tedious, imprecise, time-consuming, and sometimes of poor quality, depending on the skills and judgement of the clinician. Closed-loop control based on appropriate dynamical systems models merits investigation as a means of improving drug delivery in the intensive care unit. In this article, we discuss the challenges and opportunities of feedback control using nonnegative and compartmental system theory for the specific problem of closed-loop control of intensive care unit sedation. Several closed-loop control paradigms are investigated including adaptive control, neural network adaptive control, optimal control, and hybrid adaptive control algorithms for intensive care unit sedation.

Quantifying uncertainty bounds in anesthetic PKPD models

A major challenge faced when designing controllers to automate anesthetic drug delivery is the large variability that exists between and within patients. This intra- and inter-patient variability have been reported to lead to instability. Hence, defining and quantifying uncertainty bounds provides a mean to validate the control design, ensure its stability and assess performance. In this work, the intra- and inter-patient variability measured from thiopental induction data is used to define uncertainty bounds. It is shown that these bounds can be reduced by up to 40% when using a patient-specific model as compared to a population-normed model. It is also shown that identifying only the overall static gain of the patient system already decreases significantly this uncertainty.

Quantifying cortical activity during general anesthesia using wavelet analysis

This paper reports on a novel method for quantifying the cortical activity of a patient during general anesthesia as a surrogate measure of the patient's level of consciousness. The proposed technique is based on the analysis of a single-channel (frontal) electroencephalogram (EEG) signal using stationary wavelet transform (SWT). The wavelet coefficients calculated from the EEG are pooled into a statistical representation, which is then compared to two well-defined states: the awake state with normal EEG activity, and the isoelectric state with maximal cortical depression. The resulting index, referred to as the wavelet-based anesthetic value for central nervous system monitoring (WAV(CNS)), quantifies the depth of consciousness between these two extremes. To validate the proposed technique, we present a clinical study which explores the advantages of the WAV(CNS) in comparison with the BIS monitor (Aspect Medical Systems, MA), currently a reference in consciousness monitoring. Results show that the WAV(CNS) and BIS are well correlated (r = 0.969) during periods of steady-state despite fundamental algorithmic differences. However, in terms of dynamic behavior, the WAV(CNS) offers faster tracking of transitory changes at induction and emergence, with an average lead of 15-30 s. Furthermore, and conversely to the BIS, the WAV(CNS) regains its preinduction baseline value when patients are responding to verbal command after emergence from anesthesia. We conclude that the proposed analysis technique is an attractive alternative to BIS monitoring. In addition, we show that the WAV(CNS) dynamics can be modeled as a linear time invariant transfer function. This index is, therefore, well suited for use as a feedback sensor in advisory systems, closed-loop control schemes, and for the identification of the pharmacodynamic models of anesthetic drugs.

Closed loops in anaesthesia

Closed-loop systems are able to make their own decisions and to try to reach and maintain a preset target. As a result, they might help the anaesthetist to optimise the titration of drug administration without any overshoot, controlling physiological functions and guiding monitoring variables. Thanks to the development of fast computer technology and more reliable pharmacological effect measures, the study of automation in anaesthesia has regained popularity. This short review focuses on the most recently developed and tested feedback systems in anaesthesia. Various new approaches for controlling the administration of intravenous and inhaled hypnotic-anaesthetic drugs have recently been published. For analgesics, a framework for further research has been presented in the literature. For other drugs, such as muscle relaxants and haemodynamic agents, only short reviews can be found. Until now, most of these systems have had to be under development. The challenge is now fully to establish the safety, efficacy, reliability and utility of closed-loop anaesthesia so that it can be adopted in the clinical setting. Besides, their role in optimising the controlled variables and control models, these systems have to be tested in extreme circumstances in order to test their robustness.

A cascade feedback control approach for hypnosis

This article studies the problem of controlling the drug administration during an anesthesia process, where muscle relaxation, analgesia, and hypnosis are regulated by means of monitored administration of specific drugs. On the basis of a seventh-order nonlinear pharmacokinetic-pharmacodynamic representation of the hypnosis process dynamics, a cascade (master/slave) feedback control structure for controlling the bispectral index (BIS) is proposed. The master controller compares the measured BIS with its reference value to provide the expired isoflurane concentration reference to the slave controller. In turn, the slave controller manipulates the anesthetic isoflurane concentration entering the anesthetic system to achieve the reference from the master controller. The advantage of the proposed cascade control structure with respect to its noncascade counterpart is that the former provides operation protection against BIS measurement failures. In fact, under a BIS measurement fault, the master control feedback is broken and the slave controller operates under a safe reference value. Extensive numerical simulations are used to illustrate the functioning of the proposed cascade control structure.

On the use of multivariable piecewise-linear models for predicting human response to anesthesia

The standard modeling paradigm used to describe the relationship between input anesthetic agents and output patient endpoint variables are single-input single-output pharmacokinetic-pharmacodynamic (PK-PD) compartment models. In this paper, we propose the use of multivariable piecewise-linear models to describe the relations between inputs that include anesthesia, surgical stimuli and disturbances to a variety of patient output variables. Subspace identification methods are applied to clinical data to construct the models. A comparison of predicted and measured responses is completed, which includes predictions from PK-PD models, and piecewise-linear time-invariant models.

Closed-loop control of anaesthesia

PURPOSE OF REVIEW

Closed-loop systems are able to make decisions on their own and try to reach and maintain a preset target. As a result, they might help the anaesthesiologist in optimizing the titration of drug administration without overshooting, controlling physiological functions and guiding monitoring variables. Thanks to the development of fast computer technology and more reliable pharmacological effect measures, the study of automation in anaesthesia has regained popularity.

RECENT FINDINGS

This short review focuses on the most recently developed and tested feed-back systems in anaesthesia. Various new approaches for controlling the administration of intravenous and inhaled hypnotic-anaesthetic drugs have been published recently. For analgesics, a framework for further research has been presented in the literature. For other drugs, such as muscle relaxants and haemodynamics, a short review can be found.

SUMMARY

Until now, most of these systems are still under development. The challenge is now to establish fully the safety, efficacy, reliability and utility of closed-loop anaesthesia for its adoption into the clinical setting. Besides the optimization of controlled variables and control models, these systems have to be tested in extreme circumstances.

Unconsciousness indication using time-domain parameters extracted from mid-latency auditory evoked potentials

OBJECTIVE

The mid-latency auditory evoked potential (MLAEP) has been used to indicate depth of anaesthesia, and is usually analysed in time-domain. This work compares three techniques: Wave Deformation Parameters (PDO), Auditory Evoked Potential Index (AEPidx) and an automatic Nb-wave latency estimator (Nb), in the assessment of unconsciousness onset based on EEG under auditory stimulation.

METHODS

Ten normal adult volunteers, under no pre-anaesthetic drug administration, received propofol during two successive periods of 45 min each one (3 mg/Kg/h and 9 mg/Kg/h), being the EEG collected from 10 min previous to infusion beginning until the subjects woke up. From the time-evolution of MLAEP (averaging of successive sets of 1000 epochs) all the parameters were compared to thresholds (unconsciousness onset indication time) and the results were compared to the instant of pressing interruption of a soft-touch switch, when one assumed the volunteer became unconscious.

RESULTS

The Wilcoxon Signed-Rank test points equivalence between each of the parameters and the switch for, say, alpha = 5%. Bland-Altman diagrams revealed that the Attenuation-PDO has better agreement to the switch than Nb and AEPidx.

CONCLUSION

The results suggest that, at least to indicate unconsciousness, the most reliable effect of the anaesthetic drug on MLAEP would be the amplitude attenuation. Despite the high dependence on noise due to its time-domain basis, the Attenuation-PDO seems to be adequate to assess depth of anaesthesia.

A new paradigm for the closed-loop intraoperative administration of analgesics in humans

We present a new paradigm for the closed-loop administration of analgesics during general anesthesia. The manipulated variable in the control system is the infusion rate of the opiate alfentanil, administered intravenously through a computer-controlled infusion pump (CCIP). The outputs to be controlled are the patient's mean arterial pressure (MAP) and the drug concentration in the plasma. Maintaining MAP within appropriate ranges provides optimal treatment of the patient's reactions to surgical stimuli. Maintaining plasma drug concentrations close to a reference value specified by the anesthesiologist allows to titrate analgesic administration to qualitative clinical end-points of insufficient analgesia. MAP is acquired invasively through a catheter cannula. Since plasma drug concentrations cannot be measured on-line, they are estimated via a pharmacokinetic model. We describe an explicit model-predictive controller which achieves the above-mentioned objectives. An upper constraint on drug concentrations is maintained to avoid overdosing. Constraints on the MAP are introduced to trigger a prompt controller reaction during hypertensive and hypotensive periods. Measurement artifacts in the MAP signal are rejected to prevent harmful misbehavior of the controller. We discuss the results of the clinical validation of the controller on humans.