Robust control of burst suppression for medical coma

Robust adaptive deep brain stimulation control of in-silico non-stationary Parkinsonian neural oscillatory dynamics

Objective. Closed-loop deep brain stimulation (DBS) is a promising therapy for Parkinson's disease (PD) that works by adjusting DBS patterns in real time from the guidance of feedback neural activity. Current closed-loop DBS mainly uses threshold-crossing on-off controllers or linear time-invariant (LTI) controllers to regulate the basal ganglia (BG) Parkinsonian beta band oscillation power. However, the critical cortex-BG-thalamus network dynamics underlying PD are nonlinear, non-stationary, and noisy, hindering accurate and robust control of Parkinsonian neural oscillatory dynamics.Approach. Here, we develop a new robust adaptive closed-loop DBS method for regulating the Parkinsonian beta oscillatory dynamics of the cortex-BG-thalamus network. We first build an adaptive state-space model to quantify the dynamic, nonlinear, and non-stationary neural activity. We then construct an adaptive estimator to track the nonlinearity and non-stationarity in real time. We next design a robust controller to automatically determine the DBS frequency based on the estimated Parkinsonian neural state while reducing the system's sensitivity to high-frequency noise. We adopt and tune a biophysical cortex-BG-thalamus network model as an in-silico simulation testbed to generate nonlinear and non-stationary Parkinsonian neural dynamics for evaluating DBS methods.Main results. We find that under different nonlinear and non-stationary neural dynamics, our robust adaptive DBS method achieved accurate regulation of the BG Parkinsonian beta band oscillation power with small control error, bias, and deviation. Moreover, the accurate regulation generalizes across different therapeutic targets and consistently outperforms current on-off and LTI DBS methods.Significance. These results have implications for future designs of closed-loop DBS systems to treat PD.

Delayed closed-loop neurostimulation for the treatment of pathological brain rhythms in mental disorders: a computational study

Mental disorders are among the top most demanding challenges in world-wide health. A large number of mental disorders exhibit pathological rhythms, which serve as the disorders characteristic biomarkers. These rhythms are the targets for neurostimulation techniques. Open-loop neurostimulation employs stimulation protocols, which are rather independent of the patients health and brain state in the moment of treatment. Most alternative closed-loop stimulation protocols consider real-time brain activity observations but appear as adaptive open-loop protocols, where e.g., pre-defined stimulation sets in if observations fulfil pre-defined criteria. The present theoretical work proposes a fully-adaptive closed-loop neurostimulation setup, that tunes the brain activities power spectral density (PSD) according to a user-defined PSD. The utilized brain model is non-parametric and estimated from the observations via magnitude fitting in a pre-stimulus setup phase. Moreover, the algorithm takes into account possible conduction delays in the feedback connection between observation and stimulation electrode. All involved features are illustrated on pathological α- and γ-rhythms known from psychosis. To this end, we simulate numerically a linear neural population brain model and a non-linear cortico-thalamic feedback loop model recently derived to explain brain activity in psychosis.

Predictive neuromodulation of cingulo-frontal neural dynamics in major depressive disorder using a brain-computer interface system: A simulation study

Introduction

Deep brain stimulation (DBS) is a promising therapy for treatment-resistant major depressive disorder (MDD). MDD involves the dysfunction of a brain network that can exhibit complex nonlinear neural dynamics in multiple frequency bands. However, current open-loop and responsive DBS methods cannot track the complex multiband neural dynamics in MDD, leading to imprecise regulation of symptoms, variable treatment effects among patients, and high battery power consumption.

Methods

Here, we develop a closed-loop brain-computer interface (BCI) system of predictive neuromodulation for treating MDD. We first use a biophysically plausible ventral anterior cingulate cortex (vACC)-dorsolateral prefrontal cortex (dlPFC) neural mass model of MDD to simulate nonlinear and multiband neural dynamics in response to DBS. We then use offline system identification to build a dynamic model that predicts the DBS effect on neural activity. We next use the offline identified model to design an online BCI system of predictive neuromodulation. The online BCI system consists of a dynamic brain state estimator and a model predictive controller. The brain state estimator estimates the MDD brain state from the history of neural activity and previously delivered DBS patterns. The predictive controller takes the estimated MDD brain state as the feedback signal and optimally adjusts DBS to regulate the MDD neural dynamics to therapeutic targets. We use the vACC-dlPFC neural mass model as a simulation testbed to test the BCI system and compare it with state-of-the-art open-loop and responsive DBS treatments of MDD.

Results

We demonstrate that our dynamic model accurately predicts nonlinear and multiband neural activity. Consequently, the predictive neuromodulation system accurately regulates the neural dynamics in MDD, resulting in significantly smaller control errors and lower DBS battery power consumption than open-loop and responsive DBS.

Discussion

Our results have implications for developing future precisely-tailored clinical closed-loop DBS treatments for MDD.

Methodological aspects of studying the mechanisms of consciousness

There are at least two approaches to the definition of consciousness. In the first case, certain aspects of consciousness, called qualia, are considered inaccessible for research from a third person and can only be described through subjective experience. This approach is inextricably linked with the so-called "hard problem of consciousness", that is, the question of why consciousness has qualia or how any physical changes in the environment can generate subjective experience. With this approach, some aspects of consciousness, by definition, cannot be explained on the basis of external observations and, therefore, are outside the scope of scientific research. In the second case, a priori constraints do not constrain the field of scientific investigation, and the best explanation of the experience in the first person is included as a possible subject of empirical research. Historically, in the study of cause-and-effect relationships in biology, it was customary to distinguish between proximate causation and ultimate causation existing in biological systems. Immediate causes are based on the immediate influencing factors [1]. Proximate causation has evolutionary explanations. When studying biological systems themselves, such an approach is undoubtedly justified, but it often seems insufficient when studying the interaction of consciousness and the brain [2,3]. Current scientific communities proceed from the assumption that the physical substrate for the generation of consciousness is a neural network that unites various types of neurons located in various brain structures. Many neuroscientists attach a key role in this process to the cortical and thalamocortical neural networks. This question is directly related to experimental and clinical research in the field of disorder of consciousness. Progress in this area of medicine depends on advances in neuroscience in this area and is also a powerful source of empirical information. In this area of consciousness research, a large amount of experimental data has been accumulated, and in this review an attempt was made to generalize and systematize.

Continuous action deep reinforcement learning for propofol dosing during general anesthesia

PURPOSE

Anesthesiologists simultaneously manage several aspects of patient care during general anesthesia. Automating administration of hypnotic agents could enable more precise control of a patient's level of unconsciousness and enable anesthesiologists to focus on the most critical aspects of patient care. Reinforcement learning (RL) algorithms can be used to fit a mapping from patient state to a medication regimen. These algorithms can learn complex control policies that, when paired with modern techniques for promoting model interpretability, offer a promising approach for developing a clinically viable system for automated anesthestic drug delivery.

METHODS

We expand on our prior work applying deep RL to automated anesthetic dosing by now using a continuous-action model based on the actor-critic RL paradigm. The proposed RL agent is composed of a policy network that maps observed anesthetic states to a continuous probability density over propofol-infusion rates and a value network that estimates the favorability of observed states. We train and test three versions of the RL agent using varied reward functions. The agent is trained using simulated pharmacokinetic/pharmacodynamic models with randomized parameters to ensure robustness to patient variability. The model is tested on simulations and retrospectively on nine general anesthesia cases collected in the operating room. We utilize Shapley additive explanations to gain an understanding of the factors with the greatest influence over the agent's decision-making.

RESULTS

The deep RL agent significantly outperformed a proportional-integral-derivative controller (median episode median absolute performance error 1.9% ± 1.8 and 3.1% ± 1.1). The model that was rewarded for minimizing total doses performed the best across simulated patient demographics (median episode median performance error 1.1% ± 0.5). When run on real-world clinical datasets, the agent recommended doses that were consistent with those administered by the anesthesiologist.

CONCLUSIONS

The proposed approach marks the first fully continuous deep RL algorithm for automating anesthestic drug dosing. The reward function used by the RL training algorithm can be flexibly designed for desirable practices (e.g. use less anesthetic) and bolstered performances. Through careful analysis of the learned policies, techniques for interpreting dosing decisions, and testing on clinical data, we confirm that the agent's anesthetic dosing is consistent with our understanding of best-practices in anesthesia care.

A simulation-based comparative analysis of PID and LQG control for closed-loop anesthesia delivery

Closed loop anesthesia delivery (CLAD) systems can help anesthesiologists efficiently achieve and maintain desired anesthetic depth over an extended period of time. A typical CLAD system would use an anesthetic marker, calculated from physiological signals, as real-time feedback to adjust anesthetic dosage towards achieving a desired set-point of the marker. Since control strategies for CLAD vary across the systems reported in recent literature, a comparative analysis of common control strategies can be useful. For a nonlinear plant model based on well-established models of compartmental pharmacokinetics and sigmoid-Emax pharmacodynamics, we numerically analyze the set-point tracking performance of three output-feedback linear control strategies: proportional-integral-derivative (PID) control, linear quadratic Gaussian (LQG) control, and an LQG with integral action (ILQG). Specifically, we numerically simulate multiple CLAD sessions for the scenario where the plant model parameters are unavailable for a patient and the controller is designed based on a nominal model and controller gains are held constant throughout a session. Based on the numerical analyses performed here, conditioned on our choice of model and controllers, we infer that in terms of accuracy and bias PID control performs better than ILQG which in turn performs better than LQG. In the case of noisy observations, ILQG can be tuned to provide a smoother infusion rate while achieving comparable steady state response with respect to PID. The numerical analysis framework and findings reported here can help CLAD developers in their choice of control strategies. This paper may also serve as a tutorial paper for teaching control theory for CLAD.

Multi-model generalised predictive control for intravenous anaesthesia under inter-individual variability

Inter-individual variability possesses a major challenge in the regulation of hypnosis in anesthesia. Understanding the variability towards anesthesia effect is expected to assist the design of controller for anesthesia regulation. However, such studies are still very scarce in the literature. This study aims to analyze the inter-individual variability in propofol pharmacokinetics/pharmacodynamics (PK/PD) model and proposed a suitable controller to tackle the variability. This study employed Sobol' sensitivity analysis to identify significance parameters in propofol PK/PD model that affects the model output Bispectral Index (BIS). Parameters' range is obtained from reported clinical data. Based on the finding, a multi-model generalized predictive controller was proposed to regulate propofol in tackling patient variability. [Formula: see text] (concentration that produces 50% of the maximum effect) was found to have a highly-determining role on the uncertainty of BIS. In addition, the Hill coefficient, [Formula: see text], was found to be significant when there is a drastic input, especially during the induction phase. Both of these parameters only affect the process gain upon model linearization. Therefore, a predictive controller based on switching of model with different process gain is proposed. Simulation result shows that it is able to give a satisfactory performance across a wide population. Both the parameters [Formula: see text] and [Formula: see text], which are unknown before anesthesia procedure, were found to be highly significant in contributing the uncertainty of BIS. Their range of variability must be considered during the design and evaluation of controller. A linear controller may be sufficient to tackle most of the variability since both [Formula: see text] and [Formula: see text] would be translated into process gain upon linearization.

Credibility Evidence for Computational Patient Models Used in the Development of Physiological Closed-Loop Controlled Devices for Critical Care Medicine

Physiological closed-loop controlled medical devices automatically adjust therapy delivered to a patient to adjust a measured physiological variable. In critical care scenarios, these types of devices could automate, for example, fluid resuscitation, drug delivery, mechanical ventilation, and/or anesthesia and sedation. Evidence from simulations using computational models of physiological systems can play a crucial role in the development of physiological closed-loop controlled devices; but the utility of this evidence will depend on the credibility of the computational model used. Computational models of physiological systems can be complex with numerous non-linearities, time-varying properties, and unknown parameters, which leads to challenges in model assessment. Given the wide range of potential uses of computational patient models in the design and evaluation of physiological closed-loop controlled systems, and the varying risks associated with the diverse uses, the specific model as well as the necessary evidence to make a model credible for a use case may vary. In this review, we examine the various uses of computational patient models in the design and evaluation of critical care physiological closed-loop controlled systems (e.g., hemodynamic stability, mechanical ventilation, anesthetic delivery) as well as the types of evidence (e.g., verification, validation, and uncertainty quantification activities) presented to support the model for that use. We then examine and discuss how a credibility assessment framework (American Society of Mechanical Engineers Verification and Validation Subcommittee, V&V 40 Verification and Validation in Computational Modeling of Medical Devices) for medical devices can be applied to computational patient models used to test physiological closed-loop controlled systems.

Developing a personalized closed-loop controller of medically-induced coma in a rodent model

OBJECTIVE

Personalized automatic control of medically-induced coma, a critical multi-day therapy in the intensive care unit, could greatly benefit clinical care and further provide a novel scientific tool for investigating how the brain response to anesthetic infusion rate changes during therapy. Personalized control would require real-time tracking of inter- and intra-subject variabilities in the brain response to anesthetic infusion rate while simultaneously delivering the therapy, which has not been achieved. Current control systems for medically-induced coma require a separate offline model fitting experiment to deal with inter-subject variabilities, which would lead to therapy interruption. Removing the need for these offline interruptions could help facilitate clinical feasbility. In addition, current systems do not track intra-subject variabilities. Tracking intra-subject variabilities is essential for studying whether or how the brain response to anesthetic infusion rate changes during therapy. Further, such tracking could enhance control precison and thus help facilitate clinical feasibility.

APPROACH

Here we develop a personalized closed-loop anesthetic delivery (CLAD) system in a rodent model that tracks both inter- and intra-subject variabilities in real time while simultaneously controlling the anesthetic in closed loop. We tested the CLAD in rats by administrating propofol to control the electroencephalogram (EEG) burst suppression. We first examined whether the CLAD can remove the need for offline model fitting interruption. We then used the CLAD as a tool to study whether and how the brain response to anesthetic infusion rate changes as a function of changes in the depth of medically-induced coma. Finally, we studied whether the CLAD can enhance control compared with prior systems by tracking intra-subject variabilities.

MAIN RESULTS

The CLAD precisely controlled the EEG burst suppression in each rat without performing offline model fitting experiments. Further, using the CLAD, we discovered that the brain response to anesthetic infusion rate varied during control, and that these variations correlated with the depth of medically-induced coma in a consistent manner across individual rats. Finally, tracking these variations reduced control bias and error by more than 70% compared with prior systems.

SIGNIFICANCE

This personalized CLAD provides a new tool to study the dynamics of brain response to anesthetic infusion rate and has significant implications for enabling clinically-feasible automatic control of medically-induced coma.

Variability in pharmacologically-induced coma for treatment of refractory status epilepticus

Objective

To characterize the amount of EEG suppression achieved in refractory status epilepticus (RSE) patients treated with pharmacologically-induced coma (PIC).

Methods

We analyzed EEG recordings from 35 RSE patients between 21–84 years-old who received PIC that target burst suppression and quantified the amount of EEG suppression using the burst suppression probability (BSP). Then we measured the variability of BSPs with respect to a reference level of BSP 0.8 ± 0.15. Finally, we also measured the variability of BSPs with respect to the amount of intravenous anesthetic drugs (IVADs) received by the patients.

Results

Patients remained in the reference BSP range for only 8% (median, interquartile range IQR [0, 29] %) of the total time under treatment. The median time with BSP below the reference range was 84% (IQR [37, 100] %). BSPs in some patients drifted significantly over time despite constant infusion rates of IVADs. Similar weight-normalized infusion rates of IVADs in different patients nearly always resulted in distinct BSPs (probability 0.93 (IQR [0.82, 1.0]).

Conclusion

This study quantitatively identified high variability in the amount of EEG suppression achieved in clinical practice when treating RSE patients. While some of this variability may arise from clinicians purposefully deviating from clinical practice guidelines, our results show that the high variability also arises in part from significant inter- and intra- individual pharmacokinetic/pharmacodynamic variation. Our results indicate that the delicate balance between maintaining sufficient EEG suppression in RSE patients and minimizing IVAD exposure in clinical practice is challenging to achieve. This may affect patient outcomes and confound studies seeking to determine an optimal amount of EEG suppression for treatment of RSE. Therefore, our analysis points to the need for developing an alternative paradigm, such as vigilant anesthetic management as happens in operating rooms, or closed-loop anesthesia delivery, for investigating and providing induced-coma therapy to RSE patients.

Design, implementation, and evaluation of a physiological closed-loop control device for medically-induced coma

Concerns regarding reliability and safety, as well as uncertainties about what constitutes adequate performance evaluation, have impeded the clinical translation of PCLC devices. We describe an attempt to address these challenges through design, implementation, and evaluation of a PCLC device for delivering medically-induced coma, with the intention to eventually conduct a clinical trial. This device works by automatically adjusting the infusion rate of propofol - a general anesthetic - in response to an electroencephalogram (EEG) pattern called burst suppression. We also designed and implemented a computational patient model which interfaces with hardware and produces realistic EEG signals in response to propofol infusion. The computational patient model is used in hardware-in-the-loop studies to evaluate the behavior of our PCLC device under realistic perturbations. Finally, we have tested the performance of our PCLC device in rodents. Results from these studies suggest that closed-loop control of medically-induced coma in humans is feasible and robust. Consequently, our work produced a PCLC device and relevant pre-clinical evidence in support of a pilot clinical trial.

Model-based drug administration: current status of target-controlled infusion and closed-loop control

PURPOSE OF REVIEW

Drug administration might be optimized by incorporating pharmacokinetic-dynamic (PK/PD) principles and control engineering theories. This review gives an update of the actual status of target-controlled infusion (TCI) and closed-loop computer-controlled drug administration and the ongoing research in the field.

RECENT FINDINGS

TCI is becoming mature technology clinically used in many countries nowadays with proven safety. Nevertheless, changing populations might require adapting the established PK/PD models. As TCI requires accurate PK/PD models, new models have been developed which should now be incorporated into the pumps to allow more general use of this technology. Closed-loop administration of hypnotic drugs using an electro-encephalographic-derived-controlled variable has been well studied and has been shown to outperform manual administration. Computer administration for other drugs and fluids have been studied recently. Feasibility has been shown for systems controlling multiple components of anaesthesia, but more work is required to show clinical safety and efficiency.

SUMMARY

Evidence in the literature is increasing that TCI and closed-loop technology could assist the anaesthetists to optimize drug administration during anaesthesia.