Development and usability testing of a patient digital twin for critical care education: a mixed methods study

Background

Digital twins are computerized patient replicas that allow clinical interventions testing in silico to minimize preventable patient harm. Our group has developed a novel application software utilizing a digital twin patient model based on electronic health record (EHR) variables to simulate clinical trajectories during the initial 6 h of critical illness. This study aimed to assess the usability, workload, and acceptance of the digital twin application as an educational tool in critical care.

Methods

A mixed methods study was conducted during seven user testing sessions of the digital twin application with thirty-five first-year internal medicine residents. Qualitative data were collected using a think-aloud and semi-structured interview format, while quantitative measurements included the System Usability Scale (SUS), NASA Task Load Index (NASA-TLX), and a short survey.

Results

Median SUS scores and NASA-TLX were 70 (IQR 62.5-82.5) and 29.2 (IQR 22.5-34.2), consistent with good software usability and low to moderate workload, respectively. Residents expressed interest in using the digital twin application for ICU rotations and identified five themes for software improvement: clinical fidelity, interface organization, learning experience, serious gaming, and implementation strategies.

Conclusion

A digital twin application based on EHR clinical variables showed good usability and high acceptance for critical care education.

Development and Verification of a Digital Twin Patient Model to Predict Specific Treatment Response During the First 24 Hours of Sepsis

To develop and verify a digital twin model of critically ill patient using the causal artificial intelligence approach to predict the response to specific treatment during the first 24 hours of sepsis.

DESIGN

Directed acyclic graphs were used to define explicitly the causal relationship among organ systems and specific treatments used. A hybrid approach of agent-based modeling, discrete-event simulation, and Bayesian network was used to simulate treatment effect across multiple stages and interactions of major organ systems (cardiovascular, neurologic, renal, respiratory, gastrointestinal, inflammatory, and hematology). Organ systems were visualized using relevant clinical markers. The application was iteratively revised and debugged by clinical experts and engineers. Agreement statistics was used to test the performance of the model by comparing the observed patient response versus the expected response (primary and secondary) predicted by digital twin.

SETTING

Medical ICU of a large quaternary- care academic medical center in the United States.

PATIENTS OR SUBJECTS

Adult (> 18 year yr old), medical ICU patients were included in the study.

INTERVENTIONS

No additional interventions were made beyond the standard of care for this study.

MEASUREMENTS AND MAIN RESULTS

During the verification phase, model performance was prospectively tested on 145 observations in a convenience sample of 29 patients. Median age was 60 years (54-66 d) with a median Sequential Organ Failure Assessment score of 9.5 (interquartile range, 5.0-14.0). The most common source of sepsis was pneumonia, followed by hepatobiliary. The observations were made during the first 24 hours of the ICU admission with one-step interventions, comparing the output in the digital twin with the real patient response. The agreement between the observed versus and the expected response ranged from fair (kappa coefficient of 0.41) for primary response to good (kappa coefficient of 0.65) for secondary response to the intervention. The most common error detected was coding error in 50 observations (35%), followed by expert rule error in 29 observations (20%) and timing error in seven observations (5%).

CONCLUSIONS

We confirmed the feasibility of development and prospective testing of causal artificial intelligence model to predict the response to treatment in early stages of critical illness. The availability of qualitative and quantitative data and a relatively short turnaround time makes the ICU an ideal environment for development and testing of digital twin patient models. An accurate digital twin model will allow the effect of an intervention to be tested in a virtual environment prior to use on real patients.

Comparison of Routine and Computer-Guided Glucose Management for Glycemic Control in Critically Ill Patients

BACKGROUND

Glycemic control is crucial for reducing morbidity and mortality in critically ill patients. A standardized approach to glycemic control using a computer-guided protocol may help maintain blood glucose level within a target range and prevent human-induced medical errors.

OBJECTIVE

To determine the effectiveness of a computer-guided glucose management protocol for glycemic control in intensive care patients.

METHODS

This controlled, open-label implementation study involved 66 intensive care patients: 33 in the intervention group and 33 in the control group. The blood glucose level target range was established as 120 to 180 mg/dL. The control group received the clinic's routine glycemic monitoring approach, and the intervention group received monitoring using newly developed glycemic control software. At the end of the study, nurse perceptions and satisfaction were determined using a questionnaire.

RESULTS

The rates of hyperglycemia and hypoglycemia were lower and the blood glucose level was more successfully maintained in the target range in the intervention group than in the control group (P < .001). The time to achieve the target range was shorter and less insulin was used in the intervention group than in the control group (P < .05). Nurses reported higher levels of satisfaction with the computerized protocol, which they found to be more effective and reliable than routine clinical practice.

CONCLUSIONS

The computerized protocol was more effective than routine clinical practice in achieving glycemic control. It was also associated with higher nurse satisfaction levels.

Prompting with electronic checklist improves clinician performance in medical emergencies: a high-fidelity simulation study

Background

Inefficient processes of care delivery during acute resuscitation can compromise the “Golden Hour,” the time when quick interventions can rapidly determine the course of the patient’s outcome. Checklists have been shown to be an effective tool for standardizing care models. We developed a novel electronic tool, the Checklist for Early Recognition and Treatment of Acute Illness (CERTAIN) to facilitate standardized evaluation and treatment approach for acutely decompensating patients. The checklist was enforced by the use of a “prompter,” a team member separate from the leader who records and reviews pertinent CERTAIN algorithms and verbalizes these to the team. Our hypothesis was that the CERTAIN model, with the use of the tool and a prompter, can improve clinician performance and satisfaction in the evaluation of acute decompensating patients in a simulated environment.

Methods

Volunteer clinicians with valid adult cardiac life support (ACLS) certification were invited to test the CERTAIN model in a high-fidelity simulation center. The first session was used to establish a baseline evaluation in a standard clinical resuscitation scenario. Each subject then underwent online training before returning to a simulation center for a live didactic lecture, software knowledge assessment, and practice scenarios. Each subject was then evaluated on a scenario with a similar content to the baseline. All subjects took a post-experience satisfaction survey. Video recordings of the pre-and post-test sessions were evaluated using a validated method by two blinded reviewers.

Results

Eighteen clinicians completed baseline and post-education sessions. CERTAIN prompting was associated with reduced omissions of critical tasks (46 to 32%, p < 0.01) and 12 out of 14 general assessment tasks were completed in a more timely manner. The post-test survey indicated that 72% subjects felt better prepared during an emergency scenario using the CERTAIN model and 85% would want to be treated with the CERTAIN if they were critically ill.

Conclusion

Prompting with electronic checklist improves clinicians’ performance and satisfaction when dealing with medical emergencies in high-fidelity simulation environment.

Electronic supplementary material

The online version of this article (10.1186/s12245-018-0185-8) contains supplementary material, which is available to authorized users.

Checklist for Early Recognition and Treatment of Acute Illness (CERTAIN): evolution of a content management system for point-of-care clinical decision support

Background

The Checklist for Early Recognition and Treatment of Acute Illness (CERTAIN) is an international collaborative project with the overall objective of standardizing the approach to the evaluation and treatment of critically ill patients world-wide, in accordance with best-practice principles. One of CERTAIN’s key features is clinical decision support providing point-of-care information about common acute illness syndromes, procedures, and medications in an index card format.

Methods

This paper describes 1) the process of developing and validating the content for point-of-care decision support, and 2) the content management system that facilitates frequent peer-review and allows rapid updates of content across different platforms (CERTAIN software, mobile apps, pdf-booklet) and different languages.

Results

Content was created based on survey results of acute care providers and validated using an open peer-review process. Over a 3 year period, CERTAIN content expanded to include 67 syndrome cards, 30 procedure cards, and 117 medication cards. 127 (59 %) cards have been peer-reviewed so far. Initially MS Word® and Dropbox® were used to create, store, and share content for peer-review. Recently Google Docs® was used to make the peer-review process more efficient. However, neither of these approaches met our security requirements nor has the capacity to instantly update the different CERTAIN platforms.

Conclusion

Although we were able to successfully develop and validate a large inventory of clinical decision support cards in a short period of time, commercially available software solutions for content management are suboptimal. Novel custom solutions are necessary for efficient global point of care content system management.

Electronic supplementary material

The online version of this article (doi:10.1186/s12911-016-0367-3) contains supplementary material, which is available to authorized users.

Development and validation of clinical performance assessment in simulated medical emergencies: an observational study

Background

Critical illness is a time-sensitive process which requires practitioners to process vast quantities of data and make decisions rapidly. We have developed a tool, the Checklist for Early Recognition and Treatment of Acute Illness (CERTAIN), aimed at enhancing care delivery in such situations. To determine the efficacy of CERTAIN and similar cognitive aids, we developed rubric for evaluating provider performance in a simulated medical resuscitation environments.

Methods

We recruited 18 clinicians with current valid ACLS certification for evaluation in three simulated medical scenarios designed to mimic typical medical decompensation events routinely experienced in clinical care. Subjects were stratified as experienced or novice based on prior critical care training. A checklist of critical actions was designed using face validity for each scenario to evaluate task completion and performance. Simulation sessions were video recorded and scored by two independent raters. Construct validity was assessed under the assumption that experienced clinicians should perform better than novice clinicians on each task. Reliability was assessed as percentage agreement, kappa statistics and Bland-Altman plots as appropriate.

Results

Eleven experts and seven novices completed evaluation. The overall agreement on common checklist item completion was 84.8 %. The overall model achieved face validity and was consistent with our construct, with experienced clinicians trending towards better performance compared to novices for accuracy and speed of task completion.

Conclusions

A standardized video assessment tool has potential to provide a valid and reliable method to assess 12 performances of clinicians facing simulated medical emergencies.

Electronic supplementary material

The online version of this article (doi:10.1186/s12873-015-0066-x) contains supplementary material, which is available to authorized users.

Checklist for early recognition and treatment of acute illness: International collaboration to improve critical care practice

Processes to ensure world-wide best-practice for critical care delivery are likely to minimize preventable death, disability and costly complications for any healthcare system’s sickest patients, but no large-scale efforts have so far been undertaken towards these goals. The advances in medical informatics and human factors engineering have provided possibility for novel and user-friendly clinical decision support tools that can be applied in a complex and busy hospital setting. To facilitate timely and accurate best-practice delivery in critically ill patients international group of intensive care unit (ICU) physicians and researchers developed a simple decision support tool: Checklist for Early Recognition and Treatment of Acute Illness (CERTAIN). The tool has been refined and tested in high fidelity simulated clinical environment and has been shown to improve performance of clinical providers faced with simulated emergencies. The aim of this international educational intervention is to implement CERTAIN into clinical practice in hospital settings with variable resources (included those in low income countries) and evaluate the impact of the tool on the care processes and patient outcomes. To accomplish our aims, CERTAIN will be uniformly available on either mobile or fixed computing devices (as well as a backup paper version) and applied in a standardized manner in the ICUs of diverse hospitals. To ensure the effectiveness of the proposed intervention, access to CERTAIN is coupled with structured training of bedside ICU providers.

A survey on critical care resources and practices in low- and middle-income countries

BACKGROUND

Timely and appropriate care is the key to achieving good outcomes in acutely ill patients, but the effectiveness of critical care may be limited in resource-limited settings.

OBJECTIVES

This study sought to understand how to implement best practices in intensive care units (ICU) in low- and middle-income countries (LMIC) and to develop a point-of-care training and decision-support tool.

METHODS

An internationally representative group of clinicians performed a 22-item capacity-and-needs assessment survey in a convenience sample of 13 ICU in Eastern Europe (4), Asia (4), Latin America (3), and Africa (2), between April and July 2012. Two ICU were from low-income, 2 from low-middle-income, and 9 from upper-middle-income countries. Clinician respondents were asked about bed capacity, patient characteristics, human resources, available medications and equipment, access to education, and processes of care.

RESULTS

Thirteen clinicians from each of 13 hospitals (1 per ICU) responded. Surveyed hospitals had median of 560 (interquartile range [IQR]: 232, 1,200) beds. ICU had a median of 9 (IQR: 7, 12) beds and treated 40 (IQR: 20, 67) patients per month. Many ICU had ≥ 1 staff member with some formal critical care training (n = 9, 69%) or who completed Fundamental Critical Care Support (n = 7, 54%) or Advanced Cardiac Life Support (n = 9, 69%) courses. Only 2 ICU (15%) used any kind of checklists for acute resuscitation. Ten (77%) ICU listed lack of trained staff as the most important barrier to improving the care and outcomes of critically ill patients.

CONCLUSIONS

In a convenience sample of 13 ICU from LMIC, specialty-trained staff and standardized processes of care such as checklists are frequently lacking. ICU needs-assessment evaluations should be expanded in LMIC as a global priority, with the goal of creating and evaluating context-appropriate checklists for ICU best practices.

Temporal trends of ventilator-associated pneumonia incidence and the effect of implementing health-care bundles in a suburban community

BACKGROUND

Recent changes in critical care delivery, including the widespread implementation of health-care bundles, were aimed at reducing complications of critical illness, in particular ventilator-associated pneumonia (VAP), but no population-based study evaluated its effectiveness.

METHODS

Using a previously validated electronic medical record database, we identified adult (≥ 18 years old) critically ill patients from Olmsted County, Minnesota, requiring mechanical ventilation for ≥ 48 h from January 2003 to December 2009. Trained intensivists identified cases of VAP according to different established clinical definitions. The incidence and outcome of VAP was compared before and after implementation of the so-called "VAP bundle."

RESULTS

The median age, severity of illness, proportion of surgical patients, and patients with neurologic disease increased over time (P &lt; .05 for trend in all). Regardless of the definition used, the VAP rate remained similar throughout the study period and did not change with the introduction of the VAP bundle. According to previous Centers for Disease Control and Prevention criteria, the yearly estimates of the VAP incidence ranged between 7.1 and 10.4 cases per 1,000 ventilator-days, with an age-adjusted incidence of 3.1 vs 5.6 per 100,000 population (P = .54 for trends). Standardized hospital mortality ratio of patients at high risk to develop VAP significantly decreased from 1.7 (95% CI, 0.8-3.0) to 0.7 (95% CI, 0.3-1.4; P = .0003 for trend).

CONCLUSIONS

The incidence of VAP was unaffected by the implementation of the VAP bundle. Secular changes in hospital mortality are unlikely to be attributed to the VAP bundle per se.

Outcome of adverse events and medical errors in the intensive care unit: a systematic review and meta-analysis

Adverse events and medical errors (AEs/MEs) are more likely to occur in the intensive care unit (ICU). Information about the incidence and outcomes of such events is conflicting. A systematic review and meta-analysis were conducted to examine the effects of MEs/AEs on mortality and hospital and ICU lengths of stay among ICU patients. Potentially eligible studies were identified from 4 major databases. Of 902 studies screened, 12 met the inclusion criteria, 10 of which are included in the quantitative analysis. Patients with 1 or more MEs/AEs (vs no MEs/AEs) had a nonsignificant increase in mortality (odds ratio = 1.5; 95% confidence interval [CI] = 0.98-2.14) but significantly longer hospital and ICU stays; the mean difference (95% CI) was 8.9 (3.3-14.7) days for hospital stay and 6.8 (0.2-13.4) days for ICU. The ICU environment is associated with a substantial incidence of MEs/AEs, and patients with MEs/AEs have worse outcomes than those with no MEs/AEs.

Initial ventilator settings for critically ill patients

The lung-protective mechanical ventilation strategy has been standard practice for management of acute respiratory distress syndrome (ARDS) for more than a decade. Observational data, small randomized studies and two recent systematic reviews suggest that lung protective ventilation is both safe and potentially beneficial in patients who do not have ARDS at the onset of mechanical ventilation. Principles of lung-protective ventilation include: a) prevention of volutrauma (tidal volume 4 to 8 ml/kg predicted body weight with plateau pressure <30 cmH₂O); b) prevention of atelectasis (positive end-expiratory pressure ≥5 cmH₂O, as needed recruitment maneuvers); c) adequate ventilation (respiratory rate 20 to 35 breaths per minute); and d) prevention of hyperoxia (titrate inspired oxygen concentration to peripheral oxygen saturation (SpO₂) levels of 88 to 95%). Most patients tolerate lung protective mechanical ventilation well without the need for excessive sedation. Patients with a stiff chest wall may tolerate higher plateau pressure targets (approximately 35 cmH₂O) while those with severe ARDS and ventilator asynchrony may require a short-term neuromuscular blockade. Given the difficulty in timely identification of patients with or at risk of ARDS and both the safety and potential benefit in patients without ARDS, lung-protective mechanical ventilation is recommended as an initial approach to mechanical ventilation in both perioperative and critical care settings.

The Efficacy of Ultrasound-Guided Transversus Abdominis Plane Block in Patients Undergoing Hysterectomy

The study objective of this prospective, double-blind randomised controlled study was to evaluate the efficacy of ultrasound guided transversus abdominis plane (TAP) block and bupivacaine infiltration of the skin and subcutaneous tissue of the wound in patients undergoing hysterectomy. Patients were randomly allocated to three groups: a control group (n=18) and TAP block group (n=18) received bilateral TAP blocks with saline and bupivacaine respectively, and an infiltration group (n=19) received skin and subcutaneous wound tissue infiltration with bupivacaine at the end of surgery. After surgery patients received patient-controlled intravenous tramadol and were assessed for pain and tramadol consumption at 1, 2, 4, 6 and 24 hours.

Both the TAP and infiltration groups had lower movement and rest pain scores than the control group, with lower scores in the TAP group than the infiltration group at 6 and 24 hours. Total tramadol consumption was significantly lower in the TAP group than in the other groups at all time points. We concluded that ultrasound-guided TAP block reduced rest and movement pain after total abdominal hysterectomy and was more effective than superficial wound infiltration for postoperative pain management.

Transforaminal epidural steroid injection for the treatment of severe back pain caused by cement leakage during kyphoplasty procedure

Kyphoplasty is a minimally invasive procedure that is increasingly used to treat pain caused by compression fractures of vertebral bodies. A 56-year-old woman who had a compression fracture on the vertebral body of L5 vertebra was admitted to the Algology Department with a severe low back and leg pain. Kyphoplasty was planned for pain relief. She suffered severe pain in her back and left leg immediately after the procedure because of a leakage of injected cement through the fracture line. After injection of triamcinolone and bupivacaine transforaminally into the L5-S1 anterior epidural space, her pain complaints ended. If radicular pain symptoms caused by cement leakage are secondary to a chemically mediated non-cellular inflammatory reaction, transforaminal epidural steroid injection should be useful.