Prospective External Validation of a Commercial Model Predicting the Likelihood of Inpatient Admission From the Emergency Department

STUDY OBJECTIVE

Early notification of admissions from the emergency department (ED) may allow hospitals to plan for inpatient bed demand. This study aimed to assess Epic's ED Likelihood to Occupy an Inpatient Bed predictive model and its application in improving hospital bed planning workflows.

METHODS

All ED adult (18 years and older) visits from September 2021 to August 2022 at a large regional health care system were included. The primary outcome was inpatient admission. The predictive model is a random forest algorithm that uses demographic and clinical features. The model was implemented prospectively, with scores generated every 15 minutes. The area under the receiver operator curves (AUROC) and precision-recall curves (AUPRC) were calculated using the maximum score prior to the outcome and for each prediction independently. Test characteristics and lead time were calculated over a range of model score thresholds.

RESULTS

Over 11 months, 329,194 encounters were evaluated, with an incidence of inpatient admission of 25.4%. The encounter-level AUROC was 0.849 (95% confidence interval [CI], 0.848 to 0.851), and the AUPRC was 0.643 (95% CI, 0.640 to 0.647). With a prediction horizon of 6 hours, the AUROC was 0.758 (95% CI, 0.758 to 0.759,) and the AUPRC was 0.470 (95% CI, 0.469 to 0.471). At a predictive model threshold of 40, the sensitivity was 0.49, the positive predictive value was 0.65, and the median lead-time warning was 127 minutes before the inpatient bed request.

CONCLUSION

The Epic ED Likelihood to Occupy an Inpatient Bed model may improve hospital bed planning workflows. Further study is needed to determine its operational effect.

Mobile Robotic Platform for Contactless Vital Sign Monitoring

The COVID-19 pandemic has accelerated methods to facilitate contactless evaluation of patients in hospital settings. By minimizing in-person contact with individuals who may have COVID-19, healthcare workers can prevent disease transmission and conserve personal protective equipment. Obtaining vital signs is a ubiquitous task that is commonly done in person by healthcare workers. To eliminate the need for in-person contact for vital sign measurement in the hospital setting, we developed Dr. Spot, a mobile quadruped robotic system. The system includes IR and RGB cameras for vital sign monitoring and a tablet computer for face-to-face medical interviewing. Dr. Spot is teleoperated by trained clinical staff to simultaneously measure the skin temperature, respiratory rate, and heart rate while maintaining social distancing from patients and without removing their mask. To enable accurate, contactless measurements on a mobile system without a static black body as reference, we propose novel methods for skin temperature compensation and respiratory rate measurement at various distances between the subject and the cameras, up to 5 m. Without compensation, the skin temperature MAE is 1.3°C. Using the proposed compensation method, the skin temperature MAE is reduced to 0.3°C. The respiratory rate method can provide continuous monitoring with a MAE of 1.6 BPM in 30 s or rapid screening with a MAE of 2.1 BPM in 10 s. For the heart rate estimation, our system is able to achieve a MAE less than 8 BPM in 10 s measured in arbitrary indoor light conditions at any distance below 2 m.

A 1-year training program in emergency medicine for physicians in Karachi, Pakistan: Evaluation of learner and program outcomes

BACKGROUND

Pakistan has an underdeveloped and overburdened emergency care system, with most emergency departments (EDs) staffed by physicians not formally trained in emergency medicine (EM). As of January 2020, only nine Pakistani institutions were providing formal EM specialty training; therefore, a training program of shorter duration is needed in the interim.

METHODS

The Certification Program in Emergency Medicine (CPEM) is a 1-year training program in EM consisting of two arms: CPEM-Clinical (CPEM-C), which includes physicians from The Indus Hospital (TIH) ED, and CPEM-Didactic (CPEM-D), including physicians from EDs across Karachi. Both groups participate in weekly conferences, including didactics, small-group discussions, workshops, and journal clubs. CPEM-C learners also receive clinical mentorship from visiting international and TIH EM faculty. Both groups were assessed with preprogram, midterm, and final examinations as well as on clinical skills. Additionally, both groups provided regular feedback on program content and administration.

RESULTS

Twenty-five of the 32 initially enrolled learners completed the program in June 2019. Scores on a matched set of 50 questions administered in the pretest and final examination improved by an average of 15.1% (p < 0.005) for CPEM-C learners and 8.5% (p < 0.0005) for CPEM-D learners, with 93% of learners showing improvement. Clinical evaluations of CPEM-C and CPEM-D learners during the first and fourth quarters showed an average improvement of 1.1 out of 5 (p < 0.05) and 1.2 out of 9 (p < 0.0005) points, respectively. Learner evaluations of the program were overall positive.

CONCLUSIONS

CPEM demonstrated significant improvement in test scores and clinical evaluations in both program arms. Evaluations also suggested that the program was well received. These data, along with CPEM's ability to train physicians from multiple institutions using low-cost, innovative educational strategies, suggest that it may be an effective, transferable mechanism for the expedited development of EM in Pakistan and countries where EM is developing as a specialty.

Assessment of the Acceptability and Feasibility of Using Mobile Robotic Systems for Patient Evaluation

IMPORTANCE

Before the widespread implementation of robotic systems to provide patient care during the COVID-19 pandemic occurs, it is important to understand the acceptability of these systems among patients and the economic consequences associated with the adoption of robotics in health care settings.

OBJECTIVE

To assess the acceptability and feasibility of using a mobile robotic system to facilitate health care tasks.

DESIGN, SETTING, AND PARTICIPANTS

This study included 2 components: a national survey to examine the acceptability of using robotic systems to perform health care tasks in a hospital setting and a single-site cohort study of patient experiences and satisfaction with the use of a mobile robotic system to facilitate triage and telehealth tasks in the emergency department (ED). The national survey comprised individuals living in the US who participated in a sampling-based survey via an online analytic platform. Participants completed the national survey between August 18 and August 21, 2020. The single-site cohort study included patients living in the US who presented to the ED of a large urban academic hospital providing quaternary care in Boston, Massachusetts between April and August 2020. All data were analyzed from August to October 2020.

EXPOSURES

Participants in the national survey completed an online survey to measure the acceptability of using a mobile robotic system to perform health care tasks (facilitating telehealth interviews, acquiring vital signs, obtaining nasal or oral swabs, placing an intravenous catheter, performing phlebotomy, and turning a patient in bed) in a hospital setting in the contexts of general interaction and interaction during the COVID-19 pandemic. Patients in the cohort study were exposed to a mobile robotic system, which was controlled by an ED clinician and used to facilitate a triage interview. After exposure, patients completed an assessment to measure their satisfaction with the robotic system.

MAIN OUTCOMES AND MEASURES

Acceptability of the use of a mobile robotic system to facilitate health care tasks in a hospital setting (national survey) and feasibility and patient satisfaction regarding the use of a mobile robotic system in the ED (cohort study).

RESULTS

For the national survey, 1154 participants completed all acceptability questions, representing a participation rate of 35%. After sample matching, a nationally representative sample of 1000 participants (mean [SD] age, 48.7 [17.0] years; 535 women [53.5%]) was included in the analysis. With regard to the usefulness of a robotic system to perform specific health care tasks, the response of "somewhat useful" was selected by 373 participants (37.3%) for facilitating telehealth interviews, 350 participants (35.0%) for acquiring vital signs, 307 participants (30.7%) for obtaining nasal or oral swabs, 228 participants (22.8%) for placing an intravenous catheter, 249 participants (24.9%) for performing phlebotomy, and 371 participants (37.1%) for turning a patient in bed. The response of "extremely useful" was selected by 287 participants (28.7%) for facilitating telehealth interviews, 413 participants (41.3%) for acquiring vital signs, 192 participants (19.2%) for obtaining nasal or oral swabs, 159 participants (15.9%) for placing an intravenous catheter, 167 participants (16.7%) for performing phlebotomy, and 371 participants (37.1%) for turning a patient in bed. In the context of the COVID-19 pandemic, the median number of individuals who perceived the application of robotic systems to be acceptable for completing telehealth interviews, obtaining nasal and oral swabs, placing an intravenous catheter, and performing phlebotomy increased. For the ED cohort study, 51 individuals were invited to participate, and 41 participants (80.4%) enrolled. One participant was unable to complete the study procedures because of a signaling malfunction in the robotic system. Forty patients (mean [SD] age, 45.8 [2.7] years; 29 women [72.5%]) completed the mobile robotic system-facilitated triage interview, and 37 patients (92.5%) reported that the interaction was satisfactory. A total of 33 participants (82.5%) reported that their experience of receiving an interview facilitated by a mobile robotic system was as satisfactory as receiving an in-person interview from a clinician.

CONCLUSIONS AND RELEVANCE

In this study, a mobile robotic system was perceived to be acceptable for use in a broad set of health care tasks among survey respondents across the US. The use of a mobile robotic system enabled the facilitation of contactless triage interviews of patients in the ED and was considered acceptable among participants. Most patients in the ED rated the quality of mobile robotic system-facilitated interaction to be equivalent to in-person interaction with a clinician.

Addressing Racism in Medicine Through a Resident-Led Health Equity Retreat

Racism impacts patient care and clinical training in emergency medicine (EM), but dedicated racism training is not required in graduate medical education. We designed an innovative health equity retreat to teach EM residents about forms of racism and skills for responding to racial inequities in clinical environments. The three-hour retreat occurred during the residency didactic conference to maximize resident participation. We prioritized facilitated reflection on residents’ own experiences of race and racism in medicine in order to emphasize these concepts’ relevance to all participants. We used workshop, small group, and panel formats to optimize interactivity and discussion. Post-retreat survey respondents indicated that the curriculum successfully promoted awareness of racism in the workplace. Participants also expressed interest in continued discussions about racism in medicine as well as desire for greater faculty and nursing participation in the curriculum. Residency programs should consider incorporating similar educational sessions in core didactic curricula.

Development and validation of a novel prediction score for cardiac tamponade in emergency department patients with pericardial effusion

Aims

Determining which patients with pericardial effusion require urgent intervention can be challenging. We sought to develop a novel, simple risk prediction score for patients with pericardial effusion.

Methods and results

Adult patients admitted through the emergency department (ED) with pericardial effusion were retrospectively evaluated. The overall cohort was divided into a derivation and validation cohort for the generation and validation of a novel risk score using logistic regression. The primary outcome was a pericardial drainage procedure or death attributed to cardiac tamponade within 24 h of ED arrival. Among 195 eligible patients, 102 (52%) experienced the primary outcome. Four variables were selected for the novel score: systolic blood pressure < 100 mmHg (1.5 points), effusion diameter [1–2 cm (0 points), 2–3 cm (1.5 points), >3 cm (2 points)], right ventricular diastolic collapse (2 points), and mitral inflow velocity variation > 25% (1 point). The need for pericardial drainage within 24 h was stratified as low (<2 points), intermediate (2–4 points), or high (≥4 points), which corresponded to risks of 8.1% [95% confidence interval (CI) 3.0–16.8%], 63.8% [95% CI 50.1–76.0%], and 93.7% [95% CI 84.5–98.2%]. The area under the curve of the simplified score was 0.94 for the derivation and 0.91 for the validation cohort.

Conclusion

Among ED patients with pericardial effusion, a four-variable prediction score consisting of systolic blood pressure, effusion diameter, right ventricular collapse, and mitral inflow velocity variation can accurately predict the need for urgent pericardial drainage. Prospective validation of this novel score is warranted.

Digital triage: Novel strategies for population health management in response to the COVID-19 pandemic

The COVID-19 pandemic has created unique challenges for the U.S. healthcare system due to the staggering mismatch between healthcare system capacity and patient demand. The healthcare industry has been a relatively slow adopter of digital innovation due to the conventional belief that humans need to be at the center of healthcare delivery tasks. However, in the setting of the COVID-19 pandemic, artificial intelligence (AI) may be used to carry out specific tasks such as pre-hospital triage and enable clinicians to deliver care at scale. Recognizing that the majority of COVID-19 cases are mild and do not require hospitalization, Partners HealthCare (now Mass General Brigham) implemented a digitally-automated pre-hospital triage solution to direct patients to the appropriate care setting before they showed up at the emergency department and clinics, which would otherwise consume resources, expose other patients and staff to potential viral transmission, and further exacerbate supply-and-demand mismatching. Although the use of AI has been well-established in other industries to optimize supply and demand matching, the introduction of AI to perform tasks remotely that were traditionally performed in-person by clinical staff represents a significant milestone in healthcare operations strategy.