Relationship between nursing documentation and patients' mortality

Does nurse use of a standardized flowsheet to document communication with advanced providers provide a mechanism to detect pulse oximetry failures? A retrospective study of electronic health record data

BACKGROUND

Pulse oximetry guides clinical decisions, yet does not uniformly identify hypoxemia. We hypothesized that nursing documentation of notifying providers, facilitated by a standardized flowsheet for documenting communication to providers (physicians, nurse practitioners, and physician assistants), may increase when hypoxemia is present, but undetected by the pulse oximeter, in events termed "occult hypoxemia."

OBJECTIVE

To compare nurse documentation of provider notification in the 4 h preceding cases of occult hypoxemia, normal oxygenation, and evident hypoxemia confirmed by an arterial blood gas reading.

METHODS

We conducted a retrospective study using electronic health record data from patients with COVID-19 at five hospitals in a healthcare system with paired SpO2 and SaO2 readings (measurements within 10 min of oxygen saturation levels in arterial blood, SaO2, and by pulse oximetry, SpO2). We applied multivariate logistic regression to assess if having any nursing documentation of provider notification in the 4 h prior to a paired reading confirming occult hypoxemia was more likely compared to a paired reading confirming normal oxygen status, adjusting for characteristics significantly associated with nursing documentation. We applied conditional logistic regression to assess if having any nursing documentation of provider notification was more likely in the 4-hour window preceding a paired reading compared to the 4-hour window 24 h earlier separately for occult hypoxemia, visible hypoxemia, and normal oxygenation.

RESULTS

There were data from 1910 patients hospitalized with COVID-19 who had 44,972 paired readings and an average of 26.5 (34.5) nursing documentation of provider notification events. The mean age was 63.4 (16.2). Almost half (866/1910, 45.3 %) were White, 701 (36.7 %) were Black, and 239 (12.5 %) were Hispanic. Having any nursing documentation of provider notification was 46 % more common in the 4 h before an occult hypoxemia paired reading compared to a normal oxygen status paired reading (OR 1.46, 95 % CI: 1.28-1.67). Comparing the 4 h immediately before the reading to the 4 h one day preceding the paired reading, there was a higher likelihood of having any nursing documentation of provider notification for both evident (OR 1.45, 95 % CI 1.24-1.68) and occult paired readings (OR 1.26, 95 % CI 1.04-1.53).

CONCLUSION

This study finds that nursing documentation of provider notification significantly increases prior to confirmed occult hypoxemia, which has potential in proactively identifying occult hypoxemia and other clinical issues. There is potential value to encouraging standardized documentation of nurse concern, including communication to providers, to facilitate its inclusion in clinical decision-making.

Multisite Pragmatic Cluster-Randomized Controlled Trial of the CONCERN Early Warning System

Importance

Late predictions of hospitalized patient deterioration, resulting from early warning systems (EWS) with limited data sources and/or a care team's lack of shared situational awareness, contribute to delays in clinical interventions. The COmmunicating Narrative Concerns Entered by RNs (CONCERN) Early Warning System (EWS) uses real-time nursing surveillance documentation patterns in its machine learning algorithm to identify patients' deterioration risk up to 42 hours earlier than other EWSs.

Objective

To test our a priori hypothesis that patients with care teams informed by the CONCERN EWS intervention have a lower mortality rate and shorter length of stay (LOS) than the patients with teams not informed by CONCERN EWS.

Design

One-year multisite, pragmatic controlled clinical trial with cluster-randomization of acute and intensive care units to intervention or usual-care groups.

Setting

Two large U.S. health systems.

Participants

Adult patients admitted to acute and intensive care units, excluding those on hospice/palliative/comfort care, or with Do Not Resuscitate/Do Not Intubate orders.

Intervention

The CONCERN EWS intervention calculates patient deterioration risk based on nurses' concern levels measured by surveillance documentation patterns, and it displays the categorical risk score (low, increased, high) in the electronic health record (EHR) for care team members.

Main Outcomes and Measures

Primary outcomes: in-hospital mortality, LOS; survival analysis was used. Secondary outcomes: cardiopulmonary arrest, sepsis, unanticipated ICU transfers, 30-day hospital readmission.

Results

A total of 60 893 hospital encounters (33 024 intervention and 27 869 usual-care) were included. Both groups had similar patient age, race, ethnicity, and illness severity distributions. Patients in the intervention group had a 35.6% decreased risk of death (adjusted hazard ratio [HR], 0.644; 95% confidence interval [CI], 0.532-0.778; P<.0001), 11.2% decreased LOS (adjusted incidence rate ratio, 0.914; 95% CI, 0.902-0.926; P<.0001), 7.5% decreased risk of sepsis (adjusted HR, 0.925; 95% CI, 0.861-0.993; P=.0317), and 24.9% increased risk of unanticipated ICU transfer (adjusted HR, 1.249; 95% CI, 1.093-1.426; P=.0011) compared with patients in the usual-care group.

Conclusions and Relevance

A hospital-wide EWS based on nursing surveillance patterns decreased in-hospital mortality, sepsis, and LOS when integrated into the care team's EHR workflow.

Trial Registration

ClinicalTrials.gov Identifier: NCT03911687.

Big data research in nursing: A bibliometric exploration of themes and publications

AIMS

To comprehend the current research hotspots and emerging trends in big data research within the global nursing domain.

DESIGN

Bibliometric analysis.

METHODS

The quality articles for analysis indexed by the science core collection were obtained from the Web of Science database as of February 10, 2023.The descriptive, visual analysis and text mining were realized by CiteSpace and VOSviewer.

RESULTS

The research on big data in the nursing field has experienced steady growth over the past decade. A total of 45 core authors and 17 core journals around the world have contributed to this field. The author's keyword analysis has revealed five distinct clusters of research focus. These encompass machine/deep learning and artificial intelligence, natural language processing, big data analytics and data science, IoT and cloud computing, and the development of prediction models through data mining. Furthermore, a comparative examination was conducted with data spanning from 1980 to 2016, and an extended analysis was performed covering the years from 1980 to 2019. This bibliometric mapping comparison allowed for the identification of prevailing research trends and the pinpointing of potential future research hotspots within the field.

CONCLUSIONS

The fusion of data mining and nursing research has steadily advanced and become more refined over time. Technologically, it has expanded from initial natural language processing to encompass machine learning, deep learning, artificial intelligence, and data mining approach that amalgamates multiple technologies. Professionally, it has progressed from addressing patient safety and pressure ulcers to encompassing chronic diseases, critical care, emergency response, community and nursing home settings, and specific diseases (Cardiovascular diseases, diabetes, stroke, etc.). The convergence of IoT, cloud computing, fog computing, and big data processing has opened new avenues for research in geriatric nursing management and community care. However, a global imbalance exists in utilizing big data in nursing research, emphasizing the need to enhance data science literacy among clinical staff worldwide to advance this field.

CLINICAL RELEVANCE

This study focused on the thematic trends and evolution of research on the big data in nursing research. Moreover, this study may contribute to the understanding of researchers, journals, and countries around the world and generate the possible collaborations of them to promote the development of big data in nursing science.

Nursing patient record practice and associated factors among nurses working in North Shewa Zone public hospitals, Ethiopia

Background

Nursing documentation is an essential component of nursing practice and has the potential to improve patient care outcomes. Poor documentation of nursing care activities among nurses has been shown to have negative impacts on healthcare quality.

Objective

To assess the nursing documentation practice and its associated factors among nurses working in the North Shewa Zone public hospitals, Ethiopia.

Method

An institution-based cross-sectional study was conducted at the North Shewa Zone public hospitals. A simple random sampling technique was used to select 421 nurses. A pretested, structured, self-administered questionnaire was used to gather the data. Data were entered into Epi Data version 3.1, and SPSS version 25 was used for further analysis. Binary logistic regressions were performed to identify the independent predictors of nursing documentation practice. Adjusted odds ratio was calculated and a p-value less than 0.05 with 95% confidence interval (CI) was considered as statistically significant.

Result

A total of 421 respondents took part, giving the survey a 100% response rate. The overall good practice of nursing care documentation was 51.1%, 95% CI (46.6, 55.8). In addition, 43.2%, 95% CI (38.5, 48.0) and 35.6%, 95% CI (30.9, 40.1), of nurses had good knowledge of and favorable attitudes toward nursing care documentation. Availability of operational standards for nursing documentation [adjusted odds ratio (AOR) = 1.76; 95% CI: 1.18, 2.64], availability of documenting sheets (AOR = 1.51; 95% CI: 1.01, 2.29), and a monitoring system (AOR = 1.61; 95% CI: 1.07, 2.41) were significantly associated with nursing care documentation practice.

Conclusion

Nearly half of nursing care was not documented. The practice of nursing care documentation was significantly influenced by the availability of operational standards, documenting sheets, and monitoring systems. To improve the documentation practice, a continuous monitoring system and access to operational standards and documenting sheets are needed.

Variability in Nursing Documentation Patterns across Patients' Hospital Stays

This study explores the variability in nursing documentation patterns in acute care and ICU settings, focusing on vital signs and note documentation, and examines how these patterns vary across patients' hospital stays, documentation types, and comorbidities. In both acute care and critical care settings, there was significant variability in nursing documentation patterns across hospital stays, by documentation type, and by patients' comorbidities. The results suggest that nurses adapt their documentation practices in response to their patients' fluctuating needs and conditions, highlighting the need to facilitate more individualized care and tailored documentation practices. The implications of these findings can inform decisions on nursing workload management, clinical decision support tools, and EHR optimizations.

When to err is inhuman: An examination of the influence of artificial intelligence-driven nursing care on patient safety

Artificial intelligence, as a nonhuman entity, is increasingly used to inform, direct, or supplant nursing care and clinical decision-making. The boundaries between human- and nonhuman-driven nursing care are blurred with the advent of sensors, wearables, camera devices, and humanoid robots at such an accelerated pace that the critical evaluation of its influence on patient safety has not been fully assessed. Since the pivotal release of To Err is Human, patient safety is being challenged by the dynamic healthcare environment like never before, with nursing at a critical juncture to steer the course of artificial intelligence integration in clinical decision-making. This paper presents an overview of artificial intelligence and its application in healthcare and highlights the implications which affect nursing as a profession, including perspectives on nursing education and training recommendations. The legal and policy challenges which emerge when artificial intelligence influences the risk of clinical errors and safety issues are discussed.

Nurses' Documentation Practice and Associated Factors in Eight Public Hospitals, Amhara Region, Ethiopia: A Cross-Sectional Study

Background

Nursing care documentation, which is the record of nursing care that is planned for and delivered to individual patients, can enhance patient outcomes while advancing the nursing profession. However, its practice and associated factors among Ethiopian nurses are not well investigated.

Objective

To assess the level of nursing care documentation practice and associated factors among nurses working at public hospitals in Ethiopia.

Methods

An institutional-based cross-sectional study was conducted from May 1 to 30, 2022. A total of 378 nurses and corresponding charts were randomly selected with a multistage sampling technique. Self-administered structured questionnaires and structured checklists were used to collect data about independent variables and nurses' documentation practice, respectively. Epi Data 4.6 was used for data entry and SPSS version 25 for analysis. Descriptive statistics and binary logistic regression analysis have been employed. The STROBE checklist was used to report the study.

Results

In this study, 372 nurses participated, and 30.4% (95% confidence interval [CI]: 26%-35%) of them had good nursing care documentation practice. Adequate knowledge about nursing care documentation(adjusted odds ratio [AOR] = 4.16, 95% CI: [2.36-7.33]), favorable attitude toward nursing care documentation (AOR = 3.43, 95% CI: [1.85-6.36]), adequacy of documenting sheets (AOR = 2.02, 95% CI: [1.14-3.59]), adequacy of time (AOR = 3.85, 95% CI: [2.11-7.05]), nurse-to-patient ratio (AOR = 2.78, 95% CI: [1.13-6.84]), and caring patients who had no stress, anxiety, pain, and distress (AOR = 3.56, 95% CI: [1.69-7.52]) were significantly associated with proper nursing care documentation practices.

Conclusion

Nursing documentation practice was poor in this study compared to the health sector transformation in quality standards due to the identified factors. Improving nurses' knowledge and attitude toward nursing care documentation and increasing access to documentation materials can contribute to improving documentation practice.

Diagnostic suspicion bias and machine learning: Breaking the awareness deadlock for sepsis detection

Many early warning algorithms are downstream of clinical evaluation and diagnostic testing, which means that they may not be useful when clinicians fail to suspect illness and fail to order appropriate tests. Depending on how such algorithms handle missing data, they could even indicate "low risk" simply because the testing data were never ordered. We considered predictive methodologies to identify sepsis at triage, before diagnostic tests are ordered, in a busy Emergency Department (ED). One algorithm used "bland clinical data" (data available at triage for nearly every patient). The second algorithm added three yes/no questions to be answered after the triage interview. Retrospectively, we studied adult patients from a single ED between 2014-16, separated into training (70%) and testing (30%) cohorts, and a final validation cohort of patients from four EDs between 2016-2018. Sepsis was defined per the Rhee criteria. Investigational predictors were demographics and triage vital signs (downloaded from the hospital EMR); past medical history; and the auxiliary queries (answered by chart reviewers who were blinded to all data except the triage note and initial HPI). We developed L2-regularized logistic regression models using a greedy forward feature selection. There were 1164, 499, and 784 patients in the training, testing, and validation cohorts, respectively. The bland clinical data model yielded ROC AUC's 0.78 (0.76-0.81) and 0.77 (0.73-0.81), for training and testing, respectively, and ranged from 0.74-0.79 in four hospital validation. The second model which included auxiliary queries yielded 0.84 (0.82-0.87) and 0.83 (0.79-0.86), and ranged from 0.78-0.83 in four hospital validation. The first algorithm did not require clinician input but yielded middling performance. The second showed a trend towards superior performance, though required additional user effort. These methods are alternatives to predictive algorithms downstream of clinical evaluation and diagnostic testing. For hospital early warning algorithms, consideration should be given to bias and usability of various methods.

Health professionals' routine practice documentation and its associated factors in a resource-limited setting: a cross-sectional study

OBJECTIVES

Documenting routine practice is significant for better diagnosis, treatment, continuity of care and medicolegal issues. However, health professionals' routine practice documentation is poorly practised. Therefore, this study aimed to assess health professionals' routine practice documentation and associated factors in a resource-limited setting.

METHODS

An institution-based cross-sectional study design was used from 24 March up to 19 April 2022. Stratified random sampling and a pretested self-administered questionnaire were used among 423 samples. Epi Info V.7.1 and STATA V.15 software were used for data entry and analysis, respectively. Descriptive statistics and a logistic regression model were employed to describe the study subjects and to measure the strength of association between dependent and independent variables, respectively. A variable with a p value of <0.2 in bivariate logistic regression was considered for multivariable logistic regression. In multivariable logistic regression, ORs with 95% CIs and a p value of <0.05 were considered to determine the strength of association between dependent and independent variables.

RESULTS

Health professionals' documentation practice was 51.1% (95% CI: 48.64 to 53.1). Lack of motivation (adjusted OR (AOR): 0.41, 95% CI: 0.22 to 0.76), good knowledge (AOR: 1.35, 95% CI: 0.72 to 2.97), taking training (AOR: 4.18, 95% CI: 2.99 to 8.28), using electronic systems (AOR: 2.19, 95% CI: 1.36 to 3.28), availability of standard documentation tools (AOR: 2.45, 95% CI: 1.35 to 4.43) were statistically associated factors.

CONCLUSIONS

Health professionals' documentation practice is good. Lack of motivation, good knowledge, taking training, using electronic systems and the availability of documentation tools were significant factors. Stakeholders should provide additional training, and encourage professionals to use an electronic system for documentation practices.

Diagnostic accuracy of clinical outcome prediction using nursing data in intensive care patients: A systematic review

BACKGROUND

Nursing data consist of observations of patients' conditions and information on nurses' clinical judgment based on critically ill patients' behavior and physiological signs. Nursing data in electronic health records were recently emphasized as important predictors of patients' deterioration but have not been systematically reviewed.

OBJECTIVE

We conducted a systematic review of prediction models using nursing data for clinical outcomes, such as prolonged hospital stay, readmission, and mortality in intensive care patients, compared to physiological data only. In addition, the type of nursing data used in prediction model developments was investigated.

DESIGN

A systematic review.

METHODS

PubMed, CINAHL, Cochrane CENTRAL, EMBASE, IEEE Xplore Digital Library, Web of Science, and Scopus were searched. Clinical outcome prediction models using nursing data for intensive care patients were included. Clinical outcomes were prolonged hospital stay, readmission, and mortality. Data were extracted from selected studies such as study design, data source, outcome definition, sample size, predictors, reference test, model development, model performance, and evaluation. The risk of bias and applicability was assessed using the Prediction model Risk of Bias Assessment Tool checklist. Descriptive summaries were produced based on paired forest plots and summary receiver operating characteristic curves.

RESULTS

Sixteen studies were included in the systematic review. The data types of predictors used in prediction models were categorized as physiological data, nursing data, and clinical notes. The types of nursing data consisted of nursing notes, assessments, documentation frequency, and flowsheet comments. The studies using physiological data as a reference test showed higher predictive performance in combined data or nursing data than in physiological data. The overall risk of bias indicated that most of the included studies have a high risk.

CONCLUSIONS

This study was conducted to identify and review the diagnostic accuracy of clinical outcome prediction using nursing data in intensive care patients. Most of the included studies developed models using nursing notes, and other studies used nursing assessments, documentation frequency, and flowsheet comments. Although the findings need careful interpretation due to the high risk of bias, the area under the curve scores of nursing data and combined data were higher than physiological data alone. It is necessary to establish a strategy in prediction modeling to utilize nursing data, clinical notes, and physiological data as predictors, considering the clinical context rather than physiological data alone.

REGISTRATION

The protocol for this study is registered with PROSPERO (registration number: CRD42021273319).

Original Research: Practice Variations in Documenting Neurologic Examinations in Non-Neuroscience ICUs

BACKGROUND

In critical care units, the neurologic examination (neuro exam) is used to detect changes in neurologic function. Serial neuro exams are a hallmark of monitoring in neuroscience ICUs. But less is known about neuro exams that are performed in non-neuroscience ICUs. This knowledge gap likely contributes to the insufficient guidance on what constitutes an adequate neuro exam for patients admitted to a non-neuroscience ICU.

PURPOSE

The study purpose was to explore existing practices for documenting neuro exams in ICUs that don't routinely admit patients with a primary neurologic injury.

METHODS

A single-center, prospective, observational study examined documented neuro exams performed in medical, surgical, and cardiovascular ICUs. A comprehensive neuro exam assesses seven domains that can be divided into 20 components. In this study, each component was scored as present (documentation was found) or absent (documentation was not found); a domain was scored as present if one or more of its components had been documented.

RESULTS

There were 1,482 assessments documented on 120 patients over a one-week period. A majority of patients were male (56%), White (71%), non-Hispanic (77%), and over 60 years of age (50%). Overall, assessments of the domains of consciousness, injury severity, and cranial nerve function were documented 80% of the time or more. Assessments of the domains of pain, motor function, and sensory function were documented less than 60% of the time, and that of speech less than 5% of the time. Statistically significant differences in documentation were found between the medical, surgical, and cardiovascular ICUs for the domains of speech, cranial nerve function, and pain. There were no significant differences in documentation frequency between day and night shift nurses. Documentation practices were significantly different for RNs versus providers.

CONCLUSIONS

Our findings show that the frequency and specific components of neuro exam documentation vary significantly across nurses, providers, and ICUs. These findings are relevant for nurses and providers and may help to improve guidance for neurologic assessment of patients in non-neurologic ICUs. Further studies exploring variance in documentation practices and their implications for courses of treatment and patient outcomes are warranted.

What do we know about experiencing end-of-life in burn intensive care units? A scoping review

OBJECTIVES

The aim of this article is to review and synthesize the evidence on end-of-life in burn intensive care units.

METHODS

Systematic scoping review: Preferred Reporting Items for Systemic Reviews extension for Scoping Reviews was used as a reporting guideline. Searches were performed in 3 databases, with no time restriction and up to September 2021.

RESULTS

A total of 16,287 documents were identified; 18 were selected for analysis and synthesis. Three key themes emerged: (i) characteristics of the end-of-life in burn intensive care units, including end-of-life decisions, decision-making processes, causes, and trajectories of death; (ii) symptom control at the end-of-life in burn intensive care units focusing on patients' comfort; and (iii) concepts, models, and designs of the care provided to burned patients at the end-of-life, mainly care approaches, provision of care, and palliative care.

SIGNIFICANCE OF RESULTS

End-of-life care is a major step in the care provided to critically ill burned patients. Dying and death in burn intensive care units are often preceded by end-of-life decisions, namely forgoing treatment and do-not-attempt to resuscitate. Different dying trajectories were described, suggesting the possibility to develop further studies to identify triggers for palliative care referral. Symptom control was not described in detail. Palliative care was rarely involved in end-of-life care for these patients. This review highlights the need for early and high-quality palliative and end-of-life care in the trajectories of critically ill burned patients, leading to an improved perception of end-of-life in burn intensive care units. Further research is needed to study the best way to provide optimal end-of-life care and foster integrated palliative care in burn intensive care units.

Admission and Mortality Patterns in Intensive Care Delivery at Enugu State University of Science and Technology Teaching Hospital: A Three-Year Retrospective Study

Background

The intensive care unit (ICU) provides critical care to high-risk patients to prevent morbidity and mortality. This requires closer monitoring and better management than the care provided to patients in normal admission wards and non-critical care units. Mortality rates in ICUs in developing countries are remarkably high compared to rates in more developed countries. Evaluating outcomes of treatment is a way to improve the quality of care. Therefore, this study was conducted to review the pattern of admission and outcome in the ICU of Enugu State University of Science and Technology Teaching Hospital (ESUT-TH).

Methodology

This study was a three-year retrospective, descriptive review of all patients admitted to the ICU of ESUT-TH between January 1, 2019, and December 21, 2021. Data were collected from admissions and discharge registers of the ICU ward. Data were analyzed and expressed as frequencies and percentages. Categorical parameters were compared using the chi-squared test, and the significance level was set at p < 0.05.

Results

A total of 179 patients were admitted in the three-year period. Of them, 49.2% were postoperative patients while 21.2% were admitted from the accident and emergency unit. There were a total of 74 (41.3%) medical cases and 81 (45.3%) surgical cases, and the rest were unspecified. Among surgical cases, 19% were from the general surgery department followed by obstetrics and gynecology (18.4%) and neurosurgery (16.8%). Cerebrovascular accidents and traumatic brain injury were the most common specific diagnoses recorded among ICU admitted patients. The most common reason for admission was close monitoring of high-risk patients. The mortality rate during the studied period was 34.1%, and this was significantly associated with patient age and type of illness at presentation (p < 0.05). Stratified by year of admission, the highest rate of mortality was noted in the year 2020 (46.7%).

Conclusion

There is a high level of mortality among ICU admissions in our center. This calls for the improvement of intensive care delivery in the healthcare facility, including training and retraining of manpower and provision of essential facilities for high-quality healthcare delivery.

Detecting Language Associated With Home Healthcare Patient's Risk for Hospitalization and Emergency Department Visit

BACKGROUND

About one in five patients receiving home healthcare (HHC) services are hospitalized or visit an emergency department (ED) during a home care episode. Early identification of at-risk patients can prevent these negative outcomes. However, risk indicators, including language in clinical notes that indicate a concern about a patient, are often hidden in narrative documentation throughout their HHC episode.

OBJECTIVE

The aim of the study was to develop an automated natural language processing (NLP) algorithm to identify concerning language indicative of HHC patients' risk of hospitalizations or ED visits.

METHODS

This study used the Omaha System-a standardized nursing terminology that describes problems/signs/symptoms that can occur in the community setting. First, five HHC experts iteratively reviewed the Omaha System and identified concerning concepts indicative of HHC patients' risk of hospitalizations or ED visits. Next, we developed and tested an NLP algorithm to identify these concerning concepts in HHC clinical notes automatically. The resulting NLP algorithm was applied on a large subset of narrative notes (2.3 million notes) documented for 66,317 unique patients ( n = 87,966 HHC episodes) admitted to one large HHC agency in the Northeast United States between 2015 and 2017.

RESULTS

A total of 160 Omaha System signs/symptoms were identified as concerning concepts for hospitalizations or ED visits in HHC. These signs/symptoms belong to 31 of the 42 available Omaha System problems. Overall, the NLP algorithm showed good performance in identifying concerning concepts in clinical notes. More than 18% of clinical notes were detected as having at least one concerning concept, and more than 90% of HHC episodes included at least one Omaha System problem. The most frequently documented concerning concepts were pain, followed by issues related to neuromusculoskeletal function, circulation, mental health, and communicable/infectious conditions.

CONCLUSION

Our findings suggest that concerning problems or symptoms that could increase the risk of hospitalization or ED visit were frequently documented in narrative clinical notes. NLP can automatically extract information from narrative clinical notes to improve our understanding of care needs in HHC. Next steps are to evaluate which concerning concepts identified in clinical notes predict hospitalization or ED visit.

Factors Influencing Clinician Trust in Predictive Clinical Decision Support Systems for In-Hospital Deterioration: Qualitative Descriptive Study

BACKGROUND

Clinician trust in machine learning-based clinical decision support systems (CDSSs) for predicting in-hospital deterioration (a type of predictive CDSS) is essential for adoption. Evidence shows that clinician trust in predictive CDSSs is influenced by perceived understandability and perceived accuracy.

OBJECTIVE

The aim of this study was to explore the phenomenon of clinician trust in predictive CDSSs for in-hospital deterioration by confirming and characterizing factors known to influence trust (understandability and accuracy), uncovering and describing other influencing factors, and comparing nurses' and prescribing providers' trust in predictive CDSSs.

METHODS

We followed a qualitative descriptive methodology conducting directed deductive and inductive content analysis of interview data. Directed deductive analyses were guided by the human-computer trust conceptual framework. Semistructured interviews were conducted with nurses and prescribing providers (physicians, physician assistants, or nurse practitioners) working with a predictive CDSS at 2 hospitals in Mass General Brigham.

RESULTS

A total of 17 clinicians were interviewed. Concepts from the human-computer trust conceptual framework-perceived understandability and perceived technical competence (ie, perceived accuracy)-were found to influence clinician trust in predictive CDSSs for in-hospital deterioration. The concordance between clinicians' impressions of patients' clinical status and system predictions influenced clinicians' perceptions of system accuracy. Understandability was influenced by system explanations, both global and local, as well as training. In total, 3 additional themes emerged from the inductive analysis. The first, perceived actionability, captured the variation in clinicians' desires for predictive CDSSs to recommend a discrete action. The second, evidence, described the importance of both macro- (scientific) and micro- (anecdotal) evidence for fostering trust. The final theme, equitability, described fairness in system predictions. The findings were largely similar between nurses and prescribing providers.

CONCLUSIONS

Although there is a perceived trade-off between machine learning-based CDSS accuracy and understandability, our findings confirm that both are important for fostering clinician trust in predictive CDSSs for in-hospital deterioration. We found that reliance on the predictive CDSS in the clinical workflow may influence clinicians' requirements for trust. Future research should explore the impact of reliance, the optimal explanation design for enhancing understandability, and the role of perceived actionability in driving trust.

Clinical notes: An untapped opportunity for improving risk prediction for hospitalization and emergency department visit during home health care

BACKGROUND/OBJECTIVE

Between 10 and 25% patients are hospitalized or visit emergency department (ED) during home healthcare (HHC). Given that up to 40% of these negative clinical outcomes are preventable, early and accurate prediction of hospitalization risk can be one strategy to prevent them. In recent years, machine learning-based predictive modeling has become widely used for building risk models. This study aimed to compare the predictive performance of four risk models built with various data sources for hospitalization and ED visits in HHC.

METHODS

Four risk models were built using different variables from two data sources: structured data (i.e., Outcome and Assessment Information Set (OASIS) and other assessment items from the electronic health record (EHR)) and unstructured narrative-free text clinical notes for patients who received HHC services from the largest non-profit HHC organization in New York between 2015 and 2017. Then, five machine learning algorithms (logistic regression, Random Forest, Bayesian network, support vector machine (SVM), and Naïve Bayes) were used on each risk model. Risk model performance was evaluated using the F-score and Precision-Recall Curve (PRC) area metrics.

RESULTS

During the study period, 8373/86,823 (9.6%) HHC episodes resulted in hospitalization or ED visits. Among five machine learning algorithms on each model, the SVM showed the highest F-score (0.82), while the Random Forest showed the highest PRC area (0.864). Adding information extracted from clinical notes significantly improved the risk prediction ability by up to 16.6% in F-score and 17.8% in PRC.

CONCLUSION

All models showed relatively good hospitalization or ED visit risk predictive performance in HHC. Information from clinical notes integrated with the structured data improved the ability to identify patients at risk for these emergent care events.

Usability of novel major TraumaApp for digital data collection

Background

Delivery of major trauma care is complex and often fast paced. Clear and comprehensive documentation is paramount to support effective communication during complex clinical care episodes, and to allow collection of data for audit, research and continuous improvement. Clinical events are typically recorded on paper-based records that are developed for individual centres or systems. As one of the priorities laid out by the Scottish Trauma Network project was to develop an electronic data collection system, the TraumaApp was created as a data collection tool for major trauma that could be adopted worldwide.

Methods

The study was performed as a service evaluation based at the Queen Elizabeth University Hospital Emergency Department. Fifty staff members were recruited in pairs and listened to five paired major trauma standby and handover recordings. Participants were randomised to input data to the TraumaApp and one into the existing paper proforma. The time taken to input data add into was measured, along with time for clarifications and any errors made. Those using the app completed a System Usability Score.

Results

No statistically significant difference was demonstrated between times taken for data entry for the digital and paper documentation, apart from the Case 5 Handover (p < 0.05). Case 1 showed a significantly higher time for clarifications and number of errors with digital data collection (p = 0.01 and p = 1.79E-05 respectively). There were no other differences between data for the app and the proforma. The mean System Usability score for this cohort was 75 out of 100, with a standard deviation of 17 (rounded to nearest integer).

Conclusion

Digital real-time recording of clinical events using a tool such as the TraumaApp is comparable to completion of paper proforma. The System Usability Score for the TraumaApp was above the internationally validated standard of acceptable usability. There was no evidence of improvement in use over time or familiarity, most likely due to the brevity of the assessments and the refined user interface. This would benefit from further research, exploring data completeness and a potential mixed methods approach to explore training requirements for use of the TraumaApp.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12873-022-00578-9.

Understanding Clinician Macrocognition to Inform the Design of a Congenital Heart Disease Clinical Decision Support System

Background and Objectives

Children with congenital heart disease (CHD) are at risk of deterioration in the face of common childhood illnesses, and their resuscitation and acute treatment requires guidance of CHD experts. Many children with CHD, however, present to their local emergency departments (ED) with gastrointestinal and respiratory symptoms that closely mimic symptoms of CHD related heart failure. This can lead to incorrect or delayed diagnosis and treatment where CHD expertise is limited. An understanding of the differences in cognitive decision-making processes between CHD experts and ED physicians can inform how best to support ED physicians when treating CHD patients.

Methods

Cardiac intensivists (CHD experts) and pediatric emergency department physicians (ED physicians) in a major academic cardiac center were interviewed using the critical decision method. Interview transcripts were coded deductively based on Schubert and Klein's macrocognitive frameworks and inductively to allow for new or modified characterization of dimensions.

Results

In total, 6 CHD experts and 7 ED physicians were interviewed for this study. Although both CHD experts and ED physicians spent a lot of time sensemaking, their approaches to sensemaking differed. CHD experts reported readily recognizing the physiology of complex congenital heart disease and focused primarily on ruling out cardiac causes for the presenting illness. ED physicians reported a delay in attributing the signs and symptoms of the presenting illness to congenital heart disease, because these clinical findings were often non-specific, and thus explored different diagnoses. CHD experts moved quickly to treatment and more time anticipating potential problems and making specific contingency plans, while ED physicians spent more time gathering a range of data prior to arriving at a diagnosis. These findings were then applied to develop a prototype web-based decision support application for patients with CHD.

Conclusion

There are differences in the cognitive processes used by CHD experts and ED physicians when managing CHD patients. An understanding of differences in the cognitive processes used by CHD experts and ED physicians can inform the development of potential interventions, such as clinical decision support systems and training pathways, to support decision making pertaining to the acute treatment of pediatric CHD patients.

Effect of clinical decision support systems on emergency medicine physicians' decision-making: A pilot scenario-based simulation study

BACKGROUND AND OBJECTIVES

Children with congenital heart disease (CHD) are predisposed to rapid deterioration in the face of common childhood illnesses. When they present to their local emergency departments (ED) with acute illness, rapid and accurate diagnosis and treatment is crucial to recovery and survival. Previous studies have shown that ED physicians are uncomfortable caring for patients with CHD and there is a lack of actionable guidance to aid in their decision making. To support ED physicians' key decision components (sensemaking, anticipation, and managing complexity) when managing CHD patients, a Clinical Decision Support System (CDSS) was previously designed. This pilot study evaluates the effect of this CDSS on ED physicians' decision making compared to usual care without clinical decision support.

METHODS

In a pilot scenario-based simulation study with repeated measures, ED physicians managed mock CHD patients with and without the CDSS. We compared ED physicians' CHD-specific and general decision-making processes (e.g., recognizing sepsis, starting antibiotics, and managing symptoms) with and without the use of CDSS. The frequency of participants' utterances related to each key decision components of sensemaking, anticipation, and managing complexity were coded and statistically analyzed for significance.

RESULTS

Across all decision-making components, the CDSS significantly increased ED physicians' frequency of "CHD specific utterances" (Mean = 5.43, 95%CI: 3.7-7.2) compared to the without CDSS condition (Mean = 2.05, 95%CI: 0.3-3.8) whereas there was no significant difference in frequencies of "general utterances" when using CDSS (Mean = 4.62, 95%CI: 3.1-6.1) compared to without CDSS (Mean = 5.14 95%CI: 4.4-5.9).

CONCLUSION

A CDSS that integrates key decision-making components (sensemaking, anticipation, and managing complexity) can trigger and enrich communication between clinicians and enhance the clinical management of CHD patients. For patients with complex and subspecialized diseases such as CHD, a well-designed CDSS can become part of a multifaceted solution that includes knowledge translation, broader communication around interpretation of information, and access to additional expertise to support CHD specific decision-making.

Evaluating Completeness of Discrete Data on Physical Functioning for Children With Cerebral Palsy in a Pediatric Rehabilitation Learning Health System

OBJECTIVE

The purpose of this study was to determine the extent that physical function discrete data elements (DDE) documented in electronic health records (EHR) are complete within pediatric rehabilitation settings.

METHODS

A descriptive analysis on completeness of EHR-based DDEs detailing physical functioning for children with cerebral palsy was conducted. Data from an existing pediatric rehabilitation research learning health system data network, consisting of EHR data from 20 care sites in a pediatric specialty health care system, were leveraged. Completeness was calculated for unique data elements, unique outpatient visits, and unique outpatient records.

RESULTS

Completeness of physical function DDEs was low across 5766 outpatient records (10.5%, approximately 2 DDEs documented). The DDE for Gross Motor Function Classification System level was available for 21% (n = 3746) outpatient visits and 38% of patient records. Ambulation level was the most frequently documented DDE. Intercept only mixed effects models demonstrated that 21.4% and 45% of the variance in completeness for DDEs and the Gross Motor Function Classification System, respectively, across unique patient records could be attributed to factors at the individual care site level.

CONCLUSION

Values of physical function DDEs are missing in designated fields of the EHR infrastructure for pediatric rehabilitation providers. Although completeness appears limited for these DDEs, our observations indicate that data are not missing at random and may be influenced by system-level standards in clinical documentation practices between providers and factors specific to individual care sites. The extent of missing data has significant implications for pediatric rehabilitation quality measurement. More research is needed to understand why discrete data are missing in EHRs and to further elucidate the professional and system-level factors that influence completeness and missingness.

IMPACT

Completeness of DDEs reported in this study is limited and presents a significant opportunity to improve documentation and standards to optimize EHR data for learning health system research and quality measurement in pediatric rehabilitation settings.

Artificial Intelligence in Predicting Cardiac Arrest: Scoping Review

BACKGROUND

Cardiac arrest is a life-threatening cessation of activity in the heart. Early prediction of cardiac arrest is important, as it allows for the necessary measures to be taken to prevent or intervene during the onset. Artificial intelligence (AI) technologies and big data have been increasingly used to enhance the ability to predict and prepare for the patients at risk.

OBJECTIVE

This study aims to explore the use of AI technology in predicting cardiac arrest as reported in the literature.

METHODS

A scoping review was conducted in line with the guidelines of the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) extension for scoping reviews. Scopus, ScienceDirect, Embase, the Institute of Electrical and Electronics Engineers, and Google Scholar were searched to identify relevant studies. Backward reference list checks of the included studies were also conducted. Study selection and data extraction were independently conducted by 2 reviewers. Data extracted from the included studies were synthesized narratively.

RESULTS

Out of 697 citations retrieved, 41 studies were included in the review, and 6 were added after backward citation checking. The included studies reported the use of AI in the prediction of cardiac arrest. Of the 47 studies, we were able to classify the approaches taken by the studies into 3 different categories: 26 (55%) studies predicted cardiac arrest by analyzing specific parameters or variables of the patients, whereas 16 (34%) studies developed an AI-based warning system. The remaining 11% (5/47) of studies focused on distinguishing patients at high risk of cardiac arrest from patients who were not at risk. Two studies focused on the pediatric population, and the rest focused on adults (45/47, 96%). Most of the studies used data sets with a size of <10,000 samples (32/47, 68%). Machine learning models were the most prominent branch of AI used in the prediction of cardiac arrest in the studies (38/47, 81%), and the most used algorithm was the neural network (23/47, 49%). K-fold cross-validation was the most used algorithm evaluation tool reported in the studies (24/47, 51%).

CONCLUSIONS

AI is extensively used to predict cardiac arrest in different patient settings. Technology is expected to play an integral role in improving cardiac medicine. There is a need for more reviews to learn the obstacles to the implementation of AI technologies in clinical settings. Moreover, research focusing on how to best provide clinicians with support to understand, adapt, and implement this technology in their practice is also necessary.

The Communicating Narrative Concerns Entered by Registered Nurses (CONCERN) Clinical Decision Support Early Warning System: Protocol for a Cluster Randomized Pragmatic Clinical Trial

Background

Every year, hundreds of thousands of inpatients die from cardiac arrest and sepsis, which could be avoided if those patients’ risk for deterioration were detected and timely interventions were initiated. Thus, a system is needed to convert real-time, raw patient data into consumable information that clinicians can utilize to identify patients at risk of deterioration and thus prevent mortality and improve patient health outcomes. The overarching goal of the COmmunicating Narrative Concerns Entered by Registered Nurses (CONCERN) study is to implement and evaluate an early warning score system that provides clinical decision support (CDS) in electronic health record systems. With a combination of machine learning and natural language processing, the CONCERN CDS utilizes nursing documentation patterns as indicators of nurses’ increased surveillance to predict when patients are at the risk of clinical deterioration.

Objective

The objective of this cluster randomized pragmatic clinical trial is to evaluate the effectiveness and usability of the CONCERN CDS system at 2 different study sites. The specific aim is to decrease hospitalized patients’ negative health outcomes (in-hospital mortality, length of stay, cardiac arrest, unanticipated intensive care unit transfers, and 30-day hospital readmission rates).

Methods

A multiple time-series intervention consisting of 3 phases will be performed through a 1-year period during the cluster randomized pragmatic clinical trial. Phase 1 evaluates the adoption of our algorithm through pilot and trial testing, phase 2 activates optimized versions of the CONCERN CDS based on experience from phase 1, and phase 3 will be a silent release mode where no CDS is viewable to the end user. The intervention deals with a series of processes from system release to evaluation. The system release includes CONCERN CDS implementation and user training. Then, a mixed methods approach will be used with end users to assess the system and clinician perspectives.

Results

Data collection and analysis are expected to conclude by August 2022. Based on our previous work on CONCERN, we expect the system to have a positive impact on the mortality rate and length of stay.

Conclusions

The CONCERN CDS will increase team-based situational awareness and shared understanding of patients predicted to be at risk for clinical deterioration in need of intervention to prevent mortality and associated harm.

Trial Registration

ClinicalTrials.gov NCT03911687; https://clinicaltrials.gov/ct2/show/NCT03911687

International Registered Report Identifier (IRRID)

DERR1-10.2196/30238

Ethical dilemmas in nursing documentation

BACKGROUND

Nursing documentation is an essential aspect of ethical nursing care. Lack of awareness of ethical dilemmas in nursing documentation may increase the risk of patient harm. Considering this, ethical dilemmas within nursing documentation need to be explored.

AIM

To explore ethical dilemmas in nurses' conversations about nursing documentation.

RESEARCH DESIGN, PARTICIPANTS AND CONTEXT

The study used a qualitative design. Participants were registered nurses from a Patient Hotel at a Danish University Hospital. Data were collected in three focus groups with a total of 12 participants. Data analysis consisted of qualitative content analysis inspired by Graneheim and Lundman.

ETHICAL CONSIDERATION

This study was conducted in accordance with the ethical principles of research and regulations in terms of confidentiality, anonymity and provision of informed consent.

FINDINGS

Ethical dilemmas were strongly present in nurses' conversations about nursing documentation. These dilemmas were demonstrated in two themes: (1) a dilemma between respecting patients' autonomy and not causing harm, which was visible in nurses' navigation between written documentation and oral tradition, and (2) a dilemma concerning justice and fair distribution of goods, which was visible in nurses' balancing between documenting deviations and proof of nursing practice.

DISCUSSION

Ethical dilemmas in nursing documentation regarding respecting patients' autonomy and not causing harm accentuated discussions on professional responsibility and patient participation in clinical decisions. Dilemmas in justice and fair distribution of goods emphasised discussions on trust in relationships versus trust in electronic health records.

CONCLUSION

Actual tendencies in the healthcare system may increase ethical dilemmas in nursing documentation. Sharing otherwise invisible and individual experiences of ethical dilemmas in nursing documentation among nurses, nurse leaders and decision-makers will enable addressing these in reflections and discussions as well as in considering adjustments of conditions for nursing documentation.

Increasing accuracy in documentation through the application of a structured emergency nursing framework: A multisite quasi-experimental study

AIMS AND OBJECTIVES

To determine if the use of an emergency nursing framework improves the accuracy of clinical documentation.

BACKGROUND

Accurate clinical documentation is a nursing professional responsibility essential for high-quality and safe patient care. The use of the emergency nursing framework "HIRAID" (History, Identify Red flags, Assessment, Interventions, Diagnostics, reassessment and communication) improves emergency nursing care by reducing treatment delays and improving escalation of clinical deterioration. The effect of HIRAID on the accuracy of nursing documentation is unknown.

DESIGN

A quasi-experimental pre-post study was conducted and the report was guided by the strengthening the reporting of observational studies in epidemiology (STROBE) checklist.

METHODS

HIRAID was implemented in four regional/rural Australian emergency departments (ED) using a range of behaviour change strategies. The blinded electronic healthcare records of 120 patients with a presenting problem of shortness of breath, abdominal pain or fever were reviewed. Quantity measures of completeness and qualitative measures of completeness and linguistic correctness of documentation adapted from the D-Catch tool were used to assess accuracy. Differences between pre-post groups were analysed using Wilcoxon rank-sum and two-sample t-tests for continuous variables. Pearson's Chi-square and Fisher exact tests were used for the categorical data.

RESULTS

The number of records containing the essential assessment components of emergency care increased significantly from pre- to post-implementation of HIRAID. This overall improvement was demonstrated in both paediatric and adult populations and for all presentation types. Both the quantitative and qualitative measures of documentation on patient history and physical assessment findings improved significantly.

CONCLUSION

Use of HIRAID improves the accuracy of clinical documentation of the patient history and physical assessment in both adult and paediatric populations.

RELEVANCE TO CLINICAL PRACTICE

The emergency nursing framework "HIRAID" is recommended for use in clinical practice to increase the documentation accuracy performed by emergency nurses.

Sentiment Analysis for Necessary Preview of 30-Day Mortality in Sepsis Patients and the Control Strategies

This study was to preview the risk of 30-day mortality in sepsis patients using sentiment analysis. The clinical data of patients and nursing notes were collected from the Medical Information Mart for Intensive Care (MIMIC-III) database. The factors influencing 30-day mortality were analyzed using the Cox regression model. And, the prognostic index (PI) was estimated. The receiver operating characteristic (ROC) curve was used to determine the PI cut-off point and assess the prediction ability of the model. In total, 1844 of 3560 patients were eligible for the study, with a 30-day mortality of 37.58%. Multivariate Cox analysis showed that sentiment polarity scores, sentiment subjectivity scores, simplified acute physiology score (SAPS)-II, age, and intensive care unit (ICU) types were all associated with the risk of 30-day mortality (P < 0.05). In the preview of 30-day mortality, the area under the curve (AUC) of ROC was 0.78 (95%CI: 0.74–0.81,P < 0.001) when the cut-off point of PI was 0.467. The documented notes from nurses were described for the first time. Sentiment scores measured in nursing notes are associated with the risk of 30-day mortality in sepsis patients and may improve the preview of 30-day mortality.

A Review of Mortality Risk Prediction Models in Smartphone Applications

Healthcare professionals in healthcare systems need access to freely available, real-time, evidence-based mortality risk prediction smartphone applications to facilitate resource allocation. The objective of this study is to evaluate the quality of smartphone mobile health applications that include mortality prediction models, and corresponding information quality. We conducted a systematic review of commercially available smartphone applications in Google Play for Android, and iTunes for iOS smartphone applications. We performed initial screening, data extraction, and rated smartphone application quality using the Mobile Application Rating Scale: user version (uMARS). The information quality of smartphone applications was evaluated using two patient vignettes, representing low and high risk of mortality, based on critical care data from the Medical Information Mart for Intensive Care (MIMIC) III database. Out of 3051 evaluated smartphone applications, 33 met our final inclusion criteria. We identified 21 discrete mortality risk prediction models in smartphone applications. The most common mortality predicting models were Sequential Organ Failure Assessment (SOFA) (n = 15) and Acute Physiology and Clinical Health Assessment II (n = 13). The smartphone applications with the highest quality uMARS scores were Observation-NEWS 2 (4.64) for iOS smartphones, and MDCalc Medical Calculator (4.75) for Android smartphones. All SOFA-based smartphone applications provided consistent information quality with the original SOFA model for both the low and high-risk patient vignettes. We identified freely available, high-quality mortality risk prediction smartphone applications that can be used by healthcare professionals to make evidence-based decisions in critical care environments.

Systematic review of current natural language processing methods and applications in cardiology

Natural language processing (NLP) is a set of automated methods to organise and evaluate the information contained in unstructured clinical notes, which are a rich source of real-world data from clinical care that may be used to improve outcomes and understanding of disease in cardiology. The purpose of this systematic review is to provide an understanding of NLP, review how it has been used to date within cardiology and illustrate the opportunities that this approach provides for both research and clinical care. We systematically searched six scholarly databases (ACM Digital Library, Arxiv, Embase, IEEE Explore, PubMed and Scopus) for studies published in 2015-2020 describing the development or application of NLP methods for clinical text focused on cardiac disease. Studies not published in English, lacking a description of NLP methods, non-cardiac focused and duplicates were excluded. Two independent reviewers extracted general study information, clinical details and NLP details and appraised quality using a checklist of quality indicators for NLP studies. We identified 37 studies developing and applying NLP in heart failure, imaging, coronary artery disease, electrophysiology, general cardiology and valvular heart disease. Most studies used NLP to identify patients with a specific diagnosis and extract disease severity using rule-based NLP methods. Some used NLP algorithms to predict clinical outcomes. A major limitation is the inability to aggregate findings across studies due to vastly different NLP methods, evaluation and reporting. This review reveals numerous opportunities for future NLP work in cardiology with more diverse patient samples, cardiac diseases, datasets, methods and applications.

Sentiment Analysis Based on the Nursing Notes on In-Hospital 28-Day Mortality of Sepsis Patients Utilizing the MIMIC-III Database

In medical visualization, nursing notes contain rich information about a patient's pathological condition. However, they are not widely used in the prediction of clinical outcomes. With advances in the processing of natural language, information begins to be extracted from large-scale unstructured data like nursing notes. This study extracted sentiment information in nursing notes and explored its association with in-hospital 28-day mortality in sepsis patients. The data of patients and nursing notes were extracted from the MIMIC-III database. A COX proportional hazard model was used to analyze the relationship between sentiment scores in nursing notes and in-hospital 28-day mortality. Based on the COX model, the individual prognostic index (PI) was calculated, and then, survival was analyzed. Among eligible 1851 sepsis patients, 580 cases suffered from in-hospital 28-day mortality (dead group), while 1271 survived (survived group). Significant differences were shown between two groups in sentiment polarity, Simplified Acute Physiology Score II (SAPS-II) score, age, and intensive care unit (ICU) type (all P < 0.001). Multivariate COX analysis exhibited that sentiment polarity (HR: 0.499, 95% CI: 0.409-0.610, P < 0.001) and sentiment subjectivity (HR: 0.710, 95% CI: 0.559-0.902, P = 0.005) were inversely associated with in-hospital 28-day mortality, while the SAPS-II score (HR: 1.034, 95% CI: 1.029-1.040, P < 0.001) was positively correlated with in-hospital 28-day mortality. The median death time of patients with PI ≥ 0.561 was significantly earlier than that of patients with PI < 0.561 (13.5 vs. 49.8 days, P < 0.001). In conclusion, sentiments in nursing notes are associated with the in-hospital 28-day mortality and survival of sepsis patients.

Subcategorizing EHR diagnosis codes to improve clinical application of machine learning models

BACKGROUND

Electronic health record (EHR) data is commonly used for secondary purposes such as research and clinical decision support. However, reuse of EHR data presents several challenges including but not limited to identifying all diagnoses associated with a patient's clinical encounter. The purpose of this study was to assess the feasibility of developing a schema to identify and subclassify all structured diagnosis codes for a patient encounter.

METHODS

To develop a subclassification schema we used EHR data from an interhospital transport data repository that contained complete hospital encounter level data. Eight discrete data sources containing structured diagnosis codes were identified. Diagnosis codes were normalized using the Unified Medical Language System and additional EHR data were combined with standardized terminologies to create and validate the subcategories. We then employed random forest to assess the usefulness of the new subcategorized diagnoses to predict post-interhospital transfer mortality by building 2 models, one using standard diagnosis codes, and one using the new subcategorized diagnosis codes.

RESULTS

Six subcategories of diagnoses were identified and validated. The subcategories included: primary or admitting diagnoses (10%), past medical, surgical or social history (9%), problem list (20%), comorbidity (24%), discharge diagnoses (6%), and unmapped diagnoses (31%). The subcategorized model outperformed the standard model, achieving a training AUROC of 0.97 versus 0.95 and testing model AUROC of 0.81 versus 0.46.

DISCUSSION

Our work demonstrates that merging structured diagnosis codes with additional EHR data and secondary data sources provides additional information to understand the role of diagnosis throughout a clinical encounter and improves predictive model performance. Further work is necessary to assess if subcategorizing produces benefits in interpreting the results of prognostic models and/or operationalizing the results in clinical decision support applications.

Orthopaedic nurses' experiences with real-time documentation in a high-tech ward: A qualitative study

BACKGROUND

Real-time documentation is a novel process that changes nursing workflow; however, nurses' experiences of real-time documentation are unknown.

AIM

This study aimed to explore nurses' experiences with real-time documentation in an orthopaedic ward.

DESIGN

This qualitative study took a phenomenological-hermeneutic approach.

METHODS

Data were generated from three semi-structured focus group interviews with 18 nurses from an orthopaedic ward. Data analysis was based on Ricoeur's theory of narrative and interpretation and included naïve reading, structural analysis and critical interpretation, and discussion.

RESULTS

Five themes emerged from the structural analysis: 1) nurses were initially sceptical and outside of their comfort zone; 2) implementation required support from the head nurse and other colleagues; 3) increased time with patients led to better relationships, but nurses lacked time for reflection; 4) increased patient involvement could also present challenges; and 5) documentation became more integrated into orthopaedic nursing.

CONCLUSION

Real-time documentation improved orthopaedic nursing documentation and increased patient involvement. Nurses spent more time with patients, leading to better relationships, but they had decreased time with their colleagues and the opportunity to reflect. Real-time documentation leads to changes in workflow, so, nurses should be provided with training and the opportunity to reflect.

Utilizing timestamps of longitudinal electronic health record data to classify clinical deterioration events

OBJECTIVE

To propose an algorithm that utilizes only timestamps of longitudinal electronic health record data to classify clinical deterioration events.

MATERIALS AND METHODS

This retrospective study explores the efficacy of machine learning algorithms in classifying clinical deterioration events among patients in intensive care units using sequences of timestamps of vital sign measurements, flowsheets comments, order entries, and nursing notes. We design a data pipeline to partition events into discrete, regular time bins that we refer to as timesteps. Logistic regressions, random forest classifiers, and recurrent neural networks are trained on datasets of different length of timesteps, respectively, against a composite outcome of death, cardiac arrest, and Rapid Response Team calls. Then these models are validated on a holdout dataset.

RESULTS

A total of 6720 intensive care unit encounters meet the criteria and the final dataset includes 830 578 timestamps. The gated recurrent unit model utilizes timestamps of vital signs, order entries, flowsheet comments, and nursing notes to achieve the best performance on the time-to-outcome dataset, with an area under the precision-recall curve of 0.101 (0.06, 0.137), a sensitivity of 0.443, and a positive predictive value of 0. 092 at the threshold of 0.6.

DISCUSSION AND CONCLUSION

This study demonstrates that our recurrent neural network models using only timestamps of longitudinal electronic health record data that reflect healthcare processes achieve well-performing discriminative power.

Healthcare Process Modeling to Phenotype Clinician Behaviors for Exploiting the Signal Gain of Clinical Expertise (HPM-ExpertSignals): Development and evaluation of a conceptual framework

OBJECTIVE

There are signals of clinicians' expert and knowledge-driven behaviors within clinical information systems (CIS) that can be exploited to support clinical prediction. Describe development of the Healthcare Process Modeling Framework to Phenotype Clinician Behaviors for Exploiting the Signal Gain of Clinical Expertise (HPM-ExpertSignals).

MATERIALS AND METHODS

We employed an iterative framework development approach that combined data-driven modeling and simulation testing to define and refine a process for phenotyping clinician behaviors. Our framework was developed and evaluated based on the Communicating Narrative Concerns Entered by Registered Nurses (CONCERN) predictive model to detect and leverage signals of clinician expertise for prediction of patient trajectories.

RESULTS

Seven themes-identified during development and simulation testing of the CONCERN model-informed framework development. The HPM-ExpertSignals conceptual framework includes a 3-step modeling technique: (1) identify patterns of clinical behaviors from user interaction with CIS; (2) interpret patterns as proxies of an individual's decisions, knowledge, and expertise; and (3) use patterns in predictive models for associations with outcomes. The CONCERN model differentiated at risk patients earlier than other early warning scores, lending confidence to the HPM-ExpertSignals framework.

DISCUSSION

The HPM-ExpertSignals framework moves beyond transactional data analytics to model clinical knowledge, decision making, and CIS interactions, which can support predictive modeling with a focus on the rapid and frequent patient surveillance cycle.

CONCLUSIONS

We propose this framework as an approach to embed clinicians' knowledge-driven behaviors in predictions and inferences to facilitate capture of healthcare processes that are activated independently, and sometimes well before, physiological changes are apparent.

Exploring prevalence of wound infections and related patient characteristics in homecare using natural language processing

We aimed to create and validate a natural language processing algorithm to extract wound infection-related information from nursing notes. We also estimated wound infection prevalence in homecare settings and described related patient characteristics. In this retrospective cohort study, a natural language processing algorithm was developed and validated against a gold standard testing set. Cases with wound infection were identified using the algorithm and linked to Outcome and Assessment Information Set data to identify related patient characteristics. The final version of the natural language processing vocabulary contained 3914 terms and expressions related to the presence of wound infection. The natural language processing algorithm achieved overall good performance (F-measure = 0.88). The presence of wound infection was documented for 1.03% (n = 602) of patients without wounds, for 5.95% (n = 3232) of patients with wounds, and 19.19% (n = 152) of patients with wound-related hospitalisation or emergency department visits. Diabetes, peripheral vascular disease, and skin ulcer were significantly associated with wound infection among homecare patients. Our findings suggest that nurses frequently document wound infection-related information. The use of natural language processing demonstrated that valuable information can be extracted from nursing notes which can be used to improve our understanding of the care needs of people receiving homecare. By linking findings from clinical nursing notes with additional structured data, we can analyse related patients' characteristics and use them to develop a tailored intervention that may potentially lead to reduced wound infection-related hospitalizations.

Identifying nursing documentation patterns associated with patient deterioration and recovery from deterioration in critical and acute care settings

OBJECTIVES

Nursing documentation behavior within electronic health records may reflect a nurse's concern about a patient and can be used to predict patient deterioration. Our study objectives were to quantify variations in nursing documentation patterns, confirm those patterns and variations with clinicians, and identify which patterns indicate patient deterioration and recovery from clinical deterioration events in the critical and acute care settings.

METHODS

We collected patient data from electronic health records and conducted a regression analysis to identify different nursing documentation patterns associated with patient outcomes resulting from clinical deterioration events in the intensive care unit (ICU) and acute care unit (ACU). The primary outcome measures were whether patients were discharged alive from the hospital or expired during their hospital encounter. Secondary outcome measures were clinical deterioration events.

RESULTS

In the ICU, the increased documentation of heart rate, body temperature, and withheld medication administrations were significantly associated with inpatient mortality. In the ACU, the documentation of blood pressure, respiratory rate with comments, singular vital signs, and withheld medications were significantly related to inpatient mortality. In contrast, the documentation of heart rate and "as needed" medication administrations were significantly associated with patient survival to discharge in the ACU.

CONCLUSION

We successfully identified and confirmed the clinical relevancy of the nursing documentation patterns indicative of patient deterioration and recovery from clinical deterioration events in both the ICU and ACU.

Attitude Towards Documentation and Its Associated Factors Among Nurses Working in Public Hospitals of Hawassa City Administration, Southern Ethiopia

Background

Nursing documentation is the record of nursing care that has been planned and delivered to individual clients by qualified nurses or under the direction of qualified nurses. Various studies have shown that documentation is still a critical issue in both high- and low-income countries, especially in Sub-Saharan Africa like Ethiopia. However, there is a paucity of data in Ethiopia, the attitude of nurses towards nursing care documentation, particularly in the study setting. Therefore, this study aimed to assess the nurse's attitude towards documentation and associated factors in Hawassa City administration public hospitals, Southern Ethiopia.

Methods

Institutional based cross-sectional study was conducted among 422 nurses from March 01 to 30, 2020. A simple random sampling technique was applied to select the study participants. Data were collected using a self-administered questionnaire. Statistical package of social science (SPSS) version 20.0 software was used for analysis. The association between the attitude of nurses towards documentation and predictors was determined using multivariable logistic regression analysis. The level of statistical significance was determined at a p-value of less than 0.05.

Result

Among 413 nurses who participated in the study, 58.8% [95% CI of 54.5% to 63.7%] of them had a favorable attitude towards documentation. Work setting [AOR = 1.94 (95% CI: 1.23-3.05)] and Knowledge [AOR = 3.28 (95% CI: 2.08-5.16)], were significantly associated factors with nurses' attitude towards documentation.Conclusion and Recommendations: More than half of the study participants had a favorable attitude towards documentation. Working unit and knowledge were factors associated with nurse's attitude toward nursing care documentation. Therefore, increasing nurse's knowledge about documentation and managing working units effectively are recommended to increase the nurses' attitude toward documentation.

Artificial intelligence in nursing: Priorities and opportunities from an international invitational think-tank of the Nursing and Artificial Intelligence Leadership Collaborative

Aim

To develop a consensus paper on the central points of an international invitational think‐tank on nursing and artificial intelligence (AI).

Methods

We established the Nursing and Artificial Intelligence Leadership (NAIL) Collaborative, comprising interdisciplinary experts in AI development, biomedical ethics, AI in primary care, AI legal aspects, philosophy of AI in health, nursing practice, implementation science, leaders in health informatics practice and international health informatics groups, a representative of patients and the public, and the Chair of the ITU/WHO Focus Group on Artificial Intelligence for Health. The NAIL Collaborative convened at a 3‐day invitational think tank in autumn 2019. Activities included a pre‐event survey, expert presentations and working sessions to identify priority areas for action, opportunities and recommendations to address these. In this paper, we summarize the key discussion points and notes from the aforementioned activities.

Implications for nursing

Nursing's limited current engagement with discourses on AI and health posts a risk that the profession is not part of the conversations that have potentially significant impacts on nursing practice.

Conclusion

There are numerous gaps and a timely need for the nursing profession to be among the leaders and drivers of conversations around AI in health systems.

Impact

We outline crucial gaps where focused effort is required for nursing to take a leadership role in shaping AI use in health systems. Three priorities were identified that need to be addressed in the near future: (a) Nurses must understand the relationship between the data they collect and AI technologies they use; (b) Nurses need to be meaningfully involved in all stages of AI: from development to implementation; and (c) There is a substantial untapped and an unexplored potential for nursing to contribute to the development of AI technologies for global health and humanitarian efforts.

Real-Time Electronic Health Record Mortality Prediction During the COVID-19 Pandemic: A Prospective Cohort Study

Objective

To rapidly develop, validate, and implement a novel real-time mortality score for the COVID-19 pandemic that improves upon sequential organ failure assessment (SOFA) for decision support for a Crisis Standards of Care team.

Materials and Methods

We developed, verified, and deployed a stacked generalization model to predict mortality using data available in the electronic health record (EHR) by combining 5 previously validated scores and additional novel variables reported to be associated with COVID-19-specific mortality. We verified the model with prospectively collected data from 12 hospitals in Colorado between March 2020 and July 2020. We compared the area under the receiver operator curve (AUROC) for the new model to the SOFA score and the Charlson Comorbidity Index.

Results

The prospective cohort included 27 296 encounters, of which 1358 (5.0%) were positive for SARS-CoV-2, 4494 (16.5%) required intensive care unit care, 1480 (5.4%) required mechanical ventilation, and 717 (2.6%) ended in death. The Charlson Comorbidity Index and SOFA scores predicted mortality with an AUROC of 0.72 and 0.90, respectively. Our novel score predicted mortality with AUROC 0.94. In the subset of patients with COVID-19, the stacked model predicted mortality with AUROC 0.90, whereas SOFA had AUROC of 0.85.

Discussion

Stacked regression allows a flexible, updatable, live-implementable, ethically defensible predictive analytics tool for decision support that begins with validated models and includes only novel information that improves prediction.

Conclusion

We developed and validated an accurate in-hospital mortality prediction score in a live EHR for automatic and continuous calculation using a novel model that improved upon SOFA.

Interdisciplinary Optimization of Admission Documentation: Reducing the Bloat

Moving toward the electronic health record increases the quality of information gathered. However, nurses argue that the electronic health record is an added burden. The aim of this study was to evaluate the removal of duplicative or unnecessary fields and reordering fields on the admission form to increase documentation that is meaningful to the patient story. A team of approximately 60 interdisciplinary clinicians engaged in document review to evaluate the importance of each field and removal or modification based on those findings. After a review of the 251 fields, the authors reduced the form to 124 fields, and the percentage of unfields by 31%. After outlier removal, the average time to complete the admission form decreased by 2.88 minutes. The new form showed a reduction of 36.71% of the use of the free text advance directive. Additionally, nurses' perceptions of the form significantly improved from pretest to posttest in terms of satisfaction with the form, time to complete, usability and usefulness, question flow, and length of the form. This study shows that an interdisciplinary team can effectively work together to optimize the Adult Admission History Form, increasing the quality of documentation while reducing the time to complete.

Using nursing notes to improve clinical outcome prediction in intensive care patients: A retrospective cohort study

OBJECTIVE

Electronic health record documentation by intensive care unit (ICU) clinicians may predict patient outcomes. However, it is unclear whether physician and nursing notes differ in their ability to predict short-term ICU prognosis. We aimed to investigate and compare the ability of physician and nursing notes, written in the first 48 hours of admission, to predict ICU length of stay and mortality using 3 analytical methods.

MATERIALS AND METHODS

This was a retrospective cohort study with split sampling for model training and testing. We included patients ≥18 years of age admitted to the ICU at Beth Israel Deaconess Medical Center in Boston, Massachusetts, from 2008 to 2012. Physician or nursing notes generated within the first 48 hours of admission were used with standard machine learning methods to predict outcomes.

RESULTS

For the primary outcome of composite score of ICU length of stay ≥7 days or in-hospital mortality, the gradient boosting model had better performance than the logistic regression and random forest models. Nursing and physician notes achieved area under the curves (AUCs) of 0.826 and 0.796, respectively, with even better predictive power when combined (AUC, 0.839).

DISCUSSION

Models using only nursing notes more accurately predicted short-term prognosis than did models using only physician notes, but in combination, the models achieved the greatest accuracy in prediction.

CONCLUSIONS

Our findings demonstrate that statistical models derived from text analysis in the first 48 hours of ICU admission can predict patient outcomes. Physicians' and nurses' notes are both uniquely important in mortality prediction and combining these notes can produce a better predictive model.

Physician understanding, explainability, and trust in a hypothetical machine learning risk calculator

OBJECTIVE

Implementation of machine learning (ML) may be limited by patients' right to "meaningful information about the logic involved" when ML influences healthcare decisions. Given the complexity of healthcare decisions, it is likely that ML outputs will need to be understood and trusted by physicians, and then explained to patients. We therefore investigated the association between physician understanding of ML outputs, their ability to explain these to patients, and their willingness to trust the ML outputs, using various ML explainability methods.

MATERIALS AND METHODS

We designed a survey for physicians with a diagnostic dilemma that could be resolved by an ML risk calculator. Physicians were asked to rate their understanding, explainability, and trust in response to 3 different ML outputs. One ML output had no explanation of its logic (the control) and 2 ML outputs used different model-agnostic explainability methods. The relationships among understanding, explainability, and trust were assessed using Cochran-Mantel-Haenszel tests of association.

RESULTS

The survey was sent to 1315 physicians, and 170 (13%) provided completed surveys. There were significant associations between physician understanding and explainability (P < .001), between physician understanding and trust (P < .001), and between explainability and trust (P < .001). ML outputs that used model-agnostic explainability methods were preferred by 88% of physicians when compared with the control condition; however, no particular ML explainability method had a greater influence on intended physician behavior.

CONCLUSIONS

Physician understanding, explainability, and trust in ML risk calculators are related. Physicians preferred ML outputs accompanied by model-agnostic explanations but the explainability method did not alter intended physician behavior.

Contribution of Free-Text Comments to the Burden of Documentation: Assessment and Analysis of Vital Sign Comments in Flowsheets

Background

Documentation burden is a common problem with modern electronic health record (EHR) systems. To reduce this burden, various recording methods (eg, voice recorders or motion sensors) have been proposed. However, these solutions are in an early prototype phase and are unlikely to transition into practice in the near future. A more pragmatic alternative is to directly modify the implementation of the existing functionalities of an EHR system.

Objective

This study aims to assess the nature of free-text comments entered into EHR flowsheets that supplement quantitative vital sign values and examine opportunities to simplify functionality and reduce documentation burden.

Methods

We evaluated 209,055 vital sign comments in flowsheets that were generated in the Epic EHR system at the Vanderbilt University Medical Center in 2018. We applied topic modeling, as well as the natural language processing Clinical Language Annotation, Modeling, and Processing software system, to extract generally discussed topics and detailed medical terms (expressed as probability distribution) to investigate the stories communicated in these comments.

Results

Our analysis showed that 63.33% (6053/9557) of the users who entered vital signs made at least one free-text comment in vital sign flowsheet entries. The user roles that were most likely to compose comments were registered nurse, technician, and licensed nurse. The most frequently identified topics were the notification of a result to health care providers (0.347), the context of a measurement (0.307), and an inability to obtain a vital sign (0.224). There were 4187 unique medical terms that were extracted from 46,029 (0.220) comments, including many symptom-related terms such as “pain,” “upset,” “dizziness,” “coughing,” “anxiety,” “distress,” and “fever” and drug-related terms such as “tylenol,” “anesthesia,” “cannula,” “oxygen,” “motrin,” “rituxan,” and “labetalol.”

Conclusions

Considering that flowsheet comments are generally not displayed or automatically pulled into any clinical notes, our findings suggest that the flowsheet comment functionality can be simplified (eg, via structured response fields instead of a text input dialog) to reduce health care provider effort. Moreover, rich and clinically important medical terms such as medications and symptoms should be explicitly recorded in clinical notes for better visibility.

Real-Time Electronic Health Record Mortality Prediction During the COVID-19 Pandemic: A Prospective Cohort Study

Background

The SARS-CoV-2 virus has infected millions of people, overwhelming critical care resources in some regions. Many plans for rationing critical care resources during crises are based on the Sequential Organ Failure Assessment (SOFA) score. The COVID-19 pandemic created an emergent need to develop and validate a novel electronic health record (EHR)-computable tool to predict mortality.

Research Questions

To rapidly develop, validate, and implement a novel real-time mortality score for the COVID-19 pandemic that improves upon SOFA.

Study Design and Methods

We conducted a prospective cohort study of a regional health system with 12 hospitals in Colorado between March 2020 and July 2020. All patients >14 years old hospitalized during the study period without a do not resuscitate order were included. Patients were stratified by the diagnosis of COVID-19. From this cohort, we developed and validated a model using stacked generalization to predict mortality using data widely available in the EHR by combining five previously validated scores and additional novel variables reported to be associated with COVID-19-specific mortality. We compared the area under the receiver operator curve (AUROC) for the new model to the SOFA score and the Charlson Comorbidity Index.

Results

We prospectively analyzed 27,296 encounters, of which 1,358 (5.0%) were positive for SARS-CoV-2, 4,494 (16.5%) included intensive care unit (ICU)-level care, 1,480 (5.4%) included invasive mechanical ventilation, and 717 (2.6%) ended in death. The Charlson Comorbidity Index and SOFA scores predicted overall mortality with an AUROC of 0.72 and 0.90, respectively. Our novel score predicted overall mortality with AUROC 0.94. In the subset of patients with COVID-19, we predicted mortality with AUROC 0.90, whereas SOFA had AUROC of 0.85.

Interpretation

We developed and validated an accurate, in-hospital mortality prediction score in a live EHR for automatic and continuous calculation using a novel model, that improved upon SOFA.

Take Home Points

Study Question: Can we improve upon the SOFA score for real-time mortality prediction during the COVID-19 pandemic by leveraging electronic health record (EHR) data?Results: We rapidly developed and implemented a novel yet SOFA-anchored mortality model across 12 hospitals and conducted a prospective cohort study of 27,296 adult hospitalizations, 1,358 (5.0%) of which were positive for SARS-CoV-2. The Charlson Comorbidity Index and SOFA scores predicted all-cause mortality with AUROCs of 0.72 and 0.90, respectively. Our novel score predicted mortality with AUROC 0.94.Interpretation: A novel EHR-based mortality score can be rapidly implemented to better predict patient outcomes during an evolving pandemic.

Identifying Urinary Tract Infection-Related Information in Home Care Nursing Notes

Objectives:

Urinary tract infection (UTI) is common in home care but not easily captured with standard assessment. This study aimed to examine the value of nursing notes in detecting UTI signs and symptoms in home care.

Design:

The study developed a natural language processing (NLP) algorithm to automatically identify UTI-related information in nursing notes.

Setting and Participants:

Home care visit notes (n = 1,149,586) and care coordination notes (n = 1,461,171) for 89,459 patients treated in the largest nonprofit home care agency in the United States during 2014.

Measures:

We generated 6 categories of UTI-related information from literature and used the Unified Medical Language System (UMLS) to identify a preliminary list of terms. The NLP algorithm was tested on a gold standard set of 300 clinical notes annotated by clinical experts. We used structured Outcome and Assessment Information Set data to extract the frequency of UTI-related emergency department (ED) visits or hospitalizations and explored time-patterns in documentation of UTI-related information.

Results:

The NLP system achieved very good overall performance (F measure = 0.9, 95% CI: 0.87–0.93) based on the test results obtained by using the notes for patients admitted to the ED or hospital due to UTI. UTI-related information was significantly more prevalent (P < .01 for all the tests) in home care episodes with UTI-related ED admission or hospitalization vs the general patient population; 81% of home care episodes with UTI-related hospitalization or ED admission had at least 1 category of UTI-related information vs 21.6% among episodes without UTI-related hospitalization or ED admission. Frequency of UTI-related information documentation increased in advance of UTI-related hospitalization or ED admission, peaking within a few days before the event.

Conclusions and Implications:

Information in nursing notes is often overlooked by stakeholders and not integrated into predictive modeling for decision-making support, but our findings highlight their value in early risk identification and care guidance. Health care administrators should consider using NLP to extract clinical data from nursing notes to improve early detection and treatment, which may lead to quality improvement and cost reduction.

A bibliometric analysis and visualization of medical data mining research

BACKGROUND

Data mining technology used in the field of medicine has been widely studied by scholars all over the world. But there is little research on medical data mining (MDM) from the perspectives of bibliometrics and visualization, and the research topics and development trends in this field are still unclear.

METHODS

This paper has applied bibliometric visualization software tools, VOSviewer 1.6.10 and CiteSpace V, to study the citation characteristics, international cooperation, author cooperation, and geographical distribution of the MDM.

RESULTS

A total of 1575 documents are obtained, and the most frequent document type is article (1376). SHAN NH is the most productive author, with the highest number of publications of 12, and the Gillies's article (750 times citation) is the most cited paper. The most productive country and institution in MDM is the USA (559) and US FDA (35), respectively. The Journal of Biomedical Informatics, Expert Systems with Applications and Journal of Medical Systems are the most productive journals, which reflected the nature of the research, and keywords "classification (790)" and "system (576)" have the strongest strength. The hot topics in MDM are drug discovery, medical imaging, vaccine safety, and so on. The 3 frontier topics are reporting system, precision medicine, and inflammation, and would be the foci of future research.

CONCLUSION

The present study provides a panoramic view of data mining methods applied in medicine by visualization and bibliometrics. Analysis of authors, journals, institutions, and countries could provide reference for researchers who are fresh to the field in different ways. Researchers may also consider the emerging trends when deciding the direction of their study.

Quality of Care: Ecological Study for the Evaluation of Completeness and Accuracy in Nursing Assessment

Nursing documentation is an important proxy of the quality of care, and quality indicators in nursing assessment can be used to assess and improve the quality of care in health care institutions. The study aims to evaluate the completeness and the accuracy of nursing assessment, analyzing the compilation of pain assessment and nutritional status (body mass index (BMI)) in computerized nursing records, and how it is influenced by four variables: nurse to patient ratio, diagnosis related group weight (DRG), seniority of charge nurse, and type of ward (medical, surgical or other). The observational ecological pilot study was conducted between September and October 2018 in an Italian Tertiary-Level Teaching Hospital. The nursing documentation analyzed for the ‘Assessment’ phase included 12,513 records, 50.4% concerning pain assessment, and 45% BMI. The nurse–patient ratio showed a significant direct association with the assessment of nutritional status (p = 0.032). The average weight DRG has a negative influence on pain and BMI assessment; the surgical units positively correlate with the compilation of nursing assessment (BMI and pain). The nursing process is an essential component for the continuous improvement in the quality of care. Nurses need to be accountable to improve their knowledge and skills in nursing documentation.

Development and validation of early warning score system: A systematic literature review

OBJECTIVES

This review aims to: 1) evaluate the quality of model reporting, 2) provide an overview of methodology for developing and validating Early Warning Score Systems (EWSs) for adult patients in acute care settings, and 3) highlight the strengths and limitations of the methodologies, as well as identify future directions for EWS derivation and validation studies.

METHODOLOGY

A systematic search was conducted in PubMed, Cochrane Library, and CINAHL. Only peer reviewed articles and clinical guidelines regarding developing and validating EWSs for adult patients in acute care settings were included. 615 articles were extracted and reviewed by five of the authors. Selected studies were evaluated based on the Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD) checklist. The studies were analyzed according to their study design, predictor selection, outcome measurement, methodology of modeling, and validation strategy.

RESULTS

A total of 29 articles were included in the final analysis. Twenty-six articles reported on the development and validation of a new EWS, while three reported on validation and model modification. Only eight studies met more than 75% of the items in the TRIPOD checklist. Three major techniques were utilized among the studies to inform their predictive algorithms: 1) clinical-consensus models (n = 6), 2) regression models (n = 15), and 3) tree models (n = 5). The number of predictors included in the EWSs varied from 3 to 72 with a median of seven. Twenty-eight models included vital signs, while 11 included lab data. Pulse oximetry, mental status, and other variables extracted from electronic health records (EHRs) were among other frequently used predictors. In-hospital mortality, unplanned transfer to the intensive care unit (ICU), and cardiac arrest were commonly used clinical outcomes. Twenty-eight studies conducted a form of model validation either within the study or against other widely-used EWSs. Only three studies validated their model using an external database separate from the derived database.

CONCLUSION

This literature review demonstrates that the characteristics of the cohort, predictors, and outcome selection, as well as the metrics for model validation, vary greatly across EWS studies. There is no consensus on the optimal strategy for developing such algorithms since data-driven models with acceptable predictive accuracy are often site-specific. A standardized checklist for clinical prediction model reporting exists, but few studies have included reporting aligned with it in their publications. Data-driven models are subjected to biases in the use of EHR data, thus it is particularly important to provide detailed study protocols and acknowledge, leverage, or reduce potential biases of the data used for EWS development to improve transparency and generalizability.

Mixed-methods approach to understanding clinician macrocognition in the design of a clinical decision support tool: a study protocol

INTRODUCTION

The anatomic variants of congenital heart disease (CHD) are multiple. The increased survival of these patients and disposition into communities has led to an increase in their acute presentation to non-CHD experts in primary care clinics and emergency departments. Given the vulnerability and fragility of these patients in the face of acute illness, new clinical decision support systems (CDSS) are urgently needed to better translate the best practice recommendations for the care of these patients. This study aims to understand the perceived confidence and macrocognitive processes of non-CHD experts (emergency medicine physicians) and CHD experts (paediatric cardiac intensivists) when treating children with CHD during acute illness and apply this to optimise the design of a CDSS (MyHeartPass^™) for these patients.

METHODS AND ANALYSIS

The first phase of the study involves a survey of non-CHD experts and CHD experts to understand their perceived confidence as it relates to treating acutely ill patients with CHD. The second phase is a qualitative cognitive task analysis using critical decision method to characterise and compare the macrocognitive processes used by non-CHD experts and CHD experts during the critical decision making. In phases 3 and 4, heuristic evaluation and usability testing of the CDSS will be completed. These results will be used to inform design changes to the chosen CDSS (MyHeartPass^™). In the final phase, a within-participant simulation design will be used to study the effect of the CDSS on clinical decision making compared with baseline (without use of CDSS).

ETHICS AND DISSEMINATION

Ethics approval from The Hospital for Sick Children in Toronto, Ontario, Canada has been obtained for all phases. Results will be published in peer-reviewed journals and presented at relevant conferences. On successful completion of these studies, it is anticipated that there will be a controlled implementation of the redesigned CDSS.

Leveraging Clinical Expertise as a Feature - not an Outcome - of Predictive Models: Evaluation of an Early Warning System Use Case

Identifying patients at risk of deterioration in the hospital and intervening more quickly to prevent adverse events is a top patient safety priority. Early warning scores (EWS) identify at risk patients, but there is much opportunity for improvement particularly related to increasing lead time - the time from an alert trigger to adverse event (e.g., cardiac arrest, death). Our team develops healthcare process models of clinical concern (HPM-CC) and in this work has identified documentation signals that are proxies of nurses concern and can be used to predict patient risk earlier than current EWS systems that rely only on physiological data. We compared the performance of a validated EWS - the MEWS - to our novel model (MEWS-CC) comprised of MEWS criteria plus 3 proxy variables of nursing concern. MEWS-CC performed similarly to MEWS, with the added benefit of increased the time from EWS trigger to event by 5-26 hours.