A Bayesian model for triage decision support

Artificial Intelligence-Based Application Provides Accurate Medical Triage Advice When Compared to Consensus Decisions of Healthcare Providers

Accurate medical triage is essential for improving patient outcomes and efficient healthcare delivery. Patients increasingly rely on artificial intelligence (AI)-based applications to access healthcare information, including medical triage advice. We assessed the accuracy of triage decisions provided by an AI-based application. We presented 50 clinical vignettes to the AI-based application, seven emergency medicine providers, and five internal medicine physicians. We compared the triage decisions of the AI-based application to those of the individual providers as well as their consensus decisions. When compared to the human clinicians’ consensus triage decisions, the AI-based application performed equal or better than individual human clinicians.

Moral Decision Making in Human-Agent Teams: Human Control and the Role of Explanations

With the progress of Artificial Intelligence, intelligent agents are increasingly being deployed in tasks for which ethical guidelines and moral values apply. As artificial agents do not have a legal position, humans should be held accountable if actions do not comply, implying humans need to exercise control. This is often labeled as Meaningful Human Control (MHC). In this paper, achieving MHC is addressed as a design problem, defining the collaboration between humans and agents. We propose three possible team designs (Team Design Patterns), varying in the level of autonomy on the agent’s part. The team designs include explanations given by the agent to clarify its reasoning and decision-making. The designs were implemented in a simulation of a medical triage task, to be executed by a domain expert and an artificial agent. The triage task simulates making decisions under time pressure, with too few resources available to comply with all medical guidelines all the time, hence involving moral choices. Domain experts (i.e., health care professionals) participated in the present study. One goal was to assess the ecological relevance of the simulation. Secondly, to explore the control that the human has over the agent to warrant moral compliant behavior in each proposed team design. Thirdly, to evaluate the role of agent explanations on the human’s understanding in the agent’s reasoning. Results showed that the experts overall found the task a believable simulation of what might occur in reality. Domain experts experienced control over the team’s moral compliance when consequences were quickly noticeable. When instead the consequences emerged much later, the experts experienced less control and felt less responsible. Possibly due to the experienced time pressure implemented in the task or over trust in the agent, the experts did not use explanations much during the task; when asked afterwards they however considered these to be useful. It is concluded that a team design should emphasize and support the human to develop a sense of responsibility for the agent’s behavior and for the team’s decisions. The design should include explanations that fit with the assigned team roles as well as the human cognitive state.

Current review and next steps for artificial intelligence in multiple sclerosis risk research

In the last few decades, the prevalence of multiple sclerosis (MS), a chronic inflammatory disease of the nervous system, has increased, particularly in Northern European countries, the United States, and United Kingdom. The promise of artificial intelligence (AI) and machine learning (ML) as tools to address problems in MS research has attracted increasing interest in these methods. Bayesian networks offer a clear advantage since they can integrate data and causal knowledge allowing for visualizing interactions between dependent variables and potential confounding factors. A review of AI/ML research methods applied to MS found 216 papers using terms "Multiple Sclerosis", "machine learning", "artificial intelligence", "Bayes", and "Bayesian", of which 90 were relevant and recently published. More than half of these involve the detection and segmentation of MS lesions for quantitative analysis; however clinical and lifestyle risk factor assessment and prediction have largely been ignored. Of those that address risk factors, most provide only association studies for some factors and often fail to include the potential impact of confounding factors and bias (especially where these have causal explanations) that could affect data interpretation, such as reporting quality and medical care access in various countries. To address these gaps in the literature, we propose a causal Bayesian network approach to assessing risk factors for MS, which can address deficiencies in current epidemiological methods of producing risk measurements and makes better use of observational data.

A Bayesian Model to Predict Survival After Left Ventricular Assist Device Implantation

OBJECTIVES

This study investigates the use of a Bayesian statistical models to predict survival at various time points in patients undergoing left ventricular assist device (LVAD) implantation.

BACKGROUND

LVADs are being increasingly used in patients with end-stage heart failure. Appropriate patient selection continues to be key in optimizing post-LVAD outcomes.

METHODS

Data used for this study were derived from 10,277 adult patients from the INTERMACS (Inter-Agency Registry for Mechanically Assisted Circulatory Support) who had a primary LVAD implanted between January 2012 and December 2015. Risk for mortality was calculated retrospectively for various time points (1, 3, and 12 months) after LVAD implantation, using multiple pre-implantation variables. For each of these endpoints, a separate tree-augmented naïve Bayes model was constructed using the most predictive variables.

RESULTS

A set of 29, 26, and 31 pre-LVAD variables were found to be predictive at 1, 3, and 12 months, respectively. Predictors of 1-month mortality included low Inter-Agency Registry for Mechanically Assisted Circulatory Support profile, number of acute events in the 48 h before surgery, temporary mechanical circulatory support, and renal and hepatic dysfunction. Variables predicting 12-month mortality included advanced age, frailty, device strategy, and chronic renal disease. The accuracy of all Bayesian models was between 76% and 87%, with an area under the receiver operative characteristics curve of between 0.70 and 0.71.

CONCLUSIONS

A Bayesian prognostic model for predicting survival based on the comprehensive INTERMACS registry provided highly accurate predictions of mortality based on pre-operative variables. These models may facilitate clinical decision-making while screening candidates for LVAD therapy.

Self-triage for acute primary care via a smartphone application: Practical, safe and efficient?

Background

Since the start of out-of-hours (OOH) primary care clinics, the number of patient consultations has been increasing. Triage plays an important role in patient selection for a consultation, and in providing reassurance and self-management advice.

Objective

We aimed to investigate whether the smartphone application “Should I see a doctor?” (in Dutch:”moet ik naar de dokter?”) could guide patients in appropriate consultation at OOH clinics by focusing on four topics: 1) app usage, 2) user satisfaction, 3) whether the app provides the correct advice, and 4) whether users intend to follow the advice.

Design and setting

A prospective, cross-sectional study amongst app users in a routine primary care setting.

Methods

The app is a self-triage tool for acute primary care. A built-in questionnaire asked users about the app’s clarity, their satisfaction and whether they intended to follow the app’s advice (n = 4456). A convenience sample of users was phoned by a triage nurse (reference standard) to evaluate whether the app’s advice corresponded with the outcome of the triage call (n = 126). Suggestions of phoned participants were listed.

Results

The app was used by patients of all ages, also by parents for their children, and mostly for abdominal pain, skin disorders and cough. 58% of users received the advice to contact the clinic, 34% a self-care advice and 8% to wait-and-see. 65% of users intended to follow the app’s advice. The app was rated as ‘neutral’ to ‘very clear’ by 87%, and 89% were ‘neutral’ to ‘very satisfied’. In 81% of participants the app’s advice corresponded to the triage call outcome, with sensitivity, specificity, positive- and negative predictive values of 84%, 74%, 88% and 67%, respectively.

Conclusion

The app “Should I see a doctor?” could be a valuable tool to guide patients in contacting the OOH primary care clinic for acute care. To further improve the app’s safety and efficiency, triaging multiple symptoms should be facilitated, and more information should be provided to patients receiving a wait-and-see advice.

A Semantic-Based Model for Triage Patients in Emergency Departments

Triage is a process performed in an emergency department that aims to sort patients according to their need for care. When performed speedily and correctly, this process can potentially increase the chances of survival for a patient with serious complications. This study aims to develop a computer model, called UbiTriagem, which supports the process of triage using the concepts of web semantics and ubiquitous computing focused on healthcare. For evaluating the proposal, we performed an analysis of scenario-driven triage based on previously determined ratings. In addition, we conducted a usability evaluation in emergency department with the developed prototype with two user groups: nurses and patients. The main scientific contribution is the automatic triage assessment based on the gathering of patient data on mobile devices, performed automatically through the use of a reasoning technique in an ontology. The results for all evaluations were very positive. The automatic triage assessment has been assertive in 93.3% of the cases and, after adjustments in the model, in 100% of the cases. Regarding user satisfaction, we obtained rates of 98.7% and 96% when considering perception of utility and ease of use, respectively.

Automatic learning of mortality in a CPN model of the systemic inflammatory response syndrome

The aim of this paper is to apply machine learning as a method to refine a manually constructed CPN for the assessment of the severity of the systemic inflammatory response syndrome (SIRS).The goal of tuning the CPN is to create a scoring system that uses only objective data, compares favourably with other severity-scoring systems and differentiates between sepsis and non-infectious SIRS. The resulting model, the Learned-Age (L_A) -Sepsis CPN has good discriminatory ability for the prediction of 30-day mortality with an area under the ROC curve of 0.79. This result compares well to existing scoring systems. The L_A-Sepsis CPN also has a modest ability to discriminate between sepsis and non-infectious SIRS.

A new Bayesian network-based risk stratification model for prediction of short-term and long-term LVAD mortality

Existing risk assessment tools for patient selection for left ventricular assist devices (LVADs) such as the Destination Therapy Risk Score and HeartMate II Risk Score (HMRS) have limited predictive ability. This study aims to overcome the limitations of traditional statistical methods by performing the first application of Bayesian analysis to the comprehensive Interagency Registry for Mechanically Assisted Circulatory Support dataset and comparing it to HMRS. We retrospectively analyzed 8,050 continuous flow LVAD patients and 226 preimplant variables. We then derived Bayesian models for mortality at each of five time end-points postimplant (30 days, 90 days, 6 month, 1 year, and 2 years), achieving accuracies of 95%, 90%, 90%, 83%, and 78%, Kappa values of 0.43, 0.37, 0.37, 0.45, and 0.43, and area under the receiver operator characteristic (ROC) of 91%, 82%, 82%, 80%, and 81%, respectively. This was in comparison to the HMRS with an ROC of 57% and 60% at 90 days and 1 year, respectively. Preimplant interventions, such as dialysis, ECMO, and ventilators were major contributing risk markers. Bayesian models have the ability to reliably represent the complex causal relations of multiple variables on clinical outcomes. Their potential to develop a reliable risk stratification tool for use in clinical decision making on LVAD patients encourages further investigation.

Evaluation of symptom checkers for self diagnosis and triage: audit study

OBJECTIVE

To determine the diagnostic and triage accuracy of online symptom checkers (tools that use computer algorithms to help patients with self diagnosis or self triage).

DESIGN

Audit study.

SETTING

Publicly available, free symptom checkers.

PARTICIPANTS

23 symptom checkers that were in English and provided advice across a range of conditions. 45 standardized patient vignettes were compiled and equally divided into three categories of triage urgency: emergent care required (for example, pulmonary embolism), non-emergent care reasonable (for example, otitis media), and self care reasonable (for example, viral upper respiratory tract infection).

MAIN OUTCOME MEASURES

For symptom checkers that provided a diagnosis, our main outcomes were whether the symptom checker listed the correct diagnosis first or within the first 20 potential diagnoses (n=770 standardized patient evaluations). For symptom checkers that provided a triage recommendation, our main outcomes were whether the symptom checker correctly recommended emergent care, non-emergent care, or self care (n=532 standardized patient evaluations).

RESULTS

The 23 symptom checkers provided the correct diagnosis first in 34% (95% confidence interval 31% to 37%) of standardized patient evaluations, listed the correct diagnosis within the top 20 diagnoses given in 58% (55% to 62%) of standardized patient evaluations, and provided the appropriate triage advice in 57% (52% to 61%) of standardized patient evaluations. Triage performance varied by urgency of condition, with appropriate triage advice provided in 80% (95% confidence interval 75% to 86%) of emergent cases, 55% (47% to 63%) of non-emergent cases, and 33% (26% to 40%) of self care cases (P<0.001). Performance on appropriate triage advice across the 23 individual symptom checkers ranged from 33% (95% confidence interval 19% to 48%) to 78% (64% to 91%) of standardized patient evaluations.

CONCLUSIONS

Symptom checkers had deficits in both triage and diagnosis. Triage advice from symptom checkers is generally risk averse, encouraging users to seek care for conditions where self care is reasonable.

Cardiac Health Risk Stratification System (CHRiSS): a Bayesian-based decision support system for left ventricular assist device (LVAD) therapy

This study investigated the use of Bayesian Networks (BNs) for left ventricular assist device (LVAD) therapy; a treatment for end-stage heart failure that has been steadily growing in popularity over the past decade. Despite this growth, the number of LVAD implants performed annually remains a small fraction of the estimated population of patients who might benefit from this treatment. We believe that this demonstrates a need for an accurate stratification tool that can help identify LVAD candidates at the most appropriate point in the course of their disease. We derived BNs to predict mortality at five endpoints utilizing the Interagency Registry for Mechanically Assisted Circulatory Support (INTERMACS) database: containing over 12,000 total enrolled patients from 153 hospital sites, collected since 2006 to the present day, and consisting of approximately 230 pre-implant clinical variables. Synthetic minority oversampling technique (SMOTE) was employed to address the uneven proportion of patients with negative outcomes and to improve the performance of the models. The resulting accuracy and area under the ROC curve (%) for predicted mortality were 30 day: 94.9 and 92.5; 90 day: 84.2 and 73.9; 6 month: 78.2 and 70.6; 1 year: 73.1 and 70.6; and 2 years: 71.4 and 70.8. To foster the translation of these models to clinical practice, they have been incorporated into a web-based application, the Cardiac Health Risk Stratification System (CHRiSS). As clinical experience with LVAD therapy continues to grow, and additional data is collected, we aim to continually update these BN models to improve their accuracy and maintain their relevance. Ongoing work also aims to extend the BN models to predict the risk of adverse events post-LVAD implant as additional factors for consideration in decision making.

Knowledge discovery in clinical decision support systems for pain management: a systematic review

OBJECTIVE

The occurrence of pain accounts for billions of dollars in annual medical expenditures; loss of quality of life and decreased worker productivity contribute to indirect costs. As pain is highly subjective, clinical decision support systems (CDSSs) can be critical for improving the accuracy of pain assessment and offering better support for clinical decision-making. This review is focused on computer technologies for pain management that allow CDSSs to obtain knowledge from the clinical data produced by either patients or health care professionals.

METHODS AND MATERIALS

A comprehensive literature search was conducted in several electronic databases to identify relevant articles focused on computerised systems that constituted CDSSs and include data or results related to pain symptoms from patients with acute or chronic pain, published between 1992 and 2011 in the English language. In total, thirty-nine studies were analysed; thirty-two were selected from 1245 citations, and seven were obtained from reference tracking.

RESULTS

The results highlighted the following clusters of computer technologies: rule-based algorithms, artificial neural networks, nonstandard set theory, and statistical learning algorithms. In addition, several methodologies were found for content processing such as terminologies, questionnaires, and scores. The median accuracy ranged from 53% to 87.5%.

CONCLUSIONS

Computer technologies that have been applied in CDSSs are important but not determinant in improving the systems' accuracy and the clinical practice, as evidenced by the moderate correlation among the studies. However, these systems play an important role in the design of computerised systems oriented to a patient's symptoms as is required for pain management. Several limitations related to CDSSs were observed: the lack of integration with mobile devices, the reduced use of web-based interfaces, and scarce capabilities for data to be inserted by patients.

A randomized controlled trial comparing health and quality of life of lung transplant recipients following nurse and computer-based triage utilizing home spirometry monitoring

BACKGROUND

Lung transplantation is now a standard intervention for patients with advanced lung disease. Home monitoring of pulmonary function and symptoms has been used to follow the progress of lung transplant recipients in an effort to improve care and clinical status. The study objective was to determine the relative performance of a computer-based Bayesian algorithm compared with a manual nurse decision process for triaging clinical intervention in lung transplant recipients participating in a home monitoring program.

MATERIALS AND METHODS

This randomized controlled trial had 65 lung transplant recipients assigned to either the Bayesian or nurse triage study arm. Subjects monitored and transmitted spirometry and respiratory symptoms daily to the data center using an electronic spirometer/diary device. Subjects completed the Short Form-36 (SF-36) survey at baseline and after 1 year. End points were change from baseline after 1 year in forced expiratory volume at 1 s (FEV1) and quality of life (SF-36 scales) within and between each study arm.

RESULTS

There were no statistically significant differences between groups in FEV1 or SF-36 scales at baseline or after 1 year.: Results were comparable between nurse and Bayesian system for detecting changes in spirometry and symptoms, providing support for using computer-based triage support systems as remote monitoring triage programs become more widely available.

CONCLUSIONS

The feasibility of monitoring critical patient data with a computer-based decision system is especially important given the likely economic constraints on the growth in the nurse workforce capable of providing these early detection triage services.

Comparison of adaptive neuro-fuzzy inference system and artificial neutral networks model to categorize patients in the emergency department

Unexpected disease outbreaks and disasters are becoming primary issues facing our world. The first points of contact either at the disaster scenes or emergency department exposed the frontline workers and medical physicians to the risk of infections. Therefore, there is a persuasive demand for the integration and exploitation of heterogeneous biomedical information to improve clinical practice, medical research and point of care. In this paper, a primary triage model was designed using two different methods: an adaptive neuro-fuzzy inference system (ANFIS) and artificial neural network (ANN).When the patient is presented at the triage counter, the system will capture their vital signs and chief complains beside physiology stat and general appearance of the patient. This data will be managed and analyzed in the data server and the patient's emergency status will be reported immediately. The proposed method will help to reduce the queue time at the triage counter and the emergency physician's burden especially duringdisease outbreak and serious disaster. The models have been built with 2223 data set extracted from the Emergency Department of the Universiti Kebangsaan Malaysia Medical Centre to predict the primary triage category. Multilayer feed forward with one hidden layer having 12 neurons has been used for the ANN architecture. Fuzzy subtractive clustering has been used to find the fuzzy rules for the ANFIS model. The results showed that the RMSE, %RME and the accuracy which evaluated by measuring specificity and sensitivity for binary classificationof the training data were 0.14, 5.7 and 99 respectively for the ANN model and 0.85, 32.00 and 96.00 respectively for the ANFIS model. As for unseen data the root mean square error, percentage the root mean square error and the accuracy for ANN is 0.18, 7.16 and 96.7 respectively, 1.30, 49.84 and 94 respectively for ANFIS model. The ANN model was performed better for both training and unseen data than ANFIS model in term of generalization. It was therefore chosen as the technique to develop the primary triage prediction model. This primary triage model will be combined with the secondary triage prediction model to produce the final triage category as a tool to assist the medical officer in the emergency department.

Factors associated with prolonged stay in a pediatric emergency observation unit of an urban tertiary children's hospital in China

OBJECTIVES

This study aimed to examine the factors associated with increased length of stay (LOS > 24 hours) in the pediatric emergency observation unit (OU) of an urban tertiary children's hospital in China.

METHODS

This study was a retrospective cohort study. We retrieved and examined all the records of patients (age, 0-16 years) who were admitted to the OU (n = 10,852) during July 1, 2008, to June 30, 2009. The primary outcome was LOS and prolonged stay (LOS > 24 hours). We also performed a sensitivity analysis by using LOS of 3 days or greater and LOS of 6 days or greater as dependent variables in logistic regression and compared with LOS of greater than 24 hours regression to examine the robustness of the associations.

RESULTS

The overall mean (SD) LOS was 24.0 (24.4) hours; 31.3% had LOS of greater than 24 hours, of which the mean (SD) LOS was 50.2 (28.6) hours. The following factors were associated with LOS of greater than 24 hours: age, 28 days to 3 months (odds ratio, [OR], 1.87; 95% confidence interval, 1.36-2.59) and older than 3 months to 12 months (OR, 1.83; 95% CI, 1.35-2.50) compared with age 0 to 28 days; neurologic diseases (OR, 1.50; 95% CI, 1.31-1.72), infectious diseases (OR, 2.00; 95% CI, 1.61-2.49), and visits for non-respiratory-related signs and symptoms (OR, 2.00; 95% CI, 1.61-2.49); acuity level of emergent (OR, 1.79; 95% CI, 1.57-2.04); procedures (OR, 7.09; 95% CI, 4.16-12.10); emergency transfusions (OR, 1.33; 95% CI, 1.01-1.75); staffed by residents (OR, 1.12; 95% CI, 1.01-1.24); and patients living in low-annual gross domestic product districts (OR, 1.14; 95% CI, 1.01-1.29). Arrival at evening (OR, 0.54; 95% CI, 0.49-0.60) and overnight (OR, 0.43; 95% CI, 0.38-0.49) were less likely to have LOS of greater than 24 hours than arrival during day shifts.

CONCLUSIONS

We identified some risk factors for prolonged stay in an OU. These factors are the starting points in understanding issues related to prolonged stay and are needed to assess efficiency and quality of care in pediatric emergency department and OU. Our results have provided information basis for making improvements in the system and may be important considerations for similar institutions, which encounter similar challenges.

Interconnection between biological abnormalities in borderline personality disorder: use of the Bayesian networks model

There is agreement in that strengthening the sets of neurobiological data would reinforce the diagnostic objectivity of many psychiatric entities. This article attempts to use this approach in borderline personality disorder (BPD). Assuming that most of the biological findings in BPD reflect common underlying pathophysiological processes we hypothesized that most of the data involved in the findings would be statistically interconnected and interdependent, indicating biological consistency for this diagnosis. Prospectively obtained data on scalp and sleep electroencephalography (EEG), clinical neurologic soft signs, the dexamethasone suppression and thyrotropin-releasing hormone stimulation tests of 20 consecutive BPD patients were used to generate a Bayesian network model, an artificial intelligence paradigm that visually illustrates eventual associations (or inter-dependencies) between otherwise seemingly unrelated variables. The Bayesian network model identified relationships among most of the variables. EEG and TSH were the variables that influence most of the others, especially sleep parameters. Neurological soft signs were linked with EEG, TSH, and sleep parameters. The results suggest the possibility of using objective neurobiological variables to strengthen the validity of future diagnostic criteria and nosological characterization of BPD.

A comparison of parental and nursing assessments of level of illness or injury in a pediatric emergency department

BACKGROUND

The 5-tier Emergency Severity Index (ESI) score is a well-accepted, validated triage tool with good interrater reliability. Parental perception of illness severity has not been compared to ESI score.

OBJECTIVE

This study compares parental assessment of severity of illness to triage nurse acuity.

DESIGN

Prospective and descriptive.

SETTING

Large, urban pediatric emergency department (ED).

PARTICIPANTS

Parents/guardians of patients younger than 18 years.

INTERVENTION

The triage nurse assigned an ESI score, and the parent/ guardian assigned all patients a severity score on a scale of 1 to 5 (1, most sick and 5, least sick). Mean severity scores were compared between the groups.

RESULTS

There were 142 participants with a mean patient age of 6.15 years. The mean participant and nurse severity scores were 3.01 and 3.35, respectively, with an intraclass correlation coefficient of 0.203 (P = 0.008). Most frequently, the parent/guardian and triage nurse assigned the same score (n = 44, 31%). Seventy-six percent of the parent/ guardian scores were within 1 point of the triage nurse score.

CONCLUSIONS

Close agreement exists between parent/guardian and nurse ESI scores, illustrating objectivity in parent/guardian assessments. This study provides a springboard for future studies regarding ED use after educating families on ED triage.

Multilingual chief complaint classification for syndromic surveillance: an experiment with Chinese chief complaints

Purpose

Syndromic surveillance is aimed at early detection of disease outbreaks. An important data source for syndromic surveillance is free-text chief complaints (CCs), which may be recorded in different languages. For automated syndromic surveillance, CCs must be classified into predefined syndromic categories to facilitate subsequent data aggregation and analysis. Despite the fact that syndromic surveillance is largely an international effort, existing CC classification systems do not provide adequate support for processing CCs recorded in non-English languages. This paper reports a multilingual CC classification effort, focusing on CCs recorded in Chinese.

Methods

We propose a novel Chinese CC classification system leveraging a Chinese-English translation module and an existing English CC classification approach. A set of 470 Chinese key phrases was extracted from about one million Chinese CC records using statistical methods. Based on the extracted key phrases, the system translates Chinese text into English and classifies the translated CCs to syndromic categories using an existing English CC classification system.

Results

Compared to alternative approaches using a bilingual dictionary and a general-purpose machine translation system, our approach performs significantly better in terms of positive predictive value (PPV or precision), sensitivity (recall), specificity, and F measure (the harmonic mean of PPV and sensitivity), based on a computational experiment using real-world CC records.

Conclusions

Our design provides satisfactory performance in classifying Chinese CCs into syndromic categories for public health surveillance. The overall design of our system also points out a potentially fruitful direction for multilingual CC systems that need to handle languages beyond English and Chinese.

Evolution and challenges in the design of computational systems for triage assistance

Compared with expert systems for specific disease diagnosis, knowledge-based systems to assist decision making in triage usually try to cover a much wider domain but can use a smaller set of variables due to time restrictions, many of them subjective so that accurate models are difficult to build. In this paper, we first study criteria that most affect the performance of systems for triage assistance. Such criteria include whether principled approaches from machine learning can be used to increase accuracy and robustness and to represent uncertainty, whether data and model integration can be performed or whether temporal evolution can be modeled to implement retriage or represent medication responses. Following the most important criteria, we explore current systems and identify some missing features that, if added, may yield to more accurate triage systems.

Prospective evaluation of the MET-AP system providing triage plans for acute pediatric abdominal pain

BACKGROUND

Children with acute abdominal pain (AP) are frequently assessed in the Emergency Department (ED). Though the majority of patients have benign causes, uncertainty during the physician's initial assessment may result in unnecessary tests and prolonged observation before a definitive disposition decision can be made. A rule-based mobile clinical decision support system, Mobile Emergency Triage-Abdominal Pain (MET-AP), has been developed to recommend an appropriate triage plan (discharge, consult surgery or observe/investigate) early in the ED visit, with the goal of promoting ED efficiencies and improved patient outcomes.

OBJECTIVE

To prospectively evaluate the accuracy of MET-AP to recommend the correct triage plan when used during the initial assessment by staff emergency physicians (EPs) and residents in a tertiary care pediatric ED.

DESIGN

Prospective cohort study. Staff EPs and/or residents examined children, aged 1-16 years, with acute, non-traumatic AP of less than 10 days duration. Details of their initial assessment, along with their blinded prediction of the correct triage plan, were recorded electronically. Inter-observer assessments were collected, where possible. Telephone and chart follow-up at 10-14 days was conducted to determine the patient's outcome/diagnosis, and thus the gold standard triage plan appropriate for the patient's visit.

MEASUREMENTS

Accuracy of MET-AP to recommend the correct triage plan (i.e., to match the gold standard plan); accuracy of physicians to predict the correct triage plan; inter-observer agreement between staff EPs and residents for each clinical attribute recorded within MET-AP.

RESULTS

Over 8 months, 574 patients with AP completed follow-up (10% appendicitis, 13% other pathology, 77% benign/resolving conditions). For patient assessments by the staff EP (n=457), the MET-AP recommendation was correct for 72% of patients (95% CI's: 67.9-76.1), while the physician's prediction was correct in 70% of cases (65.9-74.2) (p=0.518). However, staff EP triage plans were more conservative than those generated by MET-AP, and a small number of patients whose triage plan should have been "consult surgery" would have been "discharged" by MET-AP. For resident assessments (n=339), MET-AP and physician accuracies were slightly lower, but not statistically different from staff results or from each other. Inter-observer agreement on most attributes was moderate to near perfect.

CONCLUSION

MET-AP shows promise in recommending the correct triage plan with similar overall accuracy to experienced pediatric EPs, but requires further research to improve accuracy and safety. MET-AP can be used on all pediatric ED patients with AP and is capable of producing a triage plan recommendation without requiring a complete set of patient information.