Residents' numeric inputting error in computerized physician order entry prescription

Automated entry of paper-based patient-reported outcomes: Applying deep learning to the Japanese orthopaedic association back pain evaluation questionnaire

BACKGROUND

Health-related patient-reported outcomes (HR-PROs) are crucial for assessing the quality of life among individuals experiencing low back pain. However, manual data entry from paper forms, while convenient for patients, imposes a considerable tallying burden on collectors. In this study, we developed a deep learning (DL) model capable of automatically reading these paper forms.

METHODS

We employed the Japanese Orthopaedic Association Back Pain Evaluation Questionnaire, a globally recognized assessment tool for low back pain. The questionnaire comprised 25 low back pain-related multiple-choice questions and three pain-related visual analog scales (VASs). We collected 1305 forms from an academic medical center as the training set, and 483 forms from a community medical center as the test set. The performance of our DL model for multiple-choice questions was evaluated using accuracy as a categorical classification task. The performance for VASs was evaluated using the correlation coefficient and absolute error as regression tasks.

RESULT

In external validation, the mean accuracy of the categorical questions was 0.997. When outputs for categorical questions with low probability (threshold: 0.9996) were excluded, the accuracy reached 1.000 for the remaining 65 % of questions. Regarding the VASs, the average of the correlation coefficients was 0.989, with the mean absolute error being 0.25.

CONCLUSION

Our DL model demonstrated remarkable accuracy and correlation coefficients when automatic reading paper-based HR-PROs during external validation.

Residents' subjective mental workload during computerized prescription entry

To examine residents' subjective mental workload when they enter prescriptions in a computerized physician order entry (CPOE) system. Twenty-two residents completed six prescribing tasks in which two factors were manipulated: numerical input method and level of urgency. Data on demographic characteristics, familiarity with CPOE, and pretest performance were collected. The subjective mental workload was measured by the National Aeronautics and Space Administration-Task Load Index (NASA-TLX). Temporal demand (Mean = 34.48) contributed most to residents' workload on the CPOE task, followed by Performance (Mean = 29.23). No significant associations were found between workload and demographic characteristics, CPOE familiarity, or pretest CPOE performance (p's > .05). A 3 × 2 repeated-measures ANOVA yielded main effects of numerical input method [F (2, 19) = 88.358, p < .001, η² = .900] and level of urgency [F (1, 21) = 169.654, p < .001, η² = .890], and interaction of input method and urgency [F (2, 20) = 87.427, p < .001, η² = .900]. Residents' major sources of workload during the CPOE prescription were temporal demand and performance. Prescriptions entered by the row of numbers exhibited the highest workload. Workload increased with higher level of urgency. It is necessary to emphasize the negative impact of subjective workload, especially in prescription task under urgent situation. Further researches focus on medical staff's workload are encouraged to ensure patient safety.

Key Experimental Factors of Machine Learning-Based Identification of Surgery Cancellations

This study aimed to provide effective methods for the identification of surgeries with high cancellation risk based on machine learning models and analyze the key factors that affect the identification performance. The data covered the period from January 1, 2013, to December 31, 2014, at West China Hospital in China, which focus on elective urologic surgeries. All surgeries were scheduled one day in advance, and all cancellations were of institutional resource- and capacity-related types. Feature selection strategies, machine learning models, and sampling methods are the most discussed topic in general machine learning researches and have a direct impact on the performance of machine learning models. Hence, they were considered to systematically generate complete schemes in machine learning-based identification of surgery cancellations. The results proved the feasibility and robustness of identifying surgeries with high cancellation risk, with the considerable maximum of area under the curve (AUC) (0.7199) for random forest model with original sampling using backward selection strategy. In addition, one-side Delong test and sum of square error analysis were conducted to measure the effects of feature selection strategy, machine learning model, and sampling method on the identification of surgeries with high cancellation risk, and the selection of machine learning model was identified as the key factors that affect the identification of surgeries with high cancellation risk. This study offers methodology and insights for identifying the key experimental factors for identifying surgery cancellations, and it is helpful to further research on machine learning-based identification of surgeries with high cancellation risk.

Clinical decision support system, using expert consensus-derived logic and natural language processing, decreased sedation-type order errors for patients undergoing endoscopy

OBJECTIVE

Determination of appropriate endoscopy sedation strategy is an important preprocedural consideration. To address manual workflow gaps that lead to sedation-type order errors at our institution, we designed and implemented a clinical decision support system (CDSS) to review orders for patients undergoing outpatient endoscopy.

MATERIALS AND METHODS

The CDSS was developed and implemented by an expert panel using an agile approach. The CDSS queried patient-specific historical endoscopy records and applied expert consensus-derived logic and natural language processing to identify possible sedation order errors for human review. A retrospective analysis was conducted to evaluate impact, comparing 4-month pre-pilot and 12-month pilot periods.

RESULTS

22 755 endoscopy cases were included (pre-pilot 6434 cases, pilot 16 321 cases). The CDSS decreased the sedation-type order error rate on day of endoscopy (pre-pilot 0.39%, pilot 0.037%, Odds Ratio = 0.094, P-value < 1e-8). There was no difference in background prevalence of erroneous orders (pre-pilot 0.39%, pilot 0.34%, P = .54).

DISCUSSION

At our institution, low prevalence and high volume of cases prevented routine manual review to verify sedation order appropriateness. Using a cohort-enrichment strategy, a CDSS was able to reduce number of chart reviews needed per sedation-order error from 296.7 to 3.5, allowing for integration into the existing workflow to intercept rare but important ordering errors.

CONCLUSION

A workflow-integrated CDSS with expert consensus-derived logic rules and natural language processing significantly reduced endoscopy sedation-type order errors on day of endoscopy at our institution.

Complexities, variations, and errors of numbering within clinical notes: the potential impact on information extraction and cohort-identification

BACKGROUND

Numbers and numerical concepts appear frequently in free text clinical notes from electronic health records. Knowledge of the frequent lexical variations of these numerical concepts, and their accurate identification, is important for many information extraction tasks. This paper describes an analysis of the variation in how numbers and numerical concepts are represented in clinical notes.

METHODS

We used an inverted index of approximately 100 million notes to obtain the frequency of various permutations of numbers and numerical concepts, including the use of Roman numerals, numbers spelled as English words, and invalid dates, among others. Overall, twelve types of lexical variants were analyzed.

RESULTS

We found substantial variation in how these concepts were represented in the notes, including multiple data quality issues. We also demonstrate that not considering these variations could have substantial real-world implications for cohort identification tasks, with one case missing > 80% of potential patients.

CONCLUSIONS

Numbering within clinical notes can be variable, and not taking these variations into account could result in missing or inaccurate information for natural language processing and information retrieval tasks.

Machine learning for identification of surgeries with high risks of cancellation

Surgery cancellations waste scarce operative resources and hinder patients' access to operative services. In this study, the Wilcoxon and chi-square tests were used for predictor selection, and three machine learning models - random forest, support vector machine, and XGBoost - were used for the identification of surgeries with high risks of cancellation. The optimal performances of the identification models were as follows: sensitivity - 0.615; specificity - 0.957; positive predictive value - 0.454; negative predictive value - 0.904; accuracy - 0.647; and area under the receiver operating characteristic curve - 0.682. Of the three models, the random forest model achieved the best performance. Thus, the effective identification of surgeries with high risks of cancellation is feasible with stable performance. Models and sampling methods significantly affect the performance of identification. This study is a new application of machine learning for the identification of surgeries with high risks of cancellation and facilitation of surgery resource management.

The impact of computerized physician order entry on prescription orders: A quasi-experimental study in Iran

Background: One way to reduce medical errors associated with physician orders is computerized physician order entry (CPOE) software. This study was conducted to compare prescription orders between 2 groups before and after CPOE implementation in a hospital. Methods: We conducted a before-after prospective study in 2 intensive care unit (ICU) wards (as intervention and control wards) in the largest tertiary public hospital in South of Iran during 2014 and 2016. All prescription orders were validated by a clinical pharmacist and an ICU physician. The rates of ordering the errors in medical orders were compared before (manual ordering) and after implementation of the CPOE. A standard checklist was used for data collection. For the data analysis, SPSS Version 21, descriptive statistics, and analytical tests such as McNemar, chi-square, and logistic regression were used. Results: The CPOE significantly decreased 2 types of errors, illegible orders and lack of writing the drug form, in the intervention ward compared to the control ward (p< 0.05); however, the 2 errors increased due to the defect in the CPOE (p< 0.001). The use of CPOE decreased the prescription errors from 19% to 3% (p= 0.001), However, no differences were observed in the control ward (p<0.05). In addition, more errors occurred in the morning shift (p< 0.001). Conclusion: In general, the use of CPOE significantly reduced the prescription errors. Nonetheless, more caution should be exercised in the use of this system, and its deficiencies should be resolved. Furthermore, it is recommended that CPOE be used to improve the quality of delivered services in hospitals.

Decreasing Prescribing Errors During Pediatric Emergencies: A Randomized Simulation Trial

OBJECTIVE

To evaluate whether a clinical aid providing precalculated medication doses decreases prescribing errors among residents during pediatric simulated cardiopulmonary arrest and anaphylaxis.

METHODS

A crossover randomized trial was conducted in a tertiary care hospital simulation center with residents rotating in the pediatric emergency department. The intervention was a reference book providing weight-based precalculated doses. The control group used a card providing milligram-per-kilogram doses. The primary outcome was the presence of a prescribing error, defined as a dose varying by ≥20% from the recommended dose or by incorrect route. Residents were involved in 2 sets of paired scenarios and were their own control group. Primary analysis was the difference in mean prescribing error proportions between both groups.

RESULTS

Forty residents prescribed 1507 medications or defibrillations during 160 scenarios. The numbers of prescribing errors per 100 bolus medications or defibrillations were 5.1 (39 out of 762) and 7.5 (56 out of 745) for the intervention and control, respectively, a difference of 2.4 (95% confidence interval [CI], -0.1 to 5.0). However, the intervention was highly associated with lower risk of 10-fold error for bolus medications (odds ratio 0.27; 95% CI, 0.10 to 0.70). For medications administered by infusion, prescribing errors occurred in 3 out of 76 (4%) scenarios in the intervention group and 13 out of 76 (22.4%) in the control group, a difference of 13% (95% CI, 3 to 23).

CONCLUSIONS

A clinical aid providing precalculated medication doses was not associated with a decrease in overall prescribing error rates but was highly associated with a lower risk of 10-fold error for bolus medications and for medications administered by continuous infusion.