Factors Influencing Problem List Use in Electronic Health Records-Application of the Unified Theory of Acceptance and Use of Technology

Impact of electronic medical record utilization on obesity screening and intervention for obese patients with endometrial cancer

OBJECTIVE

To identify the prevalence of obesity documented within the electronic medical record problem list.

METHODS

We conducted a retrospective cohort study of adult patients with obesity and endometrial cancer receiving care from January 2018 to March 2021 at a single institution. Obesity intervention was defined as receipt of at least one of the following: referral to weight loss clinic, referral to a nutritionist, completion of obesity intervention tab, or documentation of weight loss counseling. Our secondary objectives were to (1) identify the prevalence of completed obesity interventions, (2) identify the number of patients who have achieved weight loss since their initial visit, and (3) identify covariates associated with presence of obesity on problem list, completion of obesity interventions, and weight loss.

RESULTS

We identified 372 patients who met inclusion criteria. Of eligible patients, 202 (54%) had obesity documented on their problem list and 171 (46%) completed at least one obesity intervention. Within our cohort, 195 (52%) patients achieved weight loss from diagnosis or initial clinical encounter at our institution to most recent clinical encounter with median weight loss of 3.9 kg (IQR 1.5-8.0). In the multivariable logistic regressions, patients with obesity on the problem list were approximately twice as likely to have completion of obesity intervention (OR 1.91, 95% CI 1.09, 3.35, p=0.024). Although presence of obesity on the problem list was not associated with weight loss, completion of health maintenance obesity intervention tab in the electronic medical record (Epic) was associated with weight loss (OR 2.77, 95% CI 1.11, 6.89, p=0.03).

CONCLUSIONS

Only half of obese endometrial cancer patients had documentation of obesity within the electronic medical record problem list. The electronic medical record could be leveraged to achieve compliance with weight loss interventions. Further investigation on how the electronic medical record can be optimized to help patients achieve weight loss is needed.

Evaluation of SNOMED CT Grouper Accuracy and Coverage in Organizing the Electronic Health Record Problem List by Clinical System: Observational Study

Background

The problem list (PL) is a repository of diagnoses for patients' medical conditions and health-related issues. Unfortunately, over time, our PLs have become overloaded with duplications, conflicting entries, and no-longer-valid diagnoses. The lack of a standardized structure for review adds to the challenges of clinical use. Previously, our default electronic health record (EHR) organized the PL primarily via alphabetization, with other options available, for example, organization by clinical systems or priority settings. The system's PL was built with limited groupers, resulting in many diagnoses that were inconsistent with the expected clinical systems or not associated with any clinical systems at all. As a consequence of these limited EHR configuration options, our PL organization has poorly supported clinical use over time, particularly as the number of diagnoses on the PL has increased.

Objective

We aimed to measure the accuracy of sorting PL diagnoses into PL system groupers based on Systematized Nomenclature of Medicine Clinical Terms (SNOMED CT) concept groupers implemented in our EHR.

Methods

We transformed and developed 21 system- or condition-based groupers, using 1211 SNOMED CT hierarchal concepts refined with Boolean logic, to reorganize the PL in our EHR. To evaluate the clinical utility of our new groupers, we extracted all diagnoses on the PLs from a convenience sample of 50 patients with 3 or more encounters in the previous year. To provide a spectrum of clinical diagnoses, we included patients from all ages and divided them by sex in a deidentified format. Two physicians independently determined whether each diagnosis was correctly attributed to the expected clinical system grouper. Discrepancies were discussed, and if no consensus was reached, they were adjudicated by a third physician. Descriptive statistics and Cohen κ statistics for interrater reliability were calculated.

Results

Our 50-patient sample had a total of 869 diagnoses (range 4-59; median 12, IQR 9-24). The reviewers initially agreed on 821 system attributions. Of the remaining 48 items, 16 required adjudication with the tie-breaking third physician. The calculated κ statistic was 0.7. The PL groupers appropriately associated diagnoses to the expected clinical system with a sensitivity of 97.6%, a specificity of 58.7%, a positive predictive value of 96.8%, and an F1-score of 0.972.

Conclusions

We found that PL organization by clinical specialty or condition using SNOMED CT concept groupers accurately reflects clinical systems. Our system groupers were subsequently adopted by our vendor EHR in their foundation system for PL organization.

Correctly structured problem lists lead to better and faster clinical decision-making in electronic health records compared to non-curated problem lists: A single-blinded crossover randomized controlled trial

BACKGROUND

Correctly structured problem lists in electronic health records (EHRs) offer major benefits to patient care. Without structured lists, diagnosis information is often scatteredly documented in free text, which may contribute to errors and inefficient information retrieval. This study aims to assess whether EHRs with correctly structured problem lists result in better and faster clinical decision-making compared to non-curated problem lists.

METHODS

Two versions of two patient records (A and B) were created in an EHR training environment: one version included diagnosis information structured and coded on the problem list ("correctly structured problem list"), the other version had missing problem list diagnoses and diagnosis information partly documented in free text ("non-curated problem list"). In this single-blinded crossover randomized controlled trial, healthcare providers, who can prescribe medications, from two Dutch university medical center locations first evaluated a randomized version of patient A, then B. Participants were asked to motivate their answer to two medication prescription questions. One (test) question required information similarly presented in both record versions. The second (comparison) question required information documented on problem lists and/or in notes. The primary outcome measure was the correctness of the motivated answer to the comparison question. Secondary outcome measure was the time to answer and motivate both questions correctly.

RESULTS

As planned, 160 participants enrolled. Two were excluded for not meeting inclusion criteria. Correctly structured problem lists increased providers' ability to answer the comparison question correctly (56.3 % versus 33.5 %, McNemar odds ratio 2.80 (1.65-4.93) 95 %-CI). Median time to answer both questions correctly was significantly lower for EHRs with correctly structured problem lists (Wilcoxon-signed-rank test p = 0.00002, with incorrect answers coded equally at slowest time).

CONCLUSIONS

Correctly structured problem lists lead to better and faster clinical decision-making. Increased structured problem lists usage may be warranted for which implementation policies should be developed.

Use of the patient portal among older adults with diagnosed dementia and their care partners

INTRODUCTION

Care partners are at the forefront of dementia care, yet little is known about patient portal use in the context of dementia diagnosis.

METHODS

We conducted an observational cohort study of date/time-stamped patient portal use for a 5-year period (October 3, 2017-October 2, 2022) at an academic health system. The cohort consisted of 3170 patients ages 65+ with diagnosed dementia with 2+ visits within 24 months. Message authorship was determined by manual review of 970 threads involving 3065 messages for 279 patients.

RESULTS

Most (71.20%) older adults with diagnosed dementia were registered portal users but far fewer (10.41%) had a registered care partner with shared access. Care partners authored most (612/970, 63.09%) message threads, overwhelmingly using patient identity credentials (271/279, 97.13%).

DISCUSSION

The patient portal is used by persons with dementia and their care partners. Organizational efforts that facilitate shared access may benefit the support of persons with dementia and their care partners. Highlights Patient portal registration and use has been increasing among persons with diagnosed dementia. Two thirds of secure messages from portal accounts of patients with diagnosed dementia were identified as being authored by care partners, primarily using patient login credentials. Care partners who accessed the patient portal using their own identity credentials through shared access demonstrate similar levels of activity to patients without dementia. Organizational initiatives should recognize and support the needs of persons with dementia and their care partners by encouraging awareness, registration, and use of proper identity credentials, including shared, or proxy, portal access.

Effective and feasible interventions to improve structured EHR data registration and exchange: A concept mapping approach and exploration of practical examples in the Netherlands

BACKGROUND

Data in Electronic Health Records (EHRs) is often poorly structured and standardized, which hampers data reuse. Research described some examples of interventions to increase and improve structured and standardized data, such as guidelines and policies, training and user friendly EHR interfaces. However, little is known about the translation of this knowledge into practical solutions. Our study aimed to specify the most effective and feasible interventions that enable better structured and standardized EHR data registration and described practical examples of successfully implemented interventions.

METHODS

A concept mapping approach was used to determine feasible interventions that were considered to be effective or have been successfully implemented in Dutch hospitals. A focus group was held with Chief Medical Information Officers and Chief Nursing Information Officers. After interventions were determined, multidimensional scaling and cluster analysis were performed to categorize sorted interventions using Groupwisdom™, an online tool for concept mapping. Results are presented as Go-Zone plots and cluster maps. Following, semi-structured interviews were conducted to describe practical examples of successful interventions.

RESULTS

Interventions were classified into seven clusters ranked from highest to lowest perceived effectiveness: (1) education on usefulness and need; (2) strategic and (3) tactical organizational policies; (4) national policy; (5) monitoring and adjusting data (6) structure of and support from the EHR and (7) support in the registration process (EHR independent). Interviewees emphasized the following interventions proven successful in their practice: an enthusiastic ambassador per specialty who is responsible for educating peers by increasing awareness of the direct benefit of structured and standardized data registration; dashboards for continuous feedback on data quality; and EHR functionalities that support (automating) the registration process.

CONCLUSIONS

Our study provided a list of effective and feasible interventions including practical examples of interventions that have been successful. Organizations should continue to share their best practices to learn from and attempted interventions to prevent implementation of ineffective interventions.

Use of an open-source electronic health record to establish a "virtual hospital": A tale of two curricula

BACKGROUND

The electronic health record (EHR) is central to medical informatics. Its use is also recognized as an important skill for future clinicians. Typically, medical students' first exposure to an EHR is when they start their clinical internships, and medical informatics students may or may not get experience with an EHR before graduation. We describe the process of implementing an open-source EHR in two curricula: Medicine and Medical informatics. For medical students, the primary goals were to allow students to practice analyzing information from the EHR, creating therapeutic plans, and communicating with their colleagues via the EHR before they start their first clinical rotations. For medical informatics students, the primary goal was to give students hands-on experience with creating decision support in an EHR.

APPROACH

We used the OpenMRS electronic health record with a custom decision support module based on Arden Syntax. Medical students needed a secure, stable environment to practice medical reasoning. Medical informatics students needed a more isolated system to experiment with the EHR's internal configuration. Both student groups needed synthetic patient cases that were realistic, but in different aspects. For medical students, it is essential that these cases are clinically consistent, and events unfold in a logical order. By contrast, synthetic data for medical informatics students should mimic the data quality problems found in real patient data.

OUTCOMES

Medical informatics students show more mature reasoning about data quality issues and workflow integration than prior to using the EHR. Comments on both course evaluations have been positive, including comments on how working with a real-world EHR provides a realistic experience.

CONCLUSION

The open-source EHR OpenMRS has proven to be a valuable addition to both the medicine and medical informatics curriculum. Both sets of students experience use of the EHR as giving them valuable, realistic learning experiences.

Accuracy of the electronic health record's problem list in describing multimorbidity in patients with heart failure in the emergency department

Patients with heart failure (HF) often suffer from multimorbidity. Rapid assessment of multimorbidity is important for minimizing the risk of harmful drug-disease and drug-drug interactions. We assessed the accuracy of using the electronic health record (EHR) problem list to identify comorbid conditions among patients with chronic HF in the emergency department (ED). A retrospective chart review study was performed on a random sample of 200 patients age ≥65 years with a diagnosis of HF presenting to an academic ED in 2019. We assessed participant chronic conditions using: (1) structured chart review (gold standard) and (2) an EHR-based algorithm using the problem list. Chronic conditions were classified into 37 disease domains using the Agency for Healthcare Research Quality's Elixhauser Comorbidity Software. For each disease domain, we report the sensitivity, specificity, positive predictive value, and negative predictive of using an EHR-based algorithm. We calculated the intra-class correlation coefficient (ICC) to assess overall agreement on Elixhauser domain count between chart review and problem list. Patients with HF had a mean of 5.4 chronic conditions (SD 2.1) in the chart review and a mean of 4.1 chronic conditions (SD 2.1) in the EHR-based problem list. The five most prevalent domains were uncomplicated hypertension (90%), obesity (42%), chronic pulmonary disease (38%), deficiency anemias (33%), and diabetes with chronic complications (30.5%). The positive predictive value and negative predictive value of using the EHR-based problem list was greater than 90% for 24/37 and 32/37 disease domains, respectively. The EHR-based problem list correctly identified 3.7 domains per patient and misclassified 2.0 domains per patient. Overall, the ICC in comparing Elixhauser domain count was 0.77 (95% CI: 0.71-0.82). The EHR-based problem list captures multimorbidity with moderate-to-good accuracy in patient with HF in the ED.

Inaccurate recording of routinely collected data items influences identification of COVID-19 patients

Background

During the Coronavirus disease 2019 (COVID-19) pandemic it became apparent that it is difficult to extract standardized Electronic Health Record (EHR) data for secondary purposes like public health decision-making. Accurate recording of, for example, standardized diagnosis codes and test results is required to identify all COVID-19 patients. This study aimed to investigate if specific combinations of routinely collected data items for COVID-19 can be used to identify an accurate set of intensive care unit (ICU)-admitted COVID-19 patients.

Methods

The following routinely collected EHR data items to identify COVID-19 patients were evaluated: positive reverse transcription polymerase chain reaction (RT-PCR) test results; problem list codes for COVID-19 registered by healthcare professionals and COVID-19 infection labels. COVID-19 codes registered by clinical coders retrospectively after discharge were also evaluated. A gold standard dataset was created by evaluating two datasets of suspected and confirmed COVID-19-patients admitted to the ICU at a Dutch university hospital between February 2020 and December 2020, of which one set was manually maintained by intensivists and one set was extracted from the EHR by a research data management department. Patients were labeled ‘COVID-19′ if their EHR record showed diagnosing COVID-19 during or right before an ICU-admission. Patients were labeled ‘non-COVID-19′ if the record indicated no COVID-19, exclusion or only suspicion during or right before an ICU-admission or if COVID-19 was diagnosed and cured during non-ICU episodes of the hospitalization in which an ICU-admission took place. Performance was determined for 37 queries including real-time and retrospective data items. We used the F₁ score, which is the harmonic mean between precision and recall. The gold standard dataset was split into one subset including admissions between February and April and one subset including admissions between May and December to determine accuracy differences.

Results

The total dataset consisted of 402 patients: 196 ‘COVID-19′ and 206 ‘non-COVID-19′ patients. F₁ scores of search queries including EHR data items that can be extracted real-time ranged between 0.68 and 0.97 and for search queries including the data item that was retrospectively registered by clinical coders F₁ scores ranged between 0.73 and 0.99. F₁ scores showed no clear pattern in variability between the two time periods.

Conclusions

Our study showed that one cannot rely on individual routinely collected data items such as coded COVID-19 on problem lists to identify all COVID-19 patients. If information is not required real-time, medical coding from clinical coders is most reliable. Researchers should be transparent about their methods used to extract data. To maximize the ability to completely identify all COVID-19 cases alerts for inconsistent data and policies for standardized data capture could enable reliable data reuse.

Reporting Outcomes of Pediatric Intensive Care Unit Patients to Referring Physicians via an Electronic Health Record-Based Feedback System

BACKGROUND

 Many critically ill children are initially evaluated in front-line settings by clinicians with variable pediatric training before they are transferred to a pediatric intensive care unit (PICU). Because clinicians learn from past performance, communicating outcomes of patients back to front-line clinicians who provide pediatric emergency care could be valuable; however, referring clinicians do not consistently receive this important feedback.

OBJECTIVES

 Our aim was to determine the feasibility, usability, and clinical relevance of a semiautomated electronic health record (EHR)-supported system developed at a single institution to deliver timely and relevant PICU patient outcome feedback to referring emergency department (ED) physicians.

METHODS

 Guided by the Health Information Technology Safety Framework, we iteratively designed, implemented, and evaluated a semiautomated electronic feedback system leveraging the EHR in one institution. After conducting interviews and focus groups with stakeholders to understand the PICU-ED health care work system, we designed the EHR-supported feedback system by translating stakeholder, organizational, and usability objectives into feedback process and report requirements. Over 6 months, we completed three cycles of implementation and evaluation, wherein we analyzed EHR access logs, reviewed feedback reports sent, performed usability testing, and conducted physician interviews to determine the system's feasibility, usability, and clinical relevance.

RESULTS

 The EHR-supported feedback process is feasible with timely delivery and receipt of feedback reports. Usability testing revealed excellent Systems Usability Scale scores. According to physicians, the process was well-integrated into their clinical workflows and conferred minimal additional workload. Physicians also indicated that delivering and receiving consistent feedback was relevant to their clinical practice.

CONCLUSION

 An EHR-supported system to deliver timely and relevant PICU patient outcome feedback to referring ED physicians was feasible, usable, and important to physicians. Future work is needed to evaluate impact on clinical practice and patient outcomes and to investigate applicability to other clinical settings involved in similar care transitions.

Comparing ascertainment of chronic condition status with problem lists versus encounter diagnoses from electronic health records

OBJECTIVE

To assess and compare electronic health record (EHR) documentation of chronic disease in problem lists and encounter diagnosis records among Community Health Center (CHC) patients.

MATERIALS AND METHODS

We assessed patient EHR data in a large clinical research network during 2012-2019. We included CHCs who provided outpatient, older adult primary care to patients age ≥45 years, with ≥2 office visits during the study. Our study sample included 1 180 290 patients from 545 CHCs across 22 states. We used diagnosis codes from 39 Chronic Condition Warehouse algorithms to identify chronic conditions from encounter diagnoses only and compared against problem list records. We measured correspondence including agreement, kappa, prevalence index, bias index, and prevalence-adjusted bias-adjusted kappa.

RESULTS

Overlap of encounter diagnosis and problem list ascertainment was 59.4% among chronic conditions identified, with 12.2% of conditions identified only in encounters and 28.4% identified only in problem lists. Rates of coidentification varied by condition from 7.1% to 84.4%. Greatest agreement was found in diabetes (84.4%), HIV (78.1%), and hypertension (74.7%). Sixteen conditions had <50% agreement, including cancers and substance use disorders. Overlap for mental health conditions ranged from 47.4% for anxiety to 59.8% for depression.

DISCUSSION

Agreement between the 2 sources varied substantially. Conditions requiring regular management in primary care settings may have a higher agreement than those diagnosed and treated in specialty care.

CONCLUSION

Relying on EHR encounter data to identify chronic conditions without reference to patient problem lists may under-capture conditions among CHC patients in the United States.

Demand for Mobile Health in Developing Countries During COVID-19: Vietnamese's Perspectives from Different Age Groups and Health Conditions

BACKGROUND

Vietnam's economy and intellectual standards have witnessed significant development, improving conditions for residents to acquire novel mHealth applications. Additionally, the outbreak of the COVID-19 pandemic has influenced Vietnamese awareness of healthcare; however, previous studies have only been clinician-centered rather than customer-centered.

METHODS

This study addresses this literature gap by interviewing 50 Vietnamese participants grouped by age, namely Generation X, Generation Y, and Generation Z, and health conditions, namely whether participants or family members have chronic illness. The study utilized semi-structured and in-depth interviews to collect the data and used thematic analysis to analyze the data under the unified theory of acceptance and use of technology framework.

RESULTS

Most participants were willing to adopt this technology and demanded a convenient and user-friendly one-stop-shop solution, endorsements from credible and authoritative sources, and professional customer services. However, each group also had distinctive demands and behaviors.

CONCLUSION

This study contributes theoretically by providing context-rich demand for Vietnamese customers across three generations and healthcare conditions during the COVID-19 pandemic and comparing their behavior with pre-COVID literature. While this research provides helpful information for potential app developers, this study also suggests that mHealth developers and policymakers should pay more attention to the differences in the demand of age groups and health conditions.

Building a Shared, Scalable, and Sustainable Source for the Problem-Oriented Medical Record: Developmental Study

Background

Since the creation of the problem-oriented medical record, the building of problem lists has been the focus of many studies. To date, this issue is not well resolved, and building an appropriate contextualized problem list is still a challenge.

Objective

This paper aims to present the process of building a shared multipurpose common problem list at the Geneva University Hospitals. This list aims to bridge the gap between clinicians’ language expressed in free text and secondary uses requiring structured information.

Methods

We focused on the needs of clinicians by building a list of uniquely identified expressions to support their daily activities. In the second stage, these expressions were connected to additional information to build a complex graph of information. A list of 45,946 expressions manually extracted from clinical documents was manually curated and encoded in multiple semantic dimensions, such as International Classification of Diseases, 10th revision; International Classification of Primary Care 2nd edition; Systematized Nomenclature of Medicine Clinical Terms; or dimensions dictated by specific usages, such as identifying expressions specific to a domain, a gender, or an intervention. The list was progressively deployed for clinicians with an iterative process of quality control, maintenance, and improvements, including the addition of new expressions or dimensions for specific needs. The problem management of the electronic health record allowed the measurement and correction of encoding based on real-world use.

Results

The list was deployed in production in January 2017 and was regularly updated and deployed in new divisions of the hospital. Over 4 years, 684,102 problems were created using the list. The proportion of free-text entries decreased progressively from 37.47% (8321/22,206) in December 2017 to 18.38% (4547/24,738) in December 2020. In the last version of the list, over 14 dimensions were mapped to expressions, among which 5 were international classifications and 8 were other classifications for specific uses. The list became a central axis in the electronic health record, being used for many different purposes linked to care, such as surgical planning or emergency wards, or in research, for various predictions using machine learning techniques.

Conclusions

This study breaks with common approaches primarily by focusing on real clinicians’ language when expressing patients’ problems and secondarily by mapping whatever is required, including controlled vocabularies to answer specific needs. This approach improves the quality of the expression of patients’ problems while allowing the building of as many structured dimensions as needed to convey semantics according to specific contexts. The method is shown to be scalable, sustainable, and efficient at hiding the complexity of semantics or the burden of constraint-structured problem list entry for clinicians. Ongoing work is analyzing the impact of this approach on how clinicians express patients’ problems.

Factors affecting the manual linking of clinical progress notes to problems in daily clinical practice: A retrospective quantitative analysis and cross sectional survey

This cross sectional study examines how patient characteristics, doctor characteristics, and doctors' education and attitudes affect the extent to which doctors link progress notes to clinical problems. The independent effects of patient characteristics on the linking of notes was examined with a mixed model logistic regression. The effects of doctor characteristics and doctors' education and attitudes on the link ratio was analyzed with univariate analysis of variance. A survey was used to obtain arguments and attitudes on linking notes. For "patient characteristics", the odds of linking increased with an increase in the number of problems or hospital days, decreased, with an increase in the number of involved doctors, medical specialties or the number of notes. For "doctor characteristics", the link ratio increased with more work experience. For "doctors' education and attitudes", the link ratio increased with more familiarity in linking notes and belief in the added value of problem oriented charting. "Overview" was the most cited reason for linking; "I don't know how" the most cited reason for not linking. There is a huge variation within and between all disciplines. Important arguments, for and against, are found. Recommendations for policymakers and medical leadership are given to maximize the benefits.

Human Factors and Organizational Issues in Health Informatics: Innovations and Opportunities

OBJECTIVE

Human factors and ergonomics (HF/E) frameworks and methods are becoming embedded in the health informatics community. There is now broad recognition that health informatics tools must account for the diverse needs, characteristics, and abilities of end users, as well as their context of use. The objective of this review is to synthesize the current nature and scope of HF/E integration into the health informatics community.

METHODS

Because the focus of this synthesis is on understanding the current integration of the HF/E and health informatics research communities, we manually reviewed all manuscripts published in primary HF/E and health informatics journals during 2020.

RESULTS

HF/E-focused health informatics studies included in this synthesis focused heavily on EHR customizations, specifically clinical decision support customizations and customized data displays, and on mobile health innovations. While HF/E methods aimed to jointly improve end user safety, performance, and satisfaction, most HF/E-focused health informatics studies measured only end user satisfaction.

CONCLUSION

HF/E-focused health informatics researchers need to identify and communicate methodological standards specific to health informatics, to better synthesize findings across resource intensive HF/E-focused health informatics studies. Important gaps in the HF/E design and evaluation process should be addressed in future work, including support for technology development platforms and training programs so that health informatics designers are as diverse as end users.

Contextual property detection in Dutch diagnosis descriptions for uncertainty, laterality and temporality

Background

Accurate, coded problem lists are valuable for data reuse, including clinical decision support and research. However, healthcare providers frequently modify coded diagnoses by including or removing common contextual properties in free-text diagnosis descriptions: uncertainty (suspected glaucoma), laterality (left glaucoma) and temporality (glaucoma 2002). These contextual properties could cause a difference in meaning between underlying diagnosis codes and modified descriptions, inhibiting data reuse. We therefore aimed to develop and evaluate an algorithm to identify these contextual properties.

Methods

A rule-based algorithm called UnLaTem (Uncertainty, Laterality, Temporality) was developed using a single-center dataset, including 288,935 diagnosis descriptions, of which 73,280 (25.4%) were modified by healthcare providers. Internal validation of the algorithm was conducted with an independent sample of 980 unique records. A second validation of the algorithm was conducted with 996 records from a Dutch multicenter dataset including 175,210 modified descriptions of five hospitals. Two researchers independently annotated the two validation samples. Performance of the algorithm was determined using means of the recall and precision of the validation samples. The algorithm was applied to the multicenter dataset to determine the actual prevalence of the contextual properties within the modified descriptions per specialty.

Results

For the single-center dataset recall (and precision) for removal of uncertainty, uncertainty, laterality and temporality respectively were 100 (60.0), 99.1 (89.9), 100 (97.3) and 97.6 (97.6). For the multicenter dataset for removal of uncertainty, uncertainty, laterality and temporality it was 57.1 (88.9), 86.3 (88.9), 99.7 (93.5) and 96.8 (90.1). Within the modified descriptions of the multicenter dataset, 1.3% contained removal of uncertainty, 9.9% uncertainty, 31.4% laterality and 9.8% temporality.

Conclusions

We successfully developed a rule-based algorithm named UnLaTem to identify contextual properties in Dutch modified diagnosis descriptions. UnLaTem could be extended with more trigger terms, new rules and the recognition of term order to increase the performance even further. The algorithm’s rules are available as additional file 2. Implementing UnLaTem in Dutch hospital systems can improve precision of information retrieval and extraction from diagnosis descriptions, which can be used for data reuse purposes such as decision support and research.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12911-021-01477-y.

Preliminary Research on the Social Attitudes toward AI’s Involvement in Christian Education in Vietnam: Promoting AI Technology for Religious Education

Artificial intelligence innovations, such as chatbots and specialized education suggestion tools, provide potential interactive and on-demand pedagogical engagement between non-Christians and Christians with Christianity. However, there is little empirical research on the readiness, acceptance, and adoption of religious education involvement of AI in a secular state such as Vietnam. This research addresses the literature gap by providing an entrepreneurial analysis and customer perspectives on the ideas of AI involvement in religious education. Specifically, the study explores whether the Vietnamese across different ages accept and have enough skills to adopt AI in Christian education innovation. The interview sample is 32 participants, selected based on their religious orientation (Christians and non-Christians) and age (Generation X, Generation Y, and Generation Z). Most respondents are open to AI application in religious education except for Church personnel. However, only Generation Z are fully prepared to adopt this innovation. Theoretically, the research customizes the Unified Theory of Acceptance and Use of Technology model into religious innovation context. Practically, this research acts as market research on the demand for AI’s religious innovation in Vietnam, an insight for future religious tech entrepreneurs.