Detection and characterization of usability problems in structured data entry interfaces in dentistry

Using precision periodontal health care chart (PPHCC) to improve periodontal health

OBJECTIVES

Developing a Precision Periodontal Health Care Chart (PPHCC) in the electronic dental record (EDR) system and evaluating its clinical usability and effects on clinical outcomes.

MATERIALS AND METHODS

A survey with ten questions based on the System Usability Scale (SUS) and six questions about assessing clinical impact was used to evaluate the satisfaction of periodontitis patients and care providers with PPHCC before and after non-surgical periodontal therapy (NSPT). The clinical outcomes, including probing depth (PD), interdental clinical attachment loss (CAL), and bleeding on probing (BOP), in patients who used PPHCC (PC) were compared to those in patients without using PPHCC (control). The associations between risk assessments included in PPHCC and clinical outcomes of NSPT were also analyzed.

RESULTS

The mean scores of SUS questions at the initial periodontal examination were 74.26 ± 18.89 (n = 37) for patients and 88.31 ± 14.14 (n = 37) for care providers. The mean scores of SUS questions at re-evaluation were 74.84 ± 17.78 (n = 16) for patients and 89.63 ± 13.48 (n = 20) for care providers. The changes in the percentages of teeth with interdental CAL 1-2 mm (p = 0.019) and CAL 3-4 mm (p = 0.026) at the re-evaluation visit were significantly different between the PC and control groups, but the other parameters were not.

CONCLUSIONS

Both patients and care providers were satisfied with using PPHCC in the clinic. However, the short-term clinical outcomes in the PC group were similar to those in the control group.

CLINICAL RELEVANCE

PPHCC, as a tool for delivering clinical and educational information, can motivate patients to control periodontitis and assist clinicians in making a personalized treatment plan.

Evaluating and improving the usability of a mHealth platform to assess postoperative dental pain

Objectives

The use of interactive mobile health (mHealth) applications to monitor patient-reported postoperative pain outcomes is an emerging area in dentistry that requires further exploration. This study aimed to evaluate and improve the usability of an existing mHealth application.

Materials and methods

The usability of the application was assessed iteratively using a 3-phase approach, including a rapid cognitive walkthrough (Phase I), lab-based usability testing (Phase II), and in situ pilot testing (Phase III). The study team conducted Phase I, while providers and patients participated in Phase II and III.

Results

The rapid cognitive walkthrough identified 23 potential issues that could negatively impact user experience, with the majority classified as system issues. The lab-based usability testing yielded 141 usability issues.; 43% encountered by patients and 57% by dentists. Usability problems encountered during pilot testing included undelivered messages due to mobile phone carrier and service-related issues, errors in patients' phone number data entry, and problems in provider training.

Discussion

Through collaborative and iterative work with the vendor, usability issues were addressed before launching a trial to assess its efficacy.

Conclusion

The usability of the mHealth application for postoperative dental pain was remarkably improved by the iterative analysis and interdisciplinary collaboration.

Do Cone Beam CT Picture Archiving and Communication Systems Viewer Interfaces Meet the Expectations of Dental Professionals From a Usability Perspective?

Background Cone beam computed tomography (CBCT) has revolutionized dental and maxillofacial imaging by providing high-resolution 3D visualizations, essential for accurate diagnosis and treatment planning. Despite its clinical advancements, the usability of CBCT viewer interfaces, which play a crucial role in the effective interpretation of imaging data, remains a critical concern. Objective This study aims to evaluate the usability of a CBCT viewer interface in clinical settings, focusing on the effectiveness, efficiency, and satisfaction perspectives, to identify potential areas for improvement. Methods Twenty-two participants (N=22) were assigned the task of locating the mental foramen in a mandible dataset, selected randomly, using the multiplanar reconstruction (MPR) mode of a CBCT viewer interface on a local network. The task's effectiveness was gauged by the completion rate, while efficiency was measured through the duration of the task, the number of mouse clicks, and the cursor's path in both pixels and meters. Satisfaction or perceived usability was evaluated using the system usability scale (SUS-TR), and computer system usability questionnaire (T-CSUQ), among other scales, with participants also providing open-ended feedback. Demographic characteristics served as classification variables. Results All participants completed the given task. No demography-related differences in efficiency were observed. Perceived usability (SUS-TR: 60.68±19.58, T-CSUQ: 43.63±16.34) was below the industry standards, categorizing the system as a detractor. Commonly reported issues included accessing the MPR window, cursor behavior, and unclear error messages. The mean SUS-TR score negatively correlated with efficiency-related variables (p<0.05 for each). Conclusions The study's findings indicate that the CBCT viewer interface does not fully meet dental professionals' usability expectations, as evidenced by the task's completion rate, efficiency metrics, and below-average usability scores. Despite the successful task completion by all participants, the identified issues in interface design, such as difficulties in accessing the MPR window and unclear error messages, highlight significant areas for improvement. To enhance user satisfaction and efficiency, future developments of CBCT viewer interfaces should incorporate feedback from end-users and prioritize user-friendly design principles.

The relationship between electronic health records user interface features and data quality of patient clinical information: an integrative review

OBJECTIVES

Electronic health records (EHRs) user interfaces (UI) designed for data entry can potentially impact the quality of patient information captured in the EHRs. This review identified and synthesized the literature evidence about the relationship of UI features in EHRs on data quality (DQ).

MATERIALS AND METHODS

We performed an integrative review of research studies by conducting a structured search in 5 databases completed on October 10, 2022. We applied Whittemore & Knafl's methodology to identify literature, extract, and synthesize information, iteratively. We adapted Kmet et al appraisal tool for the quality assessment of the evidence. The research protocol was registered with PROSPERO (CRD42020203998).

RESULTS

Eleven studies met the inclusion criteria. The relationship between 1 or more UI features and 1 or more DQ indicators was examined. UI features were classified into 4 categories: 3 types of data capture aids, and other methods of DQ assessment at the UI. The Weiskopf et al measures were used to assess DQ: completeness (n = 10), correctness (n = 10), and currency (n = 3). UI features such as mandatory fields, templates, and contextual autocomplete improved completeness or correctness or both. Measures of currency were scarce.

DISCUSSION

The paucity of studies on UI features and DQ underscored the limited knowledge in this important area. The UI features examined had both positive and negative effects on DQ. Standardization of data entry and further development of automated algorithmic aids, including adaptive UIs, have great promise for improving DQ. Further research is essential to ensure data captured in our electronic systems are high quality and valid for use in clinical decision-making and other secondary analyses.

Development and Validation of SafeHIT: An Instrument to Assess the Self-Reported Safe Use of Health Information Technology

BACKGROUND

Implementing health information technology (HIT) may cause unintended consequences and safety risks when incorrectly designed and used. Yet, the tools to assess self-reported safe use of HIT are not well established.

OBJECTIVE

This study aims to develop and validate SafeHIT, an instrument to assess self-reported safe use of HIT among health care practitioners.

METHODS

Systematic literature review and a semistructured interview with 31 experts were adopted to generate SafeHIT instrument items. In total, 450 physicians from various departments at three Malaysian public hospitals participated in the questionnaire survey to validate SafeHIT. Exploratory factor analysis and confirmatory factor analysis (CFA) were undertaken to explore the items that best represent a specific construct and to confirm the reliability and validity of the SafeHIT, respectively.

RESULTS

The final SafeHIT consisted of 14 constructs and 58 items in total. The result of the CFA confirmed that all constructs demonstrated adequate convergent and discriminant validity.

CONCLUSION

A reliable and valid theoretically underpinned measure of determinants of safe HIT use behavior has been developed. Understanding external factors that influence safe HIT use is useful for developing targeted interventions that favor the quality and safety of health care.

Natural language processing for clinical notes in dentistry: A systematic review

OBJECTIVE

To identify and synthesise research on applications of natural language processing (NLP) for information extraction and retrieval from clinical notes in dentistry.

MATERIALS AND METHODS

A predefined search strategy was applied in EMBASE, CINAHL and Medline. Studies eligible for inclusion were those that that described, evaluated, or applied NLP to clinical notes containing either human or simulated patient information. Quality of the study design and reporting was independently assessed based on a set of questions derived from relevant tools including CHecklist for critical Appraisal and data extraction for systematic Reviews of prediction Modelling Studies (CHARMS). A narrative synthesis was conducted to present the results.

RESULTS

Of the 17 included studies, 10 developed and evaluated NLP methods and 7 described applications of NLP-based information retrieval methods in dental records. Studies were published between 2015 and 2021, most were missing key details needed for reproducibility, and there was no consistency in design or reporting. The 10 studies developing or evaluating NLP methods used document classification or entity extraction, and 4 compared NLP methods to non-NLP methods. The quality of reporting on NLP studies in dentistry has modestly improved over time.

CONCLUSIONS

Study design heterogeneity and incomplete reporting of studies currently limits our ability to synthesise NLP applications in dental records. Standardisation of reporting and improved connections between NLP methods and applied NLP in dentistry may improve how we can make use of clinical notes from dentistry in population health or decision support systems.

PROTOCOL REGISTRATION

PROSPERO CRD42021227823.

Current state of dental informatics in the field of health information systems: a scoping review

Background

Over the past 50 years, dental informatics has developed significantly in the field of health information systems. Accordingly, several studies have been conducted on standardized clinical coding systems, data capture, and clinical data reuse in dentistry.

Methods

Based on the definition of health information systems, the literature search was divided into three specific sub-searches: “standardized clinical coding systems,” “data capture,” and “reuse of routine patient care data.” PubMed and Web of Science were searched for peer-reviewed articles. The review was conducted following the PRISMA-ScR protocol.

Results

A total of 44 articles were identified for inclusion in the review. Of these, 15 were related to “standardized clinical coding systems,” 15 to “data capture,” and 14 to “reuse of routine patient care data.” Articles related to standardized clinical coding systems focused on the design and/or development of proposed systems, on their evaluation and validation, on their adoption in academic settings, and on user perception. Articles related to data capture addressed the issue of data completeness, evaluated user interfaces and workflow integration, and proposed technical solutions. Finally, articles related to reuse of routine patient care data focused on clinical decision support systems centered on patient care, institutional or population-based health monitoring support systems, and clinical research.

Conclusions

While the development of health information systems, and especially standardized clinical coding systems, has led to significant progress in research and quality measures, most reviewed articles were published in the US. Clinical decision support systems that reuse EDR data have been little studied. Likewise, few studies have examined the working environment of dental practitioners or the pedagogical value of using health information systems in dentistry.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12903-022-02163-9.

What Oncologists Want: Identifying Challenges and Preferences on Diagnosis Data Entry to Reduce EHR-Induced Burden and Improve Clinical Data Quality

PURPOSE

Accurate recording of diagnosis (DX) data in electronic health records (EHRs) is important for clinical practice and learning health care. Previous studies show statistically stable patterns of data entry in EHRs that contribute to inaccurate DX, likely because of a lack of data entry support. We conducted qualitative research to characterize the preferences of oncological care providers on cancer DX data entry in EHRs during clinical practice.

METHODS

We conducted semistructured interviews and focus groups to uncover common themes on DX data entry preferences and barriers to accurate DX recording. Then, we developed a survey questionnaire sent to a cohort of oncologists to verify the generalizability of our initial findings. We constrained our participants to a single specialty and institution to ensure similar clinical backgrounds and clinical experience with a single EHR system.

RESULTS

A total of 12 neuro-oncologists and thoracic oncologists were involved in the interviews and focus groups. The survey developed from these two initial thrusts was distributed to 19 participants yielding a 94.7% survey response rate. Clinicians reported similar user interface experiences, barriers, and dissatisfaction with current DX entry systems including repetitive entry operations, difficulty in finding specific DX options, time-consuming interactions, and the need for workarounds to maintain efficiency. The survey revealed inefficient DX search interfaces and challenging entry processes as core barriers.

CONCLUSION

Oncologists seem to be divided between specific DX data entry and time efficiency because of current interfaces and feel hindered by the burdensome and repetitive nature of EHR data entry. Oncologists' top concern for adopting data entry support interventions is ensuring that it provides significant time-saving benefits and increasing workflow efficiency. Future interventions should account for time efficiency, beyond ensuring data entry effectiveness.

Catch Me if You Can: Acute Events Hidden in Structured Chronic Disease Diagnosis Descriptions Show Detectable Recording Patterns in EHR

Our previous research shows that structured cancer DX description data accuracy varied across electronic health record (EHR) segments (e.g. encounter DX, problem list, etc.). We provide initial evidence corroborating these findings in EHRs from patients with diabetes. We hypothesized that the odds of recording an "uncontrolled diabetes" DX increased after a hemoglobin A1c result above 9% and that this rate would vary across EHR segments. Our statistical models revealed that each DX indicating uncontrolled diabetes was 2.6% more likely to occur post-A1c>9% overall (adj-p=.0005) and 3.9% after controlling for EHR segment (adj-p<.0001). However, odds ratios varied across segments (1.021 Collapse

Key Words Collapse

MESH Headings

• Chronic Disease
• Datasets as Topic
• Diabetes Mellitus/diagnosis
• Electronic Health Records
• Glycated Hemoglobin/analysis
• Humans
• Machine Learning
• Medical Informatics/methods
• Models, Statistical
• Odds Ratio

Collapse

Grants

• UL1 TR001420 NCATS NIH HHS

Collapse

Affiliation(s)

Franck Diaz-Garelli University of North Carolina at Charlotte. Charlotte, NC
Kristin M Lenoir Wake Forest School of Medicine, Winston Salem, NC
Brian J Wells Wake Forest School of Medicine, Winston Salem, NC

Collapse

How to interact with medical terminologies? Formative usability evaluations comparing three approaches for supporting the use of MedDRA by pharmacovigilance specialists

BACKGROUND

Medical terminologies are commonly used in medicine. For instance, to answer a pharmacovigilance question, pharmacovigilance specialists (PVS) search in a pharmacovigilance database for reports in relation to a given drug. To do that, they first need to identify all MedDRA terms that might have been used to code an adverse reaction in the database, but terms may be numerous and difficult to select as they may belong to different parts of the hierarchy. In previous studies, three tools have been developed to help PVS identify and group all relevant MedDRA terms using three different approaches: forms, structured query-builder, and icons. Yet, a poor usability of the tools may increase PVS' workload and reduce their performance. This study aims to evaluate, compare and improve the three tools during two rounds of formative usability evaluation.

METHODS

First, a cognitive walkthrough was performed. Based on the design recommendations obtained from this evaluation, designers made modifications to their tools to improve usability. Once this re-engineering phase completed, six PVS took part in a usability test: difficulties, errors and verbalizations during their interaction with the three tools were collected. Their satisfaction was measured through the System Usability Scale. The design recommendations issued from the tests were used to adapt the tools.

RESULTS

All tools had usability problems related to the lack of guidance in the graphical user interface (e.g., unintuitive labels). In two tools, the use of the SNOMED CT to find MedDRA terms hampered their use because French PVS were not used to it. For the most obvious and common terms, the icons-based interface would appear to be more useful. For the less frequently used MedDRA terms or those distributed in different parts of the hierarchy, the structured query-builder would be preferable thanks to its great power and flexibility. The form-based tool seems to be a compromise.

CONCLUSION

These evaluations made it possible to identify the strengths of each tool but also their weaknesses to address them before further evaluation. Next step is to assess the acceptability of tools and the expressiveness of their results to help identify and group MedDRA terms.

User-centered design principles in the development of clinical research tools

BACKGROUND

Increasing and sustaining the engagement of participants in clinical research studies is a goal for clinical investigators, especially for studies that require long-term or frequent involvement of participants. Technology can be used to reduce barriers to participation by providing multiple options for clinical data entry and form submission. However, electronic systems used in clinical research studies should be user-friendly while also ensuring data quality. Directly involving study participants in evaluating the effectiveness and usability of electronic tools may promote wider adoption, maintain involvement, and increase user satisfaction of the technology. While developers of healthcare applications have incorporated user-centered designs, these methods remain uncommon in the design of clinical study tools such as patient-reported outcome surveys or electronic data capture digital health tools.

METHODS

Our study evaluated whether the clinical research setting may benefit from implementing user-centered design principles. Study participants were recruited to test the web-based form for the Measurement to Understand the Reclassification of Disease of Cabarrus/Kannapolis (MURDOCK) Study Community Translational Population Health Registry and Biorepository that would enable them to complete their study forms electronically. The study enrollment form collects disease history, conditions, smoking status, medications, and other information. The system was initially evaluated by the data management team through traditional user-acceptance testing methods. During the tool evaluation phase, a decision was made to incorporate a small-scale usability study to directly test the system.

RESULTS

Results showed that a majority of participants found the system easy to use. Of the eight required tasks, 75% were completed successfully. Of the 72 heuristics violated, language was the most frequent violation.

CONCLUSION

Our study showed that user-centered usability methods can identify important issues and capture information that can enhance the participant's experience and may improve the quality of study tools.

Factors Influencing Problem List Use in Electronic Health Records-Application of the Unified Theory of Acceptance and Use of Technology

BACKGROUND

Problem-oriented electronic health record (EHR) systems can help physicians to track a patient's status and progress, and organize clinical documentation, which could help improving quality of clinical data and enable data reuse. The problem list is central in a problem-oriented medical record. However, current problem lists remain incomplete because of the lack of end-user training and inaccurate content of underlying terminologies. This leads to modifications of diagnosis code descriptions and use of free-text notes, limiting reuse of data.

OBJECTIVES

We aimed to investigate factors that influence acceptance and actual use of the problem list, and used these to propose recommendations, to increase the value of problem lists for (re)use.

METHODS

Semistructured interviews were conducted with physicians, heads of medical departments, and data quality experts, who were invited through snowball sampling. The interviews were transcribed and coded. Comments were fitted in constructs of the validated framework unified theory of acceptance user technology (UTAUT), and were discussed in terms of facilitators and barriers.

RESULTS

In total, 24 interviews were conducted. We found large variability in attitudes toward problem list use. Barriers included uncertainty about the responsibility for maintaining the problem list and little perceived benefits. Facilitators included the (re)design of policies, improved (peer-to-peer) training to increase motivation, and positive peer feedback and monitoring. Motivation is best increased through sharing benefits relevant in the care process, such as providing overview, timely generation of discharge or referral letters, and reuse of data. Furthermore, content of the underlying terminology should be improved and the problem list should be better presented in the EHR system.

CONCLUSION

To let physicians accept and use the problem list, policies and guidelines should be redesigned, and prioritized by supervising staff. Additionally, peer-to-peer training on the benefits of using the problem list is needed.

Workflow Differences Affect Data Accuracy in Oncologic EHRs: A First Step Toward Detangling the Diagnosis Data Babel

PURPOSE

Diagnosis (DX) information is key to clinical data reuse, yet accessible structured DX data often lack accuracy. Previous research hints at workflow differences in cancer DX entry, but their link to clinical data quality is unclear. We hypothesized that there is a statistically significant relationship between workflow-describing variables and DX data quality.

METHODS

We extracted DX data from encounter and order tables within our electronic health records (EHRs) for a cohort of patients with confirmed brain neoplasms. We built and optimized logistic regressions to predict the odds of fully accurate (ie, correct neoplasm type and anatomic site), inaccurate, and suboptimal (ie, vague) DX entry across clinical workflows. We selected our variables based on correlation strength of each outcome variable.

RESULTS

Both workflow and personnel variables were predictive of DX data quality. For example, a DX entered in departments other than oncology had up to 2.89 times higher odds of being accurate (P < .0001) compared with an oncology department; an outpatient care location had up to 98% fewer odds of being inaccurate (P < .0001), but had 458 times higher odds of being suboptimal (P < .0001) compared with main campus, including the cancer center; and a DX recoded by a physician assistant had 85% fewer odds of being suboptimal (P = .005) compared with those entered by physicians.

CONCLUSION

These results suggest that differences across clinical workflows and the clinical personnel producing EHR data affect clinical data quality. They also suggest that the need for specific structured DX data recording varies across clinical workflows and may be dependent on clinical information needs. Clinicians and researchers reusing oncologic data should consider such heterogeneity when conducting secondary analyses of EHR data.

Recording of diagnoses in public primary oral health care in a retrospective longitudinal observational study in a Finnish town: Underrepresentation of periodontitis diagnoses

OBJECTIVES

This study investigates which oral diagnoses public primary dental care dentists record.

METHODS

An observational register-based retrospective follow-up study was performed in the public primary oral health care of a Finnish town after the dentists were advised to mark the diagnoses in their practices. The rate of recorded diagnoses resulting from visits to the public primary care dentists was studied. The assessed diagnoses were recorded with the 10th revision of the International Classification of Diseases. The distribution of diagnoses was recorded during a 2-year follow-up period.

RESULTS

The most frequent diagnosis groups were dental caries (K02, 38.6%), other diseases of dental hard tissues (K03, 14.9%), diseases of pulp and periapical tissues (K04, 11.4%), periodontal diseases (K05, 9.7%), and different types of bone fractures (S02, 8.1%). Periodontitis was underrepresented.

CONCLUSIONS

In public primary oral health care, there may be difficulties in adequate recording of certain chronic diseases.

Need for diagnostic-centric care in dentistry: A case study from the Marshfield Clinic Health System

BACKGROUND

The study objective was to evaluate the workflow of dental providers who use the existing electronic dental record (EDR) system at a large regional health care system to establish a diagnostic-centric culture as part of their dental practice. A further goal focused on identifying when improvements to the workflow and design of the EDR may be indicated.

METHODS

Dental procedures performed on patients and corresponding International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) diagnoses were retrospectively mined from Marshfield Clinic's enterprise data warehouse. All dental procedures performed were selected and paired with corresponding diagnostic codes documented by dental providers. Frequency of documented diagnosis was further analyzed by characterizing correspondence with their ranking order in the diagnosis column with and without a scroll bar within the EDR user interface (UI). Accuracy of selecting appropriate ICD-9-CM for the corresponding Code on Dental Procedure and Nomenclature (CDT) was checked for 10% (n = 6,187) of the procedure-diagnosis pairs.

RESULTS

Of the 61,511 unique procedures documented using 147 CDTs, 11% (6,914 procedures) had a corresponding "not available" option associated under the diagnoses column, whereas 89% (54,597) of dental procedures were associated with a corresponding ICD-9-CM diagnostic code. Overall tendency of dental providers to select the first or last options from the diagnostic list with a scroll bar was noted. Appropriateness of documenting corresponding ICD-9-CM to CDT procedures indicated 98% accuracy.

CONCLUSION

EDR UI design greatly affected documentation process. Redesigning the EDR UI from the results will increase both the quality and utility of clinical documentation.

Investigating the structure-function relationship using Goldmann V standard automated perimetry where glaucomatous damage is advanced

PURPOSE

To investigate if the structure-function relationship between circumpapillary retinal nerve fibre layer (cpRNFL) thickness and visual field (VF) thresholds is stronger when using the Goldmann V target rather than the Goldman III target where glaucomatous damage is advanced.

METHODS

Optical coherence tomography (OCT) and VF (Humphrey Field Analyzer 24-2 or 30-2) measurements with Goldmann III (SITA standard) and V (full-threshold) targets were carried out in 51 eyes of 51 patients with primary open angle glaucoma. The relationship between cpRNFL thicknesses in supero- and infero-temporal sectors, and VF sensitivity with the Goldmann III or V target was investigated.

RESULT

Visual field sensitivities (dB) both with the Goldmann III target and Goldmann V target showed a floor effect in the structure-function relationship against cpRNFL thickness, at approximately 60 μm. There was no significant relationship between visual field sensitivity measured with the Goldmann V target (dB scale: p = 0.12, 1/Lambert scale: p = 0.40; linear mixed models) and cpRNFL thickness, when corresponding visual field sensitivity, measured with the Goldmann III target, was <20 dB.

CONCLUSION

There was no improvement in the structure-function relationship using the Goldmann V target (full-threshold), compared to using the Goldmann III target (SITA standard), where glaucomatous damage was advanced.

A tale of three subspecialties: Diagnosis recording patterns are internally consistent but Specialty-Dependent

BACKGROUND

Structured diagnosis (DX) are crucial for secondary use of electronic health record (EHR) data. However, they are often suboptimally recorded. Our previous work showed initial evidence of variable DX recording patterns in oncology charts even after biopsy records are available.

OBJECTIVE

We verified this finding's internal and external validity. We hypothesized that this recording pattern would be preserved in a larger cohort of patients for the same disease. We also hypothesized that this effect would vary across subspecialties.

METHODS

We extracted DX data from EHRs of patients treated for brain, lung, and pancreatic neoplasms, identified through clinician-led chart reviews. We used statistical methods (i.e., binomial and mixed model regressions) to test our hypotheses.

RESULTS

We found variable recording patterns in brain neoplasm DX (i.e., larger number of distinct DX-OR = 2.2, P < 0.0001, higher descriptive specificity scores-OR = 1.4, P < 0.0001-and much higher entropy after the BX-OR = 3.8 P = 0.004 and OR = 8.0, P < 0.0001), confirming our initial findings. We also found strikingly different patterns for lung and pancreas DX. Although both seemed to have much lower DX sequence entropy after the BX-OR = 0.198, P = 0.015 and OR = 0.099, P = 0.015, respectively compared to OR = 3.8 P = 0.004). We also found statistically significant differences between the brain dataset and both the lung (P < 0.0001) and pancreas (0.009

CONCLUSION

Our results suggest that disease-specific DX entry patterns exist and are established differently by clinical subspecialty. These differences should be accounted for during clinical data reuse and data quality assessments but also during EHR entry system design to maximize accurate, precise and consistent data entry likelihood.

Collapse

Key Words

• clinical data management
• data quality
• electronic health records
• learning healthcare system
• secondary use of clinical data

Collapse

MESH Headings Collapse

Grants

• K12 GM102773 NIGMS NIH HHS
• UL1 TR001420 NCATS NIH HHS

Collapse

Affiliation(s)

Jose-Franck Diaz-Garelli Wake Forest Baptist Medical Center, Winston Salem, North Carolina, USA
Roy Strowd Wake Forest Baptist Medical Center, Winston Salem, North Carolina, USA
Tamjeed Ahmed Wake Forest Baptist Medical Center, Winston Salem, North Carolina, USA
Brian J Wells Wake Forest Baptist Medical Center, Winston Salem, North Carolina, USA
Rebecca Merrill Wake Forest Baptist Medical Center, Winston Salem, North Carolina, USA
Javier Laurini Wake Forest Baptist Medical Center, Winston Salem, North Carolina, USA
Boris Pasche Wake Forest Baptist Medical Center, Winston Salem, North Carolina, USA
Umit Topaloglu Wake Forest Baptist Medical Center, Winston Salem, North Carolina, USA

Collapse

Lost in Translation: Diagnosis Records Show More Inaccuracies After Biopsy in Oncology Care EHRs

The use of diagnosis (DX) data is crucial to secondary use of electronic health record (EHR) data, yet accessible structured DX data often lack in accuracy. DX descriptions associated with structured DX codes vary even after recording biopsy results; this may indicate poor data quality. We hypothesized that biopsy reports in cancer care charts do not improve intrinsic DX data quality. We analyzed DX data for a manually well-annotated cohort of patients with brain neoplasms. We built statistical models to predict the number of fully-accurate (i.e., correct neoplasm type and anatomical location) and inaccurate DX (i.e. type or location contradicts cohort data) descriptions. We found some evidence of statistically larger numbers of fully-accurate (RR=3.07, p=0.030) but stronger evidence of much larger numbers of inaccurate DX (RR=12.3, p=0.001 and RR=19.6, p<0.0001) after biopsy result recording. Still, 65.9% of all DX records were neither fully-accurate nor fully-inaccurate. These results suggest EHRs must be modified to support more reliable DX data recording and secondary use of EHR data.

Cost Comparison of an On-Premise IT Solution with a Cloud-Based Solution for Electronic Health Records in a Dental School Clinic

Electronic health records (EHRs) are increasingly moving towards cloud-based web environments. While cloud-based EHRs claim substantial benefits at reduced cost, little cost-benefit research exists for dental schools. The aim of this study was to examine the cost-benefits of a cloud-based EHR compared to an on-premise client-server EHR in the University of Michigan School of Dentistry (U-M Dent). Data were collected in 2016 from the U-M Dent cost-benefit comparison of tangible and intangible factors associated with implementing a new EHR, using the Total Cost of Ownership (TCO) framework from EDUCAUSE. The TCO framework assessed three factors: foundational (overarching aspects: three items), qualitative (intangibles: 56 items), and quantitative (actual costs). Stakeholders performed factor grading, and relative assessment scores were derived for each item as well as the overall factor. The cloud-based EHR solution received higher foundational and qualitative factor summary scores. The overall cost of an on-premise solution over a two-year period was approximately $2,000,000 higher than a cloud-based solution. Cloud solutions did not carry any hidden costs, while such costs accounted for 8% (~$540,000) of the overall costs of the on-premise solution. Across the two-year period, both one-time and ongoing costs were higher for the on-premise solution than the cloud-based solution (by 40.5% and 20.5%, respectively). This study found that a cloud-based EHR system in the U-M Dent offered significant cost savings and unique benefits that were not available with the on-premise EHR solution. Based on cost, the U-M Dent has made a case for cloud-based EHR systems.

Usability Evaluation of Electronic Health Record System around Clinical Notes Usage-An Ethnographic Study

Background A substantial gap exists between current Electronic Health Record (EHR) usability and potential optimal usability. One of the fundamental reasons for this discrepancy is poor incorporation of a User-Centered Design (UCD) approach during the Graphical User Interface (GUI) development process. Objective To evaluate usability strengths and weaknesses of two widely implemented EHR GUIs for critical clinical notes usage tasks. Methods Twelve Internal Medicine resident physicians interacting with one of the two EHR systems (System-1 at Location-A and System-2 at Location-B) were observed by two usability evaluators employing an ethnographic approach. User comments and observer findings were analyzed for two critical tasks: (1) clinical notes entry and (2) related information-seeking tasks. Data were analyzed from two standpoints: (1) usability references categorized by usability evaluators as positive, negative, or equivocal and (2) usability impact of each feature measured through a 7-point severity rating scale. Findings were also validated by user responses to a post observation questionnaire. Results For clinical notes entry, System-1 surpassed System-2 with more positive (26% vs. 12%) than negative (12% vs. 34%) usability references. Greatest impact features on EHR usability (severity score pertaining to each feature) for clinical notes entry were: autopopulation (6), screen options (5.5), communication (5), copy pasting (4.5), error prevention (4.5), edit ability (4), and dictation and transcription (3.5). Both systems performed equally well on information-seeking tasks and features with greatest impacts on EHR usability were navigation for notes (7) and others (e.g., looking for ancillary data; 5.5). Ethnographic observations were supported by follow-up questionnaire responses. Conclusion This study provides usability-specific insights to inform future, improved, EHR interface that is better aligned with UCD approach.

Update on Electronic Dental Record and Clinical Computing Adoption Among Dental Practices in the United States

This study sought to re-characterize trends and factors affecting electronic dental record (EDR) and technologies adoption by dental practices and the impact of the Health Information Technology for Economic and Clinical Health (HITECH) act on adoption rates through 2012. A 39-question survey was disseminated nationally over 3 months using a novel, statistically-modeled approach informed by early response rates to achieve a predetermined sample. EDR adoption rate for clinical support was 52%. Adoption rates were higher among: (1) younger dentists; (2) dentists ≤ 15 years in practice; (3) females; and (4) group practices. Top barriers to adoption were EDR cost/expense, cost-benefit ratio, electronic format conversion, and poor EDR usability. Awareness of the Federal HITECH incentive program was low. The rate of chairside computer implementation was 72%. Adoption of EDR in dental offices in the United States was higher in 2012 than electronic health record adoption rates in medical offices and was not driven by the HITECH program. Patient portal adoption among dental practices in the United States remained low.

Goldmann V Standard Automated Perimetry Underestimates Central Visual Sensitivity in Glaucomatous Eyes with Increased Axial Length

PURPOSE

To investigate the effect of axial length (AL) on the structure-function relationship between retinal nerve fiber layer (RNFL) thickness measurements and visual field (VF) sensitivity measured with Goldmann III and V.

METHOD

There were 85 eyes of 85 patients with primary open angle glaucoma included in the current study. Optical coherence tomography and VF (Humphrey Field Analyzer 24-2 or 30-2) measurements with Goldmann III and V targets were carried out in all patients. The optic disc and the VF were divided into six clusters and the relationship between circumpapillary RNFL (cpRNFL) thickness and VF sensitivity (with Goldmann III or V), age, and AL were investigated in each cluster.

RESULT

Visual sensitivity with Goldmann III (19.3 ± 11.7 dB, mean ± standard deviation) was significantly lower than that with Goldmann V (24.6 ± 11.0 dB, P < 0.001, linear mixed model). Visual sensitivities with both Goldmann III and V were significantly correlated with cpRNFL thickness in all clusters. Visual sensitivity decreased with increasing AL in the nasal retinal area for both targets, however, this phenomenon was only observed with the Goldmann V target in the temporal area.

CONCLUSION

Visual sensitivity measured with the size V target decreases with increasing AL in the temporal area, which corresponds to the papillomacular bundle. In the nasal retinal area, visual sensitivity decreases with the increase of AL for both Goldmann III and Goldmann V.

TRANSLATIONAL RELEVANCE

Careful consideration is needed when measuring visual sensitivity using Goldmann V target in glaucomatous eyes with increased AL.

A Competition between Care Teams Improved Recording of Diagnoses in Primary Dental Care: A Longitudinal Follow-Up Study

Introduction

A playful competition was launched in a primary dental health care system to improve the recording of diagnoses into an electronic patient chart system and to study what diagnoses were used in primary dental care.

Methods

This was a longitudinal follow-up study with public sector primary dental care practices in a Finnish city. A one-year-lasting playful competition between the dental care teams was launched and the monthly percentage of dentists' visits with recorded diagnosis before, during, and after the intervention was recorded. The assessed diagnoses were recorded with the International Classification of Diseases (ICD-10).

Results

Before the competition, the level of diagnosis recordings was practically zero. At the end of this intervention, about 25% of the visits had a recorded diagnosis. Two years after the competition, this percentage was 35% without any additional measures. The most frequent diagnoses were dental caries (K02, 38.6%), other diseases of hard tissues of teeth (K03, 14.8%), and diseases of pulp and periapical tissues (K04, 11.4%).

Conclusions

Commitment to the idea that recording of diagnoses was beneficial improved the recording of dental diagnoses. However, the diagnoses obtained did not accurately reflect the reputed prevalence of oral diseases in the Finnish population.

The association between photoreceptor layer thickness measured by optical coherence tomography and visual sensitivity in glaucomatous eyes

PURPOSE

To assess the thickness of the photoreceptor layer in the macular region in glaucomatous eyes.

METHOD

Humphrey 10-2 visual field (VF) testing was carried out and mean threshold (mTH) was calculated in 118 eyes from 118 patients with open angle glaucoma. Macular optical coherence tomography (OCT) measurements (RS 3000, Nidek Co.ltd., Aichi, Japan) were also carried out in all eyes. Thickness measurements were recorded in the outer segment and retinal pigment epithelium (OS+RPE), the nerve fiber layer (NFL), the ganglion cell layer and inner plexiform layer (GCL+IPL), the inner nuclear layer and outer plexiform layer (INL+OPL) and the outer nuclear layer and inner segment (ONL+IS). The relationship between mTH and the thickness of these five different layers was investigated. Additionally, the influence of OS+RPE on mTH was investigated using partial correlation eliminating the effect of other variables of NFL, GCL+IPL, INL+OPL, ONL+IS, age, gender and axial length.

RESULTS

The thickness of the OS+RPE layer was significantly decreased with the decrease of mTH (coefficient = 0.63 p <0.001). Partial correlation analysis suggested OS+RPE thickness is significantly (coefficient = 0.31, p <0.001) related to mTH, independent from NFL, GCL+IPL, INL+OPL, ONL+IS, age, gender and axial length.

CONCLUSIONS

The thickness of the RPE+OS layer appears to be related to visual sensitivity in glaucoma.

The association between structure-function relationships and cognitive impairment in elderly glaucoma patients

Accurate measurement of visual field (VF) is important in accessing glaucoma, however this may not be achieved in patients with dementia or mild cognitive impairment (CI). We investigated the association between CI and structure-function relationships in elderly glaucoma patients. The study included 94 eyes of 51 glaucoma patients aged ≥75 years with no diagnoses of dementia. CI was assessed using the Mini Mental State Examination (MMSE). Using the leave-one-out cross-validation, the mean deviation (MD) of the Humphrey 30-2 VF was predicted from measurements of optical coherence tomography, and the relationship between the squared prediction error and the MMSE score, together with age, fixation loss (FL), false positive (FP), and false negative (FN) percentages that were analyzed using the linear mixed model. A high prevalence of MCI or dementia was observed in the elderly population. The squared prediction error value of the MD was 17.0 ± 21.1 (mean ± standard deviation). The squared prediction error increased with decreasing MMSE total score, but age, FL, FP, and FN were not related. Careful consideration is needed when interpreting the VF results of these patients, because VF can be over- or underestimated, as suggested by the decreased structure-function relationships.

Dental Providers' Perspectives on Diagnosis-Driven Dentistry: Strategies to Enhance Adoption of Dental Diagnostic Terminology

The routine use of standardized diagnostic terminologies (DxTMs) in dentistry has long been the subject of academic debate. This paper discusses the strategies suggested by a group of dental stakeholders to enhance the uptake of DxTMs. Through unstructured interviewing at the 'Toward a Diagnosis-Driven Profession' National Conference held on 19 March 2016 in Los Angeles, CA, USA participants were asked how enthusiastic they were about implementing and consistently using DxTMs at their work. They also brainstormed on strategies to improve the widespread use of DxTMs. Their responses are summarized by recursive abstraction and presented in themes. Conference participants were very enthusiastic about using a DxTM in their place of work. Participants enumerated several strategies to make DxTMs more appealing including: the use of mandates, a value proposition for providers, communication and education, and integration with EHRs and existing systems. All groups across the dental healthcare delivery spectrum will need to work together for the success of the widespread and consistent use of DxTMs. Understanding the provider perspective is however the most critical step in achieving this goal, as they are the group who will ultimately be saddled with the critical task of ensuring DxTM use at the point of care.

Evaluating quality of dental care among patients with diabetes: Adaptation and testing of a dental quality measure in electronic health records

BACKGROUND

Patients with diabetes are at increased risk of developing oral complications, and annual dental examinations are an endorsed preventive strategy. The authors evaluated the feasibility and validity of implementing an automated electronic health record (EHR)-based dental quality measure to determine whether patients with diabetes received such evaluations.

METHODS

The authors selected a Dental Quality Alliance measure developed for claims data and adapted the specifications for EHRs. Automated queries identified patients with diabetes across 4 dental institutions, and the authors manually reviewed a subsample of charts to evaluate query performance. After assessing the initial EHR measure, the authors defined and tested a revised EHR measure to capture better the oral care received by patients with diabetes.

RESULTS

In the initial and revised measures, the authors used EHR automated queries to identify 12,960 and 13,221 patients with diabetes, respectively, in the reporting year. Variations in the measure scores across sites were greater with the initial measure (range, 36.4-71.3%) than with the revised measure (range, 78.8-88.1%). The automated query performed well (93% or higher) for sensitivity, specificity, and positive and negative predictive values for both measures.

CONCLUSIONS

The results suggest that an automated EHR-based query can be used successfully to measure the quality of oral health care delivered to patients with diabetes. The authors also found that using the rich data available in EHRs may help estimate the quality of care better than can relying on claims data.

PRACTICAL IMPLICATIONS

Detailed clinical patient-level data in dental EHRs may be useful to dentists in evaluating the quality of dental care provided to patients with diabetes.

Adoption of dental innovations: The case of a standardized dental diagnostic terminology

BACKGROUND

Standardized dental diagnostic terminologies (SDDxTs) were introduced decades ago. Their use has been on the rise, accompanying the adoption of electronic health records (EHRs). One of the most broadly used terminologies is the Dental Diagnostic System (DDS). Our aim was to assess the adoption of SDDxTs by US dental schools by using the Rogers diffusion of innovations framework, focusing on the DDS.

METHODS

The authors electronically surveyed clinic deans in all US dental schools (n = 61) to determine use of an EHR and SDDxT, perceived barriers to adoption of an SDDxT, and the effect of implementing an SDDxT on clinical productivity.

RESULTS

The response rate was 57%. Of the 35 responses, 91% reported using an EHR to document patient care, with 84% using axiUm; 41% used the DDS. Fifty-four percent of those who did not use an SDDxT had considered adopting the DDS, but 38% had not, citing barriers such as complexity and compatibility.

CONCLUSIONS

Adoption of an SDDxT, particularly the DDS, is on the rise. Nevertheless, a large number of institutions are in the Rogers late majority and laggards categories with respect to adoption. Several factors may discourage adoption, including the inability to try out the terminology on a small scale, poor usability within the EHR, the fact that it would be a cultural shift in practice, and a perception of unclear benefits. However, the consolidation of the DDS and American Dental Association terminology efforts stands to encourage adoption.

PRACTICAL IMPLICATIONS

The successful adoption of dental innovation depends not only on the intrinsic merit of the innovation, as some useful innovations do not achieve widespread traction. As such, it is important for health care providers to understand how to disseminate their ideas in order to ensure traction and widespread adoption.

Medical care providers' perspectives on dental information needs in electronic health records

BACKGROUND

The authors conducted this study to identify the most relevant patient dental information in a medical-dental integrated electronic health record (iEHR) necessary for medical care providers to inform holistic treatment.

METHODS

The authors collected input from a diverse sample of 65 participants from a large, regional health system representing 13 medical specialties and administrative units. The authors collected feedback from participants through 11 focus group sessions. Two independent reviewers analyzed focus group transcripts to identify major and minor themes.

RESULTS

The authors identified 336 of 385 annotations that most medical care providers coded as relevant. Annotations strongly supporting relevancy to clinical practice aligned with 18 major thematic categories, with the top 6 categories being communication, appointments, system design, medications, treatment plan, and dental alerts.

CONCLUSIONS

Study participants identified dental data of highest relevance to medical care providers and recommended implementation of user-friendly access to dental data in iEHRs as crucial to holistic care delivery.

PRACTICAL IMPLICATIONS

Identification of the patients' dental information most relevant to medical care providers will inform strategies for improving the integration of that information into the medical-dental iEHR.

Measuring Practicing Clinicians' Information Literacy. An Exploratory Analysis in the Context of Panel Management

BACKGROUND

As healthcare moves towards technology-driven population health management, clinicians must adopt complex digital platforms to access health information and document care.

OBJECTIVES

This study explored information literacy, a set of skills required to effectively navigate population health information systems, among primary care providers in one Veterans' Affairs (VA) medical center.

METHODS

Information literacy was assessed during an 8-month randomized trial that tested a population health (panel) management intervention. Providers were asked about their use and comfort with two VA digital tools for panel management at baseline, 16 weeks, and post-intervention. An 8-item scale (range 0-40) was used to measure information literacy (Cronbach's α=0.84). Scores between study arms and provider types were compared using paired t-tests and ANOVAs. Associations between self-reported digital tool use and information literacy were measured via Pearson's correlations.

RESULTS

Providers showed moderate levels of information literacy (M= 27.4, SD 6.5). There were no significant differences in mean information literacy between physicians (M=26.4, SD 6.7) and nurses (M=30.5, SD 5.2, p=0.57 for difference), or between intervention (M=28.4, SD 6.5) and control groups (M=25.1, SD 6.2, p=0.12 for difference). Information literacy was correlated with higher rates of self-reported information system usage (r=0.547, p=0.001). Clinicians identified data access, accuracy, and interpretability as potential information literacy barriers.

CONCLUSIONS

While exploratory in nature, cautioning generalizability, the study suggests that measuring and improving clinicians' information literacy may play a significant role in the implementation and use of digital information tools, as these tools are rapidly being deployed to enhance communication among care teams, improve health care outcomes, and reduce overall costs.

Strategic Shift to a Diagnostic Model of Care in a Multi-Site Group Dental Practice

Background

Documenting standardized dental diagnostic terms represents an emerging change for how dentistry is practiced. We focused on a mid-sized dental group practice as it shifted to a policy of documenting patients’ diagnoses using standardized terms in the electronic health record.

Methods

Kotter’s change framework was translated into interview questions posed to the senior leadership in a mid-size dental group practice. In addition, quantitative content analyses were conducted on the written policies and forms before and after the implementation of standardized diagnosis documentation to assess the extent to which the forms and policies reflected the shift. Three reviewers analyzed the data individually and reached consensuses where needed.

Results

Kotter’s guiding change framework explained the steps taken to 97 percent utilization rate of the Electronic Health Record and Dental Diagnostic Code. Of the 96 documents included in the forms and policy analysis, 31 documents were officially updated but only two added a diagnostic element.

Conclusion

Change strategies established in the business literature hold utility for dental practices seeking diagnosis-centered care.

Practical Implications

A practice that shifts to a diagnosis-driven care philosophy would be best served by ensuring that the change process follows a leadership framework that is calibrated to the organization’s culture.

Attitudes toward and beliefs about the use of a dental diagnostic terminology: A survey of dental care providers in a dental practice

BACKGROUND

Attitudes and views are critical to the adoption of innovation. Although there have been broadening calls for a standardized dental diagnostic terminology, little is known about the views of private practice dental team members regarding the adoption of such a terminology.

METHODS

The authors developed a survey by using validated questions identified through literature review. Domain experts' input allowed for further modifications. The authors administered the final survey electronically to 814 team members at a multioffice practice based in the US Pacific Northwest.

RESULTS

Response proportion was 92%. The survey had excellent reliability (Cronbach α coefficient = 0.87). Results suggested that participants showed, in general, positive attitudes and beliefs about using a standardized diagnostic terminology in their practices. Additional written comments by participants highlighted the potential for improved communication with use of the terminology.

CONCLUSIONS

Dental care providers and staff in 1 multioffice practice showed positive attitudes about the use of a diagnostic terminology; specifically, they believed it would improve communication between the dentist and patient, as well as among providers, while expressing some concerns about whether using standardized dental diagnostic terms helps clinicians to deliver better dental care.

PRACTICAL IMPLICATIONS

As the dental profession is advancing toward the use of standardized diagnostic terminology, successful implementation will require that dental team leaders prepare their teams by gauging their attitude about the use of such a terminology.

Preprocessing structured clinical data for predictive modeling and decision support. A roadmap to tackle the challenges

BACKGROUND

EHR systems have high potential to improve healthcare delivery and management. Although structured EHR data generates information in machine-readable formats, their use for decision support still poses technical challenges for researchers due to the need to preprocess and convert data into a matrix format. During our research, we observed that clinical informatics literature does not provide guidance for researchers on how to build this matrix while avoiding potential pitfalls.

OBJECTIVES

This article aims to provide researchers a roadmap of the main technical challenges of preprocessing structured EHR data and possible strategies to overcome them.

METHODS

Along standard data processing stages - extracting database entries, defining features, processing data, assessing feature values and integrating data elements, within an EDPAI framework -, we identified the main challenges faced by researchers and reflect on how to address those challenges based on lessons learned from our research experience and on best practices from related literature. We highlight the main potential sources of error, present strategies to approach those challenges and discuss implications of these strategies.

RESULTS

Following the EDPAI framework, researchers face five key challenges: (1) gathering and integrating data, (2) identifying and handling different feature types, (3) combining features to handle redundancy and granularity, (4) addressing data missingness, and (5) handling multiple feature values. Strategies to address these challenges include: cross-checking identifiers for robust data retrieval and integration; applying clinical knowledge in identifying feature types, in addressing redundancy and granularity, and in accommodating multiple feature values; and investigating missing patterns adequately.

CONCLUSIONS

This article contributes to literature by providing a roadmap to inform structured EHR data preprocessing. It may advise researchers on potential pitfalls and implications of methodological decisions in handling structured data, so as to avoid biases and help realize the benefits of the secondary use of EHR data.

How Do Residents Spend Their Shift Time? A Time and Motion Study With a Particular Focus on the Use of Computers

PURPOSE

To understand how much time residents spend using computers compared with other activities, and what residents use computers for.

METHOD

This time and motion study was conducted in June and July 2010 at NewYork-Presbyterian/Columbia University Medical Center with seven residents (first-, second-, and third-year) on the general medicine service. An experienced observer shadowed residents during a single day shift, captured all their activities using an iPad application, and took field notes. The activities were captured using a validated taxonomy of clinical activities, expanded to describe computer-based activities with a greater level of detail.

RESULTS

Residents spent 364.5 minutes (50.6%) of their shift time using computers, compared with 67.8 minutes (9.4%) interacting with patients. In addition, they spent 292.3 minutes (40.6%) talking with others in person, 186.0 minutes (25.8%) handling paper notes, 79.7 minutes (11.1%) in rounds, 80.0 minutes (11.1%) walking or waiting, and 54.0 minutes (7.5%) talking on the phone. Residents spent 685 minutes (59.6%) multitasking. Computer-based documentation activities amounted to 189.9 minutes (52.1%) of all computer-based activities time, with 128.7 minutes (35.3%) spent writing notes and 27.3 minutes (7.5%) reading notes composed by others.

CONCLUSIONS

The study showed that residents spent considerably more time interacting with computers (over 50% of their shift time) than in direct contact with patients (less than 10% of their shift time). Some of this may be due to an increasing reliance on computing systems for access to patient data, further exacerbated by inefficiencies in the design of the electronic health record.

Assessing and comparing the usability of Chinese EHRs used in two Peking University hospitals to EHRs used in the US: A method of RUA

OBJECTIVES

The objective of this study was to investigate the usability level of Chinese hospital Electronic Health Records (EHRs) by assessing the completion times of EHRs for seven "meaningful use (MU)" relevant tasks conducted at two Chinese tertiary hospitals and comparing the results to those of relevant research conducted in US EHRs.

METHODS

Using Rapid Usability Assessment (RUA) developed by the National Center for Cognitive Informatics and Decision Making (NCCD), the usability of EHRs from two Peking University hospitals was assessed using a three-step Keystroke Level Model (KLM) in a laboratory environment.

RESULTS

(1) The total EHR task completion time for 7 MU relevant test tasks showed no significant differences between the two Chinese EHRs and their US counterparts, in which the time for thinking was relatively large and comprised 35.6% of the total time. The time for the electronic physician order was the largest. (2) For specific tasks, the mean completion times of the 2 hospital EHR systems spent on recording, modifying and searching (RMS) the medication orders were similar to those for the RMS radioactive tests; the mean time spent on the RMS laboratory test orders were much less. (3) There were 85 usability problems identified in the 2 hospital EHR systems.

DISCUSSION

In Chinese EHRs, a substantial amount of time is required to complete tasks relevant to MU targets and many preventable usability problems can be discovered. The task completion time of the 2 Chinese EHR systems was a little shorter than in the 5 reported US EHR systems, while the differences in smoking status and CPOE tasks were obvious; one main reason for these differences was the use of structured data entry.

CONCLUSIONS

The efficiency of Chinese and US EHRs was not significantly different. The key to improving the efficiency of both systems lies in expediting the Computerized physician order entry (CPOE) task. Many usability problems can be identified using heuristic assessments and improved by corresponding actions.

Periodontal Management by Risk Assessment: A Pragmatic Approach

An evidence-based periodontal disease risk assessment and diagnosis system has been developed and combined with a clinical decision support and management program to improve treatment and measure patient outcomes.

BACKGROUND

There is little agreement on a universally accepted periodontal risk assessment, periodontal diagnosis, and treatment management tool and their incorporation into dental practice to improve patient care. This article highlights the development and use of a practical periodontal management and risk assessment program that can be implemented in dental settings.

METHODS

The approach taken by Willamette Dental Group to develop a periodontal disease risk assessment, periodontal diagnosis, and treatment management tool is described using evidence-based best practices. With goals of standardized treatment interventions while maintaining personalized care and improved communication, this process is described to facilitate its incorporation into other dental settings.

CONCLUSIONS

Current electronic health records can be leveraged to enhance patient-centered care through the use of risk assessments and standardized guidelines to more effectively assess, diagnose, and treat patients to improve outcomes. Dental hygienists, and other committed providers, with their emphasis on prevention of periodontal disease can be principal drivers in creation and implementation of periodontal risk assessments and personalized treatment planning. Willamette Dental Group believes that such evidence-based tools can advance dentistry to new diagnostic and treatment standards.

Drivers Advancing Oral Health in a Large Group Dental Practice Organization

Three change drivers are being implemented to high standards of patient centric and evidence-based oral health care within the context of a large multispecialty dental group practice organization based on the commitment of the dental hygienist chief operating officer and her team.

BACKGROUND AND PURPOSE

A recent environmental scan elucidated 6 change drivers that can impact the provision of oral health care. Practitioners who can embrace and maximize aspects of these change drivers will move dentistry forward and create future opportunities. This article explains how 3 of these change drivers are being applied in a privately held, accountable risk-bearing entity that provides individualized treatment programs for more than 417,000 members. To facilitate integration of the conceptual changes related to the drivers, a multi-institutional, multidisciplinary, highly functioning collaborative work group was formed.

METHODS AND APPROACH

The document Dental Hygiene at a Crossroads for Change(1) inspired the first author, a dental hygienist in a unique position as chief operating officer of a large group practice, to pursue evidence-based organizational change and to impact the quality of patient care. This was accomplished by implementing technological advances including dental diagnosis terminology in the electronic health record, clinical decision support, standardized treatment guidelines, quality metrics, and patient engagement to improve oral health outcomes at the patient and population levels. The systems and processes used to implement 3 change drivers into a large multi-practice dental setting is presented to inform and inspire others to implement change drivers with the potential for advancing oral health.

CONCLUSIONS

Technology implementing best practices and improving patient engagement are excellent drivers to advance oral health and are an effective use of oral health care dollars. Improved oral health can be leveraged through technological advances to improve clinical practice.

Usability and Safety in Electronic Medical Records Interface Design: A Review of Recent Literature and Guideline Formulation

OBJECTIVE

The objectives of this study were to (a) review electronic medical record (EMR) and related electronic health record (EHR) interface usability issues, (b) review how EMRs have been evaluated with safety analysis techniques along with any hazard recognition, and (c) formulate design guidelines and a concept for enhanced EMR interfaces with a focus on diagnosis and documentation processes.

BACKGROUND

A major impact of information technology in health care has been the introduction of EMRs. Although numerous studies indicate use of EMRs to increase health care quality, there remain concerns with usability issues and safety.

METHOD

A literature search was conducted using Compendex, PubMed, CINAHL, and Web of Science databases to find EMR research published since 2000. Inclusion criteria included relevant English-language papers with subsets of keywords and any studies (manually) identified with a focus on EMR usability.

RESULTS

Fifty studies met the inclusion criteria. Results revealed EMR and EHR usability problems to include violations of natural dialog, control consistency, effective use of language, effective information presentation, and customization principles as well as a lack of error prevention, minimization of cognitive load, and feedback. Studies focusing on EMR system safety made no objective assessments and applied only inductive reasoning methods for hazard recognition.

CONCLUSION

On the basis of the identified usability problems and structure of safety analysis techniques, we provide EMR design guidelines and a design concept focused on the diagnosis process and documentation.

APPLICATION

The design guidelines and new interface concept can be used for prototyping and testing enhanced EMRs.

Classification of antecedents towards safety use of health information technology: A systematic review

OBJECTIVES

This paper provides a systematic review of safety use of health information technology (IT). The first objective is to identify the antecedents towards safety use of health IT by conducting systematic literature review (SLR). The second objective is to classify the identified antecedents based on the work system in Systems Engineering Initiative for Patient Safety (SEIPS) model and an extension of DeLone and McLean (D&M) information system (IS) success model.

METHODS

A systematic literature review (SLR) was conducted from peer-reviewed scholarly publications between January 2000 and July 2014. SLR was carried out and reported based on the preferred reporting items for systematic reviews and meta-analyses (PRISMA) statement. The related articles were identified by searching the articles published in Science Direct, Medline, EMBASE, and CINAHL databases. Data extracted from the resultant studies included are to be analysed based on the work system in Systems Engineering Initiative for Patient Safety (SEIPS) model, and also from the extended DeLone and McLean (D&M) information system (IS) success model.

RESULTS

55 articles delineated to be antecedents that influenced the safety use of health IT were included for review. Antecedents were identified and then classified into five key categories. The categories are (1) person, (2) technology, (3) tasks, (4) organization, and (5) environment. Specifically, person is attributed by competence while technology is associated to system quality, information quality, and service quality. Tasks are attributed by task-related stressor. Organisation is related to training, organisation resources, and teamwork. Lastly, environment is attributed by physical layout, and noise.

CONCLUSIONS

This review provides evidence that the antecedents for safety use of health IT originated from both social and technical aspects. However, inappropriate health IT usage potentially increases the incidence of errors and produces new safety risks. The review cautions future implementation and adoption of health IT to carefully consider the complex interactions between social and technical elements propound in healthcare settings.

Are three methods better than one? A comparative assessment of usability evaluation methods in an EHR

OBJECTIVE

To comparatively evaluate the effectiveness of three different methods involving end-users for detecting usability problems in an EHR: user testing, semi-structured interviews and surveys.

MATERIALS AND METHODS

Data were collected at two major urban dental schools from faculty, residents and dental students to assess the usability of a dental EHR for developing a treatment plan. These included user testing (N=32), semi-structured interviews (N=36), and surveys (N=35).

RESULTS

The three methods together identified a total of 187 usability violations: 54% via user testing, 28% via the semi-structured interview and 18% from the survey method, with modest overlap. These usability problems were classified into 24 problem themes in 3 broad categories. User testing covered the broadest range of themes (83%), followed by the interview (63%) and survey (29%) methods.

DISCUSSION

Multiple evaluation methods provide a comprehensive approach to identifying EHR usability challenges and specific problems. The three methods were found to be complementary, and thus each can provide unique insights for software enhancement. Interview and survey methods were found not to be sufficient by themselves, but when used in conjunction with the user testing method, they provided a comprehensive evaluation of the EHR.

CONCLUSION

We recommend using a multi-method approach when testing the usability of health information technology because it provides a more comprehensive picture of usability challenges.

Advancing cognitive engineering methods to support user interface design for electronic health records

BACKGROUND

Despite many decades of research on the effective development of clinical systems in medicine, the adoption of health information technology to improve patient care continues to be slow, especially in ambulatory settings. This applies to dentistry as well, a primary care discipline with approximately 137,000 practitioners in the United States. A critical reason for slow adoption is the poor usability of clinical systems, which makes it difficult for providers to navigate through the information and obtain an integrated view of patient data.

OBJECTIVE

In this study, we documented the cognitive processes and information management strategies used by dentists during a typical patient examination. The results will inform the design of a novel electronic dental record interface.

METHODS

We conducted a cognitive task analysis (CTA) study to observe ten general dentists (five general dentists and five general dental faculty members, each with more than two years of clinical experience) examining three simulated patient cases using a think-aloud protocol.

RESULTS

Dentists first reviewed the patient's demographics, chief complaint, medical history and dental history to determine the general status of the patient. Subsequently, they proceeded to examine the patient's intraoral status using radiographs, intraoral images, hard tissue and periodontal tissue information. The results also identified dentists' patterns of navigation through patient's information and additional information needs during a typical clinician-patient encounter.

CONCLUSION

This study reinforced the significance of applying cognitive engineering methods to inform the design of a clinical system. Second, applying CTA to a scenario closely simulating an actual patient encounter helped with capturing participants' knowledge states and decision-making when diagnosing and treating a patient. The resultant knowledge of dentists' patterns of information retrieval and review will significantly contribute to designing flexible and task-appropriate information presentation in electronic dental records.

User participation in healthcare IT development: a developers' viewpoint in Finland

BACKGROUND AND PURPOSE

Recent research showed that physicians in Finland were highly critical of their information technology (IT) systems. They were also critical of the methods of collaboration with the developers of the health IT systems (HITS) in use at the time of the questionnaire. This study turned the set-up around and asked systems developers the same questions about collaboration. What is developers' view on end user participation in HITS development at the moment? How would developers wish end users to participate in systems development? Do the developers' views differ from the physicians' (end users') views of the current state of collaboration in developing IT systems?

METHODS

A web-based questionnaire study was conducted in one of the major HITS provider companies in Finland among all developers, including software developers and customer support and sales personnel. Both quantitative and free-text questions of a previous study were adapted for the purpose. The responses were analyzed with qualitative and basic quantitative methods.

RESULTS

The response rate of the questionnaire was 37% and 136 responses were received. The developers who responded were experienced workers; 81% of the respondents had 6 years or more of work experience in IT systems development and 35% of them had 6 years or more of work experience in the healthcare domain. Almost three-quarters (72%) of the respondents agreed with the statement 'I work with users'. Almost all the developers (90%) thought that they are interested in user feedback and also 81% thought that they take the end users' opinions and experiences into account when developing software. A majority of the developers (57%) considered that corrections and modifications are currently not implemented quickly enough. The most popular means of user participation were that 'users would present their work and needs related to it in their workplace' (76%), followed by user groups (75%). The developers suggested many traditional user-centered and usability design methods, too. The developers' views were compared to the views of the physicians who primarily used the case company's products. The views were in direct opposition on whether developers are interested in end users' views (90% of the developers agreed, vs. 60% of the physicians disagreed) and take them into account (81% of the developers agreed, vs. 63% of the physicians disagreed), as well as on user groups (favored by 75% of the developers vs. 14% of the physicians). The majority of the respondents, both developers (57%) and physicians (74%), were dissatisfied with the pace of implementation of corrections and modifications.

CONCLUSIONS

Both physicians and developers seem to be "willing but not able" to collaborate with each other. Possible reasons for the differences in views include the fact that there is no return channel of communication on what happened to the end users' feedback, and that developers collaborate with customer representatives who are not end users. It is obvious that there are one or more spots along the route between the "end developers" and end users where there is a breakdown of the information flow.

Usability assessment of an electronic health record in a comprehensive dental clinic

In this paper we present the development and usability of an electronic health record (EHR) system in a comprehensive dental clinic.The graphic user interface of the system was designed to consider the concept of cognitive ergonomics.The cognitive task analysis was used to evaluate the user interface of the EHR by identifying all sub-tasks and classifying them into mental or physical operators, and to predict task execution time required to perform the given task. We randomly selected 30 cases that had oral examinations for routine clinical care in a comprehensive dental clinic. The results were based on the analysis of 4 prototypical tasks performed by ten EHR users. The results showed that on average a user needed to go through 27 steps to complete all tasks for one case. To perform all 4 tasks of 30 cases, they spent about 91 min (independent of system response time) for data entry, of which 51.8 min were spent on more effortful mental operators. In conclusion, the user interface can be improved by reducing the percentage of mental effort required for the tasks.