Canadian employers' perspectives on a new framework for health informatics competencies

Competencies are the knowledge, skills, and abilities needed to operate and perform successfully in the workplace. Due to the evolving nature of health informatics, it is important continuously examine and refine competencies in this field. In this study, we administered a questionnaire to Canadian employers (N = 29) of health informatics cooperative education (co-op) students to garner their feedback on competencies within a New Health Informatics Professional Competencies Framework. Overall, the findings supported this new framework. An average of ratings within each of the four competency categories revealed that participants perceived Management Science to be the most important, followed by Information & Computer Science, then Health Science and finally Data Science. Further, at least 20 (69 %) respondents rated nine of the 12 competencies as important. Of the 12 competencies, Biological and Clinical Science was rated the lowest. Findings from this study can potentially be used to inform curricula, career progression, and hiring practices in health informatics. Future work includes refining the questionnaire to assess the competencies more comprehensively and potentially exploring the importance of more transferable skills or general competencies (e.g., communication, problem-solving). Additionally, we want to survey a broader sample of health informatics professionals and integrate recent national and international work on health informatics competencies. Future work is also recommended towards the development of a maturity model for competencies of more experienced health informatics professionals.

Tabular, Annotated, Visual, or Trends + Contextual Information? Preferences for Online Laboratory Results Displays

People are increasingly offered access to their personal health information (e.g., laboratory results, clinical notes, diagnostic imaging results). However, this information is the same as that used by health care providers with clinical expertise and training in medical terminology, which citizens typically do not have. In this study, we examined participants (N = 24) preferences for four different types of displays for online laboratory (lab) results: Tabular, Annotated, Visual, and Trends + Contextual Information. The Friedman test of difference comparing participants' ratings of the four displays was significant, χ2(3)=10.8, P=.013, and the Wilcoxon signed rank pairwise comparison tests revealed that participants rated the visual lab results display significantly more favourably than the traditional display (Z=-2.746, P=.006). These findings indicate that many people prefer lab results displayed using more visual cues and some perceived this format as easier to understand than the other display formats. Given the importance of people accessing, understanding, and using their own health information, it is crucial for displays and systems to provide a better user experience. Displaying data (e.g., lab results) visually is one possible way to improve interpretability of personal health information provided to the public.

Simulating Telemedicine, Medication Reconciliation, and Social Determinants: A Novel Instructional Approach to Health Systems Competencies

While medication reconciliation is necessary to reduce errors, it is often challenging to gather an accurate history in the clinic. Telemedicine offers a relative advantage over clinic and hospital-based interviews by enabling the clinician to inspect the home environment, review pill bottles, and identify social determinants affecting adherence, such as financial instability. To be effective, however, clinicians must be trained in best-practice interview methods and the proper use of telemedicine. There is very little information in the literature describing the best strategies for teaching students or measuring competencies in telemedicine. Therefore, we created an educational module with a telemedicine simulation and an evaluation rubric. We piloted this module with 48 medical and physician assistant students. Most students could complete a virtual interview and gather a medication history. However, only half identified an over-the-counter medication missing from the list. Most students were either entrustable or approaching entrustment in the six telemedicine competencies measured in this simulation. This simulation is valuable for teaching students about medication reconciliation, using telemedicine to close gaps in access to care, and identifying health-related social needs affecting medication adherence.

Citizens' Access to Online Health Information - An International Survey of IMIA Member Countries

Citizens' access to their online health information is pivotal. Therefore, this study examines citizens' access to their online health information across countries and healthcare settings. The study is based on a survey design targeting the 98 IMIA representatives of the national societies. Results indicate that Test results and Medications are the two types of online information that citizens in most cases have access to. Ten countries provide citizens access to all the different types of information included in the study. That relatively few countries provide citizens access to all the included types of online health information underscores the importance of continuous emphasis on accessibility and research within this field.

Teledermatology: Simulating Hybrid Workflows for Telemedicine Education

Given the importance of telemedicine in improving healthcare access for underserved patients, professional students need experience using virtual clinical workflows. We developed an educational workshop with (1) readings, (2) a knowledge assessment test, (3) dermatology and teledermatology lectures, (5) a telemedicine simulation with a standardized patient, and (6) a debriefing session. The simulation included a "hybrid" workflow with live videoconferencing and store-and-forward image review. We measured student performance using three American Association of Medical Colleges (AAMC) Telemedicine Competencies for medical education. Ninety-eight medical and physician assistant students completed this workshop between 2021 and 2022, and 80% were entrustable or approaching entrustment in each competency. Some students struggled with data collection and technology use. Our results suggest that this workshop offers a practical and generalizable way to teach about multiple virtual workflows and strengthen students' telemedicine competencies.

Interpreting Laboratory Results with Complementary Health Information: A Human Factors Perspective

The desire to access personal and high-quality health information electronically is increasing, not only in Canada, but globally. With the advent of the COVID - 19 pandemic the desire and demand for telemedicine and timely access to personal health data such as online laboratory (lab) results has increased substantially. This study examines citizens' perspectives of being provided with high-quality information about a specific lab test (i.e., potassium) in the same display as a trend graph. Therefore, the objective of this study is to test how participants managed this additional information about the context of the test, understood, and applied it. The researchers analyzed the responses of semi-structured interviews with Canadian participants (N=24) using conventional content analysis. This paper examined four themes related to providing complementary information concurrently with lab results in the same display: 1) Benefits of Collocated Information, 2) Information Overload, 3) Misinterpretation, 4) Confusion. This study provided examples of some of the difficulties that the participants faced accessing their lab values online, while navigating and discerning complimentary high-quality health information available in their patient portal.

Assessing Telemedicine Competencies: Developing and Validating Learner Measures for Simulation-Based Telemedicine Training

In 2021, the Association of American Medical Colleges published Telehealth Competencies Across the Learning Continuum, a roadmap for designing telemedicine curricula and evaluating learners. While this document advances educators' shared understanding of telemedicine's core content and performance expectations, it does not include turn-key-ready evaluation instruments. At the University of Oklahoma School of Community Medicine, we developed a year-long telemedicine curriculum for third-year medical and second-year physician assistant students. We used the AAMC framework to create program objectives and instructional simulations. We designed and piloted an assessment rubric for eight AAMC competencies to accompany the simulations. In this monograph, we describe the rubric development, scores for students participating in simulations, and results comparing inter-rater reliability between faculty and standardized patient evaluators. Our preliminary work suggests that our rubric provides a practical method for evaluating learners by faculty during telemedicine simulations. We also identified opportunities for additional reliability and validity testing.

Identifying Failure Modes in Telemedicine: An Instructional Needs Assessment

Technology failures in telehealth are common, and clinicians need the skills to diagnose and manage them at the point of care. However, there are issues beyond technology failures mediating the effective use of telehealth. We must teach best-practice procedures for conducting telemedicine visits and include in instructional simulations commonly encountered failure modes so students can build their skills. To this end, we recruited medical students to conduct a Healthcare Failure Modes and Effects Analysis (HFMEA) to predict failures in telemedicine, their potential causes, and the consequences to develop and teach prevention strategies. Sixteen students observed telehealth appointments independently. Based on their observations, we identified four categories of failures in telemedicine: technical issues, patient safety, communication, and social and structural determinants. We proposed a normalized workflow that included management and prevention strategies. Our findings can inform the creation of new curricula.

Differential Perceptions of What Constitutes a Medical Error Associated with Electronic Medical Records

Perceptions of errors associated with healthcare information technology (HIT) often depend on the context and position of the viewer. HIT vendors posit very different causes of errors than clinicians, implementation teams, or IT staff. Even within the same hospital, members of departments and services often implicate other departments. Organizations may attribute errors to external care partners that refer patients, such as nursing homes or outside clinics. Also, the various clinical roles within an organization (e.g., physicians, nurses, pharmacists) can conceptualize errors and their root causes differently. Overarching all these perceptual factors, the definitions, mechanisms, and incidence of HIT-related errors are remarkably conflictual. There is neither a universal standard for defining or counting these errors. This paper attempts to enumerate and clarify the issues related to differential perceptions of medical errors associated with HIT. It then suggests solutions.

Applying Logic to the Healthcare Journey

Cross-disciplinary approaches to remediate complex healthcare service delivery issues may have merit. This study aims to establish the potential benefits of applying service design and evaluative research concepts in healthcare. Specifically, this study aims to demonstrate how a Customer Journey Map and a Logic Model could be used in unison to identify and remedy service delivery gaps to reduce barriers to care. This study provides systems thinking approach to solving operational issues in healthcare.

Humanizing Big Data and Detailing Social Determinants of Health via Information Visualizations

The pandemic has had devastating impacts on humanity and the global healthcare sector. An analysis into the social determinants of health, in particular racial and ethnic disparities may explain why certain population groups have been disproportionately affected by COVID-19. The objective of this study is to humanize and personify numerical data. Additionally, COVID-19 population data will be stratified via three data visualization tools (i.e., a persona, a journey map, Sankey diagram) to create a Visualized Combined Experience (VCE) Diagram to illustrate the micro, and macro, perspectives of marginalized individuals across the continuum of care.

Automation vs. Innovation: Unexplored Strategies to Improve Virtual Care

While there is a global desire to increase digital health capacity, digital health should transform health services delivery rather than simply automate - or worse - replicate existing practices. Failing to capitalize on this transformative potential misses an opportunity to engage patients and other users to provide a more person-centered experience. However, digital transformation done recklessly can disrupt workflow, alienate users, and jeopardize patient safety, as we have observed with implementation of many digital health tools. This paper uses a telemedicine example to provide insight into how digital health innovation can be a meaningful enabler of health system transformation. Examining different ways to leverage digital health technologies is crucial to best capitalize on their potential.

Consumers' Needs for Laboratory Results Portals: Questionnaire Study

BACKGROUND

Over the last decade, there has been an increase in the number of health care consumers (ie, patients, citizens, and laypeople) with access to their laboratory results through portals. However, many portals are not designed with the consumer in mind, which can limit communication effectiveness and consumer empowerment.

OBJECTIVE

We aimed to study design facilitators and barriers affecting consumer use of a laboratory results portal. We sought to identify modifiable design attributes to inform future interface specifications and improve patient safety.

METHODS

A web-based questionnaire with open- and closed-ended items was distributed to consumers in British Columbia, Canada. Open-ended items with affinity diagramming and closed-ended questions with descriptive statistics were analyzed.

RESULTS

Participants (N=30) preferred reviewing their laboratory results through portals rather than waiting to see their provider. However, respondents were critical of the interface design (ie, interface usability, information completeness, and display clarity). Scores suggest there are display issues impacting communication that require urgent attention.

CONCLUSIONS

There are modifiable usability, content, and display issues associated with laboratory results portals that, if addressed, could arguably improve communication effectiveness, patient empowerment, and health care safety.

Exploring Patient Journey Mapping and the Learning Health System: Scoping Review

BACKGROUND

Journey maps are visualization tools that can facilitate the diagrammatical representation of stakeholder groups by interest or function for comparative visual analysis. Therefore, journey maps can illustrate intersections and relationships between organizations and consumers using products or services. We propose that some synergies may exist between journey maps and the concept of a learning health system (LHS). The overarching goal of an LHS is to use health care data to inform clinical practice and improve service delivery processes and patient outcomes.

OBJECTIVE

The purpose of this review was to assess the literature and establish a relationship between journey mapping techniques and LHSs. Specifically, in this study, we explored the current state of the literature to answer the following research questions: (1) Is there a relationship between journey mapping techniques and an LHS in the literature? (2) Is there a way to integrate the data from journey mapping activities into an LHS? (3) How can the data gleaned from journey map activities be used to inform an LHS?

METHODS

A scoping review was conducted by querying the following electronic databases: Cochrane Database of Systematic Reviews (Ovid), IEEE Xplore, PubMed, Web of Science, Academic Search Complete (EBSCOhost), APA PsycInfo (EBSCOhost), CINAHL (EBSCOhost), and MEDLINE (EBSCOhost). Two researchers applied the inclusion criteria and assessed all articles by title and abstract in the first screen, using Covidence. Following this, a full-text review of included articles was done, with relevant data extracted, tabulated, and assessed thematically.

RESULTS

The initial search yielded 694 studies. Of those, 179 duplicates were removed. Following this, 515 articles were assessed during the first screening phase, and 412 were excluded, as they did not meet the inclusion criteria. Next, 103 articles were read in full, and 95 were excluded, resulting in a final sample of 8 articles that satisfied the inclusion criteria. The article sample can be subsumed into 2 overarching themes: (1) the need to evolve service delivery models in health care, and (2) the potential value of using patient journey data in an LHS.

CONCLUSIONS

This scoping review demonstrated the gap in knowledge regarding integrating the data from journey mapping activities into an LHS. Our findings highlighted the importance of using the data from patient experiences to enrich an LHS and provide holistic care. To satisfy this gap, the authors intend to continue this investigation to establish the relationship between journey mapping and the concept of LHSs. This scoping review will serve as phase 1 of an investigative series. Phase 2 will entail the creation of a holistic framework to guide and streamline data integration from journey mapping activities into an LHS. Lastly, phase 3 will provide a proof of concept to demonstrate how patient journey mapping activities could be integrated into an LHS.

Updating professional competencies in health informatics: A scoping review and consultation with subject matter experts

INTRODUCTION

The discipline of health informatics emerged to address the need for uniquely skilled professionals to design, develop, implement, and evaluate health information technology. Core competencies are an essential pre-requisite for establishing a professional discipline such as health informatics. In 2012, Digital Health Canada released a framework (DHC Framework) for Canadian health informatics competencies. Multiple perspectives on health informatics competencies have evolved to reflect global and unique country contexts. In this paper, we will describe a two-phase study in which we ultimately developed a new framework for health informatics competencies.

METHODS

In Phase 1, we conducted a scoping review of to identify health informatics competencies from research articles and grey literature from professional associations. Of 1038 articles identified in the search, ultimately 38 met our inclusion criteria and were subject to in-depth analysis. We summarized our findings from this phase into a preliminary framework of health informatics competencies and then in Phase 2, we shared these findings with subject matter experts (SMEs; N = 5) to garner their feedback. The SMEs were all instructors in health informatics in Canada and held various roles (director, professor, advisor, and co-operative education coordinator). We used their insights into the current and forecasted Canadian health informatics landscape to iteratively develop a new framework until we achieved consensus amongst the subject matter experts.

RESULTS

In Phase 1, all competencies of the DHC Framework were supported by the literature. However, we also identified two emergent competencies: Human Factors and Data Science. In Phase 2, consultations with SMEs guided the introduction of one new competency category and seven new competencies. One competency was renamed and two were removed from the DHC Framework. Additionally, we added new terms that encompass the framework and labelled the core of the framework Health Informatics Professionalism.

DISCUSSION

We found that the DHC Framework did not capture all necessary competencies required by health informatics professionals. Based on the literature and consultations with SMEs, we extended the DHC Framework to better reflect the current Canadian context and propose a new Health Informatics Core Competencies Framework. The new framework can be used to inform Canadian health informatics programs to ensure graduates are equipped for careers in health informatics. Future work includes validating the new framework with Canadian health informatics employers to assess whether this new framework adequately reflects their needs, and more detail may be required to define specific skills necessary in each competency.

Towards the Adoption of Novel Visualizations in Public Health

Visualizations form an important part of public health informatics (PHI) communications. Visualizing data facilitates discussion, aids understanding, makes patterns apparent, promotes analysis, and fosters recall. How rare are novel visualizations in the PHI literature? In Phase 1, we used a rapid review methodology to test the commonness of the Sankey diagram in the PHI theory literature via an automated text search for key terms. In Phase 2, we prototype an uncommon chart type. A total of 27 relvant papers were searched and a computer-generated Sankey diagram was prototyped. PHI professionals have access to visualization tools emerging from social media and niche systems. PHI literature underutilizes uncommon visualizations requiring programming expertise. The authors advocate for: multi-disciplinary teamwork, technical education, the use of open visualization tools, and further adoption of visualization for public health professionals.

An Evaluation Guide and Decision Support Tool for Journey Maps in Healthcare and Beyond

The journey map concept evolved out of the service design field and is still relatively new in the healthcare landscape [1]. Journey maps are visualizations that effectively highlight organizational issues and allow stakeholder groups to be depicted by interest or function for a comparative visual analysis [2]. There are five journey map approaches: 1) Mental (Cognitive) Model Map, 2) Customer Journey Map, 3) Experience Map, 4) Service Blueprint Map, 5) Spatial Map. The objective of this article is three-fold: 1) quantify and delineate the journey mapping visualization techniques utilized from the phase 1 scoping review [2], 2) create a Journey Map Evaluation Guide, 3) create a Journey Map Decision Support Tool to facilitate a standardized method for journey map selection. For those less familiar with journey mapping, this framework can serve as a decision-making tool to facilitate the most effective choice among the different journey mapping visualization approaches. The tools presented in this study can provide a mechanism to standardize the assessment, classification and utilization of journey maps in the healthcare sector and industries abound.

Contextualizing Online Laboratory (lab) Results and Mapping the Patient Journey

The 21st century has brought forth unprecedented technological advances, such as the advent of portable digital devices [1]. This trend has also permeated the health care sector, with the introduction of digital health services, like providing citizens with access to their online laboratory (lab) results. This qualitative study will illustrate the patient journey, namely participant 16 (P16), to address the research question: what phases does a person go through when accessing their lab results online? The findings revealed that lab results were accessed from two types of devices a tablet (e.g., portable computer) when at home and a mobile phone when away from home. We also found that interpretation of results can be a challenge and it was unclear if P16 was able to understand her lab results. To illustrate the complexity of interpreting and accessing online lab results, the authors created a Customer Journey Map to contextualize the experiences of P16. The journey map depicts a combination of factors such as: eHealth literacy, limited access to providers, difficulty interpreting lab test results. Additionally, recommendations for online lab portal functionality enhancements were discovered through the mapping exercise. This study demonstrated that along with providing citizens with access to digital health technologies and services, considerations to eHealth literacy, the digital divide and health equity are paramount. As evidenced by the visualization, journey maps hold promise to serve as efficient tools to build empathy and identify the unique needs and perspectives of citizens.

The Utilization of Health Informatics Interventions in the COVID-19 Pandemic: A Scoping Review

On March 11, 2020, the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), the highly infectious virus that causes coronavirus disease (COVID-19), was characterized by the World Health Organization (WHO) as a global pandemic [1,2]. Due to its highly contagious nature, COVID-19 has catalyzed the introduction of non-pharmaceutical interventions such as social distancing and quarantine measures [6]. Thus, the pandemic has shifted society to become reliant on healthcare technologies. The objective of this scoping review is to establish what health informatics interventions have been applied, validated and tested globally during the COVID-19 pandemic. The findings demonstrated a range of 12 types of health informatics interventions with various global applications and use. As evidenced by the intervention heterogeneity, the necessity to adopt a global cohesive strategy to improve human safety through the utilization of smart, efficient, and communicable technologies is vital.

Hidden in Plain Sight: Overlooked Results and Other Errors in Evaluating Online Laboratory Results

People are increasingly accessing their own laboratory (lab) results online. However, Canadians may be expected to use different systems to access their results, depending upon where they are tested (e.g., community lab vs. hospital), and these results may be displayed differently. This study examined the extent to which participants without medical expertise (N = 25) made errors identifying lab results (i.e., missing or mis-identifying abnormal results) in a mock report. Six participants overlooked each of the flagged values, 20 participants missed an abnormal result that was not flagged, and 2 participants mis-identified a normal value as out of range. We describe potential causes of these errors and the implications for the design of consumer-facing lab results.

Pre- and Post-Redesign Usability Assessment of a Telemedicine Interface Based on Subjective Metrics

Usability problems in the interaction between patients and telemedicine platforms has been recognized as a deterrent in the public's acceptance and use of this alternative healthcare delivery method. Therefore, evaluating the usability of telemedicine provider websites, with a focus on potential patients' first interaction with telemedicine, is a critical research inquiry. To this end, a novel survey was developed to conduct an unmoderated remote usability test (URUT) of the Teladoc website. Teladoc is one of the largest providers of Direct-to-Consumer (DTC) telemedicine. The Teladoc Website Usability Survey (TWUS) instrument collected both objective task completion success metrics and subjective user feedback. A codebook was developed to categorize design features and user interface aspects that affected usability. The TWUS and codebook demonstrated value in identifying usability problems with the Teladoc interface and can be applied in other telemedicine or Health Information Technology (HIT) usability studies. Identifying and addressing usability issues is an important approach to increase the widespread acceptance and adoption of these healthcare delivery technologies.

Overcoming Challenges to Inclusive User-based Testing of Health Information Technology with Vulnerable Older Adults: Recommendations from a Human Factors Engineering Expert Inquiry

OBJECTIVES

Involving representative users in usability testing of health information technology (HIT) is central to user-centered design. However, (vulnerable) older adults as representative users have unique requirements. Aging processes may affect physical capabilities and cognitive skills, which can hamper testing with this demographic and may require special attention and revised protocols. This study was performed to provide expert-based recommendations for HIT user-testing with (vulnerable) older adults to support inclusive HIT design and evaluation.

METHODS

First, we conducted a structured workshop with ten experts in HIT implementation and research, recruited through purposeful sampling, to generate insights into how characteristics of older adults may influence user-testing. Next, five Human Factor researchers experienced in HIT user-testing with (vulnerable) older adults validated the results and provided additional textual insights to gain consensus on the most important recommendations. A thematic analysis was performed on the resulting inquiries. Applied codes were based on the User-Centered Design framework.

RESULTS

The analysis resulted in nine recommendations for user-testing of HIT with older adults, divided into three main themes: (1) empathetic approach and trust-building, (2) new requirements for testing and study design, and (3) adjustments to usability evaluation methods. For each theme a checklist of relevant items to follow-up on the recommendation is provided.

CONCLUSIONS

The recommendations generated through expert inquiry contribute to more effective usability testing of HIT with older adults. This provides an important step towards improved accessibility of HIT amongst older adults through inclusive user-centered design.

Using Simulations to Train Medical Students for Unanticipated Technology Failures in Telemedicine

Simulations offer a safe environment for health professional training and the opportunity to predictably and consistently introduce events or variables that may be rare or dangerous in a live setting. Exposing trainees to unanticipated events during simulations can improve their ability to adapt and improvise. The COVID-19 pandemic accelerated the adoption of telehealth worldwide and highlighted the need for better training in health professional schools. In the United States, the Association of American Medical Colleges (AAMC) published new telehealth competency standards in 2021. The AAMC stated that health care providers should be aware of the risks of technology failures, capable of troubleshooting them, and lead systems interventions to improve safety. However, the AAMC does not provide guidance on the specific failures or solutions. In this study, we developed a set of technology failures that can be simulated in a telehealth curriculum. We incorporated one technology failure into a simulated telehealth encounter and gathered students' (N = 53) feedback on the exercise. Students' feedback was overwhelmingly positive. They agreed that integrating technology failures into telehealth simulations provides important practice managing these events during clinical encounters. While telehealth is an important healthcare delivery modality that can improve access-to-care, it is imperative to train medical students to navigate technology failures so that can adeptly manage these issues in clinical practice.

Why Do People Use Online Lab Results and What Do They Look For: A Qualitative Study

Laboratory (lab) test results are increasingly available online for patient review. However, there is a dearth of research with respect to users' information needs, goals, and information processing strategies. In this exploratory qualitative study, we interviewed a sample of (N = 25) online lab results users to understand their objectives and search targets. We transcribed their responses and used affinity diagramming to identify themes in their responses. Our analysis identified six reasons why people look at their online lab results (i.e., health status, reassurance, health education, speed, self-management, and patient safety) and two themes about what people look for (i.e., abnormal and normal values, trends). Knowing what drives users and what information they are looking for can inform the design of online lab reporting, improve usefulness, and better satisfy user needs.

Improving Shared Decision-Making Using Cognitive Effort-Optimization

Diabetes Prevention Programs (DPPs) can prevent or delay type 2 diabetes (T2D). However, the participation rates in DPPs have been limited. Many individuals at risk of developing diabetes have difficulties making healthy choices because of the cognitive effort required to understand the risks, the role of biomarkers, the consequences of inaction and the actions required to delay or avoid development of T2D. We report on the design and development of a prototype digital tool that decreases cognitive effort for people at risk of developing T2D using the effort-optimized intervention framework.

Are Personal Health Records (PHRs) Facilitating Patient Safety? A Scoping Review

Personal Health Records (PHRs) are poised to improve patient safety, however the mechanism(s) in which they improve safety is not clear. To this end, we conducted a scoping review with the following objectives: 1) explore the extent of the evidence that PHRs improve patient safety, 2) determine where PHR research has been done per International Medical Informatics Association (IMIA) Represented Region [1], 3) to identify the PHR naming convention(s) used per IMIA Region [1]. The findings revealed that there is limited evidence that PHRs improve patient safety. The results also revealed heterogeneity in PHR nomenclature and how they were used in healthcare settings. However, the overarching theme of the study, was that future research is needed to ensure that PHRs are designed and used in a patient safety context with human factors and usability considerations.

Teaching Teledermatology: A Simulation Pilot for Health Professional Students

We developed a teledermatology simulation to give medical and physician assistant students practice with live videoconferencing and store-and-forward workflows. The simulation included (1) pre-session reading; (2) a brief teledermatology didactic; (3) a simulated encounter with a standardized patient; and (4) faculty-led debriefs. The faculty observed students during the simulation and distributed a post-session learner satisfaction survey. Although students had mixed feelings about the simulation, 88% said the workshop met or exceeded expectations.

A Telepsychiatry Simulation for Suicide Assessment: Teaching Telemedicine Safety Competencies

Educators must provide controlled scenarios for health professional students to develop patient safety competencies related to telemedicine, including when and how to escalate care. We developed a telepsychiatry workshop to give students experience with a high-stakes mental health condition. The workshop included (1) pre-session readings; (2) didactics on mood disorders and telepsychiatry; (3) a motivational interviewing exercise; (4) a simulated telemedicine encounter; and (5) a faculty-led group debrief. We evaluated teaching effectiveness using a competency assessment with three scales: (1) medical knowledge; (2) interpersonal and communication skills; and (3) telemedicine competencies. Between 0 and 59% of students were entrustable for each telemedicine competency. Our workshop demonstrates how to teach students about the safe use of telehealth technology and provides practice triaging mental health conditions commonly encountered in primary care and mental health telemedicine clinics.

Opportunities to Improve COVID-19 Dashboard Designs for the Public

Many organizations created COVID-19 dashboards to communicate epidemiologic statistics or community health capabilities with the public. In this paper we used dashboard heuristics to identify common violations observed in COVID-19 dashboards targeted to citizens. Many of the faults we identified likely stem from failing to include users in the design of these dashboards. We urge health information dashboard designers to implement design principles and test dashboards with representative users to ensure that their tools are satisfying user needs.

The Application of a Novel, Context Specific, Remote, Usability Assessment Tool to Conduct a Pre-Redesign and Post-Redesign Usability Comparison of a Telemedicine Website

The usability of telemedicine provider websites is an essential component of increasing the public's awareness of, understanding of, and safe and effective use of telemedicine. A newly developed usability data collection tool was used to conduct a usability assessment of the Teladoc website pre- and post- a redesign. The results suggest that the Teladoc website had better usability prior to the redesign. The Teladoc Website Usability Survey (TWUS) developed for the Teladoc website usability assessment can be modified and used in future usability studies of telemedicine provider interfaces. Usability research that takes place in specific context of use settings is more valuable to identify usability problems.

COVID-19 Vaccine Hesitancy in Canada: Content Analysis of Tweets Using the Theoretical Domains Framework

Background

With the approval of two COVID-19 vaccines in Canada, many people feel a sense of relief, as hope is on the horizon. However, only about 75% of people in Canada plan to receive one of the vaccines.

Objective

The purpose of this study is to determine the reasons why people in Canada feel hesitant toward receiving a COVID-19 vaccine.

Methods

We screened 3915 tweets from public Twitter profiles in Canada by using the search words “vaccine” and “COVID.” The tweets that met the inclusion criteria (ie, those about COVID-19 vaccine hesitancy) were coded via content analysis. Codes were then organized into themes and interpreted by using the Theoretical Domains Framework.

Results

Overall, 605 tweets were identified as those about COVID-19 vaccine hesitancy. Vaccine hesitancy stemmed from the following themes: concerns over safety, suspicion about political or economic forces driving the COVID-19 pandemic or vaccine development, a lack of knowledge about the vaccine, antivaccine or confusing messages from authority figures, and a lack of legal liability from vaccine companies. This study also examined mistrust toward the medical industry not due to hesitancy, but due to the legacy of communities marginalized by health care institutions. These themes were categorized into the following five Theoretical Domains Framework constructs: knowledge, beliefs about consequences, environmental context and resources, social influence, and emotion.

Conclusions

With the World Health Organization stating that one of the worst threats to global health is vaccine hesitancy, it is important to have a comprehensive understanding of the reasons behind this reluctance. By using a behavioral science framework, this study adds to the emerging knowledge about vaccine hesitancy in relation to COVID-19 vaccines by analyzing public discourse in tweets in real time. Health care leaders and clinicians may use this knowledge to develop public health interventions that are responsive to the concerns of people who are hesitant to receive vaccines.

The Medium Is the Message: How Do Canadian University Students Want Digital Medication Information?

(1) Background: To facilitate optimal prescription medication benefits and safety, it is important that people are informed about their prescription medications. As we shift towards using the digital medium to communicate medication information, it is important to address the needs and preferences of different user groups so that they are more likely to read and use this information. In this study, we examined what digital medication information (DMI) format Canadian University students want and why. (2) Methods: This study was a qualitative investigation of young (aged 18-35) Canadian University students' (N = 36) preferences and rationale supporting these preferences with respect to three potential formats for providing DMI: email, a mobile application (app), and online. Reported advantages and disadvantages of each of the three DMI formats were identified and categorized into unique themes. (3) Results: Findings from this study suggest that Canadian University Students most want to receive DMI by email, followed by a mobile app, and finally they were least receptive to online DMI. Participants provided diverse themes of reasons supporting their preferences. (4) Conclusions: Different user groups may have different needs with respect to receiving DMI. The themes from this study suggest that using a formative evaluation framework for assessing different DMI formats may be useful in future research. Email may be the best way to share DMI with younger, generally healthy, Canadian University students who are on few medications. Further research is required to explore whether other mediums for DMI are more appropriate for users with other characteristics (e.g., older and less educated) and contexts (e.g., polypharmacy and complex conditions). Given the flexibility of digital information, DMI could plausibly be provided in multiple formats and could allow users to choose the option they like best and would be most likely to use.

Differences in Memory, Perceptions, and Preferences of Multimedia Consumer Medication Information: Experimental Performance and Self-Report Study

Background

Electronic health resources are becoming prevalent. However, consumer medication information (CMI) is still predominantly text based. Incorporating multimedia into CMI (eg, images, narration) may improve consumers’ memory of the information as well as their perceptions and preferences of these materials.

Objective

This study examined whether adding images and narration to CMI impacted patients’ (1) memory, (2) perceptions of comprehensibility, utility, or design quality, and (3) overall preferences.

Methods

We presented 36 participants with CMI in 3 formats: (1) text, (2) text + images, and (3) narration + images, and subsequently asked them to recall information. After seeing all 3 CMI formats, participants rated the formats in terms of comprehensibility, utility, and design quality, and ranked them from most to least favorite.

Results

Interestingly, no significant differences in memory were observed (F_2,70=0.1, P=0.901). Thus, this study did not find evidence to support multimedia or modality principles in the context of CMI. Despite the absence of effects on memory, the CMI format significantly impacted perceptions of the materials. Specifically, participants rated the text + images format highest in terms of comprehensibility (χ²₂=26.5, P<.001) and design quality (χ²₂=35.69, P<.001). Although the omnibus test suggested a difference in utility ratings as well (χ²₂=8.21, P=.016), no significant differences were found after correcting for multiple comparisons. Consistent with perception findings, the preference ranks yielded a significant difference (χ²₂=26.00, P<.001), whereby participants preferred the text + images format overall. Indeed, 75% (27/36) of participants chose the text + images format as their most favorite. Thus, although there were no objective memory differences between the formats, we observed subjective differences in comprehensibility, design quality, and overall preferences.

Conclusions

This study revealed that although multimedia did not appear to influence memory of CMI, it did impact participants’ opinions about the materials. The lack of observed differences in memory may have been due to ceiling effects, memory rather than understanding as an index of learning, the fragmented nature of the information in CMI itself, or the size or characteristics of the sample (ie, young, educated subjects with adequate health literacy skills). The differences in the subjective (ie, perceptions and preferences) and objective (ie, memory) results highlight the value of using both types of measures. Moreover, findings from this study could be used to inform future research on how CMI could be designed to better suit the preferences of consumers and potentially increase the likelihood that CMI is used. Additional research is warranted to explore whether multimedia impacts memory of CMI under different conditions (eg, older participants, subjects with lower levels of health literacy, more difficult stimuli, or extended time for decay).

Online Medication Information for Citizens: A Comparison of Demands on eHealth Literacy

Many people take prescription medications and need information about the risks and benefits associated with taking them. Citizens are increasingly turning to the internet for health information and medication information is no exception. There are a variety of websites that offer Online Medication Information for Citizens (OMIC). This study compared six such websites using the Health Literacy Online (HLO) Checklist as a framework. Additionally, we conducted a detailed analysis of the individual content in each OMIC for three different medications. We identified several strengths and weaknesses of the different websites in terms of how they were designed and written and their appropriateness for users with limited eHealth literacy.

User Experience Theories, Models, and Frameworks: A Focused Review of the Healthcare Literature

User experience (UX) theories, models, and frameworks (TMFs) help scope user-centered design activities, aid in the selection of constructs and measures, and contextualize findings within a larger knowledge base. However, the fragmentation of literature across disciplines and the inconsistent use of TMFs makes integrating concepts and selecting UX tools challenging. Therefore, we conducted a focused review of the healthcare literature to identify a succinct list of UX-specific TMFs for academic UX researchers and industry practitioners alike.

Narratives and Stories: Novel Approaches to Improving Patient-Facing Information Resources and Patient Engagement

Patient-centered healthcare requires development of materials for health consumers that increase health literacy, enrich the provider-patient dialog, empower shared decision-making, and improve downstream outcomes. Unfortunately, evidence suggests current methods of communication, including print and electronic media, are inadequate. The Narrative Theory of Learning is grounded in the premise that humans define their experiences and form cognitive structures (e.g., new learning, novel concepts) within the context of narratives. Simply put, humans remember stories better than fragmented bits of information. Therefore, we propose leveraging the power of narratives and stories to improve the efficacy and impact of consumer health applications. We describe several examples of future technologies that could incorporate narrative techniques and present a call to action for future research and development.

We Built It, But They Are Not Coming: Exploring Deterrents to Consumer Medication Information Use

Given the prevalence of prescription medication use, it is important that consumers are aware of the benefits and risks of taking their prescribed medications. One approach to informing consumers in North America is to provide them with Consumer Medication Information (CMI), the paper leaflets given to consumers when they fill a prescription for the first time. Unfortunately, reported use rates of written medication information are quite low. As part of a broader study investigating memory, perceptions, preferences and information needs around CMI, this study specifically examined reported deterrents to CMI use. Findings from this study revealed three areas that appear to influence CMI use: 1) Documentation, how CMI is designed and what it contains; 2) Provision, how and when CMI is given to consumers; and 3) Context, what the individual's characteristics and experiences are. These three factors warrant further investigation to reveal more of their unique facets and their relative influences on CMI use. That is, some aspects may be more influential than others.

Development of a Video Coding Scheme for Understanding Human-Computer Interaction and Clinical Decision Making

The usability of healthcare information technology has become a major issue in health informatics. There have been many reports of systems that have been deemed unusable by end users such as clinicians and a growing body of usability studies have been reported in the literature. The issue of how to fruitfully analyze and code usability study data in a meaningful way that can lead to optimized and more efficient systems has remained to be fully detailed. In this paper we describe our work in developing and organizing a principled video coding scheme that builds from our previous work in a couple of areas. We include video coding categories we have developed for understanding problems and issues with human-computer interaction. In addition, we integrate this coding scheme with categories we have used to characterize human cognition, such as clinical reasoning and decision making, in isolation of technology use. The resultant new scheme thus incorporates coding categories that can used to evaluate both usability issues (applying categories from human-computer interaction) and human cognition, in order to assess the impact of technology on clinical reasoning and decision making.

Putting Guidelines in the Hands of Patients: A Heuristic Evaluation of a Consumer Mobile Application

Preventive clinical practice guidelines are valuable. However, they are often difficult for primary care providers to implement in the time sensitive primary care setting and alternate approaches warrant exploration. An app called CANBeWell was developed in an effort to allow consumers to identify appropriate guidelines for themselves. Two investigators conducted a heuristic evaluation to identify potential eHealth literacy and usability issues. Several recommendations were made to make CANBeWell easier for consumers to use and understand. CANBeWell is a promising app for deploying preventive guidelines directly to health consumers. Usability testing is planned for the next iteration to ensure that this app meets the needs and capabilities of health consumers.

Usability and eHealth Literacy Evaluation of a Mobile Health Application Prototype to Track Diagnostic Imaging Examinations

In the absence of a pan-Canadian electronic health record, unnecessary duplicate diagnostic imaging (DI) examinations are sometimes ordered when a physician is not aware of prior DI or prior DI is not available. Research suggests that physicians may ask their patient to recall their DI history; however, patient recall of DI can be unreliable. As a potential solution, a patient-facing mobile health application (app) prototype was developed for users to record their DI. The app was designed to be usable and inclusive to users of all health literacy levels. The aim of this paper is to demonstrate how eHealth literacy and usability heuristics can be used during the design phases of app development.

Are Health Literacy and eHealth Literacy the Same or Different?

Many researchers assume that there is a relationship between health literacy and eHealth literacy, yet it is not clear whether the literature supports this assumption. The purpose of this study was to determine if there was a relationship between health and eHealth literacy. To this end, participants' (n = 36) scores on the Newest Vital Sign (NVS, a health literacy measure) were correlated with the eHealth Literacy Scale (eHEALS, an eHealth literacy measure). This analysis revealed no relationship (r = -.041, p = .81) between the two variables. This finding suggests that eHealth Literacy and health literacy are dissimilar. Several possible explanations of the pattern of results are proposed. Currently, it does not seem prudent to use the eHEALS as the sole measure of eHealth literacy, but rather researchers should continue to complement it with a validated health literacy screening tool.

All Consumer Medication Information Is Not Created Equal: Implications for Medication Safety

Many Canadians take prescription medications. These medications often have both benefits and potential consequences associated with their use. Additionally, instructions for how to administer some medications (e.g., inhalers, eye drops) maybe be critical in maximizing the beneficial effects of using medications. This study examined the Consumer Medication Information (CMI) from a leading Canadian pharmacy and revealed none of the 10 CMI contained information about allergic reactions, overdoses, or drug interactions. This lack of information may come at the expense patient safety. Additionally, much of the content identified as important was not readily differentiable from the surrounding text. Further, inhalers were the only category of medication that did not have specific use instructions but instead directed consumers to consult other resources. Thus, there are opportunities to augment CMI to improve the safe and effective use of medications. The shortcomings of CMI identified in this study represent important considerations for CMI as it is currently delivered (printed text) and opportunities to improve upon CMI as it is beginning to be offered online.

Consumer Medication Information: Similarities and Differences Between Three Canadian Pharmacies

Prescription medication use is prevalent. When a new prescription medication is dispensed, Consumer Medication Information (CMI) is provided to communicate various important aspects of the medication (e.g., benefits, administration instructions, potential side effects). However, CMI is not regulated and differs from pharmacy to pharmacy. This study explores the similarities and differences between the CMI from three pharmacies (two paper print outs and one online source) for a single medication. The three CMI were assessed in terms of readability and utility. This evaluation revealed drastic differences in the length of the CMI (Range = 453 to 2 337 words). The online CMI was longer, described more topics and provided more detail than the print versions. Although online CMI has the advantage of interactivity to expedite navigation to specific topics of interest (e.g., heading links) and searching for key words, this CMI was not layered but rather presented as one long continuous page. Consumers with lower eHealth literacy skills may be deterred by the length of the document. As CMI makes the shift to online presentation an improved understanding of optimal information organization and media presentation will be needed.

A Comparison of Danish and Canadian Consumer Medication Information

Many people around the world use prescription medications. Consumers often require information about their medications to support taking them safely and effectively. One source of such information is Consumer Medication Information (CMI). Canadians typically receive printed CMI when a new prescription is filled whereas Danes have the online resource min.medicin.dk. This study compared the content and design of Danish and Canadian CMI. Danish CMI satisfied seven of the 11 content utility criteria (developed in previous work) identified as supporting the safe and effective medication use. However, Danish CMI provided a more information about how frequently possible side effects occur and multimedia (e.g., images, videos) directions for some medications. This study examined some of the similarities and differences between how Canadians and Danes are informed about medications. However, further research is required to determine what content and methods of delivery are most beneficial in supporting safe and effective medication use.

Isolating the Effects of a Mobile Phone on the Usability and Safety of eHealth Software Applications

Mobile phones are used in conjunction with mobile eHealth software applications. These mobile software applications can be used to access, review and document clinical information. The objective of this research was to explore the relationship between mobile phones, usability and safety. Clinical simulations and semi-structured interviews were used to investigate this relationship. The findings revealed that mobile phones may lead to specific types of usability issues that may introduce some types of errors.

How to Present Evidence-Based Usability Design Principles Dedicated to Medication-Related Alerting Systems to Designers and Evaluators? Results from a Workshop

Medication alerting system use errors and lack of adoption are often attributed to usability issues. Previous work has used evidence from the literature to reveal usability principles specific to medication alerting systems and identify potential consequences of violating these principles. The current study sought to explore how best to convey these principles to designers and evaluators of these systems to facilitate their work. To this aim, a workshop with 19 participants was used to generate ideas and opinions on how to deliver these topic-specific design principles in a way that would be most helpful for them. Participants generated ideas for how (e.g., a collaborative, continuously updated forum) and what (e.g., illustrations, checklists, evidence sources and strength, consequences of violations) information is most useful to disseminate usability principles for medication alerting systems. Participants, especially designers, expressed desire to use these principles in practice and avoid previously documented mistakes and therefore make design and evaluation of these systems more effective and efficient. Those insights are discussed in terms of feasibility and logistical challenges to developing the proposed documentation). To move this work forward, a more collaborative approach of Human Factors specialists in medical informatics is necessary.

InformedTogether: Usability Evaluation of a Web-Based Decision Aid to Facilitate Shared Advance Care Planning for Severe Chronic Obstructive Pulmonary Disease

Background

Advance care planning may help patients receive treatments that better align with their goals for care. We developed a Web-based decision aid called InformedTogether to facilitate shared advance care planning between chronic obstructive pulmonary disease (COPD) patients and their doctors.

Objective

Our objective was to assess the usability of the InformedTogether decision aid, including whether users could interact with the decision aid to engage in tasks required for shared decision making, whether users found the decision aid acceptable, and implications for redesign.

Methods

We conducted an observational study with 15 patients and 8 doctors at two ethnically and socioeconomically diverse outpatient clinics. Data included quantitative and qualitative observations of patients and doctors using the decision aid on tablet or laptop computers and data from semistructured interviews. Patients were shown the decision aid by a researcher acting as the doctor. Pulmonary doctors were observed using the decision aid independently and asked to think aloud (ie, verbalize their thoughts). A thematic analysis was implemented to explore key issues related to decision aid usability.

Results

Although patients and doctors found InformedTogether acceptable and would recommend that doctors use the decision aid with COPD patients, many patients had difficulty understanding the icon arrays that were used to communicate estimated prognoses and could not articulate the definitions of the two treatment choices—Full Code and Do Not Resuscitate (DNR). Minor usability problems regarding content, links, layout, and consistency were also identified and corresponding recommendations were outlined. In particular, participants suggested including more information about potential changes in quality of life resulting from the alternative advance directives. Some doctor participants thought the decision aid was too long and some thought it may cause nervousness among patients due to the topic area.

Conclusions

A decision aid for shared advance care planning for severe COPD was found acceptable to most COPD patients and their doctors. However, many patient participants did not demonstrate understanding of the treatment options or prognostic estimates. Many participants endorsed the use of the decision aid between doctors and their patients with COPD, although they desired more information about quality of life. The design must optimize comprehensibility, including revising the presentation of statistical information in the icon array, and feasibility of integration into clinical workflow, including shortening the decision aid.

Using personal health records to scaffold perceived self-efficacy for health promotion

According to Bandura (1977), believing in one's ability to achieve a goal is one of the best predictors that a goal will be accomplished. Given its predictive power, the concept of belief in one's ability to succeed, or perceived self-efficacy, is well researched for its influence on health promotion. It has been argued that a paradigm shift must occur away from illness treatment towards illness prevention and health promotion, for healthcare to accommodate the needs of the population. Personal Health Records (PHRs) may be a tool to help facilitate this paradigm shift. PHRs are repositories of information that individuals can use to access, manage, and share their personal health information. An extension of Bandura's model of self-efficacy will be presented here which identifies opportunities for PHRs to enhance perceived self-efficacy through mastery, social modeling, social persuasion, and physiological state. Bolstering self-efficacy through PHR tools will expand the utility of PHRs beyond self-management to also facilitate health promotion and illness prevention and gains in self-efficacy are also likely to transcend into other areas of consumers' lives.