Features of effective medical knowledge resources to support point of care learning: a focus group study

Identifying primary care clinicians' preferences for, barriers to, and facilitators of information-seeking in clinical practice in Singapore: a qualitative study

BACKGROUND

The growth of medical knowledge and patient care complexity calls for improved clinician access to evidence-based resources. This study aimed to explore the primary care clinicians' preferences for, barriers to, and facilitators of information-seeking in clinical practice in Singapore.

METHODS

A convenience sample of ten doctors and ten nurses was recruited. We conducted semi-structured face-to-face in-depth interviews. The interviews were recorded, transcribed verbatim, and analysed using thematic content analysis.

RESULTS

Of the 20 participants, eight doctors and ten nurses worked at government-funded polyclinics and two doctors worked in private practice. Most clinicians sought clinical information daily at the point-of-care. The most searched-for information by clinicians in practice was less common conditions. Clinicians preferred evidence-based resources such as clinical practice guidelines and UpToDate®. Clinical practice guidelines were mostly used when they were updated or based on memory. Clinicians also commonly sought answers from their peers. Furthermore, clinicians frequently use smartphones to access the Google search engine and UpToDate® app. The barriers to accessing clinical information included the lack of time, internet surfing separation of work computers, limited search functions in the organisation's server, and limited access to medical literature databases. The facilitators of accessing clinical information included convenience, easy access, and trustworthiness of information sources.

CONCLUSION

Most primary care clinicians in our study sought clinical information at the point-of-care daily and reported increasing use of smartphones for information-seeking. Future research focusing on interventions to improve access to credible clinical information for primary care clinicians at the point-of-care is recommended.

TRIAL REGISTRATION

This study has been reviewed by NHG Domain Specific Review Board (NHG DSRB) (the central ethics committee) for ethics approval. NHG DSRB Reference Number: 2018/01355 (31/07/2019).

Evaluating the Feasibility and Acceptance of a Mobile Clinical Decision Support System in a Resource-Limited Country: Exploratory Study

BACKGROUND

In resource-limited countries, access to specialized health care services such as dermatology is limited. Clinical decision support systems (CDSSs) offer innovative solutions to address this challenge. However, the implementation of CDSSs is commonly associated with unique challenges. VisualDx-an exemplar CDSS-was recently implemented in Botswana to provide reference materials in support of the diagnosis and management of dermatological conditions. To inform the sustainable implementation of VisualDx in Botswana, it is important to evaluate the intended users' perceptions about the technology.

OBJECTIVE

This study aims to determine health care workers' acceptance of VisualDx to gauge the feasibility of future adoption in Botswana and other similar health care systems.

METHODS

The study's design was informed by constructs of the Technology Acceptance Model. An explanatory, sequential, mixed methods study involving surveys and semistructured interviews was conducted. The REDCap (Research Electronic Data Capture; Vanderbilt University) platform supported web-based data capture from March 2021 through August 2021. In total, 28 health care workers participated in the study. Descriptive statistics were generated and analyzed using Excel (Microsoft Corp), and thematic analysis of interview transcripts was performed using Delve software.

RESULTS

All survey respondents (N=28) expressed interest in using mobile health technology to support their work. Before VisualDx, participants referenced textbooks, journal articles, and Google search engines. Overall, participants' survey responses showed their confidence in VisualDx (18/19, 95%); however, some barriers were noted. Frequently used VisualDx features included generating a differential diagnosis through manual entry of patient symptoms (330/681, 48.5% of total uses) or using the artificial intelligence feature to analyze skin conditions (150/681, 22% of total uses). Overall, 61% (17/28) of the survey respondents were also interviewed, and 4 thematic areas were derived.

CONCLUSIONS

Participants' responses indicated their willingness to accept VisualDx. The ability to access information quickly without internet connection is crucial in resource-constrained environments. Selected enhancements to VisualDx may further increase its feasibility in Botswana. Study findings can serve as the basis for improving future CDSS studies and innovations in Botswana and similar resource-limited countries.

Organizing and Sharing Medical Knowledge Among Anesthesiology and Intensive care Residents: Evaluating Existing Practices and the Feasibility of Implementing a Dedicated Multiplatform Application

Treating patients with up-to-date medical knowledge is an ongoing goal for healthcare workers and implies efficient knowledge management at the point of care. Widely available mobile wireless technologies influence practices but a significant gap remains between technological possibilities and actual usage. The purpose of this study was to analyze residents' baseline practices in managing medical knowledge and to evaluate the use and impact of an innovative multiplatform application dedicated to anesthesiology and intensive care residents. This study took place in Rennes Teaching Hospital and comprised two distinct surveys. First, in April 2018, all residents received a ten-items online survey focusing on managing medical knowledge. Then, through a second online survey constituted of ten items, we sought to assess the use of a new multiplatform cloud-based application named "DansMaBlouse", dedicated to sharing and indexing medical knowledge, in anesthesiology and intensive care residents. Among 148 residents that answered the evaluation survey, the most sought out pieces of information in clinical setting were a phone or fax number (74%), drugs' characteristics (68%) and expert guidelines (57%). The main sources were senior staff (68%), medical databases (60%) and an Internet search engine (59%). Computers and smartphones were more frequently used than bound paper notebooks. After implementation of the multiplatform application DansMaBlouse, fifty-nine (82%) of the 72 residents that answered the evaluation survey reported using the application and 39% used it more than ten times. Among application users, 90% found it easy to use and 92% agreed that it improved point-of-care access to knowledge. Accessing appropriate medical knowledge at the point of care remains an issue for residents and can be improved by a multiplatform application combining personal and shared up-to-date resources.

Clinician acceptability of an antibiotic prescribing knowledge support system for primary care: a mixed-method evaluation of features and context

BACKGROUND

Overprescribing of antibiotics is a major concern as it contributes to antimicrobial resistance. Research has found highly variable antibiotic prescribing in (UK) primary care, and to support more effective stewardship, the BRIT Project (Building Rapid Interventions to optimise prescribing) is implementing an eHealth Knowledge Support System. This will provide unique individualised analytics information to clinicians and patients at the point of care. The objective of the current study was to gauge the acceptability of the system to prescribing healthcare professionals and highlight factors to maximise intervention uptake.

METHODS

Two mixed-method co-design workshops were held online with primary care prescribing healthcare professionals (n = 16). Usefulness ratings of example features were collected using online polls and online whiteboards. Verbal discussion and textual comments were analysed thematically using inductive (participant-centred) and deductive perspectives (using the Theoretical Framework of Acceptability).

RESULTS

Hierarchical thematic coding generated three overarching themes relevant to intervention use and development. Clinician concerns (focal issues) were safe prescribing, accessible information, autonomy, avoiding duplication, technical issues and time. Requirements were ease and efficiency of use, integration of systems, patient-centeredness, personalisation, and training. Important features of the system included extraction of pertinent information from patient records (such as antibiotic prescribing history), recommended actions, personalised treatment, risk indicators and electronic patient communication leaflets. Anticipated acceptability and intention to use the knowledge support system was moderate to high. Time was identified as a focal cost/ burden, but this would be outweighed if the system improved patient outcomes and increased prescribing confidence.

CONCLUSION

Clinicians anticipate that an eHealth knowledge support system will be a useful and acceptable way to optimise antibiotic prescribing at the point of care. The mixed method workshop highlighted issues to assist person-centred eHealth intervention development, such as the value of communicating patient outcomes. Important features were identified including the ability to efficiently extract and summarise pertinent information from the patient records, provide explainable and transparent risk information, and personalised information to support patient communication. The Theoretical Framework of Acceptability enabled structured, theoretically sound feedback and creation of a profile to benchmark future evaluations. This may encourage a consistent user-focused approach to guide future eHealth intervention development.

Experiences and Attitudes of Elementary School Students and Their Parents Toward Online Learning in China During the COVID-19 Pandemic: Questionnaire Study

Background

Due to widespread SARS-CoV-2 infection, an emergency homeschooling plan was rigorously implemented throughout China.

Objective

This study aimed to investigate the experiences and attitudes of elementary school students and their parents (two generations from the same family) toward online learning in China during the pandemic.

Methods

A 16-item questionnaire was distributed at the 10-day and 40-day marks after the start of the first online course to 867 parent-child pairs and 141 parent-child pairs, respectively. The questionnaire was comprised of questions pertaining to course and homework completeness, effectiveness, reliability, and abundance as well as the students’ enthusiasm for taking part in online classes and their satisfaction with the courses.

Results

Our findings indicate that 90.7% (786/867) of students exhibited high or moderate enthusiasm for participating in online classes. However, most students performed poorly in online learning classes and after-school homework. With regard to satisfaction, parents' and students' average scores were 7.35 and 7.25, respectively (10-point scoring system). During the second stage of this study, parents' positive evaluations for online learning declined, including those for the effectiveness and reliability of the courses. Furthermore, the proportion of students who completed the courses and homework on time decreased; this difference proved statistically significant (P=.047). The parents’ and students’ overall satisfaction with online learning also declined during the second stage (parents: 7.21; students: 7.23); however, the difference in overall satisfaction between the two stages was not statistically significant (parents: P=.53; students: P=.60). Several of the parents (315/867, 36.2%) indicated that assisting with and supervising the students’ online learning resulted in increased stress. Further, 36% of parents expressed dissatisfaction with or provided suggestions for online learning; most parents and students hoped to return to face-to-face classes (parents: 823/867, 94.9%; students: 811/867, 93.5%). Finally, our results presented the following six main issues that parents were the most concerned about: (1) disappointment regarding timely interaction in courses; (2) apprehensiveness about students’ understanding of the course; (3) the increased burden of annoying adult responsibilities; (4) concern about children's eyesight; (5) the idea that teachers’ explanations were not detailed enough; and (6) concerns about the decline of students' interest in and attention toward online courses.

Conclusions

Online learning can prevent the spread of infectious diseases while still allowing elementary school students to attain knowledge. However, in our study, children’s completion of the courses and homework were not satisfactory. Furthermore, their parents often experienced stress and had many concerns and complaints. Measures such as increasing the interactivity of the courses and prohibiting teachers from assigning tasks to parents could improve the effectiveness of these courses and the mental health of parents and students.

Barriers and facilitators to clinical information seeking: a systematic review

OBJECTIVE

The study sought to identify barriers to and facilitators of point-of-care information seeking and use of knowledge resources.

MATERIALS AND METHODS

We searched MEDLINE, Embase, PsycINFO, and Cochrane Library from 1991 to February 2017. We included qualitative studies in any language exploring barriers to and facilitators of point-of-care information seeking or use of electronic knowledge resources. Two authors independently extracted data on users, study design, and study quality. We inductively identified specific barriers or facilitators and from these synthesized a model of key determinants of information-seeking behaviors.

RESULTS

Forty-five qualitative studies were included, reporting data derived from interviews (n = 26), focus groups (n = 21), ethnographies (n = 6), logs (n = 4), and usability studies (n = 2). Most studies were performed within the context of general medicine (n = 28) or medical specialties (n = 13). We inductively identified 58 specific barriers and facilitators and then created a model reflecting 5 key determinants of information-seeking behaviors: time includes subthemes of time availability, efficiency of information seeking, and urgency of information need; accessibility includes subthemes of hardware access, hardware speed, hardware portability, information restriction, and cost of resources; personal skills and attitudes includes subthemes of computer literacy, information-seeking skills, and contextual attitudes about information seeking; institutional attitudes, cultures, and policies includes subthemes describing external individual and institutional information-seeking influences; and knowledge resource features includes subthemes describing information-seeking efficiency, information content, information organization, resource familiarity, information credibility, information currency, workflow integration, compatibility of recommendations with local processes, and patient educational support.

CONCLUSIONS

Addressing these determinants of information-seeking behaviors may facilitate clinicians' question answering to improve patient care.

Low performance of internal medicine senior residents in in-service examinations

We observed an unprecedented and consistent low performance of senior residents as compared with juniors in monthly examinations. This compelled us to evaluate systematically and compare the scores of senior residents with their junior colleagues. This retrospective observational study was conducted in April 2020 among internal medicine residents. Residents in first and second year of their training were labelled as junior residents. Residents in third or fourth year of their training were labelled as senior residents. Comparison of mean scores of each resident level was done separately both for monthly formative multiple-choice questions tests, and summative yearly end of term examinations. We discussed the possible reasons as well. There were 67 residents in year 2018 and 69 in 2019. There is no significant difference between scores of monthly examinations of years 2018 and 2019 among residents of each level. Rather, in March and December 2018, junior residents perform better than senior residents with p values of 0.01 and 0.04, respectively. In February and September 2019, senior residents performed better than junior residents with p value of 0.02. Similarly, there is no significant difference in scores of residents of each level in end-of-term examinations of years 2018 and 2019 with p values 0.18 and 0.25, respectively. Performance of senior residents in our residency programme in in-service examinations is relatively low as compared with their junior colleagues. There is a need to evaluate reasons for this relatively low performance of senior residents.

What Influences Choice of Continuing Medical Education Modalities and Providers? A National Survey of U.S. Physicians, Nurse Practitioners, and Physician Assistants

PURPOSE

To explore what influences clinicians in selecting continuing medical education (CME) activities in the United States.

METHOD

In August 2018, the authors conducted an Internet-based national survey, sampling 100 respondents from each of 5 groups: family medicine physicians, internal medicine and hospitalist physicians, medicine specialist physicians, nurse practitioners, and physician assistants. In total, 1,895 clinicians were invited and 500 (26%) responded. Questions addressed the selection and anticipated use of CME delivery modalities and perceived characteristics of specific CME providers. Response formats used best-worst scaling or 5-point ordinal response options.

RESULTS

The factors identified as most important in selecting CME activities were topic (best-worst scaling net positivity 0.54), quality of content (0.51), availability of CME credit (0.43), and clinical practice focus (0.41), while referral frequency (-0.57) ranked lowest. The activities that the respondents anticipated using most in the future were live (mean 3.8 [1 = not likely, 5 = very likely]), online (mean 3.5), point-of-care (mean 3.5), and print-based (mean 3.5) activities. For online CME, the features of greatest appeal were that learning could be done when clinicians had time (mean 4.4), at their own pace (mean 4.2), and at lower cost (mean 4.2). For live CME, the features of greatest appeal were that the subject was best taught using this modality (mean 4.0), or the activity was located in a destination spot (mean 4.0) or a regional location (mean 3.9). When rating specific CME providers, most academic institutions received relatively high ratings for research focus and clinical practice focus, whereas commercial providers had slightly higher ratings for ease of access. Responses were generally similar across clinician types and age groups.

CONCLUSIONS

Physicians, nurse practitioners, and physician assistants are interested in using a variety of CME delivery modalities. Appealing features of online and live CME were different.

Use of online knowledge base in primary health care and correlation to health care quality: an observational study

Background

Evidence-based information available at the point of care improves patient care outcomes. Online knowledge bases can increase the application of evidence-based medicine and influence patient outcome data which may be captured in quality registries. The aim of this study was to explore the effect of use of an online knowledge base on patient experiences and health care quality.

Methods

The study was conducted as a retrospective, observational study of 24 primary health care centers in Sweden exploring their use of an online knowledge base. Frequency of use was compared to patient outcomes in two national quality registries. A socio-economic Care Need Index was applied to assess whether the burden of care influenced the results from those quality registries. Non-parametric statistical methods and linear regression were used.

Results

Frequency of knowledge base use showed two groups: frequent and non-frequent users, with a significant use difference between the groups (p < 0.001). Outcome data showed significant higher values for all seven National Primary Care Patient Survey dimensions in the frequent compared to the non-frequent knowledge base users (p < 0.001), whereas 10 out of 11 parameters in the National Diabetes Register showed no differences between the groups (p > 0.05). Adjusting for Care Need Index had almost no effect on the outcomes for the groups.

Conclusions

Frequent users of a national online knowledge base received higher ratings on patient experiences, but figures on health care quality in diabetes showed near to no correlation. The findings indicate that some effects may be attributed to the use of knowledge bases and requires a controlled evaluation.

How searching under time pressure impacts clinical decision making

Objective:

Clinicians encounter many questions during patient encounters that they cannot answer. While search systems (e.g., PubMed) can help clinicians find answers, clinicians are typically busy and report that they often do not have sufficient time to use such systems. The objective of this study was to assess the impact of time pressure on clinical decisions made with the use of a medical literature search system.

Design:

In stage 1, 109 final-year medical students and practicing clinicians were presented with 16 clinical questions that they had to answer using their own knowledge. In stage 2, the participants were provided with a search system, similar to PubMed, to help them to answer the same 16 questions, and time pressure was simulated by limiting the participant's search time to 3, 6, or 9 minutes per question.

Results:

Under low time pressure, the correct answer rate significantly improved by 32% when the participants used the search system, whereas under high time pressure, this improvement was only 6%. Also, under high time pressure, participants reported significantly lower confidence in the answers, higher perception of task difficulty, and higher stress levels.

Conclusions:

For clinicians and health care organizations operating in increasingly time-pressured environments, literature search systems become less effective at supporting accurate clinical decisions. For medical search system developers, this study indicates that system designs that provide faster information retrieval and analysis, rather than traditional document search, may provide more effective alternatives.

Barriers and facilitators to use of a clinical evidence technology in the management of skin problems in primary care: insights from mixed methods

Objective:

Few studies have examined the impact of a single clinical evidence technology (CET) on provider practice or patient outcomes from the provider's perspective. A previous cluster-randomized controlled trial with patient-reported data tested the effectiveness of a CET (i.e., VisualDx) in improving skin problem outcomes but found no significant effect. The objectives of this follow-up study were to identify barriers and facilitators to the use of the CET from the perspective of primary care providers (PCPs) and to identify reasons why the CET did not affect outcomes in the trial.

Methods:

Using a convergent mixed methods design, the authors had PCPs complete a post-trial survey and participate in interviews about using the CET for managing patients' skin problems. Data from both methods were integrated.

Results:

PCPs found the CET somewhat easy to use but only occasionally useful. Less experienced PCPs used the CET more frequently. Data from interviews revealed barriers and facilitators at four steps of evidence-based practice: clinical question recognition, information acquisition, appraisal of relevance, and application with patients. Facilitators included uncertainty in dermatology, intention for use, convenience of access, diagnosis and treatment support, and patient communication. Barriers included confidence in dermatology, preference for other sources, interface difficulties, presence of irrelevant information, and lack of decision impact.

Conclusion:

PCPs found the CET useful for diagnosis, treatment support, and patient communication. However, the barriers of interface difficulties, irrelevant search results, and preferred use of other sources limited its positive impact on patient skin problem management.

Characterizing physicians' information needs related to a gap in knowledge unmet by current evidence

OBJECTIVE

The study sought to explore information needs arising from a gap in clinicians' knowledge that is not met by current evidence and identify possible areas of use and target groups for a future clinical decision support system (CDSS), which will guide clinicians in cases where no evidence exists.

MATERIALS AND METHODS

We interviewed 30 physicians in a large academic medical center, analyzed transcripts using deductive thematic analysis, and developed a set of themes of information needs related to a gap in knowledge unmet by current evidence. We conducted additional statistical analyses to identify the correlation between clinical experience, clinical specialty, settings of clinical care, and the characteristics of the needs.

RESULTS

This study resulted in a set of themes and subthemes of information needs arising from a gap in current evidence. Experienced physicians and inpatient physicians had more questions and the number of questions did not decline with clinical experience. The main areas of information needs included patients with comorbidities, elderly and children, new drugs, and rare disorders. To address these questions, clinicians most often used a commercial tool, guidelines, and PubMed. While primary care physicians preferred the commercial tool, specialty physicians sought more in-depth knowledge.

DISCUSSION

The current medical evidence appeared to be inadequate in covering specific populations such as patients with multiple comorbidities and elderly, and was sometimes irrelevant to complex clinical scenarios. Our findings may suggest that experienced and inpatient physicians would benefit from a CDSS that generates evidence in real time at the point of care.

CONCLUSIONS

We found that physicians had information needs, which arose from the gaps in current medical evidence. This study provides insights on how the CDSS that aims at addressing these needs should be designed.

The Efficiency of Online "Inked" Videos Versus Recorded PowerPoint Lectures on Teaching Pathophysiology to Medical Students in Pre-Clerkship Years: A Pilot Study

BACKGROUND

As online learning develops an increasingly important role in medical education, new online teaching modalities are arising all the time. One such teaching modality that is gaining popularity among medical students is the "inked" video, a type of animated video that utilizes a virtual blackboard. Student reviews suggest that the dynamic style of the inked video allows it to teach more efficiently than traditional teaching modalities, but currently there is no quantitative evidence to support or guide the use of this teaching modality.

HYPOTHESIS

When compared to the traditional recorded PowerPoint lecture, online inked videos teach the same pathophysiology concepts to pre-clerkship medical students in a shorter amount of time.

METHOD

A randomized, crossover-design study was conducted with second-year medical students at the David Geffen School of Medicine at University of California, Los Angeles (n = 22). Students were randomized to 2 groups. Each group received 2 lessons on 2 different pathophysiology topics: one via a shorter inked video and the other via a longer recorded PowerPoint lecture. Two sets of dependent variables were used to test modality effects: immediate post-tests scores and delayed retention post-test scores. A perceptions survey was also administered to assess student preferences between the 2 teaching modalities.

RESULTS

Students performed similarly on immediate and delayed post-tests for the shorter inked videos and the longer recorded PowerPoint lectures (P > .05). Students reported greater engagement (P < .05) and greater satisfaction with learning (P < .05) with the inked videos than the recorded PowerPoint lectures.

CONCLUSIONS

This study, although limited by its small-scale and single-institution design, provides preliminary evidence that online inked videos may be a more efficient and non-inferior alternative to recorded PowerPoint lectures.

Management Reasoning: Implications for Health Professions Educators and a Research Agenda

Substantial research has illuminated the clinical reasoning processes involved in diagnosis (diagnostic reasoning). Far less is known about the processes entailed in patient management (management reasoning), including decisions about treatment, further testing, follow-up visits, and allocation of limited resources. The authors' purpose is to articulate key differences between diagnostic and management reasoning, implications for health professions education, and areas of needed research.Diagnostic reasoning focuses primarily on classification (i.e., assigning meaningful labels to a pattern of symptoms, signs, and test results). Management reasoning involves negotiation of a plan and ongoing monitoring/adjustment of that plan. A diagnosis can usually be established as correct or incorrect, whereas there are typically multiple reasonable management approaches. Patient preferences, clinician attitudes, clinical contexts, and logistical constraints should not influence diagnosis, whereas management nearly always involves prioritization among such factors. Diagnostic classifications do not necessarily require direct patient interaction, whereas management prioritizations require communication and negotiation. Diagnoses can be defined at a single time point (given enough information), whereas management decisions are expected to evolve over time. Finally, management is typically more complex than diagnosis.Management reasoning may require educational approaches distinct from those used for diagnostic reasoning, including teaching distinct skills (e.g., negotiating with patients, tolerating uncertainty, and monitoring treatment) and developing assessments that account for underlying reasoning processes and multiple acceptable solutions.Areas of needed research include if and how cognitive processes differ for management and diagnostic reasoning, how and when management reasoning abilities develop, and how to support management reasoning in clinical practice.

Impact of Clinicians' Use of Electronic Knowledge Resources on Clinical and Learning Outcomes: Systematic Review and Meta-Analysis

Background

Clinicians use electronic knowledge resources, such as Micromedex, UpToDate, and Wikipedia, to deliver evidence-based care and engage in point-of-care learning. Despite this use in clinical practice, their impact on patient care and learning outcomes is incompletely understood. A comprehensive synthesis of available evidence regarding the effectiveness of electronic knowledge resources would guide clinicians, health care system administrators, medical educators, and informaticians in making evidence-based decisions about their purchase, implementation, and use.

Objective

The aim of this review is to quantify the impact of electronic knowledge resources on clinical and learning outcomes.

Methods

We searched MEDLINE, Embase, PsycINFO, and the Cochrane Library for articles published from 1991 to 2017. Two authors independently screened studies for inclusion and extracted outcomes related to knowledge, skills, attitudes, behaviors, patient effects, and cost. We used random-effects meta-analysis to pool standardized mean differences (SMDs) across studies.

Results

Of 10,811 studies screened, we identified 25 eligible studies published between 2003 and 2016. A total of 5 studies were randomized trials, 22 involved physicians in practice or training, and 10 reported potential conflicts of interest. A total of 15 studies compared electronic knowledge resources with no intervention. Of these, 7 reported clinician behaviors, with a pooled SMD of 0.47 (95% CI 0.27 to 0.67; P<.001), and 8 reported objective patient effects with a pooled SMD of 0.19 (95% CI 0.07 to 0.32; P=.003). Heterogeneity was large (I²>50%) across studies. When compared with other resources—7 studies, not amenable to meta-analytic pooling—the use of electronic knowledge resources was associated with increased frequency of answering questions and perceived benefits on patient care, with variable impact on time to find an answer. A total of 2 studies compared different implementations of the same electronic knowledge resource.

Conclusions

Use of electronic knowledge resources is associated with a positive impact on clinician behaviors and patient effects. We found statistically significant associations between the use of electronic knowledge resources and improved clinician behaviors and patient effects. When compared with other resources, the use of electronic knowledge resources was associated with increased success in answering clinical questions, with variable impact on speed. Comparisons of different implementation strategies of the same electronic knowledge resource suggest that there are benefits from allowing clinicians to choose to access the resource, versus automated display of resource information, and from integrating patient-specific information. A total of 4 studies compared different commercial electronic knowledge resources, with variable results. Resource implementation strategies can significantly influence outcomes but few studies have examined such factors.

Impact of a Search Engine on Clinical Decisions Under Time and System Effectiveness Constraints: Research Protocol

BACKGROUND

Many clinical questions arise during patient encounters that clinicians are unable to answer. An evidence-based medicine approach expects that clinicians will seek and apply the best available evidence to answer clinical questions. One commonly used source of such evidence is scientific literature, such as that available through MEDLINE and PubMed. Clinicians report that 2 key reasons why they do not use search systems to answer questions is that it takes too much time and that they do not expect to find a definitive answer. So, the question remains about how effectively scientific literature search systems support time-pressured clinicians in making better clinical decisions. The results of this study are important because they can help clinicians and health care organizations to better assess their needs with respect to clinical decision support (CDS) systems and evidence sources. The results and data captured will contribute a significant data collection to inform the design of future CDS systems to better meet the needs of time-pressured, practicing clinicians.

OBJECTIVE

The purpose of this study is to understand the impact of using a scientific medical literature search system on clinical decision making. Furthermore, to understand the impact of realistic time pressures on clinicians, we vary the search time available to find clinical answers. Finally, we assess the impact of improvements in search system effectiveness on the same clinical decisions.

METHODS

In this study, 96 practicing clinicians and final year medical students are presented with 16 clinical questions which they must answer without access to any external resource. The same questions are then represented to the clinicians; however, in this part of the study, the clinicians can use a scientific literature search engine to find evidence to support their answers. The time pressures of practicing clinicians are simulated by limiting answer time to one of 3, 6, or 9 min per question. The correct answer rate is reported both before and after search to assess the impact of the search system and the time constraint. In addition, 2 search systems that use the same user interface, but which vary widely in their search effectiveness, are employed so that the impact of changes in search system effectiveness on clinical decision making can also be assessed.

RESULTS

Recruiting began for the study in June 2018. As of the April 4, 2019, there were 69 participants enrolled. The study is expected to close by May 30, 2019, with results to be published in July.

CONCLUSIONS

All data collected in this study will be made available at the University of Queensland's UQ eSpace public data repository.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

DERR1-10.2196/12803.

Supporting Students With Electronic Health Record-Embedded Learning Aids: A Mixed-Methods Study

BACKGROUND

Students often perceive workplace-based learning as disconnected from what they learn in medical school. Interventions that deal with this issue regularly involve feedback and/or learning aids. Feedback has frequently been encouraged in previous research, whereas the use of aids is less understood.

OBJECTIVE

This study aims to investigate the added value of learning aids in making the connection and enhancing the transfer of learning between medical school and workplace-based learning.

METHODS

First-year students in postgraduate general practice training participated in a mixed-methods study. Within a quasi-experimental design, two conditions were investigated: (1) students having access to electronic health record (EHR)-embedded learning aids and feedback and (2) students only receiving feedback. Semistructured interviews were conducted and analyzed according to the thematic analysis approach.

RESULTS

Forty-four students participated in this study. No significant difference was found between the two conditions (t₄₂=-0.511, P=.61, 95% CI -4.86 to 2.90). Nevertheless, students used the aids frequently and found them useful. Given that the aids were familiar to students and contained practice-based instructions in an easily accessible format, they were perceived as feasible to use during workplace-based learning. They also appeared to stimulate transfer of learning, self-confidence, reflection, and interaction between student and supervisor.

CONCLUSIONS

Access to EHR-embedded learning aids offers additional support during, but also before and after, patient encounters. The aids can be easily implemented into workplace-based learning.

Automatic identification of recent high impact clinical articles in PubMed to support clinical decision making using time-agnostic features

OBJECTIVES

Finding recent clinical studies that warrant changes in clinical practice ("high impact" clinical studies) in a timely manner is very challenging. We investigated a machine learning approach to find recent studies with high clinical impact to support clinical decision making and literature surveillance.

METHODS

To identify recent studies, we developed our classification model using time-agnostic features that are available as soon as an article is indexed in PubMed®, such as journal impact factor, author count, and study sample size. Using a gold standard of 541 high impact treatment studies referenced in 11 disease management guidelines, we tested the following null hypotheses: (1) the high impact classifier with time-agnostic features (HI-TA) performs equivalently to PubMed's Best Match sort and a MeSH-based Naïve Bayes classifier; and (2) HI-TA performs equivalently to the high impact classifier with both time-agnostic and time-sensitive features (HI-TS) enabled in a previous study. The primary outcome for both hypotheses was mean top 20 precision.

RESULTS

The differences in mean top 20 precision between HI-TA and three baselines (PubMed's Best Match, a MeSH-based Naïve Bayes classifier, and HI-TS) were not statistically significant (12% vs. 3%, p = 0.101; 12% vs. 11%, p = 0.720; 12% vs. 25%, p = 0.094, respectively). Recall of HI-TA was low (7%).

CONCLUSION

HI-TA had equivalent performance to state-of-the-art approaches that depend on time-sensitive features. With the advantage of relying only on time-agnostic features, the proposed approach can be used as an adjunct to help clinicians identify recent high impact clinical studies to support clinical decision-making. However, low recall limits the use of HI-TA for literature surveillance.

Interactive Visual Displays for Interpreting the Results of Clinical Trials: Formative Evaluation With Case Vignettes

Background

At the point of care, evidence from randomized controlled trials (RCTs) is underutilized in helping clinicians meet their information needs.

Objective

To design interactive visual displays to help clinicians interpret and compare the results of relevant RCTs for the management of a specific patient, and to conduct a formative evaluation with physicians comparing interactive visual versus narrative displays.

Methods

We followed a user-centered and iterative design process succeeded by development of information display prototypes as a Web-based application. We then used a within-subjects design with 20 participants (8 attendings and 12 residents) to evaluate the usability and problem-solving impact of the information displays. We compared subjects’ perceptions of the interactive visual displays versus narrative abstracts.

Results

The resulting interactive visual displays present RCT results side-by-side according to the Population, Intervention, Comparison, and Outcome (PICO) framework. Study participants completed 19 usability tasks in 3 to 11 seconds with a success rate of 78% to 100%. Participants favored the interactive visual displays over narrative abstracts according to perceived efficiency, effectiveness, effort, user experience and preference (all P values <.001).

Conclusions

When interpreting and applying RCT findings to case vignettes, physicians preferred interactive graphical and PICO-framework-based information displays that enable direct comparison of the results from multiple RCTs compared to the traditional narrative and study-centered format. Future studies should investigate the use of interactive visual displays to support clinical decision making in care settings and their effect on clinician and patient outcomes.

Comprehensive process model of clinical information interaction in primary care: results of a "best-fit" framework synthesis

Objective

To describe a new, comprehensive process model of clinical information interaction in primary care (Clinical Information Interaction Model, or CIIM) based on a systematic synthesis of published research.

Materials and Methods

We used the "best fit" framework synthesis approach. Searches were performed in PubMed, Embase, the Cumulative Index to Nursing and Allied Health Literature (CINAHL), PsycINFO, Library and Information Science Abstracts, Library, Information Science and Technology Abstracts, and Engineering Village. Two authors reviewed articles according to inclusion and exclusion criteria. Data abstraction and content analysis of 443 published papers were used to create a model in which every element was supported by empirical research.

Results

The CIIM documents how primary care clinicians interact with information as they make point-of-care clinical decisions. The model highlights 3 major process components: (1) context, (2) activity (usual and contingent), and (3) influence. Usual activities include information processing, source-user interaction, information evaluation, selection of information, information use, clinical reasoning, and clinical decisions. Clinician characteristics, patient behaviors, and other professionals influence the process.

Discussion

The CIIM depicts the complete process of information interaction, enabling a grasp of relationships previously difficult to discern. The CIIM suggests potentially helpful functionality for clinical decision support systems (CDSSs) to support primary care, including a greater focus on information processing and use. The CIIM also documents the role of influence in clinical information interaction; influencers may affect the success of CDSS implementations.

Conclusion

The CIIM offers a new framework for achieving CDSS workflow integration and new directions for CDSS design that can support the work of diverse primary care clinicians.

Information needs of generalists and specialists using online best-practice algorithms to answer clinical questions

Objective

To better understand clinician information needs and learning opportunities by exploring the use of best-practice algorithms across different training levels and specialties.

Methods

We developed interactive online algorithms (care process models [CPMs]) that integrate current guidelines, recent evidence, and local expertise to represent cross-disciplinary best practices for managing clinical problems. We reviewed CPM usage logs from January 2014 to June 2015 and compared usage across specialty and provider type.

Results

During the study period, 4009 clinicians (2014 physicians in practice, 1117 resident physicians, and 878 nurse practitioners/physician assistants [NP/PAs]) viewed 140 CPMs a total of 81 764 times. Usage varied from 1 to 809 views per person, and from 9 to 4615 views per CPM. Residents and NP/PAs viewed CPMs more often than practicing physicians. Among 2742 users with known specialties, generalists ( N  = 1397) used CPMs more often (mean 31.8, median 7 views) than specialists ( N  = 1345; mean 6.8, median 2; P  < .0001). The topics used by specialists largely aligned with topics within their specialties. The top 20% of available CPMs (28/140) collectively accounted for 61% of uses. In all, 2106 clinicians (52%) returned to the same CPM more than once (average 7.8 views per topic; median 4, maximum 195). Generalists revisited topics more often than specialists (mean 8.8 vs 5.1 views per topic; P  < .0001).

Conclusions

CPM usage varied widely across topics, specialties, and individual clinicians. Frequently viewed and recurrently viewed topics might warrant special attention. Specialists usually view topics within their specialty and may have unique information needs.

A task completion framework to support single-interaction IR research

Purpose

A conceptual model describes important factors within a system and how they relate to one another. They are important because they help to identify system changes that can yield the greatest improvement. Within information retrieval (IR), most research is directed towards multi-document retrieval and a multi-interaction IR user scenario. There are few, if any, IR conceptual models supporting minimal or single-interaction IR (siIR) user scenarios, however the need for siIR systems is growing rapidly. The purpose of this paper is to take the first step towards constructing a task-oriented conceptual model and experimental framework to support siIR research.

Design/methodology/approach

A first principles approach is employed to develop a task-oriented conceptual model, called bridging information retrieval (BIR). This model is contrasted with the concept of relevance, a central factor within IR research.

Findings

BIR introduces the central concept of bridging information (BI) as the objective of IR systems. BI is the additional information a user requires to complete a task, beyond their innate knowledge. The relationship between BI and relevance is determined.

Research limitations/implications

The theoretical basis of BIR is derived axiomatically; however the resulting system evaluation model is speculative.

Practical implications

The proposed operational framework offers researchers a systematic approach to designing and evaluating siIR systems.

Originality/value

This work contributes a novel task-oriented IR conceptual model and evaluation framework, both centred around the concept of BI for siIR. It also contributes a novel search task classification method.

Educational Technologies for Physician Continuous Professional Development: A National Survey

PURPOSE

To determine the past experiences with, current use of, and anticipated use of online learning and simulation-based education among practicing U.S. physicians, and how findings vary by age.

METHOD

The authors surveyed 4,648 randomly sampled board-certified U.S. physicians, September 2015 to April 2016, using Internet-based and paper questionnaires. Survey items (some optional) addressed past and current technology usage, perceived technology effectiveness, and anticipated future use of specific technology innovations.

RESULTS

Of 988 respondents, 444 completed optional items. Of these, 429/442 (97.1%) had used online learning and 372/442 (84.2%) had used simulation-based education in the past five years. Desire for more online learning was modest (mean [standard deviation], 4.6 [1.5]; 1 = strongly disagree, 7 = strongly agree), as was desire for more simulation-based education (4.2 [1.7]). Both online learning and simulation-based education were perceived as effective (5.2 [1.4]; 5.0 [1.4]). Physicians believed they possess adequate skills for online learning (5.8 [1.2]) and that point-of-care learning is vital to effective patient care (5.3 [1.3]). Only 39.0% used objective performance data to guide their learning choices, although 64.6% agreed that such information would be useful. The highest-rated innovations included a central repository for listing educational opportunities and tracking continuing education credits, an app to award credit for answering patient-focused questions, 5-minute and 20-minute clinical updates, and an e-mailed "question of the week." Responses to most survey items were similar across age groups.

CONCLUSIONS

Practicing physicians generally seem receptive and prepared to use a variety of educational technologies, regardless of age.

Impact of electronic clinical decision support on adherence to guideline-recommended treatment for hyperlipidaemia, atrial fibrillation and heart failure: protocol for a cluster randomised trial

INTRODUCTION

Clinical practice guidelines facilitate optimal clinical practice. Point of care access, interpretation and application of such guidelines, however, is inconsistent. Informatics-based tools may help clinicians apply guidelines more consistently. We have developed a novel clinical decision support tool that presents guideline-relevant information and actionable items to clinicians at the point of care. We aim to test whether this tool improves the management of hyperlipidaemia, atrial fibrillation and heart failure by primary care clinicians.

METHODS/ANALYSIS

Clinician care teams were cluster randomised to receive access to the clinical decision support tool or passive access to institutional guidelines on 16 May 2016. The trial began on 1 June 2016 when access to the tool was granted to the intervention clinicians. The trial will be run for 6 months to ensure a sufficient number of patient encounters to achieve 80% power to detect a twofold increase in the primary outcome at the 0.05 level of significance. The primary outcome measure will be the percentage of guideline-based recommendations acted on by clinicians for hyperlipidaemia, atrial fibrillation and heart failure. We hypothesise care teams with access to the clinical decision support tool will act on recommendations at a higher rate than care teams in the standard of care arm.

ETHICS AND DISSEMINATION

The Mayo Clinic Institutional Review Board approved all study procedures. Informed consent was obtained from clinicians. A waiver of informed consent and of Health Insurance Portability and Accountability Act (HIPAA) authorisation for patients managed by clinicians in the study was granted. In addition to publication, results will be disseminated via meetings and newsletters.

TRIAL REGISTRATION NUMBER

NCT02742545.

Professional Development Perceptions and Practices Among U.S. Physicians: A Cross-Specialty National Survey

PURPOSE

Professional development (PD)-both for-credit continuing medical education (CME) and informal self-directed or point-of-care learning-is vital to all physicians. The authors sought to understand physicians' PD perceptions and practices and how these vary by specialty and practice type.

METHOD

The authors administered an Internet and paper survey, from September 2015 to April 2016, to randomly sampled U.S. physicians. Survey items addressed perceived PD needs and barriers and how physicians identify knowledge/skills gaps.

RESULTS

Of 4,648 invitees, 988 (21.6%) responded. Respondents believed that they already know what they need to learn (mean 5.8 [1 = strongly disagree; 7 = strongly agree]), can answer clinical questions using available resources (5.9), and want credit for learning during patient care (5.1). They did not strongly desire help identifying learning gaps (4.0) or indicate difficulty accumulating CME credits (3.1). Most PD was done during personal time (5.5). Competencies regarding medical knowledge/skills, wellness, informatics, and practice/systems improvement were rated the highest priority, while research, teaching, and professionalism were rated the lowest. The most important sources used to identify knowledge/skills gaps were immediate patient care needs (4.1 [1 = not important; 5 = extremely important]), personal awareness (3.8), and practice updates (3.7). The most important barriers were time (3.5) and cost (2.9). Differences by specialty and practice type were generally small and not statistically significant.

CONCLUSIONS

Physicians feel confident in identifying their own learning needs, perceive medical knowledge/skills as their highest-priority need, and desire more credit for learning during patient care.

Development of a tool to assess the completeness of drug information sources for health care professionals: A Delphi study

The aim of this study was to create a standard set of essential drug information items as a tool to assess the completeness of any type of drug information source, regardless of its length, using a Delphi consensus panel of European health care professionals. A compilation of drug-related information items was performed by searching several resources for health care professionals and a final list of 162 items was obtained. Fifty-seven experts in drug information from 23 different European countries were invited to participate in a three-round Delphi technique to obtain consensus on items considered essential and non-essential content of information. Consensus for the first, second, and third rounds was defined as ≥90%, ≥80%, and ≥75% agreement, respectively. Of the 57 experts invited, 32 completed the first round, 27 the second, and 29 the third. Consensus was achieved for 28.3% of the items in the first round, 49.3% in the second, and 58.3% in the third. The final cumulative consensus was 67.7% (n = 126) for items considered essential and 16.1% (n = 30) for items considered non-essential. The final tool obtained to assess the completeness of drug information sources was composed by 126 essential items grouped into 11 sections. This tool allows for the comparison of different information sources for the same medicine and the information content for different medicines in the same source.

Automatic identification of high impact articles in PubMed to support clinical decision making

OBJECTIVES

The practice of evidence-based medicine involves integrating the latest best available evidence into patient care decisions. Yet, critical barriers exist for clinicians' retrieval of evidence that is relevant for a particular patient from primary sources such as randomized controlled trials and meta-analyses. To help address those barriers, we investigated machine learning algorithms that find clinical studies with high clinical impact from PubMed®.

METHODS

Our machine learning algorithms use a variety of features including bibliometric features (e.g., citation count), social media attention, journal impact factors, and citation metadata. The algorithms were developed and evaluated with a gold standard composed of 502 high impact clinical studies that are referenced in 11 clinical evidence-based guidelines on the treatment of various diseases. We tested the following hypotheses: (1) our high impact classifier outperforms a state-of-the-art classifier based on citation metadata and citation terms, and PubMed's® relevance sort algorithm; and (2) the performance of our high impact classifier does not decrease significantly after removing proprietary features such as citation count.

RESULTS

The mean top 20 precision of our high impact classifier was 34% versus 11% for the state-of-the-art classifier and 4% for PubMed's® relevance sort (p=0.009); and the performance of our high impact classifier did not decrease significantly after removing proprietary features (mean top 20 precision=34% vs. 36%; p=0.085).

CONCLUSION

The high impact classifier, using features such as bibliometrics, social media attention and MEDLINE® metadata, outperformed previous approaches and is a promising alternative to identifying high impact studies for clinical decision support.

When Educational Material Is Delivered: A Mixed Methods Content Validation Study of the Information Assessment Method

BACKGROUND

The Information Assessment Method (IAM) allows clinicians to report the cognitive impact, clinical relevance, intention to use, and expected patient health benefits associated with clinical information received by email. More than 15,000 Canadian physicians and pharmacists use the IAM in continuing education programs. In addition, information providers can use IAM ratings and feedback comments from clinicians to improve their products.

OBJECTIVE

Our general objective was to validate the IAM questionnaire for the delivery of educational material (ecological and logical content validity). Our specific objectives were to measure the relevance and evaluate the representativeness of IAM items for assessing information received by email.

METHODS

A 3-part mixed methods study was conducted (convergent design). In part 1 (quantitative longitudinal study), the relevance of IAM items was measured. Participants were 5596 physician members of the Canadian Medical Association who used the IAM. A total of 234,196 ratings were collected in 2012. The relevance of IAM items with respect to their main construct was calculated using descriptive statistics (relevance ratio R). In part 2 (qualitative descriptive study), the representativeness of IAM items was evaluated. A total of 15 family physicians completed semistructured face-to-face interviews. For each construct, we evaluated the representativeness of IAM items using a deductive-inductive thematic qualitative data analysis. In part 3 (mixing quantitative and qualitative parts), results from quantitative and qualitative analyses were reviewed, juxtaposed in a table, discussed with experts, and integrated. Thus, our final results are derived from the views of users (ecological content validation) and experts (logical content validation).

RESULTS

Of the 23 IAM items, 21 were validated for content, while 2 were removed. In part 1 (quantitative results), 21 items were deemed relevant, while 2 items were deemed not relevant (R=4.86% [N=234,196] and R=3.04% [n=45,394], respectively). In part 2 (qualitative results), 22 items were deemed representative, while 1 item was not representative. In part 3 (mixing quantitative and qualitative results), the content validity of 21 items was confirmed, and the 2 nonrelevant items were excluded. A fully validated version was generated (IAM-v2014).

CONCLUSIONS

This study produced a content validated IAM questionnaire that is used by clinicians and information providers to assess the clinical information delivered in continuing education programs.

Context-sensitive decision support (infobuttons) in electronic health records: a systematic review

OBJECTIVE

Infobuttons appear as small icons adjacent to electronic health record (EHR) data (e.g., medications, diagnoses, or test results) that, when clicked, access online knowledge resources tailored to the patient, care setting, or task. Infobuttons are required for "Meaningful Use" certification of US EHRs. We sought to evaluate infobuttons' impact on clinical practice and identify features associated with improved outcomes.

METHODS

We conducted a systematic review, searching MEDLINE, EMBASE, and other databases from inception to July 6, 2015. We included and cataloged all original research in any language describing implementation of infobuttons or other context-sensitive links. Studies evaluating clinical implementations with outcomes of usage or impact were reviewed in greater detail. Reviewers worked in duplicate to select articles, evaluate quality, and abstract information.

RESULTS

Of 599 potential articles, 77 described infobutton implementation. The 17 studies meriting detailed review, including 3 randomized trials, yielded the following findings. Infobutton usage frequency ranged from 0.3 to 7.4 uses per month per potential user. Usage appeared to be influenced by EHR task. Five studies found that infobuttons are used less often than non-context-sensitive links (proportionate usage 0.20-0.34). In 3 studies, users answered their clinical question in > 69% of infobutton sessions. Seven studies evaluated alternative approaches to infobutton design and implementation. No studies isolated the impact of infobuttons on objectively measured patient outcomes.

CONCLUSIONS

Weak evidence suggests that infobuttons can help providers answer clinical questions. Research on optimal infobutton design and implementation, and on the impact on patient outcomes and provider behaviors, is needed.

Strategies for Translating Evidence-Based Medicine in Lung Cancer into Community Practice

The landscape of non-small cell lung cancer (NSCLC) treatment has rapidly evolved over the past decade. This is exemplified by the use of molecular targeted agents, immunotherapies, and newer technologies such as stereotactic body radiotherapy (SBRT). As the translation of preclinical discoveries into clinical practice continues, the effective dissemination and implementation of evidence-based treatment of NSCLC will remain a foremost challenge for oncologists. To further extend evidence-based medicine into the community setting, community oncologists are being engaged on multiple fronts including leadership and participation in national clinical trials and utilization of internet-based resources.

Physicians' perception of alternative displays of clinical research evidence for clinical decision support - A study with case vignettes

OBJECTIVE

To design alternate information displays that present summaries of clinical trial results to clinicians to support decision-making; and to compare the displays according to efficacy and acceptability.

METHODS

A 6-between (information display presentation order) by 3-within (display type) factorial design. Two alternate displays were designed based on Information Foraging theory: a narrative summary that reduces the content to a few sentences; and a table format that structures the display according to the PICO (Population, Intervention, Comparison, Outcome) framework. The designs were compared with the summary display format available in PubMed. Physicians were asked to review five clinical studies retrieved for a case vignette; and were presented with the three display formats. Participants were asked to rate their experience with each of the information displays according to a Likert scale questionnaire.

RESULTS

Twenty physicians completed the study. Overall, participants rated the table display more highly than either the text summary or PubMed's summary format (5.9vs. 5.4vs. 3.9 on a scale between 1 [strongly disagree] and 7 [strongly agree]). Usefulness ratings of seven pieces of information, i.e. patient population, patient age range, sample size, study arm, primary outcome, results of primary outcome, and conclusion, were high (average across all items=4.71 on a 1 to 5 scale, with 1=not at all useful and 5=very useful). Study arm, primary outcome, and conclusion scored the highest (4.9, 4.85, and 4.85 respectively). Participants suggested additional details such as rate of adverse effects.

CONCLUSION

The table format reduced physicians' perceived cognitive effort when quickly reviewing clinical trial information and was more favorably received by physicians than the narrative summary or PubMed's summary format display.

Unique Educational Opportunities for PCPs and Specialists Arising From Electronic Consultation Services

Health care reform should be driven by the goals of better patient experience, improved population health, lower per capita costs, and improved provider satisfaction. Electronic consultation (eConsult) services have been adopted by several jurisdictions in the United States, Canada, and Europe to improve access to specialists by primary care providers (PCPs) and are being heralded as a key component for delivery of coordinated care. The primary intent of an eConsult service is to provide PCPs with efficient, timely, direct access to specialist expertise to help guide the management of their patients, reduce the need for unnecessary face-to-face specialty consultations, and improve the quality of the initial face-to-face consultation when needed, through the preconsultative communication.In addition to improving access to care, eConsult services have been praised by PCPs and specialists for their educational value, in particular their ability to enrich practice-based learning. Less recognized, but equally important from the educational perspective, include the abilities of eConsult programs to promote reflection by PCPs and specialists, improve collegiality and professionalism between primary and specialist care, inform continuing professional development activities and maintenance of certification, and enhance training programs' teaching of effective communication and care coordination.As eConsult services become increasingly available, the medical community must leverage the educational opportunities inherent in eConsult programs to further improve the delivery of coordinated specialty care. The educational role of eConsults should be considered as a priority outcome in their evaluation and must be highlighted and optimized in next iterations of eConsult systems design.

Electronic resources preferred by pediatric hospitalists for clinical care

OBJECTIVES

There is little research on pediatric hospitalists' use of evidence-based resources. The aim of this study was to determine the electronic resources that pediatric hospitalists prefer.

METHODS

Using a web-based survey, the authors determined hospitalists' preferred electronic resources, as well as their attitudes toward lifelong learning, practice, and experience characteristics.

RESULTS

One hundred sixteen hospitalists completed the survey. The most preferred resource for general information, patient handouts, and treatment was UpToDate. Online search engines were ranked second for general information and patient handouts.

CONCLUSIONS

Pediatric hospitalists tend to utilize less rigorous electronic resources such as UpToDate and Google. These results can set a platform for discussing the quality of resources that pediatric hospitalists use.

Development and implementation of an online clinical pathway for adult chronic kidney disease in primary care: a mixed methods study

Background

Primary care physicians and other primary health care professionals from Alberta, Canada identified a clinical pathway as a potential tool to facilitate uptake of clinical practice guidelines for the diagnosis, management and referral of adults with chronic kidney disease. We describe the development and implementation of a chronic kidney disease clinical pathway (CKD-CP; www.ckdpathway.ca).

Methods

The CKD-CP was developed and implemented based on the principles of the Knowledge-To-Action Cycle framework. We used a mixed methods approach to identify the usability and feasibility of the CKD-CP. This included individual interviews, an online survey and website analytics, to gather data on barriers and facilitators to use, perceived usefulness and characteristics of users. Results are reported using conventional qualitative content analysis and descriptive statistics.

Results

Eighteen individual interviews were conducted with primary care physicians, nephrologists, pharmacists and nurse practitioners to identify themes reflecting both barriers and facilitators to integrating the CKD-CP into clinical practice. Themes identified included: communication, work efficiency and confidence. Of the 159 participants that completed the online survey, the majority (52 %) were first time CKD-CP users. Among those who had previously used the CKD-CP, 94 % agreed or strongly agreed that the pathway was user friendly, provided useful information and increased their knowledge and confidence in the care of patients with CKD. Between November 2014 and July 2015, the CKD-CP website had 10,710 visits, 67 % of which were new visitors. The 3 most frequently visited web pages were home, diagnose and medical management. Canada, Indonesia and the United States were the top 3 countries accessing the website during the 9 month period.

Conclusions

An interactive, online, point-of-care tool for primary care providers can be developed and implemented to assist in the care of patients with CKD. Our findings are important for making refinements to the CKD –CP website via ongoing discussions with end-users and the development team, along with continued dissemination using multiple strategies.

Electronic supplementary material

The online version of this article (doi:10.1186/s12911-016-0350-z) contains supplementary material, which is available to authorized users.

Public Health Platforms: An Emerging Informatics Approach to Health Professional Learning and Development

Health informatics has a major role to play in optimising the management and use of data, information and knowledge in health systems. As health systems undergo digital transformation, it is important to consider informatics approaches not only to curriculum content but also to the design of learning environments and learning activities for health professional learning and development. An example of such an informatics approach is the use of large-scale, integrated public health platforms on the Internet as part of health professional learning and development. This article describes selected examples of such platforms, with a focus on how they may influence the direction of health professional learning and development.

Formative evaluation of a patient-specific clinical knowledge summarization tool

OBJECTIVE

To iteratively design a prototype of a computerized clinical knowledge summarization (CKS) tool aimed at helping clinicians finding answers to their clinical questions; and to conduct a formative assessment of the usability, usefulness, efficiency, and impact of the CKS prototype on physicians' perceived decision quality compared with standard search of UpToDate and PubMed.

MATERIALS AND METHODS

Mixed-methods observations of the interactions of 10 physicians with the CKS prototype vs. standard search in an effort to solve clinical problems posed as case vignettes.

RESULTS

The CKS tool automatically summarizes patient-specific and actionable clinical recommendations from PubMed (high quality randomized controlled trials and systematic reviews) and UpToDate. Two thirds of the study participants completed 15 out of 17 usability tasks. The median time to task completion was less than 10s for 12 of the 17 tasks. The difference in search time between the CKS and standard search was not significant (median=4.9 vs. 4.5m in). Physician's perceived decision quality was significantly higher with the CKS than with manual search (mean=16.6 vs. 14.4; p=0.036).

CONCLUSIONS

The CKS prototype was well-accepted by physicians both in terms of usability and usefulness. Physicians perceived better decision quality with the CKS prototype compared to standard search of PubMed and UpToDate within a similar search time. Due to the formative nature of this study and a small sample size, conclusions regarding efficiency and efficacy are exploratory.

Classification of Clinically Useful Sentences in MEDLINE

OBJECTIVE

In a previous study, we investigated a sentence classification model that uses semantic features to extract clinically useful sentences from UpToDate, a synthesized clinical evidence resource. In the present study, we assess the generalizability of the sentence classifier to Medline abstracts.

METHODS

We applied the classification model to an independent gold standard of high quality clinical studies from Medline. Then, the classifier trained on UpToDate sentences was optimized by re-retraining the classifier with Medline abstracts and adding a sentence location feature.

RESULTS

The previous classifier yielded an F-measure of 58% on Medline versus 67% on UpToDate. Re-training the classifier on Medline improved F-measure to 68%; and to 76% (p<0.01) after adding the sentence location feature.

CONCLUSIONS

The classifier's model and input features generalized to Medline abstracts, but the classifier needed to be retrained on Medline to achieve equivalent performance. Sentence location provided additional contribution to the overall classification performance.

Disrupting Electronic Health Records Systems: The Next Generation

The health care system suffers from both inefficient and ineffective use of data. Data are suboptimally displayed to users, undernetworked, underutilized, and wasted. Errors, inefficiencies, and increased costs occur on the basis of unavailable data in a system that does not coordinate the exchange of information, or adequately support its use. Clinicians' schedules are stretched to the limit and yet the system in which they work exerts little effort to streamline and support carefully engineered care processes. Information for decision-making is difficult to access in the context of hurried real-time workflows. This paper explores and addresses these issues to formulate an improved design for clinical workflow, information exchange, and decision making based on the use of electronic health records.

Evaluation of the use of clinical decision support and online resources for pharmacogenomics education

AIM

To assess impact and value of using clinical decision support (CDS) to drive providers toward online pharmacogenomics education.

MATERIALS & METHODS

CDS was used to target prescribers of codeine/tramadol, send an educational email, display alert/inbox and provide links to an online resource. Providers were surveyed to assess impact.

RESULTS

Of the methods used to target providers, educational email was more effective (7.2%). Survey response rate was 29.2% (n = 528/1817). Of respondents, 57.4% reported opening the email and 27.1% accessed the online resource. Of those accessing the resource, 89% found it useful and learned something new about pharmacogenomics.

CONCLUSION

The impact of using CDS to target pharmacogenomics education was limited. However, providers accessing the online resource found it useful and educational.

Analysis of PubMed User Sessions Using a Full-Day PubMed Query Log: A Comparison of Experienced and Nonexperienced PubMed Users

Background

PubMed is the largest biomedical bibliographic information source on the Internet. PubMed has been considered one of the most important and reliable sources of up-to-date health care evidence. Previous studies examined the effects of domain expertise/knowledge on search performance using PubMed. However, very little is known about PubMed users’ knowledge of information retrieval (IR) functions and their usage in query formulation.

Objective

The purpose of this study was to shed light on how experienced/nonexperienced PubMed users perform their search queries by analyzing a full-day query log. Our hypotheses were that (1) experienced PubMed users who use system functions quickly retrieve relevant documents and (2) nonexperienced PubMed users who do not use them have longer search sessions than experienced users.

Methods

To test these hypotheses, we analyzed PubMed query log data containing nearly 3 million queries. User sessions were divided into two categories: experienced and nonexperienced. We compared experienced and nonexperienced users per number of sessions, and experienced and nonexperienced user sessions per session length, with a focus on how fast they completed their sessions.

Results

To test our hypotheses, we measured how successful information retrieval was (at retrieving relevant documents), represented as the decrease rates of experienced and nonexperienced users from a session length of 1 to 2, 3, 4, and 5. The decrease rate (from a session length of 1 to 2) of the experienced users was significantly larger than that of the nonexperienced groups.

Conclusions

Experienced PubMed users retrieve relevant documents more quickly than nonexperienced PubMed users in terms of session length.

An automated clinical alert system for newly-diagnosed atrial fibrillation

OBJECTIVE

Clinical decision support systems that notify providers of abnormal test results have produced mixed results. We sought to develop, implement, and evaluate the impact of a computer-based clinical alert system intended to improve atrial fibrillation stroke prophylaxis, and identify reasons providers do not implement a guideline-concordant response.

MATERIALS AND METHODS

We conducted a cohort study with historical controls among patients at a tertiary care hospital. We developed a decision rule to identify newly-diagnosed atrial fibrillation, automatically notify providers, and direct them to online evidence-based management guidelines. We tracked all notifications from December 2009 to February 2010 (notification period) and applied the same decision rule to all patients from December 2008 to February 2009 (control period). Primary outcomes were accuracy of notification (confirmed through chart review) and prescription of warfarin within 30 days.

RESULTS

During the notification period 604 notifications were triggered, of which 268 (44%) were confirmed as newly-diagnosed atrial fibrillation. The notifications not confirmed as newly-diagnosed involved patients with no recent electrocardiogram at our institution. Thirty-four of 125 high-risk patients (27%) received warfarin in the notification period, compared with 34 of 94 (36%) in the control period (odds ratio, 0.66 [95% CI, 0.37-1.17]; p = 0.16). Common reasons to not prescribe warfarin (identified from chart review of 151 patients) included upcoming surgical procedure, choice to use aspirin, and discrepancy between clinical notes and the medication record.

CONCLUSIONS

An automated system to identify newly-diagnosed atrial fibrillation, notify providers, and encourage access to management guidelines did not change provider behaviors.

An inter-professional approach to personalized medicine education: one institution's experience

Personalized medicine offers the promise of better diagnoses, targeted therapies and individualized treatment plans. Pharmacogenomics is an integral component of personalized medicine; it aids in the prediction of an individual's response to medications. Despite growing public acceptance and emerging clinical evidence, this rapidly expanding field of medicine is slow to be adopted and utilized by healthcare providers, although many believe that they should be knowledgeable and able to apply pharmacogenomics in clinical practice. Institutional infrastructure must be built to support pharmacogenomic implementation. Multidisciplinary education for healthcare providers is a critical component for pharmacogenomics to achieve its full potential to optimize patient care. We describe our recent experience at the Mayo Clinic implementing pharmacogenomics education in a large, academic healthcare system facilitated by the Mayo Clinic Center for Individualized Medicine.

Applying new strategies for the national adaptation, updating, and dissemination of trustworthy guidelines: results from the Norwegian adaptation of the Antithrombotic Therapy and the Prevention of Thrombosis, 9th Ed: American College of Chest Physicians Evidence-Based Clinical Practice Guidelines

BACKGROUND

The Antithrombotic Therapy and the Prevention of Thrombosis, 9th Edition: American College of Chest Physicians Evidence-Based Clinical Practice Guidelines (AT9) represent trustworthy international guidelines for antithrombotic treatment and thromboprophylaxis. We describe major changes to the format and content resulting from applying new strategies for guideline adaptation and dissemination.

METHODS

A Norwegian guideline panel of 46 experts completed a structured and systematic adaptation process, updated the recommendations based on new evidence, and rewrote the recommendations in an electronic multilayered presentation format. We published the adapted guideline using the web-based Making GRADE the Irresistible Choice Guideline Authoring and Publication Platform.

RESULTS

We applied a novel presentation format to 333 recommendations from 11 of the 15 management chapters in AT9 and condensed and restructured them into 249 recommendations in a multilayered format. We added additional relevant information, such as 29 best-practice statements about new oral anticoagulants and practical information sections for 121 recommendations. Common reasons for modifications included feasibility of the recommendations in a national context, disagreement with applied baseline risk estimates, and reevaluation of the balance between the benefits and harms of interventions in relation to assumed typical patient preferences and values. The adapted guideline was published and disseminated online in November 2013.

CONCLUSIONS

New strategies for adapting, updating, and disseminating trustworthy guidelines proved feasible and will provide Norwegian health-care professionals and patients with up-to-date guidance tailored to national circumstances.

A comprehensive information technology system to support physician learning at the point of care

MayoExpert is a multifaceted information system integrated with the electronic medical record (EMR) across Mayo Clinic's multisite health system. It was developed as a technology-based solution to manage information, standardize clinical practice, and promote and document learning in clinical contexts. Features include urgent test result notifications; models illustrating expert-approved care processes; concise, expert-approved answers to frequently asked questions (FAQs); a directory of topic-specific experts; and a portfolio for provider licensure and credentialing. The authors evaluate MayoExpert's reach, effectiveness, adoption, implementation, and maintenance. Evaluation data sources included usage statistics, user surveys, and pilot studies.As of October 2013, MayoExpert was available at 94 clinical sites in 12 states and contained 1,368 clinical topics, answers to 7,640 FAQs, and 92 care process models. In 2012, MayoExpert was accessed at least once by 2,578/3,643 (71%) staff physicians, 900/1,374 (66%) midlevel providers, and 1,728/2,291 (75%) residents and fellows. In a 2013 survey of MayoExpert users with 536 respondents, all features were highly rated (≥67% favorable). More providers reported using MayoExpert to answer questions before/after than during patient visits (68% versus 36%). During November 2012 to April 2013, MayoExpert sent 1,660 notifications of new-onset atrial fibrillation and 1,590 notifications of prolonged QT. MayoExpert has become part of routine clinical and educational operations, and its care process models now define Mayo Clinic best practices. MayoExpert's infrastructure and content will continue to expand with improved templates and content organization, new care process models, additional notifications, better EMR integration, and improved support for credentialing activities.