Do online information retrieval systems help experienced clinicians answer clinical questions?

Investigating the role of internet-based educational application in the dental sciences

The study intended to identify the significance of the online information retrieval system (IRS) in evidence-based dentistry (EBD). Thus, the researchers apply a set of pre-and post-tests to evaluate the current knowledge of clinicians and students on online IRS. Materials and Methods: The researchers followed repeated measure design in this study. They applied random sampling technique for conducting pre-and post-test assessment. Five scenarios based EBD were developed to evaluate the performance of the participants. The researchers employed two phases in order to achieve the study's objective. In the first phase, 98 clinicians and 70 students were invited to attend three out of five clinical scenarios using IRS. In the second phase, the participants were invited to participate in a 15-min lecture presented by the researchers on the searching strategies and guidelines to apply keywords for searching the evidence using IRS. A significant level of p < 0.05 was obtained from the statistical analysis using the SPSS program version 16. Results: Of the 98 clinicians, only 37 responded to the questionnaire, with a response rate of 37.8 %. On the other hand, out of 70 students, 23 responded to the questionnaire, with a response rate of 32.8 %. In the pre-test, clinicians and students correctly answered 58.3 % of scenario questions. However, the data analysis outcome revealed that only 40.5 % of participants provided a relevant evidence source after an internet search. The students spent an average of 9 min to complete the task, whereas clinicians spent 16 min. After the completion of the lecture, 23 students and clinicians responded to the pre-test, whereas 10 responded to the post-test. Most students believed that the lecture was helpful and recommended similar types of lectures to be presented in the curriculum. The study findings highlight that the percentage of evidence provided in the "pre-test" was 60 %, which was improved in the post-test to 73.3 %. Conclusion: The experimental outcome suggests that internet-based educational applications enhance students' learning strategies. Additionally, the IRS supports clinicians in retrieving effective materials for treating their patients. Furthemore, there is a demand for extracurricular activity to improve the search strategies of clinicians and students to strengthen EBD.

Cluster-randomized implementation trial of two facilitation strategies to implement a novel information and communications technology at the Veterans Health Administration

BACKGROUND

Information and communication technologies (ICTs) improve quality and efficiency of healthcare, but effective practices for implementing new ICTs are unknown. From 2019 to 2021, the Veterans Health Administration (VHA) implemented FLOW3, an ICT that facilitates prosthetic limb care. The goal of this study was to compare the impact of two facilitation strategies on FLOW3 adoption, implementation, and sustainment.

METHODS

FLOW3 is a computerized workflow management system comprised of three applications that facilitate the three steps for prosthesis authorization. During VHA's implementation of FLOW3, we randomized 60 VHA sites to basic or enhanced facilitation groups. Basic facilitation included a manualized training toolkit and office hours. Enhanced facilitation included basic facilitation plus monthly learning collaboratives and site-specific performance reports. Outcomes included time to adoption of FLOW3 and complete FLOW3 utilization rates during implementation and sustainment periods. We compared outcomes between sites assigned to basic versus enhanced facilitation groups. Results were calculated using both intent-to-treat (ITT) and dose-response analyses. The dose-response analysis used a per-protocol approach and required sites in the enhanced facilitation group to join two of six learning collaboratives; sites that attended fewer were reassigned to the basic group.

RESULTS

Randomization assigned 30 sites to enhanced facilitation and 30 to basic. Eighteen of 30 randomized sites were included in the enhanced facilitation group for dose-response analysis. During the implementation period, enhanced facilitation sites were significantly more likely to completely utilize FLOW3 than basic facilitation sites (HR: 0.17; 95% CI: 1.18, 4.53, p = 0.02) based on ITT analysis. In the dose-response analysis, the enhanced group was 2.32 (95% CI: 1.18, 4.53) times more likely to adopt FLOW3 than basic group (p = 0.014).

CONCLUSIONS

Enhanced facilitation including a learning collaborative and customized feedback demonstrated greater likelihood for sites to complete a prosthetics consult using FLOW3 throughout our study. We identified statistically significant differences in likelihood of adoption using the dose-response analysis and complete utilization rate using ITT analysis during the implementation period. All sites that implemented FLOW3 demonstrated improvement in completion rate during the sustainment period, but the difference between facilitation groups was not statistically significant. Further study to understand sustainability is warranted.

Addressing Disparities in Physician Access to Information in Support of Evidence-based Practice

The purpose of this study is to determine if Minnesota physicians have access to information resources needed to support evidence-based practice (EBP), which supports a culture of safety and patient-centered care. A survey was used to determine Minnesota physicians' need for, and access to, evidence-based clinical information. A total of 877 responses (6.4% response rate) were included in the data analysis. Participants spent 24 min daily seeking answers to clinical questions and averaged 4.41 questions per day that could not be immediately answered. Physicians reported high levels of information needs met (85.8%), though they reported limited access to drug resources, citation databases, systematic reviews, and full-text books and articles. Results also showed use of unreliable sources to support decision-making. A key finding was the extent to which workplace affiliation broadens disparities in information access. National and regional approaches can work to support EBP by reducing the information gap caused by workplace affiliation and other barriers. Further research should be done to identify partnerships, funding, infrastructure, and support to address these gaps.

How searching under time pressure impacts clinical decision making

Objective:

Clinicians encounter many questions during patient encounters that they cannot answer. While search systems (e.g., PubMed) can help clinicians find answers, clinicians are typically busy and report that they often do not have sufficient time to use such systems. The objective of this study was to assess the impact of time pressure on clinical decisions made with the use of a medical literature search system.

Design:

In stage 1, 109 final-year medical students and practicing clinicians were presented with 16 clinical questions that they had to answer using their own knowledge. In stage 2, the participants were provided with a search system, similar to PubMed, to help them to answer the same 16 questions, and time pressure was simulated by limiting the participant's search time to 3, 6, or 9 minutes per question.

Results:

Under low time pressure, the correct answer rate significantly improved by 32% when the participants used the search system, whereas under high time pressure, this improvement was only 6%. Also, under high time pressure, participants reported significantly lower confidence in the answers, higher perception of task difficulty, and higher stress levels.

Conclusions:

For clinicians and health care organizations operating in increasingly time-pressured environments, literature search systems become less effective at supporting accurate clinical decisions. For medical search system developers, this study indicates that system designs that provide faster information retrieval and analysis, rather than traditional document search, may provide more effective alternatives.

Evaluating the Impact of the Grading and Assessment of Predictive Tools Framework on Clinicians and Health Care Professionals' Decisions in Selecting Clinical Predictive Tools: Randomized Controlled Trial

BACKGROUND

While selecting predictive tools for implementation in clinical practice or for recommendation in clinical guidelines, clinicians and health care professionals are challenged with an overwhelming number of tools. Many of these tools have never been implemented or evaluated for comparative effectiveness. To overcome this challenge, the authors developed and validated an evidence-based framework for grading and assessment of predictive tools (the GRASP framework). This framework was based on the critical appraisal of the published evidence on such tools.

OBJECTIVE

The aim of the study was to examine the impact of using the GRASP framework on clinicians' and health care professionals' decisions in selecting clinical predictive tools.

METHODS

A controlled experiment was conducted through a web-based survey. Participants were randomized to either review the derivation publications, such as studies describing the development of the predictive tools, on common traumatic brain injury predictive tools (control group) or to review an evidence-based summary, where each tool had been graded and assessed using the GRASP framework (intervention group). Participants in both groups were asked to select the best tool based on the greatest validation or implementation. A wide group of international clinicians and health care professionals were invited to participate in the survey. Task completion time, rate of correct decisions, rate of objective versus subjective decisions, and level of decisional conflict were measured.

RESULTS

We received a total of 194 valid responses. In comparison with not using GRASP, using the framework significantly increased correct decisions by 64%, from 53.7% to 88.1% (88.1/53.7=1.64; t₁₉₃=8.53; P<.001); increased objective decision making by 32%, from 62% (3.11/5) to 82% (4.10/5; t₁₈₉=9.24; P<.001); decreased subjective decision making based on guessing by 20%, from 49% (2.48/5) to 39% (1.98/5; t₁₈₈=-5.47; P<.001); and decreased prior knowledge or experience by 8%, from 71% (3.55/5) to 65% (3.27/5; t₁₈₇=-2.99; P=.003). Using GRASP significantly decreased decisional conflict and increased the confidence and satisfaction of participants with their decisions by 11%, from 71% (3.55/5) to 79% (3.96/5; t₁₈₈=4.27; P<.001), and by 13%, from 70% (3.54/5) to 79% (3.99/5; t₁₈₈=4.89; P<.001), respectively. Using GRASP decreased the task completion time, on the 90th percentile, by 52%, from 12.4 to 6.4 min (t₁₉₃=-0.87; P=.38). The average System Usability Scale of the GRASP framework was very good: 72.5% and 88% (108/122) of the participants found the GRASP useful.

CONCLUSIONS

Using GRASP has positively supported and significantly improved evidence-based decision making. It has increased the accuracy and efficiency of selecting predictive tools. GRASP is not meant to be prescriptive; it represents a high-level approach and an effective, evidence-based, and comprehensive yet simple and feasible method to evaluate, compare, and select clinical predictive tools.

Impact of a Search Engine on Clinical Decisions Under Time and System Effectiveness Constraints: Research Protocol

BACKGROUND

Many clinical questions arise during patient encounters that clinicians are unable to answer. An evidence-based medicine approach expects that clinicians will seek and apply the best available evidence to answer clinical questions. One commonly used source of such evidence is scientific literature, such as that available through MEDLINE and PubMed. Clinicians report that 2 key reasons why they do not use search systems to answer questions is that it takes too much time and that they do not expect to find a definitive answer. So, the question remains about how effectively scientific literature search systems support time-pressured clinicians in making better clinical decisions. The results of this study are important because they can help clinicians and health care organizations to better assess their needs with respect to clinical decision support (CDS) systems and evidence sources. The results and data captured will contribute a significant data collection to inform the design of future CDS systems to better meet the needs of time-pressured, practicing clinicians.

OBJECTIVE

The purpose of this study is to understand the impact of using a scientific medical literature search system on clinical decision making. Furthermore, to understand the impact of realistic time pressures on clinicians, we vary the search time available to find clinical answers. Finally, we assess the impact of improvements in search system effectiveness on the same clinical decisions.

METHODS

In this study, 96 practicing clinicians and final year medical students are presented with 16 clinical questions which they must answer without access to any external resource. The same questions are then represented to the clinicians; however, in this part of the study, the clinicians can use a scientific literature search engine to find evidence to support their answers. The time pressures of practicing clinicians are simulated by limiting answer time to one of 3, 6, or 9 min per question. The correct answer rate is reported both before and after search to assess the impact of the search system and the time constraint. In addition, 2 search systems that use the same user interface, but which vary widely in their search effectiveness, are employed so that the impact of changes in search system effectiveness on clinical decision making can also be assessed.

RESULTS

Recruiting began for the study in June 2018. As of the April 4, 2019, there were 69 participants enrolled. The study is expected to close by May 30, 2019, with results to be published in July.

CONCLUSIONS

All data collected in this study will be made available at the University of Queensland's UQ eSpace public data repository.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

DERR1-10.2196/12803.

Examining the role of MEDLINE as a patient care information resource: an analysis of data from the Value of Libraries study

OBJECTIVE

This study analyzed data from a study on the value of libraries to understand the specific role that the MEDLINE database plays in relation to other information resources that are available to health care providers and its role in positively impacting patient care.

METHODS

A previous study on the use of health information resources for patient care obtained 16,122 responses from health care providers in 56 hospitals about how providers make decisions affecting patient care and the role of information resources in that process. Respondents indicated resources used in answering a specific clinical question from a list of 19 possible resources, including MEDLINE. Study data were examined using descriptive statistics and regression analysis to determine the number of information resources used and how they were used in combination with one another.

RESULTS

Health care professionals used 3.5 resources, on average, to aid in patient care. The 2 most frequently used resources were journals (print and online) and the MEDLINE database. Using a higher number of information resources was significantly associated with a higher probability of making changes to patient care and avoiding adverse events. MEDLINE was the most likely to be among consulted resources compared to any other information resource other than journals.

CONCLUSIONS

MEDLINE is a critical clinical care tool that health care professionals use to avoid adverse events, make changes to patient care, and answer clinical questions.

McMaster Optimal Aging Portal: an evidence-based database for geriatrics-focused health professionals

Objective

The objective of this work was to provide easy access to reliable health information based on good quality research that will help health care professionals to learn what works best for seniors to stay as healthy as possible, manage health conditions and build supportive health systems. This will help meet the demands of our aging population that clinicians provide high quality care for older adults, that public health professionals deliver disease prevention and health promotion strategies across the life span, and that policymakers address the economic and social need to create a robust health system and a healthy society for all ages.

Results

The McMaster Optimal Aging Portal’s (Portal) professional bibliographic database contains high quality scientific evidence about optimal aging specifically targeted to clinicians, public health professionals and policymakers. The database content comes from three information services: McMaster Premium LiteratUre Service (MacPLUS™), Health Evidence™ and Health Systems Evidence. The Portal is continually updated, freely accessible online, easily searchable, and provides email-based alerts when new records are added. The database is being continually assessed for value, usability and use. A number of improvements are planned, including French language translation of content, increased linkages between related records within the Portal database, and inclusion of additional types of content. While this article focuses on the professional database, the Portal also houses resources for patients, caregivers and the general public, which may also be of interest to geriatric practitioners and researchers.

Embedding a Medical Search Engine Within an Electronic Health Record

This study investigates an information retrieval tool embedded in an electronic health record (EHR). 1-Search provides a single search for retrieving information from a variety of content sources. 1-Search's usefulness and impact were determined by measuring the extent of physicians' information needs, pre- and post-implementation user satisfaction, and the impact of 1-Search on clinical decision-making. Results support incorporation of 1-Search into the EHR, the continued use of 1-Search, and further development.

A simple heuristic for Internet-based evidence search in primary care: a randomized controlled trial

BACKGROUND

General practitioners (GPs) are confronted with a wide variety of clinical questions, many of which remain unanswered.

METHODS

In order to assist GPs in finding quick, evidence-based answers, we developed a learning program (LP) with a short interactive workshop based on a simple three-step-heuristic to improve their search and appraisal competence (SAC). We evaluated the LP effectiveness with a randomized controlled trial (RCT). Participants (intervention group [IG] n=20; control group [CG] n=31) rated acceptance and satisfaction and also answered 39 knowledge questions to assess their SAC. We controlled for previous knowledge in content areas covered by the test.

RESULTS

Main outcome - SAC: within both groups, the pre-post test shows significant (P=0.00) improvements in correctness (IG 15% vs CG 11%) and confidence (32% vs 26%) to find evidence-based answers. However, the SAC difference was not significant in the RCT.

OTHER MEASURES

Most workshop participants rated "learning atmosphere" (90%), "skills acquired" (90%), and "relevancy to my practice" (86%) as good or very good. The LP-recommendations were implemented by 67% of the IG, whereas 15% of the CG already conformed to LP recommendations spontaneously (odds ratio 9.6, P=0.00). After literature search, the IG showed a (not significantly) higher satisfaction regarding "time spent" (IG 80% vs CG 65%), "quality of information" (65% vs 54%), and "amount of information" (53% vs 47%).

CONCLUSION

Long-standing established GPs have a good SAC. Despite high acceptance, strong learning effects, positive search experience, and significant increase of SAC in the pre-post test, the RCT of our LP showed no significant difference in SAC between IG and CG. However, we suggest that our simple decision heuristic merits further investigation.

Electronic Health Record Tools to Care for At-Risk Older Drivers: A Quality Improvement Project

PURPOSE OF THE STUDY

Evaluating driving safety of older adults is an important health topic, but primary care providers (PCP) face multiple barriers in addressing this issue. The study's objectives were to develop an electronic health record (EHR)-based Driving Clinical Support Tool, train PCPs to perform driving assessments utilizing the tool, and systematize documentation of assessment and management of driving safety issues via the tool.

DESIGN AND METHODS

The intervention included development of an evidence-based Driving Clinical Support Tool within the EHR, followed by training of internal medicine providers in the tool's content and use. Pre- and postintervention provider surveys and chart review of driving-related patient visits were conducted. Surveys included self-report of preparedness and knowledge to evaluate at-risk older drivers and were analyzed using paired t-test. A chart review of driving-related office visits compared documentation pre- and postintervention including: completeness of appropriate focused history and exam, identification of deficits, patient education, and reporting to appropriate authorities when indicated.

RESULTS

Data from 86 providers were analyzed. Pre- and postintervention surveys showed significantly increased self-assessed preparedness (p < .001) and increased driving-related knowledge (p < .001). Postintervention charts showed improved documentation of correct cognitive testing, more referrals/consults, increased patient education about community resources, and appropriate regulatory reporting when deficits were identified.

IMPLICATIONS

Focused training and an EHR-based clinical support tool improved provider self-reported preparedness and knowledge of how to evaluate at-risk older drivers. The tool improved documentation of driving-related issues and led to improved access to interdisciplinary care coordination.

Automated identification of molecular effects of drugs (AIMED)

INTRODUCTION

Genomic profiling information is frequently available to oncologists, enabling targeted cancer therapy. Because clinically relevant information is rapidly emerging in the literature and elsewhere, there is a need for informatics technologies to support targeted therapies. To this end, we have developed a system for Automated Identification of Molecular Effects of Drugs, to help biomedical scientists curate this literature to facilitate decision support.

OBJECTIVES

To create an automated system to identify assertions in the literature concerning drugs targeting genes with therapeutic implications and characterize the challenges inherent in automating this process in rapidly evolving domains.

METHODS

We used subject-predicate-object triples (semantic predications) and co-occurrence relations generated by applying the SemRep Natural Language Processing system to MEDLINE abstracts and ClinicalTrials.gov descriptions. We applied customized semantic queries to find drugs targeting genes of interest. The results were manually reviewed by a team of experts.

RESULTS

Compared to a manually curated set of relationships, recall, precision, and F2 were 0.39, 0.21, and 0.33, respectively, which represents a 3- to 4-fold improvement over a publically available set of predications (SemMedDB) alone. Upon review of ostensibly false positive results, 26% were considered relevant additions to the reference set, and an additional 61% were considered to be relevant for review. Adding co-occurrence data improved results for drugs in early development, but not their better-established counterparts.

CONCLUSIONS

Precision medicine poses unique challenges for biomedical informatics systems that help domain experts find answers to their research questions. Further research is required to improve the performance of such systems, particularly for drugs in development.

How do junior medical officers use online information resources? A survey

BACKGROUND

Online information resources function dually as important learning tools and sources of the latest evidence-based recommendations for junior medical officers (JMOs). However, little is currently known about how JMOs utilise this information when providing care for their patients. This study aimed to examine the usage and experience of online information resources amongst JMOs in South Australia to ascertain (i) the type of resources accessed, (ii) the frequency, (iii) the intended purpose, and (iv) the perceived reliability.

METHODS

A survey instrument using multiple choices, five-point Likert scales and free-text comments was developed and distributed through SurveyMonkey to South Australian JMOs between 1 May 2014 and 30 June 2014.

RESULTS

Of the 142 surveyed, 100 JMOs (70.4%) used online information resources as their first approach over all other resources available. JMOs overwhelmingly (94.4%, n = 134) used online information resources at least once per day, with the most frequent purpose for use being information regarding prescription medication (82.4%, n = 117, reported 'very frequent' use). JMOs stated online resources were necessary to perform their work and, of the different types of information accessed, they rated peer-reviewed resources as the most reliable.

CONCLUSIONS

JMOs strongly rely upon online clinical information in their everyday practice. Importantly, provision of these resources assists JMOs in their education and clinical performance.

Dissemination of Evidence-Based Practice to Directors of Nursing by an Outreach Campaign in Taiwan

BACKGROUND

Directors of nursing (DONs) have an important influence in the dissemination of evidence-based practice (EBP) in hospital settings. The current study examined how the knowledge, skills, and behaviors of DONs changed when EBP was implemented during a 5-year, nationwide promotional campaign providing EBP-related information resources and promotional activities in regional hospitals in Taiwan.

METHOD

Cross-sectional questionnaire surveys for a nationwide representative sample of DONs were conducted in 2007, 2009, and 2011 to examine views related to EBP, including changes in beliefs, attitudes, knowledge, skills, behaviors, and barriers.

RESULTS

This study enrolled 267 DONs in 2007, 257 in 2009, and 287 in 2011. During the study period, DONs' EBP knowledge and skills increased, but their beliefs and attitudes did not significantly change. Furthermore, the use of Internet-based resources, including web portals, electronic textbooks, electronic journals, and evidence-based online databases, increased. Most barriers significantly declined after the intervention.

CONCLUSION

DONs' knowledge, skills, and behaviors regarding EBP increased after the multifaceted intervention. The data suggest this outreach program is useful in disseminating EBP implementation to DONs.

Basic ICT adoption and use by general practitioners: an analysis of primary care systems in 31 European countries

Background

There is general consensus that appropriate development and use of information and communication technologies (ICT) are crucial in the delivery of effective primary care (PC). Several countries are defining policies to support and promote a structural change of the health care system through the introduction of ICT. This study analyses the state of development of basic ICT in PC systems of 31 European countries with the aim to describe the extent of, and main purposes for, computer use by General Practitioners (GPs) across Europe. Additionally, trends over time have been analysed.

Methods

Descriptive statistical analysis was performed on data from the QUALICOPC (Quality and Costs of Primary Care in Europe) survey, to describe the geographic differences in the general use of computer, and in specific computerized clinical functions for different health-related purposes such as prescribing, medication checking, generating health records and research for medical information on the Internet.

Results

While all the countries have achieved a near-universal adoption of a computer in their primary care practices, with only a few countries near or under the boundary of 90 %, the computerisation of primary care clinical functions presents a wide variability of adoption within and among countries and, in several cases (such as in the southern and central-eastern Europe), a large room for improvement.

Conclusions

At European level, more efforts could be done to support southern and central-eastern Europe in closing the gap in adoption and use of ICT in PC. In particular, more attention seems to be need on the current usages of the computer in PC, by focusing policies and actions on the improvement of the appropriate usages that can impact on quality and costs of PC and can facilitate an interconnected health care system. However, policies and investments seem necessary but not sufficient to achieve these goals. Organizational, behavioural and also networking aspects should be taken in consideration.

eHealth 2015 Special Issue: Impact of Electronic Health Records on the Completeness of Clinical Documentation Generated during Diabetic Retinopathy Consultations

BACKGROUND

Two years ago, the Diabetic Retinopathy (DRP) and Traumatology clinic of the Department of Ophthalmology and Optometrics at the Medical University of Vienna, Austria switched from paper-based to electronic health records. A customized electronic health record system (EHR-S) was implemented.

OBJECTIVES

To assess the completeness of information documented electronically compared with manually during patient visits.

METHODS

The Preferred Practice Pattern for Diabetic Retinopathy published by the American Academy of Ophthalmology was distilled into a list of medical features grouped into categories to be assessed and documented during the management of patients with DRP. The last seventy paper-based records and all electronic records generated since the switch were analyzed and graded for the presence of features on the list and the resulting scores compared.

RESULTS

In all categories, clinical documentation was more complete in the EHR group.

CONCLUSIONS

In our setting, the implementation of an EHR-S showed a statistically significant positive impact on documentation completeness.

If the results of an article are noteworthy, read the entire article; do not rely on the abstract alone

Clinicians typically update their knowledge by reading articles on the Internet. Easy access to the articles' abstracts and a lack of time to access other information sources creates a risk that therapeutic or diagnostic decisions will be made after reading just the abstracts. Occasionally, however, the abstracts of articles from clinical trials that have not obtained statistically significant differences in the primary study endpoint have reported other positive results, for example, of a secondary endpoint or a subgroup analysis. The article, however, correctly reports all results, including those of the primary endpoint. In the abstract, the safety information of the experimental treatment is usually deficient. The whole article should be read if a clinical decision is to be made.

Care episode retrieval: distributional semantic models for information retrieval in the clinical domain

Patients' health related information is stored in electronic health records (EHRs) by health service providers. These records include sequential documentation of care episodes in the form of clinical notes. EHRs are used throughout the health care sector by professionals, administrators and patients, primarily for clinical purposes, but also for secondary purposes such as decision support and research. The vast amounts of information in EHR systems complicate information management and increase the risk of information overload. Therefore, clinicians and researchers need new tools to manage the information stored in the EHRs. A common use case is, given a--possibly unfinished--care episode, to retrieve the most similar care episodes among the records. This paper presents several methods for information retrieval, focusing on care episode retrieval, based on textual similarity, where similarity is measured through domain-specific modelling of the distributional semantics of words. Models include variants of random indexing and the semantic neural network model word2vec. Two novel methods are introduced that utilize the ICD-10 codes attached to care episodes to better induce domain-specificity in the semantic model. We report on experimental evaluation of care episode retrieval that circumvents the lack of human judgements regarding episode relevance. Results suggest that several of the methods proposed outperform a state-of-the art search engine (Lucene) on the retrieval task.

Automating data extraction in systematic reviews: a systematic review

BACKGROUND

Automation of the parts of systematic review process, specifically the data extraction step, may be an important strategy to reduce the time necessary to complete a systematic review. However, the state of the science of automatically extracting data elements from full texts has not been well described. This paper performs a systematic review of published and unpublished methods to automate data extraction for systematic reviews.

METHODS

We systematically searched PubMed, IEEEXplore, and ACM Digital Library to identify potentially relevant articles. We included reports that met the following criteria: 1) methods or results section described what entities were or need to be extracted, and 2) at least one entity was automatically extracted with evaluation results that were presented for that entity. We also reviewed the citations from included reports.

RESULTS

Out of a total of 1190 unique citations that met our search criteria, we found 26 published reports describing automatic extraction of at least one of more than 52 potential data elements used in systematic reviews. For 25 (48 %) of the data elements used in systematic reviews, there were attempts from various researchers to extract information automatically from the publication text. Out of these, 14 (27 %) data elements were completely extracted, but the highest number of data elements extracted automatically by a single study was 7. Most of the data elements were extracted with F-scores (a mean of sensitivity and positive predictive value) of over 70 %.

CONCLUSIONS

We found no unified information extraction framework tailored to the systematic review process, and published reports focused on a limited (1-7) number of data elements. Biomedical natural language processing techniques have not been fully utilized to fully or even partially automate the data extraction step of systematic reviews.

The Development and Use of Evidence Summaries for Point of Care Information Systems: A Streamlined Rapid Review Approach

BACKGROUND

A systematic review of evidence is the research method which underpins the traditional approach to evidence-based health care. As systematic reviews follow a rigorous methodology, they can take a substantial amount of time to complete ranging in duration from 6 months to 2 years. Rapid reviews have been proposed as a method to provide summaries of the literature in a more timely fashion.

AIM

The aim of this paper is to outline our experience of developing evidence summaries in the context of a point of care resource as a contribution to the emerging field of rapid review methodologies.

METHODS

Evidence summaries are defined as a synopsis that summarizes existing international evidence on healthcare interventions or activities. These summaries are based on structured searches of the literature and selected evidence-based healthcare databases. Following the search, all studies are assessed for internal validity using an abridged set of critical appraisal tools. Once developed, they undergo three levels of peer review by internal and external experts.

RESULTS

As of November 2014, there are 2458 evidence summaries that have been created across a range of conditions to inform evidence-based healthcare practices. In addition, there is ongoing development of various new evidence summaries on a wide range of topics. Approximately 60-70 new evidence summaries are published every month, covering research in various medical specialty areas. All summaries are updated annually.

LINKING EVIDENCE TO ACTION

Systematic reviews, although the ideal type of research to inform practice, often do not meet the needs of users at the point of care. This article describes the development framework for the creation of evidence summaries, a type of rapid review. Although evidence summaries may result in a less rigorous process of development, they can be useful for improving practice at the point of care.

Improving Chronic Diseases Management Through the Development of an Evidence-Based Resource

OBJECTIVE

There is a large gap between evidence and practice within health care, particularly within the field of chronic disease. To reduce this gap and improve the management of chronic disease, a collaborative partnership between two schools within a large university and two industry partners (a large regional rural hospital and a rural community health center) in rural Victoria, Australia, was developed. The aim of the collaboration was to promote the development of translation science and the implementation of evidence-based health care in chronic disease with a specific focus on developing evidence-based resources that are easily accessed by clinicians.

METHODS

A working group consisting of members of the collaborating organizations and an internationally renowned expert reference group was formed. The group acted as a steering committee and was tasked with developing a taxonomy of the resources. In addition, a peer review process of all resources was established. A corresponding reference group consisting of researchers and clinicians who are clinical experts in various fields was involved in the review process. The resources developed by the group include evidence summaries and recommended practices made available on a web-based database, which can be accessed via subscription by clinicians and researchers worldwide.

RESULTS

As of mid-2014, there were 109 new evidence summaries and 25 recommended practices detailing the best available evidence on topics related to chronic disease management including asthma, diabetes, heart failure, dementia, and others. Training sessions and a newsletter were developed for clinicians within the node to enable them to use the content effectively.

LINKING EVIDENCE TO ACTION

This paper describes the processes involved in the successful development of the collaborative partnership and its evolution into producing a valuable resource for the translation of evidence into practice in the areas of chronic disease management. The resource developed is being used by clinicians to inform practice and support their clinical decision making.

Improving diagnostic accuracy using EHR in emergency departments: A simulation-based study

It is widely believed that Electronic Health Records (EHR) improve medical decision-making by enabling medical staff to access medical information stored in the system. It remains unclear, however, whether EHR indeed fulfills this claim under the severe time constraints of Emergency Departments (EDs). We assessed whether accessing EHR in an ED actually improves decision-making by clinicians. A simulated ED environment was created at the Israel Center for Medical Simulation (MSR). Four different actors were trained to simulate four specific complaints and behavior and 'consulted' 26 volunteer ED physicians. Each physician treated half of the cases (randomly) with access to EHR, and their medical decisions were compared to those where the physicians had no access to EHR. Comparison of diagnostic accuracy with and without access showed that accessing the EHR led to an increase in the quality of the clinical decisions. Physicians accessing EHR were more highly informed and thus made more accurate decisions. The percentage of correct diagnoses was higher and these physicians were more confident in their diagnoses and made their decisions faster.

Evaluating a federated medical search engine: tailoring the methodology and reporting the evaluation outcomes

BACKGROUND

Federated medical search engines are health information systems that provide a single access point to different types of information. Their efficiency as clinical decision support tools has been demonstrated through numerous evaluations. Despite their rigor, very few of these studies report holistic evaluations of medical search engines and even fewer base their evaluations on existing evaluation frameworks.

OBJECTIVES

To evaluate a federated medical search engine, MedSocket, for its potential net benefits in an established clinical setting.

METHODS

This study applied the Human, Organization, and Technology (HOT-fit) evaluation framework in order to evaluate MedSocket. The hierarchical structure of the HOT-factors allowed for identification of a combination of efficiency metrics. Human fit was evaluated through user satisfaction and patterns of system use; technology fit was evaluated through the measurements of time-on-task and the accuracy of the found answers; and organization fit was evaluated from the perspective of system fit to the existing organizational structure.

RESULTS

Evaluations produced mixed results and suggested several opportunities for system improvement. On average, participants were satisfied with MedSocket searches and confident in the accuracy of retrieved answers. However, MedSocket did not meet participants' expectations in terms of download speed, access to information, and relevance of the search results. These mixed results made it necessary to conclude that in the case of MedSocket, technology fit had a significant influence on the human and organization fit. Hence, improving technological capabilities of the system is critical before its net benefits can become noticeable.

CONCLUSIONS

The HOT-fit evaluation framework was instrumental in tailoring the methodology for conducting a comprehensive evaluation of the search engine. Such multidimensional evaluation of the search engine resulted in recommendations for system improvement.

Comparing fusion techniques for the ImageCLEF 2013 medical case retrieval task

Retrieval systems can supply similar cases with a proven diagnosis to a new example case under observation to help clinicians during their work. The ImageCLEFmed evaluation campaign proposes a framework where research groups can compare case-based retrieval approaches. This paper focuses on the case-based task and adds results of the compound figure separation and modality classification tasks. Several fusion approaches are compared to identify the approaches best adapted to the heterogeneous data of the task. Fusion of visual and textual features is analyzed, demonstrating that the selection of the fusion strategy can improve the best performance on the case-based retrieval task.

Using electronic health record systems to optimize admission decisions: the Creatinine case study

Many medical organizations have implemented electronic health record (EHR) and health information exchange (HIE) networks to improve medical decision-making. This study evaluated the contribution of EHR and HIE networks to physicians by investigating whether health information technology can lead to more efficient admission decisions by reducing redundant admissions in the stressful environment of emergency. Log-files were retrieved from an integrative and interoperable EHR that serves seven main Israeli hospitals. The analysis was restricted to a group of patients seen in the emergency departments who were administered a Creatinine test. The assessment of the contribution of EHR to admission decisions used various statistical analyses and track log-file analysis. We showed that using the EHR contributes to more efficient admission decisions and reduces the number of avoidable admissions. In particular, there was a reduction in readmissions when patient history was viewed. Using EHR can help respond to the international problem of avoidable hospital readmissions.

Net improvement of correct answers to therapy questions after pubmed searches: pre/post comparison

BACKGROUND

Clinicians search PubMed for answers to clinical questions although it is time consuming and not always successful.

OBJECTIVE

To determine if PubMed used with its Clinical Queries feature to filter results based on study quality would improve search success (more correct answers to clinical questions related to therapy).

METHODS

We invited 528 primary care physicians to participate, 143 (27.1%) consented, and 111 (21.0% of the total and 77.6% of those who consented) completed the study. Participants answered 14 yes/no therapy questions and were given 4 of these (2 originally answered correctly and 2 originally answered incorrectly) to search using either the PubMed main screen or PubMed Clinical Queries narrow therapy filter via a purpose-built system with identical search screens. Participants also picked 3 of the first 20 retrieved citations that best addressed each question. They were then asked to re-answer the original 14 questions.

RESULTS

We found no statistically significant differences in the rates of correct or incorrect answers using the PubMed main screen or PubMed Clinical Queries. The rate of correct answers increased from 50.0% to 61.4% (95% CI 55.0%-67.8%) for the PubMed main screen searches and from 50.0% to 59.1% (95% CI 52.6%-65.6%) for Clinical Queries searches. These net absolute increases of 11.4% and 9.1%, respectively, included previously correct answers changing to incorrect at a rate of 9.5% (95% CI 5.6%-13.4%) for PubMed main screen searches and 9.1% (95% CI 5.3%-12.9%) for Clinical Queries searches, combined with increases in the rate of being correct of 20.5% (95% CI 15.2%-25.8%) for PubMed main screen searches and 17.7% (95% CI 12.7%-22.7%) for Clinical Queries searches.

CONCLUSIONS

PubMed can assist clinicians answering clinical questions with an approximately 10% absolute rate of improvement in correct answers. This small increase includes more correct answers partially offset by a decrease in previously correct answers.

Decision making in family medicine: randomized trial of the effects of the InfoClinique and Trip database search engines

OBJECTIVE

To compare the ability of users of 2 medical search engines, InfoClinique and the Trip database, to provide correct answers to clinical questions and to explore the perceived effects of the tools on the clinical decision-making process.

DESIGN

Randomized trial.

SETTING

Three family medicine units of the family medicine program of the Faculty of Medicine at Laval University in Quebec city, Que.

PARTICIPANTS

Fifteen second-year family medicine residents.

INTERVENTION

Residents generated 30 structured questions about therapy or preventive treatment (2 questions per resident) based on clinical encounters. Using an Internet platform designed for the trial, each resident answered 20 of these questions (their own 2, plus 18 of the questions formulated by other residents, selected randomly) before and after searching for information with 1 of the 2 search engines. For each question, 5 residents were randomly assigned to begin their search with InfoClinique and 5 with the Trip database.

MAIN OUTCOME MEASURES

The ability of residents to provide correct answers to clinical questions using the search engines, as determined by third-party evaluation. After answering each question, participants completed a questionnaire to assess their perception of the engine's effect on the decision-making process in clinical practice.

RESULTS

Of 300 possible pairs of answers (1 answer before and 1 after the initial search), 254 (85%) were produced by 14 residents. Of these, 132 (52%) and 122 (48%) pairs of answers concerned questions that had been assigned an initial search with InfoClinique and the Trip database, respectively. Both engines produced an important and similar absolute increase in the proportion of correct answers after searching (26% to 62% for InfoClinique, for an increase of 36%; 24% to 63% for the Trip database, for an increase of 39%; P = .68). For all 30 clinical questions, at least 1 resident produced the correct answer after searching with either search engine. The mean (SD) time of the initial search for each question was 23.5 (7.6) minutes with InfoClinique and 22.3 (7.8) minutes with the Trip database (P = .30). Participants' perceptions of each engine's effect on the decision-making process were very positive and similar for both search engines.

CONCLUSION

Family medicine residents' ability to provide correct answers to clinical questions increased dramatically and similarly with the use of both InfoClinique and the Trip database. These tools have strong potential to increase the quality of medical care.

How do medical doctors use a web-based oncology protocol system? A comparison of Australian doctors at different levels of medical training using logfile analysis and an online survey

BACKGROUND

Electronic decision support is commonplace in medical practice. However, its adoption at the point-of-care is dependent on a range of organisational, patient and clinician-related factors. In particular, level of clinical experience is an important driver of electronic decision support uptake. Our objective was to examine the way in which Australian doctors at different stages of medical training use a web-based oncology system (http://www.eviq.org.au).

METHODS

We used logfiles to examine the characteristics of eviQ registrants (2009-2012) and patterns of eviQ use in 2012, according to level of medical training. We also used a web-based survey to evaluate the way doctors at different levels of medical training use the online system and to elicit perceptions of the system's utility in oncology care.

RESULTS

Our study cohort comprised 2,549 eviQ registrants who were hospital-based medical doctors across all levels of training. 65% of the cohort used eviQ in 2012, with 25% of interns/residents, 61% of advanced oncology trainees and 47% of speciality-qualified oncologists accessing eviQ in the last 3 months of 2012. The cohort accounted for 445,492 webhits in 2012. On average, advanced trainees used eviQ up to five-times more than other doctors (42.6 webhits/month compared to 22.8 for specialty-qualified doctors and 7.4 webhits/month for interns/residents). Of the 52 survey respondents, 89% accessed eviQ's chemotherapy protocols on a daily or weekly basis in the month prior to the survey. 79% of respondents used eviQ at least weekly to initiate therapy and to support monitoring (29%), altering (35%) or ceasing therapy (19%). Consistent with the logfile analysis, advanced oncology trainees report more frequent eviQ use than doctors at other stages of medical training.

CONCLUSIONS

The majority of the Australian oncology workforce are registered on eviQ. The frequency of use directly mirrors the clinical role of doctors and attitudes about the utility of eviQ in decision-making. Evaluations of this kind generate important data for system developers and medical educators to drive improvements in electronic decision support to better meet the needs of clinicians. This end-user focus will optimise the uptake of systems which will translate into improvements in processes of care and patient outcomes.

Comparing data accuracy between structured abstracts and full-text journal articles: implications in their use for informing clinical decisions

BACKGROUND

The abstract is the most frequently read section of a research article. The use of 'Consensus Abstracts', a clinician-oriented web application formatted for mobile devices to search MEDLINE/PubMed, for informing clinical decisions was proposed recently; however, inaccuracies between abstracts and the full-text article have been shown. Efforts have been made to improve quality.

METHODS

We compared data in 60 recent-structured abstracts and full-text articles from six highly read medical journals.

RESULTS

Data inaccuracies were identified and then classified as either clinically significant or not significant. Data inaccuracies were observed in 53.33% of articles ranging from 3.33% to 45% based on the IMRAD format sections. The Results section showed the highest discrepancies (45%) although these were deemed to be mostly not significant clinically except in one. The two most common discrepancies were mismatched numbers or percentages (11.67%) and numerical data or calculations found in structured abstracts but not mentioned in the full text (40%). There was no significant relationship between journals and the presence of discrepancies (Fisher's exact p value =0.3405). Although we found a high percentage of inaccuracy between structured abstracts and full-text articles, these were not significant clinically.

CONCLUSIONS

The inaccuracies do not seem to affect the conclusion and interpretation overall. Structured abstracts appear to be informative and may be useful to practitioners as a resource for guiding clinical decisions.

Assisted knowledge discovery for the maintenance of clinical guidelines

Background

Improving antibiotic prescribing practices is an important public-health priority given the widespread antimicrobial resistance. Establishing clinical practice guidelines is crucial to this effort, but their development is a complex task and their quality is directly related to the methodology and source of knowledge used.

Objective

We present the design and the evaluation of a tool (KART) that aims to facilitate the creation and maintenance of clinical practice guidelines based on information retrieval techniques.

Methods

KART consists of three main modules 1) a literature-based medical knowledge extraction module, which is built upon a specialized question-answering engine; 2) a module to normalize clinical recommendations based on automatic text categorizers; and 3) a module to manage clinical knowledge, which formalizes and stores clinical recommendations for further use. The evaluation of the usability and utility of KART followed the methodology of the cognitive walkthrough.

Results

KART was designed and implemented as a standalone web application. The quantitative evaluation of the medical knowledge extraction module showed that 53% of the clinical recommendations generated by KART are consistent with existing clinical guidelines. The user-based evaluation confirmed this result by showing that KART was able to find a relevant antibiotic for half of the clinical scenarios tested. The automatic normalization of the recommendation produced mixed results among end-users.

Conclusions

We have developed an innovative approach for the process of clinical guidelines development and maintenance in a context where available knowledge is increasing at a rate that cannot be sustained by humans. In contrast to existing knowledge authoring tools, KART not only provides assistance to normalize, formalize and store clinical recommendations, but also aims to facilitate knowledge building.

A comparison of the accuracy of clinical decisions based on full-text articles and on journal abstracts alone: a study among residents in a tertiary care hospital

BACKGROUND

Many clinicians depend solely on journal abstracts to guide clinical decisions.

OBJECTIVES

This study aims to determine if there are differences in the accuracy of responses to simulated cases between resident physicians provided with an abstract only and those with full-text articles. It also attempts to describe their information-seeking behaviour.

METHODS

Seventy-seven resident physicians from four specialty departments of a tertiary care hospital completed a paper-based questionnaire with clinical simulation cases, then randomly assigned to two intervention groups-access to abstracts-only and access to both abstracts and full-text. While having access to medical literature, they completed an online version of the same questionnaire.

FINDINGS

The average improvement across departments was not significantly different between the abstracts-only group and the full-text group (p=0.44), but when accounting for an interaction between intervention and department, the effect was significant (p=0.049) with improvement greater with full-text in the surgery department. Overall, the accuracy of responses was greater after the provision of either abstracts-only or full-text (p<0.0001). Although some residents indicated that 'accumulated knowledge' was sufficient to respond to the patient management questions, in most instances (83% of cases) they still sought medical literature.

CONCLUSIONS

Our findings support studies that doctors will use evidence when convenient and current evidence improved clinical decisions. The accuracy of decisions improved after the provision of evidence. Clinical decisions guided by full-text articles were more accurate than those guided by abstracts alone, but the results seem to be driven by a significant difference in one department.

Terminology challenges implementing the HL7 context-aware knowledge retrieval ('Infobutton') standard

Point-of-care information needs are common and frequently unmet. One solution to this problem is the use of Infobuttons, which are context-sensitive links from electronic health records (EHR) to knowledge resources, sometimes involving an intermediate broker known as an Infobutton Manager. Health Level Seven (HL7) has developed the Context-Aware Knowledge Retrieval (Infobutton) standard to standardize the integration between EHR systems and knowledge resources. While the standard specifies a set of context attributes and standard terminologies, it leaves to knowledge resources the flexibility to decide how to use these attributes and terminologies to retrieve the most relevant content. This paper describes some of the challenges faced by knowledge resources in trying to locate the most relevant content based on the attribute values for a given Infobutton request. Various approaches to content retrieval are discussed, including the role of indexing with standardized codes, the role of text-based search engines together with their ranking algorithms, and the role of hybrid approaches. Knowledge resource developers must carefully consider business rules, heuristics, and precision/recall tradeoffs when implementing the HL7 Infobutton standard.

A Multi-Tiered Perspective on Healthcare Interoperability

The current healthcare delivery paradigm is defined by integrative care delivery across disparate providers and services. Therefore the ability to deliver efficient and effective healthcare services is dependent on designing and implementing interoperable systems. However, the notion of interoperability is multifaceted and complex. Although the exchange of data is often described as analogous with interoperability it must be remembered that healthcare is a process oriented domain and clinical, management, organizational and other processes must be considered as part of interoperability. This chapter discusses healthcare delivery and the role interoperability plays in supporting its delivery. First, the chapter provides a background on healthcare interoperability from multiple perspectives. Then it presents a case study of collaborative care delivery and uses it to outline specific interoperability requirements. The chapter then uses these requirements to develop a multi-tiered framework of healthcare interoperability, concluding with a discussion of the implications of the framework for interoperability research and for systems design to support integrated healthcare delivery.

Emergency department physician internet use during clinical encounters

OBJECTIVE

This study explored the Internet log files from emergency department workstations to determine search patterns, compared them to discharge diagnoses, and the emergency medicine curriculum as a way to quantify physician search behaviors.

METHODS

The log files from the computers from January 2006 to March 2010 were mapped to the EM curriculum and compared to discharge diagnoses to explore search terms and website usage by physicians and students.

RESULTS

Physicians in the ED averaged 1.35 searches per patient encounter using Google.com and UpToDate.com 83.9% of the time. The most common searches were for drug information (23.1%) by all provider types. The majority of the websites utilized were in the third tier evidence level for evidence-based medicine (EBM).

CONCLUSION

We have shown a need for a readily accessible drug knowledge base within the EMR for decision support as well as easier access to first and second tier EBM evidence.

Information retrieval - Swedish specialist student nurses` strategies for finding clinical evidence

The care that intra-operative nurses perform has to be based on scientific knowledge. It is therefore vitally important that they have access to different databases and the Internet, as well as knowledge of the search procedure to find evidence for best practice. This means that specialist nurses should be proficient in the search procedure, interpretation of data, and implementation of available knowledge.The aim of this study was therefore to highlight the search strategies of specialist student nurses for finding scientific knowledge with regard to specific clinical issues. An additional aim was to assess their ability to critically evaluate scientific articles.

METHODS

The participants, n 16, all students enrolled in a university programme for specialist nursing in anaesthesia care, were required to find support for six questions, by using scientific knowledge databases. The tasks that required support are related to their future profession and were divided into three main areas: pre-, intra-, and post-operative care. Two distinct questions for each main area had to be answered. The data was analysed quantitatively using manifest content analysis as a basis.

RESULT

This study reveals that the student nurses mostly used the CINAHL and PubMed databases to search for answers related to the areas in focus. The keywords the participants used differed between the individuals and were applied more frequently than MESH terms. In addition, the critical evaluation of articles of scientific value that were found was not optimal. The study demonstrated that most of the participants were unable to complete all the required tasks. With regard to the initial questions, all 16 participants provided answers, while only eight student nurses completed the final questions.

CONCLUSIONS

The specialist student nurses in this study used the databases Chinal and PubMed to find scientific knowledge with the help of MESH-terms and keywords. Further research is needed to understand how education of the specialist student nurses should be carried out, in order to optimise their search strategies and critical evaluation of scientific articles.

The role of ICT in supporting disruptive innovation: a multi-site qualitative study of nurse practitioners in emergency departments

BACKGROUND

The disruptive potential of the Nurse Practitioner (NP) is evident in their ability to offer services traditionally provided by primary care practitioners and their provision of a health promotion model of care in response to changing health trends. No study has qualitatively investigated the role of the Emergency NP in Australia, nor the impact of Information and Communication Technology (ICT) on this disruptive workforce innovation. This study aimed to investigate ways in which Nurse Practitioners (NP) have incorporated the use of ICT as a mechanism to support their new clinical role within Emergency Departments.

METHODS

A cross-sectional qualitative study was undertaken in the Emergency Departments (EDs) of two large Australian metropolitan public teaching hospitals. Semi-structured, in-depth interviews were conducted with five nurse practitioners, four senior physicians and five senior nurses. Transcribed interviews were analysed using a grounded theory approach to develop themes in relation to the conceptualisation of the ED nurse practitioner role and the influences of ICT upon the role. Member checking of results was achieved by revisiting the sites to clarify findings with participants and further explore emergent themes.

RESULTS

The role of the ENP was distinguished from those of Emergency nurses and physicians by two elements: advanced practice and holistic care, respectively. ICT supported the advanced practice dimension of the NP role in two ways: availability and completeness of electronic patient information enhanced timeliness and quality of diagnostic and therapeutic decision-making, expediting patient access to appropriate care. The ubiquity of patient data sourced from a central database supported and improved quality of communication between health professionals within and across sites, with wider diffusion of the Electronic Medical Record holding the potential to further facilitate team-based, holistic care.

CONCLUSIONS

ICT is a facilitator through which the disruptive impact of NPs can be extended. However, integration of ICT into work practices without detracting from provider-patient interaction is crucial to ensure utilisation of such interventions and realisation of potential benefits.

Internet use for information seeking in clinical practice: a cross-sectional survey among French general practitioners

BACKGROUND

Medical information needs regarding patient care are particularly large for general practitioners (GPs). The Internet seems to be a relevant but underused tool to seek medical information.

OBJECTIVE

We aimed to describe the characteristics of the French GPs using the Internet for information seeking, to identify the barriers to its use and the factors that could facilitate it.

METHOD

We conducted a cross-sectional survey among GPs currently practicing in France, using an online questionnaire, in July 2009. We analysed the answers of 721 respondents.

RESULTS

Most of the respondents used the Internet to seek information. They were significantly younger, worked in group practice, had Internet training and had Internet access at the practice. The main barriers were related to the physician (lack of knowledge or specific skills), to the practice conditions (lack of time, concerns about relationship with patient, financial non-recognition) and to the information (information overload, quality concerns, low relevance, language barrier). Practitioners wanted more reliable and more relevant documents for daily practice. Websites with already selected resources could increase the GPs use of the Internet for medical information seeking.

CONCLUSION

The reported obstacles were largely common with those previously described in other countries, except the language barrier and the financial non-recognition. Even if the generalization of our results to all French GPs should be cautious, the study provided better insights into the obstacles to the Internet use to seek clinical information in family practice and the factors that could facilitate it.

Impact of knowledge resources linked to an electronic health record on frequency of unnecessary tests and treatments

INTRODUCTION

Electronic knowledge resources have the potential to rapidly provide answers to clinicians' questions. We sought to determine clinicians' reasons for searching these resources, the rate of finding relevant information, and the perceived clinical impact of the information they retrieved.

METHODS

We asked general internists, family physicians, and clinical nurse practitioners to complete the Information Assessment Method (IAM) survey after searching 1 of 2 electronic knowledge resources linked in the electronic health record. IAM stimulates reflection on the relevance, cognitive impact, use, and potential health outcomes of retrieved clinical information.

RESULTS

Forty-two clinicians rated 502 searches (mean 12, range 1-48) and reported finding information 75% (n = 375) of the time. The most common reasons for searching were to address a clinical question (411, 82%) and for curiosity (75, 15%). In 68% of the rated searches (341), participants indicated they would use the retrieved information for at least 1 patient. In 31% (157) of rated searches, clinicians expected the retrieved information to benefit the patient by avoiding an unnecessary or inappropriate treatment, diagnostic procedure, or preventive intervention.

CONCLUSIONS

Searches in electronic knowledge resources frequently yield relevant information that may benefit the patient by, for example, avoiding an inappropriate diagnostic procedure or treatment. Knowing that searches for answers to clinical questions can result in patient health benefits should intensify efforts to encourage clinicians to pursue answers to their questions.

Public access and use of health research: an exploratory study of the National Institutes of Health (NIH) Public Access Policy using interviews and surveys of health personnel

BACKGROUND

In 2008, the National Institutes of Health (NIH) Public Access Policy mandated open access for publications resulting from NIH funding (following a 12-month embargo). The large increase in access to research that will take place in the years to come has potential implications for evidence-based practice (EBP) and lifelong learning for health personnel.

OBJECTIVE

This study assesses health personnel's current use of research to establish whether grounds exist for expecting, preparing for, and further measuring the impact of the NIH Public Access Policy on health care quality and outcomes in light of time constraints and existing information resources.

METHODS

In all, 14 interviews and 90 surveys of health personnel were conducted at a community-based clinic and an independent teaching hospital in 2010. Health personnel were asked about the research sources they consulted and the frequency with which they consulted these sources, as well as motivation and search strategies used to locate articles, perceived level of access to research, and knowledge of the NIH Public Access Policy.

RESULTS

In terms of current access to health information, 65% (57/88) of the health personnel reported being satisfied, while 32% (28/88) reported feeling underserved. Among the sources health personnel reported that they relied upon and consulted weekly, 83% (73/88) reported turning to colleagues, 77% (67/87) reported using synthesized information resources (eg, UpToDate and Cochrane Systematic Reviews), while 32% (28/88) reported that they consulted primary research literature. The dominant resources health personnel consulted when actively searching for health information were Google and Wikipedia, while 27% (24/89) reported using PubMed weekly. The most prevalent reason given for accessing research on a weekly basis, reported by 35% (31/88) of survey respondents, was to help a specific patient, while 31% (26/84) were motivated by general interest in research.

CONCLUSIONS

The results provide grounds for expecting the NIH Public Access Policy to have a positive impact on EBP and health care more generally given that between a quarter and a third of participants in this study (1) frequently accessed research literature, (2) expressed an interest in having greater access, and (3) were aware of the policy and expect it to have an impact on their accessing research literature in the future. Results also indicate the value of promoting a greater awareness of the NIH policy, providing training and education in the location and use of the literature, and continuing improvements in the organization of biomedical research for health personnel use.

Exploiting domain information for Word Sense Disambiguation of medical documents

Objective

Current techniques for knowledge-based Word Sense Disambiguation (WSD) of ambiguous biomedical terms rely on relations in the Unified Medical Language System Metathesaurus but do not take into account the domain of the target documents. The authors' goal is to improve these methods by using information about the topic of the document in which the ambiguous term appears.

Design

The authors proposed and implemented several methods to extract lists of key terms associated with Medical Subject Heading terms. These key terms are used to represent the document topic in a knowledge-based WSD system. They are applied both alone and in combination with local context.

Measurements

A standard measure of accuracy was calculated over the set of target words in the widely used National Library of Medicine WSD dataset.

Results and discussion

The authors report a significant improvement when combining those key terms with local context, showing that domain information improves the results of a WSD system based on the Unified Medical Language System Metathesaurus alone. The best results were obtained using key terms obtained by relevance feedback and weighted by inverse document frequency.

Towards spoken clinical-question answering: evaluating and adapting automatic speech-recognition systems for spoken clinical questions

OBJECTIVE

To evaluate existing automatic speech-recognition (ASR) systems to measure their performance in interpreting spoken clinical questions and to adapt one ASR system to improve its performance on this task.

DESIGN AND MEASUREMENTS

The authors evaluated two well-known ASR systems on spoken clinical questions: Nuance Dragon (both generic and medical versions: Nuance Gen and Nuance Med) and the SRI Decipher (the generic version SRI Gen). The authors also explored language model adaptation using more than 4000 clinical questions to improve the SRI system's performance, and profile training to improve the performance of the Nuance Med system. The authors reported the results with the NIST standard word error rate (WER) and further analyzed error patterns at the semantic level.

RESULTS

Nuance Gen and Med systems resulted in a WER of 68.1% and 67.4% respectively. The SRI Gen system performed better, attaining a WER of 41.5%. After domain adaptation with a language model, the performance of the SRI system improved 36% to a final WER of 26.7%.

CONCLUSION

Without modification, two well-known ASR systems do not perform well in interpreting spoken clinical questions. With a simple domain adaptation, one of the ASR systems improved significantly on the clinical question task, indicating the importance of developing domain/genre-specific ASR systems.