Clinical questions raised by clinicians at the point of care: a systematic review

Common pitfalls and mistakes in the set-up, analysis and interpretation of results in network meta-analysis: what clinicians should look for in a published article

OBJECTIVE

Several tools have been developed to evaluate the extent to which the findings from a network meta-analysis would be valid; however, applying these tools is a time-consuming task and often requires specific expertise. Clinicians have little time for critical appraisal, and they need to understand the key elements that help them select network meta-analyses that deserve further attention, optimising time and resources. This paper is aimed at providing a practical framework to assess the methodological robustness and reliability of results from network meta-analysis.

METHODS

As a working example, we selected a network meta-analysis about drug treatments for generalised anxiety disorder, which was published in 2011 in the British Medical Journal. The same network meta-analysis was previously used to illustrate the potential of this methodology in a methodological paper published in JAMA.

RESULTS

We reanalysed the 27 studies included in this network following the methods reported in the original article and compared our findings with the published results. We showed how different methodological approaches and the presentation of results can affect conclusions from network meta-analysis. We divided our results into three sections, according to the specific issues that should always be addressed in network meta-analysis: (1) understanding the evidence base, (2) checking the statistical analysis and (3) checking the reporting of findings.

CONCLUSIONS

The validity of the results from network meta-analysis depends on the plausibility of the transitivity assumption. The risk of bias introduced by limitations of individual studies must be considered first and judgement should be used to infer about the plausibility of transitivity. Inconsistency exists when treatment effects from direct and indirect evidence are in disagreement. Unlike transitivity, inconsistency can be always evaluated statistically, and it should be specifically investigated and reported in the published paper. Network meta-analysis allows researchers to list treatments in preferential order; however, in this paper we demonstrated that rankings could be misleading if based on the probability of being the best. Clinicians should always be interested in the effect sizes rather than the naive rankings.

Automatic identification of high impact articles in PubMed to support clinical decision making

OBJECTIVES

The practice of evidence-based medicine involves integrating the latest best available evidence into patient care decisions. Yet, critical barriers exist for clinicians' retrieval of evidence that is relevant for a particular patient from primary sources such as randomized controlled trials and meta-analyses. To help address those barriers, we investigated machine learning algorithms that find clinical studies with high clinical impact from PubMed®.

METHODS

Our machine learning algorithms use a variety of features including bibliometric features (e.g., citation count), social media attention, journal impact factors, and citation metadata. The algorithms were developed and evaluated with a gold standard composed of 502 high impact clinical studies that are referenced in 11 clinical evidence-based guidelines on the treatment of various diseases. We tested the following hypotheses: (1) our high impact classifier outperforms a state-of-the-art classifier based on citation metadata and citation terms, and PubMed's® relevance sort algorithm; and (2) the performance of our high impact classifier does not decrease significantly after removing proprietary features such as citation count.

RESULTS

The mean top 20 precision of our high impact classifier was 34% versus 11% for the state-of-the-art classifier and 4% for PubMed's® relevance sort (p=0.009); and the performance of our high impact classifier did not decrease significantly after removing proprietary features (mean top 20 precision=34% vs. 36%; p=0.085).

CONCLUSION

The high impact classifier, using features such as bibliometrics, social media attention and MEDLINE® metadata, outperformed previous approaches and is a promising alternative to identifying high impact studies for clinical decision support.

Unanswered clinical questions: a survey of specialists and primary care providers

Objective

With the myriad of cases presented to clinicians every day at our integrated academic health system, clinical questions are bound to arise. Clinicians need to recognize these knowledge gaps and act on them. However, for many reasons, clinicians might not seek answers to these questions. Our goal was to investigate the rationale and process behind these unanswered clinical questions. Subsequently, we explored the use of biomedical information resources among specialists and primary care providers and identified ways to promote more informed clinical decision making.

Methods

We conducted a survey to assess how practitioners identify and respond to information gaps, their background knowledge of search tools and strategies, and their usage of and comfort level with technology.

Results

Most of the 292 respondents encountered clinical questions at least a few times per week. While the vast majority often or always pursued answers, time was the biggest barrier for not following through on questions. Most respondents did not have any formal training in searching databases, were unaware of many digital resources, and indicated a need for resources and services that could be provided at the point of care.

Conclusions

While the reasons for unanswered clinical questions varied, thoughtful review of the responses suggested that a combination of educational strategies, embedded librarian services, and technology applications could help providers pursue answers to their clinical questions, enhance patient safety, and contribute to patient-based, self-directed learning.

DrugSemantics: A corpus for Named Entity Recognition in Spanish Summaries of Product Characteristics

For the healthcare sector, it is critical to exploit the vast amount of textual health-related information. Nevertheless, healthcare providers have difficulties to benefit from such quantity of data during pharmacotherapeutic care. The problem is that such information is stored in different sources and their consultation time is limited. In this context, Natural Language Processing techniques can be applied to efficiently transform textual data into structured information so that it could be used in critical healthcare applications, being of help for physicians in their daily workload, such as: decision support systems, cohort identification, patient management, etc. Any development of these techniques requires annotated corpora. However, there is a lack of such resources in this domain and, in most cases, the few ones available concern English. This paper presents the definition and creation of DrugSemantics corpus, a collection of Summaries of Product Characteristics in Spanish. It was manually annotated with pharmacotherapeutic named entities, detailed in DrugSemantics annotation scheme. Annotators were a Registered Nurse (RN) and two students from the Degree in Nursing. The quality of DrugSemantics corpus has been assessed by measuring its annotation reliability (overall F=79.33% [95%CI: 78.35-80.31]), as well as its annotation precision (overall P=94.65% [95%CI: 94.11-95.19]). Besides, the gold-standard construction process is described in detail. In total, our corpus contains more than 2000 named entities, 780 sentences and 226,729 tokens. Last, a Named Entity Classification module trained on DrugSemantics is presented aiming at showing the quality of our corpus, as well as an example of how to use it.

A call to action: The controversy of and rationale for competency-based medical education

Although medical education has enjoyed many successes over the last century, there is a recognition that health care is too often unsafe and of poor quality. Errors in diagnosis and treatment, communication breakdowns, poor care coordination, inappropriate use of tests and procedures, and dysfunctional collaboration harm patients and families around the world. These issues reflect on our current model of medical education and raise the question: Are physicians being adequately prepared for twenty-first century practice? Multiple reports have concluded the answer is "no." Concurrent with this concern is an increasing interest in competency-based medical education (CBME) as an approach to help reform medical education. The principles of CBME are grounded in providing better and safer care. As interest in CBME has increased, so have criticisms of the movement. This article summarizes and addresses objections and challenges related to CBME. These can provide valuable feedback to improve CBME implementation and avoid pitfalls. We strongly believe medical education reform should not be reduced to an "either/or" approach, but should blend theories and approaches to suit the needs and resources of the populations served. The incorporation of milestones and entrustable professional activities within existing competency frameworks speaks to the dynamic evolution of CBME, which should not be viewed as a fixed doctrine, but rather as a set of evolving concepts, principles, tools, and approaches that can enable important reforms in medical education that, in turn, enable the best outcomes for patients.

Context-sensitive decision support (infobuttons) in electronic health records: a systematic review

OBJECTIVE

Infobuttons appear as small icons adjacent to electronic health record (EHR) data (e.g., medications, diagnoses, or test results) that, when clicked, access online knowledge resources tailored to the patient, care setting, or task. Infobuttons are required for "Meaningful Use" certification of US EHRs. We sought to evaluate infobuttons' impact on clinical practice and identify features associated with improved outcomes.

METHODS

We conducted a systematic review, searching MEDLINE, EMBASE, and other databases from inception to July 6, 2015. We included and cataloged all original research in any language describing implementation of infobuttons or other context-sensitive links. Studies evaluating clinical implementations with outcomes of usage or impact were reviewed in greater detail. Reviewers worked in duplicate to select articles, evaluate quality, and abstract information.

RESULTS

Of 599 potential articles, 77 described infobutton implementation. The 17 studies meriting detailed review, including 3 randomized trials, yielded the following findings. Infobutton usage frequency ranged from 0.3 to 7.4 uses per month per potential user. Usage appeared to be influenced by EHR task. Five studies found that infobuttons are used less often than non-context-sensitive links (proportionate usage 0.20-0.34). In 3 studies, users answered their clinical question in > 69% of infobutton sessions. Seven studies evaluated alternative approaches to infobutton design and implementation. No studies isolated the impact of infobuttons on objectively measured patient outcomes.

CONCLUSIONS

Weak evidence suggests that infobuttons can help providers answer clinical questions. Research on optimal infobutton design and implementation, and on the impact on patient outcomes and provider behaviors, is needed.

The Team-Based Approach to Undiagnosed and Rare Diseases

Patients with undiagnosed or rare diseases often remain without a diagnosis for many years. Many are misdiagnosed or treated symptomatically without having an identified underlying disease process. Health care providers in general practice and subspecialists are equipped to diagnose diseases commonly seen. Most practitioners are unlikely to be familiar with uncommon manifestations of a common disorder and have little or no experience with rare diseases. Multidisciplinary teams are effective in reviewing patients with undiagnosed and rare diseases and in developing a new diagnostic strategy for appropriate evaluation. A medical librarian and an access coordinating navigator are essential members of the team.

Physicians' perception of alternative displays of clinical research evidence for clinical decision support - A study with case vignettes

OBJECTIVE

To design alternate information displays that present summaries of clinical trial results to clinicians to support decision-making; and to compare the displays according to efficacy and acceptability.

METHODS

A 6-between (information display presentation order) by 3-within (display type) factorial design. Two alternate displays were designed based on Information Foraging theory: a narrative summary that reduces the content to a few sentences; and a table format that structures the display according to the PICO (Population, Intervention, Comparison, Outcome) framework. The designs were compared with the summary display format available in PubMed. Physicians were asked to review five clinical studies retrieved for a case vignette; and were presented with the three display formats. Participants were asked to rate their experience with each of the information displays according to a Likert scale questionnaire.

RESULTS

Twenty physicians completed the study. Overall, participants rated the table display more highly than either the text summary or PubMed's summary format (5.9vs. 5.4vs. 3.9 on a scale between 1 [strongly disagree] and 7 [strongly agree]). Usefulness ratings of seven pieces of information, i.e. patient population, patient age range, sample size, study arm, primary outcome, results of primary outcome, and conclusion, were high (average across all items=4.71 on a 1 to 5 scale, with 1=not at all useful and 5=very useful). Study arm, primary outcome, and conclusion scored the highest (4.9, 4.85, and 4.85 respectively). Participants suggested additional details such as rate of adverse effects.

CONCLUSION

The table format reduced physicians' perceived cognitive effort when quickly reviewing clinical trial information and was more favorably received by physicians than the narrative summary or PubMed's summary format display.

Unique Educational Opportunities for PCPs and Specialists Arising From Electronic Consultation Services

Health care reform should be driven by the goals of better patient experience, improved population health, lower per capita costs, and improved provider satisfaction. Electronic consultation (eConsult) services have been adopted by several jurisdictions in the United States, Canada, and Europe to improve access to specialists by primary care providers (PCPs) and are being heralded as a key component for delivery of coordinated care. The primary intent of an eConsult service is to provide PCPs with efficient, timely, direct access to specialist expertise to help guide the management of their patients, reduce the need for unnecessary face-to-face specialty consultations, and improve the quality of the initial face-to-face consultation when needed, through the preconsultative communication.In addition to improving access to care, eConsult services have been praised by PCPs and specialists for their educational value, in particular their ability to enrich practice-based learning. Less recognized, but equally important from the educational perspective, include the abilities of eConsult programs to promote reflection by PCPs and specialists, improve collegiality and professionalism between primary and specialist care, inform continuing professional development activities and maintenance of certification, and enhance training programs' teaching of effective communication and care coordination.As eConsult services become increasingly available, the medical community must leverage the educational opportunities inherent in eConsult programs to further improve the delivery of coordinated specialty care. The educational role of eConsults should be considered as a priority outcome in their evaluation and must be highlighted and optimized in next iterations of eConsult systems design.

Generating disease-pertinent treatment vocabularies from MEDLINE citations

OBJECTIVE

Healthcare communities have identified a significant need for disease-specific information. Disease-specific ontologies are useful in assisting the retrieval of disease-relevant information from various sources. However, building these ontologies is labor intensive. Our goal is to develop a system for an automated generation of disease-pertinent concepts from a popular knowledge resource for the building of disease-specific ontologies.

METHODS

A pipeline system was developed with an initial focus of generating disease-specific treatment vocabularies. It was comprised of the components of disease-specific citation retrieval, predication extraction, treatment predication extraction, treatment concept extraction, and relevance ranking. A semantic schema was developed to support the extraction of treatment predications and concepts. Four ranking approaches (i.e., occurrence, interest, degree centrality, and weighted degree centrality) were proposed to measure the relevance of treatment concepts to the disease of interest. We measured the performance of four ranks in terms of the mean precision at the top 100 concepts with five diseases, as well as the precision-recall curves against two reference vocabularies. The performance of the system was also compared to two baseline approaches.

RESULTS

The pipeline system achieved a mean precision of 0.80 for the top 100 concepts with the ranking by interest. There were no significant different among the four ranks (p=0.53). However, the pipeline-based system had significantly better performance than the two baselines.

CONCLUSIONS

The pipeline system can be useful for an automated generation of disease-relevant treatment concepts from the biomedical literature.

Identifying complexity in infectious diseases inpatient settings: An observation study

BACKGROUND

Understanding complexity in healthcare has the potential to reduce decision and treatment uncertainty. Therefore, identifying both patient and task complexity may offer better task allocation and design recommendation for next-generation health information technology system design.

OBJECTIVE

To identify specific complexity-contributing factors in the infectious disease domain and the relationship with the complexity perceived by clinicians.

METHOD

We observed and audio recorded clinical rounds of three infectious disease teams. Thirty cases were observed for a period of four consecutive days. Transcripts were coded based on clinical complexity-contributing factors from the clinical complexity model. Ratings of complexity on day 1 for each case were collected. We then used statistical methods to identify complexity-contributing factors in relationship to perceived complexity of clinicians.

RESULTS

A factor analysis (principal component extraction with varimax rotation) of specific items revealed three factors (eigenvalues>2.0) explaining 47% of total variance, namely task interaction and goals (10 items, 26%, Cronbach's Alpha=0.87), urgency and acuity (6 items, 11%, Cronbach's Alpha=0.67), and psychosocial behavior (4 items, 10%, Cronbach's alpha=0.55). A linear regression analysis showed no statistically significant association between complexity perceived by the physicians and objective complexity, which was measured from coded transcripts by three clinicians (Multiple R-squared=0.13, p=0.61). There were no physician effects on the rating of perceived complexity.

CONCLUSION

Task complexity contributes significantly to overall complexity in the infectious diseases domain. The different complexity-contributing factors found in this study can guide health information technology system designers and researchers for intuitive design. Thus, decision support tools can help reduce the specific complexity-contributing factors. Future studies aimed at understanding clinical domain-specific complexity-contributing factors can ultimately improve task allocation and design for intuitive clinical reasoning.

Does integrated training in evidence-based medicine (EBM) in the general practice (GP) specialty training improve EBM behaviour in daily clinical practice? A cluster randomised controlled trial

OBJECTIVES

Evidence-based medicine (EBM) is an important element in the general practice (GP) specialty training. Studies show that integrating EBM training into clinical practice brings larger benefits than stand-alone modules. However, these studies have neither been performed in GP nor assessed EBM behaviour of former trainees in daily clinical practice.

SETTING

GP specialty training in the Netherlands.

PARTICIPANTS

All 82 third year GP trainees who started their final third year in 2011 were approached for inclusion, of whom 79 (96%) participated: 39 in the intervention group and 40 in the control group.

INTERVENTION

Integrated EBM training, in which EBM is embedded closely within the clinical context by joint assignments for the trainee and supervisor in daily practice, and teaching sessions based on dilemmas from actual patient consultations.

COMPARISON

Stand-alone EBM training at the institute only.

PRIMARY AND SECONDARY OUTCOMES

Our primary outcome was EBM behaviour, assessed by measuring guideline adherence (incorporating rational, motivated deviation) and information-seeking behaviour. Our secondary outcomes were EBM attitude and EBM knowledge. Data were acquired using logbooks and questionnaires, respectively. Analyses were performed using mixed models.

RESULTS

Logbook data were available from 76 (96%) of the participating trainees at baseline (7614 consultations), 60 (76%) at the end of the third year (T1, 4973 consultations) and 53 (67%) 1 year after graduation (T2, 3307 consultations). We found no significant differences in outcomes between the 2 groups, with relative risks for guideline adherence varying between 0.96 and 0.99 (95% CI 0.86 to 1.11) at T1, and 0.99 and 1.10 (95% CI 0.92 to 1.25) at T2, and for information-seeking behaviour between 0.97 and 1.16 (95% CI 0.70 to 1.91) and 0.90 and 1.10 (95% CI 0.70 to 1.32), respectively.

CONCLUSIONS

Integrated EBM training compared with stand-alone EBM training does not improve EBM behaviour, attitude or knowledge of (future) GPs.

Integrating Genomic Resources with Electronic Health Records using the HL7 Infobutton Standard

BACKGROUND

The Clinical Genome Resource (ClinGen) Electronic Health Record (EHR) Workgroup aims to integrate ClinGen resources with EHRs. A promising option to enable this integration is through the Health Level Seven (HL7) Infobutton Standard. EHR systems that are certified according to the US Meaningful Use program provide HL7-compliant infobutton capabilities, which can be leveraged to support clinical decision-making in genomics.

OBJECTIVES

To integrate genomic knowledge resources using the HL7 infobutton standard. Two tactics to achieve this objective were: (1) creating an HL7-compliant search interface for ClinGen, and (2) proposing guidance for genomic resources on achieving HL7 Infobutton standard accessibility and compliance.

METHODS

We built a search interface utilizing OpenInfobutton, an open source reference implementation of the HL7 Infobutton standard. ClinGen resources were assessed for readiness towards HL7 compliance. Finally, based upon our experiences we provide recommendations for publishers seeking to achieve HL7 compliance.

RESULTS

Eight genomic resources and two sub-resources were integrated with the ClinGen search engine via OpenInfobutton and the HL7 infobutton standard. Resources we assessed have varying levels of readiness towards HL7-compliance. Furthermore, we found that adoption of standard terminologies used by EHR systems is the main gap to achieve compliance.

CONCLUSION

Genomic resources can be integrated with EHR systems via the HL7 Infobutton standard using OpenInfobutton. Full compliance of genomic resources with the Infobutton standard would further enhance interoperability with EHR systems.

Barriers in seeking health information from primary healthcare facilities in Pakistan

Primary care physicians (PCPs) need a wide range of health information resources to manage their patients. The present study aimed to investigate the barriers that obstruct the health information seeking behavior of primary care physicians in Pakistan from their practice setting. A survey was carried out in 77 basic health units (BHUs), seven rural health centers (RHCs), and two tehsil headquarters (THQs) of Multan District, Pakistan. A three-part questionnaire containing participants’ demographic information, barriers in seeking health information, and required sources and services, was distributed to primary care physicians through post, email, and personal visits to participating healthcare facilities. The data was analyzed statistically using SPSS. Chi square and Kruskal Wallis tests were applied to determine if there were any significant differences in demographic information of different healthcare facilities participants, the barriers they faced in seeking health information, and the required sources and services available to them. The major barriers included: no onsite library, no health sciences librarian, scarcity of health information resources, inaccessibility of health information system, and lack of information and technological infrastructure. Consultation with seniors, improved connectivity and document delivery services were among the sources and services most required by primary care physicians. The study concluded that primary care physicians encounter multiple barriers that obstruct their health-related information seeking behavior, which compromises patient care. Medical librarians should design outreach library and information services in health sciences and health information literacy programs to overcome the barriers primary care physicians face while seeking health information from their practice settings.

Practitioner Review: On the trustworthiness of clinical practice guidelines - a systematic review of the quality of methods used to develop guidelines in child and youth mental health

BACKGROUND

Numerous practice guidelines (PGs) relevant to child and youth mental health (CYMH) are available, but their quality is uncertain. We used systematic review methodology to identify the methods employed to develop PGs in CYMH and assess whether they align with international quality standards.

METHODS

We used prespecified inclusion criteria to search for CYMH PGs (2009-2014) in journals of professional associations or websites of organizations who produce or house PGs. Eligible PGs and organization websites were screened to identify PG development methods. Two reviewers assessed the alignment of the PG development methods with PG quality criteria using the Appraisal of Guidelines for Research and Evaluation (AGREE II) domains and Institute of Medicine (IOM) standards.

RESULTS

Five sets of eligible development methods were identified in 70 eligible PGs. Three sets adhered to all (National Institute for Health and Care Excellence; Scottish Intercollegiate Guidelines Network) or most (U.S. Preventive Services Task Force) AGREE II domains and IOM standards, and were used to develop 31.4% of PGs. The two remaining sets of development methods had important weaknesses (e.g. lack of mandatory rigorous systematic reviews, multidisciplinary development groups, or transparent conflict of interest methods) and were associated with 21.4% of PGs. No development methods could be identified in 40.0% of PGs; ineligible development methods were referenced in 7.1% of PGs.

CONCLUSIONS

Up to 69% of available CYMH PGs may have been developed using methods that do not align with AGREE II quality criteria or IOM standards. The quality of available CYMH PGs needs to be assessed, and strategies designed to guide practitioners to high quality PGs and facilitate adherence by PG developers to international quality standards are needed.

Evaluation of an Electronic Module for Reconciling Medications in Home Health Plans of Care

OBJECTIVES

Transitions in patient care pose an increased risk to patient safety. One way to reduce this risk is to ensure accurate medication reconciliation during the transition. Here we present an evaluation of an electronic medication reconciliation module we developed to reduce the transition risk in patients referred for home healthcare.

METHODS

Nineteen physicians with experience in managing home health referrals were recruited to participate in this within-subjects experiment. Participants completed medication reconciliation for three clinical cases in each of two conditions. The first condition (paper-based) simulated current practice - reconciling medication discrepancies between a paper plan of care (CMS 485) and a simulated Electronic Health Record (EHR). For the second condition (electronic) participants used our medication reconciliation module, which we integrated into the simulated EHR. To evaluate the effectiveness of our medication reconciliation module, we employed repeated measures ANOVA to test the hypotheses that the module will: 1) Improve accuracy by reducing the number of unaddressed medication discrepancies, 2) Improve efficiency by reducing the reconciliation time, 3) have good perceived usability.

RESULTS

The improved accuracy hypothesis is supported. Participants left more discrepancies unaddressed in the paper-based condition than the electronic condition, F (1,1) = 22.3, p < 0.0001 (Paper Mean = 1.55, SD = 1.20; Electronic Mean = 0.45, SD = 0.65). However, contrary to our efficiency hypothesis, participants took the same amount of time to complete cases in the two conditions, F (1, 1) =0.007, p = 0.93 (Paper Mean = 258.7 seconds, SD = 124.4; Electronic Mean = 260.4 seconds, SD = 158.9). The usability hypothesis is supported by a composite mean ability and confidence score of 6.41 on a 7-point scale, 17 of 19 participants preferring the electronic system and an SUS rating of 86.5.

CONCLUSION

We present the evaluation of an electronic medication reconciliation module that increases detection and resolution of medication discrepancies compared to a paper-based process. Further work to integrate medication reconciliation within an electronic medical record is warranted.

Interactive use of online health resources: a comparison of consumer and professional questions

OBJECTIVE

To understand how consumer questions on online resources differ from questions asked by professionals, and how such consumer questions differ across resources.

MATERIALS AND METHODS

Ten online question corpora, 5 consumer and 5 professional, with a combined total of over 40 000 questions, were analyzed using a variety of natural language processing techniques. These techniques analyze questions at the lexical, syntactic, and semantic levels, exposing differences in both form and content.

RESULTS

Consumer questions tend to be longer than professional questions, more closely resemble open-domain language, and focus far more on medical problems. Consumers ask more sub-questions, provide far more background information, and ask different types of questions than professionals. Furthermore, there is substantial variance of these factors between the different consumer corpora.

DISCUSSION

The form of consumer questions is highly dependent upon the individual online resource, especially in the amount of background information provided. Professionals, on the other hand, provide very little background information and often ask much shorter questions. The content of consumer questions is also highly dependent upon the resource. While professional questions commonly discuss treatments and tests, consumer questions focus disproportionately on symptoms and diseases. Further, consumers place far more emphasis on certain types of health problems (eg, sexual health).

CONCLUSION

Websites for consumers to submit health questions are a popular online resource filling important gaps in consumer health information. By analyzing how consumers write questions on these resources, we can better understand these gaps and create solutions for improving information access.This article is part of the Special Focus on Person-Generated Health and Wellness Data, which published in the May 2016 issue, Volume 23, Issue 3.

Classification of clinically useful sentences in clinical evidence resources

Most patient care questions raised by clinicians can be answered by online clinical knowledge resources. However, important barriers still challenge the use of these resources at the point of care.

OBJECTIVE

To design and assess a method for extracting clinically useful sentences from synthesized online clinical resources that represent the most clinically useful information for directly answering clinicians' information needs.

MATERIALS AND METHODS

We developed a Kernel-based Bayesian Network classification model based on different domain-specific feature types extracted from sentences in a gold standard composed of 18 UpToDate documents. These features included UMLS concepts and their semantic groups, semantic predications extracted by SemRep, patient population identified by a pattern-based natural language processing (NLP) algorithm, and cue words extracted by a feature selection technique. Algorithm performance was measured in terms of precision, recall, and F-measure.

RESULTS

The feature-rich approach yielded an F-measure of 74% versus 37% for a feature co-occurrence method (p<0.001). Excluding predication, population, semantic concept or text-based features reduced the F-measure to 62%, 66%, 58% and 69% respectively (p<0.01). The classifier applied to Medline sentences reached an F-measure of 73%, which is equivalent to the performance of the classifier on UpToDate sentences (p=0.62).

CONCLUSIONS

The feature-rich approach significantly outperformed general baseline methods. This approach significantly outperformed classifiers based on a single type of feature. Different types of semantic features provided a unique contribution to overall classification performance. The classifier's model and features used for UpToDate generalized well to Medline abstracts.

Robust ultrasensitive tunneling-FET biosensor for point-of-care diagnostics

For point-of-care (POC) applications, robust, ultrasensitive, small, rapid, low-power, and low-cost sensors are highly desirable. Here, we present a novel biosensor based on a complementary metal oxide semiconductor (CMOS)-compatible silicon nanowire tunneling field-effect transistor (SiNW-TFET). They were fabricated “top-down” with a low-cost anisotropic self-stop etching technique. Notably, the SiNW-TFET device provided strong anti-interference capacity by applying the inherent ambipolarity via both pH and CYFRA21-1 sensing. This offered a more robust and portable general protocol. The specific label-free detection of CYFRA21-1 down to 0.5 fgml⁻¹ or ~12.5 aM was achieved using a highly responsive SiNW-TFET device with a minimum sub-threshold slope (SS) of 37 mVdec⁻¹. Furthermore, real-time measurements highlighted the ability to use clinically relevant samples such as serum. The developed high performance diagnostic system is expected to provide a generic platform for numerous POC applications.

What is a medical decision? A taxonomy based on physician statements in hospital encounters: a qualitative study

OBJECTIVE

The medical literature lacks a comprehensive taxonomy of decisions made by physicians in medical encounters. Such a taxonomy might be useful in understanding the physician-centred, patient-centred and shared decision-making in clinical settings. We aimed to identify and classify all decisions emerging in conversations between patients and physicians.

DESIGN

Qualitative study of video recorded patient-physician encounters.

PARTICIPANTS AND SETTING

380 patients in consultations with 59 physicians from 17 clinical specialties and three different settings (emergency room, ward round, outpatient clinic) in a Norwegian teaching hospital. A randomised sample of 30 encounters from internal medicine was used to identify and classify decisions, a maximum variation sample of 20 encounters was used for reliability assessments, and the remaining encounters were analysed to test for applicability across specialties.

RESULTS

On the basis of physician statements in our material, we developed a taxonomy of clinical decisions--the Decision Identification and Classification Taxonomy for Use in Medicine (DICTUM). We categorised decisions into 10 mutually exclusive categories: gathering additional information, evaluating test results, defining problem, drug-related, therapeutic procedure-related, legal and insurance-related, contact-related, advice and precaution, treatment goal, and deferment. Four-coder inter-rater reliability using Krippendorff's α was 0.79.

CONCLUSIONS

DICTUM represents a precise, detailed and comprehensive taxonomy of medical decisions communicated within patient-physician encounters. Compared to previous normative frameworks, the taxonomy is descriptive, substantially broader and offers new categories to the variety of clinical decisions. The taxonomy could prove helpful in studies on the quality of medical work, use of time and resources, and understanding of why, when and how patients are or are not involved in decisions.

Usability and Acceptance of the Librarian Infobutton Tailoring Environment: An Open Access Online Knowledge Capture, Management, and Configuration Tool for OpenInfobutton

BACKGROUND

The Librarian Infobutton Tailoring Environment (LITE) is a Web-based knowledge capture, management, and configuration tool with which users can build profiles used by OpenInfobutton, an open source infobutton manager, to provide electronic health record users with context-relevant links to online knowledge resources.

OBJECTIVE

We conducted a multipart evaluation study to explore users' attitudes and acceptance of LITE and to guide future development.

METHODS

The evaluation consisted of an initial online survey to all LITE users, followed by an observational study of a subset of users in which evaluators' sessions were recorded while they conducted assigned tasks. The observational study was followed by administration of a modified System Usability Scale (SUS) survey.

RESULTS

Fourteen users responded to the survey and indicated good acceptance of LITE with feedback that was mostly positive. Six users participated in the observational study, demonstrating average task completion time of less than 6 minutes and an average SUS score of 72, which is considered good compared with other SUS scores.

CONCLUSIONS

LITE can be used to fulfill its designated tasks quickly and successfully. Evaluators proposed suggestions for improvements in LITE functionality and user interface.

Formative evaluation of a patient-specific clinical knowledge summarization tool

OBJECTIVE

To iteratively design a prototype of a computerized clinical knowledge summarization (CKS) tool aimed at helping clinicians finding answers to their clinical questions; and to conduct a formative assessment of the usability, usefulness, efficiency, and impact of the CKS prototype on physicians' perceived decision quality compared with standard search of UpToDate and PubMed.

MATERIALS AND METHODS

Mixed-methods observations of the interactions of 10 physicians with the CKS prototype vs. standard search in an effort to solve clinical problems posed as case vignettes.

RESULTS

The CKS tool automatically summarizes patient-specific and actionable clinical recommendations from PubMed (high quality randomized controlled trials and systematic reviews) and UpToDate. Two thirds of the study participants completed 15 out of 17 usability tasks. The median time to task completion was less than 10s for 12 of the 17 tasks. The difference in search time between the CKS and standard search was not significant (median=4.9 vs. 4.5m in). Physician's perceived decision quality was significantly higher with the CKS than with manual search (mean=16.6 vs. 14.4; p=0.036).

CONCLUSIONS

The CKS prototype was well-accepted by physicians both in terms of usability and usefulness. Physicians perceived better decision quality with the CKS prototype compared to standard search of PubMed and UpToDate within a similar search time. Due to the formative nature of this study and a small sample size, conclusions regarding efficiency and efficacy are exploratory.

Therapeutic enquiries about biological agents as a tool to identify safety aspects and patterns of use

Background

Biotechnological agents (BA) are increasingly being used in clinical practice. We aimed to determine, whether enquiries about them to a therapeutic consultation service have also become more frequent, and to describe the information requested in these consultations.

Methods

We retrospectively reviewed 14 104 therapeutic consultations collected in a computerised database between 2000 and 2014. Enquiries about BA (monoclonal antibodies, fusion proteins or cytokine antagonists) were chosen. Information on the type of BA, underlying condition, type of enquiry and affiliation of the enquirer was retrieved and compared with data from consultations about other agents.

Results

During the study period, 365 enquiries about 30 different BA were received. Only 4% of them were received before 2004, while 48.8% were received after 2010. Rituximab, infliximab, adalimumab and etanercept were most frequently enquired about. Agent selection (n=184) and/or adverse effects (n=174) were the most frequent reasons for making an enquiry. Most enquiries about an agent selection were made about an off-label use (n=164), mainly for systemic autoimmune diseases (n=61). Over half of the enquiries about adverse effects were about their teratogenic potential (n=96). Enquiries about BA more often requested an opinion (87.7% vs 77.7%) were made by physicians (89.9% vs 76.9%), from a hospital (81.6% vs 44.5%) and regarded a specific patient (87.4% vs 74.5%).

Conclusions

Therapeutic consultations about BA are increasing. Most of them are related to uncertainties of health professionals regarding any new medicine: their off-label use, actual adverse effects or the teratogenic potential of the involved agents.

GP trainees' in-consultation information-seeking: associations with human, paper and electronic sources

BACKGROUND

Answering clinical questions arising from patient care can improve that care and offers an opportunity for adult learning. It is also a vital component in practising evidence-based medicine. GPs' sources of in-consultation information can be human or non-human (either hard copy or electronic).

OBJECTIVES

To establish the prevalence and associations of GP trainees' in-consultation information-seeking, and to establish the prevalence of use of different sources of information (human, hard copy and electronic) and the associations of choosing particular sources.

METHODS

A cross-sectional analysis of data (2010-13) from an ongoing cohort study of Australian GP trainees' consultations. Once each 6-month training term, trainees record detailed data of 60 consecutive consultations. The primary outcome was whether the trainee sought in-consultation information for a problem/diagnosis. Secondary outcomes were whether information-seeking was from a human (GP, other specialist or other health professional) or from a non-human source (electronic or hard copy), and whether a non-human source was electronic or hard copy.

RESULTS

Six hundred forty-five trainees (response rate 94.3%) contributed data for 84,723 consultations including 131,583 problems/diagnoses. In-consultation information was sought for 15.4% (95% confidence interval=15.3-15.6) of problems/diagnoses. Sources were: GP in 6.9% of problems/diagnoses, other specialists 0.9%, other health professionals 0.6%, electronic sources 6.5% and hard-copy sources 1.5%. Associations of information-seeking included younger patient age, trainee full-time status and earlier training stage, longer consultation duration, referring the patient, organizing follow-up and generating learning goals. Associations of choosing human information sources (over non-human sources) were similar, but also included the trainee's training organization. Associations of electronic rather than hard-copy information-seeking included the trainee being younger, the training organization and information-seeking for management rather than diagnosis.

CONCLUSION

Trainee information-seeking is mainly from GP colleagues and electronic sources. Human information-sources are preferentially sought for more complex problems, even by these early-career GPs who have trained in the 'internet era'.

Automatically finding relevant citations for clinical guideline development

OBJECTIVE

Literature database search is a crucial step in the development of clinical practice guidelines and systematic reviews. In the age of information technology, the process of literature search is still conducted manually, therefore it is costly, slow and subject to human errors. In this research, we sought to improve the traditional search approach using innovative query expansion and citation ranking approaches.

METHODS

We developed a citation retrieval system composed of query expansion and citation ranking methods. The methods are unsupervised and easily integrated over the PubMed search engine. To validate the system, we developed a gold standard consisting of citations that were systematically searched and screened to support the development of cardiovascular clinical practice guidelines. The expansion and ranking methods were evaluated separately and compared with baseline approaches.

RESULTS

Compared with the baseline PubMed expansion, the query expansion algorithm improved recall (80.2% vs. 51.5%) with small loss on precision (0.4% vs. 0.6%). The algorithm could find all citations used to support a larger number of guideline recommendations than the baseline approach (64.5% vs. 37.2%, p<0.001). In addition, the citation ranking approach performed better than PubMed's "most recent" ranking (average precision +6.5%, recall@k +21.1%, p<0.001), PubMed's rank by "relevance" (average precision +6.1%, recall@k +14.8%, p<0.001), and the machine learning classifier that identifies scientifically sound studies from MEDLINE citations (average precision +4.9%, recall@k +4.2%, p<0.001).

CONCLUSIONS

Our unsupervised query expansion and ranking techniques are more flexible and effective than PubMed's default search engine behavior and the machine learning classifier. Automated citation finding is promising to augment the traditional literature search.

Moving From Digitalization to Digitization in Cardiovascular Care: Why Is it Important, and What Could it Mean for Patients and Providers?

So far, the digitization of health care is best exemplified by electronic medical records, which have been far from favorably or uniformly accepted. However, properly implemented digitization can enable better patient outcomes, improve convenience, potentially lower healthcare costs, and possibly lead to much greater physician satisfaction. Precision (also known as personalized or individualized) medicine is frequently discussed today, but, in reality, it is what physicians have attempted to do as best they could for millennia. But now we have new tools that can begin to give us a much more high-definition view of our patients; from affordable and rapid genetic testing to wearable sensors that track a wide range of important physiologic parameters continuously. Although seemingly counterintuitive, the digitization of health care can also markedly improve the physician-patient relationship, allowing more time for human interaction when care is bolstered by digital technologies that better individualize diagnostics and treatments.

New Mexico practitioners' access to and satisfaction with online clinical information resources: an interview study using qualitative data analysis software

QUESTIONS

What information resources are available to health care practitioners not affiliated with the University of New Mexico? How satisfied are they with those resources?

SETTING

The state is rural and medically underserved.

METHODS

The authors interviewed practitioners, using a nine-item guide. Interview transcripts were coded using QSR NVivo 9 software.

MAIN RESULTS

Fifty-one practitioners were interviewed. Most use online information resources. Many have access to a point-of-care resource within an electronic health records system. They often expressed dissatisfaction with available patient education resources.

CONCLUSION

New Mexico practitioners routinely use electronic information resources but indicate they need better patient information.

eHealth 2015 Special Issue: Impact of Electronic Health Records on the Completeness of Clinical Documentation Generated during Diabetic Retinopathy Consultations

BACKGROUND

Two years ago, the Diabetic Retinopathy (DRP) and Traumatology clinic of the Department of Ophthalmology and Optometrics at the Medical University of Vienna, Austria switched from paper-based to electronic health records. A customized electronic health record system (EHR-S) was implemented.

OBJECTIVES

To assess the completeness of information documented electronically compared with manually during patient visits.

METHODS

The Preferred Practice Pattern for Diabetic Retinopathy published by the American Academy of Ophthalmology was distilled into a list of medical features grouped into categories to be assessed and documented during the management of patients with DRP. The last seventy paper-based records and all electronic records generated since the switch were analyzed and graded for the presence of features on the list and the resulting scores compared.

RESULTS

In all categories, clinical documentation was more complete in the EHR group.

CONCLUSIONS

In our setting, the implementation of an EHR-S showed a statistically significant positive impact on documentation completeness.

[Analysis of the question-answer activity of a hospital pharmacy. Example of the handling of drug interactions]

INTRODUCTION

The medical care of patients generates questions among healthcare professionals. Some will necessitate an advanced research. The hospital pharmacist is at the interface between prescribers, caregivers and the medicines and is requested to answer these requests. Studies conducted in other countries showed that this question-answer activity represents a significant amount of time in daily work. In France, this topic was poorly explored. The objective of our work was to study the volume and the type of questions, the clinical situations, the time required, the medicines implicated and the sources of information used.

MATERIALS AND METHODS

A prospective study was conducted in the pharmacy of a university hospital. All the requests answered by the pharmaceutical team, which needed a specific research, analysis and writing of an answer were collected.

RESULTS

A hundred and one questions were analyzed, originating from doctors or medicals interns. Almost half concerned drug interactions, and among them, almost a fourth were not mentioned in the Summary of Product Characteristics of the medicines involved. A pharmaceutical advice was provided in 91.5% of the cases. Time dedicated to the research varied between less than 30 minutes and more than 8 hours.

DISCUSSION AND CONCLUSION

This study illustrates the question-answer activity of a hospital pharmacy, which is currently not taken into account as an indicator of pharmaceutical activity. A large part concerns analysis and management of drug interactions and requires a significant amount of pharmaceutical time.

A model to support shared decision making in electronic health records systems

Shared decision making (SDM) is an approach to medical care based on collaboration between provider and patient, with both sharing in medical decisions. When patients' values and preferences are incorporated in decision making, care is more appropriate, ethically sound, and often lower in cost. However, SDM is difficult to implement in routine practice because of the time required for SDM methods, the lack of integration of SDM approaches into electronic health record (EHR) systems, and absence of explanatory mechanisms for providers on the results of patients' use of decision aids. This article discusses potential solutions, including the concept of a "personalize button" for EHRs. Leveraging a 4-phase clinical model for SDM, this article describes how computer decision support (CDS) technologies integrated into EHRs can help ensure that health care is delivered in a way that is respectful of those preferences. The architecture described herein, called CDS for SDM, is built on recognized standards that are currently integrated into certification requirements for EHRs as part of meaningful use regulations. While additional work is needed on modeling of preferences and on techniques for rapid communication models of preferences to clinicians, unless EHRs are redesigned to support SDM around and during clinical encounters, they are likely to continue to be an unintended barrier to SDM. With appropriate development, EHRs could be a powerful tool to promote SDM by reminding providers of situations for SDM and monitoring ongoing care to ensure treatments are consistent with patients' preferences.

Clinical questions raised by providers in the care of older adults: a prospective observational study

OBJECTIVE

To characterise clinical questions raised by providers in the care of complex older adults in order to guide the design of interventions that can help providers answer these questions.

MATERIALS AND METHODS

To elicit clinical questions, we observed and audio recorded outpatient visits at three healthcare organisations. At the end of each appointment, providers were asked to identify clinical questions raised in the visit. Providers rated their questions based on their urgency, importance to the patient's care and difficulty in finding a useful answer to. Transcripts of the audio recordings were analysed to identify ageing-specific factors that may have contributed to the nature of the questions.

RESULTS

We observed 36 patient visits with 10 providers at the three study sites. Providers raised 70 clinical questions (median of 2 clinical questions per patient seen; range 0-12), pursued 50 (71%) and successfully answered 34 (68%) of the questions they pursued. Overall, 36 (51%) of providers' questions were not answered. Over one-third of the questions were about treatment alternatives and adverse effects. All but two clinical questions were motivated either directly or indirectly by issues related to ageing, such as the normal physiological changes of ageing and diseases with higher prevalence in the elderly.

CONCLUSIONS

The frequency of clinical questions was higher than in previous studies conducted in general primary care patient populations. Clinical questions were predominantly influenced by ageing-related issues. We propose a series of recommendations that may be used to guide the design of solutions to help providers answer their clinical questions in the care of older adults.