A randomized effectiveness trial of a clinical informatics consult service: impact on evidence-based decision-making and knowledge implementation

Evaluating a Large Language Model's Ability to Answer Clinicians' Requests for Evidence Summaries

Objective

This study investigated the performance of a generative artificial intelligence (AI) tool using GPT-4 in answering clinical questions in comparison with medical librarians' gold-standard evidence syntheses.

Methods

Questions were extracted from an in-house database of clinical evidence requests previously answered by medical librarians. Questions with multiple parts were subdivided into individual topics. A standardized prompt was developed using the COSTAR framework. Librarians submitted each question into aiChat, an internally-managed chat tool using GPT-4, and recorded the responses. The summaries generated by aiChat were evaluated on whether they contained the critical elements used in the established gold-standard summary of the librarian. A subset of questions was randomly selected for verification of references provided by aiChat.

Results

Of the 216 evaluated questions, aiChat's response was assessed as "correct" for 180 (83.3%) questions, "partially correct" for 35 (16.2%) questions, and "incorrect" for 1 (0.5%) question. No significant differences were observed in question ratings by question category (p=0.39). For a subset of 30% (n=66) of questions, 162 references were provided in the aiChat summaries, and 60 (37%) were confirmed as nonfabricated.

Conclusions

Overall, the performance of a generative AI tool was promising. However, many included references could not be independently verified, and attempts were not made to assess whether any additional concepts introduced by aiChat were factually accurate. Thus, we envision this being the first of a series of investigations designed to further our understanding of how current and future versions of generative AI can be used and integrated into medical librarians' workflow.

Exploring interprofessional collaboration and attitudes of health sciences librarians

Objective:

This study assessed health sciences librarians' attitudes toward interprofessional collaboration using the Interdisciplinary Education Perception Scale (IEPS) and gathered information on their involvement with interprofessional activities.

Methods:

The authors sent a survey to librarians in the Medical Library Association's (MLA's) Interprofessional Education Special Interest Group and Research Section consisting of the IEPS and questions about their prior and current experiences with interprofessional practice and education (IPE). We compared mean IEPS scores between each MLA group and several other demographic factors to assess differences in attitudes. We also compared librarians' IEPS scores with those of previously published health professional students' IEPS scores and thematically analyzed two open-ended questions.

Results:

Health sciences librarians' scores on the IEPS indicated positive attitudes toward IPE. There were no statistically significant differences between any group. Health sciences librarians' mean IEPS score was similar to the mean score of health professions students from a prior study. The most commonly reported interprofessional activity was teaching or facilitating learning activities for health professions students; fewer served on committees or engaged in non-curricular activities such as grand rounds and book clubs.

Conclusion:

Health sciences librarians in this study reported positive attitudes toward IPE, in line with the majority of other previously studied health professionals. Years of experience, previous health professional careers, and experience supporting IPE as a librarian had little bearing on the responses to the survey. This suggests that health sciences librarians have positive attitudes toward IPE, regardless of whether they directly support IPE programs or participate in interprofessional activities.

Why equating all evidence searches to systematic reviews defies their role in information seeking

All too often the quality and rigor of topic investigations is inaccurately conveyed to information professionals, resulting in a mischaracterization of the research, which, if left unchecked and published, may in turn mislead potential readers. Accurately understanding and categorizing the types of topic investigation searches that are requested of information professionals is critical to both meeting requestors’ needs and reflecting their intended methodological approaches. Information professionals’ expertise can be an invaluable resource to guide users through the investigative and publication process.

The new roles of medical librarians in medical research

Purpose

This study aims to highlight the role of librarians as an essential element in medical research. For this purpose, the primary research process was divided into three phases: before, during and after. Then, the roles of librarians associated with each phase were separated and the viewpoint of researchers and librarians on the importance of these roles were considered and compared.

Design/methodology/approach

This comparative, descriptive-causal research was conducted using the census method. Birjand University, a type-2 university in the field of Medical Sciences according to the rating of the Iranian Ministry of Health and Medical Education, was selected for the study. The participants were all faculty members and all librarians working in the university’s libraries. The data collection tool was a questionnaire made by authors. Its validity was confirmed by four professors of Library Science, and questionnaire design expert and its reliability was confirmed with a Cronbach's alpha of 0.89. Data were analyzed with the Statistical Package for the Social Sciences, using descriptive statistics.

Findings

From the perspective of researchers, awareness-raising for open access resources, determination of standard subject keywords on the basis of Medical Subject Headings for articles and scientific texts and using scientific research findings as a basis for preventing duplicate studies in future research are the most important roles for librarians in the three stages of medical research. From the perspective of librarians, the use of knowledge management skills, searching scientific information as review of the literature and also selecting standard keywords to search the databases and providing health-care professionals with the findings of latest scientific research have the highest place in the different stages of the research lifecycle.

Originality/value

The difference between the viewpoints of librarians and researchers about the role of medical librarians at the various stages of the research lifecycle shows that there are significant gaps between the librarians’ services and users’ expectations. It is expected that through learning modern professional skills, medical librarians can assume new roles in medical research and make their capabilities known and available to researchers.

Evaluating the impact of clinical librarians on clinical questions during inpatient rounds

Objective

The investigation sought to determine the effects of a clinical librarian (CL) on inpatient team clinical questioning quality and quantity, learner self-reported literature searching skills, and use of evidence-based medicine (EBM).

Methods

Clinical questioning was observed over 50 days of inpatient pediatric and internal medicine attending rounds. A CL was present for 25 days and absent for 25 days. Questioning was compared between groups. Question quality was assessed by a blinded evaluator, who used a rubric adapted from the Fresno Test of Competence in Evidence-Based Medicine. Team members were surveyed to assess perceived impacts of the CL on rounds.

Results

Rounds with a CL (CLR) were associated with significantly increased median number of questions asked (5 questions CLR vs. 3 NCLR; p<0.01) and answered (3 CLR vs. 2 NCLR; p<0.01) compared to rounds without a CL (NCLR). CLR were also associated with increased mean time spent asking (1.39 minutes CLR vs. 0.52 NCLR; p<0.01) and answering (2.15 minutes CLR vs. 1.05 NCLR; p=0.02) questions. Rounding time per patient was not significantly different between CLR and NCLR. Questions during CLR were 2 times higher in adapted Fresno Test quality than during NCLR (p<0.01). Select participants described how the CL's presence improved their EBM skills and care decisions.

Conclusions

Inpatient CLR were associated with more and improved clinical questioning and subjectively perceived to improve clinicians' EBM skills. CLs may directly affect patient care; further study is required to assess this. CLs on inpatient rounds may be an effective means for clinicians to learn and use EBM skills.

Answering medical questions at the point of care: a cross-sectional study comparing rapid decisions based on PubMed and Epistemonikos searches with evidence-based recommendations developed with the GRADE approach

INTRODUCTION

Using the best current evidence to inform clinical decisions remains a challenge for clinicians. Given the scarcity of trustworthy clinical practice guidelines providing recommendations to answer clinicians' daily questions, clinical decision support systems (ie, assistance in question identification and answering) emerge as an attractive alternative. The trustworthiness of the recommendations achieved by such systems is unknown.

OBJECTIVE

To evaluate the trustworthiness of a question identification and answering system that delivers timely recommendations.

DESIGN

Cross-sectional study.

METHODS

We compared the responses to 100 clinical questions related to inpatient management provided by two rapid response methods with 'Gold Standard' recommendations. One of the rapid methods was based on PubMed and the other on Epistemonikos database. We defined our 'Gold Standard' as trustworthy published evidence-based recommendations or, when unavailable, recommendations developed locally by a panel of six clinicians following the Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach. Recommendations provided by the rapid strategies were classified as potentially misleading or reasonable. We also determined if the potentially misleading recommendations could have been avoided with the appropriate implementation of searching and evidence summary tools.

RESULTS

We were able to answer all of the 100 questions with both rapid methods. Of the 200 recommendations obtained, 6.5% (95% CI 3% to 9.9%) were classified as potentially misleading and 93.5% (95% CI 90% to 96.9%) as reasonable. 6 of the 13 potentially misleading recommendations could have been avoided by the appropriate usage of the Epistemonikos matrix tool or by constructing summary of findings tables. No significant differences were observed between the evaluated rapid response methods.

CONCLUSION

A question answering service based on the GRADE approach proved feasible to implement and provided appropriate guidance for most identified questions. Our approach could help stakeholders in charge of managing resources and defining policies for patient care to improve evidence-based decision-making in an efficient and feasible manner.

Testing a videogame intervention to recalibrate physician heuristics in trauma triage: study protocol for a randomized controlled trial

Background

Between 30 and 40 % of patients with severe injuries receive treatment at non-trauma centers (under-triage), largely because of physician decision making. Existing interventions to improve triage by physicians ignore the role that intuition (heuristics) plays in these decisions. One such heuristic is to form an initial impression based on representativeness (how typical does a patient appear of one with severe injuries). We created a video game (Night Shift) to recalibrate physician’s representativeness heuristic in trauma triage.

Methods

We developed Night Shift in collaboration with emergency medicine physicians, trauma surgeons, behavioral scientists, and game designers. Players take on the persona of Andy Jordan, an emergency medicine physician, who accepts a new job in a small town. Through a series of cases that go awry, they gain experience with the contextual cues that distinguish patients with minor and severe injuries (based on the theory of analogical encoding) and receive emotionally-laden feedback on their performance (based on the theory of narrative engagement). The planned study will compare the effect of Night Shift with that of an educational program on physician triage decisions and on physician heuristics. Psychological theory predicts that cognitive load increases reliance on heuristics, thereby increasing the under-triage rate when heuristics are poorly calibrated. We will randomize physicians (n = 366) either to play the game or to review an educational program, and will assess performance using a validated virtual simulation. The validated simulation includes both control and cognitive load conditions. We will compare rates of under-triage after exposure to the two interventions (primary outcome) and will compare the effect of cognitive load on physicians’ under-triage rates (secondary outcome). We hypothesize that: a) physicians exposed to Night Shift will have lower rates of under-triage compared to those exposed to the educational program, and b) cognitive load will not degrade triage performance among physicians exposed to Night Shift as much as it will among those exposed to the educational program.

Discussion

Serious games offer a new approach to the problem of poorly-calibrated heuristics in trauma triage. The results of this trial will contribute to the understanding of physician quality improvement and the efficacy of video games as behavioral interventions.

Trial registration

clinicaltrials.gov; NCT02857348; August 2, 2016.

Assessing the National Library of Medicine's Informationist Awards

OBJECTIVE

To understand the experience of the informationist recipients of NLM-funded Administrative Supplements for Informationist Services and gather evidence for their impact on NIH-funded biomedical research.

METHODS

A mixed methods approach consisting of a survey of principal investigators and a focus group of informationists.

RESULTS

Informationists appeared to have a positive impact on their team's research, especially in the areas of data storage, data management planning, data organization, and literature searching. In addition, many informationists felt that their involvement had increased their research skills and made them true research partners. Assessing their own impact was a challenge for the award recipients, and questions remain about the best evaluation methods. The overall experience of the informationists and researchers was mixed but largely positive.

CONCLUSION

The NLM-funded informationist supplement award appears to be a successful mechanism for immersing informationists into research teams and improving data management in the supported projects.

Impact of a GRADE-based medical question answering system on physician behaviour: a randomised controlled trial

Physicians are frequently faced with questions related to their patients’ care that they cannot answer. A vast number of randomised trials have tested a wide variety of behaviour-changing strategies designed to improve practitioners’ evidence utilisation, but systematic reviews have concluded that the effects are generally small and inconsistent. We conducted a randomised controlled trial to determine whether a question identification and solving system, using structured evidence summaries with recommendations, would change physician’s behavior related to the care of their hospitalised patients. The trial was conducted at the secondary level, internal medicine ward. Relevant clinical questions were the units of randomisation; 14 clinicians participated in the study. The question identification and answering system was carried out using evidence summaries with recommendations based on the Grades of Recommendation, Assessment, Development and Evaluation (GRADE) approach stressing influence on clinician behaviour (decision/recommendation concordance). During 131 morning reports, 553 questions were identified (4.2 questions per meeting). 398 were excluded because they were not about diagnostic or therapeutic interventions or because their answers could not have impact on clinician behaviour, and 31 were excluded because of lack of time to answer them, leaving 124 included questions. The proportion of clinical decisions concordant with the proposed recommendations was 79%in the intervention arm and 44% in the control arm: relative risk 1.8 (95% CI 1.3 to 2.4), number of evidence summaries needed to change a care decision for one question raised was 3 (95% CI 2 to 6). A question identification and answering system was feasible, effectively performed and significantly influenced clinician behaviour related to the care of hospitalised patients, which suggests that interventions facilitating accessibility and interpretability of the best available evidence at the point of care have the potential to significantly impact on the quality of healthcare.

Effects of librarian-provided services in healthcare settings: a systematic review

OBJECTIVE

To assess the effects of librarian-provided services in healthcare settings on patient, healthcare provider, and researcher outcomes.

MATERIALS AND METHODS

Medline, CINAHL, ERIC, LISA (Library and Information Science Abstracts), and the Cochrane Central Register of Controlled Trials were searched from inception to June 2013. Studies involving librarian-provided services for patients encountering the healthcare system, healthcare providers, or researchers were eligible for inclusion. All librarian-provided services in healthcare settings were considered as an intervention, including hospitals, primary care settings, or public health clinics.

RESULTS

Twenty-five articles fulfilled our eligibility criteria, including 22 primary publications and three companion reports. The majority of studies (15/22 primary publications) examined librarians providing instruction in literature searching to healthcare trainees, and measured literature searching proficiency. Other studies analyzed librarian-provided literature searching services and instruction in question formulation as well as the impact of librarian-provided services on patient length of stay in hospital. No studies were found that investigated librarians providing direct services to researchers or patients in healthcare settings.

CONCLUSIONS

Librarian-provided services directed to participants in training programs (eg, students, residents) improve skills in searching the literature to facilitate the integration of research evidence into clinical decision-making. Services provided to clinicians were shown to be effective in saving time for health professionals and providing relevant information for decision-making. Two studies indicated patient length of stay was reduced when clinicians requested literature searches related to a patient's case.

The effect of a clinical medical librarian on in-patient care outcomes

OBJECTIVE

The research sought to determine the effect of a clinical medical librarian (CML) on outcomes of in-patients on the internal medicine service.

METHODS

A prospective study was performed with two internal medicine in-patient teams. Team 1 included a CML who accompanied the team on daily rounds. The CML answered questions posed at the point of care immediately or in emails post-rounds. Patients on Team 2, which did not include a CML, as well as patients who did not require consultation by the CML on Team 1, served as the control population. Numerous clinical and library metrics were gathered on each question.

RESULTS

Patients on Team 1 who required an answer to a clinical question were more ill and had a longer length of stay, higher costs, and higher readmission rates compared to those in the control group. Using a matched pair analysis, we showed no difference in clinical outcomes between the intervention group and the control group.

CONCLUSIONS

This study is the largest attempt to prospectively measure changes in patient outcomes when physicians were accompanied by a CML on rounds. This approach may serve as a model for further studies to define when and how CMLs are most effective.

Strategically aligning a mandala of competencies to advance a transformative vision

OBJECTIVE

This paper offers insight into the processes that have shaped the Eskind Biomedical Library's (EBL's) strategic direction and its alignment to the institution's transformative vision.

SETTING

The academic biomedical library has a notable track record for developing and pioneering roles for information professionals focused on a sophisticated level of information provision that draws from and fuels practice evolutions.

STRATEGY

The medical center's overall transformative vision informs the creation of a fully aligned library strategic plan designed to effectively contribute to the execution of key organizational goals. Annual goals reflect organizational priorities and contain quantifiable and measurable deliverables. Two strategic themes, facilitating genetic literacy and preserving community history, are described in detail to illustrate the concept of goal setting.

CONCLUSION

The strategic planning model reflects EBL's adaptation to the ever-changing needs of its organization. The paper provides a characterization of a workable model that can be replicated by other institutions.

Implementation of evidence-based practice across medical, nursing, pharmacological and allied healthcare professionals: a questionnaire survey in nationwide hospital settings

Background

Implementation of evidence-based practice (EBP) is regarded as core competence to improve healthcare quality. In the current study, we investigated the EBP of six groups of professionals: physicians, nurses, pharmacists, physical therapists, technicians, and other allied healthcare personnel.

Methods

A structured questionnaire survey of regional hospitals throughout Taiwan was conducted by post in 2011. Questionnaires were mailed to all healthcare workers of 11 randomly selected hospitals. Linear and logistic regression models were used to examine predictors for implementing EBP.

Results

In total, 6,160 returned questionnaires, including 645 from physicians, 4,206 from nurses, 430 from pharmacists, 179 from physical therapists, 537 from technicians, and 163 from other allied healthcare professionals, were valid for the analysis. Physicians and pharmacists were more aware of EBP than were the other professional groups (p < 0.001). Positive attitudes toward and beliefs in EBP were significantly lower among nurses than in the other groups (p < 0.001). Physicians had more sufficient knowledge and skills of EBP than did the other professionals (p < 0.001); in addition, they implemented EBP for clinical decision-making more often and perceived fewer personal barriers to EBP (p < 0.001). Multivariate logistic regression analyses showed that EBP implementation was associated with the following characteristics of participants: EBP training, having a faculty position, academic degree, one's profession, and perceptions (beliefs, attitudes, knowledge, skills and barriers).

Conclusions

This study depicts various levels of EBP implementation among medical, nursing, pharmacological, and allied healthcare personnel. There were significant differences in their implementation of EBP. We observed that certain factors were associated with EBP implementation, including personal backgrounds and perceptions toward EBP. The data suggest that strategies for enhancing EBP implementation should differ for various groups of professionals.

Expert searcher, teacher, content manager, and patient advocate: an exploratory study of clinical librarian roles

OBJECTIVE

The research explored the roles of practicing clinical librarians embedded in a patient care team.

METHODS

Six clinical librarians from Canada and one from the United States were interviewed to elicit detailed descriptions of their clinical roles and responsibilities and the context in which these were performed.

RESULTS

Participants were embedded in a wide range of clinical service areas, working with a diverse complement of health professionals. As clinical librarians, participants wore many hats, including expert searcher, teacher, content manager, and patient advocate. Unique aspects of how these roles played out included a sense of urgency surrounding searching activities, the broad dissemination of responses to clinical questions, and leverage of the roles of expert searcher, teacher, and content manager to advocate for patients.

CONCLUSIONS

Detailed role descriptions of clinical librarians embedded in patient care teams suggest possible new practices for existing clinical librarians, provide direction for training new librarians working in patient care environments, and raise awareness of the clinical librarian specialty among current and budding health information professionals.

The effect of a clinical medical librarian on in-patient care outcomes

OBJECTIVE

The research sought to determine the effect of a clinical medical librarian (CML) on outcomes of in-patients on the internal medicine service.

METHODS

A prospective study was performed with two internal medicine in-patient teams. Team 1 included a CML who accompanied the team on daily rounds. The CML answered questions posed at the point of care immediately or in emails post-rounds. Patients on Team 2, which did not include a CML, as well as patients who did not require consultation by the CML on Team 1, served as the control population. Numerous clinical and library metrics were gathered on each question.

RESULTS

Patients on Team 1 who required an answer to a clinical question were more ill and had a longer length of stay, higher costs, and higher readmission rates compared to those in the control group. Using a matched pair analysis, we showed no difference in clinical outcomes between the intervention group and the control group.

CONCLUSIONS

This study is the largest attempt to prospectively measure changes in patient outcomes when physicians were accompanied by a CML on rounds. This approach may serve as a model for further studies to define when and how CMLs are most effective.

Evidence-based medicine at the intersection of research interests between academic health sciences librarians and medical educators: a review of the literature

OBJECTIVES

In 2008, the Association of Academic Health Sciences Libraries established an Education Research Task Force (ERTF) to plan research addressing research priorities outlined in key Association of American Medical Colleges reports. ERTF members conducted a literature review to describe the state of collaborative research at the intersection of medical education and health sciences librarianship. Analysis of initial results revealed instruction in evidence-based medicine (EBM) was a shared interest and is thus the focus of this review.

METHODS

Searches on EBM teaching programs were conducted, and results were posted to a shared online citation management service. Individual articles were assessed and assigned metadata describing subject matter, scope, and format.

RESULTS

Article analysis identified key themes. Most papers were descriptive narratives of curricular development. Evaluation studies were also prominent and often based on student satisfaction or self-reported competency. A smaller number of controlled studies provide evidence of impacts of librarian involvement in EBM instruction.

CONCLUSIONS

Scholarship of EBM instruction is of common interest between medical educators and health sciences librarians. Coauthorship between the groups and distribution of literature points to a productive collaboration. An emerging literature of controlled studies measuring the impact of cross-disciplinary efforts signals continued progress in the arena of EBM instruction.

Standards for reporting randomized controlled trials in medical informatics: a systematic review of CONSORT adherence in RCTs on clinical decision support

INTRODUCTION

The Consolidated Standards for Reporting Trials (CONSORT) were published to standardize reporting and improve the quality of clinical trials. The objective of this study is to assess CONSORT adherence in randomized clinical trials (RCT) of disease specific clinical decision support (CDS).

METHODS

A systematic search was conducted of the Medline, EMBASE, and Cochrane databases. RCTs on CDS were assessed against CONSORT guidelines and the Jadad score.

RESULT

32 of 3784 papers identified in the primary search were included in the final review. 181 702 patients and 7315 physicians participated in the selected trials. Most trials were performed in primary care (22), including 897 general practitioner offices. RCTs assessing CDS for asthma (4), diabetes (4), and hyperlipidemia (3) were the most common. Thirteen CDS systems (40%) were implemented in electronic medical records, and 14 (43%) provided automatic alerts. CONSORT and Jadad scores were generally low; the mean CONSORT score was 30.75 (95% CI 27.0 to 34.5), median score 32, range 21-38. Fourteen trials (43%) did not clearly define the study objective, and 11 studies (34%) did not include a sample size calculation. Outcome measures were adequately identified and defined in 23 (71%) trials; adverse events or side effects were not reported in 20 trials (62%). Thirteen trials (40%) were of superior quality according to the Jadad score (≥3 points). Six trials (18%) reported on long-term implementation of CDS.

CONCLUSION

The overall quality of reporting RCTs was low. There is a need to develop standards for reporting RCTs in medical informatics.

Evidence-based medicine: is the evidence out there for primary care clinicians?

BACKGROUND

Finding evidence to answer clinical questions is essential to the practice of evidence-based medicine (EBM). However, practising EBM in primary care is thought to be problematic because of concerns about whether evidence exists to answer specific questions.

OBJECTIVES

To determine the highest level of evidence per question; to ascertain the number of questions unanswered because of a lack of evidence; to establish the frequency with which guidelines answered questions; and to determine the domain of websites used to answer questions.

METHODS

Clinical questions were identified from two primary care answering services: ATTRACT and National Library for Health (NLH) Primary Care Answering Service. The types of evidence used to answer the question were noted, including whether this was from systematic reviews or meta-analyses (level one evidence) or from randomised controlled trials (level two). The data were collected from March to June 2008.

RESULTS

Level 1 or level 2 evidence answered 11% of questions. Sixteen per cent were unanswered because of a lack of evidence. Over 40% of questions were answered using guidelines. Forty-three per cent of questions were answered with one type of evidence and 24% with two.

CONCLUSION

Guidelines are useful resources for primary care clinicians, answering two-fifths of questions.

Can evidence-based medicine and clinical quality improvement learn from each other?

The considerable gap between what we know from research and what is done in clinical practice is well known. Proposed responses include the Evidence-Based Medicine (EBM) and Clinical Quality Improvement. EBM has focused more on ‘doing the right things’—based on external research evidence—whereas Quality Improvement (QI) has focused more on ‘doing things right’—based on local processes. However, these are complementary and in combination direct us how to ‘do the right things right’. This article examines the differences and similarities in the two approaches and proposes that by integrating the bedside application, the methodological development and the training of these complementary disciplines both would gain.

Evaluating clinical librarian services: a systematic review

BACKGROUND

  Previous systematic reviews have indicated limited evidence and poor quality evaluations of clinical librarian (CL) services. Rigorous evaluations should demonstrate the value of CL services, but guidance is needed before this can be achieved.

OBJECTIVES

  To undertake a systematic review which examines models of CL services, quality, methods and perspectives of clinical librarian service evaluations.

METHODS

  Systematic review methodology and synthesis of evidence, undertaken collaboratively by a group of 8 librarians to develop research and critical appraisal skills.

RESULTS

  There are four clear models of clinical library service provision. Clinical librarians are effective in saving health professionals time, providing relevant, useful information and high quality services. Clinical librarians have a positive effect on clinical decision making by contributing to better informed decisions, diagnosis and choice of drug or therapy. The quality of CL studies is improving, but more work is needed on reducing bias and providing evidence of specific impacts on patient care. The Critical Incident Technique as part of a mixed method approach appears to offer a useful approach to demonstrating impact.

CONCLUSIONS

  This systematic review provides practical guidance regarding the evaluation of CL services. It also provides updated evidence regarding the effectiveness and impact of CL services. The approach used was successful in developing research and critical appraisal skills in a group of librarians.

The informationist: building evidence for an emerging health profession

BACKGROUND

To encourage evidence-based practice, an Annals of Internal Medicine editorial called for a new professional on clinical teams: an informationist trained in science or medicine as well as information science.

OBJECTIVES

The study explored the effects of informationists on information behaviors of clinical research teams, specifically, frequency of seeking information for clinical or research decisions, range of resources consulted, perceptions about access to information, confidence in adequacy of literature searches, and effects on decision making and practice. It also explored perceptions about training and experience needed for successful informationists.

METHODS

Exploratory focus groups and key interviews were followed by baseline and follow-up surveys conducted with researchers and clinicians receiving the service. Survey data were analyzed with Pearson's chi-square or Fisher's exact test.

RESULTS

Comparing 2006 to 2004 survey responses, the researchers found that study participants reported: seeking answers to questions more frequently, spending more time seeking or using information, believing time was less of an obstacle to finding or using information, using more information resources, and feeling greater satisfaction with their ability to find answers. Participants' opinions on informationists' qualifications evolved to include both subject knowledge and information searching expertise.

CONCLUSION

Over time, clinical research teams with informationists demonstrated changes in their information behaviors, and they valued an informationist's subject matter expertise more.