Physicians' use of medical knowledge resources: preliminary theoretical framework and findings

Needs-Assessment for an Artificial Intelligence-Based Chatbot for Pharmacists in HIV Care: Results from a Knowledge-Attitudes-Practices Survey

BACKGROUND

Pharmacists need up-to-date knowledge and decision-making support in HIV care. We aim to develop MARVIN-Pharma, an adapted artificial intelligence-based chatbot initially for people with HIV, to assist pharmacists in considering evidence-based needs.

METHODS

From December 2022 to December 2023, an online needs-assessment survey evaluated Québec pharmacists' knowledge, attitudes, involvement, and barriers relative to HIV care, alongside perceptions relevant to the usability of MARVIN-Pharma. Recruitment involved convenience and snowball sampling, targeting National HIV and Hepatitis Mentoring Program affiliates.

RESULTS

Forty-one pharmacists (28 community, 13 hospital-based) across 15 Québec municipalities participated. Participants perceived their HIV knowledge as moderate (M = 3.74/6). They held largely favorable attitudes towards providing HIV care (M = 4.02/6). They reported a "little" involvement in the delivery of HIV care services (M = 2.08/5), most often ART adherence counseling, refilling, and monitoring. The most common barriers reported to HIV care delivery were a lack of time, staff resources, clinical tools, and HIV information/training, with pharmacists at least somewhat agreeing that they experienced each (M ≥ 4.00/6). On average, MARVIN-Pharma's acceptability and compatibility were in the 'undecided' range (M = 4.34, M = 4.13/7, respectively), while pharmacists agreed to their self-efficacy to use online health services (M = 5.6/7).

CONCLUSION

MARVIN-Pharma might help address pharmacists' knowledge gaps and barriers to HIV treatment and care, but pharmacist engagement in the chatbot's development seems vital for its future uptake and usability.

Situational cues surrounding family physicians seeking external resources while self-monitoring in practice

Many models of safe and effective clinical decision making in medical practice emphasize the importance of recognizing moments of uncertainty and seeking help accordingly. This is not always done effectively, but we know little about what cues prompt health professionals to call on resources beyond their own knowledge or skill set. Such information would offer guidance regarding how systems might be designed to offer better individual support. In this study, the authors explored the situational factors that are present during moments of uncertainty that lead primary care physicians to access external resources. To do so, a generic qualitative exploratory analysis was conducted on 72 narratives collected through audio recorder-based, self-observational, journaling completed by 12 purposively selected family physicians. Participants were asked to provide a detailed descriptive account of the circumstances surrounding their consultation of external resources immediately after 6 sequential patient encounters in which they felt compelled to seek such support. Thematic analysis of the transcripts was performed to better understand participants' experiences of the social, contextual, and personal features surrounding decisions to seek support. When doing so we observed that specific features of patient encounters were routinely present when physicians decided to access external sources for help. These included medical aspects of the case (e.g., complex presentations), social aspects (e.g., the presence of another individual), and personal factors (e.g., feeling a need for reassurance). External resources were seen as an opportunity for verification, a mechanism to increase patient satisfaction, and a means through which to defend decision-making. Accessing such resources appeared to influence the physician-patient relationship for various reasons. Recognition and further study of the cues that prompt use of external information will further our understanding of physicians' behavioural responses to challenging/uncertain situations, highlight mechanisms through which a culture of self-directed assessment seeking might be encouraged, and offer guidance regarding ways in which physicians can be encouraged to practice mindfully. Our results make it clear that reasons for which primary care physicians seek the support of external resources may be multifactorial and, therefore, one should be cautious when inferring reasons for the pursuit of such support.

Psychiatrists' reflections on a medication-free program for patients with psychosis

BACKGROUND

Based on a collective action from a number of Norwegian user organizations for mental health, the Norwegian government demanded the regional health authorities to establish a medication-free treatment option for patients with severe mental illnesses. This change in governmental health policy has several direct implications, including making antipsychotic medication to a greater extent optional for patients with active psychosis.

AIMS

The sim of this study was to investigate psychiatrists' reflections on the public medication-free program and how they think it will affect patients' adherence to antipsychotics generally.

METHOD

We used a thematic analytic approach within an interpretative phenomenological framework. Consecutively semi-structured qualitative interviews were conducted with 23 psychiatrists (16 female, with 4-35 years of specialist experience).

RESULTS

Thematic analysis revealed four main interrelated themes. Psychiatrists considered medication-free treatment as an unscientific option for a stigmatized patient group; they believed that the advent of such a program is due to proposals from some dissatisfied users and their supporters and not from the majority of patients; in spite of active psychotic symptoms and lack of insight, patients had a crucial impact on choice of treatment and accordingly on adherence to medication; and psychiatrists reported that they in spite of governmental instructions utilized professionalism against unscientific ideology.

CONCLUSIONS

Despite all the internal and external pressure that the psychiatrists reported being exposed to, this did not affect their professional integrity in medical decisions based on guidelines, expertise and research studies. They believed that this treatment option would exacerbate negative attitudes towards medication and further worsen already existing adherence issues.

Determinants of knowledge-sharing networks in primary care

BACKGROUND

Around the world, health reforms are increasingly fostering collaboration and integration among primary care physicians with the aim of facilitating knowledge sharing and evidence-informed decision-making. Although extant research on this topic is abundant, the evidence and results regarding social and organizational factors affecting the formation of knowledge-sharing networks in this setting are inconclusive.

PURPOSE

The aim of this article is to explore multiple theoretical mechanisms explaining the formation of knowledge-sharing networks among primary care physicians across relevant clinical areas.

METHODOLOGY/APPROACH

The data are collected from two local health authorities in the Italian National Health Service that are responsible for delivering primary care in two Italian regions. Exponential random graph models are used to test the hypotheses.

FINDINGS

Our findings indicate that knowledge-sharing networks are highly correlated across clinical areas. In addition, knowledge-sharing networks are highly reciprocal and clustered. We also observe that formal models adopted to foster collaboration have remarkably different effects on the formation of knowledge networks, depending upon the diverse knowledge management approaches adopted in the surveyed local health authorities.

PRACTICE IMPLICATIONS

Primary care organizations need to develop and implement knowledge management practices in order to help physicians in identifying knowledge domain experts as well as to support connections through formal groupings and incentives.

Comprehensive process model of clinical information interaction in primary care: results of a "best-fit" framework synthesis

Objective

To describe a new, comprehensive process model of clinical information interaction in primary care (Clinical Information Interaction Model, or CIIM) based on a systematic synthesis of published research.

Materials and Methods

We used the "best fit" framework synthesis approach. Searches were performed in PubMed, Embase, the Cumulative Index to Nursing and Allied Health Literature (CINAHL), PsycINFO, Library and Information Science Abstracts, Library, Information Science and Technology Abstracts, and Engineering Village. Two authors reviewed articles according to inclusion and exclusion criteria. Data abstraction and content analysis of 443 published papers were used to create a model in which every element was supported by empirical research.

Results

The CIIM documents how primary care clinicians interact with information as they make point-of-care clinical decisions. The model highlights 3 major process components: (1) context, (2) activity (usual and contingent), and (3) influence. Usual activities include information processing, source-user interaction, information evaluation, selection of information, information use, clinical reasoning, and clinical decisions. Clinician characteristics, patient behaviors, and other professionals influence the process.

Discussion

The CIIM depicts the complete process of information interaction, enabling a grasp of relationships previously difficult to discern. The CIIM suggests potentially helpful functionality for clinical decision support systems (CDSSs) to support primary care, including a greater focus on information processing and use. The CIIM also documents the role of influence in clinical information interaction; influencers may affect the success of CDSS implementations.

Conclusion

The CIIM offers a new framework for achieving CDSS workflow integration and new directions for CDSS design that can support the work of diverse primary care clinicians.

Describing knowledge encounters in healthcare: a mixed studies systematic review and development of a classification

Background

Implementation science seeks to promote the uptake of research and other evidence-based findings into practice, but for healthcare professionals, this is complex as practice draws on, in addition to scientific principles, rules of thumb and a store of practical wisdom acquired from a range of informational and experiential sources. The aims of this review were to identify sources of information and professional experiences encountered by healthcare workers and from this to build a classification system, for use in future observational studies, that describes influences on how healthcare professionals acquire and use information in their clinical practice.

Methods

This was a mixed studies systematic review of observational studies. DATA SOURCES: OVID MEDLINE and Embase and Google Scholar were searched using terms around information, knowledge or evidence and sharing, searching and utilisation combined with terms relating to healthcare groups. ELIGIBILITY: Studies were eligible if one of the intentions was to identify information or experiential encounters by healthcare workers. DATA EXTRACTION: Data was extracted by one author after piloting with another. STUDY APPRAISAL: Studies were assessed using the Mixed Methods Appraisal Tool (MMAT). PRIMARY OUTCOME: The primary outcome extracted was the information source or professional experience encounter. ANALYSIS: Similar encounters were grouped together as single constructs. Our synthesis involved a mixed approach using the top-down logic of the Bliss Bibliographic Classification System (BC2) to generate classification categories and a bottom-up approach to develop descriptive codes (or “facets”) for each category, from the data. The generic terms of BC2 were customised by an iterative process of thematic content analysis. Facets were developed by using available theory and keeping in mind the pragmatic end use of the classification.

Results

Eighty studies were included from which 178 discreet knowledge encounters were extracted. Six classification categories were developed: what information or experience was encountered; how was the information or experience encountered; what was the mode of encounter; from whom did the information originate or with whom was the experience; how many participants were there; and where did the encounter take place. For each of these categories, relevant descriptive facets were identified.

Conclusions

We have sought to identify and classify all knowledge encounters, and we have developed a faceted description of key categories which will support richer descriptions and interrogations of knowledge encounters in healthcare research.

Electronic supplementary material

The online version of this article (doi:10.1186/s13012-017-0564-1) contains supplementary material, which is available to authorized users.

New challenges for clinical informatics: knowledge management tools

It is widely recognized that medicine has reached a crisis point. Doctors can no longer memorize or effectively apply the vast amounts of scientific knowledge that are relevant to their clinical practice. Estimates suggest that human knowledge doubles every 33 years [1], while the expansion of medical knowledge is currently estimated at doubling about every 19 years. In contrast, our intellectual capacity has remained practically static over the last thousand or so years [2].

Many have looked to Information Technology (IT) to solve the problem of information overload in medicine. Simply converting existing information resources into an electronic form, however, and distributing or making them accessible to users, is far from adequate and can often exacerbate the problem of information overload. Efficient organization and distribution of knowledge is one of medicine’s biggest challenges, and there is much talk about the new paradigm of ‘knowledge management tools’ which will transform the way medicine is practised.

A method for the development of disease-specific reference standards vocabularies from textual biomedical literature resources

OBJECTIVE

Disease-specific vocabularies are fundamental to many knowledge-based intelligent systems and applications like text annotation, cohort selection, disease diagnostic modeling, and therapy recommendation. Reference standards are critical in the development and validation of automated methods for disease-specific vocabularies. The goal of the present study is to design and test a generalizable method for the development of vocabulary reference standards from expert-curated, disease-specific biomedical literature resources.

METHODS

We formed disease-specific corpora from literature resources like textbooks, evidence-based synthesized online sources, clinical practice guidelines, and journal articles. Medical experts annotated and adjudicated disease-specific terms in four classes (i.e., causes or risk factors, signs or symptoms, diagnostic tests or results, and treatment). Annotations were mapped to UMLS concepts. We assessed source variation, the contribution of each source to build disease-specific vocabularies, the saturation of the vocabularies with respect to the number of used sources, and the generalizability of the method with different diseases.

RESULTS

The study resulted in 2588 string-unique annotations for heart failure in four classes, and 193 and 425 respectively for pulmonary embolism and rheumatoid arthritis in treatment class. Approximately 80% of the annotations were mapped to UMLS concepts. The agreement among heart failure sources ranged between 0.28 and 0.46. The contribution of these sources to the final vocabulary ranged between 18% and 49%. With the sources explored, the heart failure vocabulary reached near saturation in all four classes with the inclusion of minimal six sources (or between four to seven sources if only counting terms occurred in two or more sources). It took fewer sources to reach near saturation for the other two diseases in terms of the treatment class.

CONCLUSIONS

We developed a method for the development of disease-specific reference vocabularies. Expert-curated biomedical literature resources are substantial for acquiring disease-specific medical knowledge. It is feasible to reach near saturation in a disease-specific vocabulary using a relatively small number of literature sources.

The Information Seeking and Communication Model

Purpose

– This study investigated the application in the field of healthcare of a recently developed model of information seeking and communication. The purpose of this paper is to test the model’s validity and to identify insights that it may provide.

Design/methodology/approach

– To investigate the model’s application to information users, the findings from published literature on physicians’ information behaviour were studied. To investigate its application to information providers, interviews were carried out with staff working for the National Institute for Health and Care Excellence and with employees of pharmaceutical companies. The findings were examined using deductive content analysis.

Findings

– The findings endorse the validity of the model, with minor modifications. The model provides practical insights into the behaviour of both users and providers of information and the factors that influence them. It can be used to identify ways in which information behaviour may be positively modified in both finding and communicating healthcare information.

Originality/value

– This research demonstrates the practical value of a new model of information behaviour which was developed using insights from earlier models. In doing so it answers criticisms that research in library and information science often fails to build on previous research and that it has little practical usefulness.

Should we be teaching information management instead of evidence-based medicine?

To encourage high-quality patient care guided by the best evidence, many medical schools and residencies are teaching techniques for critically evaluating the medical literature. While a large step forward in many regards, these skills of evidence-based medicine are necessary but not sufficient for the practice of contemporary medicine and surgery. Incorporating the best evidence into the real world of busy clinical practice requires the applied science of information management. Clinicians must learn the techniques and skills to focus on finding, evaluating, and using information at the point of care. This information must be both relevant to themselves and their patients and be valid. WHERE ARE WE NOW?: Today, orthopaedic surgery is in the post-Flexner era of passive didactic learning combined with the practical experience of surgery as taught by supervising experts. The medical student and house officer fill their memory with mountains of facts and classic references 'just in case' that information is needed. With libraries and now internet repositories of orthopaedic information, all orthopaedic knowledge can be readily accessed without having to store much in one's memory. Evidence is often trumped by the opinion of a teacher or expert in the field. WHERE DO WE NEED TO GO?: To improve the quality of orthopaedic surgery there should be application of the best evidence, changing practice where needed when evidence is available. To apply evidence, the evidence has to find a way into practice without the long pipeline of change that now exists. Evidence should trump opinion and unfounded practices. HOW DO WE GET THERE?: To create a curriculum and learning space for information management requires effort on the part of medical schools, residency programs and health systems. Internet sources need to be created that have the readily available evidence-based answers to patient issues so surgeons do not need to spend all the time necessary to research the questions on their own. Information management is built on a platform created by EBM but saves the surgeon time and improves accuracy by having experts validate the evidence and make it easily available.

Challenges in multidisciplinary cancer care among general surgeons in Canada

Background

While many factors can influence the way that cancer care is delivered, including the way that evidence is packaged and disseminated, little research has evaluated how health care professionals who manage cancer patients seek and use this information to identify whether and how this could be supported. Through interviews we identified that general surgeons experience challenges in coordinating care for complex cancer patients whose management is not easily addressed by guidelines, and conducted a population-based survey of general surgeon information needs and information seeking practices to extend these findings.

Methods

General surgeons with privileges at acute care hospitals in Ontario, Canada were mailed a questionnaire to solicit information needs (task, importance), information seeking (source, frequency of and reasons for use), key challenges and suggested solutions. Non-responders received up to three reminder packages. Significant differences among sub-groups (age, setting) were examined statistically (Kruskal Wallis, Mann Whitney, Chi Square). Standard qualitative methods were used to thematically analyze open-ended responses.

Results

The response rate was 44.2% (170/385) representing all 14 health regions. System resource constraints (60.4%), comorbidities (56.4%) and physiologic factors (51.8%) were top-ranked issues creating information needs. Local surgical colleagues (84.6%), other local colleagues (82.2%) and the Internet (81.1%) were top-ranked sources of information, primarily due to familiarity and speed of access. No resources were considered to be highly applicable to patient care. Challenges were related to limitations in diagnostics and staging, operative resources, and systems to support multidisciplinary care, together accounting for 76.0% of all reported issues. Findings did not differ significantly by surgeon age or setting of care.

Conclusion

General surgeons appear to use a wide range of information resources but they may not address the complex needs of many cancer patients. Decision-making is challenged by informational and logistical issues related to the coordination of multidisciplinary care. This suggests that limitations in system capacity may, in part, contribute to variable guideline compliance. Further research is required to evaluate the appropriateness of information seeking, and both concurrent and consecutive mechanisms by which to achieve multidisciplinary care.

How do doctors use information in real-time? A qualitative study of internal medicine resident precepting

BACKGROUND

Despite the importance of evidence-based medicine in medical education, little observational research exists on how doctors-in-training seek and use evidence from information resources in ambulatory care.

OBJECTIVE

To describe information exchange behaviour by internal medicine residents and attendings in ambulatory resident clinic precepting rooms.

DESIGN

We observed resident behaviour and audiotaped resident-attending doctor interactions during precepting sessions.

PARTICIPANTS

Participating residents included 70 of an eligible 89 residents and 28 of 34 eligible attendings from one large academic internal medicine residency programme in the Midwestern USA. Residents were observed during 95 separate precepting interactions at four ambulatory sites.

APPROACH

Using a qualitative approach, we analysed transcripts and field notes of observed behaviours and interactions looking for themes of information exchange. Coders discussed themes which were refined using feedback from an interdisciplinary panel.

RESULTS

Four themes of information exchange behaviour emerged: (i) questioning behaviours that were used as part of the communication process in which the resident and attending doctor could reason together; (ii) searching behaviour of non-human knowledge sources occurred in a minority of precepting interations; (iii) unsolicited knowledge offering and (iv) answering behaviours were important means of exchanging information.

CONCLUSIONS

Most clinic interactions between resident and attending doctors relied heavily on spoken deliberation without resorting to the scientific literature or other published information resources. These observations suggest a range of factors that may moderate information exchange behaviour in the precepting context including relationships, space and efficiency. Future research should aim to more readily adapt information resources to the relationships and practice context of precepting.

What factors are associated with the integration of evidence retrieval technology into routine general practice settings?

BACKGROUND

Information retrieval systems have the potential to improve patient care but little is known about the variables which influence clinicians' uptake and use of systems in routine work.

AIM

To determine which factors influenced use of an online evidence retrieval system.

DESIGN OF STUDY

Computer logs and pre- and post-system survey analysis of a 4-week clinical trial of the Quick Clinical online evidence system involving 227 general practitioners across Australia.

RESULTS

Online evidence use was not linked to general practice training or clinical experience but female clinicians conducted more searches than their male counterparts (mean use=14.38 searches, S.D.=11.68 versus mean use=8.50 searches, S.D.=9.99; t=2.67, d.f.=157, P=0.008). Practice characteristics such as hours worked, type and geographic location of clinic were not associated with search activity. Information seeking was also not related to participants' perceived information needs, computer skills, training nor Internet connection speed. Clinicians who reported direct improvements in patient care as a result of system use had significantly higher rates of system use than other users (mean use=12.55 searches, S.D.=13.18 versus mean use=8.15 searches, S.D.=9.18; t=2.322, d.f.=154 P=0.022). Comparison of participants' views pre- and post- the trial, showed that post-trial clinicians expressed more positive views about searching for information during a consultation (chi(2)=27.40, d.f.=4, P< or =0.001) and a significantly greater number reported seeking information between consultations as a result of having access to an online evidence system in their consulting rooms (chi(2)=9.818, d.f.=2, P=0.010).

CONCLUSION

Clinicians' use of an online evidence system was directly related to their reported experiences of improvements in patient care. Post-trial clinicians positively changed their views about having time to search for information and pursued more questions during clinic hours.

The role of collegial interaction in continuing professional development

INTRODUCTION

Many physicians seek information from colleagues over other sources, highlighting the important role of interaction in continuing professional development (CPD). To guide the development of CPD opportunities, this study explored the nature of cancer-related questions faced by general surgeons, and how interaction with colleagues addressed those questions.

METHODS

This study involved thematic analysis of field notes collected through observation and transcripts of telephone interviews with 20 surgeons, two pathologists, one medical oncologist, and one radiation oncologist affiliated with six community hospitals participating in multidisciplinary cancer conferences by videoconference in one region of Ontario, Canada.

RESULTS

Six multidisciplinary cancer conferences (MCCs) were observed between April and September 2006, and 11 interviews were conducted between December 2006 and January 2007. Sharing of clinical experience made possible collective decision making for complex cancer cases. Physicians thought that collegial interaction improved awareness of current evidence, patient satisfaction with treatment plans, appropriate care delivery, and continuity. By comparing proposed treatment with that of the group and gaining exposure to decision making for more cases than they would see in their own practices, physicians developed clinical expertise that could be applied to future cases. Little collegial interaction occurred outside these organized sessions.

DISCUSSION

These findings highlight the role of formally coordinated collegial interaction as an important means of CPD for general surgeons. Investment may be required for infrastructure to support such efforts and for release of health professional time for participation. Further research is required to examine direct and indirect outcomes of collegial interaction.

Obtaining and providing health information in the community pharmacy setting

Community pharmacists are a valuable information resource for patients and other healthcare providers. The advent of new information technology, most notably the Internet, coupled with the rapid availability of new healthcare information, has fueled this demand. Pharmacy students must receive training that enables them to meet this need. Community advanced pharmacy practice experiences (APPEs) provide an excellent opportunity for students to develop and master drug information skills in a real-world setting. Preceptors must ensure that students are familiar with drug information resources and can efficiently identify the most useful resource for a given topic. Students must also be trained to assess the quality of resources and use this information to effectively respond to drug or health information inquiries. This article will discuss key aspects of providing drug information in the community pharmacy setting and can serve as a guide and resource for APPE preceptors.

Teaching evidence-based medicine: should we be teaching information management instead?

To encourage high-quality patient care guided by the best evidence, many medical schools and residencies are teaching techniques for critically evaluating the medical literature. While a large step forward, these skills of evidence-based medicine are necessary but not sufficient for the practice of contemporary medicine. Incorporating the best evidence into the real world of busy clinical practice requires the applied science of information management. Clinicians must learn the techniques and skills to focus on finding, evaluating, and using information at the point of care. This information must be both relevant to themselves and their patients as well as being valid. The authors discuss the need to teach the applied science of information management along with, or perhaps even instead of, teaching the basic science of evidence-based medicine. All students, residents, and practicing physicians need three skills to practice the best medicine: the ability to select foraging--"keeping up"--tools that filter information for relevance and validity, the skill to select and use a hunting--"just in time"--information tool that presents prefiltered information easily and in a quickly accessible form at the point of care, and the ability to make decisions by combining the best patient-oriented evidence with patient-centered care, placing the evidence in perspective with the needs and desires of the patient. This teaching of information management skills will prepare students and residents for a practice of medicine that requires lifelong learning.

Answering physicians' clinical questions: obstacles and potential solutions

OBJECTIVE

To identify the most frequent obstacles preventing physicians from answering their patient-care questions and the most requested improvements to clinical information resources.

DESIGN

Qualitative analysis of questions asked by 48 randomly selected generalist physicians during ambulatory care.

MEASUREMENTS

Frequency of reported obstacles to answering patient-care questions and recommendations from physicians for improving clinical information resources.

RESULTS

The physicians asked 1,062 questions but pursued answers to only 585 (55%). The most commonly reported obstacle to the pursuit of an answer was the physician's doubt that an answer existed (52 questions, 11%). Among pursued questions, the most common obstacle was the failure of the selected resource to provide an answer (153 questions, 26%). During audiotaped interviews, physicians made 80 recommendations for improving clinical information resources. For example, they requested comprehensive resources that answer questions likely to occur in practice with emphasis on treatment and bottom-line advice. They asked for help in locating information quickly by using lists, tables, bolded subheadings, and algorithms and by avoiding lengthy, uninterrupted prose.

CONCLUSION

Physicians do not seek answers to many of their questions, often suspecting a lack of usable information. When they do seek answers, they often cannot find the information they need. Clinical resource developers could use the recommendations made by practicing physicians to provide resources that are more useful for answering clinical questions.

Clinical team functioning and IT innovation: a study of the diffusion of a point-of-care online evidence system

OBJECTIVES

To investigate the association between clinical team functioning and diffusion (awareness, use, and impact) of a 24-hour online evidence retrieval system. To examine the relationships between clinical team characteristics and the adoption of the online evidence system.

DESIGN

18 clinical teams, consisting of 180 clinicians from three Australian hospitals, were identified and studied. Teams were categorized as small ( 15).

MEASUREMENTS

Clinical team functioning was assessed using the Team Climate Inventory (TCI). Awareness, use, and impact of an online evidence retrieval system were measured using a self-administered questionnaire. The relationships between TCI scores and awareness, use, and impact were examined using t-tests and one-way ANOVAs. Chi square analyses were used to examine differences between small and large teams. RESULTS were interpreted within a diffusion of innovations framework.

RESULTS

Clinical team functioning was not related to awareness or use of the online evidence retrieval system. However, clinical team functioning was significantly associated with the impact of online evidence in terms of reported experience of improved patient care following system use. Clinicians in small teams ( 15) teams.

CONCLUSIONS

Team functioning had the greatest impact on the fourth stage of innovation diffusion, the effective use of online evidence for clinical care. This supports Rogers' diffusion of innovation theory, to the effect that different types of communication about an innovation are important at different stages in the diffusion process. Members of small teams were more aware of the system than members of large teams. Team functioning is amenable to improvement through interventions. The findings suggest that the role of team climate in the diffusion of information systems is a promising area for future research.

Variation in the use of online clinical evidence: a qualitative analysis

OBJECTIVE

To investigate factors influencing variations in clinicians' use of an online evidence retrieval system.

SETTING

Public hospitals in New South Wales, Australia.

METHOD

Web log analysis demonstrated considerable variation in rates of evidence use by clinicians at different hospitals. Focus groups and interviews were held with 61 staff from three hospitals, two with high rates of use and one with a low rate of use, to explore variation in evidence use.

RESULTS

Differences between hospitals' and professional groups' (doctors, nurses and allied health) use of online evidence could be explained by organizational, professional and cultural factors. These included the presence of champions, organizational cultures which supported evidence-based practice (EBP), and database searching skills of individual clinicians. Staff shortages, ease of access and time taken to use the online evidence system were cited as barriers to use at the low use site, but no objective differences in these measures were found between the high and low use sites.

CONCLUSION

Social and cultural factors were found to be better discriminators of high and low evidence use than technical factors.

How doctors learn: education and learning across the medical-school-to-practice trajectory

People have needs they address daily over the course of their lives. They address these needs through identifying and resolving problems in ways that reflect their identities. This means that education, including medical education, is an identity development process because it provides people with more and better ways to satisfy their needs. Also, education does this by providing them with the knowledge and skills they use to occupy identifiable niches in society. These niches are describable in terms of what people do and the ways in which they relate to others. Medical education, in other words, allows medical students and residents to take on physicians' identities, and their identities go a long way toward defining when, where, and how they will address their psychosocial needs. Doctors begin developing the skills and knowledge they need in medical school and continue developing them through residency and into practice. However, there are important differences among medical students, residents, and practicing physicians, and these differences have an impact on how the individuals satisfy their psychosocial needs. These differences also bear on how well they will satisfy their needs later in their careers; i.e., the things physicians-in-training do to satisfy their psychosocial needs in the course of learning to become doctors are expected to prepare them to address their psychosocial needs later in life in their roles as both healers and professionals. It is against this background of describing how education helps physicians solve problems and so address their recurring psychosocial needs that the author presents 12 conclusions concerning physicians' learning.

Rationality in medical decision making: a review of the literature on doctors' decision-making biases

The objectives of this study were to describe ways in which doctors make suboptimal diagnostic and treatment decisions, and to discuss possible means of alleviating those biases, using a review of past studies from the psychological and medical decision-making literatures. A number of biases can affect the ways in which doctors gather and use evidence in making diagnoses. Biases also exist in how doctors make treatment decisions once a definitive diagnosis has been made. These biases are not peculiar to the medical domain but, rather, are manifestations of suboptimal reasoning to which people are susceptible in general. None the less, they can have potentially grave consequences in medical settings, such as erroneous diagnosis or patient mismanagement. No surefire methods exist for eliminating biases in medical decision making, but there is some evidence that the adoption of an evidence-based medicine approach or the incorporation of formal decision analytic tools can improve the quality of doctors' reasoning. Doctors' reasoning is vulnerable to a number of biases that can lead to errors in diagnosis and treatment, but there are positive signs that means for alleviating some of these biases are available.

Accessing heterogeneous sources of evidence to answer clinical questions

The large and rapidly growing number of information sources relevant to health care, and the increasing amounts of new evidence produced by researchers, are improving the access of professionals and students to valuable information. However, seeking and filtering useful, valid information can be still very difficult. An online information system that conducts searches based on individual patient data can have a beneficial influence on the particular patient's outcome and educate the healthcare worker. In this paper, we describe the underlying model for a system that aims to facilitate the search for evidence based on clinicians' needs. This paper reviews studies of information needs of clinicians, describes principles of information retrieval, and examines the role that standardized terminologies can play in the integration between a clinical system and literature resources, as well as in the information retrieval process. The paper also describes a model for a digital library system that supports the integration of clinical systems with online information sources, making use of information available in the electronic medical record to enhance searches and information retrieval. The model builds on several different, previously developed techniques to identify information themes that are relevant to specific clinical data. Using a framework of evidence-based practice, the system generates well-structured questions with the intent of enhancing information retrieval. We believe that by helping clinicians to pose well-structured clinical queries and including in them relevant information from individual patients' medical records, we can enhance information retrieval and thus can improve patient-care.

The potential for research-based information in public health: identifying unrecognised information needs

OBJECTIVE

To explore whether there is a potential for greater use of research-based information in public health practice in a local setting. Secondly, if research-based information is relevant, to explore the extent to which this generates questioning behaviour.

DESIGN

Qualitative study using focus group discussions, observation and interviews.

SETTING

Public health practices in Norway.

PARTICIPANTS

52 public health practitioners.

RESULTS

In general, the public health practitioners had a positive attitude towards research-based information, but believed that they had few cases requiring this type of information. They did say, however, that there might be a potential for greater use. During five focus groups and six observation days we identified 28 questions/cases where it would have been appropriate to seek out research evidence according to our definition. Three of the public health practitioners identified three of these 28 cases as questions for which research-based information could have been relevant. This gap is interpreted as representing unrecognised information needs.

CONCLUSIONS

There is an unrealised potential in public health practice for more frequent and extensive use of research-based information. The practitioners did not appear to reflect on the need for scientific information when faced with new cases and few questions of this type were generated.

A taxonomy of generic clinical questions: classification study

OBJECTIVE

To develop a taxonomy of doctors' questions about patient care that could be used to help answer such questions.

DESIGN

Use of 295 questions asked by Oregon primary care doctors to modify previously developed taxonomy of 1101 clinical questions asked by Iowa family doctors.

SETTING

Primary care practices in Iowa and Oregon.

PARTICIPANTS

Random samples of 103 Iowa family doctors and 49 Oregon primary care doctors.

MAIN OUTCOME MEASURES

Consensus among seven investigators on a meaningful taxonomy of generic questions; interrater reliability among 11 individuals who used the taxonomy to classify a random sample of 100 questions: 50 from Iowa and 50 from Oregon.

RESULTS

The revised taxonomy, which comprised 64 generic question types, was used to classify 1396 clinical questions. The three commonest generic types were "What is the drug of choice for condition x?" (150 questions, 11%); "What is the cause of symptom x?" (115 questions, 8%); and "What test is indicated in situation x?" (112 questions, 8%). The mean interrater reliability among 11 coders was moderate (kappa=0.53, agreement 55%).

CONCLUSIONS

Clinical questions in primary care can be categorised into a limited number of generic types. A moderate degree of interrater reliability was achieved with the taxonomy developed in this study. The taxonomy may enhance our understanding of doctors' information needs and improve our ability to meet those needs.

Is the Internet useful for clinical practice?

BACKGROUND

Internet use in dentistry is increasing. The authors surveyed dentists to determine how they are using the Internet to support patient care and how useful they perceive the Internet to be.

MATERIALS AND METHODS

The authors sent a 25-question survey to 455 subscribers to the Internet Dental Forum, an Internet discussion list. Participants completed the survey through a Web page or by e-mail during an 18-day period in December 1998. The survey included questions about when they began using the Internet and their motivations for doing so, their positive and negative experiences, where they use the Internet and why, their use of information resources--including electronic information resources--and demographics.

RESULTS

The authors received 314 surveys from 438 participants, a 72 percent response rate. Participants reported demographic characteristics similar to national averages and were highly computer-literate. Most considered the Internet essential for their practice and had increased their Internet use for professional reasons over time. Ninety-seven percent of respondents used the Internet at home, and 68 percent used it in the office. Only 5 percent used the Internet in the treatment area. The primary reason cited by participants for using the Internet was to keep up in general, followed by to answer patient-specific questions and to provide patient education. Reduction of professional isolation was one of the most frequently cited advantages of using the Internet.

CONCLUSIONS

The authors found that some dentists use the Internet to support clinical practice and that it helped dentists adopt new techniques for patient treatment and obtain information on new materials or products.

CLINICAL IMPLICATIONS

The Internet appears to support clinical practice mostly indirectly, by helping users keep up in general, rather than by answering specific clinical questions.

Evaluation of physician preferences for guideline implementation

Although current literature supports the use of evidence-based clinical practice guidelines (CPGs) by physicians, there is limited research concerning operational issues that may be inhibiting effective CPG implementation. The objective of our research was to increase understanding of clinical practice patterns by identifying physician preferences for CPG accessibility, format, content and learning strategies. Semistructured interviews were conducted with resident and faculty physicians in an academic medical center after they were presented with a CPG during treatment of a patient with acute pancreatitis. The results of our study revealed that physicians prefer CPGs in the form of evidence-based algorithms with treatment-specific information that is placed on the front of the patient chart during treatment. In addition, they felt that discussion of the guideline with colleagues, reminder notes/stickers on front of the patient chart, and verbal reminders from a nurse were the most effective means of encouraging utilization.

Realizing the promise: delivering pulmonary continuing medical education over the Internet

STUDY OBJECTIVES

Continuing medical education (CME) is meant to bridge the gap between new scientific observations and clinical practice. However, traditional CME has not been effective at altering the behaviors of physicians. One reason for this failure of traditional CME programs may be their inflexibility. In traditional CME, the clinician does not choose the topic, the pace of the program, or the place of learning, and the CME material cannot be easily delivered to the point of care where the clinician needs the information. Computers and computer networks have the potential to accomplish these goals. CME has begun to appear on the Internet; however, there have been few evaluations of its usefulness, acceptance, and effectiveness. Over the last 18 months, we have developed three on-line pulmonary CME programs, and we have delivered them on the Virtual Hospital, the University of Iowa's digital health sciences library on the Internet. We report our initial experience with this CME material.

DESIGN

We measured the frequency with which the Internet-delivered CME is accessed by monitoring page accessions and by using a log file analysis program (Analog 1.2.3; University of Cambridge Statistical Laboratory; Cambridge, UK). In addition, we collected all completed CME examinations and evaluation forms submitted by registered users.

MEASUREMENTS AND RESULTS

We have found that the frequency with which the Internet-delivered CME is accessed has continued to increase with time (2.3-fold increase over 18 months), that evaluations of technical and content issues are strongly favorable, and that some clinicians have been willing to pay to receive CME through the medium of the Internet.

CONCLUSIONS

We feel that with adequate peer review and quality control, physicians will use the Internet-delivered CME. However, several obstacles to wide use remain. These obstacles include issues regarding training in using the Internet for physicians, reluctance of physicians to participate in on-line commerce, and the current unavailability of CME to be delivered in small-grained quantities to the point of care. As these issues are addressed, we feel that on-line CME will represent an increasingly important CME medium for clinicians.

Information needs and information-seeking behaviors of on-call radiology residents

RATIONALE AND OBJECTIVES

The authors performed a baseline analysis of the information needs and information-seeking behaviors of on-call radiology residents to learn how to better serve their on-call information needs in the future.

MATERIALS AND METHODS

Print, electronic, and human information resources were made conveniently available to on-call radiology residents. A prospective, descriptive study was performed by using a modification of the critical incident technique to gather data from the residents every morning after being on call. Residents were asked to recall questions that arose, whether they sought answers to the questions, where they searched for answers, whether their search was successful, and whether they believed that patient care was affected.

RESULTS

The residents had 182 questions that encompassed all organ systems, imaging modalities, and aspects of radiologic practice. Residents sought answers to 138 of the 182 questions (76%) by using a wide variety of resources, with staff members and textbooks being the most common. They found answers to 114 of 138 questions (83%); 100 of the 114 (88%) answered questions affected patient care.

CONCLUSION

The ideal way to meet the information needs of on-call radiology residents is to devise an authoritative information resource that is compatible with their information-seeking behaviors.

Usage analysis of a primary care medical resource on the Internet

Physicians and patients need convenient access to quality medical information. This study's goal was to place a medical resource on the World-Wide Web (WWW), allow access to it through a simple to use interface, and analyze the usage of such a resource. The Family Practice Handbook (TFPH) was digitized and placed onto the WWW. Usage data was obtained from June 1995-June 1996. 118,804 individuals accessed TFPH viewing 409,711 pages of information. A broad spectrum of topics was accessed. TFPH proved to be an extremely popular resource, servicing the broad information needs of an international audience. These preliminary findings suggest the future promise of Internet medical resources.

Sources of referral information: a marketing analysis of physician behavior

The referral process is an important means of obtaining patients and it is necessary to determine ways of influencing the referral process to increase the patient base. This article reports research based on a survey of the referral habits of 806 primary care physicians. The results are examined in the context of physician receptivity to marketer-controlled versus health services sources of referral information.

Intelligent medical information filtering

This paper describes an intelligent information filtering system to assist users to be notified of updates to new and relevant medical information. Among the major problems users face is the large volume of medical information that is generated each day, and the need to filter and retrieve relevant information. The Internet has dramatically increased the amount of electronically accessible medical information and reduced the cost and time needed to publish. The opportunity of the Internet for the medical profession and consumers is to have more information to make decisions and this could potentially lead to better medical decisions and outcomes. However, without the assistance from professional medical librarians, retrieving new and relevant information from databases and the Internet remains a challenge. Many physicians do not have access to the services of a medical librarian. Most physicians indicate on surveys that they do not prefer to retrieve the literature themselves, or visit libraries because of the lack of recent materials, poor organisation and indexing of materials, lack of appropriate and available material, and lack of time. The information filtering system described in this paper records the online web browsing behaviour of each user and creates a user profile of the index terms found on the web pages visited by the user. A relevance-ranking algorithm then matches the user profiles to the index terms of new health care web pages that are added each day. The system creates customised summaries of new information for each user. A user can then connect to the web site to read the new information. Relevance feedback buttons on each page ask the user to rate the usefulness of the page to their immediate information needs. Errors in relevance ranking are reduced in this system by having both the user profile and medical information represented in the same representation language using a controlled vocabulary. This system also updates the user profiles, automatically relieving this burden from the user, but also allowing the user to explicitly state preferences. An initial evaluation of this system was done with health consumers using a web site on consumer health. It was found that users often modified their criteria for what they considered relevant not only between browsing sessions but also during a session. A user's criteria for what is relevant is constantly changing as they interact with the information. New revised metrics of recall and precision are needed to account for the partially relevant judgements and the dynamically changing criteria of users. Future research, development, and evaluation of interactive information retrieval systems will need to take into account the users' dynamically changing criteria of relevance.

An industrial process view of information delivery to support clinical decision making: implications for systems design and process measures

Clinical decision making is driven by information in the form of patient data and clinical knowledge. Currently prevalent systems used to store and retrieve this information have high failure rates, which can be traced to well-established system constraints. The authors use an industrial process model of clinical decision making to expose the role of these constraints in increasing variability in the delivery of relevant clinical knowledge and patient data to decision-making clinicians. When combined with nonmodifiable human cognitive and memory constraints, this variability in information delivery is largely responsible for the high variability of decision outcomes. The model also highlights the supply characteristics of information, a view that supports the application of industrial inventory management concepts to clinical decision support. Finally, the clinical decision support literature is examined from a process-improvement perspective with a focus on decision process components related to information retrieval. Considerable knowledge gaps exist related to clinical decision support process measurement and improvement.

Information seeking in primary care: how physicians choose which clinical questions to pursue and which to leave unanswered

Primary care physicians have many questions about optimal care while they are seeing patients, but they pursue only about 30% of their questions. The authors designed a study to determine the factors that motivate physicians to pursue answers to some of their questions, while leaving the majority of their questions unanswered. They interviewed 49 non-academic primary care physicians during office hours to record clinical questions as they arose in the course of patient care. The physicians then recorded their perceptions of each question with respect to 12 factors expected to motivate information seeking. Two to five days after the interview, each physician was telephoned to determine which questions had been pursued. In a multiple logistic-regression model only two factors were significant predictors of pursuit of new information: the physician's belief that a definitive answer existed, and the urgency of the patient's problem. Other factors, including the difficulty of finding the answer, potential malpractice liability, potential help or harm to the patient, and self-perceived knowledge of the problem, were not significant in the model. Primary care physicians are significantly more likely to pursue answers to their clinical questions when they believe that definitive answers to those questions exist, and when they perceive the patient's problem to be urgent. Medical information systems must be shown to have direct and immediate benefits to solving the problems of patient care if they are to be more widely used by practitioners.

Expanding the concept of medical information: an observational study of physicians' information needs

Obtaining and managing clinically relevant information constitutes a major problem for physicians, for which the development of automated tools is often proposed as a solution. However, designing and implementing appropriate automated solutions presumes knowledge of physicians' information needs. We describe an empirical study of information needs in four clinical settings in internal medicine in a university teaching hospital. In contrast to the retrospective data often used in previous studies, this research used ethnographic techniques to facilitate direct observation of communication about information needs. On the basis of this experience, we address two main issues: how to identify and interpret expressions of information needs in medicine and how to broaden our conception of "information needs" to account for the empirical data.