Seeing the random forest through the decision trees. Supporting learning health systems from histopathology with machine learning models: Challenges and opportunities

This paper discusses some overlooked challenges faced when working with machine learning models for histopathology and presents a novel opportunity to support "Learning Health Systems" with them. Initially, the authors elaborate on these challenges after separating them according to their mitigation strategies: those that need innovative approaches, time, or future technological capabilities and those that require a conceptual reappraisal from a critical perspective. Then, a novel opportunity to support "Learning Health Systems" by integrating hidden information extracted by ML models from digitalized histopathology slides with other healthcare big data is presented.

A scoping review reveals candidate quality indicators of knowledge translation and implementation science practice tools

OBJECTIVES

To identify candidate quality indicators from existing tools that provide guidance on how to practice knowledge translation and implemenation science (KT practice tools) across KT domains (dissemination, implementation, sustainability, and scalability).

STUDY DESIGN AND SETTING

We conducted a scoping review using the Joanna Briggs Institute Manual for Evidence Synthesis. We systematically searched multiple electronic databases and the gray literature. Documents were independently screened, selected, and extracted by pairs of reviewers. Data about the included articles, KT practice tools, and candidate quality indicators were analyzed, categorized, and summarized descriptively.

RESULTS

Of 43,060 titles and abstracts that were screened from electronic databases and gray literature, 850 potentially relevant full-text articles were identified, and 253 articles were included in the scoping review. Of these, we identified 232 unique KT practice tools from which 27 unique candidate quality indicators were generated. The identified candidate quality indicators were categorized according to the development (n = 17), evaluation (n = 5) and adaptation (n = 3) of the tools, and engagement of knowledge users (n = 2). No tools were identified that appraised the quality of KT practice tools.

CONCLUSIONS

The development of a quality appraisal instrument of KT practice tools is needed. The results will be further refined and finalized in order to develop a quality appraisal instrument for KT practice tools.

Public Health Surveillance of Behavioral Cancer Risk Factors During the COVID-19 Pandemic: Sentiment and Emotion Analysis of Twitter Data

BACKGROUND

The COVID-19 pandemic and its associated public health mitigation strategies have dramatically changed patterns of daily life activities worldwide, resulting in unintentional consequences on behavioral risk factors, including smoking, alcohol consumption, poor nutrition, and physical inactivity. The infodemic of social media data may provide novel opportunities for evaluating changes related to behavioral risk factors during the pandemic.

OBJECTIVE

We explored the feasibility of conducting a sentiment and emotion analysis using Twitter data to evaluate behavioral cancer risk factors (physical inactivity, poor nutrition, alcohol consumption, and smoking) over time during the first year of the COVID-19 pandemic.

METHODS

Tweets during 2020 relating to the COVID-19 pandemic and the 4 cancer risk factors were extracted from the George Washington University Libraries Dataverse. Tweets were defined and filtered using keywords to create 4 data sets. We trained and tested a machine learning classifier using a prelabeled Twitter data set. This was applied to determine the sentiment (positive, negative, or neutral) of each tweet. A natural language processing package was used to identify the emotions (anger, anticipation, disgust, fear, joy, sadness, surprise, and trust) based on the words contained in the tweets. Sentiments and emotions for each of the risk factors were evaluated over time and analyzed to identify keywords that emerged.

RESULTS

The sentiment analysis revealed that 56.69% (51,479/90,813) of the tweets about physical activity were positive, 16.4% (14,893/90,813) were negative, and 26.91% (24,441/90,813) were neutral. Similar patterns were observed for nutrition, where 55.44% (27,939/50,396), 15.78% (7950/50,396), and 28.79% (14,507/50,396) of the tweets were positive, negative, and neutral, respectively. For alcohol, the proportions of positive, negative, and neutral tweets were 46.85% (34,897/74,484), 22.9% (17,056/74,484), and 30.25% (22,531/74,484), respectively, and for smoking, they were 41.2% (11,628/28,220), 24.23% (6839/28,220), and 34.56% (9753/28,220), respectively. The sentiments were relatively stable over time. The emotion analysis suggests that the most common emotion expressed across physical activity and nutrition tweets was trust (69,495/320,741, 21.67% and 42,324/176,564, 23.97%, respectively); for alcohol, it was joy (49,147/273,128, 17.99%); and for smoking, it was fear (23,066/110,256, 20.92%). The emotions expressed remained relatively constant over the observed period. An analysis of the most frequent words tweeted revealed further insights into common themes expressed in relation to some of the risk factors and possible sources of bias.

CONCLUSIONS

This analysis provided insight into behavioral cancer risk factors as expressed on Twitter during the first year of the COVID-19 pandemic. It was feasible to extract tweets relating to all 4 risk factors, and most tweets had a positive sentiment with varied emotions across the different data sets. Although these results can play a role in promoting public health, a deeper dive via qualitative analysis can be conducted to provide a contextual examination of each tweet.

Enhanced recovery support for people with eating disorders during the COVID-19 pandemic: quality improvement using a web-based, stepped-care programme in Canada

BACKGROUND

Throughout the COVID-19 pandemic, the number of individuals struggling with eating disorders (EDs) increased substantially. Body Brave (a not-for-profit) created and implemented a web-based stepped-care Recovery Support Programme (RSP) to improve access to community-based ED services. This quality improvement study describes the RSP and assesses its ability to deliver timely access to treatment and platform engagement.

METHODS

We conducted a retrospective cohort study comparing access to, and use of Body Brave services 6 months before and 12 months after implementation of the RSP platform (using 6-month increments for two postimplementation periods). Primary programme quality measures included registration requests, number of participants onboarded and time to access services; secondary measures included use of RSP action plans, attendance for recovery sessions and workshops, number of participants accessing treatment and text-based patient experience data.

RESULTS

A substantial increase in registration requests was observed during the first postimplementation period compared with the preimplementation period (176.5 vs 85.5; p=0.028). When compared with the preimplementation period, the second postimplementation observed a significantly larger percentage of successfully onboarded participants (76.6 vs 37.9; p<0.01) and a reduction in the number of days to access services (2 days vs 31 days; p<0.01). Although participant feedback rates were low, many users found the RSP helpful, easy to access, user-friendly and were satisfied overall. Users provided suggestions for improvement (eg, a platform instructional video, offer multiple times of day for live sessions and drop-in hours).

CONCLUSIONS

Although clinical benefit needs to be assessed, our findings demonstrate that the RSP enabled participants to quickly onboard and access initial services and have informed subsequent improvements. Understanding initial programme effects and usage will help assess the feasibility of adapting and expanding the RSP across Canada to address the urgent need for low-barrier, patient-centred ED care.

Deep Learning to Refine the Identification of High-Quality Clinical Research Articles from the Biomedical Literature: Performance Evaluation

BACKGROUND

Identifying practice-ready evidence-based journal articles in medicine is a challenge due to the sheer volume of biomedical research publications. Newer approaches to support evidence discovery apply deep learning techniques to improve the efficiency and accuracy of classifying sound evidence.

OBJECTIVE

To determine how well deep learning models using variants of Bidirectional Encoder Representations from Transformers (BERT) identify high-quality evidence with high clinical relevance from the biomedical literature for consideration in clinical practice.

METHODS

We fine-tuned variations of BERT models (BERT_BASE, BioBERT, BlueBERT, and PubMedBERT) and compared their performance in classifying articles based on methodological quality criteria. The dataset used for fine-tuning models included titles and abstracts of >160,000 PubMed records from 2012-2020 that were of interest to human health which had been manually labeled based on meeting established critical appraisal criteria for methodological rigor. The data was randomly divided into 80:10:10 sets for training, validating, and testing. In addition to using the full unbalanced set, the training data was randomly undersampled into four balanced datasets to assess performance and select the best performing model. For each of the four sets, one model that maintained sensitivity (recall) at ≥99% was selected and were ensembled. The best performing model was evaluated in a prospective, blinded test and applied to an established reference standard, the Clinical Hedges dataset.

RESULTS

In training, three of the four selected best performing models were trained using BioBERT_BASE. The ensembled model did not boost performance compared with the best individual model. Hence a solo BioBERT-based model (named DL-PLUS) was selected for further testing as it was computationally more efficient. The model had high recall (>99%) and 60% to 77% specificity in a prospective evaluation conducted with blinded research associates and saved >60% of the work required to identify high quality articles.

CONCLUSIONS

Deep learning using pretrained language models and a large dataset of classified articles produced models with improved specificity while maintaining >99% recall. The resulting DL-PLUS model identifies high-quality, clinically relevant articles from PubMed at the time of publication. The model improves the efficiency of a literature surveillance program, which allows for faster dissemination of appraised research.

Identifying candidate quality indicators of tools that support the practice of knowledge translation: a scoping review protocol

OBJECTIVE

The objective of this scoping review is to identify and characterize relevant knowledge translation methods tools (those that provide guidance for optimized knowledge translation practice) to uncover candidate quality indicators to inform a future quality assessment tool for knowledge translation strategies.

INTRODUCTION

Knowledge translation strategies (defined as including knowledge translation interventions, tools, and products) target various knowledge users, including patients, clinicians, researchers, and policy-makers. The development and use of strategies that support knowledge translation practice have been rapidly increasing, making it difficult for knowledge users to decide which to use. There is limited evidence-based guidance or measures to help assess the overall quality of knowledge translation strategies.

INCLUSION CRITERIA

Empirical and non-empirical documents will be considered if they explicitly describe a knowledge translation methods tool and its development, evaluation or validation, methodological strengths or limitations, and/or use over time. The review will consider a knowledge translation methods tool if it falls within at least one knowledge translation domain (ie, implementation, dissemination, sustainability, scalability, integrated knowledge translation) in the health field.

METHODS

We will conduct a systematic search of relevant electronic databases and gray literature. The search strategy will be developed iteratively by an experienced medical information specialist and peer-reviewed with the PRESS checklist. The search will be limited to English-only documents published from 2005 onward. Documents will be independently screened, selected, and extracted by 2 researchers. Data will be analyzed and summarized descriptively, including the characteristics of the included documents, knowledge translation methods tools, and candidate quality indicators.

SCOPING REVIEW REGISTRATION

Open Science Framework ( https://osf.io/chxvq ).

Frontline connect: Evaluating a virtual technology program to enhance patient and provider communication during COVID-19

RATIONALE

Since the beginning of the COVID-19 pandemic, many hospitals have reduced in-hospital visitation. In these situations, virtual communication tools have helped maintain interaction between parties. The Frontline Connect program was designed to address communication and patient care challenges by providing data-enabled devices to clinical staff in hospitals.

OBJECTIVE

This study aimed to identify areas of improvement for the Frontline Connect program by: (a) evaluating communication needs, user experience, and program satisfaction; and (b) identifying potential barriers to device access or use.

METHODS

We administered pre-implementation needs assessment, post-use, and exit surveys to healthcare staff at a pilot hospital site in Ontario. Recruitment was through email lists and site champions using convenience sampling. We descriptively analysed survey responses and compared the initial need statements to post-implementation use-cases identified by users.

RESULTS

We received 139 needs assessments, 31 user experience assessments, and 47 exit survey responses. Most device use occurred in the emergency department and intensive care units and was facilitated by social workers, nurses, and physicians to connect patients, families, and care providers. Pre-implementation concerns were related to infection control, data security, and device privacy. In the exit survey, these were replaced by other concerns including Internet connectivity and time-intensiveness. Device utility and ease-of-use were rated 9.7/10 and 9.6/10 respectively in the user experience survey, though overall experience was rated 7.2/10 in the exit survey. Overall, respondents viewed the devices as useful and we agree with participants who suggested increased program promotion and training would likely improve adoption.

CONCLUSIONS

We found that our virtual technology program for facilitating communication was positively perceived. Survey feedback indicates that a rapid rollout in response to urgent pandemic-related needs was feasible, though program logistics could be improved. The current work supports the need to improve, standardize, and sustain virtual communication programs in hospitals.

Considerations for Conducting Bring Your Own “Device” (BYOD) Clinical Studies

Background

Digital health technologies are attracting attention as novel tools for data collection in clinical research. They present alternative methods compared to in-clinic data collection, which often yields snapshots of the participants' physiology, behavior, and function that may be prone to biases and artifacts, e.g., white coat hypertension, and not representative of the data in free-living conditions. Modern digital health technologies equipped with multi-modal sensors combine different data streams to derive comprehensive endpoints that are important to study participants and are clinically meaningful. Used for data collection in clinical trials, they can be deployed as provisioned products where technology is given at study start or in a bring your own “device” (BYOD) manner where participants use their technologies to generate study data.

Summary

The BYOD option has the potential to be more user-friendly, allowing participants to use technologies that they are familiar with, ensuring better participant compliance, and potentially reducing the bias that comes with introducing new technologies. However, this approach presents different technical, operational, regulatory, and ethical challenges to study teams. For example, BYOD data can be more heterogeneous, and recruiting historically underrepresented populations with limited access to technology and the internet can be challenging. Despite the rapid increase in digital health technologies for clinical and healthcare research, BYOD use in clinical trials is limited, and regulatory guidance is still evolving.

Key Messages

We offer considerations for academic researchers, drug developers, and patient advocacy organizations on the design and deployment of BYOD models in clinical research. These considerations address: (1) early identification and engagement with internal and external stakeholders; (2) study design including informed consent and recruitment strategies; (3) outcome, endpoint, and technology selection; (4) data management including compliance and data monitoring; (5) statistical considerations to meet regulatory requirements. We believe that this article acts as a primer, providing insights into study design and operational requirements to ensure the successful implementation of BYOD clinical studies.

Infectious diseases, comorbidities and outcomes in hospitalized people who inject drugs (PWID)

Injection drug use poses a public health challenge. Clinical experience indicates that people who inject drugs (PWID) are hospitalized frequently for infectious diseases, but little is known about outcomes when admitted. Charts were identified from local hospitals between 2013-2018 using consultation lists and hospital record searches. Included individuals injected drugs in the past six months and presented with infection. Charts were accessed using the hospital information system, undergoing primary and secondary reviews using Research Electronic Data Capture (REDCap). The Wilcoxon rank-sum test was used for comparisons between outcome categories. Categorical data were summarized as count and frequency, and compared using Fisher's exact test. Of 240 individuals, 33% were admitted to the intensive care unit, 36% underwent surgery, 12% left against medical advice (AMA), and 9% died. Infectious diagnoses included bacteremia (31%), abscess (29%), endocarditis (29%), cellulitis (20%), sepsis (10%), osteomyelitis (9%), septic arthritis (8%), pneumonia (7%), discitis (2%), meningitis/encephalitis (2%), or other (7%). Sixty-six percent had stable housing and 60% had a family physician. Fifty-four percent of patient-initiated discharges were seen in the emergency department within 30 days and 29% were readmitted. PWID are at risk for infections. Understanding their healthcare trajectory is essential to improve their care.

Tracking discussions of complementary, alternative, and integrative medicine in the context of the COVID-19 pandemic: a month-by-month sentiment analysis of Twitter data

BACKGROUND

Coronavirus disease 2019 (COVID-19) is a novel infectious disease caused by the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2). Despite the paucity of evidence, various complementary, alternative and integrative medicines (CAIMs) have been being touted as both preventative and curative. We conducted sentiment and emotion analysis with the intent of understanding CAIM content related to COVID-19 being generated on Twitter across 9 months.

METHODS

Tweets relating to CAIM and COVID-19 were extracted from the George Washington University Libraries Dataverse Coronavirus tweets dataset from March 03 to November 30, 2020. We trained and tested a machine learning classifier using a large, pre-labelled Twitter dataset, which was applied to predict the sentiment of each CAIM-related tweet, and we used a natural language processing package to identify the emotions based on the words contained in the tweets.

RESULTS

Our dataset included 28 713 English-language Tweets. The number of CAIM-related tweets during the study period peaked in May 2020, then dropped off sharply over the subsequent three months; the fewest CAIM-related tweets were collected during August 2020 and remained low for the remainder of the collection period. Most tweets (n = 15 612, 54%) were classified as positive, 31% were neutral (n = 8803) and 15% were classified as negative (n = 4298). The most frequent emotions expressed across tweets were trust, followed by fear, while surprise and disgust were the least frequent. Though volume of tweets decreased over the 9 months of the study, the expressed sentiments and emotions remained constant.

CONCLUSION

The results of this sentiment analysis enabled us to establish key CAIMs being discussed at the intersection of COVID-19 across a 9-month period on Twitter. Overall, the majority of our subset of tweets were positive, as were the emotions associated with the words found within them. This may be interpreted as public support for CAIM, however, further qualitative investigation is warranted. Such future directions may be used to combat misinformation and improve public health strategies surrounding the use of social media information.

Patient and physician perspectives on the use and outcome measures of mHealth apps: Exploratory survey and focus group study

Objective

Factors that physicians and patients consider when making decisions about using or recommending health apps are not well understood. We explored these factors to better assess how to support such decision making.

Methods

We conducted an exploratory cross-sectional study in Ontario using qualitative focus groups and quantitative surveys. 133 physicians and 94 community dwelling adults completed online surveys and we held two focus groups of nine community dwelling participants who had cardiovascular risk factors and an interest in using mHealth apps. Quantitative survey data was analyzed descriptively. Focus groups were audio-recorded and transcribed verbatim prior to inductive thematic content analysis. We integrated the results from the surveys and focus groups to understand factors that influence physicians' and patients' selection and use of such apps.

Results

Physicians recommend apps to patients but the level of evidence they prefer to use to guide selection did not align with what they were currently using. Patients trusted recommendations and reviews from medical organizations and healthcare professionals when selecting apps and were motivated to continue using apps when they supported goal setting and tracking, data sharing, decision making, and empowerment.

Conclusions

The findings highlight the significance of evaluating mHealth apps based on metrics that patients and physicians value beyond usage and clinical outcome data. Patients engage with apps that support them in confidently managing their health. Increased training and awareness of apps and creating a more rigorous evidence base showing the value of apps to supporting health goals will support greater adoption and acceptance of mHealth apps.

Online Educational Tools for Caregivers of People with Dementia: A Scoping Literature Review

BACKGROUND

Informal caregivers of people with dementia provide the majority of health-based care to people with dementia. Providing this care requires knowledge and access to resources, which caregivers often do not receive. We set out to evaluate the effect of online educational tools on informal caregiver self-efficacy, quality of life, burden/stress, depression, and anxiety, and to identify effective processes for online educational tool development.

METHODS

We conducted a scoping review of articles on online educational interventions for informal caregivers of people with dementia searching CINAHL, MEDLINE, EMBASE, and PubMed from 1990 to March 2018, with an updated search conducted in 2020. The identified articles were screened and the data were charted.

RESULTS

33 articles that reported on 24 interventions were included. There is some evidence that online interventions improve caregiver-related outcomes such as self-efficacy, depression, dementia knowledge, and quality of life; and decrease caregiver burden. Common findings across the studies included the need for tailored, stage-specific information applicable to the caregiver's situation and the use of psychosocial techniques to develop the knowledge components of the interventions.

CONCLUSION

We demonstrate the importance of having caregivers and health-care professionals involved at all stages of tool conceptualization and development. Online tools should be evaluated with robust trials that focus on how increased knowledge and development approaches affect caregiver-related outcomes.

A Deep Learning Approach to Refine the Identification of High-Quality Clinical Research Articles From the Biomedical Literature: Protocol for Algorithm Development and Validation

Background

A barrier to practicing evidence-based medicine is the rapidly increasing body of biomedical literature. Use of method terms to limit the search can help reduce the burden of screening articles for clinical relevance; however, such terms are limited by their partial dependence on indexing terms and usually produce low precision, especially when high sensitivity is required. Machine learning has been applied to the identification of high-quality literature with the potential to achieve high precision without sacrificing sensitivity. The use of artificial intelligence has shown promise to improve the efficiency of identifying sound evidence.

Objective

The primary objective of this research is to derive and validate deep learning machine models using iterations of Bidirectional Encoder Representations from Transformers (BERT) to retrieve high-quality, high-relevance evidence for clinical consideration from the biomedical literature.

Methods

Using the HuggingFace Transformers library, we will experiment with variations of BERT models, including BERT, BioBERT, BlueBERT, and PubMedBERT, to determine which have the best performance in article identification based on quality criteria. Our experiments will utilize a large data set of over 150,000 PubMed citations from 2012 to 2020 that have been manually labeled based on their methodological rigor for clinical use. We will evaluate and report on the performance of the classifiers in categorizing articles based on their likelihood of meeting quality criteria. We will report fine-tuning hyperparameters for each model, as well as their performance metrics, including recall (sensitivity), specificity, precision, accuracy, F-score, the number of articles that need to be read before finding one that is positive (meets criteria), and classification probability scores.

Results

Initial model development is underway, with further development planned for early 2022. Performance testing is expected to star in February 2022. Results will be published in 2022.

Conclusions

The experiments will aim to improve the precision of retrieving high-quality articles by applying a machine learning classifier to PubMed searching.

International Registered Report Identifier (IRRID)

DERR1-10.2196/29398

Health programmes and services addressing the prevention and management of infectious diseases in people who inject drugs in Canada: a systematic integrative review

OBJECTIVES

People who inject drugs (PWID) experience a high burden of injection drug use-related infectious disease and challenges in accessing adequate care. This study sought to identify programmes and services in Canada addressing the prevention and management of infectious disease in PWID.

DESIGN

This study employed a systematic integrative review methodology. Electronic databases (PubMed, CINAHL and Web of Science Core Collection) and relevant websites were searched for literature published between 2008 and 2019 (last search date was 6 June 2019). Eligible articles and documents were required to address injection or intravenous drug use and health programmes or services relating to the prevention or management of infectious diseases in Canada.

RESULTS

This study identified 1607 unique articles and 97 were included in this study. The health programmes and services identified included testing and management of HIV and hepatitis C virus (n=27), supervised injection facilities (n=19), medication treatment for opioid use disorder (n=12), integrated infectious disease and addiction programmes (n=10), needle exchange programmes (n=9), harm reduction strategies broadly (n=6), mobile care initiatives (n=5), peer-delivered services (n=3), management of IDU-related bacterial infections (n=2) and others (n=4). Key implications for policy, practice and future research were identified based on the results of the included studies, which include addressing individual and systemic factors that impede care, furthering evaluation of programmes and the need to provide comprehensive care to PWID, involving medical care, social support and harm reduction.

CONCLUSIONS

These results demonstrate the need for expanded services across a variety of settings and populations. Our study emphasises the importance of addressing social and structural factors that impede infectious disease care for PWID. Further research is needed to improve evaluation of health programmes and services and contextual factors surrounding accessing services or returning to care.

PROSPERO REGISTRATION NUMBER

CRD42020142947.

Machine Learning Approaches to Retrieve High-Quality, Clinically Relevant Evidence From the Biomedical Literature: Systematic Review

BACKGROUND

The rapid growth of the biomedical literature makes identifying strong evidence a time-consuming task. Applying machine learning to the process could be a viable solution that limits effort while maintaining accuracy.

OBJECTIVE

The goal of the research was to summarize the nature and comparative performance of machine learning approaches that have been applied to retrieve high-quality evidence for clinical consideration from the biomedical literature.

METHODS

We conducted a systematic review of studies that applied machine learning techniques to identify high-quality clinical articles in the biomedical literature. Multiple databases were searched to July 2020. Extracted data focused on the applied machine learning model, steps in the development of the models, and model performance.

RESULTS

From 3918 retrieved studies, 10 met our inclusion criteria. All followed a supervised machine learning approach and applied, from a limited range of options, a high-quality standard for the training of their model. The results show that machine learning can achieve a sensitivity of 95% while maintaining a high precision of 86%.

CONCLUSIONS

Machine learning approaches perform well in retrieving high-quality clinical studies. Performance may improve by applying more sophisticated approaches such as active learning and unsupervised machine learning approaches.

eHealth Implementation Issues in Low-Resource Countries: Model, Survey, and Analysis of User Experience

BACKGROUND

The implementation of eHealth in low-resource countries (LRCs) is challenged by limited resources and infrastructure, lack of focus on eHealth agendas, ethical and legal considerations, lack of common system interoperability standards, unreliable power, and shortage of trained workers.

OBJECTIVE

The aim of this study is to describe and study the current situation of eHealth implementation in a small number of LRCs from the perspectives of their professional eHealth users.

METHODS

We developed a structural equation model that reflects the opinions of professional eHealth users who work on LRC health care front lines. We recruited country coordinators from 4 LRCs to help recruit survey participants: India, Egypt, Nigeria, and Kenya. Through a web-based survey that focused on barriers to eHealth implementation, we surveyed 114 participants. We analyzed the information using a structural equation model to determine the relationships among the constructs in the model, including the dependent variable, eHealth utilization.

RESULTS

Although all the model constructs were important to participants, some constructs, such as user characteristics, perceived privacy, and perceived security, did not play a significant role in eHealth utilization. However, the constructs related to technology infrastructure tended to reduce the impact of concerns and uncertainties (path coefficient=-0.32; P=.001), which had a negative impact on eHealth utilization (path coefficient=-0.24; P=.01). Constructs that were positively related to eHealth utilization were implementation effectiveness (path coefficient=0.45; P<.001), the countries where participants worked (path coefficient=0.29; P=.004), and whether they worked for privately or publicly funded institutions (path coefficient=0.18; P<.001). As exploratory research, the model had a moderately good fit for eHealth utilization (adjusted R2=0.42).

CONCLUSIONS

eHealth success factors can be categorized into 5 groups; our study focused on frontline eHealth workers' opinions concerning 2 of these groups: technology and its support infrastructure and user acceptance. We found significant disparities among the responses from different participant groups. Privately funded organizations tended to be further ahead with eHealth utilization than those that were publicly funded. Moreover, participant comments identified the need for more use of telemedicine in remote and rural regions in these countries. An understanding of these differences can help regions or countries that are lagging in the implementation and use of eHealth technologies. Our approach could also be applied to detailed studies of the other 3 categories of success factors: short- and long-term funding, organizational factors, and political or legislative aspects.

Feasibility of a Web-Based Platform (Trial My App) to Efficiently Conduct Randomized Controlled Trials of mHealth Apps For Patients With Cardiovascular Risk Factors: Protocol For Evaluating an mHealth App for Hypertension

Background

Mobile health (mHealth) interventions can improve health by improving cardiovascular risk factors, but their adoption in care by physicians and patients is untapped. Few mHealth apps have been evaluated in clinical trials, and due to the fast pace of technological development, those previously evaluated are often outdated by the time trial results are available. Given the rapid pace of change in this field, it is not feasible to rigorously evaluate mHealth apps with current methodologies.

Objective

The overall aim of this pilot study was to test the feasibility of using a web research platform called Trial My App to conduct efficient and rigorous web-based randomized controlled trials (RCTs) of mHealth apps relevant to patients with cardiovascular risk factors by evaluating an app that targets hypertension.

Methods

For this study, 200 participants with suboptimally controlled hypertension will be recruited through advertisements in newsletters, media, and the internet, as well as through referrals from their health care providers. Screening, consent, randomization, and collection of patient-important health confidence and self-management ability outcomes will be conducted online through the Trial My App research platform. Participants will be randomized into 2 groups: 100 that will use an mHealth app for tracking hypertension and 100 that will be considered as an educational control. All participants will complete questionnaires at 0, 1, 3 and 6 months after enrolment. A substudy to validate the method of blood pressure readings and the consistency of data entered through Trial My App will be conducted with 40 participants.

Results

The development of the Trial My App web platform has been completed. The creation of survey instruments has been completed in collaboration with our patient partners and advisory board. Recruitment is expected to begin in the first quarter of 2021; data collection and analysis are expected to be completed approximately 1 year after study commencement. Results will be disseminated through conferences and publications. The primary outcomes of this study include the feasibility of conducting an RCT using the Trial My App platform by reporting recruitment, retention, and completion statistics. We will validate app-entered data with a standard 7-day home blood pressure measurement method. Lastly, the pilot, nonblinded RCT will assess the effectiveness of the mHealth app in improving the control of hypertension compared with the control of hypertension in the educational control group.

Conclusions

This study will determine if it is feasible to use the Trial My App web-based platform to evaluate the effectiveness of mHealth apps for patients with cardiovascular risk factors. As more mHealth apps are evaluated in RCTs, patients will be able to select apps that meet their needs and physicians will be able to make evidence-based recommendations to their patients for apps aimed at improving cardiovascular health.

Trial Registration

ClinicalTrials.gov NCT04528654; https://clinicaltrials.gov/ct2/show/NCT04528654

International Registered Report Identifier (IRRID)

PRR1-10.2196/26155

Health programmes and services addressing the prevention and management of infectious diseases in persons who inject drugs in Canada: a systematic integrative review protocol

INTRODUCTION

Injection drug use (IDU) and intravenous drug use (IVDU) are of concern to the people using drugs, their families and health systems. One of the complications of IDU/IVDU is the risk of infection. Clinical experience has shown that persons who inject drugs (PWID) are hospitalised and re-hospitalised frequently. In Canada there are sparse data about the reasons for which PWID are admitted to hospital and their health trajectories, especially for infectious diseases. There are special concerns regarding PWID with infections who leave the hospital against medical advice and those who leave with a peripherally inserted central catheter line in place for administration of long-term antibiotics or other therapies. Improving our understanding of current programmes and services addressing the prevention and management of infectious diseases and their complications in PWID could lead to focused interventions to enhance care in this population.

METHODS AND ANALYSIS

An integrative systematic review allows for inclusion of a variety of methodologies to understand a health issue from different viewpoints. PubMed, CINAHL, Web of Science Databases and websites of the Public Health Agency of Canada, Canadian Institute for Substance Use Research, and Canadian Centre on Substance Use and Addiction will be searched using terms for infectious diseases, drug use and geography (Canada) and limited to the last 10 years (2009-2019). The Quality Appraisal Tool in Studies with Diverse Designs will be used to appraise the quality of identified studies and documents. Quantitative, qualitative or mixed methods data synthesis will be used as needed.

ETHICS AND DISSEMINATION

This study is a secondary analysis of publicly available documents; therefore, no ethics approval is required. This information will inform a research agenda to further investigate interventions that aim to address these issues.

PROSPERO REGISTRATION NUMBER

CRD42020142947.

Advice for Junior Faculty Regarding Academic Promotion: What Not to Worry About, and What to Worry About

Junior faculty in many universities must go through the promotion process to advance from entry level, e.g., assistant professorship to associate Professor, and ultimately to professorship. The process may often be stressful for some junior faculty, mostly due to some uncertainty about how to optimise their chances of successful promotion. In this paper, we summarise some strategies that would enhance their chances of a smooth promotion based on experiences from junior faculty and senior faculty who have served on tenure and promotion committees. These strategies include understanding the promotion process at your institution; optimizing publications as first or senior author, securing research funding as principal investigator, teaching effectively, providing service efficiently; developing good time management and priority setting skills, finding excellent mentors, and targeting opportunities for collaboration. We also encourage junior faculty to be pro-active about promotion.

Feasibility and implementation of a healthy lifestyles program in a community setting in Ontario, Canada: protocol for a pragmatic mixed methods pilot study

INTRODUCTION

Rates of chronic conditions, such as diabetes, cardiovascular disease and obesity are increasing in Canada and internationally. There are effective lifestyle interventions that are known to improve chronic conditions. However, there is often a gap in 'how to' make lifestyle changes. Mental health and other determinants of health play a role in the development and progression of chronic conditions. Changing habits takes time and requires the use of multiple techniques, including mental health and behavioural change strategies, based on a person's needs. A new, multidisciplinary, person-centred and evidence-based and practice-based programme has been created to address these needs. This proposal aims to evaluate the feasibility and implementation of this programme and to determine changes in participant-directed and clinical outcomes through a pilot study.

METHODS AND ANALYSIS

A pragmatic mixed methods design will be used to study multiple dimensions of the year-long healthy lifestyles programme. The pilot study includes a randomised controlled trial, with 30 participants randomised to either the programme or to a comparator arm, and qualitative components to determine the feasibility of the programme, including recruitment and retention, data missing rates and resources needed to run this programme. Changes in participant-directed and clinical outcomes will be measured. Descriptive statistics, t-tests and repeated measures analysis of variance (ANOVA) for within group comparisons and generalised estimating equations for between group analyses will be used. Qualitative interviews of programme staff and healthcare providers and family focus groups will be used to further enhance the findings and improve the programme.

ETHICS AND DISSEMINATION

Approval from the Hamilton Integrated Research Ethics Board (HiREB) has been obtained. Informed consent will be obtained prior to enrolling any participant into the study. Participant IDs will be used during data collection and entry. Peer-reviewed publications and presentations will target researchers, health professionals and stakeholders.

TRIAL REGISTRATION NUMBER

ClinicalTrials.gov Identifier: NCT03258138.HiREB project number: 3793.

Computer-Aided Detection for Breast Cancer Screening in Clinical Settings: Scoping Review

Background

With the growth of machine learning applications, the practice of medicine is evolving. Computer-aided detection (CAD) is a software technology that has become widespread in radiology practices, particularly in breast cancer screening for improving detection rates at earlier stages. Many studies have investigated the diagnostic accuracy of CAD, but its implementation in clinical settings has been largely overlooked.

Objective

The aim of this scoping review was to summarize recent literature on the adoption and implementation of CAD during breast cancer screening by radiologists and to describe barriers and facilitators for CAD use.

Methods

The MEDLINE database was searched for English, peer-reviewed articles that described CAD implementation, including barriers or facilitators, in breast cancer screening and were published between January 2010 and March 2018. Articles describing the diagnostic accuracy of CAD for breast cancer detection were excluded. The search returned 526 citations, which were reviewed in duplicate through abstract and full-text screening. Reference lists and cited references in the included studies were reviewed.

Results

A total of nine articles met the inclusion criteria. The included articles showed that there is a tradeoff between the facilitators and barriers for CAD use. Facilitators for CAD use were improved breast cancer detection rates, increased profitability of breast imaging, and time saved by replacing double reading. Identified barriers were less favorable perceptions of CAD compared to double reading by radiologists, an increase in recall rates of patients for further testing, increased costs, and unclear effect on patient outcomes.

Conclusions

There is a gap in the literature between CAD’s well-established diagnostic accuracy and its implementation and use by radiologists. Generally, the perceptions of radiologists have not been considered and details of implementation approaches for adoption of CAD have not been reported. The cost-effectiveness of CAD has not been well established for breast cancer screening in various populations. Further research is needed on how to best facilitate CAD in radiology practices in order to optimize patient outcomes, and the views of radiologists need to be better considered when advancing CAD use.

AIMD - a validated, simplified framework of interventions to promote and integrate evidence into health practices, systems, and policies

Background

Proliferation of terms describing the science of effectively promoting and supporting the use of research evidence in healthcare policy and practice has hampered understanding and development of the field. To address this, an international Terminology Working Group developed and published a simplified framework of interventions to promote and integrate evidence into health practices, systems, and policies. This paper presents results of validation work and a second international workgroup meeting, culminating in the updated AIMD framework [Aims, Ingredients, Mechanism, Delivery].

Methods

Framework validity was evaluated against terminology schemas (n = 51); primary studies (n = 37); and reporting guidelines (n = 10). Framework components were independently categorized as fully represented, partly represented, or absent by two researchers. Opportunities to refine the framework were systematically recorded. A meeting of the expanded international Terminology Working Group updated the framework by reviewing and deliberating upon validation findings and refinement proposals.

Results

There was variation in representativeness of the components across the three types of literature, in particular for the component ‘causal mechanisms’. Analysis of primary studies revealed that representativeness of this concept lowered from 92 to 68% if only explicit, rather than explicit and non-explicit references to causal mechanisms were included.

All components were very well represented in reporting guidelines, however the level of description of these was lower than in other types of literature. Twelve opportunities were identified to improve the framework, 9 of which were operationalized at the meeting. The updated AIMD framework comprises four components: (1) Aims: what do you want your intervention to achieve and for whom? (2) Ingredients: what comprises the intervention? (3) Mechanisms: how do you propose the intervention will work? and (4) Delivery: how will you deliver the intervention?

Conclusions

The draft simplified framework was validated with reference to a wide range of relevant literature and improvements have enhanced useability. The AIMD framework could aid in the promotion of evidence into practice, remove barriers to understanding how interventions work, enhance communication of interventions and support knowledge synthesis. Future work needs to focus on developing and testing resources and educational initiatives to optimize use of the AIMD framework in collaboration with relevant end-user groups.

Electronic supplementary material

The online version of this article (doi:10.1186/s12874-017-0314-8) contains supplementary material, which is available to authorized users.

Limits of search filter development

OBJECTIVE

The research attempted to develop search filters for biomedical literature databases that improve retrieval of studies of clinical relevance for the nursing and rehabilitation professions.

METHODS

Diagnostic testing framework compared machine-culled and practitioner-nominated search terms with a hand-tagged clinical literature database.

RESULTS

We were unable to: (1) develop filters for nursing, likely because of the overlapping and expanding scope of practice for nurses in comparison with medical professionals, or (2) develop filters for rehabilitation, because of its broad scope and the profession's multifaceted understanding of "health and ability."

CONCLUSIONS

We found limitations on search filter development for these health professions: nursing and rehabilitation.

A scoping review of classification schemes of interventions to promote and integrate evidence into practice in healthcare

BACKGROUND

Many models and frameworks are currently used to classify or describe knowledge translation interventions to promote and integrate evidence into practice in healthcare.

METHODS

We performed a scoping review of intervention classifications in public health, clinical medicine, nursing, policy, behaviour science, improvement science and psychology research published to May 2013 by searching MEDLINE, PsycINFO, CINAHL and the grey literature. We used five stages to map the literature: identifying the research question; identifying relevant literature; study selection; charting the data; collating, summarizing, and reporting results.

RESULTS

We identified 51 diverse classification schemes, including 23 taxonomies, 15 frameworks, 8 intervention lists, 3 models and 2 other formats. Most documents were public health based, 55% included a literature or document review, and 33% were theory based.

CONCLUSIONS

This scoping review provides an overview of schemes used to classify interventions which can be used for evaluation, comparison and validation of existing and emerging models. The collated taxonomies can guide authors in describing interventions; adequate descriptions of interventions will advance the science of knowledge translation in healthcare.

Towards a common terminology: a simplified framework of interventions to promote and integrate evidence into health practices, systems, and policies

BACKGROUND

A wide range of diverse and inconsistent terminology exists in the field of knowledge translation. This limits the conduct of evidence syntheses, impedes communication and collaboration, and undermines knowledge translation of research findings in diverse settings. Improving uniformity of terminology could help address these challenges. In 2012, we convened an international working group to explore the idea of developing a common terminology and an overarching framework for knowledge translation interventions.

FINDINGS

Methods included identifying and summarizing existing frameworks, mapping together a subset of those frameworks, and convening a multi-disciplinary group to begin working toward consensus. The group considered four potential approaches to creating a simplified framework: melding existing taxonomies, creating a framework of intervention mechanisms rather than intervention strategies, using a consensus process to expand one of the existing models/frameworks used by the group, or developing a new consensus framework.

CONCLUSIONS

The work group elected to draft a new, simplified consensus framework of interventions to promote and integrate evidence into health practices, systems and policies. The framework will include four key components: strategies and techniques (active ingredients), how they function (causal mechanisms), how they are delivered (mode of delivery), and what they aim to change (intended targets). The draft framework needs to be further developed by feedback and consultation with the research community and tested for usefulness through application and evaluation.

Engaging with a Wiki related to knowledge translation: a survey of WhatisKT Wiki users

Background

In 2008, WhatisKT wiki was launched as a collaborative platform for knowledge translation (KT) researchers and stakeholders to debate the use and definitions of KT-related terms. The wiki has definitions for over 110 terms from disciplines including health care, information technology, education, accounting, and business. WhatisKT wiki has over 115 registered users. Approximately 73,000 unique visitors have visited the wiki since 2008. Despite annual increases in visitors and regular maintenance of the wiki, no visitors have contributed content or started a discussion.

Objective

We surveyed wiki users to gain an understanding of the perceived value of the website, reasons for not engaging in the wiki, and suggestions to facilitate collaboration and improve the usability of the wiki.

Methods

We surveyed three cohorts: KT Canada members who were previously invited to join the wiki, registered wiki members, and unregistered visitors. The first two cohorts completed a Web-based survey that included the System Usability Scale (SUS) questionnaire to assess usability; additionally 3 participants were interviewed. Unregistered wiki visitors were surveyed with polls posted on the wiki. The study received ethics approval from the McMaster University Faculty of Health Sciences Research Ethics Board.

Results

Twenty-three participants completed the Web-based and SUS surveys; 15 participants indicated that they would collaborate on the wiki. The mean SUS score of 67 (95% CI 56-77) indicated that the wiki could be considered for design improvements. Study participants indicated that the wiki could be improved by email notification regarding new terms, better grouping of terms, user friendly interface, and training for users interested in editing content.

Conclusions

The findings from this survey will be used to enhance the design and content of WhatisKT wiki. Further feedback from participants will be used to make the wiki an ideal collaboration platform for KT researchers interested in terminology.

Net improvement of correct answers to therapy questions after pubmed searches: pre/post comparison

BACKGROUND

Clinicians search PubMed for answers to clinical questions although it is time consuming and not always successful.

OBJECTIVE

To determine if PubMed used with its Clinical Queries feature to filter results based on study quality would improve search success (more correct answers to clinical questions related to therapy).

METHODS

We invited 528 primary care physicians to participate, 143 (27.1%) consented, and 111 (21.0% of the total and 77.6% of those who consented) completed the study. Participants answered 14 yes/no therapy questions and were given 4 of these (2 originally answered correctly and 2 originally answered incorrectly) to search using either the PubMed main screen or PubMed Clinical Queries narrow therapy filter via a purpose-built system with identical search screens. Participants also picked 3 of the first 20 retrieved citations that best addressed each question. They were then asked to re-answer the original 14 questions.

RESULTS

We found no statistically significant differences in the rates of correct or incorrect answers using the PubMed main screen or PubMed Clinical Queries. The rate of correct answers increased from 50.0% to 61.4% (95% CI 55.0%-67.8%) for the PubMed main screen searches and from 50.0% to 59.1% (95% CI 52.6%-65.6%) for Clinical Queries searches. These net absolute increases of 11.4% and 9.1%, respectively, included previously correct answers changing to incorrect at a rate of 9.5% (95% CI 5.6%-13.4%) for PubMed main screen searches and 9.1% (95% CI 5.3%-12.9%) for Clinical Queries searches, combined with increases in the rate of being correct of 20.5% (95% CI 15.2%-25.8%) for PubMed main screen searches and 17.7% (95% CI 12.7%-22.7%) for Clinical Queries searches.

CONCLUSIONS

PubMed can assist clinicians answering clinical questions with an approximately 10% absolute rate of improvement in correct answers. This small increase includes more correct answers partially offset by a decrease in previously correct answers.

WhatisKT wiki: a case study of a platform for knowledge translation terms and definitions--descriptive analysis

Background

More than a hundred terms, often with unclear definitions and varying emphases, are used by health research and practice communities across the world who are interested in getting the best possible evidence applied (e.g., knowledge translation, implementation science, diffusion of innovations, and technology transfer). This makes finding published evidence difficult and can result in reduced, misinterpreted, or challenging interactions among professionals. Open dialogue and interaction among various professionals is needed to achieve consolidation of vocabulary. We use case report methods to describe how we sought to build an online tool to present the range of terms and facilitate the dialogue process across groups and disciplines interested in harnessing research evidence for healthcare.

Methods

We used a wiki platform from Wikispaces to present the problem of terminology and make a case and opportunity for collaboration on usage. Wikis are web sites where communities of users can collaborate online to build content and discuss progress. We gathered terms related to getting research into practice, sought published definitions, and posted these on the wiki (WhatisKT http://whatiskt.wikispaces.com/). We built the wiki in mid-2008 and promoted it through various groups and publications. This report describes the content of the site, our promotion efforts, use of the site, and how the site was used for collaboration up to the end of 2011.

Results

The WhatisKT wiki site now includes more than 120 pages. Traffic to the site has increased substantially from an average of 200 monthly visits in 2008 to 1700 in 2011. Visitors from 143 countries viewed the wiki in 2011, compared with 12 countries in 2008. However, most use has been limited to short term accesses of about 40 seconds per visit, and discussion of consolidation and solidifying terminology is conspicuously absent.

Conclusions

Although considerable interest exists in the terms and definitions related to getting research into practice based on increasing numbers of accesses, use of the WhatisKT wiki site for anything beyond quick lookups was minimal. Additional efforts must be directed towards increasing the level of interaction among the members of the site to encourage collaboration on term use.

How current are leading evidence-based medical textbooks? An analytic survey of four online textbooks

BACKGROUND

The consistency of treatment recommendations of evidence-based medical textbooks with more recently published evidence has not been investigated to date. Inconsistencies could affect the quality of medical care.

OBJECTIVE

To determine the frequency with which topics in leading online evidence-based medical textbooks report treatment recommendations consistent with more recently published research evidence.

METHODS

Summarized treatment recommendations in 200 clinical topics (ie, disease states) covered in four evidence-based textbooks--UpToDate, Physicians' Information Education Resource (PIER), DynaMed, and Best Practice--were compared with articles identified in an evidence rating service (McMaster Premium Literature Service, PLUS) since the date of the most recent topic updates in each textbook. Textbook treatment recommendations were compared with article results to determine if the articles provided different, new conclusions. From these findings, the proportion of topics which potentially require updating in each textbook was calculated.

RESULTS

478 clinical topics were assessed for inclusion to find 200 topics that were addressed by all four textbooks. The proportion of topics for which there was 1 or more recently published articles found in PLUS with evidence that differed from the textbooks' treatment recommendations was 23% (95% CI 17-29%) for DynaMed, 52% (95% CI 45-59%) for UpToDate, 55% (95% CI 48-61%) for PIER, and 60% (95% CI 53-66%) for Best Practice (χ(2) (3)=65.3, P<.001). The time since the last update for each textbook averaged from 170 days (range 131-209) for DynaMed, to 488 days (range 423-554) for PIER (P<.001 across all textbooks).

CONCLUSIONS

In online evidence-based textbooks, the proportion of topics with potentially outdated treatment recommendations varies substantially.

How well are journal and clinical article characteristics associated with the journal impact factor? a retrospective cohort study

OBJECTIVE

Journal impact factor (JIF) is often used as a measure of journal quality. A retrospective cohort study determined the ability of clinical article and journal characteristics, including appraisal measures collected at the time of publication, to predict subsequent JIFs.

METHODS

Clinical research articles that passed methods quality criteria were included. Each article was rated for relevance and newsworthiness by 3 to 24 physicians from a panel of more than 4,000 practicing clinicians. The 1,267 articles (from 103 journals) were divided 60∶40 into derivation (760 articles) and validation sets (507 articles), representing 99 and 88 journals, respectively. A multiple regression model was produced determining the association of 10 journal and article measures with the 2007 JIF.

RESULTS

Four of the 10 measures were significant in the regression model: number of authors, number of databases indexing the journal, proportion of articles passing methods criteria, and mean clinical newsworthiness scores. With the number of disciplines rating the article, the 5 variables accounted for 61% of the variation in JIF (R(2) = 0.607, 95% CI 0.444 to 0.706, P<0.001).

CONCLUSION

For the clinical literature, measures of scientific quality and clinical newsworthiness available at the time of publication can predict JIFs with 60% accuracy.

Search filters can find some but not all knowledge translation articles in MEDLINE: an analytic survey

OBJECTIVE

Advances from health research are not well applied giving rise to over- and underuse of resources and inferior care. Knowledge translation (KT), actions and processes of getting research findings used in practice, can improve research application. The KT literature is difficult to find because of nonstandardized terminology, rapid evolution of the field, and it is spread across several domains. We created multiple search filters to retrieve KT articles from MEDLINE.

STUDY DESIGN AND SETTING

Analytic survey using articles from 12 journals tagged as having KT content and also as describing a KT application or containing a KT theory.

RESULTS

Of 2,594 articles, 579 were KT articles of which 201 were about KT applications and 152 about KT theory. Search filter sensitivity (retrieval efficiency) maximized at 83%-94% with specificity (no retrieval of irrelevant material) approximately 50%. Filter performances were enhanced with multiple terms, but these filters often had reduced specificity. Performance was higher for KT applications and KT theory articles. These filters can select KT material although many irrelevant articles also will be retrieved.

CONCLUSION

KT search filters were developed and tested, with good sensitivity but suboptimal specificity. Further research must improve their performance.

The effectiveness of integrated health information technologies across the phases of medication management: a systematic review of randomized controlled trials

OBJECTIVE

The US Agency for Healthcare Research and Quality funded an evidence report to address seven questions on multiple aspects of the effectiveness of medication management information technology (MMIT) and its components (prescribing, order communication, dispensing, administering, and monitoring).

MATERIALS AND METHODS

Medline and 11 other databases without language or date limitations to mid-2010. Randomized controlled trials (RCTs) assessing integrated MMIT were selected by two independent reviewers. Reviewers assessed study quality and extracted data. Senior staff checked accuracy.

RESULTS

Most of the 87 RCTs focused on clinical decision support and computerized provider order entry systems, were performed in hospitals and clinics, included primarily physicians and sometimes nurses but not other health professionals, and studied process changes related to prescribing and monitoring medication. Processes of care improved for prescribing and monitoring mostly in hospital settings, but the few studies measuring clinical outcomes showed small or no improvements. Studies were performed most frequently in the USA (n=63), Europe (n=16), and Canada (n=6).

DISCUSSION

Many studies had limited description of systems, installations, institutions, and targets of the intervention. Problems with methods and analyses were also found. Few studies addressed order communication, dispensing, or administering, non-physician prescribers or pharmacists and their MMIT tools, or patients and caregivers. Other study methods are also needed to completely understand the effects of MMIT.

CONCLUSIONS

Almost half of MMIT interventions improved the process of care, but few studies measured clinical outcomes. This large body of literature, although instructive, is not uniformly distributed across settings, people, medication phases, or outcomes.

The economics of health information technology in medication management: a systematic review of economic evaluations

OBJECTIVE

To conduct a systematic review and synthesis of the evidence surrounding the cost-effectiveness of health information technology (HIT) in the medication process.

MATERIALS AND METHODS

Peer-reviewed electronic databases and gray literature were searched to identify studies on HIT used to assist in the medication management process. Articles including an economic component were reviewed for further screening. For this review, full cost-effectiveness analyses, cost-utility analyses and cost-benefit analyses, as well as cost analyses, were eligible for inclusion and synthesis.

RESULTS

The 31 studies included were heterogeneous with respect to the HIT evaluated, setting, and economic methods used. Thus the data could not be synthesized, and a narrative review was conducted. Most studies evaluated computer decision support systems in hospital settings in the USA, and only five of the studied performed full economic evaluations.

DISCUSSION

Most studies merely provided cost data; however, useful economic data involves far more input. A full economic evaluation includes a full enumeration of the costs, synthesized with the outcomes of the intervention.

CONCLUSION

The quality of the economic literature in this area is poor. A few studies found that HIT may offer cost advantages despite their increased acquisition costs. However, given the uncertainty that surrounds the costs and outcomes data, and limited study designs, it is difficult to reach any definitive conclusion as to whether the additional costs and benefits represent value for money. Sophisticated concurrent prospective economic evaluations need to be conducted to address whether HIT interventions in the medication management process are cost-effective.

Personal health records: a scoping review

Electronic personal health record systems (PHRs) support patient centered healthcare by making medical records and other relevant information accessible to patients, thus assisting patients in health self-management. We reviewed the literature on PHRs including design, functionality, implementation, applications, outcomes, and benefits. We found that, because primary care physicians play a key role in patient health, PHRs are likely to be linked to physician electronic medical record systems, so PHR adoption is dependent on growth in electronic medical record adoption. Many PHR systems are physician-oriented, and do not include patient-oriented functionalities. These must be provided to support self-management and disease prevention if improvements in health outcomes are to be expected. Differences in patient motivation to use PHRs exist, but an overall low adoption rate is to be expected, except for the disabled, chronically ill, or caregivers for the elderly. Finally, trials of PHR effectiveness and sustainability for patient self-management are needed.

Retrieval of diagnostic and treatment studies for clinical use through PubMed and PubMed's Clinical Queries filters

OBJECTIVE

Clinical Queries filters were developed to improve the retrieval of high-quality studies in searches on clinical matters. The study objective was to determine the yield of relevant citations and physician satisfaction while searching for diagnostic and treatment studies using the Clinical Queries page of PubMed compared with searching PubMed without these filters.

MATERIALS AND METHODS

Forty practicing physicians, presented with standardized treatment and diagnosis questions and one question of their choosing, entered search terms which were processed in a random, blinded fashion through PubMed alone and PubMed Clinical Queries. Participants rated search retrievals for applicability to the question at hand and satisfaction.

RESULTS

For treatment, the primary outcome of retrieval of relevant articles was not significantly different between the groups, but a higher proportion of articles from the Clinical Queries searches met methodologic criteria (p=0.049), and more articles were published in core internal medicine journals (p=0.056). For diagnosis, the filtered results returned more relevant articles (p=0.031) and fewer irrelevant articles (overall retrieval less, p=0.023); participants needed to screen fewer articles before arriving at the first relevant citation (p<0.05). Relevance was also influenced by content terms used by participants in searching. Participants varied greatly in their search performance.

DISCUSSION

Clinical Queries filtered searches returned more high-quality studies, though the retrieval of relevant articles was only statistically different between the groups for diagnosis questions.

CONCLUSION

Retrieving clinically important research studies from Medline is a challenging task for physicians. Methodological search filters can improve search retrieval.

Enabling medication management through health information technology (Health IT)

OBJECTIVES

The objective of the report was to review the evidence on the impact of health information technology (IT) on all phases of the medication management process (prescribing and ordering, order communication, dispensing, administration and monitoring as well as education and reconciliation), to identify the gaps in the literature and to make recommendations for future research.

DATA SOURCES

We searched peer-reviewed electronic databases, grey literature, and performed hand searches. Databases searched included MEDLINE®, Embase, CINAHL (Cumulated Index to Nursing and Allied Health Literature), Cochrane Database of Systematic Reviews, International Pharmaceutical Abstracts, Compendex, Inspec (which includes IEEE Xplore), Library and Information Science Abstracts, E-Prints in Library and Information Science, PsycINFO, Sociological Abstracts, and Business Source Complete. Grey literature searching involved Internet searching, reviewing relevant Web sites, and searching electronic databases of grey literatures. AHRQ also provided all references in their e-Prescribing, bar coding, and CPOE knowledge libraries.

METHODS

Paired reviewers looked at citations to identify studies on a range of health IT used to assist in the medication management process (MMIT) during multiple levels of screening (titles and abstracts, full text and final review for assignment of questions and data abstrction). Randomized controlled trials and cohort, case-control, and case series studies were independently assessed for quality. All data were abstracted by one reviewer and examined by one of two different reviewers with content and methods expertise.

RESULTS

40,582 articles were retrieved. After duplicates were removed, 32,785 articles were screened at the title and abstract phase. 4,578 full text articles were assessed and 789 articles were included in the final report. Of these, 361 met only content criteria and were listed without further abstraction. The final report included data from 428 articles across the seven key questions. Study quality varied according to phase of medication management. Substantially more studies, and studies with stronger comparative methods, evaluated prescribing and monitoring. Clinical decision support systems (CDSS) and computerized provider order entry (CPOE) systems were studied more than any other application of MMIT. Physicians were more often the subject of evaluation than other participants. Other health care professionals, patients, and families are important but not studied as thoroughly as physicians. These nonphysicians groups often value different aspects of MMIT, have diverse needs, and use systems differently. Hospitals and ambulatory clinics were well-represented in the literature with less emphasis placed on long-term care facilities, communities, homes, and nonhospital pharmacies. Most studies evaluated changes in process and outcomes of use, usability, and knowledge, skills, and attitudes. Most showed moderate to substantial improvement with implementation of MMIT. Economics studies and those with clinical outcomes were less frequently studied. Those articles that did address economics and clinical outcomes often showed equivocal findings on the effectiveness and cost-effectiveness of MMIT systems. Qualitative studies provided evidence of strong perceptions, both positive and negative, of the effects of MMIT and unintended consequences. We found little data on the effects of forms of medications, conformity, standards, and open source status. Much descriptive literature discusses implementation issues but little strong evidence exists. Interest is strong in MMIT and more groups and institutions will implement systems in the next decades, especially with the Federal Government's push toward more health IT to support better and more cost-effective health care.

CONCLUSIONS

MMIT is well-studied, although on closer examination of the literature the evidence is not uniform across phases of medication management, groups of people involved, or types of MMIT. MMIT holds the promise of improved processes; clinical and economics studies and the understanding of sustainability issues are lacking.

Health information technology to facilitate communication involving health care providers, caregivers, and pediatric patients: a scoping review

BACKGROUND

Pediatric patients with health conditions requiring follow-up typically depend on a caregiver to mediate at least part of the necessary two-way communication with health care providers on their behalf. Health information technology (HIT) and its subset, information communication technology (ICT), are increasingly being applied to facilitate communication between health care provider and caregiver in these situations. Awareness of the extent and nature of published research involving HIT interventions used in this way is currently lacking.

OBJECTIVE

This scoping review was designed to map the health literature about HIT used to facilitate communication involving health care providers and caregivers (who are usually family members) of pediatric patients with health conditions requiring follow-up.

METHODS

Terms relating to care delivery, information technology, and pediatrics were combined to search MEDLINE, EMBASE, and CINAHL for the years 1996 to 2008. Eligible studies were selected after three rounds of duplicate screening in which all authors participated. Data regarding patient, caregiver, health care provider, HIT intervention, outcomes studied, and study design were extracted and maintained in a Microsoft Access database. Stage of research was categorized using the UK's Medical Research Council (MRC) framework for developing and evaluating complex interventions. Quantitative and qualitative descriptive summaries are presented.

RESULTS

We included 104 eligible studies (112 articles) conducted in 17 different countries and representing 30 different health conditions. The most common conditions were asthma, type 1 diabetes, special needs, and psychiatric disorder. Most studies (88, 85%) included children 2 to 12 years of age, and 73 (71%) involved home care settings. Health care providers operated in hospital settings in 96 (92%) of the studies. Interventions featured 12 modes of communication (eg, Internet, intranets, telephone, video conferencing, email, short message service [SMS], and manual downloading of information) used to facilitate 15 categories of functions (eg, support, medication management, education, and monitoring). Numerous patient, caregiver, and health care relevant outcomes have been measured. Most outcomes concerned satisfaction, use, usability, feasibility, and resource use, although behavior changes and quality of life were also reported. Most studies (57 studies, 55%) were pilot phase, with a lesser proportion of development phase (24 studies, 23%) and evaluation phase (11 studies, 11%) studies. HIT interventions addressed several recurring themes in this review: establishing continuity of care, addressing time constraints, and bridging geographical barriers.

CONCLUSIONS

HIT used in pediatric care involving caregivers has been implemented differently in a range of disease settings, with varying needs influencing the function, form and synchronicity of information transfer. Although some authors have followed a phased approach to development, evaluation and implementation, a greater emphasis on methodological standards such as the MRC guidance for complex interventions would produce more fruitful programs of development and more useful evaluations in the future. This review will be especially helpful to those deciding on areas where further development or research into HIT for this purpose may be warranted.

A cross-sectional study of the number and frequency of terms used to refer to knowledge translation in a body of health literature in 2006: a Tower of Babel?

BACKGROUND

The study of implementing research findings into practice is rapidly growing and has acquired many competing names (e.g., dissemination, uptake, utilization, translation) and contributing disciplines. The use of multiple terms across disciplines pose barriers to communication and progress for applying research findings. We sought to establish an inventory of terms describing this field and how often authors use them in a collection of health literature published in 2006.

METHODS

We refer to this field as knowledge translation (KT). Terms describing aspects of KT and their definitions were collected from literature, the internet, reports, textbooks, and contact with experts. We compiled a database of KT and other articles by reading 12 healthcare journals representing multiple disciplines. All articles published in these journals in 2006 were categorized as being KT or not. The KT articles (all KT) were further categorized, if possible, for whether they described KT projects or implementations (KT application articles), or presented the theoretical basis, models, tools, methods, or techniques of KT (KT theory articles). Accuracy was checked using duplicate reading. Custom designed software determined how often KT terms were used in the titles and abstracts of articles categorized as being KT.

RESULTS

A total of 2,603 articles were assessed, and 581 were identified as KT articles. Of these, 201 described KT applications, and 153 included KT theory. Of the 100 KT terms collected, 46 were used by the authors in the titles or abstracts of articles categorized as being KT. For all 581 KT articles, eight terms or term variations used by authors were highly discriminating for separating KT and non-KT articles (p < 0.001): implementation, adoption, quality improvement, dissemination, complex intervention (with multiple endings), implementation (within three words of) research, and complex intervention. More KT terms were associated with KT application articles (n = 13) and KT theory articles (n = 18).

CONCLUSIONS

We collected 100 terms describing KT research. Authors used 46 of them in titles and abstracts of KT articles. Of these, approximately half discriminated between KT and non-KT articles. Thus, the need for consolidation and consistent use of fewer terms related to KT research is evident.

Finding knowledge translation articles in CINAHL

BACKGROUND

The process of moving research into practice has a number of names including knowledge translation (KT). Researchers and decision makers need to be able to readily access the literature on KT for the field to grow and to evaluate the existing evidence.

METHODS

To develop and validate search filters for finding KT articles in the database Cumulative Index to Nursing and Allied Health (CINAHL). A gold standard database was constructed by hand searching and classifying articles from 12 journals as KT Content, KT Applications and KT Theory.

MAIN OUTCOME MEASURES

Sensitivity, specificity, precision, and accuracy of the search filters.

RESULTS

Optimized search filters had fairly low sensitivity and specificity for KT Content (58.4% and 64.9% respectively), while sensitivity and specificity increased for retrieving KT Application (67.5% and 70.2%) and KT Theory articles (70.4% and 77.8%).

CONCLUSION

Search filter performance was suboptimal marking the broad base of disciplines and vocabularies used by KT researchers. Such diversity makes retrieval of KT studies in CINAHL difficult.

Prediction of citation counts for clinical articles at two years using data available within three weeks of publication: retrospective cohort study

OBJECTIVE

To determine if citation counts at two years could be predicted for clinical articles that pass basic criteria for critical appraisal using data within three weeks of publication from external sources and an online article rating service.

DESIGN

Retrospective cohort study.

SETTING

Online rating service, Canada.

PARTICIPANTS

1274 articles from 105 journals published from January to June 2005, randomly divided into a 60:40 split to provide derivation and validation datasets.

MAIN OUTCOME MEASURES

20 article and journal features, including ratings of clinical relevance and newsworthiness, routinely collected by the McMaster online rating of evidence system, compared with citation counts at two years.

RESULTS

The derivation analysis showed that the regression equation accounted for 60% of the variation (R2=0.60, 95% confidence interval 0.538 to 0.629). This model applied to the validation dataset gave a similar prediction (R2=0.56, 0.476 to 0.596, shrinkage 0.04; shrinkage measures how well the derived equation matches data from the validation dataset). Cited articles in the top half and top third were predicted with 83% and 61% sensitivity and 72% and 82% specificity. Higher citations were predicted by indexing in numerous databases; number of authors; abstraction in synoptic journals; clinical relevance scores; number of cited references; and original, multicentred, and therapy articles from journals with a greater proportion of articles abstracted.

CONCLUSION

Citation counts can be reliably predicted at two years using data within three weeks of publication.

Which journals do primary care physicians and specialists access from an online service?

OBJECTIVE

The study sought to determine which online journals primary care physicians and specialists not affiliated with an academic medical center access and how the accesses correlate with measures of journal quality and importance.

METHODS

Observational study of full-text accesses made during an eighteen-month digital library trial was performed. Access counts were correlated with six methods composed of nine measures for assessing journal importance: ISI impact factors; number of high-quality articles identified during hand-searches of key clinical journals; production data for ACP Journal Club, InfoPOEMs, and Evidence-Based Medicine; and mean clinician-provided clinical relevance and newsworthiness scores for individual journal titles.

RESULTS

Full-text journals were accessed 2,322 times by 87 of 105 physicians. Participants accessed 136 of 348 available journal titles. Physicians often selected journals with relatively higher numbers of articles abstracted in ACP Journal Club. Accesses also showed significant correlations with 6 other measures of quality. Specialists' access patterns correlated with 3 measures, with weaker correlations than for primary care physicians.

CONCLUSIONS

Primary care physicians, more so than specialists, chose full-text articles from clinical journals deemed important by several measures of value. Most journals accessed by both groups were of high quality as measured by this study's methods for assessing journal importance.

Seed output and the seed bank in Vallisneria americana (Hydrocharitaceae)

Seed banks and sexual reproduction are known to be significant in colonization and re-establishment of some aquatic macrophyte communities. For highly clonal aquatic macrophytes, however, there is a lack of information on seed production and seed fate as compared with annual sexual species. The seed bank for three populations of Vallisneria americana in the Huron-Erie corridor of the Great Lakes was sampled and quantified in the spring of 1994, and related to seed production in the previous season at these sites. Seed deposition rates during 1994 were also assessed. Sites varied in the proportion of plants flowering and in their tertiary sex ratios, but did not differ in seed numbers produced per unit area. The size of the seed bank was not significantly related to the previous season's seed output, and estimates of seed deposition in the following year tended to be approximately tenfold greater than seed densities found in the seed bank. The stages between seed production and subsequent seed germination are generally very dynamic, with dispersal, mortality, and predation as likely regulating factors. The potential for seedling establishment in V. americana needs to be assessed more fully before the role of seeds in population processes can be determined.