Are we measuring the right end-points? Variables that affect the impact of computerised decision support on patient outcomes: a systematic review

The effects of cancer clinical decision support systems on patient-reported outcomes: A systematic review

PURPOSE

The implementation of high-quality decision-making support are integral to ensuring the delivery of quality cancer care and subsequently achieving positive patient outcomes. Decision Support Systems (DSS) are increasingly used, however it is not known what the effects are beyond supporting the decision-making process. We aimed to identify and synthesize the available literature regarding the effects of DSS on patient-reported outcomes both during and after cancer treatment.

METHODS

A systematic review was conducted using dual processes to identify empirical literature that reported an evaluation of DSS interventions and patient-reported outcomes. We appraised study quality using the Mixed Methods Appraisal Tool (MMAT). Data were narratively synthesized.

RESULTS

We included 15 studies, categorized as symptom assessment interventions or interactive educational interventions. Findings were mixed regarding the effectiveness of DSS interventions in improving total symptom distress and severity, whereas the majority were effective in reducing mean scores for worst and usual pain. Interventions were not effective in improving other health-related patient-reported outcomes including quality of life, global distress, depression, or self-efficacy and there were mixed effects for reducing decisional conflict. There was moderate to high patient adherence to the interventions and generally high satisfaction and acceptability, yet minimal evidence for the effect of DSS interventions in clinician adherence to intervention recommendations.

CONCLUSIONS

Including patient-reported outcomes in the evaluation of DSS is critical to understand their impact. Inconsistencies in reporting of interventions may, however, be a contributing factor to heterogeneous effects of clinical DSS regarding a broad range of patient-reported outcomes.

Clinical decision support systems-based interventions to improve medication outcomes: A systematic literature review on features and effects

Background: Clinical decision support systems (CDSSs) interventions were used to improve the life quality and safety in patients and also to improve practitioner performance, especially in the field of medication. Therefore, the aim of the paper was to summarize the available evidence on the impact, outcomes and significant factors on the implementation of CDSS in the field of medicine. Methods: This study is a systematic literature review. PubMed, Cochrane Library, Web of Science, Scopus, EMBASE, and ProQuest were investigated by 15 February 2017. The inclusion requirements were met by 98 papers, from which 13 had described important factors in the implementation of CDSS, and 86 were medicated-related. We categorized the system in terms of its correlation with medication in which a system was implemented, and our intended results were examined. In this study, the process outcomes (such as; prescription, drug-drug interaction, drug adherence, etc.), patient outcomes, and significant factors affecting the implementation of CDSS were reviewed. Results: We found evidence that the use of medication-related CDSS improves clinical outcomes. Also, significant results were obtained regarding the reduction of prescription errors, and the improvement in quality and safety of medication prescribed. Conclusion: The results of this study show that, although computer systems such as CDSS may cause errors, in most cases, it has helped to improve prescribing, reduce side effects and drug interactions, and improve patient safety. Although these systems have improved the performance of practitioners and processes, there has not been much research on the impact of these systems on patient outcomes.

An Electronic Medical Record-Derived Individualized Performance Metric to Measure Risk-Adjusted Adherence with Perioperative Prophylactic Bundles for Health Care Disparity Research and Implementation Science

BACKGROUND

Health care disparity persists despite vigorous countermeasures. Clinician performance is paramount for equitable care processes and outcomes. However, precise and valid individual performance measures remain elusive.

OBJECTIVES

We sought to develop a generalizable, rigorous, risk-adjusted metric for individual clinician performance (MIP) derived directly from the electronic medical record (EMR) to provide visual, personalized feedback.

METHODS

We conceptualized MIP as risk responsiveness, i.e., administering an increasing number of interventions contingent on patient risk. We embedded MIP in a hierarchical statistical model, reflecting contemporary nested health care delivery. We tested MIP by investigating the adherence with prophylactic bundles to reduce the risk of postoperative nausea and vomiting (PONV), retrieving PONV risk factors and prophylactic antiemetic interventions from the EMR. We explored the impact of social determinants of health on MIP.

RESULTS

We extracted data from the EMR on 25,980 elective anesthesia cases performed at Penn State Milton S. Hershey Medical Center between June 3, 2018 and March 31, 2019. Limiting the data by anesthesia Current Procedural Terminology code and to complete cases with PONV risk and antiemetic interventions, we evaluated the performance of 83 anesthesia clinicians on 2,211 anesthesia cases. Our metric demonstrated considerable variance between clinicians in the adherence to risk-adjusted utilization of antiemetic interventions. Risk seemed to drive utilization only in few clinicians. We demonstrated the impact of social determinants of health on MIP, illustrating its utility for health science and disparity research.

CONCLUSION

The strength of our novel measure of individual clinician performance is its generalizability, as well as its intuitive graphical representation of risk-adjusted individual performance. However, accuracy, precision and validity, stability over time, sensitivity to system perturbations, and acceptance among clinicians remain to be evaluated.

The Effects of Integrated IT Support on the Prehospital Stroke Process: Results from a Realistic Experiment

Stroke is a serious condition and the stroke chain of care is a complex. The present study aims to explore the impact of a computerised decision support system (CDSS) for the prehospital stroke process, with focus on work processes and performance. The study used an exploratory approach with a randomised controlled crossover design in a realistic contextualised simulation experiment. The study compared clinical performance among 11 emergency medical services (EMS) teams of 22 EMS clinicians using (1) a computerised decision support system (CDSS) and (2) their usual paper-based process support. Data collection consisted of video recordings, postquestionnaires and post-interviews, and data were analysed using a combination of qualitative and quantitative approaches. In this experiment, using a CDSS improved patient assessment, decision making and compliance to process recommendations. Minimal impact of the CDSS was found on EMS clinicians’ self-efficacy, suggesting that even though the system was found to be cumbersome to use it did not have any negative effects on self-efficacy. Negative effects of the CDSS include increased on-scene time and a cognitive burden of using the system, affecting patient interaction and collaboration with team members. The CDSS’s overall process advantage to the prehospital stroke process is assumed to lead to a prehospital care that is both safer and of higher quality. The key to user acceptance of a system such as this CDSS is the relative advantages of improved documentation process and the resulting patient journal. This could improve the overall prehospital stroke process.

A systematic review of trials evaluating success factors of interventions with computerised clinical decision support

BACKGROUND

Computerised clinical decision support (CDS) can potentially better inform decisions, and it can help with the management of information overload. It is perceived to be a key component of a learning health care system. Despite its increasing implementation worldwide, it remains uncertain why the effect of CDS varies and which factors make CDS more effective.

OBJECTIVE

To examine which factors make CDS strategies more effective on a number of outcomes, including adherence to recommended practice, patient outcome measures, economic measures, provider or patient satisfaction, and medical decision quality.

METHODS

We identified randomised controlled trials, non-randomised trials, and controlled before-and-after studies that directly compared CDS implementation with a given factor to CDS without that factor by searching CENTRAL, MEDLINE, EMBASE, and CINAHL and checking reference lists of relevant studies. We considered CDS with any objective for any condition in any healthcare setting. We included CDS interventions that were either displayed on screen or provided on paper and that were directed at healthcare professionals or targeted at both professionals and patients. The reviewers screened the potentially relevant studies in duplicate. They extracted data and assessed risk of bias in independent pairs or individually followed by a double check by another reviewer. We summarised results using medians and interquartile ranges and rated our certainty in the evidence using the GRADE system.

RESULTS

We identified 66 head-to-head trials that we synthesised across 14 comparisons of CDS intervention factors. Providing CDS automatically versus on demand led to large improvements in adherence. Displaying CDS on-screen versus on paper led to moderate improvements and making CDS more versus less patient-specific improved adherence modestly. When CDS interventions were combined with professional-oriented strategies, combined with patient-oriented strategies, or combined with staff-oriented strategies, then adherence improved slightly. Providing CDS to patients slightly increased adherence versus CDS aimed at the healthcare provider only. Making CDS advice more explicit and requiring users to respond to the advice made little or no difference. The CDS intervention factors made little or no difference to patient outcomes. The results for economic outcomes and satisfaction outcomes were sparse.

CONCLUSION

Multiple factors may affect the success of CDS interventions. CDS may be more effective when the advice is provided automatically and displayed on-screen and when the suggestions are more patient-specific. CDS interventions combined with other strategies probably also improves adherence. Providing CDS directly to patients may also positively affect adherence. The certainty of the evidence was low to moderate for all factors.

TRIAL REGISTRATION

PROSPERO, CRD42016033738.

Effectiveness of self-management support interventions for people with comorbid diabetes and chronic kidney disease: a systematic review and meta-analysis

BACKGROUND

Self-management support interventions may potentially delay kidney function decline and associated complications in patients with comorbid diabetes and chronic kidney disease. However, the effectiveness of these interventions remains unclear. We investigated the effectiveness of current self-management support interventions and their specific components and elements in improving patient outcomes.

METHODS

Electronic databases were systematically searched from January 1, 1994, to December 19, 2017. Eligible studies were randomized controlled trials on self-management support interventions for adults with comorbid diabetes and chronic kidney disease. Primary outcomes were systolic blood pressure, diastolic blood pressure, estimated glomerular filtration rate, and glycated hemoglobin. Secondary outcomes included self-management activity, health service utilization, health-related quality of life, medication adherence, and death.

RESULTS

Of the 48 trials identified, eight studies (835 patients) were eligible. There was moderate-quality evidence that self-management support interventions improved self-management activity (standard mean difference 0.56, 95% CI 0.15 to 0.97, p < 0.007) compared to usual care. There was low-quality evidence that self-management support interventions reduced systolic blood pressure (mean difference - 4.26 mmHg, 95% CI - 7.81 to - 0.70, p = 0.02) and glycated hemoglobin (mean difference - 0.5%, 95% CI - 0.8 to - 0.1, p = 0.01) compared to usual care.

CONCLUSIONS

Self-management support interventions may improve self-care activities, systolic blood pressure, and glycated hemoglobin in patients with comorbid diabetes and chronic kidney disease. It was not possible to determine which self-management components and elements were more effective, but interventions that utilized provider reminders, patient education, and goal setting were associated with improved outcomes. More evidence from high-quality studies is required to support future self-management programs.

SYSTEMATIC REVIEW REGISTRATION

PROSPERO CRD42015017316 .

Impact on process results of clinical decision support systems (CDSSs) applied to medication use: overview of systematic reviews

Objective

The purpose of this overview (systematic review of systematic reviews) is to evaluate the impact of clinical decision support systems (CDSS) applied to medication use in the care process.

Methods

A search for systematic reviews that address CDSS was performed on Medline following Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) and Cochrane recommendations. Terms related to CDSS and systematic reviews were used in combination with Boolean operators and search field tags to build the electronic search strategy. There was no limitation of date or language for inclusion. We included revisions that investigated, as a main or secondary objective, changes in process outcomes. The Revised Assessment of Multiple Systematic Reviews (R-AMSTAR) score was used to evaluate the quality of the studies.

Results

The search retrieved 954 articles. Five articles were added through manual search, totaling an initial sample of 959 articles. After screening and reading in full, 44 systematic reviews met the inclusion criteria. In the medication-use processes where CDSS was used, the most common stages were prescribing (n=38 (86.36%) and administering (n=12 (27.27%)). Most of the systematic reviews demonstrated improvement in the health care process (30/44 - 68.2%). The main positive results were related to improvement of the quality of prescription by the physicians (14/30 - 46.6%) and reduction of errors in prescribing (5/30 - 16.6%). However, the quality of the studies was poor, according to the score used.

Conclusion

CDSSs represent a promising technology to optimize the medication-use process, especially related to improvement in the quality of prescriptions and reduction of prescribing errors, although higher quality studies are needed to establish the predictors of success in these systems.

Improving specialist drug prescribing in primary care using task and error analysis: an observational study

BACKGROUND

Electronic prescribing has benefited from computerised clinical decision support systems (CDSSs); however, no published studies have evaluated the potential for a CDSS to support GPs in prescribing specialist drugs.

AIM

To identify potential weaknesses and errors in the existing process of prescribing specialist drugs that could be addressed in the development of a CDSS.

DESIGN AND SETTING

Semi-structured interviews with key informants followed by an observational study involving GPs in the UK.

METHOD

Twelve key informants were interviewed to investigate the use of CDSSs in the UK. Nine GPs were observed while performing case scenarios depicting requests from hospitals or patients to prescribe a specialist drug. Activity diagrams, hierarchical task analysis, and systematic human error reduction and prediction approach analyses were performed.

RESULTS

The current process of prescribing specialist drugs by GPs is prone to error. Errors of omission due to lack of information were the most common errors, which could potentially result in a GP prescribing a specialist drug that should only be prescribed in hospitals, or prescribing a specialist drug without reference to a shared care protocol. Half of all possible errors in the prescribing process had a high probability of occurrence.

CONCLUSION

A CDSS supporting GPs during the process of prescribing specialist drugs is needed. This could, first, support the decision making of whether or not to undertake prescribing, and, second, provide drug-specific parameters linked to shared care protocols, which could reduce the errors identified and increase patient safety.

Effects of health information technology on patient outcomes: a systematic review

OBJECTIVE

To systematically review studies assessing the effects of health information technology (health IT) on patient safety outcomes.

MATERIALS AND METHODS

The authors employed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement methods. MEDLINE, Cumulative Index to Nursing Allied Health (CINAHL), EMBASE, and Cochrane Library databases, from 2001 to June 2012, were searched. Descriptive and comparative studies were included that involved use of health IT in a clinical setting and measured effects on patient safety outcomes.

RESULTS

Data on setting, subjects, information technology implemented, and type of patient safety outcomes were all abstracted. The quality of the studies was evaluated by 2 independent reviewers (scored from 0 to 10). A total of 69 studies met inclusion criteria. Quality scores ranged from 1 to 9. There were 25 (36%) studies that found benefit of health IT on direct patient safety outcomes for the primary outcome measured, 43 (62%) studies that either had non-significant or mixed findings, and 1 (1%) study for which health IT had a detrimental effect. Neither the quality of the studies nor the rate of randomized control trials performed changed over time. Most studies that demonstrated a positive benefit of health IT on direct patient safety outcomes were inpatient, single-center, and either cohort or observational trials studying clinical decision support or computerized provider order entry.

DISCUSSION AND CONCLUSION

Many areas of health IT application remain understudied and the majority of studies have non-significant or mixed findings. Our study suggests that larger, higher quality studies need to be conducted, particularly in the long-term care and ambulatory care settings.

Reducing diagnostic errors in primary care. A systematic meta-review of computerized diagnostic decision support systems by the LINNEAUS collaboration on patient safety in primary care

Background: Computerized diagnostic decision support systems (CDDSS) have the potential to support the cognitive task of diagnosis, which is one of the areas where general practitioners have greatest difficulty and which accounts for a significant proportion of adverse events recorded in the primary care setting.

Objective: To determine the extent to which CDDSS may meet the requirements of supporting the cognitive task of diagnosis, and the currently perceived barriers that prevent the integration of CDDSS with electronic health record (EHR) systems.

Methods: We conducted a meta-review of existing systematic reviews published in English, searching MEDLINE, Embase, PsycINFO and Web of Knowledge for articles on the features and effectiveness of CDDSS for medical diagnosis published since 2004. Eligibility criteria included systematic reviews where individual clinicians were primary end users. Outcomes we were interested in were the effectiveness and identification of specific features of CDDSS on diagnostic performance.

Results: We identified 1970 studies and excluded 1938 because they did not fit our inclusion criteria. A total of 45 articles were identified and 12 were found suitable for meta-review. Extraction of high-level requirements identified that a more standardized computable approach is needed to knowledge representation, one that can be readily updated as new knowledge is gained. In addition, a deep integration with the EHR is needed in order to trigger at appropriate points in cognitive workflow.

Conclusion: Developing a CDDSS that is able to utilize dynamic vocabulary tools to quickly capture and code relevant diagnostic findings, and coupling these with individualized diagnostic suggestions based on the best-available evidence has the potential to improve diagnostic accuracy, but requires evaluation.

Outpatient diabetes clinical decision support: current status and future directions

Outpatient clinical decision support systems have had an inconsistent impact on key aspects of diabetes care. A principal barrier to success has been low use rates in many settings. Here, we identify key aspects of clinical decision support system design, content and implementation that are related to sustained high use rates and positive impacts on glucose, blood pressure and lipid management. Current diabetes clinical decision support systems may be improved by prioritizing care recommendations, improving communication of treatment-relevant information to patients, using such systems for care coordination and case management and integrating patient-reported information and data from remote devices into clinical decision algorithms and interfaces.

Systematic review of clinical decision support interventions with potential for inpatient cost reduction

Background

Healthcare costs are increasing rapidly and at an unsustainable rate in many countries, and inpatient hospitalizations are a significant driver of these costs. Clinical decision support (CDS) represents a promising approach to not only improve care but to reduce costs in the inpatient setting. The purpose of this study was to systematically review trials of CDS interventions with the potential to reduce inpatient costs, so as to identify promising interventions for more widespread implementation and to inform future research in this area.

Methods

To identify relevant studies, MEDLINE was searched up to July 2013. CDS intervention studies with the potential to reduce inpatient healthcare costs were identified through titles and abstracts, and full text articles were reviewed to make a final determination on inclusion. Relevant characteristics of the studies were extracted and summarized.

Results

Following a screening of 7,663 articles, 78 manuscripts were included. 78.2% of studies were controlled before-after studies, and 15.4% were randomized controlled trials. 53.8% of the studies were focused on pharmacotherapy. The majority of manuscripts were published during or after 2008. 70.5% of the studies resulted in statistically and clinically significant improvements in an explicit financial measure or a proxy financial measure. Only 12.8% of the studies directly measured the financial impact of an intervention, whereas the financial impact was inferred in the remainder of studies. Data on cost effectiveness was available for only one study.

Conclusions

Significantly more research is required on the impact of clinical decision support on inpatient costs. In particular, there is a remarkable gap in the availability of cost effectiveness studies required by policy makers and decision makers in healthcare systems.

The utility of an online diagnostic decision support system (Isabel) in general practice: a process evaluation

OBJECTIVES

To evaluate the utility of Isabel, an online diagnostic decision support system developed by Isabel Healthcare primarily for secondary medical care, in the general practice setting.

DESIGN

Focus groups were conducted with clinicians to understand why and how they used the system. A modified online post-use survey asked practitioners about its impact on their decision-making. Normalization process theory (NPT) was used as a theoretical framework to determine whether the system could be incorporated into routine clinical practice.

SETTING

The system was introduced by NHS County Durham and Darlington in the UK in selected general practices as a three-month pilot.

PARTICIPANTS

General practitioners and nurse practitioners who had access to Isabel as part of the Primary Care Trust's pilot.

MAIN OUTCOME MEASURES

General practitioners' views, experiences and usage of the system.

RESULTS

Seven general practices agreed to pilot Isabel. Two practices did not subsequently use it. The remaining five practices conducted searches on 16 patients. Post-use surveys (n = 10) indicated that Isabel had little impact on diagnostic decision-making. Focus group participants stated that, although the diagnoses produced by Isabel in general did not have an impact on their decision-making, they would find the tool useful if it were better tailored to the primary care setting. Our analysis concluded that normalization was not likely to occur in its current form.

CONCLUSIONS

Isabel was of limited utility in this short pilot study and may need further modification for use in general practice.

Health care worker perspectives inform optimization of patient panel-support tools: a qualitative study

Electronic decision-support systems appear to enhance care, but improving both tools and work practices may optimize outcomes. Using qualitative methods, the authors' aim was to evaluate perspectives about using the Patient Panel-Support Tool (PST) to better understand health care workers' attitudes toward, and adoption and use of, a decision-support tool. In-depth interviews were conducted to elicit participant perspectives about the PST-an electronic tool implemented in 2006 at Kaiser Permanente Northwest. The PST identifies "care gaps" and recommendations in screening, medication use, risk-factor control, and immunizations for primary care panel patients. Primary care physician (PCP) teams were already grouped (based on performance pre- and post-PST introduction) into lower, improving, and higher percent-of-care-needs met. Participants were PCPs (n=21), medical assistants (n=11), and quality and other health care managers (n=20); total n=52. Results revealed that the most commonly cited benefit of the PST was increased in-depth knowledge of patient panels, and empowerment of staff to do quality improvement. Barriers to PST use included insufficient time, competing demands, suboptimal staffing, tool navigation, documentation, and data issues. Facilitators were strong team staff roles, leadership/training for tool implementation, and dedicated time for tool use. Higher performing PCPs and their assistants more often described a detailed team approach to using the PST. In conclusion, PCP teams and managers provided important perspectives that could help optimize use of panel-support tools to improve future outcomes. Improvements are needed in tool function and navigation; training; staff accountability and role clarification; and panel management time.

Efficacy and safety of a TIA/stroke electronic support tool (FASTEST) trial: study protocol

BACKGROUND

Strokes are a common cause of adult disability and mortality worldwide. Transient ischaemic attacks (TIA) are associated with a high risk of subsequent stroke, and rapid intervention has the potential to reduce stroke burden. This study will assess a novel electronic decision support (EDS) tool to allow general practitioners (GPs) to implement evidence-based care rapidly without full reliance on specialists.

METHODS/DESIGN

This is a cluster randomized controlled trial comparing TIA/stroke management of GPs with access to the EDS tool versus usual care. The intervention period is 12 months with a 3-month follow-up period for individual patients. Primary outcomes consist of stroke within 90 days of presenting event and adherence to the New Zealand national TIA guideline.

DISCUSSION

A positive study will provide strong evidence for widespread implementation of this tool in practice and has the potential to improve key outcomes for patients and reduce the burden of stroke.

TRIAL REGISTRATION

Australia New Zealand Clinical Trials Registry ACTRN12611000792921.

Examining the evidence of the impact of health information technology in primary care: an argument for participatory research with health professionals and patients

PURPOSE

Health information technology represents a promising avenue to improve health care delivery. How can we use lessons learnt from existing health information technologies in primary care to inform the optimal design of newer developments such as personal health records?

METHODS

The results of systematic literature reviews about the impact of different information systems on health outcomes in primary care are critically discussed in a narrative synthesis, with a focus on their implications for the development of personal health records.

RESULTS

Given the proliferation of systematic reviews and randomized controlled trials, high quality evidence for health information technology in primary care is accumulating with mixed results. The heterogeneity of systems being compared and the quality of research can no longer account for these findings. One potential explanation may be that systems originally designed for acute care settings are being implemented in primary care. Early studies evaluating personal health records suggest that targeting patient outcomes directly and adapting systems to patients' needs may be part of the solution.

CONCLUSION

In order to develop personal health records for primary care, studies are needed that involve the users, namely patients and primary care health professionals, in the design and evaluation of these systems from their inception. Participatory research is a recommended methodological approach.

Overview of systematic reviews of the effectiveness of reminders in improving healthcare professional behavior

OBJECTIVE

The purpose of this project was to conduct an overview of existing systematic reviews to evaluate the effectiveness of reminders in changing professional behavior in clinical settings.

MATERIALS AND METHODS

Relevant systematic reviews of reminder interventions were identified through searches in MEDLINE, EMBASE, DARE and the Cochrane Library in conjunction with a larger project examining professional behavioral change interventions. Reviews were appraised using AMSTAR, a validated tool for assessing the quality of systematic reviews. As most reviews only reported vote counting, conclusions about effectiveness for each review were based on a count of positive studies. If available, we also report effect sizes. Conclusions were based on the findings from higher quality and current systematic reviews.

RESULTS

Thirty-five reviews were eligible for inclusion in this overview. Ten reviews examined the effectiveness of reminders generally, 5 reviews focused on specific health care settings, 14 reviews concentrated on specific behaviors and 6 reviews addressed specific patient populations. The quality of the reviews was variable (median = 3, range = 1 to 8). Seven reviews had AMSTAR scores >5 and were considered in detail. Five of these seven reviews demonstrated positive effects of reminders in changing provider behavior. Few reviews used quantitative pooling methods; in one high quality and current review, the overall observed effects were moderate with an absolute median improvement in performance of 4.2% (IQR: 0.5% to 6.6%).

DISCUSSION

The results support that modest improvements can occur with the use of reminders. The effect size is consistent with other interventions that have been used to improve professional behavior.

CONCLUSION

Reminders appear effective in improving different clinical behaviors across a range of settings.

Decisions about lumping vs. splitting of the scope of systematic reviews of complex interventions are not well justified: a case study in systematic reviews of health care professional reminders

OBJECTIVES

Lumping and splitting refer to the scope of a systematic review question, where lumped reviews are broad and split are narrow. The objective was to determine the frequency of lumping and splitting in systematic reviews of reminder interventions, assess how review authors justified their decisions about the scope of their reviews, and explore how review authors cited other systematic reviews in the field.

STUDY DESIGN AND SETTING

A descriptive approach involving a content analysis and citation bibliometric study of an overview of 31 systematic reviews of reminder interventions.

RESULTS

Twenty-four of 31 reminder reviews were split, most frequently across one category (population, intervention, study design, outcome). Review authors poorly justified their decisions about the scope of their reviews and tended not to cite other similar reviews.

CONCLUSION

This study demonstrates that for systematic reviews of reminder interventions, splitting is more common than lumping, with most reviews split by condition or targeted behavior. Review authors poorly justify the need for their review and do not cite relevant literature to put their reviews in the context of the available evidence. These factors may have contributed to a proliferation of systematic reviews of reminders and an overall disorganization of the literature.

Methodologies for assessing telemedicine: a systematic review of reviews

BACKGROUND AND OBJECTIVES

Previous reviews have expressed concerns about the quality of telemedicine studies. There is debate about shortcomings and appropriate methodologies. The aim of this review of systematic reviews of telemedicine is to summarize methodologies used in telemedicine research, discuss knowledge gaps and recommendations and suggest methodological approaches for further research.

METHODS

We conducted a review of systematic reviews of telemedicine according to a protocol listing explicit methods, selection criteria, data collection and quality assessment procedures. We included reviews where authors explicitly addressed and made recommendations for assessment methodologies. We did a qualitative analysis of the reviews included, sensitized by two broad methodological positions; positivist and naturalistic approaches. The analysis focused on methodologies used in the primary studies included in the reviews as reported by the review authors, and methodological recommendations made by the review authors.

RESULTS

We identified 1593 titles/abstracts. We included 50 reviews that explicitly addressed assessment methodologies. One group of reviews recommended larger and more rigorously designed controlled studies to assess the impacts of telemedicine; a second group proposed standardisation of populations, and/or interventions and outcome measures to reduce heterogeneity and facilitate meta-analysis; a third group recommended combining quantitative and qualitative research methods; and others applying different naturalistic approaches including methodologies addressing mutual adaptations of services and users; politically driven action research and formative research aimed at collaboration to ensure capacity for improvement of services in natural settings.

CONCLUSIONS

Larger and more rigorous studies are crucial for the production of evidence of effectiveness of unambiguous telemedicine services for pre defined outcome measures. Summative methodologies acknowledging telemedicine as complex innovations and outcomes as partly contingent on values, meanings and contexts are also important. So are formative, naturalistic methodologies that acknowledge telemedicine as ongoing collaborative achievements and engage with stakeholders, including patients to produce and conceptualise new and effective telemedicine innovations.

Decision-support tool in prehospital care: a systematic review of randomized trials

OBJECTIVE

The objective of this study was to evaluate the effects of the decision support tool (DST) on the assessment of the acutely ill or injured out-of-hospital patient.

METHODS

This study included systematic reviews of randomized controlled trials (RCT) where the DST was compared to usual care in and out of the hospital setting. The databases scanned include: (1) Cochrane Reviews (up to January 2010); (2) Cochrane Controlled Clinical Trials (1979 to January 2010); (3) Cinahl (1986 to January 2010); and (4) Pubmed/Medline (1926 to January 2010). In addition, information was gathered from related magazines, prehospital home pages, databases for theses, conferences, grey literature and ongoing trials.

RESULTS

Use of the DST in prehospital care may have the possibility to decrease "time to definitive care" and improve diagnostic accuracy among prehospital personnel, but more studies are needed.

CONCLUSIONS

The amount of data in this review is too small to be able to draw any reliable conclusions about the impact of the use of the DST on prehospital care. The research in this review indicates that there are very few RCTs that evaluate the use of the DST in prehospital care.

The role of computerized decision support in reducing errors in selecting medicines for prescription: narrative review

This narrative review includes a summary of research examining prescribing errors, prescription decision making and the role computerized decision support plays in this decision-making process. A reduction in medication prescribing errors, specifically a reduction in the selection of inappropriate medications, is expected to result from the implementation of an effective computerized decision support system. Previous research has investigated the impact of the implementation of electronic systems on medication errors more broadly. This review examines the specific characteristics of decision support systems that may contribute to fewer knowledge-based mistakes in prescribing, and critically appraises the large volume of information available on the decision-making process of selecting medicines for prescription. The results highlight a need for work investigating what decision strategies are used by doctors with different levels of expertise in the prescribing of medications. The nature of the relationship between decision support and decision performance is not well understood and future research is needed to determine the mechanisms by which computerized decision support influences medication selection.

Effects of clinical decision-support systems on practitioner performance and patient outcomes: a synthesis of high-quality systematic review findings

OBJECTIVE

To synthesize the literature on clinical decision-support systems' (CDSS) impact on healthcare practitioner performance and patient outcomes.

DESIGN

Literature search on Medline, Embase, Inspec, Cinahl, Cochrane/Dare and analysis of high-quality systematic reviews (SRs) on CDSS in hospital settings. Two-stage inclusion procedure: (1) selection of publications on predefined inclusion criteria; (2) independent methodological assessment of preincluded SRs by the 11-item measurement tool, AMSTAR. Inclusion of SRs with AMSTAR score 9 or above. SRs were thereafter rated on level of evidence. Each stage was performed by two independent reviewers.

RESULTS

17 out of 35 preincluded SRs were of high methodological quality and further analyzed. Evidence that CDSS significantly impacted practitioner performance was found in 52 out of 91 unique studies of the 16 SRs examining this effect (57%). Only 25 out of 82 unique studies of the 16 SRs reported evidence that CDSS positively impacted patient outcomes (30%).

CONCLUSIONS

Few studies have found any benefits on patient outcomes, though many of these have been too small in sample size or too short in time to reveal clinically important effects. There is significant evidence that CDSS can positively impact healthcare providers' performance with drug ordering and preventive care reminder systems as most clear examples. These outcomes may be explained by the fact that these types of CDSS require a minimum of patient data that are largely available before the advice is (to be) generated: at the time clinicians make the decisions.

A review on systematic reviews of health information system studies

The purpose of this review is to consolidate existing evidence from published systematic reviews on health information system (HIS) evaluation studies to inform HIS practice and research. Fifty reviews published during 1994-2008 were selected for meta-level synthesis. These reviews covered five areas: medication management, preventive care, health conditions, data quality, and care process/outcome. After reconciliation for duplicates, 1276 HIS studies were arrived at as the non-overlapping corpus. On the basis of a subset of 287 controlled HIS studies, there is some evidence for improved quality of care, but in varying degrees across topic areas. For instance, 31/43 (72%) controlled HIS studies had positive results using preventive care reminders, mostly through guideline adherence such as immunization and health screening. Key factors that influence HIS success included having in-house systems, developers as users, integrated decision support and benchmark practices, and addressing such contextual issues as provider knowledge and perception, incentives, and legislation/policy.

The impact of eHealth on the quality and safety of health care: a systematic overview

BACKGROUND

There is considerable international interest in exploiting the potential of digital solutions to enhance the quality and safety of health care. Implementations of transformative eHealth technologies are underway globally, often at very considerable cost. In order to assess the impact of eHealth solutions on the quality and safety of health care, and to inform policy decisions on eHealth deployments, we undertook a systematic review of systematic reviews assessing the effectiveness and consequences of various eHealth technologies on the quality and safety of care.

METHODS AND FINDINGS

We developed novel search strategies, conceptual maps of health care quality, safety, and eHealth interventions, and then systematically identified, scrutinised, and synthesised the systematic review literature. Major biomedical databases were searched to identify systematic reviews published between 1997 and 2010. Related theoretical, methodological, and technical material was also reviewed. We identified 53 systematic reviews that focused on assessing the impact of eHealth interventions on the quality and/or safety of health care and 55 supplementary systematic reviews providing relevant supportive information. This systematic review literature was found to be generally of substandard quality with regards to methodology, reporting, and utility. We thematically categorised eHealth technologies into three main areas: (1) storing, managing, and transmission of data; (2) clinical decision support; and (3) facilitating care from a distance. We found that despite support from policymakers, there was relatively little empirical evidence to substantiate many of the claims made in relation to these technologies. Whether the success of those relatively few solutions identified to improve quality and safety would continue if these were deployed beyond the contexts in which they were originally developed, has yet to be established. Importantly, best practice guidelines in effective development and deployment strategies are lacking.

CONCLUSIONS

There is a large gap between the postulated and empirically demonstrated benefits of eHealth technologies. In addition, there is a lack of robust research on the risks of implementing these technologies and their cost-effectiveness has yet to be demonstrated, despite being frequently promoted by policymakers and "techno-enthusiasts" as if this was a given. In the light of the paucity of evidence in relation to improvements in patient outcomes, as well as the lack of evidence on their cost-effectiveness, it is vital that future eHealth technologies are evaluated against a comprehensive set of measures, ideally throughout all stages of the technology's life cycle. Such evaluation should be characterised by careful attention to socio-technical factors to maximise the likelihood of successful implementation and adoption.

Harnessing health IT for improved cardiovascular risk management

Robyn Whittaker and colleagues argue that IT-based programs can improve cardiovascular disease management and patient empowerment, but must be accompanied by supportive social and political environments and active patient and clinician engagement.

Computerized clinical decision support for prescribing: provision does not guarantee uptake

There is wide variability in the use and adoption of recommendations generated by computerized clinical decision support systems (CDSSs) despite the benefits they may bring to clinical practice. We conducted a systematic review to explore the barriers to, and facilitators of, CDSS uptake by physicians to guide prescribing decisions. We identified 58 studies by searching electronic databases (1990-2007). Factors impacting on CDSS use included: the availability of hardware, technical support and training; integration of the system into workflows; and the relevance and timeliness of the clinical messages. Further, systems that were endorsed by colleagues, minimized perceived threats to professional autonomy, and did not compromise doctor-patient interactions were accepted by users. Despite advances in technology and CDSS sophistication, most factors were consistently reported over time and across ambulatory and institutional settings. Such factors must be addressed when deploying CDSSs so that improvements in uptake, practice and patient outcomes may be achieved.

Informatics for Infectious Disease Research and Control

The goal of infectious disease informatics is to optimize the clinical and public health management of infectious diseases through improvements in the development and use of antimicrobials, the design of more effective vaccines, the identification of biomarkers for life-threatening infections, a better understanding of host-pathogen interactions, and biosurveillance and clinical decision support. Infectious disease informatics can lead to more targeted and effective approaches for the prevention, diagnosis and treatment of infections through a comprehensive review of the genetic repertoire and metabolic profiles of a pathogen. The developments in informatics have been critical in boosting the translational science and in supporting both reductionist and integrative research paradigms.

Effectiveness of electronic guideline-based implementation systems in ambulatory care settings - a systematic review

Background

Electronic guideline-based decision support systems have been suggested to successfully deliver the knowledge embedded in clinical practice guidelines. A number of studies have already shown positive findings for decision support systems such as drug-dosing systems and computer-generated reminder systems for preventive care services.

Methods

A systematic literature search (1990 to December 2008) of the English literature indexed in the Medline database, Embase, the Cochrane Central Register of Controlled Trials, and CRD (DARE, HTA and NHS EED databases) was conducted to identify evaluation studies of electronic multi-step guideline implementation systems in ambulatory care settings. Important inclusion criterions were the multidimensionality of the guideline (the guideline needed to consist of several aspects or steps) and real-time interaction with the system during consultation. Clinical decision support systems such as one-time reminders for preventive care for which positive findings were shown in earlier reviews were excluded. Two comparisons were considered: electronic multidimensional guidelines versus usual care (comparison one) and electronic multidimensional guidelines versus other guideline implementation methods (comparison two).

Results

Twenty-seven publications were selected for analysis in this systematic review. Most designs were cluster randomized controlled trials investigating process outcomes more than patient outcomes. With success defined as at least 50% of the outcome variables being significant, none of the studies were successful in improving patient outcomes. Only seven of seventeen studies that investigated process outcomes showed improvements in process of care variables compared with the usual care group (comparison one). No incremental effect of the electronic implementation over the distribution of paper versions of the guideline was found, neither for the patient outcomes nor for the process outcomes (comparison two).

Conclusions

There is little evidence at the moment for the effectiveness of an increasingly used and commercialised instrument such as electronic multidimensional guidelines. After more than a decade of development of numerous electronic systems, research on the most effective implementation strategy for this kind of guideline-based decision support systems is still lacking. This conclusion implies a considerable risk towards inappropriate investments in ineffective implementation interventions and in suboptimal care.

Do computerised clinical decision support systems for prescribing change practice? A systematic review of the literature (1990-2007)

BACKGROUND

Computerised clinical decision support systems (CDSSs) are used widely to improve quality of care and patient outcomes. This systematic review evaluated the impact of CDSSs in targeting specific aspects of prescribing, namely initiating, monitoring and stopping therapy. We also examined the influence of clinical setting (institutional vs ambulatory care), system- or user-initiation of CDSS, multi-faceted vs stand alone CDSS interventions and clinical target on practice changes in line with the intent of the CDSS.

METHODS

We searched Medline, Embase and PsychINFO for publications from 1990-2007 detailing CDSS prescribing interventions. Pairs of independent reviewers extracted the key features and prescribing outcomes of methodologically adequate studies (experiments and strong quasi-experiments).

RESULTS

56 studies met our inclusion criteria, 38 addressing initiating, 23 monitoring and three stopping therapy. At the time of initiating therapy, CDSSs appear to be somewhat more effective after, rather than before, drug selection has occurred (7/12 versus 12/26 studies reporting statistically significant improvements in favour of CDSSs on = 50% of prescribing outcomes reported). CDSSs also appeared to be effective for monitoring therapy, particularly using laboratory test reminders (4/7 studies reporting significant improvements in favour of CDSSs on the majority of prescribing outcomes). None of the studies addressing stopping therapy demonstrated impacts in favour of CDSSs over comparators. The most consistently effective approaches used system-initiated advice to fine-tune existing therapy by making recommendations to improve patient safety, adjust the dose, duration or form of prescribed drugs or increase the laboratory testing rates for patients on long-term therapy. CDSSs appeared to perform better in institutional compared to ambulatory settings and when decision support was initiated automatically by the system as opposed to user initiation. CDSSs implemented with other strategies such as education were no more successful in improving prescribing than stand alone interventions. Cardiovascular disease was the most studied clinical target but few studies demonstrated significant improvements on the majority of prescribing outcomes.

CONCLUSION

Our understanding of CDSS impacts on specific aspects of the prescribing process remains relatively limited. Future implementation should build on effective approaches including the use of system-initiated advice to address safety issues and improve the monitoring of therapy.

Decision support systems for antibiotic prescribing

PURPOSE OF REVIEW

To explore recent developments in computerized evidence-based guidelines and decision support systems that have been designed to improve the effectiveness and efficiency of antibiotic prescribing.

RECENT FINDINGS

The most frequently utilized decision support systems are electronic guidelines and protocols, especially for empirical selection of antibiotics. The majority of decision support systems result in improvement in clinical performance and, in at least half of the published trials, in patient outcomes. Despite the reported successes of individual applications, the safety of electronic prescribing systems in routine practice has been identified recently as an issue of potential concern. Bioinformatics-assisted prescribing may contribute to reducing the complexities of prescribing combinations of antimicrobials in the era of multidrug resistance.

SUMMARY

The reemerging interest in prescribing decision support reflects the recent change in emphasis from support for diagnostic decisions towards support for patient management and from systems targeting a broad range of clinical diagnoses to task specific and condition-specific decision aids.

Chronic care model and shared care in diabetes: randomized trial of an electronic decision support system

OBJECTIVE

To assess the effect of a specialist telemedicine intervention for improving diabetes care using the chronic care model (CCM).

PARTICIPANTS AND METHODS

As part of the CCM, 97 primary care physicians at 6 primary care practices in Rochester, MN, referred 639 patients to an on-site diabetes educator between July 1, 2001, and December 31, 2003. On first referral, physicians were centrally randomized to receive a telemedicine intervention (specialty advice and evidence-based messages regarding medication management for cardiovascular risk) or no intervention, keeping outcome assessors and data analysts blinded to group assignment. After each subsequent clinical encounter, endocrinologists reviewed an abstract from the patient's electronic medical record and provided management recommendations and supporting evidence to intervention physicians via e-mail. Control physicians received e-mail with periodic generic information about cardiovascular risk reduction in diabetes. Outcome measures included diabetes care processes (diabetes test completion), outcomes (metabolic and cardiovascular risk factors, estimated coronary artery disease risk), and patient costs (payer perspective).

RESULTS

During the intervention, 951 (70%) of the 1361 endocrinology reviews detected performance gaps and resulted in a message; primary care physicians reported using 49% of messages in patient care. With a mean of 21 months' follow-up, the intervention, compared with control, did not significantly enhance metabolic outcomes or reduce estimated risk of coronary artery disease (adjusted mean difference, -1%; 95% confidence interval, -19% to 17%). The intervention group incurred lower costs (P=.02) but not in diabetes-related costs.

CONCLUSION

Specialty telemedicine did not significantly enhance the value of CCM in primary care.