A scoping review of online repositories of quality improvement projects, interventions and initiatives in healthcare

Quality measurement and surveillance platforms in critically ill children: A scoping review

BACKGROUND/AIM

The objective of this study was to describe current surveillance platforms which support routine quality measurement in paediatric critical care.

METHOD

Scoping review. The search strategy consisted of a traditional database and grey literature search as well as expert consultation. Surveillance platforms were eligible for inclusion if they collected measures of quality in critically ill children.

RESULTS

The search strategy identified 21 surveillance platforms, collecting 57 unique outcome (70%), process (23%), and structural (7%) quality measures. Hospital-associated infections were the most commonly collected outcome measure across all platforms (n = 11; 52%). In general, case definitions were not harmonised across platforms, with the exception of nationally mandated hospital-associated infections (e.g., central line-associated blood stream infection). Data collection relied on manual coding. Platforms typically did not provide an evidence-based rationale for measures collected, with no identifiable reports of co-designed, consensus-derived measures or consumer involvement in measure selection or prioritisation.

CONCLUSIONS

Quality measurement in critically ill children lacks uniformity in definition which limits local and international benchmarking. Current surveillance activities for critically ill children focus heavily on outcome measurement, with process, structural, and patient-reported measures largely overlooked. Long-term outcome measures were not routinely collected. Harmonisation of paediatric intensive care unit quality measures is needed and can be achieved using prioritisation and consensus/co-design methods.

A scoping review of local quality improvement using data from UK perioperative National Clinical Audits

Background

Significant resources are invested in the UK to collect data for National Clinical Audits (NCAs), but it is unclear whether and how they facilitate local quality improvement (QI). The perioperative setting is a unique context for QI due to its multidisciplinary nature and history of measurement. It is unclear which NCAs evaluate perioperative care, to what extent their data have been used for QI, and which factors influence this usage.

Methods

NCAs were identified from the directories held by Healthcare Quality Improvement Partnership (HQIP), Scottish Healthcare Audits and the Welsh National Clinical Audit and Outcome Review Advisory Committee. QI reports were identified by the following: systematically searching MEDLINE, CINAHL Plus, Web of Science, Embase, Google Scholar and HMIC up to December 2019, hand-searching grey literature and consulting relevant stakeholders. We charted features describing both the NCAs and the QI reports and summarised quantitative data using descriptive statistics and qualitative themes using framework analysis.

Results

We identified 36 perioperative NCAs in the UK and 209 reports of local QI which used data from 19 (73%) of these NCAs. Six (17%) NCAs contributed 185 (89%) of these reports. Only one NCA had a registry of local QI projects. The QI reports were mostly brief, unstructured, often published by NCAs themselves and likely subject to significant reporting bias. Factors reported to influence local QI included the following: perceived data validity, measurement of clinical processes as well as outcomes, timely feedback, financial incentives, sharing of best practice, local improvement capabilities and time constraints of clinicians.

Conclusions

There is limited public reporting of UK perioperative NCA data for local QI, despite evidence of improvement of most NCA metrics at the national level. It is therefore unclear how these improvements are being made, and it is likely that opportunities are being missed to share learning between local sites. We make recommendations for how NCAs could better support the conduct, evaluation and reporting of local QI and suggest topics which future research should investigate.

Trial registration

The review was registered with the International Prospective Register of Systematic Reviews (PROSPERO: CRD42018092993).

Supplementary Information

The online version contains supplementary material available at 10.1186/s13741-022-00273-0.

Reporting standards, outcomes and costs of quality improvement studies in Ireland: a scoping review

OBJECTIVES

To profile the aims and characteristics of quality improvement (QI) initiatives conducted in Ireland, to review the quality of their reporting and to assess outcomes and costs.

DESIGN

Scoping review.

DATA SOURCES

Systematic searches were conducted in PubMed, Web of Science, Embase, Google Scholar, Lenus and rian.ie. Two researchers independently screened abstracts (n=379) and separately reviewed 43 studies identified for inclusion using a 70-item critique tool. The tool was based on the Quality Improvement Minimum Quality Criteria Set (QI-MQCS), an appraisal instrument for QI intervention publications, and health economics reporting criteria. After reaching consensus, the final dataset was analysed using descriptive statistics. To support interpretations, findings were presented at a national stakeholder workshop.

ELIGIBILITY CRITERIA

QI studies implemented and evaluated in Ireland and published between January 2015 and April 2020.

RESULTS

The 43 studies represented various QI interventions. Most studies were peer-reviewed publications (n=37), conducted in hospitals (n=38). Studies mainly aimed to improve the 'effectiveness' (65%), 'efficiency' (53%), 'timeliness' (47%) and 'safety' (44%) of care. Fewer aimed to improve 'patient-centredness' (30%), 'value for money' (23%) or 'staff well-being' (9%). No study aimed to increase 'equity'. Seventy per cent of studies described 14 of 16 QI-MQCS dimensions. Least often studies reported the 'penetration/reach' of an initiative and only 35% reported health outcomes. While 53% of studies expressed awareness of costs, only eight provided at least one quantifiable figure for costs or savings. No studies assessed the cost-effectiveness of the QI.

CONCLUSION

Irish QI studies included in our review demonstrate varied aims and high reporting standards. Strategies are needed to support greater stimulation and dissemination of QI beyond the hospital sector and awareness of equity issues as QI work. Systematic measurement and reporting of costs and outcomes can be facilitated by integrating principles of health economics in QI education and guidelines.

Publication and related bias in quantitative health services and delivery research: a multimethod study

Background

Bias in the publication and reporting of research findings (referred to as publication and related bias here) poses a major threat in evidence synthesis and evidence-based decision-making. Although this bias has been well documented in clinical research, little is known about its occurrence and magnitude in health services and delivery research.

Objectives

To obtain empirical evidence on publication and related bias in quantitative health services and delivery research; to examine current practice in detecting/mitigating this bias in health services and delivery research systematic reviews; and to explore stakeholders’ perception and experiences concerning such bias.

Methods

The project included five distinct but interrelated work packages. Work package 1 was a systematic review of empirical and methodological studies. Work package 2 involved a survey (meta-epidemiological study) of randomly selected systematic reviews of health services and delivery research topics (n = 200) to evaluate current practice in the assessment of publication and outcome reporting bias during evidence synthesis. Work package 3 included four case studies to explore the applicability of statistical methods for detecting such bias in health services and delivery research. In work package 4 we followed up four cohorts of health services and delivery research studies (total n = 300) to ascertain their publication status, and examined whether publication status was associated with statistical significance or perceived ‘positivity’ of study findings. Work package 5 involved key informant interviews with diverse health services and delivery research stakeholders (n = 24), and a focus group discussion with patient and service user representatives (n = 8).

Results

We identified only four studies that set out to investigate publication and related bias in health services and delivery research in work package 1. Three of these studies focused on health informatics research and one concerned health economics. All four studies reported evidence of the existence of this bias, but had methodological weaknesses. We also identified three health services and delivery research systematic reviews in which findings were compared between published and grey/unpublished literature. These reviews found that the quality and volume of evidence and effect estimates sometimes differed significantly between published and unpublished literature. Work package 2 showed low prevalence of considering/assessing publication (43%) and outcome reporting (17%) bias in health services and delivery research systematic reviews. The prevalence was lower among reviews of associations than among reviews of interventions. The case studies in work package 3 highlighted limitations in current methods for detecting these biases due to heterogeneity and potential confounders. Follow-up of health services and delivery research cohorts in work package 4 showed positive association between publication status and having statistically significant or positive findings. Diverse views concerning publication and related bias and insights into how features of health services and delivery research might influence its occurrence were uncovered through the interviews with health services and delivery research stakeholders and focus group discussion conducted in work package 5.

Conclusions

This study provided prima facie evidence on publication and related bias in quantitative health services and delivery research. This bias does appear to exist, but its prevalence and impact may vary depending on study characteristics, such as study design, and motivation for conducting the evaluation. Emphasis on methodological novelty and focus beyond summative assessments may mitigate/lessen the risk of such bias in health services and delivery research. Methodological and epistemological diversity in health services and delivery research and changing landscape in research publication need to be considered when interpreting the evidence. Collection of further empirical evidence and exploration of optimal health services and delivery research practice are required.

Study registration

This study is registered as PROSPERO CRD42016052333 and CRD42016052366.

Funding

This project was funded by the National Institute for Health Research (NIHR) Health Services and Delivery Research programme and will be published in full in Health Services and Delivery Research; Vol. 8, No. 33. See the NIHR Journals Library website for further project information.

Publication and related biases in health services research: a systematic review of empirical evidence

Background

Publication and related biases (including publication bias, time-lag bias, outcome reporting bias and p-hacking) have been well documented in clinical research, but relatively little is known about their presence and extent in health services research (HSR). This paper aims to systematically review evidence concerning publication and related bias in quantitative HSR.

Methods

Databases including MEDLINE, EMBASE, HMIC, CINAHL, Web of Science, Health Systems Evidence, Cochrane EPOC Review Group and several websites were searched to July 2018. Information was obtained from: (1) Methodological studies that set out to investigate publication and related biases in HSR; (2) Systematic reviews of HSR topics which examined such biases as part of the review process. Relevant information was extracted from included studies by one reviewer and checked by another. Studies were appraised according to commonly accepted scientific principles due to lack of suitable checklists. Data were synthesised narratively.

Results

After screening 6155 citations, four methodological studies investigating publication bias in HSR and 184 systematic reviews of HSR topics (including three comparing published with unpublished evidence) were examined. Evidence suggestive of publication bias was reported in some of the methodological studies, but evidence presented was very weak, limited in both quality and scope. Reliable data on outcome reporting bias and p-hacking were scant. HSR systematic reviews in which published literature was compared with unpublished evidence found significant differences in the estimated intervention effects or association in some but not all cases.

Conclusions

Methodological research on publication and related biases in HSR is sparse. Evidence from available literature suggests that such biases may exist in HSR but their scale and impact are difficult to estimate for various reasons discussed in this paper.

Systematic review registration

PROSPERO 2016 CRD42016052333.

Key Characteristics of Rehabilitation Quality Improvement Publications: Scoping Review From 2010 to 2016

OBJECTIVE

To characterize the peer-reviewed quality improvement (QI) literature in rehabilitation.

DATA SOURCES

Five electronic databases were searched for English-language articles from 2010 to 2016. Keywords for QI and safety management were searched for in combination with keywords for rehabilitation content and journals. Secondary searches (eg, references-list scanning) were also performed.

STUDY SELECTION

Two reviewers independently selected articles using working definitions of rehabilitation and QI study types; of 1016 references, 112 full texts were assessed for eligibility.

DATA EXTRACTION

Reported study characteristics including study focus, study setting, use of inferential statistics, stated limitations, and use of improvement cycles and theoretical models were extracted by 1 reviewer, with a second reviewer consulted whenever inferences or interpretation were involved.

DATA SYNTHESIS

Fifty-nine empirical rehabilitation QI studies were found: 43 reporting on local QI activities, 7 reporting on QI effectiveness research, 8 reporting on QI facilitators or barriers, and 1 systematic review of a specific topic. The number of publications had significant yearly growth between 2010 and 2016 (P=.03). Among the 43 reports on local QI activities, 23.3% did not explicitly report any study limitations; 39.5% did not used inferential statistics to measure the QI impact; 95.3% did not cite/mention the appropriate reporting guidelines; only 18.6% reported multiple QI cycles; just over 50% reported using a model to guide the QI activity; and only 7% reported the use of a particular theoretical model. Study sites and focuses were diverse; however, nearly a third (30.2%) examined early mobilization in intensive care units.

CONCLUSIONS

The number of empirical, peer-reviewed rehabilitation QI publications is growing but remains a tiny fraction of rehabilitation research publications. Rehabilitation QI studies could be strengthened by greater use of extant models and theory to guide the QI work, consistent reporting of study limitations, and use of inferential statistics.

The research activities of Ontario's large community acute care hospitals: a scoping review

Background

Ontario’s large community hospitals (LCHs) provide care to 65% of the province’s hospitalized patients, yet we know very little about their research activities. By searching for research publications from 2013 to 2015, we will describe the extent, type and collaborative nature of Ontario’s LCHs’ research activities.

Methods

We conducted a scoping review by searching PubMed, Embase and the Cumulative Index to Nursing and Allied Health Literature databases from January 1, 2013 until December 31, 2015 for all publication types whose author(s) was affiliated with any of the 44 LCHs. Articles were screened and abstracted by three reviewers, independently. The data were charted and results described using summary statistics, scatter plots, and bar charts.

Results

We included 798 publications from 39 LCHs and 454 authors. The median number of publications was 7 (Interquartile range (IQR) 23). Observational study design was most commonly reported in over 50% of publications. Program evaluation was the focus in 40% of publications. Primary LCH authorship was observed for 535 publications. Over 25% and 65% of the publications were attributable to 24 authors and 9 LCHs, respectively. There was minimal collaboration both within (21.2%) and between (7.8%) LCHs. LCH size and geographic proximity to academic hospitals had minimal impact on research activity.

Conclusions

Ontario’s LCHs publish infrequently, collaborate infrequently, and their role in translational research activity is not well defined. A future survey questionnaire to LCH researchers identified through this review is planned to both validate and elicit their interpretations of our study findings and opinions about LCH involvement in research.