1
|
Schults JA, Rickard CM, Charles K, Rahiman S, Millar J, Baveas T, Long D, Kleidon TM, Macfarlane F, Mehta NM, Runnegar N, Hall L. Quality measurement and surveillance platforms in critically ill children: A scoping review. Aust Crit Care 2022:S1036-7314(22)00097-2. [PMID: 36117039 DOI: 10.1016/j.aucc.2022.07.006] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/28/2022] [Revised: 07/24/2022] [Accepted: 07/31/2022] [Indexed: 10/14/2022] Open
Abstract
BACKGROUND/AIM The objective of this study was to describe current surveillance platforms which support routine quality measurement in paediatric critical care. METHOD Scoping review. The search strategy consisted of a traditional database and grey literature search as well as expert consultation. Surveillance platforms were eligible for inclusion if they collected measures of quality in critically ill children. RESULTS The search strategy identified 21 surveillance platforms, collecting 57 unique outcome (70%), process (23%), and structural (7%) quality measures. Hospital-associated infections were the most commonly collected outcome measure across all platforms (n = 11; 52%). In general, case definitions were not harmonised across platforms, with the exception of nationally mandated hospital-associated infections (e.g., central line-associated blood stream infection). Data collection relied on manual coding. Platforms typically did not provide an evidence-based rationale for measures collected, with no identifiable reports of co-designed, consensus-derived measures or consumer involvement in measure selection or prioritisation. CONCLUSIONS Quality measurement in critically ill children lacks uniformity in definition which limits local and international benchmarking. Current surveillance activities for critically ill children focus heavily on outcome measurement, with process, structural, and patient-reported measures largely overlooked. Long-term outcome measures were not routinely collected. Harmonisation of paediatric intensive care unit quality measures is needed and can be achieved using prioritisation and consensus/co-design methods.
Collapse
Affiliation(s)
- Jessica A Schults
- School of Nursing, Midwifery and Social Work, University of Queensland, St Lucia, Queensland, Australia; Metro North Hospital and Health Service, Queensland, Australia; Child Health Research Centre, Faculty of Medicine, University of Queensland, St Lucia, Queensland, Australia; School of Nursing and Midwifery Griffith University, Queensland, Australia; Paediatric Intensive Care Unit Queensland Children's Hospital, South Brisbane, Queensland, Australia.
| | - Claire M Rickard
- School of Nursing, Midwifery and Social Work, University of Queensland, St Lucia, Queensland, Australia; Metro North Hospital and Health Service, Queensland, Australia; School of Nursing and Midwifery Griffith University, Queensland, Australia
| | - Karina Charles
- School of Nursing, Midwifery and Social Work, University of Queensland, St Lucia, Queensland, Australia; Child Health Research Centre, Faculty of Medicine, University of Queensland, St Lucia, Queensland, Australia; School of Nursing and Midwifery Griffith University, Queensland, Australia; Paediatric Intensive Care Unit Queensland Children's Hospital, South Brisbane, Queensland, Australia
| | - Sarfaraz Rahiman
- Paediatric Intensive Care Unit Queensland Children's Hospital, South Brisbane, Queensland, Australia; School of Medicine, University of Queensland, St Lucia, Queensland, Australia
| | - Johnny Millar
- Paediatric Intensive Care Unit, Royal Children's Hospital, Melbourne, Victoria, Australia; Murdoch Children's Research Institute, Victoria, Australia; Department of Paediatrics, University of Melbourne, Victoria, Australia; Centre for Outcome and Resource Evaluation, Australian and New Zealand Intensive Care Society, Melbourne, Australia
| | - Thimitra Baveas
- Paediatric Intensive Care Unit Queensland Children's Hospital, South Brisbane, Queensland, Australia
| | - Debbie Long
- Paediatric Intensive Care Unit Queensland Children's Hospital, South Brisbane, Queensland, Australia; School of Nursing, Centre for Healthcare Transformation, Queensland University of Technology, Brisbane, Queensland, Australia
| | - Tricia M Kleidon
- School of Nursing, Midwifery and Social Work, University of Queensland, St Lucia, Queensland, Australia; School of Nursing and Midwifery Griffith University, Queensland, Australia; Department of Anaesthesia and Pain Management, Queensland Children's Hospital, South Brisbane, Queensland, Australia
| | - Fiona Macfarlane
- Department of Anaesthesia and Pain Management, Queensland Children's Hospital, South Brisbane, Queensland, Australia
| | - Nilesh M Mehta
- Perioperative & Critical Care Center for Outcomes Research (PC-CORE), USA; Department of Anesthesiology, Critical Care & Pain Medicine, Division of Critical Care Medicine, Boston Children's Hospital, USA; Harvard Medical School, Boston, USA
| | - Naomi Runnegar
- School of Medicine, University of Queensland, St Lucia, Queensland, Australia; Infection Management, Princess Alexandra Hospital, Woolloongabba, Qld, Australia
| | - Lisa Hall
- School of Public Health, Faculty of Medicine, University of Queensland, St Lucia, Queensland, Australia
| |
Collapse
|
2
|
Wagstaff D, Warnakulasuriya S, Singleton G, Moonesinghe SR, Fulop N, Vindrola-Padros C. A scoping review of local quality improvement using data from UK perioperative National Clinical Audits. Perioper Med (Lond) 2022; 11:43. [PMID: 36031654 PMCID: PMC9422140 DOI: 10.1186/s13741-022-00273-0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/20/2021] [Accepted: 06/22/2022] [Indexed: 11/28/2022] Open
Abstract
Background Significant resources are invested in the UK to collect data for National Clinical Audits (NCAs), but it is unclear whether and how they facilitate local quality improvement (QI). The perioperative setting is a unique context for QI due to its multidisciplinary nature and history of measurement. It is unclear which NCAs evaluate perioperative care, to what extent their data have been used for QI, and which factors influence this usage. Methods NCAs were identified from the directories held by Healthcare Quality Improvement Partnership (HQIP), Scottish Healthcare Audits and the Welsh National Clinical Audit and Outcome Review Advisory Committee. QI reports were identified by the following: systematically searching MEDLINE, CINAHL Plus, Web of Science, Embase, Google Scholar and HMIC up to December 2019, hand-searching grey literature and consulting relevant stakeholders. We charted features describing both the NCAs and the QI reports and summarised quantitative data using descriptive statistics and qualitative themes using framework analysis. Results We identified 36 perioperative NCAs in the UK and 209 reports of local QI which used data from 19 (73%) of these NCAs. Six (17%) NCAs contributed 185 (89%) of these reports. Only one NCA had a registry of local QI projects. The QI reports were mostly brief, unstructured, often published by NCAs themselves and likely subject to significant reporting bias. Factors reported to influence local QI included the following: perceived data validity, measurement of clinical processes as well as outcomes, timely feedback, financial incentives, sharing of best practice, local improvement capabilities and time constraints of clinicians. Conclusions There is limited public reporting of UK perioperative NCA data for local QI, despite evidence of improvement of most NCA metrics at the national level. It is therefore unclear how these improvements are being made, and it is likely that opportunities are being missed to share learning between local sites. We make recommendations for how NCAs could better support the conduct, evaluation and reporting of local QI and suggest topics which future research should investigate. Trial registration The review was registered with the International Prospective Register of Systematic Reviews (PROSPERO: CRD42018092993). Supplementary Information The online version contains supplementary material available at 10.1186/s13741-022-00273-0.
Collapse
|
3
|
McCarthy SE, Jabakhanji SB, Martin J, Flynn MA, Sørensen J. Reporting standards, outcomes and costs of quality improvement studies in Ireland: a scoping review. BMJ Open Qual 2021; 10:bmjoq-2020-001319. [PMID: 34341016 PMCID: PMC8330587 DOI: 10.1136/bmjoq-2020-001319] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/15/2020] [Accepted: 07/08/2021] [Indexed: 11/04/2022] Open
Abstract
OBJECTIVES To profile the aims and characteristics of quality improvement (QI) initiatives conducted in Ireland, to review the quality of their reporting and to assess outcomes and costs. DESIGN Scoping review. DATA SOURCES Systematic searches were conducted in PubMed, Web of Science, Embase, Google Scholar, Lenus and rian.ie. Two researchers independently screened abstracts (n=379) and separately reviewed 43 studies identified for inclusion using a 70-item critique tool. The tool was based on the Quality Improvement Minimum Quality Criteria Set (QI-MQCS), an appraisal instrument for QI intervention publications, and health economics reporting criteria. After reaching consensus, the final dataset was analysed using descriptive statistics. To support interpretations, findings were presented at a national stakeholder workshop. ELIGIBILITY CRITERIA QI studies implemented and evaluated in Ireland and published between January 2015 and April 2020. RESULTS The 43 studies represented various QI interventions. Most studies were peer-reviewed publications (n=37), conducted in hospitals (n=38). Studies mainly aimed to improve the 'effectiveness' (65%), 'efficiency' (53%), 'timeliness' (47%) and 'safety' (44%) of care. Fewer aimed to improve 'patient-centredness' (30%), 'value for money' (23%) or 'staff well-being' (9%). No study aimed to increase 'equity'. Seventy per cent of studies described 14 of 16 QI-MQCS dimensions. Least often studies reported the 'penetration/reach' of an initiative and only 35% reported health outcomes. While 53% of studies expressed awareness of costs, only eight provided at least one quantifiable figure for costs or savings. No studies assessed the cost-effectiveness of the QI. CONCLUSION Irish QI studies included in our review demonstrate varied aims and high reporting standards. Strategies are needed to support greater stimulation and dissemination of QI beyond the hospital sector and awareness of equity issues as QI work. Systematic measurement and reporting of costs and outcomes can be facilitated by integrating principles of health economics in QI education and guidelines.
Collapse
Affiliation(s)
- Siobhán Eithne McCarthy
- Graduate School of Healthcare Management, RCSI University of Medicine and Health Sciences, Dublin, Ireland
| | - Samira Barbara Jabakhanji
- Healthcare Outcomes Research Centre, RCSI University of Medicine and Health Sciences, Dublin, Ireland
| | - Jennifer Martin
- National Quality Improvement Team, Health Service Executive, Dublin, Ireland
| | - Maureen Alice Flynn
- National Quality Improvement Team, Health Service Executive, Dublin, Ireland
| | - Jan Sørensen
- Healthcare Outcomes Research Centre, RCSI University of Medicine and Health Sciences, Dublin, Ireland
| |
Collapse
|
4
|
Ayorinde AA, Williams I, Mannion R, Song F, Skrybant M, Lilford RJ, Chen YF. Publication and related bias in quantitative health services and delivery research: a multimethod study. HEALTH SERVICES AND DELIVERY RESEARCH 2020. [DOI: 10.3310/hsdr08330] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/22/2022] Open
Abstract
Background
Bias in the publication and reporting of research findings (referred to as publication and related bias here) poses a major threat in evidence synthesis and evidence-based decision-making. Although this bias has been well documented in clinical research, little is known about its occurrence and magnitude in health services and delivery research.
Objectives
To obtain empirical evidence on publication and related bias in quantitative health services and delivery research; to examine current practice in detecting/mitigating this bias in health services and delivery research systematic reviews; and to explore stakeholders’ perception and experiences concerning such bias.
Methods
The project included five distinct but interrelated work packages. Work package 1 was a systematic review of empirical and methodological studies. Work package 2 involved a survey (meta-epidemiological study) of randomly selected systematic reviews of health services and delivery research topics (n = 200) to evaluate current practice in the assessment of publication and outcome reporting bias during evidence synthesis. Work package 3 included four case studies to explore the applicability of statistical methods for detecting such bias in health services and delivery research. In work package 4 we followed up four cohorts of health services and delivery research studies (total n = 300) to ascertain their publication status, and examined whether publication status was associated with statistical significance or perceived ‘positivity’ of study findings. Work package 5 involved key informant interviews with diverse health services and delivery research stakeholders (n = 24), and a focus group discussion with patient and service user representatives (n = 8).
Results
We identified only four studies that set out to investigate publication and related bias in health services and delivery research in work package 1. Three of these studies focused on health informatics research and one concerned health economics. All four studies reported evidence of the existence of this bias, but had methodological weaknesses. We also identified three health services and delivery research systematic reviews in which findings were compared between published and grey/unpublished literature. These reviews found that the quality and volume of evidence and effect estimates sometimes differed significantly between published and unpublished literature. Work package 2 showed low prevalence of considering/assessing publication (43%) and outcome reporting (17%) bias in health services and delivery research systematic reviews. The prevalence was lower among reviews of associations than among reviews of interventions. The case studies in work package 3 highlighted limitations in current methods for detecting these biases due to heterogeneity and potential confounders. Follow-up of health services and delivery research cohorts in work package 4 showed positive association between publication status and having statistically significant or positive findings. Diverse views concerning publication and related bias and insights into how features of health services and delivery research might influence its occurrence were uncovered through the interviews with health services and delivery research stakeholders and focus group discussion conducted in work package 5.
Conclusions
This study provided prima facie evidence on publication and related bias in quantitative health services and delivery research. This bias does appear to exist, but its prevalence and impact may vary depending on study characteristics, such as study design, and motivation for conducting the evaluation. Emphasis on methodological novelty and focus beyond summative assessments may mitigate/lessen the risk of such bias in health services and delivery research. Methodological and epistemological diversity in health services and delivery research and changing landscape in research publication need to be considered when interpreting the evidence. Collection of further empirical evidence and exploration of optimal health services and delivery research practice are required.
Study registration
This study is registered as PROSPERO CRD42016052333 and CRD42016052366.
Funding
This project was funded by the National Institute for Health Research (NIHR) Health Services and Delivery Research programme and will be published in full in Health Services and Delivery Research; Vol. 8, No. 33. See the NIHR Journals Library website for further project information.
Collapse
Affiliation(s)
- Abimbola A Ayorinde
- Division of Health Sciences, Warwick Medical School, University of Warwick, Coventry, UK
| | - Iestyn Williams
- Health Services Management Centre, School of Social Policy, University of Birmingham, Birmingham, UK
| | - Russell Mannion
- Health Services Management Centre, School of Social Policy, University of Birmingham, Birmingham, UK
| | - Fujian Song
- Norwich Medical School, University of East Anglia, Norwich, UK
| | - Magdalena Skrybant
- Institute of Applied Health Research, University of Birmingham, Birmingham, UK
| | - Richard J Lilford
- Institute of Applied Health Research, University of Birmingham, Birmingham, UK
| | - Yen-Fu Chen
- Division of Health Sciences, Warwick Medical School, University of Warwick, Coventry, UK
| |
Collapse
|
5
|
Ayorinde AA, Williams I, Mannion R, Song F, Skrybant M, Lilford RJ, Chen YF. Publication and related biases in health services research: a systematic review of empirical evidence. BMC Med Res Methodol 2020; 20:137. [PMID: 32487022 PMCID: PMC7268600 DOI: 10.1186/s12874-020-01010-1] [Citation(s) in RCA: 9] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/28/2019] [Accepted: 05/07/2020] [Indexed: 02/08/2023] Open
Abstract
Background Publication and related biases (including publication bias, time-lag bias, outcome reporting bias and p-hacking) have been well documented in clinical research, but relatively little is known about their presence and extent in health services research (HSR). This paper aims to systematically review evidence concerning publication and related bias in quantitative HSR. Methods Databases including MEDLINE, EMBASE, HMIC, CINAHL, Web of Science, Health Systems Evidence, Cochrane EPOC Review Group and several websites were searched to July 2018. Information was obtained from: (1) Methodological studies that set out to investigate publication and related biases in HSR; (2) Systematic reviews of HSR topics which examined such biases as part of the review process. Relevant information was extracted from included studies by one reviewer and checked by another. Studies were appraised according to commonly accepted scientific principles due to lack of suitable checklists. Data were synthesised narratively. Results After screening 6155 citations, four methodological studies investigating publication bias in HSR and 184 systematic reviews of HSR topics (including three comparing published with unpublished evidence) were examined. Evidence suggestive of publication bias was reported in some of the methodological studies, but evidence presented was very weak, limited in both quality and scope. Reliable data on outcome reporting bias and p-hacking were scant. HSR systematic reviews in which published literature was compared with unpublished evidence found significant differences in the estimated intervention effects or association in some but not all cases. Conclusions Methodological research on publication and related biases in HSR is sparse. Evidence from available literature suggests that such biases may exist in HSR but their scale and impact are difficult to estimate for various reasons discussed in this paper. Systematic review registration PROSPERO 2016 CRD42016052333.
Collapse
Affiliation(s)
- Abimbola A Ayorinde
- Warwick Centre for Applied Health Research & Delivery, Division of Health Sciences, Warwick Medical School, University of Warwick, Coventry, UK
| | - Iestyn Williams
- Health Services Management Centre, School of Social Policy, University of Birmingham, Birmingham, UK
| | - Russell Mannion
- Health Services Management Centre, School of Social Policy, University of Birmingham, Birmingham, UK
| | - Fujian Song
- Norwich Medical School, University of East Anglia, Norwich, UK
| | - Magdalena Skrybant
- Institute of Applied Health Research, University of Birmingham, Birmingham, UK
| | - Richard J Lilford
- Institute of Applied Health Research, University of Birmingham, Birmingham, UK
| | - Yen-Fu Chen
- Warwick Centre for Applied Health Research & Delivery, Division of Health Sciences, Warwick Medical School, University of Warwick, Coventry, UK.
| |
Collapse
|
6
|
Jesus TS, Papadimitriou C, Pinho CS, Hoenig H. Key Characteristics of Rehabilitation Quality Improvement Publications: Scoping Review From 2010 to 2016. Arch Phys Med Rehabil 2017; 99:1141-1148.e4. [PMID: 28965737 DOI: 10.1016/j.apmr.2017.08.491] [Citation(s) in RCA: 11] [Impact Index Per Article: 1.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/27/2017] [Revised: 08/14/2017] [Accepted: 08/18/2017] [Indexed: 10/18/2022]
Abstract
OBJECTIVE To characterize the peer-reviewed quality improvement (QI) literature in rehabilitation. DATA SOURCES Five electronic databases were searched for English-language articles from 2010 to 2016. Keywords for QI and safety management were searched for in combination with keywords for rehabilitation content and journals. Secondary searches (eg, references-list scanning) were also performed. STUDY SELECTION Two reviewers independently selected articles using working definitions of rehabilitation and QI study types; of 1016 references, 112 full texts were assessed for eligibility. DATA EXTRACTION Reported study characteristics including study focus, study setting, use of inferential statistics, stated limitations, and use of improvement cycles and theoretical models were extracted by 1 reviewer, with a second reviewer consulted whenever inferences or interpretation were involved. DATA SYNTHESIS Fifty-nine empirical rehabilitation QI studies were found: 43 reporting on local QI activities, 7 reporting on QI effectiveness research, 8 reporting on QI facilitators or barriers, and 1 systematic review of a specific topic. The number of publications had significant yearly growth between 2010 and 2016 (P=.03). Among the 43 reports on local QI activities, 23.3% did not explicitly report any study limitations; 39.5% did not used inferential statistics to measure the QI impact; 95.3% did not cite/mention the appropriate reporting guidelines; only 18.6% reported multiple QI cycles; just over 50% reported using a model to guide the QI activity; and only 7% reported the use of a particular theoretical model. Study sites and focuses were diverse; however, nearly a third (30.2%) examined early mobilization in intensive care units. CONCLUSIONS The number of empirical, peer-reviewed rehabilitation QI publications is growing but remains a tiny fraction of rehabilitation research publications. Rehabilitation QI studies could be strengthened by greater use of extant models and theory to guide the QI work, consistent reporting of study limitations, and use of inferential statistics.
Collapse
Affiliation(s)
- Tiago S Jesus
- Global Health and Tropical Medicine & World Health Organization Collaborating Center for Health Workforce Policy and Planning, Institute of Hygiene and Tropical Medicine, NOVA University of Lisbon, Lisbon, Portugal.
| | | | - Cátia S Pinho
- ISVOUGA-Superior Institute of Entre Douro e Vouga, Santa Maria da Feira, Portugal
| | - Helen Hoenig
- Physical Medicine and Rehabilitation Service, Durham Veterans Administration Medical Center, Durham, NC; Division of Geriatrics, Department of Medicine, Duke University Medical Center, Durham, NC
| |
Collapse
|
7
|
The research activities of Ontario's large community acute care hospitals: a scoping review. BMC Health Serv Res 2017; 17:566. [PMID: 28814304 PMCID: PMC5559849 DOI: 10.1186/s12913-017-2517-4] [Citation(s) in RCA: 9] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/12/2017] [Accepted: 08/07/2017] [Indexed: 11/18/2022] Open
Abstract
Background Ontario’s large community hospitals (LCHs) provide care to 65% of the province’s hospitalized patients, yet we know very little about their research activities. By searching for research publications from 2013 to 2015, we will describe the extent, type and collaborative nature of Ontario’s LCHs’ research activities. Methods We conducted a scoping review by searching PubMed, Embase and the Cumulative Index to Nursing and Allied Health Literature databases from January 1, 2013 until December 31, 2015 for all publication types whose author(s) was affiliated with any of the 44 LCHs. Articles were screened and abstracted by three reviewers, independently. The data were charted and results described using summary statistics, scatter plots, and bar charts. Results We included 798 publications from 39 LCHs and 454 authors. The median number of publications was 7 (Interquartile range (IQR) 23). Observational study design was most commonly reported in over 50% of publications. Program evaluation was the focus in 40% of publications. Primary LCH authorship was observed for 535 publications. Over 25% and 65% of the publications were attributable to 24 authors and 9 LCHs, respectively. There was minimal collaboration both within (21.2%) and between (7.8%) LCHs. LCH size and geographic proximity to academic hospitals had minimal impact on research activity. Conclusions Ontario’s LCHs publish infrequently, collaborate infrequently, and their role in translational research activity is not well defined. A future survey questionnaire to LCH researchers identified through this review is planned to both validate and elicit their interpretations of our study findings and opinions about LCH involvement in research.
Collapse
|