Reaching consensus on reporting patient and public involvement (PPI) in research: methods and lessons learned from the development of reporting guidelines

INTRODUCTION

Patient and public involvement (PPI) is inconsistently reported in health and social care research. Improving the quality of how PPI is reported is critical in developing a higher quality evidence base to gain a better insight into the methods and impact of PPI. This paper describes the methods used to develop and gain consensus on guidelines for reporting PPI in research studies (updated version of the Guidance for Reporting Patient and Public Involvement (GRIPP2)).

METHODS

There were three key stages in the development of GRIPP2: identification of key items for the guideline from systematic review evidence of the impact of PPI on health research and health services, a three-phase online Delphi survey with a diverse sample of experts in PPI to gain consensus on included items and a face-to-face consensus meeting to finalise and reach definitive agreement on GRIPP2. Challenges and lessons learnt during the development of the reporting guidelines are reported.

DISCUSSION

The process of reaching consensus is vital within the development of guidelines and policy directions, although debate around how best to reach consensus is still needed. This paper discusses the critical stages of consensus development as applied to the development of consensus for GRIPP2 and discusses the benefits and challenges of consensus development.

Comparison of two methods to estimate adverse events in the IBEAS Study (Ibero-American study of adverse events): cross-sectional versus retrospective cohort design

BACKGROUND

Adverse events (AEs) epidemiology is the first step to improve practice in the healthcare system. Usually, the preferred method used to estimate the magnitude of the problem is the retrospective cohort study design, with retrospective reviews of the medical records. However this data collection involves a sophisticated sampling plan, and a process of intensive review of sometimes very heavy and complex medical records. Cross-sectional survey is also a valid and feasible methodology to study AEs.

OBJECTIVES

The aim of this study is to compare AEs detection using two different methodologies: cross-sectional versus retrospective cohort design.

SETTING

Secondary and tertiary hospitals in five countries: Argentina, Colombia, Costa Rica, Mexico and Peru.

PARTICIPANTS

The IBEAS Study is a cross-sectional survey with a sample size of 11 379 patients. The retrospective cohort study was obtained from a 10% random sample proportional to hospital size from the entire IBEAS Study population.

METHODS

This study compares the 1-day prevalence of the AEs obtained in the IBEAS Study with the incidence obtained through the retrospective cohort study.

RESULTS

The prevalence of patients with AEs was 10.47% (95% CI 9.90 to 11.03) (1191/11 379), while the cumulative incidence of the retrospective cohort study was 19.76% (95% CI 17.35% to 22.17%) (215/1088). In both studies the highest risk of suffering AEs was seen in Intensive Care Unit (ICU) patients. Comorbid patients and patients with medical devices showed higher risk.

CONCLUSION

The retrospective cohort design, although requires more resources, allows to detect more AEs than the cross-sectional design.

A Transcription and Translation Protocol for Sensitive Cross-Cultural Team Research

Assurance of transcript accuracy and quality in interview-based qualitative research is foundational for data accuracy and study validity. Based on our experience in a cross-cultural ethnographic study of women's pelvic organ prolapse, we provide practical guidance to set up step-by-step interview transcription and translation protocols for team-based research on sensitive topics. Beginning with team decisions about level of detail in transcription, completeness, and accuracy, we operationalize the process of securing vendors to deliver the required quality of transcription and translation. We also share rubrics for assessing transcript quality and the team protocol for managing transcripts (assuring consistency of format, insertion of metadata, anonymization, and file labeling conventions) and procuring an acceptable initial translation of Spanish-language interviews. Accurate, complete, and systematically constructed transcripts in both source and target languages respond to the call for more transparency and reproducibility of scientific methods.

Holding Firm: Power, Push-Back, and Opportunities in Navigating the Liminal Space of Critical Qualitative Health Research

Critical qualitative health researchers typically occupy and navigate liminal academic spaces and statuses, with one foot planted in the arts and social sciences and the other in biomedical science. We are at once marginalized and empowered, and this liminality presents both challenges and opportunities. In this article, we draw on our experiences of being (often the lone) critical qualitative health scholars on thesis advisory committees and dissertation examinations, as well as our experiences of publishing and securing funding, to illuminate how power and knowledge relations create conditions that shape the nature of our roles. We share strategies we have developed for standing our theoretical and methodological ground. We discuss how we use the power of our liminality to hold firm, push back, and push forward, to ensure that critical qualitative research is not further relegated to the margins and its quality and integrity sustained.

Where's the LGBT in integrated care research? A systematic review

INTRODUCTION

Lesbian, gay, bisexual, and transgender (LGBT) individuals experience more negative health outcomes compared with their heterosexual peers. The health disparities are often related to family and social rejection of the LGBT individuals. Integrated care, and Medical Family Therapy in particular, may aid in addressing the systemic nature of the negative health outcomes.

METHOD

To better understand the current state of the integrated care literature on addressing the health needs of LGBT individuals, a systematic review of the research literature was conducted from January 2000 to January 2016 for articles including integrated health care interventions for LGBT populations. Independent reviewers coded identified articles.

RESULTS

Only 8 research articles met criteria for inclusion out of the 2,553 initially identified articles in the search. Results indicated a lack of integrated care research on health care and health needs of LGBT individuals, and none of the articles addressed the use of family or systemic-level interventions.

DISCUSSION

Implications for future research and the need for better education training are discussed. (PsycINFO Database Record

Redundant systematic reviews on the same topic in surgery: a study protocol for a meta-epidemiological investigation

INTRODUCTION

We are witnessing an explosive increase in redundant and overlapping publications of systematic reviews and meta-analyses (SRs/MAs) on the same topic, which often present conflicting results and interpretations, in the current medical literature. They represent wasted efforts on the part of investigators and peer reviewers and may confuse and possibly mislead clinicians and policymakers. Here, we present a protocol for a meta-epidemiological investigation to describe how often there are overlapping SRs/MAs on the same topic, to assess the quality of these multiple publications, and to investigate the causes of discrepant results between multiple SRs/MAs in the field of major surgery.

METHODS AND ANALYSIS

We will use MEDLINE/PubMed to identify all SRs/MAs of randomised controlled trials (RCTs) published in 2015 regarding major surgical interventions. After identifying the 'benchmark' SRs/MAs published in 2015, a process of screening in MEDLINE will be carried out to identify the previous SRs/MAs of RCTs on the same topic that were published within 5 years of the 'benchmark' SRs/MAs. We will tabulate the number of previous SRs/MAs on the same topic of RCTs, and then describe their variations in numbers of RCTs included, sample sizes, effect size estimates and other characteristics. We will also assess the differences in quality of each SR/MA using A Measurement Tool to Assess Systematic Reviews (AMSTAR) score. Finally, we will investigate the potential reasons to explain the discrepant results between multiple SRs/MAs.

ETHICS AND DISSEMINATION

No formal ethical approval and informed consent are required because this study will not collect primary individual data. The intended audiences of the findings include clinicians, healthcare researchers and policymakers. We will publish our findings as a scientific report in a peer-reviewed journal.

TRIAL REGISTRATION NUMBER

In PROSPERO CRD42017059077, March 2017.

Assessing the impact of healthcare research: A systematic review of methodological frameworks

BACKGROUND

Increasingly, researchers need to demonstrate the impact of their research to their sponsors, funders, and fellow academics. However, the most appropriate way of measuring the impact of healthcare research is subject to debate. We aimed to identify the existing methodological frameworks used to measure healthcare research impact and to summarise the common themes and metrics in an impact matrix.

METHODS AND FINDINGS

Two independent investigators systematically searched the Medical Literature Analysis and Retrieval System Online (MEDLINE), the Excerpta Medica Database (EMBASE), the Cumulative Index to Nursing and Allied Health Literature (CINAHL+), the Health Management Information Consortium, and the Journal of Research Evaluation from inception until May 2017 for publications that presented a methodological framework for research impact. We then summarised the common concepts and themes across methodological frameworks and identified the metrics used to evaluate differing forms of impact. Twenty-four unique methodological frameworks were identified, addressing 5 broad categories of impact: (1) 'primary research-related impact', (2) 'influence on policy making', (3) 'health and health systems impact', (4) 'health-related and societal impact', and (5) 'broader economic impact'. These categories were subdivided into 16 common impact subgroups. Authors of the included publications proposed 80 different metrics aimed at measuring impact in these areas. The main limitation of the study was the potential exclusion of relevant articles, as a consequence of the poor indexing of the databases searched.

CONCLUSIONS

The measurement of research impact is an essential exercise to help direct the allocation of limited research resources, to maximise research benefit, and to help minimise research waste. This review provides a collective summary of existing methodological frameworks for research impact, which funders may use to inform the measurement of research impact and researchers may use to inform study design decisions aimed at maximising the short-, medium-, and long-term impact of their research.

Interventions to improve hospital patient satisfaction with healthcare providers and systems: a systematic review

BACKGROUND

Many hospital systems seek to improve patient satisfaction as assessed by the Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS) surveys. A systematic review of the current experimental evidence could inform these efforts and does not yet exist.

METHODS

We conducted a systematic review of the literature by searching electronic databases, including MEDLINE and EMBASE, the six databases of the Cochrane Library and grey literature databases. We included studies involving hospital patients with interventions targeting at least 1 of the 11 HCAHPS domains, and that met our quality filter score on the 27-item Downs and Black coding scale. We calculated post hoc power when appropriate.

RESULTS

A total of 59 studies met inclusion criteria, out of these 44 did not meet the quality filter of 50% (average quality rating 27.8%±10.9%). Of the 15 studies that met the quality filter (average quality rating 67.3%±10.7%), 8 targeted the Communication with Doctors HCAHPS domain, 6 targeted Overall Hospital Rating, 5 targeted Communication with Nurses, 5 targeted Pain Management, 5 targeted Communication about Medicines, 5 targeted Recommend the Hospital, 3 targeted Quietness of the Hospital Environment, 3 targeted Cleanliness of the Hospital Environment and 3 targeted Discharge Information. Significant HCAHPS improvements were reported by eight interventions, but their generalisability may be limited by narrowly focused patient populations, heterogeneity of approach and other methodological concerns.

CONCLUSIONS

Although there are a few studies that show some improvement in HCAHPS score through various interventions, we conclude that more rigorous research is needed to identify effective and generalisable interventions to improve patient satisfaction.

Understanding qualitative research in health care

Qualitative studies are often used to research phenomena that are difficult to quantify numerically.^1,2 These may include concepts, feelings, opinions, interpretations and meanings, or why people behave in a certain way. Although qualitative research is often described in opposition to quantitative research, the approaches are complementary, and many researchers use mixed methods in their projects, combining the strengths of both approaches.² Many comprehensive texts exist on qualitative research methodology including those with a focus on healthcare related research.^2-4 Here we give a brief introduction to the rationale, methods and quality assessment of qualitative research.

Statistics, Adjusted Statistics, and Maladjusted Statistics

Statistical adjustment is a ubiquitous practice in all quantitative fields that is meant to correct for improprieties or limitations in observed data, to remove the influence of nuisance variables or to turn observed correlations into causal inferences. These adjustments proceed by reporting not what was observed in the real world, but instead modeling what would have been observed in an imaginary world in which specific nuisances and improprieties are absent. These techniques are powerful and useful inferential tools, but their application can be hazardous or deleterious if consumers of the adjusted results mistake the imaginary world of models for the real world of data. Adjustments require decisions about which factors are of primary interest and which are imagined away, and yet many adjusted results are presented without any explanation or justification for these decisions. Adjustments can be harmful if poorly motivated, and are frequently misinterpreted in the media's reporting of scientific studies. Adjustment procedures have become so routinized that many scientists and readers lose the habit of relating the reported findings back to the real world in which we live.

Balancing the Optimal and the Feasible: A Practical Guide for Setting Up Patient Registries for the Collection of Real-World Data for Health Care Decision Making Based on Dutch Experiences

OBJECTIVES

The aim of this article was to provide practical guidance in setting up patient registries to facilitate real-world data collection for health care decision making.

METHODS

This guidance was based on our experiences and involvement in setting up patient registries in oncology in the Netherlands. All aspects were structured according to 1) mission and goals ("the Why"), 2) stakeholders and funding ("the Who"), 3) type and content ("the What"), and 4) identification and recruitment of patients, data handling, and pharmacovigilance ("the How").

RESULTS

The mission of most patient registries is improving patient health by improving the quality of patient care; monitoring and evaluating patient care is often the primary goal ("the Why"). It is important to align the objectives of the registry and agree on a clear and functional governance structure with all stakeholders ("the Who"). There is often a trade off between reliability, validity, and specificity of data elements and feasibility of data collection ("the What"). Patient privacy should be carefully protected, and address (inter-)national and local regulations. Patient registries can reveal unique safety information, but it can be challenging to comply with pharmacovigilance guidelines ("the How").

CONCLUSIONS

It is crucial to set up an efficient patient registry that serves its aims by collecting the right data of the right patient in the right way. It can be expected that patient registries will become the new standard alongside randomized controlled trials due to their unique value.

Cost-Effectiveness Thresholds in Global Health: Taking a Multisectoral Perspective

Good health is a function of a range of biological, environmental, behavioral, and social factors. The consumption of quality health care services is therefore only a part of how good health is produced. Although few would argue with this, the economic framework used to allocate resources to optimize population health is applied in a way that constrains the analyst and the decision maker to health care services. This approach risks missing two critical issues: 1) multiple sectors contribute to health gain and 2) the goods and services produced by the health sector can have multiple benefits besides health. We illustrate how present cost-effectiveness thresholds could result in health losses, particularly when considering health-producing interventions in other sectors or public health interventions with multisectoral outcomes. We then propose a potentially more optimal second best approach, the so-called cofinancing approach, in which the health payer could redistribute part of its budget to other sectors, where specific nonhealth interventions achieved a health gain more efficiently than the health sector's marginal productivity (opportunity cost). Likewise, other sectors would determine how much to contribute toward such an intervention, given the current marginal productivity of their budgets. Further research is certainly required to test and validate different measurement approaches and to assess the efficiency gains from cofinancing after deducting the transaction costs that would come with such cross-sectoral coordination.

A proposed national research and development agenda for population health informatics: summary recommendations from a national expert workshop

OBJECTIVE

The Johns Hopkins Center for Population Health IT hosted a 1-day symposium sponsored by the National Library of Medicine to help develop a national research and development (R&D) agenda for the emerging field of population health informatics (PopHI).

MATERIAL AND METHODS

The symposium provided a venue for national experts to brainstorm, identify, discuss, and prioritize the top challenges and opportunities in the PopHI field, as well as R&D areas to address these.

RESULTS

This manuscript summarizes the findings of the PopHI symposium. The symposium participants' recommendations have been categorized into 13 overarching themes, including policy alignment, data governance, sustainability and incentives, and standards/interoperability.

DISCUSSION

The proposed consensus-based national agenda for PopHI consisted of 18 priority recommendations grouped into 4 broad goals: (1) Developing a standardized collaborative framework and infrastructure, (2) Advancing technical tools and methods, (3) Developing a scientific evidence and knowledge base, and (4) Developing an appropriate framework for policy, privacy, and sustainability. There was a substantial amount of agreement between all the participants on the challenges and opportunities for PopHI as well as on the actions that needed to be taken to address these.

CONCLUSION

PopHI is a rapidly growing field that has emerged to address the population dimension of the Triple Aim. The proposed PopHI R&D agenda is comprehensive and timely, but should be considered only a starting-point, given that ongoing developments in health policy, population health management, and informatics are very dynamic, suggesting that the agenda will require constant monitoring and updating.

Action research healthcare: Focus on patients, improve quality, drive down costs

We discuss action research healthcare as a transformative approach that continuously innovates in healthcare, attending to the "quadruple" aim. This article is shaped around a decade of evidence in Sweden. At the heart of healthcare action research is the endeavour to "learn by doing" with the participation of key stakeholders, including the patient. Experience suggests that an action research approach is particularly relevant when treating patients with chronic diseases and complex care needs. This inclusion is itself a social learning process and is key to realizing the improved outcomes. Insights from objective quantitative studies are balanced with personal and inter-subjective dialogue that aligns different parts of a system in a movement towards improvement. Close-up non-defensive self-inquiry in the company of colleagues, with trust dynamics building over time, may be a key point of leverage for such systemic improvement activities.

Primary and secondary patient data in contrast: the use of observational studies like RABBIT

The study of secondary patient data, particularly represented by claims data, has increased in recent years. The strength of this approach involves easy access to data that have been generated for administrative purposes. By contrast, collection of primary data for research is time-consuming and may therefore appear outdated. Both administrative data and data collected prospectively in clinical care can address similar research questions concerning effectiveness and safety of treatments. Therefore, why should we invest the precious time of rheumatologists to generate primary patient data? This article will outline some features of primary patient data collection illustrated by the German biologics register RABBIT (Rheumatoid arthritis: observation of biologic therapy). RABBIT is a long-term observational cohort study that was initiated more than 15 years ago. We will discuss as quality indicators: (i) study design, (ii) type of documentation, standardisation of (iii) clinical and (iv) safety data, (v) monitoring of the longitudinal follow-up, (vi) losses to follow-up as well as (vii) the possibilities to link the data base. The impact of these features on interpretation and validity of results is illustrated using recent publications. We conclude that high quality and completeness of data prospectively-collected offers many advantages over large quantities of non-standardised data collected in an unsupervised manner. We expect the enthusiasm about the use of secondary patient data to decline with more awareness of their methodological limitations while studies with primary patient data like RABBIT will maintain and broaden their impact on daily clinical practice.

A review of research on direct-care staff data collection regarding the severity and function of challenging behavior in individuals with intellectual and developmental disabilities

In working with individuals with intellectual and developmental disabilities (IDDs), it is direct care staff who are often required to collect data on individuals' behavior which is used as the basis for implementation of empirically based approaches for intervention and treatment. Due to limited resources, indirect and descriptive measures of challenging behaviors are employed to analyze the function of individuals' behaviors in place of the preferred method of multimodal assessment, which includes experimental functional analysis. To ensure the most effective services and support to individuals with IDDs, accurate and consistent data collection is critical. In this article, we highlight the importance of accurate data collection practices, conduct a comparison of data collection methods, and discuss limitations .… and barriers for staff. The article concludes with recommendations for best practices and future research.

Modeling Psychological Empowerment Among Youth Involved in Local Tobacco Control Efforts

The American Legacy Foundation funded 13 state health departments for their Statewide Youth Movement Against Tobacco Use in September 2000. Its goal was to create statewide tobacco control initiatives implemented with youth leadership. The underlying theory behind these initiatives was that tobacco control efforts can best be accomplished by empowering youth. To evaluate these initiatives, the authors developed a conceptual framework for youth empowerment that was used as a guide in developing standardized cross-site measures. This article describes the domains and attributes used to operationalize psychological empowerment as an outcome of youth involvement in these initiatives and presents results of our two-stage structural equation modeling. We conclude with a summary of lessons learned to date and recommendations for applying these findings to work in the field.

Development of Measures of Organizational Leadership for Health Promotion

This study describes the development and psychometric evaluation of four scales measuring leadership for health promotion at an organizational level in the baseline survey ( n = 144) of the Alberta Heart Health Project. Content validity was established through a series of focus groups and expert opinion appraisals, pilot testing of a draft based on capacity assessment instruments developed by other provinces involved in the Canadian Heart Health Initiative, and the literature. Psychometric analyses provided empirical evidence of the construct validity and reliability of the organizational leadership scales. Principal component analysis verified the unidimensionality of the leadership scales of (a) Practices for Organizational Learning, (b) Wellness Planning, (c) Workplace Climate, and (d) Organization Member Development. Scale alpha coefficients ranged between .79 and .91 thus establishing good to high scale internal consistencies. These measures can be used by both researchers and practitioners for the assessment of organizational leadership for health promotion and heart health promotion.

Sample Size Requirements for Discrete-Choice Experiments in Healthcare: a Practical Guide

Discrete-choice experiments (DCEs) have become a commonly used instrument in health economics and patient-preference analysis, addressing a wide range of policy questions. An important question when setting up a DCE is the size of the sample needed to answer the research question of interest. Although theory exists as to the calculation of sample size requirements for stated choice data, it does not address the issue of minimum sample size requirements in terms of the statistical power of hypothesis tests on the estimated coefficients. The purpose of this paper is threefold: (1) to provide insight into whether and how researchers have dealt with sample size calculations for healthcare-related DCE studies; (2) to introduce and explain the required sample size for parameter estimates in DCEs; and (3) to provide a step-by-step guide for the calculation of the minimum sample size requirements for DCEs in health care.