New horizons in the implementation and research of comprehensive geriatric assessment: knowing, doing and the 'know-do' gap

In this paper, we outline the relationship between the need to put existing applied health research knowledge into practice (the 'know-do gap') and the need to improve the evidence base (the 'know gap') with respect to the healthcare process used for older people with frailty known as comprehensive geriatric assessment (CGA). We explore the reasons for the know-do gap and the principles of how these barriers to implementation might be overcome. We explore how these principles should affect the conduct of applied health research to close the know gap. We propose that impaired flow of knowledge is an important contributory factor in the failure to implement evidence-based practice in CGA; this could be addressed through specific knowledge mobilisation techniques. We describe that implementation failures are also produced by an inadequate evidence base that requires the co-production of research, addressing not only effectiveness but also the feasibility and acceptability of new services, the educational needs of practitioners, the organisational requirements of services, and the contribution made by policy. Only by tackling these issues in concert and appropriate proportion, will the know and know-do gaps for CGA be closed.

Psychotherapy Outcome Research: Issues and Questions

Emphasis on identifying evidence-based therapies (EBTs) has increased markedly. Lists of EBTs are the rationale for recommendations for how psychotherapy provider training programs should be evaluated, professional competence assessed, and licensure and reimbursement policies structured. There are however methodological concerns that limit the external validity of EBTs. Among the most salient is the circularity inherent in randomized control trials (RCTs) of psychotherapy that constrains the manner in which the psychological problems are defined, psychotherapy can be practiced, and change evaluated. RCT studies favor therapies that focus of specific symptoms and can be described in a manual, administered reliably across patients, completed in relatively few sessions, and involve short-term evaluations of outcome. The epistemological assumptions of a natural science approach to psychotherapy research limit how studies are conducted and assessed in ways that that advantage symptom-focused approaches and disadvantage those approaches that seek to bring broad recovery-based changes. Research methods that are not limited to RCTs and include methodology to minimize the effects of "therapist allegiance" are necessary for valid evaluations of therapeutic approaches that seek to facilitate changes that are broader than symptom reduction. Recent proposals to adopt policies that dictate training, credentialing, and reimbursement based on lists of EBTs unduly limit how psychotherapy can be conceptualized and practiced, and are not in the best interests of the profession or of individuals seeking psychotherapy services.

Clinical Cancer Registries - Are They Up for Health Services Research?

Clinical cancer registries are a valuable data source for health services research (HSR). HSR is in need of high quality routine care data for its evaluations. However, the secondary use of routine data - such as documented cancer cases in a disease registry - poses new challenges in terms of data quality, IT-management, documentation processes and data privacy. In the clinical cancer registry Heilbronn-Franken, real-world data from the Giessen Tumor Documentation System (GTDS) was utilized for analyses of patients' disease processes and guideline adherence in follow-up care. A process was developed to map disease state definitions to fields of the GTDS database and extract patients' disease progress information. Thus, the disease process of sub-cohorts could be compared to each other, e.g., comparison of disease free survival of HER2 (human epidermal growth factor receptor 2)-positive and -negative women who were treated with Trastuzumab, a targeted therapy applied in breast cancer. In principle, such comparisons are feasible and of great value for HSR as they depict a routine care setting of a diverse patient cohort. Yet, local documentation practice, missing flow of information from external health care providers or small sub-cohorts impede the analyses of clinical cancer registries data bases and usage for HSR.

Research is 'a step into the unknown': an exploration of pharmacists' perceptions of factors impacting on research participation in the NHS

OBJECTIVE

This study explored National Health Service (NHS) pharmacists' perceptions and experiences of pharmacist-led research in the workplace.

DESIGN

Semistructured, face-to-face discussions continued until distinct clusters of opinion characteristics formed. Verbatim transcripts of audio-recordings were subjected to framework analysis.

SETTING

Interviews were carried out with 54 pharmacists with diverse backgrounds and roles from general practices and secondary care in the UK's largest health authority.

RESULTS

The purpose and potential of health services research (HSR) was understood and acknowledged to be worthwhile by participants, but a combination of individual and system-related themes tended to make participation difficult, except when this was part of formal postgraduate education leading to a qualification. Lack of prioritisation was routinely cited as the greatest barrier, with motivation, confidence and competence as additional impediments. System-related themes included lack of practical support and pharmacy professional issues. A minority of highly motivated individuals managed to embed research participation into routine activity.

CONCLUSIONS

Most pharmacists realised the desirability and necessity of research to underpin pharmacy service expansion, but a combination of individual and professional level changes is needed to increase activity. Our findings provide a starting point for better understanding the mindset of hospital-based and general practice-based pharmacists towards research, as well as their perceived barriers and supports.

Building an Australasian paramedicine research agenda: a narrative review

The need for paramedicine research has been recognised internationally through efforts to develop out-of-hospital research agendas in several developed countries. Australasia has a substantial paramedicine research capacity compared to the discipline internationally and is well positioned as a potential leader in the drive towards evidence-based policy and practice in paramedicine. Our objective was to draw on international experiences to identify and recommend the best methodological approach that should be employed to develop an Australasian paramedicine research agenda. A search and critical appraisal process was employed to produce an overview of the literature related to the development of paramedicine research agendas throughout the world. Based on these international experiences, and our own analysis of the Australasian context, we recommend that a mixed methods approach be used to develop an inclusive Australasian Paramedicine Research Agenda. This approach will capture the views and interests of a wide range of expert stakeholders through multiple data collection strategies, including interviews, roundtable discussions and an online Delphi consensus survey. Paramedic researchers and industry leaders have the opportunity to use this multidisciplinary process of inquiry to develop a paramedicine research agenda that will provide a framework for the development of a culture of open evaluation, innovation and improvement. This research agenda would assess the progress of paramedicine research in Australia and New Zealand, map the research capacity of the paramedicine discipline, paramedic services, universities and professional organisations, identify current strengths and opportunities, make recommendations to capitalize on opportunities, and identify research priorities. Success will depend on ensuring the participation of a representative sample of expert stakeholders, fostering an open and collaborative roundtable discussion, and adhering to a predefined approach to measure consensus on each topic.

The NIHR Public Health Research Programme: responding to local authority research needs in the United Kingdom

The remit of the National Institute for Health Research Public Health Research (PHR) Programme is to evaluate public health interventions, providing new knowledge on the benefits, costs, acceptability and wider impacts of interventions, set outside of the National Health Service, intended to improve the health of the public and reduce inequalities. This paper illustrates how the PHR Programme is providing new knowledge for public health decision makers, based on the nine key areas for local authority public health action, described by the King's Fund. Many funded PHR projects are evaluating interventions, applied in a range of settings, across the identified key areas for local authority influence. For example, research has been funded on children and young people, and for some of the wider determinants of health, such as housing and travel. Other factors, such as spatial planning, or open and green spaces and leisure, are less represented in the PHR Programme. Further opportunities in research include interventions to improve the health of adolescents, adults in workplaces, and communities. Building evidence for public health interventions at local authority level is important to prioritise and implement effective changes to improve population health.

European Code against Cancer 4th Edition: Process of reviewing the scientific evidence and revising the recommendations

The European Code Against Cancer is a set of recommendations to give advice on cancer prevention. Its 4th edition is an update of the 3rd edition, from 2003. Working Groups of independent experts from different fields of cancer prevention were appointed to review the recommendations, supported by a Literature Group to provide scientific and technical support in the assessment of the scientific evidence, through systematic reviews of the literature. Common procedures were developed to guide the experts in identifying, retrieving, assessing, interpreting and summarizing the scientific evidence in order to revise the recommendations. The Code strictly followed the concept of providing advice to European Union citizens based on the current best available science. The advice, if followed, would be expected to reduce cancer risk, referring both to avoiding or reducing exposure to carcinogenic agents or changing behaviour related to cancer risk and to participating in medical interventions able to avert specific cancers or their consequences. The information sources and procedures for the review of the scientific evidence are described here in detail. The 12 recommendations of the 4th edition of the European Code Against Cancer were ultimately approved by a Scientific Committee of leading European cancer and public health experts.

On the Brink of Genuinely Collaborative Care: Experience-Based Co-Design in Mental Health

Inpatient mental health services in the United Kingdom are currently dissatisfactory for service-users and staff. For young people with psychosis, being hospitalized is often distressing, and can lead to disengagement with mental health services. This article describes how we took three qualitative research studies about hospitalization in early psychosis (exploring the perspectives of service-users, parents, and staff) and translated them into service improvements developed in collaboration with a range of stakeholders, including service-users, carers, community and inpatient staff, and management. We used an adapted form of experience-based co-design (EBCD), a participatory action-research method for collaboratively improving health care services. The use of EBCD is still relatively novel in mental health settings, and we discuss how we adapted the methodology, and some of the implications of using EBCD with vulnerable populations in complex services. We reflect on both the disappointments and successes and give some recommendations for future research and methodological development.

Inside the "Black Box" of a Knowledge Translation Program in Applied Health Research

In this article, we present the findings of a participatory realistic evaluation of a 5-year program of health care research intended to promote the translation of knowledge into routine clinical practice. The program was one of the nine pilot Collaborations for Leadership in Applied Health Research and Care funded by the English National Institute for Health Research between 2008 and 2013. Our aim was to delineate the mechanisms by which, and circumstances in which, some projects carried out under the program achieved success in knowledge translation while others were frustrated. Using qualitative methods, we examined how closer collaboration between academics and clinicians worked in four purposefully chosen case studies. In a synthesis of the findings, we produced a "black box" model of how knowledge translation was enabled by the activation of nine mechanisms. These are summarized in the form of five simple rules for promoting knowledge translation through collaborations based on principles of coproduction.

Squire 2.0 (Standards for Quality Improvement Reporting Excellence): revised publication guidelines from a detailed consensus process

Since the publication of Standards for Quality Improvement Reporting Excellence (SQUIRE 1.0) guidelines in 2008, the science of the field has advanced considerably. In this manuscript, we describe the development of SQUIRE 2.0 and its key components. We undertook the revision between 2012 and 2015 using (1) semistructured interviews and focus groups to evaluate SQUIRE 1.0 plus feedback from an international steering group, (2) two face-to-face consensus meetings to develop interim drafts, and (3) pilot testing with authors and a public comment period. SQUIRE 2.0 emphasizes the reporting of 3 key components of systematic efforts to improve the quality, value, and safety of health care: the use of formal and informal theory in planning, implementing, and evaluating improvement work; the context in which the work is done; and the study of the intervention(s). SQUIRE 2.0 is intended for reporting the range of methods used to improve health care, recognizing that they can be complex and multidimensional. It provides common ground to share these discoveries in the scholarly literature (www.squire-statement.org).

Use of standardised patients to assess quality of tuberculosis care: a pilot, cross-sectional study

BACKGROUND

Existing studies of the quality of tuberculosis care have relied on recall-based patient surveys, questionnaire surveys of knowledge, and prescription or medical record analysis, and the results mostly show the health-care provider's knowledge rather than actual practice. No study has used standardised patients to assess clinical practice. Therefore we aimed to assess quality of care for tuberculosis using such patients.

METHODS

We did a pilot, cross-sectional validation study of a convenience sample of consenting private health-care providers in low-income and middle-income areas of Delhi, India. We recruited standardised patients in apparently good health from the local community to present four cases (two of presumed tuberculosis and one each of confirmed tuberculosis and suspected multidrug-resistant tuberculosis) to a randomly allocated health-care provider. The key objective was to validate the standardised-patient method using three criteria: negligible risk and ability to avoid adverse events for providers and standardised patients, low detection rates of standardised patients by providers, and data accuracy across standardised patients and audio verification of standardised-patient recall. We also used medical vignettes to assess providers' knowledge of presumed tuberculosis. Correct case management was benchmarked using Standards for Tuberculosis Care in India (STCI).

FINDINGS

Between Feb 2, and March 28, 2014, we recruited and trained 17 standardised patients who had 250 interactions with 100 health-care providers, 29 of whom were qualified in allopathic medicine (ie, they had a Bachelor of Medicine & Surgery [MBBS] degree), 40 of whom practised alternative medicine, and 31 of whom were informal health-care providers with few or no qualifications. The interactions took place between April 1, and April 23, 2014. The proportion of detected standardised patients was low (11 [5%] detected out of 232 interactions among providers who completed the follow-up survey), and standardised patients' recall correlated highly with audio recordings (r=0·63 [95% CI 0·53-0·79]), with no safety concerns reported. The mean consultation length was 6 min (95% CI 5·5-6·6) with a mean of 6·18 (5·72-6·64) questions or examinations completed, representing 35% (33-38) of essential checklist items. Across all cases, only 52 (21% [16-26]) of 250 were correctly managed. Correct management was higher among MBBS-qualified doctors than other types of health-care provider (adjusted odds ratio 2·41 [95% CI 1·17-4·93]; p=0·0166). Of the 69 providers who completed the vignette, knowledge in the vignettes was more consistent with STCI than their actual clinical practice-eg, 50 (73%) ordered a chest radiograph or sputum test during the vignette compared with seven (10%) during the standardised-patient interaction; OR 0·04 (95% CI 0·02-0·11); p<0·0001.

INTERPRETATION

Standardised patients can be successfully implemented to assess tuberculosis care. Our data suggest a big gap between private provider knowledge and practice. Additional work is needed to substantiate our pilot data, understand the know-do gap in provider behaviour, and to identify the best approach to measure and improve the quality of tuberculosis care in India.

FUNDING

Grand Challenges Canada, the Bill & Melinda Gates Foundation, Knowledge for Change Program, and the World Bank Development Research Group.

Analyzing Data Generated Through Deliberative Dialogue: Bringing Knowledge Translation Into Qualitative Analysis

Deliberative dialogue (DD) is a knowledge translation strategy that can serve to generate rich data and bridge health research with action. An intriguing alternative to other modes of generating data, the purposeful and evidence-informed conversations characteristic of DD generate data inclusive of collective interpretations. These data are thus dialogic, presenting complex challenges for qualitative analysis. In this article, we discuss the nature of data generated through DD, orienting ourselves toward a theoretically grounded approach to analysis. We offer an integrated framework for analysis, balancing analytical strategies of categorizing and connecting with the use of empathetic and suspicious interpretive lenses. In this framework, data generation and analysis occur in concert, alongside engaging participants and synthesizing evidence. An example of application is provided, demonstrating nuances of the framework. We conclude with reflections on the strengths and limitations of the framework, suggesting how it may be relevant in other qualitative health approaches.

Introducing Evidence Through Research "Push": Using Theory and Qualitative Methods

A multitude of factors can influence the uptake and implementation of complex interventions in health care. A plethora of theories and frameworks recognize the need to establish relationships, understand organizational dynamics, address context and contingency, and engage key decision makers. Less attention is paid to how theories that emphasize relational contexts can actually be deployed to guide the implementation of an intervention. The purpose of the article is to demonstrate the potential role of qualitative research aligned with theory to inform complex interventions. We detail a study underpinned by theory and qualitative research that (a) ensured key actors made sense of the complex intervention at the earliest stage of adoption and (b) aided initial engagement with the intervention. We conclude that using theoretical approaches aligned with qualitative research can provide insights into the context and dynamics of health care settings that in turn can be used to aid intervention implementation.

Exploring the Hidden Barriers in Knowledge Translation: A Case Study Within an Academic Community

Debates about knowledge translation (KT) typically focus on the research-practice gap, which appears to be premised on the assumption that academics are a homogeneous collective, sharing a common view. We argue that a number of hidden barriers need to be addressed related to the understanding, interpretation, ability, and commitment to translate knowledge within academic communities. We explore this by presenting a qualitative case study in a health sciences faculty. Applying organizational and management theory, we discuss different types of boundaries and the resultant barriers generated, ranging from diversity in understanding and perceptions of KT to varying motivations and incentives to engage in translational activity. We illustrate how we are using the empirical findings to inform the development of a KT strategy that targets the identified barriers. Investing in this internal KT-focused activity is an important step to maximize the potential of future collaborations between producers and users of research in health care.

Developing and evaluating guidelines for patient and public involvement (PPI) in research

PURPOSE

A growing literature reports the benefits and challenges of patient and public involvement (PPI) in research; nevertheless, understanding PPI in research design remains under-developed. The purpose of this paper is to report learning experiences from involving service users as research partners in two projects that developed and evaluated guidelines for good practice in this regard. The main objective was to evaluate these guidelines.

DESIGN/METHODOLOGY/APPROACH

PPI research guidelines were developed through five workshops involving service users/patients, carers, health and social care professionals/managers and academics. Using a participatory qualitative approach, these guidelines were evaluated through mapping them against the two service user research partners' experience within another project.

FINDINGS

The guidelines were found to be fit for purpose, as they allowed problems to be easily identified and reassurance that required standards were being met. Both academic and service user research partners learned and gained relevant skills. Two service user research partners also found their daily living skills unexpectedly enhanced by project participation.

ORIGINALITY/VALUE

The PPI guidelines, the authors developed were produced by consensus involving several stakeholders. Service users involved as research partners in the project experienced unanticipated personal benefits.

Policy recommendations for rare disease centres of expertise

AIM

Rare diseases are a serious public health concern and are a priority in the EU. This study aims to develop policy recommendations for rare disease centres of expertise (CoEs) in order to improve standards and quality of care.

SUBJECT AND METHODS

A modified 3-round Delphi technique was used. Participants included rare diseases patients, carers, patient representatives and healthcare professionals (HCPs) from CoEs in two countries-Denmark and the UK.

RESULTS

The results suggest the need to make improvements within current CoE environments, access to CoEs and the need for coordination and cooperation of services within and outside CoEs. It is recommended that CoEs are not overly 'medicalised', while at the same time they should be established as research facilities. The importance of including patient representatives in CoE performance management was also highlighted. Raising awareness and provision of appropriate training amongst non-specialist HCPs is seen as a priority for early and correct diagnosis and ensuring high quality care. Similarly, provision of targeted information about patients' illness and care was considered essential along with access to social assistance within CoEs.

CONCLUSIONS

Policy recommendations were developed in areas previously recognised as having gaps. Their implementation is expected to strengthen and improve current care provision for rare disease patients. In member states where national plans and strategies are being developed, it is recommended to replicate the methodological approach used in this study as it has proven to be a helpful tool in rare disease centres of expertise policy development.

Clinical Outcome Assessments: Conceptual Foundation-Report of the ISPOR Clinical Outcomes Assessment - Emerging Good Practices for Outcomes Research Task Force

An outcome assessment, the patient assessment used in an endpoint, is the measuring instrument that provides a rating or score (categorical or continuous) that is intended to represent some aspect of the patient's health status. Outcome assessments are used to define efficacy endpoints when developing a therapy for a disease or condition. Most efficacy endpoints are based on specified clinical assessments of patients. When clinical assessments are used as clinical trial outcomes, they are called clinical outcome assessments (COAs). COAs include any assessment that may be influenced by human choices, judgment, or motivation. COAs must be well-defined and possess adequate measurement properties to demonstrate (directly or indirectly) the benefits of a treatment. In contrast, a biomarker assessment is one that is subject to little, if any, patient motivational or rater judgmental influence. This is the first of two reports by the ISPOR Clinical Outcomes Assessment - Emerging Good Practices for Outcomes Research Task Force. This report provides foundational definitions important for an understanding of COA measurement principles. The foundation provided in this report includes what it means to demonstrate a beneficial effect, how assessments of patients relate to the objective of showing a treatment's benefit, and how these assessments are used in clinical trial endpoints. In addition, this report describes intrinsic attributes of patient assessments and clinical trial factors that can affect the properties of the measurements. These factors should be considered when developing or refining assessments. These considerations will aid investigators designing trials in their choice of using an existing assessment or developing a new outcome assessment. Although the focus of this report is on the development of a new COA to define endpoints in a clinical trial, these principles may be applied more generally. A critical element in appraising or developing a COA is to describe the treatment's intended benefit as an effect on a clearly identified aspect of how a patient feels or functions. This aspect must have importance to the patient and be part of the patient's typical life. This meaningful health aspect can be measured directly or measured indirectly when it is impractical to evaluate it directly or when it is difficult to measure. For indirect measurement, a concept of interest (COI) can be identified. The COI must be related to how a patient feels or functions. Procedures are then developed to measure the COI. The relationship of these measurements with how a patient feels or functions in the intended setting and manner of use of the COA (the context of use) could then be defined. A COA has identifiable attributes or characteristics that affect the measurement properties of the COA when used in endpoints. One of these features is whether judgment can influence the measurement, and if so, whose judgment. This attribute defines four categories of COAs: patient reported outcomes, clinician reported outcomes, observer reported outcomes, and performance outcomes. A full description as well as explanation of other important COA features is included in this report. The information in this report should aid in the development, refinement, and standardization of COAs, and, ultimately, improve their measurement properties.

What Guidance are Economists Given on How to Present Economic Evaluations for Policymakers? A Systematic Review

OBJECTIVES

To systematically review health economic guidelines for information on how to present health economic evaluations and consider implications for nontechnical audiences such as policymakers.

METHODS

Electronic databases and supplementary sources were searched for economic evaluation guidelines. Guidelines were critically appraised. Descriptive characteristics, standard formats, supports for nontechnical audiences, presentation approaches, and common reporting recommendations were extracted. Frequencies were tabulated and trends identified.

RESULTS

Thirty-one guidelines were included. Twenty-two guidelines include a standard reporting format with some sample tables and graphs. Common presentation approaches include well-cited tables of data sources, transparent model diagrams and descriptions, disaggregated results, and tabular and graphical displays of sensitivity analyses. Despite most guidelines being funded by policymakers, only five guidelines provided advice on presenting economic evaluations to noneconomists. However, 11 guidelines included a glossary of economic terminology for nontechnical readers. Common concepts that may require further explanation include differences in economic perspectives, appropriateness of time horizons, how economic outcomes such as quality-adjusted life-years relate to their component clinical outcomes, and choice of sensitivity analyses.

CONCLUSIONS

Health economists have consistent presentation formats and common reporting elements that should be considered when developing user-friendly explanations for general audiences. These overlap with policymakers' informational needs but may not be sufficient for understanding by nontechnical audiences. Developing presentation formats and tools that incorporate viewpoints of both economists and noneconomists will allow for better application of the results of economic evaluations and enhance the transparency and legitimacy of decision-making processes that are informed by economic evaluations.

Protocol--the RAMESES II study: developing guidance and reporting standards for realist evaluation

INTRODUCTION

Realist evaluation is an increasingly popular methodology in health services research. For realist evaluations (RE) this project aims to: develop quality and reporting standards and training materials; build capacity for undertaking and critically evaluating them; produce resources and training materials for lay participants, and those seeking to involve them.

METHODS

To achieve our aims, we will: (1) Establish management and governance infrastructure; (2) Recruit an interdisciplinary Delphi panel of 35 participants with diverse relevant experience of RE; (3) Summarise current literature and expert opinion on best practice in RE; (4) Run an online Delphi panel to generate and refine items for quality and reporting standards; (5) Capture 'real world' experiences and challenges of RE-for example, by providing ongoing support to realist evaluations, hosting the RAMESES JISCmail list on realist research, and feeding problems and insights from these into the deliberations of the Delphi panel; (6) Produce quality and reporting standards; (7) Collate examples of the learning and training needs of researchers, students, reviewers and lay members in relation to RE; (8) Develop, deliver and evaluate training materials for RE and deliver training workshops; and (9) Develop and evaluate information and resources for patients and other lay participants in RE (eg, draft template information sheets and model consent forms) and; (10) Disseminate training materials and other resources.Planned outputs: (1) Quality and reporting standards and training materials for RE. (2) Methodological support for RE. (3) Increase in capacity to support and evaluate RE. (4) Accessible, plain-English resources for patients and the public participating in RE.

DISCUSSION

The realist evaluation is a relatively new approach to evaluation and its overall place in the is not yet fully established. As with all primary research approaches, guidance on quality assurance and uniform reporting is an important step towards improving quality and consistency.

The SQUIRE Guidelines: an evaluation from the field, 5 years post release

Background

The Standards for Quality Improvement Reporting Excellence (SQUIRE) Guidelines were published in 2008 to increase the completeness, precision and accuracy of published reports of systematic efforts to improve the quality, value and safety of healthcare. Since that time, the field has expanded. We asked people from the field to evaluate the Guidelines, a novel approach to a first step in revision.

Methods

Evaluative design using focus groups and semi-structured interviews with 29 end users and an advisory group of 18 thinkers in the field. Sampling of end users was purposive to achieve variation in work setting, geographic location, area of expertise, manuscript writing experience, healthcare improvement and research experience.

Results

Study participants reported that SQUIRE was useful in planning a healthcare improvement project, but not as helpful during writing because of redundancies, uncertainty about what was important to include and lack of clarity in items. The concept "planning the study of the intervention" (item 10) was hard for many participants to understand. Participants varied in their interpretation of the meaning of item 10b "the concept of the mechanism by which changes were expected to occur". Participants disagreed about whether iterations of an intervention should be reported. Level of experience in writing, knowledge of the science of improvement and the evolving meaning of some terms in the field are hypothesised as the reasons for these findings.

Conclusions

The original SQUIRE Guidelines help with planning healthcare improvement work, but are perceived as complicated and unclear during writing. Key goals of the revision will be to clarify items where conflict was identified and outline the key components necessary for complete reporting of improvement work.

Unmet Need: Improving mHealth Evaluation Rigor to Build the Evidence Base

mHealth-the use of mobile technologies for health-is a growing element of health system activity globally, but evaluation of those activities remains quite scant, and remains an important knowledge gap for advancing mHealth activities. In 2010, the World Health Organization and Columbia University implemented a small-scale survey to generate preliminary data on evaluation activities used by mHealth initiatives. The authors describe self-reported data from 69 projects in 29 countries. The majority (74%) reported some sort of evaluation activity, primarily nonexperimental in design (62%). The authors developed a 6-point scale of evaluation rigor comprising information on use of comparison groups, sample size calculation, data collection timing, and randomization. The mean score was low (2.4); half (47%) were conducting evaluations with a minimum threshold (4+) of rigor, indicating use of a comparison group, while less than 20% had randomized the mHealth intervention. The authors were unable to assess whether the rigor score was appropriate for the type of mHealth activity being evaluated. What was clear was that although most data came from mHealth projects pilots aimed for scale-up, few had designed evaluations that would support crucial decisions on whether to scale up and how. Whether the mHealth activity is a strategy to improve health or a tool for achieving intermediate outcomes that should lead to better health, mHealth evaluations must be improved to generate robust evidence for cost-effectiveness assessment and to allow for accurate identification of the contribution of mHealth initiatives to health systems strengthening and the impact on actual health outcomes.

Organization and evaluation of generalist palliative care in a Danish hospital

BACKGROUND

Hospitals have a responsibility to ensure that palliative care is provided to all patients with life-threatening illnesses. Generalist palliative care should therefore be acknowledged and organized as a part of the clinical tasks. However, little is known about the organization and evaluation of generalist palliative care in hospitals. Therefore the aim of the study was to investigate the organization and evaluation of generalist palliative care in a large regional hospital by comparing results from existing evaluations.

METHODS

Results from three different data sets, all aiming to evaluate generalist palliative care, were compared retrospectively. The data-sets derived from; 1. a national accreditation of the hospital, 2. a national survey and 3. an internal self-evaluation performed in the hospital. The data were triangulated to investigate the organization and evaluation of palliative care in order to identify concordances and/or discrepancies.

RESULTS

The triangulation indicated poor validity of the results from existing methods used to evaluate palliative care in hospitals. When the datasets were compared, several discrepancies occurred with regard to the organization and the performance of generalist palliative care. Five types of discrepancies were found in 35 out of 56 sections in the fulfilment of the national accreditation standard for palliative care. Responses from the hospital management and the department managements indicated that generalist palliative care was organized locally--if at all--within the various departments and with no overall structure or policy.

CONCLUSIONS

This study demonstrates weaknesses in the existing evaluation methods for generalist palliative care and highlights the lack of an overall policy, organization and goals for the provision of palliative care in the hospital. More research is needed to focus on the organization of palliative care and to establish indicators for high quality palliative care provided by the hospital. The lack of valid indicators, both for the hospital's and the departments' provision of palliative care, calls for more qualitative insight in the clinical staff's daily work including their culture and acceptance of the provision of palliative care.

An educational review of the statistical issues in analysing utility data for cost-utility analysis

The aim of cost-utility analysis is to support decision making in healthcare by providing a standardised mechanism for comparing resource use and health outcomes across programmes of work. The focus of this paper is the denominator of the cost-utility analysis, specifically the methodology and statistical challenges associated with calculating QALYs from patient-level data collected as part of a trial. We provide a brief description of the most common questionnaire used to calculate patient level utility scores, the EQ-5D, followed by a discussion of other ways to calculate patient level utility scores alongside a trial including other generic measures of health-related quality of life and condition- and population-specific questionnaires. Detail is provided on how to calculate the mean QALYs per patient, including discounting, adjusting for baseline differences in utility scores and a discussion of the implications of different methods for handling missing data. The methods are demonstrated using data from a trial. As the methods chosen can systematically change the results of the analysis, it is important that standardised methods such as patient-level analysis are adhered to as best as possible. Regardless, researchers need to ensure that they are sufficiently transparent about the methods they use so as to provide the best possible information to aid in healthcare decision making.