The Funding, Administrative, and Policy Influences on the Evaluation of Primary Prevention Programs in Australia

Developing evaluation capacities in integrated care projects: Lessons from a scientific support mission implemented in Belgium

The capacity of self-assessment, to learn from experience, to make information-based decisions, and to adapt over time are essential drivers of success for any project aiming at healthcare system change. Yet, many of those projects are managed by healthcare providers' teams with little evaluation capacity. In this article, we describe the support mission delivered by an interdisciplinary scientific team to 12 integrated care pilot projects in Belgium, mobilizing a set of tools and methods: a dashboard gathering population health indicators, a significant event reporting method, an annual report, and the development of a sustainable "learning community." The article provides a reflexive return on the design and implementation of such interventions aimed at building organizational evaluation capacity. Some lessons were drawn from our experience, in comparison with the broader evaluation literature: The provided support should be adapted to the various needs and contexts of the beneficiary organizations, and it has to foster experience-based learning and requires all stakeholders to adopt a learning posture. A long-time, secure perspective should be provided for organizations, and the availability of data and other resources is an essential precondition for successful work.

An assessment of program evaluation methods and quality in Australian prevention agencies

ISSUE ADDRESSED

This study aimed to examine evaluation methods and quality in Australian health promotion agencies and the factors associated with this. The evidence base for prevention strategies is limited, with the evidence generated through program evaluation by health promotion and disease prevention agencies lacking rigour. Despite the need to improve the quality of evaluation, there is limited evidence of what influences evaluation quality in the prevention field.

METHODS

Data were collected using the Evaluation Practice Analysis Survey and an audit and appraisal of evaluation reports. Descriptive analysis was used to examine evaluation characteristics and multivariable regression was used to explore the association between evaluation and organisational attributes and evaluation quality.

RESULTS

In total, 392 evaluation reports were reviewed from 78 government and non-government agencies. Process evaluation was conducted most frequently, followed by impact evaluation. Overall evaluation quality was low (median 24.5%). In multivariable regression analysis, only two factors were associated with evaluation quality: health promotion budget (ratio of geometric means 1.53 [95% CI 1.02-2.29]); and, conducting statewide or national prevention programs (1.38 [95% CI 1.05-1.82]).

CONCLUSIONS

The findings show that the potential to improve evaluation quality is greatest in smaller organisations that deliver health promotion at a local or regional scale. SO WHAT?: By improving the rigour of existing evaluation, there is opportunity to build the evidence base for prevention strategies, which highlights the importance of embedding the enablers of program learning and evidence generation within health promotion and prevention organisations.