The evidence needed to demonstrate impact: A synthesis of the evidence from a phased social and emotional wellbeing intervention

A study to determine if and how bereavement support programs provided by Irish and Canadian hospices are evaluated

A study of the bereavement programs provided by hospices in Ireland and the Canadian province of Alberta was done to determine if and how these are evaluated for efficacy. All but one hospice offered multiple different bereavement programs, with routine in-house evaluations of all programs performed. In all cases, staff and client unsolicited and solicited comments made during and at the end of each program provided data for continuing program refinement. Most hospices also routinely employed a self-devised questionnaire to gain specific sought information to retain or change programs. Other information, including news about program developments elsewhere, was also actively sought for program adoption or for the improvement of existing programs. With most bereavement programs having been in place for many years, participants were confident their programs are needed, safe, and effective. Their continuing quest for high-quality programming, however, meant they were active in monitoring these programs and in seeking developments in this field.

A police-led community response to Child abuse and Youth Sexual Violence and Abuse in Indigenous communities in Far North Queensland: "Speak Up. Be strong. Be Heard."

BACKGROUND

Child abuse and Youth Sexual Violence and Abuse (YSVA) are persistent social issues across the globe. The development and implementation of effective prevention strategies are a common focus for those working at the coalface. The Cairns Child Protection Investigation Unit of the Queensland Police Service (QPS) developed and implemented the "Speak Up. Be Strong. Be Heard." (SUBSBH) initiative. This police-led multi-component child abuse prevention initiative has been implemented in 26 Aboriginal and Torres Strait Islander communities across the Far North Queensland Police District since June 2016.

OBJECTIVE

The aim of this research was to evaluate the success of the SUBSBH initiative.

PARTICIPANTS AND SETTING

Existing data held by QPS were examined. These data include statistics on reporting of YSVA offences, internal program documents and reports, and evaluation feedback forms completed prior to this evaluation study. Information collected via these sources pertained to 26 Indigenous communities within the Far North Queensland Police District. The above-mentioned feedback forms were completed by 307 participants, of whom approximately 90% are Indigenous.

METHODS

This study adopted desktop analysis and triangulation through a range of qualitative and quantitative data to ensure robust and rigorous evaluation of the SUBSBH initiative.

RESULTS

The study found that the initiative was successful in meeting basic accepted practice for child abuse and YSVA prevention programs, receiving positive participant feedback on the educational program, achieving the initiative's objective to increase reporting of YSVA, and achieving cost-efficiency in meeting outcomes. Importantly, the increase in reporting of YSVA was statistically significant.

CONCLUSION

This study contributes to current understanding regarding the implementation of multi-component child abuse prevention initiatives and provides an example of a cost-efficient police-led community response to child abuse and YSVA in Indigenous communities. The findings may guide responses in other communities which grapple with this critical social issue.

CQI approaches for evaluating management development programs: A case study with health service managers from geographically remote settings

Health systems are known for being complex. Yet, there is a paucity of evidence about programs that successfully develop competent frontline managers to navigate these complex systems. There is even less evidence about developing frontline managers in areas of contextual complexity such as geographically remote and isolated health services. This study used a customised management development program containing continuous quality improvement (CQI) approaches to determine whether additional levels of evaluation could provide evidence for program impact. Generalisability is limited by the small sample size; however, the findings suggest that continuous improvement approaches, such as action learning workplace-based CQI projects not only provide for real-world application of the manager's learning; they can potentially produce the type of data needed to conduct evaluations for organisational impact and cost-benefits. The case study contributes to the literature in an area where there is a scarcity of empirical research. Further, this study proposes a pragmatic method for using CQI approaches with existing management development programs to generate the type of data needed for multi-level evaluation.

Community-based participatory action research on rheumatic heart disease in an Australian Aboriginal homeland: Evaluation of the 'On track watch' project

Strategies to date have been ineffective in reducing high rates of rheumatic heart disease (RHD) in Australian Aboriginal people; a disease caused by streptococcal infections. A remote Aboriginal community initiated a collaboration to work towards elimination of RHD. Based in 'both-way learning' (reciprocal knowledge co-creation), the aim of this study was to co-design, implement and evaluate community-based participatory action research (CBPAR) to achieve this vision. Activities related to understanding and addressing RHD social determinants were delivered through an accredited course adapted to meet learner and project needs. Theory-driven evaluation linking CBPAR to empowerment was applied. Data collection comprised focus groups, interviews, observation, and co-development and use of measurement tools such as surveys. Data analysis utilised process indicators from national guidelines for Aboriginal health research, and outcome indicators derived from the Wallerstein framework. Findings include the importance of valuing traditional knowledges and ways of learning such as locally-meaningful metaphors to explore unfamiliar concepts; empowerment through critical thinking and community ownership of knowledge about RHD and research; providing practical guidance in implementing empowering and decolonising principles / theories. Lessons learned are applicable to next stages of the RHD elimination strategy which must include scale-up of community leadership in research agenda-setting and implementation.

Assessing research impact: Australian Research Council criteria and the case of Family Wellbeing research

Researchers worldwide are increasingly reporting the societal impact of their research as part of national research productivity assessments. However, the challenges they encounter in developing their impact case studies against specified government assessment criteria and how pitfalls can be mitigated are not reported. This paper examines the key steps taken to develop an Aboriginal Family Wellbeing (FWB) empowerment research impact case study in the context of an Australian Research Council (ARC) pilot research impact assessment exercise and the challenges involved in applying the ARC criteria. The requirement that researchers demonstrate how their institutions support them to conduct impactful research has the potential to create supportive environments for researchers to be more responsive to the needs of users outside academia. However, the 15-year reference period for the associated research underpinning the reported impact and the focus on researcher's current institutional affiliation constitute potential constraints to demonstrating the true impact of research. For researchers working with Indigenous people, relationships that build over long periods of time, irrespective of university affiliation, are critical to conducting impactful research. A more open-ended time-frame, with no institutional restrictions for the 'associated research' provides the best opportunity to demonstrate the true benefits of research not only for Indigenous people but for Australian society more broadly.