Surgical treatment of interhemispheric arachnoid cysts

OBJECTIVE

In children, interhemispheric arachnoid cysts (IHACs) are rare lesions often associated with corpus callosum dysgenesis. It is still controversial about surgical treatments for IHACs. We aim to report our experience with pediatric IHAC patients and evaluate surgical courses and neurological developments.

METHODS

Pediatric IHACs treated between 2001 and 2021 were reviewed retrospectively. IHAC was observed until they represented rapid cyst enlargement or neurological symptoms. Cyst fenestration was done by microscope or endoscope, depending on the IHAC's location. Cyst size and corpus callosum dysgenesis were evaluated with neuroimaging. Neurological development was assessed from medical records at the last follow-up.

RESULTS

Fifteen children received cyst fenestration surgery (mean age 11.4 months). Eleven patients (73.3%) under observation showed rapid cyst enlargement in a short period (median 5 months). Cysto-ventriculostomy (CVS) and cysto-cisternostomy (CCS) regressed the cyst size significantly (p = 0.003). The median follow-up duration was 51 months (range 14-178 months). Corpus callosum dysgenesis was observed in eleven patients (73.3%, complete = 5, partial = 6). Among eight patients (53.3%) having developmental delay, five patients (33.3%) showed speech delay, including one patient with intractable seizures.

CONCLUSION

Pediatric IHACs frequently present within 1 year after birth, with rapid cyst enlargement. CVS and CCS were effective in regressing the cyst size. Corpus callosum dysgenesis accompanied by IHAC might have a risk of language achievement; however, development delay could rely on multifactorial features, such as epilepsy or other brain anomalies.

Creating a developmental framework for evaluating RRI implementation in research organisations

The evaluation of Responsible Research and Innovation (RRI) actions presents important challenges for the key stakeholders engaged in the process of RRI implementation, such as policy makers, programme managers, and researchers. While there is a considerable body of literature on the conceptualization of responsibility in research and a plethora of attempts to practice RRI, there is a need for increased attention to the monitoring and evaluation of case studies of RRI implementations in research organisations, in particular regarding their structural change effects. This paper aims to discuss a contextualised developmental framework for evaluating RRI implementation in research organisations, with a specific focus on achieving structural change through tailor-made action plans. The framework, developed through RRI evaluation work in the field of biosciences, adopts a systemic and process-oriented perspective, encompassing participatory, anticipatory, reflexive, and responsive dimensions. Concrete empirical examples from bioscience organizations are provided to illustrate how the framework relates to specific conditions, experiences, and solutions, demonstrating how conceptual insights have emerged from real-life practices and data analysis. While the framework was initially customized for the specific contexts of six bioscience research organizations, it holds potential for broader relevance and applicability in addressing challenges related to RRI design, implementation, and evaluation.

An evaluation of the Compassion, Inclusion, and Engagement initiative: learning from PWLE and communities across British Columbia

BACKGROUND

The Compassion, Inclusion and Engagement initiative (CIE) was a social contact intervention that operated in British Columbia between 2015 and 2021. The primary objective of CIE was to increase the participation of people with lived experience of substance use (PWLE) in the planning, design, implementation, and evaluation of harm reduction supports and services.

CASE PRESENTATION

CIE used the developmental evaluation methodology outcome mapping to define and measure progress towards its goals. Developmental evaluation emphasizes learning in contrast to other forms of evaluation which are often more focused on determining the value or success of a project or programme based on predetermined criteria. Outcome mapping is a relational practice which acknowledges that change is achieved by an initiative's partners and the role of the initiative is to provide access to resources, ideas and opportunities that can facilitate and support change.

CONCLUSIONS

Through the implementation and evaluation of CIE, it became clear that directly supporting PWLE facilitated more meaningful and lasting change than solely working to improve the health and social services that supported them. The impacts of the CIE initiative extend far beyond the outcomes of any of the dialogues it facilitated and are largely the result of an increase in social capital. CIE engagements created the opportunity for change by inviting people most affected by the toxic drug supply together with those committed to supporting them, but their ability to bring about systemic change was limited. Both PWLE and service providers noted the lack of support to attend CIE engagements, lack of support for actions that came from those engagements, and lack of PWLE inclusion in decision-making by health authorities as limiting factors for systemic change. The lack of response at a systemic level often resulted in PWLE carrying the burden of responding to toxic drug poisonings, often without resources, support, or compensation.

Formative evaluation of Zero Suicide in the emergency department: Identifying strategies to overcome implementation barriers

BACKGROUND

Zero Suicide has been widely promoted as a comprehensive suicide prevention approach in healthcare systems, yet less is known about the barriers to implementing this approach in the emergency department.

OBJECTIVES

This developmental evaluation aimed to assess emergency department providers' perceived knowledge and self-efficacy regarding suicide prevention practices, as well as apply the Consolidated Framework for Implementation Research to explore potential facilitators and barriers to implementing Zero Suicide and identify strategies to overcome barriers.

METHODS

A sequential mixed methods approach was used, including a survey assessing emergency department providers' perceived knowledge and attitudes and semi-structured interviews exploring potential determinants of implementation.

RESULTS

Survey respondents (n = 43) perceived that they have the knowledge and self-efficacy to implement clinical elements of Zero Suicide; however, interview participants (n = 18) revealed that some clinical elements are not implemented consistently and perceive implementation barriers across multiple levels, including limited training on suicide risk assessment and limited resources needed to engage and re-engage at-risk patients in their suicide care management plan and provide follow-up supportive contacts during patients' transitions in care.

CONCLUSION

To overcome identified barriers in this setting, targeted implementation strategies are needed, including integration in electronic health record systems, leadership advocacy, and ongoing staff training.

Methodological pluralism for better evaluations of complex interventions: lessons from evaluating an innovation platform in Australia

Complex interventions, such as innovation platforms, pose challenges for evaluators. A variety of methodological approaches are often required to build a more complete and comprehensive understanding of how complex interventions work. In this paper, we outline and critically appraise a methodologically pluralist evaluation of an innovation platform to strengthen primary care for Aboriginal and Torres Strait Islander Australians. In doing so, we aim to identify lessons learned from the approach taken and add to existing literature on implementing evaluations in complex settings, such as innovation platforms. The pluralist design used four evaluation approaches-developmental evaluation, principles-focused evaluation, network analysis, and framework analysis-with differing strengths and challenges. Taken together, the multiple evaluation approaches yielded a detailed description and nuanced understanding of the formation, functioning and outcomes of the innovation platform that would be difficult to achieve with any single evaluation method. While a methodologically pluralist design may place additional pressure on logistical and analytic resources available, it enables a deeper understanding of the mechanisms that underlie complex interventions.

Partnership evaluation: An application of a developmental framework to the Governance and Local Development project in Senegal

This study explores the valued-added of USAID/Senegal's Governance for Local Development (GOLD) project's integrated approach to working with four sector projects: two in health, one in water and sanitation, and one in nutrition. The study builds on a partnership evaluation framework that identifies practices related to preconditions, factors linked with success, implementation structures and processes, contribution of collaboration to performance, and collaboration outcomes. A survey instrument developed statements of positive practices and asked respondents to agree or disagree with each one. The analysis includes exploration of differences in perception. Respondents generally expressed agreement with the statements in the survey instrument associated with the various dimensions of effective partnerships. All projects mentioned the presence of partnership champions. Problems with joint action (e.g., planning, budgeting, managing) were the most frequently cited constraints. The study confirms most of the hypotheses embedded in the evaluation framework regarding partnership's potential for creating synergies and improved outcomes and affirms the framework's usefulness as a developmental evaluation tool. Positive outcomes result from intentional efforts to build relationships, cultivate champions, and design systems and processes that support a partnership culture and contribute to each partners' success as well as the success of the partnership itself.

[Application of dementia care mapping (DCM) for one year in a geriatric health services facility: Effects of developmental evaluation based on collaboration by medical and welfare staff aimed at person-centred care]

PURPOSE

This study aimed to clarify the effects of dementia care mapping (DCM) for one year in a healthcare center for older adults. DCM was conducted between September 2016 and August 2017. The care staff include nurses and caregivers in a narrow sense, medical staff, such as a physician, physical therapists, and occupational therapists worked on DCM as care staff in this study.

RESULTS

There were 24 participants, with an average work experience of 7.21 (±4.74) years. In comparison to the baseline evaluation, the final assessment of self-efficacy through person-centred care showed significant improvement in 'Forecasting and Problem Solving on the Job' within 'Perceived Job Competence of Care Workers'. Six main categories of content were extracted from focus group interviews: 'Awareness,' 'Change of Elderly People under the Care of Staff throughout the Development of Mapping', 'Affirmative Feelings of Care Staff for Mapping', 'Negative Feelings for Mapping', 'Need for the Efficacy and Efficiency of the Mapping', and 'Mapping Based on the Age of the Participant and Future Prospects for Mapping'. The results of person-centred care showed that both the older patients and the staff noticed changes through the development of mapping.

CONCLUSION

The developmental evaluation, based on collaboration by medical and welfare staff can improve self-efficacy through the practice of person-centred care and improves the ability to solve problems during the provision of care.

Using developmental evaluation and community-based participatory research to develop a model of supportive housing

Community-based participatory research (CBPR) and developmental evaluation (DE) have emerged over recent decades as separate approaches for addressing complex social issues. Current literature offers little with respect to the use of CBPR and DE in combination, although the two approaches are complementary. Through the current paper, we outline how CBPR and DE were used to develop a model of supportive housing for teen families. More specifically, we describe the structures and processes that contributed to this development, including (1) our partnership approach, (2) pooled resources, (3) regular opportunities for collaboration and reflection, (4) integration of multiple data sources, (5) ongoing feedback and knowledge dissemination, and (6) adjustments to program practices. We end by providing insights into the lessons that we learned through this project. Through this paper, we describe how researchers and community partners can collaboratively use CBPR and DE to develop a program model in complex community settings. Insights are offered that will be important for researchers, evaluators, and practitioners seeking to develop programming in response to complex community issues.

Inclusion and equity in education: Current policy reform in Nova Scotia, Canada

This article aims to explore the context of inclusive education policy in Canada, and to highlight the particular case of inclusive education policy reform in the province of Nova Scotia. As with most other provinces and territories, inclusive education policy in Nova Scotia has broadened to include a lens of equity, with a focus on not only students with special education needs, but all students – particularly those most often marginalized by and within Canadian school systems. The article reflects on the first phase of the developmental evaluation process which took place prior to full implementation of the policy. Four interconnected key themes emerge: 1) the shifting roles and identities for educators and specialized staff; 2) the changing roles of classroom teachers; 3) the importance of support to ensure effective universal and differentiated classroom practices; and 4) the professional learning of school staff. Although situated within the Nova Scotian and the national Canadian context, the discussion and implications can readily be applied to international systems engaged in developing and implementing broad inclusive education policy.

Using developmental evaluation to enhance continuous reflection, learning and adaptation of an innovation platform in Australian Indigenous primary healthcare

Effective efforts to strengthen health systems need diverse, multi-stakeholder networks working together on complex or 'wicked' problems such as prevention and control of chronic diseases, solutions to which go beyond the role and capability of one organisation. The contextual complexities inherent in 'wicked' problems mean that solutions warrant a systems approach that encompasses innovation and new ways of thinking about, facilitating and implementing collective decision-making processes and change practices.Innovation platforms are a mechanism for facilitating communication and collaboration among diverse stakeholders, promoting joint action and stimulating innovation. Developmental evaluation is an approach that is increasingly being used to evaluate innovative and emergent programmes and projects, as it enables evaluators to provide real-time feedback so that evaluation findings can be used to guide development and adaptations. Developmental evaluation emphasises learning and adaptation, and aligns well with the implementation of innovation platforms that have continuous reflection, learning and adaptation as a specific design principle.Here, we outline our rationale for applying a developmental evaluation to enhance the formation, functioning and outcomes of an innovation platform aimed at accelerating and strengthening large-scale quality improvement efforts in Australian Aboriginal and Torres Strait Islander primary healthcare. We provide examples to explain how the developmental evaluation findings were used for adaptation of the innovation platform and assess to what extent our application of developmental evaluation was consistent with, and reflective of, its essential principles.Our evaluation aligned strongly with the principles of developmental evaluation, and the approach we took was well suited to situations with a developmental purpose, innovation niche and complexity such as innovation platforms. As a result, along with the increasing interest in multi-stakeholder platforms (e.g. innovation platforms) and the inherent challenges with evaluating these complex networks, we anticipate our use of this approach being of interest globally.

From plan to action: Adapting evaluation to serve the developmental needs of a newly-funded multidisciplinary research center

The Southeastern Coastal Center for Agricultural Health and Safety (SCCAHS) is one of many newly-funded federal research centers, housing five multidisciplinary research projects and seven pilot projects, and serving a multi-state region. In the early stages of such a complex project, with multiple teams separated by geography and disciplines, the evaluation program has been integral in connecting internal and external stakeholders at the center and project levels. We used a developmental evaluation (DE) framework to respond to the complex political environment surrounding agricultural health and safety in the southeast; to engage external stakeholders in guiding the center's research and outreach trajectories; to support center research teams in a co-creation process to develop logic models and tailored indicators; and to provide timely and feedback within the center to address communications gaps identified by the evaluation program. By using DE principles to shape monitoring and evaluation approaches, our evaluation program has adapted to the dynamic circumstances presented as our center's progress has been translated from a plan in a grant proposal to implementation.

Using developmental evaluation to support knowledge translation: reflections from a large-scale quality improvement project in Indigenous primary healthcare

BACKGROUND

Developmental evaluation is a growing area of evaluation practice, advocated for informing the adaptive development of change initiatives in complex social environments. The utilisation focus, complexity perspective and systems thinking of developmental evaluation suggest suitability for evaluating knowledge translation initiatives in primary healthcare. However, there are few examples in the literature to guide its use in these contexts and in Indigenous settings. In this paper, we reflect on our experience of using developmental evaluation to implement a large-scale knowledge translation research project in Australian Aboriginal and Torres Strait Islander primary healthcare. Drawing on principles of knowledge translation and key features of developmental evaluation, we debate the key benefits and challenges of applying this approach to engage diverse stakeholders in using aggregated quality improvement data to identify and address persistent gaps in care delivery.

DISCUSSION

The developmental evaluation enabled the team to respond to stakeholder feedback and apply learning in real-time to successfully refine theory-informed research and engagement processes, tailor the presentation of findings to stakeholders and context, and support the project's dissemination and knowledge co-production aim. It thereby contributed to the production of robust, useable research findings for informing policy and system change. The use of developmental evaluation appeared to positively influence stakeholders' use of the project reports and their responses to the findings. Challenges included managing a high volume of evaluation data and multiple evaluation purposes, balancing facilitative sense-making processes and change with task-focused project management, and lack of experience in using this evaluation approach. Use of an embedded evaluator with facilitation skills and background knowledge of the project helped to overcome these challenges, as did similarities observed between features of developmental evaluation and continuous quality improvement.

CONCLUSION

Our experience of developmental evaluation confirmed our expectations of the potential value of this approach for strengthening improvement interventions and implementation research, and particularly for adapting healthcare innovations in Indigenous settings. In our project, developmental evaluation successfully encompassed evaluation, project adaptation, capacity development and knowledge translation. Further work is warranted to apply this approach more widely to improve primary healthcare initiatives and outcomes, and to evaluate implementation research.

Using developmental evaluation as a system of organizational learning: An example from San Francisco

In the last 20 years, developmental evaluation has emerged as a promising approach to support organizational learning in emergent social programs. Through a continuous system of inquiry, reflection, and application of knowledge, developmental evaluation serves as a system of tools, methods, and guiding principles intended to support constructive organizational learning. However, missing from the developmental evaluation literature is a nuanced framework to guide evaluators in how to elevate the organizational practices and concepts most relevant for emergent programs. In this article, we describe and reflect on work we did to develop, pilot, and refine an integrated pilot framework. Drawing on established developmental evaluation inquiry frameworks and incorporating lessons learned from applying the pilot framework, we put forward the Evaluation-led Learning framework to help fill that gap and encourage others to implement and refine it. We posit that without explicitly incorporating the assessments at the foundation of the Evaluation-led Learning framework, developmental evaluation's ability to affect organizational learning in productive ways will likely be haphazard and limited.

A strategy for monitoring and evaluating massive open online courses

We argue that the complex, innovative and adaptive nature of Massive Open Online Course (MOOC) initiatives poses particular challenges to monitoring and evaluation, in that any evaluation strategy will need to follow a systems approach. This article aims to guide organizations implementing MOOCs through a series of steps to assist them in developing a strategy to monitor, improve, and judge the merit of their initiatives. We describe how we operationalise our strategy by first defining the different layers of interacting agents in a given MOOC system. We then tailor our approach to these different layers. Specifically, a two-pronged approach was developed, where we suggest that individual projects be assessed through performance monitoring; assessment criteria for which would be defined at the outset to include coverage, participation, quality and student achievement. In contrast, the success of an overall initiative should be considered within a more adaptive, emergent evaluation inquiry framework. We present the inquiry framework we developed for MOOC initiatives, and show how this framework might be used to develop evaluation questions and an assessment methodology. We also define the more fixed indicators and measures for project performance monitoring. Our strategy is described as it was developed to inform the evaluation of a MOOC initiative at the University of Cape Town (UCT), South Africa.

Piloting online WellnessRx learning modules: demonstration of developmental evaluation

WellnessRx is a health initiative focusing on healthy living through education, knowledge translation, and community engagement. Stakeholders of WellnessRx identified web-based education learning modules on nutrition and physical education as a priority to be integrated into existing health sciences curricula, as well as adapted for use by health professionals. Five learning modules were created with essential knowledge, skills, attitudes and resources or tools for health professional students and practitioners. As part of the 'developmental evaluation framework' for WellnessRx, two of these modules were piloted within two health professional student programs. This paper describes the pilot-evaluation experience involving student surveys, focus groups and interviews, and faculty perspectives. For both modules, student pre-post knowledge assessments indicated some improvements in post-module knowledge. Post module evaluations by students indicated benefits with the online delivery being flexible for access, self-health, case-based assessments and useful nutrition and physical activity guides. Challenges for students included their time to do the modules and the activity expectations. Instructors felt each module could be better targeted to different years within an undergraduate program. Through developmental evaluation, the pilot results along with recommendations and lessons learned provided the direction needed to further develop the WellnessRx logic model and proposed learning modules.

Developmental evaluation and the 'Stronger Economies Together' initiative in the United States

This article describes a developmental evaluation and explains its impact on the Stronger Economies Together (SET) initiative of the U.S. Department of Agriculture in collaboration with the Nation's four Regional Rural Development Centers and Land-Grant universities. Through a dynamic process, this evaluation of the early phases of an initiative led to continuous program alterations based on feedback. The relationship of the evaluation team to the initiative's coordinating team enabled seamless transfer of observations, suggestions, and recommendations to decision makers. The multidisciplinary character of the evaluation team provided a diverse set of perspectives with a depth of subject matter and knowledge from relevant fields. One lesson is that developmental evaluators must be flexible, nimble, creative, and adaptive. When expected data are imperfect or nonexistent, the team must collect alternate information and make recommendations to improve data collection. As the initiative proceeded and modifications came about, the evaluation team had to recognize the changes in the program and focus on different questions. This experience with developmental evaluation provides insights into how interdisciplinary teams may need to change course and conduct a developmental evaluation when a formative evaluation was originally envisioned.