Integrating impact evaluation in the design and implementation of monitoring marine protected areas

A diverse portfolio of marine protected areas can better advance global conservation and equity

Marine protected areas (MPAs) are widely used for ocean conservation, yet the relative impacts of various types of MPAs are poorly understood. We estimated impacts on fish biomass from no-take and multiple-use (fished) MPAs, employing a rigorous matched counterfactual design with a global dataset of >14,000 surveys in and around 216 MPAs. Both no-take and multiple-use MPAs generated positive conservation outcomes relative to no protection (58.2% and 12.6% fish biomass increases, respectively), with smaller estimated differences between the two MPA types when controlling for additional confounding factors (8.3% increase). Relative performance depended on context and management: no-take MPAs performed better in areas of high human pressure but similar to multiple-use in remote locations. Multiple-use MPA performance was low in high-pressure areas but improved significantly with better management, producing similar outcomes to no-take MPAs when adequately staffed and appropriate use regulations were applied. For priority conservation areas where no-take restrictions are not possible or ethical, our findings show that a portfolio of well-designed and well-managed multiple-use MPAs represents a viable and potentially equitable pathway to advance local and global conservation.

Effects of management objectives and rules on marine conservation outcomes

Understanding the relative effectiveness and enabling conditions of different area-based management tools is essential for supporting efforts that achieve positive biodiversity outcomes as area-based conservation coverage increases to meet newly set international targets. We used data from a coastal social-ecological monitoring program in 6 Indo-Pacific countries to analyze whether social, ecological, and economic objectives and specific management rules (temporal closures, fishing gear-specific, species-specific restrictions) were associated with coral reef fish biomass above sustainable yield levels across different types of area-based management tools (i.e., comparing those designated as marine protected areas [MPAs] with other types of area-based management). All categories of objectives, multiple combinations of rules, and all types of area-based management had some sites that were able to sustain high levels of reef fish biomass-a key measure for coral reef functioning-compared with reference sites with no area-based management. Yet, the same management types also had sites with low biomass. As governments advance their commitments to the Kunming-Montreal Global Biodiversity Framework and the target to conserve 30% of the planet's land and oceans by 2030, we found that although different types of management can be effective, most of the managed areas in our study regions did not meet criteria for effectiveness. These findings underscore the importance of strong management and governance of managed areas and the need to measure the ecological impact of area-based management rather than counting areas because of their designation.

Participation, not penalties: Community involvement and equitable governance contribute to more effective multiuse protected areas

Accelerating ecosystem degradation has spurred proposals to vastly expand the extent of protected areas (PAs), potentially affecting the livelihoods and well-being of indigenous peoples and local communities (IPLCs) worldwide. The benefits of multiuse PAs that elevate the role of IPLCs in management have long been recognized. However, quantitative examinations of how resource governance and the distribution of management rights affect conservation outcomes are vital for long-term sustainability. Here, we use a long-term, quasi-experimental monitoring dataset from four Indonesian marine PAs that demonstrates that multiuse PAs can increase fish biomass, but incorporating multiple governance principles into management regimes and enforcing rules equitably are critical to achieve ecological benefits. Furthermore, we show that PAs predicated primarily on enforcing penalties can be less effective than those where IPLCs have the capacity to engage in management. Our results suggest that well-governed multiuse PAs can achieve conservation objectives without undermining the rights of IPLCs.

Inferring Economic Impacts from a Program's Physical Outcomes: An Application to Forest Protection in Thailand

Economists typically estimate the average treatment effect on the treated (ATT) when evaluating government programs. The economic interpretation of the ATT can be ambiguous when program outcomes are measured in purely physical terms, as they often are in evaluations of environmental programs (e.g., avoided deforestation). This paper presents an approach for inferring economic impacts from physical outcomes when the ATT is estimated using propensity-score matching. For the case of forest protection, we show that a protection program's ex post economic impact, as perceived by the government agency responsible for protection decisions, can be proxied by a weighted ATT, with the weights derived from the propensity of being treated (i.e., protected). We apply this new metric to mangrove protection in Thailand during 1987-2000. We find that the government's protection program avoided the loss of 12.8% of the economic value associated with the protected mangrove area. This estimate is about a quarter smaller than the conventional ATT for avoided deforestation, 17.3 percentage points. The difference between the two measures indicates that the program tended to be less effective at reducing deforestation in locations where the government perceived the net benefits of protection as being greater, which is the opposite of the relationship that would characterize a maximally effective program.

The MPA Guide: A framework to achieve global goals for the ocean

[Figure: see text].

The mismeasure of conservation

One of the basic purposes of protected areas and other effective area-based conservation interventions is to achieve conservation impact, the sum of avoided biodiversity loss and promoted recovery relative to outcomes without protection. In the context of the Convention on Biological Diversity's negotiations on the post-2020 Global Biodiversity Framework, we find that targets for area-based interventions are framed overwhelmingly with measures that fail to inform decision-makers about impact and that risk diverting limited resources away from achieving it. We show that predicting impact in space and time is feasible and can provide the basis for global guidance for jurisdictions to develop targets for conservation impact and shift investment priorities to areas where impact can be most effectively achieved.

Evaluating socio-economic and conservation impacts of management: A case study of time-area closures on Georges Bank

Globally, economies and marine ecosystems are increasingly dependent on sustainable fisheries management (SFM) to balance social, economic, and conservation needs. The overarching objectives of SFM are to maximize both conservation and socio-economic benefits, while minimizing short-term socio-economic costs. A number of tools have been developed to achieve SFM objectives, ranging from fishery specific to ecosystem-based strategies. Closures are a common SFM tool used to balance the trade-off between socio-economic and conservation considerations; they vary in scope from small-scale temporary closures to large-scale permanent networks. Unfortunately, closures are frequently implemented without a plan for monitoring or assessing whether SFM objectives are met. In situations in which a monitoring plan is not in place we propose that commonly available fishery data can often be used to evaluate whether management tools are effective in meeting SFM objectives. Here, we present a case study of closures on Georges Bank that shows how fishery data can be analyzed to perform such an assessment. Since 2006, on the Canadian side of Georges Bank, seasonal scallop fishery closures have been implemented with the aim of reducing by-catch of Atlantic cod (Gadus morhua) and yellowtail flounder (Pleuronectes ferruginea) during spawning. In lieu of data from a dedicated monitoring program, we analyzed data from Vessel Monitoring Systems (VMS), fishery logbooks, and a scallop survey to assess the impact of these closures on the scallop fishery, and use observer data (i.e. by-catch) to assess the effectiveness of these closures in meeting their conservation objective. While compliance for these time-area closures was high, the closures did not significantly displace fishing activity and overall there was limited evidence of an impact on the scallop fishery. Further, the discard rates for both cod and yellowtail were above average when their respective closures were active. These results suggest that improvements to the closures design and/or other measures may be required to achieve the desired SFM objectives.

Catalyzing sustainable fisheries management through behavior change interventions

Small-scale fisheries are an important livelihood and primary protein source for coastal communities in many of the poorest regions in the world, yet many are overfished and thus require effective and scalable management solutions. Positive ecological and socioeconomic responses to management typically lag behind immediate costs borne by fishers from fishing pressure reductions necessary for fisheries recovery. These short-term costs challenge the long-term success of these interventions. However, social marketing may increase perceptions of management benefits before ecological and socioeconomic benefits are fully realized, driving new social norms and ultimately long-term sustainable behavior change. By conducting underwater visual surveys to quantify ecological conditions and by conducting household surveys with community members to quantify their perceptions of management support and socioeconomic conditions, we assessed the impact of a standardized small-scale fisheries management intervention that was implemented across 41 sites in Brazil, Indonesia, and the Philippines. The intervention combines TURF reserves (community-based territorial use rights for fishing coupled with no-take marine reserves) with locally tailored social-marketing behavior change campaigns. Leveraging data across 22 indicators and 4 survey types, along with data from 3 control sites, we found that ecological and socioeconomic impacts varied and that communities supported the intervention and were already changing their fishing practices. These results suggest that communities were developing new social norms and fishing more sustainably before long-term ecological and socioeconomic benefits of fisheries management materialized.

Adaptive social impact management for conservation and environmental management

Concerns about the social consequences of conservation have spurred increased attention the monitoring and evaluation of the social impacts of conservation projects. This has resulted in a growing body of research that demonstrates how conservation can produce both positive and negative social, economic, cultural, health, and governance consequences for local communities. Yet, the results of social monitoring efforts are seldom applied to adaptively manage conservation projects. Greater attention is needed to incorporating the results of social impact assessments in long-term conservation management to minimize negative social consequences and maximize social benefits. We bring together insights from social impact assessment, adaptive management, social learning, knowledge coproduction, cross-scale governance, and environmental planning to propose a definition and framework for adaptive social impact management (ASIM). We define ASIM as the cyclical process of monitoring and adaptively managing social impacts over the life-span of an initiative through the 4 stages of profiling, learning, planning, and implementing. We outline 14 steps associated with the 4 stages of the ASIM cycle and provide guidance and potential methods for social-indicator development, predictive assessments of social impacts, monitoring and evaluation, communication of results, and identification and prioritization of management responses. Successful ASIM will be aided by engaging with best practices - including local engagement and collaboration in the process, transparent communication of results to stakeholders, collective deliberation on and choice of interventions, documentation of shared learning at the site level, and the scaling up of insights to inform higher-level conservation policies-to increase accountability, trust, and perceived legitimacy among stakeholders. The ASIM process is broadly applicable to conservation, environmental management, and development initiatives at various scales and in different contexts.

A novel framework for analyzing conservation impacts: evaluation, theory, and marine protected areas

Environmental conservation initiatives, including marine protected areas (MPAs), have proliferated in recent decades. Designed to conserve marine biodiversity, many MPAs also seek to foster sustainable development. As is the case for many other environmental policies and programs, the impacts of MPAs are poorly understood. Social-ecological systems, impact evaluation, and common-pool resource governance are three complementary scientific frameworks for documenting and explaining the ecological and social impacts of conservation interventions. We review key components of these three frameworks and their implications for the study of conservation policy, program, and project outcomes. Using MPAs as an illustrative example, we then draw upon these three frameworks to describe an integrated approach for rigorous empirical documentation and causal explanation of conservation impacts. This integrated three-framework approach for impact evaluation of governance in social-ecological systems (3FIGS) accounts for alternative explanations, builds upon and advances social theory, and provides novel policy insights in ways that no single approach affords. Despite the inherent complexity of social-ecological systems and the difficulty of causal inference, the 3FIGS approach can dramatically advance our understanding of, and the evidentiary basis for, effective MPAs and other conservation initiatives.

Drivers of coral reef marine protected area performance

Coral reefs are severely threatened and a principal strategy for their conservation is marine protected areas (MPAs). However the drivers of MPA performance are complex and there are likely to be trade-offs between different types of performance (e.g. conservation or welfare related outcomes). We compiled a global dataset from expert knowledge for 76 coral reef MPAs in 33 countries and identified a set of performance measures reflecting ecological and socio-economic outcomes, achievement of aims and reduction of threats, using spatial or temporal comparisons wherever possible. We wanted to test the extent to which distinct types of performance occurred simultaneously, understood as win-win outcomes. Although certain performance measures were correlated, most were not, suggesting trade-offs that limit the usefulness of composite performance scores. Hypotheses were generated as to the impact of MPA features, aims, location, management and contextual variables on MPA performance from the literature. A multivariate analysis was used to test hypotheses as to the relative importance of these "drivers" on eight uncorrelated performance measures. The analysis supported some hypotheses (e.g. benefit provision for the local community improved performance), but not others (e.g. higher overall budget and more research activity did not). Factors endogenous to the MPA (such as size of the no-take area) were generally more significant drivers of performance than exogenous ones (such as national GDP). Different types of performance were associated with different drivers, exposing the trade-offs inherent in management decisions. The study suggests that managers are able to influence MPA performance in spite of external threats and could inform adaptive management by providing an approach to test for the effects of MPA features and management actions in different contexts and so to inform decisions for allocation of effort or funds to achieve specific goals.

Making parks make a difference: poor alignment of policy, planning and management with protected-area impact, and ways forward

Policy and practice around protected areas are poorly aligned with the basic purpose of protection, which is to make a difference. The difference made by protected areas is their impact, defined in program evaluation as the outcomes arising from protection relative to the counterfactual of no protection or a different form of protection. Although impact evaluation of programs is well established in fields such as medicine, education and development aid, it is rare in nature conservation. We show that the present weak alignment with impact of policy targets and operational objectives for protected areas involves a great risk: targets and objectives can be achieved while making little difference to the conservation of biodiversity. We also review potential ways of increasing the difference made by protected areas, finding a poor evidence base for the use of planning and management ‘levers’ to better achieve impact. We propose a dual strategy for making protected areas more effective in their basic role of saving nature, outlining ways of developing targets and objectives focused on impact while also improving the evidence for effective planning and management.

Measuring the difference made by conservation initiatives: protected areas and their environmental and social impacts

Success in conservation depends on our ability to reduce human pressures in areas that harbour biological diversity and ecosystem services. Legally protecting some of these areas through the creation of protected areas is a key component of conservation efforts globally. To develop effective protected area networks, practitioners need credible, scientific evidence about the degree to which protected areas affect environmental and social outcomes, and how these effects vary with context. Such evidence has been lacking, but the situation is changing as conservation scientists adopt more sophisticated research designs for evaluating protected areas' past impacts and for predicting their future impacts. Complementing these scientific advances, conservation funders and practitioners are paying increasing attention to evaluating their investments with more scientifically rigorous evaluation designs. This theme issue highlights recent advances in the science of protected area evaluations and explores the challenges to developing a more credible evidence base that can help societies achieve their goals of protecting nature while enhancing human welfare.

Impact evaluation to communicate and improve conservation non-governmental organization performance: the case of Conservation International

The rising prominence of more rigorous approaches to measuring conservation outcomes has included greater adoption of impact evaluation by conservation non-governmental organizations (CNGOs). Within the scientific literature, however, little consideration has been given to the unique and specific roles of CNGOs in advancing impact evaluation. We explore these issues in the context of one CNGO-Conservation International (CI)-and its experiences producing, using and funding impact evaluations over the past decade. We examine the contributions of impact evaluation to CI's mission at three different stages of CI's strategy: innovation, demonstration and amplification. Furthermore, we review incentives and barriers encountered by CI in its 10+ years' experience in impact evaluation. More coordinated and strategic use of impact evaluation by CNGOs would facilitate learning and promote accountability across the conservation community.

Evaluating Payments for Environmental Services: Methodological Challenges

Over the last fifteen years, Payments for Environmental Services (PES) schemes have become very popular environmental policy instruments, but the academic literature has begun to question their additionality. The literature attempts to estimate the causal effect of these programs by applying impact evaluation (IE) techniques. However, PES programs are complex instruments and IE methods cannot be directly applied without adjustments. Based on a systematic review of the literature, this article proposes a framework for the methodological process of designing an IE for PES schemes. It revises and discusses the methodological choices at each step of the process and proposes guidelines for practitioners.