Assessment of Evaluation Tools for Integrated Surveillance of Antimicrobial Use and Resistance Based on Selected Case Studies

Integrated surveillance systems for antibiotic resistance in a One Health context: a scoping review

BACKGROUND

Antibiotic resistance (ABR) has emerged as a major threat to health. Properly informed decisions to mitigate this threat require surveillance systems that integrate information on resistant bacteria and antibiotic use in humans, animals, and the environment, in line with the One Health concept. Despite a strong call for the implementation of such integrated surveillance systems, we still lack a comprehensive overview of existing organizational models for integrated surveillance of ABR. To address this gap, we conducted a scoping review to characterize existing integrated surveillance systems for ABR.

METHODS

The literature review was conducted using the PRISMA guidelines. The selected integrated surveillance systems were assessed according to 39 variables related to their organization and functioning, the socio-economic and political characteristics of their implementation context, and the levels of integration reached, together with their related outcomes. We conducted two distinct, complementary analyses on the data extracted: a descriptive analysis to summarize the characteristics of the integrated surveillance systems, and a multiple-correspondence analysis (MCA) followed by a hierarchical cluster analysis (HCA) to identify potential typology for surveillance systems.

RESULTS

The literature search identified a total of 1330 records. After the screening phase, 59 references were kept from which 14 integrated surveillance systems were identified. They all operate in high-income countries and vary in terms of integration, both at informational and structural levels. The different systems combine information from a wide range of populations and commodities -in the human, animal and environmental domains, collection points, drug-bacterium pairs, and rely on various diagnostic and surveillance strategies. A variable level of collaboration was found for the governance and/or operation of the surveillance activities. The outcomes of integration are poorly described and evidenced. The 14 surveillance systems can be grouped into four distinct clusters, characterized by integration level in the two dimensions. The level of resources and regulatory framework in place appeared to play a major role in the establishment and organization of integrated surveillance.

CONCLUSIONS

This study suggests that operationalization of integrated surveillance for ABR is still not well established at a global scale, especially in low and middle-income countries and that the surveillance scope is not broad enough to obtain a comprehensive understanding of the complex dynamics of ABR to appropriately inform mitigation measures. Further studies are needed to better characterize the various integration models for surveillance with regard to their implementation context and evaluate the outcome of these models.

Implementation of One Health surveillance systems: Opportunities and challenges - lessons learned from the OH-EpiCap application

As the complexity of health systems has increased over time, there is an urgent need for developing multi-sectoral and multi-disciplinary collaborations within the domain of One Health (OH). Despite the efforts to promote collaboration in health surveillance and overcome professional silos, implementing OH surveillance systems in practice remains challenging for multiple reasons. In this study, we describe the lessons learned from the evaluation of OH surveillance using OH-EpiCap (an online evaluation tool for One Health epidemiological surveillance capacities and capabilities), the challenges identified with the implementation of OH surveillance, and the main barriers that contribute to its sub-optimal functioning, as well as possible solutions to address them. We conducted eleven case studies targeting the multi-sectoral surveillance systems for antimicrobial resistance in Portugal and France, Salmonella in France, Germany, and the Netherlands, Listeria in The Netherlands, Finland and Norway, Campylobacter in Norway and Sweden, and psittacosis in Denmark. These evaluations facilitated the identification of common strengths and weaknesses, focusing on the organization and functioning of existing collaborations and their impacts on the surveillance system. Lack of operational and shared leadership, adherence to FAIR data principles, sharing of techniques, and harmonized indicators led to poor organization and sub-optimal functioning of OH surveillance systems. In the majority of studied systems, the effectiveness, operational costs, behavioral changes, and population health outcomes brought by the OH surveillance over traditional surveillance (i.e. compartmentalized into sectors) have not been evaluated. To this end, the establishment of a formal governance body with representatives from each sector could assist in overcoming long-standing barriers. Moreover, demonstrating the impacts of OH-ness of surveillance may facilitate the implementation of OH surveillance systems.

Moving towards One Health surveillance of antibiotic resistance in France: a semi-quantitative evaluation of the level of collaboration within the national surveillance system

Objectives

Collaboration between surveillance programmes is the keystone of One Health surveillance and international organizations call for integrated surveillance systems to manage antibiotic resistance (ABR). In France, the ABR surveillance system covers human, animal, food and the environment sectors, but appears to be fragmented, questioning its level of integration. This study aimed to evaluate collaboration within this system and to formulate recommendations towards more integration.

Methods

ECoSur, a semi-quantitative tool, was used to evaluate collaboration between surveillance programmes. A total of 31 attributes were evaluated using information from the literature and 52 interviews with surveillance actors from all four sectors. Evaluation results were visualized via three output figures displaying aspects related to governance and functionality of collaboration. Results were validated by an expert committee.

Results

Overall, the French collaborative strategy for ABR surveillance was well formalized and relevant to its objectives. However, a cross-sectoral coordination body was lacking to help with its practical implementation. The environmental sector was largely uncovered, but its integration appeared necessary to meet the strategy objectives. Data sharing and joint data analyses between programmes were insufficient, mainly due to limited resources and data interoperability issues. Collaboration was operational for internal and external communication of the results. Twelve recommendations were suggested to decision makers to foster collaboration within the French surveillance system and feed future strategies against ABR.

Conclusions

This first evaluation of collaboration within the French ABR surveillance system produced concrete recommendations to move towards One Health integrated surveillance. Both the approach and the findings could be of interest to other countries.

Development of a cross-sectoral antimicrobial resistance capability assessment framework

Antimicrobial resistance (AMR) is an urgent and growing global health concern, and a clear understanding of existing capacities to address AMR, particularly in low-income and middle-income countries (LMICs), is needed to inform national priorities, investment targets and development activities. Across LMICs, there are limited data regarding existing mechanisms to address AMR, including national AMR policies, current infection prevention and antimicrobial prescribing practices, antimicrobial use in animals, and microbiological testing capacity for AMR. Despite the development of numerous individual tools designed to inform policy formulation and implementation or surveillance interventions to address AMR, there is an unmet need for easy-to-use instruments that together provide a detailed overview of AMR policy, practice and capacity. This paper describes the development of a framework comprising five assessment tools which provide a detailed assessment of country capacity to address AMR within both the human and animal health sectors. The framework is flexible to meet the needs of implementers, as tools can be used separately to assess the capacity of individual institutions or as a whole to align priority-setting and capacity-building with AMR National Action Plans (NAPs) or national policies. Development of the tools was conducted by a multidisciplinary team across three phases: (1) review of existing tools; (2) adaptation of existing tools; and (3) piloting, refinement and finalisation. The framework may be best used by projects which aim to build capacity and foster cross-sectoral collaborations towards the surveillance of AMR, and by LMICs wishing to conduct their own assessments to better understand capacity and capabilities to inform future investments or the implementation of NAPs for AMR.

Evaluating the OH-EpiCap tool using the Danish integrated surveillance program for AMU and AMR as a case study

Antimicrobial resistance (AMR) is considered a One Health (OH) challenge, ideally demanding concerted efforts from the animal, human and environmental side. DANMAP, the Danish Integrated Antimicrobial Resistance Monitoring and Research Program, is monitoring AMR and antimicrobial use in animals and humans. OH-EpiCap is an evaluation tool, developed to address essential elements in OH surveillance systems, such as the dimensions of the organization, operational activities and the impact of the surveillance activities. We aimed to evaluate DANMAP using OH-EpiCap and hereby assessed the suitability of OH-EpiCap to evaluate integrated AMR surveillance systems. During the evaluation, the strengths and weaknesses of DANMAP concerning the "OH-ness" of the program were discussed. Furthermore, possible adaptations of the standard operating procedures and governance structure were addressed. Attention was paid to the ability and easiness of DANMAP to cope with current and future challenges connected to integrated AMR surveillance. It was concluded that DANMAP has a strong OH approach covering relevant aspects for humans and animals, whereas environmental aspects are missing. OH-EpiCap proved to be straightforward to use and provided valuable insights. The authors recommend OH-EpiCap to be used by health authorities and stakeholders. It is not suitable for the technical evaluation of a surveillance program.

Evaluation of the One Health-Ness of 20 Years of Antimicrobial Resistance Surveillance in Norway

We evaluated the One Health-ness (OH-ness) of the surveillance system for antimicrobial resistance (AMR) in Norway by using the recently developed "Evaluation tool for One Health epidemiological surveillance capacities and capabilities" (OH-EpiCap tool). First, we defined the Norwegian AMR surveillance system that we would evaluate. The tool was applied by a group of stakeholders (key persons in the Norwegian AMR surveillance programmes and authors of this paper). The evaluation was performed using a consensus approach. The evaluation resulted in an overall OH-ness score of 68% across all three dimensions included in the tool: Organisation, Operation, and Impact. Suggestions for improvement were only indicated within the areas of internal evaluation and operational costs, whereas most of the indicators included in the tool showed good adherence to the One Health principles. By performing this internal evaluation, we recognized that AMR surveillance in the environment needs to be included in a more systematic and standardized way to improve the OH-ness as defined by the quadripartite organisations. Last but not least, it was beneficial to bring key stakeholders together to conduct the evaluation. It increased a joint perception of the OH-ness of AMR surveillance in Norway and encouraged further collaboration in the future.

Users' perception of the OH-EpiCap evaluation tool based on its application to nine national antimicrobial resistance surveillance systems

Introduction

Antimicrobial resistance (AMR) is a One Health (OH) challenge. To achieve or maintain an effective and efficient AMR surveillance system, it is crucial to evaluate its performance in meeting the proposed objectives, while complying with resource restrictions. The OH-EpiCap tool was created to evaluate the degree of compliance of hazard surveillance activities with essential OH concepts across the following dimensions: organization, operational activities, and impact of the surveillance system. We present feedback on the application of the OH-EpiCap tool from a user's perspective, based on the use of the tool to evaluate nine national AMR surveillance systems, each with different contexts and objectives.

Methods

The OH-EpiCap was assessed using the updated CoEvalAMR methodology. This methodology allows the evaluation of the content themes and functional aspects of the tool and captures the user's subjective experiences via a strengths, weaknesses, opportunities, and threats (SWOT) approach.

Results and Discussion

The results of the evaluation of the OH-EpiCap are presented and discussed. The OH-EpiCap is an easy-to-use tool, which can facilitate a fast macro-overview of the application of the OH concept to AMR surveillance. When used by specialists in the matter, an evaluation using OH-EpiCap can serve as a basis for the discussion of possible adaptations of AMR surveillance activities or targeting areas that may be further investigated using other evaluation tools.

OH-EpiCap: a semi-quantitative tool for the evaluation of One Health epidemiological surveillance capacities and capabilities

Although international health agencies encourage the development of One Health (OH) surveillance, many systems remain mostly compartmentalized, with limited collaborations among sectors and disciplines. In the framework of the OH European Joint Programme "MATRIX" project, a generic evaluation tool called OH-EpiCap has been developed to enable individual institutes/governments to characterize, assess and monitor their own OH epidemiological surveillance capacities and capabilities. The tool is organized around three dimensions: organization, operational activities, and impact of the OH surveillance system; each dimension is then divided into four targets, each including four indicators. A semi-quantitative questionnaire enables the scoring of each indicator, with four levels according to the degree of satisfaction in the studied OH surveillance system. The evaluation is conducted by a panel of surveillance representatives (during a half-day workshop or with a back-and-forth process to reach a consensus). An R Shiny-based web application facilitates implementation of the evaluation and visualization of the results, and includes a benchmarking option. The tool was piloted on several foodborne hazards (i.e., Salmonella, Campylobacter, Listeria), emerging threats (e.g., antimicrobial resistance) and other zoonotic hazards (psittacosis) in multiple European countries in 2022. These case studies showed that the OH-EpiCap tool supports the tracing of strengths and weaknesses in epidemiological capacities and the identification of concrete and direct actions to improve collaborative activities at all steps of surveillance. It appears complementary to the existing EU-LabCap tool, designed to assess the capacity and capability of European microbiology laboratories. In addition, it provides opportunity to reinforce trust between surveillance stakeholders from across the system and to build a good foundation for a professional network for further collaboration.

Characterisation of antimicrobial usage in Danish pigs in 2020

Introduction

Denmark is one of the world's largest exporters of pigs and pig meat, so the sector plays an important role in the national antimicrobial use (AMU). The Danish government has run antimicrobial stewardship programs in collaboration with the pig industry for more than 25 years. These have resulted in substantial overall reductions in total AMU and limiting the use of fluoroquinolones, the 3rd and 4th generation cephalosporines and the polymyxin colistin. To understand where further reductions in AMU could take place, it is necessary to investigate which antimicrobials are being used, how, and for which reasons.

Materials and methods

We characterized the AMU in the Danish pig sector in 2020, providing new analytical insights based on data retrieved from the VetStat database. The AMU data were segmented into classes, routes of administration, treatment indications and age groups, and interpreted as an outcome of the interventions taken. We evaluated the current AMU regarding choice of antimicrobial class. Moreover, we discussed how to further improve the antimicrobial stewardship in Danish pig production to achieve additional reductions without jeopardizing animal welfare. Where relevant, two pig veterinary specialists were consulted.

Results

In 2020, 43.3 mg antimicrobials per population correction unit (PCU) were ascribed to the Danish pig sector. There was practically no use of fluoroquinolones, 3rd and 4th generation cephalosporins and polymyxins. Weaners related to 45% of the total AMU in pigs when measured in tonnes and 81% when measured in defined animal daily doses, of these 76% were ascribed to gastrointestinal indications and overall, 83% were administered perorally.

Conclusion

To enable further reductions in AMU, it should be investigated how and when to replace group treatments (e.g., all animals in section or a pen) with individual treatments. Moreover, prevention of disease and promotion of animal health should be prioritized, e.g., through focus on feed, vaccination, biosecurity, and disease eradication.

Capturing systematically users' experience of evaluation tools for integrated AMU and AMR surveillance

Tackling antimicrobial resistance (AMR) is a goal for many countries. Integrated surveillance of antimicrobial use (AMU) and resistance is a prerequisite for effective risk mitigation. Regular evaluation of any surveillance is needed to ensure its effectiveness and efficiency. The question is how to evaluate specifically integrated surveillance for AMU and AMR. In an international network called CoEvalAMR, we have developed guidelines for selection of the most appropriate tools for such an evaluation. Moreover, we have assessed different evaluation tools as examples using a country case format and a methodology with a focus on the user's experience. This paper describes the updated methodology, which consists of a brief introduction to the case and to the tool separately. Moreover, there are 12 functional aspects and nine content themes which should be scored using a 4-tiered scale. Additionally, four Strengths, Weaknesses, Opportunities, Threats (SWOT) questions should be addressed. Results are illustrated using radar diagrams. An example of application of the updated methodology is given using the ECoSur evaluation tool. No tool can cover all evaluation aspects comprehensively in a user-friendly manner, so the choice of tool must be based upon the specific evaluation purpose. Moreover, adequate resources, time and training are needed to obtain useful outputs from the evaluation. Our updated methodology can be used by tool users to share their experience with available tools, and hereby assist other users in identifying the most suited tool for their evaluation purpose. Additionally, tool developers can get valuable information for further improvements of their tool.

A systematic approach toward progressive improvement of national antimicrobial resistance surveillance systems in food and agriculture sectors

The first Food and Agriculture Organization of the United Nations (FAO) Action Plan on antimicrobial resistance (AMR), published in 2016, identified the need to develop capacity for AMR surveillance and monitoring in food and agriculture sectors. As part of this effort, FAO has developed the "Assessment Tool for Laboratories and AMR Surveillance Systems" (FAO-ATLASS) to assist countries in systematically assessing their AMR surveillance system in food and agriculture. FAO-ATLASS includes two different modules for surveillance and laboratory assessment. Each module includes two questionnaires that collect either qualitative or semi-quantitative data to describe and score the performance of national AMR surveillance system data production network, data collection and analysis, governance, communication and overall sustainability in a standardized manner. Based on information captured in the questionnaire by trained assessors (1) tables and figures describing the outputs of the surveillance system are automatically generated (2) a Progressive Improvement Pathway (PIP) stage, ranging from "1-limited" to "5-sustainable", is assigned to each laboratory assessed in the country, each area of the surveillance system and also to the overarching national AMR surveillance system. FAO-ATLASS allows national authorities to implement a strategic stepwise approach to improving their AMR surveillance systems via the FAO-ATLASS PIP system and provides an evidence base for actions and advocacy. The implementation of FAO-ATLASS at regional and global levels can contribute to harmonize and better coordinate strategies aimed at implementing an integrated AMR surveillance system under the One Health approach.

A One Health Evaluation of the Surveillance Systems on Tick-Borne Diseases in the Netherlands, Spain and Italy

Simple Summary

Ixodid ticks and tick-borne diseases are expanding their geographical range, but surveillance activities vary among countries. We analysed the surveillance systems in place in the Netherlands, Spain and Italy, to identify ideal elements to monitor tick-borne diseases, by using a One Health evaluation protocol. We identified differences among the three surveillance systems, with the Dutch initiative showing a high level of transdisciplinary collaboration, good identification of the actors and engagement of the public in research and education. Measurable outcomes have been generated, such as the reduction in tick bites and the discovery of new pathogens and tick species. In Italy and Spain, surveillance systems are based on compulsory notification to health authorities; legislation seems relevant but law enforcement alongside the availability of economic resources is rather fragmented and limited to the most severe diseases. The non-scientific community is marginally considered and collaborations are limited to local initiatives. Research activities in both countries have mostly contributed to gaining knowledge on the distribution of tick species and the discovery of new pathogens. Although all TBD surveillance plans comply with the EU regulations, the initiatives characterised by trans-disciplinary collaboration may be more effective for the surveillance and prevention of tick-transmitted diseases.

Abstract

To identify ideal elements for the monitoring and prevention of tick-borne diseases (TBD), we analysed the surveillance systems in place in the Netherlands, Spain and Italy. We applied a semi-quantitative evaluation to identify outcomes and assess the degree of One Health implementation. Differences emerged in the surveillance initiatives, as well as the One Health scores. The Dutch surveillance is dominated by a high level of transdisciplinary and trans-sectoral collaboration, enabling communication and data sharing among actors. Different project-based monitoring, research and educational activities are centrally coordinated and the non-scientific community is actively involved. All this yielded measurable health outcomes. In Italy and Spain, TBD surveillance and reporting systems are based on compulsory notification. Law enforcement, alongside dedicated time and availability of economic resources, is fragmented and limited to the most severe health issues. Veterinary and human medicine are the most involved disciplines, with the first prevailing in some contexts. Stakeholders are marginally considered and collaborations limited to local initiatives. Research activities have mostly contributed to gaining knowledge on the distribution of tick vectors and discovery of new pathogens. Although all TBD surveillance plans comply with EU regulations, initiatives characterised by transdisciplinary collaboration may be more effective for the surveillance and prevention of TBD.

Evaluation of the French surveillance system for epidemiological surveillance of antimicrobial resistance in the community and nursing homes

Background

Antimicrobial resistance (AMR) has been widely recognized as a major public health issue, which can be addressed through effective AMR surveillance systems. In 2018, a national surveillance programme for AMR in the community and nursing homes called Mission PRIMO was established in France. It builds on an existing network called MedQual-Ville that had been monitoring AMR mainly in the west of France community since 2003.

Objectives and Methods

To evaluate the MedQual-Ville surveillance activities and to formulate practical recommendations for improvement, using a semi-quantitative evaluation framework called OASIS.

Results

The evaluation showed that MedQual-Ville is overall a well-performing surveillance system. Its major strengths rely on excellent coordination and internal communication with clinical laboratories that participate on a voluntary basis. Surveillance objectives and procedures are clear to all participants. Hence, the quality and reliability of the data being produced is very high. At this stage, the major area for improvement is representativeness, with poor coverage achieved in several densely populated areas. Besides, the utility and impact of surveillance data could be improved by strengthening communication towards end-users, especially local prescribers.

Conclusions

There is currently no European programme or guidance for AMR surveillance in the community and nursing homes. Our results partly fill this gap, by evaluating how surveillance is being performed in France and providing recommendations that could be applicable to other countries with similar health systems. This work also highlighted the relevance of OASIS for evaluation of surveillance systems in the human sector.

Electronic information systems for One Health surveillance of antimicrobial resistance: a systematic scoping review

INTRODUCTION

Electronic information systems (EIS) that implement a 'One Health' approach by integrating antimicrobial resistance (AMR) data across the human, animal and environmental health sectors, have been identified as a global priority. However, evidence on the availability, technical capacities and effectiveness of such EIS is scarce.

METHODS

Through a qualitative synthesis of evidence, this systematic scoping review aims to: identify EIS for AMR surveillance that operate across human, animal and environmental health sectors; describe their technical characteristics and capabilities; and assess whether there is evidence for the effectiveness of the various EIS for AMR surveillance. Studies and reports between 1 January 2000 and 21 July 2021 from peer-reviewed and grey literature in the English language were included.

RESULTS

26 studies and reports were included in the final review, of which 27 EIS were described. None of the EIS integrated AMR data in a One Health approach across all three sectors. While there was a lack of evidence of thorough evaluations of the effectiveness of the identified EIS, several surveillance system effectiveness indicators were reported for most EIS. Standardised reporting of the effectiveness of EIS is recommended for future publications. The capabilities of the EIS varied in their technical design features, in terms of usability, data display tools and desired outputs. EIS that included interactive features, and geospatial maps are increasingly relevant for future trends in AMR data analytics.

CONCLUSION

No EIS for AMR surveillance was identified that was designed to integrate a broad range of AMR data from humans, animals and the environment, representing a major gap in global efforts to implement One Health approaches to address AMR.

Evaluating Integrated Surveillance for Antimicrobial Use and Resistance in England: A Qualitative Study

Integrated surveillance systems for antimicrobial use (AMU) and antimicrobial resistance (AMR) require regular evaluation to ensure the effectiveness and efficiency of the system. An important step in the evaluation is to choose an appropriate tool for the purpose of the evaluation. The “Integrated Surveillance System Evaluation” (ISSE) framework is a conceptual framework that was developed to evaluate One Health (OH) integration in surveillance system for AMU/AMR. This study aimed to evaluate the performance and value of integrated surveillance system for AMU/AMR in England by applying the ISSE framework, which was used to develop data collection protocols and define the study design. A qualitative study using semi-structured interviews was conducted to collect the data and analyse it thematically. Eighteen stakeholders from human, animal, food and environment sectors that are involved in AMU/AMR surveillance were interviewed. Four main themes emerged from the analysis: (1) Cross-sectoral integration in the surveillance system for AMU/AMR; (2) Production of OH outputs and outcomes; (3) Drivers and barriers to cross-sectoral collaboration; and 4) Need for more cross-sectoral collaboration. The findings showed that there were links between integrated surveillance information, decision making and interventions. However, there were only few OH examples, such as the UK AMR contingency plan, where the potential of cross-sectoral collaboration was fully exploited. A lot of the benefits described were related to the generation of information and increase in knowledge and understanding without links to how the information generated was used. While these intangible benefits have a value on their own, being able to link surveillance information and mitigation measures would help to enhance the value of integrated surveillance. In terms of improvement, the main areas identified were the development of more harmonised methods for data collection and analysis, provision of resources dedicated to cross-sectoral collaboration, improved coordination, and collection of surveillance data from the environment and from companion animals. By identifying links between OH surveillance information produced and various outputs and outcomes; this study helped to understand the wider benefits of integrated surveillance for AMU/AMR in England and provided insights on how the system could be improved and efficiency increased.

One Health Evaluation of Antimicrobial Use and Resistance Surveillance: A Novel Tool for Evaluating Integrated, One Health Antimicrobial Resistance and Antimicrobial Use Surveillance Programs

We describe the development, application and utility of our novel, One Health Evaluation of Antimicrobial Use and Resistance Surveillance (OHE-AMURS) tool that we created to evaluate progress toward integrated, One Health surveillance of antimicrobial resistance (AMR) and antimicrobial use (AMU) as a complex system in Canada. We conducted a qualitative inquiry into the current state of policy and programs for integrated AMR/AMU surveillance using explicit and tacit knowledge. To assess the "messy" state of public health surveillance program development, we synthesized recommendations from previous reports by the National Collaborating Centre for Infectious Diseases and the Canadian Council of Chief Veterinary Officers; conducted an environmental scan to find all federal, provincial, and territorial AMR/AMU surveillance programs in Canada; and conducted semi-structured interviews with Canadian subject matter experts. To integrate evidence from these different sources we adapted two published tools to create a new evaluation matrix, deriving 36 components of the ideal integrated AMR/AMU surveillance system. Our two-way matrix tool allowed us to examine seven common, foundational elements of sustainable programs for each component, and assign a stage of development/sustainability ranking for each component according to the matrix definitions. Our adaptable novel tool allowed for granular and repeatable assessment of the many components of a complex surveillance system. The assessment proved robust and exacting to ensure transparency in our methods and results. The matrix allows flexible assignment of program components based on program principles, and stages can be adapted to evaluate any aspect of an AMR/AMU surveillance or other multi-faceted, multi-jurisdictional system. Future refinement should include an assessment of the scope of surveillance components.