1
|
Patel C, Sargent GM, Tinessia A, Mayfield H, Chateau D, Ali A, Tuibeqa I, Sheel M. Measuring what matters: Context-specific indicators for assessing immunisation performance in Pacific Island Countries and Areas. PLOS GLOBAL PUBLIC HEALTH 2024; 4:e0003068. [PMID: 39052626 PMCID: PMC11271932 DOI: 10.1371/journal.pgph.0003068] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/12/2024] [Accepted: 06/18/2024] [Indexed: 07/27/2024]
Abstract
Increasing countries' access to data can improve immunisation coverage through evidence-based decision-making. However, data collection and reporting is resource-intensive, so needs to be pragmatic, especially in low-and-middle-income countries. We aimed to identify which indicators are most important for measuring, and improving, national immunisation performance in Pacific Island Countries (PICs). We conducted an expert elicitation study, asking 13 experts involved in delivering immunisation programs, decision-makers, health information specialists, and global development partners across PICs to rate 41 indicators based on their knowledge of the feasibility and relevance of each indicator. We also asked experts their preferences for indicators to be retained or removed from a list of indicators for PICs. Experts participated in two rating rounds, with a discussion on the reasons for ratings before the second round. We calculated mean scores for feasibility and relevance, and ranked indicators based on experts' preferences and mean scores. We used framework analysis to identify reasons for selecting indicators. Experts agreed that certain indicators were essential to measure (e.g. data use in program planning and measles vaccination coverage), but preferences varied for most indicators. Preferences to include indicators in a set of indicators for PICs moderately correlated with scores for relevance (r = 0.68) and feasibility (r = 0.56). In discussions, experts highlighted usefulness for decision-making and ease of data collection, reporting and interpretation as the main reasons driving indicator selection. Country-specific factors such as health system factors, roles and influence of various immunisation actors, and macro-level factors (namely population size, distribution and mobility) affected relevance and feasibility, leading us to conclude that a single set of indicators for all PICs is inappropriate. Rather than having a strict set of indicators that all countries must measure and report against, performance indicators should be flexible, country-specific, and selected in consultation with immunisation actors who collect and use the data.
Collapse
Affiliation(s)
- Cyra Patel
- National Centre for Epidemiology and Population Health, Australian National University, Canberra, Australian Capital Territory, Australia
| | - Ginny M. Sargent
- National Centre for Epidemiology and Population Health, Australian National University, Canberra, Australian Capital Territory, Australia
| | - Adeline Tinessia
- Sydney School of Public Health, Faculty of Medicine and Health, The University of Sydney, Camperdown, New South Wales, Australia
| | - Helen Mayfield
- UQ Centre for Clinical Research, The University of Queensland, Brisbane, Queensland, Australia
| | - Dan Chateau
- National Centre for Epidemiology and Population Health, Australian National University, Canberra, Australian Capital Territory, Australia
| | - Akeem Ali
- World Health Organization, Seoul, Republic of Korea
| | - Ilisapeci Tuibeqa
- Department of Paediatrics, Colonial War Memorial Hospital, Suva, Fiji
| | - Meru Sheel
- National Centre for Epidemiology and Population Health, Australian National University, Canberra, Australian Capital Territory, Australia
- Sydney School of Public Health, Faculty of Medicine and Health, The University of Sydney, Camperdown, New South Wales, Australia
- Sydney Institute for Infectious Diseases, Faculty of Medicine and Health, The University of Sydney, Westmead, New South Wales, Australia
| |
Collapse
|
2
|
Rendell N, Rosewell A, Lokuge K, Field E. Common Features of Selection Processes of Health System Performance Indicators in Primary Healthcare: A Systematic Review. Int J Health Policy Manag 2022; 11:2805-2815. [PMID: 35368205 PMCID: PMC10105193 DOI: 10.34172/ijhpm.2022.6239] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/15/2021] [Accepted: 03/06/2022] [Indexed: 11/09/2022] Open
Abstract
BACKGROUND Health system performance indicators are widely used to assess primary healthcare (PHC) performance. Despite the numerous tools and some convergence on indicator criteria, there is not a clear understanding of the common features of indicator selection processes. We aimed to review the literature to identify papers that document indicator selection processes for health system performance indicators in PHC. METHODS We searched the online databases Scopus, Medline, and CINAHL, as well as the grey literature, without time restrictions, initially on July 31, 2019 followed by an update November 13, 2020. Empirical studies or reports were included if they described the selection of health system performance indicators or frameworks, that included PHC indicators. A combination of the process focussed research question and qualitative analysis meant a quality appraisal tool or assessment of bias could not meaningfully be applied to assess individual studies. We undertook an inductive analysis based on potential indicator selection processes criteria, drawn from health system performance indicator appraisal tools reported in the literature. RESULTS We identified 16 503 records of which 28 were included in the review. Most studies used a descriptive case study design. We found no consistent variations between indicator selection processes of health systems of high income and low- or lower-middle income countries. Identified common features of selection processes for indicators in PHC include literature review or adaption of an existing framework as an initial step; a consensus building process with stakeholders; structuring indicators into categories; and indicator criteria focusing on validity and feasibility. The evidence around field testing with utility and consideration of reporting burden was less clear. CONCLUSION Our findings highlight several characteristics of health system indicator selection processes. These features provide the groundwork to better understand how to value indicator selection processes in PHC.
Collapse
Affiliation(s)
- Nicole Rendell
- National Centre for Epidemiology and Population Health, Australian National University, Canberra, ACT, Australia
| | - Alexander Rosewell
- School of Public Health and Community Medicine, University of New South Wales, Sydney, NSW, Australia
| | - Kamalini Lokuge
- National Centre for Epidemiology and Population Health, Australian National University, Canberra, ACT, Australia
| | - Emma Field
- National Centre for Epidemiology and Population Health, Australian National University, Canberra, ACT, Australia
- Menzies School of Health Research, Brisbane, QLD, Australia
| |
Collapse
|
3
|
de Lima IB, Bernadi FA, Yamada DB, Vinci ALT, Rijo RPCL, Alves D, Furegato ARF. The use of indicators for the management of Mental Health Services. Rev Lat Am Enfermagem 2021; 29:e3409. [PMID: 33852681 PMCID: PMC8040785 DOI: 10.1590/1518-8345.4202.3409] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/16/2020] [Accepted: 07/31/2020] [Indexed: 11/22/2022] Open
Abstract
OBJECTIVE to identify indicators that can be used in the management of Mental Health Services. METHOD an integrative review in which we adopted the Population, Concept, and Context strategy to formulate the following Guiding Question: "Which indicators can be used for the management of mental health services?". RESULTS a total of 22 articles were included and divided into two main groups: countries with initial high income (54%) as well as low- and middle-income countries (46%). We identified 5 studies that had experienced the use of indicators, 5 studies that had reported partial implementation, 9 studies that did not report use or implementation, 1 study on the indicator selection process, 1 as an implementation pilot, and a final study with a discussion for implementation. High-income countries also find it difficult to implement mental health indicators. The main difficulties in adopting the use of indicators are lack of basic mental health services, financial resources, legislation, political interest, and guidelines for its management. CONCLUSION it is unusual to find a descriptive comparison of quality monitoring programs at the system level in the technical-scientific literature related to mental health indicators.
Collapse
Affiliation(s)
- Inacia Bezerra de Lima
- Universidade de São Paulo, Escola de Enfermagem de Ribeirão Preto, PAHO/WHO Collaborating Centre for Nursing Research Development, Ribeirão Preto, SP, Brazil
- Scholarship holder at the Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES), Brazil
| | | | - Diego Bettiol Yamada
- Scholarship holder at the Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES), Brazil
- Universidade de São Paulo, Faculdade de Medicina de Ribeirão Preto, Ribeirão Preto, SP, Brazil
| | | | | | - Domingos Alves
- Universidade de São Paulo, Faculdade de Medicina de Ribeirão Preto, Ribeirão Preto, SP, Brazil
| | - Antonia Regina Ferreira Furegato
- Universidade de São Paulo, Escola de Enfermagem de Ribeirão Preto, PAHO/WHO Collaborating Centre for Nursing Research Development, Ribeirão Preto, SP, Brazil
| |
Collapse
|
4
|
Ahuja S, Khan A, Goulding L, Bansal RK, Shidhaye R, Thornicroft G, Jordans M. Evaluation of a new set of indicators for mental health care implemented in Madhya Pradesh, India: a mixed methods study. Int J Ment Health Syst 2020; 14:7. [PMID: 32055254 PMCID: PMC7011241 DOI: 10.1186/s13033-020-0341-4] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 06/10/2019] [Accepted: 01/27/2020] [Indexed: 11/29/2022] Open
Abstract
BACKGROUND Mental health information systems are, in general, inadequate and unreliable in India. We have developed key mental health indicators for measuring mental health service coverage in primary care. This study aims to evaluate the use of these new indicators in seven health care facilities in Sehore District of Madhya Pradesh in India. METHODS The study employed a mixed methods approach. We conducted: a qualitative study (n = 26) with health workers, Health Management Information Systems (HMIS) staff, project coordinators and supervisors; a review of case records (n = 61 at time 1 and n = 74 at time 2); and a structured questionnaire (n = 26) with health workers. The performance, user-friendliness, appropriateness, perceived utility and sustainability of the use of new mental health indicators was assessed. RESULTS High levels of completion, and correctness of completion, of the new mental health indicators were found for the case records. The simplicity of the forms, as well as technical support from the project team, contributed to acceptability and feasibility of implementation. Perceived sustainability of the new forms was, however, affected by the overstretched primary care staff. Further work is needed to support the integration of mental health with routine HMIS. CONCLUSION This study demonstrated that the implementation of key mental health service delivery indicators in Sehore District primary care facilities was feasible. Technical assistance was imperative in maintaining the performance of the indicators over the two studied time points. The integration of mental health indicators in routine health information systems, and political buy-in, are needed to monitor and sustain community mental health programmes in India.
Collapse
Affiliation(s)
- Shalini Ahuja
- Centre for Global Mental Health, Health Services and Population Research Department, Institute of Psychiatry, Psychology & Neuroscience, King’s College London, London, UK
- Centre for Implementation Science, Institute of Psychiatry, Psychology & Neuroscience, King’s College London, De Crespigny Park, London, SE5 8AF UK
| | - Azaz Khan
- Programme for Improving Mental Health Care, Sangath, Bhopal India
| | - Lucy Goulding
- Centre for Implementation Science, Institute of Psychiatry, Psychology & Neuroscience, King’s College London, De Crespigny Park, London, SE5 8AF UK
| | | | | | - Graham Thornicroft
- Centre for Global Mental Health, Health Services and Population Research Department, Institute of Psychiatry, Psychology & Neuroscience, King’s College London, London, UK
| | - Mark Jordans
- Centre for Global Mental Health, Health Services and Population Research Department, Institute of Psychiatry, Psychology & Neuroscience, King’s College London, London, UK
| |
Collapse
|