Effect of Supportive Supervision on Performance of Malaria Rapid Diagnostic Tests in Sub-Saharan Africa

Can Outreach Training and Supportive Supervision Improve Competency in Malaria Service Delivery? An Evaluation in Cameroon, Ghana, Niger, and Zambia

Outreach Training and Supportive Supervision (OTSS) of malaria services at health facilities has been adopted by numerous malaria-endemic countries. The OTSS model is characterized by a hands-on method to enhance national guidelines and supervision tools, train supervisors, and perform supervision visits. An independent evaluation was conducted to evaluate the effectiveness of OTSS on health worker competence in the clinical management of malaria, parasitological diagnosis, and prevention of malaria in pregnancy. From 2018 to 2021, health facilities in Cameroon, Ghana, Niger, and Zambia received OTSS visits during which health workers were observed directly during patient consultations, and supervisors completed standardized checklists to assess their performance. Mixed-effects logistic regression models were developed to assess the impact of increasing OTSS visit number on a set of eight program-generated outcome indicators, including overall competency and requesting a confirmatory malaria test appropriately. Seven of eight outcome indicators showed evidence of beneficial effects of increased OTSS visits. Odds of health workers reaching competency thresholds for the malaria-in-pregnancy checklist increased by more than four times for each additional OTSS visit (odds ratio [OR], 4.62; 95% CI, 3.62-5.88). Each additional OTSS visit was associated with almost four times the odds of the health worker foregoing antimalarial prescriptions for patients who tested negative for malaria (OR, 3.80; 95% CI, 2.35-6.16). This evaluation provides evidence that successive OTSS visits result in meaningful improvements in indicators linked to quality case management of patients attending facilities for malaria diagnosis and treatment, as well as quality malaria prevention services received by women attending antenatal services.

Outreach Training and Supportive Supervision for Quality Malaria Service Delivery: A Qualitative Evaluation in 11 Sub-Saharan African Countries

Quality improvement of malaria services aims to ensure that more patients receive accurate diagnosis, appropriate treatment, and referral. The Outreach Training and Supportive Supervision Plus (OTSS+) approach seeks to improve health facility readiness and provider competency through onsite supportive supervision, troubleshooting, and on-the-job training. As part of a multicomponent evaluation, qualitative research was conducted to understand the value of the OTSS+ approach for malaria quality improvement. Semistructured key informant interviews, focus group discussions, and structured health facility-based interviews were used to gather stakeholder perspectives at subnational, national, and global levels. Data were collected globally and in 11 countries implementing OTSS+; in-depth data collection was done in four: Cameroon, Ghana, Niger, and Zambia. Study sites and participants were selected purposively. Verbatim transcripts were analyzed thematically, following the Framework approach. A total of 262 participants were included in the analysis; 98 (37.4%) were supervisees, 99 (37.8%) were supervisors, and 65 (24.8%) were other stakeholders. The OTSS+ approach was perceived to improve provider knowledge and skills in malaria service delivery and to improve data and supply management indirectly. Improvements were attributed to a combination of factors. Participants valued the relevance, adaptation, and digitization of supervision checklists; the quality and amount of contact with problem-solving supervisors; and the joint identification of problems and solutions, and development of action plans. Opportunities for improvement were digitized checklist refinement, assurance of a sufficient pool of supervisors, prioritization of health facilities, action plan dissemination and follow-up, and data review and use. The OTSS+ approach was perceived to be a useful quality improvement approach for malaria services.

The use of dried tube specimens of Plasmodium falciparum in an external quality assessment programme to evaluate health worker performance for malaria rapid diagnostic testing in healthcare centres in Togo

BACKGROUND

The use of rapid diagnostic tests (RDTs) to diagnose malaria is common in sub-Saharan African laboratories, remote primary health facilities and in the community. Currently, there is a lack of reliable methods to ascertain health worker competency to accurately use RDTs in the testing and diagnosis of malaria. Dried tube specimens (DTS) have been shown to be a consistent and useful method for quality control of malaria RDTs; however, its application in National Quality Management programmes has been limited.

METHODS

A Plasmodium falciparum strain was grown in culture and harvested to create DTS of varying parasite density (0, 100, 200, 500 and 1000 parasites/µL). Using the dried tube specimens as quality control material, a proficiency testing (PT) programme was carried out in 80 representative health centres in Togo. Health worker competency for performing malaria RDTs was assessed using five blinded DTS samples, and the DTS were tested in the same manner as a patient sample would be tested by multiple testers per health centre.

RESULTS

All the DTS with 100 parasites/µl and 50% of DTS with 200 parasites/µl were classified as non-reactive during the pre-PT quality control step. Therefore, data from these parasite densities were not analysed as part of the PT dataset. PT scores across all 80 facilities and 235 testers was 100% for 0 parasites/µl, 63% for 500 parasites/µl and 93% for 1000 parasites/µl. Overall, 59% of the 80 healthcare centres that participated in the PT programme received a score of 80% or higher on a set of 0, 500 and 1000 parasites/ µl DTS samples. Sixty percent of health workers at these centres recorded correct test results for all three samples.

CONCLUSIONS

The use of DTS for a malaria PT programme was the first of its kind ever conducted in Togo. The ease of use and stability of the DTS illustrates that this type of samples can be considered for the assessment of staff competency. The implementation of quality management systems, refresher training and expanded PT at remote testing facilities are essential elements to improve the quality of malaria diagnosis.

Perspectives on Implementation Considerations and Costs of Malaria Case Management Supportive Supervision

Between 2012 and 2017, the U.S. President's Malaria Initiative-funded MalariaCare project supported national malaria control programs in sub-Saharan Africa to implement a case management quality assurance (QA) system for malaria and other febrile illnesses. A major component of the system was outreach training and supportive supervision (OTSS), whereby trained government health personnel visited health facilities to observe health-care practices using a standard checklist, to provide individualized feedback to staff, and to develop health facility-wide action plans based on observation and review of facility registers. Based on MalariaCare's experience, facilitating visits to more than 5,600 health facilities in nine countries, we found that programs seeking to implement similar supportive supervision schemes should consider ensuring the following: 1) develop a practical checklist that balances information gathering and mentorship; 2) establish basic competency criteria for supervisors and periodically assess supervisor performance in the field; 3) conduct both technical skills training and supervision skills training; 4) establish criteria for selecting facilities to conduct OTSS and determine the appropriate frequency of visits; and 5) use electronic data collection systems where possible. Cost will also be a significant consideration: the average cost per OTSS visit ranged from $44 to $333. Significant variation in costs was due to factors such as travel time, allowances for government personnel, length of the visit, and involvement of central level officials. Because the cost of conducting supportive supervision prohibits regularly visiting all health facilities, internal QA measures could also be considered as alternative or complementary activities to supportive supervision.

Introduction and Evaluation of an Electronic Tool for Improved Data Quality and Data Use during Malaria Case Management Supportive Supervision

Although on-site supervision programs are implemented in many countries to assess and improve the quality of care, few publications have described the use of electronic tools during health facility supervision. The President's Malaria Initiative-funded MalariaCare project developed the MalariaCare Electronic Data System (EDS), a custom-built, open-source, Java-based, Android application that links to District Health Information Software 2, for data storage and visualization. The EDS was used during supervision visits at 4,951 health facilities across seven countries in Africa. The introduction of the EDS led to dramatic improvements in both completeness and timeliness of data on the quality of care provided for febrile patients. The EDS improved data completeness by 47 percentage points (42-89%) on average when compared with paper-based data collection. The average time from data submission to a final data analysis product dropped from over 5 months to 1 month. With more complete and timely data available, the Ministry of Health and the National Malaria Control Program (NMCP) staff could more effectively plan corrective actions and promptly allocate resources, ultimately leading to several improvements in the quality of malaria case management. Although government staff used supervision data during MalariaCare-supported lessons learned workshops to develop plans that led to improvements in quality of care, data use outside of these workshops has been limited. Additional efforts are required to institutionalize the use of supervision data within ministries of health and NMCPs.

Effect of Supportive Supervision on Malaria Microscopy Competencies in Sub-Saharan Africa

Although light microscopy is the reference standard for diagnosing malaria, maintaining skills over time can be challenging. Between 2015 and 2017, the U.S. President's Malaria Initiative-funded MalariaCare project supported outreach training and supportive supervision (OTSS) visits at 1,037 health facilities in seven African countries to improve performance in microscopy slide preparation, staining, and reading. During these visits, supervisors observed and provided feedback to health-care workers (HCWs) performing malaria microscopy using a 30-step checklist. Of the steps observed in facilities with at least three visits, the proportion of HCWs that performed each step correctly at baseline ranged from 63.2% to 94.2%. The change in the proportion of HCWs performing steps correctly by the third visit ranged from 16.7 to 23.6 percentage points (n = 916 observations). To assess the overall improvement, facility scores were calculated based on the steps performed correctly during each visit. The mean score at baseline was 85.7%, demonstrating a high level of performance before OTSS. Regression analysis predicted an improvement in facility scores of 3.6 percentage points (P < 0.001) after three visits across all countries. In reference-level facilities with consistently high performance on microscopy procedures and parasite detection, quality assurance (QA) mechanisms could prioritize more advanced skills, such as proficiency testing for parasite counting and species identification. However, in settings with high staff turnover and declining use of microscopy in favor of rapid diagnostic tests, additional supervision visits and/or additional QA measures may be required to improve and maintain performance.