The Paediatric Admission Quality of Care (PAQC) score: designing a tool to measure the quality of early inpatient paediatric care in a low-income setting

Pairwise joint modeling of clustered and high-dimensional outcomes with covariate missingness in pediatric pneumonia care

Multiple outcomes reflecting different aspects of routine care are a common phenomenon in health care research. A common approach of handling such outcomes is multiple univariate analyses, an approach which does not allow for answering research questions pertaining to joint inference. In this study, we sought to study associations among nine pediatric pneumonia care outcomes spanning assessment, diagnosis and treatment domains of care, while circumventing the computational challenge posed by their clustered and high-dimensional nature and incompletely recorded covariates. We analyzed data from a cluster randomized trial conducted in 12 Kenyan hospitals. There were varying degrees of missingness in the covariates of interest, and these were multiply imputed using latent normal joint modeling. We used the pairwise joint modeling strategy to fit a correlated random effects joint model for the nine outcomes. This entailed fitting 36 bivariate generalized linear mixed models and deriving inference for the joint model using pseudo-likelihood theory. We also analyzed the nine outcomes separately before and after multiple imputation. We observed joint effects of patient-, clinician- and hospital-level factors on pneumonia care indicators before and after multiple imputation of missing covariates. In both pairwise joint modeling and separate univariate analysis methods, enhanced audit and feedback improved documentation and adherence to recommended clinical guidelines over time in six and five pneumonia care indicators, respectively. Additionally, multiple imputation improved precision of parameter estimates compared to complete case analysis. The strength and direction of association among pneumonia outcomes varied within and across the three domains of pneumonia care.

Oxygen systems and quality of care for children with pneumonia, malaria and diarrhoea: Analysis of a stepped-wedge trial in Nigeria

OBJECTIVES

To evaluate the effect of improved hospital oxygen systems on quality of care (QOC) for children with severe pneumonia, severe malaria, and diarrhoea with severe dehydration.

DESIGN

Stepped-wedge cluster randomised trial (unblinded), randomised at hospital-level.

SETTING

12 hospitals in south-west Nigeria.

PARTICIPANTS

7,141 children (aged 28 days to 14 years) admitted with severe pneumonia, severe malaria or diarrhoea with severe dehydration between January 2014 and October 2017.

INTERVENTIONS

Phase 1 (pulse oximetry) introduced pulse oximetry for all admitted children. Phase 2 (full oxygen system) (i) standardised oxygen equipment package, (ii) clinical education and support, (iii) technical training and support, and (iv) infrastructure and systems support.

OUTCOME MEASURES

We used quantitative QOC scores evaluating assessment, diagnosis, treatment, and monitoring practices against World Health Organization and Nigerian standards. We evaluated mean differences in QOC scores between study periods (baseline, oximetry, full oxygen system), using mixed-effects linear regression.

RESULTS

7,141 eligible participants; 6,893 (96.5%) had adequate data for analysis. Mean paediatric QOC score (maximum 6) increased from 1.64 to 3.00 (adjusted mean difference 1.39; 95% CI 1.08-1.69, p<0.001) for severe pneumonia and 2.81 to 4.04 (aMD 1.53; 95% CI 1.23-1.83, p<0.001) for severe malaria, comparing the full intervention to baseline, but did not change for diarrhoea with severe dehydration (aMD -0.12; 95% CI -0.46-0.23, p = 0.501). After excluding practices directly related to pulse oximetry and oxygen, we found aMD 0.23 for severe pneumonia (95% CI -0.02-0.48, p = 0.072) and 0.65 for severe malaria (95% CI 0.41-0.89, p<0.001) comparing full intervention to baseline. Sub-analysis showed some improvements (and no deterioration) in care processes not directly related to oxygen or pulse oximetry.

CONCLUSION

Improvements in hospital oxygen systems were associated with higher QOC scores, attributable to better use of pulse oximetry and oxygen as well as broader improvements in clinical care, with no negative distortions in care practices.

TRIAL REGISTRATION

ACTRN12617000341325.

Handling missing data in a composite outcome with partially observed components: simulation study based on clustered paediatric routine data

Composite scores are useful in providing insights and trends about complex and multidimensional quality of care processes. However, missing data in subcomponents may hinder the overall reliability of a composite measure. In this study, strategies for handling missing data in Paediatric Admission Quality of Care (PAQC) score, an ordinal composite outcome, were explored through a simulation study. Specifically, the implications of the conventional method employed in addressing missing PAQC score subcomponents, consisting of scoring missing PAQC score components with a zero, and a multiple imputation (MI)-based strategy, were assessed. The latent normal joint modelling MI approach was used for the latter. Across simulation scenarios, MI of missing PAQC score elements at item level produced minimally biased estimates compared to the conventional method. Moreover, regression coefficients were more prone to bias compared to standards errors. Magnitude of bias was dependent on the proportion of missingness and the missing data generating mechanism. Therefore, incomplete composite outcome subcomponents should be handled carefully to alleviate potential for biased estimates and misleading inferences. Further research on other strategies of imputing at the component and composite outcome level and imputing compatibly with the substantive model in this setting, is needed.

Examining which clinicians provide admission hospital care in a high mortality setting and their adherence to guidelines: an observational study in 13 hospitals

BACKGROUND

We explored who actually provides most admission care in hospitals offering supervised experiential training to graduating clinicians in a high mortality setting where practices deviate from guideline recommendations.

METHODS

We used a large observational data set from 13 Kenyan county hospitals from November 2015 through November 2018 where patients were linked to admitting clinicians. We explored guideline adherence after creating a cumulative correctness of Paediatric Admission Quality of Care (cPAQC) score on a 5-point scale (0-4) in which points represent correct, sequential progress in providing care perfectly adherent to guidelines comprising admission assessment, diagnosis and treatment. At the point where guideline adherence declined the most we dichotomised the cPAQC score and used multilevel logistic regression models to explore whether clinician and patient-level factors influence adherence.

RESULTS

There were 1489 clinicians who could be linked to 53 003 patients over a period of 3 years. Patients were rarely admitted by fully qualified clinicians and predominantly by preregistration medical officer interns (MOI, 46%) and diploma level clinical officer interns (COI, 41%) with a median of 28 MOI (range 11-68) and 52 COI (range 5-160) offering care per study hospital. The cPAQC scores suggest that perfect guideline adherence is found in ≤12% of children with malaria, pneumonia or diarrhoea with dehydration. MOIs were more adherent to guidelines than COI (adjusted OR 1.19 (95% CI 1.07 to 1.34)) but multimorbidity was significantly associated with lower guideline adherence.

CONCLUSION

Over 85% of admissions to hospitals in high mortality settings that offer experiential training in Kenya are conducted by preregistration clinicians. Clinical assessment is good but classifying severity of illness in accordance with guideline recommendations is a challenge. Adherence by MOI with 6 years' training is better than COI with 3 years' training, performance does not seem to improve during their 3 months of paediatric rotations.

Handling missing data in modelling quality of clinician-prescribed routine care: Sensitivity analysis of departure from missing at random assumption

Missing information is a major drawback in analyzing data collected in many routine health care settings. Multiple imputation assuming a missing at random mechanism is a popular method to handle missing data. The missing at random assumption cannot be confirmed from the observed data alone, hence the need for sensitivity analysis to assess robustness of inference. However, sensitivity analysis is rarely conducted and reported in practice. We analyzed routine paediatric data collected during a cluster randomized trial conducted in Kenyan hospitals. We imputed missing patient and clinician-level variables assuming the missing at random mechanism. We also imputed missing clinician-level variables assuming a missing not at random mechanism. We incorporated opinions from 15 clinical experts in the form of prior distributions and shift parameters in the delta adjustment method. An interaction between trial intervention arm and follow-up time, hospital, clinician and patient-level factors were included in a proportional odds random-effects analysis model. We performed these analyses using R functions derived from the jomo package. Parameter estimates from multiple imputation under the missing at random mechanism were similar to multiple imputation estimates assuming the missing not at random mechanism. Our inferences were insensitive to departures from the missing at random assumption using either the prior distributions or shift parameters sensitivity analysis approach.

Analysis of Hierarchical Routine Data With Covariate Missingness: Effects of Audit & Feedback on Clinicians' Prescribed Pediatric Pneumonia Care in Kenyan Hospitals

Background: Routine clinical data are widely used in many countries to monitor quality of care. A limitation of routine data is missing information which occurs due to lack of documentation of care processes by health care providers, poor record keeping, or limited health care technology at facility level. Our objective was to address missing covariates while properly accounting for hierarchical structure in routine pediatric pneumonia care.

Methods: We analyzed routine data collected during a cluster randomized trial to investigating the effect of audit and feedback (A&F) over time on inpatient pneumonia care among children admitted in 12 Kenyan hospitals between March and November 2016. Six hospitals in the intervention arm received enhance A&F on classification and treatment of pneumonia cases in addition to a standard A&F report on general inpatient pediatric care. The remaining six in control arm received standard A&F alone. We derived and analyzed a composite outcome known as Pediatric Admission Quality of Care (PAQC) score. In our analysis, we adjusted for patients, clinician and hospital level factors. Missing data occurred in patient and clinician level variables. We did multiple imputation of missing covariates within the joint model imputation framework. We fitted proportion odds random effects model and generalized estimating equation (GEE) models to the data before and after multilevel multiple imputation.

Results: Overall, 2,299 children aged 2 to 59 months were admitted with childhood pneumonia in 12 hospitals during the trial period. 2,127 (92%) of the children (level 1) were admitted by 378 clinicians across the 12 hospitals. Enhanced A&F led to improved inpatient pediatric pneumonia care over time compared to standard A&F. Female clinicians and hospitals with low admission workload were associated with higher uptake of the new pneumonia guidelines during the trial period. In both random effects and marginal model, parameter estimates were biased and inefficient under complete case analysis.

Conclusions: Enhanced A&F improved the uptake of WHO recommended pediatric pneumonia guidelines over time compared to standard audit and feedback. When imputing missing data, it is important to account for the hierarchical structure to ensure compatibility with analysis models of interest to alleviate bias.

Adherence to clinical guidelines is associated with reduced inpatient mortality among children with severe anemia in Ugandan hospitals

BACKGROUND

In resource limited settings, there is variability in the level of adherence to clinical guidelines in the inpatient management of children with common conditions like severe anemia. However, there is limited data on the effect of adherence to clinical guidelines on inpatient mortality in children managed for severe anemia.

METHODS

We analyzed data from an uncontrolled before and after in-service training intervention to improve quality of care in Lira and Jinja regional referral hospitals in Uganda. Inpatient records of children aged 0 to 5 years managed as cases of 'severe anemia (SA)' were reviewed to ascertain adherence to clinical guidelines and compare inpatient deaths in SA children managed versus those not managed according to clinical guidelines. Logistic regression analysis was conducted to evaluate the relationship between clinical care factors and inpatient deaths amongst patients managed for SA.

RESULTS

A total of 1,131 children were assigned a clinical diagnosis of 'severe anemia' in the two hospitals. There was improvement in the level of care after the in-service training intervention with more children being managed according to clinical guidelines compared to the period before, 218/510 (42.7%) vs 158/621 (25.4%) (p < 0.001). Overall, children managed according to clinical guidelines had reduced risk of inpatient mortality compared to those not managed according to clinical guidelines, [OR 0.28, (95%, CI 0.14, 0.55), p = 0.001]. Clinical care factors associated with decreased risk of inpatient death included, having pre-transfusion hemoglobin done to confirm diagnosis [OR 0.5; 95% CI 0.29, 0.87], a co-morbid diagnosis of severe malaria [OR 0.4; 95% CI 0.25, 0.76], and being reviewed after admission by a clinician [OR 0.3; 95% CI 0.18, 0.59], while a co-morbid diagnosis of severe acute malnutrition was associated with increased risk of inpatient death [OR 4.2; 95% CI 2.15, 8.22].

CONCLUSION

Children with suspected SA who are managed according to clinical guidelines have lower in-hospital mortality than those not managed according to the guidelines. Efforts to reduce inpatient mortality in SA children in resource-limited settings should focus on training and supporting health workers to adhere to clinical guidelines.

Magnitude and pattern of improvement in processes of care for hospitalised children with diarrhoea and dehydration in Kenyan hospitals participating in a clinical network

OBJECTIVE

WHO recommends optimisation of available interventions to reduce deaths of under-five children with diarrhoea and dehydration (DD). Clinical networks may help improve practice across many hospitals but experience with such networks is scarce. We describe magnitude and patterns of changes in processes of care for children with DD over the first 3 years of a clinical network.

METHODS

Observational study involving children aged 2-59 months with DD admitted to 13 hospitals participating in the clinical network. Processes of individual patient care including agreement of assessment, diagnosis and treatment according to WHO guidelines were combined using the composite Paediatric Admission Quality of Care (PAQC) score (range 0-6).

RESULTS

Data from 7657 children were analysed and improvements in PAQC scores were observed. Predicted mean PAQC score for all the hospitals at enrolment was 59.8% (95% CI: 54.7, 64.9) but showed a wide variation (variance 10.7%, 95% CI: 5.8, 19.6). Overall mean PAQC score increased by 13.8% (95% CI: 8.7-18.9, SD between hospitals: ±8.2) in the first 12 months, with an average 0.9% (95% CI: 0.3-1.5, SD ± 1.0) increase per month and plateaued thereafter, and changes were similar in two groups of hospitals joining the network at different times.

CONCLUSION

Adherence to guidelines for children admitted with DD can be improved through participation in a clinical network but improvement is limited, not uniform for all aspects of care and contexts and occurs early. Future research should address these issues.

Association of the Paediatric Admission Quality of Care score with mortality in Kenyan hospitals: a validation study

Background

Measuring the quality of hospital admission care is essential to ensure that standards of practice are met and continuously improved to reduce morbidity and mortality associated with the illnesses most responsible for inpatient deaths. The Paediatric Admission Quality of Care (PAQC) score is a tool for measuring adherence to guidelines for children admitted with acute illnesses in a low-income setting. We aimed to explore the external and criterion-related validity of the PAQC score by investigating its association with mortality using data drawn from a diverse sample of Kenyan hospitals.

Methods

We identified children admitted to Kenyan hospitals for treatment of malaria, pneumonia, diarrhoea, or dehydration from datasets from three sources: an observational study, a clinical trial, and a national cross-sectional survey. We extracted variables describing the process of care provided to patients at admission and their eventual outcomes from these data. We applied the PAQC scoring algorithm to the data to obtain a quality-of-care score for each child. We assessed external validity of the PAQC score by its systematic replication in datasets that had not been previously used to investigate properties of the PAQC score. We assessed criterion-related validity by using hierarchical logistic regression to estimate the association between PAQC score and the outcome of mortality, adjusting for other factors thought to be predictive of the outcome or responsible for heterogeneity in quality of care.

Findings

We found 19 065 eligible admissions in the three validation datasets that covered 27 hospitals, of which 12 969 (68%) were complete cases. Greater guideline adherence, corresponding to higher PAQC scores, was associated with a reduction in odds of death across the three datasets, ranging between 9% (odds ratio 0·91, 95% CI 0·84–0·99; p=0·031) and 30% (0·70, 0·63–0·78; p<0·0001) adjusted reduction per unit increase in the PAQC score, with a pooled estimate of 17% (0·83, 0·78–0·89; p<0·0001). These findings were consistent with a multiple imputation analysis that used information from all observations in the combined dataset.

Interpretation

The PAQC score, designed as an index of the technical quality of care for the three commonest causes of admission in children, is also associated with mortality. This finding suggests that it could be a meaningful summary measure of the quality of care for common inpatient conditions and supports a link between process quality and outcome. It might have potential for application in low-income countries with similar disease profiles and in which paediatric practice recommendations are based on WHO guidelines.

Funding

The Wellcome Trust.

Improving oxygen therapy for children and neonates in secondary hospitals in Nigeria: study protocol for a stepped-wedge cluster randomised trial

BACKGROUND

Oxygen is a life-saving, essential medicine that is important for the treatment of many common childhood conditions. Improved oxygen systems can reduce childhood pneumonia mortality substantially. However, providing oxygen to children is challenging, especially in small hospitals with weak infrastructure and low human resource capacity.

METHODS/DESIGN

This trial will evaluate the implementation of improved oxygen systems at secondary-level hospitals in southwest Nigeria. The improved oxygen system includes: a standardised equipment package; training of clinical and technical staff; infrastructure support (including improved power supply); and quality improvement activities such as supportive supervision. Phase 1 will involve the introduction of pulse oximetry alone; phase 2 will involve the introduction of the full, improved oxygen system package. We have based the intervention design on a theory-based analysis of previous oxygen projects, and used quality improvement principles, evidence-based teaching methods, and behaviour-change strategies. We are using a stepped-wedge cluster randomised design with participating hospitals randomised to receive an improved oxygen system at 4-month steps (three hospitals per step). Our mixed-methods evaluation will evaluate effectiveness, impact, sustainability, process and fidelity. Our primary outcome measures are childhood pneumonia case fatality rate and inpatient neonatal mortality rate. Secondary outcome measures include a range of clinical, quality of care, technical, and health systems outcomes. The planned study duration is from 2015 to 2018.

DISCUSSION

Our study will provide quality evidence on the effectiveness of improved oxygen systems, and how to better implement and scale-up oxygen systems in resource-limited settings. Our results should have important implications for policy-makers, hospital administrators, and child health organisations in Africa and globally.

TRIAL REGISTRATION

Australian New Zealand Clinical Trials Registry: ACTRN12617000341325 . Retrospectively registered on 6 March 2017.

Some sustained improvements in pneumonia case management four and five years following implementation of paediatric hospital guidelines in Lao PDR

In 2010, WHO paediatric hospital guidelines were implemented in Lao PDR, along with training workshops and feedback audits, achieving significant improvements in pneumonia case management when assessed one-year post-intervention. The sustainability of these improvements is hereby assessed, four and five years post-intervention. Medical records of children aged 1-59 months, diagnosed with pneumonia in 2010, 2011, 2014 and 2015 from a central Lao hospital were reviewed. Information relating to clinical steps in pneumonia case management was extracted and a scoring system applied based on the documentation of each clinical step, producing a pneumonia assessment score for each case. Comparisons of clinical steps and mean assessment score across study years were performed using Pearson's chi-squared and t-tests, respectively. Of 231 pneumonia cases, the mean assessment scores in 2010, 2011, 2014 and 2015 were 57%, 96%, 69% and 69% respectively, showing a significant reduction from the immediate post-intervention period (2011) to 2015 (p < 0.01). Mean assessment score in 2014/2015 was significantly higher than in 2010 (p < 0.01). The high standards of pneumonia case management in 2011 were not observed in 2014/2015 in the absence of ongoing intervention but overall quality of care remained higher than pre-intervention levels, suggesting some degree of sustainability in the long-term.