Promotion in Place: A Model for Competency-Based, Time-Variable Graduate Medical Education

PROBLEM

Competency-based medical education is increasingly regarded as a preferred framework for physician training, but implementation is limited. U.S. residency programs remain largely time based, with variable assessments and limited opportunities for individualization. Gaps in graduates' readiness for unsupervised care have been noted across specialties. Logistical barriers and regulatory requirements constrain movement toward competency-based, time-variable (CBTV) graduate medical education (GME), despite its theoretical benefits.

APPROACH

The authors describe a vision for CBTV-GME and an implementation model that can be applied across specialties. Termed "Promotion in Place" (PIP), the model relies on enhanced assessment, clear criteria for advancement, and flexibility to adjust individuals' responsibilities and time in training based on demonstrated competence. PIP allows a resident's graduation to be advanced or delayed accordingly. Residents deemed competent for early graduation can transition to attending physician status within their training institution and benefit from a period of "sheltered independence" until the standard graduation date. Residents who need extended time to achieve competency have graduation delayed to incorporate additional targeted education.

OUTCOMES

A proposal to pilot the PIP model of CBTV-GME received funding through the American Medical Association's "Reimagining Residency" initiative in 2019. Ten of 46 residency programs in a multihospital system expressed interest and pursued initial planning. Seven programs withdrew for reasons including program director transitions, uncertainty about resident reactions, and the COVID-19 pandemic. Three programs petitioned their specialty boards for exemptions from time-based training. One program was granted the needed exemption and launched a PIP pilot, now in year 4, demonstrating the feasibility of implementing this model. Implementation tools and templates are described.

NEXT STEPS

Larger-scale implementation with longer-term assessment is needed to evaluate the impact and generalizability of this CBTV-GME model.

Health Surveillance in a Down Syndrome Specialty Clinic: Implementation of EHR-integrations during the COVID pandemic

Objective

To address gaps in routine recommended care for children with Down syndrome, through quality improvement during the COVID-19 pandemic.

Study design

A retrospective chart review of patients with Down syndrome was conducted. Records of visits to the Massachusetts General Hospital Down Syndrome Program were assessed for adherence to five components of the 2011 American Academy of Pediatrics (AAP) Clinical Report, “Health Supervision for Children with Down Syndrome.” The impact of two major changes was analyzed using statistical process control charts: a planned intervention of integrations to the electronic health record (EHR) for routine health maintenance with age-based logic based on a diagnosis of Down syndrome, created and implemented in July 2020; and a natural disruption in care due to the COVID-19 pandemic, starting in March 2020.

Results

From December 2018 to March 2022, 433 patients with Down syndrome had 940 visits. During the COVID-19 pandemic, adherence to the audiology component decreased (58% to 45%, p <0.001); composite adherence decreased but later improved. Ophthalmology evaluation remained stable. Improvement in adherence to three components (TSH, hemoglobin, sleep study ever) in July 2020 coincided with EHR-integrations. Total adherence to the 5 AAP guideline components was higher for follow-up visits compared with new patient visits (69% and 61%, respectively; p < 0.01).

Conclusion

The COVID-19 pandemic influenced adherence to components of the AAP Health supervision for children with Down syndrome but improvements in adherence coincided with implementation of our intervention, and re-opening after the COVID-19 pandemic.

As each primary care pediatrician cares for 1–2 patients with Down syndrome (1), for many children the current care model involves a primary care physician providing health supervision for children with Down syndrome following the American Academy of Pediatrics’ (AAP) Clinical Report.(2–4)

The New USMLE Step 1 Paradigm: An Opportunity to Cultivate Diversity of Excellence

The February 2020 announcement that United States Medical Licensing Examination (USMLE) Step 1 results will be reported as pass/fail instead of numerical scores has been controversial. Step 1 scores have played a key role in residency selection, including screening for interviews. Although Step 1 scores are viewed as an objective criterion, they have been shown to disadvantage female and underrepresented minority applicants, cause student anxiety and financial burden, and affect student well-being. Furthermore, Step 1 scores incompletely predict applicants' overall residency performance. With this paradigm shift in Step 1 score reporting, residency programs will have fewer objective, standardized metrics for selection decisions, which may lead to greater emphasis on USMLE Step 2 Clinical Knowledge scores or yield unintended consequences, including shifting weight to metrics such as medical school reputation.Yet, greater breadth in residency selection metrics will better serve both applicants and programs. Some students excel in coursework, others in research or leadership. All factors should be recognized, and broader metrics should be implemented to promote and recognize diversity of excellence. Given the need for metrics for residency selection as well as for a more holistic approach to evaluating residency applicants, assessment during medical school should be revisited and made more meaningful. Another opportunity may involve use of situational judgment tests to predict professionalism and performance on other competencies. It will be important to evaluate the impact of the new Step 1 paradigm and related initiatives going forward. Residency application overload must also be addressed.

Increasing Vaginal Chlamydia Trachomatis Testing in Adolescent and Young Adults

OBJECTIVE

The Centers for Disease Control and Prevention recommend testing for Chlamydia trachomatis in sexually active female patients <25 years old using nucleic-acid amplification tests (NAAT) from a vaginal swab. Our providers were typically testing using the less sensitive urine NAATs. We aimed to increase the percentage of urogenital C trachomatis NAATs performed by using vaginal swabs in adolescent female patients ages 10 through 20 years from 1.4% to 25%.

METHODS

We implemented 3 interventions at 3 pediatric practices over 12 months including education, process standardization, and cross-training. We used statistical process control to analyze the effect of interventions on our primary outcome: the percentage of urogenital C trachomatis tests performed with a vaginal swab. Our balance measure was the total number of urogenital C trachomatis tests.

RESULTS

There were 818 urogenital C trachomatis tests performed: 289 before and 529 after the first intervention. Of urogenital C trachomatis tests in the preintervention time period, 1.4% were performed by using vaginal swabs. We surpassed our aim of 25% 6 weeks after the first intervention. We noted sustained improvement after the second intervention, with an average of 68.3% of tests performed by using vaginal swabs for the remaining postintervention period. There was no difference in the overall number of urogenital C trachomatis tests pre- and postintervention.

CONCLUSIONS

Using quality improvement methodology and implementing easily replicable interventions, we significantly and sustainably increased use of vaginal swabs. The interventions standardizing processes were associated with a higher impact than the educational intervention.

Use of an objective structured clinical exam (OSCE) for early identification of communication skills deficits in interns

INTRODUCTION

There is limited information about whether OSCE during GME orientation can identify trainee communication deficits before these become evident via clinical performance evaluations.

METHODS

Ninety-seven interns matriculating to eight residency programs in six specialties at four hospitals participated in a nine-station communication skills OSCE. Ratings were based on the "Kalamazoo, adapted" communication skills checklist. Possible association with intern performance evaluations was assessed by repeated-measures logistic regression and ROC curves were generated.

RESULTS

The mean OSCE score was 4.08 ± 0.27 with a range of 3.3-4.6. Baseline OSCE scores were associated with subsequent communication concerns recorded by faculty, based on 1591 evaluations. A 0.1-unit decrease in the OSCE communication score was associated with an 18% higher odds of being identified with a communication concern by faculty evaluation (odds ratio 1.18, 95% CI 1.01-1.36, p = 0.034). ROC curves did not demonstrate a "cut-off" score (AUC= 0.558). Non-faculty evaluators were 3-5 times more likely than faculty evaluators to identify communication deficits, based on 1900 evaluations.

CONCLUSIONS

Lower OSCE performance was associated with faculty communication concerns on performance evaluations; however, a "cut-off" score was not demonstrated that could identify trainees for potential early intervention. Multi-source evaluation also identified trainees with communication skills deficits.

A mixed methods approach to developing and evaluating oncology trainee education around minimization of adverse events and improved patient quality and safety

BACKGROUND

Adverse events are a significant quality and safety issue in the hospital setting due to their direct impact on patients. Additionally, such events are often handled by junior doctors due to their direct involvement with patients. As such, it is important for health care organizations to prioritize education and training for junior doctors on identifying adverse events and handling them when they occur. The Cancer Cup Challenge is an educational program focuses on quality improvement and adverse event awareness targeting for junior oncology doctors across three international sites.

METHODS

A mixed methodology was used to develop and evaluate the program. The Qstream spaced learning platform was used to disseminate information to participants, as it has been demonstrated to impact on both knowledge and behavior. Eight short case based scenarios with expert feedback were developed by a multidisciplinary advisory committee containing representatives from the international sites. At the conclusion of the course impact on participant knowledge was evaluated using analysis of the metrics collected by the Qstream platform. Additionally, an online survey and semi-structured interviews were used to evaluate engagement and perceived value by participants.

RESULTS

A total of 35 junior doctors registered to undertake the Qstream program, with 31 (88.57 %) successfully completing it. Analysis of the Qstream metrics revealed 76.57 % of cases were answered correctly on first attempt. The post-program survey received 17 responses, with 76.47 % indicating cases for the course were interesting and 82.35 % feeling cases were relevant. Finally, 14 participants consented to participate in semi-structured interviews about the program, with feedback towards the course being generally very positive.

CONCLUSIONS

Our study demonstrates that an online game is well accepted by junior doctors as a method to increase their quality improvement awareness. Developing effective and sustainable training for doctors is important to ensure positive patient outcomes are maintained in the hospital setting. This is particularly important for junior doctors as they are working closely with patients and learning skills and behaviors, which will influence their practice throughout their careers.

Use of spaced education to deliver a curriculum in quality, safety and value for postgraduate medical trainees: trainee satisfaction and knowledge

PURPOSE

Quality, patient safety and value are important topics for graduate medical education (GME). Spaced education delivers case-based content in a structured longitudinal experience. Use of spaced education to deliver quality and safety education in GME at an institutional level has not been previously evaluated.

OBJECTIVES

To implement a spaced education course in quality, safety and value; to assess learner satisfaction; and to describe trainee knowledge in these areas.

METHODS

We developed a case-based spaced education course addressing learning objectives related to quality, safety and value. This course was offered to residents and fellows about two-thirds into the academic year (March 2014) and new trainees during orientation (June 2014). We assessed learner satisfaction by reviewing the course completion rate and a postcourse survey, and trainee knowledge by the per cent of correct responses.

RESULTS

The course was offered to 1950 trainees. A total of 305 (15.6%) enrolled in the course; 265/305 (86.9%) answered at least one question, and 106/305 (34.8%) completed the course. Fewer participants completed the March programme compared with the orientation programme (42/177 (23.7%) vs 64/128 (50.0%), p<0.001). Completion rates differed by specialty, 80/199 (40.2%) in non-surgical specialties compared with 16/106 (24.5%) in surgical specialties (p=0.008). The proportion of questions answered correctly on the first attempt was 53.2% (95% CI 49.4% to 56.9%). Satisfaction among those completing the programme was high.

CONCLUSIONS

Spaced education can help deliver and assess learners' understanding of quality, safety and value principles. Offering a voluntary course may result in low completion. Learners were satisfied with their experience and were introduced to new concepts.

Pediatric resident education in quality improvement (QI): a national survey

OBJECTIVE

To assess pediatric residents' perceptions of their quality improvement (QI) education and training, including factors that facilitate learning QI and self-efficacy in QI activities.

METHODS

A 22-question survey questionnaire was developed with expert-identified key topics and iterative pretesting of questions. Third-year pediatric residents from 45 residency programs recruited from a random sample of 120 programs. Data were analyzed by descriptive statistics, chi-square tests, and qualitative content analysis.

RESULTS

Respondents included 331 residents for a response rate of 47%. Demographic characteristics resembled the national profile of pediatric residents. Over 70% of residents reported that their QI training was well organized and met their needs. Three quarters felt ready to use QI methods in practice. Those with QI training before residency were significantly more confident than those without prior QI training. However, fewer than half of respondents used standard QI methods such as PDSA cycles and run charts in projects. Residents identified faculty support, a structured curriculum, hands-on projects, and dedicated project time as key strengths of their QI educational experiences. A strong QI culture was also considered important, and was reported to be present in most programs sampled.

CONCLUSIONS

Overall, third-year pediatric residents reported positive QI educational experiences with strong faculty support and sufficient time for QI projects. However, a third of residents thought that the QI curricula in their programs needed improvement, and a quarter lacked self-efficacy in conducting future QI activities. Continuing curricular improvement, including faculty development, is warranted.

Impact of online education on intern behaviour around joint commission national patient safety goals: a randomised trial

PURPOSE

To compare the effectiveness of two types of online learning methodologies for improving the patient-safety behaviours mandated in the Joint Commission National Patient Safety Goals (NPSG).

METHODS

This randomised controlled trial was conducted in 2010 at Massachusetts General Hospital and Brigham and Women's Hospital (BWH) in Boston USA. Incoming interns were randomised to either receive an online Spaced Education (SE) programme consisting of cases and questions that reinforce over time, or a programme consisting of an online slide show followed by a quiz (SQ). The outcome measures included NPSG-knowledge improvement, NPSG-compliant behaviours in a simulation scenario, self-reported confidence in safety and quality, programme acceptability and programme relevance.

RESULTS

Both online learning programmes improved knowledge retention. On four out of seven survey items measuring satisfaction and self-reported confidence, the proportion of SE interns responding positively was significantly higher (p<0.05) than the fraction of SQ interns. SE interns demonstrated a mean 4.79 (36.6%) NPSG-compliant behaviours (out of 13 total), while SQ interns completed a mean 4.17 (32.0%) (p=0.09). Among those in surgical fields, SE interns demonstrated a mean 5.67 (43.6%) NPSG-compliant behaviours, while SQ interns completed a mean 2.33 (17.9%) (p=0.015). Focus group data indicates that SE was more contextually relevant than SQ, and significantly more engaging.

CONCLUSION

While both online methodologies improved knowledge surrounding the NPSG, SE was more contextually relevant to trainees and was engaging. SE impacted more significantly on both self-reported confidence and the behaviour of surgical residents in a simulated scenario.

Is residents' progress on individualized learning plans related to the type of learning goal set?

PURPOSE

To determine the types of learning goals residents select for their individualized learning plans (ILPs) and the relationship between goal type and progress toward achieving that goal.

METHOD

Pediatric and combined pediatric residents at 46 U.S. training programs completed a Web-based survey on ILPs in 2008-2009, describing their most important learning goal and the goals on which they made the most and least progress. Using iterative inductive review, responses were categorized into seven types (six corresponding with the Accreditation Council for Graduate Medical Education general competencies). Descriptive statistics and regression models were used to assess the relationship between goal type and progress made.

RESULTS

Of 1,739 eligible residents, 992 (57%) completed the survey; 668 (38%) had previously completed an ILP and described their learning goals. Residents were more likely to report medical knowledge (MK) (53.7%) and patient care (PC) (25.9%) goals as most important and less likely to report professionalism (1.5%) and systems-based practice (SBP) (1.0%) goals as most important. Compared with progress on MK goals, residents reported significantly greater progress on PC (odds ratio [OR]: 2.20; 95% confidence interval [CI]: 1.57-3.09) and practice-based learning and improvement teaching (OR: 2.99; 95% CI: 1.59-5.63) goals and less progress on SBP goals (OR: 0.16; 95% CI: 0.05-0.56).

CONCLUSIONS

Residents most commonly identified MK and PC learning goals as the most important. Residents made more progress on goals related to everyday tasks, such as PC and teaching, compared with goals less integrated in everyday training, such as SBP.

Review of quality measures of the most integrated health care settings for children and the need for improved measures: recommendations for initial core measurement set for CHIPRA

OBJECTIVE

To identify, assess, and make recommendations for inclusion of measures that assess the domain of "most integrated health care setting," with a specific focus on measures of the medical home, one particular mechanism for integrating care, to identify gaps in measurement; and to make recommendations for new measure development.

METHODS

We developed a conceptual framework for care integration and reviewed literature on measures assessing the presence and quality of the medical home to determine their validity, reliability, and feasibility as a proxy for care integration.

RESULTS

We identified 2 broad approaches to assessing the extent to which patients receive care that fulfills the aims of the medical home: 1) organizational assessment of practice systems and processes thought associated with achieving these desired aims (viz, the National Committee for Quality Assurance Physician Practice Connections-Patient Centered Medical Home measure and the Medical Home Index, and 2) direct assessment by patients/families of their experience of care in targeted dimensions. Based on concerns about the absence of reliability data and the feasibility of applying the practice audit/self-assessment approach on a population level for the purpose of state reporting, as well as the limited data linking performance on the specific measures with important child outcomes, we did not recommend any of the measures of organizational assessments of practice systems for inclusion in the core set as an indicator of care integration. In contrast, measures of the medical home based on items from the National Survey of Child Health on a population level of or the Consumer Assessment of Healthcare Providers and Systems for practice- and state-level assessment are more feasible, have known reliability and performance characteristics, and more closely reflect the aims of the medical home, including care integration.

CONCLUSIONS

Measures of health care integration as captured by the experience of care in a medical home can best be assessed for state-level performance through patient/family experience surveys. Better measures of care integration, care coordination, and integration of mental, developmental, and physical health into a comprehensive care system are high-priority topics for measure development.

Measuring patient and family experiences of health care for children

BACKGROUND

The Institute of Medicine considers patient centeredness a core dimension of quality. Several patient/family surveys exist to assess pediatric health care. The Children's Health Insurance Program Reauthorization Act mandates strengthening quality measurement for children, including for patient/family experience of care.

OBJECTIVES

The aim of this study was to determine what instruments exist for measuring patient/family experience of pediatric health care and which should be included in the core measurement set for assessing Medicaid and the Children's Health Insurance Program (CHIP) programs; to identify gaps in measurement; and to provide recommendations for measure development.

METHODS

We developed a conceptual framework for measuring patient/family experience of care. We conducted a review of national measure clearinghouses and of the literature to assess validity, reliability, and feasibility of existing measures, and how these measures address the conceptual framework.

RESULTS

We found valid and reliable instruments for measuring patient/family experience of care include the Consumer Assessment of Healthcare Providers and Systems (CAHPS®) instruments, Promoting Healthy Development Survey (PHDS), Young Adult Health Care Survey (YAHCS), and the National Research Corporation Picker Pediatric Inpatient Survey (NRC Picker). We identified the need for matching patients with providers and groups as a barrier for widespread use of the CAHPS® pediatric clinician & group instrument.

CONCLUSIONS

We recommended to the National Advisory Council for Healthcare Research and Quality Subcommittee on Children's Healthcare Quality Measures for Medicaid and Child Health Insurance Programs (SNAC) the CAHPS® Child Medicaid 4.0 and pediatric Clinician & Group Survey for inclusion in the initial recommended list of core measures for voluntary use by Medicaid and CHIP. The Clinician and Group Survey was not included in the list posted for public comment due to concerns at that time (December 2009) about feasibility. We also recommended that development of a child version of the CAHPS® behavioral and mental health survey now used in the adult population and of a pediatric hospital CAHPS® measure be considered high priorities for development in the next phase of Children's Health Insurance Program Reauthorization Act measurement activity. This phase should also explore methods to increase response rates and lower costs of obtaining consumer feedback.

Impact of a computerized template on antibiotic prescribing for acute respiratory infections in children and adolescents

BACKGROUND

Computerized decision support (CDS) can potentially improve patient safety and guideline adherence. The authors developed an acute respiratory illness interactive template (ARI-IT) within an electronic health record (EHR) to manage pediatric ARIs and assessed the impact on antibiotic prescribing.

METHODS

They randomized 12 practices either to receive the ARI-IT or to the control group. Antibiotic rates among all eligible ARI diagnoses were compared among control and intervention ARI visits, controlling for clustering by clinician.

RESULTS

There was no difference in total antibiotic prescriptions between control and intervention clinics. Use of the ARI-IT significantly reduced antibiotic prescriptions (31.7% vs 39.9%; P = .02) and use of macrolides (6.2% vs 9.5%; P = .02) among visits compared with those eligible visits where it was not used.

CONCLUSION

Use of the CDS reduced antibiotic prescribing and macrolide prescriptions among children with an ARI. Nonetheless, the low overall use resulted in an ineffective intervention.

Electronic health record decision support and quality of care for children with ADHD

OBJECTIVES

The objective of this study was to assess the effect of electronic health record (EHR) decision support on physician management and documentation of care for children with attention-deficit/hyperactivity disorder (ADHD).

METHODS

This study involved 79 general pediatricians in 12 pediatric primary care practices that use the same EHR who were caring for 412 children who were aged 5 to 18 years and had a previous diagnosis of ADHD. We conducted a cluster randomized trial of EHR-based decision support that included (1) clinician reminders to assess ADHD symptoms every 3 to 6 months and (2) an ADHD note template with structured fields for symptoms, treatment effectiveness, and adverse effects. The main outcome measures were (1) proportion of children with visits during the 6-month study period in which ADHD was assessed and (2) quality of documentation of ADHD assessment. Generalized estimating equations were used to control for the clustering by providers.

RESULTS

Children at intervention sites were more likely to have had a visit during the study period in which their ADHD was assessed. The ADHD template was used at 32% of visits at which patients were scheduled specifically for ADHD assessment, and its use was associated with improved documentation of symptoms, treatment effectiveness, and treatment adverse effects.

CONCLUSIONS

EHR-based decision support improved the likelihood that children with ADHD had visits for as well as care related to managing this condition. Better understanding of how to optimize provider use of the decision support and templates could promote additional improvements in care.

Factors associated with successful self-directed learning using individualized learning plans during pediatric residency

OBJECTIVE

The aim of this study was to determine whether resident or program characteristics are associated with effective self-directed learning of residents.

METHODS

A cross-sectional survey of pediatric and medicine/pediatric residents and program directors from a nationally representative sample of residency programs was conducted. Self-directed learning efficacy was measured by resident-reported progress on learning goals from their most recent individualized learning plan (ILP). Multilevel linear regression models were used to analyze the relationship between learner and program characteristics and self-directed learning efficacy of residents.

RESULTS

All program directors of participating programs (N = 46) completed the survey; the response rate from residents was 57% (992/1739). At the time of the survey, 78% of residents had previously written an ILP. Most residents achieved moderate self-directed learning efficacy. The most important factors associated with greater self-directed learning efficacy included using a system to track one's own progress in achieving learning goals, higher score on a propensity toward lifelong learning scale, and reporting greater confidence in self-directed learning abilities. Program characteristics, including program-level support for ILPs, had little or mixed association with resident self-directed learning efficacy.

CONCLUSIONS

The most important factors associated with effective self-directed learning were resident characteristics. Our findings imply that residency programs should invest their limited resources in curricula that help residents develop measurable goals and systems for tracking progress toward goal attainment. Since propensity toward lifelong learning was an important factor, medical schools and residency training programs should enhance their efforts to develop this characteristic in learners.

Health literacy and quality: focus on chronic illness care and patient safety

Despite a heightened focus on improving quality, recent studies have suggested that children only receive half of the indicated preventive, acute, or chronic care. Two major areas in need of improvement are chronic illness care and prevention of medical errors. Recently, health literacy has been identified as an important and potentially ameliorable factor for improving quality of care. Studies of adults have documented that lower health literacy is independently associated with poorer understanding of prescriptions and other medical information and worse chronic disease knowledge, self-management behaviors, and clinical outcomes. There is also growing evidence to suggest that health literacy is important in pediatric safety and chronic illness care. Adult studies have suggested that addressing literacy can lead to improved patient knowledge, behaviors, and outcomes. Early studies in the field of pediatrics have shown similar promise. There are significant opportunities to evaluate and demonstrate the importance of health literacy in improving pediatric quality of care. Efforts to address health literacy should be made to apply the 6 Institute of Medicine aims for quality-care that is safe, effective, patient centered, timely, efficient, and equitable. Efforts should also be made to consider the distinct nature of pediatric care and address the "4 D's" unique to child health: the developmental change of children over time; dependency on parents or adults; differential epidemiology of child health; and the different demographic patterns of children and their families.

Electronic results management in pediatric ambulatory care: qualitative assessment

BACKGROUND

Electronic results management may improve the reliability and efficiency of test results management, but few studies have investigated this topic in pediatrics.

METHODS

We conducted semi-structured, key informant interviews before and after implementation of electronic results management at 8 pediatric ambulatory care practices. We also surveyed all pediatricians at 18 practices (10 additional practices). All practices were members of Partners Healthcare and had been using an electronic health record when they were offered electronic results management. We assessed baseline processes for results management, barriers to electronic results management adoption, and the perceived impact of electronic results management on quality, efficiency, and provider satisfaction.

RESULTS

From interviews, we found a range of processes in place to manage test results, but all practices reported losing some results and no practice tracked all test results from the time of ordering to parent/patient notification. Practices that fully adopted electronic results management reported gains in efficiency, reliability, timeliness, and provider satisfaction, whereas some partial adopters reported decreased efficiency and increased risk of lost test results. Barriers to electronic results management adoption included lack of inclusion of all ordered tests in the electronic results management system, user interface design issues, and lack of sufficient pediatric customization. Survey results (response rate: 62%) indicated that pediatricians thought electronic results management improved the quality and efficiency of care, with 72% of pediatricians reporting safer care and 63% reporting more-effective care.

CONCLUSIONS

We found that pediatric practices have room for improvement in the management of test results, and electronic results management may be an effective method for improving the efficiency and safety of test results management. However, partial adoption of electronic results management may decrease efficiency and pose a threat to patient safety.

Practice variation in parental assessment of pediatric ambulatory care

OBJECTIVE

To assess parental experience of pediatric primary care at the level of physician practice and to determine what variation exists among practices.

METHODS

This cross-sectional survey assessed 7 pediatric primary care practices in eastern Massachusetts. Parents of children aged < or =12 years who received care between July 1999 and June 2000 were surveyed. Parents assessed practice performance in 6 areas of quality: access to care, patient education and information, patient/physician relationship, coordination and continuity of care, office staff courtesy and helpfulness, and specialty care experience.

RESULTS

Surveys were returned by 744 parents (response rate 50%). Practices performed best in the domains of patient education and access to care, and performed poorest in coordination/continuity and specialty care experience. Practice performance varied in the included domains, with significant interpractice variation in specialty care experience (range 61-83 on 100-point scale), coordination and continuity (range 64-84), and access to care (range 80-92). Items with significant variation included the physician/nurse being informed about specialist care, the physician/nurse knowing the parent/child as people, staff helpfulness, and the physician/nurse knowing what worried the parent about the child's health.

CONCLUSIONS

Parent reports of specific experiences of care revealed priorities for improvement in several areas, including coordination and continuity, specialty care experience, and interpersonal aspects of care. Performance differed among practices, providing an opportunity for practices to learn from each other. Survey-based measures of quality can help identify variation in performance and priorities for improving quality of care.

Quality of Health Care for Children

OBJECTIVE

To assess how parent reports about the inpatient care of their children vary according to the health status of children with and without chronic conditions.

DESIGN

We analyzed parent responses to the Picker Institute Pediatric Inpatient Survey.

SETTING

Thirty-nine hospitals between January 1, 1997, and December 31, 1999.

PARTICIPANTS

Overall, 12 562 parents of children who received inpatient care at participating hospitals. Main Outcome Measure Parent rating of overall quality of care.

RESULTS

Fifty-one percent of parents reported that their child had a chronic condition. Quality-of-care ratings varied according to health status and the presence of chronic conditions. Parents of children in the worst (fair or poor) health without chronic conditions reported lower quality of care (P < .001) and more care problems (P < .001) than did those with chronic conditions. Parents of children in the best (excellent, very good, or good) health tended to rate care highly, whether or not their children had chronic conditions. In a multivariable model, the decrement in perceived quality of care associated with poorer health was greater for those without than for those with chronic conditions (P < .001).

CONCLUSIONS

Although children in poor health are at risk for experiencing a lower quality of health care, parents of such children who have chronic conditions report fewer care-related problems. This may be owing to the more frequent health care interactions and better continuity of care for children with chronic conditions.

Proposal of an Algorithm for Diagnosis and Treatment of Neonatal Seizures in Developing Countries

Seizures in the neonatal period are common. They can be caused by a variety of conditions, ranging from benign, self-limited illnesses to severe, life-threatening disorders. They are often the first sign of neurologic dysfunction in neonates, and may be used as one factor in considering long-term prognosis. An important mission of the International League Against Epilepsy (ILAE) is to improve the care of patients with epilepsy. Most recently, as part of the Global Campaign against Epilepsy, ILAE, in conjunction with the World Health Organization (WHO), established a new initiative to create clinical guidelines and diagnostic and management algorithms for the care of patients with seizures that can be applied worldwide, including in developing countries with limited or varied medical resources. Created by an international panel of experts in seizure management and guideline development, this document proposes guidelines for the diagnosis and management of the most common and important conditions that cause seizures in the neonatal period. The publication of these clinical pathways for neonatal seizures will be followed by a period of field testing and comment by WHO clinicians and officials before finalization.

Quality measures for children's health care

BACKGROUND

The ability to measure and improve the quality of children's health care is of national importance. Despite the existence of numerous health care quality measures, the collective ability of measures to assess children's health care quality is unclear. A review of existing health care quality measures for children is timely for both assessing the current state of quality measures for children and identifying areas requiring additional research and development.

OBJECTIVES

To identify and collect current health care quality measures for child health and then to systematically categorize and classify measures and identify gaps in child health care quality measures requiring additional development.

DESIGN/METHODS

We first identified child health care quality instruments with assistance from staff at the Agency for Healthcare Research and Quality, experts in the field, the Computerized Needs-oriented Quality Measurement Evaluation System, the Child and Adolescent Health Measurement Initiative, and a medical literature review. From these instruments, we then selected clinical performance measures applicable to children (aged 0-18 years). We categorized the individual measures into the Institute of Medicine's framework for the National Health Care Quality Report. The framework includes health care quality domains (patient safety, effectiveness, patient-centeredness, and timeliness) and patient-perspective domains (staying healthy, getting better, living with illness, and end-of-life care). We then determined the balance of the measures (how well they assess care for all children versus children with special health care needs) and their comprehensiveness (how well the measures apply to the developmental range of children). Finally, we analyzed the ability of the measures to assess equity in care.

RESULTS

We identified 19 measure sets, and 396 individual measures were used to assess children's health care quality. The distribution of measures in the health care quality domains was: safety, 14.4%; effectiveness, 59.1%; patient-centeredness, 32.1%; and timeliness, 33.3%. The distribution of measures in the patient-perspective domains was: staying healthy, 24%; getting better, 40.2%; living with illness, 17.4%; end of life, 0%; and multidimensional, 23.5% (measures were multidimensional if they applied to >1 domain). Most of the measures were meant for use in the general pediatric population (81.1%), with a significant proportion designed for children with special health care needs (18.9%). The majority (>or=79%) of the measures could be applied to children across all age groups. However, there were relatively few measures designed specifically for each developmental stage. Regarding the use of measures to study equity in health care, 6 of the measure sets have been used in previous studies of equity. All the survey measure sets contain items that identify patients at risk for poor outcomes, and 4 are available in languages other than English. However, only 1 survey (Consumer Assessment of Health Plans) has undergone studies of cross-cultural validation. Among the measure sets based on administrative data, 3 included infant mortality, a well-known measure of health disparity.

CONCLUSIONS

There are several instruments designed to measure health care quality for children. Despite this, we found relatively few measures for assessing patient safety and living with illness and none for end-of-life care. Few measures are designed for specific age categories among children. Although equity is an overarching concern in health care quality, the application of current measures to assess disparities has been limited. These areas need additional research and development for a more complete assessment of health care quality for children.

Does a clinical pathway improve the quality of care for sickle cell anemia?

BACKGROUND

Clinical pathways are often implemented to improve care, yet their effect on quality of care and outcomes is often not evaluated. The Johns Hopkins Children's Center instituted a clinical pathway in early 1996 to improve the care for pediatric sickle cell vaso-occlusive crisis (VOC) and used a retrospective before-after study to describe how quality of care and outcomes changed after introduction of the pathway.

RESULTS

Physicians used the pathway in 43% of eligible admissions, with use decreasing over time. Patients on the pathway were more likely to receive each of its required elements than those not on the pathway (odds ratios [OR] 1.15-2.49). After pathway implementation, even patients not on the pathway were more likely to receive incentive spirometry than those admitted before pathway availability (OR 1.40). Pathway use was associated with longer length of stay (LOS) and time to oral pain medication, while readmission rates did not change.

DISCUSSION

Use of a clinical pathway improved quality of care by increasing compliance with specific care elements, with mixed results on outcomes. Pathways may improve care for all patients, including nonpathway-treated patients, by influencing underlying practice patterns. Quality improvement committees must regularly monitor outcomes after pathway implementation to evaluate the need for pathway reinforcement and refinement.

Are hospital characteristics associated with parental views of pediatric inpatient care quality?

OBJECTIVE

Patient assessments of care are increasingly being considered an important dimension of quality of care. Few studies have examined the types and extent of problems identified by parents in the care of hospitalized children and whether hospital characteristics are associated with some of these problems. The objective of this study was to describe the quality of pediatric inpatient care as perceived by parents of hospitalized children and test whether hospital characteristics (academic status, market competition, freestanding children's hospital) are associated with variations in quality.

METHODS

We performed a cross-sectional analysis of surveys from 6030 parents of children who were discharged for a medical condition from 38 hospitals that used the Picker Institute's Pediatric Inpatient Survey. The Pediatric Inpatient Survey measures 7 dimensions of inpatient care quality: partnership, coordination, information to parent, information to child, physical comfort, confidence and trust, and continuity and transition. Our main outcome measures included an overall quality of care rating (1 = poor, 5 = excellent), as well as overall and dimension-specific problem scores (0 = no problems, 100 = problems with 100% of processes asked about in the survey). We used Pearson correlation to determine the strength of association between the overall quality of care rating and dimension problem scores. We tested for associations between hospital characteristics and problem scores using linear regression models, controlling for patient health status and other socioeconomic status variables.

RESULTS

Parents on average rated their child's care as very good (mean: 4.2) but reported problems with 27% of the survey's hospital process measures. Information to the child (33%) and coordination of care (30%) had the highest problem rates. Parent communication problems correlated most strongly with overall quality of care ratings (r = -0.49). Parents of children who were hospitalized at academic health centers (AHCs) reported 4% more problems overall (29.8% vs 25.5%) and almost 9% more problems with coordination of care (34.1% vs 25.6%) compared with those at non-AHCs. Parents in more competitive markets reported almost 3% more problems than those in the less competitive ones (28.9% vs 26.3%). The freestanding children's hospital classification was not associated with overall problem scores. We found wide variation in problem scores by hospital, even among AHCs. Hospital and patient characteristics explained only 6% of the variance in problem scores.

CONCLUSIONS

Despite high subjective ratings of quality of care, measures of specific processes of care reveal significant variations among hospitals and identify areas with opportunities for improvement. Improving the quality of communication with the parent of a hospitalized child may have the most positive impact on a hospital's overall quality of care rating. AHCs and hospitals in more competitive markets may be more prone to problems. With wide variation in parental perceptions of hospital quality of care, a systems analysis of individual hospitals may provide strategies for hospitals to deliver higher quality care.