Inter-rater evaluation of a clinical scoring system in children with asthma

Derivation and Validation of an Objective Effort of Breathing Score in Critically Ill Children

OBJECTIVES

To derive and validate a score that correlates with an objective measurement of a child's effort of breathing.

DESIGN

Secondary analysis of a previously conducted observational study.

SETTING

The pediatric and cardiothoracic ICUs of a quaternary-care children's hospital.

PATIENTS

Patients more than 37 weeks gestational age to age 18 years who were undergoing extubation.

INTERVENTIONS

Effort of breathing was measured in patients following extubation using esophageal manometry to calculate pressure rate product. Simultaneously, members of a multidisciplinary team (nurse, physician, and respiratory therapist) assessed respiratory function using a previously validated tool. Elements of the tool that were significantly associated with pressure rate product in univariate analysis were identified and included in a multivariate model. An Effort of Breathing score was derived from the results of the model using data from half of the subjects (derivation cohort) and then validated using data from the remaining subjects (validation cohort) by calculating the area under the receiver operator characteristic curve for pressure rate product greater than 90th percentile and for the need for reintubation.

MEASUREMENTS AND MAIN RESULTS

Among 409 subjects, the median age was 5 months, and nearly half were cardiac surgery patients (49.1%). Retractions, stridor, and pulsus paradoxus were included in the Simple Score. Area under the receiver operator characteristic curve for pressure rate product greater than 90th percentile was 0.8359 (95% CI, 0.7996-0.8722) in the derivation cohort and 0.7930 (0.7524-0.8337) in the validation cohort. Area under the receiver operator characteristic curve for reintubation was 0.7280 (0.6807-0.7752) when all scores were analyzed individually and was 0.7548 (0.6644-0.8452) if scores from three clinicians from different disciplines were summated. Results were similar regardless of provider discipline or training.

CONCLUSIONS

A scoring system was derived and validated, performed acceptably to predict increased effort of breathing or need for advanced respiratory support and may function best when used by a team.

Experience of parents of children with autism on YouTube: are there educationally useful videos?

The aims of this study were to determine the following: first, are there educationally useful videos of parents of children with autism sharing their experiences? Second, do any of the data related to videos help in identifying useful videos? And third, what do posted comments tell us? YouTube was searched for videos of parents sharing their experiences. The following parameters were collected: title, creator, URL, duration, number of viewers, likes, dislikes, comments, days on YouTube, and country. Based on agreed-upon criteria, videos were divided independently into educationally useful and non-useful categories. A critical thematic analysis of comments was conducted. A total of 180 videos were finally identified, of which 106 (59%) provided useful information, scoring 15.3 ± 0.7 (mean ± SD); 74 (41%) were determined to be not educationally useful, scoring 8.6 ± 2.1. The differences in scores were significant (p < 0.001), but there were no significant differences between the useful and non-useful groups in terms of video parameters. No correlation was found between scores and any of the videos' parameters. In conclusion, there are videos that can be used as educational resources. The videos' parameters did not differentiate between useful and non useful. Useful videos were mostly created by professional societies and by parents. The study reflects the emerging role of YouTube in sharing experiences.

Clinical respiratory scales: which one should we use?

INTRODUCTION

There are countless clinical respiratory scales for acute dyspnoea. Most healthcare professionals choose one based on previous personal experience or following local practice, unaware of the implications of their choice. The lack of critical comparisons between those different tools has been a widespread problem that only recently has begun to be addressed via score validation studies. Here we try to assess and compare the quality criteria of measurement properties of acute dyspnoea scores. Areas covered: A literature review was conducted by searching the PubMed database. Forty-five documents were deemed eligible as they reported the use or building of clinical scales, using at least two parameters, and applied these to an acute episode of respiratory dyspnoea. Our primary focus was the description of the validity, reliability and utility of 41 suitable scoring instruments. Differences in sample selection, study design, rater profiles and potential methodological shortcomings were also addressed. Expert commentary: All acute dyspnoea scores lack complete validation. In particular, the areas of measurement error and interpretability have not been addressed correctly by any of the tools reviewed. Frequent modification of pre-existing scores (in items composition and/or name), differences in study design and discrepancies in reviewed sources also hinder the search for an adequate tool.

Reliability of Examination Findings in Suspected Community-Acquired Pneumonia

BACKGROUND

The authors of national guidelines emphasize the use of history and examination findings to diagnose community-acquired pneumonia (CAP) in outpatient children. Little is known about the interrater reliability of the physical examination in children with suspected CAP.

METHODS

This was a prospective cohort study of children with suspected CAP presenting to a pediatric emergency department from July 2013 to May 2016. Children aged 3 months to 18 years with lower respiratory signs or symptoms who received a chest radiograph were included. We excluded children hospitalized ≤14 days before the study visit and those with a chronic medical condition or aspiration. Two clinicians performed independent examinations and completed identical forms reporting examination findings. Interrater reliability for each finding was reported by using Fleiss' kappa (κ) for categorical variables and intraclass correlation coefficient (ICC) for continuous variables.

RESULTS

No examination finding had substantial agreement (κ/ICC > 0.8). Two findings (retractions, wheezing) had moderate to substantial agreement (κ/ICC = 0.6-0.8). Nine findings (abdominal pain, pleuritic pain, nasal flaring, skin color, overall impression, cool extremities, tachypnea, respiratory rate, and crackles/rales) had fair to moderate agreement (κ/ICC = 0.4-0.6). Eight findings (capillary refill time, cough, rhonchi, head bobbing, behavior, grunting, general appearance, and decreased breath sounds) had poor to fair reliability (κ/ICC = 0-0.4). Only 3 examination findings had acceptable agreement, with the lower 95% confidence limit >0.4: wheezing, retractions, and respiratory rate.

CONCLUSIONS

In this study, we found fair to moderate reliability of many findings used to diagnose CAP. Only 3 findings had acceptable levels of reliability. These findings must be considered in the clinical management and research of pediatric CAP.

Telemedicine: A Reliable Tool to Assess the Severity of Respiratory Distress in Children

BACKGROUND AND OBJECTIVES

Remote assessment of respiratory distress using telemedicine enabled audio-video conferencing (TM) is of value for medical decision-making. Our goal was to evaluate the interobserver reliability (IOR) of TM compared with face-to-face (FTF) assessment of respiratory distress in children.

METHODS

A prospective, cohort study was performed in pediatric emergency department from July 2012 to February 2013. Children (aged 0-18 years) who presented with signs of respiratory distress were included in the study. The respiratory score is a 4-item, 12-point scale (respiratory rate [1-3], retractions [0-3], dyspnea [0-3], and wheezing [0-3]) that assesses the severity of a child's respiratory distress. Each child was evaluated by a pair of observers from a pool of 25 observers. The first observer evaluated the patient FTF, and the second observer simultaneously and independently evaluated remotely via TM. The overall respiratory distress severity is based on the respiratory scale and reported as nonsevere (≤8) and severe (≥9) respiratory distress. The IOR reliability between FTF and TM assessment was measured using a 2-way mixed model, absolute agreement and average measure intraclass correlation coefficient (ICC).

RESULTS

Forty-eight patients and 135 paired observations were recorded. IOR between the FTF and TM groups for total respiratory score had an ICC of 0.95 (confidence interval 0.93-0.96) and for subscores, the ICC range was as follows: respiratory rate = 0.92, retractions = 0.85, dyspnea = 0.94, and wheezing = 0.77.

CONCLUSIONS

TM is a reliable tool to assess the severity of respiratory distress in children.

The validity and reliability of the clinical assessment of increased work of breathing in acutely ill patients

BACKGROUND

Mechanical ventilation is frequently indicated to reduce the work of breathing. Because it cannot be measured easily at the bedside, physicians rely on surrogate measurements such as patient appearance of distress and increased breathing effort.

OBJECTIVE

We determined the validity and reliability of subjectively rating the appearance of respiratory distress and the reliability of 11 signs of increased breathing effort.

SUBJECTS

The study included consecutive, acutely ill patients requiring various levels of respiratory support.

METHODS

Blinded to each other's observations, a fellow and a critical care consultant rated the severity of distress (absent, slight, moderate, severe) after observing subjects for 10 seconds and then determined the presence of the signs of increased breathing effort.

RESULTS

A total of 149 paired examinations occurred 6±6 minutes apart. The rating of respiratory distress correlated with oxygenation, respiratory rate, and 9 signs of increased work of breathing. It had the highest intraclass correlation coefficient (0.69; 95% confidence interval, 0.59-0.78). Rating distress as moderate to severe had a sensitivity of 70%, specificity of 92%, and positive likelihood ratio of 8 for the presence of 3 or more of hypoxia, tachypnea, and any sign of increased breathing effort. Agreement was moderate (κ = 0.53-0.47) for rating of distress, nasal flaring, scalene contraction, gasping, and abdominal muscle contraction, and fair (κ = 0.36-0.23) for sternomastoid contraction, tracheal tug, and thoracoabdominal paradox.

CONCLUSION

Assessing the increased work of breathing by rating the severity of respiratory distress based on subject appearance is a valid and moderately reliable sign that predicts the presence of serious respiratory dysfunction. The reliability of the individual signs of increased breathing effort is moderate at best.

Large observer variation of clinical assessment of dyspnoeic wheezing children

BACKGROUND

In children with acute dyspnoea, the assessment of severity of dyspnoea and response to treatment is often performed by different professionals, implying that knowledge of the interobserver variation of this clinical assessment is important.

OBJECTIVE

To determine intraobserver and interobserver variation in clinical assessment of children with dyspnoea.

METHODS

From September 2009 to September 2010, we recorded a convenience sample of 27 acutely wheezing children (aged 3 months-7 years) in the emergency department of a general teaching hospital in the Netherlands, on video before and after treatment with inhaled bronchodilators. These video recordings were independently assessed by nine observers scoring wheeze, prolonged expiratory phase, retractions, nasal flaring and a general assessment of dyspnoea on a Likert scale (0-10). Assessment was repeated after 2 weeks to evaluate intraobserver variation.

RESULTS

We analysed 972 observations. Intraobserver reliability was the highest for supraclavicular retractions (κ 0.84) and moderate-to-substantial for other items (κ 0.49-0.65). Interobserver reliability was considerably worse, with κ<0.46 for all items. The smallest detectable change of the dyspnoea score (>3 points) was larger than the minimal important change (<1 point), meaning that in 69% of observations a clinically important change after treatment cannot be distinguished from measurement error.

CONCLUSIONS

Intraobserver variation is modest, and interobserver variation is large for most clinical findings in children with dyspnoea. The measurement error induced by this variation is too large to distinguish potentially clinically relevant changes in dyspnoea after treatment in two-thirds of observations. The poor interobserver reliability of clinical dyspnoea assessment in children limits its usefulness in clinical practice and research, and highlights the need to use more objective measurements in these patients.

Interobserver reliability of attending physicians and bedside nurses when using an inpatient paediatric respiratory score

AIMS AND OBJECTIVES

This study aimed to determine the interobserver reliability between bedside nurses and attending physicians for a paediatric respiratory score as part of an asthma Integrated Care Pathway implementation.

BACKGROUND

An Integrated Care Pathway is one approach to improving quality of care for children hospitalised with asthma. Prior to implementation of the integrated care pathway, it was necessary to train nursing staff on the use of a respiratory assessment tool and to evaluate the interobserver reliability use of this tool.

DESIGN

Prospective study using a convenience sample of children hospitalised for a respiratory illness in an academic medical centre.

METHODS

The respiratory assessment used was the Paediatric Asthma Score. Bedside nurse-attending physician (27 different RNs and three attending paediatric hospitalists) pairs performed 71 simultaneous patient assessments on 20 patients. Intraclass correlation coefficient and kappa statistics were used to assess interobserver reliability.

RESULTS

The overall intraclass correlation coefficient was nearly perfect where κ = 0·95, 95% CI (0·92, 0·97) and overall kappa for reliability based on clinically relevant score breakpoints was also high with κ = 0·82, 95% CI (0·75, 0·90). The majority of subgroup analyses revealed substantial to almost perfect agreement across a variety of diagnoses, age ranges, and individual score components.

CONCLUSIONS

Bedside nurses, with support and training from attending physicians, can perform respiratory assessments that agree almost perfectly with those of attending physicians.

RELEVANCE TO CLINICAL PRACTICE

The use of an Integrated Care Pathway allows for optimal interprofessional collaboration between bedside nurses and attending physicians.

Systematic review: insufficient validation of clinical scores for the assessment of acute dyspnoea in wheezing children

BACKGROUND

A reliable, valid, and easy-to-use assessment of the degree of wheeze-associated dyspnoea is important to provide individualised treatment for children with acute asthma, wheeze or bronchiolitis.

OBJECTIVE

To assess validity, reliability, and utility of all available paediatric dyspnoea scores.

METHODS

Systematic review. We searched Pubmed, Cochrane library, National Guideline Clearinghouse, Embase and Cinahl for eligible studies. We included studies describing the development or use of a score, assessing two or more clinical symptoms and signs, for the assessment of severity of dyspnoea in an acute episode of acute asthma, wheeze or bronchiolitis in children aged 0-18 years. We assessed validity, reliability and utility of the retrieved dyspnoea scores using 15 quality criteria.

RESULTS

We selected 60 articles describing 36 dyspnoea scores. Fourteen scores were judged unsuitable for clinical use, because of insufficient face validity, use of items unsuitable for children, difficult scoring system or because complex auscultative skills are needed, leaving 22 possibly useful scores. The median number of quality criteria that could be assessed was 7 (range 6-11). The median number of positively rated quality criteria was 3 (range 1-5). Although most scores were easy to use, important deficits were noted in all scores across the three methodological quality domains, in particular relating to reliability and responsiveness.

CONCLUSION

None of the many dyspnoea scores has been sufficiently validated to allow for clinically meaningful use in children with acute dyspnoea or wheeze. Proper validation of existing scores is warranted to allow paediatric professionals to make a well balanced decision on the use of the dyspnoea score most suitable for their specific purpose.

Asthma vital signs at triage: home or admission (ASTHmA)

OBJECTIVES

Commonly used acute asthma scoring systems assess severity of symptoms, whereas other clinical models aim to predict hospitalization; all rely on a measure of response to treatment and use the same criteria across age ranges. This may not reflect a child's changing physiology and response to illness as he or she grows older.This study aimed to find age-specific objective predictors of hospitalization readily known at triage. The goal is to identify rapidly those who will likely need admission regardless of treatment administered or response to aggressive treatment in the emergency department (ED).

METHODS

Children between 1 and 18 years of age with a final primary ED International Classification of Diseases, Ninth Revision, diagnosis of asthma or asthma-related spectrum of disease were studied using data from the National Hospital Ambulatory Medical Care Survey. The primary outcome was hospital admission (observation unit, ward, monitored, or pediatric intensive care unit).Triage vital signs, mode of arrival, recent visits, emergency severity index score, as well as demographic and socioeconomic factors were incorporated into age-specific forward-selection multiple logistic regression models.

RESULTS

In 2,454,983 ED visits for asthma or reactive airway disease among children 1 to 18 years of age, patterns of vital sign predictors for admission varied by age group. Across all ages, diastolic hypotension at triage was an early, consistent, independent predictor of admission, especially in 1- to 3-year-olds (odds ratio, 6.27; 95% confidence interval, 6.01-6.54) and 3- to 6-year-olds (odds ratio, 17.95; 95% confidence interval, 16.80-19.17).

CONCLUSIONS

Age-specific assessment is important in the evaluation of acute asthma or reactive airway exacerbation. Diastolic hypotension may serve as an early warning indicator of severity of disease and need for hospitalization. Variability by age group in vital sign predictor for admission calls for further development or refinement of age-specific asthma assessment tools.

Effect of a clinical pathway on the hospitalisation rates of children with asthma: a prospective study

AIM

To determine the effect of implementing a clinical pathway, using evidence-based clinical practice guidelines, for the emergency care of children and adolescents with asthma.

METHODS

A prospective, before-after, controlled trial was conducted, which included patients aged 1-18 years who had acute exacerbations of asthma treated in a tertiary care paediatric emergency department. Data were collected for identical 2-month seasonal periods before and after implementation of the clinical pathway to determine hospitalisation rate and other outcomes. For 2 weeks after emergency visits, the rate at which patients returned to emergency care for worsening asthma was evaluated. A multidisciplinary panel, using national guidelines and a systematic review, developed the pathway.

RESULTS

267 patients were studied. The rate of hospitalisation was significantly lower in the post-implementation group (10/74; 13.5%) than in the pre-implementation control group (53/193; 27.5%; p = 0.02; number needed to treat 7.1). All reduction in hospitalisation occurred in children with moderate to severe asthma exacerbation. After implementation of the clinical pathway, the rate of administration of oral corticosteroids to patients with moderate or severe exacerbations increased from 71% to 92% (p = 0.01), and significantly more patients received beta2-agonists in the first hour (p = 0.02). No significant change in relapse to acute care occurred within 2 weeks (p = 0.19).

CONCLUSIONS

An evidence-based clinical pathway for children and adolescents with moderate to severe exacerbations of acute asthma markedly decreases their rate of hospitalisation without increased return to emergency care.

Current management of status asthmaticus in the pediatric ICU

Status asthmaticus (SA) in the pediatric ICU (PICU) can progress to a life-threatening emergency. The goal of management is to improve hypoxemia, improve bronchoconstriction, and decrease airway edema through the administration of continuous nebulized beta2 adrenergic agonist with intermittent anticholinergics, corticosteroids, and oxygen. Adjunctive therapies, such as magnesium, methylxanthines, intravenous beta-agonists, heliox, and noninvasive ventilation should be considered in the child who fails to respond to initial therapies. The restoration of adequate pulmonary functions, resolution of airway obstruction, and avoidance of mechanical ventilation should guide management. This article reviews the pathophysiology, assessment, and management of the child who has SA in the PICU to provide the critical care nurse with current information to facilitate optimal care.

Harnessing the potential of the internet to promote chronic illness self-management: diabetes as an example of how well we are doing

OBJECTIVES

Given the potential for the Internet to be used as a dynamic, interactive medium for providing information, changing attitudes and behaviour and enhancing social support, it is important to consider whether what is currently available online for chronic illness self-management adequately harnesses this potential. The objective of this paper was to review the content of diabetes self-management websites and to identify strengths and limitations of online diabetes self-management.

METHODS

We reviewed and coded features of 87 publicly available diabetes websites hosted by governmental, health plan, commercial, pharmaceutical, and not-for-profit organizations. We assessed whether each website was using online opportunities in the areas of interactivity, theory-based interventions, social support, and evidence-based care.

RESULTS

The majority of sites provided information, essentially using an electronic newspaper or pamphlet format. Few sites offered interactive assessments, social support or problem-solving assistance, although there were some significant differences in these characteristics across the types of site.

DISCUSSION

Current diabetes websites fall short of their potential to help consumers. Suggestions are made for ways to improve the helpfulness and interactivity of these resources.

What are patients seeking when they turn to the Internet? Qualitative content analysis of questions asked by visitors to an orthopaedics Web site

Background

More people than ever are turning to the Internet for health-related information, and recent studies indicate that the information patients find online directly affects the decisions they make about their health care. Little is known about the information needs or actual search behavior of people who use the Internet for health information.

Objective

This study analyzes what people search for when they use a health-education Web site offering information about arthritis, orthopaedics, and sports-medicine topics. Additionally, it determines who is performing these searches: is it patients, friends or relatives of patients, or neither? Finally, it examines the similarities and differences among questions submitted by Web site visitors from different countries.

Methods

Content analysis was performed on 793 free-text search queries submitted to a patient-education Web site owned and operated by the Department of Orthopaedics and Sports Medicine at the University of Washington Medical Center. The 793-query data set was coded into 3 schemes: (1) the purpose of the query, (2) the topic of the query, and (3) the relationship between the asker of the query and the patient. We determined the country from which each query was submitted by analyzing the Internet Protocol addresses associated with the queries.

Results

The 5 most frequent reasons visitors searched the Web site were to seek: (1) information about a condition, (2) information about treatment, (3) information about symptoms, (4) advice about symptoms, and (5) advice about treatment. We were able to determine the relationship between the person submitting the query and the patient in question for 178 queries. Of these, the asker was the patient in 140 cases, and the asker was a friend or relative of the patient in 38 cases. The queries were submitted from 34 nations, with most coming from the United States, Australia, the United Kingdom, and Canada. When comparing questions submitted from the United States versus those from all other countries, the 3 most frequent types of questions were the same for both groups (and were the top 3 question types listed above).

Conclusions

These results provide the University of Washington Department of Orthopaedics and Sports Medicine, as well as other organizations that provide health-information Web sites, with data about what people around the world are seeking when they turn to the Internet for health information. If Web site managers can adapt their health-information Web sites in response to these findings, patients may be able to find and use Internet-based health information more successfully, enabling them to participate more actively in their health care.