The Inter-rater Reliability of Echocardiographic Diastolic Function Evaluation Among Emergency Physician Sonographers

Assessment of E/A ratio helps emergency clinicians in the management of patients with acute dyspnea

Acute dyspnea (AD) is one of the main reasons for admission to the Emergency Department (ED). In the last years integrated ultrasound examination (IUE) of lung, heart and inferior vena cava (IVC) has become an extension of clinical examination for a fast differential diagnosis. The aim of present study is to assess the feasibility and diagnostic accuracy of E/A ratio for diagnosing acute heart failure (aHF) in patients with acute dyspnea. We included 92 patients presenting to the ED of CTO Hospital in Naples (Italy) for AD. All patients underwent IUE of lung-heart-IVC with a portable ultrasound device. Left ventricle diastolic function was assessed using pulse wave doppler at the tips of the mitral valve and E wave velocity and E/A ratio were recorded. The FINAL diagnosis was determined by two expert reviewers: acute HF or non-acute HF (non-aHF). We used 2 × 2 contingency tables to analyze sensitivity, specificity, positive predictive and negative predictive value of ultrasound parameters for the diagnosis of AD, comparing with the FINAL diagnosis. Lung ultrasound (LUS) showed high sensitivity, good specificity and accuracy in identification of patients with aHF. However, the highest accuracy was obtained by diastolic function parameters. The E/A ratio showed the highest diagnostic performance with an AUC for aHF of 0.93. In patients presenting with AD, E/A ratio is easy to obtain in a fast ultrasound protocol and showed an excellent accuracy for diagnosis of aHF.

Cardiac POCUS in Pediatric Emergency Medicine: A Narrative Review

PURPOSE OF THIS REVIEW

The cardiac point of care ultrasound (POCUS) is among the most impactful examinations in the evaluation of an ill child. This paper will review the English-language literature on cardiac POCUS in the pediatric emergency department (PED), the adult emergency literature with relevance to pediatric emergency, and other pediatric cardiac studies outside pediatric emergency with relevance to PED detection of potentially emergent pediatric cardiac pathology.

RECENT FINDINGS

Pediatric emergency physicians can reliably detect decreased left-sided systolic function and pericardial effusion using POCUS. Case reports show that pediatric emergency physicians have detected right-sided outflow tract obstruction, aortic root dilatation, and congenital cardiac disease using POCUS. Training for pediatric cardiac POCUS competency is feasible, and cardiac POCUS does not increase the burden on cardiology resources to the PED.

SUMMARY

While cardiac pathology in children is relatively rare, pediatric cardiac POCUS can incorporate a broad curriculum beyond systolic function and the presence of pericardial fluid. Further research should assess pediatric emergency physician performance in the identification of a broader range of cardiac pathology.

POCUS for Diastolic Dysfunction: A Review of the Literature

Emergency and critical care physicians frequently encounter patients presenting with dyspnea and normal left ventricular systolic function who may benefit from early diastolic evaluation to determine acute patient management. The current American Society of Echocardiography Guidelines approach to diastolic evaluation is often impractical for point of care ultrasound (POCUS) evaluation, and few studies have evaluated the potential use of a simplified approach. This article reviews the literature on the use of a simplified diastolic evaluation to assist in determining acute patient management.

Interpretation Accuracy and Interrater Agreement of Pediatric Skin and Soft Tissue Point-of-Care Ultrasound Images among Residents and Faculty

BACKGROUND

Point-of-care ultrasound (POCUS) is commonly used to facilitate care in the emergency department. Acquired images are often reviewed by local experts for educational and quality assurance purposes. However, no published study has examined the accuracy and reliability of POCUS image interpretation by multiple reviewers.

OBJECTIVES

We studied the accuracy and interrater agreement among expert and trainee reviewers of prerecorded pediatric skin and soft tissue (SST) POCUS images.

METHODS

POCUS faculty and emergency medicine (EM) residents blindly reviewed deidentified pediatric SST POCUS images and indicated whether a drainable fluid collection was present, absent, or indeterminate. This was then compared with the gold standard based on discharge diagnoses and telephone follow-up. Images rated as indeterminate were excluded from the initial analysis. Sensitivity analysis assuming indeterminate answers were inaccurate was subsequently conducted.

RESULTS

In phase 1, 6 pediatric EM POCUS directors reviewed 168 images. The overall accuracy was 79.7% (range 66.1-86.0%). The mean Cohen's kappa was 0.58 (range 0.24-0.84). Sensitivity analysis yielded an overall accuracy of 71.3% (range 56.5-76.9%) and a Cohen's kappa of 0.43 (range 0.20-0.59). In phase 2, 6 general EM POCUS faculty and 20 EM residents reviewed 120 images. The overall accuracy among residents was 72.2% (range 51.4-84.7%) and among faculty was 83.6% (range 77.9-88.8%). Sensitivity analysis yielded an overall resident accuracy of 63.0% (range 49.5-80.7) and an overall faculty accuracy of 73.9% (range 67.0-79.8%). Fleiss' kappa was 0.322 for residents and 0.461 for faculty.

CONCLUSIONS

We found moderate accuracy and fair to good interrater agreement among POCUS faculty and EM residents reviewing pediatric SST POCUS images.

Point-of-care ultrasound in primary care: a systematic review of generalist performed point-of-care ultrasound in unselected populations

BACKGROUND

Both the interest and actual extent of use of point-of-care ultrasound, PoCUS, among general practitioners or family physicians are increasing and training is also increasingly implemented in residency programs. However, the amount of research within the field is still rather limited compared to what is seen within other specialties in which it has become more established, such as in the specialty of emergency medicine. An assumption is made that what is relevant for emergency medicine physicians and their populations is also relevant to the general practitioner, as both groups are generalists working in unselected populations. This systematic review aims to examine the extent of use and to identify clinical studies on the use of PoCUS by either general practitioners or emergency physicians on indications that are relevant for the former, both in their daily practice and in out-of-hours services.

METHODS

Systematic searches were done in PubMed/MEDLINE using terms related to general practice, emergency medicine, and ultrasound.

RESULTS

On the extent of use, we identified 19 articles, as well as 26 meta-analyses and 168 primary studies on the clinical use of PoCUS. We found variable, but generally low, use among general practitioners, while it seems to be thoroughly established in emergency medicine in North America, and increasingly also in the rest of the world. In terms of clinical studies, most were on diagnostic accuracy, and most organ systems were studied; the heart, lungs/thorax, vessels, abdominal and pelvic organs, obstetric ultrasound, the eye, soft tissue, and the musculoskeletal system. The studies found in general either high sensitivity or high specificity for the particular test studied, and in some cases high total accuracy and superiority to other established diagnostic imaging modalities. PoCUS also showed faster time to diagnosis and change in management in some studies.

CONCLUSION

Our review shows that generalists can, given a certain level of pre-test probability, safely use PoCUS in a wide range of clinical settings to aid diagnosis and better the care of their patients.

Reproducibility of the assessment of myocardial function through tissue Doppler imaging in very low birth weight infants

OBJECTIVES

This study aims to analyze the variability between two trained neonatologists when performing consecutive echocardiograms using tissue Doppler imaging (TDI) and conventional methods in very low birth weight infant (VLBWI).

METHODS

Two serial echocardiograms were performed in 30 VLBWI infants. The echocardiographic parameters analyzed were tricuspid annular plane systolic excursion (TAPSE), A', E', and S' waves, and myocardial performance index acquired by TDI (MPI-TDI) of both ventricles and shortening fraction (SF). The intra-observer and inter-observer agreements and the intra-operator agreement were analyzed using quantitative and qualitative statistical methods.

RESULTS

The intra-observer agreement was very good, TAPSE, and TDI-derived parameters had an intra-class correlation (ICC) > 0.8. TDI-derived velocities had a coefficient of variation (COV) < 11%, while MPI-TDI had a COV between 20%-28%. The inter-observer agreement was excellent. There was greater variability when analyzing intra-operator agreement, with the least variable parameter being TAPSE. According to PABAK, the variability presented moderately substantial agreement.

CONCLUSIONS

Tricuspid annular plane systolic excursion is very reproducible between observers and operators. Measurements of TDI wave velocities are more reproducible than MPI-TDI. TDI is sufficiently reproducible in the VLBWI if adequate training is performed, and guidelines are followed to obtain standardized echocardiographic images.

Decision support for assessment of left ventricular diastolic function

Echocardiographic assessment of the left ventricular diastolic function (LVDF), an integrated part of evaluation of left ventricular function is still a delicate task and is performed with substantial inter-rater variability. Therefore, we aimed to create and evaluate a guidelines-based automated decision support. An algorithm was created for a hierarchical analysis of LVDF based on variables as recommended by the latest guidelines. Age-adjusted normal ranges were pooled from previously published studies into an integrated reference table. For proof-of-concept, 20 echocardiographic examinations were analyzed offline by four experienced physicians with more than 10 years of echocardiographic experience. The first assessments were to be performed as they would be in the clinical practice. Six months later, the assessments were repeated based on the 2017 ASE/EACVI guidelines. The overall inter-rater agreement for the first clinical assessments was moderate, while the guidelines-based assessments had only fair inter-rater agreement. Both kinds of manual assessment had poor agreement with the standardized automated assessment algorithm of LVDF. In conclusion, the presented automated decision support for evaluation of diastolic LV function by Doppler echocardiography is mainly based on current guidelines involving multiple parameters in combination. Incorporating age dependency aspects in our program (available for use at https://liu.se/en/research/left-ventricular-diastolic-function-decision-support) enhances the accuracy of the evaluation and reduces variability in evaluation of LVDF. The large inter-rater variation in classification in this study also underscores the usefulness of tools to support a standardized evaluation.

Emergency physician use of tissue Doppler bedside echocardiography in detecting diastolic dysfunction: an exploratory study

Introduction

This study evaluates the agreement between emergency physician (EP) assessment of diastolic dysfunction (DD) by a simplified approach using average peak mitral excursion velocity (eʹ_A) and an independent cardiologist’s diagnosis of DD by estimating left atrial (LA) pressure using American Society of Echocardiography (ASE) guidelines.

Methods

This was a secondary analysis of 48 limited bedside echocardiograms (LBE) performed as a part of a research study of patients presenting to the Emergency Department (ED) with elevated blood pressure but without decompensated heart failure. EPs diagnosed DD based on eʹ_A < 9 cm/s alone. A blinded board-certified cardiologist reviewed LBEs to estimate LA filling pressures following ASE guidelines. An unweighted kappa measure was calculated to determine agreement between EP and cardiologist.

Results

Six LBEs were deemed indeterminate by the cardiologist and excluded from the analysis. Agreement was reached in 41 out of 48 cases (85.4%). The unweighted kappa coefficient was 0.74 (95% CI 0.57–0.92). EPs identified 18 out of 20 LBEs diagnosed with diastolic dysfunction by the cardiologist.

Conclusion

There is a good agreement between (eʹ_A) by EP and cardiologist interpretation of LBEs. Future studies should investigate this simplified approach as a one-step method of screening for LV diastolic dysfunction in the ED.

Inter-observer reliability of preoperative cardiopulmonary exercise test interpretation: a cross-sectional study

BACKGROUND

Despite the increasing importance of cardiopulmonary exercise testing (CPET) for preoperative risk assessment, the reliability of CPET interpretation is unclear. We aimed to assess inter-observer reliability of preoperative CPET.

METHODS

We conducted a prospective, multi-centre, observational study of preoperative CPET interpretation. Participants were professionals with previous experience or training in CPET, assessed by a standardized questionnaire. Each participant interpreted 100 tests using standardized software. The CPET variables of interest were oxygen consumption at the anaerobic threshold (AT) and peak oxygen consumption (VO₂ peak). Inter-observer reliability was measured using intra-class correlation coefficient (ICC) with a random effects model. Results are presented as ICC with 95% confidence interval, where ICC of 1 represents perfect agreement and ICC of 0 represents no agreement.

RESULTS

Participants included 8/28 (28.6%) clinical physiologists, 10 (35.7%) junior doctors, and 10 (35.7%) consultant doctors. The median previous experience was 140 (inter-quartile range 55-700) CPETs. After excluding the first 10 tests (acclimatization) for each participant and missing data, the primary analysis of AT and VO₂ peak included 2125 and 2414 tests, respectively. Inter-observer agreement for numerical values of AT [ICC 0.83 (0.75-0.90)] and VO₂ peak [ICC 0.88 (0.84-0.92)] was good. In a post hoc analysis, inter-observer agreement for identification of the presence of a reportable AT was excellent [ICC 0.93 (0.91-0.95)] and a reportable VO₂ peak was moderate [0.73 (0.64-0.80)].

CONCLUSIONS

Inter-observer reliability of interpretation of numerical values of two commonly used CPET variables was good (>80%). However, inter-observer agreement regarding the presence of a reportable value was less consistent.