Discussion on the validity of commonly used reliability indices in sports medicine and exercise science: a critical review with data simulations

Examiner experience moderates reliability of human lower extremity muscle ultrasound measurement - a double blinded measurement error study

Structural muscle properties are critical in health and athletic settings, with magnetic resonance imaging considered the gold standard assessment procedure under static conditions due to its reliability and objectivity. Practical limitations, including cost and accessibility, have led to the increasing use of ultrasound as an alternative for skeletal muscle morphological parameters. However, ultrasound measurements are sensitive to evaluation conditions and assessor experience, which has not been sufficiently explored, yet. Therefore, this study investigated the influence of assessor experience on the reliability of ultrasound measurements. A double-blind design was used, involving an experienced assessor (> 12,000 images for several years) and multiple inexperienced assessors (< 100 images) to collect data from 39 recreationally active participants. Measurements of muscle architecture were conducted in the leg muscles over two consecutive days, generating 1,248 ultrasound images. Relative and absolute reliability were analyzed using intraclass correlation coefficients (ICCs), standard error of measurement, minimal detectable change, mean absolute error (MAE), mean absolute percentage error (MAPE) and Bland-Altman analyses. Relative reliability was good to excellent in all measurement spots and time-points for muscle thickness (ICC = 0.76-0.98) irrespective of assessor experience, except for the inter-day comparison for the gastrocnemius lateralis by the inexperienced assessors, (ICC = 0.58). The pennation angle assessment ranged from insufficient to excellent reliability (ICC = 0.18-0.94) and experience contributed greatly to better results. The random error for the inexperienced assessors was reflected in two- to three-times higher MAEs/MAPEs and limits of agreement in the Bland-Altman analyses, respectively. The findings emphasize the importance of experience and standardization in achieving reliable ultrasound data, particularly for (a) sensitive parameters like the pennation angle and/or (b) inter-day, intra-subject comparisons.