Transforming assessment of speech in children with cleft palate via online crowdsourcing

Columbia-Suicide Severity Rating Scale (C-SSRS) Reveals High Rates of Suicidality in 602 Patients With Cleft and Craniofacial Conditions

INTRODUCTION

Little is known about rates of suicidal ideation and behavior among youth with cleft lip and/or palate (CLP) and other craniofacial conditions.

METHODS

Records of patients ages 6 and older who were administered the Columbia-Suicide Severity Rating Scale (C-SSRS) Lifetime Version during routine multidisciplinary cleft or craniofacial team visits between 2019 and 2023 were examined. Demographics information, C-SSRS data, and diagnoses were assessed with statistics including t tests, the Fisher exact test, and odds ratios.

RESULTS

A total of 1140 C-SSRS questionnaires across 602 (433 CLP and 169 craniofacial) patients with an average age of 11.2±3.7 years were included. Eighty-four (13.6%) patients endorsed lifetime suicidal ideation, 9 (1.5%) had at least one instance of suicidal behavior, 30 (5.0%) endorsed nonsuicidal self-injury, and 2 (0.3%) engaged in self-injurious behavior. Compared with CLP, those with other craniofacial conditions had similar odds of endorsing suicidal ideation and behavior ( P ≥0.05). Compared with those with isolated cleft palates, CLP had greater odds of endorsing suicidal ideation and behavior, though those differences were not significant ( P ≥0.05). Incidence of suicidality was unchanged before, during, and after the COVID-19 pandemic ( P ≥0.05). Dividing patients by sex or insurance type revealed no difference in suicidality ( P ≥0.05).

CONCLUSION

Patients with CLP and craniofacial conditions have a high incidence of suicidal ideation and behavior, though levels are similar between these groups. Suicidality in these patients was not negatively impacted by the COVID-19 pandemic. Early identification of safety risks and psychosocial challenges through regular screening can facilitate connection with appropriate clinical interventions.

Voice Pathologies and the General Population's Perception

OBJECTIVES

To examine how the general population perceives voice pathology based on subjective qualities.

STUDY DESIGN

Descriptive, cross-sectional, survey-based study.

METHODS

This is an IRB-approved Qualtrics survey on Amazon MTurk for respondents ages 18 and older. Ten subjects with voice pathologies supplied voice recordings of the Rainbow Passage to be assessed by the respondents. Respondents then assessed the voice conditions on perceived qualities of intelligence, leadership ability, and employability. One-way analysis of variance (ANOVA) and Dunnett's multiple comparison test with Sidak correction compared the mean scores of the samples (alpha = 0.05).

RESULTS

A total of 1754 responses were included in the final dataset. The female control voice was scored as more likely to be a Fortune 500 leader as well as more intelligent, friendly, attractive, and employable when compared to the female vocal fry and muscle tension dysphonia (MTD) recordings (P < 0.0001). Conversely, the male MTD was the only male pathology that received a significantly lower score on friendliness, attractiveness, Fortune 500 leader status and employability than the male control (P = 0.0102, P = 0.0007, P = 0.0338, and P = 0.0039, respectively).

CONCLUSIONS

This study demonstrates the more critical appraisal of voice pathologies of female patients compared to their male counterparts. People with voice disorders are perceived as being less successful, a disadvantage to potential leadership and career opportunities.

Crowdsourced Perceptual Ratings of Voice Quality in People With Parkinson's Disease Before and After Intensive Voice and Articulation Therapies: Secondary Outcome of a Randomized Controlled Trial

PURPOSE

Limited research has examined the suitability of crowdsourced ratings to measure treatment effects in speakers with Parkinson's disease (PD), particularly for constructs such as voice quality. This study obtained measures of reliability and validity for crowdsourced listeners' ratings of voice quality in speech samples from a published study. We also investigated whether aggregated listener ratings would replicate the original study's findings of treatment effects based on the Acoustic Voice Quality Index (AVQI) measure.

METHOD

This study reports a secondary outcome measure of a randomized controlled trial with speakers with dysarthria associated with PD, including two active comparators (Lee Silverman Voice Treatment [LSVT LOUD] and LSVT ARTIC), an inactive comparator (untreated PD), and a healthy control group. Speech samples from three time points (pretreatment, posttreatment, and 6-month follow-up) were presented in random order for rating as "typical" or "atypical" with respect to voice quality. Untrained listeners were recruited through the Amazon Mechanical Turk crowdsourcing platform until each sample had at least 25 ratings.

RESULTS

Intrarater reliability for tokens presented repeatedly was substantial (Cohen's κ = .65-.70), and interrater agreement significantly exceeded chance level. There was a significant correlation of moderate magnitude between the AVQI and the proportion of listeners classifying a given sample as "typical." Consistent with the original study, we found a significant interaction between group and time point, with the LSVT LOUD group alone showing significantly higher perceptually rated voice quality at posttreatment and follow-up relative to the pretreatment time point.

CONCLUSIONS

These results suggest that crowdsourcing can be a valid means to evaluate clinical speech samples, even for less familiar constructs such as voice quality. The findings also replicate the results of the study by Moya-Galé et al. (2022) and support their functional relevance by demonstrating that the effects of treatment measured acoustically in that study are perceptually apparent to everyday listeners.

Development and Initial Validation of a Novel Professional Aesthetic Scale for the Female Abdomen

BACKGROUND

A growing body of literature aims to describe abdominal aesthetic goals in order to tailor surgical and non-surgical treatment options to meet patient goals. We aimed to integrate lay-person perceptions into the design of a novel professional aesthetic scale for the abdomen.

METHODS

An iterative process of expert consensus was used to choose 5 domains -1) abdominal muscle lines, 2) abdominal shape, 3) scar, 4) skin, and 5) umbilicus. A survey was developed to measure global and domain-specific aesthetic preferences on 5 abdomens. This was distributed through Amazon Mechanical Turk to 340 respondents. Principal component analysis was used to integrate survey data into weights for each of the scale's sub-questions. Attending plastic surgeons then rated abdomens using the final scale, and reliability and validity were calculated.

RESULTS

The final scale included eleven sub-questions - hourglass shape, bulges, hernia, infraumbilical skin, supraumbilical skin, umbilicus shape, umbilicus medialization position, umbilicus height position, semilunar lines, central midline depression, scar - within the 5 domains. Central midline depression held the highest weight (16.1%) when correlated to global aesthetic rating, followed by semilunar lines (15.8%) and infraumbilical skin (11.8%). The final scale demonstrated strong validity (Pearson r=0.99) and was rated as easy to use by 7 attending plastic surgeons.

CONCLUSION

The final scale is the first published professional aesthetic scale for the abdomen that aims to integrate lay-person opinion. In addition, this analysis and survey data provide insights into the importance of eleven components in overall aesthetic appeal of the abdomen.

Defining Ideal External Female Genital Anatomy via Crowdsourcing Analysis

BACKGROUND

Despite existing anthropometric data in the literature regarding the variation of female external genital anatomy, the ideal aesthetic characteristics have yet to be defined.

OBJECTIVES

The authors utilized crowdsourcing to better evaluate preferred anatomic characteristics of external female genitalia.

METHODS

Fifty-six total images were digitally created by altering the proportions of the labia minora, labia majora, and clitoral hood. Images with differing ratios were presented in pairs to Amazon Mechanical Turk (Seattle, WA, USA) raters. Three different experiments were performed with each varying 2 of the 3 image characteristics to permit 2-factor modeling. The Bradley-Terry-Luce model was applied to the pairwise comparisons ratings to create a rank order for each image. Preferences for each anatomic variable were compared with chi-squared tests.

RESULTS

A total of 5000 raters participated. Experiment 1 compared differing widths of the labia majora and labia minora and determined a significant preference for larger labia majora width and mid-range labia minora width (P = 0.007). Experiment 2 compared labia minora width vs clitoral hood length and showed a statistically significant preference for wider majoras (P < 0.001) but no significant preference in clitoral hood length (P = 0.54). Experiment 3 compared clitoral hood length vs labia minora width and showed a statistically significant preference for mid-range labia minora widths (P < 0.001) but no significant preference in clitoral hood length (P = 0.78).

CONCLUSIONS

Raters preferred a labia majora to labia minora width ratio of 3:1 with minimal preference in clitoral hood length.

The Feasibility of Cross-Linguistic Speech Evaluation in the Care of International Cleft Palate Patients

ABSTRACT

Many patients with cleft palate in developing countries never receive postoperative speech assessment or therapy. The use of audiovisual recordings could improve access to post-repair speech care. The present study evaluated whether English-speaking speech-language pathologists (SLPs) could assess cleft palate patients speaking an unfamiliar language (Tamil) using recorded media. Recordings obtained from Tamil-speaking participants were rated by 1 Tamil-speaking SLP and 3 English-speaking SLPs. Ratings were analyzed for inter-rater reliability and scored for percent correct. Accuracy of the English SLPs was compared with independent t tests and Analysis of Variance. Sixteen participants (mean age 14.5 years, standard deviation [SD] 7.4 years; mean age of surgery of 2.7 years, SD 3.7 years; time since surgery: 10.8 years, SD 5.7 years) were evaluated. Across the 4 SLPs, 5 speech elements were found to have moderate agreement, and the mean kappa was 0.145 (slight agreement). Amongst the English-speaking SLPs, 10 speech elements were found to have substantial or moderate agreement, and the mean kappa was 0.333 (fair agreement). Speech measures with the highest inter-rater reliability were hypernasality and consonant production errors. The average percent correct of the English SLPs was 60.7% (SD 20.2%). English SLPs were more accurate if the participant was female, under eighteen, bilingual, or had speech therapy. The results demonstrate that English SLPs without training in a specific language (Tamil) have limited potential to assess speech elements accurately. This research could guide training interventions to augment the ability of SLPs to conduct cross-linguistic evaluations and improve international cleft care by global health teams.

Crowdsourcing as a tool in the clinical assessment of intelligibility in dysarthria: How to deal with excessive variation

PURPOSE

Independent laypersons are essential in the assessment of intelligibility in persons with dysarthria (PWD), as they reflect intelligibility limitations in the most ecologically valid way, without being influenced by familiarity with the speaker. The present work investigated online crowdsourcing as a convenient method to involve lay people as listeners, with the objective of exploring how to constrain the expected variability of crowd-based judgements to make them applicable in clinical diagnostics.

METHOD

Intelligibility was assessed using a word transcription task administered via crowdsourcing. In study 1, speech samples of 23 PWD were transcribed by 18 crowdworkers each. Four methods of aggregating the intelligibility scores of randomly sampled panels of 4 to 14 listeners were compared for accuracy, i.e. the stability of the resulting intelligibility estimates across different panels, and their validity, i.e. the degree to which they matched data obtained under controlled laboratory conditions ("gold standard"). In addition, we determined an economically acceptable number of crowdworkers per speaker which is needed to obtain accurate and valid intelligibility estimates. Study 2 examined the robustness of the chosen aggregation method against downward outliers due to spamming in a larger sample of 100 PWD.

RESULTS

In study 1, an interworker aggregation method based on negative exponential weightings of the scores as a function of their distance from the "best" listener's score (exponentially weighted mean) outperformed three other methods (median value, arithmetic mean, maximum). Under cost-benefit considerations, an optimum panel size of 9 crowd listeners per examination was determined. Study 2 demonstrated the robustness of this aggregation method against spamming crowd listeners.

CONCLUSION

Though intelligibility data collected through online crowdsourcing are noisy, accurate and valid intelligibility estimates can be obtained by appropriate aggregation of the raw data. This makes crowdsourcing a suitable method for incorporating real-world perspectives into clinical dysarthria assessment.