Inter-Rater Agreement for the Milestones and Barriers Assessments of the Verbal Behavior Milestones Assessment and Placement Program (VB-MAPP)

The Effects of Discrete Trial and Natural Environment Teaching on Adaptive Behavior in Toddlers With Autism Spectrum Disorder

The literature has yet to review the differential effects of Natural Environment Teaching (NET) and Discrete Trial Teaching (DTT) on adaptive skills. A sample of 142 children diagnosed with ASD between the ages of 16 and 35 months received either DTT, NET, or both interventions (NET+ DTT). The Bayley Scales of Infant and Toddler Development (BSID) Adaptive Subscale and the Verbal Behavior Milestones Assessment and Placement Program (VB-MAPP) Barriers Assessment were used as baseline and posttest measures. Children who received NET and NET+DTT conditions showed significant improvements compared to the DTT condition indicating that the addition of NET leads to increased adaptive skills and decreased barrier behaviors in participants. DTT may also play a necessary foundational role for children with more significant delays. These results provide support for the use of a combination of teaching strategies in community-based early intervention and refine protocols for teaching adaptive skills to toddlers with ASD.

Development and Preliminary Validation of the Patient Outcome Planning Calculator (POP-C): A Tool for Determining Treatment Dosage in Applied Behavior Analysis

Board certified behavior analysts (BCBA) are responsible for determining the medically necessary treatment dosage for patients (i.e., the number of hours of therapy a patient should receive per week to optimize progress) during applied behavior analysis (ABA) therapy. However, because there is currently no standard method for making these determinations, BCBAs must rely on their own clinical judgment. Given that clinical judgment may be underdeveloped in some BCBAs, particularly those who are newly certified, more formal strategies are needed to guide decision making as it relates to medical necessity and treatment dosage. In this article we describe the development of the Patient Outcome Planning Calculator (POP-C), a standardized decision-making tool designed to assist novice practitioners in determining the medically necessary ABA treatment intensity and appropriate treatment setting for individuals with autism spectrum disorder (ASD). We present preliminary reliability data as well as construct validity data indicating statistically significant correlations between the POP-C and several norm-referenced and criterion-referenced assessments commonly used to estimate skill level and the corresponding degree of support needed within the ASD population to inform the ABA treatment model and goals.

Supplementary Information

The online version contains supplementary material available at 10.1007/s40617-023-00861-6.

Validity and Reliability Evidence for Assessments Based in Applied Behavior Analysis: A Systematic Review

The current article presents the findings from a systematic review of the available reliability and validity evidence supporting the use of criterion-referenced assessments based on the applied behavior analysis framework. We identified 46 studies that reported reliability and/or validity evidence for six assessments, 37 of which presented reliability evidence and 43 presented validity evidence. Additionally, we extracted and summarized information related to participant characteristics (e.g., age, sex, diagnosis), geographic location, and research setting (e.g., residential facility, home). Overall, we found conflicting support for the use of the assessments. When coupled with the reported usage by behavior analysis professionals, our findings suggest a misalignment between the reportedly used assessments and the number of published studies providing validity and/or reliability evidence. We found inconsistent use of measurement-related vocabulary and that many studies could have been strengthened by conducting different statistical analyses. We provide a summary of studies, findings, and offer recommendations for clinical practice and future measurement research.

Content Validity Evidence for the Verbal Behavior Milestones Assessment and Placement Program

The purpose of this study is to provide content validity evidence for the Verbal Behavior Milestones Assessment and Placement Program (VB-MAPP). A national panel of 13 experts provided an evaluation of the domain relevance, age appropriateness, method of measurement appropriateness, and domain representation across the three levels of the Milestones Assessment, Early Echoic Skills Assessment (EESA), and Barriers Assessment. Overall, the content validity evidence for the VB-MAPP Milestones, EESA, and Barriers Assessment was moderate to strong across the evaluated areas although there were areas with limited or conflicting support. The evidence suggests that the scores of the VB-MAPP provide information relevant to the target behaviors of interest but a few domains may not be fully represented by their specific items.

Reliability and validity of a novel caregiver-assessed skills system based on the ALSO conception in children with autism spectrum disorders

BACKGROUND

In responding to the potential challenges in the transition from childhood to adulthood for individuals with autism spectrum disorders (ASD) in China, a novel conception of "ALSO" was proposed to bridge the transitional needs and early intervention. To facilitate the application of ALSO in early intervention, ALSOLIFE skills assessment system (ALSOLIFE Assessment) was developed to enable caregivers to evaluate their children's skills guided by the ALSO conception. Given that the critical shortage of qualified professionals in China, many caregivers of children with ASD must function as home therapists. To address the practical needs of Chinese families of children with ASD, ALSOLIFE Assessment is designed as a free, online, technology-assisted, self-operated and behavioral intervention approaches supported system. The assessment report then further served as the basis for caregivers to deliver the tailored educational intervention to their children. Although ALSOLIFE Assessment provides caregivers a home-based intervention program, it is still unclear whether its evaluation is reliable and accurate. Therefore, we conduct this study to investigate the reliability and validity of the ALSOLIFE Assessment.

METHODS

A total of 1,050 children with ASD (1 to 10 years old) were recruited from 31 provinces of mainland China. Their caregivers participated simultaneously as ALSOLIFE Assessment evaluators. The testing results of Verbal Behavior Milestones Assessment and Placement Program (VB-MAPP) and Psychoeducational Profile-3 (PEP-3) were also collected for a portion of the child participants (VB-MAPP, N=34; PEP-3, N=31) to check criterion-related validity.

RESULTS

The reliability and validity of the ALSOLIFE Assessment satisfied psychometric requirements after the reduction from 511 to 464 items. Exploratory factor analysis (EFA) of ALSOLIFE Assessment scoring data yielded six factors, and confirmatory factor analysis (CFA) verified the best fitting construct model is a bifactorial model with one general factor and six group factors. Compared to the VB-MAPP and PEP-3, the ALSOLIFE Assessment exhibited good criterion-related validity across three levels: total scores, 6 skill domains, and 22 skill subdomains. The reliability tests indicated its strong internal consistency, test-retest reliability, and inter-rater reliability.

CONCLUSIONS

The reliability and validity of the revised ALSOLIFE Assessment (464-items) satisfied psychometric requirements. It is essential to develop validated and comprehensive evaluation tools, embedded on the needs of Chinese families of children with ASD.