Machine learning determination of applied behavioral analysis treatment plan type

Machine Learning Differentiation of Autism Spectrum Sub-Classifications

PURPOSE

Disorders on the autism spectrum have characteristics that can manifest as difficulties with communication, executive functioning, daily living, and more. These challenges can be mitigated with early identification. However, diagnostic criteria has changed from DSM-IV to DSM-5, which can make diagnosing a disorder on the autism spectrum complex. We evaluated machine learning to classify individuals as having one of three disorders of the autism spectrum under DSM-IV, or as non-spectrum.

METHODS

We employed machine learning to analyze retrospective data from 38,560 individuals. Inputs encompassed clinical, demographic, and assessment data.

RESULTS

The algorithm achieved AUROCs ranging from 0.863 to 0.980. The model correctly classified 80.5% individuals; 12.6% of individuals from this dataset were misclassified with another disorder on the autism spectrum.

CONCLUSION

Machine learning can classify individuals as having a disorder on the autism spectrum or as non-spectrum using minimal data inputs.

OpenMedLM: prompt engineering can out-perform fine-tuning in medical question-answering with open-source large language models

LLMs can accomplish specialized medical knowledge tasks, however, equitable access is hindered by the extensive fine-tuning, specialized medical data requirement, and limited access to proprietary models. Open-source (OS) medical LLMs show performance improvements and provide the transparency and compliance required in healthcare. We present OpenMedLM, a prompting platform delivering state-of-the-art (SOTA) performance for OS LLMs on medical benchmarks. We evaluated OS foundation LLMs (7B-70B) on medical benchmarks (MedQA, MedMCQA, PubMedQA, MMLU medical-subset) and selected Yi34B for developing OpenMedLM. Prompting strategies included zero-shot, few-shot, chain-of-thought, and ensemble/self-consistency voting. OpenMedLM delivered OS SOTA results on three medical LLM benchmarks, surpassing previous best-performing OS models that leveraged costly and extensive fine-tuning. OpenMedLM displays the first results to date demonstrating the ability of OS foundation models to optimize performance, absent specialized fine-tuning. The model achieved 72.6% accuracy on MedQA, outperforming the previous SOTA by 2.4%, and 81.7% accuracy on MMLU medical-subset, establishing itself as the first OS LLM to surpass 80% accuracy on this benchmark. Our results highlight medical-specific emergent properties in OS LLMs not documented elsewhere to date and validate the ability of OS models to accomplish healthcare tasks, highlighting the benefits of prompt engineering to improve performance of accessible LLMs for medical applications.

Family-Centric Applied Behavior Analysis Promotes Sustained Treatment Utilization and Attainment of Patient Goals

BACKGROUND/OBJECTIVES

 Autism spectrum disorder (ASD) is a neurodevelopmental disorder characterized by social communication difficulties and restricted repetitive behaviors or interests. Applied behavior analysis (ABA) has been shown to significantly improve outcomes for individuals on the autism spectrum. However, challenges regarding access, cost, and provider shortages remain obstacles to treatment delivery. To this end, parents were trained as parent behavior technicians (pBTs), improving access to ABA, and empowering parents to provide ABA treatment in their own homes. We hypothesized that patients diagnosed with severe ASD would achieve the largest gains in overall success rates toward skill acquisition in comparison to patients diagnosed with mild or moderate ASD. Our secondary hypothesis was that patients with comprehensive treatment plans (>25-40 hours/week) would show greater gains in skill acquisition than those with focused treatment plans (less than or equal to 25 hours/week).  Methods: This longitudinal, retrospective chart review evaluated data from 243 patients aged two to 18 years who received at least three months of ABA within our pBT treatment delivery model. Patients were stratified by utilization of prescribed ABA treatment, age, ASD severity (per the Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition), and treatment plan type (comprehensive vs. focused). Patient outcomes were assessed by examining success rates in acquiring skills, both overall and in specific focus areas (communication, emotional regulation, executive functioning, and social skills).

RESULTS

Patients receiving treatment within the pBT model demonstrated significant progress in skill acquisition both overall and within specific focus areas, regardless of cohort stratification. Patients with severe ASD showed greater overall skill acquisition gains than those with mild or moderate ASD. In addition, patients with comprehensive treatment plans showed significantly greater gains than those with focused treatment plans.

CONCLUSION

The pBT model achieved both sustained levels of high treatment utilization and progress toward patient goals. Patients showed significant gains in success rates of skill acquisition both overall and in specific focus areas, regardless of their level of treatment utilization. This study reveals that our pBT model of ABA treatment delivery leads to consistent improvements in communication, emotional regulation, executive functioning, and social skills across patients on the autism spectrum, particularly for those with more severe symptoms and those following comprehensive treatment plans.

Machine Learning Approach with Harmonized Multinational Datasets for Enhanced Prediction of Hypothyroidism in Patients with Type 2 Diabetes

Type 2 diabetes (T2D) is a global health concern with increasing prevalence. Comorbid hypothyroidism (HT) exacerbates kidney, cardiac, neurological and other complications of T2D; these risks can be mitigated pharmacologically upon detecting HT. The current HT standard of care (SOC) screening in T2D is infrequent, delaying HT diagnosis and treatment. We present a first-to-date machine learning algorithm (MLA) clinical decision tool to classify patients as low vs. high risk for developing HT comorbid with T2D; the MLA was developed using readily available patient data from harmonized multinational datasets. The MLA was trained on data from NIH All of US (AoU) and UK Biobank (UKBB) (Combined dataset) and achieved a high negative predictive value (NPV) of 0.989 and an AUROC of 0.762 in the Combined dataset, exceeding AUROCs for the models trained on AoU or UKBB alone (0.666 and 0.622, respectively), indicating that increasing dataset diversity for MLA training improves performance. This high-NPV automated tool can supplement SOC screening and rule out T2D patients with low HT risk, allowing for the prioritization of lab-based testing for at-risk patients. Conversely, an MLA output that designates a patient to be at risk of developing HT allows for tailored clinical management and thereby promotes improved patient outcomes.

Family-Centric Applied Behavior Analysis Facilitates Improved Treatment Utilization and Outcomes

Background/Objective: Autism spectrum disorder (ASD) is a neurodevelopmental condition characterized by lifelong impacts on functional social and daily living skills, and restricted, repetitive behaviors (RRBs). Applied behavior analysis (ABA), the gold-standard treatment for ASD, has been extensively validated. ABA access is hindered by limited availability of qualified professionals and logistical and financial barriers. Scientifically validated, parent-led ABA can fill the accessibility gap by overcoming treatment barriers. This retrospective cohort study examines how our ABA treatment model, utilizing parent behavior technicians (pBTs) to deliver ABA, impacts adaptive behaviors and interfering behaviors (IBs) in a cohort of children on the autism spectrum with varying ASD severity levels, and with or without clinically significant IBs. Methods: Clinical outcomes of 36 patients ages 3-15 years were assessed using longitudinal changes in Vineland-3 after 3+ months of pBT-delivered ABA treatment. Results: Within the pBT model, our patients demonstrated clinically significant improvements in Vineland-3 Composite, domain, and subdomain scores, and utilization was higher in severe ASD. pBTs utilized more prescribed ABA when children initiated treatment with clinically significant IBs, and these children also showed greater gains in their Composite scores. Study limitations include sample size, inter-rater reliability, potential assessment metric bias and schedule variability, and confounding intrinsic or extrinsic factors. Conclusion: Overall, our pBT model facilitated high treatment utilization and showed robust effectiveness, achieving improved adaptive behaviors and reduced IBs when compared to conventional ABA delivery. The pBT model is a strong contender to fill the widening treatment accessibility gap and represents a powerful tool for addressing systemic problems in ABA treatment delivery.

A Prediction Model of Autism Spectrum Diagnosis from Well-Baby Electronic Data Using Machine Learning

Early detection of autism spectrum disorder (ASD) is crucial for timely intervention, yet diagnosis typically occurs after age three. This study aimed to develop a machine learning model to predict ASD diagnosis using infants' electronic health records obtained through a national screening program and evaluate its accuracy. A retrospective cohort study analyzed health records of 780,610 children, including 1163 with ASD diagnoses. Data encompassed birth parameters, growth metrics, developmental milestones, and familial and post-natal variables from routine wellness visits within the first two years. Using a gradient boosting model with 3-fold cross-validation, 100 parameters predicted ASD diagnosis with an average area under the ROC curve of 0.86 (SD < 0.002). Feature importance was quantified using the Shapley Additive explanation tool. The model identified a high-risk group with a 4.3-fold higher ASD incidence (0.006) compared to the cohort (0.001). Key predictors included failing six milestones in language, social, and fine motor domains during the second year, male gender, parental developmental concerns, non-nursing, older maternal age, lower gestational age, and atypical growth percentiles. Machine learning algorithms capitalizing on preventative care electronic health records can facilitate ASD screening considering complex relations between familial and birth factors, post-natal growth, developmental parameters, and parent concern.

Machine Learning Approach for Improved Longitudinal Prediction of Progression from Mild Cognitive Impairment to Alzheimer's Disease

Mild cognitive impairment (MCI) is cognitive decline that can indicate future risk of Alzheimer's disease (AD). We developed and validated a machine learning algorithm (MLA), based on a gradient-boosted tree ensemble method, to analyze phenotypic data for individuals 55-88 years old (n = 493) diagnosed with MCI. Data were analyzed within multiple prediction windows and averaged to predict progression to AD within 24-48 months. The MLA outperformed the mini-mental state examination (MMSE) and three comparison models at all prediction windows on most metrics. Exceptions include sensitivity at 18 months (MLA and MMSE each achieved 0.600); and sensitivity at 30 and 42 months (MMSE marginally better). For all prediction windows, the MLA achieved AUROC ≥ 0.857 and NPV ≥ 0.800. With averaged data for the 24-48-month lookahead timeframe, the MLA outperformed MMSE on all metrics. This study demonstrates that machine learning may provide a more accurate risk assessment than the standard of care. This may facilitate care coordination, decrease healthcare expenditures, and maintain quality of life for patients at risk of progressing from MCI to AD.

Clinical Outcomes of a Hybrid Model Approach to Applied Behavioral Analysis Treatment

Objective This study examines the implementation of a hybrid applied behavioral analysis (ABA) treatment model to determine its impact on autism spectrum disorder (ASD) patient outcomes.  Methods Retrospective data were collected for 25 pediatric patients to measure progress before and after the implementation of a hybrid ABA treatment model under which therapists consistently captured session notes electronically regarding goals and patient progress. ABA treatment was streamlined for consistent delivery, with improved software utilization for tracking scheduling and progress. Eleven goals within three domains (behavioral, social, and communication) were examined.  Results After the implementation of the hybrid model, the goal success rate improved by 9.7% compared to the baseline; 41.8% of goals showed improvement, 38.4% showed a flat trend, and 19.8% showed deterioration. Multiple goals trended upwards in 76% of the patients.  Conclusion This pilot study demonstrated that enhancing the consistency with which ABA treatment is monitored/delivered can improve patient outcomes as seen through improved attainment of goals.