The influence of loss to follow-up in autism screening research: Taking stock and moving forward

BACKGROUND

How best to improve the early detection of autism spectrum disorder (ASD) is the subject of significant controversy. Some argue that universal ASD screeners are highly accurate, whereas others argue that evidence for this claim is insufficient. Relatedly, there is no clear consensus as to the optimal role of screening for making referral decisions for evaluation and treatment. Published screening research can meaningfully inform these questions-but only through careful consideration of children who do not complete diagnostic follow-up.

METHODS

We developed two simulation models that re-analyze the results of a large-scale validation study of the M-CHAT-R/F by Robins et al. (2014, Pediatrics, 133, 37). Model #1 re-analyzes screener accuracy across six scenarios, each reflecting different assumptions regarding loss to follow-up. Model #2 builds on this by closely examining differential attrition at each point of the multi-step detection process.

RESULTS

Estimates of sensitivity ranged from 40% to 94% across scenarios, demonstrating that estimates of accuracy depend on assumptions regarding the diagnostic status of children who were lost to follow-up. Across a range of plausible assumptions, data also suggest that children with undiagnosed ASD may be more likely to complete follow-up than children without ASD, highlighting the role of clinicians and caregivers in the detection process.

CONCLUSIONS

Using simulation modeling as a quantitative method to examine potential bias in screening studies, analyses suggest that ASD screening tools may be less accurate than is often reported. Models also demonstrate the critical importance of every step in a detection process-including steps that determine whether children should complete an additional evaluation. We conclude that parent and clinician decision-making regarding follow-up may contribute more to detection than is widely assumed.

Conceptualizations of health, ability to act upon those definitions and desire for supports among families from historically marginalized communities

Respect for parents' values and clinician-parent collaboration is less common among families from historically marginalized communities. We investigated how parents from marginalized communities operationalize health and their preferences for paediatric primary care. We recruited families who spoke English, Haitian Creole or Spanish with at least one child younger than 6 years old. Staff queried families' values and life experiences, perspectives on health and healthcare, social supports and resources. Fourteen interviews with the parents of 26 children were thematically analysed. Interviews revealed the following four themes: (1) parents' definitions of 'health' extend beyond physical health; (2) families' ability to actuate health definitions is complicated by poverty's impact on agency; (3) parents engage in ongoing problem recognition and identify solutions, but enacting solutions can be derailed by barriers and (4) parents want support from professionals and peers who acknowledged the hard work of parenting. Eliciting parents' multidimensional conceptualizations of health can support families' goal achievement and concern identification in the context of isolation, limited agency and few resources. Efforts to improve family centred care and reduce disparities in paediatric primary care must be responsive to the strengths, challenges, resources and priorities of marginalized families.

Operationalizing Primary Outcomes to Achieve Reach, Effectiveness, and Equity in Multilevel Interventions

When intervention scientists plan a clinical trial of an intervention, they select an outcome metric that operationalizes their definition of intervention success. The outcome metric that is selected has important implications for which interventions are eventually supported for implementation at scale and, therefore, what health benefits (including how much benefit and for whom) are experienced in a population. Particularly when an intervention is to be implemented in a population that experiences a health disparity, the outcome metric that is selected can also have implications for equity. Some outcome metrics risk exacerbating an existing health disparity, while others may decrease disparities for some but have less effect for the larger population. In this study, we use a computer to simulate implementation of a hypothetical multilevel, multicomponent intervention to highlight the tradeoffs that can occur between outcome metrics that reflect different operationalizations of intervention success. In particular, we highlight tradeoffs between overall mean population benefit and the distribution of health benefits in the population, which has direct implications for equity. We suggest that simulations like the one we present can be useful in the planning of a clinical trial for a multilevel and/or multicomponent intervention, since simulated implementation at scale can illustrate potential consequences of candidate operationalization of intervention success, such that unintended consequences for equity can be avoided.

Child social-emotional and behavioral problems and competencies contribute to changes in developmental functioning during Early Intervention

This study examined how social-emotional and behavioral (SEB) problems and competencies contribute to changes in developmental functioning among children enrolled in Part C Early Intervention (EI), a U.S. program supporting young children with developmental delays and disabilities. The sample included 1,055 children enrolled in EI from 2011-2019 (mean age at EI entry = 17 months; 64% male; 72% marginalized racial and ethnic backgrounds). Standardized developmental assessments, drawn from administrative records, characterized developmental functioning at EI entry and exit and parents reported SEB functioning. Hierarchical regression analyses revealed that SEB problems and competencies interacted in predicting change in developmental functioning from EI entry to exit. Monitoring, identifying, and addressing SEB problems and competencies may optimize developmental outcomes for young children with developmental delays and disabilities.

Pediatric Provider and Staff Burnout in Federally Qualified Community Health Centers

This study examined changes in provider and staff burnout in 4 Boston-area federally qualified community health centers (FQHCs) participating in a pediatric behavioral health integration project. Utilizing the Maslach Burnout Inventory-Human Services Survey for Medical Personnel (MBI-HSSMP), the study found that emotional exhaustion among primary care providers ( M = 20.5; confidence interval [CI], 17.8-23.2) was higher ( P = .001) than that among behavioral health clinicians ( M = 13.6; CI, 10.4-16.8) and ( P = .00005) community health workers ( M = 10.8; CI, 7.3-14.2). Emotional exhaustion among staff increased ( P = .04) from baseline ( M = 16.8; CI, 15.0-18.6) to follow-up ( M = 20.8; CI, 17.5-24.2), but burnout at follow-up was lower than national averages. FQHCs are integral in caring for marginalized patients; therefore, supporting a stable workforce by minimizing burnout is essential.

Changes Over Time in Outcomes of School-Age Children and Parents Receiving Integrated Mental Health Care in Federally Qualified Health Centers

OBJECTIVE

The purpose of this study is to understand change in parent-reported outcomes of mental health symptoms, health-related quality of life (QoL), and school-related functioning among children receiving mental health care at 3 federally qualified health centers engaging in a comprehensive pediatric mental health integration model.

METHODS

Trained personnel enrolled English- or Spanish-speaking families of 6- to 12-year-old children who had recently started receiving integrated mental health care and surveyed their parent/caregiver at 3 time points: entry into the cohort, 6-month follow-up, and 12-month follow-up (unique N = 51). Primary outcomes included validated measures of child symptoms, child health-related QoL, and child school-related functioning. Secondary outcomes focused on parental functioning and included validated measures of parental stress and depressive and internalizing symptoms. A multilevel mixed-effects generalized linear model was used to estimate the change in parent-reported outcomes over time, with inverse probability weights used to address attrition. Additional analyses were conducted to determine the degree to which changes in symptoms over time were associated with improvements in school-related functioning.

RESULTS

Over 12 months, children's mental health symptoms, health-related QoL, and school-related functional outcomes significantly improved. No changes in parental functioning were observed. In addition, improvements in mental health symptoms and health-related QoL were associated with improvements in school-related functional outcomes over time.

CONCLUSION

Findings demonstrate that outcomes of children who received integrated mental health care improved over time, both in regard to mental health and school functioning.

Associations of preschool reactive bed-sharing with sociodemographic factors, sleep disturbance, and psychopathology

OBJECTIVE

To advance understanding of early childhood bed-sharing and its clinical significance, we examined reactive bed-sharing rates, sociodemographic correlates, persistence, and concurrent and longitudinal associations with sleep disturbances and psychopathology.

METHODS

Data from a representative cohort of 917 children (mean age 3.8 years) recruited from primary pediatric clinics in a Southeastern city for a preschool anxiety study were used. Sociodemographics and diagnostic classifications for sleep disturbances and psychopathology were obtained using the Preschool Age Psychiatric Assessment (PAPA), a structured diagnostic interview administered to caregivers. A subsample of 187 children was re-assessed approximately 24.7 months after the initial PAPA interview.

RESULTS

Reactive bed-sharing was reported by 38.4% of parents, 22.9% nightly and 15.5% weekly, and declined with age. At follow-up, 48.9% of nightly bed-sharers and 88.7% of weekly bed-sharers were no longer bed-sharing. Sociodemographics associated with nightly bed-sharing were Black and (combined) American Indian, Alaska Native and Asian race and ethnicity, low income and parent education less than high school. Concurrently, bed-sharing nightly was associated with separation anxiety and sleep terrors; bed-sharing weekly was associated with sleep terrors and difficulty staying asleep. No longitudinal associations were found between reactive bed-sharing and sleep disturbances or psychopathology after controlling for sociodemographics, baseline status of the outcome and time between interviews.

CONCLUSIONS

Reactive bed-sharing is relatively common among preschoolers, varies significantly by sociodemographic factors, declines during the preschool years and is more persistent among nightly than weekly bed-sharers. Reactive bed-sharing may be an indicator of sleep disturbances and/or anxiety but there is no evidence that bed-sharing is an antecedent or consequence of sleep disturbances or psychopathology.

Association of Integrating Mental Health Into Pediatric Primary Care at Federally Qualified Health Centers With Utilization and Follow-up Care

Importance

More than 1 in 5 children in low-income families report a mental health (MH) problem, yet most face barriers accessing MH services. Integrating MH services into primary care at pediatric practices such as federally qualified health centers (FQHCs) may address these barriers.

Objective

To examine the association of a comprehensive MH integration model with health care utilization, psychotropic medication use, and MH follow-up care among Medicaid-enrolled children at FQHCs.

Design, Setting, and Participants

This retrospective cohort study used Massachusetts claims data from 2014 to 2017 to conduct difference-in-differences (DID) analyses before vs after implementation of a complete FQHC-based MH integration model. The sample included Medicaid-enrolled children aged 3 to 17 years who received primary care at 3 intervention FQHCs or 6 geographically proximal nonintervention FQHCs in Massachusetts. Data were analyzed in July 2022.

Exposures

Receipt of care at an FQHC implementing the Transforming and Expanding Access to Mental Health Care in Urban Pediatrics (TEAM UP) model, which began fully integrating MH care into pediatrics in mid-2016.

Main Outcomes and Measures

Utilization outcomes included primary care visits, MH service visits, emergency department (ED) visits, inpatient admissions, and psychotropic medication use. Follow-up visits within 7 days of a MH-related ED visit or hospitalization were also examined.

Results

Among the 20 170 unique children in the study sample, at baseline (2014), their mean (SD) age was 9.0 (4.1) years, and 4876 (51.2%) were female. In contrast to nonintervention FQHCs, TEAM UP was positively associated with primary care visits with MH diagnoses (DID, 4.35 visits per 1000 patients per quarter; 95% CI, 0.02 to 8.67 visits per 1000 patients per quarter) and MH service use (DID, 54.86 visits per 1000 patients per quarter; 95% CI, 1.29 to 108.43 visits per 1000 patients per quarter) and was negatively associated with rates of psychotropic medication use (DID, -0.4%; 95% CI -0.7% to -0.01%) and polypharmacy (DID, -0.3%; 95% CI, -0.4% to -0.1%). TEAM UP was positively associated with ED visits without MH diagnoses (DID, 9.45 visits per 1000 patients per quarter; 95% CI, 1.06 to 17.84 visits per 1000 patients per quarter), but was not significantly associated with ED visits with MH diagnoses. No statistically significant changes were observed in inpatient admissions, follow-up visits after MH ED visits, or follow-up visits after MH hospitalizations.

Conclusions and Relevance

The first 1.5 years of MH integration enhanced access to pediatric MH services while limiting the use of psychotropic medications. Additional implementation time is necessary to determine whether these changes will translate into reductions in avoidable utilization.

Overcoming tensions between family-centered care and fidelity within Early Intervention implementation research

LAY ABSTRACT

Early Intervention systems provide therapeutic services to families of young children birth to 3 years with developmental delays and are considered a natural access point to services for young children and their families. Research studies in the autism field have been interested in training providers to deliver evidence-based practices in Early Intervention systems to increase access to services for young children with an increased likelihood of being autistic. However, research has often overlooked that Early Intervention systems prioritize family-centered care, an approach to working with families that honors and respects their values and choices and that provides supports to strengthen family functioning. This commentary points out that family-centered care deserves greater attention in research being done in Early Intervention systems. We describe how family-centered care may shape how interventions are delivered, and discuss directions for future research to evaluate the impact of family-centered care alongside intervention delivery.

Prevalence and Co-Occurrence of Developmental and Emotional-Behavioral Problems in Young Children

OBJECTIVE

A limited number of studies have estimated the prevalence of emotional-behavioral disorders among young children. None have assessed their co-occurrence with developmental delays using standardized assessment tools. Our objective was to estimate the prevalence of emotional-behavioral disorders and their co-occurrence with developmental delays among young children (2-5.5 years).

METHODS

Parents of young children (N = 987) enrolled from pediatric waiting rooms completed developmental-behavioral screening questionnaires. Based on results, 585 families were invited to and 439 completed evaluations that included structured diagnostic interviews with parents to assess child psychopathology (Preschool Age Psychiatric Assessment (PAPA)), developmental testing with children (including the Bayley Scales of Infant and Toddler Development, third Edition for children ≤ 42 months; Differential Ability Scales, second Edition for older children), and videotaped observation to establish whether autism risk was sufficient to warrant further evaluation.

RESULTS

According to PAPA algorithms, 23.0% of children met criteria for a DSM-IV disorder, while 9% qualified for a developmental delay. Presence of delay doubled the odds of having a DSM-IV disorder (OR = 2.1; CI: 1.02-4.3), and presence of disorder doubled the odds of having a moderate-to-severe developmental delay (OR=2.0; CI: 1.10-3.50). Prevalence of DSM-IV disorders (48.8% (95% CI: 33.5-64.5%)) and developmental delays (57.5% (95% CI: 41.7-71.9%)) were both higher among children at risk for autism. While developmental delay did not vary by race/ethnicity, prevalence of DSM-IV disorders was lower among African-American/Black (10.6%; CI: 2.9-18.3) and Hispanic/Latino children (11.1%; CI: 4.3-17.9).

CONCLUSIONS

Developmental delays and emotional-behavioral disorders among young children seen in pediatric settings are characterized by high prevalence and substantial co-occurrence.

From Descriptive to Predictive: Linking Early Childhood Developmental and Behavioral Screening Results With Educational Outcomes in Kindergarten

OBJECTIVE

To assess the predictive value of a pediatric screening tool by linking 2 independent databases: an educational database that includes data from standardized academic assessments administered during kindergarten and a pediatric database that includes screening results.

METHODS

A database that includes results of the Survey of Well-being of Young Children (SWYC) completed during pediatric visits were linked to an educational database that includes STAR Early Literacy examinations in kindergarten. Linear multilevel regression modeling was used to examine if screening results on the developmental and behavioral sections of the most recently completed SWYC form predicted trends in the percentile rank on the STAR exam over the school year, adjusting for potential confounders.

RESULTS

Linking the 2 databases resulted in a sample of 586 children who were administered at least one SWYC evaluation between 24 and 48 months and completed at least one STAR Early Literacy examination in kindergarten. The sample represents a diverse population with 55% Hispanic children, 25% Non-Hispanic black children, and 91% of children receiving a subsidized lunch. After adjusting for confounders, children with a positive developmental or behavioral screen had significantly lower percentile ranks on the STAR exam.

CONCLUSIONS

Early developmental and behavioral screening results predicted performance on the STAR exam in kindergarten. Children with developmental and behavioral concerns may be less ready to enter kindergarten than peers without such concerns. These preliminary findings provide proof-of-principle of the potential utility of developmental screening tools in identifying children with reduced school readiness who may benefit from intervention prior to kindergarten.

Implementation of a Novel Pediatric Behavioral Health Integration Initiative

This study explores healthcare professionals' perspectives about the impact of behavioral health integration (BHI) on pediatric primary care delivery in community health centers (CHCs). A concurrent, qualitative-dominant mixed methods empirical study design was utilized, applying semi-structured interviews with healthcare professionals at the end of the implementation phase of a 3-year co-development, implementation, and evaluation process. Surveys were also administered at three time points. Via thematic analysis, emergent qualitative themes were mapped onto the Relational Coordination (RC) conceptual framework to triangulate and complement final qualitative results with quantitative results. Interview findings reveal five emergent themes aligning with RC domains. Survey results show that healthcare professionals reported increased behavioral healthcare integration into clinic practice (p = 0.0002) and increased clinic readiness to address behavioral health needs (p = 0.0010). Effective pediatric BHI and care delivery at CHCs may rely on strong professional relationships and communication. Additional research from the patient/caregiver perspective is needed.

Editorial: Evaluating the Success of Early Detection of Autism: It's Time to Move Beyond the Median

For many years, the median age of diagnosis has been the most widely used metric for evaluating changes in the early diagnosis of autism. The logic appears simple: if health and educational systems improve their ability to diagnose autism at early ages, then it would seem that the median age should fall. However, despite decades of efforts in the United States to improve the early identification, the median age of diagnosis has hardly budged. But simple logic can be deceptive. In a watershed analysis of data from the Autism and Developmental Disabilities Monitoring Network-the most prominent study of autism prevalence in the United States-Shaw and colleagues apply different metrics to existing data to yield novel findings. Citing earlier methodological work published in this journal and others, the authors report what they refer to as the "simple median age of diagnosis" (ie, the median age at the earliest recorded autism spectrum disorder diagnosis or special educational eligibility before 8 years of age) alongside 2 alternative metrics: (1) "median age at identification"-similar to the "simple median" except that it also includes children who meet case definitions for autism but are not yet identified by 8 years of age (ie, they do not have a recorded diagnosis or record of special education eligibility); and (2) cumulative incidence of autism by 4 years of age-ie, the ratio of the number of children diagnosed by 48 months of age divided by the total number of children in the population. Results have tremendous implications both for research and policy.

Integrating Pediatric Universal Behavioral Health Care at Federally Qualified Health Centers

BACKGROUND

Research supports integrated pediatric behavioral health (BH), but evidence gaps remain in ensuring equitable care for children of all ages. In response, an interdisciplinary team codeveloped a stepped care model that expands BH services at 3 federally qualified health centers (FQHCs).

METHODS

FQHCs reported monthly electronic medical record data regarding detection of BH issues, receipt of services, and psychotropic medications. Study staff reviewed charts of children with attention-deficit/hyperactivity disorder (ADHD) before and after implementation.

RESULTS

Across 47 437 well-child visits, >80% included a complete BH screen, significantly higher than the state's long-term average (67.5%; P < .001). Primary care providers identified >30% of children as having BH issues. Of these, 11.2% of children <5 years, 53.8% of 5-12 years, and 74.6% >12 years were referred for care. Children seen by BH staff on the day of referral (ie, "warm hand-off") were more likely to complete an additional BH visit than children seen later (hazard ratio = 1.37; P < .0001). There was no change in the proportion of children prescribed psychotropic medications, but polypharmacy declined (from 9.5% to 5.7%; P < .001). After implementation, diagnostic rates for ADHD more than doubled compared with baseline, follow-up with a clinician within 30 days of diagnosis increased (62.9% before vs 78.3% after; P = .03) and prescriptions for psychotropic medication decreased (61.4% before vs 43.9% after; P = .03).

CONCLUSIONS

Adding to a growing literature, results demonstrate that integrated BH care can improve services for children of all ages in FQHCs that predominantly serve marginalized populations.

Effectiveness of Screening in Early Intervention Settings to Improve Diagnosis of Autism and Reduce Health Disparities

IMPORTANCE

The American Academy of Pediatrics recommends referring children at elevated risk of autism spectrum disorder (ASD) for Part C early intervention (EI) services, but notes that EI services often fail to provide ASD screening.

OBJECTIVE

To evaluate the hypothesis that a multistage screening protocol for ASD implemented in 3 EI settings will increase autism detection, especially among Spanish-speaking families.

DESIGN, SETTING, AND PARTICIPANTS

Difference-in-differences analyses with propensity score weighting of a quasi-experimental design using administrative data on 3 implementation EI agencies and 9 comparison EI agencies from 2012 to 2018 provided by the Massachusetts Department of Public Health. Eligible children were aged 14 to 36 months, enrolled in EI, had no prior ASD diagnosis or medical condition precluding participation, and had parents who spoke English or Spanish. The final analytic sample included 33 326 unique patients assessed across 150 200 person-quarters.

EXPOSURES

Multistage ASD assessment protocol including ASD screening questionnaires, observational screener, and diagnostic evaluation.

MAIN OUTCOMES AND MEASURES

Increased incidence of ASD diagnoses as documented in Department of Public Health records and reductions in language-associated health care disparities.

RESULTS

Implementation of screening at 3 EI sites was associated with a significant increase in the rate of ASD diagnoses (incidence rate ratios [IRR], 1.6; 95% CI, 1.3-2.1; P < .001), representing an additional 8.1 diagnoses per 1000 children per quarter. Among Spanish-speaking families, screening was also associated with a significant increase in the rate of ASD diagnoses (IRR, 2.6; 95% CI, 1.6-4.3; P < .001), representing 15.4 additional diagnoses per 1000 children per quarter-a larger increase than for non-Spanish-speaking families (interaction IRR, 1.8; 95% CI, 1.0-3.1; P = .005). Exploratory analyses revealed that screening was associated with a larger increase in the rate of ASD diagnoses among boys (IRR, 1.8; 95% CI, 1.4-2.3; P < .001) than among girls (IRR, 1.1; 95% CI, 0.6-1.7; P = .84).

CONCLUSIONS AND RELEVANCE

In this study, associations between increased rates of ASD diagnoses and reductions in disparities for Spanish-speaking households support the effectiveness of multistage screening in EI. This study provides a comprehensive evaluation of ASD screening in EI settings as well as a rigorous evaluation of ASD screening in any setting with a no-screening comparison condition. Given that the intervention included multiple components, mechanisms of action warrant further research, as do disparities by child sex.

Social Determinants of Health and ADHD Symptoms in Preschool-Age Children

OBJECTIVE

Attention-Deficit/Hyperactivity Disorder (ADHD) disproportionately affects socioeconomically disadvantaged children, but for unclear reasons. We examined the association between social determinants of health (SDH) and ADHD symptoms in a national sample of preschool-age children.

METHODS

We conducted exploratory factor analysis (EFA) and exploratory structural equation modeling (ESEM) with a sample of 7,565 preschool-age children from the 2016 National Survey of Children's Health, to examine the association between ADHD symptoms and SDH.

RESULTS

EFA indicated a one-factor structure for ADHD symptoms, and three factors for SDH (socioeconomic status, access to basic needs, and caregiver well-being). Independently, all three SDH were significantly associated with higher ADHD symptoms. However, in the ESEM model, only worse caregiver well-being (β = .39, p < .01) was significantly associated with ADHD symptoms, and fully mediated the relationship between SDH and ADHD symptoms.

CONCLUSION

Addressing caregiver well-being in preschoolers with ADHD symptoms could be an early intervention strategy.

A Taxonomy of Reported Harms in Pediatric Autism Spectrum Disorder Screening: Provider and Parent Perspectives

The U.S. Preventive Services Task Force (USPSTF) report on screening for Autism Spectrum Disorder (ASD) highlighted the need for research that examines the harms potentially associated with screening so as to assess the overall net benefit of universal screening. In response, this study engages qualitative, semi-structured interviews to generate a taxonomy outlining potential harms reported by parents and providers (pediatricians and Early Intervention providers) with experience in screening young children for ASD. Potential harms emerged including: physical, psychological, social, logistical/financial, opportunity cost, attrition, and exacerbation of non-targeted disparities. Respondents reported harms being experienced by the toddlers, parents, and providers. The harms reported highlight opportunities for providers to offer resources that mitigate the potential for these unintended consequences.

Delays in autism diagnosis for U.S. Spanish-speaking families: The contribution of appointment availability

Promoting equity in health services requires an understanding of the mechanisms that produce disparities. Utilizing a sequential, mixed-methods, explanatory study design, we analyzed child-, family-, and organizational-level factors and their association with wait times for an ASD diagnostic evaluation among 353 families scheduled for English and Spanish language appointments (27% Spanish language). A subset of parents and caregivers participated in English and Spanish language focus groups to provide their perspectives on the diagnostic process. Spanish language was associated with greater completion of, and time to evaluations than English language. The only variable found to mediate associations with time-to-evaluation was appointment availability - an organizational factor. Qualitative results elucidate potential explanations for greater Spanish language evaluation completion (e.g., fewer community-based diagnostic options). Results serve as a case study to support the utility and importance of analyzing the influence of organizational-level factors on delays and disparities for childhood health and mental health services. We discuss our findings in relation to strategies that can be widely applied to support equitable services access for childhood diagnostic and intervention services.

Measuring the impact of burn injury on the parent-reported health outcomes of children 1-to-5 years: Item pool development for the Preschool1-5 Life Impact Burn Recovery Evaluation (LIBRE) Profile

Modern, reliable, and valid outcome measures are essential to understanding the health needs of young children with burn injuries. Burn-specific and age-appropriate legacy assessment tools exist for this population but are hindered by the limitations of existing paper-based instruments. The purpose of this study was to develop item pools comprised of questions appropriate for children aged 1-5 with burn injuries. Item development was based on a framework provided by previous work to develop the Preschool Life Impact Burn Recovery Evaluation (LIBRE) Conceptual Model. The Preschool LIBRE Conceptual Model work established four sub-domains of functioning for children with burns aged 1-5. Item development involved a systematic literature review, a qualitative item review process with clinical experts, and parent cognitive interviews. Four item pools were established: (1) communication and language development; (2) physical functioning; (3) psychological functioning and (4) social functioning for preschool-aged children with burn injuries. We selected and refined candidate items, recall periods, survey instructions, and response option choices through clinical and parental feedback during the qualitative review and cognitive interview processes. Item pools are currently being field-tested as part of the process to calibrate and validate the Preschool1-5 LIBRE Computer Adaptive Test (CAT) Profile.

Rapid-cycle systems modeling to support evidence-informed decision-making during system-wide implementation

Background

To “model and simulate change” is an accepted strategy to support implementation at scale. Much like a power analysis can inform decisions about study design, simulation models offer an analytic strategy to synthesize evidence that informs decisions regarding implementation of evidence-based interventions. However, simulation modeling is under-utilized in implementation science. To realize the potential of simulation modeling as an implementation strategy, additional methods are required to assist stakeholders to use models to examine underlying assumptions, consider alternative strategies, and anticipate downstream consequences of implementation. To this end, we propose Rapid-cycle Systems Modeling (RCSM)—a form of group modeling designed to promote engagement with evidence to support implementation. To demonstrate its utility, we provide an illustrative case study with mid-level administrators developing system-wide interventions that aim to identify and treat trauma among children entering foster care.

Methods

RCSM is an iterative method that includes three steps per cycle: (1) identify and prioritize stakeholder questions, (2) develop or refine a simulation model, and (3) engage in dialogue regarding model relevance, insights, and utility for implementation. For the case study, 31 key informants were engaged in step 1, a prior simulation model was adapted for step 2, and six member-checking group interviews (n = 16) were conducted for step 3.

Results

Step 1 engaged qualitative methods to identify and prioritize stakeholder questions, specifically identifying a set of inter-related decisions to promote implementing trauma-informed screening. In step 2, the research team created a presentation to communicate key findings from the simulation model that addressed decisions about programmatic reach, optimal screening thresholds to balance demand for treatment with supply, capacity to start-up and sustain screening, and availability of downstream capacity to provide treatment for those with indicated need. In step 3, member-checking group interviews with stakeholders documented the relevance of the model results to implementation decisions, insight regarding opportunities to improve system performance, and potential to inform conversations regarding anticipated implications of implementation choices.

Conclusions

By embedding simulation modeling in a process of stakeholder engagement, RCSM offers guidance to realize the potential of modeling not only as an analytic strategy, but also as an implementation strategy.

Examining the measurement equivalence of the Maslach Burnout Inventory across age, gender, and specialty groups in US physicians

Background

Disparities in US physician burnout rates across age, gender, and specialty groups as measured by the Maslach Burnout Inventory-Human Services Survey for Medical Personnel (MBI) are well documented. We evaluated whether disparities in US physician burnout are explained by differences in the MBI’s functioning across physician age, gender, and specialty groups.

Methods

We assessed the measurement equivalence of the MBI across age, gender, and specialty groups in multi-group item response theory- (IRT-) based differential item functioning (DIF) analyses using secondary, cross-sectional survey data from US physicians (n = 6577). We detected DIF using two IRT-based methods and assessed its impact by estimating the overall average difference in groups’ subscale scores attributable to DIF. We assessed DIF’s practical significance by comparing differences in individuals’ subscale scores and burnout prevalence estimates from models unadjusted and adjusted for DIF.

Results

We detected statistically significant age-, gender-, and specialty- DIF in all but one MBI item. However, in all cases, average differences in expected subscale-level scores due to DIF were < 0.10 SD on each subscale. Differences in physicians’ individual-level subscale scores and burnout symptom prevalence estimates across DIF- adjusted and unadjusted IRT models were also small (in all cases, mean absolute differences in individual subscale scores were < 0.04 z-score units; prevalence estimates differed by < 0.70%).

Conclusions

Age-, gender-, and specialty-related disparities in US physician burnout are not explained by differences in the MBI’s functioning across these demographic groups. Our findings support the use of the MBI as a valid tool to assess age-, gender-, and specialty-related disparities in US physician burnout.

Supplementary Information

The online version contains supplementary material available at 10.1186/s41687-021-00312-2.

The decision sampling framework: a methodological approach to investigate evidence use in policy and programmatic innovation

Background

Calls have been made for greater application of the decision sciences to investigate and improve use of research evidence in mental health policy and practice. This article proposes a novel method, “decision sampling,” to improve the study of decision-making and research evidence use in policy and programmatic innovation. An illustrative case study applies the decision sampling framework to investigate the decisions made by mid-level administrators when developing system-wide interventions to identify and treat the trauma of children entering foster care.

Methods

Decision sampling grounds qualitative inquiry in decision analysis to elicit information about the decision-making process. Our case study engaged mid-level managers in public sector agencies (n = 32) from 12 states, anchoring responses on a recent index decision regarding universal trauma screening for children entering foster care. Qualitative semi-structured interviews inquired on questions aligned with key components of decision analysis, systematically collecting information on the index decisions, choices considered, information synthesized, expertise accessed, and ultimately the values expressed when selecting among available alternatives.

Results

Findings resulted in identification of a case-specific decision set, gaps in available evidence across the decision set, and an understanding of the values that guided decision-making. Specifically, respondents described 14 inter-related decision points summarized in five domains for adoption of universal trauma screening protocols, including (1) reach of the screening protocol, (2) content of the screening tool, (3) threshold for referral, (4) resources for screening startup and sustainment, and (5) system capacity to respond to identified needs. Respondents engaged a continuum of information that ranged from anecdote to research evidence, synthesizing multiple types of knowledge with their expertise. Policy, clinical, and delivery system experts were consulted to help address gaps in available information, prioritize specific information, and assess “fit to context.” The role of values was revealed as participants evaluated potential trade-offs and selected among policy alternatives.

Conclusions

The decision sampling framework is a novel methodological approach to investigate the decision-making process and ultimately aims to inform the development of future dissemination and implementation strategies by identifying the evidence gaps and values expressed by the decision-makers, themselves.

Supplementary Information

The online version contains supplementary material available at 10.1186/s13012-021-01084-5.

Understanding How Parents Make Meaning of Their Child's Behaviors During Screening for Autism Spectrum Disorders: A Longitudinal Qualitative Investigation

A family's journey in understanding their child's behaviors in relation to Autism Spectrum Disorders (ASD) frequently begins with screening. This study aimed to characterize the interpretive processes that unfold for parents. We employed longitudinal interviews with 19 families engaged in a community-based multi-stage screening protocol. Parents participated in 1-6 interviews dependent upon children's length of engagement in the screening protocol; data were analyzed through modified grounded theory. Parents who moved towards understanding their child's behaviors as ASD expressed (1) sensitization to ASD symptoms, (2) differentiation from other developmental conditions, and (3) use of the ASD diagnosis to explain the etiology of concerning behaviors. Identifying interpretive processes involved during ASD screening provides new opportunities for shared decision-making.

To reduce the average age of autism diagnosis, screen preschoolers in primary care

LAY ABSTRACT

Pediatric primary care providers check for autism signs, usually using a standard checklist, at 18- and 24-month well-child visits. When the checklist shows possible autism, children should be referred for additional treatment and evaluation with an autism specialist. However, many children with autism spectrum disorder are not detected as toddlers. Low-income and minority children are particularly likely to have a late autism spectrum disorder diagnosis. Checking for autism at preschool-aged well-child visits might be one way to identify autism spectrum disorder earlier, especially for low-income and minority children.

Implications of adverse childhood experiences screening on behavioral health services: A scoping review and systems modeling analysis

Widespread implementation of adverse childhood experiences (ACEs) screening is occurring in the United States in response to policies and practice recommendations. However, limited research has established how these screening efforts impact the health care system and ultimately health outcomes. This article examines the current knowledge base on screening in medical settings. A scoping review of articles reporting on ACEs screening and prevalence in the United States was conducted. Of the 1,643 unique studies across two decades, 12 articles meeting criteria included nine on routine screening in medical settings and three on population-based surveys. A Monte Carlo simulation model was designed to synthesize evidence, identify key areas of uncertainty, and explore service system implications. Results indicated significant heterogeneity in the proportion of respondents who reported ACEs, with 6% to 64% of patients reporting 1+ ACEs and .01% to 40.7% reporting 4+ ACEs. Gaps in the literature were identified regarding cut-scores for referrals and referral completion rates. Three scenarios, modeled based on these data and past research on behavioral health screenings in pediatric primary care, demonstrated how ACEs screening may differentially impact behavioral health care systems. Priorities for future research were highlighted to refine estimates of the likely impact of ACEs screening on health care delivery. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

A commentary on interpreting the United States preventive services task force autism screening recommendation statement

In 2016, the US Preventive Services Task Force concluded that there was "insufficient" ("I" statement) evidence to support universal primary care screening for autism spectrum disorder. The statement led to controversy among research and clinical communities. Although a number of papers have since been published arguing for the potential benefit of autism spectrum disorder screening, none adequately address the potential harms of autism spectrum disorder screening. This evidence gap may relate to confusion regarding how the US Preventive Services Task Force conceptualizes and evaluates potential harm. In this commentary, we explore how the US Preventive Services Task Force operationalizes harm and discuss how the potential for harm was described in the "I" statement on autism spectrum disorder screening. This information can serve as a guide for investigators working to study the benefits and harms of autism spectrum disorder screening in order to fill the research gaps cited by the US Preventive Services Task Force report. Finally, we recommend future research directions for exploring harms of autism spectrum disorder screening, filling cited research gaps, and ultimately ensuring that the benefits of autism spectrum disorder screening truly outweigh the harms for all children and their families.

Describing the emotional exhaustion, depersonalization, and low personal accomplishment symptoms associated with Maslach Burnout Inventory subscale scores in US physicians: an item response theory analysis

PURPOSE

Current US health policy discussions regarding physician burnout have largely been informed by studies employing the Maslach Burnout Inventory (MBI); yet, there is little in the literature focused on interpreting MBI scores. We described the burnout symptoms and precision associated with MBI scores in US physicians.

METHODS

Using item response theory (IRT) analyses of secondary, cross-sectional survey data, we created response profiles describing the probability of burnout symptoms associated with US physicians' MBI emotional exhaustion (EE), depersonalization (DP), and personal accomplishment (PA) subscale scores. Response profiles were mapped to raw subscale scores and used to predict symptom endorsements at mean scores and commonly used cut-points.

RESULTS

The average US physician was likely to endorse feeling he/she is emotionally drained, used up, frustrated, and working too hard and all PA indicators once weekly or more but was unlikely to endorse feeling any DP symptoms once weekly or more. At the commonly used EE and DP cut-points of 27 and 10, respectively, a physician was unlikely to endorse feeling burned out or any DP symptoms once weekly or more. Each subscale assessed the majority of sample score ranges with ≥ 0.70 reliability.

CONCLUSIONS

We produced a crosswalk mapping raw MBI subscale scores to scaled scores and response profiles calibrated in a US physician sample. Our results can be used to better understand the meaning and precision of MBI scores in US physicians; compare individual/group MBI scores against a reference population of US physicians; and inform the selection of subscale cut-points for defining categorical physician burnout outcomes.

Comparative Accuracy of Developmental Screening Questionnaires

Importance

Universal developmental screening is widely recommended, yet studies of the accuracy of commonly used questionnaires reveal mixed results, and previous comparisons of these questionnaires are hampered by important methodological differences across studies.

Objective

To compare the accuracy of 3 developmental screening instruments as standardized tests of developmental status.

Design, Setting, and Participants

This cross-sectional diagnostic accuracy study recruited consecutive parents in waiting rooms at 10 pediatric primary care offices in eastern Massachusetts between October 1, 2013, and January 31, 2017. Parents were included if they were sufficiently literate in the English or Spanish language to complete a packet of screening questionnaires and if their child was of eligible age. Parents completed all questionnaires in counterbalanced order. Participants who screened positive on any questionnaire plus 10% of those who screened negative on all questionnaires (chosen at random) were invited to complete developmental testing. Analyses were weighted for sampling and nonresponse and were conducted from October 1, 2013, to January 31, 2017.

Exposures

The 3 screening instruments used were the Ages & Stages Questionnaire, Third Edition (ASQ-3); Parents' Evaluation of Developmental Status (PEDS); and Survey of Well-being of Young Children (SWYC): Milestones.

Main Outcomes and Measures

Reference tests administered were Bayley Scales of Infant and Toddler Development, Third Edition, for children aged 0 to 42 months, and Differential Ability Scales, Second Edition, for older children. Age-standardized scores were used as indicators of mild (80-89), moderate (70-79), or severe (<70) delays.

Results

A total of 1495 families of children aged 9 months to 5.5 years participated. The mean (SD) age of the children at enrollment was 2.6 (1.3) years, and 779 (52.1%) were male. Parent respondents were primarily female (1325 [88.7%]), with a mean (SD) age of 33.4 (6.3) years. Of the 20.5% to 29.0% of children with a positive score on each questionnaire, 35% to 60% also received a positive score on a second questionnaire, demonstrating moderate co-occurrence. Among younger children (<42 months), the specificity of the ASQ-3 (89.4%; 95% CI, 85.9%-92.1%) and SWYC Milestones (89.0%; 95% CI, 86.1%-91.4%) was higher than that of the PEDS (79.6%; 95% CI, 75.7%-83.1%; P < .001 and P = .002, respectively), but differences in sensitivity were not statistically significant. Among older children (43-66 months), specificity of the ASQ-3 (92.1%; 95% CI, 85.1%-95.9%) was higher than that of the SWYC Milestones (70.7%; 95% CI, 60.9%-78.8%) and the PEDS (73.7%; 95% CI, 64.3%-81.3%; P < .001), but sensitivity to mild delays of the SWYC Milestones (54.8%; 95% CI, 38.1%-70.4%) and of the PEDS (61.8%; 95% CI, 43.1%-77.5%) was higher than that of the ASQ-3 (23.5%; 95% CI, 9.0%-48.8%; P = .012 and P = .002, respectively). Sensitivity exceeded 70% only with respect to severe delays, with 73.7% (95% CI, 50.1%-88.6%) for the SWYC Milestones among younger children, 78.9% (95% CI, 55.4%-91.9%) for the PEDS among younger children, and 77.8% (95% CI, 41.8%-94.5%) for the PEDS among older children. Attending to parents' concerns was associated with increased sensitivity of all questionnaires.

Conclusions and Relevance

This study found that 3 frequently used screening questionnaires offer adequate specificity but modest sensitivity for detecting developmental delays among children aged 9 months to 5 years. The results suggest that trade-offs in sensitivity and specificity occurred among the questionnaires, with no one questionnaire emerging superior overall.

Multi-method process maps: An interdisciplinary approach to investigate ad hoc modifications in protocol-driven interventions

INTRODUCTION

Implementation scientists increasingly recognize that the process of implementation is dynamic, leading to ad hoc modifications that may challenge fidelity in protocol-driven interventions. However, limited attention to ad hoc modifications impairs investigators' ability to develop evidence-based hypotheses about how such modifications may impact intervention effectiveness and cost. We propose a multi-method process map methodology to facilitate the systematic data collection necessary to characterize ad hoc modifications that may impact primary intervention outcomes.

METHODS

We employ process maps (drawn from systems science), as well as focus groups and semi-structured interviews (drawn from social sciences) to investigate ad hoc modifications. Focus groups are conducted with the protocol's developers and/or planners (the implementation team) to characterize the protocol "as envisioned," while interviews conducted with frontline administrators characterize the process "as realized in practice." Process maps with both samples are used to identify when modifications occurred across a protocol-driven intervention. A case study investigating a multistage screening protocol for autism spectrum disorders (ASD) is presented to illustrate application and utility of the multi-method process maps.

RESULTS

In this case study, frontline administrators reported ad hoc modifications that potentially influenced the primary study outcome (e.g., time to ASD diagnosis). Ad hoc modifications occurred to accommodate (1) whether providers and/or parents were concerned about ASD, (2) perceptions of parental readiness to discuss ASD, and (3) perceptions of family service delivery needs and priorities.

CONCLUSION

Investigation of ad hoc modifications on primary outcomes offers new opportunities to develop empirically based adaptive interventions. Routine reporting standards are critical to provide full transparency when studying ad hoc modifications.

Selecting physician well-being measures to assess health system performance and screen for distress: Conceptual and methodological considerations

Following national calls to address burnout among U.S. physicians, healthcare systems across the nation are integrating measures of physician well-being into institutional assessments. In this paper, we review important conceptual and methodological considerations for selecting self-reported physician well-being measures to monitor health system performance and to screen individual physicians for symptoms of distress. First, we discuss the importance of selecting any given measure of physician well-being based on the degree to which evidence supports the validity of the measure within the context of its intended use. Second, we present a conceptual model explaining the relationship between physician well-being and the larger healthcare context, to assist health systems in identifying the intended goals of physician well-being assessment. Well-being assessments are metrics of individual-level physician wellness/distress and may be indicators of system-level performance. We highlight proposed roles of physician well-being as a performance metric (i.e., as a downstream effect of the medical practice environment, as a predictor of health system outcomes, and as a mediator of the practice environment's effect on health system outcomes). Using this framework, we review the evidence supporting the validity of four of the most commonly used measures of well-being in U.S. physicians, identify gaps in the literature, and present practical recommendations for healthcare organizations' selection of appropriate measurement tools. We conclude by offering directions for future research to advance the measurement of physician well-being outcomes.

Establishing New Norms for Developmental Milestones

BACKGROUND AND OBJECTIVES

Pediatric surveillance of young children depends on providers' assessment of developmental milestones, yet normative data are sparse. Our objectives were to develop new norms for common milestones to aid in clinical interpretation of milestone attainment.

METHODS

We analyzed responses to the developmental screening form of the Survey of Well-being of Young Children from 41 465 screens across 3 states. Associations between developmental status and a range of child characteristics were analyzed, and norms for individual questions were compared to guidelines regarding attainment of critical milestones from the Centers for Disease Control and Prevention (CDC).

RESULTS

A contemporary resource of normative data for developmental milestone attainment was established. Lower developmental status was associated with child age in the presence of positive behavioral screening scores (P < .01), social determinants of health (P < .01), Medicaid (P < .01), male sex (P < .01), and child race (P < .01). Comparisons between Survey of Well-being of Young Children developmental questions and CDC guidelines reveal that a high percentage of children are reported to pass milestones by the age at which the CDC states that "most children pass" and that an even higher percentage of children are reported to pass milestones by the age at which the CDC states that parents should "act early." An interactive data visualization tool that can assist clinicians in real-time developmental screening and surveillance interpretation is also provided.

CONCLUSIONS

Detailed normative data on individual developmental milestones can help clinicians guide caregivers' expectations for milestone attainment, thereby offering greater specificity to CDC guidelines.

Establishing New Norms for Developmental Milestones

BACKGROUND AND OBJECTIVES

Pediatric surveillance of young children depends on providers' assessment of developmental milestones, yet normative data are sparse. Our objectives were to develop new norms for common milestones to aid in clinical interpretation of milestone attainment.

METHODS

We analyzed responses to the developmental screening form of the Survey of Well-being of Young Children from 41 465 screens across 3 states. Associations between developmental status and a range of child characteristics were analyzed, and norms for individual questions were compared to guidelines regarding attainment of critical milestones from the Centers for Disease Control and Prevention (CDC).

RESULTS

A contemporary resource of normative data for developmental milestone attainment was established. Lower developmental status was associated with child age in the presence of positive behavioral screening scores (P < .01), social determinants of health (P < .01), Medicaid (P < .01), male sex (P < .01), and child race (P < .01). Comparisons between Survey of Well-being of Young Children developmental questions and CDC guidelines reveal that a high percentage of children are reported to pass milestones by the age at which the CDC states that "most children pass" and that an even higher percentage of children are reported to pass milestones by the age at which the CDC states that parents should "act early." An interactive data visualization tool that can assist clinicians in real-time developmental screening and surveillance interpretation is also provided.

CONCLUSIONS

Detailed normative data on individual developmental milestones can help clinicians guide caregivers' expectations for milestone attainment, thereby offering greater specificity to CDC guidelines.

State-Level Trends in the Prevalence of Autism Spectrum Disorder (ASD) from 2000 to 2012: A Reanalysis of Findings from the Autism and Developmental Disabilities Network

Since 2000, the Autism and Developmental Disabilities Network (ADDM) has published detailed prevalence estimates for autism spectrum disorder (ASD) among 8 year-olds, which are widely interpreted as the U.S. national prevalence of ASD. Although differences in state-level ASD prevalence has been reported, state-level heterogeneity has not been explored systematically. We analyzed state-level estimates and trends in ASD prevalence from 2000 to 2012 using secondary data from bi-annual ADDM reports. Heterogeneity among state-level ASD prevalence estimates were apparent in 2000 and grew between 2000 and 2012. Findings highlight the need for greater understanding of how children with ASD are identified by the medical and educational systems, which has significant implications for the state-level resources required to effectively manage ASD.

What Drives Detection and Diagnosis of Autism Spectrum Disorder? Looking Under the Hood of a Multi-stage Screening Process in Early Intervention

U.S. guidelines for detecting autism emphasize screening and also incorporate clinical judgment. However, most research focuses on the former. Among 1,654 children participating in a multi-stage screening protocol for autism, we used mixed methods to evaluate: (1) the effectiveness of a clinical decision rule that encouraged further assessment based not only on positive screening results, but also on parent or provider concern, and (2) the influence of shared decision-making on screening administration. Referrals based on concern alone were cost-effective in the current study, and reported concerns were stronger predictors than positive screens of time-to-complete referrals. Qualitative analyses suggest a dynamic relationship between parents' concerns, providers' concerns, and screening results that is central to facilitating shared decision-making and influencing diagnostic assessment.

Measuring the Impact of Burn Injury on the Parent-Reported Health Outcomes of Children 1 to 5 Years: A Conceptual Framework for Development of the Preschool Life Impact Burn Recovery Evaluation Profile CAT

Due to the rapid developmental growth in preschool-aged children, more precise measurement of the effects of burns on child health outcomes is needed. Expanding upon the Shriners Hospitals for Children/American Burn Association Burn Outcome Questionnaire 0 to 5 (BOQ0–5), we developed a conceptual framework describing domains important in assessing recovery from burn injury among preschool-aged children (1–5 years). We developed a working conceptual framework based on the BOQ0–5, the National Research Council and Institute of Medicine’s Model of Child Health, and the World Health Organization’s International Classification of Functioning, Disability, and Health for Children and Youth. We iteratively refined our framework based on a literature review, focus groups, interviews, and expert consensus meetings. Data were qualitatively analyzed using methods informed by grounded theory. We reviewed 95 pediatric assessments, conducted two clinician focus groups and six parent interviews, and consulted with 23 clinician experts. Three child health outcome domains emerged from our analysis: symptoms, functioning, and family. The symptoms domain describes parents’ perceptions of their child’s pain, skin-related discomfort, and fatigue. The functioning domain describes children’s physical functioning (gross and fine motor function), psychological functioning (internalizing, externalizing, and dysregulation behavior; trauma; toileting; resilience), communication and language development (receiving and producing meaning), and social functioning (connecting with family/peers, friendships, and play). The family domain describes family psychological and routine functioning outcomes.

Colocated Mental Health/Developmental Care

The inadequacy of mental health and developmental services for children is a widely recognized and growing problem. Although a variety of solutions have been proposed, none has been generally successful or feasible. This research describes models of colocation that have evolved in primary care settings in Massachusetts and reports on pediatricians' and their colocated colleagues' impressions of their benefits and challenges. Pediatricians in 18 practices that included a colocated mental health/developmental specialist (MH/DS) were identified through a survey administered through the state American Academy of Pediatrics Chapter, and interviewed. Practices varied widely in the professional expertise/training and roles of the MH/DSs, communication among providers, and financial arrangements. The majority of pediatricians and MH/DSs reported being pleased with their colocated arrangements, despite the costs rarely being supported by billing revenues. This study suggests that further development of such systems hold promise to meet the growing need for accessible pediatric mental and developmental health care.

Is a Positive Developmental-Behavioral Screening Score Sufficient to Justify Referral? A Review of Evidence and Theory

In their recommendations on screening for autism and developmental disabilities, the American Academy of Pediatrics recommends referral subsequent to a positive screening result. In this article, we argue that positive screening results are not always sufficient to justify a referral. We show that although positive predictive values are often low, they actually overstate the probability of having a disorder for many children who screen positive. Moreover, recommended screening thresholds are seldom set to ensure that the benefits of referral will equal or exceed the costs and risk of harm, which is a necessary condition for an optimal threshold in decision analysis. Drawing on recent recommendations for the Institute of Medicine/National Academy of Medicine, we discuss the implications of this argument for pediatric policy, education, and practice. In particular, we recommend that screening policies be revised to ensure that the costs and benefits of actions recommended in the event of a positive screen are appropriate to the screening threshold. We recommend greater focus on clinical decision-making in the education of physicians, including shared decision-making with patients and their families. Finally, we recommend broadening the scope of screening research to encompass not only the accuracy of specific screening instruments, but also their ability to improve decision-making in the context of systems of care.

Age at First Identification of Autism Spectrum Disorder: An Analysis of Two US Surveys

OBJECTIVE

Evidence regarding the age at which autism spectrum disorder (ASD) is identified is essential for improving early detection, yet many extant studies have not applied time-to-event analyses, which account for statistical biases that arise from sampling in cross-sectional surveys by adjusting for child age at time of parental report. Our objective was to estimate age distributions for first identification of ASD in national parent surveys using time-to-event analyses.

METHOD

We conducted time-to-event analyses of responses to identical questions in the 2011 to 2012 National Survey of Children's Health (n = 95,677) and the 2009 to 2010 National Survey of Children with Special Health Care Needs (n = 371,617).

RESULTS

Parents in both surveys reported that a minority of ASD cases were identified before age 3 years, and that one-third to one-half of cases were identified after 6 years. In both surveys, a majority of parents described their child's ASD severity as mild, and these parents reported the oldest age at identification (mean, 5.6 and 8.6 years). In contrast, parents who described their child's ASD as severe reported earlier age at identification (mean, 3.7 and 4.5 years). Time-to-event analyses yielded older estimates of age at identification than analyses based on raw distributions.

CONCLUSION

In two national surveys, a majority of parents of children with ASD reported identification after 3 years, when eligibility for early intervention services expires, and many reported identification of ASD after school age. Later identification of children with milder symptoms highlights the need for early screening that is sensitive to all forms of ASD, regardless of severity.

A system dynamics model of clinical decision thresholds for the detection of developmental-behavioral disorders

Background

Clinical decision-making has been conceptualized as a sequence of two separate processes: assessment of patients’ functioning and application of a decision threshold to determine whether the evidence is sufficient to justify a given decision. A range of factors, including use of evidence-based screening instruments, has the potential to influence either or both processes. However, implementation studies seldom specify or assess the mechanism by which screening is hypothesized to influence clinical decision-making, thus limiting their ability to address unexpected findings regarding clinicians’ behavior. Building on prior theory and empirical evidence, we created a system dynamics (SD) model of how physicians’ clinical decisions are influenced by their assessments of patients and by factors that may influence decision thresholds, such as knowledge of past patient outcomes. Using developmental-behavioral disorders as a case example, we then explore how referral decisions may be influenced by changes in context. Specifically, we compare predictions from the SD model to published implementation trials of evidence-based screening to understand physicians’ management of positive screening results and changes in referral rates. We also conduct virtual experiments regarding the influence of a variety of interventions that may influence physicians’ thresholds, including improved access to co-located mental health care and improved feedback systems regarding patient outcomes.

Results

Results of the SD model were consistent with recent implementation trials. For example, the SD model suggests that if screening improves physicians’ accuracy of assessment without also influencing decision thresholds, then a significant proportion of children with positive screens will not be referred and the effect of screening implementation on referral rates will be modest—results that are consistent with a large proportion of published screening trials. Consistent with prior theory, virtual experiments suggest that physicians’ decision thresholds can be influenced and detection of disabilities improved by increasing access to referral sources and enhancing feedback regarding false negative cases.

Conclusions

The SD model of clinical decision-making offers a theoretically based framework to improve understanding of physicians’ behavior and the results of screening implementation trials. The SD model is also useful for initial testing of hypothesized strategies to increase detection of under-identified medical conditions.

Electronic supplementary material

The online version of this article (doi:10.1186/s13012-016-0517-0) contains supplementary material, which is available to authorized users.

Adolescent Perceptions of Cholesterol Screening Results: "Young Invincibles" or Developing Adults?

PURPOSE

Guidelines recommend cholesterol screening for all adolescents and young adults (AYAs) ages 17-21 years. Little is known about how screening results impact perceptions of AYA health.

METHODS

We recruited 37 AYAs and 35 parents of AYAs with differing risk for abnormal cholesterol results: (1) familial hypercholesterolemia; (2) obesity; and (3) generally healthy. Participants completed quantitative health status ratings using visual analog scales (VASs) and semistructured interviews regarding three hypothetical cholesterol screening scenarios: (1) high likelihood of cardiovascular disease (CVD) before age 40 years ("high risk"); (2) some risk of CVD before age 70 years ("moderate risk"); and (3) low risk for CVD despite a strong family history of CVD ("low risk"). We analyzed VAS data with logistic regression and qualitative data with a priori and emergent coding using multiple coders.

RESULTS

Each group perceived all three cholesterol screening scenarios as comparatively less than perfect health; the high-risk result fell furthest from perfect health. Although there was no significant difference between AYAs and parents in VAS ratings, qualitative analyses revealed AYAs were more likely than parents to discount the impact of moderate-risk results because of longer length of time before predicted CVD.

CONCLUSIONS

AYAs' and parents' perceptions of the impact of cholesterol screening results on AYA health varied by presented scenario, ranging from mild to significant decreases in perceptions of AYA health. As universal cholesterol screening continues to be adopted in this age group, further studies of the real-life impact on AYA risk perceptions and subsequent behavior are warranted.

Thresholds and accuracy in screening tools for early detection of psychopathology

BACKGROUND

The accuracy of any screening instrument designed to detect psychopathology among children is ideally assessed through rigorous comparison to 'gold standard' tests and interviews. Such comparisons typically yield estimates of what we refer to as 'standard indices of diagnostic accuracy', including sensitivity, specificity, positive predictive value (PPV), and negative predictive value. However, whereas these statistics were originally designed to detect binary signals (e.g., diagnosis present or absent), screening questionnaires commonly used in psychology, psychiatry, and pediatrics typically result in ordinal scores. Thus, a threshold or 'cut score' must be applied to these ordinal scores before accuracy can be evaluated using such standard indices. To better understand the tradeoffs inherent in choosing a particular threshold, we discuss the concept of 'threshold probability'. In contrast to PPV, which reflects the probability that a child whose score falls at or above the screening threshold has the condition of interest, threshold probability refers specifically to the likelihood that a child whose score is equal to a particular screening threshold has the condition of interest.

METHOD

The diagnostic accuracy and threshold probability of two well-validated behavioral assessment instruments, the Child Behavior Checklist Total Problem Scale and the Strengths and Difficulties Questionnaire total scale were examined in relation to a structured psychiatric interview in three de-identified datasets.

RESULTS

Although both screening measures were effective in identifying groups of children at elevated risk for psychopathology in all samples (odds ratios ranged from 5.2 to 9.7), children who scored at or near the clinical thresholds that optimized sensitivity and specificity were unlikely to meet criteria for psychopathology on gold standard interviews.

CONCLUSIONS

Our results are consistent with the view that screening instruments should be interpreted probabilistically, with attention to where along the continuum of positive scores an individual falls.