1
|
Pillai M, Posada J, Gardner RM, Hernandez-Boussard T, Bannett Y. Measuring quality-of-care in treatment of young children with attention-deficit/hyperactivity disorder using pre-trained language models. J Am Med Inform Assoc 2024; 31:949-957. [PMID: 38244997 PMCID: PMC10990536 DOI: 10.1093/jamia/ocae001] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/26/2023] [Revised: 12/07/2023] [Accepted: 01/03/2024] [Indexed: 01/22/2024] Open
Abstract
OBJECTIVE To measure pediatrician adherence to evidence-based guidelines in the treatment of young children with attention-deficit/hyperactivity disorder (ADHD) in a diverse healthcare system using natural language processing (NLP) techniques. MATERIALS AND METHODS We extracted structured and free-text data from electronic health records (EHRs) of all office visits (2015-2019) of children aged 4-6 years in a community-based primary healthcare network in California, who had ≥1 visits with an ICD-10 diagnosis of ADHD. Two pediatricians annotated clinical notes of the first ADHD visit for 423 patients. Inter-annotator agreement (IAA) was assessed for the recommendation for the first-line behavioral treatment (F-measure = 0.89). Four pre-trained language models, including BioClinical Bidirectional Encoder Representations from Transformers (BioClinicalBERT), were used to identify behavioral treatment recommendations using a 70/30 train/test split. For temporal validation, we deployed BioClinicalBERT on 1,020 unannotated notes from other ADHD visits and well-care visits; all positively classified notes (n = 53) and 5% of negatively classified notes (n = 50) were manually reviewed. RESULTS Of 423 patients, 313 (74%) were male; 298 (70%) were privately insured; 138 (33%) were White; 61 (14%) were Hispanic. The BioClinicalBERT model trained on the first ADHD visits achieved F1 = 0.76, precision = 0.81, recall = 0.72, and AUC = 0.81 [0.72-0.89]. Temporal validation achieved F1 = 0.77, precision = 0.68, and recall = 0.88. Fairness analysis revealed low model performance in publicly insured patients (F1 = 0.53). CONCLUSION Deploying pre-trained language models on a variable set of clinical notes accurately captured pediatrician adherence to guidelines in the treatment of children with ADHD. Validating this approach in other patient populations is needed to achieve equitable measurement of quality of care at scale and improve clinical care for mental health conditions.
Collapse
Affiliation(s)
- Malvika Pillai
- Department of Biomedical Data Science, Stanford University School of Medicine, Stanford, CA 94305, United States
| | - Jose Posada
- Computer Science Department, University of the North, Barranquilla 080020, Colombia
| | - Rebecca M Gardner
- Department of Epidemiology and Population Health, Stanford University School of Medicine, Stanford, CA 94305, United States
| | - Tina Hernandez-Boussard
- Department of Biomedical Data Science, Stanford University School of Medicine, Stanford, CA 94305, United States
| | - Yair Bannett
- Division of Developmental-Behavioral Pediatrics, Department of Pediatrics, Stanford University School of Medicine, Stanford, CA 94304, United States
| |
Collapse
|
2
|
Roberts MD, Loubeau JK, Hasan S, Rabin M, Sikov J, Baul TD, Brigham R, Melissa G, Singh R, Cassidy K, Spencer AE. A Quality Improvement Initiative for Detection of Attention-Deficit/Hyperactivity Disorder in an Urban, Academic Safety Net Hospital. J Dev Behav Pediatr 2024; 45:e121-e128. [PMID: 38552001 PMCID: PMC11141212 DOI: 10.1097/dbp.0000000000001257] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 04/16/2024]
Abstract
OBJECTIVE Improve detection of Attention Deficit/Hyperactivity Disorder (ADHD) in a safety net, hospital-based, academic pediatric practice by optimizing screening with the Pediatric Symptom Checklist attention score (PSC-AS) and further evaluation with the Vanderbilt ADHD Diagnostic Rating Scale (VADRS). METHODS We implemented a multi-component intervention by (1) optimizing electronic medical record (EMR) features; (2) adjusting clinic operational workflow; and (3) creating a decision-making algorithm for pediatric primary care clinicians (PPCCs). We extracted 4 outcomes manually from the EMR (pediatrician acknowledgment of a positive PSC-AS, documentation of a plan for further evaluation, distribution of VADRS, and completion of at least 1 VADRS). Outcomes were measured monthly in run charts compared to the pre-intervention control period, and implementation was optimized with Plan-Do-Study-Act cycles. RESULTS PPCCs were significantly more likely to acknowledge a positive PSC-AS in the intervention versus control (65.3% vs 41.5%; p < 0.001), although this did not change documentation of a plan (70% vs 67.1%; p -value = 0.565). Significantly more children with a positive PSC-AS were distributed a parent or teacher VADRS in the intervention versus control (30.6% vs 17.7%; p -value = 0.0059), but the percentage of returned VADRS rating scales did not improve (12.9% vs 9.2%; p -value = 0.269). CONCLUSION Our ADHD detection quality improvement initiative improved use of the PSC-AS to identify attention problems and distribution of VADRS diagnostic rating scales, but additional interventions are needed to improve the completion of ADHD evaluations in primary care to ensure that children are appropriately identified and offered evidence-based care.
Collapse
Affiliation(s)
- Mona Doss Roberts
- Department of Pediatrics, Boston University Chobanian & Avedisian School of Medicine, Boston Medical Center, Boston, MA, USA
| | - J. Krystel Loubeau
- Department of Psychiatry, Boston University Chobanian & Avedisian School of Medicine, Boston Medical Center, Boston, MA, USA
| | - Syeda Hasan
- Department of Psychiatry, Boston University Chobanian & Avedisian School of Medicine, Boston Medical Center, Boston, MA, USA
| | - Megan Rabin
- Department of Psychiatry, Boston University Chobanian & Avedisian School of Medicine, Boston Medical Center, Boston, MA, USA
| | - Jennifer Sikov
- Department of Psychology, Florida International University, Miami, FL
| | - Tithi D. Baul
- Department of Psychiatry, Boston University Chobanian & Avedisian School of Medicine, Boston Medical Center, Boston, MA, USA
| | - Rebecca Brigham
- Department of Psychiatry, Boston University Chobanian & Avedisian School of Medicine, Boston Medical Center, Boston, MA, USA
| | - Gillooly Melissa
- Department of Pediatrics, Boston University Chobanian & Avedisian School of Medicine, Boston Medical Center, Boston, MA, USA
| | - Ruby Singh
- Massachusetts General Hospital, Boston, MA, USA
| | | | - Andrea E. Spencer
- Department of Psychiatry, Boston University Chobanian & Avedisian School of Medicine, Boston Medical Center, Boston, MA, USA
- Pritzker Department of Psychiatry and Behavioral Health, Ann & Robert H. Lurie Children’s Hospital of Chicago, Chicago, IL
| |
Collapse
|
3
|
Golson ME, Brunson McClain M, O'Dell SM, Gormley MJ, Roanhorse TT, Yang NJ, Kettlewell P, Shahidullah JD. Assessment and Management of Attention-Deficit/Hyperactivity Disorder: Pediatric Resident Perspectives on Training and Practice. Clin Pediatr (Phila) 2023; 62:1513-1522. [PMID: 36995005 DOI: 10.1177/00099228231163687] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 03/31/2023]
Abstract
Attention-deficit/hyperactivity disorder (ADHD) is a common presenting concern in primary care. This study examined the relationship between pediatric residency training program characteristics and residents' ADHD knowledge, attitudes, and comfort in providing ADHD services. Given the familiarity that pediatric chief residents have with the training and experiences within their residency programs, a 30-item survey was mailed to pediatric chief residents. A total of 100 residents returned their surveys (response rate 49.5%) and were included in the descriptive quantitative and thematic qualitative analyses. The majority of participants rated their ADHD knowledge as at least average. However, approximately half of the participants were comfortable with screening, and less than half were comfortable with managing stimulant medication or behavioral treatments. Participants emphasized the importance of interprofessional collaboration, clinical experiences, and integrated ADHD education throughout training. These results emphasize the importance of improved training in screening, diagnosing, and managing ADHD to increase resident comfort regarding these practices.
Collapse
Affiliation(s)
- Megan E Golson
- Department of Psychology, Utah State University, Logan, UT, USA
| | | | | | | | | | - Nai-Jiin Yang
- Department of Psychology, Utah State University, Logan, UT, USA
| | | | | |
Collapse
|
4
|
Laur C, Ladak Z, Hall A, Solbak NM, Nathan N, Buzuayne S, Curran JA, Shelton RC, Ivers N. Sustainability, spread, and scale in trials using audit and feedback: a theory-informed, secondary analysis of a systematic review. Implement Sci 2023; 18:54. [PMID: 37885018 PMCID: PMC10604689 DOI: 10.1186/s13012-023-01312-0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/19/2023] [Accepted: 10/05/2023] [Indexed: 10/28/2023] Open
Abstract
BACKGROUND Audit and feedback (A&F) is a widely used implementation strategy to influence health professionals' behavior that is often tested in implementation trials. This study examines how A&F trials describe sustainability, spread, and scale. METHODS This is a theory-informed, descriptive, secondary analysis of an update of the Cochrane systematic review of A&F trials, including all trials published since 2011. Keyword searches related to sustainability, spread, and scale were conducted. Trials with at least one keyword, and those identified from a forward citation search, were extracted to examine how they described sustainability, spread, and scale. Results were qualitatively analyzed using the Integrated Sustainability Framework (ISF) and the Framework for Going to Full Scale (FGFS). RESULTS From the larger review, n = 161 studies met eligibility criteria. Seventy-eight percent (n = 126) of trials included at least one keyword on sustainability, and 49% (n = 62) of those studies (39% overall) frequently mentioned sustainability based on inclusion of relevant text in multiple sections of the paper. For spread/scale, 62% (n = 100) of trials included at least one relevant keyword and 51% (n = 51) of those studies (31% overall) frequently mentioned spread/scale. A total of n = 38 studies from the forward citation search were included in the qualitative analysis. Although many studies mentioned the need to consider sustainability, there was limited detail on how this was planned, implemented, or assessed. The most frequent sustainability period duration was 12 months. Qualitative results mapped to the ISF, but not all determinants were represented. Strong alignment was found with the FGFS for phases of scale-up and support systems (infrastructure), but not for adoption mechanisms. New spread/scale themes included (1) aligning affordability and scalability; (2) balancing fidelity and scalability; and (3) balancing effect size and scalability. CONCLUSION A&F trials should plan for sustainability, spread, and scale so that if the trial is effective, the benefits can continue. A deeper empirical understanding of the factors impacting A&F sustainability is needed. Scalability planning should go beyond cost and infrastructure to consider other adoption mechanisms, such as leadership, policy, and communication, that may support further scalability. TRIAL REGISTRATION Registered with Prospero in May 2022. CRD42022332606.
Collapse
Affiliation(s)
- Celia Laur
- Women's College Hospital Institute for Health System Solutions and Virtual Care, 76 Grenville Street, Toronto, ON, M5S 1B2, Canada.
- Institute of Health Policy, Management and Evaluation, Health Sciences Building, University of Toronto, 155 College Street, Suite 425, Toronto, ON, M5T 3M6, Canada.
| | - Zeenat Ladak
- Women's College Hospital Institute for Health System Solutions and Virtual Care, 76 Grenville Street, Toronto, ON, M5S 1B2, Canada
- Ontario Institute for Studies in Education, University of Toronto, 252 Bloor Street West, Toronto, ON, M5S 1V6, Canada
| | - Alix Hall
- School of Medicine and Public Health, The University of Newcastle, Newcastle, NSW, Australia
- National Centre of Implementation Science, The University of Newcastle, Newcastle, NSW, Australia
- Hunter Medical Research Institute, New Lambton Heights, NSW, Australia
- Hunter New England Population Health, Hunter New England Local Health District, Newcastle, NSW, Australia
| | - Nathan M Solbak
- Physician Learning Program, Continuing Medical Education and Professional Development, Cumming School of Medicine, University of Calgary, 3280 Hospital Drive NW, Calgary, Alberta, T2N 4Z6, Canada
- Health Quality Programs, Queen's University, 92 Barrie Street, Kingston, ON, K7L 3N6, Canada
| | - Nicole Nathan
- School of Medicine and Public Health, The University of Newcastle, Newcastle, NSW, Australia
- National Centre of Implementation Science, The University of Newcastle, Newcastle, NSW, Australia
- Hunter Medical Research Institute, New Lambton Heights, NSW, Australia
- Hunter New England Population Health, Hunter New England Local Health District, Newcastle, NSW, Australia
| | - Shewit Buzuayne
- Women's College Hospital Institute for Health System Solutions and Virtual Care, 76 Grenville Street, Toronto, ON, M5S 1B2, Canada
| | - Janet A Curran
- School of Nursing, Faculty of Health, Dalhousie University, Halifax, NS, B3H 4R2, Canada
| | - Rachel C Shelton
- Department of Sociomedical Sciences, Mailman School of Public Health, Columbia University, New York, NY, USA
| | - Noah Ivers
- Women's College Hospital Institute for Health System Solutions and Virtual Care, 76 Grenville Street, Toronto, ON, M5S 1B2, Canada
- Institute of Health Policy, Management and Evaluation, Health Sciences Building, University of Toronto, 155 College Street, Suite 425, Toronto, ON, M5T 3M6, Canada
- Department of Family and Community Medicine, University of Toronto, 500 University Ave, Toronto, M5G 1V7, Canada
| |
Collapse
|
5
|
O'Dell SM, Gormley MJ, Schlieder V, Klinger T, DeHart K, Kettlewell PW, Kulchak Rahm A. Perspectives of Rural Primary Care Clinicians on Pediatric Attention-Deficit/Hyperactivity Disorder Care. J Dev Behav Pediatr 2022; 43:273-282. [PMID: 35353792 DOI: 10.1097/dbp.0000000000001056] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 03/31/2021] [Accepted: 11/15/2021] [Indexed: 11/25/2022]
Abstract
OBJECTIVE Despite efficacious treatments, evidence-based guidelines, and increased availability of integrated behavioral health care, youth coping with attention-deficit/hyperactivity disorder (ADHD) receive suboptimal care. More research is needed to understand and address care gaps, particularly within rural health systems that face unique challenges. We conducted a qualitative study within a predominantly rural health system with a pediatric-integrated behavioral health care program to address research gaps and prepare for quality improvement initiatives, including primary care clinician (PCC) trainings and clinical decision support tools in the electronic health record (EHR). METHOD Semistructured interviews were conducted with 26 PCCs representing clinics within the health system. Interview guides were based on the Practical Robust Implementation and Sustainability Model to elicit PCC views regarding determinants of current practices and suggestions to guide quality improvement efforts. We used thematic analysis to identify patterns of responding that were common across participants. RESULTS PCCs identified several internal and external contextual factors as determinants of current practices. Of note, PCCs recommended increased access to continuing education trainings held in clinic over lunch and delivered in less than 30 minutes. Suggested improvements to the EHR included incorporating parent and teacher versions of the Vanderbilt Rating Scale into the EHR, documentation templates aligned with evidence-based guidelines, and alerts and suggestions to aid medication management during appointments. CONCLUSION Future research to identify implementation strategies to help rural PCCs adopt innovations are needed given the increased responsibility for managing ADHD care and intractable gaps in access to behavioral health care in rural regions.
Collapse
Affiliation(s)
- Sean M O'Dell
- Departments of Population Health Sciences
- Psychiatry and Behavioral Health, Geisinger, Danville, PA
| | - Matthew J Gormley
- Psychiatry and Behavioral Health, Geisinger, Danville, PA
- Department of Educational Psychology, University of Nebraska-Lincoln, Lincoln, NE
| | | | - Tracey Klinger
- Investigator Initiated Research Operations, Geisinger, Danville, PA
| | - Kathy DeHart
- Department of Pediatrics, Geisinger, Danville, PA
| | | | | |
Collapse
|
6
|
Lakes KD, Cibrian FL, Schuck S, Nelson M, Hayes GR. Digital health interventions for youth with ADHD: A systematic review. COMPUTERS IN HUMAN BEHAVIOR REPORTS 2022. [DOI: 10.1016/j.chbr.2022.100174] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/18/2022] Open
|
7
|
Jenssen BP, Thayer J, Nekrasova E, Grundmeier RW, Fiks AG. Innovation in the pediatric electronic health record to realize a more effective platform. Curr Probl Pediatr Adolesc Health Care 2022; 52:101109. [PMID: 34895836 DOI: 10.1016/j.cppeds.2021.101109] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 02/08/2023]
Abstract
Commercial electronic health records (EHRs) were first developed to automate business processes. As EHRs developed, design principles focused on transferring existing paper-based documentation to comparable electronic forms. In addition, a strong industry focus on adult healthcare settings and quality measures has limited attention and resources for high priority EHR functionality needed for the unique health care of children. The objective of this paper is to provide a review of innovation in the EHR, that includes a variety of established and emerging technologies that may help realize a more effective EHR in child health settings. A more effective EHR would serve as an electronic hub. Existing EHR infrastructure could provide the foundation upon which new technologies and approaches branch and extend, enabling more rapid and customizable innovation to better meet shifting stakeholder and end-user needs. Among many areas for improvement, key goals of innovation could include technology that relieves ambulatory primary care clinician documentation burden, identifies needs, and supports improved care coordination and outcomes, focused on the following key areas: identification of child and family care needs, decision support, documentation, care coordination, and family communication.
Collapse
Affiliation(s)
- Brian P Jenssen
- Department of Pediatrics, Perelman School of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania, USA; The Possibilities Project, Center for Pediatric Clinical Effectiveness and PolicyLab, Children's Hospital of Philadelphia (CHOP), USA; Department of Biomedical and Health Informatics, Children's Hospital of Philadelphia, Philadelphia, Pennsylvania, USA.
| | - Jeritt Thayer
- Department of Biomedical and Health Informatics, Children's Hospital of Philadelphia, Philadelphia, Pennsylvania, USA
| | - Ekaterina Nekrasova
- The Possibilities Project, Center for Pediatric Clinical Effectiveness and PolicyLab, Children's Hospital of Philadelphia (CHOP), USA
| | - Robert W Grundmeier
- Department of Pediatrics, Perelman School of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania, USA; The Possibilities Project, Center for Pediatric Clinical Effectiveness and PolicyLab, Children's Hospital of Philadelphia (CHOP), USA; Department of Biomedical and Health Informatics, Children's Hospital of Philadelphia, Philadelphia, Pennsylvania, USA
| | - Alexander G Fiks
- Department of Pediatrics, Perelman School of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania, USA; The Possibilities Project, Center for Pediatric Clinical Effectiveness and PolicyLab, Children's Hospital of Philadelphia (CHOP), USA; Department of Biomedical and Health Informatics, Children's Hospital of Philadelphia, Philadelphia, Pennsylvania, USA
| |
Collapse
|
8
|
Guevara JP, Power TJ, Bevans K, Snitzer L, Leavy S, Stewart D, Broomfield C, Shah S, Grundmeier R, Michel JJ, Berkowitz S, Blum NJ, Bryan M, Griffis H, Fiks AG. Improving Care Management in Attention-Deficit/Hyperactivity Disorder: An RCT. Pediatrics 2021; 148:peds.2020-031518. [PMID: 34281997 DOI: 10.1542/peds.2020-031518] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Accepted: 05/12/2021] [Indexed: 11/24/2022] Open
Abstract
OBJECTIVES To compare the effectiveness of care management combined with a patient portal versus a portal alone for communication among children with attention-deficit/hyperactivity disorder (ADHD). METHODS Randomized controlled trial conducted at 11 primary care practices. Children aged 5 to 12 years old with ADHD were randomly assigned to care management + portal or portal alone. The portal included parent-reported treatment preferences and goals, medication side effects, and parent- and teacher-reported ADHD symptom scales. Care managers provided education to families; communicated quarterly with parents, teachers, and clinicians; and coordinated care. The main outcome, changes in the Vanderbilt Parent Rating Scale (VPRS) score as a measure of ADHD symptoms, was assessed using intention-to-treat analysis. RESULTS A total of 303 eligible children (69% male; 46% Black) were randomly assigned, and 273 (90%) completed the study. During the 9-month study, parents in the care management + portal arm communicated inconsistently with care managers (mean 2.2; range 0-6) but similarly used the portal (mean 2.3 vs 2.2) as parents in the portal alone arm. In multivariate models, VPRS scores decreased over time (Adjusted β = -.015; 95% confidence interval -0.023 to -0.07) in both groups, but there were no intervention-by-time effects (Adjusted β = .000; 95% confidence interval -0.011 to 0.012) between groups. Children who received ≥2 care management sessions had greater reductions in VPRS scores than those with fewer sessions. CONCLUSIONS Results did not provide evidence that care management combined with a patient portal was different from portal use alone among children with ADHD. Both groups demonstrated similar reductions in ADHD symptoms. Those families with greater care management engagement demonstrated greater reductions than those with less engagement.
Collapse
Affiliation(s)
| | - Thomas J Power
- Child and Adolescent Psychiatry and Behavioral Sciences, Children's Hospital of Philadelphia and Perelman School of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania
| | - Katherine Bevans
- Department of Health and Rehabilitation Sciences, College of Public Health, Temple University, Philadelphia, Pennsylvania
| | - Lisa Snitzer
- Mental Health Partnerships, Philadelphia, Pennsylvania
| | | | - Denise Stewart
- City of Philadelphia Water Department, Philadelphia, Pennsylvania
| | | | | | | | | | - Steven Berkowitz
- Department of Psychiatry, University of Colorado, Denver, Colorado
| | | | - Matthew Bryan
- Department of Biostatistics, Epidemiology, and Informatics, Perelman School of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania
| | - Heather Griffis
- Healthcare Analytics Unit, Children's Hospital of Philadelphia, Philadelphia, Pennsylvania
| | | |
Collapse
|
9
|
Mautone JA, Wolk CB, Cidav Z, Davis MF, Young JF. Strategic Implementation Planning for Integrated Behavioral Health Services in Pediatric Primary Care. IMPLEMENTATION RESEARCH AND PRACTICE 2021; 2. [PMID: 34337415 PMCID: PMC8320620 DOI: 10.1177/2633489520987558] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/16/2022] Open
Abstract
Background Delivering physical and behavioral health services in a single setting is associated with improved quality of care and reduced health care costs. Few health systems implementing integrated care develop conceptual models and targeted measurement strategies a priori with an eye toward adoption, implementation, sustainment, and evaluation. This is a broad challenge in the field, which can make it difficult to disentangle why implementation is or is not successful. Method This paper discusses strategic implementation and evaluation planning for a pediatric integrated care program in a large health system. Our team developed a logic model, which defines resources and community characteristics, program components, evaluation activities, short-term activities, and intermediate and anticipated long-term patient-, clinician-, and practice-related outcomes. The model was designed based on research and stakeholder input to support strategic implementation and evaluation of the program. For each aspect of the logic model, a measurement battery was selected. Initial implementation data and intermediate outcomes from a pilot in five practices in a 30-practice pediatric primary care network are presented to illustrate how the logic model and evaluation plan have been used to guide the iterative process of program development. Results A total of 4,619 office visits were completed during the two years of the pilot. Primary care clinicians were highly satisfied with the integrated primary care program and provided feedback on ways to further improve the program. Members of the primary care team and behavioral health providers rated the program as being relatively well integrated into the practices after the second year of the pilot. Conclusions This logic model and evaluation plan provide a template for future projects integrating behavioral health services in non-specialty mental health settings, including pediatric primary care, and can be used broadly to provide structure to implementation and evaluation activities and promote replication of effective initiatives.
Collapse
Affiliation(s)
- Jennifer A Mautone
- Department of Child & Adolescent Psychiatry & Behavioral Sciences, Children's Hospital of Philadelphia.,Department of Psychiatry, Perelman School of Medicine at University of Pennsylvania
| | - Courtney Benjamin Wolk
- Department of Psychiatry, Perelman School of Medicine at University of Pennsylvania.,Leonard Davis Institute for Health Economics at University of Pennsylvania
| | - Zuleyha Cidav
- Department of Psychiatry, Perelman School of Medicine at University of Pennsylvania
| | - Molly F Davis
- Department of Psychiatry, Perelman School of Medicine at University of Pennsylvania.,Penn Implementation Science Center at the Leonard Davis Institute of Health Economics (PISCE@LDI)
| | - Jami F Young
- Department of Child & Adolescent Psychiatry & Behavioral Sciences, Children's Hospital of Philadelphia.,Department of Psychiatry, Perelman School of Medicine at University of Pennsylvania
| |
Collapse
|
10
|
Cotterill S, Tang MY, Powell R, Howarth E, McGowan L, Roberts J, Brown B, Rhodes S. Social norms interventions to change clinical behaviour in health workers: a systematic review and meta-analysis. HEALTH SERVICES AND DELIVERY RESEARCH 2020. [DOI: 10.3310/hsdr08410] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/05/2023] Open
Abstract
Background
A social norms intervention seeks to change the clinical behaviour of a target health worker by exposing them to the values, beliefs, attitudes or behaviours of a reference group or person. These low-cost interventions can be used to encourage health workers to follow recommended professional practice.
Objective
To summarise evidence on whether or not social norms interventions are effective in encouraging health worker behaviour change, and to identify the most effective social norms interventions.
Design
A systematic review and meta-analysis of randomised controlled trials.
Data sources
The following databases were searched on 24 July 2018: Ovid MEDLINE (1946 to week 2 July 2018), EMBASE (1974 to 3 July 2018), Cumulative Index to Nursing and Allied Health Literature (1937 to July 2018), British Nursing Index (2008 to July 2018), ISI Web of Science (1900 to present), PsycINFO (1806 to week 3 July 2018) and Cochrane trials (up to July 2018).
Participants
Health workers took part in the study.
Interventions
Behaviour change interventions based on social norms.
Outcome measures
Health worker clinical behaviour, for example prescribing (primary outcome), and patient health outcomes, for example blood test results (secondary), converted into a standardised mean difference.
Methods
Titles and abstracts were reviewed against the inclusion criteria to exclude any that were clearly ineligible. Two reviewers independently screened the remaining full texts to identify relevant papers. Two reviewers extracted data independently, coded for behaviour change techniques and assessed quality using the Cochrane risk-of-bias tool. We performed a meta-analysis and presented forest plots, stratified by behaviour change technique. Sources of variation were explored using metaregression and network meta-analysis.
Results
A total of 4428 abstracts were screened, 477 full texts were screened and findings were based on 106 studies. Most studies were in primary care or hospitals, targeting prescribing, ordering of tests and communication with patients. The interventions included social comparison (in which information is given on how peers behave) and credible source (which refers to communication from a well-respected person in support of the behaviour). Combined data suggested that interventions that included social norms components were associated with an improvement in health worker behaviour of 0.08 standardised mean differences (95% confidence interval 0.07 to 0.10 standardised mean differences) (n = 100 comparisons), and an improvement in patient outcomes of 0.17 standardised mean differences (95% confidence interval 0.14 to 0.20) (n = 14), on average. Heterogeneity was high, with an overall I
2 of 85.4% (primary) and 91.5% (secondary). Network meta-analysis suggested that three types of social norms intervention were most effective, on average, compared with control: credible source (0.30 standardised mean differences, 95% confidence interval 0.13 to 0.47); social comparison combined with social reward (0.39 standardised mean differences, 95% confidence interval 0.15 to 0.64); and social comparison combined with prompts and cues (0.33 standardised mean differences, 95% confidence interval 0.22 to 0.44).
Limitations
The large number of studies prevented us from requesting additional information from authors. The trials varied in design, context and setting, and we combined different types of outcome to provide an overall summary of evidence, resulting in a very heterogeneous review.
Conclusions
Social norms interventions are an effective method of changing clinical behaviour in a variety of health service contexts. Although the overall result was modest and very variable, there is the potential for social norms interventions to be scaled up to target the behaviour of a large population of health workers and resulting patient outcomes.
Future work
Development of optimised credible source and social comparison behaviour change interventions, including qualitative research on acceptability and feasibility.
Study registration
This study is registered as PROSPERO CRD42016045718.
Funding
This project was funded by the National Institute for Health Research (NIHR) Health Services and Delivery Research programme and will be published in full in Health Services and Delivery Research; Vol. 8, No. 41. See the NIHR Journals Library website for further project information.
Collapse
Affiliation(s)
- Sarah Cotterill
- Centre for Biostatistics, School of Health Sciences, Faculty of Biology Medicine and Health, The University of Manchester, Manchester Academic Health Science Centre, Manchester, UK
| | - Mei Yee Tang
- Centre for Biostatistics, School of Health Sciences, Faculty of Biology Medicine and Health, The University of Manchester, Manchester Academic Health Science Centre, Manchester, UK
| | - Rachael Powell
- Manchester Centre for Health Psychology, Division of Psychology and Mental Health, School of Health Sciences, Faculty of Biology Medicine and Health, The University of Manchester, Manchester Academic Health Science Centre, Manchester, UK
| | - Elizabeth Howarth
- Centre for Biostatistics, School of Health Sciences, Faculty of Biology Medicine and Health, The University of Manchester, Manchester Academic Health Science Centre, Manchester, UK
| | - Laura McGowan
- Manchester Centre for Health Psychology, Division of Psychology and Mental Health, School of Health Sciences, Faculty of Biology Medicine and Health, The University of Manchester, Manchester Academic Health Science Centre, Manchester, UK
| | - Jane Roberts
- Outreach and Evidence Search Service, Library and E-learning Service, Northern Care Alliance, NHS Group, Royal Oldham Hospital, Oldham, UK
| | - Benjamin Brown
- Health e-Research Centre, Farr Institute for Health Informatics Research, Faculty of Biology Medicine and Health, The University of Manchester, Manchester Academic Health Science Centre, Manchester, UK
- Centre for Primary Care, School of Health Sciences, Faculty of Biology Medicine and Health, The University of Manchester, Manchester Academic Health Science Centre, Manchester, UK
| | - Sarah Rhodes
- Centre for Biostatistics, School of Health Sciences, Faculty of Biology Medicine and Health, The University of Manchester, Manchester Academic Health Science Centre, Manchester, UK
| |
Collapse
|
11
|
Michel JJ, Erinoff E, Tsou AY. More Guidelines than states: variations in U.S. lead screening and management guidance and impacts on shareable CDS development. BMC Public Health 2020; 20:127. [PMID: 31996264 PMCID: PMC6990572 DOI: 10.1186/s12889-020-8225-8] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/01/2019] [Accepted: 01/15/2020] [Indexed: 11/10/2022] Open
Abstract
BACKGROUND Pediatric lead exposure in the United States (U.S.) remains a preventable public health crisis. Shareable electronic clinical decision support (CDS) could improve lead screening and management. However, discrepancies between federal, state and local recommendations could present significant challenges for implementation. METHODS We identified publically available guidance on lead screening and management. We extracted definitions for elevated lead and recommendations for screening, follow-up, reporting, and management. We compared thresholds and level of obligation for management actions. Finally, we assessed the feasibility of development of shareable CDS. RESULTS We identified 54 guidance sources. States offered different definitions of elevated lead, and recommendations for screening, reporting, follow-up and management. Only 37 of 48 states providing guidance used the Center for Disease Control (CDC) definition for elevated lead. There were 17 distinct management actions. Guidance sources indicated an average of 5.5 management actions, but offered different criteria and levels of obligation for these actions. Despite differences, the recommendations were well-structured, actionable, and encodable, indicating shareable CDS is feasible. CONCLUSION Current variability across guidance poses challenges for clinicians. Developing shareable CDS is feasible and could improve pediatric lead screening and management. Shareable CDS would need to account for local variability in guidance.
Collapse
Affiliation(s)
- Jeremy J Michel
- Department of Biomedical and Health Informatics, The Children's Hospital of Philadelphia, 2716 South Street, Philadelphia, PA, 19146, USA.
- Department of Pediatrics, Perelman School of Medicine at the University of Pennsylvania, Philadelphia, PA, 19146, USA.
- ECRI Institute Center for Clinical Evidence and Guidelines, Plymouth Meeting, PA, 19462, USA.
| | - Eileen Erinoff
- ECRI Institute Center for Clinical Evidence and Guidelines, Plymouth Meeting, PA, 19462, USA
| | - Amy Y Tsou
- ECRI Institute Center for Clinical Evidence and Guidelines, Plymouth Meeting, PA, 19462, USA
- Michael J Crescenz Veterans Affairs Medical Center, Philadelphia, PA, 19104, USA
| |
Collapse
|
12
|
Chronic Care for Attention-Deficit/Hyperactivity Disorder: Clinical Management from Childhood Through Adolescence. J Dev Behav Pediatr 2020; 41 Suppl 2S:S99-S104. [PMID: 31996572 PMCID: PMC9295618 DOI: 10.1097/dbp.0000000000000772] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 11/25/2022]
Abstract
OBJECTIVE Impairments generally persist when children diagnosed with attention-deficit/hyperactivity disorder (ADHD) mature into adolescence. To examine changes in ADHD care during the transition from childhood to adolescence, we conducted a retrospective, longitudinal cohort study of patients diagnosed with ADHD before the age of 10 years to assess changes from preadolescence through adolescence in (1) frequency by which primary care providers offer ADHD care to patients, (2) range of concerns assessed during patient encounters, and (3) treatments implemented or recommended. METHODS We identified patients from 3 practices included in a large primary care network who (1) were born between 1996 and 1997, (2) were diagnosed with ADHD before the age of 10 years, and (3) received primary care continuously from age 9 through late adolescence. Clinical care was compared among patients in preadolescence (age 9-11), early adolescence (age 12-14), and late adolescence (age 15-18). RESULTS Children diagnosed with ADHD before the age of 10 years were less likely to have a documented visit for ADHD during late adolescence (41% of patients) compared with preadolescence (63%, p < 0.001). Evidence of monitoring for depression, suicide, and substance abuse increased from preadolescence to adolescence (p < 0.001) and occurred in about 90% of adolescent patients. However, monitoring for risky sexual activity occurred in only about 50% of adolescents. Discussions of medication diversion and driver readiness were essentially not documented. CONCLUSION The findings raise concerns about how primary care providers manage adolescents with a history of ADHD. Improving monitoring of risky sexual behavior and driver readiness and providing patient education about medication diversion are needed.
Collapse
|
13
|
Huang KY, Lee D, Nakigudde J, Cheng S, Gouley KK, Mann D, Schoenthaler A, Chokshi S, Kisakye EN, Tusiime C, Mendelsohn A. Use of Technology to Promote Child Behavioral Health in the Context of Pediatric Care: A Scoping Review and Applications to Low- and Middle-Income Countries. Front Psychiatry 2019; 10:806. [PMID: 31798470 PMCID: PMC6865208 DOI: 10.3389/fpsyt.2019.00806] [Citation(s) in RCA: 10] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 05/28/2019] [Accepted: 10/11/2019] [Indexed: 12/13/2022] Open
Abstract
Background: The burden of mental, neurological, and substance (MNS) disorders is greater in low- and middle-income countries (LMICs). The rapid growth of digital health (i.e., eHealth) approaches offer new solutions for transforming pediatric mental health services and have the potential to address multiple resource and system barriers. However, little work has been done in applying eHealth to promote young children's mental health in LMICs. It is also not clear how eHealth has been and might be applied to translating existing evidence-based practices/strategies (EBPs) to enable broader access to child mental health interventions and services. Methods: A scoping review was conducted to summarize current eHealth applications and evidence in child mental health. The review focuses on 1) providing an overview of existing eHealth applications, research methods, and effectiveness evidence in child mental health promotion (focused on children of 0-12 years of age) across diverse service contexts; and 2) drawing lessons learned from the existing research about eHealth design strategies and usability data in order to inform future eHealth design in LMICs. Results: Thirty-two (32) articles fitting our inclusion criteria were reviewed. The child mental health eHealth studies were grouped into three areas: i) eHealth interventions targeting families that promote child and family wellbeing; ii) eHealth for improving school mental health services (e.g., promote school staff's knowledge and management skills); and iii) eHealth for improving behavioral health care in the pediatric care system (e.g., promote use of integrated patient-portal and electronic decision support systems). Most eHealth studies have reported positive impacts. Although most pediatric eHealth studies were conducted in high-income countries, many eHealth design strategies can be adapted and modified to fit LMIC contexts. Most user-engagement strategies identified from high-income countries are also relevant for populations in LMICs. Conclusions: This review synthesizes patterns of eHealth use across a spectrum of individual/family and system level of eHealth interventions that can be applied to promote child mental health and strengthen mental health service systems. This review also summarizes critical lessons to guide future eHealth design and delivery models in LMICs. However, more research in testing combinations of eHealth strategies in LMICs is needed.
Collapse
Affiliation(s)
- Keng-Yen Huang
- Department of Population Health, New York University School of Medicine, New York, NY, United States
| | - Douglas Lee
- College of Osteopathic Medicine, New York Institute of Technology, New York, NY, United States
| | - Janet Nakigudde
- Department of Psychiatry, Makerere University, Kampala, Uganda
| | - Sabrina Cheng
- Department of Population Health, New York University School of Medicine, New York, NY, United States
| | - Kathleen Kiely Gouley
- Department of Population Health, New York University School of Medicine, New York, NY, United States
| | - Devin Mann
- Department of Population Health, New York University School of Medicine, New York, NY, United States
| | - Antoinette Schoenthaler
- Department of Population Health, New York University School of Medicine, New York, NY, United States
| | - Sara Chokshi
- Department of Population Health, New York University School of Medicine, New York, NY, United States
| | | | | | - Alan Mendelsohn
- Department of Population Health, New York University School of Medicine, New York, NY, United States
| |
Collapse
|
14
|
Michel JJ, Mayne S, Grundmeier RW, Guevara JP, Blum NJ, Power TJ, Coffin E, Miller JM, Fiks AG. Sharing of ADHD Information between Parents and Teachers Using an EHR-Linked Application. Appl Clin Inform 2018; 9:892-904. [PMID: 30566963 DOI: 10.1055/s-0038-1676087] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/11/2023] Open
Abstract
BACKGROUND Appropriate management of attention-deficit/hyperactivity disorder (ADHD) involves parents, clinicians, and teachers. Fragmentation of interventions between different settings can lead to suboptimal care and outcomes. Electronic systems can bridge gaps across settings. Our institution developed an email-based software to collect ADHD information from parents and teachers, which delivered data directly to the clinician within the electronic health record (EHR). OBJECTIVE We sought to adapt our institution's existing EHR-linked system for ADHD symptom monitoring to support communication between parents and teachers and then to assess child characteristics associated with sharing of ADHD information. METHODS We updated our software to support automated sharing of ADHD information between parents and teachers. Sharing was optional for parents but obligatory for teachers. We conducted a retrospective cohort study involving 590 patients at 31 primary care sites to evaluate a system for sharing of ADHD-specific health information between parents and teachers. We used multivariable logistic regression to estimate associations between child characteristics and parental sharing. We further investigated the association between child characteristics and viewing of survey results delivered through the electronic communication system. RESULTS Most parents (64%) elected to share survey results with teachers at the first opportunity and the vast majority (80%) elected to share all possible information. Parents who elected to share usually continue sharing at subsequent opportunities (89%). Younger child age and performance impairments were associated with increased likelihood of sharing. However, parents viewed only 16% of teacher submitted surveys and teachers only viewed 30% of surveys shared by parents. CONCLUSION This study demonstrates that electronic systems to capture ADHD information from parents and teachers can be adapted to support communication between them, and that parents are amenable to sharing ADHD information with teachers. However, strategies are needed to encourage viewing of shared information.
Collapse
Affiliation(s)
- Jeremy J Michel
- Department of Pediatrics, Perelman School of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania, United States.,Department of Biomedical and Health Informatics, The Children's Hospital of Philadelphia, Philadelphia, Pennsylvania, United States
| | - Stephanie Mayne
- Center for Pediatric Clinical Effectiveness, The Children's Hospital of Philadelphia, Philadelphia, Pennsylvania, United States.,PolicyLab: Center to Bridge Research, Practice, and Policy, The Children's Hospital of Philadelphia, Philadelphia, Pennsylvania, United States
| | - Robert W Grundmeier
- Department of Pediatrics, Perelman School of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania, United States.,Department of Biomedical and Health Informatics, The Children's Hospital of Philadelphia, Philadelphia, Pennsylvania, United States
| | - James P Guevara
- Department of Pediatrics, Perelman School of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania, United States.,PolicyLab: Center to Bridge Research, Practice, and Policy, The Children's Hospital of Philadelphia, Philadelphia, Pennsylvania, United States.,Leonard Davis Institute of Health Economics, University of Pennsylvania, Philadelphia, Pennsylvania, United States
| | - Nathan J Blum
- Division of Developmental and Behavioral Pediatrics, The Children's Hospital of Philadelphia, Philadelphia, Pennsylvania, United States
| | - Thomas J Power
- Department of Child and Adolescent Psychiatry and Behavioral Sciences, The Children's Hospital of Philadelphia, Philadelphia, Pennsylvania, United States
| | - Emily Coffin
- The Children's Hospital of Philadelphia, Philadelphia, Pennsylvania, United States
| | - Jeffrey M Miller
- Department of Biomedical and Health Informatics, The Children's Hospital of Philadelphia, Philadelphia, Pennsylvania, United States
| | - Alexander G Fiks
- Department of Pediatrics, Perelman School of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania, United States.,Department of Biomedical and Health Informatics, The Children's Hospital of Philadelphia, Philadelphia, Pennsylvania, United States.,Center for Pediatric Clinical Effectiveness, The Children's Hospital of Philadelphia, Philadelphia, Pennsylvania, United States
| |
Collapse
|