Impact of electronic health record transition on behavioral health screening in a large pediatric practice

Screening Infants Through Adolescents for Social/Emotional/Behavioral Problems in a Pediatric Network

OBJECTIVE

To assess changes in screening completion in a diverse, 7-clinic network after making annual screening for social/emotional/behavioral (SEB) problems the standard of care for all infant through late adolescent-aged patients and rolling out a fully automated screening system tied to the electronic medical record and patient portal.

METHODS

In 2017, the Massachusetts General Hospital made SEB screening using the age-appropriate version of the Pediatric Symptom Checklist the standard of care in its pediatric clinics for all patients aged 2.0 months to 17.9 years. Billing records identified all well-child visits between January 1, 2016 and December 31, 2019. For each visit, claims were searched for billing for an SEB screen and the electronic data warehouse was queried for an electronically administered screen. A random sample of charts was reviewed for other evidence of screening. Chi-square analyses and generalized estimating equations assessed differences in screening over time and across demographic groups.

RESULTS

Screening completion (billing and/or electronic) significantly increased from 2016 (37.2%) through 2019 (2017 [46.2%] vs 2018 [66.8%] vs 2019 [70.9%]; χ² (3) =112652.33, P < .001), with an even higher prevalence found after chart reviews. Most clinics achieved screening levels above 90% by the end of 2019. Differences among demographic groups were small and dependent on whether data were aggregated at the clinic or system level.

CONCLUSIONS

Following adoption of a best-practice policy and implementation of an electronic system, SEB screening increased in all age groups and clinics. Findings demonstrate that the AAP recommendation for routine psychosocial assessment is feasible and sustainable.

Assessment of Variation in Electronic Health Record Capabilities and Reported Clinical Quality Performance in Ambulatory Care Clinics, 2014-2017

IMPORTANCE

Electronic health records (EHRs) are widely promoted to improve the quality of health care, but information about the association of multifunctional EHRs with broad measures of quality in ambulatory settings is scarce.

OBJECTIVE

To assess the association between EHRs with different degrees of capabilities and publicly reported ambulatory quality measures in at least 3 clinical domains of care.

DESIGN, SETTING, AND PARTICIPANTS

This cross-sectional and longitudinal study was conducted using survey responses from 1141 ambulatory clinics in Minnesota, Washington, and Wisconsin affiliated with a health system that responded to the Healthcare Information and Management Systems Society Annual Survey and reported performance measures in 2014 to 2017. Statistical analysis was performed from July 10, 2019, through February 26, 2021.

MAIN OUTCOMES AND MEASURES

A composite measure of EHR capability that considered 50 EHR capabilities in 7 functional domains, grouped into the following ordered categories: no functional EHR, EHR underuser, EHR, neither underuser or superuser, EHR superuser; as well as a standardized composite of ambulatory clinical performance measures that included 3 to 25 individual measures (median, 13 individual measures).

RESULTS

In 2014, 381 of 746 clinics (51%) were EHR superusers; this proportion increased in each subsequent year (457 of 846 clinics [54%] in 2015, 510 of 881 clinics [58%] in 2016, and 566 of 932 clinics [61%] in 2017). In each cross-sectional analysis year, EHR superusers had better clinical quality performance than other clinics (adjusted difference in score: 0.39 [95% CI, 0.12-0.65] in 2014; 0.29 [95% CI, -0.01 to 0.59] in 2015; 0.26 [95% CI, -0.05 to 0.56] in 2016; and 0.20 [95% CI, -0.04 to 0.45] in 2017). This difference in scores translates into an approximately 9% difference in a clinic's rank order in clinical quality. In longitudinal analyses, clinics that progressed to EHR superuser status had only slightly better gains in clinical quality between 2014 and 2017 compared with the gains in clinical quality of clinics that were static in terms of their EHR status (0.10 [95% CI, -0.13 to 0.32]). In an exploratory analysis, different types of EHR capability progressions had different degrees of associated improvements in ambulatory clinical quality (eg, progression from no functional EHR to a status short of superuser, 0.06 [95% CI, -0.40 to 0.52]; progression from EHR underuser to EHR superuser, 0.18 [95% CI, -0.14 to 0.50]).

CONCLUSIONS AND RELEVANCE

Between 2014 and 2017, ambulatory clinics in Minnesota, Washington, and Wisconsin with EHRs having greater capabilities had better composite measures of clinical quality than other clinics, but clinics that gained EHR capabilities during this time had smaller increases in clinical quality that were not statistically significant.

Health Care Reform, Length of Stay, and Readmissions for Child Mental Health Hospitalizations

BACKGROUND

Health care reform may impact inpatient mental health services by increasing access and changing insurer incentives. We examined whether implementation of the 2014 Affordable Care Act (ACA) was associated with changes in psychiatric length of stay (LOS) and 30-day readmissions for pediatric patients.

METHODS

We conducted an interrupted time-series analysis to evaluate LOS and 30-day readmissions during the 30 months before and 24 months after ACA implementation, with a 6-month wash-out period, on patients aged 4 to 17 years who were discharged from the psychiatry unit of a children's hospital. Differences by payer (Medicaid versus non-Medicaid) were examined in moderated interrupted time series. Logistic regression was used to examine the association between psychiatric LOS and 30-day readmissions.

RESULTS

There were 1874 encounters in the pre-ACA period and 2186 encounters in the post-ACA period. Compared with pre-ACA implementation, post-ACA implementation was associated with LOS that was significantly decreasing over time (pre-ACA versus post-ACA slope difference: -0.10 days per encounter per month [95% confidence interval -0.17 to -0.02]; P = .01), especially for Medicaid-insured patients (pre-ACA versus post-ACA slope difference: -0.14 days per encounter per month [95% confidence interval -0.26 to -0.01]; P = .03). The overall proportion of 30-day readmissions increased significantly (pre-ACA 6%, post-ACA 10%; P < .05 for the difference). We found no association between LOS and 30-day readmissions.

CONCLUSIONS

ACA implementation was associated with a decline in psychiatric inpatient LOS over time, especially for those on Medicaid, and an increase in 30-day readmissions. LOS was not associated with 30-day inpatient readmissions. Further investigation to understand the drivers of these patterns is warranted.

Written Versus Verbal Food Insecurity Screening in One Primary Care Clinic

OBJECTIVE

Clinics are increasingly interested in identifying food insecurity (FI), but there are limited data on how to implement FI screening. Our objective was to determine the difference in FI disclosure rates by parents/guardians screened by a written questionnaire compared to verbally.

METHODS

The study occurred in 1 pediatric primary care clinic in which we screened for FI using the 2-item Hunger Vital Sign. We used interrupted time series to evaluate the effect of changing from the clinician verbal screening to a written questionnaire. Screening results were extracted for all well-child visits from 4/2017 to 10/2018 for children age 0 to 18 years. The outcome was the proportion who screened positive for FI 9 months before and 9 months after the implementation of the written questionnaire. We estimated the difference in the level and trend of positive screens using ordinary least squares regression using Newey-West standard errors and adjusting for autocorrelation.

RESULTS

In 7996 well-child visits, 1141 patients (14.3%) screened positive. In bivariate analysis, there was a significant difference in the FI disclosure rates between patients screened by written questionnaire compared to verbally (16.3% vs 10.4%, P < .001). In interrupted time series, changing to the written questionnaire was associated with a significant increase in FI disclosure rates (β = .04, 95% confidence interval: 0.01, 0.07; P = .02). There was no significant change in the trend in disclosure rates.

DISCUSSION

Multiple barriers exist to effectively implementing FI screening in clinical care. Changing from a verbal to a written questionnaire resulted in an immediate and significant increase in the number of parents/guardians who reported FI.

How an Electronic Medical Record System Facilitates and Demonstrates Effective Psychosocial Screening in Pediatric Primary Care

Using questionnaires, administrative claims, and chart review data, the current study explored the impact of using an electronic medical record system to administer, score, and store the Pediatric Symptom Checklist (PSC-17) during annual pediatric well-child visits. Within a sample of 1773 Medicaid-insured outpatients, the electronic system demonstrated that 90.5% of cases completed a PSC-17 screen electronically, billing codes indicating a screen was administered agreed with the existence of a questionnaire in the chart in 98.8% of cases, the classification of risk based on PSC-17 scores agreed with the classification of risk based on the Current Procedural Terminology code modifiers in 72.9% of cases, and 90.0% of clinicians' progress notes mentioned PSC-17 score in treatment planning. Using an electronic approach to psychosocial screening in pediatrics facilitated the use of screening information gathered during the clinical visit and allowed for enhanced tracking of outcomes and quality monitoring.

Effect of an Electronic Health Record Transition on the Provision of Recommended Well Child Services in Pediatric Primary Care Practices

We sought to determine the effect of transitioning between electronic health record (EHR) systems on the quality of preventive care in a large pediatric primary care network. To study this, we performed a retrospective chart analysis of 42 primary care practices from the Pediatric Physicians' Organization at Children's who transitioned EHRs. We reviewed 24 random encounters per week distributed evenly across 6 age categories before, during, and after a transition period. We reviewed encounter documentation for age-appropriate well child services, per American Academy of Pediatrics/Bright Futures guidelines. Logistic regression and statistical process control analysis were used. In the pretransition period, 84.5% of all recommended elements were documented versus 86.4% posttransition (P = .04). Documentation of age-appropriate anticipatory guidance showed significant positive change (69.0% to 80.2%, P = .005), but it was the only subdomain with a statistically significant increase. These increases suggest that EHR transitions have the opportunity to affect the delivery of preventive care.

Unintended Consequences of Nationwide Electronic Health Record Adoption: Challenges and Opportunities in the Post-Meaningful Use Era

The US health system has recently achieved widespread adoption of electronic health record (EHR) systems, primarily driven by financial incentives provided by the Meaningful Use (MU) program. Although successful in promoting EHR adoption and use, the program, and other contributing factors, also produced important unintended consequences (UCs) with far-reaching implications for the US health system. Based on our own experiences from large health information technology (HIT) adoption projects and a collection of key studies in HIT evaluation, we discuss the most prominent UCs of MU: failed expectations, EHR market saturation, innovation vacuum, physician burnout, and data obfuscation. We identify challenges resulting from these UCs and provide recommendations for future research to empower the broader medical and informatics communities to realize the full potential of a now digitized health system. We believe that fixing these unanticipated effects will demand efforts from diverse players such as health care providers, administrators, HIT vendors, policy makers, informatics researchers, funding agencies, and outside developers; promotion of new business models; collaboration between academic medical centers and informatics research departments; and improved methods for evaluations of HIT.

Comprehensive methodology to monitor longitudinal change patterns during EHR implementations: a case study at a large health care delivery network

OBJECTIVE

To test a systematic methodology to monitor longitudinal change patterns on quality, productivity, and safety outcomes during a large-scale commercial Electronic Health Record (EHR) implementation.

MATERIALS AND METHODS

Our method combines an interrupted time-series design with control sites and 41 consensus outcomes including quality (11 measures), productivity (20 measures), and safety (10 measures). The intervention consisted of a phased commercial EHR implementation at a large health care delivery network. Four medium-size hospitals and 39 clinics from 5 geographic regions implementing the new EHR were compared against a parallel control consisting of one medium-size and one large hospital and 10 clinics that had not implemented the new EHR at the time of this study. We collected monthly data from February 2013 to July 2017.

RESULTS

The proposed methodology was successfully implemented and significant changes were observed in most measured variables. A significant change attributable to the intervention was observed in 12 (29%) measures in three or more regions; in 32 (78%) measures in two or more regions; and in 40 (98%) measures in at least one region. A similar pattern (i.e., same impact in three or more regions) was detected for nine (22%) measures, a mixed pattern (i.e., same impact in two regions, and different impact in other regions) was detected for nine (22%) measures, and an inconsistent pattern (i.e., did not detect the same impact across regions) was detected for 23 (56%) measures.

DISCUSSION

Using a formal methodology to assess changes in a set of consensus measures, we detected various patterns of impact and mixed time-sensitive effects. With an increasing adoption of EHR systems, it is critical for health care organizations to systematically monitor their EHR implementations. The proposed method provides a robust and consistent approach to monitor EHR implementations longitudinally allowing for continuous monitoring after the system becomes stable in order to avoid unexpected effects.

CONCLUSION

Our results and methodology can guide the broader medical and informatics communities by informing what and how to continuously monitor EHR impact on quality, productivity, and safety.

Experiences of General Practitioners and Practice Support Staff Using a Health and Lifestyle Screening App in Primary Health Care: Implementation Case Study

Background

Technology-based screening of young people for mental health disorders and health compromising behaviors in general practice increases the disclosure of sensitive health issues and improves patient-centered care. However, few studies investigate how general practitioners (GPs) and practice support staff (receptionists and practice managers) integrate screening technology into their routine work, including the problems that arise and how the staff surmount them.

Objective

The aim of this study was to investigate the implementation of a health and lifestyle screening app, Check Up GP, for young people aged 14 to 25 years attending an Australian general practice.

Methods

We conducted an in-depth implementation case study of Check Up GP in one general practice clinic, with methodology informed by action research. Semistructured interviews and focus groups were conducted with GPs and support staff at the end of the implementation period. Data were thematically analyzed and mapped to normalization process theory constructs. We also analyzed the number of times we supported staff, the location where young people completed Check Up GP, and whether they felt they had sufficient privacy and received a text messaging (short message service, SMS) link at the time of taking their appointment.

Results

A total of 4 GPs and 10 support staff at the clinic participated in the study, with all except 3 receptionists participating in the final interviews and focus groups. During the 2-month implementation period, the technology and administration of Check Up GP was iterated through 4 major quality improvement cycles in response to the needs of the staff. This resulted in a reduction in the average time taken to complete Check Up GP from 14 min to 10 min, improved SMS text messaging for young people, and a more consistent description of the app by receptionists to young people. In the first weeks of implementation, researchers needed to regularly support staff with the app’s administration; however, this support decreased over time, even as usage rose slightly. The majority of young people (73/87, 84%) completed Check Up GP in the waiting room, with less than half (35/80, 44%) having received an SMS from the clinic with a link to the tool. Participating staff valued Check Up GP, particularly its facilitation of youth-friendly practice. However, there was at first a lack of organizational systems and capacity to implement the app and also initially a reliance on researchers to facilitate the process.

Conclusions

The implementation of a screening app in the dynamic and time-restricted general practice setting presents a range of technical and administrative challenges. Successful implementation of a screening app is possible but requires adequate time and intensive facilitation. More resources, external to staff, are needed to drive and support sustainable technology innovation and implementation in general practice settings.

The Impact of the Massachusetts Behavioral Health Child Screening Policy on Service Utilization

OBJECTIVE

In 2008, Massachusetts Medicaid implemented a pediatric behavioral health (BH) screening mandate. This study conducted a population-level, longitudinal policy analysis to determine the impact of the policy on ambulatory, emergency, and inpatient BH care in comparison with use of these services in California, where no similar policy exists.

METHODS

With Medicaid Analytic Extract (MAX) data, an interrupted time-series analysis with control series design was performed to assess changes in service utilization in the 18 months (January 2008-June 2009) after a BH screening policy was implemented in Massachusetts and to compare service utilization with California's. Outcomes included population rates of BH screening, BH-related outpatient visits, BH-related emergency department visits, BH-related hospitalizations, and psychotropic drug use. Medicaid-eligible children from January 1, 2006, to December 31, 2009, with at least ten months of Medicaid eligibility who were older than 4.5 years and younger than 18 years were included.

RESULTS

Compared with rates in California, Massachusetts rates of BH screening and BH-related outpatient visits rose significantly after Massachusetts implemented its screening policy. BH screening rose about 13 per 1,000 youths per month during the first nine months, and BH-related outpatient visits rose to about 4.5 per 1,000 youths per month (p<.001). Although BH-related emergency department visits, hospitalization and psychotropic drug use increased, there was no difference between the states in rate of increase.

CONCLUSIONS

The goal of BH screening is to identify previously unidentified children with BH issues and provide earlier treatment options. The short-term outcomes of the Massachusetts policy suggest that screening at preventive care visits led to more BH-related outpatient visits among vulnerable children.

Health information technology adoption: Understanding research protocols and outcome measurements for IT interventions in health care

OBJECTIVE

To classify and characterize the variables commonly used to measure the impact of Information Technology (IT) adoption in health care, as well as settings and IT interventions tested, and to guide future research.

MATERIALS AND METHODS

We conducted a descriptive study screening a sample of 236 studies from a previous systematic review to identify outcome measures used and the availability of data to calculate these measures. We also developed a taxonomy of commonly used measures and explored setting characteristics and IT interventions.

RESULTS

Clinical decision support is the most common intervention tested, primarily in non-hospital-based clinics and large academic hospitals. We identified 15 taxa representing the 79 most commonly used measures. Quality of care was the most common category of these measurements with 62 instances, followed by productivity (11 instances) and patient safety (6 instances). Measures used varied according to type of setting, IT intervention and targeted population.

DISCUSSION

This study provides an inventory and a taxonomy of commonly used measures that will help researchers select measures in future studies as well as identify gaps in their measurement approaches. The classification of the other protocol components such as settings and interventions will also help researchers identify underexplored areas of research on the impact of IT interventions in health care.

CONCLUSION

A more robust and standardized measurement system and more detailed descriptions of interventions and settings are necessary to enable comparison between studies and a better understanding of the impact of IT adoption in health care settings.

Adoption Factors of the Electronic Health Record: A Systematic Review

Background

The Health Information Technology for Economic and Clinical Health (HITECH) was a significant piece of legislation in America that served as a catalyst for the adoption of health information technology. Following implementation of the HITECH Act, Health Information Technology (HIT) experienced broad adoption of Electronic Health Records (EHR), despite skepticism exhibited by many providers for the transition to an electronic system. A thorough review of EHR adoption facilitator and barriers provides ongoing support for the continuation of EHR implementation across various health care structures, possibly leading to a reduction in associated economic expenditures.

Objective

The purpose of this review is to compile a current and comprehensive list of facilitators and barriers to the adoption of the EHR in the United States.

Methods

Authors searched Cumulative Index of Nursing and Allied Health Literature (CINAHL) and MEDLINE, 01/01/2012–09/01/2015, core clinical/academic journals, MEDLINE full text, and evaluated only articles germane to our research objective. Team members selected a final list of articles through consensus meetings (n=31). Multiple research team members thoroughly read each article to confirm applicability and study conclusions, thereby increasing validity.

Results

Group members identified common facilitators and barriers associated with the EHR adoption process. In total, 25 adoption facilitators were identified in the literature occurring 109 times; the majority of which were efficiency, hospital size, quality, access to data, perceived value, and ability to transfer information. A total of 23 barriers to adoption were identified in the literature, appearing 95 times; the majority of which were cost, time consuming, perception of uselessness, transition of data, facility location, and implementation issues.

Conclusions

The 25 facilitators and 23 barriers to the adoption of the EHR continue to reveal a preoccupation on cost, despite incentives in the HITECH Act. Limited financial backing and outdated technology were also common barriers frequently mentioned during data review. Future public policy should include incentives commensurate with those in the HITECH Act to maintain strong adoption rates.

Physician and Parent Perspectives on Psychosocial and Emotional Data Entry in the Electronic Medical Record in a Pediatric Setting

Objective:

This pilot study was conducted to evaluate physician and patient preferences for documentation of emotional and psychosocial information in the electronic medical record (EMR).

Methods:

Pediatricians from an academic medical center and parents of patients in an academic pediatric rheumatology practice were surveyed on 10 different elements using Likert-type scale items (1 = not at all important, 10 = extremely important). The importance of the proposed categories was evaluated by means testing and pairwise comparisons of the responses.

Results:

Responses were obtained from 45 physicians and 35 parents. The overall mean scores for physicians and parents were 7.70 and 7.44, respectively. Scores on personality, friends, and school differed between physicians and parents, but those differences were not significant after adjustment for multiple comparisons (P = .13, .17, and .26, respectively). Fears, special requests, and special needs were in the high-score group for both physicians and parents.

Conclusion:

Physicians and parents reported that the incorporation of emotional and psychosocial information into the EMR added value to the health care of children.

Behavioral health services following implementation of screening in Massachusetts Medicaid children

OBJECTIVES

To determine the relationship of child behavioral health (BH) screening results to receipt of BH services in Massachusetts Medicaid (MassHealth) children.

METHODS

After a court decision, Massachusetts primary care providers were mandated to conduct BH screening at well-child visits and use a Current Procedural Terminology code along with a modifier indicating whether a BH need was identified. Using MassHealth claims data, a cohort of continuously enrolled (July 2007-June 2010) children was constructed. The salient visit (first use of the modifier, screening code, or claim in fiscal year 2009) was considered a reference point to examine BH history and postscreening BH services. Bivariate and multivariate logistic regression analyses were performed to determine predictors of postscreening BH services.

RESULTS

Of 261,160 children in the cohort, 45% (118,464) were screened and 37% had modifiers. Fifty-seven percent of children screening positive received postscreening BH services compared with 22% of children screening negative. However, only 30% of newly identified children received BH services. The strongest predictors of postscreening BH services for children without a BH history were being in foster care (odds ratio, 10.38; 95% confidence interval, 9.22-11.68) and having a positive modifier (odds ratio, 3.79; 95% confidence interval, 3.53-4.06).

CONCLUSIONS

Previous BH history, a positive modifier, and foster care predicted postscreening BH services. Only one-third of newly identified children received services. Thus although screening is associated with an increase in BH recognition, it may be insufficient to improve care. Additional strategies may be needed to enhance engagement in BH services.

Use of interrupted time series analysis in evaluating health care quality improvements

Interrupted time series (ITS) analysis is arguably the strongest quasi-experimental research design. ITS is particularly useful when a randomized trial is infeasible or unethical. The approach usually involves constructing a time series of population-level rates for a particular quality improvement focus (eg, rates of attention-deficit/hyperactivity disorder [ADHD] medication initiation) and testing statistically for a change in the outcome rate in the time periods before and time periods after implementation of a policy/program designed to change the outcome. In parallel, investigators often analyze rates of negative outcomes that might be (unintentionally) affected by the policy/program. We discuss why ITS is a useful tool for quality improvement. Strengths of ITS include the ability to control for secular trends in the data (unlike a 2-period before-and-after t test), ability to evaluate outcomes using population-level data, clear graphical presentation of results, ease of conducting stratified analyses, and ability to evaluate both intended and unintended consequences of interventions. Limitations of ITS include the need for a minimum of 8 time periods before and 8 after an intervention to evaluate changes statistically, difficulty in analyzing the independent impact of separate components of a program that are implemented close together in time, and existence of a suitable control population. Investigators must also be careful not to make individual-level inferences when population-level rates are used to evaluate interventions (though ITS can be used with individual-level data). A brief description of ITS is provided, including a fully implemented (but hypothetical) study of the impact of a program to reduce ADHD medication initiation in children younger than 5 years old and insured by Medicaid in Washington State. An example of the database needed to conduct an ITS is provided, as well as SAS code to implement a difference-in-differences model using preschool-age children in California as a comparison group.