Assessing Nurse Competency in the Veterans Health Administration Registered Nurse Transition-to-Practice Residency Program: Item Pool Content Validation

BACKGROUND

New graduate registered nurse (RN) competencies are complex and difficult to measure. Psychometrically sound tools are needed to evaluate competencies for nurses participating in nurse residencies.

PURPOSE

Project aims were to develop an item pool for a tool to measure new graduate RN competencies for the Veterans Health Administration RN Transition-to-Practice Residency Program; validate item pool content; and use consensus methods to improve item pool content validity.

METHODS

A sequential, mixed-methods design was used. Item pool creation, content validation, and revisions included a collaborative process with the evaluation team, operational stakeholders, and subject matter experts (SMEs).

RESULTS

Inclusion of SMEs in item development enhanced item pool content validity to measure nurse competency. Stakeholder feedback ensured programmatic logistical and evaluation concerns were met.

CONCLUSIONS

Engaging SMEs in conceptualization, item development, and aligning existing standards enhanced item pool content validity to measure nurse competencies for new graduate RNs.

Machine learning to develop a predictive model of pressure injury in persons with spinal cord injury

STUDY DESIGN

A 5-year longitudinal, retrospective, cohort study.

OBJECTIVES

Develop a prediction model based on electronic health record (EHR) data to identify veterans with spinal cord injury/diseases (SCI/D) at highest risk for new pressure injuries (PIs).

SETTING

Structured (coded) and text EHR data, for veterans with SCI/D treated in a VHA SCI/D Center between October 1, 2008, and September 30, 2013.

METHODS

A total of 4709 veterans were available for analysis after randomly selecting 175 to act as a validation (gold standard) sample. Machine learning models were created using ten-fold cross validation and three techniques: (1) two-step logistic regression; (2) regression model employing adaptive LASSO; (3) and gradient boosting. Models based on each method were compared using area under the receiver-operating curve (AUC) analysis.

RESULTS

The AUC value for the gradient boosting model was 0.62 (95% CI = 0.54-0.70), for the logistic regression model it was 0.67 (95% CI = 0.59-0.75), and for the adaptive LASSO model it was 0.72 (95% CI = 0.65-80). Based on these results, the adaptive LASSO model was chosen for interpretation. The strongest predictors of new PI cases were having fewer total days in the hospital in the year before the annual exam, higher vs. lower weight and most severe vs. less severe grade of injury based on the American Spinal Cord Injury Association (ASIA) Impairment Scale.

CONCLUSIONS

While the analyses resulted in a potentially useful predictive model, clinical implications were limited because modifiable risk factors were absent in the models.

Reliability and validity of the Veterans Administration Mobility Screening and Solutions Tool

Objectives

The Veterans Administration (VA) Mobility Screening and Solutions Tool (VA MSST) was developed to screen a patient’s safe mobility level ‘in the moment’ and provide clinical decision support related to the use of safe patient handling and mobility (SPHM) equipment. This evidence-based flowchart tool is a common language tool that enables any healthcare worker at any time to accurately measure and communicate patient mobility and transfer equipment needs across disciplines and settings.

Methods

The VA MSST has four levels and differentiates between the need for powered and non-powered equipment depending on the patient’s independence. Subject matter experts wrote scenarios for interrater reliability and validity testing. The initial VA MSST draft iteration was reviewed by 163 VA staff (mostly physical therapists and occupational therapists) amongst simulation scenarios and provided content validity, and additional insight and suggestions. Revisions were made to create the final VA MSST which was evaluated by over 200 healthcare workers from varied disciplines (including medical doctors, advanced practice registered nurses, registered nurses, licensed practical nurses, certified nursing assistants, occupational therapists, physical therapists, speech therapists, radiology and ultrasound technicians, etc.). An instruction video and eighteen scenario videos were embedded in an online survey. The survey intended to demonstrate the interrater reliability and validity (concurrent and construct) of the VA MSST. Over 500 VA staff (raters) received a survey invitation via email.

Results

Raters (N = 230) from multiple disciplines and healthcare settings independently screened patient mobility status for each of 18 scenarios using the VA MSST. The raters were diverse in their age and years of experience. The estimated interrater reliability (IRR) for VA MSST was excellent and statistically significant with an estimated Krippendorff’s alpha (ICC (C, k)) of 0.998 [95% CI: 0.996–0.999]. Eighty-two percent of raters reported that overall VA MSST instructions were clear or very clear and understandable. VA MSST ratings made by technicians and nursing assistants group correlated strongly (r = 0.99, p < 0.001) with the ‘gold standard’ (experienced physical therapists), suggesting a high concurrent validity of the tool. The VA MSST significantly discriminated between the different levels of patient mobility required for safe mobilization as intended (each difference, p < 0.0001); this suggests a good construct validity.

Conclusions

The VA MSST is an evidence-based flowchart screening and decision support tool that demonstrates excellent interrater reliability across disciplines and settings. VA MSST has strong face and content validity, as well as good concurrent and construct validity.

Concussion coach for postconcussive symptoms: A randomized, controlled trial of a smartphone application with Afghanistan and Iraq war Veterans

Objective: Investigate the effectiveness of Concussion Coach, an interactive smartphone application, as a treatment for residual neurobehavioral symptoms and distress in Veterans with a history of mild traumatic brain injury (mild TBI). Methods: Veterans with mild TBI were randomized to Concussion Coach (n = 238) or Treatment-as-Usual (TAU) (n = 241) in a 3-month randomized controlled trial. Primary outcome measures included postconcussive symptom (PCS) severity as measured by the Neurobehavioral Symptom Inventory (NSI), and psychological distress as measured by the Brief Symptom Inventory-18 (BSI-18). Measures of self-efficacy, social support, and comfort with technology were administered as potential moderators and mediators. An intention-to-treat (ITT) analysis was performed (N = 461: Concussion Coach = 231 and TAU = 230) using Bayesian Network (BN)modeling. Results: The probability of decreased PCS severity was significantly greater for those assigned to Concussion Coach, .35 [.32,.37], than for TAU, .29 (.27, .32), with an odds ratio (OR) of 1.29. Also, Concussion Coach showed a significantly greater probability of increased self-efficacy (.36 [.32, .39]) than did TAU (.28 [.25, .30], OR = 1.42). In turn, self-efficacy (increased vs. decreased) showed a significantly greater probability of decreased PCS severity (.51 [.47, .54] vs. .27 [.24, .30], OR = 2.71) and decreased psychological distress (.53 [.49, .56] vs. .32 [.29, .35], OR = 2.35), suggesting that self-efficacy may have mediated Concussion Coach effects. Conclusions: Concussion Coach is effective at reducing PCS severity and psychological distress. Increased self-efficacy/perception of self-management of symptoms may be key to successful treatment of residual symptoms in those with history of concussion.

Veterans with Traumatic Brain Injury-related Ocular Injury and Vision Dysfunction: Recommendations for Rehabilitation

SIGNIFICANCE

We know the prevalence of traumatic brain injury (TBI)-related vision impairment and ocular injury symptoms. Lacking is an understanding of health care utilization to treat these symptoms. Utilization knowledge is important to structuring access to treatment, identifying clinical training needs, and providing evidence of the effectiveness of treatment.

PURPOSE

This article reports rehabilitation, glasses/contacts, and imaging/photography/video recommendations made by optometrists and ophthalmologists as part of the Department of Veterans Affairs-mandated Performance of Traumatic Brain Injury Specific Ocular Health and Visual Functioning Examination administered to veterans with TBI at Department of Veterans Affairs polytrauma specialty facilities.

METHODS

Using a retrospective design, natural language processing, and descriptive and regression statistics, data were analyzed for 2458 Operation Enduring Freedom/Operation Iraqi Freedom veterans who were administered the mandated examination between 2008 and 2017.

RESULTS

Of the 2458 veterans, vision rehabilitation was recommended for 24%, glasses/contacts were recommended for 57%, and further imaging/photography/video testing was recommended for 58%. Using key words in the referral, we determined that 37% of veterans were referred to blind rehabilitation, 16% to occupational therapy, and 3% to low-vision clinics. More than 50% of the referrals could have been treated by blind rehabilitation, occupational therapy, or low-vision clinics. Rehabilitation referrals were significantly associated with younger age, floaters, photosensitivity, double vision, visual field and balance deficits, dizziness, and difficulty reading. In comparison, prescriptions for glasses and contacts were associated with older age, photosensitivity, blurred vision, decreased visual field and night vision, difficulty reading, and dry eye. Imaging/photography/video testing was associated with floaters, photosensitivity, and headache.

CONCLUSIONS

Findings delineate service delivery models available to veterans with TBI-related vision impairment. The challenge these data address is the lack of clear paths from diagnosis of TBI to identification of vision dysfunction deficits to specialized vision rehabilitation, and finally to community reintegration and community based-vision rehabilitation.

A preliminary evaluation of full practice authority of advance practice registered nurses in the Veterans Health Administration

BACKGROUND

Responding to National Academy of Medicine and National Council of State Boards of Nursing recommendations, the Department of Veterans Health Affairs (VHA) implemented full practice authority (FPA) for Advanced Practice Registered Nurses in VHA medical centers (VAMCs) in 2017.

PURPOSE

To evaluate FPA policy implementation's impact on quality indicators including access to care as measured by new patient appointments in primary, specialty and mental health services.

METHODS

Linear growth models compared early (n = 85) vs. late (n = 55) FPA implementing VAMCs on the trajectories of each of the three quality indicators.

FINDINGS

Early FPA implementing VAMCs showed greater rates of improvement over time in new patient appointments completed within 30 days of preferred date for primary care (p = .003), specialty care (p = 0.05), and mental health (p = 0.001).

DISCUSSION

VAMCs that started implementation of FPA policy early showed greater improvement in access to care for Veterans over time than VAMCs that did not.

Evaluating an Evidence-Based Practice Curriculum for Nurses Entering Clinical Practice in the Veterans Health Administration

Introduction

The Veterans Health Administration (VHA) registered nurse (RN) Transition to Practice (TTP) program is a 1-year comprehensive, standardized curriculum taught for entry-level nurses to assist them in transitioning to VA-trained, competent, professional RNs. The TTP program includes revised modules on Evidence-Based Practice (EBP) clinical decision making. The revised curriculum emphasizes EBP as a problem-solving approach to clinical decision making rather than a project-based approach to implement practice changes. The goal of this quality improvement project was to evaluate the content, delivery, and outcomes of a revised Evidence-Based Practice Curriculum (EBPC) for use in the VHA RN TTP program.

Methods

Focus groups were conducted with TTP coordinators, who teach the program and facility EBP content experts from 32 VHA Medical Centers. All attended a three-day face-to-face training at a central location. Qualitative data were managed and analyzed with a rapid assessment process.

Discussion

Leaders within and outside of organizations are commonly believed to affect the success of implementing and sustaining any program or initiative through their influence on organizational climate, leadership processes, and leadership alignment across multiple levels of leadership. Our findings were in line with other research showing that leaders should prioritize EBP and fuel it with resources to create sustainable change.

Conclusions

In conclusion, the EBPC was reviewed very favorably by all who planned to use it in their facilities in teaching the content to practicing registered nurses. Future evaluation will focus on the degree to which faculty use the program, how they use the modules, and what feedback nurses provide after exposure to EBPC.

Action Ethnography of Community Reintegration for Veterans and Military Service Members With Traumatic Brain Injury: Protocol for a Mixed Methods Study

BACKGROUND

Numerous studies of community reintegration (CR) in traumatic brain injury (TBI) have been conducted in civilian populations, but research is limited in veteran and military service member populations. Little is known about how knowledge from civilian studies translates into veterans' experiences and needs. The US Department of Veterans Health Administration (VHA) recognizes the distinctive health care needs of post-9/11 veteran and military service members, particularly with TBI, including the need to bridge health and rehabilitation-related services from acute care and inpatient settings to veteran and military service members' homes and communities to facilitate CR.

OBJECTIVE

The goal of this study is to better understand the experiences of veterans with complicated mild, moderate, or severe TBI; their families; and CR workers as veterans and servicemembers transition to and sustain living in communities. This paper describes the rationale, design, and methods used to reach this goal.

METHODS

This five-year longitudinal mixed methods study uses both a community-engaged research (CEnR) approach and an ethnographic approach. The sample includes 30 veterans and service members with TBI, 13 family caregivers, 11 CR specialists, 16 key stakeholders, and 82 community events. Interviews and observations are coded and analyzed using hierarchical coding schemes and thematic analysis. Analyses include data from surveys, interviews, and participant observations. Content analysis is used to highlight the complex social context of reintegration and to triangulate quantitative data. Egocentric (personal) social network analysis is used to examine the support system a veteran or service member has in place to facilitate reintegration.

RESULTS

Study enrollment and data collection are completed. Data analyses are underway.

CONCLUSIONS

The results of this study may provide a heightened understanding of environmental factors affecting CR in complicated mild, moderate, or severe TBI. Veteran, servicemember and family voices and insights provide VHA clinicians and policy makers with an ecological view of CR that is grounded in the life experiences of veterans, military service members, and families. The results of this study provide a roadmap for designing and testing interventions to maximize CR in a variety of domains. The longitudinal ethnographic approach allows for capturing detailed experiences within the naturalistic context. CEnR allows collaborative assessment of the social context of reintegration with community members.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

DERR1-10.2196/14170.

Doctoral-prepared nurses in the Veterans Health Administration: A cross-sectional survey

BACKGROUND

Doctoral-prepared nurses with diverse skillsets are required to meet nursing care needs in a complex and changing healthcare environment. A better understanding of the roles of doctoral-prepared nurses in Veterans Health Administration (VHA) workforce will help leverage their expertise to meet the needs of Veterans.

PURPOSE

Assess the current roles of doctoral-prepared nurses within the VHA.

METHOD

A cross-sectional survey was used to collect information on doctoral-prepared nurses within the VHA in 2016. Multiple strategies were used to identify doctoral-prepared nurses to recruit for an online survey. Survey invitations were sent electronically to unique individuals identified (N = 2403).

RESULTS

Responses were received from 1015 nurses (42.2% response rate), with 929 nurses with a doctoral degree identified. DNP/DNAP degrees were most common (55%), followed by a PhD or DNS (33%). Significant differences were noted between nurses in different doctoral education categories across four main roles: research, clinical, educational, and administration.

CONCLUSIONS

This survey generated the first comprehensive list of VHA doctoral-prepared nurses. Findings are being used by the VHA Office of Nursing Services to align degree types, duties of different positions, functional statements, and position expectations. Results support the continued need for collaboration between nurses with PhDs and DNPs to achieve research and clinical goals.

One hertz versus ten hertz repetitive TMS treatment of PTSD: A randomized clinical trial

The purpose of this trial was to test whether right prefrontal cortex 1 Hz versus 10 Hz rTMS provides a significantly greater improvement in PTSD symptoms and/or function. Veterans 18 to 50 years of age suffering from PTSD were randomized to right prefrontal 1 Hz rTMS [2400 pulses/session] versus right prefrontal 10 Hz rTMS [2400 pulses/session]. The treatments were performed 5 days a week for 6 weeks with a 3-week taper using the NeuroStar system. There were one month and three months post treatment follow-up evaluations. Forty-four participants were enrolled with 17 being randomized to 1 Hz rTMS and 18 to 10 Hz rTMS. Both groups had significant improvement in PTSD and depression scores from baseline to the end of acute treatment. The 10 Hz group but not the 1 Hz group demonstrated significant improvement in function. Although both groups demonstrated significant improvement in PTSD and depression symptoms, a significant advantage for either the 1 Hz or 10 Hz frequency group on any of the scales acquired was not demonstrated. Further work is required with larger samples sizes to test whether low or high frequency is superior or if individual differences would indicate the more effective frequency.

Leveraging Electronic Health Care Record Information to Measure Pressure Ulcer Risk in Veterans With Spinal Cord Injury: A Longitudinal Study Protocol

Background

Pressure ulcers (PrUs) are a frequent, serious, and costly complication for veterans with spinal cord injury (SCI). The health care team should periodically identify PrU risk, although there is no tool in the literature that has been found to be reliable, valid, and sensitive enough to assess risk in this vulnerable population.

Objective

The immediate goal is to develop a risk assessment model that validly estimates the probability of developing a PrU. The long-term goal is to assist veterans with SCI and their providers in preventing PrUs through an automated system of risk assessment integrated into the veteran’s electronic health record (EHR).

Methods

This 5-year longitudinal, retrospective, cohort study targets 12,344 veterans with SCI who were cared for in the Veterans Health Administration (VHA) in fiscal year (FY) 2009 and had no record of a PrU in the prior 12 months. Potential risk factors identified in the literature were reviewed by an expert panel that prioritized factors and determined if these were found in structured data or unstructured form in narrative clinical notes for FY 2009-2013. These data are from the VHA enterprise Corporate Data Warehouse that is derived from the EHR structured (ie, coded in database/table) or narrative (ie, text in clinical notes) data for FY 2009-2013.

Results

This study is ongoing and final results are expected in 2017. Thus far, the expert panel reviewed the initial list of risk factors extracted from the literature; the panel recommended additions and omissions and provided insights about the format in which the documentation of the risk factors might exist in the EHR. This list was then iteratively refined through review and discussed with individual experts in the field. The cohort for the study was then identified, and all structured, unstructured, and semistructured data were extracted. Annotation schemas were developed, samples of documents were extracted, and annotations are ongoing. Operational definitions of structured data elements have been created and steps to create an analytic dataset are underway.

Conclusions

To our knowledge, this is the largest cohort employed to identify PrU risk factors in the United States. It also represents the first time natural language processing and statistical text mining will be used to expand the number of variables available for analysis. A major strength of this quantitative study is that all VHA SCI centers were included in the analysis, reducing potential for selection bias and providing increased power for complex statistical analyses. This longitudinal study will eventually result in a risk prediction tool to assess PrU risk that is reliable and valid, and that is sensitive to this vulnerable population.

Processes and outcomes of the veterans health administration safe patient handling program: study protocol

Background

Health care workers, such as nurses, nursing aides, orderlies, and attendants, who manually move patients, are consistently listed in the top professions for musculoskeletal injuries (MSIs) by the Bureau of Labor Statistics. These MSIs are typically caused by high-risk patient caregiving activities. In 2008, a safe patient handling (SPH) program was implemented in all 153 Veterans Administration Medical Centers (VAMCs) throughout the United States to reduce patient handling injuries.

Objective

The goal of the present study is to evaluate the effects associated with the national implementation of a comprehensive SPH program. The primary objectives of the research were to determine the effectiveness of the SPH program in improving direct care nursing outcomes and to provide a context for understanding variations in program results across sites over time. Secondary objectives of the present research were to evaluate the effectiveness of the program in reducing direct and indirect costs associated with patient handling, to explore the potential mediating and moderating mechanisms, and to identify unintended consequences of implementing the program.

Methods

This 3-year longitudinal study used mixed methods of data collection at 6- to 9-month intervals. The analyses will include data from surveys, administrative databases, individual and focus group interviews, and nonparticipant observations. For this study, a 3-tiered measurement plan was used. For Tier 1, the unit of analysis was the facility, the data source was the facility coordinator or administrative data, and all 153 VAMCs participated. For Tier 2, frontline caregivers and program peer leaders at 17 facilities each completed different surveys. For Tier 3, six facilities completed qualitative site visits, which included individual interviews, focus groups, and nonparticipant observations. Multiple regression models were proposed to test the effects of SPH components on nursing outcomes related to patient handling. Content analysis and constant comparative analysis were proposed for qualitative data analysis to understand the context of implementation and to triangulate quantitative data.

Results

All three tiers of data for this study have been collected. We are now in the analyses and writing phase of the project, with the possibility for extraction of additional administrative data. The focus of this paper is to describe the SPH program, its evaluation study design, and its data collection procedures. This study evaluates the effects associated with the national implementation of a comprehensive SPH program that was implemented in all 153 VAMCs throughout the United States to reduce patient handling injuries.

Conclusions

To our knowledge, this is the largest evaluation of an SPH program in the United States. A major strength of this observational study design is that all VAMCs implemented the program and were included in Tier 1 of the study; therefore, population sampling bias is not a concern. Although the design lacks a comparison group for testing program effects, this longitudinal field study design allows for capturing program dose-response effects within a naturalistic context. Implementation of the VA-wide SPH program afforded the opportunity for rigorous evaluation in a naturalistic context. Findings will guide VA operations for policy and decision making about resources, and will be useful for health care, in general, outside of the VA, in implementation and impact of an SPH program.

Effects of a universal classroom behavior management program in first and second grades on young adult behavioral, psychiatric, and social outcomes

BACKGROUND

The Good Behavior Game (GBG), a method of classroom behavior management used by teachers, was tested in first- and second-grade classrooms in 19 Baltimore City Public Schools beginning in the 1985-1986 school year. The intervention was directed at the classroom as a whole to socialize children to the student role and reduce aggressive, disruptive behaviors, confirmed antecedents of later substance abuse and dependence disorders, smoking, and antisocial personality disorder. This article reports on impact to ages 19-21.

METHODS

In five poor to lower-middle class, mainly African American urban areas, three or four schools were matched and within each set randomly assigned to one of three conditions: (1) GBG, (2) a curriculum-and-instruction program directed at reading achievement, or (3) the standard program. Balanced assignment of children to classrooms was made, and then, within intervention schools, classrooms and teachers were randomly assigned to intervention or control.

RESULTS

By young adulthood significant impact was found among males, particularly those in first grade who were more aggressive, disruptive, in reduced drug and alcohol abuse/dependence disorders, regular smoking, and antisocial personality disorder. These results underline the value of a first-grade universal prevention intervention. REPLICATION: A replication was implemented with the next cohort of first-grade children with the same teachers during the following school year, but with diminished mentoring and monitoring of teachers. The results showed significant GBG impact for males on drug abuse/dependence disorders with some variation. For other outcomes the effects were generally smaller but in the predicted direction.

Impact of the Good Behavior Game, a universal classroom-based behavior intervention, on young adult service use for problems with emotions, behavior, or drugs or alcohol

BACKGROUND

The Good Behavior Game (GBG) is a classroom behavior management strategy focused on socializing children to the role of student and aimed at reducing early aggressive, disruptive behavior, a confirmed antecedent to service use. The GBG was tested in a randomized field trial in 19 elementary schools with two cohorts of children as they attended first and second grades. This article reports on the impact of the GBG on service use through young adulthood.

METHODS

Three or four schools in each of five urban areas were matched and randomly assigned to one of three conditions: (1) GBG, (2) an intervention aimed at academic achievement, or (3) the standard program of the school system. Children were assigned to classrooms to ensure balance, and teachers and classrooms were randomly assigned to intervention conditions.

RESULTS

This study provides evidence of a positive impact of a universal preventive intervention on later service use by males, although not by females, for problems with emotions, behavior, or drugs or alcohol. For both cohorts, males in GBG classrooms who had been rated as highly aggressive, disruptive by their teachers in the fall of first grade had a lower rate of school-based service use than their counterparts in control classrooms. REPLICATION: The design employed two cohorts of students. Although both first- and second-grade teachers received less training and support with the second cohorts of students than with the first cohort, the impact of GBG was similar across both cohorts.

Methods for testing theory and evaluating impact in randomized field trials: intent-to-treat analyses for integrating the perspectives of person, place, and time

Randomized field trials provide unique opportunities to examine the effectiveness of an intervention in real world settings and to test and extend both theory of etiology and theory of intervention. These trials are designed not only to test for overall intervention impact but also to examine how impact varies as a function of individual level characteristics, context, and across time. Examination of such variation in impact requires analytical methods that take into account the trial's multiple nested structure and the evolving changes in outcomes over time. The models that we describe here merge multilevel modeling with growth modeling, allowing for variation in impact to be represented through discrete mixtures--growth mixture models--and nonparametric smooth functions--generalized additive mixed models. These methods are part of an emerging class of multilevel growth mixture models, and we illustrate these with models that examine overall impact and variation in impact. In this paper, we define intent-to-treat analyses in group-randomized multilevel field trials and discuss appropriate ways to identify, examine, and test for variation in impact without inflating the Type I error rate. We describe how to make causal inferences more robust to misspecification of covariates in such analyses and how to summarize and present these interactive intervention effects clearly. Practical strategies for reducing model complexity, checking model fit, and handling missing data are discussed using six randomized field trials to show how these methods may be used across trials randomized at different levels.