Validation of the conceptual research utilization scale: an application of the standards for educational and psychological testing in healthcare

How context links to best practice use in long-term care homes: a mixed methods study

BACKGROUND

Context (work environment) plays a crucial role in implementing evidence-based best practices within health care settings. Context is multi-faceted and its complex relationship with best practice use by care aides in long-term care (LTC) homes are understudied. This study used an innovative approach to investigate how context elements interrelate and influence best practice use by LTC care aides.

METHODS

In this secondary analysis study, we combined coincidence analysis (a configurational comparative method) and qualitative analysis to examine data collected through the Translating Research in Elder Care (TREC) program. Coincidence analysis of clinical microsystem (care unit)-level data aggregated from a survey of 1,506 care aides across 36 Canadian LTC homes identified configurations (paths) of context elements linked consistently to care aides' best practices use, measured with a scale of conceptual research use (CRU). Qualitative analysis of ethnographic case study data from 3 LTC homes (co-occurring with the survey) further informed interpretation of the configurations.

RESULTS

Three paths led to very high CRU at the care unit level: very high leadership; frequent use of educational materials; or a combination of very high social capital (teamwork) and frequent communication between care aides and clinical educators or specialists. Conversely, 2 paths led to very low CRU, consisting of 3 context elements related to unfavorable conditions in relationships, resources, and formal learning opportunities. Our qualitative analysis provided insights into how specific context elements served as facilitators or barriers for best practices. This qualitative exploration was especially helpful in understanding 2 of the paths, illustrating the pivotal role of leadership and the function of teamwork in mitigating the negative impact of time constraints.

CONCLUSIONS

Our study deepens understanding of the complex interrelationships between context elements and their impact on the implementation of best practices in LTC homes. The findings underscore that there is no singular, universal bundle of context-related elements that enhance or hinder best practice use in LTC homes.

Measuring older adults' wellbeing when transitioning into assisted living facilities: a confirmatory factor analysis of the Mueller assessment of transition (MAT)

OBJECTIVES

Older adults' wellbeing during the transition into an assisted living facility (ALF) is not well understood and may influence their wellbeing. The Mueller Assessment of Transition (MAT) was created to measure the impact of transition on older adults' wellbeing. Early developmental testing of the MAT revealed a hypothesized model with two constructs (adjustment strategies and constraints to wellbeing). Therefore, the purpose of this study was to confirm the factor structure of the MAT with a representative sample of older adults transitioning into ALFs.

METHODS

In a nationwide sample, 108 older adult participants completed the MAT to measure wellbeing when relocating into ALFs. Confirmatory factor analysis (CFA) assessed the structural validity of the MAT. Internal consistency was evaluated, and chi-square tests of association for regional differences in MAT scores were also conducted.

RESULTS

The CFA produced strong fit indices to confirm the hypothesized 2-factor (constraints to wellbeing and adjustment strategies) model of the MAT. Cronbach's alpha for the internal consistency was 0.784 and chi-square test indicated no significant regional differences.

CONCLUSION

The MAT was established as a valid and reliable standardized assessment. Implications for using the MAT as a tool to measure older adults' wellbeing and future research are discussed.

Nursing assistants' use of best practices and pain in older adults living in nursing homes

BACKGROUND

Inadequate pain management persists in nursing homes. Nursing assistants provide the most direct care in nursing homes, and significantly improving the quality of care requires their adoption of best care practices informed by the best available evidence. We assessed the association between nursing assistants' use of best practices and residents' pain levels.

METHODS

We performed a cross-sectional analysis of data collected between September 2019 and February 2020 from a stratified random sample of 87 urban nursing homes in western Canada. We linked administrative data (the Resident Assessment Instrument-Minimum Data Set [RAI-MDS], 2.0) for 10,093 residents and survey data for 3547 nursing assistants (response rate: 74.2%) at the care unit level. Outcome of interest was residents' pain level, measured by the pain scale derived from RAI-MDS, 2.0. The exposure variable was nursing assistants' use of best practices, measured with validated self-report scales and aggregated to the unit level. Two-level random-intercept multinomial logistic regression accounted for the clustering effect of residents within care units. Covariates included resident demographics and clinical characteristics and characteristics of nursing assistants, unit, and nursing home.

RESULTS

Of the residents, 3305 (30.3%) were identified as having pain. On resident care units with higher levels of best practice use among nursing assistants, residents had 32% higher odds of having mild pain (odds ratio, 1.32; 95% confidence interval, 1.01-1.71; p = 0.040), compared with residents on care units with lower levels of best practice use among nursing assistants. The care units did not differ in reported moderate or severe pain among residents.

CONCLUSIONS

We observed that higher unit-level best practice use among nursing assistants was associated with mild resident pain. This association warrants further research to identify key individual and organizational factors that promote effective pain assessment and management.

Development and validation of a new tool: The sarcopenia knowledge questionnaire

We developed a new questionnaire-the Sarcopenia Knowledge Questionnaire (SKQ)-to evaluate the level of awareness about sarcopenia among older adults and tested the reliability and validity of this tool. A total of 293 older adults completed the questionnaire. The SKQ comprises three domains including 23 items: screening and diagnosis (10 items), sarcopenia outcomes (7 items), and lifestyle factors (6 items). The Cronbach's α value was 0.969, which indicated excellent internal consistency. The SKQ correlated well with the Mandarin Multidimensional Health Literacy Questionnaire (r = 0.511; p < 0.001), confirming its moderate convergent validity. The absolute values of the critical ratio ranged from 9.90 to 25.82 (p < 0.001), indicating satisfactory item discrimination. Thus, the SKQ appears to be a valid and reliable instrument for evaluating the knowledge of older adults about sarcopenia.

Safer Care for Older Persons in (residential) Environments (SCOPE): a pragmatic controlled trial of a care aide-led quality improvement intervention

BACKGROUND

The increased complexity of residents and increased needs for care in long-term care (LTC) have not been met with increased staffing. There remains a need to improve the quality of care for residents. Care aides, providers of the bulk of direct care, are well placed to contribute to quality improvement efforts but are often excluded from so doing. This study examined the effect of a facilitation intervention enabling care aides to lead quality improvement efforts and improve the use of evidence-informed best practices. The eventual goal was to improve both the quality of care for older residents in LTC homes and the engagement and empowerment of care aides in leading quality improvement efforts.

METHODS

Intervention teams participated in a year-long facilitative intervention which supported care aide-led teams to test changes in care provision to residents using a combination of networking and QI education meetings, and quality advisor and senior leader support. This was a controlled trial with random selection of intervention clinical care units matched 1:1 post hoc with control units. The primary outcome, between group change in conceptual research use (CRU), was supplemented by secondary staff- and resident-level outcome measures. A power calculation based upon pilot data effect sizes resulted in a sample size of 25 intervention sites.

RESULTS

The final sample included 32 intervention care units matched to 32 units in the control group. In an adjusted model, there was no statistically significant difference between intervention and control units for CRU or in secondary staff outcomes. Compared to baseline, resident-adjusted pain scores were statistically significantly reduced (less pain) in the intervention group (p=0.02). The level of resident dependency significantly decreased statistically for residents whose teams addressed mobility (p<0.0001) compared to baseline.

CONCLUSIONS

The Safer Care for Older Persons in (residential) Environments (SCOPE) intervention resulted in a smaller change in its primary outcome than initially expected resulting in a study underpowered to detect a difference. These findings should inform sample size calculations of future studies of this nature if using similar outcome measures. This study highlights the problem with measures drawn from current LTC databases to capture change in this population. Importantly, findings from the trial's concurrent process evaluation provide important insights into interpretation of main trial data, highlight the need for such evaluations of complex trials, and suggest the need to consider more broadly what constitutes "success" in complex interventions.

TRIAL REGISTRATION

ClinicalTrials.gov , NCT03426072, registered August 02, 2018, first participant site April, 05, 2018.

Validity and Reliability of a Questionnaire on the Knowledge, Attitudes, Perceptions and Practices toward Food Poisoning among Malaysian Secondary School Students: A Pilot Study

Children in school settings are at risk of contracting food poisoning due to inadequate food safety practices and safe eating behaviors. This research aimed to develop a valid and reliable questionnaire on the knowledge, attitudes, perceptions, and practices (KAP2) toward food poisoning and its prevention among secondary school students. The questionnaire was developed by considering the Health Belief Model (HBM). A pilot study using a cross-sectional survey was conducted in Tangkak, Johor, among 30 selected students using a convenience sampling method. A pre-test was conducted on 15 secondary school students aged 13–17 years old prior to the pilot study, and they were excluded from the pilot study. The constructed knowledge was assessed using the difficulty and discrimination indices. Meanwhile, the reliability of the attitude, practice, and perception components in the questionnaire were assessed using Cronbach’s alpha. Regarding knowledge (34 items), the difficulty index showed that most items (n = 28) were easy, while one was difficult, and the remaining five were within an acceptable range. In addition, the discrimination index of the knowledge component (34 items) showed that twenty-two, three, and one had good, acceptable, and excellent ranges, respectively. Only eight items had a low discrimination power. All items of the attitude and practice components (10 items for each) showed a corrected item-total correlation value of >0.30. Only four out of twenty-one items of the perception component showed an unacceptable range of <0.30. However, following a discussion with the experts, all items were retained. With the incorporation of the HBM, the 75-item adapted food poisoning KAP2 questionnaire is valid and reliable. It can be utilized to measure and generate food poisoning KAP2 among secondary school students in Malaysia.

Psychometric Properties of the Access of Older Adults to Outpatient Primary-Care Health Services Scale

This study evaluates the psychometric properties of the Access of Older Adults to Outpatient Primary-Care Health Services Scale (AOAOPHSS), in research conducted among 707 Mexican older adults selected by convenience from 14 rural and one urban locations. The AOAOPHSS explores 10 dimensions of two integrated subscales: Accessibility and Personal Abilities. Data analysis was performed in five phases. First, potentially biased responses were identified. Second, the response efficiency of the items and their association with external variables were evaluated. Third, the basic properties of the scores for the subscales' dimensions of the AOAOPHSS were identified using non-parametric Mokken Scaling Analysis (MSA). Fourth, the Structural Equation Modeling methodology was used to identify the properties of the internal structure of the latent construct. Finally, reliability and internal consistency were evaluated at both score and item levels. The following findings emerged. 13 items with inefficient response options were removed, and 24 were retained using the MSA. The latent structure of the latter was defined based on 21 items of five Accessibility Subscale dimensions. Its internal consistency reliability ranged between 0.67 and 0.81 (omega coefficients) and between 0.61 and 0.78 (alpha coefficients). Accordingly, this paper discusses the overall implications of using the Accessibility Subscale.

The Influence of Context on Implementation and Improvement: Protocol for a Mixed Methods, Secondary Analyses Study

BACKGROUND

Caring for the well-being of older adults is one of the greatest challenges in modern societies. Improving the quality of care and life for older adults and the work lives of their care providers calls for effective knowledge translation of evidence-based best practices.

OBJECTIVE

This study's purpose is to contribute to knowledge translation by better understanding the roles of organizational context (workplace environment) and facilitation (process or role) in implementation and improvement success. Our study has 2 goals: (1) to advance knowledge translation science by further developing and testing the Promoting Action on Research Implementation in Health Services framework (which outlines how implementation relies on the interplay of context, facilitation, and evidence) and (2) to advance research by optimizing implementation success via tailoring of modifiable elements of organizational context and facilitation.

METHODS

This is secondary analyses of 15 years of longitudinal data from the Translating Research in Elder Care (TREC) program's multiple data sources. This research is ongoing in long-term care (LTC) homes in western Canada. TREC data include the following: 5 waves of survey collection, 2 clinical trials, and regular ongoing outcome data for LTC residents. We will use a sequential exploratory and confirmatory mixed methods design. We will analyze qualitative and quantitative data holdings in an iterative process: (1) comprehensive reanalysis of qualitative data to derive hypotheses, (2) quantitative modeling to test hypotheses, and (3) action cycles to further refine and integrate qualitative and quantitative analyses. The research team includes 4 stakeholder panels: (1) system decision- and policy makers, (2) care home managers, (3) direct care staff, and (4) a citizen engagement group of people living with dementia and family members of LTC residents. A fifth group is our panel of external scientific advisors. Each panel will engage periodically, providing their perspectives on project direction and findings.

RESULTS

This study is funded by the Canadian Institutes of Health Research. Ethics approval was obtained from the University of Alberta (Pro00096541). The results of the secondary analyses are expected by the end of 2023.

CONCLUSIONS

The project will advance knowledge translation science by deepening our understanding of the roles of context, the interactions between context and facilitation, and their influence on resident and staff quality outcomes. Importantly, findings will inform understanding of the mechanisms by which context and facilitation affect the success of implementation and offer insights into factors that influence the implementation success of interventions in nursing homes.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

DERR1-10.2196/40611.

Content Validity of a Scale Designed to Measure the Access of Older Adults to Outpatient Health Services

The objective of this work was to validate the content of a scale formulated in Spanish for older adults in Mexico, with the aim of comprehensively measuring the access of this population group to outpatient primary-care services. To this end, we carried out a methodological content-validity study in four stages: (1) construction of the scale; (2) evaluation of item legibility; (3) quantitative content evaluation by two groups of judges selected by convenience: participant-judges including older adults with adequate reading comprehension, surveyed in person (n = 23), and expert-judges comprised of researchers specialized in the fields of health services, psychometrics and aging, surveyed online (n = 7); and (4) collection of qualitative feedback from several of the participant-judges (older adults, n = 4). The content was validated both by sequentially examining the level of consensus in the responses of both groups of judges, using the Tastle and Wierman method, and by calculating Aiken's Validity Coefficient with a 90% confidence interval. The scale contained 65 items pertaining to 10 dimensions of two major constructs: accessibility (n = 39) and personal abilities (n = 26). Five items were eliminated in accordance with the minimum-consensus criterion (0.5). This is the first psychometric scale to be developed in Mexico with the view of integrating the characteristics of health-care services and the abilities of the older adults in a single questionnaire designed to measure the access of this population group to outpatient primary-care services.

Validity, reliability, and acceptability of the Evidence-Informed Decision-Making (EIDM) competence measure

Valid, reliable, and acceptable tools for assessing self-reported competence in evidence-informed decision-making (EIDM) are required to provide insight into the current status of EIDM knowledge, skills, attitudes/beliefs, and behaviours for registered nurses working in public health. The purpose of this study was to assess the validity, reliability, and acceptability of the EIDM Competence Measure. A psychometric study design was employed guided by the Standards for Educational and Psychological Testing and general measurement development principles. All registered nurses working across 16 public health units in Ontario, Canada were invited to complete the newly developed EIDM Competence Measure via an online survey. The EIDM Competence Measure is a self-reported tool consisting of four EIDM subscales: 1) knowledge; 2) skills; 3) attitudes/beliefs; and 4) behaviours. Acceptability was measured by completion time and percentage of missing data of the original 40-item tool. The internal structure of the tool was first assessed through item-subscale total and item-item correlations within subscales for potential item reduction of the original 40-item tool. Following item reduction which resulted in a revised 27-item EIDM Competence Measure, a principal component analysis using an oblique rotation was performed to confirm the four subscale structure. Validity based on relationships to other variables was assessed by exploring associations between EIDM competence attributes and individual factors (e.g., years of nursing experience, education) and organizational factors (e.g., resource allocation). Internal reliability within each subscale was analyzed using Cronbach’s alphas. Across 16 participating public health units, 201 nurses (mean years as a registered nurse = 18.1, predominantly female n = 197; 98%) completed the EIDM Competence Measure. Overall missing data were minimal as 93% of participants completed the entire original 40-item tool (i.e., no missing data), with 7% of participants having one or more items with missing data. Only one participant (0.5%) had >10% of missing data (i.e., more than 4 out of 40 items with data missing). Mean completion time was 7 minutes and 20 seconds for the 40-item tool. Extraction of a four-factor model based on the 27-item version of the scale showed substantial factor loadings (>0.4) that aligned with the four EIDM subscales of knowledge, skills, attitudes/beliefs, and behaviours. Significant relationships between EIDM competence subscale scores and education, EIDM training, EIDM project involvement, and supportive organizational culture were observed. Cronbach’s alphas exceeded minimum standards for all subscales: knowledge (α = 0.96); skills (α = 0.93); attitudes/beliefs (α = 0.80); and behaviours (α = 0.94).

Psychometric properties and correlates of Chinese version of Perceived Stress Scale (CPSS-10) in people with common mental disorders with different employment Statuses

Background/objectives

Several studies have indicated that stress is associated with common mental disorders, and work stress trebles the risk of developing them. However, a validated assessment tool for measuring and establishing psychological stress correlates in this group of clients remains unavailable. The objectives of the present study were to examine the psychometric properties of the Chinese version of the Perceived Stress Scale-10 (CPSS-10) on people with common mental disorders with different employment statuses and explore its correlates.

Methods

Two hundred and fifty-two participants with common mental disorders were recruited. The data were analysed through exploratory factor and confirmatory analyses to investigate construct validity. The convergent and discriminant validities were examined based on their correlation with other measures, while the internal consistency was estimated using Cronbach's α coefficient. A t-test was used to detect differences between groups. The CPSS-10 correlates were explored using multiple linear regression analysis.

Results

Principal component analysis with varimax rotation yielded two factors, which accounted for 63.82% of the total variance, while confirmatory factor analysis confirmed its factor structure. The CPSS-10 had a positively moderate to strong correlation with other measures, thereby indicating its acceptable convergent and discriminant validities. The internal consistency ranged from acceptable to good for the two subscales and ten overall items, while the item-total correlation was adequate except for the seventh item. There were no group differences in gender nor employment status. Finally, the CPSS-10 predictors were studied.

Conclusion

The CPSS-10 is a reliable and valid instrument for people with common mental disorders with different employment statuses.

Development and validation of a knowledge, attitude, and practice questionnaire regarding cardiovascular diseases in an Iranian general population

Background

Studies on knowledge, attitude, and practice (KAP) can be valuable for public health to help to develop targeted educational programs and assess the effectiveness of intervention programs. The purpose of this study was to develop and examine the validity and reliability of a questionnaire on knowledge, attitude, and practice (KAP) regarding cardiovascular diseases (CVDs), their risk factors, and symptoms among an Iranian general population.

Methods

This cross-sectional study was conducted on an Iranian population older than 20 years referred to some of Tehran’s healthcare centers. An initial 62-item questionnaire was developed, and the face, content, and construct validities were assessed.

Results

In all, 300 adults with a mean age (SD) of 39.79 (12.1) years participated in this study. Based on the results of the content validity, a questionnaire with 30 essential items was designed. Exploratory factor analysis suggested a four-factor subscale with 29 finalized items (CVD-KAP29), and acceptable goodness of fit indices was demonstrated by confirmatory factor analysis. The Cronbach’s alpha and McDonald’s ω coefficients were higher than 0.60 for all domains except the nutrition and smoking subscales.

Conclusions

Results provided evidence of the validity of the CVD-KAP29 for KAP studies for cardiovascular diseases in the general population.

How Does the Facilitation Effort of Clinical Educators Interact With Aspects of Organizational Context to Affect Research Use in Long-term Care? Evidence From CHAID Analysis

PURPOSE

Organizational context influences the effect of facilitation efforts on research use in care settings. The interactions of these factors are complex. Therefore, the use of traditional statistical methods to examine their interrelationships is often impractical. Big Data analytics can automatically detect patterns within the data. We applied the chi-squared automatic interaction detection (CHAID) algorithm and classification tree technique to explore the dynamic and interdependent relationships between the implementation science concepts-context, facilitation, and research use.

DESIGN

Observational, cross-sectional study based on survey data collected from a representative sample of nursing homes in western Canada.

METHODS

We assessed three major constructs: (a) Conceptual research utilization (CRU) using the CRU scale; (b) facilitation of research use measured by the frequency of contacts between the frontline staff and a clinical educator, or person who brings new ideas to the care unit; and (c) organizational context at the unit level using the Alberta Context Tool (ACT). CHAID analysis was performed to detect the interactions between facilitation and context variables. Results were illustrated in a classification tree to provide a straightforward visualization.

FINDINGS

Data from 312 care units in three provinces were included in the final analysis. Results indicate significant multiway interactions between facilitation and various aspects of the organizational context, including leadership, culture, evaluation, structural resources, and organizational slack (staffing). Findings suggested the preconditions of the care settings where research use can be maximized.

CONCLUSIONS

CHAID analysis helped transform data into usable knowledge. Our findings provide insight into the dynamic relationships of facilitators' efforts and organizational context, and how these factors' interplay and their interdependence together may influence research use.

CLINICAL RELEVANCE

Knowledge of the combined effects of facilitators' efforts and various aspects of organizational context on research use can contribute to effective strategies to narrow the evidence-practice gap in care settings.

Development and content validation of a measure to assess evidence-informed decision-making competence in public health nursing

There are professional expectations for public health nurses to develop competencies in evidence-informed decision-making (EIDM) due to its potential for improved client outcomes. Robust tools to assess EIDM competence can encourage increased EIDM engagement and uptake. This study aimed to develop and validate the content of a measure to assess EIDM competence among public health nurses. A four-stage process, based on measure development principles and the Standards for Educational and Psychological Testing, was used to develop and refine items for a new EIDM competence measure: a) content coverage assessment of existing measures; b) identification of existing measures for use and development of items; c) validity assessment based on content; d) validity assessment based on response process. An EIDM competence measurement tool consisting of EIDM knowledge, skills, attitudes/beliefs, and behaviour items was developed using conceptual literature and existing measures (Evidence-Based Practice Competency Tool and Evidence-Based Practice Beliefs Scale) to address limitations of existing EIDM tools identified from the content coverage assessment. Item content validity index ratings ranged from 0.64–1.00. Qualitative themes from validity assessment based on content and response process included word changes to improve clarity, reducing item redundancy, separating multi-component items, and ensuring items reflect nursing role expectations. Upon determining its reliability and validity, there is potential for the EIDM competence measure to be used in: public health nursing practice to identify competence gaps and strengths to facilitate professional development activities; in research to support development of strategies to build EIDM capacity; and for curriculum planning and development across nursing education programs.

Capturing the impact of cultural differences in residency

BACKGROUND

Postgraduate training is a period in which residents develop both their medical competency and their professional identity in an environment of apprenticeship. As situated learning suggests, a critical dimension of such apprenticeship is the mode through which trainees can legitimately participate in the practice before they become experts, in this case physicians. One source of barriers to participation is cultural difference between learner and the clinical environment.

OBJECTIVE

To assess the extent cultural differences create barriers for residents, particularly but not exclusively for international medical graduates (IMGs).

METHODS

In 2014-15 a questionnaire was developed with subscales assessing areas such as sense of hierarchy, individuality versus teamwork, and risk tolerance. We refined the instrument by subjecting it to a review panel of experts in postgraduate education followed by "think aloud" sessions with residents.

RESULTS

Piloting this instrument yielded a Cronbach's alpha of 0.675. When administered to a larger group of residents and faculty representing many specialties, the Impact of Cultural Differences on Residency Experiences (ICDRE) questionnaire revealed a few items for which the Canadian Medical Graduates and International Medical Graduates differed in their mean opinion. The groups were not substantially different overall, but we did observe an interesting diversity of cultural beliefs within each group.

CONCLUSIONS

We suggest that the ICDRE may be useful in identifying beliefs which may present challenges to an individual resident or in capturing trends in a resident population so that a specialty program can address the trends proactively. The instrument also provides language with which to anchor preceptors' evaluations of residents' professionalism and may serve as an interventional coaching tool.

Accelerating the conceptual use of behavioral health research in juvenile court decision-making: study protocol

BACKGROUND

The youth criminal-legal system is under heavy political scrutiny with multiple calls for significant transformation. Leaders within the system are faced with rethinking traditional models and are likely to benefit from behavioral health research evidence as they redesign systems. Little is known about how juvenile court systems access and use behavioral health research evidence; further, the field lacks a validated survey measure of behavioral health research use that can be used to evaluate the effectiveness of evidence dissemination interventions for policy and system leaders. Conceptual research use is a particularly salient construct for system reform as it describes the process of shifting awareness and the consideration of new frameworks for action. A tool designed to measure the conceptual use of behavioral health research would advance the field's ability to develop effective models of research evidence dissemination, including collaborative planning models to support the use of behavioral health research in reforms of the criminal-legal system.

METHODS

The ARC Study is a longitudinal, cohort and measurement validation study. It will proceed in two phases. The first phase will focus on measure development using established methods of construct validity (theoretical review, Delphi methods for expert review, cognitive interviewing). The second phase will involve gathering responses from the developed survey to examine scale psychometrics using Rasch analyses, change sensitivity analyses, and associations between research use exposure and conceptual research use among juvenile court leaders. We will recruit juvenile court leaders (judges, administrators, managers, supervisors) from 80 juvenile court jurisdictions with an anticipated sample size of n = 520 respondents.

DISCUSSION

The study will introduce a new measurement tool for the field that will advance implementation science methods for the study of behavioral health research evidence use in complex policy and decision-making interventions. To date, there are few validated survey measures of conceptual research use and no measures that are validated for measuring change in conceptual frameworks over time among agency leaders. While the study is most directly related to leaders in the youth criminal-legal system, the findings are expected to be informative for research focused on leadership and decision-making in diverse fields.

Research utilization process model: A cyclical, spiral, and developmental process to provide conclusive research knowledge in health professions education

Background: Research utilization plays an important role in evidence-based practice. Health professions education is a research-based discipline. Conclusive research knowledge is ranked as high level of evidence in evidence-based practice. This study was conducted to develop a model to reach conclusive research knowledge in health professions education.

Methods: This study was conducted in 2 phases. In the first phase, a qualitative content analysis of the literature was performed, which provided research utilization concepts to design a model. In the second phase, a group of 5 consultants of health professions education designed a preliminary model and revised it to develop an appropriate model for research utilization in health professions education.

Results: The indicators mentioned in 24 documents were extracted and later categorized into 25 items. Then, 8 concepts emerged, including problem identification, research design and implementation, research reports, research publication, presentation, research access, adoption, and research use. These concepts were arranged to design a model for research utilization in health professions education.

Conclusion: This study illustrated a cyclical, spiral, and developmental model to reach conclusive research knowledge. This model emphasizes both primary and secondary research projects to reach conclusive research knowledge. Also, it distinguishes between aligned transfer and cooperative presentation in research knowledge dissemination. This model reveals the importance of the process use in health profession education. Also, it distinguishes between use and misuse of research findings.

Predictors of Nurses' Research Use in Canadian Long-term Care Homes

OBJECTIVES

We examined the influence of individual characteristics and organizational context features on nurses' self-reported use of research evidence in long-term care (LTC) homes.

DESIGN

A cross-sectional analysis of survey data collected in the Translating Research in Elder Care program.

SETTING AND PARTICIPANTS

756 nurses (registered nurses and licensed practical nurses) from 89 LTC homes in Western Canada.

METHODS

Generalized estimating equation modeling was used to identify which individual characteristics and organizational context features significantly predicted (P < .05) 3 kinds of self-reported research use by nurses: instrumental (the direct application of research findings), conceptual (using research findings to change thinking), and persuasive (using research findings to convince others).

RESULTS

Nurses reported a moderate to high level of research use. There were no significant differences in mean research use scores by nursing role. Only 2 variables were associated with all 3 kinds of research use: having a positive attitude toward research, and availability of structural and electronic resources. Additional variables associated with instrumental research use were problem-solving ability, engaging in formal interactions (eg, education sessions), and better perceptions of organizational slack-staff (the availability of sufficient staff). Additional variables associated with conceptual research use were self-determination and job efficacy. Finally, additional variables associated with persuasive research use were belief suspension (the ability to suspend previously held beliefs), organizational citizenship behavior (one's voluntary commitment to the organization), self-determination, job efficacy, evaluation, and better perceptions of organizational slack-time (perceived availability of extra time).

CONCLUSIONS AND IMPLICATIONS

Conceptual and persuasive research use were most strongly influenced by individual characteristics, whereas instrumental research use was predicted equally by individual and organizational variables. Nurses working in LTC are positioned in leadership roles; by targeting both the individual- and organizational-level predictors of nurses' research use, they can improve conditions for individuals living in LTC.

A Russian validation study of the Coma Recovery Scale-Revised (CRS-R)

INTRODUCTION

The aim of the study was to develop and validate a Russian adaptation of the Coma Recovery Scale-Revised (CRS-R).

SUBJECTS AND METHODS

We evaluated 58 patients with chronic disorders of consciousness (> 4 weeks post-injury, DOC) of various etiology and two patients in a locked-in state at different stages after coma. We tested sensitivity for changes over 1 week, reliability, criterion validity and diagnostic sensitivity of the Russian adaptation of the CRS-R in comparison with the Russian adaptations of Full Outline of UnResponsiveness Score (FOUR), and Glasgow Coma Scale (GCS).

RESULTS

We obtained good sensitivity for changes in neurological status over week (p < 0.0001) and good test-retest reliability (r = 0.997, p < 0.0001) of the CRS-R. Inter-rater reliability was good (κ = 0.99, p < 0.001). We showed high internal consistency (α = 0.87) of the scale and good criterion validity between other scales (r = 0.597 for GCS, and r = 0.900 for FOUR). CRS-R also demonstrated a higher sensitivity in differential diagnosis of DOC, as compared to GCS, and FOUR Score (p < 0.001).

CONCLUSION

The results show that the Russian version of the CRS-R is a valid and sensitive tool for the evaluation of patients with chronic DOC, which can be used for differential diagnosis and for recovery assessment.

Importance of clinical educators to research use and suggestions for better efficiency and effectiveness: results of a cross-sectional survey of care aides in Canadian long-term care facilities

OBJECTIVE

This study explored the effect of clinical educators as facilitators of research use and how it may be modified by organisational context in the settings.

DESIGN

Cross-sectional observational study.

SETTING

A representative sample of 91 residential long-term care (LTC) facilities across Western Canada.

PARTICIPANTS

We used surveys to collect data from the frontline care aides and information about the organisational context of the care units.

OUTCOME MEASURE AND EXPLANATORY VARIABLES

We assessed research use (the outcome) with the Conceptual Research Utilization (CRU) scale. Explanatory variables in the multiple regression analysis were facilitation, organisational context and the interaction terms. Facilitation was measured by the frequency of contacts between care aides and clinical educator or person who brings new ideas about resident care. Three core organisational context variables were measured using the Alberta Context Tool.

RESULTS

We included data of 3873 care aides from 294 care units in the LTC facilities. We found significant associations between CRU and facilitation, leadership, culture and evaluation. Interactions of facilitation x leadership and facilitation x culture were negative. The coefficient of the facilitation x evaluation term in the regression model was positive (0.019, 95% CI 0.012 to 0.026), suggesting synergistic effects between facilitation and a well-developed process to evaluate care quality using relevant data.

CONCLUSIONS

Findings indicate clinical educators are effective facilitators of research use among the care aides, but the effect is modified by organisational context. For greatest impact, managers can direct efforts of the clinical educators to care units where leadership and culture ratings are lowest, but a proficient feedback and evaluation process is in place. This understanding enables managers to deploy clinical educators (a scarce resource in LTC settings) most efficiently.

Predicting high school students' health-promoting lifestyle: a test of the information, motivation, behavioral skills model

Background and aim Expenditure on adolescents' health promotion is critical to enhancing quality of life and fundamental for the prosperity of countries. This study tried to investigate the relationships between information-motivation-behavioral skills (IMB) model constructs as predictors of high school student's health-promoting lifestyle (HPL) in Southwest Iran. Methods This cross-sectional descriptive-analytic study was conducted on 400 students in Southwest Iran. Sample size required for this study was obtained using Soper's structural equation modeling (SEM) calculator. The stratified random sampling method was used in this study. A path analysis was taken to evaluate the HPL main determinants utilizing SEM. Results Information only had a significant indirect relationship with adolescents' HPL. But, both motivation and behavioral skills constructs had significant direct and indirect relationship with adolescents' HPL. The behavioral skill construct was the most imperative predictor variable. In general, behavioral skills construct (75%), motivation (66%) and knowledge (50%) constructs had the highest impact on HPL. The IMB model was able to explain 71% of the variance in adolescents' HPL. Conclusion The findings of the study showed that the IMB model was appropriate for explaining adolescents' HPL. It is necessary that actions are taken to improve behavioral skills and promote adolescents' HPL.

Development and feasibility of Mandarin-language bone health scales for adolescents with cancer in Taiwan

Cancer during adolescence increases the risk for bone mass deficiency later in life. Adolescents with cancer must learn to improve their bone health to avoid osteoporosis. In the present cross-sectional study, we developed and tested scales to assess the bone health self-efficacy and beliefs of adolescents with cancer in Taiwan. Test development followed three stages: item generation and scale formatting, examination of content validity, and validation of psychometric properties with a sample of 100 adolescents with cancer. Through the validation process, this research generated the seven-item Bone Health Self-Efficacy Scale and the 13-item Bone Health Belief Scale. Multiple indices demonstrated construct validity. Cronbach's alphas (0.809 and 0.705) demonstrated internal consistency. No items caused a drop in Cronbach's alpha of 10%, all inter-item correlations were <0.800, and the factor loadings for all items reached 0.400, demonstrating item appropriateness. The present study provides initial evidence of the scales' accessibility and feasibility for adolescents with cancer who speak Mandarin. These scales might also help clinical nurses evaluate the effectiveness of bone health education provided to adolescents with cancer.

The Influence of Context on Utilizing Research Evidence for Pain Management in Jordanian Pediatric Intensive Care Units (PICU)

PURPOSE

The purpose of this study was to gain a beginning understanding of the contextual factors that influence the use of research for pain management in Jordanian Pediatric Intensive Care Units (PICUs).

DESIGN AND METHODS

A paper or online questionnaire was used to collect data on instrumental research use (IRU) and conceptual research use (CRU) and ten contextual variables from 73 registered nurses working in four Jordanian PICUs. The Pearson product-moment correlation coefficient was used to test the relationship between continuous (demographic and contextual) factors and IRU and CRU. One way ANOVA and independent t-test were used to examine the differences between sociodemographic variables and IRU and CRU. Generalized Estimating Equations (GEE) was used to determine the demographic and contextual factors that influenced research use. We modeled the significant variables identified by bivariate correlation, t-test, and ANOVA at (p≤0.10).

RESULTS

Nine of the contextual factors significantly and positively correlated with the IRU for pain assessment, eight with the IRU for pain treatment, and six with the CRU for pain management (including assessment and treatment). Hospital type (public) predicted the IRU for pain assessment. Social capital, structural, and electronic resources predicted the IRU for pain treatment. Social capital predicted the CRU for pain management.

CONCLUSION

Context influences Jordanian PICU nurses' use of research for pain management.

PRACTICE IMPLICATIONS

Concentrating on modifiable contextual factors may positively influence Jordanian PICU nurses' use of research for pain management. This influence may extend to reduce children's pain in Jordanian PICUs.

Organizational supports used by private child and family serving agencies to facilitate evidence use: a mixed methods study protocol

Background

Challenges to evidence use are well documented. Less well understood are the formal supports—e.g., technical infrastructure, inter-organizational relationships—organizations may put in place to help overcome these challenges. This study will identify supports for evidence use currently used by private child and family serving agencies delivering publicly funded behavioral health and/or human services; examine contextual, organizational, and managerial factors associated with use of such supports; and determine how identified supports affect evidence use by staff at multiple levels of the organization.

Methods

We will use a sequential explanatory mixed methods design, with study activities occurring in two sequential phases: In phase 1, quantitative survey data collected from managers of private child and family serving agencies in six states (CA, IN, KY, MO, PA, and WI) and analyzed using both regression and qualitative comparative analysis (QCA) will identify organizational supports currently being used to facilitate evidence use and examine the contextual, organizational, and managerial factors associated with the use of such supports. In phase 2, data from phase 1 will be used to select a purposive sample of 12 agencies for in-depth case studies. In those 12 agencies, semi-structured interviews with key informants and managers, focus groups with frontline staff, and document analysis will provide further insight into agencies’ motivation for investing in organizational supports for evidence use and the facilitators and barriers encountered in doing so. Semi-structured interviews with managers and focus groups with frontline staff will also assess whether and how identified supports affect evidence use at different levels of the organization (senior executives, middle managers, frontline supervisors, and frontline staff). Within- and between-case analyses supplemented by QCA will identify combinations of factors associated with the highest and lowest levels of staff evidence use.

Discussion

This study will inform efforts to improve sustainment, scale-up, and spread of evidence by providing insight into organizational and managerial strategies that facilitate evidence use, the contexts in which these strategies are most effective, and their effect on evidence use by staff at different levels of the organization.

The Pediatric/Adolescent Shoulder Survey (PASS): A Reliable Youth Questionnaire With Discriminant Validity and Responsiveness to Change

BACKGROUND

Health-related quality of life tools for shoulder pathology in adults have been developed, such as the Disabilities of the Arm, Shoulder, and Hand (DASH) or QuickDASH. However, some content is not applicable to or readily grasped by a younger population.

PURPOSE

To evaluate the psychometric properties of a new 13-item pediatric/adolescent shoulder survey (PASS).

STUDY DESIGN

Cohort study (diagnosis); Level of evidence, 2.

METHODS

An initial development cohort of children was given the PASS, QuickDASH, and single assessment numeric evaluation (SANE) in a prospective fashion. Internal reliability and concurrent validity were evaluated. Subsequent to this, the PASS, along with the QuickDASH, was incorporated into routine clinical evaluation of shoulder injuries. The secondary cohort was established via a retrospective review of patients (age <18 years) evaluated in our sports clinics for shoulder complaints. Discriminant validity and floor/ceiling effects were evaluated. A subset of patients with scores available both pre- and within 6 months postoperatively was analyzed to assess responsiveness to treatment.

RESULTS

Overall internal reliability of the shoulder survey was good in the development cohort of 127 patients, with Cronbach α = 0.86. Total score averaged 59.7% ± 17.9% in this cohort (range, 18.75%-100%). Significant correlations between all surveys were observed (r = 0.51-0.79, P < .05). A total of 127 surveys were available for the secondary cohort. The QuickDASH demonstrated significant differences between sexes with regard to scores (P < .05). Both tools were able to discriminate between patients with acute injuries and diminished clinical examinations (P < .05). A significant correlation was observed between duration of symptoms and scores on the PASS. The subset of 25 patients with pre- and postoperative data showed significant change in total score and the symptoms/limitations domain of the PASS, with significant correlation with length of time since surgery.

CONCLUSION

This 13-question pediatric/adolescent-focused shoulder survey shows excellent psychometric properties, good discriminant ability, and early evidence of responsiveness to change, especially when compared with an adult-formatted questionnaire.

Factor Structure, Reliability and Measurement Invariance of the Alberta Context Tool and the Conceptual Research Utilization Scale, for German Residential Long Term Care

We translated the Canadian residential long term care versions of the Alberta Context Tool (ACT) and the Conceptual Research Utilization (CRU) Scale into German, to study the association between organizational context factors and research utilization in German nursing homes. The rigorous translation process was based on best practice guidelines for tool translation, and we previously published methods and results of this process in two papers. Both instruments are self-report questionnaires used with care providers working in nursing homes. The aim of this study was to assess the factor structure, reliability, and measurement invariance (MI) between care provider groups responding to these instruments. In a stratified random sample of 38 nursing homes in one German region (Metropolregion Rhein-Neckar), we collected questionnaires from 273 care aides, 196 regulated nurses, 152 allied health providers, 6 quality improvement specialists, 129 clinical leaders, and 65 nursing students. The factor structure was assessed using confirmatory factor models. The first model included all 10 ACT concepts. We also decided a priori to run two separate models for the scale-based and the count-based ACT concepts as suggested by the instrument developers. The fourth model included the five CRU Scale items. Reliability scores were calculated based on the parameters of the best-fitting factor models. Multiple-group confirmatory factor models were used to assess MI between provider groups. Rather than the hypothesized ten-factor structure of the ACT, confirmatory factor models suggested 13 factors. The one-factor solution of the CRU Scale was confirmed. The reliability was acceptable (>0.7 in the entire sample and in all provider groups) for 10 of 13 ACT concepts, and high (0.90-0.96) for the CRU Scale. We could demonstrate partial strong MI for both ACT models and partial strict MI for the CRU Scale. Our results suggest that the scores of the German ACT and the CRU Scale for nursing homes are acceptably reliable and valid. However, as the ACT lacked strict MI, observed variables (or scale scores based on them) cannot be compared between provider groups. Rather, group comparisons should be based on latent variable models, which consider the different residual variances of each group.

[Barriers and motivations of nurses for conducting research in Intensive Care Units and Emergency Medical Service]

BACKGROUND

The implementation of evidence based practice is essential in clinical practice. However, it is still a challenge in critical care patients.

AIM

To identify the barriers for conducting research that nursing professionals perceive in intensive care and medical emergency departments, as well as to investigate the areas of interest and motivations to carry out research projects.

METHOD

Cross-sectional and multicentre study carried out in 4 intensive care units and in one Medical Emergency Department emergency pre-hospital carein Catalonia on 2014. The instrument used was The Barriers to Research Utilization Scale which had been previously validated into Spanish. A descriptive and bivariate analysis was performed. A statistical significance of P<.05 was assumed.

RESULTS

One hundred seventy-two questionnaires were obtained (69.9% response). Of the total, 135 were from critical care, 27 to pre-hospital care, and 10 from both. Just over half (57.3%) had research experience, although 44.4% had related training. The questionnaire dimension considered most relevant was organisational characteristics. The most important barriers were: there is not enough time at work [3.11 (SD 1.21)], physicians do not collaborate in its implementation [2.99 (SD 1.22)], and nurses are isolated with respect to other professionals [2.86 (SD 1.32)]. Significant differences were observed in the barriers according to research experience and work place. The main motivation was to be updated in critical patient care.

CONCLUSIONS

The main barriers perceived are related to the organisation. There are differences in the barriers according to research experience and work place.

Validation of the Turkish Cervical Cancer and Human Papilloma Virus Awareness Questionnaire

AIM

The aim of this study was to determine the validity and reliability of the 'Cervical Cancer and Human Papilloma Virus Awareness Questionnaire' among fertility age women by adapting the scale into Turkish.

BACKGROUND

Cervical cancer is the fourth most commonly form seen among women. Death from cervical cancer ranks third among causes and is one of the most preventable forms of cancer.

METHODS

This cross-sectional study included 360 women from three family health centres between January 5 and June 25, 2014.

RESULTS

Internal consistency showed that the Kuder-Richardson 21 reliability coefficient in the first part was 0.60, Cronbach's alpha reliability coefficient was 0.61 in the second part. The Kaiser-Meyer-Olkin value of the items on the scale was 0.712. The Barlett test was significant. The confirmatory factor analysis indicated that the model matched the data adequately.

CONCLUSION

This study shows that the Turkish version of the instrument is a valid and reliable tool to evaluate knowledge, perceptions and preventive behaviours of women regarding human papilloma virus and cervical cancer.

IMPLICATIONS FOR NURSING AND POLICY MAKERS

Nurses who work in the clinical and primary care settings need to screen, detect and refer women who may be at risk from cervical cancer.

Development and testing of two implementation tools to measure components of professional pharmacy service fidelity

RATIONALE, AIMS AND OBJECTIVES

There is a need to evaluate both service process and implementation outcomes as professional services are being implemented into pharmacy practice. Fidelity is an implementation outcome, which may be used for service optimization, by associating service components to patient outcomes, as well as use in process evaluation. The objective of this study was to develop tools to measure components of fidelity, specifically, an adherence index (adherence of the service provider to the elements of the service) and a patient responsiveness scale for the professional pharmacy service, medication review with follow-up.

METHODS

The procedure described by DeVellis was followed to develop the tools. An expert panel was used to create items and establish content validity. Primary data were collected from 190 service provider pharmacists from 128 pharmacies across 11 provinces of Spain using Spanish version tools as part of an ongoing implementation study (English translations appended to the online version of the article as supplementary material). An initial assessment of item functionality was performed using descriptive statistics and item discrimination for both tools. The patient responsiveness scale's internal consistency was confirmed by calculating Cronbach's alpha coefficient and inter-item correlations. In addition, for the patient responsiveness scale, the number of factors to retain was based on Kaiser criterion, parallel analysis and Cattell's scree test and the number of items was optimized as guided by iterative exploratory factor analysis (EFA).

RESULTS

Acceptability of both tools was high. An adherence index of 39 items was developed. After five EFA iterations, four items were removed, resulting in a reliable, 12-item, two-factor patient responsiveness scale, explaining 53.9% of total variance.

CONCLUSIONS

Two tools for measuring implementation fidelity, an adherence index and a patient responsiveness scale, have been developed and tested. Future assessment, in particular to establish criterion validity, is recommended.

Using conjoint analysis to develop a system of scoring policymakers' use of research in policy and program development

Background

The importance of utilising the best available research evidence in the development of health policies, services, and programs is increasingly recognised, yet few standardised systems for quantifying policymakers’ research use are available. We developed a comprehensive measurement and scoring tool that assesses four domains of research use (i.e. instrumental, conceptual, tactical, and imposed). The scoring tool breaks down each domain into its key subactions like a checklist. Our aim was to develop a tool that assigned appropriate scores to each subaction based on its relative importance to undertaking evidence-informed health policymaking. In order to establish the relative importance of each research use subaction and generate this scoring system, we conducted conjoint analysis with a sample of knowledge translation experts.

Methods

Fifty-four experts were recruited to undertake four choice surveys. Respondents were shown combinations of research use subactions called profiles, and rated on a 1 to 9 scale whether each profile represented a limited (1–3), moderate (4–6), or extensive (7–9) example of research use. Generalised Estimating Equations were used to analyse respondents’ choice data, which calculated a utility coefficient for each subaction. A large utility coefficient indicated that a subaction was particularly influential in guiding experts’ ratings of extensive research use.

Results

Utility coefficients were calculated for each subaction, which became the points assigned to the subactions in the scoring system. The following subactions yielded the largest utilities and were regarded as the most important components of each research use domain: using research to directly influence the core of the policy decision; using research to inform alternative perspectives to deal with the policy issue; using research to persuade targeted stakeholders to support a predetermined decision; and using research because it was a mandated requirement by the policymaker’s organisation.

Conclusions

We have generated an empirically derived and context-sensitive means of measuring and scoring the extent to which policymakers used research to inform the development of a policy document. The scoring system can be used by organisations to not only quantify the extent of their research use, but also to provide them with insights into potential strategies to improve subsequent research use.

Electronic supplementary material

The online version of this article (doi:10.1186/s12961-015-0022-y) contains supplementary material, which is available to authorized users.

Reliability and Validity of the Alberta Context Tool (ACT) with Professional Nurses: Findings from a Multi-Study Analysis

Although organizational context is central to evidence-based practice, underdeveloped measurement hinders its assessment. The Alberta Context Tool, comprised of 59 items that tap 10 modifiable contextual concepts, was developed to address this gap. The purpose of this study to examine the reliability and validity of scores obtained when the Alberta Context Tool is completed by professional nurses across different healthcare settings. Five separate studies (N = 2361 nurses across different care settings) comprised the study sample. Reliability and validity were assessed. Cronbach's alpha exceeded 0.70 for 9/10 Alberta Context Tool concepts. Item-total correlations exceeded acceptable standards for 56/59 items. Confirmatory Factor Analyses coordinated acceptably with the Alberta Context Tool's proposed latent structure. The mean values for each Alberta Context Tool concept increased from low to high levels of research utilization(as hypothesized) further supporting its validity. This study provides robust evidence for reliability and validity of scores obtained with the Alberta Context Tool when administered to professional nurses.

An evaluation of chronic pain questionnaires in the adult population

BACKGROUND AND PURPOSE

Considering pain's subjectivity, measurement and its processes are indispensable to clinicians and researchers. Development and testing methods of recently published chronic pain questionnaires were analyzed to determine the state of measurement in chronic pain.

METHODS

There were 8 questionnaires analyzed against 28 criteria, which combined specific testing standards and commonly accepted reliability statistics.

RESULTS

Only 1 questionnaire received a rating of good method quality. The 7 remaining questionnaires received a rating of poor method quality.

CONCLUSIONS

Newly developed chronic pain self-report questionnaires revealed deficiencies in construction and testing methods. It is proposed that an adapted version of the Standards of Educational and Psychological Testing serves as a useful guide for developing and testing new health questionnaires.

Utilization of research findings for health policy making and practice: evidence from three case studies in Bangladesh

Background

In striving to contribute towards improved health outcomes, health research institutions generate and accumulate huge volumes of relevant but often underutilized data. This study explores activities undertaken by researchers from the International Centre for Diarrhoeal Disease Research, Bangladesh (icddr,b), an international research institution that promotes the utilization of their findings in the policymaking processes in Bangladesh.

Methods

The study used an exploratory case study design and employed qualitative methods to explore activities implemented to promote research utilization and the extent to which researchers felt that their findings contributed to the policymaking process. Data were collected between September and December 2011 through key informant interviews, focus group discussions with study investigators, and database and document reviews. We reviewed findings from 19 reproductive health studies conducted and completed by icddr,b researchers between 2001 and 2011. We interviewed 21 key informants, including 13 researchers, two policy makers, and six programme implementers. Data were entered into Microsoft Word and analyzed manually following a thematic framework approach. Following the World Health Organization/Turning Research into Practice (WHO/TRIP) framework, three case studies of how research findings were utilized in the policymaking processes in Bangladesh were documented.

Results

Activities implemented to promote research utilization included conducting dissemination workshops, publishing scientific papers, developing policy briefs, providing technical assistance to policymakers and programme implementers, holding one-on-one meetings, and joining advocacy networks. The majority of the researchers (12 of 13) reported that their study findings were utilized to influence policymaking processes at different levels. However, some researchers reported being unaware of whether and how their findings were utilized. As regards actual utilization of research findings, the evidence from the three case studies indicate that research findings can be utilized instrumentally, conceptually and symbolically, and at different stages within the policymaking process, including agenda setting and policy formulation and implementation.

Conclusions

The results show that research findings from icddr,b were promoted and utilized in health policymaking processes in Bangladesh using a variety of utilization approaches. These results suggest a need for using multiple approaches to promote utilization of research findings in health policymaking processes.

Screening for Voice Disorders in Older Adults (Rastreamento de Alterações Vocais em Idosos--RAVI)--Part I: Validity Evidence Based on Test Content and Response Processes

PURPOSE

To identify the validity evidence based on the content and response processes of the Rastreamento de Alterações Vocais em Idosos (RAVI; "Screening for Voice Disorders in Older Adults"), an epidemiologic screening for voice disorders in older adults.

STUDY DESIGN

This is a prospective, nonrandomized, cross-sectional, validation study.

METHODS

Criteria for defining the domains and elaborating the questions were established to confirm the validity evidence based on the content. A multidisciplinary committee of 19 experts evaluated the questions, and the relevance and representation of the domains were analyzed using the Content Validity Index for Items (CVI-I) and the Content Validity Index (CVI), respectively. For validity evidence based on the response processes, 40 individuals of both sexes, aged ≥60 years, were stratified by demographic and socioeconomic condition. They responded to the RAVI, made suggestions, and their verbal and nonverbal reactions were observed.

RESULTS

The first version of the RAVI consisted of 20 questions related to sensations and perceptions associated with the voice. Although the CVI value of 0.80 was satisfactory, the CVI-I and the suggestions of the expert committee indicated that the scale needed to be reformulated. Consultation with older adults indicated a need for further adjustment. The preliminary version of the RAVI consisted of 16 questions.

CONCLUSIONS

The two aspects of validity evidence described in the present study were essential for adapting the questions to better fit the construct of the questionnaire. Other aspects of validity evidence and reliability analysis will be described in part II of this study.

Using conjoint analysis to develop a system to score research engagement actions by health decision makers

Background

Effective use of research to inform policymaking can be strengthened by policymakers undertaking various research engagement actions (e.g., accessing, appraising, and applying research). Consequently, we developed a thorough measurement and scoring tool to assess whether and how policymakers undertook research engagement actions in the development of a policy document. This scoring tool breaks down each research engagement action into its key ‘subactions’ like a checklist. The primary aim was to develop the scoring tool further so that it assigned appropriate scores to each subaction based on its effectiveness for achieving evidence-informed policymaking. To establish the relative effectiveness of these subactions, we conducted a conjoint analysis, which was used to elicit the opinions and preferences of knowledge translation experts.

Method

Fifty-four knowledge translation experts were recruited to undertake six choice surveys. Respondents were exposed to combinations of research engagement subactions called ‘profiles’, and rated on a 1–9 scale whether each profile represented a limited (1–3), moderate (4–6), or extensive (7–9) example of each research engagement action. Generalised estimating equations were used to analyse respondents’ choice data, where a utility coefficient was calculated for each subaction. A large utility coefficient indicates that a subaction was influential in guiding experts’ ratings of extensive engagement with research.

Results

The calculated utilities were used as the points assigned to the subactions in the scoring system. The following subactions yielded the largest utilities and were regarded as the most important components of engaging with research: searching academic literature databases, obtaining systematic reviews and peer-reviewed research, appraising relevance by verifying its applicability to the policy context, appraising quality by evaluating the validity of the method and conclusions, engaging in thorough collaborations with researchers, and undertaking formal research projects to inform the policy in question.

Conclusions

We have generated an empirically-derived and context-sensitive method of measuring and scoring the extent to which policymakers engaged with research to inform policy development. The scoring system can be used by organisations to quantify staff research engagement actions and thus provide them with insights into what types of training, systems, and tools might improve their staff’s research use capacity.

Electronic supplementary material

The online version of this article (doi:10.1186/s12961-015-0013-z) contains supplementary material, which is available to authorized users.

The Nocturia Impact Diary: a self-reported impact measure to complement the voiding diary

BACKGROUND

Nocturia is a chronic, fluctuating disease that disrupts sleep and has a wide-ranging impact on quality of life. Valid tools to measure the patient-reported impact of nocturia are essential for evaluating the value of treatment, but the available tools are suboptimal.

OBJECTIVES

This study reports the development and validation of the Nocturia Impact Diary-an augmented form of the Nocturia Quality of Life questionnaire designed to be completed in conjunction with the widely used 3-day voiding diary.

METHODS

The process comprised three steps: Step 1: Development of a concept pool using the Nocturia Quality of Life questionnaire and data from relevant studies; Step 2: Content validity study; Step 3: Psychometric testing of construct validity, reliability, and sensitivity of the diary in a randomized, placebo-controlled study in patients with nocturia.

RESULTS

Step 1: Fourteen items and 4 domains were included in the first draft of the diary. Step 2: Twenty-three patients with nocturia participated in the cognitive debriefing study. Items were adjusted accordingly, and the content validity was high. Step 3: Fifty-six patients were randomized to desmopressin orally disintegrating tablet or placebo. The diary demonstrated high construct validity, with good sensitivity and a good fit to Rasch model, as well as high internal consistency, discriminatory ability, and acceptable sensitivity to change. Results indicated that the diary was unidimensional.

CONCLUSIONS

The Nocturia Impact Diary is a convenient, validated patient-reported outcome measure. It should be used in conjunction with a voiding diary to capture the real-life consequences of nocturia and its treatment.

Linguistic validation of the Alberta Context Tool and two measures of research use, for German residential long term care

Background

To study the association between organizational context and research utilization in German residential long term care (LTC), we translated three Canadian assessment instruments: the Alberta Context Tool (ACT), Estabrooks’ Kinds of Research Utilization (RU) items and the Conceptual Research Utilization Scale. Target groups for the tools were health care aides (HCAs), registered nurses (RNs), allied health professionals (AHPs), clinical specialists and care managers. Through a cognitive debriefing process, we assessed response processes validity–an initial stage of validity, necessary before more advanced validity assessment.

Methods

We included 39 participants (16 HCAs, 5 RNs, 7 AHPs, 5 specialists and 6 managers) from five residential LTC facilities. We created lists of questionnaire items containing problematic items plus items randomly selected from the pool of remaining items. After participants completed the questionnaires, we conducted individual semi-structured cognitive interviews using verbal probing. We asked participants to reflect on their answers for list items in detail. Participants’ answers were compared to concept maps defining the instrument concepts in detail. If at least two participants gave answers not matching concept map definitions, items were revised and re-tested with new target group participants.

Results

Cognitive debriefings started with HCAs. Based on the first round, we modified 4 of 58 ACT items, 1 ACT item stem and all 8 items of the RU tools. All items were understood by participants after another two rounds. We included revised HCA ACT items in the questionnaires for the other provider groups. In the RU tools for the other provider groups, we used different wording than the HCA version, as was done in the original English instruments. Only one cognitive debriefing round was needed with each of the other provider groups.

Conclusion

Cognitive debriefing is essential to detect and respond to problematic instrument items, particularly when translating instruments for heterogeneous, less well educated provider groups such as HCAs. Cognitive debriefing is an important step in research tool development and a vital component of establishing response process validity evidence. Publishing cognitive debriefing results helps researchers to determine potentially critical elements of the translated tools and assists with interpreting scores.

Precision of the Conceptual Research Utilization Scale

Background and Purpose: Conceptual research utilization (CRU) is one indicator of an optimum practice environment that leads to improved patient and organizational outcomes. Yet, its measurement has not been adequately addressed. In this study, we investigated precision of scores obtained with a new CRU scale using item response theory (IRT) methods. Methods: We analyzed the responses from 1,349 health care aides from 30 Canadian nursing homes using Samejima’s (1969, 1996) graded response model (GRM). Results: Findings suggest that the CRU scale is most precise at low to average trait levels with significantly less precision at higher trait levels. Conclusions: The scale showed acceptable precision at low to average trait levels. New items and/or different response options that capture higher trait levels are needed. Future development of the scale is discussed.

German translation of the Alberta Context Tool and two measures of research use: methods, challenges and lessons learned

Background

Understanding the relationship between organizational context and research utilization is key to reducing the research-practice gap in health care. This is particularly true in the residential long term care (LTC) setting where relatively little work has examined the influence of context on research implementation. Reliable, valid measures and tools are a prerequisite for studying organizational context and research utilization. Few such tools exist in German. We thus translated three such tools (the Alberta Context Tool and two measures of research use) into German for use in German residential LTC. We point out challenges and strategies for their solution unique to German residential LTC, and demonstrate how resolving specific challenges in the translation of the health care aide instrument version streamlined the translation process of versions for registered nurses, allied health providers, practice specialists, and managers.

Methods

Our translation methods were based on best practices and included two independent forward translations, reconciliation of the forward translations, expert panel discussions, two independent back translations, reconciliation of the back translations, back translation review, and cognitive debriefing.

Results

We categorized the challenges in this translation process into seven categories: (1) differing professional education of Canadian and German care providers, (2) risk that German translations would become grammatically complex, (3) wordings at risk of being misunderstood, (4) phrases/idioms non-existent in German, (5) lack of corresponding German words, (6) limited comprehensibility of corresponding German words, and (7) target persons’ unfamiliarity with activities detailed in survey items. Examples of each challenge are described with strategies that we used to manage the challenge.

Conclusion

Translating an existing instrument is complex and time-consuming, but a rigorous approach is necessary to obtain instrument equivalence. Essential components were (1) involvement of and co-operation with the instrument developers and (2) expert panel discussions, including both target group and content experts. Equivalent translated instruments help researchers from different cultures to find a common language and undertake comparative research. As acceptable psychometric properties are a prerequisite for that, we are currently carrying out a study with that focus.

Optimizing the mobility of residents with dementia: a pilot study promoting healthcare aide uptake of a simple mobility innovation in diverse nursing home settings

BACKGROUND

Almost 90 percent of nursing home residents have some type of mobility limitation. Many spend most of their waking hours lying in bed or sitting. Such inactivity can negatively affect residents' health and general well-being. This pilot study aimed to assess (1) the effect of the sit-to-stand activity on mobility outcomes of nursing home residents, (2) the effect of an audit-and-feedback intervention on uptake of the sit-to-stand activity by healthcare aides, and (3) the contextual factors influencing uptake of the sit-to-stand activity by healthcare aides.

METHODS

This quasi-experimental pilot study was conducted in two nursing homes in western Canada. Twenty-six residents with dementia completed the sit-to-stand activity with 56 healthcare aides during daily care; separately, 71 healthcare aides completed a research use and context survey. Preliminary mobility feedback was presented to healthcare aides in one site. Resident mobility was measured using the 30-second sit-to-stand test. Healthcare aide uptake of the activity was measured using documentation flowsheets and a survey-based measure. Context was measured using the Alberta Context Tool. Mobility and uptake outcomes were analyzed over time and by site with analysis of covariance. Spearman and Pearson correlations were used to correlate context data with research use.

RESULTS

Residents who more frequently completed the sit-to-stand activity were more likely to maintain or improve mobility compared with those who completed it less frequently (F=4.46; p=0.046, after adjustment for age). Uptake for one site was significantly different from the other (t-score=2.67; p=0.01, after adjustment for resident covariates). The audit-and-feedback intervention was associated with increased uptake of the activity from pre-intervention to post-intervention (t-score=-2.48; p=0.02). More context domains correlated significantly with aides' use of conceptual research and information sources in one site than the other.

CONCLUSIONS

The sit-to-stand activity is a promising means to maintain or improve transfer ability of nursing home residents with dementia. In the nursing home with initially weak uptake, strengthened uptake followed an audit-and-feedback intervention. Activity participation was higher in the site with stronger correlations between context and measured research use. Results are sufficiently promising to warrant proceeding with a full clinical trial.

The influence of organizational context on the use of research by nurses in Canadian pediatric hospitals

Background

Organizational context is recognized as an important influence on the successful implementation of research by healthcare professionals. However, there is relatively little empirical evidence to support this widely held view.

Methods

The objective of this study was to identify dimensions of organizational context and individual (nurse) characteristics that influence pediatric nurses’ self-reported use of research. Data on research use, individual, and contextual variables were collected from registered nurses (N = 735) working on 32 medical, surgical and critical care units in eight Canadian pediatric hospitals using an online survey. We used Generalized Estimating Equation modeling to account for the correlated structure of the data and to identify which contextual dimensions and individual characteristics predict two kinds of self-reported research use: instrumental (direct) and conceptual (indirect).

Results

Significant predictors of instrumental research use included: at the individual level; belief suspension-implement, research use in the past, and at the hospital unit (context) level; culture, and the proportion on nurses possessing a baccalaureate degree or higher. Significant predictors of conceptual research use included: at the individual nurse level; belief suspension-implement, problem solving ability, use of research in the past, and at the hospital unit (context) level; leadership, culture, evaluation, formal interactions, informal interactions, organizational slack-space, and unit specialty.

Conclusions

Hospitals, by focusing attention on modifiable elements of unit context may positively influence nurses’ reported use of research. This influence of context may extend to the adoption of best practices in general and other innovative or quality interventions.

Advancing the argument for validity of the Alberta Context Tool with healthcare aides in residential long-term care

BACKGROUND

Organizational context has the potential to influence the use of new knowledge. However, despite advances in understanding the theoretical base of organizational context, its measurement has not been adequately addressed, limiting our ability to quantify and assess context in healthcare settings and thus, advance development of contextual interventions to improve patient care. We developed the Alberta Context Tool (the ACT) to address this concern. It consists of 58 items representing 10 modifiable contextual concepts. We reported the initial validation of the ACT in 2009. This paper presents the second stage of the psychometric validation of the ACT.

METHODS

We used the Standards for Educational and Psychological Testing to frame our validity assessment. Data from 645 English speaking healthcare aides from 25 urban residential long-term care facilities (nursing homes) in the three Canadian Prairie Provinces were used for this stage of validation. In this stage we focused on: (1) advanced aspects of internal structure (e.g., confirmatory factor analysis) and (2) relations with other variables validity evidence. To assess reliability and validity of scores obtained using the ACT we conducted: Cronbach's alpha, confirmatory factor analysis, analysis of variance, and tests of association. We also assessed the performance of the ACT when individual responses were aggregated to the care unit level, because the instrument was developed to obtain unit-level scores of context.

RESULTS

Item-total correlations exceeded acceptable standards (> 0.3) for the majority of items (51 of 58). We ran three confirmatory factor models. Model 1 (all ACT items) displayed unacceptable fit overall and for five specific items (1 item on adequate space for resident care in the Organizational Slack-Space ACT concept and 4 items on use of electronic resources in the Structural and Electronic Resources ACT concept). This prompted specification of two additional models. Model 2 used the 7 scaled ACT concepts while Model 3 used the 3 count-based ACT concepts. Both models displayed substantially improved fit in comparison to Model 1. Cronbach's alpha for the 10 ACT concepts ranged from 0.37 to 0.92 with 2 concepts performing below the commonly accepted standard of 0.70. Bivariate associations between the ACT concepts and instrumental research utilization levels (which the ACT should predict) were statistically significant at the 5% level for 8 of the 10 ACT concepts. The majority (8/10) of the ACT concepts also showed a statistically significant trend of increasing mean scores when arrayed across the lowest to the highest levels of instrumental research use.

CONCLUSIONS

The validation process in this study demonstrated additional empirical support for construct validity of the ACT, when completed by healthcare aides in nursing homes. The overall pattern of the data was consistent with the structure hypothesized in the development of the ACT and supports the ACT as an appropriate measure for assessing organizational context in nursing homes. Caution should be applied in using the one space and four electronic resource items that displayed misfit in this study with healthcare aides until further assessments are made.