Development of the Patient Education Materials Assessment Tool (PEMAT): a new measure of understandability and actionability for print and audiovisual patient information

OBJECTIVE

To develop a reliable and valid instrument to assess the understandability and actionability of print and audiovisual materials.

METHODS

We compiled items from existing instruments/guides that the expert panel assessed for face/content validity. We completed four rounds of reliability testing, and produced evidence of construct validity with consumers and readability assessments.

RESULTS

The experts deemed the PEMAT items face/content valid. Four rounds of reliability testing and refinement were conducted using raters untrained on the PEMAT. Agreement improved across rounds. The final PEMAT showed moderate agreement per Kappa (Average K=0.57) and strong agreement per Gwet's AC1 (Average=0.74). Internal consistency was strong (α=0.71; Average Item-Total Correlation=0.62). For construct validation with consumers (n=47), we found significant differences between actionable and poorly-actionable materials in comprehension scores (76% vs. 63%, p<0.05) and ratings (8.9 vs. 7.7, p<0.05). For understandability, there was a significant difference for only one of two topics on consumer numeric scores. For actionability, there were significant positive correlations between PEMAT scores and consumer-testing results, but no relationship for understandability. There were, however, strong, negative correlations between grade-level and both consumer-testing results and PEMAT scores.

CONCLUSIONS

The PEMAT demonstrated strong internal consistency, reliability, and evidence of construct validity.

PRACTICE IMPLICATIONS

The PEMAT can help professionals judge the quality of materials (available at: http://www.ahrq.gov/pemat).

Evaluation of methods used for estimating content validity

The assessment of content validity is a critical and complex step in the development process of instruments which are frequently used to measure complex constructs in social and administrative pharmacy research. The aims of this study were to investigate the elements of content validity; to describe a practical approach for assessing content validity; and to discuss existing content validity indices. This is a narrative review of the assessment and quantification of content validity. It describes the key stages of conducting the content validation study and discusses the quantification and evaluation of the content validity estimates. Content validity provides evidence about the degree to which elements of an assessment instrument are relevant to and representative of the targeted construct for a particular assessment purpose. The assessment of content validity relies on using a panel of experts to evaluate instrument elements and rate them based on their relevance and representativeness to the content domain. It is a three-stage process that includes; the development stage, judgment and quantifying stage, and revising and reconstruction stage. To quantify the expert judgments, several indices have been discussed in this paper such as the content validity ratio (CVR), content validity index (CVI), modified-Kappa, and some agreement indices. A practical guide describes the process of content validity evaluation is provided. In summary, content validation processes and content validity indices are essential factors in the instrument development process, should be treated and reported as important as other types of construct validation. Determining item CVI and reporting an overall CVI are important components necessary to instruments especially when the instrument is used to measure health outcomes or to guide a clinical decision making. Content validity deserves a rigorous assessment process as the obtained information from this process are invaluable for the quality of the newly developed instrument.

Development and validation of a new tool to measure the facilitators, barriers and preferences to exercise in people with osteoporosis

Background

Despite the widely known benefits of exercise and physical activity, adherence rates to these activities are poor. Understanding exercise facilitators, barriers, and preferences may provide an opportunity to personalize exercise prescription and improve adherence. The purpose of this study was to develop the Personalized Exercise Questionnaire (PEQ) to identify these facilitators, barriers, and preferences to exercise in people with osteoporosis.

Methods

This study comprises two phases, instrument design and judgmental evidence. A panel of 42 experts was used to validate the instrument through quantitative (content validity) and qualitative (cognitive interviewing) methods. Content Validity Index (CVI) is the most commonly used method to calculate content validity quantitatively. There are two kinds of CVI: Item-CVI (I-CVI) and Scale-level CVI (S-CVI).

Results

Preliminary versions of this tool showed high content validity of individual items (I-CVI range: 0.50 to 1.00) and moderate to high overall content validity of the PEQ (S-CVI/UA = 0.63; S-CVI/Ave = 0.91). Through qualitative methods, items were improved until saturation was achieved. The tool consists of 6 domains and 38 questions. The 6 domains are: 1) support network; 2) access; 3) goals; 4) preferences; 5) feedback and tracking; and 6) barriers. There are 35 categorical questions and 3 open-ended items.

Conclusions

Using an iterative approach, the development and evaluation of the PEQ demonstrated high item-content validity for assessing the facilitators, barriers, and preferences to exercise in people with osteoporosis. Upon further validation it is expected that this measure might be used to develop more client-centered exercise programs, and potentially improve adherence.

Electronic supplementary material

The online version of this article (10.1186/s12891-017-1914-5) contains supplementary material, which is available to authorized users.

Improving the normalization of complex interventions: part 2 - validation of the NoMAD instrument for assessing implementation work based on normalization process theory (NPT)

INTRODUCTION

Successful implementation and embedding of new health care practices relies on co-ordinated, collective behaviour of individuals working within the constraints of health care settings. Normalization Process Theory (NPT) provides a theory of implementation that emphasises collective action in explaining, and shaping, the embedding of new practices. To extend the practical utility of NPT for improving implementation success, an instrument (NoMAD) was developed and validated.

METHODS

Descriptive analysis and psychometric testing of an instrument developed by the authors, through an iterative process that included item generation, consensus methods, item appraisal, and cognitive testing. A 46 item questionnaire was tested in 6 sites implementing health related interventions, using paper and online completion. Participants were staff directly involved in working with the interventions. Descriptive analysis and consensus methods were used to remove redundancy, reducing the final tool to 23 items. Data were subject to confirmatory factor analysis which sought to confirm the theoretical structure within the sample.

RESULTS

We obtained 831 completed questionnaires, an average response rate of 39% (range: 22-77%). Full completion of items was 50% (n = 413). The confirmatory factor analysis showed the model achieved acceptable fit (CFI = 0.95, TLI = 0.93, RMSEA = 0.08, SRMR = 0.03). Construct validity of the four theoretical constructs of NPT was supported, and internal consistency (Cronbach's alpha) were as follows: Coherence (4 items, α = 0.71); Collective Action (7 items, α = 0.78); Cognitive Participation (4 items, α = 0.81); Reflexive Monitoring (5 items, α = 0.65). The normalisation scale overall, was highly reliable (20 items, α = 0.89).

CONCLUSIONS

The NoMAD instrument has good face validity, construct validity and internal consistency, for assessing staff perceptions of factors relevant to embedding interventions that change their work practices. Uses in evaluating and guiding implementation are proposed.

Measuring patient satisfaction with health care treatment using the Short Assessment of Patient Satisfaction measure delivered superior and robust satisfaction estimates

OBJECTIVES

Reviews of patient satisfaction suggest seven dimensions, each of which should be assessed. This study reports development of a short generic patient satisfaction measure for use in routine clinical practice.

STUDY DESIGN AND SETTINGS

Participants were randomly recruited from two Australian incontinence clinics. Participants completed a follow-up questionnaire including patient satisfaction items. Iterative Mokken and Rasch analyses derived the Short Assessment of Patient Satisfaction (SAPS) scale from the item bank.

RESULTS

The SAPS psychometric properties illustrated the following features, namely its descriptive system covers all seven patient satisfaction dimensions, there were no misfitting items, and the scale exceeded the Loevinger H criteria for a strong unidimensional scale. The reliability of the SAPS was Cronbach α=0.86. When discriminatory function was examined, the SAPS scale was more sensitive than two other generic patient satisfaction instruments.

CONCLUSION

The SAPS scale is based on a firm theoretical model of patient satisfaction and its descriptive system covers the known dimensions contributing to patient satisfaction. Its internal psychometric properties exceeded standard psychometric standards, and it discriminated at least as well as other longer patient satisfaction measures. Although it needs further validation, the study results suggest that it may be useful for assessing patient satisfaction with health care.

Improving the normalization of complex interventions: part 1 - development of the NoMAD instrument for assessing implementation work based on normalization process theory (NPT)

BACKGROUND

Understanding and measuring implementation processes is a key challenge for implementation researchers. This study draws on Normalization Process Theory (NPT) to develop an instrument that can be applied to assess, monitor or measure factors likely to affect normalization from the perspective of implementation participants.

METHODS

An iterative process of instrument development was undertaken using the following methods: theoretical elaboration, item generation and item reduction (team workshops); item appraisal (QAS-99); cognitive testing with complex intervention teams; theory re-validation with NPT experts; and pilot testing of instrument.

RESULTS

We initially generated 112 potential questionnaire items; these were then reduced to 47 through team workshops and item appraisal. No concerns about item wording and construction were raised through the item appraisal process. We undertook three rounds of cognitive interviews with professionals (n = 30) involved in the development, evaluation, delivery or reception of complex interventions. We identified minor issues around wording of some items; universal issues around how to engage with people at different time points in an intervention; and conceptual issues around the types of people for whom the instrument should be designed. We managed these by adding extra items (n = 6) and including a new set of option responses: 'not relevant at this stage', 'not relevant to my role' and 'not relevant to this intervention' and decided to design an instrument explicitly for those people either delivering or receiving an intervention. This version of the instrument had 53 items. Twenty-three people with a good working knowledge of NPT reviewed the items for theoretical drift. Items that displayed a poor alignment with NPT sub-constructs were removed (n = 8) and others revised or combined (n = 6). The final instrument, with 43 items, was successfully piloted with five people, with a 100% completion rate of items.

CONCLUSION

The process of moving through cycles of theoretical translation, item generation, cognitive testing, and theoretical (re)validation was essential for maintaining a balance between the theoretical integrity of the NPT concepts and the ease with which intended respondents could answer the questions. The final instrument could be easily understood and completed, while retaining theoretical validity. NoMAD represents a measure that can be used to understand implementation participants' experiences. It is intended as a measure that can be used alongside instruments that measure other dimensions of implementation activity, such as implementation fidelity, adoption, and readiness.

Development and validation of an instrument to measure nurses' compassion competence

AIM

To develop and psychometrically validate the Compassion Competence Scale.

BACKGROUND

Compassion is a vital asset in the nursing profession; thus, it is necessary to develop a suitable instrument for assessment.

METHODS

The 49-item preliminary instrument was developed using concept analysis (hybrid model). Content and face validity were confirmed by 10 experts and 10 nurses, respectively, and the number of items was reduced to 18. The tool was tested on 660 nurses working at three tertiary hospitals in South Korea in 2013.

RESULTS

We selected 17 items for the final scale and extracted three factors (variance explained: 55.9%): communication, sensitivity, and insight. Correlations with the Emotional Competence Scale (.68), Compassionate Love Scale (.62), and Interpersonal Reactivity Index (.41) supported the scale's convergent validity. The reliability (Cronbach's α and test-retest reliability) was acceptable.

CONCLUSIONS

The Compassion Competence Scale is a simple, effective screening tool for identifying compassion competence among nurses.

Reliability and validity of Japanese versions of the UCLA loneliness scale version 3 for use among mothers with infants and toddlers: a cross-sectional study

Background

Mothers with infants and toddlers are a potential target population for the prevention or alleviation of feelings of loneliness. However, the theory and methods for measuring loneliness among mothers with infants and toddlers have yet to be standardized worldwide, including in Japan. Our goal was to evaluate the reliability and validity of the Japanese version of the UCLA Loneliness Scale Version 3 (UCLA-LS3-J), as well as two short-form versions—the 10-item UCLA-LS3 (SF-10) and the 3-item UCLA-LS3 (SF-3)—for the measurement of loneliness in mothers with infants and toddlers in Japan.

Methods

This cross-sectional study was conducted using a self-report questionnaire. The target population was 430 mothers with infants and toddlers who visited a community health center in Yokohama City in Japan. Questionnaire items encompassed the UCLA-LS3-J, as well as demographic data, the feeling for childrearing scale, and measures of social networks and subjective health status. The reliability and validity of the UCLA-LS3-J and its two short-form versions (SF-3 and SF-10) were determined via IBM SPSS Amos and SPSS Statistics 22.

Results

Questionnaires were returned by 248 mothers (valid response rate: 57.7%) aged 32.7 ± 4.6 (mean ± SD) years. The mean score on the UCLA-LS3-J was 38.4 ± 9.7 (range 20.0–73.0), with a normal distribution. When confirmatory factor analysis was carried out (for a single factor model), the goodness of fit of the model was almost identical to that of the original UCLA-LS3 version for the UCLA-LS3-J: (GFI = 0.882, AGFI = 0.840, CFI = 0.932, RMSEA = 0.066) and SF-10: (GFI = 0.942, AGFI = 0.900, CFI = 0.956, RMSEA = 0.081). The SF-3 model also showed an acceptable fit. The UCLA-LS3-J total score was significantly correlated with the total score on the SF-10 (r = 0.965) and SF-3 (r = 0.868). The Cronbach’s α coefficient of the UCLA-LS3-J was 0.926, while those of the SF-10 and SF-3 were 0.888 and 0.790, respectively. The score on the UCLA-LS3-J was positively correlated with childcare burden (r = .319, p < 0.001) and negatively correlated with social networks (r = −.438, p < 0.001).

Conclusions

This study indicated that the reliability and validity of the UCLA-LS3-J as well as its two short-form versions were adequate for assessing loneliness in mothers with infants and toddlers in Japan.

Electronic supplementary material

The online version of this article (10.1186/s12905-019-0792-4) contains supplementary material, which is available to authorized users.

Development and validation of the Body and Appearance Self-Conscious Emotions Scale (BASES)

The purpose of these studies was to develop a psychometrically sound measure of shame, guilt, authentic pride, and hubristic pride for use in body and appearance contexts. In Study 1, 41 potential items were developed and assessed for item quality and comprehension. In Study 2, a panel of experts (N=8; M=11, SD=6.5 years of experience) reviewed the scale and items for evidence of content validity. Participants in Study 3 (n=135 males, n=300 females) completed the BASES and various body image, personality, and emotion scales. A separate sample (n=155; 35.5% male) in Study 3 completed the BASES twice using a two-week time interval. The BASES subscale scores demonstrated evidence for internal consistency, item-total correlations, concurrent, convergent, incremental, and discriminant validity, and 2-week test-retest reliability. The 4-factor solution was a good fit in confirmatory factor analysis, reflecting body-related shame, guilt, authentic and hubristic pride subscales of the BASES. The development and validation of the BASES may help advance body image and self-conscious emotion research by providing a foundation to examine the unique antecedents and outcomes of these specific emotional experiences.

Development and validation of Nurses' Moral Courage Scale

BACKGROUND

Moral courage is required at all levels of nursing. However, there is a need for development of instruments to measure nurses' moral courage.

OBJECTIVES

The objective of this study is to develop a scale to measure nurses' self-assessed moral courage, to evaluate the scale's psychometric properties, and to briefly describe the current level of nurses' self-assessed moral courage and associated socio-demographic factors.

RESEARCH DESIGN

In this methodological study, non-experimental, cross-sectional exploratory design was applied. The data were collected using Nurses' Moral Courage Scale and analysed statistically.

PARTICIPANTS AND RESEARCH CONTEXT

The data were collected from a convenience sample of 482 nurses from four different clinical fields in a major university hospital in Finland for the final testing of the scale. The pilot comprised a convenience sample of 129 nurses.

ETHICAL CONSIDERATIONS

The study followed good scientific inquiry guidelines. Ethical approval was obtained from the university ethics committee and permission to conduct the study from the participating hospital.

FINDINGS

Psychometric evaluation showed that the 4-sub-scale, 21-item Nurses' Moral Courage Scale demonstrates good reliability and validity at its current state of development showing a good level of internal consistency for a new scale, the internal consistency values ranging from 0.73 to 0.82 for sub-scales and 0.93 for the total scale, thus well exceeding the recommended Cronbach's alpha value of >0.7. Principal component analysis and confirmatory factor analysis supported the theoretical construct of Nurses' Moral Courage Scale. Face validity and expert panel assessments markedly contributed to the relevance of items in establishing content validity.

DISCUSSION AND CONCLUSION

Nurses' Moral Courage Scale provides a new generic instrument intended for measuring nurses' self-assessed moral courage. Recognizing the importance of moral courage as a part of nurses' moral competence and its assessment offers possibilities to develop interventions and educational programs for enhancement of moral courage. Research should focus on further validation measures of Nurses' Moral Courage Scale in international contexts.

Development and evaluation of the content validity, practicability and feasibility of the Innovative dementia-oriented Assessment system for challenging behaviour in residents with dementia

Background

One of the most difficult issues for care staff is the manifestation of challenging behaviour among residents with dementia. The first step in managing this type of behaviour is analysing its triggers. A structured assessment instrument can facilitate this process and may improve carers’ management of the situation. This paper describes the development of an instrument designed for this purpose and an evaluation of its content validity and its feasibility and practicability in nursing homes.

Methods

The development process and evaluation of the content validity were based on Lynn’s methodology (1998). A literature review (steps 1 + 2) provided the theoretical framework for the instrument and for item formation. Ten experts (step 3) evaluated the first version of the instrument (the Innovative dementia-oriented Assessment (IdA®)) regarding its relevance, clarity, meaningfulness and completeness; content validity indices at the scale-level (S-CVI) and item-level (I-CVI) were calculated. Health care workers (step 4) evaluated the second version in a workshop. Finally, the instrument was introduced to 17 units in 11 nursing homes in a field study (step 5), and 60 care staff members assessed its practicability and feasibility.

Results

The IdA® used the need-driven dementia-compromised behaviour (NDB) model as a theoretical framework. The literature review and expert-based panel supported the content validity of the IdA®. At the item level, 77% of the ratings had a CVI greater than or equal to 0.78. The majority of the question-ratings (84%, n = 154) and answer-ratings (69%, n = 122) showed valid results, with none below 0.50. The health care workers confirmed the understandability, completeness and plausibility of the IdA®. Steps 3 and 4 led to further item clarification. The carers in the study considered the instrument helpful for reflecting challenging behaviour and beneficial for the care of residents with dementia. Negative ratings referred to the time required and the lack of effect on residents´ behaviour.

Conclusions

There was strong evidence supporting the content validity of the IdA®. Despite the substantial length and time requirement, the instrument was considered helpful for analysing challenging behaviour. Thus, further research on the psychometric qualities, implementation aspects and effectiveness of the IdA® in understanding challenging behaviour is needed.

Electronic supplementary material

The online version of this article (doi:10.1186/s12913-017-2469-8) contains supplementary material, which is available to authorized users.

Physical Literacy Knowledge Questionnaire: feasibility, validity, and reliability for Canadian children aged 8 to 12 years

BACKGROUND

Physical literacy is defined as the motivation, confidence, physical competence, and knowledge and understanding to engage in physical activity for life. Physical literacy knowledge and understanding encompasses movement (how to move), performance (evaluation of movement), and health and fitness (value of exercise, need for relaxation and sleep, etc.). This paper describes the development and evaluation of a standardized assessment of physical literacy knowledge and understanding for Canadian children in grades 4, 5, and 6.

METHODS

Proposed Physical Literacy Knowledge Questionnaire (PLKQ) content was identified through expert consultation and a review of provincial/territorial physical education curricula for grades 4 to 6. Open-ended questions verified language and generated response options. Feasibility was assessed via completion time and error frequency. Item validity assessed scores by age, gender, and teacher ratings of student knowledge. Test-retest reliability was assessed over short (2-day) and long (7-day) intervals.

RESULTS

Subsets of 678 children (54% girls, 10.1 ± 1.0 years of age) completed the feasibility and validity assessments. Response errors (missing or duplicate responses, etc.) were minimal (2% or less) except for one question (7% error) about the use of safety gear during physical activity. A Delphi process among experts in children's physical activity and fitness achieved consensus on the core content and supported an item analysis to finalize item selection. As expected, knowledge scores increased with age (partial eta² = 0.07) but were not related to gender (p = 0.63). Teacher ratings of children's knowledge of physical activity behaviour (r = 0.13, p = 0.01) and fitness (r = 0.12, p = 0.03), but not movement skill (r = 0.07, p = 0.19) were associated with PLKQ scores. Test-retest reliability for PLKQ score and individual questions was substantial to excellent for 71% of comparisons over a 2-day interval, but lower over a 7-day interval (53% substantial or excellent). Items with low reliability had high or low proportions of correct responses.

CONCLUSIONS

This study provides feasibility and validity evidence for the Physical Literacy Knowledge Questionnaire as an assessment of physical literacy knowledge for Canadian children in grades 4, 5, and 6. Completion rates were high and knowledge scores increased with age. Streamlining of the content in accordance with Delphi panel recommendations would further enhance feasibility, but would also focus the content on items with limited reliability. Future studies of alternative item wording and responses are recommended to enhance test-retest reliability.

Emerging good practices for Translatability Assessment (TA) of Patient-Reported Outcome (PRO) measures

This paper presents emerging Good Practices for Translatability Assessment (TA) of Patient-Reported Outcome (PRO) Measures. The ISOQOL Translation and Cultural Adaptation Special Interest Group (TCA-SIG) undertook the review of several TA approaches, with the collaboration of organizations who are involved in conducting TA, and members of the TCA-SIG. The effort led to agreement by the writing group on Good Practices for 1) the terminology to be used in referring to translatability process, 2) the best definition of TA, 3) the methodology that is recommended at each step of the process, 4) the persons involved in TA, 5) the timing of assessment, 6) the review criteria for TA, and 7) the recommendations to be made at the end of the TA process. With input from the TCA-SIG membership and in consultation with experts in the field, these emerging good practices can guide the future use of TA in the development of PROs.

Development and psychometric testing of a Clinical Reasoning Evaluation Simulation Tool (CREST) for assessing nursing students' abilities to recognize and respond to clinical deterioration

BACKGROUND

The development of clinical reasoning skills in recognising and responding to clinical deterioration is essential in pre-registration nursing education. Simulation has been increasingly used by educators to develop this skill.

OBJECTIVE

To develop and evaluate the psychometric properties of a Clinical Reasoning Evaluation Simulation Tool (CREST) for measuring clinical reasoning skills in recognising and responding to clinical deterioration in a simulated environment.

DESIGN

A scale development with psychometric testing and mixed methods study.

PARTICIPANTS/SETTINGS

Nursing students and academic staff were recruited at a university.

METHOD

A three-phase prospective study was conducted. Phase 1 involved the development and content validation of the CREST; Phase 2 included the psychometric testing of the tool with 15 second-year and 15 third-year nursing students who undertook the simulation-based assessment; Phase 3 involved the usability testing of the tool with nine academic staff through a survey questionnaire and focus group discussion.

RESULTS

A 10-item CREST was developed based on a model of clinical reasoning. A content validity of 0.93 was obtained from the validation of 15 international experts. The construct validity was supported as the third-year students demonstrated significantly higher (p<0.001) clinical reasoning scores than the second-year students. The concurrent validity was also supported with significant positive correlations between global rating scores and almost all subscale scores, and the total scores. The predictive validity was supported with an existing tool. The internal consistency was high with a Cronbach's alpha of 0.92. A high inter-rater reliability was demonstrated with an intraclass correlation coefficient of 0.88. The usability of the tool was rated positively by the nurse educators but the need to ease the scoring process was highlighted.

CONCLUSIONS

A valid and reliable tool was developed to measure the effectiveness of simulation in developing clinical reasoning skills for recognising and responding to clinical deterioration.

Initial development and psychometric testing of an instrument to measure the quality of children's end-of-life care

Background

The field of pediatric palliative care is hindered by the lack of a well-defined, reliable, and valid method for measuring the quality of end-of-life care.

Methods

The study purpose was to develop and test an instrument to measure mothers’ perspectives on the quality of care received before, at the time of, and following a child’s death. In Phase 1, key components of quality end-of-life care for children were synthesized through a comprehensive review of research literature. These key components were validated in Phase 2 and then extended through focus groups with bereaved parents. In Phase 3, items were developed to assess structures, processes, and outcomes of quality end-of-life care then tested for content and face validity with health professionals. Cognitive testing was conducted through interviews with bereaved parents. In Phase 4, bereaved mothers were recruited through 10 children’s hospitals/hospices in Canada to complete the instrument, and psychometric testing was conducted.

Results

Following review of 67 manuscripts and 3 focus groups with 10 parents, 141 items were initially developed. The overall content validity index for these items was 0.84 as rated by 7 health professionals. Based on feedback from health professionals and cognitive testing with 6 parents, a 144-item instrument was finalized for further testing. In Phase 4, 128 mothers completed the instrument, 31 of whom completed it twice. Test-retest reliability, internal consistency, and construct validity were demonstrated for six subscales: Connect With Families, Involve Parents, Share Information With Parents, Share Information Among Health Professionals, Support Parents, and Provide Care at Death. Additional items with content validity were grouped in four domains: Support the Child, Support Siblings, Provide Bereavement Follow-up, and Structures of Care. Forty-eight items were deleted through psychometric testing, leaving a 95-item instrument.

Conclusions

There is good initial evidence for the reliability and validity of this new quality of end-of-life care instrument as a mechanism for evaluative feedback to health professionals, health systems, and policy makers to improve children’s end-of-life care.

Electronic supplementary material

The online version of this article (doi:10.1186/1472-684X-14-1) contains supplementary material, which is available to authorized users.

Development of the Patient Scale of the Patient and Observer Scar Assessment Scale (POSAS) 3.0: a qualitative study

PURPOSE

The Patient and Observer Scar Assessment Scale (POSAS) is widely used for measurements of scar quality. This encompasses visual, tactile and sensory characteristics of the scar. The Patient Scale of previous POSAS versions was lacking input from patients. Therefore, the aim of this study was to develop the POSAS3.0, Patient Scale with involvement of adults patients with all scar types, complying with the highest clinimetric standards.

METHODS

From February 2018 to April 2019, a series of six focus group interviews were performed in the Netherlands and Australia to identify scar quality characteristics that adults with scars consider to be important. All focus groups were transcribed, anonymized and analysed using a thematic analysis. Relevant characteristics were formulated into items, resulting in a Dutch and English version of the Patient Scale. These drafts were pilot tested in Australia, the Netherlands and the United Kingdom, and refined accordingly.

RESULTS

A total of 21 relevant scar quality characteristics were identified during the focus groups. Two distinct versions of the POSAS3.0, Patient Scale were developed. The Generic version contains 16 items and can be used for all scar types, except linear scars. The Linear Scar version of the Patient Scale contains the same 16 items, with an extra item referring to the widening of scar margins. All included items are rated on a verbal rating scale with five response options.

CONCLUSION

Two versions of the POSAS3.0 Patient Scale were developed. Further field tests are being performed to establish the measurement properties and scoring algorithm of the scales.

Development, validation and psychometric analysis of the diabetic foot self-care questionnaire of the University of Malaga, Spain (DFSQ-UMA)

This paper assessed the reliability and construct validity of a tool to evaluate the foot self-care of diabetic patients. The education of diabetic patients about their foot care is a major issue to avoid complications like amputations and ulcers. Specific tools aimed to assess patient's knowledge in this area are needed. The study had two phases: in Phase 1, item-generation was carried out through a literature review, expert review by a Delphi technique and cognitive interviews with diabetic patients for testing readability and comprehension. In Phase 2, diabetic patients participated in a cross-sectional study for a psychometric evaluation (reliability and construct validity) was carried out on a sample of type I and II diabetic patients. The study was conducted at the University of Malaga (Spain), podiatric clinics and a Diabetic Foot Unit between October 2012 and March 2013. After psychometric-test analyses on a sample of 209 diabetic patients, the questionnaire resulted in 16 questions. Cronbach's alpha was 0.89 after removing 4 items because of their low reliability. Inter-item correlations gave a mean value of 0.34 (range: 0.06-0.74). The rotated solution showed a 3-factor structure (self-care, foot care, and footwear and socks) that jointly accounted for 60.88% of the variance observed. The correlation between the questionnaire scores and HbA1c was significant and inverse, (r = -0.15; p < 0.01). The findings show that the questionnaire is a valid and reliable tool for evaluating foot self-care behavior in diabetic patients.

Development and psychometric properties of the tool for assessing LGBTQI+ health training (TALHT) in pre-licensure nursing curricula

BACKGROUND

There is a lack of comprehensive lesbian, gay, bisexual, transgender, queer, and intersex (LGBTQI+) content in pre-licensure nursing curricula. LGBTQI+ people commonly experience mistreatment from nurses and healthcare providers due to a lack of knowledge or personal biases. To date, few instruments exist to guide LGBTQI+-specific curricular improvement.

OBJECTIVES/DESIGN/SETTING/PARTICIPANT/METHODS

Johns Hopkins School of Nursing LGBTQI+ Health Initiative and Emory University School of Nursing developed the Tool for Assessing LGBTQI+ Health Training (TALHT) to assist in evaluating gaps and redundancies in LGBTQI+ health content within pre-licensure nursing curricula. Face and content validity were evaluated among experts in LGBTQI+ health, curricular development, nursing education, and measure development (N = 22). The TALHT was modified based on expert feedback and a pilot with pre-licensure faculty (N = 13) to evaluate utility and acceptability of the refined measure.

RESULTS

Face and content validity evaluation of the 60-item iteration of the TALHT showed that the majority of items were consistently relevant among external expert reviewers. However, the clarity and readability of items were lacking. We triangulated the quantitative and qualitative data from experts to adapt and reduce the number of items in the TALHT to 20 items. Pilot testing of the revised 20-item measure found that utility and acceptability of the tool were rated highly among pre-licensure faculty.

CONCLUSION

The validation of the TALHT comes at a time when this type of instrument is clearly needed. The Future of Nursing 2020-2030 report calls for nursing education to incorporate competencies in caring for diverse populations to promote health equity. The Essentials: Core Competencies for Professional Nursing Education calls for a transition to operationalization of competency-based nursing education. The TALHT provides faculty and programs with a valid and reliable means to operationalize implementation of those competencies as they pertain to LGBTQI+ health.

Adaptation and testing of psychosocial assessment instruments for cross-cultural use: an example from the Thailand Burma border

Background

The purpose of this study was to develop valid and reliable instruments to assess priority psychosocial problems and functioning among adult survivors of systematic violence from Burma living in Thailand.

Methods

The process involved four steps: 1) instrument drafting and piloting; 2) reliability and validity testing; 3) instrument revision; and 4) retesting revised instrument.

Results

A total of N = 158 interviews were completed. Overall subscales showed good internal consistency (0.73-0.92) and satisfactory combined test-retest/inter rater reliability (0.63-0.84). Criterion validity, was not demonstrated for any scale. The alcohol and functioning scales underperformed and were revised (step 3) and retested (step 4). Upon retesting, the function scale showed good internal consistency reliability (0.91-0.92), and the alcohol scale showed acceptable internal consistency (0.79) and strong test-retest/inter-rater reliability (0.86-0.89).

Conclusions

This paper describes the importance and process of adaptation and testing, illustrated by the experiences and results for selected instruments in this population.

The Swedish version of the Normalization Process Theory Measure S-NoMAD: translation, adaptation, and pilot testing

BACKGROUND

The original British instrument the Normalization Process Theory Measure (NoMAD) is based on the four core constructs of the Normalization Process Theory: Coherence, Cognitive Participation, Collective Action, and Reflexive Monitoring. They represent ways of thinking about implementation and are focused on how interventions can become part of everyday practice.

AIM

To translate and adapt the original NoMAD into the Swedish version S-NoMAD and to evaluate its psychometric properties based on a pilot test in a health care context including in-hospital, primary, and community care contexts.

METHODS

A systematic approach with a four-step process was utilized, including forward and backward translation and expert reviews for the test and improvement of content validity of the S-NoMAD in different stages of development. The final S-NoMAD version was then used for process evaluation in a pilot study aimed at the implementation of a new working method for individualized care planning. The pilot was executed in two hospitals, four health care centres, and two municipalities in a region in northern Sweden. The S-NoMAD pilot results were analysed for validity using confirmatory factor analysis, i.e. a one-factor model fitted for each of the four constructs of the S-NoMAD. Cronbach's alpha was used to ascertain the internal consistency reliability.

RESULTS

In the pilot, S-NoMAD data were collected from 144 individuals who were different health care professionals or managers. The initial factor analysis model showed good fit for two of the constructs (Coherence and Cognitive Participation) and unsatisfactory fit for the remaining two (Collective Action and Reflexive Monitoring) based on three items. Deleting those items from the model yielded a good fit and good internal consistency (alphas between 0.78 and 0.83). However, the estimation of correlations between the factors showed that the factor Reflexive Monitoring was highly correlated (around 0.9) with the factors Coherence and Collective Action.

CONCLUSIONS

The results show initial satisfactory psychometric properties for the translation and first validation of the S-NoMAD. However, development of a highly valid and reliable instrument is an iterative process, requiring more extensive validation in various settings and populations. Thus, in order to establish the validity and reliability of the S-NoMAD, additional psychometric testing is needed.

Development of the Student Evidence-based Practice Questionnaire (S-EBPQ)

BACKGROUND

The assessment of learning and teaching of Evidence-based Practice (EBP) in nursing is an important issue, yet few tools have been developed specifically for use with student nurses. Therefore, the Evidence-based Practice Questionnaire (EBPQ), which has been successfully used to measure EBP in nurses and nurse educators, was revised to develop a Student version (S-EBPQ).

OBJECTIVE

The purpose of the study was to develop a student version of the Evidence-based Practice Questionnaire (EBPQ) and test its psychometric properties with a UK undergraduate student population.

DESIGN

Instrument development study.

PARTICIPANTS AND METHOD

Two hundred and forty-four undergraduate nursing students from an English University were recruited over a three year period to complete the EBPQ. This data was submitted to reliability analysis based on Item Response Theory and Exploratory Factor Analysis to explore construct validity.

RESULTS

Principal Component Analysis demonstrated evidence for the S-EBPQ's construct validity, and analyses comparing the subscale scores of students in their first and second years of studies identified evidence for the tool's convergent validity. Descriptive statistics, correlation coefficients and reliability estimates demonstrated evidence for the S-EBPQ's internal reliability, and item facility and discrimination.

CONCLUSION

The S-EBPQ appears to be a psychometrically robust measure of EBP use, attitudes, and knowledge and skills (regarding the retrieval and evaluation of evidence, and the application and sharing of EBP). It may therefore provide an effective means of evaluating learning of EBP with undergraduate nursing students.

Reliability and validity of the Pragmatics Observational Measure (POM): a new observational measure of pragmatic language for children

There is a need for a reliable and valid assessment of childhood pragmatic language skills during peer-peer interactions. This study aimed to evaluate the psychometric properties of a newly developed pragmatic assessment, the Pragmatic Observational Measure (POM). The psychometric properties of the POM were investigated from observational data of two studies - study 1 involved 342 children aged 5-11 years (108 children with ADHD; 108 typically developing playmates; 126 children in the control group), and study 2 involved 9 children with ADHD who attended a 7-week play-based intervention. The psychometric properties of the POM were determined based on the COnsensus-based Standards for the selection of health status Measurement INstruments (COSMIN) taxonomy of psychometric properties and definitions for health-related outcomes; the Pragmatic Protocol was used as the reference tool against which the POM was evaluated. The POM demonstrated sound psychometric properties in all the reliability, validity and interpretability criteria against which it was assessed. The findings showed that the POM is a reliable and valid measure of pragmatic language skills of children with ADHD between the age of 5 and 11 years and has clinical utility in identifying children with pragmatic language difficulty.

Development and psychometric testing of the nursing student mentors' competence instrument (MCI): A cross-sectional study

BACKGROUND

Mentors require competence at a diverse array of skills to mentor students during clinical practice. According to the latest evidence, competence at mentoring includes: knowledge, skills and attributes of individual students' learning objectives, core elements of nursing, learning processes, a reciprocal and trustful relationship, feedback, evaluation, cooperation with stakeholders, and the mentor's personal qualities.

OBJECTIVE

The purpose of the study was to test psychometric properties of a mentor's competence instrument developed to self-evaluate mentors' competence at mentoring nursing students in clinical practice.

DESIGN

A cross-sectional, descriptive, explorative study design was used.

SETTINGS

Data were collected from mentors at five university hospitals in Finland in 2016.

PARTICIPANTS

A total of 576 mentors participated in this study.

METHODS

The instrument was developed through systematic review, experts' evaluations, and pilot versions of the instrument tested in previous studies. The construct validity and reliability of the instrument were tested using exploratory factor analysis (EFA) with promax rotation and Cronbach's alpha.

RESULTS

A 10-factor model showed that the instrument has acceptable construct validity. Cronbach's alpha values for the subscales observed ranged from 0.76 to 0.90.

CONCLUSIONS

The instrument exhibited acceptable psychometric properties, thereby proving itself a valuable tool for evaluating mentors' competence at mentoring students. Further assessments of its reliability, validity and generality for measuring mentor's competence for mentoring students in different contexts and cultures are recommended.

Development and Evaluation of a Reverse-Entry Ion Source Orbitrap Mass Spectrometer

As a step towards development of a high-resolution ion mobility mass spectrometer using the orbitrap mass analyzer platform, we describe herein a novel reverse-entry ion source (REIS) coupled to the higher-energy C-trap dissociation (HCD) cell of an orbitrap mass spectrometer with extended mass range. Development of the REIS is a first step in the development of a drift tube ion mobility-orbitrap MS. The REIS approach retains the functionality of the commercial instrument ion source which permits the uninterrupted use of the instrument during development as well as performance comparisons between the two ion sources. Ubiquitin (8.5 kDa) and lipid binding to the ammonia transport channel (AmtB, 126 kDa) protein complex were used as model soluble and membrane proteins, respectively, to evaluate the performance of the REIS instrument. Mass resolution obtained with the REIS is comparable to that obtained using the commercial ion source. The charge state distributions for ubiquitin and AmtB obtained on the REIS are in agreement with previous studies which suggests that the REIS-orbitrap EMR retains native structure in the gas phase. Graphical Abstract ᅟ.

Developing a family-reported measure of experiences with home-based pediatric palliative and hospice care: a multi-method, multi-stakeholder approach

Background

Many children with serious illnesses are receiving palliative and end-of-life care from pediatric palliative and hospice care teams at home (PPHC@Home). Despite the growth in PPHC@Home, no standardized measures exist to evaluate whether PPHC@Home provided in the U.S. meets the needs and priorities of children and their families.

Methods

We developed and conducted a preliminary evaluation of a family-reported measure of PPHC@Home experiences using a multi-method, multi-stakeholder approach. Our instrument development process consisted of four phases. Item identification and development (Phase 1) involved a comprehensive literature search of existing instruments, guidelines, standards of practice, and PPHC@Home outcome studies, as well as guidance from a PPHC stakeholder panel. Phase 2 involved the initial item prioiritization and reduction using a discrete choice experiment (DCE) with PPHC professionals and parent advocates. Phase 3 involved a second DCE with bereaved parents and parents currently receiving care for their child to further prioritize and winnow the items to a set of the most highly-valued items. Finally, we conducted cognitive interviews with parents to provide information about the content validity and clarity of the newly-developed instrument (Phase 4).

Results

Items were compiled predominantly from three existing instruments. Phase 2 participants included 34 PPHC providers, researchers, and parent advocates; Phase 3 participants included 47 parents; and Phase 4 participants included 11 parents. At the completion of Phase 4, the Experiences of Palliative and Hospice Care for Children and Caregivers at Home (EXPERIENCE@Home) Measure contains 22 of the most highly-valued items for evaluating PPHC@Home. These items include “The care team treats my child’s physical symptoms so that my child has as good a quality of life as possible”, “I have regular access to on-call services from our care team”, and “The nurses have the knowledge, skills, and experience to support my child’s palliative or hospice care at home.”

Conclusions

The EXPERIENCE@Home Measure is the first known to specifically measure family-reported experiences with PPHC@Home in the U.S. Future work will include formal psychometric evaluation with a larger sample of parents, as well as evaluation of the clinical utility of the instrument with PPHC@Home teams.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12904-020-00703-0.