How to develop a condition-specific PROM

Developing new patient-reported outcome measures (PROMs) for application in clinical studies can be necessary if an adequate PROM does not exist. For adequate measurement, it is essential that the PROM has face validity (ie, is perceived to be relevant by clinicians and researchers) and has high content validity (ie, content relevance and content coverage for the targeted patient group). The steps needed to create PROMs that possess face and content validity for a specific condition are described in this paper. Face validity is achieved by item identification and generation through literature review. Content validity is confirmed through repetitive cognitive interviews of patients from the targeted patient group in order to generate a consensus-based pilot-version of the new PROM. This qualitative process ensures that items are appropriately worded, understandable, and minimizes doubts about how items should be answered. A practical example of this process is presented, which shows the development of the Knee Numeric-Entity Evaluation Score (KNEES-ACL), a condition-specific PROM for patients with deficiency of the anterior cruciate ligament (ACL).

Content Validation of an Instrument for the Assessment of School Teachers' Levels of Knowledge of Diabetes through Expert Judgment

The objective of this study was to describe the content validation, through expert judgment, of a questionnaire for determining the level of knowledge that school teachers have of diabetes in order to design relevant educational interventions to improve the health of school-aged children. This psychometric study involved 15 experts who assessed each of the items in the instrument. The results revealed that the strength of agreement shown by the questionnaire ranged from substantial to almost perfect in its four dimensions, with the characteristics of “sufficiency” and “relevance” having the highest scores (0.982 and 0.903, respectively) based on the judgments made by the participating experts. Regarding statistical significance, the characteristics “sufficiency”, with p < 0.001, and “relevance”, with p = 0.001, were particularly relevant. The overall degree of understandability for the new version of the instrument was high (91.54%). The psychometric results obtained from validation of the “grado de conocimientos sobre diabetes en docentes del ámbito escolar” (Spanish for “level of knowledge of diabetes in school teachers”)—GCDDaE questionnaire through expert judgment and the pre-test indicate that it is recommended for use as it is both relevant and quick and easy to administer.

Development and Validation of Adherence Score for Subcutaneous Biologic Disease-Modifying Antirheumatic Drugs

Background: The biologic disease-modifying antirheumatic drugs (bDMARDs) are currently incorporated as part of the pharmacotherapy management of inflammatory arthritis (IA). Adherence to bDMARDs is crucial to ensure treatment success in IA. However, most of the recent studies evaluated adherence level in patients using subcutaneous injections of bDMARDs utilized the indirect methods adapted from adherence assessment for oral medication. Aim: This study aimed to develop a questionnaire to assess adherence to the self-injectable subcutaneous bDMARDs. Methods: The development of the Subcutaneous bDMARDs Adherence Score (SCADS) involved evaluation of content validity. Literature reviews provide the basis for domain identification and item formation. Four experts evaluated the instrument by using a four-point ordinal scale with a rubric scoring on relevance, importance, and clarity of each item in measuring the overarching construct. The item-level content validity index (I-CVI) and the scale-level content validity index (S-CVI) were calculated. The factor structure and internal consistency reliability of SCADS were estimated using principal component analysis (PCA) and Cronbach's alpha, respectively. Results: Both S-CVI/UA (universal agreement) and the average item-level content validity index (S-CVI/Ave) (average) for the entire instrument showed excellent criteria with a value of >0.90. Cronbach's alpha coefficient value for SCADS was 0.707 indicating good internal consistency. All items showed corrected item-total correlation coefficients above 0.244. Questionnaire items with a factor loading of 0.30 or above were considered in the final factor solution. The factor analysis resulted in 3-factor solutions, which corresponded to 66.62% of the total variance. Conclusion: The SCADS is a consistent and reliable instrument for evaluating adherence among IA patients using the subcutaneous bDMARDs. It is simple to use, yet comprehensive but still requiring further clinical and international validation.

Transcultural adaptation, content validity and reliability of the instrument 'Picture My Participation' for children and youth with and without intellectual disabilities in mainland China

BACKGROUND

'Picture My Participation' (PMP) is a validated questionnaire for assessing participation in everyday activities by children with disabilities in low and middle income countries, but it is not yet available in simplified Chinese.

AIM

To describe the cross-cultural adaptation of the simplified Chinese version of 'Picture My Participation' (PMP-C; Simplified) and explore its validity and reliability.

METHODS

A cross-sectional study using convenience sampling was conducted using PMP-C (Simplified) with structured interviews supported by pictures for children and youth with and without intellectual disabilities (ID) in mainland China. The validity of the PMP-C (Simplified) was demonstrated by face validity and content validity while the reliability was evaluated for internal consistency and test-retest reliability.

RESULTS

Five items were slightly modified and eight pictures were revised to improve their fit with the culture of mainland China. All the items in PMP-C (Simplified) had excellent content validity, and face validity. The internal consistency, reliability coefficient and test-retest reliability of the subscale attendance for children and youth with and without ID were excellent.

CONCLUSION

Preliminary evidence of the content validity of PMP-C (Simplified) items and reliability of the subscale attendance for use with children and youth in mainland China has been gathered. However, psychometric properties in terms of construct validity for the whole instrument and reliability for the engagement subscale need further exploration.

On the Practical Consequences of Misfit in Mokken Scaling

Mokken scale analysis is a popular method to evaluate the psychometric quality of clinical and personality questionnaires and their individual items. Although many empirical papers report on the extent to which sets of items form Mokken scales, there is less attention for the effect of violations of commonly used rules of thumb. In this study, the authors investigated the practical consequences of retaining or removing items with psychometric properties that do not comply with these rules of thumb. Using simulated data, they concluded that items with low scalability had some influence on the reliability of test scores, person ordering and selection, and criterion-related validity estimates. Removing the misfitting items from the scale had, in general, a small effect on the outcomes. Although important outcome variables were fairly robust against scale violations in some conditions, authors conclude that researchers should not rely exclusively on algorithms allowing automatic selection of items. In particular, content validity must be taken into account to build sensible psychometric instruments.

Content Validity of Parental Stressor Scale: Pediatric Intensive Care Unit (PSS:PICU)

Around 10% to 42% of parents suffer from post-traumatic stress disorder following child's Pediatric Intensive Care Unit (PICU) admission. What is needed is an understanding of factors associated with parental stress at the beginning of the PICU admission to guide strategies to prevent the development of stress-related morbidity. Only one measure exists to assess sources of PICU-related parental stress, the Parental Stressor Scale: Pediatric Intensive Care Unit (PSS:PICU). However, this scale has not been modified in over 30 years. The purpose of this study is to examine the content validity and descriptive statistics of the PSS:PICU, propose initial refinements of the scale based on interviews and quantitative analyses, and identify threats to validity in this measure using a convergent parallel mixed methods design. Three validity threats were identified: construct underrepresentation, construct-irrelevant variance, and item redundancy. Suggested scale refinements were to delete 17, revise 7, retain 15, and add 16 items for future testing.

Assessing the Palliative and End-of-Life Care Education-Practice-Competence Triad in Intensive Care Units: Content Validity, Feasibility, and Reliability of a New Tool

BACKGROUND

To date, a comprehensive, psychometrically robust instrument to assess palliative and end-of-life (PEOL) care education, practice, and perceived competence among intensive care unit (ICU) nurses does not exist.

OBJECTIVE

To examine content validity and reliability of a proposed instrument to assess the PEOL care education-practice- competence triad among ICU nurses.

METHODS

An international modified e-Delphi and a cross-sectional pilot questionnaire survey. The Delphi involved 23 panelists from 11 countries. The pilot study involved 40 staff nurses and 3 nurse managers from 3 adult ICUs in a randomly selected hospital in Egypt. An instrument was developed and judged for content validity by international panelists, and then pretested in a pilot study, where data were collected at 2 time points using self-administered questionnaires, followed by cognitive interviews. Test-retest reliability was examined using intraclass correlation (ICC), standard error of measurement (SEM), and repeatability coefficient (RC).

RESULTS

The panelists confirmed content validity of the proposed instrument, and staff nurses confirmed its comprehensibility. At the level of the instrument's total scores, the lowest ICC was .9 (95% confidence interval: .8-.9); and the highest SEM and RC were 4.8 and 13.3, respectively.

CONCLUSIONS

The PEOL Care Index is a comprehensive, comprehensible, content valid, and reliable instrument to assess the PEOL care education-practice-competence triad among ICU nurses. Construct and criterion validities need to be confirmed in future studies. Applicability of the PEOL Care Index in different settings and cultures needs to be examined.

Preliminary test-retest reliability of the Wheelchair Satisfaction Questionnaire

PURPOSE

Economic realities in lower-and-middle-income countries (LMICs) present an increased need for outcome measures for wheelchair efficacy, as these measures enable optimized use of funds. As the provision level of wheelchairs is low in these areas, and many wheelchairs are inappropriate for their intended users, use of funds based on evidence is especially necessary. The Wheelchair Satisfaction Questionnaire (WSQ) was designed to be a snapshot of a wheelchair user's level of satisfaction with their wheelchair. The WSQ is comprised of 16 visual analogue scale questions. Each question includes an option for a qualitative explanatory comment. The current study examined initial test-retest reliability of the WSQ.

MATERIALS AND METHODS

The WSQ was administered twice to the same set of wheelchair users who were secondary students at a school for students with disabilities. A demonstration was given to the participants to explain how to mark the analogue scale. Participants were instructed to answer each item honestly and without peer input. A one-week time span separated test and retest. Scores for both sessions were entered into SPSS. An interclass coefficient of 0.70 or above indicates acceptable test-retest reliability.

RESULTS

The ages of participants ranged from 13 to 24, with a mean age of 17.86 years. Sixty-five participants completed the questionnaire fully in both test and retest and were included in this study. A high degree of reliability was found between scores from both tests. The interclass coefficient was r(63) = .863, p = .01, indicating statistically significant agreement between test and retest.

CONCLUSION

The results support the WSQ as a reliable measure, confirming the WSQ as a reliable tool for user feedback on wheelchair function. Because the WSQ is designed to provide user feedback with enough granularity to give data on particular aspects of wheelchair structure and function, data can facilitate repair and modifications to wheelchair parts. Studies using the WSQ to assess specific wheelchair types could indicate consistent patterns of user satisfaction and dissatisfaction, revealing relevant design issues. The WSQ is designed to give wheelchair users a voice that can empower user centred modification and design changes to facilitate improved health, opportunity, and social interaction.Implications for RehabilitationResults from the WSQ could enable manufacturers to utilize user feedback to improve the design of wheelchairs for use in LMIC's, and providers could be better-informed in the selection of wheelchair types for specific environments.The WSQ could provide immediate user feedback to inform wheelchair modification and selection to best serve a particular user.The WSQ could be used in clinical settings over time to collect longitudinal data from wheelchair users, which could identify the most commonly perceived reasons for user dissatisfaction in a particular clinical setting.

A Qualitative Study to Assess the Content Validity of the 24-Hour Migraine Quality of Life Questionnaire in Patients with Migraine

Objective

A concept elicitation, cognitive debriefing, and usability study was undertaken to: (1) explore migraine symptoms and day‐to‐day impacts; (2) determine the comprehensiveness and comprehensibility of the previously developed 24‐Hour Migraine Quality of Life Questionnaire electronic patient‐reported outcome (24‐Hr MQoLQ ePRO) items, and the appropriateness and understanding of the recall period, response options, and instructions; and (3) assess the usability on an electronic hand‐held device.

Methods

Eleven United States English‐speaking people with episodic migraine were recruited to participate in one‐on‐one interviews, which followed methods appropriate for concept elicitation, cognitive debriefing, and usability testing. Interviews were audio‐recorded, transcribed, and analyzed following the constant comparative method.

Results

Participants had a mean age of 42 years, and 8 were female. Through spontaneous mention or probing, all concepts of the 24‐Hr MQoLQ ePRO were endorsed by a majority of the participants. Cognitive interviewing confirmed the 24‐Hr MQoLQ ePRO instructions were clear, meaningful, and important to assess as symptoms and day‐to‐day impacts experienced as a result of migraine. Overall impressions of the ePRO device were overwhelmingly favorable, and the ePRO device was preferred to paper and pencil by all participants. Participant responses regarding the level of headache pain that would be acceptable in order to continue to go about daily activities ranged from 3 to 6, on a scale of 0 to 10, with 0 being “no headache” and 10 being “the worst headache.”

Conclusions

The 24‐Hr MQoLQ ePRO is content‐valid and appropriate for inclusion in future acute treatment for migraine studies designed to measure the symptoms and health‐related quality of life of migraine.

Validation of a Sensor-Fitted Simulator for Upper Airway Examination

OBJECTIVE

To validate a simulator for upper airway examination, fitted with sensors, for use as an academic tool for learning how to conduct examination of the upper airway and for evaluation of that learning.

STUDY DESIGN

Validation study.

SETTING

Undergraduate medical education.

SUBJECTS AND METHODS

A group of 18 fifth-year medical students and another of 6 otorhinolaryngology specialists conducted 6 examinations each with the simulator. To investigate concurrent validity, we calculated the correlation between damage scores provided by the simulator and damage assessment by a specialist. To evaluate construct validity, we compared both groups with regard to damage scores, technical procedure, and time spent. To examine content and face validity, we used questionnaires based on a 5-point Likert scale.

RESULTS

For concurrent validity, the correlation between the simulator's damage scores and the specialist's damage assessment was high: Spearman's ρ was 0.828 (P < .001). For construct validity, the group of students differed from the group of specialists in damage scores (P = .027) and in technical procedures (P < .001) but not in time spent. For content validity, all questionnaire statements were scored highly, and both groups had similar average scores. For face validity, the group of specialists considered the simulator to be realistic, and all statements on the questionnaire were rated with at least 4/5.

CONCLUSION

Concurrent, construct, content, and face validity have been demonstrated for a sensor-fitted simulator for upper airway examination, which is therefore accurate enough to be used as an academic tool for learning and evaluation of learning.

Validity and Reliability of the New Basic Functional Assessment Protocol (BFA)

The global evaluation of motion patterns can examine the synchrony of neuromuscular control, range of motion, strength, resistance, balance and coordination needed to complete the movement. Visual assessments are commonly used to detect risk factors. However, it is essential to define standardized field-based tests that can evaluate with accuracy. The aims of the study were to design a protocol to evaluate fundamental motor patterns (FMP), and to analyze the validity and reliability of an instrument created to provide information about the quality of movement in FMP. Five tasks were selected: Overhead Squat (OHS); Hurdle Step (HS); Forward Step Down (FSD); Shoulder Mobility (SM); Active Stretching Leg Raise (ASLR). A list of variables was created for the evaluation of each task. Ten qualified judges assessed the validity of the instrument, while six external observers performed inter-intra reliability. The results show that the instrument is valid according to the experts' opinion; however, the reliability shows values below those established. Thus, the instrument was considered unreliable, so it is recommended to repeat the reliability process by performing more training sessions for the external observers. The present study creates the basic functional assessment (BFA), a new protocol which comprises five tasks and an instrument to evaluate FMP.

How do physiotherapists understand and interpret the "Pain Attitudes and Beliefs Scale"? A cognitive interview study

Background: The Pain Attitudes and Beliefs Scale (PABS) for physiotherapists aims to differentiate between clinicians' biomedical and biopsychosocial treatment orientations regarding nonspecific low back pain (LBP). Objective: To study the content validity of the Norwegian PABS by following international guidelines: exploring its relevance, comprehensibility and comprehensiveness. Methods: Cognitive interviews were performed using the Three-Step Test Interview, consisting of think-aloud techniques, retrospective probing and in-depth interviews. Eleven Norwegian physiotherapists with a diversity of professional backgrounds participated. Results: The participants encountered little difficulty in completing the PABS. All items were deemed relevant and important but five items had ambiguous formulations which can easily be handled. The biomedical subscale appeared to be a comprehensive representation of biomedical treatment orientation. The biopsychosocial subscale was found to lack items concerning cognitive behavioral aspects of LBP management, such as patient education, therapeutic alliance, shared decision making and graded exposure. Conclusions: This study provides empirical evidence that the Norwegian version of the PABS-PT is relevant and comprehensible, provided some minor adjustments. The biopsychosocial subscale, however, lacks comprehensiveness, as it is not able to capture important aspects of contemporary biopsychosocial best practice care. Measurement of biopsychosocial treatment orientation may therefore be incomplete.

Content Validation through Expert Judgement of an Instrument on the Nutritional Knowledge, Beliefs, and Habits of Pregnant Women

The aim of this study was to conduct content validation through expert judgement of an instrument which explores the nutritional knowledge, beliefs, and habits during pregnancy. This is a psychometric study in which 14 experts participated in the evaluation of each of the questionnaire items, which were divided into two blocks according to the characteristics of sufficiency, clarity, coherence, and relevance. Fleiss’ κ statistic was used to measure strength of agreement. A pre-test with 102 participants was conducted to measure the degree of understandability of the instrument. The strength of agreement obtained for each of the dimensions was almost perfect. For each pair of experts, strength of agreement ranged between substantial and almost perfect. Sufficiency was the characteristic of the questionnaire that obtained the highest values in the two blocks, and was also the most statistically significant (p < 0.001). Coherence was the most statistically significant characteristic in the first block (p = 0.030). Clarity was the most statistically significant characteristic in the second block (p = 0.037). The wording of five of the twenty original items was corrected. The new version of the instrument attained a high degree of understandability. The results suggest that the instrument is valid and may therefore be applied.

Functioning in schizophrenia: a Delphi study covering the perspective of physiotherapists

PURPOSE

This qualitative study explores the barriers, personal characteristics/resources, and environmental factors that experienced physiotherapists identify as relevant in the assessment and treatment of persons living with schizophrenia, and whether the identified aspects are represented in the International Classification of Functioning, Disability and Health Core Sets for schizophrenia.

METHODS

A three-round Delphi study with physiotherapists was conducted between April and July 2018. In the first round, participants had to list all the aspects they considered to be relevant when assessing and/or treating individuals with schizophrenia, and they were asked six open-ended questions. Their responses were linked to categories. In the second and third rounds, physiotherapists had to judge whether each category/personal factor was relevant for describing functioning in schizophrenia.

RESULTS

Thirteen of 22 eligible physiotherapists from eight countries responded to the first round, and 10 completed all three rounds. Eighty-two (84.5%) of the 97 categories in the Comprehensive Core Set for schizophrenia and all 25 categories in the Brief Core Set were considered relevant. A total of five categories were additionally identified.

CONCLUSIONS

The barriers, personal characteristics/resources, and environmental factors from the physiotherapists' perspective have been identified. The results largely confirm the content validity of the Core Sets for schizophrenia.Implications for rehabilitationThis study shows which barriers, personal characteristics/resources, and environmental factors in persons with schizophrenia are relevant from physiotherapists' perspective and should be integrated in the rehabilitation process.The content validity of the Comprehensive and Brief Core Sets for schizophrenia is largely supported from the physiotherapists' perspective and therefore could be used in the assessment of functioning in persons with schizophrenia.The Comprehensive and Brief ICF Core Sets for schizophrenia could be used to plan and assess multidisciplinary rehabilitation interventions.

“DLQI Seems to be ‘Action’, and Skindex-29 Seems to be ‘Emotion’”: Qualitative Study of the Perceptions of Patients with Psoriasis or Eczema on Two Common Dermatology-specific Quality of Life Measures

Little is known about which quality of life measure best captures the lived experience of people with a chronic skin condition. The purpose of this study was to explore patients’ views on the Dermatology Life Quality Index (DLQI) and Skindex-29. Participants were adults (n = 28) with a diagnosis of eczema or psoriasis who completed the DLQI and the Skindex-29 before being interviewed about the content and format of these questionnaires. Interviews were analysed using content analysis. Participants were generally satisfied with length and layout of both questionnaires. However, the majority preferred the Skindex-29 for its ease of understanding, use of a longer recall period and incorporation of items on a variety of emotions. Participants reported both questionnaires failing to incorporate important aspects of their lives, for instance impact on professional relationships. Participants voiced limitations in both measures but overall felt Skindex-29 better captured their lived experience.

A Practical Guide for Item Generation in Measure Development: Insights From the Development of a Patient-Reported Experience Measure of Compassion

BACKGROUND AND PURPOSE

Although various measure development guidelines exist, practical guidance on how to systematically generate items is nascent. This article provides practical guidance on item generation in measure development and the use of a Table of Specifications (TOS) in this process.

METHODS

In addition to a review of the literature, the item generation process within an ongoing study to develop a valid and reliable patient-reported measure of compassion is provided.

RESULTS

Consensus on an initial pool of 109 items and their response scale was achieved with the aid of a TOS.

CONCLUSIONS

Dynamic, experiential, and relational care constructs such as compassion lie at the heart of nursing. Practical guidance on item generation is needed to allow nurses to identify, measure, and improve compassion in research and practice.

Development and psychometric properties of the public attitude towards vaccination scale - Health belief model

AIM

The aim of this study was to develop and psychometrically evaluate the Public Attitude Towards Vaccination Scale - Health Belief Model.

DESIGN

A methodological and prospective psychometric study.

METHOD

A three-phase construct was used to develop the Public Attitude Towards Vaccination Scale - Health Belief Model and to determine its psychometric properties: (1) creation of the item pool/conceptualization; (2) evaluation of the items; and (3) psychometric evaluation. This scale was tested using the construct validity (exploratory and confirmatory factor analyses) and the reliability analysis. A psychometric assessment of the scale was conducted with 586 individuals. Data were collected between January - April 2018.

RESULTS

Items of the scale were obtained by appraising the literature concerning vaccination and the other Health Belief Model scale and conducting interviews with mothers. The content validity ratio of this scale calculated according to experts' opinions ranged between 0.769 and 1.00. According to the exploratory factor analysis, there were five factors with an eigenvalue higher than 1 in the scale. These five factors accounted for 68.9% of the total variance. In confirmatory factor analysis, values of fit indices were excellent or acceptable. This scale had high internal consistency and test-retest reliability.

CONCLUSION

This study successfully developed the Public Attitude Towards Vaccination Scale - Health Belief Model. In addition to researchers, this scale can be used by nurses while providing counselling for people with vaccine hesitancy/refusal.

IMPACT

This measurement tool can be used to understand and address 'vaccine hesitancy' by researchers. The results of the research using this measurement tool will provide valuable information to policymakers for preventing vaccine hesitancy. The validity and reliability of this scale can easily be conducted in different languages.

Development of a Novel Hematological Malignancy Specific Patient-Reported Outcome Measure (HM-PRO): Content Validity

BACKGROUND

The quality of life of patients at all stages of hematological malignancy is greatly affected by the disease and its treatment. There is a wide range of health-related quality of life (HRQoL) issues important to these patients. Any new instrument developed to measure HRQoL of such patients should be content valid, i.e., the items should be comprehensively relevant to the patients and their health condition. The aim of the present study was to examine content validity of a hematological malignancy specific patient reported outcome measure (HM-PRO) developed for use in routine clinical practice.

METHODS

Following literature review and semi-structured interviews, the generated themes and sub-themes were discussed to develop the prototype version of the HM-PRO. A 4-step approach was used for content validation: initial testing and cognitive interviewing; item rating; content validity panel meeting; final field testing and cognitive interviewing. Additional questions related to patients' perception of recall period and preferred sentence structure (i.e., question or statement) of the items were also asked during cognitive interviews.

RESULTS

The content analysis of 129 transcribed semi-structured interviews resulted in the prototype version of the instrument consisting of 58 items grouped into two parts: Part A (impact/HRQoL - 34 items) and Part B (signs and symptoms - 24 items). The initial testing showed intra-class correlation coefficient (ICC) of >0.8 for both Part A and Part B. Item rating for language clarity, completeness, relevance, and response scale by experts and patients showed content validity index for scales average >0.8 for both Part A and Part B, except 0.64 for relevance for Part A by the patient panel. The final testing of the revised version of the instrument showed the Cronbach's alpha value of 0.91 for Part A and 0.76 for Part B, suggesting high internal consistency, and ICC of 0.91 for Part A and 0.76 for Part B. The recall period of "today" for Part-A and "last 3 days" for Part-B were the patients' preferred "recall period." Furthermore, the patients expressed preference to the HM-PRO items as statements.

CONCLUSION

The findings of this study confirm that the HM-PRO possesses a strong content validity, includes all the issues important to patients and is easy to read, understand and respond to spontaneously.

A novel patient-reported outcome for paediatric localized scleroderma: a qualitative assessment of content validity

BACKGROUND

According to current standards, no existing patient-reported outcome (PRO) measures have high-quality validity evidence for use with individuals diagnosed with paediatric localized scleroderma (LS). This severely hinders patient-centred LS-focused research, including much needed clinical trials.

OBJECTIVES

To develop a valid health-related quality of life measure for individuals with paediatric LS and to qualitatively evaluate its content validity using a patient-centred approach.

METHODS

Previously collected qualitative data from youth with LS and their caregivers was used to develop items. The resulting item set was administered in a clinical setting to participants aged 8-18 years old. Cognitive interviews were used to evaluate time to survey completion, readability/understanding of the items, appropriateness of the recall period and construct representation.

RESULTS

Seventeen children and adolescents with LS participated in the study. Interviews supported readability, understanding of the items and appropriateness of the recall period in individuals > 10 years old. Revisions were made to simplify the instructions and to be more inclusive of different subtypes of LS. Three items were added to improve content representation.

CONCLUSIONS

Content validity was supported by the patient-centred development process of the outcome measure and via direct feedback from individuals with LS and their families. Although an important first step, the resulting PRO, termed the Localized Scleroderma Quality of Life Instrument, should be further evaluated in a larger sample before being implemented. What's already known about this topic? No current health-related quality of life (HRQoL) measures have been created using direct input from children and adolescents with localized scleroderma (LS). When compared with qualitative reports of HRQoL impact in youth with all LS subtypes, no existing patient-reported outcome (PRO) measures have appropriate content validity for individuals with paediatric LS. What does this study add? This study proposes a novel LS-specific PRO and is the first qualitative assessment of content validity for any PRO measure in this population. Results from cognitive interviews with children and adolescents support the content validity of the newly developed item set and its ability to capture HRQoL impact in a clinical context. What are the clinical implications of this work? Incorporating a content-valid PRO of HRQoL impact into clinical practice would allow for the valid, ongoing capture of patient experience in LS. Although content validity is an important and necessary step in the process of evaluating validity, items within this novel measure will undergo additional psychometric evaluation before implementation in research and clinical settings.

Measuring shared decision making in oncology: Development and first testing of the iSHAREpatient and iSHAREphysician questionnaires

BACKGROUND

Existing measures to assess shared decision making (SDM) have often been developed based on an ill-defined underlying construct, and many assess physician behaviours only or focus on a single patient-physician encounter.

OBJECTIVE

To (a) develop a patient and a physician questionnaire to measure SDM in oncology and (b) determine their content validity and comprehensibility.

METHODS

A systematic review of SDM models and an oncology-specific SDM model informed the domains of the SDM construct. We formulated items for each SDM domain. Cancer patients and physicians rated content validity in an online questionnaire. We assumed a formative measurement model and performed online field-testing in cancer patients to inform further item reduction. We tested item comprehension in cognitive interviews with cancer patients and physicians.

RESULTS

We identified 17 domains and formulated 132 items. Twelve cancer patients rated content validity at item level, and 11 physicians rated content validity at domain level. We field-tested the items among 131 cancer patients and conducted cognitive interviews with eight patients and five physicians. These phases resulted in the 15-item iSHAREpatient and 15-item iSHAREphysician questionnaires, covering 13 domains.

CONCLUSIONS

We thoroughly developed the iSHARE questionnaires. They both assess patient and physician behaviours and cover the entire SDM process rather than a single consultation.

Identifying and Prioritizing Effective Factors on Outsourcing in Public Hospitals Using Fuzzy BWM

The outsourcing of health services has gained prominence over the past decades. Because numerous factors affect outsourcing in the field of health services, identifying and prioritizing these factors is specifically important. This study sought to identify and prioritize the factors affecting outsourcing, and to propose a model for the effective outsourcing of hospital services in Shiraz, Iran. The study drew on an exploratory mixed research method. In the first stage, all the criteria affecting the outsourcing of activities in hospitals were identified through the theoretical framework, a literature review, and interviews with hospital experts. Next, the criteria were finalized and prioritized using the fuzzy best-worst method (BWM). Following the literature review, 34 criteria for outsourcing were identified based on the studies explored and the interviews with the experts; the criteria were categorized into seven dimensions including "strategy", "management", "economy", "quality", "security and keeping patients' records", "service", and "agility." These dimensions formed the final outsourcing model of hospitals in Shiraz. Finally, the fuzzy BWM analysis revealed that "security and keeping patient's records" had the highest priority in outsourcing-related decision-making. The findings can help hospital managers make the right decision concerning the outsourcing of hospital services. The dimensions found in this research might also have been identified in other models, although this study was different in that it concentrated on the criteria in the specialized area of hospital management, while identifying the importance and weights of all the criteria involved.

Development and validation of a quality of relationship intervention for stroke survivor-family caregiver dyads

Background: The relationship between stroke survivors and family caregivers is critical for the well-being of both dyad members. Currently, there are few interventions targeted at dyads and focused on strengthening the relationship between survivors and family caregivers.Objectives: This study reports on the development of a customizable, strengths-based, relationship-focused intervention driven by the real-world experience and advice of stroke dyads. It also describes the "tips" that survivors and family caregivers offered for dealing with relationship challenges after stroke.Methods: Content of the intervention, including relationship tips, was derived from semi-structured interviews with N= 19 stroke dyads. A modified Delphi process with a national panel of 10 subject matter experts was used to evaluate and refine the content of the intervention and the associated screening tool.Results: Seventeen domains of relationship challenges and tips were identified. Consensus was reached among experts that the intervention content was relevant to the goal of helping survivors and family caregivers maintain a strong relationship after stroke; (2) clear from the perspective of stroke survivors and family caregivers who would be using it; (3) accurate with respect to the advice being offered, and; (4) useful for helping stroke survivors and family caregivers improve the quality of their relationship.Conclusions: This study extends the limited body of research about dyadic interventions after stroke. The next steps in this line of research include feasibility testing the intervention and evaluating its efficacy in a larger trial.

Development and Validation of an Evaluation Tool to Measure the Effectiveness of a Smoking Cessation Training among Healthcare Providers in Malaysia: The Providers' Smoking Cessation Training Evaluation (ProSCiTE)

Background: In line with Article 14 of the Framework Convention for Tobacco Control, we have witnessed vast developments in smoking cessation training for healthcare providers, offering help for smokers. However, there is no specific evaluation tool to monitor and evaluate the effectiveness of these programs for future enhancement and sustainability. Objective: To develop and validate a new tool for evaluating smoking cessation training programs for healthcare providers called the Providers' Smoking Cessation Training Evaluation (ProSCiTE). Methods: The 74-item ProSCiTE tool was developed based on a review of the literature and an expert panel review. The tool was validated in a sample of 403 healthcare providers using a cross-sectional study design from July to December 2016. Content validity was assessed by the Scale-Content Validity Index (S-CVI). The construct validity of the ProSCiTE was analyzed using exploratory factor analysis (EFA) to confirm psychometric properties. Internal consistency reliability was determined using Cronbach's alpha. Results: The content validity showed that the S-CVI ranged from 0.82 to 1.00 for consistency, representativeness, relevancy, and the clarity of each construct, resulting in 67 items for the questionnaire. The construct validity of the ProSCiTE (based on eigenvalues and factor loadings to confirm the four-factor structure (attitude, self-efficacy, behavior, and barriers) with 54.74% total variance) was acceptable (Kaiser-Mayer-Olkin = 0.923; Bartlett's test of sphericity was significant, p < 0.001). The internal consistency reliability of the four-factor structure was very good, with Cronbach's alpha values at 0.89, 0.94, 0.95, and 0.90, respectively. Conclusions: This study showed that 67 items of the ProSCiTE demonstrated good content and construct validity, as well as a high internal consistency reliability for the measurement of knowledge, attitudes, self-efficacy, behavior, and barriers to smoking cessation interventions among healthcare providers. Therefore, the ProSCiTE is a valid and reliable research tool with which to evaluate the effectiveness of smoking cessation training programs.

Development and content validation of the Telenursing Interaction and Satisfaction Questionnaire (TISQ)

Background

Caller satisfaction with telephone advice nursing (TAN) is generally high, and the interaction is essential. However, a valid questionnaire exploring caller satisfaction in TAN with focus on perceived interaction is lacking.

Objective

To develop and assess content validity and test‐retest reliability of a theoretically anchored questionnaire, the Telenursing Interaction and Satisfaction Questionnaire (TISQ), that explores caller satisfaction in TAN by focusing on perceived interaction between the caller and the telenurse.

Methods

The study was performed in three stages. First, variables relevant for patient satisfaction in health care were identified through a literature search. Variables were then structured according to the Interaction Model of Client Health Behavior (IMCHB), which provided theoretical guidance. Items relevant for a TAN context were developed through consensus discussions. Then, evaluation and refinement were performed through cognitive interviews with callers and expert ratings of the Content Validity Index (CVI). Finally, test‐retest reliability of items was evaluated in a sample of 109 individuals using intraclass correlation coefficients (ICC).

Results

The TISQ consists of 60 items. Twenty items cover perceived interaction in terms of health information, affective support, decisional control and professional/technical competence. Five items cover satisfaction with interaction and five items overall satisfaction. Remaining items reflect singularity of the caller and descriptive items of the call. The TISQ was found to exhibit good content validity, and test‐retest reliability was moderate to good (ICC = 0.39‐0.84).

Conclusions

The items in the TISQ form a comprehensive and theoretically anchored questionnaire with satisfactory content validity and test‐retest reliability.

Development of a 'universal-reporter' outcome measure (UROM) for patient and healthcare professional completion: a mixed methods study demonstrating a novel concept for optimal questionnaire design

OBJECTIVES

To describe the novel concept of, and methods for developing, a 'universal-reporter' outcome measure (UROM); a single questionnaire for completion by patients and/or healthcare professionals (HCPs) when views on the same subject are required.

DESIGN

A mixed methods study with three phases-phase I: identification of relevant content domains from existing clinical tools, patient questionnaires and in-depth interviews with multistakeholders; phase II: item development using a novel approach that considered plain language in conjunction with medical terminology; and phase III: pretesting with multistakeholders using cognitive interviews.

SETTING

A case study in surgical wound assessment undertaken in two UK hospital trusts and one university setting.

PARTICIPANTS

Patients who had recently undergone general abdominal surgery and healthcare professionals involved in post-surgical wound care.

RESULTS

Phase I: In the example case study, 19 relevant content domains were identified from two clinical tools, two patient questionnaires and 19 multistakeholder interviews (nine patients, 10 HCPs). Phase II: Domains were operationalised into items and subitems (secondary components to collect further information, if relevant). The version after pretesting had 16 items, five of which included further subitems. Plain language in conjunction with medical terminology was applicable in nine (27%) items/subitems. Phase III: Pretesting with 28 patients and 14 HCPs found that the UROM was acceptable to both respondent groups. An unanticipated secondary finding of the study was that the combined use of plain language and medical terminology during questionnaire development may be a useful, novel technique for evaluating item interpretation and thereby identifying items with inadequate content validity.

CONCLUSION

UROMs are a novel approach to outcome assessment that are acceptable to both patients and HCPs. Combining plain language and medical terminology during item development is a recommended technique to improve accuracy of item interpretation and content validity during questionnaire design. More work is needed to further validate this novel approach and explore the application of UROMs to other settings.