Construction of a web-based questionnaire for longitudinal investigation of work exposure, musculoskeletal pain and performance impairments in high-performance marine craft populations

OBJECTIVE

High-performance marine craft personnel (HPMCP) are regularly exposed to vibration and repeated shock (VRS) levels exceeding maximum limitations stated by international legislation. Whereas such exposure reportedly is detrimental to health and performance, the epidemiological data necessary to link these adverse effects causally to VRS are not available in the scientific literature, and no suitable tools for acquiring such data exist. This study therefore constructed a questionnaire for longitudinal investigations in HPMCP.

METHODS

A consensus panel defined content domains, identified relevant items and outlined a questionnaire. The relevance and simplicity of the questionnaire's content were then systematically assessed by expert raters in three consecutive stages, each followed by revisions. An item-level content validity index (I-CVI) was computed as the proportion of experts rating an item as relevant and simple, and a scale-level content validity index (S-CVI/Ave) as the average I-CVI across items. The thresholds for acceptable content validity were 0.78 and 0.90, respectively. Finally, a dynamic web version of the questionnaire was constructed and pilot tested over a 1-month period during a marine exercise in a study population sample of eight subjects, while accelerometers simultaneously quantified VRS exposure.

RESULTS

Content domains were defined as work exposure, musculoskeletal pain and human performance, and items were selected to reflect these constructs. Ratings from nine experts yielded S-CVI/Ave of 0.97 and 1.00 for relevance and simplicity, respectively, and the pilot test suggested that responses were sensitive to change in acceleration and that the questionnaire, following some adjustments, was feasible for its intended purpose.

CONCLUSIONS

A dynamic web-based questionnaire for longitudinal survey of key variables in HPMCP was constructed. Expert ratings supported that the questionnaire content is relevant, simple and sufficiently comprehensive, and the pilot test suggested that the questionnaire is feasible for longitudinal measurements in the study population.

Initial development of a patient-reported outcome measure of experience with cognitive impairment associated with schizophrenia

Cognitive impairment is a serious, often distressing aspect of schizophrenia that affects patients' day-to-day lives. Although several interview-based instruments exist to assess cognitive functioning, a reliable measure developed based on the experiences of patients facing cognitive difficulties is needed to complement the objective performance-based assessments. The present article describes the initial development of a patient-reported outcome (PRO) measure to assess the subjective experience of cognitive impairment among patients with schizophrenia, the Patient-Reported Experience of Cognitive Impairment in Schizophrenia (PRECIS). The phases of development included the construction of a conceptual model based on the existing knowledge and two sets of qualitative interviews with patients: 1) concept elicitation interviews to ensure face and content validity from the perspective of people with schizophrenia and 2) cognitive debriefing of the initial item pool. Input from experts was elicited throughout the process. The initial conceptual model included seven domains. The results from concept elicitation interviews (n=80) supported these domains but yielded substantive changes to concepts within domains and to terminology. Based on these results, an initial pool of 53 items was developed to reflect the most common descriptions and languages used by the study participants. Cognitive debriefing interviews (n=22) resulted in the removal of 18 items and modification of 22 other items. The remaining 35 items represented 23 concepts within six domains plus two items assessing bother. The draft PRO measure is currently undergoing psychometric testing as a precursor to broad-based clinical and research use.

An examination of concepts in vocational rehabilitation that could not be linked to the ICF based on an analysis of secondary data

BACKGROUND

In the last few years the International Classification of Functioning, Disability and Health (ICF) has become a widely known and useful reference classification in vocational rehabilitation. It would be equally important to know which aspects of work-related health information cannot be assigned to distinct ICF categories.

OBJECTIVE

The objective of this study is to examine the concepts derived from three studies conducted within the ICF Core Set for vocational rehabilitation project, which could not be linked to distinct ICF codes in order to complement the current understanding of functioning in vocational rehabilitation.

METHODS

Secondary data analysis of the concepts from the systematic literature review, expert survey and patient focus group study of the ICF Core Set for vocational rehabilitation project that were marked as nd = not definable, nc = not covered or pf = personal factor. Nd-concepts were assigned to the biopsychosocial model of the ICF; additional ICF categories were formulated where needed. Nc-concepts were grouped into common themes not covered by the ICF. Pf-categories were linked to a proposed personal factors classification.

RESULTS

1093 nd-concepts were matched to overarching terms in the ICF, and "other specified"-categories were detailed. 1924 pf-concepts were linked to 31 second level categories of a proposed personal factors classification. 441 nc-concepts were grouped into six themes including the concept of well-being and attributes related to processes and time.

CONCLUSIONS

With concepts that emerged from the secondary analysis of data gathered during the vocational rehabilitation ICF Core Set project, we have enriched the ICF model with constructs specific to vocational rehabilitation. However, additional research is needed to further explore personal factors specific to vocational rehabilitation. The influence of themes complementary to the ICF such as well-being and quality of life on return-to-work should be further investigated.

Visual stimuli in intervention approaches for pre-schoolers diagnosed with phonological delay

The aim of this study was to develop and content validate specific speech and language intervention picture cards: The Letter-Sound (L&S) cards. The present study was also focused on assessing the influence of these cards on letter-sound correspondences and speech sound production. An expert panel of six speech and language therapists analysed and discussed the L&S cards based on several criteria previously established. A Speech and Language Therapist carried out a 6-week therapeutic intervention with a group of seven Portuguese phonologically delayed pre-schoolers aged 5;3 to 6;5. The modified Bland-Altman method revealed good agreement among evaluators, that is the majority of the values was between the agreement limits. Additional outcome measures were collected before and after the therapeutic intervention process. Results indicate that the L&S cards facilitate the acquisition of letter-sound correspondences. Regarding speech sound production, some improvements were also observed at word level. The L&S cards are therefore likely to give phonetic cues, which are crucial for the correct production of therapeutic targets. These visual cues seemed to have helped children with phonological delay develop the above-mentioned skills.

Development of the Diary for Irritable Bowel Syndrome Symptoms to Assess Treatment Benefit in Clinical Trials: Foundational Qualitative Research

BACKGROUND

Irritable bowel syndrome (IBS) is a chronic gastrointestinal disorder characterized by abdominal pain and alterations in bowel habits. Three subtypes are defined on the basis of stool patterns: diarrhea-predominant IBS, constipation-predominant IBS, and alternating or mixed IBS.

OBJECTIVES

To develop patient-reported outcome measures for qualification by the Food and Drug Administration to support product approvals and labeling in IBS; the article focuses on the qualitative research that provided the foundation for the new measures.

METHODS

Forty-nine concept elicitation and 42 cognitive debriefing interviews were conducted with subjects meeting Rome III criteria; additional criteria were imposed to yield a sample representative of the target patient population.

RESULTS

Although incomplete bowel movements, abnormal stool frequency and consistency, and abdominal pain, discomfort, and bloating were reported most frequently across concept elicitation interviews, the relative importance of specific symptoms varied by subtype. Among their five symptoms most important to treat, diarrhea-predominant and alternating or mixed IBS subjects frequently identified urgency, loose/watery stools, abdominal pain, and cramping, whereas constipation-predominant IBS subjects commonly included infrequent and incomplete bowel movements, bloating, and abdominal pain. The cognitive debriefing interviews facilitated refinement of each item set, supported minor modifications following translatability assessment, and suggested improvements to the electronic interface. Furthermore, subjects reported that every item was relevant and no concepts of importance were missing.

CONCLUSIONS

Results support the content validity of the IBS patient-reported outcome measures. A pilot study was recently initiated to inform item reduction, develop scoring algorithms, and provide preliminary psychometric information. Comprehensive psychometric evaluation and responder definition development will follow.

What do Demand-Control and Effort-Reward work stress questionnaires really measure? A discriminant content validity study of relevance and representativeness of measures

OBJECTIVES

The Demand-Control (DC) and Effort-Reward Imbalance (ERI) models predict health in a work context. Self-report measures of the four key constructs (demand, control, effort, and reward) have been developed and it is important that these measures have good content validity uncontaminated by content from other constructs. We assessed relevance (whether items reflect the constructs) and representativeness (whether all aspects of the construct are assessed, and all items contribute to that assessment) across the instruments and items.

METHODS

Two studies examined fourteen demand/control items from the Job Content Questionnaire and seventeen effort/reward items from the Effort-Reward Imbalance measure using discriminant content validation and a third study developed new methods to assess instrument representativeness. Both methods use judges' ratings and construct definitions to get transparent quantitative estimates of construct validity. Study 1 used dictionary definitions while studies 2 and 3 used published phrases to define constructs.

RESULTS

Overall, 3/5 demand items, 4/9 control items, 1/6 effort items, and 7/11 reward items were uniquely classified to the appropriate theoretical construct and were therefore 'pure' items with discriminant content validity (DCV). All pure items measured a defining phrase. However, both the DC and ERI assessment instruments failed to assess all defining aspects.

CONCLUSIONS

Finding good discriminant content validity for demand and reward measures means these measures are usable and our quantitative results can guide item selection. By contrast, effort and control measures had limitations (in relevance and representativeness) presenting a challenge to the implementation of the theories. Statement of contribution What is already known on this subject? While the reliability and construct validity of Demand-Control and Effort-Reward-Imbalance (DC and ERI) work stress measures are routinely reported, there has not been adequate investigation of their content validity. This paper investigates their content validity in terms of both relevance and representativeness and provides a model for the investigation of content validity of measures in health psychology more generally. What does this study add? A new application of an existing method, discriminant content validity, and a new method of assessing instrument representativeness. 'Pure' DC and ERI items are identified, as are constructs that are not fully represented by their assessment instruments. The findings are important for studies attempting to distinguish between the main DC and ERI work stress constructs. The quantitative results can be used to guide item selection for future studies.

Content validity of symptom-based measures for diabetic, chemotherapy, and HIV peripheral neuropathy

INTRODUCTION

No treatments for axonal peripheral neuropathy are approved by the United States Food and Drug Administration (FDA). Although patient- and clinician-reported outcomes are central to evaluating neuropathy symptoms, they can be difficult to assess accurately. The inability to identify efficacious treatments for peripheral neuropathies could be due to invalid or inadequate outcome measures.

METHODS

This systematic review examined the content validity of symptom-based measures of diabetic peripheral neuropathy, HIV neuropathy, and chemotherapy-induced peripheral neuropathy.

RESULTS

Use of all FDA-recommended methods to establish content validity was only reported for 2 of 18 measures. Multiple sensory and motor symptoms were included in measures for all 3 conditions; these included numbness, tingling, pain, allodynia, difficulty walking, and cramping. Autonomic symptoms were less frequently included.

CONCLUSIONS

Given significant overlap in symptoms between neuropathy etiologies, a measure with content validity for multiple neuropathies with supplemental disease-specific modules could be of great value in the development of disease-modifying treatments for peripheral neuropathies. Muscle Nerve 55: 366-372, 2017.

What Do You Think You Are Measuring? A Mixed-Methods Procedure for Assessing the Content Validity of Test Items and Theory-Based Scaling

The valid measurement of latent constructs is crucial for psychological research. Here, we present a mixed-methods procedure for improving the precision of construct definitions, determining the content validity of items, evaluating the representativeness of items for the target construct, generating test items, and analyzing items on a theoretical basis. To illustrate the mixed-methods content-scaling-structure (CSS) procedure, we analyze the Adult Self-Transcendence Inventory, a self-report measure of wisdom (ASTI, Levenson et al., 2005). A content-validity analysis of the ASTI items was used as the basis of psychometric analyses using multidimensional item response models (N = 1215). We found that the new procedure produced important suggestions concerning five subdimensions of the ASTI that were not identifiable using exploratory methods. The study shows that the application of the suggested procedure leads to a deeper understanding of latent constructs. It also demonstrates the advantages of theory-based item analysis.

Factors Affecting Korean Neonatal Infant Pain Care: Translation and Validation of Three Instruments

This study aimed to translate three English-language instruments-Pain Knowledge and Use (PKU), Collaboration and Satisfaction About Care Decisions (CSACD), and Environmental Complexity Scale (ECS)-into Korean and evaluate the equivalence of the instrument versions. Three Korean translators and two senior reviewers translated the instruments' 56 items using a committee approach. Eight Korean experts evaluated the cultural relevance of the translated instruments using a content validity index (CVI), and 12 Korean neonatal nurses were interviewed to assess their understanding of items. In an expert panel review, the item-CVI was less than 0.78 for 14 items. Based on cognitive interviews, four items were found to be unsuitable in Korea. Based on expert panel review and cognitive interviews, unsuitable items were modified or deleted. In another expert panel review, the scale-CVI was 1.00 for the final instruments. The findings support the validity of the Korean-language PKU, CSACD, and ECS for research application.

Development and content validity of the CENA Program for Educational Training on the Neuropsychology of Learning, with an emphasis on executive functions and attention

INTRODUCTION

The importance of executive functions (EF) in childhood development, and their role as indicators of health, well-being, professional and academic success have been demonstrated by several studies in the literature. FE are cognitive processes that aim to control and manage behavior to achieve specific goal and included skills planning, inhibition, cognitive flexibility, (executive) attention and the central executive component of working memory (WM). In the context of education, the EF are crucial for continued learning and efficient academic performance due to their involvement in several components of the educational process.

OBJECTIVE

The aim of this article was to describe the development and content validity of the CENA Program for Educational Training on the Neuropsychology of Learning, with an emphasis on executive functions and attention.

METHODS

The study involved seven specialists (four responsible for evaluating the program, and three involved in brainstorming), and was carried out in three stages:Background research: neuropsychology and education;Program development - author brainstorming andEvaluation by expert judges The goals, language and methods.

RESULTS

CENA Program were considered adequate, attesting to its content validity as a school-based neuropsychological intervention.

CONCLUSION

Teacher training in school neuropsychology may be an important area for future investment and contribute to academic achievement and student development in the Brazilian education system.

Content Validity of the Credibility and Expectancy Questionnaire in a Pain Rehabilitation Setting

BACKGROUND

Content validity, the proper reflection of the concept to be measured, is yet unknown for the Credibility and Expectancy Questionnaire (CEQ). It is frequently used in pain rehabilitation because treatment expectancy is influential on the outcome.

OBJECTIVE

To test and improve the content validity of the CEQ in a sample of patients with chronic pain in different phases of their treatment.

METHODS

A qualitative observational study design using the Three-Step Test-Interview method was used. Therein, data collection, analyses, and adaptations occur iteratively.

RESULTS

Seventeen patients with chronic pain in different stages of treatment participated through convenience sampling from the mother sample of a randomized controlled trial. The main study parameter is content validity, which is defined as (1) interpretations and responses of the participants and (2) the identification of response problems operationalized, and resulting in changes in the CEQ. For patients waiting for treatment, the written instruction of the CEQ allowed different interpretations. After changing the instructions, the CEQ became an easy-to-understand and content-valid questionnaire. For patients who had already undergone treatment, changes regarding time frame and recall period were necessary to overcome interpretation and response problems to the CEQ.

DISCUSSION

After small changes, the CEQ appeared to be a content-valid measurement instrument for patients waiting for treatment. However, for patients who had already undergone treatment, the content validity of the CEQ was less, and considerable changes were necessary.

Theoretical to Tangible: Creating a Measure of Self-Advocacy for Female Cancer Survivors

BACKGROUND AND PURPOSE

Self-advocacy is well appreciated but poorly defined concept in oncology.

METHODS

The development of the Female Self-Advocacy in Cancer Survivorship (FSACS) Scale's theoretical underpinnings and item development led to evaluations of the measure's content validity and reliability.

RESULTS

The construct of self-advocacy contains 3 subdimensions with 57 Likert-type self-report items. Content validity results (S-CVI = 0.81 and S-CVI/UA = 0.83) indicated strong relevancy of items. Reliability results for each of the 3 subdimensions supported the consistency of the FSACS Scale scores, with strong internal consistency (Cronbach's alphas = .88, .81, and .90) and test-retest reliability (PPMC r = .85, .97, and .88).

CONCLUSIONS

The FSACS Scale captures the construct of self-advocacy, and its scores show strong reliability.

The Development of a Checklist to Enhance Methodological Quality in Intervention Programs

The methodological quality of primary studies is an important issue when performing meta-analyses or systematic reviews. Nevertheless, there are no clear criteria for how methodological quality should be analyzed. Controversies emerge when considering the various theoretical and empirical definitions, especially in relation to three interrelated problems: the lack of representativeness, utility, and feasibility. In this article, we (a) systematize and summarize the available literature about methodological quality in primary studies; (b) propose a specific, parsimonious, 12-items checklist to empirically define the methodological quality of primary studies based on a content validity study; and (c) present an inter-coder reliability study for the resulting 12-items. This paper provides a precise and rigorous description of the development of this checklist, highlighting the clearly specified criteria for the inclusion of items and a substantial inter-coder agreement in the different items. Rather than simply proposing another checklist, however, it then argues that the list constitutes an assessment tool with respect to the representativeness, utility, and feasibility of the most frequent methodological quality items in the literature, one that provides practitioners and researchers with clear criteria for choosing items that may be adequate to their needs. We propose individual methodological features as indicators of quality, arguing that these need to be taken into account when designing, implementing, or evaluating an intervention program. This enhances methodological quality of intervention programs and fosters the cumulative knowledge based on meta-analyses of these interventions. Future development of the checklist is discussed.

The Pareto Analysis for Establishing Content Criteria in Surgical Training

INTRODUCTION

Current surgical training is still highly dependent on expensive operating room (OR) experience. Although there have been many attempts to transfer more training to the skills laboratory, little research is focused on which technical behaviors can lead to the highest profit when they are trained outside the OR. The Pareto principle states that in any population that contributes to a common effect, a few account for the bulk of the effect. This principle has been widely used in business management to increase company profits. This study uses the Pareto principle for establishing content criteria for more efficient surgical training.

METHOD

A retrospective study was conducted to assess verbal guidance provided by 9 supervising surgeons to 12 trainees performing 64 laparoscopic cholecystectomies in the OR. The verbal corrections were documented, tallied, and clustered according to the aimed change in novice behavior. The corrections were rank ordered, and a cumulative distribution curve was used to calculate which corrections accounted for 80% of the total number of verbal corrections.

RESULTS

In total, 253 different verbal corrections were uttered 1587 times and were categorized into 40 different clusters of aimed changes in novice behaviors. The 35 highest-ranking verbal corrections (14%) and the 11 highest-ranking clusters (28%) accounted for 80% of the total number of given verbal corrections.

CONCLUSIONS

Following the Pareto principle, we were able to identify the aspects of trainee behavior that account for most corrections given by supervisors during a laparoscopic cholecystectomy on humans. This strategy can be used for the development of new training programs to prepare the trainee in advance for the challenges encountered in the clinical setting in an OR.

Assessing Asthma Symptoms in Adolescents and Adults: Qualitative Research Supporting Development of the Asthma Daily Symptom Diary

BACKGROUND

Despite the widespread availability of patient-reported asthma questionnaires, instruments developed in accordance with present regulatory expectations are lacking. To address this gap, the Patient-Reported Outcome (PRO) Consortium's Asthma Working Group has developed a patient-reported asthma daily symptom diary (ADSD) for use in clinical research to assess outcomes and support medical product labeling claims in adults and adolescents with asthma.

OBJECTIVES

To summarize the qualitative research conducted to inform the initial development of the ADSD and to provide evidence for content validity of the instrument in accordance with the Food and Drug Administration's PRO Guidance.

METHODS

Research informing the initial development and confirming the content validity of the ADSD is summarized. This comprised a review of published qualitative research, semi-structured concept elicitation interviews (n = 55), and cognitive interviews (n = 65) with a diverse and representative sample of adults and adolescents with a clinician-confirmed diagnosis of asthma in the United States to understand the asthma symptom experience and to assess the relevance and understanding of the newly developed ADSD.

RESULTS

From the qualitative literature review and concept elicitation interviews, eight core asthma symptoms emerged. These were broadly categorized as breathing symptoms (difficulty breathing, shortness of breath, and wheezing), chest symptoms (chest tightness, chest pain, and pressure/weight on chest), and cough symptoms (cough and the presence of mucus/phlegm). Conceptual saturation was achieved and differences in the experience of participants according to socio-demographic or clinical characteristics were not observed. Subsequent testing of the ADSD confirmed participant relevance and understanding.

CONCLUSIONS

The ADSD is a new patient-reported asthma symptom diary developed in accordance with the Food and Drug Administration's PRO Guidance. Evidence to date supports the content validity of the instrument. Item performance, reliability, and construct validity will be assessed in future quantitative research.

Measurement of activity limitations and participation restrictions: examination of ICF-linked content and scale properties of the FIM and PC-PART instruments

PURPOSE

To explore the operationalization of activity and participation-related measurement constructs through comparison of item phrasing, item response categories and scoring (scale properties) for two separate instruments targeting activities of daily living.

METHOD

Personal Care Participation Assessment and Resource Tool (PC-PART) item content was linked to ICF categories using established linking rules. Previously reported ICF-linked FIM content categories and ICF-linked PC-PART content categories were compared to identify common ICF categories between the instruments. Scale properties of both instruments were compared using a patient scenario to explore the instruments' separate measurement constructs.

RESULTS

The PC-PART and FIM shared 15 of the 53 level two ICF-linked categories identified across both instruments. Examination of the instruments' scale properties for items with overlapping ICF content, and exploration through a patient scenario, provided supportive evidence that the instruments measure different constructs.

CONCLUSIONS

While the PC-PART and FIM share common ICF-linked content, they measure separate constructs. Measurement construct was influenced by the instruments' scale properties. The FIM was observed to measure activity limitations and the PC-PART measured participation restrictions. Scrutiny of instruments' scale properties in addition to item content is critical in the operationalization of activity and participation-related measurement constructs. Implications for Rehabilitation When selecting outcome measures for use in rehabilitation it is necessary to examine both the content of the instruments' items and item phrasing, response categories and scoring, to clarify the construct being measured. Measurement of activity limitations as well as participation restrictions in activities of daily living required for community life provides a more comprehensive measurement of rehabilitation outcomes than measurement of either construct alone. To measure the effects of interventions used in rehabilitation, it is necessary to select measures with relevant content and scale properties that enable evaluation of change in the constructs that are expected to change, as a result of the rehabilitation intervention.

Content Validation of a Catalog of Exercises for Judo

This study aimed to assess the content validity of a catalog of 76 judo exercises. Two groups of raters comprising 16 judo experts evaluated the following content validity indicators: Clarity of Language, Practical Pertinence, Theoretical Relevance, and the Dimension of each exercise. The results confirmed the content validity of the judo training catalog with indicators showing scores greater than 0.80. These findings suggest that all 76 judo exercises are pertinent, representative of judo training and understandable for judo coaches. Thus, this catalog of judo exercises may help judo coaches in the selection and recording of exercises.

Qualitative Development and Content Validity of the Non-small Cell Lung Cancer Symptom Assessment Questionnaire (NSCLC-SAQ), A Patient-reported Outcome Instrument

PURPOSE

The purpose of this article was to describe the process and results of the preliminary qualitative development of a new symptoms-based patient-reported outcome (PRO) measure intended for assessing treatment benefit in clinical trials of advanced non-small cell lung cancer (NSCLC).

METHODS

Individual qualitative interviews were conducted in adults with NSCLC (Stages I-IV) in the United States. Experienced interviewers conducted concept-elicitation (CE) and cognitive interviews using semistructured interview guides. The CE interview guide was used for eliciting spontaneous reports of symptom experiences along with probing to further explore and confirm concepts. Interview transcripts were coded and analyzed by professional qualitative coders, and were summarized by like content using an iterative coding framework. Data from the CE interviews were considered alongside existing literature and clinical expert opinion during an item-generation process, leading to the development of a preliminary version of the NSCLC Symptom Assessment Questionnaire (SAQ). Three waves of cognitive interviews were conducted to evaluate concept relevance, item interpretability, and structure of the draft items and to facilitate further instrument refinement.

FINDINGS

Fifty-one subjects (mean [SD] age, 64.9 [11.2] years; 51.0% women) participated in the CE interviews. A total of 1897 expressions of NSCLC-related symptoms were identified and coded in interview transcripts, representing ~42 distinct symptom concepts. A 9-item initial-draft instrument was developed for testing in 3 waves of cognitive interviews with additional subjects with NSCLC (n = 20), during which both paper and electronic versions of the instrument were evaluated and refined. Participant responses and feedback during cognitive interviews led to the removal of 2 items and substantial modifications to others.

IMPLICATIONS

The NSCLC-SAQ is a 7-item PRO measure intended for use in advanced NSCLC clinical trials to support medical product labelling. The NSCLC-SAQ uses a 7-day recall period and verbal rating scales. It was developed in accordance with the US Food and Drug Administration's PRO Guidance and scientific best practices, and the resulting qualitative interview data provide evidence of content validity. The NSCLC-SAQ has been prepared in both paper and electronic administration formats and a tablet computer-based version is currently undergoing quantitative testing to confirm its measurement properties and support US Food and Drug Administration qualification.

The Validation of a Case-Based, Cumulative Assessment and Progressions Examination

OBJECTIVE

To assess content and criterion validity, as well as reliability of an internally developed, case-based, cumulative, high-stakes third-year Annual Student Assessment and Progression Examination (P3 ASAP Exam).

METHODS

Content validity was assessed through the writing-reviewing process. Criterion validity was assessed by comparing student scores on the P3 ASAP Exam with the nationally validated Pharmacy Curriculum Outcomes Assessment (PCOA). Reliability was assessed with psychometric analysis comparing student performance over four years.

RESULTS

The P3 ASAP Exam showed content validity through representation of didactic courses and professional outcomes. Similar scores on the P3 ASAP Exam and PCOA with Pearson correlation coefficient established criterion validity. Consistent student performance using Kuder-Richardson coefficient (KR-20) since 2012 reflected reliability of the examination.

CONCLUSION

Pharmacy schools can implement internally developed, high-stakes, cumulative progression examinations that are valid and reliable using a robust writing-reviewing process and psychometric analyses.

Content validity of global measures for at-work productivity in patients with rheumatic diseases: an international qualitative study

OBJECTIVES

To identify from a patient's perspective, difficulties and differences in the comprehension of five global presenteeism measures in patients with inflammatory arthritis and OA across seven countries.

METHODS

Seventy patients with a diagnosis of inflammatory arthritis or OA in paid employment were recruited from seven countries across Europe and Canada. Patients were randomly allocated to be cognitively debriefed on 3/5 global measures [Work Productivity Scale - Rheumatoid Arthritis, Work Productivity and Activity Impairment Questionnaire (WPAI), Work Ability Index, Quality and Quantity questionnaire, and WHO Health and Work Performance Questionnaire (HPQ)], with the WPAI debriefed in all patients as a standard measure of comparison between countries and patients. NVivo was used to code the data into four themes: construct and anchor, time recall, reference frame, and attribution.

RESULTS

Discrepancies were found in the interpretation of the word performance (HPQ) between countries, with Romania and Sweden relating performance to sports rather than work. Seventy percent of patients considered that a 7-day recall (WPAI) can accurately represent how their disease affects work productivity. The compared to normal reference (Quality and Quantity questionnaire) was reportedly too ambiguous, and the comparison with colleagues (HPQ), made many feel uncomfortable. Overall, 29% of patients said the WPAI was the most relevant to them, making it the most favoured measure.

CONCLUSION

Overall, patients across countries agree that the construct of work productivity in the last 7 days can accurately reflect the impact of disease while at work. Some current constructs to assess at-work productivity are not interchangeable between languages.

Development of the Flu-PRO: a patient-reported outcome (PRO) instrument to evaluate symptoms of influenza

BACKGROUND

To develop content validity of a comprehensive patient-reported outcome (PRO) measure following current best scientific methodology to standardize assessment of influenza (flu) symptoms in clinical research.

METHODS

Stage I (Concept Elicitation): 1:1 telephone interviews with influenza-positive adults (≥18 years) in the US and Mexico within 7 days of diagnosis. Participants described symptom type, character, severity, and duration. Content analysis identified themes and developed the draft Flu-PRO instrument. Stage II (Cognitive Interviewing): The Flu-PRO was administered to a unique set of influenza-positive adults within 14 days of diagnosis; telephone interviews addressed completeness, respondent interpretation of items and ease of use.

RESULTS

Samples: Stage I: N = 46 adults (16 US, 30 Mexico); mean (SD) age: 38 (19), 39 (14) years; % female: 56%, 73%; race: 69% White, 97% Mestizo. Stage II: N = 34 adults (12 US, 22 Mexico); age: 37 (14), 39 (11) years; % female: 50%, 50%; race: 58% White, 100% Mestizo.

SYMPTOMS

Symptoms identified by >50%: coughing, weak or tired, throat symptoms, congestion, headache, weakness, sweating, chills, general discomfort, runny nose, chest (trouble breathing), difficulty sleeping, and body aches or pains. No new content was uncovered during Stage II; participants easily understood the instrument.

CONCLUSIONS

Results show the 37-item Flu-PRO is a content valid measure of influenza symptoms in adults with a confirmed diagnosis of influenza. Research is underway to evaluate the suitability of the instrument for children and adolescents. This work can form the basis for future quantitative tests of reliability, validity, and responsiveness to evaluate the measurement properties of Flu-PRO for use in clinical trials and epidemiology studies.

Concept Elicitation Within Patient-Powered Research Networks: A Feasibility Study in Chronic Lymphocytic Leukemia

OBJECTIVES

To explore the feasibility of using social media-based patient networks to gather qualitative data on patient-reported outcome (PRO) concepts relevant to chronic lymphocytic leukemia (CLL).

METHODS

Between August and November 2013, US-residing members of the PatientsLikeMe online CLL patient community completed open-ended web-based surveys designed to elicit descriptions of CLL symptoms, impacts, and treatment-related perceptions. Qualitative telephone follow-up interviews were conducted with a subsample of respondents. Survey responses and interview transcripts were coded for qualitative analysis using Atlas.ti.

RESULTS

Fifty survey responses were included in the analyses. Participants were age 60.5 ± 6.9 years, 54% female, and 96% white. When surveyed, 20% were receiving current treatment, 16% were in remission, and 64% were treatment-naïve. Among respondents, 369 descriptions of CLL symptoms were coded. Fatigue-related symptoms were expressed most frequently, with 54% reporting "fatigue," "tiredness," or both in their responses. These concepts were followed by night sweats (38%), swollen lymph nodes (32%), and frequent infections (28%). Among impacts of CLL, worry and fear (66% of respondents), depressed feelings (52%), and work limitations (50%) were noted most frequently.

CONCLUSIONS

Survey results identified constitutional symptoms of CLL included in existing PRO instruments and the literature. Although the findings suggest that qualitative data obtained through social media applications can be potentially useful in supporting concept identification for newly developed PRO instruments, they also indicate that online approaches alone may not be sufficient to achieve efficient and exhaustive concept elicitation. Further research is needed to identify whether the results can support content validity in the same way as established qualitative research methods.

Exploring Fear of Childbirth in the United States Through a Qualitative Assessment of the Wijma Delivery Expectancy Questionnaire

OBJECTIVE

To explore women's experiences while completing the Wijma Delivery Expectancy Questionnaire (W-DEQ), an instrument used to measure fear of childbirth, and to analyze the readability and applicability of the instrument within a diverse population of women in the United States.

DESIGN

Qualitative descriptive study using focus groups with women who were pregnant or had given birth in the last 5 years.

SETTING

Urban health center in the Detroit metropolitan area.

PARTICIPANTS

Participants included 22 women who participated in three focus groups.

METHODS

Focus groups were used to collect data, which were analyzed using content analysis. The Fry Readability Graph, in computer form, was used to rate the readability of the instrument.

RESULTS

Women in the focus groups identified many themes that were consistent with previous research. However, the women indicated many new factors that contributed to their fear that were previously unidentified by the W-DEQ, including fear of abandonment by their clinicians and fear of how the structure of the maternity care system affects care during childbirth.

CONCLUSION

The findings from the focus groups challenge the utility and appropriateness of the W-DEQ for use as a screening tool to identify women who are pregnant and experiencing FOC in a U.S.

The German version of the Camberwell Assessment of Need for the Elderly (CANE): evaluation of content validity and adaptation to the German-speaking context

BACKGROUND

The Camberwell Assessment of Need for the Elderly (CANE) was developed for the assessment of physical-, psychological-, and environment-related needs in the elderly. The aim of this study was to revise and adapt the German version of the CANE with regard to the content validity of the instrument.

METHODS

Following a multistage approach, face-to-face interviews using the CANE, an expert survey and a multidisciplinary consensus conference were conducted in order to evaluate the frequency and relevance of met and unmet needs in the German elderly population, and to modify the content of the CANE for the German-speaking countries.

RESULTS

In Germany, unmet physical needs including physical health, medication, eyesight/hearing/communication, mobility/falls, self-care, and continence were found to have top priority closely followed by social needs (company, intimate relationships, daytime activities, information, and abuse/neglect). Psychological needs were the lowest ranked care category. Experts' proposals for the improvement of the German version of the CANE were collected. All findings were discussed and integrated in the multidisciplinary consensus conference with the result of a revised and adapted CANE that is applicable in the German-speaking context.

CONCLUSIONS

The provision of an adapted and improved German version of the CANE may substantially contribute to a comprehensive and valid assessment of needs in the elderly population. The results of this study represent an important basis for comprehensive needs assessment in the elderly in the theoretical and practical field of healthcare and health services research.