The use of concept mapping in measurement development and evaluation: Application and future directions

Development of a quality of life measure for left ventricular assist device recipients using a mixed methods approach

BACKGROUND

Left ventricular assist device (LVAD) recipients report symptom improvement but find adjusting to life with the LVAD challenging. These challenges are unique, and existing patient-reported outcome measures (PROMs) do not reflect their experiences. This study aimed to develop a culturally relevant quality of life PROM for use with LVAD recipients in future research, design evolutions and clinical practice.

METHODS

A three-stage mixed-methods approach was used to develop a PROM: stage 1 included group concept mapping (GCM); stage 2 semi-structured qualitative interviews were conducted with 11 LVAD recipients and 10 clinicians, and a questionnaire was developed using a conceptual framework; and stage 3 used exploratory psychometric analysis of the PROM data using Rasch measurement theory. This paper presents stages 2 and 3.

RESULTS

The conceptual framework consisted of four key concepts, including general health, life with the LVAD, equipment and clothing and emotional impact. Statements from interviews and GCM were used to create items for the LVAD quality of life (LVAD-QoL). Cognitive interviews tested face validity and participant comprehension. Forty-nine participants were recruited from three UK transplant centres. PROM data were collected and analysed using Rasch analysis. Four items displayed misfit; dependency between item sets was the biggest issue (57/485 pairwise differences). After restructuring and dealing with item misfit, the LVAD-QoL conformed to the Rasch model, supporting the psychometric properties and quality of the LVAD-QoL.

CONCLUSIONS

Using a mixed-methods approach ensured the development of a robust and psychometrically sound tool for research, design evolution and clinical practice with LVAD recipients.

Empowerment Among Adolescent Girls in Nepal: A Concept Mapping Exploratory Study

BACKGROUND

The concept of empowerment is challenging to operationalize and measure; it is multidimensional, the outcomes are not always directly observable, and meanings of empowerment are highly contextual and socially and culturally situated. This study aimed to explore perspectives of empowerment among adolescent girls in Nepal to identify statements for inclusion in a context-specific empowerment measure.

METHODS

We used a participatory and mixed method research method called concept mapping in 3 districts in Nepal. Three sequential concept mapping sessions were used to solicit, organize, and process how participants responded to the prompt: "The life of an adolescent girl improves when she has/can…" The Concept Systems Global software was used to manage and analyze the concept mapping brainstorming, sorting, and rating data using established tools, such as multidimensional scaling and cluster analysis.

RESULTS

Concept mapping was conducted with 113 participants, including national experts, program staff, adolescents, and their mothers. They identified 105 items that fall into a 4-cluster solution: education and knowledge, decision-making, supports and skills, and physical infrastructure. Rating data uncovered there was some overlap between the top 10 most important items between the national-level experts and other stakeholder groups; however, several components associated with empowerment differed by stakeholder group in terms of importance.

CONCLUSION

This research represents a critical step in exploring definitions of empowerment among adolescent girls in the Nepal context and with expert input. These results led to the development of a contextually specific definition of empowerment. Researchers and practitioners interested in developing context-specific understandings of complex topics that incorporate community voices and perspectives could use a similar concept mapping approach in other countries to explore various topics with diverse populations.

Group concept mapping for health professions education scholarship

While explicit conceptual models help to inform research, they are left out of much of the health professions education (HPE) literature. One reason may be the limited understanding about how to develop conceptual models with intention and rigor. Group concept mapping (GCM) is a mixed methods conceptualization approach that has been used to develop frameworks for planning and evaluation, but GCM has not been common in HPE. The purpose of this article is to describe GCM in order to make it more accessible for HPE scholars. We recount the origins and evolution of GCM and summarize its core features: GCM can combine multiple stakeholder perspectives in a systematic and inclusive manner to generate explicit conceptual models. Based on the literature and prior experience using GCM, we detail seven steps in GCM: (1) brainstorming ideas to a specific "focus prompt," (2) preparing ideas by removing duplicates and editing for consistency, (3) sorting ideas according to conceptual similarity, (4) generating the point map through quantitative analysis, (5) interpreting cluster map options, (6) summarizing the final concept map, and (7) reporting and using the map. We provide illustrative examples from HPE studies and compare GCM to other conceptualization methods. GCM has great potential to add to the myriad of methodologies open to HPE researchers. Its alignment with principles of diversity and inclusivity, as well as the need to be systematic in applying theoretical and conceptual frameworks to practice, make it a method well suited for the complexities of contemporary HPE scholarship.

The PELI-D II Study: Development and Preliminary Validation of the Preferences for Leisure Activities Inventory (P-LAI) for Adult Day Services

OBJECTIVES

The current study aimed to develop and preliminarily validate an initial version of an instrument to assess the leisure activity preferences of people receiving adult day services (ADS).

METHODS

Based on previously conducted concept mapping steps, we identified 12 clusters of preferences for leisure activities. We adopted the structure of the Preferences for Everyday Living Inventory and phrased our cluster labels as questions to develop a first draft of the Preferences for Leisure Activities Inventory (P-LAI). We conducted cognitive interviews (n = 8) to revise, preliminarily validate, and preliminarily finalize the P-LAI.

RESULTS

The draft of the P-LAI included 25 questions. Based on the results of the cognitive interviews, the number of questions was reduced to 21, three questions were rephrased, and evidence to support the preliminary validity of the P-LAI based on the response processes was provided.

CONCLUSIONS

The P-LAI is the first instrument to assess preferences for leisure activities in the ADS environment.

CLINICAL IMPLICATIONS

The preliminary results of the P-LAI allow ADS providers and healthcare professionals to assess the preferences for leisure activities of people who receive ADS in a structured way. Based on these results, preference-based services can be designed and planned, and the person-centered philosophy of care can be further operationalized in the ADS environment.

Barriers and facilitators for oral health screening among tobacco users: a mixed-methods study

OBJECTIVES

Tobacco consumption adversely affects general and oral health and is considered one of the significant public health burdens globally. The present study aims to assess the barriers and facilitators for attending oral and dental health screening among tobacco users who seek cessation advice.

METHODOLOGY

The present mixed-methods study used group concept mapping (GCM) to identify the facilitators/barriers to attending oral health screening among young adults attending face-to-face and virtual Tobacco Cessation Clinic at King Saud University (Riyadh, Saudi Arabia) between September 2022 and April 2023. Study investigators included healthcare social workers, dental interns, and oral and maxillofacial medicinists. Information about demographics, general health, oral/dental health and tobacco use were collected using self-completed questionnaires. The barriers and facilitators were assessed following GCM by brainstorming, sorting, rating, and interpretation activities. Descriptive, multidimensional scaling and hierarchical cluster analysis were used to describe the study participants and produce concept maps of the generated statements.

RESULTS

The study included 148 participants who generated 67 statements summarised into 28 statements as facilitators or barriers. Based on a 5-point importance scale, the participants indicated the importance of facilitators under health-related cluster [e.g. when I feel pain] as the highest, followed by personal [e.g. to maintain my mouth hygiene], social [e.g. the quality of treatment] and financial clusters [e.g. the reasonable cost]. Concerning barriers, financial factors [e.g. high cost] acted as the highest-rated barrier, followed by personal [e.g. lack of dental appointments] and health-related [e.g. worry that dental problems will worsen]. The social factors were the least considerable barrier [e.g. lack of time]. Clustering these facilitators/barriers on the concept map indicated their conceptual similarity by an average stress value of 0.23.

CONCLUSION

Pain was the most important facilitator to attending oral health screening by young adults seeking tobacco cessation advice. Notable barriers included the high cost of dental treatment and the lack of scheduled appointments. Thus, oral health care providers need to consider scheduling periodic and timely dental check-ups to prevent and reduce the burden of tobacco-associated and pain-causing oral diseases.

'They treat us like machines': migrant workers' conceptual framework of labour exploitation for health research and policy

BACKGROUND

The exploitation of migrant workers ranks high on global political agendas including the Sustainable Development Goals. Research on exploited workers, using assessment tools where exploitation is defined by professional experts, indicates serious health concerns and needs. Yet, migrant workers are rarely asked about their understanding of a phenomenon they may experience. Our study aimed to conceptualise 'labour exploitation' from the perspective of migrant workers employed in manual low-skilled jobs.

METHODS

Twenty-seven Latin Americans working in London (UK) participated in Group Concept Mapping; a participatory mixed-method where qualitative data are collected to define a concept's content and then analysed using quantitative methods to generate a structured conceptual framework. Participants generated statements describing the concept content during brainstorming sessions, and structured them during sorting-rating exercises. Multi-Dimensional Scaling and Cluster Analysis were performed, generating a conceptual framework that clarified the dimensions, subdimensions and constituent statements of the concept of labour exploitation from migrant workers' perspectives.

RESULTS

Three key dimensions were identified: 'poor employment conditions and lack of protection', covering contractual arrangements and employment relations; 'disposability and abuse of power' (or 'dehumanisation') covering mechanisms or means which make migrant workers feel disposable and abused; and 'health and safety and psychosocial hazards' encompassing issues from physical and psychosocial hazards to a lack of health and social protection. 'Dehumanisation' has not been included in mainstream tools assessing exploitation, despite its importance for study participants who also described harsh situations at work including sexual, physical and verbal abuse.

CONCLUSION

Our study provides a conceptual framework of labour exploitation that gives voice to migrant workers and can be operationalised into a measure of migrant labour exploitation. It also calls for the dimension 'dehumanisation' and structural forms of coercion to be integrated into mainstream conceptualisations, and their workplace hazards to be urgently addressed.

Digital transformation and middle managers' leadership skills and behavior: a group concept mapping approach

This study, with the aim to test theory in practice, used group concept mapping to develop a comprehensive conceptualization of middle managers' leadership behaviors concerning digital transformation as a form of radical change. Participants were professionals in the largest public organization in the Netherlands (a police organization) who were dealing with digital transformation in their own practice and who enrolled in an education program on leadership and intelligence. Based on 94 unique statements, the participant-driven results revealed six thematically coherent clusters representing leadership skills and behaviors regarding improvement and results, digital technologies, cooperation, the self, change and ambivalence, and others. The stress value of 0.2234 indicated a good fit. Further analysis showed that clusters containing soft skills and people-oriented behaviors were considered the most important. These results can serve as input to support leadership development programs for middle managers to develop themselves into people-oriented, empowering leaders who can adapt their leadership approaches to fit and support change in general and technology-driven change in particular. Ultimately this will benefit their and their employees' overall well-being at work. This study is the first to investigate middle managers' leadership skills and behaviors in a large public organization that is entirely participant-driven.

Development of a conceptual model of the capacity for patients to engage in their health care: a group concept mapping study

BACKGROUND

Patient engagement is seen as a necessary component in achieving the triple aim of improved population health, improved experience of care, and lower per capita health care costs. While there has been a substantial increase in the number of tools and patient-centered initiatives designed to help patients participate in health decisions, there remains a limited understanding of engagement from the perspective of patients and a lack of measures designed to capture the multi-faceted nature of the concept.

METHODS

Development of a concept map of patient engagement followed a five-step modified Group Concept Mapping (GCM) methodology of preparation, generation, structuring, analysis and interpretation. We engaged a Project Advisory Committee at each step, along with three rounds of survey collection from clinicians and patients for element generation (272 clinicians, 61 patients), statement sorting (30 clinicians, 15 patients), and ranking and rating of statements (159 clinicians, 67 patients). The survey of three separate samples, as opposed to focus groups of 'experts,' was an intentional decision to gain a broad perspective about the concept of patient engagement. We conducted the structure and analysis steps within the groupwisdom concept mapping software.

RESULTS

The final concept map comprised 47 elements organized into 5 clusters: Relationship with Provider, Patient Attitudes and Behaviors, Access, Internal Resources and External Resources. There was considerable agreement in the way elements in each cluster were rated by patients and clinicians. An analysis of the importance of the constitutive elements of patient engagement relative to their addressability highlighted actionable items in the domain of Relationship with Provider, aimed at building trust and enabling patients to ask questions. At the same time, the analysis also identified elements traditionally considered barriers to engagement, like personal access to the internet and the patient's level of digital literacy, as difficult to address by the healthcare system, but also relatively less important for patients.

CONCLUSIONS

Through our GCM approach, incorporating perspectives of both patients and clinicians, we identified items that can be used to assess patient engagement efforts by healthcare systems. As a result, our study offers specific insight into areas that can be targeted for intervention by healthcare systems to improve patient engagement.

Identifying patient-valued outcomes for use in early phase trials of ocular surface disease interventions

BACKGROUND

Patient-reported outcomes (PROs) can be used to evaluate the impact of dry eye symptoms (DES) on daily life. Early-phase clinical trials provide an opportunity to evaluate PRO strategies. Existing measures identified through systematic review omitted important concepts that mattered to patients. The aim of our work was to develop a conceptual map of DES and assess the relative importance of identified concepts.

METHODS

Web-based group concept mapping software was used to develop a conceptual map. This semi-quantitative mixed-methods approach consists of three stages 1) statement generation, 2) thematic sorting, 3) rating of statements for importance [1 (not important), 2 (important), 3 (very important)] and relevance [1 (not my experience), 2 (sometimes my experience); 3 (definitely my experience)]. Thirty-nine participants were recruited from two UK-based patient support groups (British Sjögren's Syndrome Association, PemFriends). Three withdrew, two for health reasons and one struggled with the web-based format.

RESULTS

125 statements and six thematic clusters were generated. The Environmental Impacts cluster scored highest for importance (2.45), followed by Pain and Discomfort (2.35), Eye Treatments (2.32), Daily Impact (2.07), Psychosocial Issues (1.78) and Miscellaneous (1.78). Mapping statements against existing PRO measures confirmed a number of important missing issues including the impact of 'UV levels' (2.50), hot dry weather (2.33), the temporal aspects of pain (2.64), and issues with night-driving (2.59).

CONCLUSIONS

Group concept mapping identified important issues for people living with DES not currently captured by existing PROs, highlighting the need for additional PRO items to be considered for use in clinical trials.

British Gen Z perceptions of sustainable fisheries: developing a measurement instrument

With continued pressure upon fisheries, alongside improved fisheries management in some places, there has been a decline in fish catches with changes in the structure of the ecosystem affecting its' functionality. The use of scientific based indicator frameworks in an attempt to pursue sustainable fisheries is a common trend, however the management decisions are often socially driven and can deviate, or even, conflict with the science. This study aimed to investigate British Gen Z perceptions upon what creates a sustainable fishing industry through a series of three studies. A compilation of 82 statements, derived from practitioner indicator systems, was used by 23 participants in a concept mapping process consisting of five steps; create statements, sort & rate statements, multi-dimensional scaling of sorted units, cluster analysis, and label the clusters. The next stage of the study looked to refine the large number of statements by statistically verifying them with exploratory factor analysis and confirmatory composite analysis using split halves of a sample containing 657 participants and generating a three-factor solution of Community, Ecological Management and, Economic. The domain was then switched to the descriptive typology and a further 179 British Gen Z evaluated the fisheries at Hastings and Brixham based upon case studies which further validated the three-factor solution. The significance of this study demonstrates that consumers, who influence policy and management of fisheries through their purchasing behaviour, interpret sustainable fisheries differently to science which is evidenced by the merging of the elements of ecologically and management.

How Should Job Crafting Interventions Be Implemented to Make Their Effects Last? Protocol for a Group Concept Mapping Study

BACKGROUND

By means of job crafting (JC) employees shape and customize their job design to align it with their preferences. Research has so far shown that such bottom-up proactivity can be stimulated via JC interventions. While the overall effectiveness behind these interventions has been supported, it is unclear how to implement these interventions to make their effects lasting.

METHODS

The overall aim of this project will be to investigate how to implement JC interventions with lasting effects. We will apply a group concept mapping (GCM) methodology, which is a mixed methods approach of exploratory nature for engaging stakeholder groups in a structured conceptualization process. As part of concept mapping procedures, brainstorming sessions will be conducted with experts in job crafting to identify factors expected to make job crafting intervention effects lasting. These factors will be sorted by similarity and rated by each participant in regard to their perceived importance and feasibility to ensure lasting, sustainable effects. The data will be analyzed using multidimensional scaling (MDS), hierarchical cluster analysis, and descriptive and inferential statistics, resulting in a visual representation of conceptually distinguished clusters representing the factors influencing the sustainability of JC interventions. In the final step, a workshop will be conducted with the participants to facilitate the interpretation of the results.

RESULTS AND CONCLUSION

This study will provide knowledge relevant to organizational practitioners and scholars who want to implement JC interventions with lasting effects. Although data collected following the group concept mapping procedure is self-reported and at risk of being simplified, the method allows for a structured conceptualization process integrating different perspectives and uncovering implicit knowledge making it suitable for studying complex phenomena. The results will not only enrich the current literature concerning the effectiveness of JC interventions but also be used to develop a practitioner-oriented toolkit outlining evidence-based recommendations concerning designing and implementing, as well as evaluating JC interventions.

Living with a left ventricular assist device: Capturing recipients experiences using group concept mapping software

Background

Left ventricular assist device (LVAD) implantation significantly impacts on a recipient’s symptoms and quality of life. Capturing their experiences and post implant journey is an important part of clinical practice, research and device design evolution. Patient reported outcome measures (PROMs) are a useful tool for capturing that experience. However, patient reported outcome measures need to reflect recipients’ experiences. Discussions with a patient partner group found that none of the frequently used cardiology PROMs captured their unique experiences.

Aims

To capture the experiences and important issues for LVAD recipients. Develop a conceptual map of domains and items that should be reflected in patient reported outcomes.

Methods

Group concept mapping (GCM) web-based software was used to remotely capture and structure recipients’ experiences across a wide geographical area. GCM is a semi-quantitative mixed method consisting of 3 stages: item generation, item sorting and rating (importance, relevance and frequency). Patient partners were involved in all aspects of the study design and development.

Results

18 LVAD recipients consented to take part. 101 statements were generated and multi-dimensional scaling, and hierarchical cluster analysis identified 9 clusters. Cluster themes included: Activities, Partner/family support, Travel, Mental wellbeing, Equipment and clothing, Physical and cognitive limitations, LVAD Restrictions, LVAD Challenges and positive impact of the LVAD (LVAD Positives). LVAD Positives were scored highest across all the rating variables, e.g., frequency (2.85), relevance (2.44) and importance (2.21). Other domains rated high for importance included physical and cognitive limitations (2.19), LVAD restrictions (2.11), Partner/family support (2.02), and Equipment and clothing (2.01).

Conclusion

Online GCM software facilitated the inclusion of geographically dispersed recipients and provided useful insights into the experiences of LVAD recipients. The conceptual framework identifies important domains and items that should be prioritised and included in patient reported outcomes in future research, LVAD design evolution, and clinical practice.

Concept Mapping to Identify Content for a Performance-Based Measure of Low Luminance Vision-Related Activities of Daily Living

Purpose

The purpose of this study was to identify low luminance activities of daily living (ADL) relevant to adults with vision impairment using a concept-mapping approach.

Methods

“Group concept mapping” was utilized to identify specific ADLs that persons with vision impairment find challenging under low light conditions. In the first “brainstorming” phase, 24 adults with vision impairment from a range of eye conditions (mean age = 73 years, SD = 14 years) and 26 international low vision experts (mean experience = 22, SD = 11 years) generated statements to the focus prompt, “Thinking as broadly as possible, generate a list of statements detailing specific day-to-day activities a person with vision impairment might find challenging under low light conditions, such as in a poorly lit room or outside at dusk.” In the second phase, participants sorted activities by similarity and rated the importance of each activity. Multidimensional scaling and hierarchical cluster analysis were applied to produce concept maps showing clusters of prioritized activities.

Results

One hundred thirteen unique ideas/activities were generated, rated and sorted. Eight clusters were identified (from highest to lowest importance): hazard detection and safety outside; social interactions; navigation; near reading; selfcare and safety at home; distance spotting; searching around the home; and cooking and cleaning.

Conclusions

The conceptual framework and low luminance ADLs identified (the most important being hazard detection and safety outside, and social interactions) provide a basis for developing a performance-based measure of low luminance visual function.

Translational Relevance

A performance-based measure of low luminance vision-related ADLs is required for comprehensively and objectively assessing efficacy of eye treatments and low vision rehabilitation outcomes in adults with vision impairment.

Building a better understanding of labour exploitation's impact on migrant health: An operational framework

BACKGROUND

There is limited evidence on labour exploitation's impact on migrant health. This population is, however, often employed in manual low-skilled jobs known for poor labour conditions and exploitation risks. The lack of a common conceptualisation of labour exploitation in health research impedes the development of research measuring its effects on migrant health and, ultimately, our understanding of migrants' health needs.

AIM

To develop an operational conceptual framework of labour exploitation focusing on migrant workers in manual low-skilled jobs.

METHODS

Non-probabilistic sampling was used to recruit multidisciplinary experts on labour exploitation. An online Group Concept Mapping (GCM) was conducted. Experts: 1) generated statements describing the concept 'labour exploitation' focusing on migrants working in manual low-skilled jobs; 2) sorted generated statements into groups reflecting common themes; and 3) rated them according to their importance in characterising a situation as migrant labour exploitation. Multidimensional Scaling and Cluster Analysis were used to produce an operational framework detailing the concept content (dimensions, statements, and corresponding averaged rating).

FINDINGS

Thirty-two experts sorted and rated 96 statements according to their relative importance (1 "relatively unimportant" to 5 "extremely important"). The operational framework consists of four key dimensions of migrant labour exploitation, distributed along a continuum of severity revealed by the rating: 'Shelter and personal security' (rating: 4.47); 'Finance and migration' (4.15); 'Health and safety' (3.96); and 'Social and legal protection' (3.71).

CONCLUSION

This study is the first to both generate an empirical operational framework of migrant labour exploitation, and demonstrate the existence of a "continuum from decent work to forced labour". The framework content can be operationalised to measure labour exploitation. It paves the way to better understand how different levels of exploitation affect migrant workers' health for global policymakers, health researchers, and professionals working in the field of migrant exploitation.

Development of a Guideline to Enhance the Reporting of Concept Mapping Research: Study Protocol

Reporting guidelines are intended to enhance the clarity and transparency of research publications. Concept mapping, a mixed-methods design, has been widely used in health research. Current reporting guidelines for mixed-methods studies are not relevant for concept mapping research. The aim of this study is to develop a reporting guideline for concept mapping research following the EQUATOR network toolkit. Guideline development is in three stages: 1. A systematic review to identify key components of reporting concept mapping research, 2. A concept mapping study involving the key stakeholder groups—researchers, methodologists, peer reviewers, journal editors, statisticians, and people who have participated in concept mapping research—to identify candidate items to include in a reporting guideline, and 3. Development of a draft reporting guideline for concept mapping research. The outcome of the research will be a reporting guideline for concept mapping research.

Conceptualization and Inventory of the Sexual and Psychological Burden of Women With Pelvic Floor Complaints; A Mixed-Method Study

Introduction

Despite the fact that the literature reports various restrictions and types of distress in women with pelvic floor complaints, a comprehensive overview of women's sexual and psychological burden emerging from these complaints is lacking, which compromises our ability to assess and grasp the impact to women.

Aim

This study was performed to conceptualize women's sexual and psychological burden and create a more comprehensive overview on this topic from both women's and health care providers’ perspectives. Furthermore, this research intended to identify items to populate a to-be-developed instrument to assess sexual and psychological burden.

Methods

In Group Concept Mapping, 125 statements were used about restrictions and distress that women with pelvic floor complaints experienced. Women with, and health care providers with and without pelvic floor complaints (13 women and 3 men) sorted the statements into comprehensive self-labeled clusters and rated their nature and severity. Multidimensional scaling and hierarchical cluster analyses were performed to identify a conceptual model of coherent clusters of statements. Item-total correlations of severity scores were calculated to identify statements that can be used in future research to represent women's sexual and psychological burden.

Main Outcome Measure

A conceptual model emerged, and outcomes of item-total correlations were then examined again using the conceptual model.

Results

Seven distress clusters were identified, namely, loss of control, sexual distress, feeling insecure, feeling wronged, feeling helpless, feeling angry, and feeling disappointed. Feeling insecure appeared more pervasive than other distresses. Furthermore, 33 statements were identified that can be used in future research to develop an instrument to assess sexual and psychological burden representing both women's and health care providers’ perspectives.

Conclusion

The conceptual model and list of statements may concisely represent the sexual and psychological burden of women with pelvic floor complaints from both women's and health care providers’ perspectives on this topic.

Brand AM, Rosas S, Waterink W, et al. Conceptualization and Inventory of the Sexual and Psychological Burden of Women With Pelvic Floor Complaints; A Mixed-Method Study. Sex Med 2022;10:100504.

Seeking a deeper understanding of 'distributed health literacy': A systematic review

BACKGROUND

Previous research suggests that it would be useful to view health literacy as a set of 'distributed competencies', which can be found dispersed through the individual's social network, rather than an exclusively individual attribute. However, to date there is no focused exploration of how distributed health literacy has been defined, conceptualized or assessed in the peer-reviewed literature.

AIMS

This systematic review aimed to explore: (1) definitions and conceptual models of distributed health literacy that are available from the peer-reviewed literature; and (2) how distributed health literacy has been measured in empirical research.

METHODS

We searched MEDLINE, Embase, CINAHL, PsycInfo, Scopus, ERIC and Web of Science using truncated versions of the keywords 'literacy' and 'distributed' (within five words' distance). We collated the definitions and conceptual models of distributed health literacy, and report on how health literacy has been measured in empirical research studies. Findings related to distributed health literacy from included manuscripts were synthesized using thematic synthesis.

RESULTS

Of the 642 studies screened, 10 were included in this systematic review. The majority were empirical manuscripts reporting on qualitative research in one of five countries, with two reviews, one conceptual analysis and one quantitative study. Edwards' definition of distributed health literacy, which emphasizes the health literacy abilities, skills and practices of others that contribute to an individual's level of health literacy was widely applied in a variety of clinical and geographical settings. However, we did not identify any quantitative instruments which directly measured distributed health literacy. There was significant variability in questions used to explore the concept qualitatively, and discrepancies across studies in regard to (a) what constitutes distributed health literacy and what does not (e.g., general social support), and (b) the relationship between distributed health literacy and other constructs (e.g., public health literacy).

CONCLUSION

Although there is a widely applied definition of distributed health literacy, our review revealed that the research space would benefit from the development of the concept, both theoretically for example via conceptual distinctions between distributed health literacy and other types of social support, and empirically for example through the development of a quantitative measurement instrument.

PATIENT OR PUBLIC CONTRIBUTION

This paper is a systematic review and did not involve patients or the public.

Reflections on How Tinnitus Impacts the Lives of Children and Adolescents

Objectives The aim of this study was to generate a conceptual framework describing which aspects of children and adolescents' lives are affected by chronic tinnitus. Design Views and experiences of 32 participants from two participant groups informed this study: (a) a tinnitus group, consisting of adults who had experienced tinnitus during childhood and/or adolescence and primary carers of children/adolescents with tinnitus, and (b) a clinicians' group, consisting of clinicians who provided care for children/adolescents with tinnitus. Participants produced statements describing aspects of children/adolescents' lives that may be affected by chronic tinnitus. Key concepts were identified through the processes of sorting the statements and rating them for degree of associated impact. Result Participants identified 118 unique aspects of the lives of children/adolescents who may be affected by chronic tinnitus. These were clustered into four concepts: (a) emotional well-being, (b) academic performances, (c) social/relationa, and (d) auditory/cognitive processing. At a group level, participants rated the impact of tinnitus as above a slight degree but below a moderate degree of impact. However, individual participant's ratings indicated a range of perceived impact for each statement. Conclusions The experience of chronic tinnitus during childhood and adolescence extends beyond the mere perception of sound. The perception of tinnitus may impact a child's emotional well-being, academic performances, social/relational, and auditory/cognitive processing. The impact of tinnitus in one aspect of a child's life may influence other aspects of their life. While at a group level, participants regarded the impact of tinnitus as "somewhat more than mild" to "less than moderate"; individual participant's ratings indicate that the impact from chronic tinnitus may be highly individual and highlighted the importance of individual assessment and management. Clinically, tinnitus management during childhood and adolescence may be improved if clinicians consider the impact and manifestation of tinnitus within each child's daily life and tailor tinnitus education and management strategies accordingly.

The Clinical Sustainability Assessment Tool: measuring organizational capacity to promote sustainability in healthcare

Background

Few validated assessment tools are available to increase understanding and measure factors associated with sustainment of clinical practices, an increasingly recognized need among clinicians. We describe the development of the Clinical Sustainability Assessment Tool (CSAT), designed to assess factors that contribute to sustainable practices in clinical settings.

Methods

Sixty-four participants from clinical and research fields participated in concept mapping and were recruited to brainstorm factors that lead to sustained clinical practices. Once repeated factors were removed, participants sorted items based on similarity and rated them by importance and feasibility. Using concept mapping analyses, items were grouped into meaningful domains to develop an initial tool. We then recruited pilot sites and early adopters, for a total of 286 practicing clinicians, to pilot and evaluate the tool. Individuals were recruited from clinical settings across pediatric and adult medical and surgical subspecialties. The data were analyzed using confirmatory factor analysis (CFA) to test hypothesized subscale structure in the instrument. We used root mean square error of approximation (RMSEA) and the standardized root mean square residual (SRMR) to assess fit and thus the ability of CSAT to measure the identified domains.

Results

The concept mapping produced sorted statements that were edited into items that could be responded to, resulting in the creation of a tool with seven determinant domains and 47 items. The pilot and CFA testing resulted in a final CSAT instrument made up 35 items, five per domain. CFA results demonstrated very good fit of the seven domain structure of the CSAT (RMSEA = 0.049; SRMR = 0.049). Usability testing indicated the CSAT is brief, easy to use, easy to learn, and does not require extensive training. Additionally, the measure scored highly (18/20) on the Psychometric and Pragmatic Evidence Rating Scale (PAPERS). The seven final CSAT domains were engaged staff and leadership, engaged stakeholders, organizational readiness, workflow integration, implementation and training, monitoring and evaluation, and outcomes and effectiveness.

Conclusions

The CSAT is a new reliable assessment tool which allows for greater practical and scientific understanding of contextual factors that enable sustainable clinical practices over time.

Supplementary Information

The online version contains supplementary material available at 10.1186/s43058-021-00181-2.

Reducing the Impact of Tinnitus on Children and Adolescents' Lives: A Mixed-Methods Concept Mapping Study

OBJECTIVES

To generate a conceptual framework describing what is done to reduce the impact of chronic tinnitus on the lives of children and adolescents.

DESIGN

Views and experiences of 32 adults from two participant groups informed this concept mapping study: (i) a tinnitus group (adults who experienced tinnitus during childhood/adolescence, and primary carers of children/adolescents with tinnitus) and (ii) a clinicians' group (clinicians who provided care for children/adolescents with tinnitus). Participants produced statements describing what is done to reduce the impact of chronic tinnitus on the lives of children and adolescents who experience it. Through grouping and rating processes, they identified key concepts and inferred their associated benefit.

RESULTS

The participants generated 102 unique statements across four concepts: (1) Education, Support, and Counselling; (2) Support from Parents and Teachers; (3) Clinical Assessments and Management; and (4) Self-Management Techniques. Many statements highlighted the need for child-friendly and patient-centred care. Adults with personal experience of childhood tinnitus tended to perceive many of the statements as more beneficial than did the clinician group.

CONCLUSIONS

Although many children will develop management strategies to assist them with their tinnitus, both the adults who experienced tinnitus as children and their parents valued strategies involving clinical care, knowledge, and expertise. Participants from the tinnitus group perceived a greater degree of benefit associated with strategies from all four clusters than the clinicians' group. However, both groups perceived the greatest degree of benefit as being associated with activities and strategies within the Education, Support, and Counselling and the Clinical Assessments and Management clusters. Both groups identified that recognising the occurrence of tinnitus for children and adolescents, acknowledging the potential for associated distress, and initiating clinical care provide the nexus of effective management. Addressing the concerns and needs of parents was also perceived as valuable; hence, approaching the management of tinnitus during childhood and adolescence from a family-centred care framework is recommended.

Factors affecting clinicians' adherence to principles of diagnosis documentation: A concept mapping approach for improved decision-making

BACKGROUND

The quality of data in electronic health records (EHRs) depends on adherence of clinicians to principles of diagnosis documentation.

OBJECTIVE

A concept mapping (CM) approach was used to extract factors related to quality of clinicians' documentation that govern EHR data quality.

METHOD

Influential factors extracted from brainstorming sessions were sorted by individual participants, followed by a quantitative analysis using multidimensional scaling and cluster analysis to categorise sorted factors. Finally, a questionnaire was used to elicit the importance-feasibility of the extracted factors. Results were visualised by cluster maps and Go-Zone plots.

RESULT

Factors were classified into seven clusters: "knowledge about International Classification of Diseases and clinical coding," "need for facilitators and guidelines," "explaining the importance of the issue and defining responsibilities," "cooperation of other personnel," "codify legal requirements," "workload" and "clinical obstacles," as ranked by importance.

CONCLUSION

To enhance the quality of EHR data, a collaboration between physicians, nurses, managers and EHR developers is required. CM is an acceptable approach to meet this objective. Our findings highlight the significance of clinical coding knowledge, awareness about its importance and applicability and use of well-structured information systems. In combination, these three factors can have a strong positive impact on the quality of EHR data.

IMPLICATIONS

A list of solutions is provided for policymakers, and two interventions suggested, based on the findings of this study, including the adoption of EHRs that incorporate documentation guidelines. We further propose updated clinical training programs and a monitoring and feedback mechanism to facilitate the EHR documentation process.

Psychometric Properties of the QoL-ME: A Visual and Personalized Quality of Life Assessment App for People With Severe Mental Health Problems

Background: Quality of Life (QoL) assessment in people with severe mental health problems may benefit from improved personalization and accessibility. Therefore, an innovative, digital, visual, and personalized QoL assessment app for people with severe mental health problems was recently developed: the QoL-ME. The aim of this study was to evaluate the psychometric quality of the QoL-ME by assessing its reliability, validity, and responsiveness. Methods: To examine the reliability of the QoL-ME, the internal consistency of its subscales was assessed using Cronbach's Alpha. Correlations between the QoL-ME and the MANSA were computed to appraise the construct validity of the QoL-ME. Internal responsiveness was evaluated using the standardized response mean and external responsiveness was investigated using hierarchical regression. Results: Cronbach's Alpha's of the subscales of the QoL-ME ranged between 0.5 and 0.84. In accordance with expectations, the language-based core version of the QoL-ME correlated strongly (r = between 0.55 and 0.76) with the MANSA, whilst the picture-based additional modules of the QoL-ME correlated moderately (r = 0.3) with the MANSA. The standardized response mean was 0.23 and the regression model revealed a coefficient β of -0.01. Conclusions: The QoL-ME has adequate psychometric properties. In comparison with similar pictorial instruments, both the QoL-ME's reliability and validity can be considered as sufficient. The results indicate that the responsiveness of the QoL-ME is insufficient. Additional research is needed to evaluate and potentially modify the instrument to improve its responsiveness.

Conceptualizing social well-being in activity-based offices

Purpose

The purpose of this article is to aid conceptualization of social well-being at work by identifying its components in a contemporary office context, so adequate measures can be developed to monitor social well-being and to assess the impact of interventions in the workplace.

Design/methodology/approach

This study used existing interview data from recent post-occupancy evaluations of two large activity-based flexible offices in the Dutch public sector. Data-driven concept mapping of 182 different employees' statements on social aspects of well-being was used to find communalities in their perceptions.

Findings

From the data 14 key concepts emerged referring to employees' social needs, reactions to (anti-)social behaviour of others and perceived social affordances of the work environment. Contrary to established theory, social well-being appeared to be a context-bound phenomenon, including components of both short-term hedonic and long-term eudaimonic well-being.

Research limitations/implications

The findings serve as an inductive source for the further development of adequate measures of social well-being at work. Limitations concern the specific (cultural) setting of the cases and the use of existing data.

Practical implications

Preliminary suggestions for fostering social well-being include change management, participatory design, being alert of the identified risks of activity-based offices and supporting privacy regulation, identity marking and a sense of community, as well as a diversity of informal face-to-face interactions balanced with quiet spaces.

Originality/value

This article contributes to the conceptualization of social well-being in contemporary offices by discussing established social well-being theory and analysing real-world data, using a method novel to management research.

Care Staff Perspectives on Using Mobile Technology to Support Communication in Long-Term Care: Mixed Methods Study

BACKGROUND

Long-term care (LTC) homes provide 24-hour care for people living with complex care needs. LTC staff assist older adults living with chronic conditions such as Alzheimer disease, related dementias, and stroke, which can cause communication disorders. In addition to the complex cognitive challenges that can impact communication, further difficulties can arise from cultural-language differences between care staff and residents. Breakdowns in caregiver-resident communication can negatively impact the delivery of person-centered care. Recent advances in mobile technology, specifically mobile devices (tablets and smartphones) and their software apps, offer innovative solutions for supporting everyday communication between care staff and residents. To date, little is known about the care staff's perspectives on the different ways that mobile technology could be used to support communication with residents.

OBJECTIVE

This study aims to identify care staff's perspectives on the different ways of using devices and apps to support everyday communication with adults living in LTC homes and the priority care areas for using mobile technology to support communication with residents.

METHODS

This descriptive study employed concept mapping methods to explore care staff's perspectives about ways of using mobile technology with residents and to identify the usefulness, practicality, and probable uses of mobile technology to support communication in priority care areas. Concept mapping is an integrated mixed methods approach (qualitative and quantitative) that uses a structured process to identify priority areas for planning and evaluation. In total, 13 care staff from a single LTC home participated in this study. Concept mapping includes 2 main data collection phases: (1) statement generations through brainstorming and (2) statement structuring through sorting and rating. Brainstorming took place in person in a group session, whereas sorting and rating occurred individually after the brainstorming session. Concept mapping data were analyzed using multidimensional scaling and cluster analysis to generate numerous interpretable data maps and displays.

RESULTS

Participants generated 67 unique statements during the brainstorming session. Following the sorting and rating of the statements, a concept map analysis was performed. In total, 5 clusters were identified: (1) connect, (2) care management, (3) facilitate, (4) caregiving, and (5) overcoming barriers. Although all 5 clusters were rated as useful, with a mean score of 4.1 to 4.5 (Likert: 1-5), the care staff rated cluster 2 (care management) as highest on usefulness, practicality, and probable use of mobile technology to support communication in LTC.

CONCLUSIONS

This study provided insight into the viewpoints of care staff regarding the different ways mobile technology could be used to support caregiver-resident communication in LTC. Our findings suggest that care management, facilitating communication, and overcoming barriers are 3 priority target areas for implementing mobile health interventions to promote person-centered care and resident-centered care.

Questionnaire validation practice within a theoretical framework: a systematic descriptive literature review of health literacy assessments

OBJECTIVE

Validity refers to the extent to which evidence and theory support the adequacy and appropriateness of inferences based on score interpretations. The health sector is lacking a theoretically-driven framework for the development, testing and use of health assessments. This study used the Standards for Educational and Psychological Testing framework of five sources of validity evidence to assess the types of evidence reported for health literacy assessments, and to identify studies that referred to a theoretical validity testing framework.

METHODS

A systematic descriptive literature review investigated methods and results in health literacy assessment development, application and validity testing studies. Electronic searches were conducted in EBSCOhost, Embase, Open Access Theses and Dissertations and ProQuest Dissertations. Data were coded to the Standards' five sources of validity evidence, and for reference to a validity testing framework.

RESULTS

Coding on 46 studies resulted in 195 instances of validity evidence across the five sources. Only nine studies directly or indirectly referenced a validity testing framework. Evidence based on relations to other variables is most frequently reported.

CONCLUSIONS

The health and health equity of individuals and populations are increasingly dependent on decisions based on data collected through health assessments. An evidence-based theoretical framework provides structure and coherence to existing evidence and stipulates where further evidence is required to evaluate the extent to which data are valid for an intended purpose. This review demonstrates the use of the Standards' theoretical validity testing framework to evaluate sources of evidence reported for health literacy assessments. Findings indicate that theoretical validity testing frameworks are rarely used to collate and evaluate evidence in validation practice for health literacy assessments. Use of the Standards' theoretical validity testing framework would improve evaluation of the evidence for inferences derived from health assessment data on which public health and health equity decisions are based.

Negotiating Participatory Research Processes With Domestic Violence Program Staff to Obtain Ecologically Valid Data

The choice of research design affects not only internal and external validity but ecological validity as well. Ecological validity refers to the extent to which data are based in and relevant to real-world settings. One way for researchers to maximize the likelihood of achieving ecological validity is to design studies with community partners that use participatory approaches. Engaging deeply with community partners in determining what to study, how to measure constructs of interest, interpreting results, and using findings to create change is one means of conducting research that is likely to produce findings that are meaningful to communities. Conducting in-depth, participatory research within agencies providing crisis supports to individuals who have experienced trauma creates an additional level of complexity in the research process. This article presents a case example of how academic researchers and staff members of a nonprofit organization working with survivors of intimate partner violence collaborated on an evaluation of the community partner agency. As part of this process, the team members had to be continually aware of and responsive to the nature of crisis-focused work. We provide a reflexive account of the lessons learned and recommendations for future research.

Hit or Miss? Evaluating the Potential of a Research Niche: A Case Study in the Field of Virtual Quality Management

When knowledge is developed fast, as it is the case so often nowadays, one of the main difficulties in initiating new research in any field is to identify the domain’s specific state-of-the-art and trends. In this context, to evaluate the potential of a research niche by assisting the literature review process and to add a new and modern large-scale and automated dimension to it, the paper proposes a methodology that uses “Latent Semantic Analysis” (LSA) for identifying trends, focused within the knowledge space created at the intersection of three sustainability-related methodologies/concepts: “virtual Quality Management” (vQM), “Industry 4.0”, and “Product Life-Cycle” (PLC). The LSA was applied to a significant number of scientific papers published around these concepts to generate ontology charts that describe the knowledge structure of each by the frequency, position, and causal relation of associated notions. These notions are combined for defining the common high-density knowledge zone from where new technological solutions are expected to emerge throughout the PLC. The authors propose the concept of the knowledge space, which is characterized through specific descriptors with their own evaluation scales, obtained by processing the emerging information as identified by a combination of classic and innovative techniques. The results are validated through an investigation that surveys a relevant number of general managers, specialists, and consultants in the field of quality in the automotive sector from Romania. This practical demonstration follows each step of the theoretical approach and yields results that prove the capability of the method to contribute to the understanding and elucidation of the scientific area to which it is applied. Once validated, the method could be transferred to fields with similar characteristics.

Eliciting patient-important outcomes through group brainstorming: when is saturation reached?

PURPOSE

Group brainstorming is a technique for the elicitation of patient input that has many potential uses, however no data demonstrate concept saturation. In this study we explore concept saturation in group brainstorming performed in a single session as compared to two or three sessions.

METHODS

Fifty-two predominately African American adults patients with moderately to poorly controlled Diabetes Mellitus participated in three separate group brainstorming sessions as part of a PCORI-funded group concept mapping study examining comparing methods for the elicitation of patient important outcomes (PIOs). Brainstorming was unstructured, in response to a prompt designed to elicit PIOs in diabetes care. We combined similar brainstormed responses from all three sessions into a 'master list' of unique PIOs, and then compared the proportion obtained at each individual session, as well as those obtained in combinations of 2 sessions, to the master list.

RESULTS

Twenty-four participants generated 85 responses in session A, 14 participants generated 63 in session B, and 14 participants generated 47 in session C. Compared to the master list, the individual sessions contributed 87%, 76%, and 63% of PIOs. Session B added 3 unique PIOs not present in session A, and session C added 2 PIOs not present in either A or B. No single session achieved >90% saturation of the master list, but all 3 combinations of 2 sessions achieved > 90%.

CONCLUSIONS

Single sessions elicited only 63-87% of the patient-important outcomes obtained across all three sessions, however all combinations of two sessions elicited over 90% of the master list, suggesting that 2 sessions are sufficient for concept saturation.

TRIAL REGISTRATION

NCT02792777 . Registered 2 June 2016.

The power of the group: comparison of interviews and group concept mapping for identifying patient-important outcomes of care

BACKGROUND

Data are limited regarding how to effectively and efficiently identify patient priorities for research or clinical care. Our goal was to compare the comprehensiveness and efficiency of group concept mapping (GCM), a group participatory method, to interviews for identifying patient goals when seeking care.

METHODS

We engaged patients with moderately- to poorly-controlled diabetes mellitus in either GCM or an individual interview. The primary outcome was the comprehensiveness of GCM brainstorming (the first stage of GCM) as compared to interviews for eliciting patient-important outcomes (PIOs) related to seeking care. Secondary outcomes included 1) comprehensiveness of GCM brainstorming and interviews compared to a master list of PIOs and 2) efficiency of GCM brainstorming, the entire GCM process and interviews.

RESULTS

We engaged 89 interview participants and 52 GCM participants (across 3 iterations of GCM) to identify outcomes most important to patients when making decisions related to diabetes management. We identified 26 PIOs in interviews, 33 PIOs in the first GCM brainstorming session, and 38 PIOs across all three GCM brainstorming sessions. The initial GCM brainstorming session identified 77% (20/26) of interview PIOs, and all 3 GCM brainstorming sessions combined identified 88% (23/26). When comparing GCM brainstorming and interviews to the master list of PIOs, the initial GCM brainstorming sessions identified 80% (33/41), all 3 GCM brainstorming sessions identified 93% (38/41) and interviews identified 63% (26/41) of all PIOs. Compared to interviews, GCM brainstorming required less research team time, more patient time, and had a lowest cost. The entire GCM process still required less research team time than interviews, though required more patient time and had a higher cost than interviews.

CONCLUSIONS

GCM brainstorming is a powerful tool for effectively and efficiently identifying PIOs in certain scenarios, though it does not provide the breadth and depth of individual interviews or the higher level conceptual organization of the complete process of GCM. Selection of the optimal method for patient engagement should include consideration of multiple factors including depth of patient input desired, research team expertise, resources, and the population to be engaged.

TRIAL REGISTRATION

Registered on ClinicalTrials.gov , NCT02792777. Registration information submitted 6/2/2016, with the registration first posted on the ClinicalTrials.gov website 6/8/2016. Data collection began on 4/29/2016.

The systematic development of a complex intervention: HealthMap, an online self-management support program for people with HIV

Background

Despite persistent calls for HIV care to adopt a chronic care approach, few HIV treatment services have been able to establish service arrangements that prioritise self-management. To prevent cardiovascular and other chronic disease outcomes, the HealthMap program aims to enhance routine HIV care with opportunities for self-management support. This paper outlines the systematic process that was used to design and develop the HealthMap program, prior to its evaluation in a cluster-randomised trial.

Methods

Program development, planning and evaluation was informed by the PRECEDE-PROCOEDE Model and an Intervention Mapping approach and involved four steps: (1) a multifaceted needs assessment; (2) the identification of intervention priorities; (3) exploration and identification of the antecedents and reinforcing factors required to initiate and sustain desired change of risk behaviours; and finally (4) the development of intervention goals, strategies and methods and integrating them into a comprehensive description of the intervention components.

Results

The logic model incorporated the program’s guiding principles, program elements, hypothesised causal processes, and intended program outcomes. Grounding the development of HealthMap on a clear conceptual base, informed by the research literature and stakeholder’s perspectives, has ensured that the HealthMap program is targeted, relevant, provides transparency, and enables effective program evaluation.

Conclusions

The use of a systematic process for intervention development facilitated the development of an intervention that is patient centred, accessible, and focuses on the key determinants of health-related outcomes for people with HIV in Australia. The techniques used here may offer a useful methodology for those involved in the development and implementation of complex interventions.

Using concept mapping to develop a human rights based indicator framework to assess country efforts to strengthen rehabilitation provision and policy: the Rehabilitation System Diagnosis and Dialogue framework (RESYST)

BACKGROUND

Rehabilitation is crucial for the realization of the right to health and a proper concern of global health. Yet, reliable information to guide rehabilitation service planning is unavailable in many countries in part due to the lack of appropriate indicators. To ensure universal health coverage and meet the central imperative of "leaving no one behind" countries must be able to assess key aspects of rehabilitation policy and provision and monitor how they have discharged their human rights responsibilities towards those most disadvantaged, including people with disability. This article describes the process of developing an expert guided indicator framework to assess governments' efforts and progress in strengthening rehabilitation in line with the Convention on the Rights of Persons with Disabilities.

METHODS

A systems methodology - concept mapping - was used to capture, aggregate and confirm the knowledge of diverse stakeholders on measures thought to be useful for monitoring the implementation of the Convention with respect to health related rehabilitation. Fifty-six individuals generated a list of 107 indicators through online brainstorming which were subsequently sorted by 37 experts from the original panel into non overlapping categories. Forty-one participants rated the indicators for importance and feasibility. Multivariate statistical techniques where used to explore patterns and themes in the data and create the indicators' organizing framework which was verified and interpreted by a select number of participants.

RESULTS

A concept map of 11 clusters of indicators emerged from the analysis grouped into three broader themes: Governance and Leadership (3 clusters); Service Delivery, Financing and Oversight (6 clusters); and Human Resources (2 clusters). The indicator framework was comprehensive and well aligned with the Convention. On average, there was a moderately positive correlation between importance and feasibility of the indicators (r = .58) with experts prioritizing the indicators contained in the clusters of the Governance and Leadership domain. Two of the most important indicators arose from the Service Delivery, Financing and Oversight domain and reflect the need to monitor unmet needs and barriers in access to rehabilitation. In total, 59 indicators achieved above average score for importance and comprised the two-tiered priority set of indicators.

CONCLUSION

Concept mapping was successful in generating a shared model that enables a system's view of the most critical legal, policy and programmatic factors that must be addressed when assessing country efforts to reform, upscale and improve rehabilitation services. The Rehabilitation Systems Diagnosis and Dialogue framework provides a data driven basis for the development of standardized data collection tools to facilitate comparative analysis of rehabilitation systems. Despite agreement on the importance and feasibility of 59 indicators, further research is needed to appraise the applicability and utility of the indicators and secure a realistic assessment of rehabilitation systems.

Development of the Organisational Health Literacy Responsiveness (Org-HLR) self-assessment tool and process

BACKGROUND

The World Health Organization describes health literacy as a critical determinant of health and driver of citizen empowerment and health equity. Several studies have shown that health literacy is associated with a range of socioeconomic factors including educational attainment, financial position and ethnicity. The complexity of the health system influences how well a person is able to engage with information and services. Health organisations can empower the populations they serve and address inequity by ensuring they are health literacy responsive. The aim of this study was to develop the Organisational Health Literacy Responsiveness self-assessment tool (Org-HLR Tool), and an assessment process to support organisations with application of the tool.

METHODS

A co-design workshop with health and social service professionals was undertaken to inform the structure of the tool and assessment process. Participants critiqued existing self-assessment tools and discussed the likely utility of the data they generate. A review of widely used organisational performance assessment tools informed the structure and self-assessment process. The Organisational Health Literacy Responsiveness (Org-HLR) Framework (with seven domains/24 sub-domains) provided the structure for the assessment dimensions of the tool. The performance indicators were drawn from raw data collected during development of the Org-HLR Framework.

RESULTS

Twenty-two professionals participated in the workshop. Based on the feedback provided and a review of existing tools, a multi-stage, group-based assessment process for implementing the Org-HLR Tool was developed. The assessment process was divided into three parts; i) reflection; ii) self-rating; and iii) priority setting, each supported by a corresponding tool. The self-rating tool, consistent with the Org-HLR Framework, was divided into: External policy and funding environment; Leadership and culture; Systems, processes and policies; Access to services and programs; Community engagement and partnerships; Communication practices and standards; Workforce. Each of these had 1 to 5 sub-dimensions (24 in total), and 135 performance indicators.

CONCLUSIONS

The Org-HLR Tool and assessment process were developed to address a gap in available tools to support organisations to assess their health literacy responsiveness, and prioritise and plan their quality improvement activities. The tool is currently in the field for further utility and acceptability testing.

The preliminary measurement properties of the person-centred community care inventory (PERCCI)

Purpose

Researchers investigating person-centredness in older people’s long-term community care are hindered by the lack of appropriate measures. Studies have tended to rely on proxy indicators or generic instruments, risking invalid results. This new research aimed to develop and psychometrically test a person-centredness scale for use in older people’s community services.

Methods

Questionnaire items were sourced from groups of older people and mapped to a conceptual framework of person-centredness. A postal questionnaire in 2015–2016 tested these items with older people supported by mental health and social care services in five areas of England. Dimensionality was assessed through exploratory factor analysis and a confirmatory bifactor model, with classical item analysis removing weak items. Test–retest analysis was undertaken through a repeated postal questionnaire 3 weeks after the first.

Results

Three factors were identified, representing (i) interpersonal and (ii) organisational aspects of person-centred care; and (iii) negatively phrased items. Removing weaker items resulted in an 18-item scale. The bifactor analysis concluded the summary scale was ‘essentially unidimensional’. The Person-centred community care inventory (PERCCI) had excellent reliability, with Intra-Class Correlation Coefficient of 0.886 [95% CI 0.818–0.929]. A priori hypotheses about associations with satisfaction metrics and support variables were broadly confirmed.

Conclusions

The PERCCI has promising measurement properties and can be recommended for use in research with older adults using community mental health and social care services. Future developments must identify how sensitive the instrument is in detecting changing service quality.

The patient experience in community mental health services for older people: a concept mapping approach to support the development of a new quality measure

Background

The patient experience is a crucial part of the measurement of service quality. However, instruments to evaluate experiential quality in the community mental health care of older adults are lacking. Before designing a new instrument, clarity is needed about what is to be measured, and how care experiences are articulated by patients. The study aimed to construct a framework to describe older patients’ experience of community mental health and social care.

Methods

Concept mapping blends structured qualitative data collection with quantitative analysis in a mixed method approach. Five activities were undertaken. Patients first identified sentences describing the care experience; a card-sort exercise then grouped these thematically; multidimensional analysis portrayed these data in a map of clusters; interpretation was by patient advisers; finally a new questionnaire was designed. The research involved 22 older people with mental health problems and 29 mental health practitioners, from one region of England.

Results

Sixty-seven statements were identified that described the care experience. Analysis of card sort data revealed seven clusters, which were interpreted by patient advisers to the study as: personal qualities and relationships; communication problems; feeling powerless; in-and-out care; bureaucracy; focus on life, not just mental health; and continuity of care. These themes and the component statements were used as a foundation for later work, developing a new measure of the care experience in mental health services for older people.

Conclusions

Concept mapping has many strengths as an empirical and participant-driven means for underpinning new measurement instruments. A group of older people identified 67 candidate statements that could act as questionnaire items grouped within seven themes. Future research will establish the psychometric properties of the new measure.

Electronic supplementary material

The online version of this article (10.1186/s12913-018-3231-6) contains supplementary material, which is available to authorized users.

Towards the use of mixed methods inquiry as best practice in health outcomes research

Mixed methods research (MMR) has found an increased interest in the field of health outcomes research. Consideration for both qualitative and quantitative perspectives has become key to contextualising patient experiences in a clinically meaningful measurement framework. The purpose of this paper is to outline a process for incorporating MMR in health outcomes research to guide stakeholders in their understanding of the essence of mixed methods inquiry. In addition, this paper will outline the benefits and challenges of MMR and describe the types of support needed for designing and conducting robust MMR measurement studies. MMR involves the application of a well-defined and pre-specified research design that articulates purposely and prospectively, qualitative and quantitative components to generate an integrated set of evidence addressing a single research question. Various methodological design options are possible depending on the research question. MMR designs allow a research question to be studied thoroughly from different perspectives. When applied, it allows the strengths of one approach to complement the restrictions of another. Among other applications, MMR can be used to enhance the creation of conceptual models and development of new instruments, to interpret the meaningfulness of outcomes in a clinical study from the patient perspective, and inform health care policy. Robust MMR requires research teams with experience in both qualitative and quantitative research. Moreover, a thorough understanding of the underlying principles of MMR is recommended at the point of study conception all the way through to implementation and knowledge dissemination. The framework outlined in this paper is designed to encourage health outcomes researchers to apply MMR to their research and to facilitate innovative, patient-centred methodological solutions to address the complex challenges of the field.

A Multidimensional Tool Based on the eHealth Literacy Framework: Development and Initial Validity Testing of the eHealth Literacy Questionnaire (eHLQ)

BACKGROUND

For people to be able to access, understand, and benefit from the increasing digitalization of health services, it is critical that services are provided in a way that meets the user's needs, resources, and competence.

OBJECTIVE

The objective of the study was to develop a questionnaire that captures the 7-dimensional eHealth Literacy Framework (eHLF).

METHODS

Draft items were created in parallel in English and Danish. The items were generated from 450 statements collected during the conceptual development of eHLF. In all, 57 items (7 to 9 items per scale) were generated and adjusted after cognitive testing. Items were tested in 475 people recruited from settings in which the scale was intended to be used (community and health care settings) and including people with a range of chronic conditions. Measurement properties were assessed using approaches from item response theory (IRT) and classical test theory (CTT) such as confirmatory factor analysis (CFA) and reliability using composite scale reliability (CSR); potential bias due to age and sex was evaluated using differential item functioning (DIF).

RESULTS

CFA confirmed the presence of the 7 a priori dimensions of eHLF. Following item analysis, a 35-item 7-scale questionnaire was constructed, covering (1) using technology to process health information (5 items, CSR=.84), (2) understanding of health concepts and language (5 items, CSR=.75), (3) ability to actively engage with digital services (5 items, CSR=.86), (4) feel safe and in control (5 items, CSR=.87), (5) motivated to engage with digital services (5 items, CSR=.84), (6) access to digital services that work (6 items, CSR=.77), and (7) digital services that suit individual needs (4 items, CSR=.85). A 7-factor CFA model, using small-variance priors for cross-loadings and residual correlations, had a satisfactory fit (posterior productive P value: .27, 95% CI for the difference between the observed and replicated chi-square values: -63.7 to 133.8). The CFA showed that all items loaded strongly on their respective factors. The IRT analysis showed that no items were found to have disordered thresholds. For most scales, discriminant validity was acceptable; however, 2 pairs of dimensions were highly correlated; dimensions 1 and 5 (r=.95), and dimensions 6 and 7 (r=.96). All dimensions were retained because of strong content differentiation and potential causal relationships between these dimensions. There is no evidence of DIF.

CONCLUSIONS

The eHealth Literacy Questionnaire (eHLQ) is a multidimensional tool based on a well-defined a priori eHLF framework with robust properties. It has satisfactory evidence of construct validity and reliable measurement across a broad range of concepts (using both CTT and IRT traditions) in various groups. It is designed to be used to understand and evaluate people's interaction with digital health services.

Development and preliminary validation of a scale to measure patient uncertainty: The “Uncertainty Scale”

Research suggests that patient uncertainty related to experiencing symptoms may drive decisions to seek care. The only validated measure of patient uncertainty assesses uncertainty related to defined illness. In prior work, we engaged patients to describe uncertainty related to symptoms and used findings to develop the ‘U-Scale’ scale. In this work, we present results from preliminary scale reliability and validity testing. Psychometric testing demonstrated content validity, high internal consistency, and evidence for concurrent validity. Next steps include administration in diverse populations for continued refinement and validation, and exploration of the potential contribution of uncertainty to healthcare utilization.

Development of the organisational health literacy responsiveness (Org-HLR) framework in collaboration with health and social services professionals

BACKGROUND

The health literacy skills required by individuals to interact effectively with health services depends on the complexity of those services, and the demands they place on people. Public health and social service organisations have a responsibility to provide services and information in ways that promote equitable access and engagement, that are responsive to diverse needs and preferences, and support people to participate in decisions regarding their health and wellbeing. The aim of this study was to develop a conceptual framework describing the characteristics of health literacy responsive organisations.

METHODS

Concept mapping (CM) workshops with six groups of professionals (total N = 42) from across health and social services sectors were undertaken. An online concept mapping consultation with 153 professionals was also conducted. In these CM activities, participants responded to the seeding statement "Thinking broadly from your experiences of working in the health system, what does an organisation need to have or do in order to enable communities and community members to fully engage with information and services to promote and maintain health and wellbeing". The CM data were analysed using multidimensional scaling and hierarchical cluster analyses to derive concept maps and cluster tree diagrams. Clusters from the CM processes were then integrated by identifying themes and subthemes across tree diagrams.

RESULTS

Across the workshops, 373 statements were generated in response to the seeding statement. An additional 1206 statements were generated in the online consultation. 84 clusters were derived within the workshops and 20 from the online consultation. Seven domains of health literacy responsiveness were identified; i) External policy and funding environment; ii) Leadership and culture; iii) Systems, processes and policies; iv) Access to services and programs; v) Community engagement and partnerships; vi) Communication practices and standards; and vii) Workforce. Each domain included 1 to 5 sub-domains (24 sub-domains in total).

CONCLUSIONS

Using participatory research processes, a conceptual framework describing the characteristics, values, practices and capabilities of organisational health literacy responsiveness was derived. The framework may guide the planning and monitoring of health service and health system improvements, and has the potential to guide effective public health policy and health system reforms.

Reasons for electronic cigarette use beyond cigarette smoking cessation: A concept mapping approach

INTRODUCTION

Electronic cigarettes (ECIGs) continue to grow in popularity, however, limited research has examined reasons for ECIG use.

METHODS

This study used an integrated, mixed-method participatory research approach called concept mapping (CM) to characterize and describe adults' reasons for using ECIGs. A total of 108 adults completed a multi-module online CM study that consisted of brainstorming statements about their reasons for ECIG use, sorting each statement into conceptually similar categories, and then rating each statement based on whether it represented a reason why they have used an ECIG in the past month.

RESULTS

Participants brainstormed a total of 125 unique statements related to their reasons for ECIG use. Multivariate analyses generated a map revealing 11, interrelated components or domains that characterized their reasons for use. Importantly, reasons related to Cessation Methods, Perceived Health Benefits, Private Regard, Convenience and Conscientiousness were rated significantly higher than other categories/types of reasons related to ECIG use (p<.05). There also were significant model differences in participants' endorsement of reasons based on their demography and ECIG behaviors.

CONCLUSIONS

This study shows that ECIG users are motivated to use ECIGs for many reasons. ECIG regulations should address these reasons for ECIG use in addition to smoking cessation.