Ambient and focal attention during complex problem-solving: preliminary evidence from real-world eye movement data

Time course analysis of eye movements during free exploration of real-world scenes often reveals an increase in fixation durations together with a decrease in saccade amplitudes, which has been explained within the two visual systems approach, i.e., a transition from ambient to focal. Short fixations and long saccades during early viewing periods are classified as ambient mode of vision, which is concerned with spatial orientation and is related to simple visual properties such as motion, contrast, and location. Longer fixations and shorter saccades during later viewing periods are classified as focal mode of vision, which is concentrated in the foveal projection and is capable of object identification and its semantic categorization. While these findings are mainly obtained in the context of image exploration, the present study endeavors to investigate whether the same pattern of interplay between ambient and focal visual attention is deployed when people work on complex real-world tasks-and if so, when? Based on a re-analysis of existing data that integrates concurrent think aloud and eye tracking protocols, the present study correlated participants' internal thinking models to the parameters of their eye movements when they planned solutions to an open-ended design problem in a real-world setting. We hypothesize that switching between ambient and focal attentional processing is useful when solvers encounter difficulty compelling them to shift their conceptual direction to adjust the solution path. Individuals may prefer different attentional strategies for information-seeking behavior, such as ambient-to-focal or focal-to-ambient. The observed increase in fixation durations and decrease in saccade amplitudes during the periods around shifts in conceptual direction lends support to the postulation of the ambient-to-focal processing; however, focal-to-ambient processing is not evident. Furthermore, our data demonstrate that the beginning of a shift in conceptual direction is observable in eye movement behavior with a significant prolongation of fixation. Our findings add to the conclusions drawn from laboratory settings by providing preliminary evidence for ambient and focal processing characteristics in real-world problem-solving.

Think Aloud Testing of a Smartphone App for Lifestyle Change Among Persons at Risk of Type 2 Diabetes: Usability Study

BACKGROUND

Type 2 diabetes mellitus (DM2) is a leading cause of morbidity and mortality worldwide and is considered a global epidemic. Despite the growing evidence on the effectiveness of mobile health interventions in the management of DM2, the evidence on the effect of mobile health interventions in prevention of DM2 is sparse. Therefore, we have developed an app aiming to promote initiation of behavioral change and adherence to healthy behavior. Before commencing a small-scale randomized controlled trial to assess the feasibility of using an app for initiation and adherence of healthy behavior in people at risk of DM2, testing the usability of the app in the target population is warranted.

OBJECTIVE

The aim of this study was to assess the usability of an app among people at risk of DM2.

METHODS

A qualitative study with the use of a think aloud (TA) procedure was conducted from April to November 2022. The TA procedure consisted of 10 problem-solving tasks and a semistructured interview which was carried out after the tasks. These interviews served to gain more in-depth knowledge of the users experience of the problem-solving tasks. The TA-sessions and the postactivity interviews were recorded and transcribed verbatim, and the data were coded and analyzed following the principles of thematic analysis.

RESULTS

In total, 7 people at risk of DM2 with a median age of 66 (range 41-75) years participated in this study. The analysis resulted in the following themes: (1) user interface design; and (2) suggestions for improvements of the functionality of the app.

CONCLUSIONS

Overall, the participants were satisfied with the usability of the app. Through the TA-sessions, real time perspective on the appeal, relevance, and utility of the app were gained. Only minor changes to the functionality of the prototype app were reported as necessary to improve the usability of the app. Points of guidance from the participants in this study have been adopted and incorporated into the final design of the app now being assessed for feasibility in a small-scale randomized controlled trial.

An Unstructured Supplementary Service Data System to Verify HIV Self-Testing Among Nigerian Youths: Mixed Methods Analysis of Usability and Feasibility

BACKGROUND

Mobile health (mHealth) interventions among adolescents and young adults (AYAs) are increasingly available in African low- and middle-income countries (LMICs). For example, the unstructured supplementary service data (USSD) could be used to verify HIV self-testing (HIVST) among AYAs with poor bandwidth.

OBJECTIVE

The aim of this study is to describe the creation of an USSD platform and determine its feasibility and usability to promote the verification of HIVST results among AYAs in Nigeria.

METHODS

We developed and evaluated a USSD platform to verify HIVST results using a user-centered approach. The USSD platform guided AYAs in performing HIVST, interpreting the result, and providing linkage to care after the test. Following the usability assessment, the USSD platform was piloted. We used a mixed methods study to assess the platform's usability through a process of quantitative heuristic assessment, a qualitative think-aloud method, and an exit interview. Descriptive statistics of quantitative data and inductive thematic analysis of qualitative variables were organized.

RESULTS

A total of 19 AYAs participated in the usability test, with a median age of 19 (IQR 16-23) years. There were 11 females, 8 males, and 0 nonbinary individuals. All individuals were out-of-school AYAs. Seven of the 10 Nielsen usability heuristics assessed yielded positive results. The participants found the USSD platform easy to use, preferred the simplicity of the system, felt no need for a major improvement in the design of the platform, and were happy the system provided linkage to care following the interpretation of the HIVST results. The pilot field test of the platform enrolled 164 out-of-school AYAs, mostly young girls and women (101, 61.6%). The mean age was 17.5 (SD 3.18) years, and 92.1% (151/164) of the participants reported that they were heterosexual, while 7.9% (13/164) reported that they were gay. All the participants in the pilot study were able to conduct HIVST, interpret their results, and use the linkage to care feature of the USSD platform without any challenge. A total of 7.9% (13/164) of the AYAs had positive HIV results (reactive to the OraQuick kit).

CONCLUSIONS

This study demonstrated the usability and feasibility of using a USSD system as an alternative to mobile phone apps to verify HIVST results among Nigerian youth without smartphone access. Therefore, the use of a USSD platform has implications for the verification of HIVST in areas with low internet bandwidth. Further pragmatic trials are needed to scale up this approach.

Cognitive and Metacognitive Processes Demonstrated by Pharmacy Students When Making Therapeutic Decisions

Objective. To characterize the types of cognitive and metacognitive processes demonstrated by third-year pharmacy students during a therapeutic reasoning activity.Methods. A qualitative, descriptive study following a think-aloud protocol was used to analyze the cognitive (analytical) and metacognitive processes observed by third-year pharmacy students as they completed a 25-minute therapeutic reasoning activity. Using a deductive codebook developed from literature about reasoning, two independent coders characterized processes from students' audio-recorded, transcribed think-aloud episodes while making therapeutic decisions about simulated clinical cases.Results. A total of 40 think-aloud episodes were transcribed among the cohort. Categorization of the think-aloud transcriptions revealed a series of cognitive analytical and metacognitive processes demonstrated by students during the therapeutic decision-making activity. A total of 1792 codes were categorized as analytical processes, falling into six major themes: 69% gathering information (1232/1792), 13% processing information (227/1792), 7% making assessments (133/1792), 1% synthesizing information (19/1792), 7% articulating evidence (117/1792), and 4% making a recommendation (64/1792). In comparison to gathering information, a much lower frequency of processing and assessment was observed for students, particularly for those that were unable to resolve the case. Students' movement between major analytical processes co-occurred commonly with metacognitive processes. Of the 918 codes categorized as metacognitive processes, two major themes arose: 28% monitoring for knowledge or emotions (257/918) and 72% controlling the planning of next steps or verification of correct information (661/918). Sequencing the codes and co-occurrences of processes allowed us to propose an integrated cognitive/metacognitive model of therapeutic reasoning for students.Conclusion. This study categorizes the cognitive (analytical) and metacognitive processes engaged during pharmacy students' therapeutic reasoning process. The findings can inform current instructional practices and further research into educational activities that can strengthen pharmacy students' therapeutic reasoning skills.

Designing and Developing an eHealth Program for Patients With Persistent Physical Symptoms: Usability Study

BACKGROUND

Patients with persistent physical symptoms presenting in primary care are often affected by multiple symptoms and reduced functioning. The medical and societal costs of these patients are high, and there is a need for new interventions tailored to both the patients and health care system.

OBJECTIVE

This study aimed to examine the usability of an unguided, self-help treatment program, "My Symptoms," developed to assist patients and general practitioners in symptom management.

METHODS

In all, 11 users (4 patients with persistent physical symptoms and 7 laypeople) participated in web-based thinking-aloud interviews involving the performance of predefined tasks in the program. Thematic analysis was used to categorize the severity of usability issues. General usability heuristics were cross-referenced with the usability issues.

RESULTS

The analysis identified important usability issues related to functionality, navigation, and content. The study shows how therapeutic knowledge in some cases was lost in the translation of face-to-face therapy to a digital format. The user testing helped uncover how the functionality of the digital elements and general navigation of the program played a huge part in locating and accessing the needed treatment. Examples of redesign to mediate the therapeutic value in the digital format involving health care professionals, web developers, and users are provided. The study also highlights the differences of involving patients and laypeople in the interviews.

CONCLUSIONS

Taking the experience of common symptoms as a point of departure, patients and laypeople contributed to finding usability issues on program functionality, navigation, and content to improve the program and make the treatment more accessible to users.

The cognitive apprenticeship: advancing reasoning education by thinking aloud

Teaching clinical reasoning has long challenged educators because it requires familiarity with reasoning concepts, experience with describing thinking, and comfort with exposing uncertainty and error. We propose that teachers adopt the cognitive apprenticeship model and a method of disclosing uncertainty known as intellectual streaking. These approaches reflect a shift in the educator's mindset from transmitting medical knowledge to broadcasting cognition. We provide several examples to guide the adoption of these strategies and make recommendations for teachers and training programs to improve the teaching of clinical reasoning.

Investigating the clinical decision-making of nursing students using high-fidelity simulation, observation and think aloud: A mixed methods research study

AIMS

The aim of this study was to investigate nursing students' clinical decision-making by using high-fidelity simulation of a deteriorated patient scenario.

DESIGN

A convergent parallel mixed methods research design was used consisting of quantitative and qualitative data collection.

METHODS

Twenty-three students completed the Health Science Reasoning Test before and after the simulation between October 2015 and June 2016. They were presented with a simulated scenario and asked to 'think aloud' during and after the simulation. The students were audio-video recorded and observations were collected by the researcher.

RESULTS

There was a significant moderate increase in the 'deduction' and 'analysis' sub-scale scores and overall test score, suggestive of improved analytical decision-making processes through the simulation experience. Think-aloud and observation data identified that students predominantly applied 'forward' reasoning during the simulated 'patient's' deterioration, focusing mainly on cue acquisition. 'Backward' reasoning with a focus on cue interpretation was most prominent in the debriefing data, in line with the survey outcomes. Accurate cue interpretation of critical, key cues appeared more useful than the total number of cues in solving the main clinical case problem.

CONCLUSION

Students learn different clinical decision-making skills during the simulation compared to what they learn from debriefing. Using observation and think-aloud methods have significant benefits for researchers seeking to optimize the evaluation of the clinical decision-making process.

What do older adults think about when formulating implementation intentions for physical activity? Evidence from a qualitative study

OBJECTIVES

Physical activity is an important health behaviour especially for older adults. Forming implementation intentions is an effective strategy to implement physical activity in daily life for young and middle-aged adults. However, evidence for older adults is inconclusive. This study explored the thoughts of older adults about implementation intentions and potential barriers and facilitators while formulating them.

METHODS

Three samples of older adults from the United Kingdom (n = 8), Germany (n = 9) and Switzerland (n = 17) were prompted to think aloud while formulating implementation intentions to be more physically active. After the task, semi-structured interviews were conducted. Data were analysed thematically.

RESULTS

Participants expressed pre-established thoughts about implementation intentions (e.g. they feel too restrictive). During the formulation of implementation intentions, several barriers to creating them were reported (e.g. problems with finding cues due to absence of recurring daily routines), but participants also mentioned that forming implementation intentions acted as a facilitator for physical activity (e.g. cues as useful reminders to be active, task itself triggering self-reflection about physical activity). After the task, participants reflected on circumstances that decrease the likelihood of enacting implementation intentions (e.g. spontaneous alternative activities, weather, health-related barriers, Covid-19-related barriers), which triggered spontaneous coping planning.

CONCLUSIONS

The results on barriers and facilitators of implementation intentions and physical activity from older adults' perspectives provide starting points for improving instructions for older adults on how to create implementation intentions for physical activity. Future studies are needed to investigate whether the findings extend to implementation intentions for other behaviours.

User-Centered Design to Reduce Inappropriate Blood Transfusion Orders

BACKGROUND

To improve blood transfusion practices, we applied user-centered design (UCD) to evaluate potential changes to blood transfusion orders.

OBJECTIVES

The aim of the study is to build effective transfusion orders with different designs to improve guideline adherence.

METHODS

We developed three different versions of transfusion orders that varied how information was presented to clinicians ordering blood transfusions. We engaged 14 clinicians (residents, advanced practice providers [APPs], and attending physicians) from different specialties. We used the think aloud technique and rapid qualitative analysis to generate themes to incorporate into our modified orders.

RESULTS

Most end-users who participated in the semi-structured interviews preferred the interruptive alert design plus behavioral nudges (n = 8/14, 57%). The predominant rationale was that the in-line alert was not visually effective in capturing the end-user's attention, while the interruptive alert forced a brief stop in the workflow to consider the guidelines. All users supported the general improvements, though for different reasons, and as a result, the general improvements remained in the designs for the forthcoming trial.

CONCLUSION

The user experience uncovered through the think aloud approach produced a clear and rich understanding of potentially confounding factors in the initial design of different intervention versions. Input from end-users guided the creation of all three designs so each was addressing human factors with parity, which ensured that the results of our study reflected differences in interruptive properties of the alerts and not differences in design.

Exploring the barriers and facilitators for supporting adolescents with knee pains adherence to mobile health apps: A think-aloud study

Background

Mobile health (mHealth) applications have the potential to support adolescents' self-management of knee pain. However, ensuring adherence remains a barrier when designing mHealth concepts for adolescents.

Objective

This study aimed to explore barriers and facilitators for adhering to mHealth interventions to inform design principles.

Methods

Think-aloud tests were conducted with 12 adolescents (aged 12.5 years median) with knee pain, using a low-fidelity prototype. The prototype was informed by the authors previous work, rapid prototyping sessions with seven health professionals, and synthesis via the Behavioral Intervention Technology Model. The think-aloud tests were video recorded and analyzed thematically to identify design principles.

Results

The analysis based on user testing with adolescents with knee pain identified three themes: "user experience and feedback," "contextual challenges," and "new features" and nine subthemes. Adolescents were able to use mHealth behavioral features such as self-tracking, goal setting, education, and data visualization to capture and reflect on their knee pain developments, which facilitated use. However, adolescents struggle with timing interventions, breaking down management behaviors, and biases towards interventions were identified as internal threats to adherence. Competing activities, parental meddling, and privacy concerns were external adherence barriers. Twelve design principles were identified for integrating these insights into mHealth designs.

Conclusion

Participants' motivations for adherence were influenced by internal and external factors. While adolescents were able to use mHealth behavioral features to capture and reflect on knee pain developments, understanding how to accommodate adolescents' cognitive abilities, competing activities, and need for independence is quintessential to enhance adherence in everyday contexts.

Why Do Adults Value EQ-5D-Y-3L Health States Differently for Themselves Than for Children and Adolescents: A Think-Aloud Study

OBJECTIVES

Following protocol, adults value EQ-5D-Y-3L health states from the perspective of a 10-year-old child. It remains unclear why adults value health states differently for themselves than for a 10-year-old child and whether the latter perspective is representative of adults' preferences for the 8 to 15 years age range of the EQ-5D-Y-3L. This study examines the reasons underlying (potential) differences in adults' health-state preferences for themselves, a 10-year-old child, and 15-year-old adolescent.

METHODS

We conducted semistructured interviews using a think-aloud protocol with 25 participants who performed valuation tasks in July 2020 to August 2020. Using the Framework Method, we developed 28 categories grouped under 5 themes that illustrated the differences emerging from the interviews.

RESULTS

Participants (A) deemed it more straining to perform valuation tasks for a 10-year-old child and 15-year-old adolescent than for themselves, (B) had a stable self-image, but varied in whom they imagined as 10-year-old child and 15-year-old adolescent, (C) focused on different dimensions and levels for a 15-year-old adolescent than for a 10-year-old child and themselves, (D) had various thoughts about nonhealth-related factors that influenced their preferences, and (E) gave up relatively few life-years for a 10-year-old child and 15-year-old adolescent, also to avoid others bearing a grudge against them.

CONCLUSIONS

Our results indicate that differences in adults' health-state preferences for themselves and a 10-year-old child largely result from differences in thoughts about nonhealth-related factors. They further indicate that health-state preferences for a 10-year-old child may not be representative of such preferences for the full EQ-5D-Y-3L age range.

A web-based comprehensive head and neck cancer patient education and support needs program: Usability testing

Background: With increasing accessibility of the World Wide Web, patients are using it to obtain patient education materials (PEM). With this in mind, our group (surgeons, radiation oncologists, medical oncologists, nursing, allied health professionals and academic researchers) developed a comprehensive information resource for patients with head and neck cancer (HNC), the Head and Neck Cancer Australia, formerly Beyond Five website. The aim of this study was to determine patient usability of the Head and Neck Cancer Australia website. Methods: Usability testing (Cognitive walkthrough & Think-Aloud) in 18 patients treated for HNC was undertaken at a Cancer centre (anonymous for reviewers). Results: The tasks rated easiest by patients were finding information on psychological well-being (mean time spent: 66 s, range: 10-565), health and well-being effecting quality of life (mean time spent: 36 s, range 9-117) and carer information (mean time spent: 10 s, range 3-35). Patients indicated the website contained a lot of information, covers most topics, was a trusted source of information and a springboard to other information. Conclusions: The Head and Neck Cancer Australia website provides a wide range of information and support in multiple formats available to HNC patients. Further refinements in design, navigation and website instructions are needed to allow effective patient interaction.

Temporal Assessment of Self-Regulated Learning by Mining Students' Think-Aloud Protocols

It has been widely theorized and empirically proven that self-regulated learning (SRL) is related to more desired learning outcomes, e.g., higher performance in transfer tests. Research has shifted to understanding the role of SRL during learning, such as the strategies and learning activities, learners employ and engage in the different SRL phases, which contribute to learning achievement. From a methodological perspective, measuring SRL using think-aloud data has been shown to be more insightful than self-report surveys as it helps better in determining the link between SRL activities and learning achievements. Educational process mining on the basis of think-aloud data enables a deeper understanding and more fine-grained analyses of SRL processes. Although students’ SRL is highly contextualized, there are consistent findings of the link between SRL activities and learning outcomes pointing to some consistency of the processes that support learning. However, past studies have utilized differing approaches which make generalization of findings between studies investigating the unfolding of SRL processes during learning a challenge. In the present study with 29 university students, we measured SRL via concurrent think-aloud protocols in a pre-post design using a similar approach from a previous study in an online learning environment during a 45-min learning session, where students learned about three topics and wrote an essay. Results revealed significant learning gain and replication of links between SRL activities and transfer performance, similar to past research. Additionally, temporal structures of successful and less successful students indicated meaningful differences associated with both theoretical assumptions and past research findings. In conclusion, extending prior research by exploring SRL patterns in an online learning setting provides insights to the replicability of previous findings from online learning settings and new findings show that it is important not only to focus on the repertoire of SRL strategies but also on how and when they are used.

Content Validation of Clinician-Reported Items for a Severity Measure for CDKL5 Deficiency Disorder

CDKL5 deficiency disorder (CDD) results in early-onset seizures and severe developmental impairments. A CDD clinical severity assessment (CCSA) was previously developed with clinician and parent-report items to capture information on a range of domains. Consistent with US Food and Drug Administration (FDA) guidelines, content validation is the first step in evaluating the psychometric properties of an outcome measure. The aim of this study was to validate the content of the clinician-reported items in the CCSA (CCSA-Clinician). Eight neurologists leading the USA CDD Center of Excellence clinics were interviewed using the "think aloud" technique to critique 26 clinician-reported items. Common themes were aggregated, and a literature search of related assessments informed item modifications. The clinicians then participated in 2 consensus meetings to review themes and finalize the items. A consensus was achieved for the content of the CCSA-Clinician. Eight of the original items were omitted, 11 items were added, and the remaining 18 items were revised. The final 29 items were classified into 2 domains: functioning and neurologic impairments. This study enabled refinement of the CCSA-Clinician and provided evidence for its content validity. This preliminary validation is essential before field testing and further validation, in order to advance the instrument toward clinical trial readiness.

Development of and Experiences With an Informational Website on Early Labor: Qualitative User Involvement Study

Background

The period of regular contractions before 4 cm of cervical dilatation is often referred to as the latent phase or early labor. Women find it challenging to prepare for and cope with this phase of labor, and easily accessed web-based information from reliable sources may be useful in this preparation.

Objective

The aim of this study is to describe the development of a Norwegian website, Latens.no, for people seeking information on early labor and to explore users’ experiences with the website to increase its user-friendliness.

Methods

We developed a website using an iterative process involving a multidisciplinary research team, health personnel, users, a graphic designer, and an expert in software development. We explored the website’s user-friendliness using semistructured individual interviews and the think-aloud method. All interviews were audio recorded and transcribed. We then analyzed the participants’ feedback on the website.

Results

Participants included women who had recently given birth to their first baby (n=2), women who were pregnant with their first baby (n=4), and their partners (n=2). Results from participants’ experiences completing tasks included positive feedback related to the content of Latens.no, positive feedback related to the website’s design, and suggestions for improvement. Participants wanted to find information on early labor on the internet. Moreover, they found the information on the website relevant, trustworthy, and easy to read, and the design was attractive and easy to use. Overall, the participants performed the tasks easily, with few clicks and minimal effort.

Conclusions

The think-aloud method, while performing tasks, allowed for detailed feedback. The participants confirmed the user-friendliness of the website but at the same time provided information enabling improvement. We expect that changes made based on this user-centered design study will further increase the usability and acceptability of Latens.no.

Mobile Health for Smoking Cessation Among Disadvantaged Young Women During and After Pregnancy: User-Centered Design and Usability Study

Background

Smoking prevalence during and after pregnancy remains high among socioeconomically disadvantaged women. Mobile health (mHealth) apps with game and social support elements seem promising to support smoking cessation.

Objective

This study aims to describe the user-centered design and usability evaluation of Kindle, an mHealth app with game and social support elements, to support disadvantaged young women during and after pregnancy through the first stages of smoking cessation.

Methods

Disadvantaged women (n=9), members of their social networks (n=4), and nurses supporting these women (n=51) were informants throughout the iterative prototype development of Kindle according to the International Organization for Standardization 9241-11:2018. Specific phases included understanding the context of use through secondary analysis of qualitative interview data (phase 1), establishing the user and organizational requirements (phase 2), production of design solutions (phase 3), and usability inspection of the prototype through a heuristic evaluation (3 experts) along with user testing by a think aloud method (5 disadvantaged women and 5 nurses; phase 4). Usability problems were categorized according to the principles of the Healthcare Information and Management Systems Society.

Results

Phase 1 resulted in an understanding of the VoorZorg program and the needs of VoorZorg nurses and clients (eg, focus on early stages of change and building new supportive networks to aid clients in smoking cessation). In phase 2, we established requirements (n=22; eg, mHealth app, secure communication between nurses and clients, easy-to-use interfaces, inclusion of game elements, and tailoring at early stages of change in smoking cessation). Phase 3 resulted in a prototype of Kindle, combining the interface for nurses and clients, including the following functionalities: personal goal setting with earning points; secured chat function between nurses and other clients; and tips, diary, and profile creation. The heuristic evaluation and thinking aloud method in phase 4 revealed 78 usability problems in the interfaces. Most usability problems concerned simplicity (eg, unclear clickable button) and naturalness (eg, unclear icon).

Conclusions

The user-centered design and usability testing of the mHealth app Kindle yielded useful insights. The involvement of end users, specifically socioeconomically disadvantaged women during and after their pregnancy, resulted in a prototype that met their needs and requirements (eg, mHealth app, secure communication between nurses and clients, easy-to-use interfaces, inclusion of game elements, and tailoring to the early stages of change in smoking cessation) to achieve readiness for smoking cessation. Moreover, the usability evaluation by end users and experts revealed unique usability problems for this population. These insights allow for further optimization of Kindle and encourage future studies to engage disadvantaged populations in all phases of mHealth intervention design and usability testing.

Evaluation of Response Processes to the Danish Version of the Dutch Multifactor Fatigue Scale in Stroke Using the Three-Step Test-Interview

Validated self-report measures of post-stroke fatigue are lacking. The Dutch Multifactor Fatigue Scale (DMFS) was translated into Danish, and response process evidence of validity was evaluated. DMFS consists of 38 Likert-rated items distributed on five subscales: Impact of fatigue (11 items), Signs and direct consequences of fatigue (9), Mental fatigue (7), Physical fatigue (6), and Coping with fatigue (5). Response processes to DMFS were investigated using a Three-Step Test-Interview (TSTI) protocol, and data were analyzed using Framework Analysis. Response processes were indexed on the following categories: (i) "congruent," response processes were related to the subscale construct; (ii) "incongruent," response processes were not related to the subscale construct; (iii) "ambiguous," response processes were both congruent and incongruent or insufficient to evaluate congruency; and (iv) "confused," participants did not understand the item. Nine adults were recruited consecutively 10-34 months post-stroke (median = 26.5) at an outpatient brain injury rehabilitation center in 2019 [five females, mean age = 55 years (SD = 6.3)]. Problematic items were defined as <50% of response processes being congruent with the intended construct. Of the 38 items, five problematic items were identified, including four items of Physical fatigue and one of Mental fatigue. In addition, seven items posed various response difficulties to some participants due to syntactic complexity, vague terms, a presupposition, and a double-barrelled statement. In conclusion, findings elucidate the interpretative processes involved in responding to DMFS post-stroke, strengthen the evidence base of validity, and guide revisions to mitigate potential problems in item performance.

Influences on the Uptake of Health and Well-being Apps and Curated App Portals: Think-Aloud and Interview Study

BACKGROUND

Health and well-being smartphone apps can provide a cost-effective solution to addressing unhealthy behaviors. The selection of these apps tends to occur in commercial app stores, where thousands of health apps are available. Their uptake is often influenced by popularity indicators. However, these indicators are not necessarily associated with app effectiveness or evidence-based content. Alternative routes to app selection are increasingly available, such as via curated app portals, but little is known about people's experiences of them.

OBJECTIVE

The aim of this study is to explore how people select health apps on the internet and their views on curated app portals.

METHODS

A total of 18 UK-based adults were recruited through social media and asked during an in-person meeting to verbalize their thoughts while searching for a health or well-being app on the internet on a platform of their choice. The search was then repeated on 2 curated health app portals: the National Health Service Apps Library and the Public Health England One You App portal. This was followed by semistructured interviews. Data were analyzed using framework analysis, informed by the Capability, Opportunity, Motivation-Behavior model and the Theoretical Domains Framework.

RESULTS

Searching for health and well-being apps on the internet was described as a minefield. App uptake appeared to be influenced by participants' capabilities such as app literacy skills and health and app awareness, and opportunities including the availability of apps, app esthetics, the price of an app, and social influences. Motivation factors that seemed to affect the uptake were perceived competence, time efficiency, perceived utility and accuracy of an app, transparency about data protection, commitment and social identity, and a wide range of emotions. Social influences and the perceived utility of an app were highlighted as particularly important. Participants were not previously aware of curated portals but found the concept appealing. Curated health app portals appeared to engender trust and alleviate data protection concerns. Although apps listed on these were perceived as more trustworthy, their presentation was considered disappointing. This disappointment seemed to stem from the functionality of the portals, lack of user guidance, and lack of tailored content to an individual's needs.

CONCLUSIONS

The uptake of health and well-being apps appears to be primarily affected by social influences and the perceived utility of an app. App uptake via curated health app portals perceived as credible may mitigate concerns related to data protection and accuracy, but their implementation must better meet user needs and expectations.

Pediatric Hospitalists' Performance and Perceptions of Script Concordance Testing for Self-Assessment

OBJECTIVES

The cognitive expertise of Pediatric Hospitalists (PH) lies not in standard knowledge but in making decisions under conditions of uncertainty. To maintain expertise, PH should engage in deliberate practice via self-assessments that promote higher-level cognitive processes necessary to address problems with missing or ambiguous information. Higher levels of cognition are purported with Script Concordance Test (SCT) questions compared to Multiple Choice Questions (MCQ). To determine if PH use higher levels of cognition when answering SCT versus MCQ questions and to analyze participants' perceptions of the utility of using SCT self-assessment for deliberate practice in addressing clinical problems encountered in daily practice.

METHODS

This is a mixed methods study comparing the cognitive level expressed according to Bloom's Taxonomy by PH answering MCQ versus SCT questions using a "think aloud" (TA) exercise, followed by qualitative analysis of interviews conducted afterward.

RESULTS

A significantly greater percentage of comments were coded as higher cognitive processes (apply, analyze, evaluate, and create) for SCT versus MCQ (74% vs 19%) compared with lower order (remember, understand); chi-square P < .00001. Analysis of interviews revealed 6 themes.

CONCLUSION

SCT questions elicited higher level cognition essential to clinical reasoning compared to MCQ questions. PH-indicated MCQ questions measure standard knowledge, while SCT questions better measure decision-making under conditions of uncertainty. PH-perceived SCT could be useful for deliberate practice in Pediatric Hospital Medicine decision-making if they could compare their rationale in answering questions with that of experts.

Exploring External Factors Affecting the Intention-Behavior Gap When Trying to Adopt a Sustainable Diet: A Think Aloud Study

Not least from an ecological and health perspective, it can be posited that a broader part of consumers should practice sustainable diets. People who are already willing to do so are often confronted with the intention-behavior gap, caused by a range of internal and external factors. To eliminate these barriers requires a deeper and more comprehensive understanding of these factors and their interplay. Therefore, a think aloud study with 20 adult German participants was conducted to explore the four chosen external factors of availability, education, advertising and price. Furthermore, questionnaires for all four factors were handed out and a follow-up interview was conducted to gain additional qualitative data. Results show that these four external factors seem to have a major impact on the intention-behavior relation. According to the participants all factors interact in some way with other internal and external factors, making practicing sustainable diets a complex activity. In conclusion, the four external factors availability, education, advertising and price need to be addressed by various stakeholders within our food systems in order to move forward in the process of making sustainable diets practicable and sustainable food systems firmly established.

Exploring views on alcohol consumption and digital support for alcohol reduction in UK-based Punjabi-Sikh men: A think aloud and interview study

INTRODUCTION AND AIMS

We aimed to explore UK-based Punjabi-Sikh men's views on: (i) alcohol consumption within the community; (ii) available support for alcohol reduction; and (iii) an evidence-informed alcohol reduction app.

DESIGN AND METHODS

Semi-structured interviews and a think aloud method were employed. Participants (n = 15) were male, aged 18-27 years, identified as Punjabi-Sikh, were hazardous or harmful drinkers (i.e. had an Alcohol Use Disorders Identification Test-Consumption score of ≥5) and interested in using an app to reduce drinking. Interviews were audio-recorded, transcribed verbatim and analysed with inductive thematic analysis.

RESULTS

Six themes were developed: (i) fear of drinking to cope; (ii) clash between religious and cultural norms (i.e. an internal conflict between important values); (iii) stigmatisation of mental health issues and lack of knowledge as barriers to help seeking; (iv) perceived usefulness of goal setting, monitoring and feedback (i.e. beliefs about the utility of the app's components for reducing drinking); (v) concerns about accessibility of the app within the Punjabi-Sikh community; and (vi) desire for human support for continued app engagement.

DISCUSSION AND CONCLUSIONS

Among UK-based, Punjabi-Sikh men, clashing religious and cultural norms give rise to internal conflict about drinking. Stigmatisation of mental health issues and lack of knowledge of available support leads to reduced help seeking. Respondents believed an evidence-informed alcohol reduction app could be useful, but were concerned about accessibility within the wider community and wanted an element of human support. The potential for a combination of digital and face-to-face support should be explored.

Using a think-aloud methodology to understand online physical activity information search experiences and preferences of parents of children and youth with disabilities

BACKGROUND

The Internet is a preferred source of physical activity (PA) information. However, limited research exists regarding the experiences of parents of children and youth with disabilities (CYWD) in searching for PA programme information online. This research examined the experiences and preferences of parents of CYWD in searching for PA programme information online.

METHOD

Parents of CYWD (n = 10) participated in a think-aloud exercise while searching for PA programme information online. Following the think-aloud exercise, semi-structured interviews were used to further understand parents' experiences and preferences in searching for PA programme information online.

RESULTS

Parents identified key features that contributed to a positive online search experience. Additionally, parents noted challenges and resulting negative affect that was experienced.

CONCLUSIONS

This research can inform the development and dissemination of online PA programme information that is accessible and relevant to the preferences of parents of CYWD and can facilitate positive search experiences.

An assessment of patient perspectives on pharmacogenomics educational materials

Aim: Pharmacogenomics (PGx) holds potential to improve patient treatment; yet, effective patient educational materials are limited. Materials & methods: Using a 'think aloud' technique, we sought to understand comprehension and perceptions of a multimedia PGx results packet including a cover letter with QR code to an educational video, brochure and prototype report in the context of PGx case vignettes. Results: The cover letter and video components were viewed less favorably due to excess detail, complex jargon and technology challenges. Recommendations were to enhance comprehension and utility and to customize materials to each patient's medications or disease conditions. Conclusion: Educational materials were revised to improve comprehension and usability, and diminish concerns to better prepare patients to understand the importance of discussing test results with their provider.

Usability testing of bed information management system: A think-aloud method

Not considering the usability in designing clinical information systems causes problems in human–computer interaction and patient dissatisfaction. Therefore, in this study, the usability of the bed information management system (BIMS) was examined by think-aloud method. This cross-sectional study was conducted on the BIMS in 50 noneducational hospitals. Participants consisted of three groups including users, facilitators, and technical support. To carry out the study, a scenario consisting of four tasks was designed. Three researchers analyzed the recorded files to identify the usability problems and their severity. The mean time of the evaluation process was 20:33 ± 4:47 s. The total number of the problems identifies by users was 80 cases. Data entry and layout problems with 38 (48%) and 33 (41%) cases were the most frequently found problems, respectively. About 61% and 55% of the data entry and layout problems had a minor severity (Severity 2), respectively. Furthermore, 43 (54%) cases of the problems were resolved by the users and 32 (40%) cases by the facilitator assistance. This study showed that a large number of the problems were due to the system poor design. Furthermore, by increasing the users’ level of knowledge about the system, it is possible to enhance user-system interaction. It is recommended that before designing and implementing a system, the system should be evaluated for usability, and the users should be educated in clinical information systems.

The development of a self-help intervention to build social confidence in people living with visible skin conditions or scars: a think-aloud study

INTRODUCTION

People with a visible difference, such as scarring or a skin condition, can experience anxiety and intrusive reactions from others when in social situations. The use of products to conceal marks on the skin is provided in a number of different hospital services and by charities. However, there are relatively few psychosocial interventions available for these individuals.

OBJECTIVES

To examine the views of skin camouflage users and practitioners on the acceptability, usability and need for a specifically developed cognitive behavioural therapy (CBT) self-help booklet.

METHODS

A think-aloud protocol and descriptive form of thematic analysis were used to ascertain participants' views of this novel psychosocial intervention. Nine participants took part in think-aloud interviews that were analysed using thematic analysis. Six skin camouflage users and three skin camouflage practitioners participated in the study.

RESULTS

Support for the relevance, acceptability and usability of the booklet was found from both participants who used camouflage and those who provided it. However, some participants reported that they would envisage that some people would need additional support so as to be able to use the techniques described within the booklet.

CONCLUSIONS

This study represents an important step towards developing a brief self-help intervention for people with living with visible skin conditions or scars and demonstrates the importance of seeking feedback from experts by experience on theoretically informed psychological interventions for this patient group.

Usability testing of an electronic pain monitoring system for palliative cancer patients: A think-aloud study

Pain experienced by advanced cancer patients is often poorly controlled due to inadequate assessment. We aimed to test an electronic pain monitoring system (PainCheck) with advanced cancer patients and health professionals. In all, 29 participants (13 patients and 16 health professionals (n = 16)) used PainCheck while thinking their thoughts aloud. After the think aloud, both groups were asked about their experiences. Think-aloud and semi-structured interview data were analysed using framework analysis. Both groups were generally positive about PainCheck and found it easy to understand but they had concerns about clinical integration. Their concerns related to impact on workload, how lack of response may affect patient care and ability to engage with and use the technology. PainCheck has the capacity to be useful in clinical practice. For integration to be successful, patients and health professionals need guidance on how to use PainCheck and the expectations of users need to be clarified.

The nursing assessment of infants with bronchiolitis

Bronchiolitis is the leading cause of morbidity and hospitalization in infants under the age of one year. Supportive treatments and regular assessment remain the mainstay of care for infants admitted to hospital. Nurses play an important role in the assessment of infants with bronchiolitis; however, this is not well described in the literature and consequently little is known about what strategies nurses employ in assessing infants with bronchiolitis. The aim of this study was to explore bronchiolitis assessment in the context of nursing practice. A naturalistic inquiry study was undertaken using think aloud and retrospective probing data collection methods. The results revealed that the information gathered by nurses in their assessment of infants with bronchiolitis was varied and the process of acquiring and evaluating this information was multifaceted and holistic in nature. A close partnership between the nurse and mother was identified, and the mother's expert knowledge and ability to identify subtle changes in the infant's clinical condition over time was essential to the assessment process. The assessment partnership with families provides nurses with the most comprehensive and holistic view of the infant's clinical condition and vital assessment information could be lost if this partnership does not occur.

Factors Influencing Usability of a Smartphone App to Reduce Excessive Alcohol Consumption: Think Aloud and Interview Studies

BACKGROUND

Interventions delivered by smartphone apps have the potential to help drinkers reduce their consumption of alcohol. To optimize engagement and reduce the high rates of attrition associated with the use of digital interventions, it is necessary to ensure that an app's design and functionality is appropriate for its intended purposes and target population.

AIMS

To understand the usability of an app to help people reduce their alcohol consumption.

METHOD

The app, Drink Less, contains a core module focusing on goal setting, supplemented by five additional modules: self-monitoring and feedback, identity change, cognitive bias re-training, action planning, and social comparison. Two studies were conducted, a "think aloud" study performed with people using the app for the first time and a semistructured interview study performed after users had had access to the app for at least 2 weeks. A thematic analysis of the "think aloud" and interview transcripts was conducted by one coder and verified by a second.

RESULTS

Twenty-four participants, half of whom were women and half from disadvantaged groups, took part in the two studies. Three main themes identified in the data were "Feeling lost and unsure of what to do next," "Make the app easy to use," and "Make the app beneficial and rewarding to use." These themes reflected participants' need for (i) guidance, particularly when first using the app or when entering data; (ii) the data entry process to be simple and the navigation intuitive; (iii) neither the amount of text nor range of options to be overwhelming; (iv) the app to reward them for effort and progress; and (v) it to be clear how the app could help alcohol reduction goals be reached.

CONCLUSION

First-time and experienced users want an alcohol reduction app to be easy, rewarding, and beneficial to use. An easy-to-use app would reduce user burden, offer ongoing help, and be esthetically pleasing. A rewarding and beneficial app would provide positive reinforcement, give feedback about progress, and demonstrate credibility. Users need help when first using the app, and they need a compelling reason to continue using it.

Justify Your Answer: The Role of Written Think Aloud in Script Concordance Testing

Construct: Clinical reasoning assessment is a growing area of interest in the medical education literature. Script concordance testing (SCT) evaluates clinical reasoning in conditions of uncertainty and has emerged as an innovative tool in the domain of clinical reasoning assessment. SCT quantifies the degree of concordance between a learner and an experienced clinician and attempts to capture the breadth of responses of expert clinicians, acknowledging the significant yet acceptable variation in practice under situations of uncertainty.

BACKGROUND

SCT has been shown to be a valid and reliable clinical reasoning assessment tool. However, as SCT provides only quantitative information, it may not provide a complete assessment of clinical reasoning.

APPROACH

Think aloud (TA) is a qualitative research tool used in clinical reasoning assessment in which learners verbalize their thought process around an assigned task. This study explores the use of TA, in the form of written reflection, in SCT to assess resident clinical reasoning, hypothesizing that the information obtained from the written TA would enrich the quantitative data obtained through SCT. Ninety-one pediatric postgraduate trainees and 21 pediatricians from 4 Canadian training centers completed an online test consisting of 24 SCT cases immediately followed by retrospective written TA. Six of 24 cases were selected to gather TA data. These cases were chosen to allow all phases of clinical decision making (diagnosis, investigation, and treatment) to be represented in the TA data. Inductive thematic analysis was employed when systematically reviewing TA responses.

RESULTS

Three main benefits of adding written TA to SCT were identified: (a) uncovering instances of incorrect clinical reasoning despite a correct SCT response, (b) revealing sound clinical reasoning in the context of a suboptimal SCT response, and (c) detecting question misinterpretation.

CONCLUSIONS

Written TA can optimize SCT by demonstrating when correct examinee responses are based on guessing or uncertainty rather than robust clinical rationale. TA can also enhance SCT by allowing examinees to provide justification for responses that otherwise would have been considered incorrect and by identifying questions that are frequently misinterpreted to avoid including them in future examinations. TA also has significant value in differentiating between acceptable variations in expert clinician responses and deviance associated with faulty rationale or question misinterpretation; this could improve SCT reliability. A written TA protocol appears to be a valuable tool to assess trainees' clinical reasoning and can strengthen the quantitative assessment provided by SCT.

Content Validity of the Credibility and Expectancy Questionnaire in a Pain Rehabilitation Setting

BACKGROUND

Content validity, the proper reflection of the concept to be measured, is yet unknown for the Credibility and Expectancy Questionnaire (CEQ). It is frequently used in pain rehabilitation because treatment expectancy is influential on the outcome.

OBJECTIVE

To test and improve the content validity of the CEQ in a sample of patients with chronic pain in different phases of their treatment.

METHODS

A qualitative observational study design using the Three-Step Test-Interview method was used. Therein, data collection, analyses, and adaptations occur iteratively.

RESULTS

Seventeen patients with chronic pain in different stages of treatment participated through convenience sampling from the mother sample of a randomized controlled trial. The main study parameter is content validity, which is defined as (1) interpretations and responses of the participants and (2) the identification of response problems operationalized, and resulting in changes in the CEQ. For patients waiting for treatment, the written instruction of the CEQ allowed different interpretations. After changing the instructions, the CEQ became an easy-to-understand and content-valid questionnaire. For patients who had already undergone treatment, changes regarding time frame and recall period were necessary to overcome interpretation and response problems to the CEQ.

DISCUSSION

After small changes, the CEQ appeared to be a content-valid measurement instrument for patients waiting for treatment. However, for patients who had already undergone treatment, the content validity of the CEQ was less, and considerable changes were necessary.

Using a novel assessment of procedural proficiency provides medical educators insight into blood pressure measurement

OBJECTIVE

This investigation was performed to determine how students in a health sciences program utilize and explain techniques within blood pressure measurement using a novel assessment, and changes associated with greater curricular exposure.

METHODS

An exploratory, qualitative and quantitative study was conducted using a 'Think Aloud' design with protocol analysis. Following familiarization, participants performed the task of measuring blood pressure on a reference subject while stating their thought processes. A trained practitioner recorded each participant's procedural proficiency using a standardized rubric. There were 112 participants in the study with varying levels of curricular exposure to blood pressure measurement.

RESULTS

Four trends are noted. Specifically, a trend was observed wherein a marked increase in procedural proficiency with a plateau occurred (e.g. released cuff pressure 2-4 mmHg, 10%, 60%, 83%, 82%). Secondly, a trend was observed with improvement across groups (e.g. cuff placed snugly/smoothly on upper arm, 20%, 60%, 81%, and 91%). Other trends included a marked improvement with subsequent decrease, and an improvement without achieving proficiency (e.g. palpation of the brachial pulse, 5%, 90%, 81%, 68%, appropriate size cuff, 17%, 40%, 33%, 41%, respectively). Qualitatively, transcript interpretation resulted in a need for clarification in the way blood pressure procedure is instructed in the curriculum.

CONCLUSIONS

The current investigation provides a snapshot of proficiency in blood pressure assessment across a curriculum and highlights considerations for best instructional practices, including the use of Think Aloud. Consequently, medical educators should use qualitative and quantitative assessments concurrently to determine achievement of blood pressure skill proficiency.

Older Cancer Patients' User Experiences With Web-Based Health Information Tools: A Think-Aloud Study

Background

Health information is increasingly presented on the Internet. Several Web design guidelines for older Web users have been proposed; however, these guidelines are often not applied in website development. Furthermore, although we know that older individuals use the Internet to search for health information, we lack knowledge on how they use and evaluate Web-based health information.

Objective

This study evaluates user experiences with existing Web-based health information tools among older (≥ 65 years) cancer patients and survivors and their partners. The aim was to gain insight into usability issues and the perceived usefulness of cancer-related Web-based health information tools.

Methods

We conducted video-recorded think-aloud observations for 7 Web-based health information tools, specifically 3 websites providing cancer-related information, 3 Web-based question prompt lists (QPLs), and 1 values clarification tool, with colorectal cancer patients or survivors (n=15) and their partners (n=8) (median age: 73; interquartile range 70-79). Participants were asked to think aloud while performing search, evaluation, and application tasks using the Web-based health information tools.

Results

Overall, participants perceived Web-based health information tools as highly useful and indicated a willingness to use such tools. However, they experienced problems in terms of usability and perceived usefulness due to difficulties in using navigational elements, shortcomings in the layout, a lack of instructions on how to use the tools, difficulties with comprehensibility, and a large amount of variety in terms of the preferred amount of information. Although participants frequently commented that it was easy for them to find requested information, we observed that the large majority of the participants were not able to find it.

Conclusions

Overall, older cancer patients appreciate and are able to use cancer information websites. However, this study shows the importance of maintaining awareness of age-related problems such as cognitive and functional decline and navigation difficulties with this target group in mind. The results of this study can be used to design usable and useful Web-based health information tools for older (cancer) patients.

'The ICECAP-SCM tells you more about what I'm going through': A think-aloud study measuring quality of life among patients receiving supportive and palliative care

BACKGROUND

The ICECAP-Supportive Care Measure is a self-complete questionnaire developed to aid economic evaluation of supportive care interventions.

AIM

To determine the feasibility of completing ICECAP-Supportive Care Measure alongside EQ-5D-5L and ICECAP-A (generic measures used in economic evaluation) among patients receiving hospice care, close persons and healthcare professionals.

DESIGN

Participants were asked to 'think aloud' while completing ICECAP-Supportive Care Measure and two other generic measures used in economic evaluation, EQ-5D-5L and ICECAP-A, and then participate in a semi-structured interview. From verbatim transcripts, five raters identified the frequency of errors in comprehension, retrieval, judgement and response. Qualitative data were analysed using constant comparison.

SETTING/PARTICIPANTS

Eligible patients were identified from one UK hospice by a research nurse. Close persons and healthcare professionals were identified by the patient. In all, 72 semi-structured interviews were conducted with patients (n = 33), close persons (n = 22) and healthcare professionals (n = 17).

RESULTS

Patients and close persons reported that the ICECAP-Supportive Care Measure was most appropriate for measuring their quality of life. It appeared more meaningful, easier to complete and had fewest errors (3.9% among patients, 4.5% among close persons) compared to EQ-5D-5L (9.7% among patients, 5.5% among close persons). Healthcare professionals acknowledged the value of the ICECAP-Supportive Care Measure but had fewer errors in completing the EQ-5D-5L (3.5% versus 6.7%). They found it easier to complete because it focuses on observable health states.

CONCLUSIONS

The ICECAP-Supportive Care Measure is feasible to use and perceived as appropriate for evaluating palliative care interventions. Healthcare professionals with limited knowledge of the patient who act as proxy completers may find the measure difficult to complete.

An analysis of nursing students' decision-making in teams during simulations of acute patient deterioration

AIM

The aim of this study was to examine the decision-making of nursing students during team based simulations on patient deterioration to determine the sources of information, the types of decisions made and the influences underpinning their decisions.

BACKGROUND

Missed, misinterpreted or mismanaged physiological signs of deterioration in hospitalized patients lead to costly serious adverse events. Not surprisingly, an increased focus on clinical education and graduate nurse work readiness has resulted.

DESIGN

A descriptive exploratory design.

METHODS

Clinical simulation laboratories in three Australian universities were used to run team based simulations with a patient actor. A convenience sample of 97 final-year nursing students completed simulations, with three students forming a team. Four teams from each university were randomly selected for detailed analysis. Cued recall during video review of team based simulation exercises to elicit descriptions of individual and team based decision-making and reflections on performance were audio-recorded post simulation (2012) and transcribed.

RESULTS

Students recalled 11 types of decisions, including: information seeking; patient assessment; diagnostic; intervention/treatment; evaluation; escalation; prediction; planning; collaboration; communication and reflective. Patient distress, uncertainty and a lack of knowledge were frequently recalled influences on decisions.

CONCLUSIONS

Incomplete information, premature diagnosis and a failure to consider alternatives when caring for patients is likely to lead to poor quality decisions. All health professionals have a responsibility in recognizing and responding to clinical deterioration within their scope of practice. A typology of nursing students' decision-making in teams, in this context, highlights the importance of individual knowledge, leadership and communication.

Feasibility of the Prototype Willingness Model as the basis for school-delivered alcohol misuse prevention: A qualitative think-aloud study to explore acceptability of 'The Alcohol Smart Quiz' with adolescents and teachers

This study sought feedback from teachers and adolescents on a novel intervention to reduce alcohol misuse based on the Prototype Willingness Model. A qualitative, think-aloud interview study was conducted with 17 adolescents aged 11-14 years and nine teachers. Transcripts were analysed using thematic analysis. Adolescents found the content of the intervention appealing and credible. Teachers welcomed the content and format as a means of generating discussion. 'Moderate' drinking appeared to be an acceptable message, although difficult to define. Enacting 'if-then' plans in real social situations might be challenging and representations of drinking on social media should be considered.

Developing best practices teaching procedures for skinfold assessment: observational examination using the Think Aloud method

Skinfold assessment is valid and economical; however, it has a steep learning curve, and many programs only include one exposure to the technique. Increasing the number of exposures to skinfold assessment within an undergraduate curriculum would likely increase skill proficiency. The present study combined observational and Think Aloud methodologies to quantify procedural and cognitive characteristics of skinfold assessment. It was hypothesized that 1) increased curricular exposure to skinfold assessment would improve proficiency and 2) the combination of an observational and Think Aloud analysis would provide quantifiable areas of emphasis for instructing skinfold assessment. Seventy-five undergraduates with varied curricular exposure performed a seven-site skinfold assessment on a test subject while expressing their thoughts aloud. A trained practitioner recorded procedural observations, with transcripts generated from audio recordings to capture cognitive information. Skinfold measurements were compared with a criterion value, and bias scores were generated. Participants whose total bias fell within ±3.5% of the criterion value were proficient, with the remainder nonproficient. An independent-samples t-test was used to compare procedural and cognitive observations across experience and proficiency groups. Additional curricular exposure improved performance of skinfold assessment in areas such as the measurement of specific sites (e.g., chest, abdomen, and thigh) and procedural (e.g., landmark identification) and cognitive skills (e.g., complete site explanation). Furthermore, the Think Aloud method is a valuable tool for determining curricular strengths and weaknesses with skinfold assessment and as a pedagogical tool for individual instruction and feedback in the classroom.

Development and Acceptability of a Co-Produced Online Intervention to Prevent Alcohol Misuse in Adolescents: A Think Aloud Study

Background

The prototype willingness model (PWM) may offer an appropriate basis for explaining and preventing adolescent alcohol misuse. An intervention was developed using a co-production approach, and consisted of an online quiz featuring 10 questions linked to the PWM.

Objective

This study sought to determine the acceptability and relevance of the intervention content to young people, to incorporate their feedback into a final version.

Methods

A qualitative think aloud study with follow-up semistructured interviews was undertaken with 16 young people aged 11-15 (50%). Transcripts were analyzed using thematic analysis.

Results

The following 3 main themes relating the acceptability of the intervention were identified: “challenging expectations of alcohol education”; “motivations for drinking or not drinking,” and “the inevitability of drinking.” Participants found the intervention appealing because it was counter to their expectations. The content appeared to reflect their experiences of social pressure and drinking encounters. There was evidence that a focus on drinker/nondrinker prototypes was too narrow and that because adolescents perceived drinking as inevitable, it would be challenging to enact any plans to resist pressure to drink.

Conclusions

An online intervention based on the PWM has the potential to engage and interest adolescents. A wide range of alcohol prototypes should be targeted and a focus on short-term harms should ensure that the intervention is credible to young people.

Measuring wheelchair confidence among power wheelchair users: an adaptation of the WheelCon-M using focus groups and a think aloud process

PURPOSE

The aim of this study was to adapt the WheelCon-M, an outcome measure used to assess confidence with manual wheelchair use, into a version that is applicable for power wheelchair users (WheelCon-P).

METHODS

Adaptation of the WheelCon-M into the WheelCon-P occurred in two phases: (1) item modification was conducted using focus groups and (2) item refinement was done using a think aloud process.

RESULTS

The healthcare professionals (n = 12) were mostly female, mostly occupational therapists and practiced an average of 14 years. The power wheelchair users (n = 8) were 50% men with an average of 8 years of power wheelchair experience and a range of diagnoses. In phase 1, of the 63 WheelCon-M items, 6 remained the same, 18 were removed, 38 modified and 15 new items added to develop the WheelCon-P. In phase 2, 15 WheelCon-P items were refined.

CONCLUSIONS

This study resulted in the modification of the WheelCon-M into the WheelCon-P, a 59-item, self-report outcome measure designed to assess confidence with power wheelchair use. The next step in the evolution of this promising new measure is the assessment of its measurement properties. Implications for Rehabilitation The WheelCon-P is a new outcome measure designed to assess confidence with power wheelchair use. This study suggests that the items in this measure will be interpreted as intended.

Valuation of depression co-occurring with a somatic condition: feasibility of the time trade-off task

BACKGROUND

Health state valuations obtained from the general population are used for cost-utility analyses of health-care interventions. Currently, most studies have focused on valuations of somatic conditions, to a much lesser extent of mental states, that is, depression and even less on valuations of depression co-occurring with somatic conditions.

OBJECTIVE

We tested the feasibility of the time trade-off (TTO) task to elicit valuations for depression solitary or co-occurring with a somatic condition. Moreover, we explored person- and state-related factors that may affect valuations.

DESIGN

During semi-structured interviews, 10 individuals (five women, mean age: 36 years) used a TTO task to value vignettes describing mild and severe depression; and mild depression co-occurring with moderate and severe states of cancer, diabetes or heart disease. During valuations, participants were thinking aloud. Feasibility criteria were successful completion and difficulty/concentration (1-10); logical consistency of values; and comprehension of the TTO, based on qualitative analysis of think aloud data. Factors influencing valuations were generated from think aloud data.

RESULTS

Participants reported satisfactory levels of difficulty (mean: 1.9) and concentration (mean: 8.3) and assigned consistent values. Qualitative analysis revealed difficulties with imagining: living with depression for lifetime (n = 4); reaching the age of 80 (n = 6); and living with a somatic condition and mentally healthy (n = 6). Person- and state-related factors, for example perceived susceptibility to depression (n = 4), appeared to affect valuations.

CONCLUSION

Quantitative findings supported feasibility of the valuation protocol, yet qualitative findings indicated that certain task aspects should be readdressed. Factors influencing valuations can be explored to better understand valuations.

Getting a technology-based diabetes intervention ready for prime time: a review of usability testing studies

Consumer health technologies can educate patients about diabetes and support their self-management, yet usability evidence is rarely published even though it determines patient engagement, optimal benefit of any intervention, and an understanding of generalizability. Therefore, we conducted a narrative review of peer-reviewed articles published from 2009 to 2013 that tested the usability of a web- or mobile-delivered system/application designed to educate and support patients with diabetes. Overall, the 23 papers included in our review used mixed (n = 11), descriptive quantitative (n = 9), and qualitative methods (n = 3) to assess usability, such as documenting which features performed as intended and how patients rated their experiences. More sophisticated usability evaluations combined several complementary approaches to elucidate more aspects of functionality. Future work pertaining to the design and evaluation of technology-delivered diabetes education/support interventions should aim to standardize the usability testing processes and publish usability findings to inform interpretation of why an intervention succeeded or failed and for whom.

How do people interpret information about colorectal cancer screening: observations from a think-aloud study

Background

The English NHS Bowel Cancer Screening Programme biennially invites individuals aged 60–74 to participate in screening. The booklet, ‘Bowel Cancer Screening: The Facts' accompanies this invitation. Its primary aim is to inform potential participants about the aims, advantages and disadvantages of colorectal cancer screening.

Objective

To provide detailed commentary on how individuals process the information contained within ‘The Facts’ booklet.

Design, setting and participants

This study comprised of 18 interviews with individuals aged 45–60 and used a ‘think‐aloud’ paradigm in which participants read aloud the booklet. Participant utterances (verbal statements made in response to researcher‐led prompts) were transcribed and analysed using a combination of content and thematic analysis.

Results

A total of 776 coded utterances were analysed (mean = 43.1 per person; range = 8–95). While overall comprehension was satisfactory, several problem areas were identified such as the use of complex unfamiliar terminology and the presentation of numerical information. Specific sections such as colonoscopy risk information evoked negative emotional responses. Participants made several suggestions for ways in which comprehension might be improved.

Conclusion

Public perceptions of the NHS Bowel Cancer Screening Programme information materials indicated that specific aspects of the booklet were difficult to process. These materials may be an appropriate target to improve public understanding of the aims, benefits and disadvantages of colorectal cancer screening. These findings will contribute to a broader NIHR‐funded project that aims to design a supplementary ‘gist‐based’ information leaflet suitable for low literacy populations.