Lessons Learned: Beta-Testing the Digital Health Checklist for Researchers Prompts a Call to Action by Behavioral Scientists

Design Approaches for Developing Quality Checklists in Healthcare Organizations: A Scoping Review

Quality checklists have demonstrated benefits in healthcare and other high-reliability organizations, but there remains a gap in the understanding of design approaches and levels of stakeholder engagement in the development of these quality checklists. This scoping review aims to synthesize the current knowledge base regarding the use of various design approaches for developing quality checklists in healthcare. Secondary objectives are to explore theoretical frameworks, design principles, stakeholder involvement and engagement, and characteristics of the design methods used for developing quality checklists. The review followed the Preferred Reporting Items for Systematic Reviews 2020 checklist. Seven databases (PubMed, APA PsycInfo, CINAHL, Embase, Scopus, ACM Digital Library, and IEEE Xplore) were searched for studies using a comprehensive search strategy developed in collaboration with a health sciences librarian. Search terms included "checklist" and "user-centered design" and their related terms. The IAP2 Spectrum of Participation Framework was used to categorize studies by level of stakeholder engagement during data extraction. Twenty-nine studies met the inclusion criteria for this review. Twenty-three distinct design methods were identified that were predominantly non-collaborative in nature (e.g., interviews, surveys, and other methods that involved only one researcher and one participant at a given time). Analysis of the levels of stakeholder engagement revealed a gap in studies that empowered their stakeholders in the quality checklist design process. Highly effective, clear, and standardized methodology are needed for the design of quality checklists. Future work needs to explore how stakeholders can be empowered in the design process, and how different levels of stakeholder engagement might impact implementation outcomes.

The PICOTS-ComTeC Framework for Defining Digital Health Interventions: An ISPOR Special Interest Group Report

OBJECTIVES

Digital health definitions are abundant, but often lack clarity and precision. We aimed to develop a minimum information framework to define patient-facing digital health interventions (DHIs) for outcomes research.

METHODS

Definitions of digital-health-related terms (DHTs) were systematically reviewed, followed by a content analysis using frameworks, including PICOTS (population, intervention, comparator, outcome, timing, and setting), Shannon-Weaver Model of Communication, Agency for Healthcare Research and Quality Measures, and the World Health Organization's Classification of Digital Health Interventions. Subsequently, we conducted an online Delphi study to establish a minimum information framework, which was pilot tested by 5 experts using hypothetical examples.

RESULTS

After screening 2610 records and 545 full-text articles, we identified 101 unique definitions of 67 secondary DHTs in 76 articles, resulting in 95 different patterns of concepts among the definitions. World Health Organization system (84.5%), message (75.7%), intervention (58.3%), and technology (52.4%) were the most frequently covered concepts. For the Delphi survey, we invited 47 members of the ISPOR Digital Health Special Interest Group, 18 of whom became the Delphi panel. The first, second, and third survey rounds were completed by 18, 11, and 10 respondents, respectively. After consolidating results, the PICOTS-ComTeC acronym emerged, involving 9 domains (population, intervention, comparator, outcome, timing, setting, communication, technology, and context) and 32 optional subcategories.

CONCLUSIONS

Patient-facing DHIs can be specified using PICOTS-ComTeC that facilitates identification of appropriate interventions and comparators for a given decision. PICOTS-ComTeC is a flexible and versatile tool, intended to assist authors in designing and reporting primary studies and evidence syntheses, yielding actionable results for clinicians and other decision makers.

The Digital Therapeutics Real-World Evidence Framework: An Approach for Guiding Evidence-Based Digital Therapeutics Design, Development, Testing, and Monitoring

Digital therapeutics (DTx) are a promising way to provide safe, effective, accessible, sustainable, scalable, and equitable approaches to advance individual and population health. However, developing and deploying DTx is inherently complex in that DTx includes multiple interacting components, such as tools to support activities like medication adherence, health behavior goal-setting or self-monitoring, and algorithms that adapt the provision of these according to individual needs that may change over time. While myriad frameworks exist for different phases of DTx development, no single framework exists to guide evidence production for DTx across its full life cycle, from initial DTx development to long-term use. To fill this gap, we propose the DTx real-world evidence (RWE) framework as a pragmatic, iterative, milestone-driven approach for developing DTx. The DTx RWE framework is derived from the 4-phase development model used for behavioral interventions, but it includes key adaptations that are specific to the unique characteristics of DTx. To ensure the highest level of fidelity to the needs of users, the framework also incorporates real-world data (RWD) across the entire life cycle of DTx development and use. The DTx RWE framework is intended for any group interested in developing and deploying DTx in real-world contexts, including those in industry, health care, public health, and academia. Moreover, entities that fund research that supports the development of DTx and agencies that regulate DTx might find the DTx RWE framework useful as they endeavor to improve how DTxcan advance individual and population health.

Ethical, legal, and social implications of digital health: A needs assessment from the Society of Behavioral Medicine to inform capacity building for behavioral scientists

The ethical, legal, and social implications (ELSIs) of digital health are important when researchers and practitioners are using technology to collect, process, or store personal health data. Evidence underscores a strong need for digital health ELSI training, yet little is known about the specific ELSI topic areas that researchers and practitioners would most benefit from learning. To identify ELSI educational needs, a needs assessment survey was administered to the members of the Society of Behavioral Medicine (SBM). We sought to identify areas of ELSI proficiency and training need, and also evaluate interest and expertise in ELSI topics by career level and prior ELSI training history. The 14-item survey distributed to SBM members utilized the Digital Health Checklist tool (see recode.health/tools) and included items drawn from the four-domain framework: data management, access and usability, privacy and risk to benefit assessment. Respondents (N = 66) were majority faculty (74.2%) from psychology or public health. Only 39.4% reported receiving "formal" ELSI training. ELSI topics of greatest interest included practices that supported participant engagement, and dissemination and implementation of digital tools beyond the research setting. Respondents were least experienced in managing "bystander" data, having discussions about ELSIs, and reviewing terms of service agreements and privacy policies with participants and patients. There is opportunity for formalized ELSI training across career levels. Findings serve as an evidence base for continuous and ongoing evaluation of ELSI training needs to support scientists in conducting ethical and impactful digital health research.

Evaluating the Added Value of Digital Contact Tracing Support Tools for Citizens: Framework Development

BACKGROUND

The COVID-19 pandemic revealed that with high infection rates, health services conducting contact tracing (CT) could become overburdened, leading to limited or incomplete CT. Digital CT support (DCTS) tools are designed to mimic traditional CT, by transferring a part of or all the tasks of CT into the hands of citizens. Besides saving time for health services, these tools may help to increase the number of contacts retrieved during the contact identification process, quantity and quality of contact details, and speed of the contact notification process. The added value of DCTS tools for CT is currently unknown.

OBJECTIVE

To help determine whether DCTS tools could improve the effectiveness of CT, this study aims to develop a framework for the comprehensive assessment of these tools.

METHODS

A framework containing evaluation topics, research questions, accompanying study designs, and methods was developed based on consultations with CT experts from municipal public health services and national public health authorities, complemented with scientific literature.

RESULTS

These efforts resulted in a framework aiming to assist with the assessment of the following aspects of CT: speed; comprehensiveness; effectiveness with regard to contact notification; positive case detection; potential workload reduction of public health professionals; demographics related to adoption and reach; and user experiences of public health professionals, index cases, and contacts.

CONCLUSIONS

This framework provides guidance for researchers and policy makers in designing their own evaluation studies, the findings of which can help determine how and the extent to which DCTS tools should be implemented as a CT strategy for future infectious disease outbreaks.

Qualitative Evaluation of Family Caregivers' Experiences Participating in Knowledge and Interpersonal Skills to Develop Exemplary Relationships (KINDER): Web-Based Intervention to Improve Relationship Quality

BACKGROUND

The onset of Alzheimer disease and related dementias (AD/ADRD) can alter relationships between family caregivers and persons living with AD/ADRD, such as through the occurrence of distressful behavioral and psychological symptoms of dementia. Poorly perceived relationship quality by caregivers contributes to negative outcomes for both care partners, such as low-quality caregiving and potential mistreatment of older adults. Knowledge and Interpersonal Skills to Develop Exemplary Relationships (KINDER) is a new, web-based, asynchronous psychoeducational intervention with content informed by focus groups with family caregivers. The program was developed to prevent low-quality caregiving and potential mistreatment of older adults by focusing on building healthy caregiving relationships.

OBJECTIVE

The purpose of this study is to describe caregivers' experiences participating in KINDER to understand intervention acceptability. Of particular interest was learning how comfortable caregivers were viewing content addressing potential mistreatment, as well as whether asynchronous delivery created any barriers to participating in the intervention. Findings will inform future program refinements before efficacy testing.

METHODS

Although 23 caregivers enrolled in the KINDER parent study, only 7 of them completed the 8-week intervention. In-depth, semistructured qualitative interviews were conducted with all participants who completed the program to understand their experiences while attending KINDER and to decipher barriers to participation. We also asked participants about which program elements were most valuable and which were least valuable to them, as well as how the program could be improved. Interview transcripts were analyzed by 2 coders using thematic analysis.

RESULTS

Our findings indicate that caregivers were overall satisfied with KINDER's focus and content. Participants particularly liked how KINDER materials felt authentic and relevant to supporting healthy care relationships (Theme 1). The program's multiple components were found to be valuable, especially story-based video vignettes and readings (Theme 2). Most caregivers were comfortable viewing depictions of mistreatment and understood the importance of this content (Theme 3). Notably, while caregivers appreciated the convenience of participating in an asynchronous web-based intervention, several expressed a desire for more opportunities to speak with other caregivers (Theme 4). Technology challenges, such as a lack of clarity about automated intervention activities, deterred completion.

CONCLUSIONS

Findings from this study suggest an asynchronous web-based intervention covering sensitive topics such as mistreatment is acceptable for at least some AD/ADRD caregivers. Caregivers' comments that materials felt authentic may suggest that the integration of caregiver voices before intervention development enhanced the relevance of content. To make KINDER easier to deliver and participate in, the investigators plan to reduce the use of automation and integrate more group-based programming, as recommended by participants. Further, given the higher-than-expected dropout rate, in future studies, the investigators will collect data to determine the reasons for participants not completing study activities.

Capturing Cognitive Aging in Vivo: Application of a Neuropsychological Framework for Emerging Digital Tools

As the global burden of dementia continues to plague our healthcare systems, efficient, objective, and sensitive tools to detect neurodegenerative disease and capture meaningful changes in everyday cognition are increasingly needed. Emerging digital tools present a promising option to address many drawbacks of current approaches, with contexts of use that include early detection, risk stratification, prognosis, and outcome measurement. However, conceptual models to guide hypotheses and interpretation of results from digital tools are lacking and are needed to sort and organize the large amount of continuous data from a variety of sensors. In this viewpoint, we propose a neuropsychological framework for use alongside a key emerging approach-digital phenotyping. The Variability in Everyday Behavior (VIBE) model is rooted in established trends from the neuropsychology, neurology, rehabilitation psychology, cognitive neuroscience, and computer science literature and links patterns of intraindividual variability, cognitive abilities, and everyday functioning across clinical stages from healthy to dementia. Based on the VIBE model, we present testable hypotheses to guide the design and interpretation of digital phenotyping studies that capture everyday cognition in vivo. We conclude with methodological considerations and future directions regarding the application of the digital phenotyping approach to improve the efficiency, accessibility, accuracy, and ecological validity of cognitive assessment in older adults.

Advancing Infodemiology in a Digital Intensive Era

Collapse