A Framework for Rigorous Qualitative Research as a Component of Mixed Method Rapid-Cycle Evaluation

Food Systems Interventions for Nutrition: Lessons from 6 Program Evaluations in Africa and South Asia

Although there is growing global momentum behind food systems strategies to improve planetary and human health-including nutrition-there is limited evidence of what types of food systems interventions work. Evaluating these types of interventions is challenging due to their complex and dynamic nature and lack of fit with standard evaluation methods. In this article, we draw on a portfolio of 6 evaluations of food systems interventions in Africa and South Asia that were intended to improve nutrition. We identify key methodological challenges and formulate recommendations to improve the quality of such studies. We highlight 5 challenges: a lack of evidence base to justify the intervention, the dynamic and multifaceted nature of the interventions, addressing attribution, collecting or accessing accurate and timely data, and defining and measuring appropriate outcomes. In addition to more specific guidance, we identify 6 cross-cutting recommendations, including a need to use multiple and diverse methods and flexible designs. We also note that these evaluation challenges present opportunities to develop new methods and highlight several specific needs in this space.

Identification of barriers at the primary care provider level to improve inflammatory breast cancer diagnosis and management

The purpose of this study, based in the United States, was to evaluate knowledge gaps and barriers related to diagnosis and care of inflammatory breast cancer (IBC), a rare but lethal breast cancer subtype, amongst Primary Care Providers (PCP) as they are often the first point of contact when patients notice initial symptoms. PCP participants in the Duke University Health System, federally qualified health center, corporate employee health and community practices, nearby academic medical center, Duke physician assistant and advanced practice nurse leadership program alumni were first selected in a convenience sample and for semi-structured interviews (n = 11). Based on these data, an online survey tool was developed and disseminated (n = 78) to assess salient measures of IBC diagnosis, health disparity factors, referral and care coordination practices, COVID-19 impact, and continuing medical education (CME). PCP reported access to care and knowledge gaps in symptom recognition (mean = 3.3, range 1-7) as major barriers. Only 31 % reported ever suspecting IBC in a patient. PCP (n = 49) responded being challenged with referral delays in diagnostic imaging. Additionally, since the COVID-19 pandemic started, 63 % reported breast cancer referral delays, and 33 % reported diagnosing less breast cancer. PCP stated interest in CME in their practice for improved diagnosis and patient care, which included online (53 %), lunch time or other in-service training (33 %), patient and provider-facing websites (32 %). Challenges communicating rare cancer information, gaps in confidence in diagnosing IBC, and timely follow-up with patients and specialists underscores the need for developing PCP educational modules to improve guideline-concordant care.

A Clinical Leadership Lens on Implementing Progress Feedback in Three Countries: Development of a Multidimensional Qualitative Coding Scheme

BACKGROUND

Progress feedback, also known as measurement-based care (MBC), is the routine collection of patient-reported measures to monitor treatment progress and inform clinical decision-making. Although a key ingredient to improving mental health care, sustained use of progress feedback is poor. Integration into everyday workflow is challenging, impacted by a complex interrelated set of factors across patient, clinician, organizational, and health system levels. This study describes the development of a qualitative coding scheme for progress feedback implementation that accounts for the dynamic nature of barriers and facilitators across multiple levels of use in mental health settings. Such a coding scheme may help promote a common language for researchers and implementers to better identify barriers that need to be addressed, as well as facilitators that could be supported in different settings and contexts.

METHODS

Clinical staff, managers, and leaders from two Dutch, three Norwegian, and four mental health organizations in the USA participated in semi-structured interviews on how intra- and extra-organizational characteristics interact to influence the use of progress feedback in clinical practice, supervision, and program improvement. Interviews were conducted in the local language, then translated to English prior to qualitative coding.

RESULTS

A team-based consensus coding approach was used to refine an a priori expert-informed and literature-based qualitative scheme to incorporate new understandings and constructs as they emerged. First, this hermeneutic approach resulted in a multi-level coding scheme with nine superordinate categories and 30 subcategories. Second-order axial coding established contextually sensitive categories for barriers and facilitators.

CONCLUSIONS

The primary outcome is an empirically derived multi-level qualitative coding scheme that can be used in progress feedback implementation research and development. It can be applied across contexts and settings, with expectations for ongoing refinement. Suggestions for future research and application in practice settings are provided. Supplementary materials include the coding scheme and a detailed playbook.

Stakeholder Perspectives on Factors Related to Deprescribing Potentially Inappropriate Medications in Older Adults Receiving Dialysis

BACKGROUND

Potentially inappropriate medications, or medications that generally carry more risk of harm than benefit in older adults, are commonly prescribed to older adults receiving dialysis. Deprescribing, a systematic approach to reducing or stopping a medication, is a potential solution to limit potentially inappropriate medications use. Our objective was to identify clinicians and patient perspectives on factors related to deprescribing to inform design of a deprescribing program for dialysis clinics.

METHODS

We conducted rapid qualitative analysis of semistructured interviews and focus groups with clinicians (dialysis clinicians, primary care providers, and pharmacists) and patients (adults receiving hemodialysis aged 65 years or older and those aged 55-64 years who were prefrail or frail) from March 2019 to December 2020.

RESULTS

We interviewed 76 participants (53 clinicians [eight focus groups and 11 interviews] and 23 patients). Among clinicians, 24 worked in dialysis clinics, 18 worked in primary care, and 11 were pharmacists. Among patients, 13 (56%) were aged 65 years or older, 14 (61%) were Black race, and 16 (70%) reported taking at least one potentially inappropriate medication. We identified four themes (and corresponding subthemes) of contextual factors related to deprescribing potentially inappropriate medications: ( 1 ) system-level barriers to deprescribing (limited electronic medical record interoperability, time constraints and competing priorities), ( 2 ) undefined comanagement among clinicians (unclear role delineation, clinician caution about prescriber boundaries), ( 3 ) limited knowledge about potentially inappropriate medications (knowledge limitations among clinicians and patients), and ( 4 ) patients prioritize symptom control over potential harm (clinicians expect resistance to deprescribing, patient weigh risks and benefits).

CONCLUSIONS

Challenges to integration of deprescribing into dialysis clinics included siloed health systems, time constraints, comanagement behaviors, and clinician and patient knowledge and attitudes toward deprescribing.

Impact of Telemedicine Modality on Quality Metrics in Diverse Settings: Implementation Science-Informed Retrospective Cohort Study

BACKGROUND

Video-based telemedicine (vs audio only) is less frequently used in diverse, low socioeconomic status settings. Few prior studies have evaluated the impact of telemedicine modality (ie, video vs audio-only visits) on clinical quality metrics.

OBJECTIVE

The aim of this study was to assess telemedicine uptake and impact of visit modality (in-person vs video and phone visits) on primary care quality metrics in diverse, low socioeconomic status settings through an implementation science lens.

METHODS

Informed by the RE-AIM (Reach, Effectiveness, Adoption, Implementation, and Maintenance) framework, we evaluated telemedicine uptake, assessed targeted primary care quality metrics by visit modality, and described provider-level qualitative feedback on barriers and facilitators to telemedicine implementation.

RESULTS

We found marginally better quality metrics (ie, blood pressure and depression screening) for in-person care versus video and phone visits; de-adoption of telemedicine was marked within 2 years in our population.

CONCLUSIONS

Following the widespread implementation of telemedicine during the COVID-19 pandemic, the impact of visit modality on quality outcomes, provider and patient preferences, as well as technological barriers in historically marginalized settings should be considered.

A mixed-method approach to generate and deliver rapid-cycle evaluation feedback: lessons learned from a multicenter implementation trial in pediatric surgery

BACKGROUND

Rapid-cycle feedback loops provide timely information and actionable feedback to healthcare organizations to accelerate implementation of interventions. We aimed to (1) describe a mixed-method approach for generating and delivering rapid-cycle feedback and (2) explore key lessons learned while implementing an enhanced recovery protocol (ERP) across 18 pediatric surgery centers.

METHODS

All centers are members of the Pediatric Surgery Research Collaborative (PedSRC, www.pedsrc.org ), participating in the ENhanced Recovery In CHildren Undergoing Surgery (ENRICH-US) trial. To assess implementation efforts, we conducted a mixed-method sequential explanatory study, administering surveys and follow-up interviews with each center's implementation team 6 and 12 months following implementation. Along with detailed notetaking and iterative discussion within our team, we used these data to generate and deliver a center-specific implementation report card to each center. Report cards used a traffic light approach to quickly visualize implementation status (green = excellent; yellow = needs improvement; red = needs significant improvement) and summarized strengths and opportunities at each timepoint.

RESULTS

We identified several benefits, challenges, and practical considerations for assessing implementation and using rapid-cycle feedback among pediatric surgery centers. Regarding potential benefits, this approach enabled us to quickly understand variation in implementation and corresponding needs across centers. It allowed us to efficiently provide actionable feedback to centers about implementation. Engaging consistently with center-specific implementation teams also helped facilitate partnerships between centers and the research team. Regarding potential challenges, research teams must still allocate substantial resources to provide feedback rapidly. Additionally, discussions and consensus are needed across team members about the content of center-specific feedback. Practical considerations include carefully balancing timeliness and comprehensiveness when delivering rapid-cycle feedback. In pediatric surgery, moreover, it is essential to actively engage all key stakeholders (including physicians, nurses, patients, caregivers, etc.) and adopt an iterative, reflexive approach in providing feedback.

CONCLUSION

From a methodological perspective, we identified three key lessons: (1) using a rapid, mixed method evaluation approach is feasible in pediatric surgery and (2) can be beneficial, particularly in quickly understanding variation in implementation across centers; however, (3) there is a need to address several methodological challenges and considerations, particularly in balancing the timeliness and comprehensiveness of feedback.

TRIAL REGISTRATION

NIH National Library of Medicine Clinical Trials.

CLINICALTRIALS

gov Identifier: NCT04060303. Registered August 7, 2019, https://clinicaltrials.gov/ct2/show/NCT04060303.

Rapid evaluation of service innovations in health and social care: key considerations

Funding

This project was funded by the National Institute for Health and Care Research (NIHR) Health and Social Care Delivery Research programme (RSET: 16/138/17; BRACE: 16/138/31).

Parents' Perspectives on the Utility of Genomic Sequencing in the Neonatal Intensive Care Unit

BACKGROUND

It is critical to understand the wide-ranging clinical and non-clinical effects of genome sequencing (GS) for parents in the NICU context. We assessed parents' experiences with GS as a first-line diagnostic tool for infants with suspected genetic conditions in the NICU.

METHODS

Parents of newborns (N = 62) suspected of having a genetic condition were recruited across five hospitals in the southeast United States as part of the SouthSeq study. Semi-structured interviews (N = 78) were conducted after parents received their child's sequencing result (positive, negative, or variants of unknown significance). Thematic analysis was performed on all interviews.

RESULTS

Key themes included that (1) GS in infancy is important for reproductive decision making, preparing for the child's future care, ending the diagnostic odyssey, and sharing results with care providers; (2) the timing of disclosure was acceptable for most parents, although many reported the NICU environment was overwhelming; and (3) parents deny that receiving GS results during infancy exacerbated parent-infant bonding, and reported variable impact on their feelings of guilt.

CONCLUSION

Parents reported that GS during the neonatal period was useful because it provided a "backbone" for their child's care. Parents did not consistently endorse negative impacts like interference with parent-infant bonding.

Inclusion of autistic students: promising modalities for supporting a school team

PURPOSE

School staff who work with autistic students are at a high risk of exhaustion. More training and guidance are needed to enable them to respond to these students' needs. The purpose of this study was to design and evaluate a professional development program offered by an occupational therapist to help a team of school staff facilitate the participation of autistic students and their peers.

METHODS

A design-based research approach guided the development and evaluation of the modalities of the program. A case study was conducted in a school with nine school staff members using individual interviews, questionnaires, and a logbook.

RESULTS

The program comprised five modalities: regular presence of the occupational therapist, a needs analysis, content focused on the features of an inclusive school, individual coaching, and team coaching. The following elements emerged as particularly helpful for the participants: frequent interactions with the occupational therapist, opportunities to collaborate, personalized support, experimentation of new practices with the occupational therapist, and concrete training content that can benefit all students.

CONCLUSION

The modalities of the program appear promising to help a school team facilitate the participation of all students, including those who are autistic.Implications for rehabilitationOccupational therapists can help school teams to facilitate the participation of autistic students and their peers.A combination of individual and team coaching sessions appears promising in helping school staff adopt practice changes to make their school more inclusive.Using a multi-tiered approach can help therapists collaborate with school staff members to first find solutions that will be applicable with all of their students.

Hidden Curriculum and the Demographic Stoicism That Keeps Women and Minorities Away From Radiology: A Mixed-Methods Study of Medical Students

OBJECTIVE

To understand how women and historically underrepresented minority medical students perceive radiology as a potential career choice.

METHODS

Medical students representing a broad spectrum of radiology exposure from a single institution were invited to participate in a mixed-methods study. Participants completed a 16-item survey about demographics and perceptions of radiology. Ten focus groups were administered to probe decision making regarding career selection. The themes influencing women and historically underrepresented minority students are presented.

RESULTS

Forty-nine medical students, including 29 (59%) women and 17 (35%) underrepresented minorities, participated. Most participants (28 of 48, 58%) reported men outnumbered women in radiology. Female participants reported a lack of mentorship and role models as major concerns. Outreach efforts focused on the family-friendly nature of radiology were viewed as patronizing. Demographic improvements in the field were viewed as very slow. Forty-six percent (22 of 48) of participants indicated that radiology had a less underrepresented racial or ethnic workforce than other medical specialties. Minority participants especially noted a lack of radiology presence in mainstream media, so students have few preconceived biases. A failure to organically connect with the mostly White male radiologists because of a lack of shared background was a major barrier. Finally, participants described a hidden curriculum that pushes minority medical students away from specialty fields like radiology and toward primary care fields to address underserved communities and health care disparities.

DISCUSSION

Women and historically underrepresented minority medical students perceive major barriers to choosing a career in radiology. Radiology departments must develop sophisticated multilevel approaches to improve diversity.

Describing current use, barriers, and facilitators of patient portal messaging for research recruitment: Perspectives from study teams and patients at one institution

Introduction

The electronic health record (EHR) and patient portal are used increasingly for clinical research, including patient portal recruitment messaging (PPRM). Use of PPRM has grown rapidly; however, best practices are still developing. In this study, we examined the use of PPRM at our institution and conducted qualitative interviews among study teams and patients to understand experiences and preferences for PPRM.

Methods

We identified study teams that sent PPRMs and patients that received PPRMs in a 60-day period. We characterized these studies and patients, in addition to the patients' interactions with the PPRMs (e.g., viewed, responded). From these groups, we recruited study team members and patients for semi-structured interviews. A pragmatic qualitative inquiry framework was used by interviewers. Interviews were audio-recorded and analyzed using a rapid qualitative analysis exploratory approach.

Results

Across ten studies, 35,037 PPRMs were sent, 33% were viewed, and 17% were responded to. Interaction rates varied across demographic groups. Six study team members completed interviews and described PPRM as an efficient and helpful recruitment method. Twenty-eight patients completed interviews. They were supportive of receiving PPRMs, particularly when the PPRM was relevant to their health. Patients indicated that providing more information in the PPRM would be helpful, in addition to options to set personalized preferences.

Conclusions

PPRM is an efficient recruitment method for study teams and is acceptable to patients. Engagement with PPRMs varies across demographic groups, which should be considered during recruitment planning. Additional research is needed to evaluate and implement recommended changes by study teams and patients.

Embedding Big Qual and Team Science into Qualitative Research: Lessons from a Large-scale, Cross-site Research Study

Background

A major part of the HEALing Communities Study (HCS), launched in 2019 to address the growing opioid epidemic, is evaluating the study's intervention implementation process through an implementation science (IS) approach. One component of the IS approach involves teams with more than 20 researchers collaborating across four research sites to conduct in-depth qualitative interviews with over 300 participants at four time points. After completion of the first two rounds of data collection, we reflect upon our qualitative data collection and analysis approach. We aim to share our lessons learned about designing and applying qualitative methods within an implementation science framework.

Methods

The HCS evaluation is based on the RE-AIM/PRISM framework and incorporates interviews at four timepoints. At each timepoint, the core qualitative team of the Intervention Work Group drafts an interview guide based on the framework and insights from previous round(s) of data collection. Researchers then conduct interviews with key informants and coalition members within their respective states. Data analysis involves drafting, iteratively refining, and finalizing a codebook in a cross-site and within-site consensus processes. Interview transcripts are then individually coded by researchers within their respective states.

Results

Successes in the evaluation process includes having structured procedures for communication, data collection, and analysis, all of which are critical for ensuring consistent data collection and for achieving consensus during data analysis. Challenges include recognizing and accommodating the diversity of training and knowledge between researchers, and establishing reliable ways to securely store, manage, and share the large volumes of data.

Conclusion

Qualitative methods using a team science approach have been limited in their application in large, multi-site randomized controlled trials of health interventions. Our experience provides practical guidance for future studies with large, experientially and disciplinarily diverse teams, and teams seeking to incorporate qualitative or mixed-methods components for their evaluations.

Approaches to enabling rapid evaluation of innovations in health and social care: a scoping review of evidence from high-income countries

OBJECTIVE

The COVID-19 pandemic increased the demand for rapid evaluation of innovation in health and social care. Assessment of rapid methodologies is lacking although challenges in ensuring rigour and effective use of resources are known. We mapped reports of rapid evaluations of health and social care innovations, categorised different approaches to rapid evaluation, explored comparative benefits of rapid evaluation, and identified knowledge gaps.

DESIGN

Scoping review.

DATA SOURCES

MEDLINE, EMBASE and Health Management Information Consortium (HMIC) databases were searched through 13 September 2022.

ELIGIBILITY CRITERIA FOR SELECTING STUDIES

We included publications reporting primary research or methods for rapid evaluation of interventions or services in health and social care in high-income countries.

DATA EXTRACTION AND SYNTHESIS

Two reviewers developed and piloted a data extraction form. One reviewer extracted data, a second reviewer checked 10% of the studies; disagreements and uncertainty were resolved through consensus. We used narrative synthesis to map different approaches to conducting rapid evaluation.

RESULTS

We identified 16 759 records and included 162 which met inclusion criteria.We identified four main approaches for rapid evaluation: (1) Using methodology designed specifically for rapid evaluation; (2) Increasing rapidity by doing less or using less time-intensive methodology; (3) Using alternative technologies and/or data to increase speed of existing evaluation method; (4) Adapting part of non-rapid evaluation.The COVID-19 pandemic resulted in an increase in publications and some limited changes in identified methods. We found little research comparing rapid and non-rapid evaluation.

CONCLUSIONS

We found a lack of clarity about what 'rapid evaluation' means but identified some useful preliminary categories. There is a need for clarity and consistency about what constitutes rapid evaluation; consistent terminology in reporting evaluations as rapid; development of specific methodologies for making evaluation more rapid; and assessment of advantages and disadvantages of rapid methodology in terms of rigour, cost and impact.

Assessing Knowledge and Barriers at the Primary Care Provider Level that Contribute to Disparities in Inflammatory Breast Cancer Diagnosis and Treatment

Purpose

The purpose of this study was to evaluate knowledge gaps and barriers related to diagnosis and care of inflammatory breast cancer (IBC), a rare but most lethal breast cancer subtype, amongst Primary Care Providers (PCP) as they are often the first point of contact when patients notice initial symptoms.

Methods

PCP participants within Duke University Health System, federally qualified health center, corporate employee health and community practices, nearby academic medical center, Duke physician assistant, and nurse leadership program alumni were first selected in a convenience sample (n=11) for semi-structured interviews (n=11). Based on these data, an online survey tool was developed and disseminated (n=78) to assess salient measures of IBC diagnosis, health disparity factors, referral and care coordination practices, COVID impact, and continued medical education (CME).

Results

PCP reported access to care and knowledge gaps in symptom recognition (mean = 3.3, range 1-7) as major barriers. Only 31% reported ever suspecting IBC in a patient. PCP (n=49) responded being challenged with referral delays in diagnostic imaging. Additionally, since the COVID-19 pandemic started, 63% reported breast cancer referral delays, and 33% reported diagnosing less breast cancer. PCP stated interest in CME in their practice for improved diagnosis and patient care, which included online (53%), lunch time or other in-service training (33%), patient and provider-facing websites (32%).

Conclusions

Challenges communicating rare cancer information, gaps in confidence in diagnosing IBC, and timely follow-up with patients and specialists underscores the need for developing PCP educational modules to improve guideline-concordant care.

Key stakeholder perspectives on challenges and opportunities for rural HPV vaccination in North and South Carolina

The objective of this study was to identify factors at the individual, provider, and systems levels that serve as challenges or opportunities for increasing adolescent vaccination-including Human Papillomavirus (HPV) vaccination-in rural communities in the southern United States (US). As part of a broader study to increase HPV vaccine uptake in the southern US, we conducted in-depth interviews with vaccination stakeholders representing public health and education agencies in North Carolina (NC) and South Carolina (SC). Fourteen key stakeholders were recruited using purposive sampling to obtain insights into challenges and solutions to rural-urban disparities in HPV vaccination coverage. Stakeholders were also queried about their experiences and attitudes toward school-based vaccination promotion programs and campaigns. We used a rapid qualitative approach to analyze the data. Stakeholders identified factors at the individual, provider, and systems levels that serve as challenges to vaccination in rural communities. Similar to previous studies, stakeholders mentioned challenges with healthcare access and vaccine-related misconceptions that pose barriers to HPV vaccination for rural residents. Systems-level challenges identified included limited access to high-speed internet in rural areas that may impact providers' ability to interface with state-level digital systems such as the vaccination registry. Stakeholders identified a number of opportunities to increase HPV vaccination coverage, including through school-based health promotion programs. Stakeholders strongly supported school-based programs and approaches to strengthen confidence and demand for HPV vaccination and to help address persistent social determinants and system level factors that pose challenges to HPV vaccination coverage in many rural areas.

Rapid evaluation of COVID-19 related service and practice changes in health and human services using tailored methods

The COVID-19 pandemic required substantive delivery and practice changes for government services under tight timeframes and high public scrutiny. These urgently implemented service changes provided the opportunity for evaluators to support decision-makers to understand the impact of adaptations for those delivering and receiving health and human services. Tailored rapid evaluation methods (REM) provide a pragmatic approach to generating timely information for evidence-based policy and decision-making under these conditions. Drawing from features of a range of existing rapid evaluation models, as well as developmental and utilization-focussed evaluation theory, this article outlines the design and implementation of a novel REM approach and considers the benefits of both tailoring and standardizing rapid evaluation approaches to meet end-user needs. The tailored REM approach and mixed methods are contextualized and compared to other documented rapid evaluation models to demonstrate the purpose and value of customization. This article builds on previous descriptions of the implementation of a novel REM approach to provide a comparative account of tailored rapid evaluation methods. The article outlines the drivers that led to the selected tailoring of the REM approach, and shares lessons learned in the context of the COVID-19 pandemic by a large internal government evaluation unit (Department of Health and Human Services) in Victoria, Australia. The customized features of REM ensure that it can consider the experiences of those delivering and receiving services, and inform near-term decision-making on programme and policy design in emergency and fast-paced contexts. The article shares a case study of a rapid evaluation of telehealth in pediatric care to demonstrate insights from tailoring the REM approach in practice. The REM method was utilized with the aim of delivering findings in a time-sensitive manner to rapidly inform decision making for policy-makers. Key enablers for the tailored REM protocol include the use of multi-disciplinary teams, flexible evaluation design, and a participatory approach that facilitates stakeholder involvement throughout delivery. Insights from the case study and methods presented seek to inform practice for evaluators who intend to or may want to tailor their own rapid evaluation model in resource and time-limited settings.

The experience of conducting collaborative and intensive pragmatic qualitative (CLIP-Q) research to support rapid public health and healthcare innovation

A key challenge for qualitative methods in applied health research is the fast pace that can characterize the public health and health and care service landscape, where there is a need for research informed by immediate pragmatic questions and relevant findings are required quickly to inform decision-making. The COVID-19 pandemic accelerated the pace at which evidence was needed to inform urgent public health and healthcare decision-making. This required qualitative researchers to step up to the challenge of conducting research at speed whilst maintaining rigor and ensuring the findings are credible. This article illustrates how working with multidisciplinary, collaborative teams and the tailoring of qualitative methods to be more pragmatic and efficient can provide timely and credible results. Using time-limited case studies of applied qualitative health research drawn from the work of the Behavioral and Qualitative Science Team from the National Institute for Health and Care Research Applied Research Collaboration West (NIHR ARC West), we illustrate our collaborative and intensive pragmatic qualitative (CLIP-Q) approach. CLIP-Q involves (i) collaboration at all stages of the design, conduct and implementation of projects and, where possible, co-production with people with lived experience, (ii) an intensive team-based approach to data collection and analysis at pace, and (iii) pragmatic study design and efficient strategies at each stage of the research process. The case studies include projects conducted pre COVID-19 and during the first wave of the pandemic, where urgent evidence was required in weeks rather than months to inform rapid public health and healthcare decision making.

Religion, politics and COVID-19 risk perception among urban residents in Malawi

INTRODUCTION

Majority of Malawians have not yet adopted COVID-19 mitigation measures despite having knowledge about its infectivity, morbidity, and fatality. Understanding drivers of hesitancy to adoption of COVID-19 mitigation measures is critical as it can inform prevention programs. This study explores Malawians' COVID-19 risk perception, and the associated constraints in the adoption of mitigation efforts. A Health Belief Model (HBM) approach was used to understand perceived factors that undermine public health COVID-19 messages to reduce the spread of the pandemic in Malawi.

METHODS

The study applied rapid appraisal and photovoice qualitative inquiry to comprehend risk perception regarding COVID-19. We purposively selected 52 participants from three major cities in Malawi. Audio and video interviews were transcribed verbatim, and transcripts were coded manually to derive key themes and concepts.

RESULTS

The study identified that social factors particularly religious and political beliefs influenced COVID-19 risk perception. Specific religious beliefs pertaining to individuals recognizing signs of the 'Christian apocalypse' were particularly associated with lower risk perceptions. Politically, participants believed COVID-19 lockdown measures were a ploy by the then-ruling party to remain in power.

CONCLUSION

The study suggests that religious beliefs and political environment undermine self -perceived risk of contracting COVID-19 among urban dwellers in Malawi. We recommend that diverse actors in Malawi should collaborate to promote the dissemination of accurate COVID-19 discourses and reduce the severity of the pandemic's impact in Malawi.

Radiology Stereotypes, Application Barriers, and Hospital Integration: A Mixed-methods Study of Medical Student Perceptions of Radiology

RATIONALE AND OBJECTIVES

Limited exposure to radiology by medical students can perpetuate negative stereotypes and hamper recruitment efforts. The purpose of this study is to understand medical students' perceptions of radiology and how they change based on medical education and exposure.

MATERIALS AND METHODS

A single-institution mixed-methods study included four groups of medical students with different levels of radiology exposure. All participants completed a 16-item survey regarding demographics, opinions of radiology, and perception of radiology stereotypes. Ten focus groups were administered to probe perceptions of radiology. Focus groups were coded to identify specific themes in conjunction with the survey results.

RESULTS

Forty-nine participants were included. Forty-two percent of participants had positive opinions of radiology. Multiple radiology stereotypes were identified, and false stereotypes were diminished with increased radiology exposure. Opinions of the impact of artificial intelligence on radiology closely aligned with positive or negative views of the field overall. Multiple barriers to applying for a radiology residency position were identified including board scores and lack of mentorship. COVID-19 did not affect perceptions of radiology. There was broad agreement that students do not enter medical school with many preconceived notions of radiology, but that subsequent exposure was generally positive. Exposure both solidified and eliminated various stereotypes. Finally, there was general agreement that radiology is integral to the health system with broad exposure on all services.

CONCLUSION

Medical student perceptions of radiology are notably influenced by exposure and radiology programs should take active steps to engage in medical student education.

What makes home environments favorable to independence: perspectives of autistic people and their parents

PURPOSE

Few autistic adolescents and adults manage to integrate a home that enables them to fully exercise their independence, even if that is a desire shared by many of them. Creating residential environments that are favorable to their independence at home is a promising, yet poorly explored, avenue. The aim of this study conducted in Québec (Canada) is to identify the main environmental factors influencing their independence at home from the perspectives of autistic people and their parents.

MATERIALS AND METHODS

The walking interview method was used to collect the perspectives of 10 dyads composed of an autistic person and at least one of their parents.

RESULTS

Participants identified several factors, including: (a) support from parents, (b) support from extended social network, (c) a physical environment that meets one's needs, (d) clear time indicators, (e) opportunities to perform life habits in other settings, and (f) support from professionals. They also gave many examples of concrete ways to implement these factors at home and suggested elements to consider when modifying the home environment.

CONCLUSIONS

The results emphasize the need to involve both autistic people and their parents during the evaluation and implementation of these factors to optimize their benefits. IMPLICATIONS FOR REHABILITATIONModifying the residential environments of autistic people is a promising way to foster their independence at home.The environmental factors of supportive home environments identified in this study can be used as a starting point when designing home environments for autistic people.As the needs and preferences of autistic people vary, they must be involved in the selection and the implementation of modifications in their home environments.Parents have a key role to play to support the development of their youth's independence at home.

Evaluating fitness-for-use of electronic health records in pragmatic clinical trials: reported practices and recommendations

OBJECTIVE

To empirically explore how pragmatic clinical trials (PCTs) that used real-world data (RWD) assessed study-specific fitness-for-use.

METHODS

We conducted interviews and surveys with PCT teams who used electronic health record (EHR) data to ascertain endpoints. The survey cataloged key concerns about RWD, activities used to assess data fitness-for-use, and related barriers encountered by study teams. Patterns and commonalities across trials were used to develop recommendations for study-specific fitness-for-use assessments.

RESULTS

Of 15 invited trial teams, 7 interviews were conducted. Of 31 invited trials, 15 responded to the survey. Most respondents had prior experience using RWD (93%). Major concerns about EHR data were data reliability, missingness or incompleteness of EHR elements, variation in data quality across study sites, and presence of implausible or incorrect values. Although many PCTs conducted fitness-for-use activities (eg, data quality assessments, 11/14, 79%), less than a quarter did so before choosing a data source. Fitness-for-use activities, findings, and resulting study design changes were not often publically documented. Overall costs and personnel costs were barriers to fitness-for-use assessments.

DISCUSSION

These results support three recommendations for PCTs that use EHR data for endpoint ascertainment. Trials should detail the rationale and plan for study-specific fitness-for-use activities, conduct study-specific fitness-for-use assessments early in the prestudy phase to inform study design changes before the trial begins, and share results of fitness-for-use assessments and description of relevant challenges and facilitators.

CONCLUSION

These recommendations can help researchers and end-users of real-world evidence improve characterization of RWD reliability and relevance in the PCT-specific context.

Acceptability of, and preferences for, remote consulting during COVID-19 among older patients with two common long-term musculoskeletal conditions: findings from three qualitative studies and recommendations for practice

Background

Guidance for choosing face-to-face vs remote consultations (RCs) encourages clinicians to consider patient preferences, however, little is known about acceptability of, and preferences for RCs, particularly amongst patients with musculoskeletal conditions. This study aimed to explore the acceptability of, and preferences for, RC among patients with osteoporosis and rheumatoid arthritis.

Methods

Three UK qualitative studies, exploring patient experiences of accessing and receiving healthcare, undertaken during the pandemic, with people with osteoporosis and rheumatoid arthritis. Study team members agreed a consistent approach to conduct rapid deductive analysis using the Theoretical Framework of Acceptability (TFA) on transcripts from each data set relating to RC, facilitated by group meetings to discuss interpretations. Findings from the three studies were pooled.

Results

Findings from 1 focus group and 64 interviews with 35 people were included in the analysis. Participants’ attitudes to RC, views on fairness (ethicality) and sense-making (intervention coherence) varied according to their needs within the consultation and views of the pandemic. Some participants valued the reduced burden associated with RC, while others highly valued non-verbal communication and physical examination associated with face-to-face consults (opportunity costs). Some participants described low confidence (self-efficacy) in being able to communicate in RCs and others perceived RCs as ineffective, in part due to suboptimal communication.

Conclusions

Acceptability of, and preferences for RC appear to be influenced by societal, healthcare provider and personal factors and in this study, were not condition-dependant. Remote care by default has the potential to exacerbate health inequalities and needs nuanced implementation.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12891-022-05273-1.

Using rapid evaluation methods to assess service delivery changes: Lessons learned for evaluation practice during the COVID-19 pandemic

The COVID-19 pandemic required large-scale service delivery changes for government, and provided the opportunity for evaluators to step up and support decision makers to understand the impact of these changes. Rapid evaluation methods (REM) provide a pragmatic approach for generating timely information for evidence-based policy and decision-making. Grounded in developmental and utilisation-focused evaluation theory, REM incorporates a team-based, mixed methods design, executed over a 6-8-week period. Customised rubrics were used to rigorously assess effectiveness and scalability of practice changes to inform COVID-19 response planning. REM is an alternative approach to full-scale evaluation models frequently implemented to assess policies and programs. Adapted use of REM suggests that meaningful insights can be gained through use of smaller scale evaluations. This article shares lessons learned from a novel rapid evaluation method applied in the context of the COVID-19 pandemic. The rapid evaluation approach was implemented to provide real-time insights and evaluative conclusions for 15 program and practice adaptations across Victorian health and human service settings. The article shares insights about the practical applicability of balancing rigour and timeliness when implementing a rapid evaluation, and strengths and limitations of working within a fast-paced evaluation framework. Findings can inform evaluative practice in resource and time-limited settings.

Utilizing Multilingual Methods and Rapid Analysis for Global Qualitative Research During a Pandemic

Historically, qualitative research has complemented quantitative biologic and epidemiologic studies to provide a more complete understanding of pandemics. The COVID-19 pandemic has generated unique and novel challenges for qualitative researchers, who have embraced creative solutions including virtual focus groups and rapid analyses to continue their work. We present our experience conducting a multilingual global qualitative study of healthcare resilience among teams of pediatric oncology professionals during the COVID-19 pandemic. We provide an in-depth description of our methodology and an analysis of factors we believe contributed to our study’s success including our use of technology, engagement of a large multilingual team, global partnerships, and framework-based rapid analysis. We hope these techniques may be useful to qualitative researchers conducting studies during the current pandemic, as well as for all pediatric oncology studies including multiple languages or geographically disparate subjects.

Integrating PrEP delivery in public health family planning clinics: a protocol for a pragmatic stepped wedge cluster randomized trial in Kenya

Background

Adolescent girls and young women account for a disproportionate fraction of new HIV infections in Africa and are a priority population for HIV prevention, including provision of pre-exposure prophylaxis (PrEP). Anchoring PrEP delivery to care settings like family planning (FP) services that women already access routinely may offer an efficient platform to reach HIV at-risk women. However, context-specific implementation science evaluation is needed.

Methods

The Family Planning Plus Project is a prospective, pragmatic implementation evaluation, designed as a stepped wedge, cluster randomized trial, at 12 clinics in Kenya. In collaboration with the Kenya Ministry of Health and Kisumu County Department of Health, we will introduce integration of HIV risk screening and PrEP delivery in public health FP clinics. The core multifaceted implementation strategies to integrate PrEP in FP clinics will include: (1) PrEP delivery by existing FP clinic staff, (2) health provider training, (3) PrEP technical assistance to coach and mentor providers, (4) joint supervision with Kisumu County health officials, and (5) stakeholder engagement. All core components of PrEP delivery—including screening for HIV risk, HIV testing, dispensing, adherence and risk reduction counseling, assessment of side effects, and provision of refills, or safety assessment—will be conducted by existing FP clinic staff as part of a standard care service package. The goal is to catalyze sustainable scale-up within existing infrastructures beyond the project. We will rigorously evaluate implementation outcomes and impact, using the RE-AIM (Reach, Effectiveness, Adoption, Implementation, Maintenance) framework, and we will use Organizational Readiness for Implementing Change (ORIC) and the Consolidated Framework for Implementation Science Research (CFIR) to assess readiness to implement and contextual enablers and barriers of implementation, including how clinics innovate efficient delivery systems.

Discussion

Anchoring PrEP delivery to existing FP systems and staffing has tremendous potential to address barriers that women face in accessing HIV prevention and PrEP care, including lack of time, cost, and stigma of visiting a facility solely for HIV prevention. The FP Plus Project will initiate preparation for full-scale and sustainable model of integration of comprehensive HIV prevention services, including PrEP implementation, in public health FP clinics in low-income settings.

Trial registration

Registered with ClinicalTrials.gov on December 14, 2020: NCT04666792

Evaluating the Effect a of Handout on Community Pharmacists' Opioid Safety Counseling

Community pharmacists are the most accessible healthcare professionals to counsel patients about opioid risks and safety. Resources such as handouts are needed to improve pharmacists' self-efficacy about opioid safety counseling. This study aims to understand the effects and usefulness of handouts in opioid risk and safety counseling in community pharmacists. Three community pharmacies participated in this study for three weeks, where five pharmacists participated in completing a survey about the process and integration of a handout in opioid medication consultation. Pharmacists filled the survey after counseling patients for opioid medication/s. Field observations were conducted at one of these sites as well. A total of 57 consults were recorded via surveys in these pharmacies. Only using the handout to guide the conversation was rated much less useful than integrating the handout and showing it to patient in the consult (β = 0.94, adjusted R2 = 0.29, p< .00001). Satisfaction about the patient education provided increased if opioid risks and safety topics were discussed (β = 0.7, adjusted R2 = 0.32, p = 0.00015). Patients seemed engaged in the consults, which was evident from the head nods, questions asked, and attention toward the pharmacist. Effective communication with patients or patient caregivers about opioid safety can be accomplished by using and modeling use of this handout and by providing structured instructions to use this handout optimally.

The Therapeutic Odyssey: Positioning Genomic Sequencing in the Search for a Child's Best Possible Life

Background: The desire of parents to obtain a genetic diagnosis for their child with intellectual disability and associated symptoms has long been framed as a diagnostic odyssey, an arduous and sometimes perilous journey focused on the goal of identifying a cause for the child's condition.Methods: Semi-structured interviews (N = 60) were conducted with parents of children (N = 59, aged 2-24 years) with intellectual disability and/or developmental delay (IDD) who underwent genome sequencing at a single pediatric multispecialty clinic. Interviews were conducted after parents received their child's sequencing result (positive findings, negative findings, or variants of unknown significance). Thematic analysis was performed on all interviews.Results: Parents reported that obtaining a genetic diagnosis was one important step in their overall goal of helping their child live their best life possible life. They intended to use the result as a tool to help their child by seeking the correct school placement and obtaining benefits and therapeutic services.Conclusions: For the parents of children with IDD, the search for a genetic diagnosis is best conceptualized as a part of parents' ongoing efforts to leverage various diagnoses to obtain educational and therapeutic services for their children. Cleaving parents' search for a genetic diagnosis from these broader efforts obscures the value that some parents place on a sequencing result in finding and tailoring therapies and services beyond the clinic. Interviews with parents reveal, therefore, that genomic sequencing is best understood as one important stage of an ongoing therapeutic odyssey that largely takes place outside the clinic. Findings suggest the need to expand translational research efforts to contextualize a genetic diagnosis within parents' broader efforts to obtain educational and therapeutic services outside clinical contexts.

Monitoring implementation in program evaluation with direct audio coding

This project explored the reliability and utility of transcription in coding qualitative data across two studies in a program evaluation context. The first study tested the method of direct audio coding, or coding audio files without transcripts, using qualitative data software. The presence and frequency of codes applied in direct audio coding and traditional transcription coding were compared and the two methods produced similar results. Direct audio coding was then employed in an evaluation study to monitor implementation and the method was found to be reliable. Implications are discussed with considerations for both researchers and practitioners.

Value and Limitations of Broad Brush Surveys Used in Community-Randomized Trials in Southern Africa

We describe and reflect on a rapid qualitative survey approach called "Broad Brush Survey" (BBS) used in six community-randomized trials (CRTs)/studies in Zambia and South Africa (2004-2018) to document, compare, classify, and communicate community features systematically for public health and multidisciplinary research ends. BBS is based on a set sequence of participatory qualitative methods and fieldwork carried out prior to a CRT intervention and/or research by social scientists to generate rapid community profiles using four key indicators: physical features, social organization, networks, and community narratives. Profiling makes apparent similarities and differences, enabling comparison across communities and can be facilitated by an ideal model of open-closed systems. Findings have provided practical outputs (e.g., community profiles) and academic opportunities (e.g., community typologies). The BBS approach enables complex social landscapes to be incorporated in CRTs. This method has proven to be useful, adaptable and to have multidisciplinary appeal.