Practical Guide to Mixed Methods

Advanced and alternative research methods for the acute care surgeon scientist

Clinical research has evolved significantly over the last few decades to include many advanced and alternative study designs to answer unique questions. Recognizing a potential knowledge gap, the AAST Associate Member Council and Educational Development Committee created a research course at the 2022 Annual Meeting in Chicago to introduce junior researchers to these methodologies. This manuscript presents a summary of this AAST Annual Meeting session, and reviews topics including hierarchical modeling, geospatial analysis, patient-centered outcomes research, mixed methods designs, and negotiating complex issues in multicenter trials.

The Effects of Quality Assurance System Implementation on Work Well-Being and Patient Safety: Protocol for a Mixed Methods Study

BACKGROUND

Systematic monitoring of work atmosphere and patient safety incidents is a necessary part of a quality assurance system, particularly an accredited system like the Joint Commission International (JCI). How the implementation of quality assurance systems affects well-being at work and patient safety is unclear. Evidence shows that accreditation improves workplace atmosphere and well-being. Thus, the assumption that an increase in employees' well-being at work improves patient safety is reasonable.

OBJECTIVE

This study aims to describe the protocol for monitoring the effects of implementing the quality assurance system of JCI at Orton Orthopedic Hospital on employees' well-being (primary outcome) and patient safety (secondary outcome).

METHODS

Quantitative (questionnaires and register data) and qualitative (semistructured interviews) methods will be used. In addition, quantitative data will be collected from register data. Both quantitative and register data will be analyzed. Register data analysis will be performed using generalized linear models with an appropriate distribution and link function. The study timeline covers the time before, during, and after the start of the accreditation process. The collected data will be used to compare job satisfaction, as a part of the well-being questionnaire, and the development of patient safety during the accreditation process.

RESULTS

The results of the quality assurance system implementation illuminate its possible effects on the patient's safety and job satisfaction. The repeatability and internal consistency reliability of the well-being questionnaire will be reported. Data collection will begin in May, 2024. It will be followed by data analysis and the results are expected to be published by 2025.

CONCLUSIONS

The planned study will contribute to the evaluation of the effects of JCI accreditation in terms of well-being at work and patient safety.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

PRR1-10.2196/45200.

Prescriber Uncertainty as Opportunity to Improve Care of Type 2 Diabetes with Chronic Kidney Disease: Mixed Methods Study

BACKGROUND

Over 5 million patients in the United States have type 2 diabetes mellitus (T2D) with chronic kidney disease (CKD); antidiabetic drug selection for this population is complex and has important implications for outcomes.

OBJECTIVE

To better understand how providers choose antidiabetic drugs in T2D with CKD DESIGN: Mixed methods. Interviews with providers underwent qualitative analysis using grounded theory to identify themes related to antidiabetic drug prescribing. A provider survey used vignettes and direct questions to quantitatively assess prescribers' knowledge and preferences. A retrospective cohort analysis of real-world prescribing data assessed the external validity of the interview and survey findings.

PARTICIPANTS

Primary care physicians, endocrinologists, nurse-practitioners, and physicians' assistants were eligible for interviews; primary care physicians and endocrinologists were eligible for the survey; prescribing data were derived from adult patients with serum creatinine data.

MAIN MEASURES

Interviews were qualitative; for the survey and retrospective cohort, proportion of patients receiving metformin was the primary outcome.

KEY RESULTS

Interviews with 9 providers identified a theme of uncertainty about guidelines for prescribing antidiabetic drugs in patients with T2D and CKD. The survey had 105 respondents: 74 primary care providers and 31 endocrinologists. Metformin was the most common choice for patients with T2D and CKD. Compared to primary care providers, endocrinologists were less likely to prescribe metformin at levels of kidney function at which it is contraindicated and more likely to correctly answer a question about metformin's contraindications (71% versus 41%) (p < .05). Real-world data were consistent with survey findings, and further showed low rates of use of sodium-glucose cotransporter 2 inhibitors and glucagon-like peptide 1 receptor agonists (<10%) in patients with eGFR below 60 ml/min/1.73m2.

CONCLUSIONS

Providers are unsure how to treat T2D with CKD and incompletely informed as to existing guidelines. This suggests opportunities to improve care.

Patient Preferences and Satisfaction With Decisions in Stage-III Melanoma: A Mixed Methods Study

INTRODUCTION

Rapid accumulation of data in surgical and medical oncology has changed the treatment landscape for patients with stage-III melanoma, introducing options for active surveillance and adjuvant systemic therapy; however, these options have increased the complexity of decision making.

METHODS

We conducted an explanatory sequential mixed-methods study consisting of surveys and semistructured interviews among patients diagnosed with stage-III melanoma at a single institution from August 2019 to December 2021. The survey included the validated 30-point satisfaction with decision scale (SWD). The interview guide was developed using a shared decision-making framework.

RESULTS

Twenty-six participants completed the survey (response rate 40%) and 17 were interviewed. In the survey, 69% of participants reported receiving a recommendation for active surveillance and 23% received a recommendation for adjuvant systemic therapy. Overall SWD for treatment of the lymph node basin and adjuvant systemic therapy was high at 27.94 and 26.21 out of 30, respectively. In the interviews, participants stressed the importance of the physician's recommendation as well as the desire to minimize intervention and avoid potential side effects in their decisions. However, they demonstrated persistent knowledge gaps in their understanding of the treatment options.

CONCLUSIONS

Like other cancer types where the option for active surveillance exists, the physician's recommendation is influential in shaping decisions for patients with stage-III melanoma. Physicians can improve shared decision making in this complex treatment landscape through improved multidisciplinary collaboration and mechanisms for ensuring patients' understanding of the treatment options.

Evaluating Shared Decision-Making in Treatment Selection for Dupuytren Contracture: A Mixed Methods Approach

BACKGROUND

Patients with Dupuytren contracture can receive a variety of surgical and nonsurgical treatments. The extent to which patients participate in the shared decision-making process is unclear.

METHODS

An explanatory-sequential mixed-methods study was conducted. Participants completed the Nine-Item Shared Decision-Making Questionnaire and the brief Michigan Hand Outcomes Questionnaire before completing semi-structured interviews in which they described their experience with selecting treatment.

RESULTS

Thirty participants [25 men (83%) and five women (17%); mean age, 69 years (range, 51 to 84 years)] received treatment for Dupuytren contracture (11 collagenase injection, six needle aponeurotomy, and 13 limited fasciectomy). Adjusted mean scores for the Shared Decision-Making Questionnaire and brief Michigan Hand Outcomes Questionnaire were 71 (SD 20) and 77 (SD 16), respectively, indicating a high degree of shared decision-making and satisfaction. Patients who received limited fasciectomy accepted invasiveness and prolonged recovery time because they believed it provided a long-term solution. Patients chose needle aponeurotomy and collagenase injection because the treatments were perceived as safer and more convenient and permitted rapid return to daily activities, which was particularly valued by patients who were employed or had bilateral contractures.

CONCLUSIONS

Physicians should help patients choose a treatment that aligns with the patient's preferences for long-term versus short-term results, recovery period and postoperative rehabilitation, and risk of complications, because patients used this information to assist in their treatment selection. Areas of improvement for shared decision-making include equal presentation of all treatments and ensuring realistic patient expectations regarding the chronic and recurrent nature of Dupuytren contracture regardless of treatment received.

Experiences of children with central venous access devices: a mixed-methods study

BACKGROUND

Our study aims to explore the experience of having a central venous access device (CVAD) from the perspective of the child and family and how movements within and outside of hospital environments influence this experience.

METHODS

A mixed-methods study was conducted across Children's Health Queensland (Australia), including inpatient and home-care settings. Children less than 18 years with CVADs were eligible and followed for 3 months or CVAD removal. A subgroup of primary caregivers participated in semi-structured interviews. Quantitative and qualitative measures of child and family CVAD experiences were explored.

RESULTS

In total, 163 patients with 200 CVADs were recruited and followed for 6993 catheter days (3329 [48%] inpatients; 3147 [45%] outpatients; 517 [7%] home). Seventeen participants were interviewed. Experiences of having a CVAD were complex but predominantly positive primarily related to personalized CVAD care, healthcare quality, and general wellbeing. Their experience was shaped by their movements through hospital and home environments, including care variation and distress with procedures. Device selection and insertion location further influenced experience, including safety, impairments in activities of daily living, school, and recreation.

CONCLUSIONS

CVAD experiences were influenced by nonmodifiable (e.g., diagnosis) and modifiable factors (e.g., education; care variation). Clinical approaches and policies that account for family and child considerations should be explored.

IMPACT

Variation in decision making and management for pediatric CVADs is accepted by many clinicians, but the influence this variation has on the health experience of children and their families is less well explored. This is the first study to draw from a broad range of children requiring CVADs to determine their experience within and outside of healthcare facilities. Interdisciplinary clinicians and researchers need to work collaboratively with children and their families to provide resources and support services to ensure they have positive experiences with CVADs, no matter where they are managed, or who they are managed by.

Improving Plastic Dressing Clinic Burden by Reducing Punch Biopsy Referrals Using a Patient Information Leaflet: A Quality Improvement Project

Background

Although being a safe intervention with low complication rates, a significant proportion of patients undergoing punch biopsies were referred to the local plastic dressing clinic (PDC) despite providing minimal utility and adding to an already heavily burdened clinic. The aim of this quality improvement (QI) project was to reduce the number of patients who had undergone a punch biopsy who were referred to PDC. The steering group included a plastic surgeon consultant, a senior outpatient department (OPD) and a PDC nurse, and a junior trainee. Process mapping and driver diagrams were used to identify a patient information leaflet as the intervention.

Methods

A patient information leaflet was created by the clinical lead, which included advice for postoperative wound care and when to seek medical attention. This leaflet was provided to every patient who underwent a minor operation for a skin lesion from September 2021. Operative notes of patients who underwent a punch biopsy were reviewed from October to November 2021 and from August to September 2022 to identify the number of patients who were referred to PDC. Concurrently informal feedback from patients and the process manager guided any necessary changes.

Results

There was a small improvement in the number of PDC referrals during October-November 2021 (46%) compared to the baseline measurements (54%). This corresponded with an absolute risk reduction (ARR) of 7%, risk ratio (RR) of 0.86, and relative risk reduction (RRR) of 14%. No changes were made at this stage. There was a much greater improvement in August-September 2022 (20%) compared to the baseline measurements with 34% ARR, 0.37 RR, and 63% RRR. Informal feedback from the process manager confirmed that every patient received a leaflet and that the patients found the leaflet very helpful.

Conclusions

The implementation of the patient information leaflet has led to a reduction in referrals of patients with punch biopsies to PDC. While this is a small and narrow project, it demonstrates the values of QI methodology to select a successful intervention. Furthermore, it corroborates with other studies that have shown that patient information leaflets empower patients and can help reduce the National Health Service (NHS) outpatient burden. Considering its simplicity, trialing a similar approach in other specialties is strongly encouraged.

Prevalence of myalgic encephalomyelitis/chronic fatigue syndrome (ME/CFS) in Australian primary care patients: only part of the story?

Background

ME/CFS is a disorder characterized by recurrent fatigue and intolerance to exertion which manifests as profound post-exertional malaise. Prevalence studies internationally have reported highly variable results due to the 20 + diagnostic criteria. For Australia, the prevalence of ME/CFS based on current case definitions is unknown.

Objectives

To report prevalence of ME/CFS in patients aged ≥ 13 years attending Australian primary care settings for years 2015–2019, and provide context for patterns of primary care attendance by people living with ME/CFS.

Methodology

Conducted in partnership with the Patient Advisory Group, this study adopted a mixed methods approach. De-identified primary care data from the national MedicineInsight program were analyzed. The cohort were regularly attending patients, i.e. 3 visits in the preceding 2 years. Crude prevalence rates were calculated for years 2015–2019, by sex, 10-year age groups, remoteness and socioeconomic status. Rates are presented per 100,000population (95% confidence intervals (CI)). Qualitative data was collected through focus groups and in-depth 1:1 interview.

Results

Qualitative evidence identified barriers to reaching diagnosis, and limited interactions with primary care due to a lack of available treatments/interventions, stigma and disbelief in ME/CFS as a condition.

In each year of interest, crude prevalence in the primary care setting ranged between 94.9/100,000 (95% CI: 91.5–98.5) and 103.9/100,000 population (95%CI: 100.3–107.7), equating to between 20,140 and 22,050 people living with ME/CFS in Australia in 2020. Higher rates were observed for age groups 50-59 years and 40-49 years. Rates were substantially higher in females (130.0–141.4/100,000) compared to males (50.9–57.5/100,000). In the context of the qualitative evidence, our prevalence rates likely represent an underestimate of the true prevalence of ME/CFS in the Australian primary care setting.

Conclusion

ME/CFS affects a substantial number of Australians. Whilst this study provides prevalence estimates for the Australian primary care setting, the qualitative evidence highlights the limitations of these. Future research should focus on using robust case ascertainment criteria in a community setting. Quantification of the burden of disease can be used to inform health policy and planning, for this understudied condition.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12889-022-13929-9.

Surgeon Variation in the Application of Robotic Technique for Abdominal Hernia Repair: A Mixed-Methods Study

INTRODUCTION

Although the utilization of robotic technique for abdominal hernia repair has increased rapidly, there is no consensus as to when it should be applied for optimal outcomes. High variability exists within surgeon practices regarding how they use this technology, and the factors that drive robotic utilization remain largely unknown. This study aims to explore the motivating factors associated with surgeons' decisions to utilize a robotic approach for abdominal hernia repair.

METHODS

An exploratory mixed-methods approach was utilized. Surgeons who performed abdominal hernia repairs were interviewed to identify impactful themes motivating surgical approach. This informed a retrospective analysis of ventral hernia repairs performed in 2020 within the Michigan Surgical Quality Collaborative. Surgeon robotic utilization rates were calculated. Among selective robotic users, multivariable regression evaluated the patient and hernia factors associated with robotic utilization.

RESULTS

Qualitative analysis of 21 interviews revealed three dominant themes in the decision to utilize robotic technology: access and resources, surgeon comfort, and market factors. Among 71 surgeons caring for 1174 hernia patients, robotic utilization rates ranged from 0% to 98% of cases. There were 27 surgeons identified as selective robotic users, who cared for 423 patients. Multivariable regression revealed that hernia location was the only factor associated with robotic technique, with non-midline hernias associated with a 4.47 (95% confidence interval 1.34-14.88) higher odds of robotic repair than epigastric hernias.

CONCLUSIONS

Major drivers of robotic technique for hernia repair were found to be perceived benefits and availability, rather than patient or hernia characteristics. These data will contribute to an understanding of surgeon decision-making and help develop improvements to patient care.

Dissatisfaction After Post-Mastectomy Breast Reconstruction: A Mixed-Methods Study

BACKGROUND

Breast reconstruction is associated with improved patient well-being after mastectomy; however, factors that contribute to post-surgical dissatisfaction remain poorly characterized.

METHODS

Adult women who underwent post-mastectomy implant-based or autologous breast reconstruction between 2015 and 2019 were recruited to participate in semi-structured interviews regarding their lived experiences with reconstructive care. Participants completed the BREAST-Q, and tabulated scores were used to dichotomize patient-reported outcomes as satisfied or dissatisfied (high or low) for each BREAST-Q domain. A convergent mixed-methods analysis was used to evaluate interviews for content related to satisfaction or dissatisfaction with breast reconstruction.

RESULTS

Overall, we interviewed 21 women and identified 17 subcodes that corresponded with the five BREAST-Q domains. Sources of dissatisfaction were found to be related to the following domains: (a) low breast satisfaction due to asymmetry, cup size, and lack of sensation and physical feeling (n = 8, 38%); (b) poor sexual well-being due to shape, look and feel (n = 7, 78% [of 9 who discussed sexual well-being]); (c) reduced physical well-being of the chest due to persistent pain and weakness (n = 11, 52%); (d) reduced abdominal well-being due to changes in abdominal strength, numbness, and posture (n = 6, 38% [of 16 who underwent abdominally based reconstruction]); and (e) low psychosocial well-being impacted by an unexpected appearance that negatively influenced self-confidence and self-identity (n = 13, 62%).

CONCLUSION

Patients may be unprepared for the physical, sexual, and psychosocial outcomes of breast reconstruction. Targeted strategies to improve preoperative education and shared decision making are needed to mitigate unexpectedness associated with breast reconstruction and related outcomes.

Factors Associated With Age of Presentation to Gender-Affirming Medical Care

BACKGROUND

Gender-incongruent youth may present to gender-affirming medical care (GAMC) later in adolescence and puberty when hormone blockers provide less benefit. Factors influencing age of presentation to GAMC have not been described.

METHODS

A sequential mixed methods study. Participants were categorized on the basis of age at presentation to GAMC. Youth presenting at ≥15 years comprised the older-presenting youth, whereas those presenting at <15 years comprised the younger-presenting youth. Caregivers were categorized on the basis of the youth's age of presentation. Twenty-four individuals were interviewed, 6 youth and 6 caregivers from each age category. Thematic analysis identified themes related to timing of presentation to GAMC. Themes differentially endorsed between older and younger youth or between caregivers of older and younger youth were used to design a questionnaire distributed to 193 youths and 187 caregivers. Responses were compared between age groups for youths and caregivers.

RESULTS

Five themes differed between age groups: validity of gender identity, gender journey barriers, influential networks, perceptions of medical therapy, and health care system interactions. Questionnaires were completed by 121 youths and 121 caregivers. Compared with younger-presenting youth, older-presenting youth recognized gender incongruence at older ages, were less likely to have caregivers who helped them access care or LGBTQ+ (lesbian, gay, bisexual, transgender, queer) family members, more often endorsed familial religious affiliations, and experienced greater youth-caregiver disagreement around importance of GAMC.

CONCLUSIONS

Family environment appears to be a key determinant of when youth present to GAMC. Whether this association occurs through affecting transgender identity formation and recognition requires further study.

Trends in Breast Cancer Treatment De-Implementation in Older Patients with Hormone Receptor-Positive Breast Cancer: A Mixed Methods Study

INTRODUCTION

Guidelines allow for the omission of sentinel lymph node biopsy (SLNB) and post-lumpectomy radiotherapy in women ≥ 70 years of age with hormone receptor-positive (HR +) breast cancer. Despite this, national data suggest these procedures have not been widely de-implemented.

OBJECTIVES

Our objectives were to evaluate trends in SLNB and post-lumpectomy radiotherapy utilization in patients who are eligible for omission, and evaluate patient preferences as a target for de-implementation of low-value care.

METHODS

We performed a sequential explanatory mixed-methods study by first analyzing an institutional database of patients ≥ 70 years of age with HR + breast cancer who received surgical treatment from 2014 to 2018. Based on the quantitative data, we conducted semi-structured interviews with women identified as high or low utilizers of breast cancer treatments to elicit patient perspectives on de-implementation.

RESULTS

SLNB and post-lumpectomy radiotherapy were performed in 68% and 43% of patients, respectively, who met the criteria for omission. There was a significant decrease in SLNB rates from 2014 to 2018. Forty-nine percent of patients were classified as high utilizers and 26% were classified as low utilizers. Qualitative analysis found that the most important factors influencing decision making regarding SLNB and post-lumpectomy radiotherapy omission for both high and low utilizers were trust in their provider and a desire for peace of mind.

CONCLUSIONS

Despite efforts to de-implement low-value care, older women with HR + breast cancer remain at risk of overtreatment. Patient perspectives suggest that multi-level de-implementation strategies will need to target provider practice patterns and patient-provider communication to promote high-quality decision making and reduction in breast cancer overtreatment.

Exploration of Factors Associated With Surgeon Deviation From Practice Guidelines for Management of Inguinal Hernias

IMPORTANCE

Despite availability of evidence-based guidelines for surgery, many patients receive guideline-discordant care. Reasons for this are largely unknown. For example, evidence-based guidelines recommend a minimally invasive approach for persons with bilateral or recurrent unilateral inguinal hernias. Benefits are also noted for primary unilateral inguinal hernia. However, findings from previous quantitative research indicate that only 26% of patients receive this treatment and only 42% of surgeons offer a minimally invasive approach, even for recurrent or bilateral hernias.

OBJECTIVE

To explore factors associated with surgeon choice of approach (minimally invasive vs open) in inguinal hernia repair as a tool to gain an understanding of guideline-discordant care.

DESIGN, SETTING, AND PARTICIPANTS

Qualitative study performed as part of a larger explanatory sequential mixed methods design. Purposive sampling was used to recruit 21 practicing surgeons from a large statewide quality collaborative who were diverse with regard to practice type, geographic location, and surgical specialty. Qualitative interviews consisted of a clinical vignette, followed by semi-structured interview questions. Through thematic analysis using qualitive data analysis software, patterns within the data were located, analyzed, and identified. All data were collected between April 24 and July 31, 2018.

EXPOSURE

Clinical vignette as part of the qualitative interviews.

MAIN OUTCOMES AND MEASURES

Capture of surgical approaches and factors motivating decision-making for inguinal hernia repair.

RESULTS

Of the 21 participating surgeons, 17 (81%) were men, 18 (86%) were White, and all were 35 years of age or older. Data revealed 3 dominant themes: surgeon preference and autonomy (eg, favoring one approach over the other), access and resources (eg, availability of robot), and patient characteristics (eg, age, comorbidities).

CONCLUSIONS AND RELEVANCE

Decision-making for the approach to inguinal hernia repair is largely influenced by surgeon preference and access to resources rather than patient factors. Although a one-size-fits-all approach is not recommended, the operative approach should ideally be informed by patient factors, including hernia characteristics. Addressing surgeon preference and available resources with a clinician-facing decision aid may provide an opportunity to optimize care for patients undergoing inguinal hernia repair.

Convergent Mixed Methods Exploration of Telehealth in Bariatric Surgery: Maximizing Provider Resources and Access

Background

Telehealth may be an important care delivery modality in reducing dropout from bariatric surgery programs which is reported globally at approximately 50%.

Methods

In this convergent mixed methods case study of a large, US healthcare system, we examine the impact of telehealth implementation in 2020 on pre-operative bariatric surgery visits and provider perspectives of telehealth use.

Results

We find that telehealth was significantly associated with a 38% reduction in no-show rate compared with the prior year. Additionally, providers had positive experiences with regard to the appropriateness and feasibility of using telehealth in the pre-operative bariatric surgery process.

Conclusions

Telehealth use in the pre-operative bariatric surgery process may lead to greater efficiency in healthcare resource utilization. Insurance providers and bariatric accreditation bodies globally should consider accepting telehealth visits and self-reported weights when determining coverage decisions to ensure access for patients.

Laying the Foundation for Bladder Health Promotion in Women and Girls

Prevention strategies have been effective in many areas of human health, yet have not been utilized for lower urinary tract symptoms (LUTS) or bladder health (BH). This commentary outlines LUTS prevention research initiatives underway within the NIH-sponsored Prevention of Lower Urinary Tract Symptoms Research Consortium (PLUS). Prevention science involves the systematic study of factors associated with health and health problems, termed protective and risk factors, respectively. PLUS is enhancing traditional prevention science approaches through use of: (1) a transdisciplinary team science approach, (2) both qualitative and quantitative research methodology (mixed methodology), and (3) community engagement. Important foundational work of PLUS includes development of clear definitions of both BH and disease, as well as a BH measurement instrument that will be validated for use in the general population, adolescents, and Latinx and Spanish-speaking women.¹ The BH measurement instrument will be used in an upcoming nationally-representative cohort study that will measure BH and investigate risk and protective factors. PLUS investigators also developed a conceptual framework to guide their research agenda; this framework organizes a broad array of candidate risk and protective factors that can be studied across the life course of girls and women.¹ As PLUS begins to fill existing knowledge gaps with new information, its efforts will undoubtedly be complemented by outside investigators to further advance the science of LUTS prevention and BH across additional populations. Once the BH community has broadened its understanding of modifiable risk and protective factors, intervention studies will be necessary to test LUTS prevention strategies and support public health efforts. LUTS providers may be able to translate this evolving evidence for individual patients under their care and act as BH advocates in their local communities.