Control selection methods in recent case-control studies conducted as part of infectious disease outbreaks

Lessons Learned From a Sequential Mixed-Mode Survey Design to Recruit and Collect Data From Case-Control Study Participants: Formative Evaluation

BACKGROUND

Sequential mixed-mode surveys using both web-based surveys and telephone interviews are increasingly being used in observational studies and have been shown to have many benefits; however, the application of this survey design has not been evaluated in the context of epidemiological case-control studies.

OBJECTIVE

In this paper, we discuss the challenges, benefits, and limitations of using a sequential mixed-mode survey design for a case-control study assessing risk factors during the COVID-19 pandemic.

METHODS

Colorado adults testing positive for SARS-CoV-2 were randomly selected and matched to those with a negative SARS-CoV-2 test result from March to April 2021. Participants were first contacted by SMS text message to complete a self-administered web-based survey asking about community exposures and behaviors. Those who did not respond were contacted for a telephone interview. We evaluated the representativeness of survey participants to sample populations and compared sociodemographic characteristics, participant responses, and time and resource requirements by survey mode using descriptive statistics and logistic regression models.

RESULTS

Of enrolled case and control participants, most were interviewed by telephone (308/537, 57.4% and 342/648, 52.8%, respectively), with overall enrollment more than doubling after interviewers called nonresponders. Participants identifying as female or White non-Hispanic, residing in urban areas, and not working outside the home were more likely to complete the web-based survey. Telephone participants were more likely than web-based participants to be aged 18-39 years or 60 years and older and reside in areas with lower levels of education, more linguistic isolation, lower income, and more people of color. While there were statistically significant sociodemographic differences noted between web-based and telephone case and control participants and their respective sample pools, participants were more similar to sample pools when web-based and telephone responses were combined. Web-based participants were less likely to report close contact with an individual with COVID-19 (odds ratio [OR] 0.70, 95% CI 0.53-0.94) but more likely to report community exposures, including visiting a grocery store or retail shop (OR 1.55, 95% CI 1.13-2.12), restaurant or cafe or coffee shop (OR 1.52, 95% CI 1.20-1.92), attending a gathering (OR 1.69, 95% CI 1.34-2.15), or sport or sporting event (OR 1.05, 95% CI 1.05-1.88). The web-based survey required an average of 0.03 (SD 0) person-hours per enrolled participant and US $920 in resources, whereas the telephone interview required an average of 5.11 person-hours per enrolled participant and US $70,000 in interviewer wages.

CONCLUSIONS

While we still encountered control recruitment challenges noted in other observational studies, the sequential mixed-mode design was an efficient method for recruiting a more representative group of participants for a case-control study with limited impact on data quality and should be considered during public health emergencies when timely and accurate exposure information is needed to inform control measures.

Use of a non-probabilistic online panel as a control group for case-control studies to investigate food and waterborne outbreaks in Lower Saxony, Germany

Established methods of recruiting population controls for case–control studies in infectious disease outbreak investigations are resource- and time-intensive, and are often subject to bias. The online panel have recently gained interest as an easy and timely method to select controls. We examined the feasibility, suitability and reliability of using an online panel to select controls for case–control studies as part of investigations of diffuse food and waterborne outbreaks. In January 2019, we deployed a web survey by email to the 277 members of a non-probabilistic online panel in Lower Saxony, Germany. We questioned them on basic sociodemographic characteristics and eating habits. They were frequency matched to cases on sex and age. Their food exposures were compared to those of traditionally recruited controls of four historical case–controls studies, which successfully investigated food and waterborne outbreaks. We used logistic regressions to assess the association between the food exposures and the disease (odds ratios). The use of a control panel successfully led to the identification of the food items in three of the four historical outbreak investigations, and their recruitment benefitted from increased speed and limited costs. Timely outbreak investigations would enable rapidly implementing control measures. We recommend the further evaluation of using panellists as controls in parallel case–control studies and case–panel studies.

Impact of Best-Fitted Control Selection on Effect Size: An Example in Functional GI Disorder Case-Control Studies

BACKGROUND

Effect sizes are the most useful quantities for communicating the practical significance of results and helping to facilitate cumulative science. We hypothesize that the selection of the best-fitted controls can significantly affect the estimated effect sizes in case-control studies. Therefore, we decided to exemplify and clarify this effect on effect size using a large data set. The objective of this study was to investigate the association among variables in functional gastrointestinal disorders (FGIDs) and mental health problems, common ailments that reduce the quality of life of a large proportion of the community worldwide.

METHOD

In this methodological study, we constitute case and control groups in our study framework using the Epidemiology of Psychological, Alimentary Health and Nutrition (SEPAHAN) dataset of 4763 participants. We devised four definitions for control in this extensive database of FGID patients and analyzed the effect of these definitions on the odds ratio (OR): 1. conventional control: without target disorder/syndrome (sample size 4040); 2. without any positive criteria: criterion-free control (sample size 1053); 3. syndrome-free control: without any disorder/syndrome (sample size 847); 4. symptom-free control: without any symptoms (sample size 204). We considered a fixed case group that included 723 patients with a Rome III-based definition of functional dyspepsia. Psychological distress, anxiety, and depression were considered as dependent variables in the analysis. Logistic regression was used for association analysis, and the odds ratio and 95% confidence interval (95%CI) for OR were reported as the effect size.

RESULTS

The estimated ORs indicate that the strength of the association in the first case-control group is the lowest, and the fourth case-control group, including controls with completely asymptomatic people, is the highest. Ascending effect sizes were obtained in the conventional, criterion-free, syndrome-free, and symptom-free control groups. These results are consistent for all three psychological disorders, psychological distress, anxiety, and depression.

CONCLUSIONS

This study shows that a precise definition of the control is mandatory in every case-control study and affects the estimated effect size. In clinical settings, the selection of symptomatic controls using the conventional definition could significantly diminish the effect size.

Development, content validation, and reliability of the Assessment of Real-World Observational Studies (ArRoWS) critical appraisal tool

PURPOSE

The objective was to develop and test a pragmatic critical appraisal tool, the Assessment of Real-World Observational Studies (ArRoWS), to quickly and easily assess the quality of real-world evidence studies using electronic health records.

METHODS

The initial ArRoWS tool was developed by identifying items frequently found in existing validated assessment instruments and adapting these items to specifically assess real-world evidence studies. The tool was revised based on recommendations from an expert panel of 14 senior academic individuals specializing in epidemiology and content validity was measured. During March 2018-January 2019, 47 large, observational studies related to cardiometabolic medicine were identified through a search algorithm and assessed by three pairs of raters using the ArRoWS tool.

RESULTS

The final version of the ArRoWS had 16 items including nine core items and seven study design-specific items with item-specific content validity indexes ranging from 0.64 to 1.00. The scale-level content validity index of the ArRoWS appraisal tool was 0.91. When the ArRoWS tool was pilot tested, the observed agreement between assessor pairs on whether the study provided high-quality real-world evidence was 85.7%, 68.8%, and 58.8%. The prevalence adjusted bias-adjusted kappa for the assessor pairs was 0.71, 0.38, and 0.18.

CONCLUSION

The ArRoWS is a simple tool to standardize the assessment of real-world evidence studies.

Online population control surveys: A new method for investigating foodborne outbreaks

In foodborne outbreak investigations, case-control and cohort studies are used to test hypotheses and identify a source, but these studies are resource-intensive and may have challenges of representativeness, temporality or accessibility. We used online surveys to collect population control data for two foodborne outbreaks and compared the data collected to our cases and existing population exposure data. Online survey population controls were comparable to cases based on age and sex. Exposure data collected through online surveys were more precise than existing control data, represented the disease-specific exposure period and could be easily modified. In one outbreak the online control exposure data differed from established population data. In both outbreaks, the information from the online population control survey supported the hypothesis of the investigation. Our findings demonstrate that online surveys were a rapid and representative way to collect responses from controls during outbreak investigations.

Are food exposures obtained through commercial market panels representative of the general population? Implications for outbreak investigations

Current methods of control recruitment for case-control studies can be slow (a particular issue for outbreak investigations), resource-intensive and subject to a range of biases. Commercial market panels are a potential source of rapidly recruited controls. Our study evaluated food exposure data from these panel controls, compared with an established reference dataset. Market panel data were collected from two companies using retrospective internet-based surveys; these were compared with reference data from the National Diet and Nutrition Survey (NDNS). We used logistic regression to calculate adjusted odds ratios to compare exposure to each of the 71 food items between the market panel and NDNS participants. We compared 2103 panel controls with 2696 reference participants. Adjusted for socio-demographic factors, exposure to 90% of foods was statistically different between both panels and the reference data. However, these differences were likely to be of limited practical importance for 89% of Panel A foods and 79% of Panel B foods. Market panel food exposures were comparable with reference data for common food exposures but more likely to be different for uncommon exposures. This approach should be considered for outbreak investigation, in conjunction with other considerations such as population at risk, timeliness of response and study resources.

Online market research panel members as controls in case-control studies to investigate gastrointestinal disease outbreaks: early experiences and lessons learnt from the UK

Established methods of recruiting population controls for case–control studies to investigate gastrointestinal disease outbreaks can be time consuming, resulting in delays in identifying the source or vehicle of infection. After an initial evaluation of using online market research panel members as controls in a case–control study to investigate a Salmonella outbreak in 2013, this method was applied in four further studies in the UK between 2014 and 2016. We used data from all five studies and interviews with members of each outbreak control team and market research panel provider to review operational issues, evaluate risk of bias in this approach and consider methods to reduce confounding and bias. The investigators of each outbreak reported likely time and cost savings from using market research controls. There were systematic differences between case and control groups in some studies but no evidence that conclusions on the likely source or vehicle of infection were incorrect. Potential selection biases introduced by using this sampling frame and the low response rate are unclear. Methods that might reduce confounding and some bias should be balanced with concerns for overmatching. Further evaluation of this approach using comparisons with traditional methods and population-based exposure survey data is recommended.