Implications of cognitive load for hypothesis generation and probability judgment

Data-Driven Hypothesis Generation in Clinical Research: What We Learned from a Human Subject Study?

Hypothesis generation is an early and critical step in any hypothesis-driven clinical research project. Because it is not yet a well-understood cognitive process, the need to improve the process goes unrecognized. Without an impactful hypothesis, the significance of any research project can be questionable, regardless of the rigor or diligence applied in other steps of the study, e.g., study design, data collection, and result analysis. In this perspective article, the authors provide a literature review on the following topics first: scientific thinking, reasoning, medical reasoning, literature-based discovery, and a field study to explore scientific thinking and discovery. Over the years, scientific thinking has shown excellent progress in cognitive science and its applied areas: education, medicine, and biomedical research. However, a review of the literature reveals the lack of original studies on hypothesis generation in clinical research. The authors then summarize their first human participant study exploring data-driven hypothesis generation by clinical researchers in a simulated setting. The results indicate that a secondary data analytical tool, VIADS-a visual interactive analytic tool for filtering, summarizing, and visualizing large health data sets coded with hierarchical terminologies, can shorten the time participants need, on average, to generate a hypothesis and also requires fewer cognitive events to generate each hypothesis. As a counterpoint, this exploration also indicates that the quality ratings of the hypotheses thus generated carry significantly lower ratings for feasibility when applying VIADS. Despite its small scale, the study confirmed the feasibility of conducting a human participant study directly to explore the hypothesis generation process in clinical research. This study provides supporting evidence to conduct a larger-scale study with a specifically designed tool to facilitate the hypothesis-generation process among inexperienced clinical researchers. A larger study could provide generalizable evidence, which in turn can potentially improve clinical research productivity and overall clinical research enterprise.

Data-driven hypothesis generation among inexperienced clinical researchers: A comparison of secondary data analyses with visualization (VIADS) and other tools

Objectives

To compare how clinical researchers generate data-driven hypotheses with a visual interactive analytic tool (VIADS, a visual interactive analysis tool for filtering and summarizing large datasets coded with hierarchical terminologies) or other tools.

Methods

We recruited clinical researchers and separated them into "experienced" and "inexperienced" groups. Participants were randomly assigned to a VIADS or control group within the groups. Each participant conducted a remote 2-hour study session for hypothesis generation with the same study facilitator on the same datasets by following a think-aloud protocol. Screen activities and audio were recorded, transcribed, coded, and analyzed. Hypotheses were evaluated by seven experts on their validity, significance, and feasibility. We conducted multilevel random effect modeling for statistical tests.

Results

Eighteen participants generated 227 hypotheses, of which 147 (65%) were valid. The VIADS and control groups generated a similar number of hypotheses. The VIADS group took a significantly shorter time to generate one hypothesis (e.g., among inexperienced clinical researchers, 258 s versus 379 s, p = 0.046, power = 0.437, ICC = 0.15). The VIADS group received significantly lower ratings than the control group on feasibility and the combination rating of validity, significance, and feasibility.

Conclusion

The role of VIADS in hypothesis generation seems inconclusive. The VIADS group took a significantly shorter time to generate each hypothesis. However, the combined validity, significance, and feasibility ratings of their hypotheses were significantly lower. Further characterization of hypotheses, including specifics on how they might be improved, could guide future tool development.

Data-driven hypothesis generation among inexperienced clinical researchers: A comparison of secondary data analyses with visualization (VIADS) and other tools

Objectives

To compare how clinical researchers generate data-driven hypotheses with a visual interactive analytic tool (VIADS, a visual interactive analysis tool for filtering and summarizing large data sets coded with hierarchical terminologies) or other tools.

Methods

We recruited clinical researchers and separated them into "experienced" and "inexperienced" groups. Participants were randomly assigned to a VIADS or control group within the groups. Each participant conducted a remote 2-hour study session for hypothesis generation with the same study facilitator on the same datasets by following a think-aloud protocol. Screen activities and audio were recorded, transcribed, coded, and analyzed. Hypotheses were evaluated by seven experts on their validity, significance, and feasibility. We conducted multilevel random effect modeling for statistical tests.

Results

Eighteen participants generated 227 hypotheses, of which 147 (65%) were valid. The VIADS and control groups generated a similar number of hypotheses. The VIADS group took a significantly shorter time to generate one hypothesis (e.g., among inexperienced clinical researchers, 258 seconds versus 379 seconds, p = 0.046, power = 0.437, ICC = 0.15). The VIADS group received significantly lower ratings than the control group on feasibility and the combination rating of validity, significance, and feasibility.

Conclusion

The role of VIADS in hypothesis generation seems inconclusive. The VIADS group took a significantly shorter time to generate each hypothesis. However, the combined validity, significance, and feasibility ratings of their hypotheses were significantly lower. Further characterization of hypotheses, including specifics on how they might be improved, could guide future tool development.

How do clinical researchers generate data-driven scientific hypotheses? Cognitive events using think-aloud protocol

Objectives

This study aims to identify the cognitive events related to information use (e.g., "Analyze data", "Seek connection") during hypothesis generation among clinical researchers. Specifically, we describe hypothesis generation using cognitive event counts and compare them between groups.

Methods

The participants used the same datasets, followed the same scripts, used VIADS (a visual interactive analysis tool for filtering and summarizing large data sets coded with hierarchical terminologies) or other analytical tools (as control) to analyze the datasets, and came up with hypotheses while following the think-aloud protocol. Their screen activities and audio were recorded and then transcribed and coded for cognitive events.

Results

The VIADS group exhibited the lowest mean number of cognitive events per hypothesis and the smallest standard deviation. The experienced clinical researchers had approximately 10% more valid hypotheses than the inexperienced group. The VIADS users among the inexperienced clinical researchers exhibit a similar trend as the experienced clinical researchers in terms of the number of cognitive events and their respective percentages out of all the cognitive events. The highest percentages of cognitive events in hypothesis generation were "Using analysis results" (30%) and "Seeking connections" (23%).

Conclusion

VIADS helped inexperienced clinical researchers use fewer cognitive events to generate hypotheses than the control group. This suggests that VIADS may guide participants to be more structured during hypothesis generation compared with the control group. The results provide evidence to explain the shorter average time needed by the VIADS group in generating each hypothesis.

Visitor arrivals forecasts amid COVID-19: A perspective from the Africa team

COVID-19 disrupted international tourism worldwide, subsequently presenting forecasters with a challenging conundrum. In this competition, we predict international arrivals for 20 destinations in two phases: (i) Ex post forecasts pre-COVID; (ii) Ex ante forecasts during and after the pandemic up to end 2021. Our results show that univariate combined with cross-sectional hierarchical forecasting techniques (THieF-ETS) outperform multivariate models pre-COVID. Scenarios were developed based on judgemental adjustment of the THieF-ETS baseline forecasts. Analysts provided a regional view on the most likely path to normal, based on country-specific regulations, macroeconomic conditions, seasonal factors and vaccine development. Results show an average recovery of 58% compared to 2019 tourist arrivals in the 20 destinations under the medium scenario; severe, it is 34% and mild, 80%.

Acute Effect of Cognitive Compromise during Physical Exercise on Self-Regulation in Early Childhood Education

Self-regulation (SR) in pre-schoolers is a strong predictor of different aspects of mental health and wellbeing. However, SR only recently has been examined concerning physical activity and its effects on cognitive performance. In the present study, 49 preschool children aged 4-5 years were submitted to classroom movement breaks (CMBs) of 15-min with different degrees of difficulty. Before beginning the intervention, SR (i.e., head, toes, knees and shoulders test, HTKS) and skill levels were assessed for tasks demand adjustment to individual resources and the counterbalanced assignment of the participants to the groups. Similarly, after the intervention, the performance on the HTKS was re-evaluated. There was a general intervention effect on the SR of pre-schoolers, regardless of the difficulty level of the task [F (3) = 11.683, p-value < 0.001, η2p = 0.438]. Nevertheless, it seems that only when CMBs stimulate the children cognitively with optimal difficulty, is it possible to obtain benefits. We recommend providing teachers with professional support when implementing physical activity breaks in their daily program to generate an individualized level of cognitive load that would allow children to reach the optimal challenge point.

Where do hypotheses come from?

Why are human inferences sometimes remarkably close to the Bayesian ideal and other times systematically biased? In particular, why do humans make near-rational inferences in some natural domains where the candidate hypotheses are explicitly available, whereas tasks in similar domains requiring the self-generation of hypotheses produce systematic deviations from rational inference. We propose that these deviations arise from algorithmic processes approximating Bayes' rule. Specifically in our account, hypotheses are generated stochastically from a sampling process, such that the sampled hypotheses form a Monte Carlo approximation of the posterior. While this approximation will converge to the true posterior in the limit of infinite samples, we take a small number of samples as we expect that the number of samples humans take is limited. We show that this model recreates several well-documented experimental findings such as anchoring and adjustment, subadditivity, superadditivity, the crowd within as well as the self-generation effect, the weak evidence, and the dud alternative effects. We confirm the model's prediction that superadditivity and subadditivity can be induced within the same paradigm by manipulating the unpacking and typicality of hypotheses. We also partially confirm our model's prediction about the effect of time pressure and cognitive load on these effects.

Using a model of hypothesis generation to predict eye movements in a visual search task

We used a model of hypothesis generation (called HyGene; Thomas, Dougherty, Sprenger, & Harbison, 2008) to make predictions regarding the deployment of attention (as assessed via eye movements) afforded by the cued recall of target characteristics before the onset of a search array. On each trial, while being eyetracked, participants were first presented with a memory prompt that was diagnostic regarding the target's color in a subsequently presented search array. We assume that the memory prompts led to the generation of hypotheses (i.e., potential target characteristics) from long-term memory into working memory to guide attentional processes and ocular-motor behavior. However, given that multiple hypotheses might be generated in response to a prompt, it has been unclear how the focal hypothesis (i.e., the hypothesis that exerts the most influence on search) affects search behavior. We tested two possibilities using first fixation data, with the assumption that the first item fixated within a search array was the focal hypothesis. We found that a model assuming that the first item generated into working memory guides overt attentional processes was most consistent with the data at both the aggregate and single-participant levels of analysis.

Evaluating the role of attention in the context of unconscious thought theory: differential impact of attentional scope and load on preference and memory

Attention is a key process used to conceptualize and define modes of thought, but we lack information about the role of specific attentional processes on preferential choice and memory in multi-attribute decision making. In this study, we examine the role of attention based on two dimensions, attentional scope and load on choice preference strength and memory using a paradigm that arguably elicits unconscious thought. Scope of attention was manipulated by using global or local processing during distraction (Experiment 1) and before the information-encoding stage (Experiment 2). Load was manipulated by using the n-back task in Experiment 1. Results from Experiment 1 show that global processing or distributed attention during distraction results in stronger preference irrespective of load but better memory only at low cognitive load. Task difficulty or load did not have any effect on preference or memory. In Experiment 2, distributed attention before attribute encoding facilitated only memory but did not influence preference. Results show that attentional processes at different stages of processing like distraction and information-encoding influence decision making processes. Scope of attention not only influences preference and memory but the manner in which attentional scope influences them depends on both load and stage of information processing. The results indicate the important role of attention in processes critical for decision making and calls for a re-evaluation of the unconscious thought theory (UTT) and the need for reconceptualizing the role of attention.

Temporal dynamics of hypothesis generation: the influences of data serial order, data consistency, and elicitation timing

The pre-decisional process of hypothesis generation is a ubiquitous cognitive faculty that we continually employ in an effort to understand our environment and thereby support appropriate judgments and decisions. Although we are beginning to understand the fundamental processes underlying hypothesis generation, little is known about how various temporal dynamics, inherent in real world generation tasks, influence the retrieval of hypotheses from long-term memory. This paper presents two experiments investigating three data acquisition dynamics in a simulated medical diagnosis task. The results indicate that the mere serial order of data, data consistency (with previously generated hypotheses), and mode of responding influence the hypothesis generation process. An extension of the HyGene computational model endowed with dynamic data acquisition processes is forwarded and explored to provide an account of the present data.