Authors' Reply to: VanderWeele et al., Chiolero, and Schooling et al

From ideas to studies: how to get ideas and sharpen them into research questions

Where do new research questions come from? This is at best only partially taught in courses or textbooks about clinical or epidemiological research. Methods are taught under the assumption that a researcher already knows the research question and knows which methods will fit that question. Similarly, the real complexity of the thought processes that lead to a scientific undertaking is almost never described in published papers. In this paper, we first discuss how to get an idea that is worth researching. We describe sources of new ideas and how to foster a creative attitude by “cultivating your thoughts”. Only a few of these ideas will make it into a study. Next, we describe how to sharpen and focus a research question so that a study becomes feasible and a valid test of the underlying idea. To do this, the idea needs to be “pruned”. Pruning a research question means cutting away anything that is unnecessary, so that only the essence remains. This includes determining both the latent and the stated objectives, specific pruning questions, and the use of specific schemes to structure reasoning. After this, the following steps include preparation of a brief protocol, conduct of a pilot study, and writing a draft of the paper including draft tables. Then you are ready to carry out your research.

For and Against Methodologies: Some Perspectives on Recent Causal and Statistical Inference Debates

I present an overview of two methods controversies that are central to analysis and inference: That surrounding causal modeling as reflected in the "causal inference" movement, and that surrounding null bias in statistical methods as applied to causal questions. Human factors have expanded what might otherwise have been narrow technical discussions into broad philosophical debates. There seem to be misconceptions about the requirements and capabilities of formal methods, especially in notions that certain assumptions or models (such as potential-outcome models) are necessary or sufficient for valid inference. I argue that, once these misconceptions are removed, most elements of the opposing views can be reconciled. The chief problem of causal inference then becomes one of how to teach sound use of formal methods (such as causal modeling, statistical inference, and sensitivity analysis), and how to apply them without generating the overconfidence and misinterpretations that have ruined so many statistical practices.