November 26, 2020, I was awarded the Distinguished Lorentz Fellowship & Prize. SPUI25 hosted an online event for the occasion. You can read a short summary here, or just watch the recorded event (above). Many questions were asked via the zoom Q&A, youtube chat and on twitter. Thank you all for these questions! I could only answer some of them during the live event. Below a list of remaining questions and my answers to them [¹].

Q1: Deterministic vs stochastic models

“I was wondering what your opinion shifting from stochastic models (which we mostly use in psychology) to more deterministic models as thinking tools are. Is this implied in the epistemological sea change?”

I often use deterministic models for ease of analysis, but derived results often apply directly to stochastic models as well. Both deterministic and stochastic models have scientific value when used thoughtfully and reflectively. My proposal for thinking tools is not to favour one over another. Instead, the ‘epistemological sea change’ refers to a conceptual shift in how we think about scientific progress in psychology. We explain this in more detail here.

Q2: Empirical underdetermination

Supposing we manage to develop a theoretical model that manages to replicate a certain effect or aspect of behaviour. What would be the next step at that point? If there are multiple ways to model such an effect, what does success teach us?

Empirical underdetermination is a feature of science (and cognition). Be that as it may, on my view the first step is not to explain effects, but instead to explain capacities. It turns out remarkably hard to come up with even one plausible explanation. Existing explanations are either limited to toy domains, have “magic” components, are intractable, or otherwise make unrealistic assumptions. Many researchers concern themselves a lot with the question you raise, but I think it distracts them from taking the first necessary steps to come up with possible explanations.

Q3: Role of Machine Learning / AI

What are your ideas in relation to using computational techniques (such as machine learning/AI) for more rigorous formalization, but without giving in to the prevalent “engineering mindset” in those fields that sometimes leads to doing even less theory-related thinking than psychology?

You hit the nail on the head! This is exactly the tension I see, too. I think the only way not to give in to the “engineering mindset” is to be aware that it is a pitfall (if ones purpose is theory building) and actively steer away from it. At the AI program I’m affiliated with, we are exploring ways to embed this better in the educational program.

Q4: Philosophy + psychology

What are some good examples of combining philosophy with psychology on the level of theory?

I think my own work provides good examples:

• van Rooij, I., & Baggio, G. (in press). Theory before the test: How to build high-verisimilitude explanatory theories in psychological science. Perspectives on Psychological Science.

• van Rooij, I., Wright, C., Kwisthout, J., & Wareham, T. (2018). Rational analysis, intractability, and the prospects of ‘as if’-explanations. Synthese, 195(2), 491–510

Work by others that also provide good examples:

• Thagard (2000). Coherence in thought and action. MIT Press.

• Oaksford, M., & Chater, N. (2007). Bayesian rationality: The probabilistic approach to human reasoning. Oxford University Press.

Q5: Empirical cycle

Isn’t it the case that scientific progress requires a process where new facts, new theories and new methods reinforce each other. Thus, new facts lead to better theories and insight, which helps us to developed better measurements and methods, those help to find new facts and then hopefully a new round of better theories, etc.

Ideally, yes. In practice, too much of psychology still hunts for ‘effects’ and too little resources go into building theories. Moreover, ‘effects’ are not the ‘facts’ to start building theories from/on. Hence, we get very little theory off the ground from the many ‘facts’ we have. I argue that we need a different starting point and different methodological tools and habits. We need a theoretical cycle to complement the known empirical cycle.

Q6: Psychopathology

Your example of sorting letters seems very much inspired by AI, i.e. mathematical processes that computers can also do. But how would a theory look like of something that computers don’t show, but only humans do, e.g. mental disorders like addiction or depression? To understand what people with these disorders do differently, wouldn’t we have to first find “effects” to describe how people with these disorders perceive reality in a different way? How would a theory of addiction or depression start without such effects?

I conjecture that even in these cases we can better start at capacities than at effects. In this commentary, we posed it as follows: “A capacity can be understood as a more or less reliable ability (or disposition or tendency) to transform some initial state into a resulting state. The resulting states need not be ‘desirable’, and the tendency of a person, given certain initial conditions, to more or less reliably converge on a particular state of mind is a capacity, too. In this light, depression and anxiety (but also mental well-being) can theoretically be seen as states to which our mental states converge under certain conditions, but not others.”

Q7: Applied research and intervention

I have a question concerning applied research: Do you think theory could help building more helpful interventions? For example for children with ADHD. Do you think not using theory to build such interventions could be compared to a trial and error approach and might therefore take much more time and be less effective?

Theories can certainly help with developing interventions, but (1) it is not why I make theories (rather I make them to understand and to explain), and (2) it is not clear that intervention is always a laudable goal (even if we have a good theory that would allow us to intervene); I’d rather use theory to empower people than to intervene.

Q8: Math and Emotions

Kenneth Gergen mentioned operant learning theory as an example of good psychology theory in the past. Those learning experiments can easily be implemented in computers because the training symbols (rewards and punishments) can be turned into numbers (+1, -1). But in many other areas of psychology (e.g. emotions), the process of “turning vague labels into numbers” seems to be much more difficult. So doesn’t the “aversion” of psychology to theory come from the difficulty to turn vague verbal labels into numbers?

Mathematics is not limited to numbers. For instance, graph theory is more about relations than about numbers. There are theories that use graph theory to model the influence of emotions on our thinking. See, for instance:

• Thagard, P. (2006). Hot Thought: Mechanisms and Applications of Emotional Cognition. MIT Press.

To answer your question of where the aversion to formal theory in psychology comes from: I think from a lack of education in real mathematics (as opposed to mere arithmetic), resulting in a lack of appreciation for the beauty, naturalness and fun of math.

Q9: Replicability as foundation

Many scientists dismiss calls for more/better theory building (and against the atheoretical chase for replicable effects) by stating that indeed everyone agrees that theory is important but replicable facts are just the first step towards having something to explain. Iris has already touched on the pitfalls of such a bottom-up approach; perhaps this question can be an opportunity to expand on that.

Indeed, many researchers take replicability as a necessary first step to make progress in science. Two counter points:

First, a focus on ‘replication first’ can actually hinder scientific progress. For elaboration on this point, I refer to the brilliant work by Berna Devezer and colleagues:

• Devezer, B. et al. (2019). Scientific discovery in a model-centric framework: Reproducibility, innovation, and epistemic diversity. PLOS ONE 14 (5): e0216125.

• Devezer, B., Navarro, D. J., Vandekerckhove, J, & Buzbas, E. O. (2020). The case for formal methodology in scientific reform. (preprint)

Second, we already know which robust phenomena need explaining: capacities. As we put it in this paper: “While effects are usually discovered empirically through intricate experiments, capacities (primary explananda) do not need to be discovered in the same way (Cummins, 2000). Just like we knew that apples fall straight from the trees (rather than move upward or sideways) before we had an explanation in terms of Newton’s theory of gravity, so too we already know that humans can learn languages, interpret complex visual and social scenes, and navigate dynamic, uncertain, culturally complex social worlds. These capacities are so complex to explain computationally or mechanistically that we do not know yet how to emulate them in artificial systems at human levels of sophistication. The priority should be the discovery not of experimentally constructed effects, but of plausible explanations of real-world capacities.”

I think this will suffice as ‘expand’ for now :)

Q10: The goal of theory

Follow-up on Kenneth’s comment: you mention that physics and chemistry etc. “solve practical problems”. Isn’t psychology also trying to solve “practical problems”, like treating mental disorders, helping to raise children, …. so shouldn’t theories in psychology also be like in physics or chemistry (rather than those by Marx and Freud)?

I believe Kenneth’s point was to challenge us to think about why we would want to make theories; not so much their form (say, verbal versus formal, or more physics versus more psychoanalysis like). On Kenneth’s pragmatist view, theories are for problem solving. I myself am, however, more a problem creator than a problem solver.

Q11: Computational complexity

How does computational complexity analysis aid thinking about phenomena? Doesn’t this analysis type only assesses our own current theories, taking them literally?

I co-wrote a book on this topic, titled Cognition and intractability (check out Chapter 1). The approach indeed assesses our current theories and takes them literally in so far as their theoretical commitments go. I think this is how things should be in theoretical research.

Let me close by thanking everyone again for the thought-provoking questions. If I missed any key questions, or you’d like to add new questions, feel free to let me know!

– Iris van Rooij