Chapter 2 (Study 1): Investigating object orientation effects across 18 languages

At the beginning of my PhD, I contributed to a study led by Dr. Sau-Chin Chen (from Tzu Chi University, in Taiwan) and coordinated by the Psychological Science Accelerator (study page available at https://psysciacc.org/002-object-orientation). This study revisited the object orientation effect, which taps into sensorimotor simulation in conceptual processing. In addition to the orientation congruency factor, this study incorporated two variables. The first variable was an individual difference measuring each participant’s ability to mentally rotate objects. The second variable was the language of testing, with 18 languages being used. The object orientation effect was not replicated. Furthermore, the mental rotation and the language variables did not interact with the effect. In the present chapter, I describe this study and discuss several aspects regarding measurement error, replication and future crosslinguistic studies in conceptual processing.


One of the seminal findings supporting the sensorimotor basis of conceptual processing was the object orientation effect. Stanfield and Zwaan (2001) observed this effect in a sentence-picture verification task. On each trial of this task, participants first read a sentence such as ‘The eagle is in the nest’. On the next screen, they saw a certain picture and had to verify whether the protagonist of the sentence was present in the picture. Thus, following the previous example, if the picture showed an eagle, participants would click on the yes button. The key manipulation of this paradigm, unbeknownst to participants, was that the protagonist shown in the picture could either match or mismatch the orientation that had been implied in the sentence. For instance, following the example sentence above, if participants saw an eagled perched on a branch, the orientation would be matched. If, instead, the eagle were flying, the orientation would be mismatched. The theory of embodiment in language comprehension sustains that people mentally simulate perceptual and motor features of meaning (Barsalou, 1999b; Pulvermüller, 1999). Therefore, according to Stanfield and Zwaan, pictures with a mismatching orientation (compared to pictures with a matching orientation) would incur processing costs, as participants would need to adjust their mental simulation of the scene. Indeed, Stanfield and Zwaan observed such an orientation match advantage in their results.

However, subsequent studies revisiting the object orientation effect revealed mixed results. The finding replicated more successfully in English than in other languages (S.-C. Chen et al., 2020; de Koning et al., 2017; Koster et al., 2018; Rommers et al., 2013; Zwaan & Pecher, 2012). One of the non-replications received a response from Zwaan (2014), who argued that the replication was underpowered—lacking a sufficient sample size—and pointed to several methodological differences with the original study. These comments are noteworthy, and will resonate throughout the present thesis. First, it should be seriously considered whether the sample sizes used in replications are sufficient. Recent research in neuroscience suggested that studying a certain question—namely, the mapping of individual differences at the structural and the functional levels—requires far more than the 25 participants that make up the average sample size in this topic. Rather, the necessary sample size is around 10,000 participants (Marek et al., 2022). In our topic of interest, it is just possible that the necessary sample sizes far exceed the figures we are used to. A second, crucial issue to consider regarding replications is the fact that the results of our research are absolutely dependent on the method we use, as we demonstrate in Study 2 within the present thesis. From the operationalisation of the research question to the statistical analysis, every decision could have a great impact on the results (Barsalou, 2019). For instance, Noah et al. (2018) found that the facial-feedback effect was eliminated when participants were aware of being recorded. Barsalou (2019) described the variables that can influence an experiment by alluding to the ‘quantum perspective’. That is, a myriad variables can affect the expression of the cognitive mechanism by which a cognitive effect arises. Furthermore, the manifestation of the effect in the experimental situation (traditionally, in the laboratory) need not be the genuine manifestation of the effect. On the contrary, the experimental setting tends to be far removed from the real-life settings of our processes of interest. Against this backdrop, it is not surprising that effects sometimes do not replicate. I do not think this realisation should lead us to a nihilistic or a fatalistic standpoint, where we give up our hopes of reaching any trustworthy conclusions. On the contrary, as Barsalou contends, we could approach the quantum perspective by scrutinising the variability that underlies the cognitive/experimental effects we study (see Cumming, 2014).3

References

Barsalou, L. W. (1999b). Perceptual symbol systems. Behavioral and Brain Sciences, 22, 577–660. https://doi.org/10.1017/S0140525X99002149
Barsalou, L. W. (2019). Establishing generalizable mechanisms. Psychological Inquiry, 30(4), 220–230. https://doi.org/10.1080/1047840X.2019.1693857
Chen, S.-C., de Koning, B. B., & Zwaan, R. A. (2020). Does object size matter with regard to the mental simulation of object orientation? Experimental Psychology, 67(1), 56–72. https://doi.org/10.1027/1618-3169/a000468
Cumming, G. (2014). The new statistics: Why and how. Psychological Science, 25(1), 7–29. https://doi.org/10.1177/0956797613504966
de Koning, B. B., Wassenburg, S. I., Bos, L. T., & van der Schoot, M. (2017). Mental simulation of four visual object properties: Similarities and differences as assessed by the sentence–picture verification task. Journal of Cognitive Psychology, 29(4), 420–432. https://doi.org/10.1080/20445911.2017.1281283
Koster, D., Cadierno, T., & Chiarandini, M. (2018). Mental simulation of object orientation and size: A conceptual replication with second language learners. Journal of the European Second Language Association, 2(1), 38. https://doi.org/10.22599/jesla.39
Marek, S., Tervo-Clemmens, B., Calabro, F. J., Montez, D. F., Kay, B. P., Hatoum, A. S., Donohue, M. R., Foran, W., Miller, R. L., Hendrickson, T. J., Malone, S. M., Kandala, S., Feczko, E., Miranda-Dominguez, O., Graham, A. M., Earl, E. A., Perrone, A. J., Cordova, M., Doyle, O., … Dosenbach, N. U. F. (2022). Reproducible brain-wide association studies require thousands of individuals. Nature, 1–7. https://doi.org/10.1038/s41586-022-04492-9
Noah, T., Schul, Y., & Mayo, R. (2018). When both the original study and its failed replication are correct: Feeling observed eliminates the facial-feedback effect. Journal of Personality and Social Psychology, 114, 657–664. https://doi.org/10.1037/pspa0000121
Pulvermüller, F. (1999). Words in the brain’s language. Behavioral and Brain Sciences, 22, 253–336. https://doi.org/10.1017/S0140525X9900182X
Rommers, J., Meyer, A. S., & Huettig, F. (2013). Object shape and orientation do not routinely influence performance during language processing. Psychological Science, 24(11), 2218–2225. https://doi.org/10.1177/0956797613490746
Stanfield, R. A., & Zwaan, R. A. (2001). The effect of implied orientation derived from verbal context on picture recognition. Psychological Science, 12(2), 153–156. https://doi.org/10.1111/1467-9280.00326
Zwaan, R. A. (2014). Replications should be performed with power and precision: A response to Rommers, Meyer, and Huettig (2013). Psychological Science, 25(1), 305–307. https://doi.org/10.1177/0956797613509634
Zwaan, R. A., & Pecher, D. (2012). Revisiting Mental Simulation in Language Comprehension: Six Replication Attempts. PLoS ONE, 7(12), e51382. https://doi.org/10.1371/journal.pone.0051382

  1. Thank you to Dr. Margriet Groen and Prof. Max Louwerse for a discussion on this topic.↩︎


Pablo Bernabeu, 2022. Licence: CC BY 4.0.
Thesis: https://doi.org/10.17635/lancaster/thesis/1795.

Online book created using the R package bookdown.