The engagement of sensory brain regions during word recognition is widely documented, yet its precise relevance is less clear. It would constitute perceptual simulation only if it has a functional role in conceptual processing. We investigated this in an Event-Related Potential (ERP) experiment implementing the conceptual modality switch paradigm. In each trial, participants verified the relation between a property word and a concept word. Orthogonally, we manipulated the conceptual modality of successive trials, and tested whether switching modalities incurred any processing costs at different stages of word recognition. Unlike previous studies, we time-locked ERPs to the first word of target trials, in order to measure the modality transitions from the beginning, and also to reduce confounds within the target trial. Further, we included different types of switch—one from auditory to visual modality, and one from haptic to visual—, which were compared to the non-switch—visual to visual. Also, one group of participants was asked to respond quickly (n = 21), and another group to respond self-paced (n = 21), whilst a few others received no constraints (n = 5). We found ERP effects in four typical time windows from 160 to 750 ms post word onset. The overall effect is characterized by a negativity for modality-switching relative to not switching, and it increases over time. Further, the effect arises with both types of switch, and influences both participant groups within anterior and posterior brain regions. The emergence of this effect in the first time window particularly suggests that sensory regions may have a functional role in conceptual processing. The increased effect later on converges with previous studies in supporting the compatibility of distributional and embodied processing. On a less conclusive note, more research may be necessary to ascertain the nature of the effect at late stages.
Bernabeu, P. (2017). Modality switches occur early and extend late in conceptual processing: evidence from ERPs [Master's thesis]. School of Humanities, Tilburg University. https://psyarxiv.com/5gjvk