Linguistic and embodied systems in conceptual processing: Variation across individuals and items

class: center, middle, inverse, title-slide

# <img src='img/LU%20logo.jpg' alt='Lancaster University logo' width='130px' style='float:right; margin-top:-6%; margin-bottom:-235%;'/><br><div style='margin-top:-3%'>Linguistic and embodied systems<br>in conceptual processing:<br>Variation across individuals and items</div>
## Pablo Bernabeu
### Supervisors:  Dermot Lynott and Louise Connell
### 24 May 2021<br><br><a href='https://www.youtube.com/watch?v=y2bopgYWYvE&ab_channel=LancasterPsychology' style='color:grey; font-size:70%'><i class='fas fa-video fa-sm'></i>  Video recording</a><br><a href='https://github.com/pablobernabeu/Language-and-embodiment-in-conceptual-processing-Variation-across-participants-words-and-paradigms' style='color:grey; font-size:70%'><i class='fab fa-github fa-sm'></i>  Data and analyses to follow</a>

---

<script src="https://use.fontawesome.com/releases/v5.15.3/js/all.js"></script>

<style>
.title-slide {
background-color: #FAFAFC;
}

.title-slide h1,
.title-slide h2,
.title-slide h3,
.title-slide h4 {
color: black;
text-shadow: none;
}
</style>

# Presentation

*Work in progress*

<br>

- Understanding the meaning of words

- Linguistic system

- Embodied system and subsystems

- **Semantic priming** study

- **Semantic decision** study

- Power analysis for next study

- Conclusions

- References

---

# Cognitive systems

- What memory systems should be considered?

- Tulving (2007): *Are There 256 Different Kinds of Memory?*

- What semantic memory systems should be considered? 
<br>(e.g., Bernabeu et al., 2021)

- Linguistic system

- Embodied system

- Perceptual

- Motor

- Emotional

- Social

- Systems: subject to research question and available research

---

# Semantic priming study

- Main data set: Semantic Priming Project (Hutchison et al., 2013)

- 512 participants, 1,661 target words

- Lexical decision = overt task

Over multiple trials, do the characters form a real word?

- Semantic priming = covert experimental design

>  <span style='color:blue;'> PRIME WORD  &nbsp; &nbsp;  target word</span>

>   &nbsp; &nbsp; &nbsp; &nbsp; PIKE  &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;  prack

- Prime always real word; target sometimes nonword.

- Prime word: 150 ms.

Intertrial interval: 50 or 1,050 ms.

Target word: until response or 3,000 ms.

---

#### Research question

Role and interplay of the linguistic and perceptual systems.

#### Variables used

- Semantic association between prime and target word

- Latent Semantic Analysis measure (Hutchison et al., 2013)

- Visual-modality difference between prime and target word

- Normed visual rating (Lynott et al., 2020)

- Individual differences: Vocabulary size (Hutchison et al., 2013)

- Individual differences: Attentional control (Hutchison et al., 2013), entered as covariate (see Yap et al., 2017 regarding the covariance of these variables).

- Interstimulus interval (Hutchison et al., 2013): 50 or 1,050 ms

- Relevant because linguistic system tends to activate before embodied system (Bernabeu et al., 2017)

---

#### Vocabulary size varies between participants and within words, thus requiring by-word random slopes (Brauer & Curtin, 2018).

---

#### Vocabulary size varies between participants and within words, thus requiring by-word random slopes (Brauer & Curtin, 2018).

---

#### Word association varies between words and within participants, thus requiring by-participant random slopes (Brauer & Curtin, 2018).

---

#### Word association varies between words and within participants, thus requiring by-participant random slopes (Brauer & Curtin, 2018).

---

# Statistical analysis applied in both studies

- Unique contribution of each variable of interest, and of some interactions among those variables.

- Linear mixed-effects models

- Maximal random effects structures. While convergence was facilitated, random slopes for the effects of interest were never removed (Brauer & Curtin, 2018; Singmann & Kellen, 2019). 
  
  - <em>P</em> values calculated using the Kenward-Roger approximation for degrees of freedom (Luke, 2017).
  
- All analyses in R

---

# Random effects (1/3): intercepts
Following Brauer and Curtin (2018)

```

lmerTest::lmer(target.RT ~

# RANDOM EFFECTS: maximal structure constructed following the 
# guidelines of Brauer and Curtin (2018; 
# https://psych.wisc.edu/Brauer/BrauerLab/wp-content/uploads/2014/04/Brauer-Curtin-2018-on-LMEMs.pdf).
# Interaction effects only require random slopes if all
# interacting variables vary within the same units.

# Random intercepts
(1 | Participant) + (1 | target_word) +

```

---

# Random effects (2/3): by-subject random slopes
Following Brauer and Curtin (2018)

```

# By-subject random slopes
(0 + z_LSA_word_association || Participant) + 
(0 + z_visual_rating_diff || Participant) + 
(0 + z_recoded_isi || Participant) + 
(0 + z_LSA_word_association : z_recoded_isi || Participant) +
(0 + z_visual_rating_diff : z_recoded_isi || Participant) +
# In the lines below, random slopes were removed following Remedy 11 
# from Table 17 in Brauer and Curtin (2018).
# (z_target_word_frequency || Participant) + 
# (z_target_orthographic_Levenshtein_distance || Participant) +

```

---

# Random effects (3/3): by-item random slopes
Following Brauer and Curtin (2018)

```

# By-item random slopes
(0 + z_vocabulary_size || target_word) +
# In the line below, random slopes were removed following Remedy 11 
# from Table 17 in Brauer and Curtin (2018).
# (0 + z_attentional_control || target_word) +

```

---

<div id="htmlwidget-da689b1938e7a50c36fe" style="width:100%;height:auto;" class="datatables html-widget"></div>
<script type="application/json" data-for="htmlwidget-da689b1938e7a50c36fe">{"x":{"filter":"none","data":[["(Intercept)","z_target_word_frequency","z_target_orthographic_Levenshtein_distance","z_attentional_control","z_vocabulary_size","z_LSA_word_association","z_visual_rating_diff","z_recoded_isi","z_attentional_control:z_LSA_word_association","z_attentional_control:z_visual_rating_diff","z_attentional_control:z_recoded_isi","z_LSA_word_association:z_vocabulary_size","z_visual_rating_diff:z_vocabulary_size","z_LSA_word_association:z_recoded_isi","z_visual_rating_diff:z_recoded_isi","z_recoded_isi:z_vocabulary_size"],["667.600338673015","-21.0571052163988","12.0327455999855","-25.8732319795719","-15.0115851088126","-3.64942269095352","1.60153710740065","11.0254672615618","-0.824713099479509","-0.234831940405368","-2.68007537202957","1.50071124150684","-0.501139519085131","0.159537086404172","-0.773941346961707","0.827541727173337"],["6.27230041741858","0.873755479864374","0.870906923573956","6.6556894889978","6.65827023962908","0.823912896827379","0.611408593062378","1.73055490838621","0.47565929460758","0.476615471289989","1.84874731352234","0.513571869994449","0.500791938538023","0.441551925195868","0.441601605252286","1.84935695949883"],["506.921301460149","1457.37149559033","1457.75868586729","494.643180474208","495.498645362691","1155.21336350724","1488.82959212044","492.087809238296","483.794770236234","489.018360411255","493.192216907582","486.171150183799","526.311352559329","163972.654053957","154380.047763073","493.891670691996"],["< .001","< .001","< .001","< .001",".025","< .001",".009","< .001",".084",".622",".148",".004",".317",".718",".08",".655"]],"container":"<table class=\"nowrap display\">\n  <thead>\n    <tr>\n      <th> <\/th>\n      <th>β<\/th>\n      <th><em>SE<\/em><\/th>\n      <th><em>df<\/em><\/th>\n      <th><em>p<\/em><\/th>\n    <\/tr>\n  <\/thead>\n<\/table>","options":{"order":[],"autoWidth":false,"orderClasses":false,"columnDefs":[{"targets":1,"render":"function(data, type, row, meta) {\n    return type !== 'display' ? data : DTWidget.formatRound(data, 2, 3, \",\", \".\");\n  }"},{"targets":2,"render":"function(data, type, row, meta) {\n    return type !== 'display' ? data : DTWidget.formatRound(data, 2, 3, \",\", \".\");\n  }"},{"targets":3,"render":"function(data, type, row, meta) {\n    return type !== 'display' ? data : DTWidget.formatRound(data, 2, 3, \",\", \".\");\n  }"},{"orderable":false,"targets":0}]}},"evals":["options.columnDefs.0.render","options.columnDefs.1.render","options.columnDefs.2.render"],"jsHooks":[]}</script>

* Role of linguistic and embodied systems

---

## Interaction: Vocabulary size by word association

---

## Interaction: Vocabulary size by visual-modality difference

---

## Interaction: Vocabulary size by interstimulus interval

---

# Higher-vocabulary participants

- less affected by the prime word--specifically, word association and visual-modality difference. This converges with Yap et al.'s (2017) finding that higher- and lower-vocabulary participants were affected by different variables. Potentially, the variables affecting higher-vocabulary participants are more selective, that is, fewer and more relevant to the task (also see Pexman & Yap, 2018).

---

## Interaction: Vocabulary size by target word frequency

---

## Interaction: Vocabulary size by orthographic Levenshtein distance of target word

---

# Semantic decision

#### Research question: Role and interplay of the linguistic and perceptual systems.

- Main data set: Calgary Semantic Decision Project (Pexman et al., 2017)

- 312 participants, 10,000 words

- Overt task: Over multiple trials, is the word abstract or concrete?

#### Variables

Similar to previous ones.

Word co-occurrence (Wingfield & Connell, 2019) instead of association.

---

# Random effects (1/3): intercepts
Following Brauer and Curtin (2018)

```

lmerTest::lmer(target.RT ~

# Random intercepts
(1 | Participant) + (1 | Word) +

```

---

# Random effects (2/3): by-subject random slopes
Following Brauer and Curtin (2018)

```

# By-subject random slopes
(0 + z_word_cooccurrence || Participant) + 
(0 + z_visual_rating || Participant) +
# In the lines below, random slopes were removed following 
# Remedy 11 from Table 17 in Brauer and Curtin (2018).
# (z_word_frequency || Participant) + 
# (z_orthographic_Levenshtein_distance || Participant) +
# (z_word_concreteness || Participant) +

```

---

# Random effects (3/3): by-item random slopes
Following Brauer and Curtin (2018)

```

# By-item random slopes
(0 + z_vocabulary_size || Word) +

```

---

<div id="htmlwidget-611e67be2f8d4b5ae1ad" style="width:100%;height:auto;" class="datatables html-widget"></div>
<script type="application/json" data-for="htmlwidget-611e67be2f8d4b5ae1ad">{"x":{"filter":"none","data":[["(Intercept)","z_word_frequency","z_orthographic_Levenshtein_distance","z_word_concreteness","z_word_cooccurrence","z_visual_rating","z_vocabulary_size","z_word_concreteness:z_vocabulary_size","z_vocabulary_size:z_word_cooccurrence","z_vocabulary_size:z_visual_rating"],["968.598367907698","-33.1526512032526","-0.976891342652453","-42.7300692799547","-5.60204168140698","-6.57321400608311","-40.4138485377047","-7.19443941008556","3.96346994485279","4.70005161155466"],["10.2284345670919","1.61132177216293","1.65643876066303","2.36072227285111","2.89081676603148","2.42908664831209","10.1856046773596","1.42073896513325","2.41600845003274","1.79407040935844"],["321.667707337402","4034.33278021112","4049.30879521526","4018.97924275463","770.576426826124","1405.9085880459","311.767960707249","3973.18807415663","384.703872612295","484.520671627406"],["< .001","< .001",".555","< .001",".053",".007","< .001","< .001",".102",".009"]],"container":"<table class=\"nowrap display\">\n  <thead>\n    <tr>\n      <th> <\/th>\n      <th>β<\/th>\n      <th><em>SE<\/em><\/th>\n      <th><em>df<\/em><\/th>\n      <th><em>p<\/em><\/th>\n    <\/tr>\n  <\/thead>\n<\/table>","options":{"order":[],"autoWidth":false,"orderClasses":false,"columnDefs":[{"targets":1,"render":"function(data, type, row, meta) {\n    return type !== 'display' ? data : DTWidget.formatRound(data, 2, 3, \",\", \".\");\n  }"},{"targets":2,"render":"function(data, type, row, meta) {\n    return type !== 'display' ? data : DTWidget.formatRound(data, 2, 3, \",\", \".\");\n  }"},{"targets":3,"render":"function(data, type, row, meta) {\n    return type !== 'display' ? data : DTWidget.formatRound(data, 2, 3, \",\", \".\");\n  }"},{"orderable":false,"targets":0}]}},"evals":["options.columnDefs.0.render","options.columnDefs.1.render","options.columnDefs.2.render"],"jsHooks":[]}</script>

* Role of linguistic and embodied systems

---

## Interaction: Vocabulary size by word co-occurrence

---

## Interaction: Vocabulary size by visual-modality difference

---

# As before, higher-vocabulary participants

- less affected by the prime word--specifically, word association and visual-modality difference. This converges with Pexman and Yap's (2018) finding that higher- and lower-vocabulary participants were affected by different variables. Potentially, the variables affecting higher-vocabulary participants are more selective, that is, fewer and more relevant to the task (also see Yap et al., 2017).

---

## Interaction: Vocabulary size by word frequency

---

## Interaction: Vocabulary size by orthographic Levenshtein distance

---

## Power analysis

For another, forthcoming study, a prospective power analysis is being conducted using the results from the two present studies. Both these studies are apt for usage as pilots in a power analysis, as they encompass larger-than-average sample sizes (Albers & Lakens, 2018; Loken & Gelman, 2017). For each effect of interest, power curves based on Monte Carlo simulations are being performed, using the `simr` package in R.

Monte Carlo simulations operate by running the statistical model specified a large number of times, under slight variations of the dependent variable. The power to detect each effect of interest is calculated by dividing the number of times that the effect was significant by the total number of iterations. For instance, if an effect is significant in 85 out of 100 simulations, the power for that effect is 85%. Importantly, it is reasonable to regard the results conservatively, due to potential bias in the published data sets (Loken & Gelman, 2017). *P* values were calculated using the Satterthwaite approximation for degrees of freedom (Luke, 2017).

<em>Preliminary</em> power for semantic decision study presented next (< 30 iterations instead of recommended 500).

---

### Word co-occurrence

---

### Visual-modality rating

---

### Vocabulary size

---

### Interaction: Vocabulary size by word co-occurrence

---

### Interaction: Vocabulary size by visual-modality rating

---

# Conclusions

- Dynamic process subject to individual and contextual variables. Systematic associations across the levels of participant, stimulus and task, consistent with roles for both linguistic and embodied systems in conceptual processing.

- Performance of higher-vocabulary participants influenced by fewer variables. This converges with previous findings suggesting that higher and lower-vocabulary participants are affected by different variables. Potentially, the variables affecting higher-vocabulary participants are more relevant to the task (Pexman & Yap, 2018; Yap et al., 2017).

- Sample size suggested by power analysis much larger than usual samples in literature (also see Loken & Gelman, 2017).

------

- Time-consuming analyses run on High-End Computing facility at Lancaster University.

---

### References (1/4)

Albers, C., & Lakens, D. (2018). When power analyses based on pilot data are biased: Inaccurate effect size estimators and follow-up bias. <em>Journal of Experimental Social Psychology, 74</em>, 187–195. https://doi.org/10.1016/j.jesp.2017.09.004

Bernabeu, P., Lynott, D., & Connell, L. (2021). <em>Preregistration: The interplay between linguistic and embodied systems in conceptual processing.</em> OSF. https://osf.io/ftydw/

Bernabeu, P., Willems, R. M., & Louwerse, M. M. (2017). Modality switch effects emerge early and increase throughout conceptual processing: Evidence from ERPs. In G. Gunzelmann, A. Howes, T. Tenbrink, & E. J. Davelaar (Eds.), <em>Proceedings of the 39th Annual Conference of the Cognitive Science Society</em> (pp. 1629-1634). Austin, TX: Cognitive Science Society.
https://mindmodeling.org/cogsci2017/papers/0318

Brauer, M., & Curtin, J. J. (2018). Linear mixed-effects models and the analysis of nonindependent data: A unified framework to analyze categorical and continuous independent variables that vary within-subjects and/or within-items. <em>Psychological Methods, 23</em>(3), 389–411. https://doi.org/10.1037/met0000159

---

### References (2/4)

Hutchison, K. A., Balota, D. A., Neely, J. H., Cortese, M. J., Cohen-Shikora, E. R., Tse, C.-S., Yap, M. J., Bengson, J. J., Niemeyer, D., & Buchanan, E. (2013). The semantic priming project. <em>Behavior Research Methods, 45</em>, 1099–1114. https://doi.org/10.3758/s13428-012-0304-z

Loken, E., & Gelman, A. (2017). Measurement error and the replication crisis. <em>Science, 355</em>(6325), 584-585. https://doi.org/10.1126/science.aal3618

Luke, S. G. (2017). Evaluating significance in linear mixed-effects models in R. <em>Behavior Research Methods, 49</em>(4), 1494–1502. https://doi.org/10.3758/s13428-016-0809-y

Lynott, D., Connell, L., Brysbaert, M., Brand, J., & Carney, J. (2019). The Lancaster Sensorimotor Norms: Multidimensional measures of perceptual and action strength for 40,000 English words. <em>Behavior Research Methods, 52</em>, 1271–1291. https://doi.org/10.3758/s13428-019-01316-z.

---

### References (3/4)

Pexman, P. M., Heard, A., Lloyd, E., & Yap, M. J. (2017). The Calgary semantic decision project: Concrete/abstract decision data for 10,000 English words. <em>Behavior Research Methods, 49</em>(2), 407–417. https://doi.org/10.3758/s13428-016-0720-6

Pexman, P. M., & Yap, M. J. (2018). Individual differences in semantic processing: Insights from the Calgary semantic decision project. <em>Journal of Experimental Psychology: Learning, Memory, and Cognition, 44</em>(7), 1091–1112. 
https://doi.org/10.1037/xlm0000499

Singmann, H., & Kellen, D. (2019). An Introduction to Mixed Models for Experimental Psychology. In D. H. Spieler & E. Schumacher (Eds.), *New Methods in Cognitive Psychology* (pp. 4--31). Hove, UK: Psychology Press. http://singmann.org/download/publications/singmann_kellen-introduction-mixed-models.pdf

---

### References (4/4)

Tulving, E. (2007). Are There 256 Different Kinds of Memory? In J. S. Nairne (Ed.), <em>The foundations of remembering: Essays in honor of Henry L. Roediger, III</em> (p. 39–52). Psychology Press.

Wingfield, C., & Connell, L. (2019). <em>Understanding the role of linguistic distributional knowledge in cognition</em>. PsyArXiv. https://doi.org/10.31234/osf.io/hpm4z

Yap, M. J., Hutchison, K. A., & Tan, L. C. (2017). Individual differences in semantic priming performance: Insights from the semantic priming project. In M. N. Jones (Ed.), <em>Frontiers of cognitive psychology. Big data in cognitive science</em> (p. 203–226). Routledge/Taylor & Francis Group.