Details on some presentations | Pablo Bernabeu

Scaling systematic reviews: A solo researcher's workflow with Gemini

Fri, 21 Nov 2025 00:00:00 +0000

Your browser does not support embedded PDFs. Download the PDF instead.

Overview

graph TD A["Scopus API retrieves papers"] --> B["Batch process PDFs in NotebookLM
to prevent data omissions"] B --> C["Extract distinct data points
from each paper"] C --> D["Gemini for
high-level synthesis"] D --> E["Human in the loop validation
through prompt engineering"] E --> F["Large scale review by a sole researcher,
saving 30+ hours per review"]

Background References

Bernabeu, P. (2024). rscopus_plus. OSF. https://doi.org/10.17605/OSF.IO/BUZQ6

Malik, M., & Sime, J. A. (2025). Teamwork, co-regulation, and socially shared regulation skills within engineering education studies: A GenAI-assisted scoping review. ASEE Annual Conference & Exposition, Montreal, Quebec, Canada. https://doi.org/10.18260/1-2--57199

Unpacking ERP responses in artificial language learning

Thu, 13 Feb 2025 00:00:00 +0000

Overview

graph TD A["Two sites: Norway (Mini-Norwegian,
Mini-English) and Spain
(Mini-Spanish, Mini-English)"] --> B["Session 1: executive functions
and language history"] B --> C["Session 2: resting-state EEG, then
gender agreement training and ERP test"] C --> D["Session 3:
differential object marking"] D --> E["Session 4:
verb-object agreement"] E --> F["Session 5: retest
of executive functions"] F --> G["Session 6 (four months later):
retention of all properties"] C --> H["P600-like effect for misplaced definite
articles in medial and posterior regions"] H --> I["Control reference point
for properties of interest"]

Longer summary

Third language acquisition often involves morphosyntactic transfer from previously acquired languages. Research suggests that crosslinguistic influence follows systematic patterns, with attention playing a role in selecting the source of transfer. This study investigates morphosyntactic transfer longitudinally using artificial languages distributed between groups in two sites: Norway (Mini-Norwegian and Mini-English) and Spain (Mini-Spanish and Mini-English).

The study consists of six sessions. Session 1 assesses attention-related cognitive functions and language history through a home-based session that includes tasks such as the digit span task to evaluate working memory, the Stroop task for inhibitory control, and a serial reaction time task to assess implicit learning. Additionally, participants complete the Language History Questionnaire (LHQ3; Li et al., 2020) to provide comprehensive background information.

Session 2 commences with resting-state electroencephalography (EEG) recordings, conducted with eyes open and closed in counterbalanced order to measure attentional skills. This is followed by training on gender agreement, a property present in both Norwegian and Spanish. Participants engage in vocabulary pre-training, grammatical training, and a behavioural test, culminating in an EEG experiment where event-related potentials (ERPs) are recorded in response to grammatical violations.

A week later, Session 3 introduces differential object marking, which is present in Spanish but absent in Norwegian and English. Training focuses solely on this new property, while the EEG experiment tests both gender agreement and differential object marking together. Next, after another week, Session 4 introduces verb-object agreement, a property absent from all three languages, using the same training and testing structure.

Session 5 involves a retest of executive functions, mirroring the tasks from Session 1 to examine longitudinal stability and pre-post changes. Due to the absence of a control group, pre-post changes are to be analysed only in relation to the baseline effects of the executive functions on Session 2 performance.

Finally, Session 6, conducted four months later, tests the retention of all grammatical properties and includes control tests to assess knowledge of the relevant properties in the natural languages.

The artificial languages were designed with minimal confounding factors by avoiding cognates and ensuring morphological consistency across languages. Linguistic stimuli were counterbalanced across conditions to prevent spurious effects, with gender and number equally distributed. Stimuli creation and presentation were facilitated through modular R scripts and OpenSesame software, ensuring reproducibility, testability and reusability.

This presentation focusses on preliminary results from the Norwegian site, with a methodological emphasis. We first examine accuracy in the grammaticality judgements, which was generally high across participants. The Mini-English group exhibited higher accuracy than the Mini-Norwegian group, particularly for the property of gender agreement.

Next, we will describe the ERP results for the various grammaticality conditions. The properties of interest did not exhibit clear effects. Instead, a P600-like positivity was observed in response to misplaced definite articles (e.g., thestreet versus the street), predominantly in medial and posterior brain regions. This positivity suggests increased syntactic processing demands when encountering ungrammatical forms. This control effect provides a useful point of reference to assess the results for the properties of interest.

The mixed-effects models for gender agreement suggested that grammaticality interacted with session number, working memory and implicit learning. Cross-linguistic differences were also evident, with the Mini-English group displaying more robust effects than the Mini-Norwegian group.

These preliminary findings suggest that language learning is influenced by attentional mechanisms, with individual differences in executive functions playing a role. The P600 effects associated with the control condition of article misplacement provide a useful point of reference. Audience feedback on the methods and interpretations will be welcomed to guide the next stages of this research.

References

Alday, P. M. (2019). How much baseline correction do we need in ERP research? Extended GLM model can replace baseline correction while lifting its limits. Psychophysiology, 56(12), e13451. https://doi.org/10.1111/psyp.13451

Bardel, C., & Falk, Y. (2012). The L2 status factor and the declarative/procedural distinction. In J. Cabrelli, S. Flynn, & J. Rothman (Eds.), Third Language Acquisition in Adulthood (pp. 61–78). John Benjamins Publishing Company. https://doi.org/10.1075/sibil.46.06bar

Brauer, M., & Curtin, J. J. (2018). Linear mixed-effects models and the analysis of nonindependent data: A unified framework to analyze categorical and continuous independent variables that vary within-subjects and/or within-items. Psychological Methods, 23(3), 389–411. https://doi.org/10.1037/met0000159

Desbordes, T., Lakretz, Y., Chanoine, V., Oquab, M., Badier, J.-M., Trébuchon, A., Carron, R., Bénar, C.-G., Dehaene, S., & King, J.-R. (2023). Dimensionality and ramping: Signatures of sentence integration in the dynamics of brains and deep language models. Journal of Neuroscience, 43(29), 5350–5364. https://doi.org/10.1523/JNEUROSCI.1163-22.2023

Flynn, S., Foley, C., & Vinnitskaya, I. (2004). The cumulative-enhancement model for language acquisition: Comparing adults’ and children’s patterns of development in first, second and third language acquisition of relative clauses. International Journal of Multilingualism, 1(1), 3–16. https://doi.org/10.1080/14790710408668175

Friederici, A. D., Steinhauer, K., & Pfeifer, E. (2002). Brain signatures of artificial language processing: Evidence challenging the critical period hypothesis. Proceedings of the National Academy of Sciences, 99(1), 529–534. https://doi.org/10.1073/pnas.012611199

Fuhs, M. W., Nesbitt, K. T., Farran, D. C., & Dong, N. (2014). Longitudinal associations between executive functioning and academic skills across content areas. Developmental Psychology, 50(6), 1698–1709. https://doi.org/10.1037/a0036633

González Alonso, J., Alemán Bañón, J., DeLuca, V., Miller, D., Pereira Soares, S. M., Puig-Mayenco, E., Slaats, S., & Rothman, J. (2020). Event related potentials at initial exposure in third language acquisition: Implications from an artificial mini-grammar study. Journal of Neurolinguistics, 56, 100939. https://doi.org/10.1016/j.jneuroling.2020.100939

González Alonso, J., Bernabeu, P., Silva, G., DeLuca, V., Poch, C., Ivanova, I., & Rothman, J. (2025). Starting from the very beginning: Unraveling third language (L3) development with longitudinal data from artificial language learning and EEG. International Journal of Multilingualism, 22(1), 119–142. https://doi.org/10.1080/14790718.2024.2415993

Grossmann, J. A., Aschenbrenner, S., Teichmann, B., & Meyer, P. (2023). Foreign language learning can improve response inhibition in individuals with lower baseline cognition: Results from a randomized controlled superiority trial. Frontiers in Aging Neuroscience, 15. https://doi.org/10.3389/fnagi.2023.1123185

Hudson Kam, C. L., & Newport, E. L. (2005). Regularizing unpredictable variation: The roles of adult and child learners in language formation and change. Language Learning and Development, 1(2), 151–195. https://doi.org/10.1080/15475441.2005.9684215

Just, M. A., & Carpenter, P. A. (1980). A theory of reading: From eye fixations to comprehension. Psychological Review, 87(4), 329–354. https://doi.org/10.1037/0033-295X.87.4.329

Kidd, E. (2012). Implicit statistical learning is directly associated with the acquisition of syntax. Developmental Psychology, 48(1), 171–184. https://doi.org/10.1037/a0025405

Kliesch, M., Pfenninger, S. E., Wieling, M., Stark, E., & Meyer, M. (2022). Cognitive benefits of learning additional languages in old adulthood? Insights from an intensive longitudinal intervention study. Applied Linguistics, 43(4), 653–676. https://doi.org/10.1093/applin/amab077

Li, P., Zhang, F., Yu, A., & Zhao, X. (2020). Language History Questionnaire (LHQ3): An enhanced tool for assessing multilingual experience. Bilingualism: Language and Cognition, 23(5), 938–944. https://doi.org/10.1017/S1366728918001153

Meister, C., Pimentel, T., Clark, T., Cotterell, R., & Levy, R. (2022). Analyzing wrap-up effects through an information-theoretic lens. In S. Muresan, P. Nakov, & A. Villavicencio (Eds.), Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers) (pp. 20–28). Association for Computational Linguistics. https://doi.org/10.18653/v1/2022.acl-short.3

Meltzer, J. A., Kates Rose, M., Le, A. Y., Spencer, K. A., Goldstein, L., Gubanova, A., Lai, A. C., Yossofzai, M., Armstrong, S. E. M., & Bialystok, E. (2023). Improvement in executive function for older adults through smartphone apps: A randomized clinical trial comparing language learning and brain training. Aging, Neuropsychology, and Cognition, 30(2), 150–171. https://doi.org/10.1080/13825585.2021.1991262

Monaghan, P., Donnelly, S., Alcock, K., Bidgood, A., Cain, K., Durrant, S., Frost, R. L. A., Jago, L. S., Peter, M. S., Pine, J. M., Turnbull, H., & Rowland, C. F. (2023). Learning to generalise but not segment an artificial language at 17 months predicts children’s language skills 3 years later. Cognitive Psychology, 147, 101607. https://doi.org/10.1016/j.cogpsych.2023.101607

Morgan-Short, K., Finger, I., Grey, S., & Ullman, M. T. (2012). Second language processing shows increased native-like neural responses after months of no exposure. PLOS ONE, 7(3), e32974. https://doi.org/10.1371/journal.pone.0032974

Pereira Soares, S. M., Kupisch, T., & Rothman, J. (2022). Testing potential transfer effects in heritage and adult L2 bilinguals acquiring a mini grammar as an additional language: An ERP approach. Brain Sciences, 12(5), 669. https://doi.org/10.3390/brainsci12050669

Rothman, J. (2011). L3 syntactic transfer selectivity and typological determinacy: The typological primacy model. Second Language Research, 27(1), 107–127. https://doi.org/10.1177/0267658310386439

Rothman, J., Alemán Bañón, J., & González Alonso, J. (2015). Neurolinguistic measures of typological effects in multilingual transfer: Introducing an ERP methodology. Frontiers in Psychology, 6. https://doi.org/10.3389/fpsyg.2015.01087

Sala, G., & Gobet, F. (2017). Does far transfer exist? Negative evidence from chess, music, and working memory training. Current Directions in Psychological Science, 26(6), 515–520. https://doi.org/10.1177/0963721417712760

Samuels, W. E., Tournaki, N., Blackman, S., & Zilinski, C. (2016). Executive functioning predicts academic achievement in middle school: A four-year longitudinal study. The Journal of Educational Research, 109(5), 478–490. https://doi.org/10.1080/00220671.2014.979913

Stowe, L. A., Kaan, E., Sabourin, L., & Taylor, R. C. (2018). The sentence wrap-up dogma. Cognition, 176, 232–247. https://doi.org/10.1016/j.cognition.2018.03.011

Swanson, H. L. (2015). Growth in working memory and inhibition predicts literacy in English language learners: A cross-sectional and longitudinal study. Memory, 23(5), 748–773. https://doi.org/10.1080/09658211.2014.927504

Uddén, J., & Männel, C. (2018). Artificial grammar learning and its neurobiology in relation to language processing and development. In S.-A. Rueschemeyer & M. G. Gaskell (Eds.), The Oxford Handbook of Psycholinguistics (p. 0). Oxford University Press. https://doi.org/10.1093/oxfordhb/9780198786825.013.33

Wendebourg, K., Öttl, B., Meurers, D., & Kaup, B. (2025). Semantic information boosts the acquisition of a novel grammatical system in different presentation formats. Language and Cognition, 17, e30. https://doi.org/10.1017/langcog.2023.47

Westergaard, M., Mitrofanova, N., Mykhaylyk, R., & Rodina, Y. (2017). Crosslinguistic influence in the acquisition of a third language: The Linguistic Proximity Model. International Journal of Bilingualism, 21(6), 666–682. https://doi.org/10.1177/1367006916648859

Smart starts: Cognitive differences predict prior knowledge involvement in language learning

Fri, 22 Nov 2024 00:00:00 +0000

Overview

graph TD A["Individual differences measured at onset:
SRT task (procedural memory),
digit-span (working memory),
Stroop task (inhibitory control)"] --> B["Longitudinal artificial language
(AL) learning paradigm"] B --> C["Grammatical and lexical similarity between
ALs and previous languages (Norwegian-English
or Spanish-English) systematically manipulated"] C --> D["Behavioral sensitivity to grammatical
violations measured after each
of three training sessions"] D --> E["Ability to capitalize on prior knowledge
modulated by procedural memory,
working memory and inhibitory control"]

Investigating language learning and morphosyntactic transfer longitudinally using artificial languages

Thu, 05 Sep 2024 00:00:00 +0000

Overview

graph TD A["Session 1: executive functions assessment
and language history questionnaire"] --> B["Session 2: resting-state EEG,
then gender agreement"] B --> C["Session 3: adds
differential object marking"] C --> D["Session 4: adds
verb-object agreement"] D --> E["Session 5: retest
of cognitive battery"] E --> F["Session 6: all properties
retested after four months"] G["Two sites: Mini-Norwegian or Mini-English in Norway,
Mini-Spanish or Mini-English in Spain"] --> B H["Each grammar session: vocabulary pre-training,
grammar training, behavioural test, ERP experiment"] --> B

References

González Alonso, J., Alemán Bañón, J., DeLuca, V., Miller, D., Pereira Soares, S. M., Puig-Mayenco, E., Slaats, S., & Rothman, J. (2020). Event related potentials at initial exposure in third language acquisition: Implications from an artificial mini-grammar study. Journal of Neurolinguistics, 56, 100939. https://doi.org/10.1016/j.jneuroling.2020.100939

Morgan-Short, K., Finger, I., Grey, S., & Ullman, M. T. (2012). Second language processing shows increased native-like neural responses after months of no exposure. PLOS ONE, 7(3), e32974. https://doi.org/10.1371/journal.pone.0032974

Pereira Soares, S. M., Kupisch, T., & Rothman, J. (2022). Testing potential transfer effects in heritage and adult L2 bilinguals acquiring a mini grammar as an additional language: An ERP approach. Brain Sciences, 12(5), Article 5. https://doi.org/10.3390/brainsci12050669

Rogala, J., Kublik, E., Krauz, R., & Wróbel, A. (2020). Resting-state EEG activity predicts frontoparietal network reconfiguration and improved attentional performance. Scientific Reports, 10(1), 5064. https://doi.org/10.1038/s41598-020-61866-7

Your browser does not support embedded PDFs. Download the PDF instead.

Making research materials Findable, Accessible, Interoperable and Reusable

Thu, 05 Sep 2024 00:00:00 +0000

Your browser does not support embedded PDFs. Download the PDF instead.

Overview

graph TD A["Base files with minimal
language components"] --> B["Modular R framework with
interoperable components"] B --> C["Controls: counterbalance gender and
number, rotate words, control frequency"] C --> D["compile all stimuli script
regenerates final stimuli"] D --> E["Stimuli saved to
session materials folder"] E --> F["Presentation in OpenSesame
across several sessions"] F --> G["Custom script time-locks EEG
to stimulus onset for ERPs"] G --> H["Accessible materials, reproducible
workflow, expandable design"]

Snippet 1

The use of code scripts facilitates the reproducibility, testability and expandability of materials.

└── stimulus_preparation
├── Norway site, base stimuli.csv
├── Spain site, base stimuli.csv
├── base_images.R
├── R_functions
│ ├── Session2_Pretraining_vocabulary.R
│ ├── Session2_Training_gender_agreement.R
│ ├── Session2_Test_gender_agreement.R
│ ├── Session2_Experiment_gender_agreement.R
...
├── compile_all_stimuli.R

Table 1

The minimal components of each language are contained in a base file.

verb_ID	verb_type	verb	verb_contrast_ID
1	copula_be	is
2	copula_be	are
3	copula_look	looks
4	copula_look	look
5	transitive	remembered	A
6	transitive	forgot	A
7	transitive	chose	B
8	transitive	refused	B

Snippet 2

Tests were set throughout the workflow to control the frequency of some categories (R code).

columns_to_check = c('noun1_gender', 'number', 'person',
'verb', 'noun1', 'wrapup_noun')
for(i in seq_along(columns_to_check)) {
column = columns_to_check[i]
number_of_unique_frequencies =
combinations %>%
filter(complete.cases(get(column)), get(column) != '') %>%
group_by(get(column)) %>% tally() %>% select(n) %>%
n_distinct()
if(number_of_unique_frequencies != 1) {
warning(paste0('Some elements in the column `', column,
'` appear more often than others.'))
}
}

Snippet 3

Seamless adjustment of parameters in each OpenSesame session (Python code).

participant_parameters =
pd.read_csv(exp.get_file('../parameters per participant/' +
var.study_site +
' site, parameters per participant.csv'))
var.resting_state_order =
participant_parameters.loc[
participant_parameters['participant'] ==
var.subject_nr]['Session2_resting_state_order'].iloc[0]
var.language =
participant_parameters.loc[participant_parameters['participant'] ==
var.subject_nr]['language'].iloc[0]

Snippet 4

Sending serial-port triggers in OpenSesame to record ERPs (Python code).

# Open the first serial port available
serialport = serial.Serial(serial.tools.list_ports.comports()[0].device)
# Send triggers to the port
def send_trigger(trigger):
serialport.write(trigger.to_bytes(length = 1, byteorder = 'big'))
# 10 ms separation from next trigger (see BrainVision Recorder manual)
time.sleep(0.01)
# reset port
serialport.write(int(0).to_bytes(length = 1, byteorder = 'big'))
return;

References

Barsalou, L. W. (2019). Establishing generalizable mechanisms. Psychological Inquiry, 30(4), 220–230. https://doi.org/10.1080/1047840X.2019.1693857

Cross, Z. R., Zou-Williams, L., Wilkinson, E. M., Schlesewsky, M., & Bornkessel-Schlesewsky, I. (2021). Mini Pinyin: A modified miniature language for studying language learning and incremental sentence processing. Behavior Research Methods, 53(3), 1218–1239. https://doi.org/10.3758/s13428-020-01473-6

González Alonso, J., Alemán Bañón, J., DeLuca, V., Miller, D., Pereira Soares, S. M., Puig-Mayenco, E., Slaats, S., & Rothman, J. (2020). Event related potentials at initial exposure in third language acquisition: Implications from an artificial mini-grammar study. Journal of Neurolinguistics, 56, 100939. https://doi.org/10.1016/j.jneuroling.2020.100939

Mitrofanova, N., Leivada, E., & Westergaard, M. (2023). Crosslinguistic influence in L3 acquisition: Evidence from artificial language learning. Linguistic Approaches to Bilingualism, 13(5), 717-742. https://doi.org/10.1075/lab.22063.mit

Pereira Soares, S. M., Kupisch, T., & Rothman, J. (2022). Testing potential transfer effects in heritage and adult L2 bilinguals acquiring a mini grammar as an additional language: An ERP approach. Brain Sciences, 12(5), Article 5. https://doi.org/10.3390/brainsci12050669

Wilkinson, M. D., Dumontier, M., Aalbersberg, Ij. J., Appleton, G., Axton, M., Baak, A., Blomberg, N., Boiten, J.-W., da Silva Santos, L. B., Bourne, P. E., Bouwman, J., Brookes, A. J., Clark, T., Crosas, M., Dillo, I., Dumon, O., Edmunds, S., Evelo, C. T., Finkers, R., … Mons, B. (2016). The FAIR Guiding Principles for scientific data management and stewardship. Scientific Data, 3(1), Article 1. https://doi.org/10.1038/sdata.2016.18

Discussion of Labotka et al. (2023)

Mon, 22 May 2023 00:00:00 +0000

Slideshare

Labotka, D., Sabo, E., Bonais, R., Gelman, S. A., & Baptista, M. (2023). Testing the effects of congruence in adult multilingual acquisition with implications for creole genesis. Cognition, 235, 105387. https://doi.org/10.1016/j.cognition.2023.105387

Discussion of Jost et al. (2019)

Mon, 30 Jan 2023 00:00:00 +0000

Slideshare

Jost, E., Brill-Schuetz, K., Morgan-Short, K., & Christiansen, M. H. (2019). Input complexity affects long-term retention of statistically learned regularities in an artificial language learning task. Frontiers in Human Neuroscience, 13, 358. https://www.frontiersin.org/articles/10.3389/fnhum.2019.00358

Linguistic and embodied systems in conceptual processing: Variation across individuals and items

Mon, 24 May 2021 00:00:00 +0000

Overview

graph TD A["Word is processed"] --> B["Linguistic system
activated first"] A --> C["Embodied system
activated thereupon"] B --> D["Variation across
individuals and items"] C --> D D --> E["Study 1:
merge existing datasets"] D --> F["Study 2: collect novel data on
vocabulary size, sensorimotor experience
and attentional control"] F --> G["Pilot studies
with larger samples"] G --> H["Simulation based power curves"]

Video

Slides

Abstract

Research in conceptual processing has suggested that the comprehension of words draws on complementary cognitive systems. In the milliseconds during which a word is processed, the linguistic system is activated first. Reading the word entity, for instance, may activate words such as being, thing and object (https://smallworldofwords.org/en/project/explore; De Deyne et al., 2019). Thereupon, the embodied system is activated, incorporating sensorimotor, emotional and social dimensions (Borghi et al., 2019). For instance, entity activates visual, auditory and head-specific meanings (https://embodiedcognitionlab.shinyapps.io/sensorimotor_norms; Lynott et al., 2020). Research has also suggested that the linguistic system is more important for relatively abstract words—e.g., attempt—, whereas the embodied system is more important for more concrete words—e.g., building (Bolognesi & Steen, 2018). The role of individual differences has also been investigated (Dils & Boroditsky, 2010), although to a lesser extent. An individual’s linguistic experience (e.g., larger vocabulary) facilitates word processing and task-relevant attention (Pexman & Yap, 2018; Yap et al., 2017), while greater sensorimotor experience enables more detailed meaning activation within specific conceptual areas (e.g., space: Vukovic & Williams, 2015). The variation across individuals and items, within both the linguistic and embodied systems, is seldom considered simultaneously. We are undertaking this in two studies. The first study (Bernabeu et al., 2021) will merge existing datasets (Lynott et al., 2020; Pexman et al., 2017; Pexman & Yap, 2018; Wingfield & Connell, 2019). The second study will collect novel data to investigate questions such as the unique roles of vocabulary size, sensorimotor experience and attentional control. To determine the sample size for the latter study, two pilot studies with larger-than-average samples were conducted, using the aforementioned datasets and that of Hutchison et al. (2013). Simulation-based, prospective power curves were performed. These pilots revealed important roles for linguistic and embodied information, vocabulary size, and attentional control, as well as statistical power considerations.

References

Bernabeu, P., Lynott, D., & Connell, L. (2021). Preregistration: The interplay between linguistic and embodied systems in conceptual processing. OSF. https://osf.io/ftydw/

Bolognesi, M., & Steen, G. (2018). Abstract concepts: Structure, processing, and modeling. Topics in Cognitive Science, 10(3), 490–500. https://doi.org/10.1111/tops.12354

Borghi, A., M., Barca, L., Binkofski, F., Castelfranchi, C., Pezzulo, G., & Tummolini, L. (2019). Words as social tools: Language, sociality and inner grounding in abstract concepts. Physics of Life Reviews, 29, 120–53. https://doi.org/10.1016/j.plrev.2018.12.001

De Deyne, S., Navarro, D. J., Perfors, A., Brysbaert, M., & Storms, G. (2019). The “Small World of Words” English word association norms for over 12,000 cue words. Behavior Research Methods, 51, 987–1006. http://dx.doi.org/10.3758/s13428-018-1115-7

Dils, A. T., & Boroditsky, L. (2010). Visual motion aftereffect from understanding motion language. Proceedings of the National Academy of Sciences, 107(37), 16396-16400. https://doi.org/10.1073/pnas.1009438107

Hutchison, K. A., Balota, D. A., Neely, J. H., Cortese, M. J., Cohen-Shikora, E. R., Tse, C.-S., Yap, M. J., Bengson, J. J., Niemeyer, D., & Buchanan, E. (2013). The semantic priming project. Behavior Research Methods, 45, 1099–1114. https://doi.org/10.3758/s13428-012-0304-z

Lynott, D., Connell, L., Brysbaert, M., Brand, J., & Carney, J. (2020). The Lancaster Sensorimotor Norms: Multidimensional measures of perceptual and action strength for 40,000 English words. Behavior Research Methods, 52, 1-21. https://doi.org/10.3758/s13428-019-01316-z

Pexman, P. M., Heard, A., Lloyd, E., & Yap, M. J. (2017). The Calgary semantic decision project: Concrete/abstract decision data for 10,000 English words. Behavior Research Methods, 49(2), 407–417. https://doi.org/10.3758/s13428-016-0720-6

Pexman, P. M., & Yap, M. J. (2018). Individual differences in semantic processing: Insights from the Calgary semantic decision project. Journal of Experimental Psychology: Learning, Memory, and Cognition, 44(7), 1091–1112. https://doi.org/10.1037/xlm0000499

Vukovic, N., & Williams, J. N. (2015). Individual differences in spatial cognition influence mental simulation of language. Cognition, 142, 110–122. https://doi.org/10.1016/j.cognition.2015.05.017

Wingfield, C., & Connell, L. (2019). Understanding the role of linguistic distributional knowledge in cognition. PsyArXiv. https://doi.org/10.31234/osf.io/hpm4z

Yap, M. J., Hutchison, K. A., & Tan, L. C. (2017). Individual differences in semantic priming performance: Insights from the semantic priming project. In M. N. Jones (Ed.), Frontiers of cognitive psychology. Big data in cognitive science (p. 203–226). Routledge/Taylor & Francis Group.

Towards reproducibility and maximally-open data

Fri, 14 May 2021 00:00:00 +0000

Slideshare

Mixed-effects models in R and a new tool for data simulation

Thu, 26 Nov 2020 00:00:00 +0000

Overview

graph TD A["Variation within factors
(participants, trials, items)"] --> B["Linear mixed-effects models:
random intercepts and random slopes"] B --> C["Fewer false positives
and false negatives"] B --> D["Fit models in R"] D --> E["Maximal approach:
all possible random effects"] E --> F["Model sometimes
fails to converge"] F --> G["Progressively lighten the
random effects structure"] D --> H["Web application
for data simulation"]

Slides

Abstract

Linear mixed-effects models (LMEMs) are used to account for variation within factors with multiple observations, such as participants, trials, items, channels, etc (for an earlier approach, see Clark, 1973). This variation is modelled in terms of random intercepts (e.g., overall variation per participant) as well as random slopes for the fixed effects (e.g., treatment effect per participant). These measures help reduce false positives and false negatives (Barr et al., 2013), and the resulting models tend to be robust to violations of assumptions (Schielzeth et al., 2020). The use of LMEMs has grown over the past decade, under various implementation forms (Meteyard & Davies, 2020). In this talk, I will look over the rationale for LMEMs, and demonstrate how to fit them in R (Brauer & Curtin, 2018; Luke, 2017). Challenges will also be covered. For instance, when using the widely-accepted ‘maximal’ approach, based on fitting all possible random effects for each fixed effect, models sometimes fail to find a solution, or ‘convergence’. Advice for the problem of nonconvergence will be demonstrated, based on the progressive lightening of the random effects structure (Singman & Kellen, 2017; for an alternative approach, especially with small samples, see Matuschek et al., 2017). At the end, on a different note, I will present a web application that facilitates data simulation for research and teaching (Bernabeu & Lynott, 2020).

References

Barr, D. J., Levy, R., Scheepers, C., & Tily, H. J. (2013). Random effects structure for confirmatory hypothesis testing: Keep it maximal. Journal of Memory and Language, 68, 255–278. http://dx.doi.org/10.1016/j.jml.2012.11.001

Bernabeu, P., & Lynott, D. (2020). Web application for the simulation of experimental data (Version 1.2). https://github.com/pablobernabeu/Experimental-data-simulation/

Brauer, M., & Curtin, J. J. (2018). Linear mixed-effects models and the analysis of nonindependent data: A unified framework to analyze categorical and continuous independent variables that vary within-subjects and/or within-items. Psychological Methods, 23(3), 389–411. https://psych.wisc.edu/Brauer/BrauerLab/wp-content/uploads/2014/04/Brauer-Curtin-2018-on-LMEMs.pdf

Clark, H. H. (1973). The language-as-fixed-effect fallacy: A critique of language statistics in psychological research. Journal of Verbal Learning and Verbal Behavior, 12(4), 335-359. https://doi.org/10.1016/S0022-5371(73)80014-3

Luke, S. G. (2017). Evaluating significance in linear mixed-effects models in R. Behavior Research Methods, 49(4), 1494–1502. https://doi.org/10.3758/s13428-016-0809-y

Matuschek, H., Kliegl, R., Vasishth, S., Baayen, H., & Bates, D. (2017). Balancing type 1 error and power in linear mixed models. Journal of Memory and Language, 94, 305–315. https://doi.org/10.1016/j.jml.2017.01.001

Meteyard, L., & Davies, R. A. (2020). Best practice guidance for linear mixed-effects models in psychological science. Journal of Memory and Language, 112, 104092. https://doi.org/10.1016/j.jml.2020.104092

Schielzeth, H., Dingemanse, N. J., Nakagawa, S., Westneat, D. F., Allegue, H, Teplitsky, C., Reale, D., Dochtermann, N. A., Garamszegi, L. Z., & Araya-Ajoy, Y. G. (2020). Robustness of linear mixed-effects models to violations of distributional assumptions. Methods in Ecology and Evolution, 00, 1– 12. https://doi.org/10.1111/2041-210X.13434

Singmann, H., & Kellen, D. (2019). An Introduction to Mixed Models for Experimental Psychology. In D. H. Spieler & E. Schumacher (Eds.), New Methods in Cognitive Psychology (pp. 4–31). Hove, UK: Psychology Press. http://singmann.org/download/publications/singmann_kellen-introduction-mixed-models.pdf

Reproducibilidad en torno a una aplicación web

Thu, 08 Oct 2020 00:00:00 +0000

Overview

graph TD A["Aplicacion web en R
(shiny, flexdashboard)"] --> B["Uso sin programar"] A --> C["Exportar registro
de actividad"] A --> D["Codigo abierto:
compartir, investigar, editar"] A --> E["Exportar codigo R
ajustado a cada usuario"] C --> F["Reproducibilidad"] D --> F E --> F

Resumen

Las aplicaciones web nos ayudan a facilitar el uso de nuestro trabajo, ya que no requieren programación para utilizarlas (ver ejemplos). Crear estas aplicaciones en R, mediante paquetes como “shiny” o “flexdashboard”, ofrece múltiples ventajas. Entre ellas destaca la reproducibilidad, tal como veremos en torno a una aplicación para la simulación de datos. Por un lado, los usuarios pueden exportar un registro de su actividad. Por otro lado, el código utilizado para crear estas aplicaciones se puede compartir, investigar y editar con la facilidad que ofrece un lenguaje de código abierto como R. Esto facilita el uso gratuito, el desarrollo colaborativo y una documentación accesible sobre cualquiera de los paquetes utilizados. Por último, la reproducibilidad se puede maximizar si se facilita a los usuarios que lo deseen la exportación de un código de R ajustado a sus requerimientos (más allá del código de la aplicación en general), lo cual añadiría a la aplicación las ventajas de un paquete de R. Esta última opción (no disponible actualmente en la aplicación de simulación, ni en la mayoría de las aplicaciones) se puede habilitar adaptando el código de la aplicación a funciones básicas de R.

Details on some presentations | Pablo Bernabeu

Scaling systematic reviews: A solo researcher's workflow with Gemini

Slides

Overview

Background References

Unpacking ERP responses in artificial language learning

PDF slides

Overview

Longer summary

References

Smart starts: Cognitive differences predict prior knowledge involvement in language learning

Overview

Investigating language learning and morphosyntactic transfer longitudinally using artificial languages

Overview

References

Poster

Making research materials Findable, Accessible, Interoperable and Reusable

Poster

Overview

Snippet 1

Table 1

Snippet 2

Snippet 3

Snippet 4

References

Discussion of Labotka et al. (2023)

Discussion of Jost et al. (2019)

Linguistic and embodied systems in conceptual processing: Variation across individuals and items

Overview

Abstract

References

Towards reproducibility and maximally-open data

Mixed-effects models in R and a new tool for data simulation

Overview

Slides

Abstract

References

Reproducibilidad en torno a una aplicación web

Overview

Resumen

Vídeo y filminas