Presentations

Abstract

Nearly all human languages have grammars with complex recursive structures. These structures pose notable learning challenges. Two distributional properties of the input may facilitate learning: the presence of semantic biases (e.g. p(barks|dog) > p(talks|dog)) and the Zipf-distribution, with short sentences being extremely more frequent than longer ones. This project tested the effect of these sources of information on statistical learning of a hierarchical center-embedding grammar, using an artificial grammar learning paradigm. Semantic biases were represented by variations in transitional probabilities between words, with a biased input (p(barks|dog) > p(talks|dog)) compared to a non-biased input (p(barks|dog) = p(talks|dog)). The Zipf distribution was compared to a flat distribution, with sentences of different lengths occurring equally often. In a 2×2 factorial design, we tested for effects of biased transitional probabilities (biased/non-biased) and the distribution of sequences with varying length (Zipf distribution/flat distribution) on implicit learning and explicit ratings of grammaticality. Preliminary results show that a Zipf-shaped and semantically biased input facilitates grammar learnability. Thus, this project contributes to understanding how we learn complex structures with long-distance dependencies: learning may be sensitive to the specific distributional properties of the linguistic input, mirroring meaningful aspects of the world and favoring short utterances.

Abstract

The neural infrastructure for sentence production and comprehension has been found to be mostly shared. The same regions are engaged during speaking and listening, with some differences in their loading depending on modality (Giglio et al., 2022). In this fMRI study (n=40), we investigated whether modality affects the connectivity between inferior frontal and temporal regions, previously found to be involved in syntactic processing across modalities, and with the cerebellum, which has been historically linked with motor aspects of production. Participants produced or listened to word sequences of increasing constituent size. We found that constituent size reliably increased the connectivity between the inferior frontal gyrus and the posterior temporal lobe in both modalities. Preliminary cerebellar results suggest that different sub-regions presented different patterns of connectivity. Connectivity between Lobule VI and (pre)motor regions was increased during production relative to comprehension. Connectivity between Crus I/II and fronto-temporal regions was instead increased as a function of constituent size, and in particular during production. These results thus show that the connectivity between fronto-temporal regions is upregulated for syntactic structure building in both sentence production and comprehension, while cortico-cerebellar connectivity is enhanced both in response to syntactic processing and during production.

Abstract

Language comprehension may be facilitated by prediction, where a listener’s eye-gaze moves towards a referent before it is mentioned if the noun is predictable. Anticipatory fixations reduce when speech contains disfluencies (hesitations/repairs). Changes to the pattern of anticipatory fixations could result from a change in prediction or an attention shift. We combined EEG and eye-tracking to study the prediction of language in naturalistic, virtual environments (experiment 1 & 2) and the influence of disfluencies on predicting language (experiment 2). Participants (n=32; preliminary n=19) listened to sentences spoken by a virtual agent in various virtual scenes (e.g., office, street) while participants’ eye-movements and EEG were recorded. Spoken sentences were predictable or unpredictable, based on the verb constraints and referents were visible or absent in the scene to be congruent or incongruent with listeners’ predictions, respectively. In experiment 2, sentences were additionally either fluent or disfluent with ahesitation (uhh). Increased processing, reflected in increased theta power, was greater either at the predictive verb onset or at unpredictable noun onset in fluent sentences, but was observed at both predictable and unpredictable noun onsets in disfluent sentences. Our findings provide preliminary evidence supporting that hesitations reduce the weight listeners place on their predictions.

Abstract

Effective communication involves recognizing the disparity between our own perspective and that of the recipient, influenced by factors such as stereotypes and accents. The extent to which Theory of Mind (ToM), the ability to ascribe beliefs to others, plays a role in this process is uncertain. We anticipate individuals to speak in line with their beliefs, but what if their words conflict with our expectations? To investigate, we devised a virtual perspective-taking experiment where we manipulated a virtual agent's beliefs. Electroencephalography data were collected as participants listened to statements from the agent that either aligned or clashed with their true or false beliefs. We focused on the N400, an event-related brain component linked to word unexpectedness. As hypothesized, statements inconsistent with the agent's true beliefs triggered more pronounced N400 responses compared to matching statements. Furthermore, we anticipated that when the agent held a false belief, this knowledge would factor into interpreting their statements. Neither statements aligned with nor those diverging from the agent's false beliefs evoked N400 responses. This can be taken as evidence that participants did take the agents perspective into account. These results strongly support the role of Theory of Mind in language comprehension.

Abstract

Human communication requires interlocutors to mutually understand each other. Previous research has suggested inter-brain synchrony as an important feature of social interaction, since it has been observed during joint attention, speech interactions and cooperative tasks. Nonetheless, it is still unknown whether inter-brain synchrony is actually related to successful face-to-face communication. Here, we use dual-EEG to study if inter-brain synchrony is modulated during episodes of successful and unsuccessful communication in clear and noisy communication settings. Dyads performed a tangram-based referential communication task with and without background noise, while both their EEG and audiovisual behavior was recorded. Other-initiated repairs were annotated in the audiovisual data and were used as indexes of unsuccessful and successful communication. More specifically, we compared inter-brain synchrony during episodes of miscommunication (repair initiations) and episodes of mutual understanding (repair solutions and acceptance phases) in the clear and the noise condition. We expect that when communication is successful, inter-brain synchrony will be stronger than when communication is unsuccessful, and we expect that these patterns will be most pronounced in the noise condition. Results are currently being analyzed and will be presented and discussed with respect to the inter-brain neural signatures underlying the process of mutual understanding in face-to-face conversation.

Abstract

Monitoring auditory feedback is important for fluent speech production as it enables correction of vocalization errors. Influence of auditory feedback is best illustrated by manipulating it during speech production. A common temporal manipulation technique is delaying auditory feedback (DAF), which disrupts speech fluency, and a common spectral manipulation technique is perturbing the pitch of auditory feedback, which elicits vocal changes. We aimed to understand whether there is a correlation between sensitivity to temporal versus spectral manipulations of auditory feedback. We collected data from 40 participants. In the DAF task, participants repeated sentences, and auditory feedback was presented with 0 or 200ms delay. In the pitch perturbation task, participants phonated the vowel /a/ for 4 seconds and pitch of the auditory feedback was shifted by ±100 or ±200 cents.Voice recordings were analyzed using LME models to test the effects of feedback manipulations. We found that DAF significantly prolonged articulation duration and increased both voice pitch and intensity. Additionaly we found that large pitch shifts elicited less compensatory responses. There was a large individual variability in sensitivity to feedback manipulations for both tasks, however there was no correlation between the sensitivity profiles between tasks, suggesting that these features are processed differently.

Abstract

Phonological word forms are maintained in long-term memory and rapidly accessed during speech. However, the neural mechanisms that support the acquisition, maintenance, and recollection of word-form memories remain unclear. Starting from the hypothesis that dendrites and cell assemblies are the neural substrate for associating phoneme sequences to words, the present study investigates the dynamics of lexical access in a biologically constrained network model and compares it to the computational principles of human spoken word recognition. The model is a spiking recurrent network of dendritic neurons with realistic physiology and connectivity. The network implements unsupervised plasticity in the excitatory and inhibitory synapses, resulting in stable associations between phonemic and word representations. The activity of word assemblies indicates that, first, lexical representations are activated incrementally; second, lexical neighbors delay correct lexical access with both cohort and rhyme groups reactivated; and third, partial phonemic mismatch degrades word recognition. We also derive the model's predictions on the lexical bias effect and show that it is consistent with the hypothesis of offline feedback, providing a plausible mechanism for perceptual learning. In conclusion, we show that the dendrites could be the missing link between the perceptual space of phonemes and the mental lexicon.

Abstract

A series of results from event-related brain potential recordings and fMRI research will be presented, suggesting that language processing does not obey strict compositionality, and, moreover immediately recruits extralinguistic information. It will also be shown that pragmatic inferences require contributions from TOM networks. This implies that an embodied account of semantics fails (under the somewhat strange assumption that the brain is not part of the body). I will put forward an embrained perspective on language processing.

Abstract

In my presentation I will summarize the results of a number of ERP and fMRI studies on the processing and integration of co-occurring speech and gestures/pantomimes. The ERP results indicate that the time course of integrating language and action (i.e., gesture) is very similar to that of integrating linguistic meaning into a sentence or discourse representation. Moreover, in both cases the Left Inferior Frontal cortex plays a central role in orchestrating the multimodal unification of language and action. This orchestration is partly done by modulating temporal areas that store representations activated by the input. I will discuss the parameters of this modulation.

Abstract

For over 20 years, researchers have investigated syntactic priming of transitives by measuring the frequency of sentence choice. These studies have shown syntactic priming for passives but weaker or absent syntactic priming for actives (e.g. [1],[2]). Until recently very few studies have reported syntactic priming by measuring reaction times of language production and none of these studies included transitive sentences. We previously found syntactic priming of both active and passive Dutch transitives in speech onsets, and interestingly the effect appeared to be stronger for actives than passives [3][4]. In order to explain the discrepancy between our results and other results reported in the literature, we hypothesized that there is a ceiling effect in the frequency of using an active transitive (in general about 94% of produced transitives are actives) but not in the speed of producing one. To confirm this hypothesis we conducted a syntactic priming experiment with a picture description task measuring both reaction times and the frequency of occurrence on the same trials. This way we could exclude the alternative explanation that the discrepancy in results is caused by differences in design or stimuli. The results of this experiment show syntactic priming effects for passives and not for actives in the frequency of occurrence (in line with the literature). However, the reaction times of producing these sentences do show syntactic priming for actives and these effects appear to be even stronger than for passives (in line with our previous data). In conclusion, our results suggest that measuring the frequency of occurrence is not enough to get a complete picture of syntactic priming. Measuring reaction times in addition to the frequency of occurrence could provide us with a more complete picture. References [1] Bock, K., & Loebell, H. (1990). Framing sentences. Cognition, 35, 1-39. [2] Hartsuiker, R. J., & Kolk, H. H. J. (1998). Syntactic persistence in Dutch sentence production. Language and Speech, 41, 143-184. [3] Menenti, L., Segaert, K., & Hagoort, P. (2008). Repetition suppression for syntax and semantics: overt speech in fMRI. Society for Neuroscience, Washington DC. [4] Segaert, K., Menenti, L., & Hagoort, P. (2009) Scanning speech with fMRI: Short and long term priming of syntax and verbs, CUNY conference on human sentence processing, Davis, CA.

Abstract

Syntactic priming has been frequently used to study syntactic processes in language production in monolinguals [1][2] and bilinguals [3]. In a previous study in language comprehension [4] we showed that passive sentences in English (the participant’s L2) can be primed by passive sentences in German (L1) and English (L2). This was manifested in faster reading times for target sentences and repetition suppression effects in left inferior frontal, left precentral and left middle temporal regions of interest in an fMRI study. However, syntactic priming in comprehension is complicated by the influence of verb repetition between prime and target [5][6]. Therefore, we conducted a reading time and fMRI study looking at the influence of verb repetition on syntactic priming. In this study of monolingual comprehension in Dutch we primed passive sentences as well as sentences with crossed-dependency structures. The reading time results revealed a syntactic priming effect for passive sentences, while the effect for crossed-dependency structure sentences interacted with the factor verb repetition. The preliminary fMRI results suggest that the repetition of passive structures leads to reductions in neural activity. The repetition of crossed dependency structures causes repetition enhancement, an increase in the BOLD response, an effect that interacts with the factor verb repetition. In conclusion, the influence of verb repetition on syntactic priming in comprehension is complex and seems to depend on the type of syntactic structure investigated. References [1] Bock K. (1986). Syntactic persistence in language production. Cognitive Psychology, 18(3), 355-387. [2] Pickering M, & Branigan H. (1999). Syntactic priming in language production. Trends in Cognitive Sciences, 3(4), 136-141. [3] Schoonbaert S, Hartsuiker RJ, & Pickering MJ. (2007). The representation of lexical and syntactic information in bilinguals: Evidence from syntactic priming. Journal of Memory and Language, 56(2), 153-171. [4] Weber K, & Indefrey P. (in press). Syntactic priming in German-English bilinguals during sentence comprehension. NeuroImage. [5] Arai M, van Gompel R, & Scheepers C. (2007). Priming ditransitive structures in comprehension. Cognitive Psychology, 54, 218-250. [6] Thothathiri M, & Snedeker J. (2008). Give and take: Syntactic priming during spoken language comprehension. Cognition, 108(1), 51-68.