Presentations

Abstract

Nearly all human languages have grammars with complex recursive structures. These structures pose notable learning challenges. Two distributional properties of the input may facilitate learning: the presence of semantic biases (e.g. p(barks|dog) > p(talks|dog)) and the Zipf-distribution, with short sentences being extremely more frequent than longer ones. This project tested the effect of these sources of information on statistical learning of a hierarchical center-embedding grammar, using an artificial grammar learning paradigm. Semantic biases were represented by variations in transitional probabilities between words, with a biased input (p(barks|dog) > p(talks|dog)) compared to a non-biased input (p(barks|dog) = p(talks|dog)). The Zipf distribution was compared to a flat distribution, with sentences of different lengths occurring equally often. In a 2×2 factorial design, we tested for effects of biased transitional probabilities (biased/non-biased) and the distribution of sequences with varying length (Zipf distribution/flat distribution) on implicit learning and explicit ratings of grammaticality. Preliminary results show that a Zipf-shaped and semantically biased input facilitates grammar learnability. Thus, this project contributes to understanding how we learn complex structures with long-distance dependencies: learning may be sensitive to the specific distributional properties of the linguistic input, mirroring meaningful aspects of the world and favoring short utterances.

Abstract

The neural infrastructure for sentence production and comprehension has been found to be mostly shared. The same regions are engaged during speaking and listening, with some differences in their loading depending on modality (Giglio et al., 2022). In this fMRI study (n=40), we investigated whether modality affects the connectivity between inferior frontal and temporal regions, previously found to be involved in syntactic processing across modalities, and with the cerebellum, which has been historically linked with motor aspects of production. Participants produced or listened to word sequences of increasing constituent size. We found that constituent size reliably increased the connectivity between the inferior frontal gyrus and the posterior temporal lobe in both modalities. Preliminary cerebellar results suggest that different sub-regions presented different patterns of connectivity. Connectivity between Lobule VI and (pre)motor regions was increased during production relative to comprehension. Connectivity between Crus I/II and fronto-temporal regions was instead increased as a function of constituent size, and in particular during production. These results thus show that the connectivity between fronto-temporal regions is upregulated for syntactic structure building in both sentence production and comprehension, while cortico-cerebellar connectivity is enhanced both in response to syntactic processing and during production.

Abstract

Language comprehension may be facilitated by prediction, where a listener’s eye-gaze moves towards a referent before it is mentioned if the noun is predictable. Anticipatory fixations reduce when speech contains disfluencies (hesitations/repairs). Changes to the pattern of anticipatory fixations could result from a change in prediction or an attention shift. We combined EEG and eye-tracking to study the prediction of language in naturalistic, virtual environments (experiment 1 & 2) and the influence of disfluencies on predicting language (experiment 2). Participants (n=32; preliminary n=19) listened to sentences spoken by a virtual agent in various virtual scenes (e.g., office, street) while participants’ eye-movements and EEG were recorded. Spoken sentences were predictable or unpredictable, based on the verb constraints and referents were visible or absent in the scene to be congruent or incongruent with listeners’ predictions, respectively. In experiment 2, sentences were additionally either fluent or disfluent with ahesitation (uhh). Increased processing, reflected in increased theta power, was greater either at the predictive verb onset or at unpredictable noun onset in fluent sentences, but was observed at both predictable and unpredictable noun onsets in disfluent sentences. Our findings provide preliminary evidence supporting that hesitations reduce the weight listeners place on their predictions.

Abstract

Effective communication involves recognizing the disparity between our own perspective and that of the recipient, influenced by factors such as stereotypes and accents. The extent to which Theory of Mind (ToM), the ability to ascribe beliefs to others, plays a role in this process is uncertain. We anticipate individuals to speak in line with their beliefs, but what if their words conflict with our expectations? To investigate, we devised a virtual perspective-taking experiment where we manipulated a virtual agent's beliefs. Electroencephalography data were collected as participants listened to statements from the agent that either aligned or clashed with their true or false beliefs. We focused on the N400, an event-related brain component linked to word unexpectedness. As hypothesized, statements inconsistent with the agent's true beliefs triggered more pronounced N400 responses compared to matching statements. Furthermore, we anticipated that when the agent held a false belief, this knowledge would factor into interpreting their statements. Neither statements aligned with nor those diverging from the agent's false beliefs evoked N400 responses. This can be taken as evidence that participants did take the agents perspective into account. These results strongly support the role of Theory of Mind in language comprehension.

Abstract

Human communication requires interlocutors to mutually understand each other. Previous research has suggested inter-brain synchrony as an important feature of social interaction, since it has been observed during joint attention, speech interactions and cooperative tasks. Nonetheless, it is still unknown whether inter-brain synchrony is actually related to successful face-to-face communication. Here, we use dual-EEG to study if inter-brain synchrony is modulated during episodes of successful and unsuccessful communication in clear and noisy communication settings. Dyads performed a tangram-based referential communication task with and without background noise, while both their EEG and audiovisual behavior was recorded. Other-initiated repairs were annotated in the audiovisual data and were used as indexes of unsuccessful and successful communication. More specifically, we compared inter-brain synchrony during episodes of miscommunication (repair initiations) and episodes of mutual understanding (repair solutions and acceptance phases) in the clear and the noise condition. We expect that when communication is successful, inter-brain synchrony will be stronger than when communication is unsuccessful, and we expect that these patterns will be most pronounced in the noise condition. Results are currently being analyzed and will be presented and discussed with respect to the inter-brain neural signatures underlying the process of mutual understanding in face-to-face conversation.

Abstract

Monitoring auditory feedback is important for fluent speech production as it enables correction of vocalization errors. Influence of auditory feedback is best illustrated by manipulating it during speech production. A common temporal manipulation technique is delaying auditory feedback (DAF), which disrupts speech fluency, and a common spectral manipulation technique is perturbing the pitch of auditory feedback, which elicits vocal changes. We aimed to understand whether there is a correlation between sensitivity to temporal versus spectral manipulations of auditory feedback. We collected data from 40 participants. In the DAF task, participants repeated sentences, and auditory feedback was presented with 0 or 200ms delay. In the pitch perturbation task, participants phonated the vowel /a/ for 4 seconds and pitch of the auditory feedback was shifted by ±100 or ±200 cents.Voice recordings were analyzed using LME models to test the effects of feedback manipulations. We found that DAF significantly prolonged articulation duration and increased both voice pitch and intensity. Additionaly we found that large pitch shifts elicited less compensatory responses. There was a large individual variability in sensitivity to feedback manipulations for both tasks, however there was no correlation between the sensitivity profiles between tasks, suggesting that these features are processed differently.

Abstract

Phonological word forms are maintained in long-term memory and rapidly accessed during speech. However, the neural mechanisms that support the acquisition, maintenance, and recollection of word-form memories remain unclear. Starting from the hypothesis that dendrites and cell assemblies are the neural substrate for associating phoneme sequences to words, the present study investigates the dynamics of lexical access in a biologically constrained network model and compares it to the computational principles of human spoken word recognition. The model is a spiking recurrent network of dendritic neurons with realistic physiology and connectivity. The network implements unsupervised plasticity in the excitatory and inhibitory synapses, resulting in stable associations between phonemic and word representations. The activity of word assemblies indicates that, first, lexical representations are activated incrementally; second, lexical neighbors delay correct lexical access with both cohort and rhyme groups reactivated; and third, partial phonemic mismatch degrades word recognition. We also derive the model's predictions on the lexical bias effect and show that it is consistent with the hypothesis of offline feedback, providing a plausible mechanism for perceptual learning. In conclusion, we show that the dendrites could be the missing link between the perceptual space of phonemes and the mental lexicon.

Abstract

Although semantic influences on verbal short-term memory (STM) performance are well-established, substantially less research has studied the influence of syntactic representation. In the present study, syntactic and semantic factors were manipulated in order to explore how both interact to influence verbal STM. Subjects performed immediate, serial recall on lists of six Dutch words composed of three sets of adjective-noun pairs, where the nouns were either common (‘de’) or neuter (‘het’) gender. The grammaticality of the word pairs was manipulated through the morphological agreement between the adjectives and nouns (either legal of illegal), and the semantics by creating more or less meaningful word pairs (e.g., big bucket vs. grateful bucket). Syntactic and semantic factors were fully crossed within-subjects and within-items yielding a 2 (Grammatical) X 2 (Meaningful) X 2 (Noun Gender) design. Results on serial order memory accuracy revealed that both grammaticality and meaningfulness improved performance, and that the factors interacted, such that the beneficial effects of grammaticality were only present for lists of meaningful items. The present results thus demonstrate that while something as simple as morphological agreement (a long-term, syntactic constraint) can improve verbal STM performance, it only seem to do so in the presence stronger semantic constraints.

Abstract

Although semantic influences on verbal short-term memory (STM) are well-documented, substantially less research has examined influences of syntactic representation. In the present study, both syntactic and semantic factors were manipulated in order to explore how each affects verbal STM. Subjects (N=20) performed immediate, serial recall on lists of six Dutch words composed of three sets of adjective-noun pairs. Lists were factorially manipulated within a 2 (Noun Gender; common vs. neuter) X 2 (Grammatical; legal vs. illegal morphological agreement) X 2 (Meaningful; more vs. less) within-subjects design. Results on serial order memory revealed significant main effects of meaningfulness and grammaticality and a meaningfulness X grammaticality interaction, whereby the effects of the grammaticality were only present for more meaningful lists. The present results demonstrate that although syntactic factors can influence verbal STM, they only seem to do so in the presence stronger semantic constraints.

Abstract

One of the physiological markers of monitoring in both
speech and non-speech tasks is the so-called error related
negativity (ERN), an event-related potential that is
typically observed after error trials. However, the ERN is
also observed after correct trials in both manual and
verbal tasks, suggesting that it might be a more general
marker for the monitoring of response conflict. The
present work tests this hypothesis in speech production
by exploring a situation where increased response conflict
naturally occurs, namely, when multiple speech
outputs are simultaneously activated. Event-related
potentials were recorded while participants named
pictures in their first and second languages. Activation of
multiple outputs was manipulated through the form
similarity between translation equivalents (i.e., cognate
status). Replicating previous results, cognates were faster
to name than non-cognates. Interestingly, responselocked
analyses not only showed a reliable ERN on correct
trials, but that the amplitude of the ERN was larger
for cognates compared to non-cognates. Thus, despite
being faster to name, cognates seem to induce more
conflict during response monitoring. This in turn indicates
that the ERN is not simply sensitive to conflicting motor
responses, but also to more abstract conflict resulting
from co-activation of multiple phonological representations.

Abstract

My contribution will focus on the neural infrastructure for deriving speaker meaning. Recent accounts have argued that simulation (i.e. the re-enactment of states of perception and action) suffice to realize comprehension. This will fail on theoretical grounds. But I will also show empirical evidence indicating that the Theory of Mind network kicks in when articularized conversational implicatures are at stake. In addition, I will show that markers of Information Structure trigger the operation of a general attention network in the surface of modulating the depth of processing.

Abstract

A central and influential idea among researchers of language is that our language faculty is organized according to Fregean compositionality, which implies that the meaning of an utterances is a function of the meaning of its parts and of the syntactic rules by which these parts are combined. The implication of this idea is that beyond word recognition, language interpretation takes place in a two-step fashion. First, the meaning of a sentence is computed. In a second step the sentence meaning is integrated with information from prior discourse, with world knowledge, with information about the speaker, and with semantic information from extralinguistic domains such as co-speech gestures or the visual world. FMRI results and results from recordings of event related brain potentials will be presented that are inconsistent with this classical Fregean model of language intepretation. Our data support a model in which knowledge about the context and the world, knowledge about concomitant information from other modalities, and knowledge about the speaker are brought to bear immediately, by the same fast-acting brain system that combines the meanings of individual words into a message-level representation. The Memory, Unification and Control (MUC) model of language accounts for these data. Resting state connectivity data, and results from Psycho-Physiological Interactions will be discussed, suggesting a division of labour between temporal and inferior frontal cortex. These results indicate that Broca’s area and adjacent cortex play an important role in semantic and syntactic unification operations. I will also discuss fMRI results that indicate the insufficiency of the Mirror Neuron Hypothesis to explain language understanding. Instead I will sketch a picture of language processing from an embrained perspective.

Abstract

There is a school of philosophers who believe that the garden of nature should be cleaned first from the conceptual weeds by qualified philosophers, before empirical researchers should be allowed to enter the scene. I will defend a different position. This is one in which, for the case of cognitive neuroscience, knowledge on brain and cognition is strongly driven by new research tools and methods, which provide new challenges for conceptual analysis.

Abstract

With more than a hundred billion neurons, and more than 100.000 kilometers of connecting wires inside our skull, the human brain is the most complex organ in the known universe. Recent developments of brain imaging techniques, allow unprecedented previews of the human brain in action. What happens in our brain when we learn, when we change our opinion, when we speak, when we experience emotion; it will all be discussed in this lecture. How increased insights into brain function will impact society will be discussed as well.

Peter Hagoort is director of the Max Planck Instute for Psycholinguistics (since November 2006), and the founding director of the Donders Centre for Cognitive Neuroimaging (1999), a cognitive neuroscience research centre at the Radboud University Nijmegen, with participation of the Universities of Maastricht, Twente, and the Max Planck Institute for Psycholinguistics. In addition, he is professor in cognitive neuroscience at the Radboud University Nijmegen. His own research interests relate to the domain of the human language faculty and how it is instantiated in the brain. In his research he applies neuroimaging techniques such as ERP, MEG, PET and fMRI to investigate the language system and its impairments as in aphasia, dyslexia and autism. At the Donders Centre he is currently heading the research group Neurocognition of Language. At the Max Planck Institute he is heading a department on the Neurobiology of Language. For his scientific contributions, the Royal Netherlands Academy of Arts Sciences (KNAW) awarded him with the Hendrik Mullerprijs in 2003. In 2004 he was awarded by the Dutch Queen with the “Knighthood of the Dutch Lion.” In 2005 he received the NWO-Spinoza Prize. Peter Hagoort is fellow of the Royal Netherlands Academy of Arts and Sciences (KNAW).

Abstract

Connectivity among language-related brain regions during resting state has consistently been observed in previous studies. The current study investigates whether and how this connectivity is altered by a language task. Twenty-four native Dutch speakers were asked to read sentences for comprehension (i.e., a 50 min. language comprehension task), and resting state fMRI data were collected before and after the task. In accordance with previous similar work (Xiang, Fonteijn, Norris, & Hagoort. (2010). Topographical functional connectivity pattern in the perisylvian language networks. Cerebral Cortex, 20, 549-560.), ROIs in left BA44, BA45 and BA47 were used as seed regions. Functional connectivity (fc) of the seed regions with left parietal and temporal areas was found, in line with Xiang et al’s observations. Moreover, comparing fc's before and after the task, we found that the task altered fc patterns. After the task, for ROI BA44 and BA45, reduced connectivity with middle and posterior temporal regions as well as with the parietal lobule were found. In contrast, we 232 Cognitive Neuroscience Society — 2011 Annual Meeting Poster Session I — Tuesday, April 5, 3:00 - 5:00 pm, Pacific Concourse observed increased connectivity with medial frontal and superior frontal gyrus. For BA47, increased connectivity with anterior temporal lobe and bilateral precentral gyrus, and reduced connectivity with visual cortex were observed. Together the results suggest that language tasks modulate the resting-state connectivity within the brain's language network, in line with previous work (Waites, Stanislavsky, Abbott, & Jackson. (2005) Effect of prior cognitive state on resting state networks measured with functional connectivity. Human Brain Mapping, 24, 59-68.).