Presentations

Abstract

Recent work suggests that monitoring in speech production may occur via domain-general mechanisms responsible for detecting response conflict. To test this hypothesis, we measured EEG as people engaged in both non-verbal (flanker) and verbal (tongue twisters) tasks designed to elicit response conflict and errors. In the flanker task, people pressed a button corresponding to whether a center arrow was facing left or right, and response conflict was induced with flanking arrows pointing in the same (congruent; >>>>>) or a different (incongruent >><>>) direction. In the tongue twister task, people read sequences of four nonwords three times in which rhymes alternated in an ABAB pattern while onset speech sounds alternated in an ABBA (tongue twister) or an ABAB (non-tongue twister) pattern (e.g., tif deev dif teev vs. tif teev dif deef). Results in the fl anker task showed standard markers of response conflict in the form of an increased N2 for incongruent relative to congruent trials as well as an error-related negativity (ERN) for incorrect trials. Behaviourally, more errors were elicited for tongue twisters relative to nontongue twister trials, and an ERN was observed on incorrect responses. Correlations between the magnitude of the N2 and ERN in the fl anker task with the magnitude of the ERN and error rates in the tongue twister task are consistent with a common underlying locus. Adaptation effects preceding and following erroneous trials in production are also presented. These results are consistent with response confl ict serving as a cue to monitoring in speech production.

Abstract

A central and influential idea among researchers of language is that our language faculty is organized according to the principle of strict compositionality, which implies that the meaning of an utterances is a function of the meaning of its parts and of the syntactic rules by which these parts are combined. The implication of this idea is that beyond word recognition, language interpretation takes place in a two-step fashion. First, the meaning of a sentence is computed. In a second step the sentence meaning is integrated with information from prior discourse, with world knowledge, with information about the speaker, and with semantic information from extralinguistic domains such as co-speech gestures or the visual world. FMRI results and results from recordings of event related brain potentials will be presented that are inconsistent with this classical model of language intepretation. Our data support a model in which knowledge about the context and the world, knowledge about concomitant information from other modalities, and knowledge about the speaker are brought to bear immediately, by the same fast-acting brain system that combines the meanings of individual words into a message-level representation. The Memory, Unification and Control (MUC) model provides a neurobiological plausible account of the underlying neural architecture. Resting state connectivity data, and results from Psycho-Physiological Interactions will be discussed, suggesting a division of labour between temporal and inferior frontal cortex. These results indicate that Broca’s area and adjacent cortex play an important role in semantic and syntactic unification operations. I will also discuss fMRI results that indicate the insufficiency of the Mirror Neuron Hypothesis to explain language understanding. Instead I will sketch a picture of language processing from an embrained perspective.

Abstract

From a functionalist perspective all that brain research is claimed to have told us is that language processing "happens somewhere north of the neck" (Jerry Fodor, 1999). I will argue why I disagree with this conclusion, for at least the following three reasons. First, one fundamental question in the language sciences is: what makes the human brain language-ready? Understanding the neural architecture that supports human language function is a crucial part of the explanandum. I will show some unique features of human perisylvian cortex based on data from Diffusion Tensor Imaging and resting state fMRI. The second argument is that even if one is only interested in the cognitive architecture of language comprehension and production, relevant evidence can be obtained from neurobiological data, both structural and functional. I will discuss the consequences of connectivity patterns in the brain for assumptions in processing models of language, and I will show fMRI data based on a repetition suppression paradigm that provide evidence for the claim that syntactic encoding and parsing are based on the same mechanism. Finally, I will argue that framing theories of sentence processing in a way that connects to other areas of cognitive neuroscience might be helpful in asking interesting and relevant new questions. I will illustrate this in the context of the Memory, Unification and Control (MUC) model of language.

Abstract

A central and influential idea among researchers of language is that our language faculty is organized according to the principle of strict compositionality, which implies that the meaning of an utterances is a function of the meaning of its parts and of the syntactic rules by which these parts are combined. The implication of this idea is that beyond word recognition, language interpretation takes place in a two-step fashion. First, the meaning of a sentence is computed. In a second step the sentence meaning is integrated with information from prior discourse, with world knowledge, with information about the speaker, and with semantic information from extralinguistic domains such as co-speech gestures or the visual world. FMRI results and results from recordings of event related brain potentials will be presented that are inconsistent with this classical model of language intepretation. Our data support a model in which knowledge about the context and the world, knowledge about concomitant information from other modalities, and knowledge about the speaker are brought to bear immediately, by the same fast-acting brain system that combines the meanings of individual words into a message-level representation. The Memory, Unification and Control (MUC) model provides a neurobiological plausible account of the underlying neural architecture. Resting state connectivity data, and results from Psycho-Physiological Interactions will be discussed, suggesting a division of labour between temporal and inferior frontal cortex. These results indicate that Broca’s area and adjacent cortex play an important role in semantic and syntactic unification operations. I will also discuss fMRI results that indicate the insufficiency of the Mirror Neuron Hypothesis to explain language understanding. Instead I will sketch a picture of language processing from an embrained perspective.

Abstract

Traditionally, language comprehension has been studied as a solitary and unimodal activity. Here, we investigate language comprehension as a joint activity, i.e., in a dynamic social context involving multiple participants in different roles with different perspectives, while taking into account the multimodal nature of facetoface communication. We simulated a triadic communication context involving a speaker alternating her gaze between two different recipients, conveying information not only via speech but gesture as well. Participants thus viewed videorecorded speechonly or speech+gesture utterances referencing objects (e.g., “he likes the laptop”/+TYPING ON LAPTOPgesture) when being addressed (direct gaze) or unaddressed (averted gaze). The videoclips were followed by two object images (laptoptowel). Participants’ task was to choose the object that matched the speaker’s message (i.e., laptop). Unaddressed recipients responded significantly slower than addressees for speechonly utterances. However, perceiving the same speech accompanied by gestures sped them up to levels identical to that of addressees. Thus, when speech processing suffers due to being unaddressed, gestures become more prominent and boost comprehension of a speaker’s spoken message. Our findings illuminate how participants process multimodal language and how this process is influenced by eye gaze, an important social cue facilitating coordination in the joint activity of conversation.

Abstract

A series of results from event-related brain potential recordings and fMRI research will be presented, suggesting that language processing does not obey strict compositionality, and, moreover immediately recruits extralinguistic information. It will also be shown that pragmatic inferences require contributions from TOM networks. This implies that an embodied account of semantics fails (under the somewhat strange assumption that the brain is not part of the body). I will put forward an embrained perspective on language processing.

Abstract

In my presentation I will summarize the results of a number of ERP and fMRI studies on the processing and integration of co-occurring speech and gestures/pantomimes. The ERP results indicate that the time course of integrating language and action (i.e., gesture) is very similar to that of integrating linguistic meaning into a sentence or discourse representation. Moreover, in both cases the Left Inferior Frontal cortex plays a central role in orchestrating the multimodal unification of language and action. This orchestration is partly done by modulating temporal areas that store representations activated by the input. I will discuss the parameters of this modulation.

Abstract

For over 20 years, researchers have investigated syntactic priming of transitives by measuring the frequency of sentence choice. These studies have shown syntactic priming for passives but weaker or absent syntactic priming for actives (e.g. [1],[2]). Until recently very few studies have reported syntactic priming by measuring reaction times of language production and none of these studies included transitive sentences. We previously found syntactic priming of both active and passive Dutch transitives in speech onsets, and interestingly the effect appeared to be stronger for actives than passives [3][4]. In order to explain the discrepancy between our results and other results reported in the literature, we hypothesized that there is a ceiling effect in the frequency of using an active transitive (in general about 94% of produced transitives are actives) but not in the speed of producing one. To confirm this hypothesis we conducted a syntactic priming experiment with a picture description task measuring both reaction times and the frequency of occurrence on the same trials. This way we could exclude the alternative explanation that the discrepancy in results is caused by differences in design or stimuli. The results of this experiment show syntactic priming effects for passives and not for actives in the frequency of occurrence (in line with the literature). However, the reaction times of producing these sentences do show syntactic priming for actives and these effects appear to be even stronger than for passives (in line with our previous data). In conclusion, our results suggest that measuring the frequency of occurrence is not enough to get a complete picture of syntactic priming. Measuring reaction times in addition to the frequency of occurrence could provide us with a more complete picture. References [1] Bock, K., & Loebell, H. (1990). Framing sentences. Cognition, 35, 1-39. [2] Hartsuiker, R. J., & Kolk, H. H. J. (1998). Syntactic persistence in Dutch sentence production. Language and Speech, 41, 143-184. [3] Menenti, L., Segaert, K., & Hagoort, P. (2008). Repetition suppression for syntax and semantics: overt speech in fMRI. Society for Neuroscience, Washington DC. [4] Segaert, K., Menenti, L., & Hagoort, P. (2009) Scanning speech with fMRI: Short and long term priming of syntax and verbs, CUNY conference on human sentence processing, Davis, CA.

Abstract

Syntactic priming has been frequently used to study syntactic processes in language production in monolinguals [1][2] and bilinguals [3]. In a previous study in language comprehension [4] we showed that passive sentences in English (the participant’s L2) can be primed by passive sentences in German (L1) and English (L2). This was manifested in faster reading times for target sentences and repetition suppression effects in left inferior frontal, left precentral and left middle temporal regions of interest in an fMRI study. However, syntactic priming in comprehension is complicated by the influence of verb repetition between prime and target [5][6]. Therefore, we conducted a reading time and fMRI study looking at the influence of verb repetition on syntactic priming. In this study of monolingual comprehension in Dutch we primed passive sentences as well as sentences with crossed-dependency structures. The reading time results revealed a syntactic priming effect for passive sentences, while the effect for crossed-dependency structure sentences interacted with the factor verb repetition. The preliminary fMRI results suggest that the repetition of passive structures leads to reductions in neural activity. The repetition of crossed dependency structures causes repetition enhancement, an increase in the BOLD response, an effect that interacts with the factor verb repetition. In conclusion, the influence of verb repetition on syntactic priming in comprehension is complex and seems to depend on the type of syntactic structure investigated. References [1] Bock K. (1986). Syntactic persistence in language production. Cognitive Psychology, 18(3), 355-387. [2] Pickering M, & Branigan H. (1999). Syntactic priming in language production. Trends in Cognitive Sciences, 3(4), 136-141. [3] Schoonbaert S, Hartsuiker RJ, & Pickering MJ. (2007). The representation of lexical and syntactic information in bilinguals: Evidence from syntactic priming. Journal of Memory and Language, 56(2), 153-171. [4] Weber K, & Indefrey P. (in press). Syntactic priming in German-English bilinguals during sentence comprehension. NeuroImage. [5] Arai M, van Gompel R, & Scheepers C. (2007). Priming ditransitive structures in comprehension. Cognitive Psychology, 54, 218-250. [6] Thothathiri M, & Snedeker J. (2008). Give and take: Syntactic priming during spoken language comprehension. Cognition, 108(1), 51-68.