Presentations

Abstract

One of the most daunting tasks of a listener is to map a continuous auditory stream onto known speech sound categories and lexical items. A major issue with this mapping problem is the variability in the acoustic realizations of sound categories, both within and across speakers. Past research has suggested listeners may use various sources of information, such as lexical knowledge or visual cues (e.g., lip-reading) to recalibrate these speech categories to the current speaker. Previous studies have focused on audiovisual recalibration of consonant categories. The present study explores whether vowel categorization, which is known to show less sharply defined category boundaries, also benefit from visual cues.
Participants were exposed to videos of a speaker pronouncing one out of two vowels (Dutch vowels /e/ and /ø/), paired with audio that was ambiguous between the two vowels. The most ambiguous vowel token was determined on an individual basis by a categorization task at the beginning of the experiment. In one group of participants, this auditory token was paired with a video of an /e/ articulation, in the other group with an /ø/ video. After exposure to these videos, it was found in an audio-only categorization task that participants had adapted their categorization behavior as a function of the video exposure. The group that was exposed to /e/ videos showed a reduction of /ø/ classifications, suggesting they had recalibrated their vowel categories based on the available visual information. These results show that listeners indeed use visual information to recalibrate vowel categories, which is in line with previous work on audiovisual recalibration in consonant categories, and lexically-guided recalibration in both vowels and consonants.
In addition, a secondary aim of the current study was to explore individual variability in audiovisual recalibration. Phoneme categories vary not only in terms of boundary location, but also in terms of boundary sharpness, or how strictly categories are distinguished. The present study explores whether this sharpness is associated with the amount of audiovisual recalibration. The results tentatively support that a fuzzy boundary is associated with stronger recalibration, suggesting that listeners’ category sharpness may be related to the weight they assign to visual information in audiovisual speech perception. If listeners with fuzzy boundaries assign more weight to visual cues, given that vowel categories have less sharp boundaries than consonants, there ought to be audiovisual recalibration for vowels as well. This is exactly what was found in the current study.

Abstract

The classroom is designed to teach children cultural inventions for which the brain is not evolutionary designed. Hence the
classroom environment has to implement cultural reycling of neuronal maps. To do this effectively it has to recruit existing neural
infrastructure. Therefore, teaching programmes have to be tailored to the possibilities and limitations of available neural architecture.
An example in case is reading, a cultural invention of a few thousand years old. Orthographies and reading methods need
to use visual cortex areas in the most optimal way. I will discuss how the characteristics of different orthographies are tailored
to the possibilities of complex cells in visual cortex. In addition, different reading methods will be evaluated in the light of our
understanding of human brain organization. I will argue that a systematic investigation of culture-brain relations is much needed
for optimizing the optimal environment.

Abstract

A series of results from event-related brain potential recordings and fMRI research will be presented, suggesting that language processing does not obey strict compositionality, and, moreover immediately recruits extralinguistic information. It will also be shown that pragmatic inferences require contributions from TOM networks. This implies that an embodied account of semantics fails (under the somewhat strange assumption that the brain is not part of the body). I will put forward an embrained perspective on language processing.

Abstract

In my presentation I will summarize the results of a number of ERP and fMRI studies on the processing and integration of co-occurring speech and gestures/pantomimes. The ERP results indicate that the time course of integrating language and action (i.e., gesture) is very similar to that of integrating linguistic meaning into a sentence or discourse representation. Moreover, in both cases the Left Inferior Frontal cortex plays a central role in orchestrating the multimodal unification of language and action. This orchestration is partly done by modulating temporal areas that store representations activated by the input. I will discuss the parameters of this modulation.

Abstract

For over 20 years, researchers have investigated syntactic priming of transitives by measuring the frequency of sentence choice. These studies have shown syntactic priming for passives but weaker or absent syntactic priming for actives (e.g. [1],[2]). Until recently very few studies have reported syntactic priming by measuring reaction times of language production and none of these studies included transitive sentences. We previously found syntactic priming of both active and passive Dutch transitives in speech onsets, and interestingly the effect appeared to be stronger for actives than passives [3][4]. In order to explain the discrepancy between our results and other results reported in the literature, we hypothesized that there is a ceiling effect in the frequency of using an active transitive (in general about 94% of produced transitives are actives) but not in the speed of producing one. To confirm this hypothesis we conducted a syntactic priming experiment with a picture description task measuring both reaction times and the frequency of occurrence on the same trials. This way we could exclude the alternative explanation that the discrepancy in results is caused by differences in design or stimuli. The results of this experiment show syntactic priming effects for passives and not for actives in the frequency of occurrence (in line with the literature). However, the reaction times of producing these sentences do show syntactic priming for actives and these effects appear to be even stronger than for passives (in line with our previous data). In conclusion, our results suggest that measuring the frequency of occurrence is not enough to get a complete picture of syntactic priming. Measuring reaction times in addition to the frequency of occurrence could provide us with a more complete picture. References [1] Bock, K., & Loebell, H. (1990). Framing sentences. Cognition, 35, 1-39. [2] Hartsuiker, R. J., & Kolk, H. H. J. (1998). Syntactic persistence in Dutch sentence production. Language and Speech, 41, 143-184. [3] Menenti, L., Segaert, K., & Hagoort, P. (2008). Repetition suppression for syntax and semantics: overt speech in fMRI. Society for Neuroscience, Washington DC. [4] Segaert, K., Menenti, L., & Hagoort, P. (2009) Scanning speech with fMRI: Short and long term priming of syntax and verbs, CUNY conference on human sentence processing, Davis, CA.

Abstract

Syntactic priming has been frequently used to study syntactic processes in language production in monolinguals [1][2] and bilinguals [3]. In a previous study in language comprehension [4] we showed that passive sentences in English (the participant’s L2) can be primed by passive sentences in German (L1) and English (L2). This was manifested in faster reading times for target sentences and repetition suppression effects in left inferior frontal, left precentral and left middle temporal regions of interest in an fMRI study. However, syntactic priming in comprehension is complicated by the influence of verb repetition between prime and target [5][6]. Therefore, we conducted a reading time and fMRI study looking at the influence of verb repetition on syntactic priming. In this study of monolingual comprehension in Dutch we primed passive sentences as well as sentences with crossed-dependency structures. The reading time results revealed a syntactic priming effect for passive sentences, while the effect for crossed-dependency structure sentences interacted with the factor verb repetition. The preliminary fMRI results suggest that the repetition of passive structures leads to reductions in neural activity. The repetition of crossed dependency structures causes repetition enhancement, an increase in the BOLD response, an effect that interacts with the factor verb repetition. In conclusion, the influence of verb repetition on syntactic priming in comprehension is complex and seems to depend on the type of syntactic structure investigated. References [1] Bock K. (1986). Syntactic persistence in language production. Cognitive Psychology, 18(3), 355-387. [2] Pickering M, & Branigan H. (1999). Syntactic priming in language production. Trends in Cognitive Sciences, 3(4), 136-141. [3] Schoonbaert S, Hartsuiker RJ, & Pickering MJ. (2007). The representation of lexical and syntactic information in bilinguals: Evidence from syntactic priming. Journal of Memory and Language, 56(2), 153-171. [4] Weber K, & Indefrey P. (in press). Syntactic priming in German-English bilinguals during sentence comprehension. NeuroImage. [5] Arai M, van Gompel R, & Scheepers C. (2007). Priming ditransitive structures in comprehension. Cognitive Psychology, 54, 218-250. [6] Thothathiri M, & Snedeker J. (2008). Give and take: Syntactic priming during spoken language comprehension. Cognition, 108(1), 51-68.