Presentations

Abstract

Das wichtigste und zugleich komplexeste Organ im menschlichen Körper ist das Gehirn.
Es ist beteiligt an allem, was uns lieb und teuer ist. Ohne Gehirn kein Gedächtnis, kein Gefühl, keine Sprache oder Wahrnehmung, so Dr. Peter Hagoort, Professor am Donders Centre for Cognition an der Radboud Universität. Der Nimweger Wissenschaftler beschäftigt sich mit den revolutionären Entwicklungen auf dem Gebiet der Scanbildtechnologie.

Abstract

Human face-to-face communication is a multi-modal activity. Recent research has shown that, during comprehension, recipients integrate information from speech with that contained in co-speech gestures (e.g., Kelly et al., 2010). The current studies take this research one step further by investigating the influence of another modality, namely eye gaze, on speech and gesture comprehension, to advance our understanding of language processing in more situated contexts. In spite of the large body of literature on processing of eye gaze, very few studies have investigated its processing in the context of communication (but see, e.g., Staudte & Crocker, 2011 for an exception). In two studies we simulated a triadic communication context in which a speaker alternated their gaze between our participant and another (alleged) participant. Participants thus viewed speech-only or speech + gesture utterances either in the role of addressee (direct gaze) or in the role of unaddressed recipient (averted gaze). In Study 1, participants (N = 32) viewed video-clips of a speaker producing speech-only (e.g. “she trained the horse”) or speech+gesture utterances conveying complementary information (e.g. “she trained the horse”+WHIPPING gesture). Participants were asked to judge whether a word displayed on screen after each video-clip matched what the speaker said or not. In half of the cases, the word matched a previously uttered word, requiring a “yes” answer. In all other cases, the word matched the meaning of the gesture the actor had performed, thus requiring a ‘no’ answer.

Abstract

During reading, people not only retrieve meaning from individual words, they also combine words into multi-word meaning representation and derive inference from it. In single word studies, action verb meaning (kick) is understood through the activation of motor areas, typically interpreted as showing the necessity of these sensori motor regions as part of a semantic circuit for language comprehension (Pulvermüller & Fadiga, 2010). But it remains unclear how this association-based theory scales up to understanding sentence meaning and how the semantic circuit subserves inference
making at the sentence level.

Abstract

Words’ meanings vary with context. When do context effects arise? The answer to this is critical for deciding between theories assuming that meanings are accessed from a stable mental lexicon and theories that suggest meanings are constructed ad hoc. On the first view, a word form activates an invariant semantic representation, which is subsequently tailored to fit the context (e.g., Evans, 2009; Machery, 2010). On an alternative view, word forms are cues to construct meaning; the information that gets activated is always co-determined by the word and its context (Elman, 2004; 2009; Lai, Hagoort, & Casasanto, 2011).

Abstract

An important property of language is that it can be used exophorically, for instance in referring to entities in the extra-linguistic context of a conversation using demonstratives such as “this” and “that”. Despite large-scale cross-linguistic descriptions of demonstrative systems, the mechanisms underlying the comprehension of such referential acts are poorly understood. Therefore, we investigated the neural mechanisms underlying demonstrative comprehension in situated contexts. Twenty-three participants were presented on a computer screen with pictures containing a speaker and two similar objects. One of the objects was close to the speaker, whereas the other was either distal from the speaker but optically close to the participant (“sagittal orientation”), or distal from both (“lateral orientation”). The speaker pointed to one object, and participants heard sentences spoken by the speaker containing a proximal (“this”) or distal (“that”) demonstrative, and a correct or incorrect noun-label (i.e., a semantic violation). EEG was recorded continuously and time-locked to the onset of demonstratives and nouns. Semantic violations on the noun-label yielded a significant, wide-spread N400 effect, regardless of the objects’ orientation. Comparing the comprehension of proximal to distal demonstratives in the sagittal orientation yielded a similar N400 effect, both for the close and the far referent. Interestingly, no demonstrative effect was found when objects were oriented laterally. Our findings suggest a similar time-course for demonstrative and noun-label processing. However, the comprehension of demonstratives depends on the spatial orientation of potential referents, whereas noun-label comprehension does not. These findings reveal new insights about the mechanisms underlying everyday demonstrative comprehension.