Presentations

Abstract

In conversation, negative responses to invitations,
requests, offers and the like more often occur with
a delay – conversation analysts talk of them as
dispreferred. Here we examine the contrastive
cognitive load ‘yes’ and ‘no’ responses make,
either when given relatively fast (300 ms) or
delayed (1000 ms). Participants heard minidialogues,
with turns extracted from a spoken
corpus, while having their EEG recorded. We find
that a fast ‘no’ evokes an N400-effect relative to a
fast ‘yes’, however this contrast is not present for
delayed responses. This shows that an immediate
response is expected to be positive – but this
expectation disappears as the response time
lengthens because now in ordinary conversation
the probability of a ‘no’ has increased.
Additionally, however, 'No' responses elicit a late
frontal positivity both when they are fast and when
they are delayed. Thus, regardless of the latency of
response, a ‘no’ response is associated with a late
positivity, since a negative response is always
dispreferred and may require an account. Together
these results show that negative responses to social
actions exact a higher cognitive load, but especially
when least expected, as an immediate response.

Abstract

Recent work in the L&C department in MPI Nijmegen has explored the processing implications of the core ecological niche for language learning and use, namely interactive conversation. It turns out that the rapidity of turn-exchange puts extreme requirements on predictive comprehension and speedy production, reflected e.g. in the trouble kids have to approach adult norms. This strong functional pressure must have implications for language typology. But what exactly? This paper explores what we have recently found out about differing processing in different word orders, and the ways in which the tough processing requirements of conversation can be buffered.

Abstract

Within the confines of this mini-plenary I’ll try to sketch how turn-taking may have played a crucial role
in molding the origins and shape of language. First, I’ll run through some of our recent findings that
reveal the intensive cognitive processing that underlies turn-taking – measuring response-timing, gaze,
the acoustics, breathing, and EEG. These findings suggest that the turn-taking system stretches cognitive
processing to the limit. Asking why the system is the way it is, I’ll advance the argument that language as
we now know it may have emerged from the growth of a rich information-encoding system in the context
of an antecedent turn-taking system, so that increasingly complex messages became squeezed into short
turns, with the consequence of extreme compression, inference enrichment of the Gricean kind, tendency
for fixed word orders, etc. Some support for this account can also be found in ontogenetic and
phylogenetic studies of turn-taking.

Abstract

The diversity of languages contrasts with the universality of much of the communicational infrastructure that makes language possible. An important component of this infrastructure is the turn-taking system of conversation, the Stephen Levinsoncore ecological niche for language use. This system puts intense pressure on language processing: cross-linguistically, we mostly respond within 200 milliseconds, even though language encoding takes at least three times as long. It can be shown using many different measures (e.g. response times, breathing, EEG) that we beat the clock by predicting what the other is going to say and starting production as soon as we can. This raises interesting questions about why this system is the way it is, what functional pressures it puts on language structure and language diversity, and how it originated, which I will briefly address. I will argue that the current system can best be understood within an evolutionary context in which the turn-taking system was antecedent to the complexities of modern language so that increasingly complex messages became squeezed into short turns, with the consequence of extreme compression, inference enrichment of the Gricean kind, a tendency for fixed word orders, amongst other things. Some support for this account can be found in ontogenetic and phylogenetic studies of turn-taking which I will briefly review.

Abstract

In spoken interactions, interlocutors carefully plan and time their utterances, minimising gaps and overlaps between consecutive turns.1 Cross-linguistic comparison indicates that spoken languages vary minimally in their turn timing.2 Pre-linguistic vocal turn taking has also been attested in the first six months of life.3 These observations suggest that the turn-taking system provides a universal basis for our linguistic capacities.4 It remains an open question, however, whether precisely-timed turn taking is solely a property of speech. It has previously been argued that, unlike speakers, signers do not attend to the one-at-a-time principle, and instead form a collaborative turn-taking floor with their interlocutors, thus having a higher degree of social tolerance for overlap.5 But recent corpus analyses of Sign Language of the Netherlands (NGT) have revealed that, although simultaneous signing is more frequent in NGT than overlapping speech in spoken languages, the additional overlap may come as a consequence of having larger and thus slower articulators.6 The beginnings and ends of signed utterances are bookended by preparatory and retractive movements — phonetically necessary articulations that do not add to the interpretation of the utterance.7 When turn timing is calculated on the basis of stroke-to-stroke turn boundaries, NGT turn timing and turn overlap are consistent with documented averages for spoken turn taking.6 This paper presents new experimental evidence supporting the psychological reality of stroke-to-stroke turn boundaries for signers by using an adapted button-press paradigm, originally developed for measuring spoken turn prediction.8 Our results indicate that signers indeed anticipated turn boundaries at the ends of turn-final strokes. These findings are the first to experimentally support the idea that signers use something like stroke-to-stroke turn boundaries to coordinate conversational turns. They also suggest that linguistic processing, represented by participant age and age of acquisition, plays a role in the ability to use precisely-timed turns in conversation.

Abstract

In spontane gesprekken wisselen gebaarders steeds vlug van beurt. Gebarentaalgebruikers moeten daarom steeds op het juiste moment naar de juiste persoon kijken. Hoe voorspellen gebaarders wanneer de beurt gaat wisselen en wie deze overneemt? Wij hebben de eerste vraag onderzocht door verschillende groepen gebarentaalgebruikers (doven en horenden, jong en oud, verschillende regios) te testen. Omdat er in Nijmegen weinig (dove) gebaarders wonen, hebben we dit gedaan in ons lab op wielen: de Gebarenbus.

Abstract

Introduction:
Underlying spatial memory and talking about spatial layouts are common cognitive processes (Haun et al. 2005). For example, to locate an object in space it is obligatory to choose a coordinate system called frame of reference in cognition as well as in its verbal expression. Coding space within different frames of reference requires different cognitive processes (e.g. Neggers et al. 2005). In relative frames of reference the origin of the coordinate system is the viewpoint of a person. In intrinsic frames of reference an object is located in relation to another object (Levinson 2003). FMRI data have suggested that different frames of reference show different patterns of neural activation (Burgess et al. 2002; Committeri et al. 2004). However, the number of existing frames of reference and their neural correlates remain controversial. In an event-related fMRI study we investigated whether differential neural networks for relative and intrinsic frames of reference can be isolated.
Methods:
In the present study an implicit sentence picture matching task was used to investigate differential neural correlates for relative and intrinsic frames of reference. Twenty-eight healthy human adults (16 women, 12 men) read a sentence describing a spatial scene followed by a picture, and decided whether the sentence matches the picture or not. Feedback was given either supporting a relative or an intrinsic frame of reference. After half of the trails the feedback switched from one reference frame to the respective other reference frame (Fig.1). Participants were instructed to respond as accurately and as quickly as possible. They responded with their right hand by pressing a key with the index finger for a correct decision and a second key with the middle finger for an incorrect judgment. Two baseline tasks were included (Fig.1): a high level baseline (c5) and a low level baseline (c6).
A 3 Tesla MRI system (Siemens TRIO, Erlangen, Germany) was used to acquire functional images of the whole brain. Using a gradient-echo echo planar scanning sequence 36 axial slices were obtained for each participant (voxel-size 3 x 3 x 3 mm, TR = 2310 ms, field of view = 192, TE = 30 ms, flip angle = 75). All functional images were acquired in one run that lasted for 50 minutes. Following the acquisition of functional images a high-resolution anatomical scan (T1-weighted MP-RAGE, 176 slices) was acquired. FMRI data were analyzed using BrainVoyager QX (Brain Innovation, Maastricht, The Netherlands). Random-effects whole brain group analyses were performed. The statistical threshold at the voxel level was set at p < 0.001, uncorrected for multiple comparisons.
Results:
Intrinsic trials as compared to baseline trials revealed increased activity in the parietal lobe and in the parahippocampal gyrus. Relative as compared to baseline trails revealed a widespread network of activity. Increased activity was observed in occipitotemporal cortices, in the parietal lobe, and in frontal areas.
We focused on the direct comparison between relative and intrinsic trials. Results showed increased activity in the left parahippocampal gyrus only for intrinsic trials as compared to relative trails. An ANOVA of the averaged beta-weights with the within factors Reference frame and Condition and the between factor Block order (relative-intrinsic and intrinsic-relative), obtained for all voxels in the parahippocampal gyrus, showed no main effect of Reference frames and Condition. A significant interaction between the factors Reference frame and Condition was observed (p < 0.05). T-contrasts showed a significant effect for intrinsic (c4) as compared to relative trials (c3; p < 0.001).
Conversely, relative as compared to intrinsic trials showed strong increased activity in the left medial frontal gyrus. An ANOVA of the beta-weights in the brain area showed no main effects. A significant interaction between the factors Reference frame and Condition was observed (p < 0.05). T-contrasts showed a significant effect for intrinsic (c4) as compared to relative trials (c3, p < 0.01).
When comparing all intrinsic and relative conditions together to the baseline we observed increased activity in the right and left frontal eye fields (Fig. 2). An ANOVA of the averaged beta-weights with the within factors Reference frame and the between factor Block order obtained for all voxels in the left frontal eye fields showed a main effect of Block order (p < 0.001) and an trend effect of Reference frame (p = 0.08). An ANOVA of the averaged beta-weights for all voxels in the right frontal eye fields showed a main effect of Block order (p < 0.05) only.
Conclusions:
Using a sentence-picture matching task, we investigated whether differential neural correlates for intrinsic and relative frames of reference can be isolated. Intrinsic trials compared to relative trials showed increased activity in the parahippocampal gyrus whereas relative trails compared to intrinsic trials revealed increased neural activity in the frontal and parietal lobe. Both frames of reference together compared to a baseline show increased activity in the frontal eye fields which was stronger for the second block. This could be related to switching of reference frames (Wallentin et al. 2008). The present results confirm studies which report the parietal lobe to be involved in relative coding (Cohen & Andersen 2002). The neural correlates of intrinsic frames of reference were previously less well investigated. The present results show differential neural networks for both frames of reference that are crucial to spatial language.
References:
Burgess, N. (2002), 'The human hippocampus and spatial and episodic memory', Neuron, vol. 36, pp. 625-641.
Cohen, Y. (2002), 'A common reference frame for movement plans in the posterior parietal cortex', Nature Reviews Neuroscience, vol. 3, pp. 553-562.
Committeri, G. (2004), 'Reference frames for spatial cognition: Different brain areas are involved in viewer-, object-, and landmark centered judgments about object location', Cognitive Neuroscience, vol. 16, pp. 1517-1535.
Haun, D. (2005), 'Bias in spatial memory: a categorical endorsement', Acta Psychologia, vol. 118, pp. 149-170.
Levinson, S. (2003), 'Space in language and cognition: Explorations in cognitive diversity', Cambridge: CUP.
Neggers, S. (2005), 'Quantifing the interactions between allo- and egocentric representation of space', Acta Psychologia, vol. 118, pp. 25-45.
Wallentin, M. (2008), 'Frontal eye fields involved in shifting frames of reference within working memory for scenes', Neuropsychologia, vol. 46, pp. 399-408.

Abstract

Action in interaction Since the core matrix for language use is interaction, the main job of language is not to express propositions or abstract meanings, but to deliver actions. For in order to respond in interaction we have to ascribe to the prior turn a primary ‘action’ – variously thought of as an ‘illocution’, ‘speech act’, ‘move’, etc. – to which we then respond. The analysis of interaction also relies heavily on attributing actions to turns, so that, e.g., sequences can be characterized in terms of actions and responses. Yet the process of action ascription remains way understudied. We don’t know much about how it is done, when it is done, nor even what kind of inventory of possible actions might exist, or the degree to which they are culturally variable. The study of action ascription remains perhaps the primary unfulfilled task in the study of language use, and it needs to be tackled from conversationanalytic,psycholinguistic, cross-linguistic and anthropological perspectives. In this talk I try to take stock of what we know, and derive a set of goals for and constraints on an adequate theory. Such a theory is likely to employ, I will suggest, a top-down plus bottom-up account of action perception, and a multi-level notion of action which may resolve some of the puzzles that have repeatedly arisen.

Abstract

Linguistic diversity and the 'interaction engine'. In this lecture I argue that our new insights into linguistic diversity require a rethink about the foundations of language. In the first part of the lecture, I outline why strong theories of language universals now look untenable. Combining typological and phylogenetic data suggests that languages are largely structured by cultural evolution, rather than a specific ‘language instinct’. In the second part, I turn to the implications: What then is the nature of the human endowment for language? I argue that there is a substantial infrastructure for language, which is distinct from language itself, and strongly universal, the ‘interaction engine’ of the title. The infrastructure involves speech capacities of course (vocal learning, vocal apparatus), intention-recognition systems (the pragmatics of Gricean meaningnn), and ethological properties of communicative interaction (turn-taking, structured interaction sequences, multimodal signals, etc.). A working hypothesis is that this base, together with general (non-language-specialized) properties of human cognition, provides enough foundation for infants to bootstrap into their local cultural linguistic tradition.

Abstract

The language sciences are about to undergo dramatic changes. The cognitive sciences have taken their object of enquiry to be the characterization of The Human Mind, and the language sciences have focused on the characterization of The Language Instinct, or Universal Grammar. This abstraction away from variation and diversity is now significantly inhibiting research – after all, diversity at all levels is a unique feature of our communication system compared to all other species. Further, the universals which have been the goal of linguistic research have evaporated in the face of increasing information about linguistic diversity and language change. The alternative Darwinian paradigm embraces the new facts about cognitive and linguistic diversity, viewing variation as the fuel for evolution, and adopts a diachronic perspective in which we can ask about the relative roles of biological and cultural evolution and their interaction. New findings suggest that language diversity is largely a product of cultural evolution under the constraints of general cognitive capacities rather than being tightly constrained by either Universal Grammar or Greenbergian universals.

Abstract

How are the senses structured by the languages we speak, the cultures we inhabit? To what extent is the encoding of perceptual experiences in languages a matter of how the mind/brain is “wired-up” and to what extent is it a question of local cultural preoccupation? The “Language of Perception” project tests the hypothesis that some perceptual domains may be more “ineffable” – i.e. difficult or impossible to put into words – than others. While cognitive scientists have assumed that proximate senses (olfaction, taste, touch) are more ineffable than distal senses (vision, hearing), anthropologists have illustrated the exquisite variation and elaboration the senses achieve in different cultural milieus. The project is designed to test whether the proximate senses are universally ineffable – suggesting an architectural constraint on cognition – or whether they are just accidentally so in Indo-European languages, so expanding the role of cultural interests and preoccupations. To address this question, a standardized set of stimuli of color patches, geometric shapes, simple sounds, tactile textures, smells and tastes have been used to elicit descriptions from speakers of more than twenty languages—including three sign languages. The languages are typologically, genetically and geographically diverse, representing a wide-range of cultures. The communities sampled vary in subsistence modes (hunter-gatherer to industrial), ecological zones (rainforest jungle to desert), dwelling types (rural and urban), and various other parameters. We examine how codable the different sensory modalities are by comparing how consistent speakers are in how they describe the materials in each modality. Our current analyses suggest that taste may, in fact, be the most codable sensorial domain across languages, followed closely by visual phenomena, such as colour and shape. Olfaction appears to be the least codable across cultures. Nevertheless, we have identified exquisite elaboration in the olfactory domains in some cultural settings, contrary to some contemporary predictions within the cognitive sciences. These results suggest that differential codability may be at least partly the result of cultural preoccupation. This shows that the senses are not just physiological phenomena but are constructed through linguistic, cultural and social practices.

Abstract

How are the senses structured by the languages we speak, the cultures we inhabit? To what extent is the encoding of perceptual experiences in languages a matter of how the mind/brain is ―wired-up‖ and to what extent is it a question of local cultural preoccupation? The ―Language of Perception‖ project tests the hypothesis that some perceptual domains may be more ―ineffable‖ – i.e. difficult or impossible to put into words – than others. While cognitive scientists have assumed that proximate senses (olfaction, taste, touch) are more ineffable than distal senses (vision, hearing), anthropologists have illustrated the exquisite variation and elaboration the senses achieve in different cultural milieus. The project is designed to test whether the proximate senses are universally ineffable – suggesting an architectural constraint on cognition – or whether they are just accidentally so in Indo-European languages, so expanding the role of cultural interests and preoccupations. To address this question, a standardized set of stimuli of color patches, geometric shapes, simple sounds, tactile textures, smells and tastes have been used to elicit descriptions from speakers of more than twenty languages—including three sign languages. The languages are typologically, genetically and geographically diverse, representing a wide-range of cultures. The communities sampled vary in subsistence modes (hunter-gatherer to industrial), ecological zones (rainforest jungle to desert), dwelling types (rural and urban), and various other parameters. We examine how codable the different sensory modalities are by comparing how consistent speakers are in how they describe the materials in each modality. Our current analyses suggest that taste may, in fact, be the most codable sensorial domain across languages. Moreover, we have identified exquisite elaboration in the olfactory domains in some cultural settings, contrary to some contemporary predictions within the cognitive sciences. These results suggest that differential codability may be at least partly the result of cultural preoccupation. This shows that the senses are not just physiological phenomena but are constructed through linguistic, cultural and social practices.