Publications

Abstract

Gesture and speech are assumed to form an integrated system during language production. Based on this view, we propose the integrated‐systems hypothesis, which explains two ways in which gesture and speech are integrated—through mutual and obligatory interactions—in language comprehension. Experiment 1 presented participants with action primes (e.g., someone chopping vegetables) and bimodal speech and gesture targets. Participants related primes to targets more quickly and accurately when they contained congruent information (speech: “chop”; gesture: chop) than when they contained incongruent information (speech: “chop”; gesture: twist). Moreover, the strength of the incongruence affected processing, with fewer errors for weak incongruities (speech: “chop”; gesture: cut) than for strong incongruities (speech: “chop”; gesture: twist). Crucial for the integrated‐systems hypothesis, this influence was bidirectional. Experiment 2 demonstrated that gesture’s influence on speech was obligatory. The results confirm the integrated‐systems hypothesis and demonstrate that gesture and speech form an integrated system in language comprehension.

Abstract

Locative expressions encode the spatial relationship between two (or more) entities. In this paper, we focus on locative expressions in signed language, which use the visual-spatial modality for linguistic expression, specifically in
Turkish Sign Language ( Türk İşaret Dili, henceforth TİD). We show that TİD uses various strategies in discourse to encode the relation between a Ground entity (i.e., a bigger and/or backgrounded entity) and a Figure entity (i.e., a
smaller entity, which is in the focus of attention). Some of these strategies exploit affordances of the visual modality for analogue representation and support evidence for modality-specific effects on locative expressions in sign languages.
However, other modality-specific strategies, e.g., the simultaneous expression of Figure and Ground, which have been reported for many other sign languages, occurs only sparsely in TİD. Furthermore, TİD uses categorical as well as analogical structures in locative expressions. On the basis of
these findings, we discuss differences and similarities between signed and spoken languages to broaden our understanding of the range of structures used in natural language (i.e., in both the visual-spatial or oral-aural modalities) to encode locative relations. A general linguistic theory of spatial relations, and specifically of locative expressions, must take all structures that
might arise in both modalities into account before it can generalize over the human language faculty.

Abstract

Demonstratives (e.g., words such as this and that in English) pivot on relationships between the item being talked about, and features of the speech act situation (e.g., where the speaker and addressee are standing or looking). However, they are only rarely investigated multi-modally, in natural language contexts. This task is designed to build a video corpus of cross-linguistically comparable discourse data for the study of “deixis in action”, while simultaneously supporting the investigation of joint attention as a factor in speaker selection of demonstratives. In the task, two or more speakers are asked to discuss and evaluate a group of similar items (e.g., examples of local handicrafts, tools, produce) that are placed within a relatively defined space (e.g., on a table). The task can additionally provide material for comparison of pointing gesture practices.

Abstract

This study investigates how children of different ages talk about a conversation that they have witnessed. 48 Turkish children, five, nine and thirteen years in age, saw a televised dialogue between two Sesame Street characters (Bert and Ernie). Afterward, they narrated what they had seen and heard. Their reports were analysed for the development of linguistic devices used to orient their listeners to the relevant properties of a conversational exchange. Each utterance in the child's narrative was analysed as to its conversational role: (1) whether the child used direct or indirect quotation frames; (2) whether the child marked the boundaries of conversational turns using speakers' names and (3) whether the child used a marker for pairing of utterances made by different speakers (agreement-disagreement, request-refusal, questioning-answering). Within pairings, children's use of (a) the temporal and evaluative connectivity markers and (b) the kind of verb of saying were identified. The data indicate that there is a developmental change in children's ability to use appropriate linguistic means to orient their listeners to the different properties of a conversation. The development and use of these linguistic means enable the child to establish different social roles in a narrative interaction. The findings are interpreted in terms of the child's social-communicative development from being a ' character' to becoming a ' narrator' and ' author' of the reported conversation in the narrative situation.