Publications

Abstract

The present paper studies how, in deictic expressions, the temporal interdependency of speech and gesture is realized in the course of motor planning and execution. Two theoretical positions were compared. On the “interactive” view the temporal parameters of speech and gesture are claimed to be the result of feedback between the two systems throughout the phases of motor planning and execution. The alternative “ballistic” view, however, predicts that the two systems are independent during the phase of motor execution, the temporal parameters having been preestablished in the planning phase. In four experiments subjects were requested to indicate which of an array of referent lights was momentarily illuminated. This was done by pointing to the light and/or by using a deictic expression (this/that light). The temporal and spatial course of the pointing movement was automatically registered by means of a Selspot opto-electronic system. By analyzing the moments of gesture initiation and apex, and relating them to the moments of speech onset, it was possible to show that, for deictic expressions, the ballistic view is very nearly correct.

Abstract

The process of speaking is traditionally regarded as a mapping of thoughts (intentions, feelings, etc.) onto language. One requirement that this mapping has to meet is that the units of information to be expressed be strictly ordered. The channel of speech largely prohibits the simultaneous expression of multiple propositions: the speaker has a linearization problem - that is, a linear order has to be determined over any knowledge structure to be formulated. This may be relatively simple if the informational structure has itself an intrinsic linear arrangement, as often occurs with event structures, but it requires special procedures if the structure is more complex, as is often the case in two- or three-dimensional spatial patterns. How, for instance, does a speaker proceed in describing his home, or the layout of his town? Two powerful constraints on linearization derive, on the one hand, from 'mutual knowledge' and, on the other, from working memory limitations. Mutual knowledge may play a role in that the listener can be expected to derive different implicatures from different orderings (compare 'she married and became pregnant' with 'she became pregnant and married'). Mutual knowledge determinants of linearization are essentially pragmatic and cultural, and dependent on the content of discourse. Working memory limitations affect linearization in that a speaker's linearization strategy will minimize memory load during the process of formulating. A multidimensional structure is broken up in such a way that the number of 'return addresses' to be kept in memory will be minimized. This is attained by maximizing the connectivity of the discourse, and by backtracking to stored addresses in a first-in-last-out fashion. These memory determinants of linearization are presumably biological, and independent of the domain of discourse. An important question is whether the linearization requirement is enforced by the oral modality of speech or whether it is a deeper modality-independent property of language use.