Human language has long been considered a unimodal activity, with the body being considered a mere vehicle
for expressing acoustic linguistic meaning. But theories of
language evolution point towards a close link between
vocal and visual communication early on in history,
pinpointing gesture as the origin of human language.
Some consider this link between gesture and
communicative vocalisations as having been temporary,
with conventionalized linguistic code eventually replacing
early bodily signaling. Others argue for this link being
permanent, positing that even fully-fledged human
language is a multi-modal phenomenon, with visual
signals forming integral components of utterances in faceto-
face conversation. My research provides evidence for
the latter. Based on this research, I will provide insights
into some of the factors and principles governing multimodal
language use in adult interaction. My talk consists
of three parts: First, I will present empirical findings
showing that movements we produce with our body are
indeed integral to spoken language and closely linked to
communicative intentions underlying speaking. Second, I
will show that bodily signals, first and foremost manual
gestures, play an active role in the coordination of
meaning during face-to-face interaction, including
fundamental processes like the grounding of referential
utterances. Third, I will present recent findings on the role
of bodily communicative acts in the psycholinguistically
challenging context of turn-taking during conversation.
Together, the data I present form the basis of a framework
aiming to capture multi-modal language use and
processing situated in face-to-face interaction, the
environment in which language first emerged, is acquired
and used most.