Presentations

Abstract

This study aims to test recent theoretical frameworks in speech motor control which claim that speech production targets are specified in auditory terms. According to such frameworks, people with better auditory acuity should have more precise speech targets. Participants performed speech perception and production tasks in a counterbalanced order. Speech perception acuity was assessed using an adaptive speech discrimination task, where participants discriminated between stimuli on a /ɪ/-/ɛ/ and a /ɑ/-/ɔ/ continuum. To assess variability in speech production, participants performed a pseudo-word reading task; formant values were measured for each recording of the vowels /ɪ/, /ɛ/, /ɑ/ and /ɔ/ in 288 pseudowords (18 per vowel, each of which was repeated 4 times). We predicted that speech production variability would correlate inversely with discrimination performance. Results confirmed this prediction as better discriminators had more distinctive vowel production targets. In addition, participants with higher auditory acuity produced vowels with smaller within-phoneme variability but spaced farther apart in vowel space. This study highlights the importance of individual differences in the study of speech motor control, and sheds light on speech production-perception interactions.

Abstract

This study aims to test a prediction of recent
theoretical frameworks in speech motor control: if
speech production targets are specified in auditory
terms, people with better auditory acuity should
have more precise speech targets.
To investigate this, we had participants perform
speech perception and production tasks in a
counterbalanced order. To assess speech perception
acuity, we used an adaptive speech discrimination
task. To assess variability in speech production,
participants performed a pseudo-word reading task;
formant values were measured for each recording.
We predicted that speech production variability to
correlate inversely with discrimination performance.
The results suggest that people do vary in their
production and perceptual abilities, and that better
discriminators have more distinctive vowel
production targets, confirming our prediction. This
study highlights the importance of individual
differences in the study of speech motor control, and
sheds light on speech production-perception
interaction.

Abstract

In investigations of feedback control during speech production, researchers have focused on two different kinds of responses to erroneous or unexpected auditory feedback. Compensation refers to online, feedback-based corrections of articulations. In contrast, adaptation refers to long-term changes in the speech production system after exposure to erroneous/unexpected feedback, which may last even after feedback is normal again. In the current study, we aimed to compare both types of feedback responses by investigating the conditions under which the system starts adapting in addition to merely compensating. Participants vocalized long vowels while they were exposed to either consistently altered auditory feedback, or to feedback that was unpredictably either altered or normal. Participants were not aware of the manipulation of auditory feedback. We predicted that both conditions would elicit compensation, whereas adaptation would be stronger when the altered feedback was consistent across trials. The results show that although there seems to be somewhat more adaptation for the consistently altered feedback condition, a substantial amount of individual variability led to statistically unreliable effects at the group level. The results stress the importance of taking into account individual differences and show that people vary widely in how they respond to altered auditory feedback.

Abstract

A recurring criticism in the field of psycholinguistics and is the lack of ecological validity of experimental designs. For example, many experiments on sentence comprehension are conducted enclosed booths, where sentences are presented word by word on a computer screen. In addition, very often participants are instructed to make judgments that relate directly to the experimental manipulation. Thus, the contexts in which these processes are studied is quite restricted, which calls into question the generalizability of the results to more naturalistic environments. A possible solution to this problem is the use of virtual reality (VR) in psycholinguistic experiments. By immersing participants into a virtual environment, ecological validity can be increased while experimental control is maintained.
In the current experiment we combine electroencephalography (EEG) and VR to look at semantic processing in a more naturalistic setting. During the experiment, participants move through a visually rich virtual restaurant. Tables and avatars are placed in the restaurant and participants are instructed to stop at each table and look at the object (e.g. a plate with a steak) in front of the avatar. Then, the avatar will produce an utterance to accompany the object (e.g. “I think this steak is very nice”), in which the noun will either match (e.g. steak) or mismatch (e.g. mandarin) with the item on the table. Based on previous research, we predict a modulation of the N400, which should be larger in the mismatch than the match condition. Implications of the use of virtual reality for experimental research will be discussed.

Abstract

Fluctuations in pupil size have been shown to reflect variations in processing demands during language
comprehension. Increases in pupil diameter have been observed as a consequence of syntactic anomalies
(Schluroff 1982), increased syntactic complexity (Just & Carpenter 1993) and lexical ambiguity (Ben-
Nun 1986). An issue that has not received attention is whether pupil size also varies due to pragmatic
manipulations. In a pupillometry experiment, we investigated whether pupil diameter is sensitive to
increased processing demands as a result of comprehending an indirect request versus a statement. During
natural conversation, communication is often indirect. For example, in an appropriate context, ''It'' cold in
here'' is a request to shut the window, rather than a statement about room temperature (Holtgraves 1994).
We tested 49 Dutch participants (mean age = 20.8). They were presented with 120 picture-sentence
combinations that could either be interpreted as an indirect request (a picture of a window with the
sentence ''it's hot here'') or as a statement (a picture of a window with the sentence ''it's nice here''). The
indirect requests were non-conventional, i.e. they did not contain directive propositional content and were
not directly related to the underlying felicity conditions (Holtgraves 2002). In order to verify that the
indirect requests were recognized, participants were asked to decide after each combination whether or
not they heard a request. Based on the hypothesis that understanding this type of indirect utterances
requires additional inferences to be made on the part of the listener (e.g., Holtgraves 2002; Searle 1975;
Van Ackeren et al. 2012), we predicted a larger pupil diameter for indirect requests than statements. The
data were analyzed using linear mixed-effects models in R, which allow for simultaneous inclusion of
participants and items as random factors (Baayen, Davidson, & Bates 2008). The results revealed a larger
mean pupil size and a larger peak pupil size for indirect requests as compared to statements. In line with
previous studies on pupil size and language comprehension (e.g., Just & Carpenter 1993), this difference
was observed within a 1.5 second window after critical word onset. We suggest that the increase in pupil
size reflects additional on-line processing demands for the comprehension of non-conventional indirect
requests as compared to statements. This supports the idea that comprehending this type of indirect
request requires capacity demanding inferencing on the part of the listener. In addition, this study
demonstrates the usefulness of pupillometry as a tool for experimental research in pragmatics.

Abstract

Fluctuations in pupil size have been shown to reflect variations in processing demands during language
comprehension. Increases in pupil diameter have been observed as a consequence of syntactic anomalies
(Schluroff 1982), increased syntactic complexity (Just & Carpenter 1993) and lexical ambiguity (Ben-
Nun 1986). An issue that has not received attention is whether pupil size also varies due to pragmatic
manipulations. In a pupillometry experiment, we investigated whether pupil diameter is sensitive to
increased processing demands as a result of comprehending an indirect request versus a statement. During
natural conversation, communication is often indirect. For example, in an appropriate context, ''It'' cold in
here'' is a request to shut the window, rather than a statement about room temperature (Holtgraves 1994).
We tested 49 Dutch participants (mean age = 20.8). They were presented with 120 picture-sentence
combinations that could either be interpreted as an indirect request (a picture of a window with the
sentence ''it's hot here'') or as a statement (a picture of a window with the sentence ''it's nice here''). The
indirect requests were non-conventional, i.e. they did not contain directive propositional content and were
not directly related to the underlying felicity conditions (Holtgraves 2002). In order to verify that the
indirect requests were recognized, participants were asked to decide after each combination whether or
not they heard a request. Based on the hypothesis that understanding this type of indirect utterances
requires additional inferences to be made on the part of the listener (e.g., Holtgraves 2002; Searle 1975;
Van Ackeren et al. 2012), we predicted a larger pupil diameter for indirect requests than statements. The
data were analyzed using linear mixed-effects models in R, which allow for simultaneous inclusion of
participants and items as random factors (Baayen, Davidson, & Bates 2008). The results revealed a larger
mean pupil size and a larger peak pupil size for indirect requests as compared to statements. In line with
previous studies on pupil size and language comprehension (e.g., Just & Carpenter 1993), this difference
was observed within a 1.5 second window after critical word onset. We suggest that the increase in pupil
size reflects additional on-line processing demands for the comprehension of non-conventional indirect
requests as compared to statements. This supports the idea that comprehending this type of indirect
request requires capacity demanding inferencing on the part of the listener. In addition, this study
demonstrates the usefulness of pupillometry as a tool for experimental research in pragmatics.

Abstract

Recent work suggests that monitoring in speech production may occur via domain-general mechanisms responsible for detecting response conflict. To test this hypothesis, we measured EEG as people engaged in both non-verbal (flanker) and verbal (tongue twisters) tasks designed to elicit response conflict and errors. In the flanker task, people pressed a button corresponding to whether a center arrow was facing left or right, and response conflict was induced with flanking arrows pointing in the same (congruent; >>>>>) or a different (incongruent >><>>) direction. In the tongue twister task, people read sequences of four nonwords three times in which rhymes alternated in an ABAB pattern while onset speech sounds alternated in an ABBA (tongue twister) or an ABAB (non-tongue twister) pattern (e.g., tif deev dif teev vs. tif teev dif deef). Results in the fl anker task showed standard markers of response conflict in the form of an increased N2 for incongruent relative to congruent trials as well as an error-related negativity (ERN) for incorrect trials. Behaviourally, more errors were elicited for tongue twisters relative to nontongue twister trials, and an ERN was observed on incorrect responses. Correlations between the magnitude of the N2 and ERN in the fl anker task with the magnitude of the ERN and error rates in the tongue twister task are consistent with a common underlying locus. Adaptation effects preceding and following erroneous trials in production are also presented. These results are consistent with response confl ict serving as a cue to monitoring in speech production.

Abstract

A central and influential idea among researchers of language is that our language faculty is organized according to the principle of strict compositionality, which implies that the meaning of an utterances is a function of the meaning of its parts and of the syntactic rules by which these parts are combined. The implication of this idea is that beyond word recognition, language interpretation takes place in a two-step fashion. First, the meaning of a sentence is computed. In a second step the sentence meaning is integrated with information from prior discourse, with world knowledge, with information about the speaker, and with semantic information from extralinguistic domains such as co-speech gestures or the visual world. FMRI results and results from recordings of event related brain potentials will be presented that are inconsistent with this classical model of language intepretation. Our data support a model in which knowledge about the context and the world, knowledge about concomitant information from other modalities, and knowledge about the speaker are brought to bear immediately, by the same fast-acting brain system that combines the meanings of individual words into a message-level representation. The Memory, Unification and Control (MUC) model provides a neurobiological plausible account of the underlying neural architecture. Resting state connectivity data, and results from Psycho-Physiological Interactions will be discussed, suggesting a division of labour between temporal and inferior frontal cortex. These results indicate that Broca’s area and adjacent cortex play an important role in semantic and syntactic unification operations. I will also discuss fMRI results that indicate the insufficiency of the Mirror Neuron Hypothesis to explain language understanding. Instead I will sketch a picture of language processing from an embrained perspective.

Abstract

From a functionalist perspective all that brain research is claimed to have told us is that language processing "happens somewhere north of the neck" (Jerry Fodor, 1999). I will argue why I disagree with this conclusion, for at least the following three reasons. First, one fundamental question in the language sciences is: what makes the human brain language-ready? Understanding the neural architecture that supports human language function is a crucial part of the explanandum. I will show some unique features of human perisylvian cortex based on data from Diffusion Tensor Imaging and resting state fMRI. The second argument is that even if one is only interested in the cognitive architecture of language comprehension and production, relevant evidence can be obtained from neurobiological data, both structural and functional. I will discuss the consequences of connectivity patterns in the brain for assumptions in processing models of language, and I will show fMRI data based on a repetition suppression paradigm that provide evidence for the claim that syntactic encoding and parsing are based on the same mechanism. Finally, I will argue that framing theories of sentence processing in a way that connects to other areas of cognitive neuroscience might be helpful in asking interesting and relevant new questions. I will illustrate this in the context of the Memory, Unification and Control (MUC) model of language.

Abstract

A central and influential idea among researchers of language is that our language faculty is organized according to the principle of strict compositionality, which implies that the meaning of an utterances is a function of the meaning of its parts and of the syntactic rules by which these parts are combined. The implication of this idea is that beyond word recognition, language interpretation takes place in a two-step fashion. First, the meaning of a sentence is computed. In a second step the sentence meaning is integrated with information from prior discourse, with world knowledge, with information about the speaker, and with semantic information from extralinguistic domains such as co-speech gestures or the visual world. FMRI results and results from recordings of event related brain potentials will be presented that are inconsistent with this classical model of language intepretation. Our data support a model in which knowledge about the context and the world, knowledge about concomitant information from other modalities, and knowledge about the speaker are brought to bear immediately, by the same fast-acting brain system that combines the meanings of individual words into a message-level representation. The Memory, Unification and Control (MUC) model provides a neurobiological plausible account of the underlying neural architecture. Resting state connectivity data, and results from Psycho-Physiological Interactions will be discussed, suggesting a division of labour between temporal and inferior frontal cortex. These results indicate that Broca’s area and adjacent cortex play an important role in semantic and syntactic unification operations. I will also discuss fMRI results that indicate the insufficiency of the Mirror Neuron Hypothesis to explain language understanding. Instead I will sketch a picture of language processing from an embrained perspective.

Abstract

Traditionally, language comprehension has been studied as a solitary and unimodal activity. Here, we investigate language comprehension as a joint activity, i.e., in a dynamic social context involving multiple participants in different roles with different perspectives, while taking into account the multimodal nature of facetoface communication. We simulated a triadic communication context involving a speaker alternating her gaze between two different recipients, conveying information not only via speech but gesture as well. Participants thus viewed videorecorded speechonly or speech+gesture utterances referencing objects (e.g., “he likes the laptop”/+TYPING ON LAPTOPgesture) when being addressed (direct gaze) or unaddressed (averted gaze). The videoclips were followed by two object images (laptoptowel). Participants’ task was to choose the object that matched the speaker’s message (i.e., laptop). Unaddressed recipients responded significantly slower than addressees for speechonly utterances. However, perceiving the same speech accompanied by gestures sped them up to levels identical to that of addressees. Thus, when speech processing suffers due to being unaddressed, gestures become more prominent and boost comprehension of a speaker’s spoken message. Our findings illuminate how participants process multimodal language and how this process is influenced by eye gaze, an important social cue facilitating coordination in the joint activity of conversation.