Presentations

Abstract

Nearly all human languages have grammars with complex recursive structures. These structures pose notable learning challenges. Two distributional properties of the input may facilitate learning: the presence of semantic biases (e.g. p(barks|dog) > p(talks|dog)) and the Zipf-distribution, with short sentences being extremely more frequent than longer ones. This project tested the effect of these sources of information on statistical learning of a hierarchical center-embedding grammar, using an artificial grammar learning paradigm. Semantic biases were represented by variations in transitional probabilities between words, with a biased input (p(barks|dog) > p(talks|dog)) compared to a non-biased input (p(barks|dog) = p(talks|dog)). The Zipf distribution was compared to a flat distribution, with sentences of different lengths occurring equally often. In a 2×2 factorial design, we tested for effects of biased transitional probabilities (biased/non-biased) and the distribution of sequences with varying length (Zipf distribution/flat distribution) on implicit learning and explicit ratings of grammaticality. Preliminary results show that a Zipf-shaped and semantically biased input facilitates grammar learnability. Thus, this project contributes to understanding how we learn complex structures with long-distance dependencies: learning may be sensitive to the specific distributional properties of the linguistic input, mirroring meaningful aspects of the world and favoring short utterances.

Abstract

The neural infrastructure for sentence production and comprehension has been found to be mostly shared. The same regions are engaged during speaking and listening, with some differences in their loading depending on modality (Giglio et al., 2022). In this fMRI study (n=40), we investigated whether modality affects the connectivity between inferior frontal and temporal regions, previously found to be involved in syntactic processing across modalities, and with the cerebellum, which has been historically linked with motor aspects of production. Participants produced or listened to word sequences of increasing constituent size. We found that constituent size reliably increased the connectivity between the inferior frontal gyrus and the posterior temporal lobe in both modalities. Preliminary cerebellar results suggest that different sub-regions presented different patterns of connectivity. Connectivity between Lobule VI and (pre)motor regions was increased during production relative to comprehension. Connectivity between Crus I/II and fronto-temporal regions was instead increased as a function of constituent size, and in particular during production. These results thus show that the connectivity between fronto-temporal regions is upregulated for syntactic structure building in both sentence production and comprehension, while cortico-cerebellar connectivity is enhanced both in response to syntactic processing and during production.

Abstract

Language comprehension may be facilitated by prediction, where a listener’s eye-gaze moves towards a referent before it is mentioned if the noun is predictable. Anticipatory fixations reduce when speech contains disfluencies (hesitations/repairs). Changes to the pattern of anticipatory fixations could result from a change in prediction or an attention shift. We combined EEG and eye-tracking to study the prediction of language in naturalistic, virtual environments (experiment 1 & 2) and the influence of disfluencies on predicting language (experiment 2). Participants (n=32; preliminary n=19) listened to sentences spoken by a virtual agent in various virtual scenes (e.g., office, street) while participants’ eye-movements and EEG were recorded. Spoken sentences were predictable or unpredictable, based on the verb constraints and referents were visible or absent in the scene to be congruent or incongruent with listeners’ predictions, respectively. In experiment 2, sentences were additionally either fluent or disfluent with ahesitation (uhh). Increased processing, reflected in increased theta power, was greater either at the predictive verb onset or at unpredictable noun onset in fluent sentences, but was observed at both predictable and unpredictable noun onsets in disfluent sentences. Our findings provide preliminary evidence supporting that hesitations reduce the weight listeners place on their predictions.

Abstract

Effective communication involves recognizing the disparity between our own perspective and that of the recipient, influenced by factors such as stereotypes and accents. The extent to which Theory of Mind (ToM), the ability to ascribe beliefs to others, plays a role in this process is uncertain. We anticipate individuals to speak in line with their beliefs, but what if their words conflict with our expectations? To investigate, we devised a virtual perspective-taking experiment where we manipulated a virtual agent's beliefs. Electroencephalography data were collected as participants listened to statements from the agent that either aligned or clashed with their true or false beliefs. We focused on the N400, an event-related brain component linked to word unexpectedness. As hypothesized, statements inconsistent with the agent's true beliefs triggered more pronounced N400 responses compared to matching statements. Furthermore, we anticipated that when the agent held a false belief, this knowledge would factor into interpreting their statements. Neither statements aligned with nor those diverging from the agent's false beliefs evoked N400 responses. This can be taken as evidence that participants did take the agents perspective into account. These results strongly support the role of Theory of Mind in language comprehension.

Abstract

Human communication requires interlocutors to mutually understand each other. Previous research has suggested inter-brain synchrony as an important feature of social interaction, since it has been observed during joint attention, speech interactions and cooperative tasks. Nonetheless, it is still unknown whether inter-brain synchrony is actually related to successful face-to-face communication. Here, we use dual-EEG to study if inter-brain synchrony is modulated during episodes of successful and unsuccessful communication in clear and noisy communication settings. Dyads performed a tangram-based referential communication task with and without background noise, while both their EEG and audiovisual behavior was recorded. Other-initiated repairs were annotated in the audiovisual data and were used as indexes of unsuccessful and successful communication. More specifically, we compared inter-brain synchrony during episodes of miscommunication (repair initiations) and episodes of mutual understanding (repair solutions and acceptance phases) in the clear and the noise condition. We expect that when communication is successful, inter-brain synchrony will be stronger than when communication is unsuccessful, and we expect that these patterns will be most pronounced in the noise condition. Results are currently being analyzed and will be presented and discussed with respect to the inter-brain neural signatures underlying the process of mutual understanding in face-to-face conversation.

Abstract

Monitoring auditory feedback is important for fluent speech production as it enables correction of vocalization errors. Influence of auditory feedback is best illustrated by manipulating it during speech production. A common temporal manipulation technique is delaying auditory feedback (DAF), which disrupts speech fluency, and a common spectral manipulation technique is perturbing the pitch of auditory feedback, which elicits vocal changes. We aimed to understand whether there is a correlation between sensitivity to temporal versus spectral manipulations of auditory feedback. We collected data from 40 participants. In the DAF task, participants repeated sentences, and auditory feedback was presented with 0 or 200ms delay. In the pitch perturbation task, participants phonated the vowel /a/ for 4 seconds and pitch of the auditory feedback was shifted by ±100 or ±200 cents.Voice recordings were analyzed using LME models to test the effects of feedback manipulations. We found that DAF significantly prolonged articulation duration and increased both voice pitch and intensity. Additionaly we found that large pitch shifts elicited less compensatory responses. There was a large individual variability in sensitivity to feedback manipulations for both tasks, however there was no correlation between the sensitivity profiles between tasks, suggesting that these features are processed differently.

Abstract

Phonological word forms are maintained in long-term memory and rapidly accessed during speech. However, the neural mechanisms that support the acquisition, maintenance, and recollection of word-form memories remain unclear. Starting from the hypothesis that dendrites and cell assemblies are the neural substrate for associating phoneme sequences to words, the present study investigates the dynamics of lexical access in a biologically constrained network model and compares it to the computational principles of human spoken word recognition. The model is a spiking recurrent network of dendritic neurons with realistic physiology and connectivity. The network implements unsupervised plasticity in the excitatory and inhibitory synapses, resulting in stable associations between phonemic and word representations. The activity of word assemblies indicates that, first, lexical representations are activated incrementally; second, lexical neighbors delay correct lexical access with both cohort and rhyme groups reactivated; and third, partial phonemic mismatch degrades word recognition. We also derive the model's predictions on the lexical bias effect and show that it is consistent with the hypothesis of offline feedback, providing a plausible mechanism for perceptual learning. In conclusion, we show that the dendrites could be the missing link between the perceptual space of phonemes and the mental lexicon.

Abstract

This study aims to test recent theoretical frameworks in speech motor control which claim that speech production targets are specified in auditory terms. According to such frameworks, people with better auditory acuity should have more precise speech targets. Participants performed speech perception and production tasks in a counterbalanced order. Speech perception acuity was assessed using an adaptive speech discrimination task, where participants discriminated between stimuli on a /ɪ/-/ɛ/ and a /ɑ/-/ɔ/ continuum. To assess variability in speech production, participants performed a pseudo-word reading task; formant values were measured for each recording of the vowels /ɪ/, /ɛ/, /ɑ/ and /ɔ/ in 288 pseudowords (18 per vowel, each of which was repeated 4 times). We predicted that speech production variability would correlate inversely with discrimination performance. Results confirmed this prediction as better discriminators had more distinctive vowel production targets. In addition, participants with higher auditory acuity produced vowels with smaller within-phoneme variability but spaced farther apart in vowel space. This study highlights the importance of individual differences in the study of speech motor control, and sheds light on speech production-perception interactions.

Abstract

This study aims to test a prediction of recent
theoretical frameworks in speech motor control: if
speech production targets are specified in auditory
terms, people with better auditory acuity should
have more precise speech targets.
To investigate this, we had participants perform
speech perception and production tasks in a
counterbalanced order. To assess speech perception
acuity, we used an adaptive speech discrimination
task. To assess variability in speech production,
participants performed a pseudo-word reading task;
formant values were measured for each recording.
We predicted that speech production variability to
correlate inversely with discrimination performance.
The results suggest that people do vary in their
production and perceptual abilities, and that better
discriminators have more distinctive vowel
production targets, confirming our prediction. This
study highlights the importance of individual
differences in the study of speech motor control, and
sheds light on speech production-perception
interaction.

Abstract

In investigations of feedback control during speech production, researchers have focused on two different kinds of responses to erroneous or unexpected auditory feedback. Compensation refers to online, feedback-based corrections of articulations. In contrast, adaptation refers to long-term changes in the speech production system after exposure to erroneous/unexpected feedback, which may last even after feedback is normal again. In the current study, we aimed to compare both types of feedback responses by investigating the conditions under which the system starts adapting in addition to merely compensating. Participants vocalized long vowels while they were exposed to either consistently altered auditory feedback, or to feedback that was unpredictably either altered or normal. Participants were not aware of the manipulation of auditory feedback. We predicted that both conditions would elicit compensation, whereas adaptation would be stronger when the altered feedback was consistent across trials. The results show that although there seems to be somewhat more adaptation for the consistently altered feedback condition, a substantial amount of individual variability led to statistically unreliable effects at the group level. The results stress the importance of taking into account individual differences and show that people vary widely in how they respond to altered auditory feedback.

Abstract

A recurring criticism in the field of psycholinguistics and is the lack of ecological validity of experimental designs. For example, many experiments on sentence comprehension are conducted enclosed booths, where sentences are presented word by word on a computer screen. In addition, very often participants are instructed to make judgments that relate directly to the experimental manipulation. Thus, the contexts in which these processes are studied is quite restricted, which calls into question the generalizability of the results to more naturalistic environments. A possible solution to this problem is the use of virtual reality (VR) in psycholinguistic experiments. By immersing participants into a virtual environment, ecological validity can be increased while experimental control is maintained.
In the current experiment we combine electroencephalography (EEG) and VR to look at semantic processing in a more naturalistic setting. During the experiment, participants move through a visually rich virtual restaurant. Tables and avatars are placed in the restaurant and participants are instructed to stop at each table and look at the object (e.g. a plate with a steak) in front of the avatar. Then, the avatar will produce an utterance to accompany the object (e.g. “I think this steak is very nice”), in which the noun will either match (e.g. steak) or mismatch (e.g. mandarin) with the item on the table. Based on previous research, we predict a modulation of the N400, which should be larger in the mismatch than the match condition. Implications of the use of virtual reality for experimental research will be discussed.

Abstract

Fluctuations in pupil size have been shown to reflect variations in processing demands during language
comprehension. Increases in pupil diameter have been observed as a consequence of syntactic anomalies
(Schluroff 1982), increased syntactic complexity (Just & Carpenter 1993) and lexical ambiguity (Ben-
Nun 1986). An issue that has not received attention is whether pupil size also varies due to pragmatic
manipulations. In a pupillometry experiment, we investigated whether pupil diameter is sensitive to
increased processing demands as a result of comprehending an indirect request versus a statement. During
natural conversation, communication is often indirect. For example, in an appropriate context, ''It'' cold in
here'' is a request to shut the window, rather than a statement about room temperature (Holtgraves 1994).
We tested 49 Dutch participants (mean age = 20.8). They were presented with 120 picture-sentence
combinations that could either be interpreted as an indirect request (a picture of a window with the
sentence ''it's hot here'') or as a statement (a picture of a window with the sentence ''it's nice here''). The
indirect requests were non-conventional, i.e. they did not contain directive propositional content and were
not directly related to the underlying felicity conditions (Holtgraves 2002). In order to verify that the
indirect requests were recognized, participants were asked to decide after each combination whether or
not they heard a request. Based on the hypothesis that understanding this type of indirect utterances
requires additional inferences to be made on the part of the listener (e.g., Holtgraves 2002; Searle 1975;
Van Ackeren et al. 2012), we predicted a larger pupil diameter for indirect requests than statements. The
data were analyzed using linear mixed-effects models in R, which allow for simultaneous inclusion of
participants and items as random factors (Baayen, Davidson, & Bates 2008). The results revealed a larger
mean pupil size and a larger peak pupil size for indirect requests as compared to statements. In line with
previous studies on pupil size and language comprehension (e.g., Just & Carpenter 1993), this difference
was observed within a 1.5 second window after critical word onset. We suggest that the increase in pupil
size reflects additional on-line processing demands for the comprehension of non-conventional indirect
requests as compared to statements. This supports the idea that comprehending this type of indirect
request requires capacity demanding inferencing on the part of the listener. In addition, this study
demonstrates the usefulness of pupillometry as a tool for experimental research in pragmatics.

Abstract

Fluctuations in pupil size have been shown to reflect variations in processing demands during language
comprehension. Increases in pupil diameter have been observed as a consequence of syntactic anomalies
(Schluroff 1982), increased syntactic complexity (Just & Carpenter 1993) and lexical ambiguity (Ben-
Nun 1986). An issue that has not received attention is whether pupil size also varies due to pragmatic
manipulations. In a pupillometry experiment, we investigated whether pupil diameter is sensitive to
increased processing demands as a result of comprehending an indirect request versus a statement. During
natural conversation, communication is often indirect. For example, in an appropriate context, ''It'' cold in
here'' is a request to shut the window, rather than a statement about room temperature (Holtgraves 1994).
We tested 49 Dutch participants (mean age = 20.8). They were presented with 120 picture-sentence
combinations that could either be interpreted as an indirect request (a picture of a window with the
sentence ''it's hot here'') or as a statement (a picture of a window with the sentence ''it's nice here''). The
indirect requests were non-conventional, i.e. they did not contain directive propositional content and were
not directly related to the underlying felicity conditions (Holtgraves 2002). In order to verify that the
indirect requests were recognized, participants were asked to decide after each combination whether or
not they heard a request. Based on the hypothesis that understanding this type of indirect utterances
requires additional inferences to be made on the part of the listener (e.g., Holtgraves 2002; Searle 1975;
Van Ackeren et al. 2012), we predicted a larger pupil diameter for indirect requests than statements. The
data were analyzed using linear mixed-effects models in R, which allow for simultaneous inclusion of
participants and items as random factors (Baayen, Davidson, & Bates 2008). The results revealed a larger
mean pupil size and a larger peak pupil size for indirect requests as compared to statements. In line with
previous studies on pupil size and language comprehension (e.g., Just & Carpenter 1993), this difference
was observed within a 1.5 second window after critical word onset. We suggest that the increase in pupil
size reflects additional on-line processing demands for the comprehension of non-conventional indirect
requests as compared to statements. This supports the idea that comprehending this type of indirect
request requires capacity demanding inferencing on the part of the listener. In addition, this study
demonstrates the usefulness of pupillometry as a tool for experimental research in pragmatics.