Presentations

Abstract

Nearly all human languages have grammars with complex recursive structures. These structures pose notable learning challenges. Two distributional properties of the input may facilitate learning: the presence of semantic biases (e.g. p(barks|dog) > p(talks|dog)) and the Zipf-distribution, with short sentences being extremely more frequent than longer ones. This project tested the effect of these sources of information on statistical learning of a hierarchical center-embedding grammar, using an artificial grammar learning paradigm. Semantic biases were represented by variations in transitional probabilities between words, with a biased input (p(barks|dog) > p(talks|dog)) compared to a non-biased input (p(barks|dog) = p(talks|dog)). The Zipf distribution was compared to a flat distribution, with sentences of different lengths occurring equally often. In a 2×2 factorial design, we tested for effects of biased transitional probabilities (biased/non-biased) and the distribution of sequences with varying length (Zipf distribution/flat distribution) on implicit learning and explicit ratings of grammaticality. Preliminary results show that a Zipf-shaped and semantically biased input facilitates grammar learnability. Thus, this project contributes to understanding how we learn complex structures with long-distance dependencies: learning may be sensitive to the specific distributional properties of the linguistic input, mirroring meaningful aspects of the world and favoring short utterances.

Abstract

The neural infrastructure for sentence production and comprehension has been found to be mostly shared. The same regions are engaged during speaking and listening, with some differences in their loading depending on modality (Giglio et al., 2022). In this fMRI study (n=40), we investigated whether modality affects the connectivity between inferior frontal and temporal regions, previously found to be involved in syntactic processing across modalities, and with the cerebellum, which has been historically linked with motor aspects of production. Participants produced or listened to word sequences of increasing constituent size. We found that constituent size reliably increased the connectivity between the inferior frontal gyrus and the posterior temporal lobe in both modalities. Preliminary cerebellar results suggest that different sub-regions presented different patterns of connectivity. Connectivity between Lobule VI and (pre)motor regions was increased during production relative to comprehension. Connectivity between Crus I/II and fronto-temporal regions was instead increased as a function of constituent size, and in particular during production. These results thus show that the connectivity between fronto-temporal regions is upregulated for syntactic structure building in both sentence production and comprehension, while cortico-cerebellar connectivity is enhanced both in response to syntactic processing and during production.

Abstract

Language comprehension may be facilitated by prediction, where a listener’s eye-gaze moves towards a referent before it is mentioned if the noun is predictable. Anticipatory fixations reduce when speech contains disfluencies (hesitations/repairs). Changes to the pattern of anticipatory fixations could result from a change in prediction or an attention shift. We combined EEG and eye-tracking to study the prediction of language in naturalistic, virtual environments (experiment 1 & 2) and the influence of disfluencies on predicting language (experiment 2). Participants (n=32; preliminary n=19) listened to sentences spoken by a virtual agent in various virtual scenes (e.g., office, street) while participants’ eye-movements and EEG were recorded. Spoken sentences were predictable or unpredictable, based on the verb constraints and referents were visible or absent in the scene to be congruent or incongruent with listeners’ predictions, respectively. In experiment 2, sentences were additionally either fluent or disfluent with ahesitation (uhh). Increased processing, reflected in increased theta power, was greater either at the predictive verb onset or at unpredictable noun onset in fluent sentences, but was observed at both predictable and unpredictable noun onsets in disfluent sentences. Our findings provide preliminary evidence supporting that hesitations reduce the weight listeners place on their predictions.

Abstract

Effective communication involves recognizing the disparity between our own perspective and that of the recipient, influenced by factors such as stereotypes and accents. The extent to which Theory of Mind (ToM), the ability to ascribe beliefs to others, plays a role in this process is uncertain. We anticipate individuals to speak in line with their beliefs, but what if their words conflict with our expectations? To investigate, we devised a virtual perspective-taking experiment where we manipulated a virtual agent's beliefs. Electroencephalography data were collected as participants listened to statements from the agent that either aligned or clashed with their true or false beliefs. We focused on the N400, an event-related brain component linked to word unexpectedness. As hypothesized, statements inconsistent with the agent's true beliefs triggered more pronounced N400 responses compared to matching statements. Furthermore, we anticipated that when the agent held a false belief, this knowledge would factor into interpreting their statements. Neither statements aligned with nor those diverging from the agent's false beliefs evoked N400 responses. This can be taken as evidence that participants did take the agents perspective into account. These results strongly support the role of Theory of Mind in language comprehension.

Abstract

Human communication requires interlocutors to mutually understand each other. Previous research has suggested inter-brain synchrony as an important feature of social interaction, since it has been observed during joint attention, speech interactions and cooperative tasks. Nonetheless, it is still unknown whether inter-brain synchrony is actually related to successful face-to-face communication. Here, we use dual-EEG to study if inter-brain synchrony is modulated during episodes of successful and unsuccessful communication in clear and noisy communication settings. Dyads performed a tangram-based referential communication task with and without background noise, while both their EEG and audiovisual behavior was recorded. Other-initiated repairs were annotated in the audiovisual data and were used as indexes of unsuccessful and successful communication. More specifically, we compared inter-brain synchrony during episodes of miscommunication (repair initiations) and episodes of mutual understanding (repair solutions and acceptance phases) in the clear and the noise condition. We expect that when communication is successful, inter-brain synchrony will be stronger than when communication is unsuccessful, and we expect that these patterns will be most pronounced in the noise condition. Results are currently being analyzed and will be presented and discussed with respect to the inter-brain neural signatures underlying the process of mutual understanding in face-to-face conversation.

Abstract

Monitoring auditory feedback is important for fluent speech production as it enables correction of vocalization errors. Influence of auditory feedback is best illustrated by manipulating it during speech production. A common temporal manipulation technique is delaying auditory feedback (DAF), which disrupts speech fluency, and a common spectral manipulation technique is perturbing the pitch of auditory feedback, which elicits vocal changes. We aimed to understand whether there is a correlation between sensitivity to temporal versus spectral manipulations of auditory feedback. We collected data from 40 participants. In the DAF task, participants repeated sentences, and auditory feedback was presented with 0 or 200ms delay. In the pitch perturbation task, participants phonated the vowel /a/ for 4 seconds and pitch of the auditory feedback was shifted by ±100 or ±200 cents.Voice recordings were analyzed using LME models to test the effects of feedback manipulations. We found that DAF significantly prolonged articulation duration and increased both voice pitch and intensity. Additionaly we found that large pitch shifts elicited less compensatory responses. There was a large individual variability in sensitivity to feedback manipulations for both tasks, however there was no correlation between the sensitivity profiles between tasks, suggesting that these features are processed differently.

Abstract

Phonological word forms are maintained in long-term memory and rapidly accessed during speech. However, the neural mechanisms that support the acquisition, maintenance, and recollection of word-form memories remain unclear. Starting from the hypothesis that dendrites and cell assemblies are the neural substrate for associating phoneme sequences to words, the present study investigates the dynamics of lexical access in a biologically constrained network model and compares it to the computational principles of human spoken word recognition. The model is a spiking recurrent network of dendritic neurons with realistic physiology and connectivity. The network implements unsupervised plasticity in the excitatory and inhibitory synapses, resulting in stable associations between phonemic and word representations. The activity of word assemblies indicates that, first, lexical representations are activated incrementally; second, lexical neighbors delay correct lexical access with both cohort and rhyme groups reactivated; and third, partial phonemic mismatch degrades word recognition. We also derive the model's predictions on the lexical bias effect and show that it is consistent with the hypothesis of offline feedback, providing a plausible mechanism for perceptual learning. In conclusion, we show that the dendrites could be the missing link between the perceptual space of phonemes and the mental lexicon.

Abstract

When talking, speakers continuously monitor the auditory feedback of their own voice to control and inform speech production processes. When speakers are provided with auditory feedback that is perturbed in real time, most of them compensate for this by opposing the feedback perturbation. For example, when speakers hear themselves at a higher pitch than intended, they would compensate by lowering their pitch. However, sometimes speakers follow the perturbation instead (i.e., raising their pitch in response to higher-than-expected pitch). Current theoretical frameworks cannot account for following responses. In the current study, we performed two experiments to investigate whether the state of the speech production system at perturbation onset may determine what type of response (opposing or following) is given. Participants vocalized while the pitch in their auditory feedback was briefly (500 ms) perturbed in half of the vocalizations. None of the participants were aware of these manipulations. Subsequently, we analyzed the pitch contour of the participants’ vocalizations. The results suggest that whether a perturbation-related response is opposing or following unexpected feedback depends on ongoing fluctuations of the production system: It initially responds by doing the opposite of what it was doing. In addition, the results show that all speakers show both following and opposing responses, although the distribution of response types varies across individuals. Both the interaction with ongoing fluctuations and the non-trivial number of following responses suggest that current speech production models are inadequate. More generally, the current study indicates that looking beyond the average response can lead to a more complete view on the nature of feedback processing in motor control. Future work should explore whether the direction of feedback-based control in domains outside of speech production will also be conditional on the state of the motor system at the time of the perturbation.

Abstract

Dyslexia is a learning disorder caused primarily by a phonological processing deficit. So far, few studies have examined whether dyslexia deficits extend to syntactic processing. We investigated how dyslexic children process syntactic structures. In a self-paced reading syntactic priming paradigm, the passive voice was repeated in mini-blocks of five sentences. These were mixed with an equal number of filler mini-blocks (actives, intransitives); the verb was repeated within all mini-blocks. The data of 20 dyslexic children (Mean(age)=12,8), native speakers of Brazilian Portuguese, were compared to that of 25 non-dyslexic children (Mean(age)=10,4 years). A repeated-measures ANOVA on reading times for the verb revealed a significant sentence repetition (p<.001) and group by sentence repetition effect (p<.001). Dyslexics demonstrated priming effects between all consecutive passive voice repetitions (all p<.05), whereas reading times for controls differed only between the first and second passive (p<.001). For active sentences, dyslexics showed priming effects only between the first and second sentences (p<.05) while controls did not show any significant effect, suggesting that the effects for passives are not solely due to the verb being repeated, but at least in part due to the repeated syntactic structure. These findings thus reveal syntactic processing differences between dyslexic and non-dyslexic children.

Abstract

In certain situations, human listeners have more difficulty in understanding speech in a multi-talker environment than in the presence of non-intelligible noise. The costs of speech-in-speech masking have been attributed to informational masking, i.e. to the competing processing of the target and the distractor speech’s information. It remains unclear what kind of information is competing, as intelligible speech and unintelligible speech-like signals (e.g. reversed, noise-vocoded, and foreign speech) differ both in linguistic content and in acoustic information. Thus, intelligible speech could be a stronger distractor than unintelligible speech because it presents closer acoustic information to the target speech, or because it carries competing linguistic information. In this study, we intended to isolate the linguistic component of speech-in-speech masking and we tested its influence on the comprehension of target speech. To do so, 24 participants performed a dichotic listening task in which the interfering stimuli consisted of 4-band noise-vocoded sentences that could become intelligible through training. The experiment included three steps: first, the participants were instructed to report the clear target speech from a mixture of one clear speech channel and one unintelligible noise-vocoded speech channel; second, they were trained on the interfering noise-vocoded sentences so that they became intelligible; third, they performed the dichotic listening task again. Crucially, before and after training, the distractor speech had the same acoustic features but not the same linguistic information. We thus predicted that the distracting noise-vocoded signal would interfere more with target speech comprehension after training than before training. To control for practice/fatigue effects, we used additional 2-band noise-vocoded sentences, which participants were not trained on, as interfering signals in the dichotic listening tasks. We expected that performance on these trials would not change after training, or would change less than that on trials with trained 4-band noise-vocoded sentences. Performance was measured under three SNR conditions: 0, -3, and -6 dB. The behavioral results are consistent with our predictions. The 4-band noise-vocoded signal interfered more with the comprehension of target speech after training (i.e. when it was intelligible) compared to before training (i.e. when it was unintelligible), but only at SNR -3dB. Crucially, the comprehension of the target speech did not change after training when the interfering signals consisted of unintelligible 2-band noise-vocoded speech sounds, ruling out a fatigue effect. In line with previous studies, the present results show that intelligible distractors interfere more with the processing of target speech. These findings further suggest that speech-in-speech interference originates, to a certain extent, from the parallel processing of competing linguistic content. A magnetoencephalography study with the same design is currently being performed, to specifically investigate the neural origins of informational masking.