Presentations

Abstract

The ability to use linguistic representations to refer to the world is a vital mechanism that gives human language its communicative power. In particular, the anaphoric use of words to refer to previously mentioned concepts (antecedents) is what allows dialogue to be coherent and meaningful. Psycholinguistic theory posits that anaphor comprehension involves reactivating an episodic memory representation of the antecedent [1-2]. Whereas this implies the involvement of memory structures, the neural processes for reference resolution are largely unknown. Here, we report time-frequency analysis of four EEG experiments [3-6], revealing the increased coupling of functional neural systems associated with coherent referring expressions compared to referentially ambiguous expressions. We performed time-frequency analysis on data from four experiments in which referentially ambiguous expressions elicited a sustained negativity in the ERP waveform compared to coherent expressions. In Experiment 1, 32 participants read 120 correct Dutch sentences with coherent or ambiguous pronouns. In Experiment 2, 31 participants listened to 90 naturally spoken Dutch mini-stories containing coherent or ambiguous NP anaphora. In Experiment 3, 22 participants each read 60 Spanish sentences with a coherent or ambiguous ellipsis determiner. In Experiment 4, 19 participants each read 180 grammatically correct English sentences containing coherent or ambiguous pronouns. Analysis was performed with Fieldtrip [7], separately for low frequency (2-30 Hz) and high frequency (25-90 Hz) activity. Power-changes per trial were computed as a relative change from a pre-CW baseline interval, average power changes were computed per subject for coherent and ambiguous conditions separately. Statistical tests used cluster-based random permutation [8]. Despite varying in modality, language and type of expression, all experiments showed larger gamma-band power around 80 Hz for coherence compared to ambiguity, within a similar time range. No differences were observed in low frequencies. In high-density EEG Experiment 4, an additional short-duration gamma-increase was observed around 40 Hz, around 300-500 ms after pronoun-onset, which was localised using Beamformer analysis [9] to left posterior parietal cortex (PPC). The 80 Hz power increase around 600-1200 ms after word onset was localised to left inferior frontal-temporal cortex. We argue that the observed gamma-band power increases reflect successful referential binding and resolution, linking incoming information to previously encountered concepts and integrates that information into the unfolding discourse representation. Specifically, we argue that this involves antecedent reactivation in the PPC episodic memory network [10-11], interacting with unification processes in the frontal-temporal language network [12]. Based on these results, and on results of patient [13] and fMRI [14] research on pronoun comprehension, we propose an initial neurobiological account of reference, by bridging the psycholinguistics of anaphora with the neurobiology of language and of episodic memory. [1] Dell et al., 1983 [2] Gerrig & McKoon, 1998 [3] Nieuwland & Van Berkum, 2006 [4] Nieuwland et al., 2007a [5] Martin et al., 2012 [6] Nieuwland, 2014 [7] Oostenveld et al., 2011 [8] Maris & Oostenveld, 2007 [9] Gross et al., 2001 [10] Shannon & Buckner, 2004 [11] Wager et al., 2005 [12] Hagoort & Indefrey, 2014 [13] Kurczek et al., 2013 [14] Nieuwland et al., 2007b

Abstract

The ability to use words to refer to the world is a vital mechanism that gives human language its communicative power. In particular, the use of words to refer to previously mentioned concepts (anaphora) is what allows dialogue to be coherent and meaningful. Psycholinguistic theory posits that anaphor comprehension involves reactivating a memory representation of the antecedent. Whereas this implies the involvement of episodic memory, the neural processes for reference resolution are largely unknown. Here, we report time-frequency analysis of four EEG experiments to reveal the increased coupling of functional neural systems associated with referring expressions that can be straightforwardly understood compared to those that cannot (referential coherence or ambiguity). Despite varying in modality, language and type of referential expression, all experiments showed larger gamma-band power for coherence compared to ambiguity. In high-density EEG Experiment 4, Beamformer analysis localised this increase to the posterior parietal cortex around 300-500 ms after onset of the anaphor and to frontal-temporal cortex around 500-1000 ms. We argue that the observed gamma-band power increases reflect successful referential binding and resolution, which links incoming information to previously encountered concepts through an interaction between the episodic memory network and the frontal-temporal language network.

Abstract

In an event-related fMRI study, we investigated to what extent semantic and
referential aspects of language comprehension recruit common or dis-
tinct neural ensembles. We compared BOLD responses to sentences containing semantically anomalous or coherent words, and to sentences containing referentially ambiguous pronouns (e.g., “Ronald told Frank that he...”), referentially failing pronouns (e.g., “Rose told Emily that he...”) or coherent pronouns. Semantic anomaly elicited activation increases in lateral prefrontal brain regions associated with semantic pro-
cessing. Referential failure elicited
activation increases in brain regions
associated with morphosyntactic processing, and additional activations
associated with elaborative inferenc
ing if readers took failing pronouns
to refer to unmentioned entities. Referential ambiguity selectively
recruited medial prefrontal regions,
suggesting that readers engaged in
problem-solving to select a unique
referent from the discourse model.
Furthermore, our results showed that semantic anomaly and referential
ambiguity recruit overlapping neural ensembles in opposite directions,
possibly reflecting the dynamic re
cruitment of semantic and episodic
processing to resolve semantically or referentially problematic situations.
These findings suggest that neurocogni
tive accounts of language compre-
hension will have to address not just how we parse a sentence and com-
bine individual word meanings, bu
t also how we determine who’s who
and what’s what during sentence and discourse comprehension

Abstract

In this event-related brain potentials (E RP) study, we explored the possibility to selectively track referential ambiguity during spoken discourse comprehension. Earlier ERP research has shown that referentially am biguous nouns (e.g., “the girl” in a two-girl context) elicit a frontal, sustained negative sh ift relative to unambiguous control words. In the current study, we examined whether this ERP effect reflec ts ‘deep’ situation model ambiguity or ‘superficial’ textbase am biguity. We contrast ed these different interpretations by investigating whether a discourse-level semantic manipulation that prevents referential ambiguity also averts t he elicitation of a referentially induced ERP effect. We compared ERPs el icited by nouns that were re ferentially non-ambiguous but were associated with two discourse entities (e .g., “the girl” with two girls introduced in the context, but one of which has died or le ft the scene), with referentially ambiguous and non-ambiguous control words. While temporally referentially ambiguous nouns elicited a frontal negative shift compared to control words, the ‘double bo und’ but referentially non-ambiguous nouns did not. These results suggest that it is possible to selectively track referential ambiguity with ERPs at the level that is most relev ant to discourse comprehension, the situation model