Presentations

Abstract

Subject-verb agreement is one of the most common
grammatical encoding operations in language
production. In many languages, morphological
inflection on verbs code for the number of the head
noun of a subject phrase (e.g., The key to the cabinets
is rusty). Despite the relative ease with which subjectverb
agreement is accomplished, people sometimes
make agreement errors (e.g., The key to the cabinets
are rusty). Such errors offer a window into the early
stages of production planning. Agreement errors are
influenced by both syntactic and semantic factors, and
are more likely to occur when a sentence contains either
conceptual or syntactic number mismatches. Little
is known about the timecourse of these influences,
however, and some controversy exists as to whether
they are independent. The current study was designed
to address these two issues using EEG. Semantic and
syntactic factors influencing number mismatch were
factorially-manipulated in a forced-choice sentence
completion paradigm. To avoid EEG artifact associated
with speaking, participants (N=20) were presented with
a noun-phrase, and pressed a button to indicate which
version of the verb ‘to be’ (is/are) should continue
the sentence. Semantic number was manipulated
using preambles that were semantically-integrated or
unintegrated. Semantic integration refers to the semantic
relationship between nouns in a noun-phrase, with
integrated items promoting conceptual-singularity.
The syntactic manipulation was the number (singular/
plural) of the local noun preceding the decision. This
led to preambles such as “The pizza with the yummy
topping(s)... “ (integated) vs. “The pizza with the tasty
bevarage(s)...” (unintegrated). Behavioral results showed
effects of both Local Noun Number and Semantic
Integration, with more errors and longer reaction times
occurring in the mismatching conditions (i.e., plural
local nouns; unintegrated subject phrases). Classic ERP
analyses locked to the local noun (0-700 ms) and to the
time preceding the response (-600 to 0 ms) showed no
systematic differences between conditions. Despite this
result, we assessed whether difference might emerge
using multivariate pattern analysis (MVPA). Using the
same epochs as above, support-vector machines with a
radial basis function were trained on the single-trial level
to classify the difference between Local Noun Number
and Semantic Integration conditions across time and
channels. Results revealed that both conditions could
be reliably classified at the single subject level, and
that classification accuracy was strongest in the epoch
preceding the response. Classification accuracy was
at chance when a classifier trained to dissociate Local
Noun Number was used to predict Semantic Integration
(and vice versa), providing some evidence of the
independence of the two effects. Significant inter-subject
variability was present in the channels and time-points
that were critical for classification, but earlier timepoints
were more often important for classifying Local Noun
Number than Semantic Integration. One result of this
variability is classification performed across subjects was
at chance, which may explain the failure to find standard
ERP effects. This study thus provides an important first
test of semantic and syntactic influences on subject-verb
agreement with EEG, and demonstrates that where
classic ERP analyses fail, MVPA can reliably distinguish
differences at the neurophysiological level.

Abstract

Prominent theories of predictive language processing assume that language production processes are used to anticipate upcoming linguistic input during comprehension (Dell & Chang, 2014; Pickering & Garrod, 2013). Here, we explore the converse case: Does a task set including production in addition to comprehension encourage prediction, compared to a task only including comprehension? To test this hypothesis, we conducted a cross-modal naming experiment (Experiment 1) including an object naming task and a self-paced reading experiment (Experiment 2) that did not include overt production. We used the same predictable (N = 40) and non-predictable (N = 40) sentences in both experiments. The sentences consisted of a fixed agent, a transitive verb and a predictable or non-predictable target word (The man drinks a beer vs. The man buys a beer). Most of the empirical work on prediction used sentences in which the target words were highly predictable (often with a mean cloze probability > .8) and thus it is little surprising that participants engaged in predictive language processing very easily. In the current sentences, the mean cloze probability in the predictable sentences was .39 (ranging from .06 to .8; zero in the non-predictable sentences). If comprehenders are more likely to engage in predictive processing when the task set involves production, we should observe more pronounced effects of prediction in Experiment 1 as compared to Experiment 2. If production does not enhance prediction, we should observe similar effects of prediction in both experiments. In Experiment 1, participants (N = 54) listened to recordings of the sentences which ended right before the spoken target word. Coinciding with the end of the playback, a picture of the target word was shown which the participants were asked to name as fast as possible. Analyses of their naming latencies revealed a statistically significant naming advantage of 106 ms on predictable over non-predictable trials. Moreover, we found that the objects’ naming advantage was predicted by the target words’ cloze probability in the sentences (r = .411, p = .016). In Experiment 2, the same sentences were used in a self-paced reading experiment. To allow for testing of potential spill-over effects, we added a neutral prepositional phrase (buys a beer from the bar keeper/drinks a beer from the shop) to each sentence. Participants (N = 54) read the sentences word-by-word, advancing by pushing the space bar. On 30% of the trials, comprehension questions were used to keep up participants' focus on comprehending the sentences. Analyses of participants’ target and post-target reading times revealed numerical advantages of 6 ms and 20 ms, respectively, in the predictable as compared to the non-predictable condition. However, in both cases, this difference was not statistically reliable (t = .757, t = 1.43) and the significant positive correlation between an item’s naming advantage and its cloze probability as seen in Experiment 1 was absent (r = .037, p = .822). Importantly, the analysis of participants' responses to the comprehension questions, showed that they understood the sentences (mean accuracy = 93%). To conclude, although both experiments used the same sentences, we observed effects of prediction only when the task included production. In Experiment 2, no evidence for anticipation was found although participants clearly understood the sentences and the method has previously been shown to be sensitive to measure prediction effects (Van Berkum et al., 2005). Our results fit with a recent study by Gollan et al. (2011) who found only a small processing advantage of predictive over non-predictive sentences in reading (using highly predictable sentences with a cloze probability > . 87) but a strong prediction effect when participants read the same sentences and carried out an additional object naming task (see also Griffin & Bock, 1998). Taken together, the studies suggest that the comprehenders' task set exerts a powerful influence on the likelihood and magnitude of predictive language processing. When the task set involves language production, as is often the case in natural conversation, comprehenders might engage in prediction to a stronger degree than in pure comprehension tasks. Being able to predict words another person is about to say might optimize the comprehension process and enable smooth turn-taking.

Abstract

When processing visual objects, we integrate visual, linguistic and spatial information to form an episodic trace. Re-activating one aspect of the episodic trace of an object re-activates the entire bundle making all integrated information available. Using the blank screen paradigm [1], researchers observed that upon processing spoken linguistic input, participants tended to make eye movements on a blank screen, fixating locations that were previously occupied by objects mentioned in the linguistic utterance or were related. Ferreira and colleagues [2] suggested that 'looking at nothing' facilitated memory retrieval. However, this claim lacks convincing empirical support. In Experiment 1, Dutch participants looked at four-object-displays. Three objects were related to a spoken target word. Given the target word 'beker' (beaker), the display featured a phonological (a bear), a shape (a bobbin), a semantic (a fork) competitor, and an unrelated distractor (an umbrella). Participants were asked to name the objects as fast as possible. Subsequently, the objects disappeared. Participants fixated the center of the screen and listened to the target word. They had to carry out a semantic judgment task (indicating in which position an object had appeared that was semantically related to the objects) or a visual shape similarity judgment (indicating the position of the object similar in shape to the target). In both conditions, we observed that participants re-fixated the empty target location before responding. The set-up of Experiment 2 was identical except that we asked participants to maintain fixating the center of the screen while listening to the spoken word and responding. Performance accuracy was significantly lower in Experiment 2 than in Experiment 1. The results indicate that memory retrieval for objects is impaired when participants are not allowed to look at relevant, though empty locations. [1] Altmann, G. (2004). Language-mediated eye movements in the absence of a visual world: the 'blank screen paradigm'. Cognition, 93(2), B79-B87. [2] Ferreira, F., Apel, J., & Henderson, J. M. (2008). Taking a new look at looking at nothing. Trends Cogn Sci, 12(11), 405-410.

Abstract

Sentence formulation requires mapping pre-verbal messages onto linguistic structures. This message-to-language
mapping is often evaluated in eye-tracking tasks where speakers describe pictured events (The dog chased the mailman).
Speakers can begin sentence formulation by quickly selecting the first-fixated character as the sentential starting point
(lexical incrementality), or generating a rudimentary sentence plan based on their construal of the event gist before
selecting a starting point (hierarchical incrementality; Kuchinsky & Bock, 2010). Lexical incrementality predicts fast
divergence of fixations while hierarchical incrementality predicts slower divergence of fixations to the two characters
within 200ms of picture onset.

Abstract

Much evidence demonstrates cerebellar involvement in language [1] but a theoretical framework about its precise role is lacking. In cerebellar motor control an influential model ascribes the cerebellum a predictive role [2]. It has been argued that cerebellar nonmotor regions perform similar computations as motor regions, and both are involved in online prediction [2]. We test this hypothesis by administering repetitive transcranial magnetic stimulation (rTMS) to the right cerebellum, a region implicated in language [3] during a predictive language task. Methods Visual World task [4]: Participants' eye movements were recorded while they listened to sentences and looked at a computer display of an agent and 4 objects, one of which (the target) was mentioned in the sentence. In the Prediction condition the object could be predicted on the basis of the verb; on Control trials it could not. We hypothesised that rTMS to the right cerebellum should make target fixation slower in the Prediction condition, but not in the Control condition. TMS protocol: TMS was delivered between two task blocks. In the cerebellar rTMS group (n = 22) the stimulation site was 1cm down and 3cm right of the inion. Participants received 10min of 1Hz rTMS. In addition, we tested two control groups. In the vertex rTMS group (n = 21), rTMS was applied at the same intensity, duration and frequency as in the cerebellar rTMS group, but over the vertex. In the no stimulation group (n = 22) the coil was placed over the cerebellar stimulation site but no pulses were delivered. Results As hypothesised, participants in the cerebellar rTMS group took longer to fixate the target after TMS in the Prediction condition but not in the Control condition (Block-by-Condition interaction: F(1,21) = 8.848, p = 0.007). This interaction was not found in either the vertex rTMS group (F(1,20) = 0.064, p = 0.802) or the no stimulation group (F(1,21) = 2.461, p = 0.132). Conclusions Here, we show that rTMS to the right cerebellum selectively affects linguistic prediction. These results provide additional evidence that the cerebellum plays a role in language and support theoretical accounts that the cerebellum contributes to nonmotor functions, as it does to motor functions, by online prediction. 1. Strick et al (2009). Cerebellum and nonmotor function. Annu Rev Neurosci, 32, 413-134 2. Miall et al (1993). Is the cerebellum a Smith predictor. J Mot Behav, 25, 203-216 3. Marien et al (2001). The lateralised linguistic cerebellum: a review and a new hypothesis. Brain and Language, 79, 580-600 4. Altmann & Kamide (1999). Incremental interpretation at verbs. Cognition, 73, 247-264

Abstract

In sentence comprehension, readers and listeners often anticipate upcoming information (e.g., Altmann & Kamide, 1999). We investigated two aspects of this process, namely 1) what is pre-activated when anticipating an upcoming word (the contents of predictions), and 2) which cognitive mechanisms are involved. The contents of predictions at the level of meaning could be restricted to functional semantic attributes (e.g., edibility; Altmann & Kamide, 1999). However, when words are processed other types of information can also be activated, such as object shape representations. It is unknown whether this type of information is already activated when upcoming words are predicted. Forty-five adult participants listened to predictable words in sentence contexts (e.g., "In 1969 Neil Armstrong was the first man to set foot on the moon.") while looking at visual displays of four objects. Their eye movements were recorded. There were three conditions: target present (e.g., a moon and three distractor objects that were unrelated to the predictable word in terms of semantics, shape, and phonology), shape competitor (e.g., a tomato and three unrelated distractors), and distractors only (e.g., rice and three other unrelated objects). Across lists, the same pictures and sentences were used in the different conditions. We found that participants already showed a significant bias for the target object (moon) over unrelated distractors several seconds before the target was mentioned, demonstrating that they were predicting. Importantly, there was also a smaller but significant shape competitor (tomato) preference starting at about a second before critical word onset, consistent with predictions involving the referent’s shape. The mechanisms of predictions could be specific to language tasks, or language could use processing principles that are also used in other domains of cognition. We investigated whether performance in non-linguistic prediction is related to prediction in language processing, taking an individual differences approach. In addition to the language processing task, the participants performed a simple cueing task (after Posner, Nissen, & Ogden, 1978). They pressed one of two buttons (left/right) to indicate the location of an X symbol on the screen. On half of the trials, the X was preceded by a neutral cue (+). On the other half, an arrow cue pointing left (<) or right (>) indicated the upcoming X's location with 80% validity (i.e., the arrow cue was correct 80% of the time). The SOA between cue and target was 500 ms. Prediction was quantified as the mean response latency difference between the neutral and valid condition. This measure correlated positively with individual participants' anticipatory target and shape competitor preference (r = .27; r = .45), and was a significant predictor of anticipatory looks in linear mixed-effects regression models of the data. Participants who showed more facilitation from the arrow cues predicted to a higher degree in the linguistic task. This suggests that prediction in language processing may use mechanisms that are also used in other domains of cognition. References Altmann, G. T. M., & Kamide, Y. (1999). Incremental interpretation at verbs: Restricting the domain of subsequent reference. Cognition, 73(3), 247-264. Posner, M. I., Nissen, M. J., & Ogden, W. C. (1978). Attended and unattended processing modes: The role of set for spatial location. In: H.L. Pick, & I.J. Saltzman (Eds.), Modes of perceiving and processing information. Hillsdale, N.J.: Lawrence Erlbaum Associates.