Research article summary (published 29 Jun 2008):

Seeing and hearing meaning: ERP and fMRI evidence of word versus picture integration into a sentence context.

Full Abstract

Understanding language always occurs within a situational context and, therefore, often implies combining streams of information from different domains and modalities. One such combination is that of spoken language and visual information, which are perceived together in a variety of ways during everyday communication. Here we investigate whether and how words and pictures differ in terms of their neural correlates when they are integrated into a previously built-up sentence context. This is assessed in two experiments looking at the time course (measuring event-related potentials, ERPs) and the locus (using functional magnetic resonance imaging, fMRI) of this integration process. We manipulated the ease of semantic integration of word and/or picture to a previous sentence context to increase the semantic load of processing. In the ERP study, an increased semantic load led to an N400 effect which was similar for pictures and words in terms of latency and amplitude. In the fMRI study, we found overlapping activations to both picture and word integration in the left inferior frontal cortex. Specific activations for the integration of a word were observed in the left superior temporal cortex. We conclude that despite obvious differences in representational format, semantic information coming from pictures and words is integrated into a sentence context in similar ways in the brain. This study adds to the growing insight that the language system incorporates (semantic) information coming from linguistic and extralinguistic domains with the same neural time course and by recruitment of overlapping brain areas.

Author information

Author/s: Willems, Roel M (RM); Ozyürek, Asli (A); Hagoort, Peter (P);

Affiliation: FC Donders Centre for Cognitive Neuroimaging, Radboud University Nijmegen, Nijmegen, The Netherlands. roel.willems(-atsign-)fcdonders.ru.nl

Journal and publication information

Publication Type: Journal Article; Research Support, Non-U.S. Gov't

Journal: Journal of cognitive neuroscience (J Cogn Neurosci), published in United States. (Language: eng)

Reference: 2008-Jul; vol 20 (issue 7) : pp 1235-49

Dates: Created 2008/06/19; Completed 2008/09/24; Revised 2008/11/21;

PMID: 18284352, status: MEDLINE (last retrieval date: 2/18/2009, IMS Date: )

Sourced from the National Library of Medicine. Abstract text and other information may be subject to copyright.

External Links for this article
(including full text providers, if available):

Click Electronic Full-text Provider Links to see options for finding the electronic full text links to this article. Note there may be a subscription or fee required for access to the full text. See our FAQ for information on finding FREE full text articles.

This article may also be located in paper journal collections available in many libraries. Use the Journal and Publication Information above to find the full article.

MeSH headings (categories)

This article was linked to the MESH Headings shown below.

Adolescent Adult Analysis of Variance Brain - blood supply; physiology Brain Mapping Computer-Assisted Instruction - methods Electroencephalography - methods Female Functional Laterality - physiology Hearing - physiology

Hearing - physiology Humans Magnetic Resonance Imaging - methods Male Mental Processes Oxygen - blood Photic Stimulation - methods Semantics Time Factors Vision, Ocular - physiology

Associated Chemicals: Oxygen (7782-44-7)

Related articles

These are the highest related articles currently in the database:

• Visual cortex activation in late-onset, Braille naive blind individuals: an fMRI study during semantic and phonological tasks with heard words. 26 Sep 2005
• Supramodal language comprehension: role of the left temporal lobe for listening and reading. 18 Feb 2007
• The functional significance of mu rhythms: translating "seeing" and "hearing" into "doing". 29 May 2005
• [Electroencephalographic studies on the relationship between the visual and auditory analyzers in patients with vascular diseases of the brain.] 30 Dec 1960
• Visual memory for shapes in deaf signers and nonsigners and in hearing signers and nonsigners: atypical lateralization and enhancement. 30 Dec 2006
• Attention to human speakers in a virtual auditory environment: brain potential evidence. 2 Mar 2008
• Chewing side preference in man correlated with handedness, footedness, eyedness and earedness. 30 Jul 1987
• [Human functional asymmetry in operator activity] 30 Aug 1980
• Central auditory processing of speech in cochlear implant users demonstrated by positron emission tomography. 30 Dec 1996
• High binaural coherence determines successful sound localization and increased activity in posterior auditory areas. 13 Sep 2005

See 100+ related articles.

Related Article Map

Legend: - FREE Full text Article. - Abstract only. - Title only. More help.

See a large map of 100+ related articles.