Cross-modal integration during vowel identification in audiovisual speech: a functional magnetic resonance imaging study.

Full Abstract

To investigate the neural substrates of the perception of audiovisual speech, we conducted a functional magnetic resonance imaging study with 28 normal volunteers. We hypothesized that the constraint provided by visually-presented articulatory speech (mouth movements) would lessen the workload for speech identification if the two were concordant, but would increase the workload if the two were discordant. In auditory attention sessions, subjects were required to identify vowels based on auditory speech. Auditory vowel stimuli were presented with concordant or discordant visible articulation movements, unrelated lip movements, and without visual input. In visual attention sessions, subjects were required to identify vowels based on the visually-presented vowel articulation movements. The movements were presented with concordant or discordant uttered vowels and noise, and without sound. Irrespective of the attended modality, concordant conditions significantly shortened the reaction time, whereas discordant conditions lengthened the reaction time. Within the neural substrates that were commonly activated by auditory and visual tasks, the mid superior temporal sulcus showed greater activity for discordant stimuli than concordant stimuli. These findings suggest that the mid superior temporal sulcus plays an important role in the auditory-visual integration process underlying vowel identification.

Author information

Author/s: Murase, Mika (M); Saito, Daisuke N (DN); Kochiyama, Takanori (T); Tanabe, Hiroki C (HC); Tanaka, Satoshi (S); Harada, Tokiko (T); Aramaki, Yu (Y); Honda, Manabu (M); Sadato, Norihiro (N);

Affiliation: Department of Physiological Sciences, The Graduate University for Advanced Studies (Sokendai), Kanagawa, Japan.

Journal and publication information

Publication Type: Journal Article; Research Support, Non-U.S. Gov't

Journal: Neuroscience letters (Neurosci Lett), published in Ireland. (Language: eng)

Reference: 2008-Mar; vol 434 (issue 1) : pp 71-6

Dates: Created 2008/03/17; Completed 2008/07/03;

PMID: 18280656, status: MEDLINE (last retrieval date: 2/18/2009, IMS Date: )

Sourced from the National Library of Medicine. Abstract text and other information may be subject to copyright.

External Links for this article
(including full text providers, if available):

Click Electronic Full-text Provider Links to see options for finding the electronic full text links to this article. Note there may be a subscription or fee required for access to the full text. See our FAQ for information on finding FREE full text articles.

This article may also be located in paper journal collections available in many libraries. Use the Journal and Publication Information above to find the full article.