Header

UZH-Logo

Maintenance Infos

The interplay of language and visual perception in working memory


Souza, Alessandra S; Skóra, Zuzanna (2017). The interplay of language and visual perception in working memory. Cognition, 166:277-297.

Abstract

How do perception and language interact to form the representations that guide our thoughts and actions over the short-term? Here, we provide a first examination of this question by investigating the role of verbal labels in a continuous visual working memory (WM) task. Across four experiments, participants retained in memory the continuous color of a set of dots which were presented sequentially (Experiments 1-3) or simultaneously (Experiment 4). At test, they reproduced the colors of all dots using a color wheel. During stimulus presentation participants were required to either label the colors (color labeling) or to repeat "bababa" aloud (articulatory suppression), hence prompting or preventing verbal labeling, respectively. We tested four competing hypotheses of the labeling effect: (1) labeling generates a verbal representation that overshadows the visual representation; (2) labeling yields a verbal representation in addition to the visual one; (3) the labels function as a retrieval cue, adding distinctiveness to items in memory; and (4) labels activate visual categorical representations in long-term memory. Collectively, our experiments show that labeling does not overshadow the visual input; it augments it. Mixture modeling showed that labeling increased the quantity and quality of information in WM. Our findings are consistent with the hypothesis that labeling activates visual long-term categorical representations which help in reducing the noise in the internal representations of the visual stimuli in WM.

Abstract

How do perception and language interact to form the representations that guide our thoughts and actions over the short-term? Here, we provide a first examination of this question by investigating the role of verbal labels in a continuous visual working memory (WM) task. Across four experiments, participants retained in memory the continuous color of a set of dots which were presented sequentially (Experiments 1-3) or simultaneously (Experiment 4). At test, they reproduced the colors of all dots using a color wheel. During stimulus presentation participants were required to either label the colors (color labeling) or to repeat "bababa" aloud (articulatory suppression), hence prompting or preventing verbal labeling, respectively. We tested four competing hypotheses of the labeling effect: (1) labeling generates a verbal representation that overshadows the visual representation; (2) labeling yields a verbal representation in addition to the visual one; (3) the labels function as a retrieval cue, adding distinctiveness to items in memory; and (4) labels activate visual categorical representations in long-term memory. Collectively, our experiments show that labeling does not overshadow the visual input; it augments it. Mixture modeling showed that labeling increased the quantity and quality of information in WM. Our findings are consistent with the hypothesis that labeling activates visual long-term categorical representations which help in reducing the noise in the internal representations of the visual stimuli in WM.

Statistics

Citations

Dimensions.ai Metrics
43 citations in Web of Science®
45 citations in Scopus®
Google Scholar™

Altmetrics

Downloads

409 downloads since deposited on 08 Feb 2018
108 downloads since 12 months
Detailed statistics

Additional indexing

Item Type:Journal Article, refereed, original work
Communities & Collections:06 Faculty of Arts > Institute of Psychology
Dewey Decimal Classification:150 Psychology
Scopus Subject Areas:Social Sciences & Humanities > Experimental and Cognitive Psychology
Social Sciences & Humanities > Language and Linguistics
Social Sciences & Humanities > Developmental and Educational Psychology
Social Sciences & Humanities > Linguistics and Language
Life Sciences > Cognitive Neuroscience
Uncontrolled Keywords:Linguistics and Language, Experimental and Cognitive Psychology, Cognitive Neuroscience, Developmental and Educational Psychology, Language and Linguistics
Language:English
Date:2017
Deposited On:08 Feb 2018 14:07
Last Modified:26 Jan 2022 15:58
Publisher:Elsevier
ISSN:0010-0277
OA Status:Green
Publisher DOI:https://doi.org/10.1016/j.cognition.2017.05.038
PubMed ID:28595141
  • Content: Accepted Version