Project Awarded: $26,500
This project aims to investigate the nature of representations in human early visual cortex. Classic neuroscientific models posit that early visual cortex represents mainly low-level visual features from retinal input. Philosophical accounts by Jesse Prinz and Peter Carruthers posit that consciousness, because it depends on activation of early sensory areas, always has a sensory format (i.e., is a percept or mental image) and is never purely semantic (i.e., a thought). Consequently, there are no conscious thoughts. We aim to challenge both neuroscientific and philosophical accounts by demonstrating that early visual areas can in fact represent semantic information, as a result of feedback from higher areas. We will achieve this by decoding auditory information of several semantic categories from fMRI activity patterns of early visual cortex in the absence of visual stimulation. We will identify which semantic sound categories are represented in early visual cortex and to which level of abstraction.
We propose to examine the semantic/sensory distinction via the abstract/concrete distinction. For example, a semantic representation of a human figure covers a vast amount of possible shapes, orientations, locations, distances, colors, etc., making it abstract. A visualization/perception of a human figure is much more concrete, with a more specific shape, orientation, location, distance, color, etc. we design our experiment in light of this distinction, such that if the decoder performs successful classifications above chance, then it is relying on abstract (hence semantic) information and not on concrete (hence sensory) information.
Demonstrating the existence of semantic representations in early visual cortex, resulting from top-down effects, will challenge the influential neuroscientific and philosophical views described above. Additionally, our results will support a novel account of feedforward vs. feedback effects on early visual cortex. According to it, whereas feedforward information coming from external stimuli or mental imagery is (roughly) concrete, feedback information stemming from high-level areas is at least sometimes abstract. In ordinary visual perception and visualization, both feedback-abstract and feedforward-concrete representations exist in early visual cortex. However, in cases where only sounds are heard, sometimes only the feedback-abstract information is present in early visual cortex.
In sum, the expected empirical results and their philosophical analysis will make room for semantic representations in early sensory areas and for thoughts in consciousness.
Petra Vetter, PhD. Professor, Department of Psychology, Royal Holloway
Assaf Weksler, PhD. Postdoctoral researcher, Department of Philosophy and Psychology, University of Haifa