Responses in certain layers in Deep Convolutional Neural Networks (DCNNs) show striking similarities to responses in certain visual brain regions (Yamins et al., PNAS, 2013;
Yamins et al., NeurIPS, 2014)
These visual brain regions exhibit topographical preferences (Levy et al., Nature Neuroscience, 2001)
Do layers in DCNNs also show topographical preferences?
We measured the responses from human brains and a DCNN after "viewing" a stimuli set of 156 natural images:
Brain Data: Subjects (n=15) viewed the 156 images many times in an fMRI scanner. We generated a Representational Dissimilarity Matrix (RDM)
for EVC, Fusiform, IT, and PHC by computing the pairwise distance between the brain activations from each pair of images.
Model Data: Hybrid-CNN (Zhou et al., NeurIPS, 2014) is an AlexNet with 5 convolutional layers
and 3 fully connected layers trained on both object and scene recognition tasks. We generated an RDM for each unit (each unit corresponds to a spatial
location in the image) within each of the 5 convolutional layers by computing the pairwise distance between the model
activations from each pair of the 156 images (156 images were not used in model training).
We then correlate (Spearman) the RDMs from the brain regions with the RDMs from the model units using RSA
(Kriegeskorte, Frontiers in Systems Neuroscience, 2008) to produce a topographical correlation map. This allows us to quantify and visualize the
topographical correspondence between brain regions and DCNNs.
Results are presented in two steps:
We replicated many previous works showing a correspondence between convolutional network layers and brain regions (Motivation point #1). Specifically in our data,
EVC: Significant in layers 1 and 2
Fusiform: Significant in layers 2, 3, 4, and 5
IT: Significant in layers 4 and 5
PHC: Significant in layers 2, 3, 4, and 5
We analyzed whether each brain regions corresponding significant network layers showed the same center/periphery bias as the brain region (See figure on left).
EVC/Layers 1-2: Shows a randomly distributed topographical preferences
Fusiform/Layers 2-5: Shows strong center bias
IT/Layers 4-5: Shows strong center bias
PHC/Layers 2-5: Shows clear periphery bias in layer 2 with a transition to a distributed pattern
Our results revealed foveally biased fusiform and IT highly correlated with unit activations of the network with the center selective
visual feld and peripherally biased PHC strongly correlated with unit activations of the network with periphery
selective receptive felds. We demonstrated for the frst time a topographical correspondence (central/periphery biases) between
ventral brain regions and unit activations of the Hybrid-CNN.
These findings support two main hypotheses in vision neuroscience:
The human visual pathway optimizes the cost function of visual recognition
The characteristics of neural tuning, internal neural representations and brain area functions along this pathway are most likely the
result of this cost function optimization (Marbelstone et al., Frontiers in Computational Neuroscience, 2016)
Explore whether the topographical biases found in the brain and model are a result of the statistics of the image or learned weights.
Use DCNNs to investigate the underlying computational principles involved in the brain's hierarchical and topographical organization.
Citation and Links
Mohsenzadeh, Y., Mullin, C., Lahner, B. et al. Emergence
of Visual Center-Periphery Spatial Organization
in Deep Convolutional Neural Networks. Sci Rep10, 4638 (2020). https://doi.org/10.1038/s41598-020-61409-0