Published eLetters

Guidelines

As a forum for professional feedback, submissions of letters are open to all. You do not need to be a subscriber. To avoid redundancy, we urge you to read other people's letters before submitting your own. Name, current appointment, place of work, and email address are required to send a letter, and will be published with your review. We also require that you declare any competing financial interests. Unprofessional submissions will not be considered or responded to.

We thank Shahab Bakhtiari for the excellent commentary on our paper. The Journal Club Article provides a detailed background regarding specificity and the debate on the locus of learning; it also suggests interesting further research directions of a non-strictly hierarchical model with readouts from different layers. While the main idea of our paper was well summarized in this article, we would like to respond to issues raised in the Article.
First, it was pointed out that the DNN model “attributed the behavioral improvement by VPL to retuning of sensory neurons”, while it is possible that “when the most informative neurons are not contributing the most in the decision, performance improvement by VPL mainly involves optimizing the readout from these neurons based on the training task”. In the paper, we did not made assumptions about where learning should occur, and we agree that VPL likely involves plasticity in more than one visual area. In fact, we have suggested that PL is a distributed process across the whole brain (Maniglia & Seitz, 2017), and that the advantage of the DNN is that allows us to model this distribution of learning across the model’s layers. In our main experiment, all the weights in the DNN model were plastic, including the readout. In the “freezing layer” experiment, the behavioral improvement could occur just by learning the readout. While our comparisons with physiological data were mainly focused on representation learning in early visual...

We thank Shahab Bakhtiari for the excellent commentary on our paper. The Journal Club Article provides a detailed background regarding specificity and the debate on the locus of learning; it also suggests interesting further research directions of a non-strictly hierarchical model with readouts from different layers. While the main idea of our paper was well summarized in this article, we would like to respond to issues raised in the Article.
First, it was pointed out that the DNN model “attributed the behavioral improvement by VPL to retuning of sensory neurons”, while it is possible that “when the most informative neurons are not contributing the most in the decision, performance improvement by VPL mainly involves optimizing the readout from these neurons based on the training task”. In the paper, we did not made assumptions about where learning should occur, and we agree that VPL likely involves plasticity in more than one visual area. In fact, we have suggested that PL is a distributed process across the whole brain (Maniglia & Seitz, 2017), and that the advantage of the DNN is that allows us to model this distribution of learning across the model’s layers. In our main experiment, all the weights in the DNN model were plastic, including the readout. In the “freezing layer” experiment, the behavioral improvement could occur just by learning the readout. While our comparisons with physiological data were mainly focused on representation learning in early visual areas, similar comparisons could be made between the changes in the readout weights of the DNN model and changes in decision areas of the brain. However, as in many other VPL models, the decision process was assumed to occur in a single layer with the most parsimonious number of units (1 in our case) reporting the decision, which makes it difficult to draw comparisons with corresponding neural population in the brain.
Second, as very briefly mentioned in our paper, a future research direction is to add connections between intermediate layers to the decision neuron. Here, we note the challenges in carrying out these simulations. The main issue is how to initialize such a network with skip-layer connections while maintaining biological similarities that the original AlexNet exhibits, which is a key premise for the current DNN model. Simply adding skip layer weights with predefined values on the current DNN model may introduce bias. We propose that such experiment should require retraining AlexNet, or other alternatives, on natural images with added skip-layer connections , and proceed with VPL experiments if the similarities with the brain found in previous research still hold in this new network.
Finally, we note that, with the fast development of modern DNN tools, unavailable at the time of our study, the opportunities for improvements on the simulations presented in our paper are abundant. We are excited to see the extent to which even greater insight into mechanisms of perceptual learning can be achieved through model comparison of a broader range of DNN model variants.

We thank Shahab Bakhtiari for the excellent commentary on our paper. The Journal Club Article provides a detailed background regarding specificity and the debate on the locus of learning; it also suggests interesting further research directions of a non-strictly hierarchical model with readouts from different layers. While the main idea of our paper was well summarized in this article, we would like to respond to issues raised in the Article.
First, it was pointed out that the DNN model “attributed the behavioral improvement by VPL to retuning of sensory neurons”, while it is possible that “when the most informative neurons are not contributing the most in the decision, performance improvement by VPL mainly involves optimizing the readout from these neurons based on the training task”. In the paper, we did not made assumptions about where learning should occur, and we agree that VPL likely involves plasticity in more than one visual area. In fact, we have suggested that PL is a distributed process across the whole brain (Maniglia & Seitz, 2017), and that the advantage of the DNN is that allows us to model this distribution of learning across the model’s layers. In our main experiment, all the weights in the DNN model were plastic, including the readout. In the “freezing layer” experiment, the behavioral improvement could occur just by learning the readout. While our comparisons with physiological data were mainly focused on representation learning in early visual...

We thank Shahab Bakhtiari for the excellent commentary on our paper. The Journal Club Article provides a detailed background regarding specificity and the debate on the locus of learning; it also suggests interesting further research directions of a non-strictly hierarchical model with readouts from different layers. While the main idea of our paper was well summarized in this article, we would like to respond to issues raised in the Article.
First, it was pointed out that the DNN model “attributed the behavioral improvement by VPL to retuning of sensory neurons”, while it is possible that “when the most informative neurons are not contributing the most in the decision, performance improvement by VPL mainly involves optimizing the readout from these neurons based on the training task”. In the paper, we did not made assumptions about where learning should occur, and we agree that VPL likely involves plasticity in more than one visual area. In fact, we have suggested that PL is a distributed process across the whole brain (Maniglia & Seitz, 2017), and that the advantage of the DNN is that allows us to model this distribution of learning across the model’s layers. In our main experiment, all the weights in the DNN model were plastic, including the readout. In the “freezing layer” experiment, the behavioral improvement could occur just by learning the readout. While our comparisons with physiological data were mainly focused on representation learning in early visual areas, similar comparisons could be made between the changes in the readout weights of the DNN model and changes in decision areas of the brain. However, as in many other VPL models, the decision process was assumed to occur in a single layer with the most parsimonious number of units (1 in our case) reporting the decision, which makes it difficult to draw comparisons with corresponding neural population in the brain.
Second, as very briefly mentioned in our paper, a future research direction is to add connections between intermediate layers to the decision neuron. Here, we note the challenges in carrying out these simulations. The main issue is how to initialize such a network with skip-layer connections while maintaining biological similarities that the original AlexNet exhibits, which is a key premise for the current DNN model. Simply adding skip layer weights with predefined values on the current DNN model may introduce bias. We propose that such experiment should require retraining AlexNet, or other alternatives, on natural images with added skip-layer connections , and proceed with VPL experiments if the similarities with the brain found in previous research still hold in this new network.
Finally, we note that, with the fast development of modern DNN tools, unavailable at the time of our study, the opportunities for improvements on the simulations presented in our paper are abundant. We are excited to see the extent to which even greater insight into mechanisms of perceptual learning can be achieved through model comparison of a broader range of DNN model variants.