Welcome to Hui Wu's website

Research Scientist, IBM Research AI

Pooling with Stochastic Spatial Sampling

Pooling with Stochastic Spatial Sampling

Feature pooling layers (e.g., max pooling) in convolutional
neural networks (CNNs) serve the dual purpose
of providing increasingly abstract representations as well
as yielding computational savings in subsequent convolutional
layers. We observe that this regularly spaced
downsampling arising from non-overlapping windows,
although intuitive from a signal processing perspective
(which has the goal of signal reconstruction), is not necessarily
optimal for learning (where the goal is to generalize).
We study this aspect and propose a novel pooling
strategy with stochastic spatial sampling (S3Pool), where
the regular downsampling is replaced by a more general
stochastic version. We observe that this general stochasticity
acts as a strong regularizer, and can also be seen as
doing implicit data augmentation by introducing distortions in the feature maps.