feature map sampling
Our proposed method need to use real data as inputs, but it is a bit unclear what data is used.
Formally, to prune a feature map with c channels, we consider applying n×c×k×k convolutional filters W on N×c×k×k input volumes X sampled from this feature map, which produces N×n output matrix Y. Here, N is the number of samples, n is the number of output channels, and kxk are the kernel size.
In the above formulation, X, Y are volumes, so they must be corresponding to each other on feature maps. Illustrated in the following figure, X is the pink patch on the left, Y is the blue patch on the right.
For channel pruning, we found that it is enough to extract 5000 images, and 10 samples per image. Optimally, every patch on a feature map should be used. However we found that it's too time consuming, and 10 samples per image is already enough.