Tied spatial transformer networks for digit recognition

This paper reports a new approach based on convolutional neural networks (CNNs), which uses spatial transformer networks (STNs). The approach, referred to as Tied Spatial Transformer Networks (TSTNs), consists of training a system which combines a localization CNN and a classification CNN whose weights are shared. The localization CNN is used for predicting an affine transform for the input image, which is then processed according to the predicted parameters and passed through the classification CNN. We have conducted initial experiments on the cluttered MNIST dataset of noisy digits, comparing the TSTN and STN with identical configurations of trainable parameters, but untied, as well as the classification CNN only, applied to the unprocessed images. In all these cases, we obtain better results using the TSTN. We conjecture that the TSTN provides a regularization effect, as compared to untied STNs. Further experiments seem to support this hypothesis.