The Learnable Typewriter: A Generative Approach to Text Analysis

We present a generative document-specific approach to character analysis and recognition in text lines. Our main idea is to build on unsupervised multi-object segmentation methods and in particular those that reconstruct images based on a limited amount of visual elements, called sprites. Taking as input a set of text lines with similar font or handwriting, our approach can learn a large number of different characters and leverage line-level annotations when available. Our contribution is twofold. First, we provide the first adaptation and evaluation of a deep unsupervised multi-object segmentation approach for text line analysis. Since these methods have mainly been evaluated on synthetic data in a completely unsupervised setting, demonstrating that they can be adapted and quantitatively evaluated on real images of text and that they can be trained using weak supervision are significant progresses. Second, we show the potential of our method for new applications, more specifically in the field of palaeography, which studies the history and variations of handwriting, and for cipher analysis. We demonstrate our approach on four very different datasets: a printed volume of the Google1000 dataset [19, 48], the Copiale cipher [2, 27], a large scale multi-font benchmark [41], and historical handwritten charters from the 12th and early 13th century [6].