Reviving medieval byzantine seals: a synthetic-to-real approach to character recognition

Training AI models to recognize characters engraved on medieval seals represents a challenging task in document analysis and cultural heritage preservation. Seal images often exhibit significant variations in style, material, state of conservation, and illumination, requiring large training sets that span a wide range of combinations. However, since collecting and annotating large sets of seal images is labor-intensive and time-consuming, synthetic training images are needed. In this work, we develop a pipeline to generate synthetic three-dimensional (i.e., computer-generated) seal images with random combinations of attributes. Our pipeline allows us to synthesize a large number of realistic, annotated, seal images under controlled combinations of attributes. We train different AI models for character classification, localization, and detection, enriching a small set of real seal or character images with a large number of synthetic ones either through fine-tuning or mixed training. Across all tasks and models, our experiments show that supplementing real samples with synthetic images boosts performance compared to training on real samples alone by margins topping 10% in some tasks.