Decoupling Articulation from Language: Phonetic Interfaces for Romanian Visual Speech Recognition

Visual Speech Recognition (VSR) for low-resource languages poses unique challenges, as limited supervision makes direct visual-to-text learning unstable and highly datadependent. We present a structured architecture for Romanian VSR that introduces a phonetic interface between visual articulation modeling and linguistic decoding. The framework learns pronunciation-aware visual representations by leveraging synchronized audio signals during training, while operating in a visual-only setting at inference time. These representations are mapped into a language-agnostic phoneme space using a Connectionist Temporal Classification objective, forming an intermediate articulatory layer independent of orthographic conventions. A pretrained large language model subsequently reconstructs well-formed Romanian text from phoneme sequences through prompt-conditioned generation. We evaluate the approach on RoVSR, a 165-hour in-the-wild Romanian audiovisual dataset, and examine both zero-shot transfer and supervised adaptation scenarios. The results indicate that separating articulatory modeling from language reconstruction provides stable transfer to Romanian and consistent improvements under lightweight fine-tuning, supporting the use of structured phonetic intermediates for lowresource visual speech recognition.