Sign language indexation within the MPEG-7 framework

In this paper, we address the issue of sign language indexation/recognition. The existing tools, like on-line Web dictionaries or other educational-oriented applications, are making exclusive use of textual annotations. However, keyword indexing schemes have strong limitations due to the ambiguity of the natural language and to the huge effort needed to manually annotate a large amount of data. In order to overcome these drawbacks, we tackle sign language indexation issue within the MPEG-7 framework and propose an approach based on linguistic properties and characteristics of sign language. The method developed introduces the concept of over time stable hand configuration instantiated on natural or synthetic prototypes. The prototypes are indexed by means of a shape descriptor which is defined as a translation, rotation and scale invariant Hough transform. A very compact representation is available by considering the Fourier transform of the Hough coefficients. Such an approach has been applied to two data sets consisting of 'Letters' and 'Words' respectively. The accuracy and robustness of the results are discussed and a complete sign language description schema is proposed.