Designing GUIs by sketch drawing and visual programming

This paper presents a new interactive UIDE that is based on visual programming and constrained sketch drawing. At the early stages of the iterative conception process, GUIs are interactively designed by drawing a "rough sketch" that acts as a first draft of the final description. This drawing is interpreted in real time by the system in order to produce the corresponding widget view (i.e. the actual visible GUI) and a graph of abstract objects that represents the GUI structure. This graph can then be easily modified by mixing visual and textual programming in a fully iterative and incremental way. This system is also based on the use of generic objects which are dynamically instantiated into actual widgets according to their structural and functional context. This scheme makes it possible to define very generic GUIs that can then be deeply refined in an efficient way at any stage of the conception process.