A benchmark of incremental model transformation tools based on an industrial case study with AADL

Incremental model transformation (IMT) tools have been proposed to improve performances of model transformations by updating only the parts of a model that need to be changed when another model on which it depends has been changed. Yet, the question is how these tools are suitable for modeling large and complex systems with rich modeling languages as used in industry. In this paper, we report the results of a benchmark of the most mature IMT tools. Particularly, we benchmark MoTE, eMoflon, VIATRA and YAMTL to evaluate their usability, maintainability and runtime performances. Our benchmark is based on a model transformation of an industrial case study using the standard architecture description language AADL. We propose a reusable evaluation framework, available for tool developers and users. Besides the capability to process large models, our benchmark also assesses IMT tool performances according to the different kinds of complex structures that typically exist in models of rich languages, as well as the complexity of the transformation specifications. Our results show the promising potential of some tools to specify sophisticated specifications and transform large models with good performance, but their use still requires the help of their developers, especially to solve serious problems with their runtime performance.