A Generic White-Box Testing Framework for Model Transformations

Theproject TETRABox(A Generic White-Box TEsting Framework for Model TRAnsformations) resides in the context of Model-Driven Engineering (MDE), which proposes the use of models to conduct software development on a higher level of abstraction. These models are intended to be transformed into code by dedicated transformations, thereby achieving quality and productivity gains. Given the prominent role of transformations in MDE and their increasing use in safety critical areas such as the aviation industry, proper means for testing the correctness of model transformations are inevitable. Although first testing frameworks have been proposed, they fall short with respect to test source model generation and fault localization, and they are hardly configurable and tightly coupled to a certain transformation language. Second, apart of these frameworks, first isolated approaches for the phase of test source model generation have been proposed, which, however, rely mostly on black-box testing techniques, thus, incorporating the source metamodels and the requirements, but neglect the transformation definition, which may lead to untested parts of the transformation definition. Finally, means for fault localization are missing, since testing approaches identify the failing of a test case, but miss to provide the failing parts of the transformation definition. For tackling these limitations, the aim of this project is to establish a comprehensive testing framework for model transformations called TETRABox, whereby we base on the experiences gained in our previous FWF-funded project TROPIC. TETRABox supports all testing phases, ranging from test source model generation to fault localization especially focusing on configurable components. To keep the framework broadly applicable, the envisioned components for testing are independent of a transformation language, allowing new languages to be incorporated by providing a transformation to the common formalism of a control flow graph. Second, to leverage white-box testing, TETRABox allows the automatic generation of test source models on basis of the transformation definition by means of symbolic execution. Finally, for fault localization, oracles offering a dedicated failure trace are employed, which are used to provide an entry point for debugging by slicing techniques. The methodology for evaluating the TETRABox framework builds on three major pillars. First, transformations of the ATL model transformation zoo will be systematically tested by means of mutation testing and the results will be compared to existing testing techniques. Second, an empirical study with students from our model engineering courses (around 200 master students every year) will be conducted, whereby errors will be seeded into existing transformations, and the students will have to spot these errors with and without the help of the TETRABox framework. Finally, dedicated workshops will be held with (inter-)national partners. Consequently, the realization of the project TETRABox would represent a major cornerstone in letting the vision of Model-Driven Engineering coming true.

Model transformations are crucial for the success of Model-Driven Engineering (MDE) which aims at specifying software systems at a higher abstraction level using a network of models before the systems are realized for specific platforms using code generation techniques. Model transformations allow to manipulate existing as well as to generate new models by including or removing (un)necessary information for given purposes (e.g., assessing performance or safety aspects). Given their prominent role in MDE and their increasing use in safety critical areas such as production systems, automotive, or smart cities, proper means for testing the correctness of model transformations are inevitable. Although first testing approaches have been proposed in the past, they fall short in crucial phases such as the selection of the most important test models as well as to locate the faults in the transformation which are revealed by the test cases. In addition, existing approaches are tightly coupled with specific model transformation languages, neglect information about the inner structure of transformations, and finally, do not consider information about the progress of the testing process which is important for regression testing. To tackle these limitations, we worked on a comprehensive testing framework for model transformations called TETRABox (A Generic White-Box TEsting Framework for Model TRAnsformations). TETRABox supports all testing phases ranging from test case selection to fault localization considering the inner structure of transformations as well as the progress of the testing process. To keep TETRABox broadly applicable, we aimed for reusable concepts which are independent of specific transformation languages, but at the same time, we instantiated these concepts for particular model transformation languages such as the ATLAS Transformation Language (ATL) - one of the most used transformation languages. Furthermore, we have developed general testing and debugging support for executable modelling languages (model transformation languages can be considered as a specific kind of executable modelling languages) in the open-source GEMOC language workbench, which supports us in the specification of existing and future model transformation languages offering out-of-the-box testing and debugging support.