Execution of UML models: a systematic review of research and practice

Software & Systems Modeling, Apr 2018

Several research efforts from different areas have focused on the execution of UML models, resulting in a diverse and complex scientific body of knowledge. With this work, we aim at identifying, classifying, and evaluating existing solutions for the execution of UML models. We conducted a systematic review in which we selected 63 research studies and 19 tools among over 5400 entries by applying a systematic search and selection process. We defined a classification framework for characterizing solutions for UML model execution, and we applied it to the 82 selected entries. Finally, we analyzed and discussed the obtained data. From the analyzed data, we drew the following conclusions: (i) There is a growing scientific interest on UML model execution; (ii) solutions providing translational execution clearly outnumber interpretive solutions; (iii) model-level debugging is supported in very few cases; (iv) only a few research studies provide evidence of industrial use, with very limited empirical evaluations; (v) the most common limitation deals with coverage of the UML language. Based on these observations, we discuss potential research challenges and implications for the future of UML model execution. Our results provide a concise overview of states of the art and practice for UML model execution intended for use by both researchers and practitioners.

A PDF file should load here. If you do not see its contents the file may be temporarily unavailable at the journal website or you do not have a PDF plug-in installed and enabled in your browser.

Alternatively, you can download the file locally and open with any standalone PDF reader:

https://link.springer.com/content/pdf/10.1007%2Fs10270-018-0675-4.pdf

Execution of UML models: a systematic review of research and practice

and the application of MDE and CBSE techniques to mobile multi-robot systems. He has (co-)authored over 60 publications in international journals Execution of UML models: a systematic review of research and practice Federico Ciccozzi 0 1 0 Malina Software Corporation , Ottawa , Canada 1 Vrije Universiteit Amsterdam , Amsterdam , The Netherlands Several research efforts from different areas have focused on the execution of UML models, resulting in a diverse and complex scientific body of knowledge. With this work, we aim at identifying, classifying, and evaluating existing solutions for the execution of UML models. We conducted a systematic review in which we selected 63 research studies and 19 tools among over 5400 entries by applying a systematic search and selection process. We defined a classification framework for characterizing solutions for UML model execution, and we applied it to the 82 selected entries. Finally, we analyzed and discussed the obtained data. From the analyzed data, we drew the following conclusions: (i) There is a growing scientific interest on UML model execution; (ii) solutions providing translational execution clearly outnumber interpretive solutions; (iii) model-level debugging is supported in very few cases; (iv) only a few research studies provide evidence of industrial use, with very limited empirical evaluations; (v) the most common limitation deals with coverage of the UML language. Based on these observations, we discuss potential research challenges and implications for the future of UML model execution. Our results provide a concise overview of states of the art and practice for UML model execution intended for use by both researchers and practitioners. 1 http://www.uml.org. UML; Model execution; Code generation; Model compilation; Model interpretation; Systematic review 1 Introduction Standardized by the Object Management Group (OMG) in 1997, the Unified Modeling Language (UML)1 has emerged and established itself as both a de facto and a de jure standard in industrial development of software systems [ 33 ]. This was due in part to its versatility, which enables its use as general-purpose language, and also to its ability to be customized through its profiling mechanisms [ 1 ] to directly Communicated by Dr. Jeff Gray. School of Innovation, Design and Engineering (IDT), Mälardalen University, Västerås, Sweden support concepts specific to individual domains, organizations, or projects. There is evidence that, in industrial practice, UML models have been used primarily for problem understanding (i.e., analysis) and documentation [ 24 ], despite the fact that a number of tools support executable variants of UML. These relied on custom semantics in combination with traditional thirdgeneration programming languages, such as C++ or Java, for specifying detailed action code. Consequently, they were not fully compliant with the UML standard, forcing users into a potentially precarious “vendor lock-in” predicament. However, the introduction of UML2 together with the definition of (i) a formal specification of the executable semantics for a subset of UML2, via the Foundational Subset For Executable UML Models (fUML)2 and (ii) a textual action language, the Action Language for Foundational UML (Alf)3, enabled compact and complete specification of complex behaviors including their algorithmic parts, such that the models based on these are fully executable provided that corresponding tools are available. 2 http://www.omg.org/spec/FUML. 3 http://www.omg.org/spec/ALF. As of today, a plethora of methods and techniques for executing UML models have been proposed (e.g., [ 26,31,47 ]). Each of these uses at least one of two possible execution strategies: translational or interpretive (see Sect. 2). However, they do this in idiosyncratic ways, so that each is applicable only in specific contexts, focusing on specific concerns and problem areas. Consequently, it is currently difficult to have a clear view of existing solutions for the execution of UML models and their characteristics. This makes it difficult for practitioners to decide whether and how to take advantage of executable UML models. The goal of this study is to identify, classify, and evaluate the states of the art and practice of solutions for executing models based on the UML family of languages (i.e., UML2, UML2 profiles, fUML, Alf). With this objective in mind, we performed a systematic investigation of trends, technical characteristics, available evidence, and limitations of current solutions for the execution of UML models in both research and industrial practice. The main contributions of this study are: • a classification framework for classifying, comparing, and evaluating solutions for executing UML models; • a systematic review of the states of the art and practice in executing UML models; • an exploration of emerging research challenges and implications for future research and practice of executing UML models. The intended audience of this study includes both (i) researchers willing to further contribute to this research area (e.g., by defining new solutions for executing UML models or improving existing ones) and (ii) practitioners who want to better understand existing and future solutions for executing UML models in order to select the one that best suits their needs. From a methodological perspective, in this study we perform a systematic review [ 27,51 ]. As part of our systematic process, we first selected 82 primary studies from over 5400 candidates for answering the research questions that we identified (see Sect. 3.1). Next, we defined a classification framework for characterizing solutions for executing UML models, which we applied to the 82 selected studies. Finally, we analyzed the obtained data to produce: (i) an overview of the states of the art and practice, and (ii) a list of emerging and future research challenges and implications. We expect that the results of these activities will (i) provide a concise overview of the states of the art and practice of UML model execution, and (ii) help researchers and practitioners in identifying the limitations of and gaps in current research as well as the potential and applicability of UML model execution in practice [28]. It is worth noting that, besides numerous systematic reviews and surveys on UML-based methods and techniques exist, we could not identify any systematic study on the execution of UML models. The only work dealing with a comparison of approaches for execution of UML, is represented by Gotti and Mbarki [ 20 ], where the authors only compare four approaches selected ad hoc. Consequently, to the best of our knowledge, this study represents the first systematic investigation of the states of the art and practice of the execution of UML models. Outline In Sect. 2 we review the requisite background that provides the context of our study. Section 3 describes in detail the choices we made in designing, conducting, and documenting the study. The results of the study are discussed in Sects. 4, 5, 6, 7, and 8. Section 9 focuses on the main findings and implications for future researchers and practitioners working on UML model execution. Section 10 provides a brief summary of the work and conclusions reached. 2 Background 2.1 From programming to MDE In the early 1950s, executables were written directly in machine code language. The need for simplification by abstracting out excessive detail about the underlying hardware led to the creation of numerous programming languages, many of which are still in use. These languages used higher-level abstractions to capture desired behaviors. Thus, programming progressed from first-generation machine code languages, to second-generation assembler languages, to what are known as third-generation or high-level programming languages (3GLs). Programs written using second- and third-generation programming languages were not directly executable, but needed to be transformed into a format (i.e., machine code) that the machine could execute. This could be done in one of two ways: • Interpretation the software program is interpreted by a language-specific virtual machine program executing on the target platform; • Translation the software program is translated into object code4 that could be executed directly on the target machine. However, increasing demands for ever more sophisticated software functionality has led to calls for more powerful 4 Object code is what a compiler produces. We use this term since it includes both machine code languages (e.g., Java bytecode) and intermediate languages (e.g., LLVM intermediate representation). and more flexible methods of programming computers. As before, this meant moving even further away from the concepts of the underlying computing machinery toward methods of expression that are closer to the problem domain—a trend that is often referred to as “raising the level of abstraction” of programming. The various developments grouped under the common heading of Model-Driven Engineering (MDE)5 were conceived with this goal in mind. At the core of MDE is the notion that models should serve as primary artifacts in software development, as opposed to being merely optional support facilities for analysis and documentation. Abstracting away unneeded (often target-related) details, models can tame complexity and allow developers to better focus on the key design concerns of their applications [ 44 ]. In other words, models allow developers (i.e., not necessarily experts in programming) to define complex functions in a more human-centric way than if using traditional programming languages. Models conform to a metamodel, defined as part of a modeling language which can be either general purpose or domain specific. From the myriad of general-purpose and domain-specific modeling languages (DSMLs) that have been defined to date, UML has emerged and established itself for industrial modeling [ 33 ]. UML is general purpose, but it provides powerful profiling mechanisms to constrain and extend the language to achieve DSMLs, via UML profiles. In this paper, we focus on execution of UML and UML profiles. If computer-based models of software systems are specified using an executable modeling language, it is possible to gradually evolve them starting with very abstract models, which may be suitable for “descriptive” purposes, into prescriptive ones that are precise and detailed enough to be executable. However, supporting the full development cycle with a single modeling language presents a very unique language design challenge: the ability to create both highly abstract (and potentially incomplete) descriptive models and very detailed prescriptive ones. One of the main goals of the MDE paradigm is to promote automation during the development process through computer-based model manipulation of models and related artifacts. A prominent example of this is the automatic generation of executable artifacts from design models by means of model transformations; there is documented evidence that the automatic production of executables from models is a key feature in MDE [ 25 ]. By shifting the focus of the development from handwritten code to models, one might reasonably expect that corresponding execution mechanisms, namely interpretation and translation (to object code), would be provided for modeling languages as well. However, for a number of reasons, including the need to exploit available middleware, expertise, and development tools, translation of models has primarily targeted high-level languages (e.g., 3GLs) instead of object code (see Fig. 1). Translation to 3GLs can be found also in execution of 3GLs;6 an example is Cfront, which translated C++ to C to enable the use of C compilers. 2.2 UML as an executable modeling language Although the initial release of UML helped in raising the level of abstraction, it was neither sufficiently powerful nor sufficiently precise to produce complete executable programs. Origins of UMLUML was introduced in the second half of the 1990s as an evolution of object-oriented programming. Its origins are at the intersection of three methodologies popular in the 1980s and early 1990s: the Booch method, Rumbaugh’s Object-Modeling Technique (OMT), and Jacobson’s Object-Oriented Software Engineering (OOSE). UML 1.x was created and standardized by the OMG in 1997. With UML 1.5, an action semantics for UML was introduced. Nevertheless, until version 2.0, UML was still considered ambiguous and partially inconsistent, especially in terms of execution semantics. UML 2.0 and aboveVersion 2.0, together with the standardisation of (i) the Foundational Subset For Executable UML Models (fUML), which gives a precise execution semantics to a subset of UML limited to composite structures, classes and activities (inclusion of UML state machines is under formalization too) [ 48 ], and (ii) a textual action language, Alf, to express complex execution behaviors, has made UML a full-fledged implementation-capable language [ 45 ]. fUML and AlffUML is a standard that defines the execution semantics of UML (e.g., how control structures conditionally execute statements); application models designed with fUML are executable by definition. fUML is intended for the definition of the execution semantics for any DSML based on UML profiles [ 48 ]. The work from Mayerhofer et al. [ 35 ] has made it possible to use fUML for defining execution semantics of any DSML based on the Meta-Object Facility standard 5 Other terms used for this purpose include Model-Based (Software) Engineering and Model-Driven Development. 6 In programming, this is called source-to-source compilation or transpilation. too. Alf is a textual surface representation for UML modeling elements, whose execution semantics is given by mapping Alf’s concrete syntax to the abstract syntax of fUML. While Alf maps to fUML in order to provide its execution semantics, its use is not limited to the context of models conforming to the fUML subset. For example, using (f)UML and Alf, it is possible to fully describe a software functionally, while exploiting the UML profile for Modeling and Analysis of Real-Time and Embedded Systems (MARTE) [ 34 ] for modeling hardware components as well as allocations of software to hardware. Example of executable UML modelIn Fig. 2 we can see a portion of a UML model representing the smart street lightning system in a concrete graphical syntax, from which executable C++ code can be automatically generated [ 10 ]. More specifically, the portion represents a lamppost system in terms of its software functionalities, physical devices and allocations. In terms of UML, LampPost_ Functional represents the root software composite component, which contains six software components. Connections between software functionalities are achieved through connectors via ports. Behavioral descriptions of the software components are defined in terms of UML state machines, for defining the overall behavior by means of states and transitions, and Alf, for specifying fine-grained actions. The state machine diagram describing the behavior of the type ManagerR is shown in the upper right corner of Fig. 2. 3 Research method This study has been carried out by following the process shown in the activity diagram in Fig. 3. It can be divided into three main phases: planning, conducting, and documenting. This is in compliance with the well-established methodology for systematic studies [ 27,51 ]. Each phase was performed by one or more members of the research team conducting the study (see Appendix 10). In order to address potential threats to validity and possible biases, we submitted the protocol describing the design of this study and the final report to external experts for independent review and criticism. Each review underwent a thorough refinement iteration of the design and reporting of this study.7 To allow easy replication and verification of our study, we made available to interested researchers a complete replication package.8 It includes a description of the review protocol, the complete list of selected studies, extracted data, and the R scripts we developed for summarizing and synthesizing our findings. In the following, we cover each phase of the process, highlighting the main activities and generated artifacts. Planning The objective of this phase was to: (i) establish the need for a review of UML model execution strategies, (ii) identify the main research questions (see Sect. 3.1), and (iii) define the protocol to be followed by the research team. Conducting In this phase, we performed the review itself by following all the activities defined in the review protocol: • Search and selection we performed a combination of automatic search and snowballing for identifying a comprehensive set of potentially relevant solutions for the execution of UML models. With snowballing, performed after the automatic search, we aimed at enlarging the set of potentially relevant solutions by considering each previously selected research study and focusing on those articles either citing or cited by it [ 49 ]. Next, the candidate solutions were filtered in order to obtain the final list of primary studies to be considered in later activities of the review. Section 3.2 describes in detail the search and selection process. • Data extraction form definition in this activity, we defined the set of parameters to be used for comparing primary studies based on the chosen research questions. This activity was systematically performed by applying a keywording process [ 39 ]. Section 3.3 describes the data extraction in detail. • Classification framework definition: in this activity we organized the parameters pertaining to the technical characteristics of a solution for executing UML models. Because these were applicable to evaluating both tools and research studies, the result was a reusable classification framework, which can be used by both researchers and practitioners (see Sect. 3.3). 7 We would like to thank the following external reviewers: Kai Petersen (Blekinge Institute of Technology), Daniel Sundmark (Mälardalen University), and Jérémie Tatibouet (CEA LIST Institute). 8 http://cs.gssi.it/umlModelsExecution. • Data extraction and quality assessment in this activity we studied the details of each primary study, and based on that, filled in the corresponding data extraction form. Completed forms were then collected and aggregated for use in the next activity. More details about this activity are presented in Sect. 3.3. Furthermore, we assessed the quality of each selected research study according to a set of well-defined quality criteria (see Sect. 6.4). • Data synthesis this activity focused on a comprehensive analysis and summary of the data extracted in the preceding activity. The main goal here was to elaborate on the extracted data in order to answer the research questions (see Sect. 3.1). This involved both quantitative and qualitative analysis of the extracted data. The details of this activity are presented in Sect. 3.4. Documenting The main activities performed in this phase were: (i) a thorough elaboration of the data extracted in the preceding phase in order to properly position the obtained results in the appropriate context from both an academic and a pragmatic point of view, (ii) an analysis of possible threats to validity, especially those identified during the definition of the review protocol, and (iii) the writing of a technical report describing the performed study. 3.1 Goal and research questions The goal of our study is to identify, classify, and evaluate the publication trends, characteristics, provided evidence, and limitations of existing solutions for the execution of UML models from a researcher’s and practitioner’s points of view. This goal was then refined into the following research questions, each of which had a defined primary objective: • RQ1—What are the publication trends of research studies about solutions for the execution of UML models? Objective: to identify and classify primary studies in order to assess interest, relevant venues, and publication types over the years. • RQ2—What are the technical characteristics of existing solutions for the execution of UML models? Objective: to classify existing solutions for executing UML models (e.g., execution strategy used, which elements of UML are supported, application domains targeted), using a systematic classification framework (see Sect. 3.3). • RQ3—What evidence exists to justify adoption of an existing solution? Objective: to investigate and qualify the quality of the evidence that could make a proposed solution applicable in practice. This is based on explicit quality criteria, such as the rigor and thoroughness of the validation strategies used (e.g., controlled experiment, industrial application, formal proofs). • RQ4—What are the limitations of existing solutions for the execution of UML models? Objective: to identify current limitations with respect to the state of the art. In this context, we focused on the limitations (and thereby needs for improvement) of the solutions as they were reported in the analyzed primary studies. 3.2 Search and selection strategy As shown in Fig. 4, our search and selection process was composed of two main sub-processes, each of which focused on a specific source of information: • Research studies search and selection this covers the research described in peer-reviewed publications in various research-oriented venues and forums such as international journals, conferences, books, workshops; • Tools search and selection this covers relevant opensource and commercial tools. Both sub-processes included a stage in which each potentially relevant approach was evaluated against a set of selection criteria (see Sects. 3.2.2, 3.2.4). To handle those cases in a cost-effective way, we used the adaptive reading depth [ 39 ], because it was not necessary to read the full text of approaches that clearly did not qualify. Also, by following the method proposed in [ 6 ], each potentially relevant approach was classified by both the principal researcher9 and the research methodologist as relevant, uncertain, or irrelevant according to the selection criteria described in Sect. 3.2.2. Approaches classified as irrelevant were immediately excluded, whereas all the other approaches were discussed, when needed, with the help of the study advisor. In the following, we give a brief description of each stage of the two search and selection sub-processes. 3.2.1 Research studies search and selection Before performing the actual search and selection of relevant research studies, we manually selected a set of pilot studies. These were selected based on the degree of domain expertise of the authors, as well as on an informal preliminary screening of available literature on the topic. The chosen pilot studies fully met our selection criteria (see Sect. 3.2.2) and are reported in the replication package of this study. The pilot studies were used to validate and refine our search and selection strategy. We describe below the main stages of the research studies search and selection activity. 9 The research team and the different roles are described in “Appendix A”. 1. Initial search In this stage, we performed automatic searches on electronic databases and indexing systems [ 27 ]. As suggested in [ 27 ], to cover as much potentially relevant literature as possible, we chose four of the largest and most complete scientific databases and indexing systems in software engineering: IEEE Xplore Digital Library, ACM Digital Library, SCOPUS, and Web of Science. uml AND (execut∗ OR translat∗ OR compilat∗ OR interpret∗ OR synthes∗ OR simulat∗ OR debug∗ OR (code AND generat∗)) AND (mde OR mda OR mdsd OR mdd OR model∗ OR diagram OR specification) AND (software OR system OR tool) Listing 1 Search string used for automatic research studies To create the search string in Listing 3.2.1, we considered the research questions in Sect. 3.1 and the set of pilot studies. To ensure consistency, the search strings were always applied to the title, abstract and keywords of papers. 2. Impurity and duplicates removal Due to the nature of the electronic databases and indexing systems, the search results could also include items that were not research papers, such as conference and workshop proceedings, international standards, textbooks, editorials. Consequently, in this stage we: (i) manually removed these results in order to have a coherent set of potentially relevant research studies, and (ii) identified and removed duplicated entries. 3. Merging In this stage all relevant results from the different databases and indexing systems used in the first stage were combined. 4. Application of the selection criteria In this stage we considered all the selected studies and filtered them according to a set of well-defined inclusion and exclusion criteria. These criteria are described in details in Sect. 3.2.2. 5. Snowballing To mitigate a potential bias with respect to the construct validity of the study, we complemented the previously described automatic search with a snowballing activity [ 22 ]. The main goal of this stage was to enlarge the set of potentially relevant studies by considering each study selected in the previous stages, and focusing on those papers that either cited or were cited by it. Technically, we performed a closed recursive backward and forward snowballing activity [ 50 ]. 3.2.2 Selection criteria for research studies We used the following inclusion criteria for the research studies: IS1 Studies proposing solutions for the execution of UML models. IS2 Studies in which UML models are: (i) the main input artifacts of the proposed execution method or technique, and (ii) executed without any manual intervention. IS3 Studies providing some kind of evaluation of the proposed solution (e.g., via formal analysis, controlled experiment, exploitation in industry, example usage). IS4 Studies subjected to peer review [ 51 ] (e.g., journal papers, book chapters, papers published as part of conference or workshop proceedings will be considered, whereas white papers will be discarded). IS5 Studies written in English. IS6 Studies for which the full text is available. We used the following exclusion criteria for the research studies: ES1 Studies that do not provide any implementation of the proposed solution. ES2 Studies published before 1997 (because UML was adopted in 1997). ES3 Secondary and tertiary studies (e.g., systematic literature reviews, surveys). ES4 Studies in the form of tutorial papers, short papers, poster papers, editorials, because they do not provide enough information. 3.2.3 Tools search and selection In parallel with the search and selection of research studies, we also performed a search and selection of tools used for executing UML models. The process we followed for searching and selecting relevant tools involved the following stages: 1. Initial search We based this stage on: (i) our personal experiences with execution of UML models, (ii) specific searches for querying generic web search engines with the search string defined in Listing 3.2.1, and (iii) exploiting knowledge garnered from existing networks of experts in the execution of UML models, e.g., by accessing forums, mailing lists. 2. Inclusion of tools from research studies During the search and selection for research studies we stumbled upon further tools used or referred by researchers. Those tools were included in the set of tools to be considered. If an included research study described the tool’s characteristics for execution of UML models, no explicit tool entry is added to the set of tools (to avoid duplication). 3. Consultation with experts In this stage we consulted experts from both industry and academia to minimize the set of selected tools. For example, this required removing duplicates due to renaming of a tool over its lifetime. 4. Application of the selection criteria In this stage, we considered all the selected tools and filtered them according to well-defined inclusion and exclusion criteria. These criteria are presented in detail in Sect. 3.2.4. In this search and selection subprocess, we focused on tools and we collected information from associated documentation (e.g., white papers, user guides, Web pages), rather than from actual scientific papers. 3.2.4 Selection criteria for tools The following tools inclusion criteria were used: IT1 Tools proposing methods or techniques for execution of UML models. IT2 Tools in which UML models are the main input artifacts of the proposed execution method or technique. The following tool exclusion criterion was used: ET1 Tools leveraging third-party tools for executing UML models. 3.3 Data extraction and classification framework definition The main goal of this activity was to create a data extraction form to be used to collect data extracted from each primary study. In our work, the data extraction form was composed of four facets, each of which addressed a specific research question (see Sect. 3.1). Specifically, for answering RQ1 (i.e., research question about publication trends), we considered standard information such as title, authors and publication details of each study. As for the other research questions, (i.e., RQ2, RQ3, RQ4), we followed a systematic process based on keywording for: (i) defining the parameters of each facet of the data extraction form, and (ii) extracting data from the primary studies accordingly. The goal of the keywording was to effectively develop an extraction form that could fit existing studies and tools, and which took their characteristics into account [ 39 ]. The activity diagram in Fig. 5 shows the process that we followed to define the data extraction form. The following describes the individual steps of this process: 1. Identify keywords and concepts We collected keywords and concepts by reading the full text of each pilot study. 2. Cluster keywords and define parameters We performed a clustering operation on collected keywords and concepts in order to organize them according to the identified characteristics. The clustering operation is very similar to the sorting phase of the grounded theory methodology [ 13 ]. More specifically, we considered all the keywords and concepts collected in the previous phase and iteratively grouped them together until a saturation of all the concepts has been achieved. The output of this stage was the initial data extraction form containing all the identified parameters, each of which represented a specific characteristic of a solution for executing UML models. The following steps were performed individually for each primary study. 3. Extract data from current study We first extracted information about the current primary study to be analyzed and then collected information based on the parameters of the data extraction form. Finally, we collected any kind of additional information that was deemed relevant but that did not fit within the data extraction form. 4. Refine data extraction form We reviewed the collected additional information. This could be due to any of the following: • the collected information was not interpreted correctly; in this case, the collected information was refined; • the parameters of the data extraction form were not representative enough for the considered primary study; in this case, the data extraction form was refined so that it better fit the collected information; previously analyzed primary studies were re-analyzed according to the refined data extraction form. The above process was complete when all primary studies were analyzed. Once we had a complete data extraction form, we isolated the facet related to the technical characteristics of the approaches (i.e., the facet related to RQ2). This was then analyzed further to establish any relationships between the parameters, and also to determine the multiplicity and possible values of each parameter. This process was performed through several iterations and led to the definition of a classification framework for solutions for UML model execution. This classification framework is described in Sect. 5. It can be used by researchers and practitioners for classifying, comparing, and evaluating solutions for executing UML models in an objective manner. 3.4 Data synthesis We designed and executed our data synthesis activity by following lessons learned and findings presented by Cruzes et al. [ 14 ]. Specifically, our data synthesis activity was divided into two main phases: vertical analysis and horizontal analysis. In vertical analysis (cf. Sects. 4–7), we analyzed the extracted data to find trends and collect information about each parameter of our data extraction form. In this case, we applied the line of argument synthesis [ 51 ]. We present the results of this vertical analysis in Sects. 4, 5, 6, 7. Moreover, the complete set of extracted data for each primary study is given in “Appendix C”. For horizontal analysis (cf. Sect. 8), we analyzed the extracted data to explore possible relations across different parameters of our data extraction form. To that end, we cross-tabulated and grouped the data and made comparisons between two or more parameters of the data extraction form. The main goal of the horizontal analysis was to uncover the existence of possible interesting relations between data pertaining to different aspects of our research. For this purpose, we used contingency tables analysis as the strategy for evaluating the actual existence of those relations. The results of the horizontal analysis are presented in Sect. 8. In both cases, we performed a combination of content analysis [ 18 ] (mainly for categorizing and coding approaches under broad thematic categories) and narrative synthesis [ 43 ] (mainly for detailed explanation and interpretation of the findings coming from the content analysis). 3.5 Threats to validity In this section, we discuss the main threats to validity of our study and how we mitigated them. External validity Concerning research studies, we employed a search strategy consisting of both automatic search and snowballing of selected studies. We use these two search strategies in combination to mitigate the threat to external validity. In both automatic search and snowballing, we did not consider gray literature (e.g., white papers) because in this phase we are focussing on the state of the art about UML models execution and peer-reviewed scientific venues are well-established publication targets for researchers. Nevertheless, given the quite high number of considered primary studies, this potential bias should not impact our study significantly. Concerning tools, we mitigated the external validity threat by applying a well-defined process for selecting relevant tools from the state of the practice. Namely, we performed (i) an initial search based on our direct experience, specific searches on generic web engines, and the knowledge from networks of experts, (ii) an analysis of the tools referred by selected research studies, and (iii) consultations with experts from both industry and academia. Finally, in our search we considered only material written in the English language. This choice may potentially have led us to discard some primary studies published in other languages. However, the English language is the most widely used language for scientific articles and tools documentation, so this bias can be reasonably deemed to have a minimal impact. Internal validity We mitigated this potential threat to validity by (i) rigorously defining the protocol of our study, and (ii) defining our data extraction form by following a well-defined and preliminarily piloted validated process (see Sect. 3.3). Regarding the validity of the synthesis of collected data, the threats are minimal, since we employed well-assessed descriptive statistics. Construct validity We are reasonably confident that we have not missed any significant relevant research studies or tools. Specifically, research studies were searched by firstly applying an automatic search on multiple data sources. We considered multiple electronic databases to get relevant studies independently of publishers’ policies (see Sect. 3.2). We are reasonably confident about the construction of the search string, since the terms used were identified by carefully following the research questions and by analyzing the set of pilot studies. Complementing the automatic search with snowballing increases our confidence in our search strategy. Tools were selected by applying the previously described search and selection process, involving our direct experience, web search engines, knowledge from existing networks of experts, selected research studies, and experts from both industry and academia. After having collected all relevant research studies and tools, we rigorously screened them according to welldocumented inclusion and exclusion criteria (see Sect. 3.2.1). This selection stage was performed by the principal researcher and the research methodologist. As suggested by Wohlin et al. [ 51 ], a random sample of 20 potentially relevant research studies and tools was identified and the inter-researcher agreement was measured using the Cohen–Kappa coefficient [ 12 ]. The results are very convincing, since we obtained a Cohen–Kappa coefficient of 0.80 for research studies and 0.91 for tools.10 Conclusion validity Firstly, we mitigated potential threats to conclusion validity by applying well-accepted systematic processes throughout our study and we documented all of them in our research protocol, so that this study can be replicated by other researchers interested in the topic. Secondly, the data extraction phase could have been another source of threats to the conclusion validity of our study, mainly because (i) other researchers may identify parameters and dimensions different from ours and (ii) our data extraction activity is based on human judgement about the solutions described in the primary studies. We mitigated this bias by (i) letting the parameters and their facets emerge from the pilot studies and refining them throughout the data extraction activity, (ii) performing an external evaluation by independent researchers who were not involved in our research, and (iii) having the data extraction process conducted by the principle researcher and validated by the secondary researcher. 4 Publication trends In this section, we describe publication trends extracted from the analyzed research studies. More specifically, we aim at answering the following research question: RQ1—What are the publication trends of research studies pertaining to solutions for the execution of UML models? To answer RQ1, for each analyzed research study, we extracted the publication year and venue. The results obtained are discussed below. 4.1 Publication year Figure 6 presents the distribution of publications on execution of UML models over time. From the collected data, we can observe that relatively few studies were published until 2007. Starting with 2008, we can see a growth in 10 The result of the Cohen Kappa statistic is considered as substantial (almost perfect) when it is above 0.60 (0.80) [ 30 ]. the number of studies. Interestingly, this increase coincides with the publication of the first beta version of the fUML specification (in the OMG process, a “beta” specification is publicly released to encourage early implementation and also to identify possible issues which are handled by a Finalization Task Force, leading to a finalized specification that is formally published—in the case of fUML, this took until 2011). Indeed, the average number of publications between 1997 and 2007 is only 1 per year. Between 2008 and 2016, it reached a value of 6. This result confirms the growing focus on and increasing need for research in UML model execution. Given the steady rate of the number of publications per year over the past 8 years, we expect this trend to continue in the future. The first research studies proposing a solution to executing UML models were published in 2000 (P62, P63). Specifically, the authors of (P62) presented a framework for generating simulation programs from UML models with the goal of predicting the performance of the system being modeled. In P63 they used the Fujaba environment to specify production control systems, generate Java code, and simulate their corresponding UML models. We also analyzed the drop of the number of research studies in 2010, but did not find any clear reasons for it. 4.2 Publication venues We classified the analyzed research studies to assess their distribution by (i) type of publication (i.e., journal, conference, or workshop paper) and (ii) targeted publication venues. Figure 7 shows the publication types of the analyzed research studies. The most common publication type is a conference paper (38/63), followed by journal papers (19/63). The low number of workshop papers may be an indicator of the fact that, given the effort required for proposing a solution for UML model execution, researchers focusing on this research topic commonly target more scientifically rewarding publications (e.g., journal papers). Nevertheless, this is changing since there are now workshops and conferences fully dedicated to the topic, creating thereby a clearly defined dedicated research community. The first step toward this change is the International Workshop on Executable Modeling (EXE), co-located with the ACM/IEEE International Conference on Model Driven Engineering Languages and Systems (MODELS), which has hosted three primary studies in 2015–2016. Table 1 shows the publication venues that hosted more than one analyzed research study (the last row of the table is an aggregate of all the publication venues with only one research study). We can see that research on UML model execution is spread across a large number of venues (more than 50 in total), spanning different research areas such as embedded systems, reconfigurable systems, visual languages and human–computer interaction, distributed computing. Under this perspective, UML model execution is perceived today as orthogonal with respect to other research areas. A consequence is that researchers actually focus more on the benefits and effects of exploiting UML model execution (e.g., a gain in terms of system reliability or reconfigurability), rather than on the specifics of the execution mechanisms. Highlights—RQ1. What are the publication trends of research studies related to solutions for the execution of UML models? From 2008 there has been a growing scientific interest on UML model execution; this positive trend has been more or less steady in the past 6 years; Conferences and journals are the most targeted publication venues, testifying that UML model execution is becoming a significant research theme; Research on UML model execution is spread across a large number of heterogeneous venues, with researchers focusing more on benefits and effects of model execution, rather than on the specific execution techniques; 5 Technical characteristics In this section, we describe the technical characteristics of the analyzed solutions for executing UML models. More specifically, it aims at answering the following research question: RQ2—What are the characteristics of existing solutions for the execution of UML models? Figure 8 presents the classification framework we defined for answering this research question. Note that this classification framework was obtained through a systematically conducted process (i.e., the keywording activity for the data extraction form), and it was applied to all the 80 primary studies of this study, which strongly suggests that it provides a general facility for evaluating and comparing approaches for execution of UML models. For reasons of brevity, we do not provide a fully detailed description of all the parameters of the classification framework; however, we do briefly elaborate on each of them when discussing the results of our study in the remainder of this section.11 5.1 UML modeling In this section we examine the characteristics of the analyzed solutions with respect to the way they model a software system with UML. 11 The interested reader can refer to our “replication” package for a thorough and extensive discussion of all characteristics of our classification framework. Required UML diagrams This parameter represents the non-empty set of UML diagrams required by the analyzed solution to be used when modeling a software system (Fig. 9a). Class (48/82), state machine (36/82), and activity (33/82) diagrams represent the most commonly required model representations, often in combination. Overall, the expected trend is that both structural and behavioral models are used when defining executable models. Used action languages To make UML models executable, in addition to well-defined execution semantics, further languages (i.e., action languages) may be needed for specifying fine-grained data and behaviors [ 36 ]. This parameter specifies which action languages were used by the analyzed solutions. Figure 9b shows that there is a remarkable number of solutions (22/82) which do not use action languages. It is interesting to note that only 1 tool (out of 19) does not use any form of action language, whereas the number of research studies not using action languages is much higher (21/63). Almost half of the solutions using an action language (37/60) leverage UML-compliant languages, but only a small portion employs the current standard, Alf (9/37). Most solutions (46/60) leverage popular programming languages as action languages, although these are not semantically consistent with UML. Applied UML profiles With this parameter, we describe whether an analyzed solution relies on any UML profile to enable execution. Note that standard UML is not itself executable, due to the presence of numerous variability points and unspecified details (even fUML, a standardized executable subset of UML, has variability points). Hence, the execution of UML models requires additional semantic details to be provided. This may be done explicitly, using a suitable UML profile that defines the necessary semantics (as in 49/82 of our primary studies), or implicitly, by solutionspecific semantics built into the model transformation or execution technologies. With this parameter, we identify the first case. Among the solutions using UML profiles (see Fig. 9c), the most commonly used standard profile is MARTE (16/49). It is interesting to note that, in a relevant number of cases (27/49), a solution-specific profile was explicitly defined. Modeling tool This parameter specifies the modeling tools which are used by the analyzed solution for UML modeling (see Fig. 9d). The most used tool appears to be Papyrus12 (19/82). The number of tool-independent solutions (14/82) is fairly low, it is worth noting that more than half of the solutions (51/82) leverage open-source tools.13 12 http://www.eclipse.org/papyrus/. 13 This should not be misinterpreted to mean that these tools dominate in terms of usage volume in industrial practice, where commercial tools still seem to be dominant. Based on the fUML standard fUML defines an execution semantics for a large subset of standard UML. This parameter identifies whether the analyzed solution conforms to the fUML specification. Different solutions may leverage different implementations as long as they conform to the specification. Among the analyzed solutions (see Fig. 9e), the majority does not conform to the semantics defined by fUML. However, it should be kept in mind that fUML was not available until 2011. If we consider the studies proposed after 2011 (37/82), we note that more than one-third (13/37) is based on fUML. Note that, in order to address the informality of UML, in approaches not based on fUML, execution semantics is enforced at model transformation level, e.g., by using 3GLs as action code in the models. Covered MDA levels Inspired by the well-known MDA [ 29 ] modeling levels, this parameter specifies whether the analyzed solution covers the platform-independent (PIM) and/or the platform-specific (PSM) modeling levels (see Figs. 9f, 10). While the majority (80/82) exploits models at the PIM level, only very few solutions (17/82) actually model the hardware platform in detail (HW). A significant number of solutions exploit both PIM and PSM models (20/82), or even combine PIM, PSM, and HW (16/82). Support for partial models This parameter specifies whether the analyzed solution supports the execution of models with elements that are tentative, missing or not well-formed [ 53 ]. This capability is particularly useful in early design phases where different design approaches are proposed and evaluated. By supporting the execution of partial models, it is possible to (1) significantly reduce the amount of time and Fig. 10 Distribution of solutions according to the MDA levels effort required to evaluate an approach and (2) increase the likelihood that unpromising approaches will be identified and discarded early in the design cycle. The analysis revealed that, to date, only one solution for execution of UML models explicitly provides support for the execution of partial models; in P10, the authors provide semantics for elements that are “tentative, missing or not well-formed” so that they can support the execution of models that are still under development. 5.2 Execution strategy provide such a feature, which suggests that model execution is (too) rarely a way to assess the model itself. Support for simulation This parameter indicates whether the analyzed solution supports simulation of UML models. Simulation is defined as execution of a model in an environment (e.g., an IDE) that is different from the ultimate intended target environment (e.g., a controller board). This can be useful for a number of reasons, including the unavailability of the target environment, the availability of tools that are not present or not available in the target environment (e.g., debugging tools), or other pragmatic or economical reasons. Among the analyzed solutions, more than half (49/82) provide support for model simulation (see Fig. 12f). It is important to note that in certain cases (such as in tools like iUML or BridgePoint) interpretive and translational approaches live under the same hood, but they are provided through different mechanisms and for different purposes; for example, simulation may be used for models validation/analysis/debugging, whereas model execution is used for running the system on the target platform. Our analysis reveals that among the 14 solutions adopting an interpretive execution strategy, 11 of them also support models simulation. In other words, even if we consider the support for simulation and the interpretive execution strategy as disjoint aspects of analyzed solutions, in the majority of the cases solutions supporting UML models simulation provide also simulation support. Production system This Boolean parameter identifies whether source models are executed on the ultimate target platform (e.g., full code generation and execution), or not (e.g., models simulated/run in the modeling tool only). The significant number of solutions (29/82) that do not provide execution on target platform,(see Fig. 12g), suggests that model execution is considered beneficial for early design assessment too. 5.2.1 Translational execution In this section, we provide the characteristics of the analyzed solutions based whether they execute UML models through translation or interpretation,14 as well as a set of parameters independent of the execution type. We start with the set of independent parameters and continue with parameters specific to translational and interpretive execution. Execution tools and technologies This parameter refers to all third-party tools and technologies used by the analyzed solutions to execute UML models (e.g., programming languages, model transformation languages). From our analysis (see Fig. 12d), we discovered that a substantial number of the solutions (40/82) employs Java as execution technology. The use of Java varies from model transformations to execution in a simulated environment. Note that most tools (16/19) heavily exploit Java, while most academic solutions (41/63) leverage technologies specifically developed for model manipulation (i.e., model transformation languages). Among them, QVT,15 Acceleo,16 and ATL.17 Model-level interactive debugging This parameter specifies whether the analyzed solution provides some level of support for interactive debugging at the level of the source UML model (e.g., support for specifying breakpoints in the models, step-by-step execution at the UML level.). As shown in Fig. 12e, only a few solutions (21/82, of which 13 are tools) Overall, Fig. 11.a shows that the number of translational solutions (70/82) clearly outnumbers the number of interpretive (14/82) solutions. From these numbers, we can conclude that the landscape of solutions applicable in real-world scenarios for the execution of UML models is dominated by translational approaches. This shows a trend of the community in choosing the pragmatic solution represented by translational execution, resulting in a myriad of repetitive solutions answering the same questions, which, considering the relevant number of research studies and tools on the topic, are not definitively answered yet. 14 Note that one approach can provide both translational and inter- Translation targets This parameter describes the target propretive mechanisms (e.g., for code generation and model simulation, gramming languages of the translated source models (see respectively). Fig. 12a). A total of 43 different languages are targeted, with 15 http://www.omg.org/spec/QVT/. the most common being Java (26/82). We found out that 3 16 https://eclipse.org/acceleo/. solutions translate UML models directly to object code. Two 17 https://wiki.eclipse.org/MMT/ATL_Transformation_Language_(ATL). of them are hybrid, since they translate UML models to a for Yes No 0 Yes No 21 32 38 61 60 (b) Translation − Software platform (c) Traceability link support 20 40 60 80 20 40 80 (e) Model−level interactive debugging (f) Support for simulation Yes 15 No 0 Yes No 33 49 55 60 80 0 20 40 80 0 20 40 60 80 (a) Translation − Translation targets SyPsVyteXJHtCAhamMDC+dovCC+aLLan# 245567 13 2326 Petri net 2 PHP 2 IEC 61131−3 code 2 Visual OBathseicr 2 30 0 20 40 60 80 (d) Execution tools and technologies Java 40 Eclipse UML2−based tool 9 Acceleo 5 QVT 5 C++ 5 XSLT 4 xpand 3 XML−based tool 3 C 2 fumlvm 2 hif 2 Other 47 0 20 40 60 (g) Production system Yes No 29 53 0 20 40 60 80 mat that is interpreted by ad hoc virtual machines. The three solutions leverage very limited subsets of UML; this makes them only suitable for limited applications and can explain their negligible impact in the community. It is interesting to note that direct translation to object code is not provided in any of the tools that we analyzed. Each solution employed a specific compiler producing executables for a specific target platform. No solutions used the same compiler nor targeted the same platform. More specifically, the well-known open-source Gnu Collection Compiler (GCC) was used in P30 for generating a simplified form of abstract syntax tree, which was then used for GCC optimizations. By following a different line of reasoning, the authors of P55 realized a model compiler, which takes as input UML models and compiles them into assembly code, from scratch. The produced assembly code can then be executed by a custom virtual machine called UML Virtual Machine (UVM). In P58, an ad hoc compiler and target platform were developed as well, where UML specifications are compiled to equivalent binary representations, which are directly executed on [an ad hoc] Abstract Execution Platform (AEP). Software platform This Boolean parameter specifies whether the generated executable is meant to run on a specific software platform (e.g., OS, user-defined middleware, runtime libraries). The landscape of solutions is balanced with respect to this parameter (see Fig. 12b). Traceability links support This parameter represents whether the analyzed solution provides mechanisms for generating explicit traceability links [ 4 ] connecting the UML models and other artifacts being executed (e.g., the source code generated by translation, the compiled binaries). Such links are crucial for certification in safety-critical domains, and are useful for a variety of purposes, including: debugging in cases where errors are encountered during execution, errors during compilation of the generated code, or for back propagation from the execution to models [ 10 ]. Fig. 12c we can see that only a small set of solutions (15/82), mostly tools (12/15), provide traceability links. A consequence of this is that these solutions are not well suited, for example, in situations where there are needs for vertical navigation from models to code and back based on precise correspondences. An example could be execution-based model assessment and optimization. 5.2.2 Interpretive execution By “interpretive” execution we mean that the UML model is interpreted to be run on the designated platform. The parameter is composed of the name of the interpretation engine used for interpreting and executing the UML models. The overall number of interpretive solutions was 14. Among them, none seems to provide a solution for the execution of UML models on the actual target platform. Instead, they focus on higher-level execution for simulation and model-based analysis. Regarding adopted interpretation engines, none stood out as the preferred one. (See Table 2.) 5.3 Intended benefits This parameter identifies the benefits that the analyzed solution is intended to support explicitly. The possible values are: correctness, meant as improving functional correctness of models (e.g., through model-based analysis), quality, meant as improving non-functional aspects of the generated executable artifact (e.g., through optimized model compilation), or production, meant as reducing the effort needed for Table 2 Interpretation engines Engine name Moka—Papyrus fUML virtual machine Ad hoc engine based on IBM RSA-RT fUML engine based on Kermeta UML interpreter engine MOCAS Populo ACTi fUML reference implementation iUML BridgePoint Studies P18, T14 P24, P25 P31 P32 P34 P23 P48 P49 T13 T11 T12 producing executable artefacts from models (e.g., through automatic code generation). As shown in Fig. 11b, in the majority of the cases solutions aim at decreasing production effort (47/82) or at improving model correctness (46/82). While most of the tools (13/19) focus on production, research studies display a fairly even distribution among the three benefits (29/63 on correctness, 34/63 on production, 24/63 on quality). 5.4 Associated process This parameter specifies whether the proposed solution is closely associated with a specific development methodology. As shown in Fig. 11c, from our analysis we can notice that only a few solutions (11/82) explicitly position themselves within a specific methodology. The only well-known methodology is Shlaer–Mellor’s, associated to the tools iUML (T11) and BridgePoint (T12). Other ad hoc methodologies are CHESS (T4, P6), MADES (P13), GenERTiCA (P5), MaCMAS (P54), and those unnamed provided in P4, P14, P21, and P60. 5.5 Extensibility This parameter shows whether the proposed execution mechanisms can be extended or customized with additional components and capabilities, including those provided by third parties (e.g., plug-in based approaches, support for ecosystems of third-party modules). As shown in Fig. 11d, proportionally, only a minimal part of research studies (5/63) consider such capabilities, while more than half of the tools (12/19) provide extensibility mechanisms. 5.6 Readiness level The Technology Readiness Level (TRL18) is proposed as a systematic metric/measurement system for assessing the maturity of a particular technology, and is defined as an integer n where 1 ≤ n ≤ 9. Consequently, the readiness level of an analyzed solution and can be one of: LOW if n ≤ 4 (i.e., if the solution was either formulated, validated or demonstrated at most in lab), MEDIUM if 5 ≤ n ≤ 6 (i.e., if the solution was either validated or demonstrated in the relevant environment), and HIGH if n ≥ 7 (i.e., if the solution was either completed, demonstrated, or proven in operational environment). We assigned a TRL value depending on how the solution was validated. It is important to note that the TRL parameter refers to the whole proposed solution (e.g., the realization of the whole tool chain, if available) and on how it has been evaluated (e.g., its feasibility has been tested via a simple example, or it has been validated in an operational environment). As shown in Fig. 11e, it turns out that most of the analyzed solutions (56/82) have LOW readiness level. Only two research studies (P3, P56) have HIGH, while the remaining (12/82) are tools. 5.7 Supported non-functional properties This parameter represents the set of non-functional properties recognized by the analyzed solution. By “recognized” we mean that the solution provides explicit mechanisms to assess, optimize, and/or reason about those non-functional properties (e.g., the solution supports analysis of the energy consumed by the system, or its performance, security.). As shown in Fig. 11f, more than half of the analyzed solutions (54/82) do not focus on reasoning on specific non-functional properties. Among the ones which do, performance is the most addressed property (25/28). 5.8 Formal specification languages This parameter displays the formal specification languages (see Table 3) eventually used in conjunction with the UML models for, e.g., proving security properties, formally verify services composition. Formal language specifications are in some cases generated from UML models, and in others defined along with them; nevertheless, primary studies did not report enough details to be able to deeply categorize the two variants. Most solutions (69/82) do not employ any for18 The TRL measurement system is employed by the Horizon 2020 European commission for the work program of 2014/2015: https://ec.europa.eu/research/participants/data/ref/h2020/ wp/2014_2015/annexes/h2020-wp1415-annex-g-trl_en.pdf. mal language. Among those that use formal languages, the majority performs model checking. Highlights—RQ2. What are the technical characteristics of existing solutions for the execution of UML models? Solutions providing translational execution clearly outnumber interpretive solutions. Interpretive solutions are mainly addressing higher-level execution (e.g., for simulation). Solutions for translation to object code only leverage very limited subsets of UML; Class, activity, and state machine are the most used diagrams; Most solutions employ 3GLs as action languages, while only very few adopted the standardized action language for UML (Alf); Currently, there is only one solution which provides support for execution of partial models; Out of the analyzed translational solutions, only a very few provide explicit traceability links, which can be exploited for consistent navigations between models and generated code; Very few solutions provide support for model-level interactive debugging (note that on the other hand most tools provide this feature); Among the very few solutions that display HIGH as readiness level only two are research studies; A very small amount of solutions explicitly provides mechanisms which enable extension and customization. 6 Provided evidence In this section, we describe the evidence provided by the analyzed research studies (52/70, since tools are excluded). More specifically, it aims at answering the following research question: RQ3—What is the evidence that motivates the adoption of existing solutions for the execution of UML models? Intuitively, the goal of answering RQ3 is to assess what is the potential for industrial adoption of research studies on UML model execution today. The underlying assumption of RQ3 is that research studies which have been (rigorously) evaluated in industry and with higher quality are more likely to be adopted in industry. This parameter represents the type of applied research method used to assess the analyzed solution. Based on [40, fig. 19], the possible values of this parameter are validation and evaluation. Validation is done in lab, whereas evaluation takes place in real-world (industrial) contexts. The latter generally provides a higher level of evidence about the practical applicability of a proposed solution, provided that the considered control groups are sufficiently diverse and independent, and that the number of participants in the studies is sufficiently large. Our analysis (see Fig. 13a) uncovered that only a very small number of the analyzed research studies (12/63) provided an evaluation. 6.2 Type of evidence This parameter describes the type of evidence (e.g., example, empirical study) exploited by the analyzed research study for Synthetic 10 Controller 10 Communication 8 Mobile robotic system 7 Media converter 7 Manufacturing 5 Business processes 5 Monitoring and sensing 4 Information system 4 Consumer electronics 4 Mobile app 3 Modeling language 2 Web 2 Mathematical function 1 80 0 the purposes of validation and/or evaluation. The different types of evidence are: • Example: 1 in-house example. • Set of examples: several in-house (not industrial) examples, several runs and comparisons of one example, even several models for the system, e.g., with different sizes. • Empirical laboratory: case studies, controlled experiments and empirical evaluations in laboratory. • Industrial example: example coming from industry and performed in laboratory. • Set of industrial examples: several industrial examples (even several models for the system, e.g., with different sizes). • Empirical industrial: case studies, controlled experiments and empirical evaluations either in industry or in realworld scenarios. • Industrial evaluation: evaluation performed by industrial actors, solution used in industry. Table 4 Quality assessment criteria ID Q1 Q2 Q3 Q4 Q5 Q6 Figure 13b shows that the majority (44/63) of the analyzed solutions rely on the least systematic and least reliable typology, that is, on examples. Of these, only a few (11/44) use multiple different examples. Out of the few that provide evidence in industrial settings (15/63), only three rely on empirical evaluations, whereas the majority of them (10/15) apply the proposed solutions on examples coming from industry. Only one research study (P6) actually performed a full-fledged industrial evaluation of the proposed solution. These results related to industrialbased evidence are actually making evident the perception that academic results on UML model execution are not fully transferred to or at least evaluated by industry. This can be considered as an interesting research gap to be explored in the future. The obtained results also show that more than half of the research studies (39/63) present the application of their proposed solution using only a single example, independently of whether the example comes from industry or not. However, reaching conclusions about a complex topic like a solution for UML model execution from a single (usually simplified) example is dubious; this may negatively affect the potential impact that a proposed solution may have. The research community can address this potential issue by evaluating their proposed solutions on a set of complementary examples or applications in order to provide different perspectives over the proposed solutions. 6.3 Type of systems for evidence This parameter describes the type of system used for providing evidence. Figure 13c presents the distribution of the analyzed studies with respect to the type of system used in obtaining evidence. The obtained results are spread across a variety of alternatives, ranging from communication applications, consumer electronics, mobile apps. This leads to the conclusion that there is no significant preference when it comes to the type of system used for providing evidence. Quality criteria Is there a clear statement of the aims of the research? Is there an adequate description of the context (e.g., industrial use, laboratory-based investigation, product) in which the research was carried out? Is there an adequate justification and description of the research design? Is there a clear statement of obtained findings, including data supporting them? Is there a critical discussion of the researchers’ roles, potential bias, and influence on the proposed research? Is there a critical discussion of potential limitations of the proposed research? Interestingly, one of the two most used types of system is the synthetic one (10/63), meaning that the proposed solution has been applied to an artificial ad hoc example which is not grounded in any specific real system. The other most used type is controller (10/63). Communication systems (8/63), manufacturing systems (5/63), and information systems(4/63) are some of the types utilized (the interested reader can refer to the figure for the complete list). Finally, we can notice that recent technological trends are reflected in the types of systems, such as mobile robotic systems (7/63) and mobile apps (3/63). This shows that UML can be successfully used for modeling and executing modern software systems. 6.4 Quality assessment results In the context of this research, we assessed the quality of each selected research study with the objectives to: (i) provide an indication of the overall level of quality of academic research on executing UML models, and (ii) complement our findings about the evidence that motivates the adoption of solutions for executing UML models (RQ3). We defined a set of criteria for assessing the quality of research studies in an objective and unbiased manner. We decided to follow the strategies already applied in [ 5,19 ], and we based our quality assessment strategy on the assessment instrument used in [17]. Specifically, we formulated the quality score of a study according to a set of criteria formulated as questions. Table 4 presents the questions we used for our study, which were inspired by those proposed in [ 5,19 ]. Each quality assessment question was answered by assigning a numerical value (1 = "yes", 0 = "no", and 0.5 = "to some extent") [19]. It is important to note that the quality criteria reported in Table 4 focus more on the quality of the paper (e.g., its clarity, rigor, precision in describing aspects of the performed research.) rather than on the quality of the research itself. The quality of research studies is important because a badly reported scientific article may suffer in terms of impact and Q1 Fig. 14 Quality assessment—analysis results 32 17 14 9 29 25 Q2 Q3 Fig. 15 Quality assessment—criteria-specific analysis results transfer to industry, mainly because it may be difficult to locate relevant information in the study itself and important information may be missing. Figure 14 shows the frequency of the total quality scores achieved by the selected research studies. The maximum score for a study is 6, i.e., scoring 1 for each question of the quality criteria. By looking at the obtained data we can tell that the majority of the studies achieved a score between 2 and 4, with only three studies falling below a score of 2, and 12 studies scoring above 4. The results tell us that the majority of the studies performed well, even though they still left something to be desired from a research quality point of view (e.g., many studies failed in describing precisely the context of their research and its potential limitations). Figure 15 shows the frequencies of the scores of our selected research studies over the six quality criteria. In this context, the good news is that the majority of research studies 6 performed well when providing a clear statement of the aims of their research (Q1) and the obtained results (Q4). Also, the justification and description of the research design (Q3) were described fairly well, with 25 studies doing it fully, 29 doing it partially, and 9 that did not report them at all. However, we can see insufficient scores with respect to the adequacy of the description of the context in which the research was carried out (Q2), with 32 studies not even saying anything about it. A similar situation can be seen for the reporting of a critical discussion of the researchers’ roles, their biases and influence on their studies (Q5), and of the potential limitations of the proposed research (Q6). Overall, this analysis reveals the main issues pertaining to the quality of documenting current research on execution of UML models. Clearly, the research community must do better on these aspects of their work. It is important to note that one of the reasons for having lower quality scores may be related to the page limits of venues where the primary studies have been submitted. In order to better investigate on this aspect, we built a contingency table between each single quality criteria Q1–Q6 and the type of publication (i.e., journal, conference, workshop). The analysis of the contingency tables reveals that the type of publication has a slight influence on the Q2 and Q3 quality criteria, where we noticed that journal papers have higher quality (i.e., a score of 1 for Q2 and Q3) with respect to overall trends. In contrast, the publication type does not play a significant role when considering Q1, Q4, Q5, Q6, and their aggregation computed by summing all Q1–Q6 criteria. Based on the results obtained, in Table 5, we list the research studies with a quality score higher than or equal to 5. They can serve as good examples of high-quality research that can inspire future research studies in the area. Highlights—RQ3. What is the evidence that motivates the adoption of existing solutions for the execution of UML models? The majority of the analyzed studies provide validation rather than evaluation (according to the definitions in [40, fig. 19]); A small number of studies provides evidence by experimentation in industrial settings; among them, only a few rely on empirical evaluation. The majority of the research studies are of good quality, although many of them fail in describing precisely the context of their research, the researchers’ roles, and potential limitations. Score This section describes the limitations (and thereby need for improvement) of the solutions as they were reported in the analyzed research studies. More specifically, it aims at answering the following research question: RQ4—What are the limitations of existing solutions for the execution of UML models? Table 6 shows the distribution of the identified limitations and needs for improvement across our primary studies. Among them, the most common type of needed improvement is expressiveness enhancement with respect to the coverage of UML concepts (31/63); this result is not surprising, given the huge number of concepts of the UML metamodel; indeed, as of today the UML metamodel contains 264 classifiers (including abstract classifiers) such as Class, Object, Activity, Transition. It is understandable that many research teams do not have the resources for providing a complete approach with respect to the full set of concepts related to specific UML diagrams since they may be hundreds. In this context, an interesting perspective is provided by the potential added value that a dedicated company or a university spin-off may give to provide more complete tool support. An example of this positive synergy is the Eclipse Papyrus project, which started as an open-source project lead by the LISE team of a research division of the French alternative energies and atomic energy commission (CEA) and is supported by an industry consortium under the Eclipse Foundation umbrella now.19 Along the same lines, another recurrent improvement among research studies is tool enhancement (20/63). Similarly to the case of expressiveness, we can trace the motivation behind this result to the limited resources of researchers when considering a large language like UML. Also, it is important to note that in some cases research groups may be more interested in providing a prototype tool for giving evidence about their scientific results, rather than in providing an industry-ready, fully functional tool. As a natural consequence of this situation, tool enhancement for a specific 19 http://projects.eclipse.org/projects/modeling.mdt.papyrus/who. research-driven solution is one of the most common needs for improvement. Other relevant possible improvements are additional analysis at model level (12/63) and additional evaluation (11/63). In order to better put into context the identified limitations, we performed a detailed analysis about the potential correlations between identified limitations and all the other parameters of our classification framework; the results of this analysis are presented in Sects. 8.13 to 8.16. Highlights—RQ4. What are the limitations of existing solutions for the execution of UML models? The most common limitation is represented by the supported expressiveness in terms of covered UML concepts. Another relevant limitation is related to tool enhancement. 8 Horizontal analysis The goal of our horizontal analysis is to investigate on possible correlations between related parameters of the classification framework. To that end, while designing the classification framework, we kept track of potentially relevant relationships that might exist between pairs of parameters (the full set of potentially relevant relations is in our replication package). In this context, we checked those relations by analyzing the contingency table [ 3 ] of each potentially relevant pair of parameters. The results of this analysis are described below. 8.1 Execution strategy versus UML diagrams As discussed in Sect. 5.1, the most commonly required UML diagrams for model execution are class diagram, activity diagram, and state machine diagram, often in combination. Nevertheless, it is interesting to understand which diagrams to define the semantics of fUML itself as well as other major sections of standard UML). At the time of this writing, at least one tool, Papyrus, already supports this type of capability. The ability of an execution environment to adopt languagespecific semantics is part of a more general need: the capacity to be customized to suit specific needs and situations. With software being used increasingly to control various physical processes (so-called cyber-physical systems [ 32 ]), the ability to readily and correctly integrate and synchronize the execution of UML model simulators into complex multiparadigm (e.g., hybrid) simulation frameworks, involving components that simulate different kinds of components has become crucial. And, in line with the previously noted gradual elimination of the boundary between design time and deployment time execution, an enhanced ability to combine UML simulators and/or execution systems with actual running systems is paramount. 9.2 Research perspectives for UML model execution As a by-product of our in-depth analysis, we also reached certain conclusions about potential research directions for executable UML models. While these are inherently subjective and speculative in nature, we feel that it is worthwhile noting them in the expectation that they might prove useful to researchers in this domain. A potentially significant need we observed pertains to what is referred to here and elsewhere as “model-level debugging.” As it can be seen from Fig 12g, only a few solutions provide support for this. Yet, what we are dealing with here is the general ability to observe and understand the behavior of an executing systems, whether it be in development or in actual field operation. Recall that a central tenet of MDE is to raise the level of abstraction of software specification such that it is closer to the problem domain. Why should this be limited just to the development environment? Given the increasing reliance of modern society on software-intensive systems, the ability to observe, understand, and control executing software is gaining in significance. (Debugging, albeit critically important, is just one specialized manifestation of this more general but under-appreciated need.) In concrete terms, this requirement translates into the general ability to observe and control executing models at an abstraction level that exactly matches the level in which they were specified. From a practical viewpoint, this can be decomposed into two finer-grained requirements: (1) the ability to select what is to be observed in an executing model and when, and (2) the ability to direct both the flow and even the pace of execution. This requires refined mechanisms to attach probes at selected points in time and space, to selectively capture and record the flow and consequences of execution (or, possibly, to slow it down to “human” rates), as well as to steer execution in the desired direction by selective injection of inputs or setting of variable values. All of this has to be done at the “model level,” or the benefits of raising the level of abstraction will not only be lost, but may actually prove to be an impediment because of the need to understand the potentially complex relationship between a model and its low-level technology-specific realization. Clearly, this is a critical avenue of research for those working on model execution. Note that the ability to control model execution is also critical when executable models are used during design space exploration. In those situations, it is desirable to produce minimal models that capture only the essence of the design approach, without investing excessive effort in specifying low-level implementation detail. Consequently, such models are inherently incomplete. This means that in the course of executing such a model, points can be reached where execution cannot proceed due to missing information or ambiguity. In such situation, the ability to “force” the model to proceed down a selected execution path becomes fundamental. To summarize, we see that there is a critical need to research and develop new and efficient methods for observing and controlling the execution of software systems developed using MDE principles, including those based on executable UML. 10 Conclusions In this study, we reported on a systematic review focused on identifying, classifying, and evaluating existing solutions, both in research and industry, for the execution of models based on the UML family of languages. The goal was to identify and assess trends, technical characteristics, available evidence, and limitations of current solutions for the benefit of both researchers and practitioners, as well as to identify a set of critical research challenges. From over 5400 entries dealing with UML model execution, we selected 63 representative research studies and 19 tools. Research studies were selected via automatic searches on electronic data sources and a closed recursive backward and forward snowballing activity, whereas tools were selected via automatic searches on generic web search engines, from research studies, and by consultations with experts. Next, we derived a classification framework to help us characterize the different solutions, which we then applied to the 82 selected solutions. Finally, we analyzed and discussed the obtained data to: (i) provide an overview of the state of the art and practice in the domain and (ii) identify emergent research challenges and associated implications. From the collected data we can conclude that: (i) there is growing scientific interest in UML model execution; (ii) currently, translational execution approaches clearly outnumber interpretive methods; (iii) there is a lack of solutions available for executing high-level (i.e., partial or incomplete) UML models; (iv) there are very few solutions available for modellevel debugging; (v) very few solutions explicitly provide mechanisms for extension and customization; (vi) there is insufficient research of industrial usage and associated empirical methods; (vii) although most of the selected research studies were of good technical quality, many of them fail to adequately describe the context in which the research was performed, the roles of the researchers, and the limitations of their contributions; (viii) the limited degree of coverage of UML concepts (and, hence, the expressiveness) is the most common limitation of the analyzed solutions. Acknowledgements This research was supported by the Knowledge Foundation through the SMARTCore (http://www.es.mdh.se/projects/ 377-SMARTCore) and MOMENTUM projects (http://www.es.mdh. se/projects/458-MOMENTUM). Appendix A—Research team Three researchers carried out this study, each of them with a specific role within the research team. Principal researcher Dr. Federico Ciccozzi, senior researcher with expertise in model-driven engineering and component-based software engineering for the development of complex systems based on domain-specific modeling languages, both UML and EMF based. This team member specializes in: definition of DSMLs (including UML profiles), automatic model manipulations through transformations for code generation, analysis, model optimization, system properties preservation (just to mention a few); he was involved in all the activities, i.e., planning the study, conducting it, and reporting. Research methodologist Dr. Ivano Malavolta, senior researcher with expertise in empirical methods applied to software systems and systematic literature reviews; he was mainly involved in (i) the planning phase of the study, and (ii) supporting the principal researcher during the whole study, e.g., by reviewing the data extraction form, the selected primary studies, the extracted data, the produced reports. Advisor Prof. Bran Selic, senior researcher with extensive expertise in model-driven engineering, complex systems, UML and related profiles, simulation and execution of UML models. He provided guidance on key decisions and during conflicts, thereby avoiding ‘endless discussions’ [ 52 ]. He also supported the other researchers during data and findings synthesis activities. From a geographical point of view, the research team is entirely distributed thereby ensuring ‘expertise’ and reinforcing ‘independence’ of the individuals’ reviews. Appendix B—Selected primary studies See Tables 7 and 8. Integration of UML models in FMI-Based co-simulation On the automated translational execution of the action language for foundational UML A Model-Driven Engineering Methodology to Design Parallel and Distributed Embedded Systems System-level design based on UML/MARTE for FPGA-based embedded real-time systems On the Generation of Full-Fledged Code from UML Ciccozzi, F. et al. Profiles and ALF for Complex Systems HDL code generation from UML/MARTE sequence Ebeid, E. et al. diagrams for verification and synthesis A formal, model-driven design flow for system simulation and multi-core implementation A Model-Driven Approach to Generate Mobile Applications for Multiple Platforms Improving the design flow for parallel and heterogeneous architectures running real-time applications: The PHARAON FP7 project Reliable execution of statechart-generated correct embedded software under soft errors Formalizing execution semantics of UML profiles with fUML models Textual, executable, translatable UML Extending UML/MARTE to Support Discrete Controller Synthesis, Application to Reconfigurable Systems-on-Chip Modeling Model-driven engineering of Manufacturing Automation Software Projects A SysML-based approach A model-based framework for developing real-time safety Ada systems Combining fUML and profiles for non-functional analysis based on model execution traces Executing and debugging UML models: an fUML extension Multi-Paradigm Semantics for Simulating SysML Models using SystemC-AMS Environment modeling and simulation for automated Iqbal, M.Z. et al. testing of soft real-time embedded software Software and Systems Modeling Berardinelli, L. et al. International ACM Sigsoft conference on Quality of 2013 software architectures Laurent, Y. et al. ACM Symposium on Applied Computing Chaves Cafe, D. et al. Specification and Forum on Design Languages Model-level, Platform-independent Debugging in the Bagherzadeh, M. et al. Joint Meeting on Foundations of Software Context of the Model-driven Development of Engineering Real-time Systems Guermazi, S. et al. Symposium on Theory of Modeling and Simulation 2016 Radjenovic, A. et al. European conference on Modelling Foundations and 2012 Applications Brazilian Symposium on Computing System Engineering Forum on Specification and Design Languages International Symposium on Object/Component/Service-Oriented Real-Time Distributed Computing ACM Symposium on Applied Computing Journal of Software: Evolution and Process European conference on Modelling foundations and 2011 applications International Symposium on Software Engineering for Adaptive and Self-Managing Systems International Workshop on Equation-Based Object-Oriented Modeling Languages and Tools European conference on Modelling foundations and 2011 applications Wada, H. et al. Riccobene, E. et al. Handbook of Research on Software Engineering and 2009 Productivity Technologies: Implications of Globalization ACM Transactions on Embedded Computing Systems Arpinen, T. et al. EURASIP Journal on Embedded Systems Vidal, J. et al. Design, Automation and Test in Europe Table 7 continued Study Title A Model Driven Approach for Android Applications Parada, A.G. et al. Development Modeling and simulation of secure wireless sensor network Diaz, A. et al. An Optimized Compilation of UML State Machines Charfi, A. et al. P27 P28 P29 P30 P31 P32 P33 P34 P35 P36 P37 P38 P39 P40 P41 P42 P43 P44 P45 P46 P47 P48 P49 P50 Symbolic execution of UML-RT state machines Achieving process modeling and execution through the combination of aspect and model-driven engineering approaches Contracts for Model Execution Verification On the Performance of UML State Machine Interpretation at Runtime Modelica code generation from ModelicaML state machines extended by asynchronous communication Code generation for UML 2 activity diagrams: Toward a comprehensive model-driven development approach From UML to Petri Nets: The PCM-Based Methodology Closing the Gap between UML-based Modeling, Simulation and Synthesis of Combined HW/SW Systems Matilda: A Generic and Customizable Framework for Direct Model Execution in Model-Driven Software Development SystemC/C-Based Model-Driven Design for Embedded Systems Performance evaluation of UML2-modeled embedded streaming applications with system-level simulation A co-design approach for embedded system modeling and code generation with UML and MARTE Framework to Simulate the Behavior of Embedded Real-Time Systems Specified in UML Models Wehrmeister, M.A. et al. Brazilian Symposium on Computing System Engineering Distefano, S. et al. IEEE Transactions on Software Engineering Mischkalla, F. et al. Design, Automation and Test in Europe SecureMDD: A Model-Driven Development Method Moebius, N. et al. for Secure Smart Card Applications Realization of UML class and state machine models Derezinska, A. et al. in the C# code generation and execution framework eUDEVS: Executable UML with DEVS Theory of Modeling and Simulation Model-driven development of composite context-aware web applications Execution and Simulation of (Profiled) UML Models Fuentes, L. et al. using Pópulo Toward a UML virtual machine: implementing an interpreter for UML 2 actions and activities Crane M.L. et al. Automatic Performance Model Transformation from Pllana, S. et al. UML to C++ International workshop on Models in software engineering Conference of the center for advanced studies on collaborative research: meeting of minds International Conference on Parallel Processing—Workshops International Conference on Availability, Reliability 2009 and Security Informatica Risco-Martin, J.L. et al. Simulation Kapitsaki, G.M. et al. Information and Software Technology P51 An Execution Framework for MARTE-based ModelsMraidha, C. et al. P52 UJECTOR: A Tool for Executable Code Generation Usman, M. et al. from UML Models P53 MDD4SOA: Model-Driven Service Orchestration Mayer, P. et al. P54 Enabling the Evolution of Service-Oriented Solutions Using an UML2 Profile and a Reference Petri Nets Execution Platform Fabra, J. et al. P55 A Model-Based Approach for Platform-Independent Schattkowsky, T. et al. Binary Components with Precise Timing and Fine-Grained Concurrency P56 FSMC+, a tool for the generation of Java code from Tiella, R. et al. statecharts P57 MDA-based approach for embedded software generation from a UML/MOF repository P58 A Model-Based Approach for Executable Specifications on Reconfigurable Hardware P59 Automatic Code Generation from a UML model to Vogel-Heuser, B. et al. JEC 61131-3 and system configuration tools P60 Embedded System Design Using Formal Model Refinement: An Approach Based on the Combined Use of UML and the B Language International Conference on Engineering of Complex Computer Systems Advanced Software Engineering and Its Applications2008 International Conference on Internet and Web Applications and Services Hawaii International Conference on System Sciences2007 International Symposium on Principles and Practice 2007 of Programming in Java International Conference on Control and Automation2005 Do Nascimento, F.A.M. et al.Symposium on Integrated Circuits and Systems Design Schattkowsky, T. et al. Design, Automation and Test in Europe Voros, N.S. et al. Design Automation for Embedded Systems Tool T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 T11 T12 T13 T14 T15 T16 T17 T18 T19 P61 Deriving executable process descriptions from UML Di Nitto, E. et al. International Conference on Software Engineering 2002 P62 A UML tool for an automatic generation of simulation programs P63 Testing and simulating production control systems using the Fujaba environment Arief, L.B. et al. Niere, J. et al. International Workshop on Software and Performance Applications of Graph Transformations with Industrial Relevance http://www.fujaba.de/ https://xtuml.org/ http://fuml.modeldriven.org https://wiki.eclipse.org/Papyrus_Qompass https://www.eclipse.org/papyrus-rt/ http://www.state-machine.com/qm/ http://www.sparxsystems.com/ Appendix C—Extracted data for each primary study Legend for Table 9. UML diagrams (UC: Use case, T: Timing, STRUCT: Comp. structure, SM: State machine, SEQ: Sequence, P: Package, O: Object, INT: Interaction, DEP: Deployment, C: Component, COMM: Communication, CL: Class, ACT: Activity, COLLAB: Collaboration, AH: Ad hoc). Action lang. (ADA: ADA, UAL: UAL, C#: C#, C: C, ALF: ALF, J: Java, C++: C++, UA: UML actions, ✗: Not supported). Profiles (MOD: ModelicaML, ✗: Not supported, PA: UML-PA, SPT: UML-SPT, SC: SystemC UML profile, RT: UML-RT, SYS: SysML, M: MARTE, AH: Ad hoc UML profile). Tool (I: Tool independent, C: CHESS, S: Sparx Enterprise Architect, RR: IBM Rational Rose, A: Artisan Real-Time Studio, RSA: IBM Rational Software Architect, EU: Eclipse UML2, P: Eclipse Papyrus). fUML ( : yes, ✗: no). MDA levels (PIM: platform-independent model, PSM: platform-specific model, HW: hardware). Legend for Table 10. Execution strategy (I: interpretation, T: translation). Intended benefits (C: correctness, P: production, Q: quality). Associated process ( : yes, ✗: no). Extensibility ( : yes, ✗: no). Readiness level (H: high, M: medium, L: low). Non-functional properties (C: code size, P: performance, A: adaptability, S: safety, SE: security, ✗: not supported). Formal languages (B: Event B, BHDL: BHDL, FFSM: Finite state machine (ad hoc), Jolie: jolie, KIV: KIV, Lisp: Lisp, NuSMV: nusmv, Z: Z, ✗: not supported). Legend for Table 11. Traceability ( : yes, ✗: no). Model debugging ( : yes, ✗: no). Simulation ( : yes, ✗: no). Production system ( : yes, ✗: no). Legend for Table 12. Software platform ( : yes, ✗: no). Legend for Table 13. Research method (V: Validation, E: Evaluation). Type of evidence (E: Example, ES: Set of experiments, EXS: Set of examples, EL: Empirical experiment in the lab, EXI: Example from industry, EXSI: Set of examples from industry, EI: Industrial empirical experiment, EVI: Industrial evaluation). Type of systems for evidence (W: Web, S: Synthetic, SE: Monitoring and sensing, MR: Mobile robotic system, A: Mobile app, MC: Media converter, MA: Mathematical function, MAN: Manufacturing, L: Modeling language, IS: Information system, C: Controller, CE: Consumer electronics, COM: Communication, B: Business processes). Identified limitations (T: Traceability enhancement, TO: Tool enhancement, S: Scalability, R: Support for runtime models update, RE: Reusability enhancement, P: Portability enhancement, PI: Platform independence enhancement, EP: Execution platform improvement, MC: Support for model checking, E: Expressiveness enhancement, EC: Execution strategies combination, ECO: Execution correctness assessment, O: Generated code optimization, COV: Better coverage of UML, EV: Additional evaluation, AN: Additional analysis of models, PS: Platformspecific limitations) (Table 14). L M U . g n a l S ro T H Y Y Y P R A ✗ S M M ✗ ✗ ✗ R M S M A A M ✗ ✗ ✗ M S M M M ✗ S ✗ ✗ M ✗ M fU ✗ ✗ ✗ ✗ ✗ ✗ ✗ ✗ ✗ ✗ ✗ ✗ ✗ ✗ ✗ ✗ ✗ ✗ ✗ ✗ ✗ ✗ r e re th th ,O H O S A , , A U , + + C F , L n C o , it F F F c A A L A A L A L A A A A U U A U U A U A U U ✗ ✗ ✗ C A ✗ U A O ✗ ✗ ✗ J U U ✗ O ✗ ✗ U r A FL teh r e h A A t W W H H , , M M M S S S P P P , , , W W H H , , M W M M M PS ,H PS P P S S , , , , W W H , H , M M M S S S P P P , , , M S P , — ) 2 s r e h U t lo A o S U U U T R E E E I P E O A I S I O O I R E I O O O O I O O I I O A R r r r r r r r r teh teh SA U teh teh teh teh teh teh r e h t R O O Q P , , E S SM P , Q E E irsaadgLm ,TLC ,,SSTEQM ,,STCR ,TCA ,,,TCOAD ,STTR ,SEQ ,,,SSTCRM ,STTR ,,,STCR ,,TECAD ,SM ,,STCR ,STCAM ,TCA T T ,SM ,,SETCQA T ,LLCBOA ,,ISTTNM ,SEQMM ,,SSEQM ,,SCOMM ,SLM M M C M M C M L C C L L C L L L ,LC ,L L C C L L C M C M O L L ,C U S A S S A S C U A C C A C C C P C C A A C C A S A S C C C P 9 e y l d 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 ab tu 3 3 3 3 3 3 3 3 3 4 4 4 4 4 4 4 4 4 4 5 5 5 5 5 5 5 5 5 5 6 T S P P P P P P P P P P P P P P P P P P P P P P P P P P P P P P T , M S , P E D , d e u n i t n o c W W W W W H H H H H , , , , , M M M M M M M S S S S S S S P P P P P P P , , , , , , , W W W H H H , , , M M M M M M S S S S S S P P P P P P , , , , , , H lrseofi T treh DO PT ,SAY treh ,CAH ,AH ,SSY H H treh H treh PT treh A H P R O ✗ ✗ M M ✗ S S O S M M A ✗ ✗ A ✗ ✗ ✗ M ✗ O A O ✗ S O P A O , + + P E D , L W H , W H , M M M M M S S S S S P P P P P , , , , , A D IM IM IM IM IM IM IM IM IM IM IM IM IM IM IM IM IM IM IM IM IM M P P P – P P P P P P P P P P P P P P P P P P M fU ✗ ✗ ✗ ✗ ✗ ✗ ✗ ✗ ✗ ✗ ✗ ✗ ✗ ✗ ✗ ✗ ✗ r r r r r r r r r r l e e e e oo th th th th ,P teh teh R SA teh teh teh teh r r e e th th T I O O A O O C O O R R O O O O I P P P O S O r e lse th r r H r fi S S O S e e A e ro Y Y , Y T th th , th H P ✗ ✗ ✗ S ✗ S M ✗ S ✗ ✗ R ✗ O O ✗ ✗ M O ✗ ✗ A J DA ,re h ,A t # # O C ,C ,+ # , C + C C ++ r C , , .ltiacnngo ,,trJehOA ,,teh++CO treh FL ,,trJehOA ,,,JF++LC ,,LDAAA ,,J++LCA ,,J++LCA ,,,trJehCC treh treh FL FL ,treh++O ++ ,++C ,,++CC A ✗ ✗ U – J O A U A U U U O O O A A C C C J ✗ M S , P D , E , O , P M E S M S M Q , M E L O S C C , , , T d e u n i t n o c 9 e y l d 1 2 3 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 ab tu 6 6 6 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 T S P P P T T T T T T T T T T T T T T T T T T T s e i t r e p o r p l a n o i t c n u f n o P A O , , , E N ✗ ✗ ✗ P P P ✗ ✗ ✗ ✗ ✗ P P ✗ ✗ P ✗ ✗ ✗ P ✗ P P P ✗ ✗ S ✗ S C R L L H L L M L L L L L L M L M M L L L L M M M L L L M L L L ✗ ✗ ✗ ✗ ✗ ✗ ✗ ✗ ✗ ✗ ✗ ✗ ✗ ✗ ✗ ✗ ✗ ✗ ✗ ✗ ✗ l e v e l s t fi e n e b y g e t a r t s fi i c e sp L l V D a r r r M H rom teh teh IV lieo teh uS ,B F O ✗ ✗ ✗ ✗ ✗ ✗ O ✗ ✗ ✗ ✗ ✗ K ✗ ✗ ✗ ✗ ✗ ✗ ✗ ✗ J O ✗ N ✗ ✗ ✗ B A , E N ✗ ✗ ✗ P ✗ ✗ ✗ P ✗ P P P ✗ S ✗ ✗ ✗ ✗ ✗ P P ✗ ✗ ✗ P ✗ P P ✗ P R L L L L L L L L L M M L L L L L L L L L L L L L L H L L L L n o i d t e u u c n e ti x n E I I I I T T T T T T T T T T T T T I I T T T T T T T T T T T o c 0 1 m r o F Z ✗ ✗ ✗ ✗ ✗ ✗ ✗ ✗ ✗ ✗ ✗ ✗ ✗ ✗ ✗ ✗ ✗ ✗ ✗ ✗ ✗ s e i t r e p o r p l a n o i t c n u f n o s t fi e n e b y g e t a r t s ✗ ✗ ✗ ✗ P ✗ ✗ ✗ ✗ ✗ ✗ ✗ ✗ + + C , re 2 2 h L L t a O M M ed M A rd ls , U U g U S n titecxuonoo liscep2LUM ,tilrsecehpE ,tlirsecehpE treh ,andPTQV ,.raephnTG ,tilrsecehpE treh ,trehTLO IF ,IF++C treh ,S++LTX treh liscep2LUM treh treh treh 2FTOM ,,TTLRVM itrcpFSO ,trehLUOM ,trehLUOM ,lcceeoLTA ,lrssaaoogC eenodC lcceeo ,lcceeoCCG E E O O O – X G O O A H H O C O E C O O O M Q M f f A P G A A s k n i s l ie ty teg ili a b r a ts ce n ra io T – ✗ ✗ ✗ ✗ ✗ ✗ ✗ ✗ ✗ ✗ ✗ ✗ – ✗ ✗ ✗ ✗ ✗ – – ✗ ✗ ✗ ✗ ✗ t u c e x e g n li rm e o d f o ta l M p e t l — e p )2 rg im a Q T – – – – – – – – – – – – – – – – – – – – – – – – – – – – – G R ( s c i t s i r e e t c in ra g a n h e c .r c l p o a L L a r h k ic te d o M M n n U U h I – A – – – – – – – – – – – – – – – M – – – – – f f – – – – – c e T 1 1 m i S ✗ ✗ ✗ ✗ ✗ ✗ ✗ ✗ ✗ ✗ ✗ ✗ ✗ ✗ ✗ ✗ w o d a h y t i c o l e V e h c a p s k n i l y t i l i b a e c a r M ✗ ✗ ✗ ✗ ✗ ✗ ✗ ✗ ✗ ✗ ✗ ✗ ✗ ✗ ✗ ✗ ✗ ✗ ✗ ✗ ✗ ✗ ✗ ✗ ✗ ✗ e n i g n d .e ta eu rp e sa co lo i in re A rm c h up T t t S e o d o C n n o I R K M A – – – – – – – – – – – – – P A – – – – – – – – – – c g n i g g u b e d l e d o s k n i l ✗ ✗ ✗ ✗ M ✗ ✗ ✗ ✗ ✗ ✗ + + t e n T ✗ ✗ ✗ ✗ ✗ ✗ ✗ ✗ – – ✗ ✗ ✗ m r o f t a l p t e g r a T – – – – – – – – – – – – – – – – – – – – – – – e n i g n e . r p r r L a d r e e k e te th th M o u n U n I – – – – – – – – – – – – – – O O f M – – – – – i t n o c 1 1 s t e g r a t r e h t m O r , fso r ++ r r ran teh ++ ,C teh ++ teh L D H V , L M X , L D C r H e m h V e t , ts O C C y , + re tem tem re ,S ## + h s s h + t y y t + ,C ,C T O – C C O C O J J O S S O C J C C – J C O A J – – ✗ ✗ – ✗ ✗ ✗ ✗ – – r e h t O r , e A th D m L a v M a J U tIrenm LUM – trehO SLDm trehO SLDm – trehO trehO trehO trehO trehO Jaav SLDm trehO LXM trehO – trehO ,trehO trehO ,trehO trehO – – e l fi l a u t x e l T e , d e o l fi m r o f t a l P ✗ ✗ ✗ ✗ – – – – ✗ ✗ ✗ ✗ ✗ ✗ – – + + L on D C r it H , M e a V C X th m , , , O r C C C , o L rfsan tseym treh LM treh treh treh treh LDH tseym tseym LDH treh treh ,XM ++ T S O J X O – – – – O O J O V J S S V J O O J – – C s p e t s e l fi m rm Lm re re re re re re L re re re re re re L re L re L te S th th th th th th M th th th th th th M th M th M In D O O O O – – – – O O U O O O O O O U O X O – – X n o i t a l s n a r T 2 2 1 1 1 – – – – 1 1 2 2 2 1 1 2 1 3 1 1 1 – – 2 d e u n i t n o c m e t s y O , n o h t y B V A ,J O D , tifrrsaaonnom ++ ,,,JSPLELDW treh treh ,,trehTLHOM treh treh ,,,S++LCVHD te ,,,JPPP++H ,trJeh ,++CAD ,,,,##++CCCA ,,treh++CAOD ,,##++CC ,,,trJeh++CC ,,,##++CCC ,,,J++CADA ,,tseyS++CCm ++ ++ ,++C ,,,##++CCA ++ T C J B O O J J O O C J J J .n C O A J J A J O J C C – – C C C C C s t c a f i t r a s p e t s O , C m e t s y S , L H V , B V , n o h t y P , P H P , J , i h p l e D , t p i r c S n o i t c l e d o l e d o l e d o m L S D , l e d o n o i t a ls r r r r r r r r n e e e e e e e e ra th th th th th th th th T 1 1 3 1 1 2 3 1 1 2 2 2 1 O 1 O 3 1 O O O O O 3 1 – – 2 2 1 O 1 Table 13 Provided evidence and quality assessment scores (RQ3), identified limitations (RQ4) Identified limitations Table 13 continued Research method Type of evidence Type of systems for evidence P01 V Table 14 Quality assessment scores (RQ3) Table 14 continued Q1 Q2 Q3 Q4 Q5 Q6 Total Q1 1 0.5 1 1 0.5 1 1 1 1 1 1 1 1 0.5 1 0.5 0.5 1 1 1 0.5 0 N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A Q2 0 0 0 0 0 0 0 0 0 0 0.5 0 0.5 0 1 0 0 0.5 0.5 0.5 0 0 N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A Q3 0.5 0.5 0 0.5 1 1 0.5 0 0.5 0 0.5 0.5 0.5 0.5 1 0.5 0.5 0.5 1 1 0 0.5 N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A Q4 1 1 1 1 1 1 0.5 1 1 1 0.5 1 0.5 1 1 1 1 0.5 1 1 1 1 N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A Q5 0 0 0 0 0.5 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0.5 0 0 N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A Q6 0 0 0 0.5 1 0 0 0.5 0 0.5 0 0 0 0 0 0 0 0 0.5 1 1 0.5 N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A Total 2.5 2 2 3 4 3 2 2.5 2.5 2.5 2.5 2.5 2.5 2 5 2 2 2.5 4 5 2.5 2 N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A 1. Abouzahra , A. , Bézivin , J. , Del Fabro , M.D. , Jouault , F. : A practical approach to bridging domain specific languages with UML profiles . In: Proceedings of the Best Practices for Model Driven Software Development at OOPSLA , vol. 5 . Citeseer ( 2005 ) 2. Abrial , J.-R.: Modeling in Event-B: System and Software Engineering . Cambridge University Press, Cambridge ( 2010 ) 3. Agresti , A. , Kateri , M. : Categorical Data Analysis . Springer, Berlin ( 2011 ) 4. Aizenbud-Reshef , N. , Nolan , B.T. , Rubin , J. , Shaham-Gafni , Y. : Model traceability . IBM Syst. J . 45 ( 3 ), 515 - 526 ( 2006 ) 5. Ali , M.S. , Babar , M.A. , Chen , L. , Stol , K.-J.: A systematic review of comparative evidence of aspect-oriented programming . Inf. Softw. Technol . 52 ( 9 ), 871 - 887 ( 2010 ) 6. Ali , N.B. , Petersen , K. : Evaluating strategies for study selection in systematic literature studies . In: International Symposium on Empirical Software Engineering and Measurement . ACM ( 2014 ) 7. Aljer , A. , Devienne , P. , Tison , S. , Boulanger , J.-L. , Mariano , G.: BHDL: Circuit design in B . In: Proceedings. Third International Conference on Application of Concurrency to System Design , 2003 , pp. 241 - 242 . IEEE ( 2003 ) 8. Charfi , A. , Mraidha , C. , Gérard , S. , Terrier , F. , Boulet , P. : Toward optimized code generation through model-based optimization . In: Proceedings of the Conference on Design, Automation and Test in Europe , pp. 1313 - 1316 ( 2010 ) 9. Ciccozzi , F. : Unicomp: a semantics-aware model compiler for optimised predictable software . In: International Conference on Software Engineering (ICSE ) 2018 - New Ideas and Emerging Results (NIER) , May 2018 . UML, Alf, fUML, compilation, modeldriven engineering, predictability, semantics 10. Ciccozzi , F. , Cicchetti , A. , Sjödin , M. : Round-trip support for extrafunctional property management in model-driven engineering of embedded systems . Inf. Softw. Technol . 55 ( 6 ), 1085 - 1100 ( 2013 ) 11. Cimatti , A. , Clarke , E. , Giunchiglia , E. , Giunchiglia , F. , Pistore , M. , Roveri , M. , Sebastiani , R. , Tacchella , A. : Nusmv 2: an opensource tool for symbolic model checking . In: Brinksma, E. , Larsen , K.G. (eds.) Computer Aided Verification, pp. 359 - 364 . Springer, Heidelberg ( 2002 ) 12. Cohen , J. : Weighted kappa: nominal scale agreement provision for scaled disagreement or partial credit . Psychol. Bull . 70 ( 4 ), 213 ( 1968 ) 13. Corbin , J.M. , Strauss , A. : Basics of Qualitative Research: Techniques and Procedures for Developing Grounded Theory . Sage Publications, Thousand Oaks ( 2014 ) 14. Cruzes , D.S. , Dybå , T.: Research synthesis in software engineering: a tertiary study . Inf. Softw. Technol . 53 ( 5 ), 440 - 455 ( 2011 ) 15. Di Francesco , P. , Malavolta , I. , Lago , P.: Research on architecting microservices: trends, focus, and potential for industrial adoption . In: 2017 IEEE International Conference on Software Architecture (ICSA) , pp. 21 - 30 . IEEE ( 2017 ) 16. Dragoni , N. , Giallorenzo , S. , Lafuente , A.L. , Mazzara , M. , Montesi , F. , Mustafin , R. , Safina , L. : Microservices: yesterday, today, and tomorrow . In: Mazzara, M. , Meyer, M. (eds.) Present and Ulterior Software Engineering, pp. 195 - 216 . Springer, Cham ( 2017 ) 17. Dybå , T. , Dingsøyr , T. : Empirical studies of agile software development: a systematic review . Inf. Softw. Technol . 50 ( 9 ), 833 - 859 ( 2008 ) 18. Franzosi , R.: Quantitative Narrative Analysis , vol. 162 . Sage , Thousand Oaks ( 2010 ) 19. Galster , M. , Weyns , D. , Tofan , D. , Michalik , B. , Avgeriou , P. : Variability in software systems: systematic literature review . IEEE Trans. Softw. Eng . 40 ( 3 ), 282 - 306 ( 2014 ) 20. Gotti , S. , Mbarki , S.: UML executable: a comparative study of UML compilers and interpreters . In: 2016 International Conference on Information Technology for Organizations Development (IT4OD) , pp. 1 - 5 ( March 2016 ) 21. Grandy , H. , Bischof , M. , Stenzel , K. , Schellhorn , G. , Reif , W. : Verification of Mondex electronic purses with KIV: from a security protocol to verified code . In: Woodcock, J . (ed.) FM 2008 : Formal Methods , pp. 165 - 180 . Springer ( 2008 ) 22. Greenhalgh , T. , Peacock , R.: Effectiveness and efficiency of search methods in systematic reviews of complex evidence: audit of primary sources . BMJ 331 ( 7524 ), 1064 - 1065 ( 2005 ) 23. Hrischuk , C. , Rolia , J. , Woodside , C.M.: Automatic generation of a software performance model using an object-oriented prototype . In: Proceedings of the Third International Workshop on Modeling, Analysis, and Simulation of Computer and Telecommunication Systems , 1995 . MASCOTS' 95 , pp. 399 - 409 . IEEE ( 1995 ) 24. Hutchinson , J. , Whittle , J. , Rouncefield , M. : Model-driven engineering practices in industry: social, organizational and managerial factors that lead to success or failure . Sci. Comput . Program. 89 , 144 - 161 ( 2014 ) 25. Hutchinson , J. , Whittle , J. , Rouncefield , M. , Kristoffersen , S. : Empirical assessment of MDE in industry . In: Proceedings of ICSE. ACM ( 2011 ) 26. Iqbal , M.Z. , Arcuri , A. , Briand , L. : Environment modeling and simulation for automated testing of soft real-time embedded software . Softw. Syst. Model . 14 ( 1 ), 483 - 524 ( 2015 ) 27. Kitchenham , B. , Brereton , P.: A systematic review of systematic review process research in software engineering . Inf. Softw. Technol . 55 ( 12 ), 2049 - 2075 ( 2013 ) 28. Kitchenham , B.A. , Charters , S. : Guidelines for performing systematic literature reviews in software engineering . Technical Report EBSE-2007-01 , Keele University and University of Durham ( 2007 ) 29. Kleppe , A.G. , Warmer , J. , Bast , W. : MDA Explained-The Model Driven Architecture: Practice and Promise . Addison-Wesley Professional , Reading ( 2003 ) 30. Landis , J.R. , Koch , G.G. : The measurement of observer agreement for categorical data . Biometrics 33 , 159 - 174 ( 1977 ) 31. Laurent , Y. , Bendraou , R. , Gervais , M.-P.: Executing and debugging UML models: an fUML extension . In: Proceedings of the 28th Annual ACM Symposium on Applied Computing , pp. 1095 - 1102 . ACM ( 2013 ) 32. Lee , E.A. , Seshia , S.A. : Introduction to Embedded Systems: A Cyber-Physical Systems Approach . MIT Press, Cambridge ( 2011 ) 33. Malavolta , I. , Lago , P. , Muccini , H. , Pelliccione , P. , Tang , A. : What industry needs from architectural languages: a survey . IEEE Trans. Softw. Eng . 39 ( 6 ), 869 - 891 ( 2013 ) 34. MARTE profile. http://www.omg.org/spec/MARTE/1.1/. Latest access: 20 Nov 2017 35. Mayerhofer , T. , Langer , P. , Wimmer , M. , Kappel , G.: xMOF: Executable DSMLs based on fUML . In: Proceedings of SLE ( 2013 ) 36. Mellor , S.J. , Tockey , S. , Arthaud , R. , Leblanc , P.: An action language for UML: proposal for a precise execution semantics . In: Bezivin, J . (ed.) The Unified Modeling Language.UML98: Beyond the Notation , pp. 307 - 318 . Springer ( 1998 ) 37. Meyes , S. : The Most Important C++ Software... Ever ( 2006 ) 38. Montesi , F. , Guidi , C. , Zavattaro , G.: Composing services with JOLIE . In: Fifth European Conference on Web Services , 2007 . ECOWS' 07 , pp. 13 - 22 . IEEE ( 2007 ) 39. Petersen , K. , Feldt , R. , Mujtaba , S. , Mattsson , M. : Systematic mapping studies in software engineering . In: Proceedings of the 12th International Conference on Evaluation and Assessment in Software Engineering, EASE'08 , pp. 68 - 77 , Swinton, UK ( 2008 ) British Computer Society 40. Petersen , K. , Vakkalanka , S. , Kuzniarz , L. : Guidelines for conducting systematic mapping studies in software engineering: an update . Inf. Softw. Technol . 64 , 1 - 18 ( 2015 ) 41. Peterson , J.L. : Petri Net Theory and the Modeling of Systems . Prentice-Hall, Englewood Cliffs ( 1981 ) 42. Potter , B. , Till , D. , Sinclair , J.: An Introduction to Formal Specification and Z. Prentice Hall PTR , Englewood Cliffs ( 1996 ) 43. Rodgers , M. , Sowden , A. , Petticrew , M. , Arai , L. , Roberts , H. , Britten , N. , Popay , J.: Testing methodological guidance on the conduct of narrative synthesis in systematic reviews effectiveness of interventions to promote smoke alarm ownership and function . Evaluation 15 ( 1 ), 49 - 73 ( 2009 ) 44. Schmidt , D.C. : Guest editor's introduction: model-driven engineering . Computer 39 ( 2 ), 25 - 31 ( 2006 ) 45. Selic , B. : The less well known UML . In: Bernardo, M. , Cortellessa , V. , Pierantonio , A . (eds.) Formal Methods for Model-Driven Engineering . Volume 7320 of Lecture Notes in Computer Science, pp. 1 - 20 . Springer, Berlin ( 2012 ) 46. Steele , G. : Common LISP: The Language . Elsevier, London ( 1990 ) 47. Tatibouët , J. , Cuccuru , A. , Gérard , S. , Terrier , F. : Formalizing execution semantics of UML profiles with fUML models . In: Model-Driven Engineering Languages and Systems , pp. 133 - 148 . Springer ( 2014 ) 48. Tatibouët , J. , Cuccuru , A. , Gérard , S. , Terrier , F. : Formalizing execution semantics of UML profiles with fUML models . In: Proceedings of MODELS , pp. 133 - 148 ( 2014 ) 49. Wohlin , C. : Guidelines for snowballing in systematic literature studies and a replication in software engineering . In: Proceedings of the 18th International Conference on Evaluation and Assessment in Software Engineering , p. 38 . ACM ( 2014 ) 50. Wohlin , C. : Guidelines for snowballing in systematic literature studies and a replication in software engineering . In: Proceedings of the 18th International Conference on Evaluation and Assessment in Software Engineering, EASE '14 , pp. 38 : 1 - 38 : 10 . ACM , New York ( 2014 ) 51. Wohlin , C. , Runeson , P. , Höst , M. , Ohlsson , M. , Regnell , B. , Wesslén , A. : Experimentation in Software Engineering . Computer Science. Springer, Berlin ( 2012 ) 52. Zhang , H., Babar , M.A. : Systematic reviews in software engineering: an empirical investigation . Inf. Softw. Technol . 55 ( 7 ), 1341 - 1354 ( 2013 ) 53. Zurowska , K. , Dingel , J.: A customizable execution engine for models of embedded systems . In: Roubtsova, E. , McNeile , A. , Kindler , E. , Gerth , C. (eds.) Behavior Modeling-Foundations and Applications , pp. 82 - 110 . Springer, Berlin ( 2015 ) Ivano Malavolta is Assistant Professor at the Vrije Universiteit Amsterdam, The Netherlands . His research focuses on software architecture, model-driven engineering (MDE), and mobile-enabled systems, especially how MDE techniques can be exploited for architecting complex and mobileenabled software systems at the right level of abstraction. He is program committee member and reviewer of international conferences and journals in his fields of interest. He is applying empirical methods to assess practices and trends in the field of software engineering. He authored more than 70 papers in international journals and peer-reviewed international conferences proceedings . He received a Ph.D. in computer science from the University of L'Aquila in 2012 . He is a member of ACM and IEEE . More information is available at http://www.ivanomalavolta.com.


This is a preview of a remote PDF: https://link.springer.com/content/pdf/10.1007%2Fs10270-018-0675-4.pdf

Federico Ciccozzi, Ivano Malavolta, Bran Selic. Execution of UML models: a systematic review of research and practice, Software & Systems Modeling, 2018, 1-48, DOI: 10.1007/s10270-018-0675-4