Artificial intelligence (AI) and the use of machine learning (ML) and deep learning (DL) technologies are becoming increasingly popular in companies. These technologies enable companies to leverage big quantities of data to improve system performance and accelerate business development. However, despite the appeal of ML/DL, there is a lack of systematic and structured methods and processes to help data scientists and other company roles and functions to develop, deploy and evolve models. In this paper, based on multi-case study research in six companies, we explore practices and challenges practitioners experience in developing ML/DL models as part of large software-intensive embedded systems. Based on our empirical findings, we derive a conceptual framework in which we identify three high-level activities that companies perform in parallel with the development, deployment and evolution of models. Within this framework, we outline activities, iterations and triggers that optimize model design as well as roles and company functions. In this way, we provide practitioners with a blueprint for effectively integrating ML/DL model development into the business to achieve better results than other (algorithmic) approaches. In addition, we show how this framework helps companies solve the challenges we have identified and discuss checkpoints for terminating the business case.

Context: With the advent of Machine Learning (ML) and especially Deep Learning (DL) technology, companies are increasingly using Artificial Intelligence (AI) in systems, along with electronics and software. Nevertheless, the end-to-end process of developing, deploying and evolving ML and DL models in companies brings some challenges related to the design and scaling of these models. For example, access to and availability of data is often challenging, and activities such as collecting, cleaning, preprocessing, and storing data, as well as training, deploying and monitoring the model(s) are complex. Regardless of the level of expertise and/or access to data scientists, companies in all embedded systems domain struggle to build high-performing models due to a lack of established and systematic design methods and processes.

Objective: The overall objective is to establish systematic and structured design methods and processes for the end-to-end process of developing, deploying and successfully evolving ML/DL models.

Method: To achieve the objective, we conducted our research in close collaboration with companies in the embedded systems domain using different empirical research methods such as case study, action research and literature review.

Results and Conclusions: This research provides six main results: First, it identifies the activities that companies undertake in parallel to develop, deploy and evolve ML/DL models, and the challenges associated with them. Second, it presents a conceptual framework for the continuous delivery of ML/DL models to accelerate AI-driven business in companies. Third, it presents a framework based on current literature to accelerate the end-to-end deployment process and advance knowledge on how to integrate, deploy and operationalize ML/DL models. Fourth, it develops a generic framework with five architectural alternatives for deploying ML/DL models at the edge. These architectural alternatives range from a centralized architecture that prioritizes (re)training in the cloud to a decentralized architecture that prioritizes (re)training at the edge. Fifth, it identifies key factors to help companies decide which architecture to choose for deploying ML/DL models. Finally, it explores how MLOps, as a practice that brings together data scientist teams and operations, ensures the continuous delivery and evolution of models. 

The adoption of continuous software engineering practices such as DevOps (Development and Operations) in business operations has contributed to significantly shorter software development and deployment cycles. Recently, the term MLOps (Machine Learning Operations) has gained increasing interest as a practice that brings together data scientists and operations teams. However, the adoption of MLOps in practice is still in its infancy and there are few common guidelines on how to effectively integrate it into existing software development practices. In this paper, we conduct a systematic literature review and a grey literature review to derive a framework that identifies the activities involved in the adoption of MLOps and the stages in which companies evolve as they become more mature and advanced. We validate this framework in three case companies and show how they have managed to adopt and integrate MLOps in their large-scale software development companies. The contribution of this paper is threefold. First, we review contemporary literature to provide an overview of the state-of-the-art in MLOps. Based on this review, we derive an MLOps framework that details the activities involved in the continuous development of machine learning models. Second, we present a maturity model in which we outline the different stages that companies go through in evolving their MLOps practices. Third, we validate our framework in three embedded systems case companies and map the companies to the stages in the maturity model. 

Machine learning and deep learning techniques are becoming increasingly popular and critical for companies as part of their systems. However, although the development and prototyping of ML/DL systems are common across companies, the transition from prototype to production-quality deployment models are challenging. One of the key challenges is how to determine the selection of an optimal architecture for AI deployment. Based on our previous research, and to offer support and guidance to practitioners, we developed a framework in which we present five architectural alternatives for AI deployment ranging from centralized to fully decentralized edge architectures. As part of our research, we validated the framework in software-intensive embedded system companies and identified key challenges they face when deploying ML/DL models. In this paper, and to further advance our research on this topic, we identify factors that help practitioners determine what architecture to select for the ML/D L model deployment. For this, we conducted a follow-up study involving interviews and workshops in seven case companies in the embedded systems domain. Based on our findings, we identify three key factors and develop a framework in which we outline how prioritization and trade-offs between these result in certain architecture. The contribution of the paper is threefold. First, we identify key factors critical for AI system deployment. Second, we present the architecture selection framework that explains how prioritization and trade-offs between key factors result in the selection of a certain architecture. Third, we discuss additional factors that may or may not influence the selection of an optimal architecture.

Since the advent of mobile computing and IoT, a large amount of data is distributed around the world. Companies are increasingly experimenting with innovative ways of implementing edge/cloud (re)training of AI systems to exploit large quantities of data to optimize their business value. Despite the obvious benefits, companies face challenges as the decision on how to implement edge/cloud (re)training depends on factors such as the task intent, the amount of data needed for (re)training, edge-to-cloud data transfer, the available computing and memory resources. Based on action research in a software-intensive embedded systems company where we study multiple use cases as well as insights from our previous collaborations with industry, we develop a generic framework consisting of five architectural alternatives to deploy AI on the edge utilizing transfer learning. We validate the framework in four additional case companies and present the challenges they face in selecting the optimal architecture. The contribution of the paper is threefold. First, we develop a generic framework consisting of five architectural alternatives ranging from a centralized architecture where cloud (re)training is given priority to a decentralized architecture where edge (re)training is instead given priority. Second, we validate the framework in a qualitative interview study with four additional case companies. As an outcome of validation study, we present two variants to the architectural alternatives identified as part of the framework. Finally, we identify the key challenges that experts face in selecting an ideal architectural alternative.

Companies across domains are rapidly engaged in shifting computational power and intelligence from centralized cloud to fully decentralized edges to maximize value delivery, strengthen security and reduce latency. However, most companies have only recently started pursuing this opportunity and are therefore at the early stage of the cloud-to-edge transition. To provide an overview of AI deployment in the context of edge/cloud/hybrid architectures, we conduct a systematic literature review and a grey literature review. To advance understanding of how to integrate, deploy, operationalize and evolve AI models, we derive a framework from existing literature to accelerate the end-to-end deployment process. The framework is organized into five phases: Design, Integration, Deployment, Operation and Evolution. We make an attempt to analyze the extracted results by comparing and contrasting them to derive insights. The contribution of the paper is threefold. First, we conduct a systematic literature review in which we review the contemporary scientific literature and provide a detailed overview of the state-of-the-art of AI deployment. Second, we review the grey literature and present the state-of-practice and experience of practitioners while deploying AI models. Third, we present a framework derived from existing literature for the end-to-end deployment process and attempt to compare and contrast SLR and GLR results.

Artificial Intelligence is becoming increasingly popular with organizations due to the success of Machine Learning and Deep Learning techniques. Using these techniques, data scientists learn from vast amounts of data to enhance behaviour in software-intensive systems. Despite the attractiveness of these techniques, however, there is a lack of systematic and structured design process for developing ML/DL models. The study uses a multiple-case study approach to explore the different activities and challenges data scientists face when developing ML/DL models in software-intensive embedded systems. In addition, we have identified seven different phases in the proposed design process leading to effective model development based on the case study. Iterations identified between phases and events which trigger these iterations optimize the design process for ML/DL models. Lessons learned from this study allow data scientists and engineers to develop high-performance ML/DL models and also bridge the gap between high demand and low supply of data scientists.