Context: With the advent of Machine Learning (ML) and especially Deep Learning (DL) technology, companies are increasingly using Artificial Intelligence (AI) in systems, along with electronics and software. Nevertheless, the end-to-end process of developing, deploying and evolving ML and DL models in companies brings some challenges related to the design and scaling of these models. For example, access to and availability of data is often challenging, and activities such as collecting, cleaning, preprocessing, and storing data, as well as training, deploying and monitoring the model(s) are complex. Regardless of the level of expertise and/or access to data scientists, companies in all embedded systems domain struggle to build high-performing models due to a lack of established and systematic design methods and processes.

Objective: The overall objective is to establish systematic and structured design methods and processes for the end-to-end process of developing, deploying and successfully evolving ML/DL models.

Method: To achieve the objective, we conducted our research in close collaboration with companies in the embedded systems domain using different empirical research methods such as case study, action research and literature review.

Results and Conclusions: This research provides six main results: First, it identifies the activities that companies undertake in parallel to develop, deploy and evolve ML/DL models, and the challenges associated with them. Second, it presents a conceptual framework for the continuous delivery of ML/DL models to accelerate AI-driven business in companies. Third, it presents a framework based on current literature to accelerate the end-to-end deployment process and advance knowledge on how to integrate, deploy and operationalize ML/DL models. Fourth, it develops a generic framework with five architectural alternatives for deploying ML/DL models at the edge. These architectural alternatives range from a centralized architecture that prioritizes (re)training in the cloud to a decentralized architecture that prioritizes (re)training at the edge. Fifth, it identifies key factors to help companies decide which architecture to choose for deploying ML/DL models. Finally, it explores how MLOps, as a practice that brings together data scientist teams and operations, ensures the continuous delivery and evolution of models. 

Context: Digitalisation is transforming software-intensive embedded systems companies by focussing on business models that utilise software, data and AI (especially Machine learning and Deep Learning (DL)). However, despite these advancements, the majority of companies still struggle to transition their models from prototypes to fully functional operational systems. This highlights the need to optimise the end-to-end process of developing, deploying and evolving ML/DL models to ensure continuous value delivery.

Objective: This thesis is structured around three primary objectives. The first objective is to identify the need of MLOps (Machine Learning Operations). Building on this understanding, the second objective is to develop frameworks for the adoption of MLOps, aiming to standardise and streamline the processes of developing, deploying and evolving ML/DL models. Finally, the third objective is to adopt MLOps practices and assess the maturity of their adoption.

Method: To achieve these objectives, we conducted research in close collaboration with various companies and used a combination of different empirical research methods, such as case studies, action research, and literature reviews.

Results and Conclusions: First, the thesis identifies the activities carried out by practitioners in companies and the challenges they face when developing, deploying and evolving models. Second, it presents a conceptual framework with three parallel and concurrent activities that companies utilise in the process of developing, deploying and evolving models. Third, it introduces a framework based on current literature to accelerate and advance knowledge on the end-to-end deployment process. Fourth, it develops a generic framework with five architectural alternatives ranging from a centralised architecture to a decentralised architecture for deploying ML/DL models at the edge. It also identifies key factors that help companies overcome their dilemma to decide which architecture to choose for deploying ML/DL models. Five, it explores how MLOps, as a practice, brings together data scientist teams and operations to ensure the continuous delivery and evolution of models. Sixth, it presents the MLOps framework, maps companies to the MLOps maturity model, and validates the MLOps framework and maturity model with other companies. It also presents critical trade-offs that practitioners made when adopting MLOps. Seventh, it presents an MLOps taxonomy that helps companies determine their maturity stage and provide tailored MLOps practices to advance.