Context:While continuous deployment is popular among web-based software development organizations, adopting continuous deployment in software-intensive system-of-systems is more challenging. On top of the challenges arising from deploying software to a single software-intensive embedded system, software-intensive system-of-systems (SiSoS) add a layer of complexity as new software undergoes an extensive field validation applied to individual components of the SiSoS, as well as the overall SiSoS, to ensure that both legacy and new functionalities are working as desired.

Objectives:This paper aims to study how SiSoS transitions to continuous deployment by exploring how continuous deployment impacts field testing and validation activities, how continuous deployment can be practiced in SiSoS, and to identify the success factors that companies need to consider when transitioning to continuous deployment.

Method:We conducted a case study at Ericsson AB focusing on the embedded software of the Third Generation Radio Access Network (3G RAN). The 3G RAN consists of two large-scale software-intensive embedded systems, representing a simple SiSoS composed of two systems. 3G RAN software was the first to transition to continuous deployment and is used as a reference case for other products within Ericsson AB.

Results:Software deployment, in addition to field testing and validation, have transitioned from being a discrete activity performed at the end of software development to a continuous process performed in parallel to software development. Further, our study reveals an orchestrating approach for software deployment, which allows pre/post validation of legacy behavior and new features in a shorter release and deployment cadence. Furthermore, we identified the essential success factors that organizations should consider when transitioning to continuous deployment.

Conclusion:Transition to continuous deployment, in addition to field testing and validation, shall be considered and planned carefully. In this paper, we provide a set of success factors and orchestration technique that helps organization when transitioning to continuous deployment in the software-intensive embedded system-of-systems context.

Context: Nowadays, the hype around artificial intelligence is at its absolute peak. Large amounts of data are collected every second of the day and a variety of tools exists to enable easy analysis of data. In practice, however, making meaningful use of it is way more challenging. For instance, affected stakeholders often struggle to specify their information needs and to interpret the results of such analyses. Objective: In this study we investigate how to enable continuous monitoring of information needs, and the generation of knowledge and insights for various stakeholders involved in the lifecycle of software-intensive products. The overarching goal is to support their decision making by providing relevant insights related to their area of responsibility. Methods: We implement multiple monitoringand decision-making frameworks for six individual, real-world cases selected from three different platforms and covering four types of stakeholders. We compare the individual procedures to derive a generic process for instantiating such frameworks as well as a model to scale it up for multiple stakeholders. Results: For one, we discovered that information needs of stakeholders are often related to a limited subset of data sources and should be specified in stages. For another, stakeholders often benefit from sharing and reusing existing components among themselves in later phases. Specifically, we identify three types of reuse: (1) Data and knowledge, (2) tools and methods, and (3) concepts. As a result, key aspects of our model are iterative feedback and specification cycles as well as the reuse of appropriate components to speed up the instantiation process and maximize the efficiency of the model. Conclusion: Our results indicate that knowledge and insights can be generated much faster and stakeholders feel the benefits of the analysis very early on by iteratively specifying information needs and by systematically sharing and reusing knowledge, tools and concepts.

Background : Developing and maintaining large scale machine learning (ML) based software systems in an in-dustrial setting is challenging. There are no well-established development guidelines, but the literature contains reports on how companies develop and maintain deployed ML-based software systems. Objective : This study aims to survey the literature related to development and maintenance of large scale ML -based systems in industrial settings in order to provide a synthesis of the challenges that practitioners face. In addition, we identify solutions used to address some of these challenges. Method : A systematic literature review was conducted and we identified 72 papers related to development and maintenance of large scale ML-based software systems in industrial settings. The selected articles were qualita-tively analyzed by extracting challenges and solutions. The challenges and solutions were thematically synthe-sized into four quality attributes: adaptability, scalability, safety and privacy. The analysis was done in relation to ML workflow, i.e. data acquisition, training, evaluation, and deployment. Results : We identified a total of 23 challenges and 8 solutions related to development and maintenance of large scale ML-based software systems in industrial settings including six different domains. Challenges were most often reported in relation to adaptability and scalability. Safety and privacy challenges had the least reported solutions. Conclusion : The development and maintenance on large-scale ML-based systems in industrial settings introduce new challenges specific for ML, and for the known challenges characteristic for these types of systems, require new methods in overcoming the challenges. The identified challenges highlight important concerns in ML system development practice and the lack of solutions point to directions for future research.

Delivering faster value to customers with online experimentation is an emerging practice in industry. Multi-Armed Bandit (MAB) based experiments have the potential to deliver even faster results with a better allocation of resources over traditional A/B experiments. However, the incorrect use of MAB-based experiments can lead to incorrect conclusions that can potentially hurt the company's business. The objective of this study is to understand the pitfalls and restrictions of using MABs in online experiments, as well as the strategies that are used to overcome them. This research uses a multiple case study method with eleven experts across five software companies and simulations to triangulate the data of some of the identified limitations. This study analyzes some limitations faced by companies using MAB and discusses strategies used to overcome them. The results are summarized into practitioners’ guidelines with criteria to select an appropriated experimental design. MAB algorithms have the potential to deliver even faster results with a better allocation of resources over traditional A/B experiments. However, potential mistakes can occur and hinder the potential benefits of such approach. Together with the provided guidelines, we aim for this paper to be used as reference material for practitioners during the design of an online experiment.

Context

Test driven development (TDD) has been extensively researched and compared to traditional approaches (test last development, TLD). Existing literature reviews show varying results for TDD.

Objective

This study investigates how the conclusions of existing literature reviews change when taking two study quality dimension into account, namely rigor and relevance.

Method

In this study a systematic literature review has been conducted and the results of the identified primary studies have been analyzed with respect to rigor and relevance scores using the assessment rubric proposed by Ivarsson and Gorschek 2011. Rigor and relevance are rated on a scale, which is explained in this paper. Four categories of studies were defined based on high/low rigor and relevance.

Results

We found that studies in the four categories come to different conclusions. In particular, studies with a high rigor and relevance scores show clear results for improvement in external quality, which seem to come with a loss of productivity. At the same time high rigor and relevance studies only investigate a small set of variables. Other categories contain many studies showing no difference, hence biasing the results negatively for the overall set of primary studies. Given the classification differences to previous literature reviews could be highlighted.

Conclusion

Strong indications are obtained that external quality is positively influenced, which has to be further substantiated by industry experiments and longitudinal case studies. Future studies in the high rigor and relevance category would contribute largely by focusing on a wider set of outcome variables (e.g. internal code quality). We also conclude that considering rigor and relevance in TDD evaluation is important given the differences in results between categories and in comparison to previous reviews.

Context

The increased use of Open Source Software (OSS) affects how software-intensive product development organizations (SIPDO) innovate and compete, moving them towards Open Innovation (OI). Specifically, software engineering tools have the potential for OI, but require better understanding regarding what to develop internally and what to acquire from outside the organization, and how to cooperate with potential competitors.

Aim

This paper aims at synthesizing a theory of openness for software engineering tools in SIPDOs, that can be utilized by managers in defining more efficient strategies towards OSS communities.

Method

We synthesize empirical evidence from a systematic mapping study, a case study, and a survey, using a narrative method. The synthesis method entails four steps: (1) Developing a preliminary synthesis, (2) Exploring the relationship between studies, (3) Assessing the validity of the synthesis, and (4) Theory formation.

Result

We present a theory of openness for OSS tools in software engineering in relation to four constructs: (1) Strategy, (2) Triggers, (3) Outcomes, and (4) Level of openness.

Conclusion

The theory reasons that openness provides opportunities to reduce the development cost and development time. Furthermore, OI positively impacts on the process and product innovation, but it requires investment by organizations in OSS communities. By betting on openness, organizations may be able to significantly increase their competitiveness.

Context: The role of product management (PM) is key for building, implementing and managing softwareintensive systems. Whereas engineering is concerned with how to build systems, PM is concerned with 'what' to build and 'why' we should build the product. The role of PM is recognized as critical for the success of any product. However, few studies explore how the role of PM is changing due to recent trends that come with digitalization and digital transformation. Objectives: Although there is prominent research on PM, few studies explore how this role is changing due to the digital transformation of the software-intensive industry. In this paper, we study how trends such as DevOps and short feedback loops, data and artificial intelligence (AI), as well as the emergence of digital ecosystems, are changing current product management practices. Methods: This study employs a qualitative approach using multi-case study research as the method. For our research, we selected five case companies in the software-intensive systems domain. Through workshop sessions, frequent meetings and interviews, we explore how DevOps and short feedback loops, data and artificial intelligence (AI), and digital ecosystems challenge current PM practices. Results: Our study yielded an in-depth understanding of how digital transformation of the software-intensive systems industry is changing current PM practices. We present empirical results from workshops and from interviews in which case company representatives share their insights on how software, data and AI impact current PM practices. Based on these results, we present a framework organized along two dimensions, i.e. a certainty dimension and an approach dimension. The framework helps structure the approaches product managers can employ to select and prioritize development of new functionality. Contributions: The contribution of this paper is a framework for 'Strategic Digital Product Management' (SDPM). The framework outlines nine approaches that product managers can employ to maximize the return on investment (RoI) of R&D using new digital technologies.

Context: Federated Learning (FL) has gained prominence as a solution for preserving data privacy in machine learning applications. However, existing FL frameworks pose challenges for software engineers due to implementation complexity, limited customization options, and scalability issues. These limitations prevent the practical deployment of FL, especially in dynamic and resource-constrained edge environments, preventing its widespread adoption. Objective: To address these challenges, we propose EdgeFL, an efficient and low-effort FL framework designed to overcome centralized aggregation, implementation complexity and scalability limitations. EdgeFL applies a decentralized architecture that eliminates reliance on a central server by enabling direct model training and aggregation among edge nodes, which enhances fault tolerance and adaptability to diverse edge environments. Methods: We conducted experiments and a case study to demonstrate the effectiveness of EdgeFL. Our approach focuses on reducing weight update latency and facilitating faster model evolution on edge devices. Results: Our findings indicate that EdgeFL outperforms existing FL frameworks in terms of learning efficiency and performance. By enabling quicker model evolution on edge devices, EdgeFL enhances overall efficiency and responsiveness to changing data patterns. Conclusion: EdgeFL offers a solution for software engineers and companies seeking the benefits of FL, while effectively overcoming the challenges and privacy concerns associated with traditional FL frameworks. Its decentralized approach, simplified implementation, combined with enhanced customization and fault tolerance, make it suitable for diverse applications and industries.