Malmö University Publications
Change search
Refine search result
123 1 - 50 of 142
CiteExportLink to result list
Permanent link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Rows per page
  • 5
  • 10
  • 20
  • 50
  • 100
  • 250
Sort
  • Standard (Relevance)
  • Author A-Ö
  • Author Ö-A
  • Title A-Ö
  • Title Ö-A
  • Publication type A-Ö
  • Publication type Ö-A
  • Issued (Oldest first)
  • Issued (Newest first)
  • Created (Oldest first)
  • Created (Newest first)
  • Last updated (Oldest first)
  • Last updated (Newest first)
  • Disputation date (earliest first)
  • Disputation date (latest first)
  • Standard (Relevance)
  • Author A-Ö
  • Author Ö-A
  • Title A-Ö
  • Title Ö-A
  • Publication type A-Ö
  • Publication type Ö-A
  • Issued (Oldest first)
  • Issued (Newest first)
  • Created (Oldest first)
  • Created (Newest first)
  • Last updated (Oldest first)
  • Last updated (Newest first)
  • Disputation date (earliest first)
  • Disputation date (latest first)
Select
The maximal number of hits you can export is 250. When you want to export more records please use the Create feeds function.
  • 1. Alégroth, Emil
    et al.
    Feldt, Robert
    Olsson Holmström, Helena
    Malmö högskola, Faculty of Technology and Society (TS), Department of Computer Science (DV).
    Transitioning Manual System Test Suites to Automated Testing: An Industrial Case Study2013In: 2013 IEEE Sixth International Conference on Software Testing, Verification and Validation, IEEE, 2013, p. 56-65Conference paper (Refereed)
    Abstract [en]

    Visual GUI testing (VGT) is an emerging technique that provides software companies with the capability to automate previously time-consuming, tedious, and fault prone manual system and acceptance tests. Previous work on VGT has shown that the technique is industrially applicable, but has not addressed the real-world applicability of the technique when used by practitioners on industrial grade systems. This paper presents a case study performed during an industrial project with the goal to transition from manual to automated system testing using VGT. Results of the study show that the VGT transition was successful and that VGT could be applied in the industrial context when performed by practitioners but that there were several problems that first had to be solved, e.g. testing of a distributed system, tool volatility. These problems and solutions have been presented together with qualitative, and quantitative, data about the benefits of the technique compared to manual testing, e.g. greatly improved execution speed, feasible transition and maintenance costs, improved bug finding ability. The study thereby provides valuable, and previously missing, contributions about VGT to both practitioners and researchers.

  • 2. Alégroth, Emil
    et al.
    Nass, Michael
    Olsson Holmström, Helena
    Malmö högskola, Faculty of Technology and Society (TS), Department of Computer Science (DV).
    JAutomate: A Tool for System and Acceptance Test Automation2013In: 2013 IEEE Sixth International Conference on Software Testing, Verification and Validation, IEEE, 2013, p. 439-446Conference paper (Refereed)
    Abstract [en]

    System- and acceptance-testing are primarily performed with manual practices in current software industry. However, these practices have several issues, e.g. they are tedious, error prone and time consuming with costs up towards 40 percent of the total development cost. Automated test techniques have been proposed as a solution to mitigate these issues, but they generally approach testing from a lower level of system abstraction, leaving a gap for a flexible, high system-level test automation technique/tool. In this paper we present JAutomate, a Visual GUI Testing (VGT) tool that fills this gap by combining image recognition with record and replay functionality for high system-level test automation performed through the system under test's graphical user interface. We present the tool, its benefits compared to other similar techniques and manual testing. In addition, we compare JAutomate with two other VGT tools based on their static properties. Finally, we present the results from a survey with industrial practitioners that identifies test-related problems that industry is currently facing and discuss how JAutomate can solve or mitigate these problems.

  • 3. Backlund, Emil
    et al.
    Bolle, Mikael
    Tichy, Matthias
    Olsson Holmström, Helena
    Malmö högskola, School of Technology (TS).
    Bosch, Jan
    Automated User Interaction Analysis for Workflow-Based Web Portals2014In: Software Business: Towards Continuous Value Delivery, Springer, 2014, p. 148-162Conference paper (Refereed)
    Abstract [en]

    Success in the software market requires constant improvement of the software. These improvements however have to directly align with the needs of the users of the software. A recent trend in software engineering is to collect post-deployment data about how users use a software system. We report in this paper about a case study with an industrial partner in which (1) we identified which data has to be collected for a web-based portal system, (2) implemented the data collection, and (3) performed an experiment comparing the collected data with answers of the test subjects in a survey.

  • 4.
    Bosch, Jan
    et al.
    Chalmers University of Technology, Gothenburg, Sweden.
    Olsson, Helena H.
    Malmö University, Faculty of Technology and Society (TS), Department of Computer Science and Media Technology (DVMT).
    Digital for real: A multicase study on the digital transformation of companies in the embedded systems domain2021In: Journal of Software: Evolution and Process, ISSN 2047-7473, E-ISSN 2047-7481, Vol. 33, no 5, article id e2333Article in journal (Refereed)
    Abstract [en]

    With digitalization and with technologies such as software, data, and artificial intelligence, companies in the embedded systems domain are experiencing a rapid transformation of their conventional businesses. While the physical products and associated product sales provide the core revenue, these are increasingly being complemented with service offerings, new data-driven services, and digital products that allow for continuous value creation and delivery to customers. However, although there is significant research on digitalization and digital transformation, few studies highlight the specific needs of embedded systems companies and what it takes to transform from a traditional towards a digital company within business domains characterized by high complexity, hardware dependencies, and safety-critical system functionality. In this paper, we capture the difference between what constitutes a traditional and a digital company and we detail the typical evolution path embedded systems companies take when transitioning towards becoming digital companies.

    Download full text (pdf)
    fulltext
  • 5. Bosch, Jan
    et al.
    Olsson, Helena Holmström
    Malmö University, Faculty of Technology and Society (TS), Department of Computer Science and Media Technology (DVMT).
    Ecosystem traps and where to find them2018In: Journal of Software: Evolution and Process, ISSN 2047-7473, E-ISSN 2047-7481, Vol. 30, no 11, article id e1961Article in journal (Refereed)
    Abstract [en]

    Today, companies operate in business ecosystems where they collaborate, compete, share, and learn from others with benefits such as to present more attractive offerings and sharing innovation costs. With ecosystems being the new way of operating, the ability to strategically reposition oneself to increase or shift power balance is becoming key for competitive advantage. However, companies run into a number of traps when trying to realize strategical changes in their ecosystems. In this paper, we identify 5 traps that companies fall into. First, the "descriptive versus prescriptive trap" is when companies assume that current boundaries between partners are immutable. Second, the "assumptions trap" is when powerful ecosystem partners assume that they understand what others regard as value-adding without validating their assumptions. Third, the "keeping it too simple trap" is when companies overlooks the effort required to align interests. Fourth, the "doing it all at once trap" is when companies disrupt an ecosystem assuming that all partners can change direction at the same time. Finally, the "planning trap" is when companies are unable to move forward without a complete plan. We provide empirical evidence for each trap, and we propose an ecosystem engagement process for how to avoid falling into these.

  • 6.
    Bosch, Jan
    et al.
    Software Center and Chalmers University of Technology.
    Olsson, Helena Holmström
    Malmö University, Faculty of Technology and Society (TS), Department of Computer Science and Media Technology (DVMT). Software Center.
    Brinne, Bjorn
    Peltarion.
    Crnkovic, Ivica
    Chalmers Artificial Intelligence Research Center and Chalmers University of Technology.
    AI Engineering: Realizing the Potential of AI2022In: IEEE Software, ISSN 0740-7459, E-ISSN 1937-4194, Vol. 39, no 6, p. 23-27Article in journal (Refereed)
    Abstract [en]

    Artificial Intelligence (AI) engineering is an engineering discipline that is concerned with all aspects of development and evolution of AI systems (that is, systems that include AI components). AI engineering is primarily an extension of software engineering, but it also includes methods and technologies from data science and AI in general.

  • 7.
    Bosch, Jan
    et al.
    Chalmers University of Technology, Sweden.
    Olsson, Helena Holmström
    Malmö University, Faculty of Technology and Society (TS), Department of Computer Science and Media Technology (DVMT).
    Crnkovic, Ivica
    Chalmers University of Technology, Sweden.
    Engineering AI Systems: A Research Agenda2021In: Artificial Intelligence Paradigms for Smart Cyber-Physical Systems / [ed] Ashish Kumar Luhach; Atilla Elçi, IGI Global, 2021, p. 1-19Chapter in book (Refereed)
    Abstract [en]

    Artificial intelligence (AI) and machine learning (ML) are increasingly broadly adopted in industry. However, based on well over a dozen case studies, we have learned that deploying industry-strength, production quality ML models in systems proves to be challenging. Companies experience challenges related to data quality, design methods and processes, performance of models as well as deployment and compliance. We learned that a new, structured engineering approach is required to construct and evolve systems that contain ML/DL components. In this chapter, the authors provide a conceptualization of the typical evolution patterns that companies experience when employing ML as well as an overview of the key problems experienced by the companies that they have studied. The main contribution of the chapter is a research agenda for AI engineering that provides an overview of the key engineering challenges surrounding ML solutions and an overview of open items that need to be addressed by the research community at large.

  • 8.
    Bosch, Jan
    et al.
    Chalmers University of Technology, Department of Computer Science & Engineering, Göteborg, Sweden .
    Olsson, Helena Holmström
    Malmö University, Faculty of Technology and Society (TS), Department of Computer Science and Media Technology (DVMT).
    Crnkovic, Ivica
    Chalmers University of Technology, Department of Computer Science & Engineering, Göteborg, Sweden .
    It Takes Three to Tango: Requirement, Outcome/data, and AI Driven Development2018In: Proceedings of the International Workshop on Software-intensive Business: Start-ups, Ecosystems and Platforms (SiBW 2018), CEUR-WS.org , 2018, p. 177-192Conference paper (Refereed)
    Abstract [en]

    Today’s software-intensive organizations are experiencing a paradigm-shift with regards to how to develop software systems. With the increasing availability and access to data and with artificial intelligence (AI) and technologies such as machine learning and deep learning emerging, the traditional requirement driven approach to software development is becoming complemented with other approaches. In addition to having development teams executing on requirements specified by product management, the development of software systems is progressing towards a data driven practice where teams receive an outcome to realize and where design decisions are taken based on continuous collection and analysis of data. On top of this, and due to artificial intelligence components being introduced to more and more software systems, learning algorithms, automatically generated models and data is replacing code and the development process is no longer only a manual effort but instead a combination of human and automated processes. In this paper, and based on multi-case study research in embedded systems and online companies, we see that companies use different approaches to software development but that they often take a requirement driven approach even if they would benefit from one of the other two. Also, we see that picking the wrong approach results in a number of problems such as e.g. inefficiency and waste of development efforts. To help address these problems, we develop a holistic development framework and we provide guidelines on how to improve effectiveness in development. The contribution of this paper is two-fold. First, we identify that there are three distinct approaches to software development; (1) Requirement driven development, (2) Outcome/data driven development and (3) AI driven development and we outline the typical problems that companies experience when using the wrong approach for the wrong purpose. Second, we provide a holistic framework with guidelines for when to use what approach to software development. 

    Download full text (pdf)
    fulltext
  • 9. Bosch, Jan
    et al.
    Olsson Holmström, Helena
    Malmö högskola, Faculty of Technology and Society (TS). Malmö högskola, Internet of Things and People (IOTAP).
    Data-Driven Continuous Evolution of Smart Systems2016In: Proceedings of the 11th International Symposium on Software Engineering for Adaptive and Self-Managing Systems, ACM Digital Library, 2016, p. 28-34Conference paper (Refereed)
    Abstract [en]

    As Marc Andreessen said in his Wall Street Journal OpEd, software is eating the world. The systems that we are building today and in the near future will exhibit levels of autonomy that will put new demands on the engineering of such systems. Although promising examples of autonomous systems exist, there is no established methodology for systematically building autonomous systems that employ modern software engineering technology such as continuous deployment and data-driven engineering. The contribution of this paper is twofold. First, it identifies and presents the challenge of continuous evolution of autonomous systems as a well-defined problem that needs to be addressed by software engineering research. Second, it presents a conceptual solution to this problem that integrates the development of new software for autonomous systems by R&D teams with systematic experimentation by autonomous systems.

  • 10.
    Bosch, Jan
    et al.
    Chalmers University.
    Olsson Holmström, Helena
    Malmö högskola, Faculty of Technology and Society (TS).
    Toward Evidence-Based Organizations Lessons from Embedded Systems, Online Games, and the Internet of Things2017In: IEEE Software, ISSN 0740-7459, E-ISSN 1937-4194, Vol. 34, no 5, p. 60-66Article in journal (Refereed)
    Abstract [en]

    Case studies investigated how companies in three domains transition to data-driven development. The results led to a model of the levels that software-intensive companies move through as they evolve from an opinionbased to an evidence-based organization.

  • 11. Bosch, Jan
    et al.
    Olsson Holmström, Helena
    Malmö högskola, Faculty of Technology and Society (TS), Department of Computer Science (DV).
    Björk, Jens
    Ljungblad, Jens
    The Early Stage Software Startup Development Model: A Framework for Supporting Lean Principles in Software Startups2013In: Lean Enterprise Software and Systems. LESS 2013., Springer, 2013, p. 1-15Conference paper (Refereed)
    Abstract [en]

    Software startups are more popular than ever and growing in numbers. They operate under conditions of extreme uncertainty and face many challenges. Often, agile development practices and lean principles are suggested as ways to increase the odds of succeeding as a startup, as they both advocate close customer collaboration and short feedback cycles focusing on delivering direct customer value. However, based on an interview study we see that despite guidance and support in terms of well-known and documented development methods, practitioners find it difficult to implement and apply these in practice. To explore this further, and to propose operational support for software startup companies, this study aims at investigating (1) what are the typical challenges when finding a product idea worth scaling, and (2) what solution would serve to address these challenges. To this end, we propose the ‘Early Stage Software Startup Development Model’ (ESSSDM). The model extends already existing lean principles, but offers novel support for practitioners for investigating multiple product ideas in parallel, for determining when to move forward with a product idea, and for deciding when to abandon a product idea. The model was evaluated in a software startup project, as well as with industry professionals within the software startup domain.

  • 12.
    Dakkak, Anas
    et al.
    Ericsson AB, Sweden.
    Bosch, Jan
    Chalmers University of Technology, Sweden.
    Olsson, Helena Holmström
    Malmö University, Faculty of Technology and Society (TS), Department of Computer Science and Media Technology (DVMT).
    Controlled Continuous Deployment: A Case Study From The Telecommunications Domain2022In: Proceedings of the International Conference on Software and System Processes and International Conference on Global Software Engineering, Association for Computing Machinery (ACM), 2022, p. 24-33Conference paper (Refereed)
    Abstract [en]

    Continuous deployment has become a widely used practice in web-based software applications. Deploying a new software version to production is a seamless automated process executed thousands of times per day. Continuous deployment reduces the time between a code commit and that commit is active in production. While continuous deployment promises many advantages to software development organizations, the adoption of continuous deployment in the software-intensive embedded systems industry is limited. Several empirical studies have highlighted the challenges associated with software-intensive embedded systems. However, very few studies, if any at all, have attempted to provide a practical approach to realize continuous deployment to these systems. This paper proposes a Controlled Continuous Deployment (CCD) approach, which considers the constraints software-intensive embedded systems have, such as high reliability and availability requirements, limited possibility for rollback after deployment, and the high volume of in-service systems in the market. We derived the approach by conducting a case study at Ericsson AB, focusing on three Radio Access Networks (RAN) technologies embedded software used in 3G, 4G, and 5G mobile networks.  

  • 13.
    Dakkak, Anas
    et al.
    Ericsson AB, Stockholm, Sweden.
    Bosch, Jan
    Chalmers University of Technology, Gothenburg, Sweden.
    Olsson, Helena Holmström
    Malmö University, Faculty of Technology and Society (TS), Department of Computer Science and Media Technology (DVMT).
    DevServOps: DevOps For Product-Oriented Product Service Systems2023In: 2023 49th Euromicro Conference on Software Engineering and Advanced Applications (SEAA), Institute of Electrical and Electronics Engineers (IEEE), 2023Conference paper (Refereed)
    Abstract [en]

    Companies producing software-intensive products do not only offer products to customers but Product Service Systems (PSS), a combination of the products and services that address customers’ needs. Further, product-related services are key in ensuring customer satisfaction as the service organization represents the company’s interface toward its customers, who operate and use the products. Therefore, while DevOps has been widely adopted in companies developing web-based applications aiming to streamline the Development and Operations activities, the projecting of DevOps as applied in web-based applications to PSS is difficult without considering the role of services. Therefore, based on a two years participant observation case study conducted at a multinational telecommunications systems provider, we propose a new and novel approach called Development-Services-Operations (DevServOps) which incorporates services as a key player facilitating an end-to-end software flow toward customers in one direction and feedback toward developers in the other direction.

  • 14.
    Dakkak, Anas
    et al.
    Ericsson AB,Stockholm,Sweden.
    Bosch, Jan
    Chalmers University of Technology,Göteborg,Sweden.
    Olsson, Helena Holmström
    Malmö University, Faculty of Technology and Society (TS), Department of Computer Science and Media Technology (DVMT).
    The Role Of Post-Release Software Traceability in Release Engineering: A Software-Intensive Embedded Systems Case Study From The Telecommunications Domain2022In: 2022 48th Euromicro Conference on Software Engineering and Advanced Applications (SEAA), Institute of Electrical and Electronics Engineers (IEEE), 2022Conference paper (Refereed)
    Abstract [en]

    Modern release engineering practices such as continuous integration and delivery have allowed software development companies to transition from a long release cycle to a shorter one. The shorter release cycle has led to more software releases available to customers. At the same time, companies developing high-volume software-intensive embedded systems often deliver patch releases and maintenance releases on top of major and minor releases to customers who pick and choose what releases apply to them and decide when to upgrade the system, if to upgrade at all. While release engineering has been studied before in web-based, desktop-based, and embedded software, the focus has been on pre-release activities. Few studies have investigated what happens after the release, particularly the role of tracing software from release to deployment in high-volume software-intensive embedded systems. To address this gap, we conducted a qualitative case study at a multi-national telecommunications systems provider focusing on Radio Access Network (RAN) software. RAN software is a complex and large-scale embedded software used in mobile networks Base Stations (BS), providing software functionality for RAN mobile technologies ranging from 2G to 5G. Our study shed light on post-release software traceability and how it is used in the release engineering process.

  • 15.
    Dakkak, Anas
    et al.
    Ericsson AB, Stockholm, Sweden..
    Bosch, Jan
    Chalmers Univ Technol, Dept Comp Sci & Engn, Gothenburg, Sweden..
    Olsson, Helena Holmström
    Malmö University, Faculty of Technology and Society (TS), Department of Computer Science and Media Technology (DVMT).
    Towards AIOps enabled services in continuously evolving software-intensive embedded systems2023In: Journal of Software: Evolution and Process, ISSN 2047-7473, E-ISSN 2047-7481Article in journal (Refereed)
    Abstract [en]

    Continuous deployment has been practiced for many years by companies developing web- and cloud-based applications. To succeed with continuous deployment, these companies have a strong collaboration culture between the operations and development teams. In addition, these companies use AI, analytics, and big data to assist with time-consuming postdeployment activities such as continuous monitoring and fault identification. Thus, the term AIOps has evolved to highlight the importance and difficulty of maintaining highly available applications in a complex and dynamic environment. In contrast, software-intensive embedded systems often provide customer product-related services, such as maintenance, optimization, and support. These services are critical for these companies as they provide significant revenue and increase customer satisfaction. Therefore, the objective of our study is to gain an in-depth understanding of the impact of continuous deployment on product-related services provided by software-intensive embedded systems companies. In addition, we aim to understand how AIOps can support continuous deployment in the context of software-intensive embedded systems. To address this objective, we conducted a case study at a large and multinational telecommunications systems provider focusing on the radio access network (RAN) systems for 4G and 5G networks. The company provides RAN products and three complementing services: rollout, optimization, and customer support. The results from the case study show that the boundaries between product-related services become blurry with continuous deployment. In addition, product-related services, which were conducted in sequence by independent projects, converge with continuous deployment and become part of the same project. Further, AIOps platforms play an important role in reducing costs and increasing postdeployment activities' efficiency and speed. These results show that continuous deployment has a profound impact on the software-intensive system's provider service organization. The service organization becomes the connection between the R&D organization and the customer. In order to cope with the increased speed of releases, deployment and postdeployment activities need to be largely automated. AIOps platforms are seen as a critical enabler in managing the increasing complexity without increasing human involvement.

  • 16.
    Dakkak, Anas
    et al.
    Ericsson AB, Torshamnsgatan 21, Stockholm, 164 83, Sweden.
    Bosch, Jan
    Department of Computer Science and Engineering, Chalmers University of Technology, Chalmersplatsen 1, Gothenburg, 412 96, Sweden.
    Olsson, Helena Holmström
    Malmö University, Faculty of Technology and Society (TS), Department of Computer Science and Media Technology (DVMT).
    Issa Mattos, David
    Department of Computer Science and Engineering, Chalmers University of Technology, Chalmersplatsen 1, Gothenburg, 412 96, Sweden.
    Continuous deployment in software-intensive system-of-systems2023In: Information and Software Technology, ISSN 0950-5849, E-ISSN 1873-6025, Vol. 159, p. 107200-107200, article id 107200Article in journal (Refereed)
    Abstract [en]

    Context:While continuous deployment is popular among web-based software development organizations, adopting continuous deployment in software-intensive system-of-systems is more challenging. On top of the challenges arising from deploying software to a single software-intensive embedded system, software-intensive system-of-systems (SiSoS) add a layer of complexity as new software undergoes an extensive field validation applied to individual components of the SiSoS, as well as the overall SiSoS, to ensure that both legacy and new functionalities are working as desired.

    Objectives:This paper aims to study how SiSoS transitions to continuous deployment by exploring how continuous deployment impacts field testing and validation activities, how continuous deployment can be practiced in SiSoS, and to identify the success factors that companies need to consider when transitioning to continuous deployment.

    Method:We conducted a case study at Ericsson AB focusing on the embedded software of the Third Generation Radio Access Network (3G RAN). The 3G RAN consists of two large-scale software-intensive embedded systems, representing a simple SiSoS composed of two systems. 3G RAN software was the first to transition to continuous deployment and is used as a reference case for other products within Ericsson AB.

    Results:Software deployment, in addition to field testing and validation, have transitioned from being a discrete activity performed at the end of software development to a continuous process performed in parallel to software development. Further, our study reveals an orchestrating approach for software deployment, which allows pre/post validation of legacy behavior and new features in a shorter release and deployment cadence. Furthermore, we identified the essential success factors that organizations should consider when transitioning to continuous deployment.

    Conclusion:Transition to continuous deployment, in addition to field testing and validation, shall be considered and planned carefully. In this paper, we provide a set of success factors and orchestration technique that helps organization when transitioning to continuous deployment in the software-intensive embedded system-of-systems context.

  • 17.
    Dakkak, Anas
    et al.
    Ericsson AB, Stockholm, Sweden..
    Munappy, Aiswarya Raj
    Chalmers Univ Technol, Gothenburg, Sweden..
    Bosch, Jan
    Chalmers Univ Technol, Gothenburg, Sweden..
    Olsson, Helena Holmström
    Malmö University, Faculty of Technology and Society (TS), Department of Computer Science and Media Technology (DVMT).
    Customer Support In The Era of Continuous Deployment: A Software-Intensive Embedded Systems Case Study2022In: 2022 IEEE 46TH ANNUAL COMPUTERS, SOFTWARE, AND APPLICATIONS CONFERENCE (COMPSAC 2022) / [ed] Leong, HV Sarvestani, SS Teranishi, Y Cuzzocrea, A Kashiwazaki, H Towey, D Yang, JJ Shahriar, H, Institute of Electrical and Electronics Engineers (IEEE), 2022, p. 914-923Conference paper (Refereed)
    Abstract [en]

    Supporting customers after they acquire the product is essential for companies producing and selling software-intensive embedded systems products. Generally, customer support is the first interaction point between the product users and the product vendor. Customer support is often engaged with answering customers' questions, troubleshooting, fault identification, and fixing product faults. While continuous deployment advocates for closer cooperation between the ones operating the software and the ones developing it, the means of such collaboration in general and the role of customer support, in particular, has not been addressed in the context of software-intensive embedded systems. Therefore, to better understand the impact that continuous deployment has on customer support and the role customer support should play in this context, we conducted a case study at a multinational company developing and selling telecommunications networks infrastructure. We focused on the 4th and 5th Generation (4G and 5G) Radio Access Networks (RAN) products, which can be considered a high volume product as they cover more than 80% of the world's population. Our study reveals that customer support needs to transition from a transaction-based and passive function triggered by customer support requests, to take an active role characterized by being proactive and preemptive to cope with the shorter operational time of a software version introduced by continuous deployment. In addition, customer support plays an essential role in making the feedback actionable by aggregating and consolidating feedback data to the R&D organization.

  • 18.
    Dakkak, Anas
    et al.
    Ericsson AB,Stockholm,Sweden.
    Zhang, Hongyi
    Chalmers University of Technology, Gothenburg, Sweden.
    Mattos, David Issa
    Chalmers University of Technology, Gothenburg, Sweden.
    Bosch, Jan
    Chalmers University of Technology, Gothenburg, Sweden.
    Olsson, Helena Holmström
    Malmö University, Faculty of Technology and Society (TS), Department of Computer Science and Media Technology (DVMT).
    Towards Continuous Data Collection from In-service Products: Exploring the Relation Between Data Dimensions and Collection Challenges2021In: 2021 28th Asia-Pacific Software Engineering Conference (APSEC), IEEE, 2021Conference paper (Refereed)
    Abstract [en]

    Data collected from in-service products play an important role in enabling software-intensive embedded systems suppliers to embrace data-driven practices. Data can be used in many different ways such as to continuously learn and improve the product, enhance post-deployment services, reduce operational cost or create a better user experience. While there is no shortage of possible use cases leveraging data from in-service products, software-intensive embedded systems companies struggle to continuously collect data from their in-service products. Often, data collection is done in an ad-hoc way and targeting specific use cases or needs. Besides, few studies have investigated data collection challenges in relation to the data dimensions, which are the minimum set of quantifiable data aspects that can define software-intensive embedded product data from a collection point of view. To help address data collection challenges, and to provide companies with guidance on how to improve this process, we conducted a case study at a large multinational telecommunications supplier focusing on data characteristics and collection challenges from the Radio Access Networks (RAN) products. We further investigated the relations of these challenges to the data dimensions to increase our understanding of how data dominions contribute to the challenges.

  • 19. Dingsøyr, Torgeir
    et al.
    Moe, Nils Brede
    Olsson Holmström, Helena
    Malmö University, Faculty of Technology and Society (TS), Department of Computer Science and Media Technology (DVMT).
    Towards an Understanding of Scaling Frameworks and Business Agility2018In: 19th International Conference On Agile Software Development (Xp '18), ACM Digital Library, 2018, article id 6Conference paper (Refereed)
    Abstract [en]

    Large development projects and programs are conducted using agile development methods, with an increasing body of advice from practitioners and from research. This sixth workshop showed in increasing interest in scaling frameworks and in topics related to achieving business agility. This article summarizes four contributed papers, discussions in "open space" format and also presents a revised research agenda for large-scale agile development.

    Download full text (pdf)
    FULLTEXT01
  • 20.
    Dzhusupova, Rimma
    et al.
    McDermott, Dept Elect & Instrumentat Control & Safety Syst, The Hague, Netherlands..
    Banotra, Richa
    McDermott, Dept Instrumentat Control & Safety Syst, The Hague, Netherlands..
    Bosch, Jan
    Chalmers Univ Technol, Dept Comp Sci & Engn, Gothenburg, Sweden..
    Olsson, Helena Holmström
    Malmö University, Faculty of Technology and Society (TS), Department of Computer Science and Media Technology (DVMT).
    Pattern Recognition Method for Detecting Engineering Errors on Technical Drawings2022In: 2022 IEEE World AI IoT Congress (AIIoT) / [ed] Paul, R, IEEE , 2022, p. 642-648Conference paper (Refereed)
  • 21.
    Dzhusupova, Rimma
    et al.
    Electrical, Instrumentation, Control & Safety Systems McDermott The Hague The Netherlands.
    Banotra, Richa
    Instrumentation, Control & Safety Systems McDermott The Hague The Netherlands.
    Bosch, Jan
    Computer Science and Engineering Chalmers University of Technology Gothenburg Sweden.
    Olsson, Helena Holmström
    Malmö University, Faculty of Technology and Society (TS), Department of Computer Science and Media Technology (DVMT).
    Using artificial intelligence to find design errors in the engineering drawings2023In: Journal of Software: Evolution and Process, ISSN 2047-7473, E-ISSN 2047-7481, Vol. 35, no 12Article in journal (Refereed)
    Abstract [en]

    Artificial intelligence is increasingly becoming important to businesses because many companies have realized the benefits of applying machine learning (ML) and deep learning (DL) in their operations. ML and DL have become attractive technologies for organizations looking to automate repetitive tasks to reduce manual work and free up resources for innovation. Unlike rule-based automation, typically used for standardized and predictable processes, machine learning, especially deep learning, can handle more complex tasks and learn over time, leading to greater accuracy and efficiency improvements. One of such promising applications is to use AI to reduce manual engineering work. This paper discusses a particular case within McDermott where the research team developed a DL model to do a quality check of complex blueprints. We describe the development and the final product of this case—AI-based software for the engineering, procurement, and construction (EPC) industry that helps to find the design mistakes buried inside very complex engineering drawings called piping and instrumentation diagrams (P&IDs). We also present a cost-benefit analysis and potential scale-up of the developed software. Our goal is to share the successful experience of AI-based product development that can substantially reduce the engineering hours and, therefore, reduce the project's overall costs. The developed solution can also be potentially applied to other EPC companies doing a similar design for complex installations with high safety standards like oil and gas or petrochemical plants because the design errors it captures are common within this industry. It also could motivate practitioners and researchers to create similar products for the various fields within engineering industry. 

    Download full text (pdf)
    fulltext
  • 22.
    Dzhusupova, Rimma
    et al.
    McDermott, Dept Elect & Instrumentat Control & Safety Syst, The Hague, Netherlands..
    Bosch, Jan
    Chalmers Univ Technol, Dept Comp Sci & Engn, Gothenburg, Sweden..
    Olsson, Helena Holmström
    Malmö University, Faculty of Technology and Society (TS), Department of Computer Science and Media Technology (DVMT).
    Challenges in developing and deploying AI in the engineering, procurement and construction industry2022In: 2022 IEEE 46th Annual Computers, Software, and Applications Conference (COMPSAC) / [ed] Leong, HV Sarvestani, SS Teranishi, Y Cuzzocrea, A Kashiwazaki, H Towey, D Yang, JJ Shahriar, H, IEEE , 2022, p. 1070-1075Conference paper (Refereed)
    Abstract [en]

    AI in the Engineering, Procurement and Construction (EPC) industry has not yet a proven track record in large-scale projects. Since AI solutions for industrial applications became available only recently, deployment experience and lessons learned are still to be built up. Several research papers exist describing the potential of AI, and many surveys and white papers have been published indicating the challenges of AI deployment in the EPC industry. However, there is a recognizable shortage of in-depth studies of deployment experience in academic literature, particularly those focusing on the experiences of EPC companies involved in large-scale project execution with high safety standards, such as the petrochemical or energy sector. The novelty of this research is that we explore in detail the challenges and obstacles faced in developing and deploying AI in a large-scale project in the EPC industry based on real-life use cases performed in an EPC company. Those identified challenges are not linked to specific technology or a company's know-how and, therefore, are universal. The findings in this paper aim to provide feedback to academia to reduce the gap between research and practice experience. They also help reveal the hidden stones when implementing AI solutions in the industry.

  • 23.
    Dzhusupova, Rimma
    et al.
    Eindhoven Univ Technol, Math & Comp Sci, Eindhoven, Netherlands..
    Bosch, Jan
    Chalmers Univ Technol, Comp Sci & Engn, Gothenburg, Sweden..
    Olsson, Helena Holmström
    Malmö University, Faculty of Technology and Society (TS), Department of Computer Science and Media Technology (DVMT).
    Choosing the right path for AI integration in engineering companies: A strategic guide2024In: Journal of Systems and Software, ISSN 0164-1212, E-ISSN 1873-1228, Vol. 210, article id 111945Article in journal (Refereed)
    Abstract [en]

    The Engineering, Procurement and Construction (EPC) businesses operating within the energy sector are recognizing the increasing importance of Artificial Intelligence (AI). Many EPC companies and their clients have realized the benefits of applying AI to their businesses in order to reduce manual work, drive productivity, and streamline future operations of engineered installations in a highly competitive industry. The current AI market offers various solutions and services to support this industry, but organizations must understand how to acquire AI technology in the most beneficial way based on their business strategy and available resources. This paper presents a framework for EPC companies in their transformation towards AI. Our work is based on examples of project execution of AI-based products development at one of the biggest EPC contractors worldwide and on insights from EPC vendor companies already integrating AI into their engineering solutions. The paper covers the entire life cycle of building AI solutions, from initial business understanding to deployment and further evolution. The framework identifies how various factors influence the choice of approach toward AI project development within large international engineering corporations. By presenting a practical guide for optimal approach selection, this paper contributes to the research in AI project management and organizational strategies for integrating AI technology into businesses. The framework might also help engineering companies choose the optimum AI approach to create business value.

    Download full text (pdf)
    fulltext
  • 24.
    Dzhusupova, Rimma
    et al.
    McDermott, The Hague, The Netherlands.
    Bosch, Jan
    Chalmers University of Technology, Gothenburg, Sweden.
    Olsson, Helena Holmström
    Malmö University, Faculty of Technology and Society (TS), Department of Computer Science and Media Technology (DVMT).
    The goldilocks framework: towards selecting the optimal approach to conducting AI projects2022In: CAIN '22: Proceedings of the 1st International Conference on AI Engineering: Software Engineering for AI, ACM Digital Library, 2022, p. 124-135Conference paper (Refereed)
    Abstract [en]

    Artificial intelligence is increasingly becoming important to businesses since many companies have realized the benefits of applying Machine Learning (ML) and Deep Learning (DL) into their operations. Nevertheless, ML/DL technologies' industrial development and deployment examples are still rare and generally confined within a small cluster of large international companies who are struggling to apply ML more broadly and deploy their use cases at a large scale. Meanwhile, current AI market has started offering various solutions and services. Thus, organizations must understand how to acquire AI technology based on their business strategy and available resources. This paper discusses the industrial experience of developing and deploying ML/DL use cases to support organizations in their transformation towards AI. We identify how various factors, like cost, schedule, and intellectual property, can be affected by the choice of approach towards ML/DL project development and deployment within large international engineering corporations. As a research result, we present a framework that covers the trade-offs between those various factors and can support engineering companies to choose the best approach based on their long-term business strategies and, therefore, would help to accomplish their ML/DL project deployment successfully.  

     

  • 25.
    Eklund, Ulrik
    et al.
    Malmö högskola, Faculty of Technology and Society (TS), Department of Computer Science (DV).
    Olsson Holmström, Helena
    Malmö högskola, Faculty of Technology and Society (TS), Department of Computer Science (DV).
    Strøm, Niels Jørgen
    Grundfos A/S, DK-8850, Bjerringbro, Denmark.
    Industrial Challenges of Scaling Agile in Mass-Produced Embedded Systems2014In: Agile Methods. Large-Scale Development, Refactoring, Testing, and Estimation: XP 2014 International Workshops, Rome, Italy, May 26-30, 2014, Revised Selected Papers, Springer, 2014, p. 30-42Conference paper (Refereed)
    Abstract [en]

    When individual teams in mechatronic organizations attempt to adopt agile software practices, these practices tend to only affect mod- ules or sub-systems. The short iterations on team level do not lead to short lead-times in launching new or updated products since the overall R&D approach on an organization level is still governed by an overall stage gate or single cycle V-model. This paper identifies challenges for future research on how to combine the predictability and planning desired of mechanical manufacturing with the dynamic capabilities of modern agile software development. Scaling agile in this context requires an expansion in two dimensions: First, scal- ing the number of involved teams. Second, traversing necessary systems engineering activities in each sprint due to the co-dependency of software and hardware development.

    Download full text (pdf)
    FULLTEXT01
  • 26.
    Fabijan, Aleksander
    et al.
    Malmö University, Faculty of Technology and Society (TS), Department of Computer Science and Media Technology (DVMT).
    Dmitriev, Pavel
    Outreach.io, Seattle, WA, USA.
    McFarland, Colin
    Skyscanner, Edinburgh, Scotland, UK.
    Vermeer, Lukas
    BOOKING.COM, Amsterdam, Netherlands.
    Olsson Holmström, Helena
    Malmö University, Faculty of Technology and Society (TS), Department of Computer Science and Media Technology (DVMT).
    Bosch, Jan
    Chalmers University, Gothenburg, Sweden.
    Experimentation growth: Evolving trustworthy A/B testing capabilities in online software companies2018In: Journal of Software: Evolution and Process, ISSN 2047-7473, E-ISSN 2047-7481, Vol. 30, no 12, article id e2113Article in journal (Refereed)
    Abstract [en]

    Companies need to know how much value their ideas deliver to customers. One of the most powerful ways to accurately measure this is by conducting online controlled experiments (OCEs). To run experiments, however, companies need to develop strong experimentation practices as well as align their organization and culture to experimentation. The main objective of this paper is to demonstrate how to run OCEs at large scale using the experience of companies that succeeded in scaling. Based on case study research at Microsoft, Booking.com, Skyscanner, and Intuit, we present our main contribution—The Experiment Growth Model. This four‐stage model addresses the seven critical aspects of experimentation and can help companies to transform their organizations into learning laboratories where new ideas can be tested with scientific accuracy. Ultimately, this should lead to better products and services.

  • 27.
    Fabijan, Aleksander
    et al.
    Malmö University, Faculty of Technology and Society (TS), Department of Computer Science and Media Technology (DVMT).
    Dmitriev, Pavel
    Olsson, Helena Holmström
    Malmö University, Faculty of Technology and Society (TS), Department of Computer Science and Media Technology (DVMT).
    Bosch, Jan
    Effective Online Controlled Experiment Analysis at Large Scale2018In: Proceedings of the EUROMICRO Conference, IEEE, 2018, p. 64-67Conference paper (Refereed)
    Abstract [en]

    Online Controlled Experiments (OCEs) are the norm in data-driven software companies because of the benefits they provide for building and deploying software. Product teams experiment to accurately learn whether the changes that they do to their products (e.g. adding new features) cause any impact (e.g. customers use them more frequently). Experiments also help reduce the risk from deploying software by minimizing the magnitude and duration of harm caused by software bugs, allowing software to be shipped more frequently. To make informed decisions in product development, experiment analysis needs to be granular with a large number of metrics over heterogeneous devices and audiences. Discovering experiment insights by hand, however, can be cumbersome. In this paper, and based on case study research at a large-scale software development company with a long tradition of experimentation, we (1) describe the standard process of experiment analysis, and (2) introduce an artifact to improve the effectiveness and comprehensiveness of this process.

  • 28.
    Fabijan, Aleksander
    et al.
    Malmö University, Faculty of Technology and Society (TS), Department of Computer Science and Media Technology (DVMT).
    Dmitriev, Pavel
    Olsson, Helena Holmström
    Malmö University, Faculty of Technology and Society (TS), Department of Computer Science and Media Technology (DVMT).
    Bosch, Jan
    Online Controlled Experimentation at Scale: An Empirical Survey on the Current State of A/B Testing2018In: Proceedings of the EUROMICRO Conference, IEEE, 2018, p. 68-72Conference paper (Refereed)
    Abstract [en]

    Online Controlled Experiments (OCEs, aka A/B tests) are one of the most powerful methods for measuring how much value new features and changes deployed to software products bring to users. Companies like Microsoft, Amazon, and Booking.com report the ability to conduct thousands of OCEs every year. However, the competences of the remainder of the online software industry remain unknown. The main objective of this paper is to reveal the current state of A/B testing maturity in the software industry based on a maturity model from our previous research. We base our findings on 44 responses from an online empirical survey. Our main contribution of this paper is the current state of experimentation maturity as operationalized by the ExG model for a convenience sample of companies doing online controlled experiments. Our findings show that, among others, companies typically develop in-house experimentation platforms, that these platforms are of various levels of maturity, and that designing key metrics - Overall Evaluation Criteria - remains the key challenge for successful experimentation.

  • 29.
    Fabijan, Aleksander
    et al.
    Malmö University, Faculty of Technology and Society (TS), Department of Computer Science and Media Technology (DVMT).
    Dmitriev, Pavel
    Olsson, Helena Holmström
    Malmö University, Faculty of Technology and Society (TS), Department of Computer Science and Media Technology (DVMT).
    Bosch, Jan
    Vermeer, Lukas
    Lewis, Dylan
    Three Key Checklists and Remedies for Trustworthy Analysis of Online Controlled Experiments at Scale2019In: 2019 IEEE/ACM 41st International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP 2019), IEEE, 2019, p. 1-10Conference paper (Refereed)
    Abstract [en]

    Online Controlled Experiments (OCEs) are transforming the decision-making process of data-driven companies into an experimental laboratory. Despite their great power in identifying what customers actually value, experimentation is very sensitive to data loss, skipped checks, wrong designs, and many other 'hiccups' in the analysis process. For this purpose, experiment analysis has traditionally been done by experienced data analysts and scientists that closely monitored experiments throughout their lifecycle. Depending solely on scarce experts, however, is neither scalable nor bulletproof. To democratize experimentation, analysis should be streamlined and meticulously performed by engineers, managers, or others responsible for the development of a product. In this paper, based on synthesized experience of companies that run thousands of OCEs per year, we examined how experts inspect online experiments. We reveal that most of the experiment analysis happens before OCEs are even started, and we summarize the key analysis steps in three checklists. The value of the checklists is threefold. First, they can increase the accuracy of experiment setup and decision-making process. Second, checklists can enable novice data scientists and software engineers to become more autonomous in setting-up and analyzing experiments. Finally, they can serve as a base to develop trustworthy platforms and tools for OCE set-up and analysis.

  • 30.
    Fabijan, Aleksander
    et al.
    Malmö högskola, Faculty of Technology and Society (TS), Department of Computer Science and Media Technology (DVMT).
    Dmitriev, Pavel
    Microsoft Analysis & Experimentation, Redmond, USA.
    Olsson, Helena Holmström
    Malmö högskola, Faculty of Technology and Society (TS), Department of Computer Science and Media Technology (DVMT). Dep. of Computer Science, Chalmers University of Tech., Göteborg, Sweden.
    Bosh, Jan
    The Benefits of Controlled Experimentation at Scale2017In: 2017 43rd Euromicro Conference on Software Engineering and Advanced Applications (SEAA), IEEE, 2017, p. 18-26Conference paper (Refereed)
    Abstract [en]

    Online controlled experiments (for example A/B tests) are increasingly being performed to guide product development and accelerate innovation in online software product companies. The benefits of controlled experiments have been shown in many cases with incremental product improvement as the objective. In this paper, we demonstrate that the value of controlled experimentation at scale extends beyond this recognized scenario. Based on an exhaustive and collaborative case study in a large software-intensive company with highly developed experimentation culture, we inductively derive the benefits of controlled experimentation. The contribution of our paper is twofold. First, we present a comprehensive list of benefits and illustrate our findings with five case examples of controlled experiments conducted at Microsoft. Second, we provide guidance on how to achieve each of the benefits. With our work, we aim to provide practitioners in the online domain with knowledge on how to use controlled experimentation to maximize the benefits on the portfolio, product and team level.

  • 31.
    Fabijan, Aleksander
    et al.
    Malmö University, Faculty of Technology and Society (TS), Department of Computer Science and Media Technology (DVMT).
    Dmitriev, Pavel
    Olsson Holmström, Helena
    Malmö University, Faculty of Technology and Society (TS), Department of Computer Science and Media Technology (DVMT).
    Bosch, Jan
    The Online Controlled Experiment Lifecycle2020In: IEEE Software, ISSN 0740-7459, E-ISSN 1937-4194, Vol. 37, no 2, p. 60-67Article in journal (Refereed)
    Abstract [en]

    Online Controlled Experiments (OCEs) enable an accurate understanding of customer value and generate millions of dollars of additional revenue at Microsoft. Unlike other techniques for learning from customers, OCEs establish an accurate and causal relationship between a change and the impact observed. Although previous research describes technical and statistical dimensions, the key phases of online experimentation are not widely known, their impact and importance are obscure, and how to establish OCEs in an organization is underexplored. In this paper, using a longitudinal in-depth case study, we address this gap by (1) presenting the Experiment Lifecycle, and (2) demonstrating with four example experiments their profound impact. We show that OECs help optimize infrastructure needs and aid in project planning and measuring team efforts, in addition to their primary goal of accurately identifying what customers value. We conclude that product development should fully integrate the Experiment Lifecycle to benefit from the OCEs.

  • 32.
    Fabijan, Aleksander
    et al.
    Malmö högskola, Faculty of Technology and Society (TS).
    Dmitriev, Pavel
    Olsson Holmström, Helena
    Malmö högskola, Faculty of Technology and Society (TS).
    Bosh, Jan
    The Evolution of Continuous Experimentation in Software Product Development: From Data to a Data-Driven Organization at Scale2017In: International Conference on Software Engineering. Proceedings, IEEE, 2017, p. 770-780Conference paper (Refereed)
    Abstract [en]

    Software development companies are increasingly aiming to become data-driven by trying to continuously experiment with the products used by their customers. Although familiar with the competitive edge that the A/B testing technology delivers, they seldom succeed in evolving and adopting the methodology. In this paper, and based on an exhaustive and collaborative case study research in a large software-intense company with highly developed experimentation culture, we present the evolution process of moving from ad-hoc customer data analysis towards continuous controlled experimentation at scale. Our main contribution is the "Experimentation Evolution Model" in which we detail three phases of evolution: technical, organizational and business evolution. With our contribution, we aim to provide guidance to practitioners on how to develop and scale continuous experimentation in software organizations with the purpose of becoming data-driven at scale.

    Download full text (pdf)
    FULLTEXT01
  • 33.
    Fabijan, Aleksander
    et al.
    Malmö högskola, Faculty of Technology and Society (TS).
    Olsson Holmström, Helena
    Malmö högskola, Faculty of Technology and Society (TS).
    Bosch, Jan
    Commodity Eats Innovation for Breakfast: A Model for Differentiating Feature Realization2016In: Product-Focused Software Process Improvement: 17th International Conference, PROFES 2016, Trondheim, Norway, November 22-24, 2016, Proceedings, Springer, 2016, p. 517-525Conference paper (Refereed)
    Abstract [en]

    Once supporting the electrical and mechanical functionality, software today became the main competitive advantage in products. However, in the companies that we study, the way in which software features are developed still reflects the traditional ‘requirements over the wall’ approach. As a consequence, individual departments prioritize what they believe is the most important and are unable to identify which features are regularly used – ‘flow’, there to be bought – ‘wow’, differentiating and that add value to customers, or which are regarded commodity. In this paper, and based on case study research in three large software-intensive companies, we (1) provide empirical evidence that companies do not distinguish between different types of features, which causes poor allocation of R&D efforts and suppresses innovation, and (2) develop a model in which we depict the activities for differentiating and working with different types of features and stakeholders.

    Download full text (pdf)
    FULLTEXT01
  • 34.
    Fabijan, Aleksander
    et al.
    Malmö högskola, Faculty of Technology and Society (TS). Malmö högskola, Internet of Things and People (IOTAP).
    Olsson Holmström, Helena
    Malmö högskola, Faculty of Technology and Society (TS). Malmö högskola, Internet of Things and People (IOTAP).
    Bosch, Jan
    Customer Feedback and Data Collection Techniques in Software R&D: A Literature Review2015In: Software Business: 6th International Conference, ICSOB 2015, Braga, Portugal, June 10-12, 2015, Proceedings, Springer, 2015, p. 139-153Conference paper (Refereed)
    Abstract [en]

    In many companies, product management struggles in getting accurate customer feedback. Often, validation and confirmation of functionality with customers takes place only after the product has been deployed, and there are no mechanisms that help product managers to continuously learn from customers. Although there are techniques available for collecting customer feedback, these are typically not applied as part of a continuous feedback loop. As a result, the selection and prioritization of features becomes far from optimal, and product deviates from what the customers need. In this paper, we present a literature review of currently recognized techniques for collecting customer feedback. We develop a model in which we categorize the techniques according to their characteristics. The purpose of this literature review is to provide an overview of current software engineering research in this area and to better understand the different techniques that are used for collecting customer feedback.

    Download full text (pdf)
    FULLTEXT01
  • 35.
    Fabijan, Aleksander
    et al.
    Malmö högskola, Faculty of Technology and Society (TS).
    Olsson Holmström, Helena
    Malmö högskola, Faculty of Technology and Society (TS).
    Bosch, Jan
    Data-Driven Decision-Making in Product R&D2015In: Agile Processes in Software Engineering and Extreme Programming: 16th International Conference, XP 2015, Helsinki, Finland, May 25-29, 2015, Proceedings, Springer, 2015, p. 350-351Conference paper (Refereed)
    Abstract [en]

    Software development companies experience the road mapping and requirements ranking process to be complex as product management (PdM) strives in getting timely and accurate feedback from the customers. Often, companies have insufficient knowledge about how their products are being used, what features the customers appreciate and which ones will generate revenue. To address this problem, this research aims at helping the companies in closing the 'open' feedback loop that exists between PdM and customers. Moreover, the research strives at exploring techniques that can be used to involve customers in continuous validation of software functionality in order to provide PdM with the evidence needed for accurate R&D investments.

  • 36.
    Fabijan, Aleksander
    et al.
    Malmö högskola, Faculty of Technology and Society (TS). Malmö högskola, Internet of Things and People (IOTAP).
    Olsson Holmström, Helena
    Malmö högskola, Faculty of Technology and Society (TS). Malmö högskola, Internet of Things and People (IOTAP).
    Bosch, Jan
    Early Value Argumentation and Prediction: An Iterative Approach to Quantifying Feature Value2015In: Product-Focused Software Process Improvement, Springer, 2015, p. 16-23Conference paper (Refereed)
    Abstract [en]

    Companies are continuously improving their practices and ways of working in order to fulfill always-changing market requirements. As an example of building a better understanding of their customers, organizations are collecting user feedback and trying to direct their R&D efforts by e.g. continuing to develop features that deliver value to the customer. We (1) develop an actionable technique that practitioners in organizations can use to validate feature value early in the development cycle, (2) validate if and when the expected value reflects on the customers, (3) know when to stop developing it, and (4) identity unexpected business value early during development and redirect R&D effort to capture this value. The technique has been validated in three experiments in two cases companies. Our findings show that predicting value for features under development helps product management in large organizations to correctly re-prioritize R&D investments.

    Download full text (pdf)
    FULLTEXT01
  • 37.
    Fabijan, Aleksander
    et al.
    Malmö högskola, Faculty of Technology and Society (TS).
    Olsson Holmström, Helena
    Malmö högskola, Faculty of Technology and Society (TS).
    Bosch, Jan
    The Lack of Sharing of Customer Data in Large Software Organizations: Challenges and Implications2016In: Agile Processes, in Software Engineering, and Extreme Programming, Springer, 2016, p. 39-52Conference paper (Refereed)
    Abstract [en]

    With agile teams becoming increasingly multi-disciplinary and including all functions, the role of customer feedback is gaining momentum. Today, companies collect feedback directly from customers, as well as indirectly from their products. As a result, companies face a situation in which the amount of data from which they can learn about their customers is larger than ever before. In previous studies, the collection of data is often identified as challenging. However, and as illustrated in our research, the challenge is not the collection of data but rather how to share this data among people in order to make effective use of it. In this paper, and based on case study research in three large software-intensive companies, we (1) provide empirical evidence that ‘lack of sharing’ is the primary reason for insufficient use of customer and product data, and (2) develop a model in which we identify what data is collected, by whom data is collected and in what development phases it is used. In particular, the model depicts critical hand-overs where certain types of data get lost, as well as the implications associated with this. We conclude that companies benefit from a very limited part of the data they collect, and that lack of sharing of data drives inaccurate assumptions of what constitutes customer value.

    Download full text (pdf)
    FULLTEXT01
  • 38.
    Fabijan, Aleksander
    et al.
    Malmö högskola, Faculty of Technology and Society (TS).
    Olsson Holmström, Helena
    Malmö högskola, Faculty of Technology and Society (TS).
    Bosch, Jan
    Time to Say 'Good Bye': Feature Lifecycle2016In: Proceedings 42nd Euromicro Conference on Software Engineering and Advanced Applications SEAA 2016, IEEE, 2016, p. 9-16Conference paper (Refereed)
    Abstract [en]

    With continuous deployment of software functionality, a constant flow of new features to products is enabled. Although new functionality has potential to deliver improvements and possibilities that were previously not available, it does not necessary generate business value. On the contrary, with fast and increasing system complexity that is associated with high operational costs, more waste than value risks to be created. Validating how much value a feature actually delivers, project how this value will change over time, and know when to remove the feature from the product are the challenges large software companies increasingly experience today. We propose and study the concept of a software feature lifecycle from a value point of view, i.e. how companies track feature value throughout the feature lifecycle. The contribution of this paper is a model that illustrates how to determine (1) when to add the feature to a product, (2) how to track and (3) project the value of the feature during the lifecycle, and how to (4) identify when a feature is obsolete and should be removed from the product.

    Download full text (pdf)
    FULLTEXT01
  • 39.
    Fabijan, Aleksander
    et al.
    Malmö högskola, Faculty of Technology and Society (TS).
    Olsson Holmström, Helena
    Malmö högskola, Faculty of Technology and Society (TS).
    Bosh, Jan
    Differentiating Feature Realization in Software Product Development2017In: Product-Focused Software Process Improvement: Product-Focused Software Process Improvement. PROFES 2017., Springer, 2017, p. 221-236Conference paper (Refereed)
    Abstract [en]

    Software is no longer only supporting mechanical and electrical products. Today, it is becoming the main competitive advantage and an enabler of innovation. Not all software, however, has an equal impact on customers. Companies still struggle to differentiate between the features that are regularly used, there to be for sale, differentiating and that add value to customers, or which are regarded commodity. Goal: The aim of this paper is to (1) identify the different types of software features that we can find in software products today, and (2) recommend how to prioritize the development activities for each of them. Method: In this paper, we conduct a case study with five large-scale software intensive companies. Results: Our main result is a model in which we differentiate between four fundamentally different types of features (e.g. ‘Checkbox’, ‘Flow’, ‘Duty’ and ‘Wow’). Conclusions: Our model helps companies in (1) differentiating between the feature types, and (2) selecting an optimal methodology for their development (e.g. ‘Output-Driven’ vs. ‘Outcome-Driven’).

    Download full text (pdf)
    FULLTEXT01
  • 40. Felderer, Michael
    et al.
    Olsson Holmström, Helena
    Malmö University, Faculty of Technology and Society (TS), Department of Computer Science and Media Technology (DVMT).
    Rabiser, Rick
    Introduction to the special issue on quality engineering and management of software-intensive systems2019In: Journal of Systems and Software, ISSN 0164-1212, E-ISSN 1873-1228, Vol. 149, p. 533-534Article in journal (Other academic)
  • 41.
    Figalist, Iris
    et al.
    Siemens AG, Corp Technol, Munich, Germany..
    Dieffenbacher, Marco
    FAU Erlangen Nuremberg, Inst Informat Syst, Erlangen, Germany..
    Eigner, Isabella
    FAU Erlangen Nuremberg, Inst Informat Syst, Erlangen, Germany..
    Bosch, Jan
    Chalmers Univ Technol, Dept Comp Sci & Engn, Gothenburg, Sweden..
    Olsson, Helena Holmström
    Malmö University, Faculty of Technology and Society (TS), Department of Computer Science and Media Technology (DVMT).
    Elsner, Christoph
    Siemens AG, Corp Technol, Erlangen, Germany..
    Mining Customer Satisfaction on B2B Online Platforms using Service Quality and Web Usage Metrics2020In: 2020 27TH ASIA-PACIFIC SOFTWARE ENGINEERING CONFERENCE (APSEC 2020), IEEE, 2020, p. 435-444Conference paper (Refereed)
    Abstract [en]

    In order to distinguish themselves from their competitors, software service providers constantly try to assess and improve customer satisfaction. However, measuring customer satisfaction in a continuous way is often time and cost intensive, or requires effort on the customer side. Especially in B2B contexts, a continuous assessment of customer satisfaction is difficult to achieve due to potential restrictions and complex provider-customer-end user setups. While concepts such as web usage mining enable software providers to get a deep understanding of how their products are used, its application to quantitatively measure customer satisfaction has not yet been studied in greater detail. For that reason, our study aims at combining existing knowledge on customer satisfaction, web usage mining, and B2B service characteristics to derive a model that enables an automated calculation of quantitative customer satisfaction scores. We apply web usage mining to validate these scores and to compare the usage behavior of satisfied and dissatisfied customers. This approach is based on domain-specific service quality and web usage metrics and is, therefore, suitable for continuous measurements without requiring active customer participation. The applicability of the model is validated by instantiating it in a real-world B2B online platform.

  • 42.
    Figalist, Iris
    et al.
    Corporate Technology, Siemens AG, 81739, Munich, Germany.
    Elsner, Christoph
    Corporate Technology, Siemens AG, 81739, Munich, Germany.
    Bosch, Jan
    Department of Computer Science and Engineering, Chalmers University of Technology, Hörselgången 11, 412 96, Göteborg, Sweden.
    Olsson, Helena Holmström
    Malmö University, Faculty of Technology and Society (TS), Department of Computer Science and Media Technology (DVMT).
    An End-to-End Framework for Productive Use of Machine Learning in Software Analytics and Business Intelligence Solutions2020In: Product-Focused Software Process Improvement: 21st International Conference, PROFES 2020, Turin, Italy, November 25–27, 2020, Proceedings / [ed] Maurizio Morisio; Marco Torchiano; Andreas Jedlitschka, Springer, 2020, p. 217-233Conference paper (Refereed)
    Abstract [en]

    Nowadays, machine learning (ML) is an integral component in a wide range of areas, including software analytics (SA) and business intelligence (BI). As a result, the interest in custom ML-based software analytics and business intelligence solutions is rising. In practice, however, such solutions often get stuck in a prototypical stage because setting up an infrastructure for deployment and maintenance is considered complex and time-consuming. For this reason, we aim at structuring the entire process and making it more transparent by deriving an end-to-end framework from existing literature for building and deploying ML-based software analytics and business intelligence solutions. The framework is structured in three iterative cycles representing different stages in a model’s lifecycle: prototyping, deployment, update. As a result, the framework specifically supports the transitions between these stages while also covering all important activities from data collection to retraining deployed ML models. To validate the applicability of the framework in practice, we compare it to and apply it in a real-world ML-based SA/BI solution.

  • 43.
    Figalist, Iris
    et al.
    Siemens Corp Technol, Otto Hahn Ring 6, D-81739 Munich, Germany..
    Elsner, Christoph
    Siemens Corp Technol, Otto Hahn Ring 6, D-81739 Munich, Germany..
    Bosch, Jan
    Chalmers Univ Technol, Dept Comp Sci & Engn, Horselgangen 11, S-41296 Gothenburg, Sweden..
    Olsson, Helena Holmström
    Malmö University, Faculty of Technology and Society (TS), Department of Computer Science and Media Technology (DVMT).
    Breaking the vicious circle: A case study on why AI for software analytics and business intelligence does not take off in practice2022In: Journal of Systems and Software, ISSN 0164-1212, E-ISSN 1873-1228, Vol. 184, article id 111135Article in journal (Refereed)
    Abstract [en]

    In recent years, the application of artificial intelligence (AI) has become an integral part of a wide range of areas, including software engineering. By analyzing various data sources generated in software engineering, it can provide valuable insights into customer behavior, product performance, bugs and errors, and many more. In practice, however, AI for software analytics and business intelligence often remains at a prototypical stage, and the results are rarely used to make decisions based on data. To understand the underlying causes of this phenomenon, we conduct an explanatory case study consisting of and interview study and a survey on the challenges of realizing and utilizing artificial intelligence in the context of software-intensive businesses. As a result, we identify a vicious circle that prevents practitioners from moving from prototypical AI-based analytics to continuous and productively usable software analytics and business intelligence solutions. In order to break the vicious circle in a targeted manner, we identify a set of solutions based on existing literature as well as the previously conducted interviews and survey. Finally, these solutions are validated by a focus group of experts. (C) 2021 Elsevier Inc. All rights reserved.

  • 44.
    Figalist, Iris
    et al.
    Siemens Corporate Technology, Munich, Germany.
    Elsner, Christoph
    Siemens Corporate Technology, Munich, Germany.
    Bosch, Jan
    Chalmers.
    Olsson, Helena Holmström
    Malmö University, Faculty of Technology and Society (TS), Department of Computer Science and Media Technology (DVMT).
    Breaking the Vicious Circle: Why AI for software analytics and business intelligence does not take off in practice2020In: 2020 46th Euromicro Conference on Software Engineering and Advanced Applications (SEAA), IEEE, 2020, p. 5-12Conference paper (Refereed)
    Abstract [en]

    In recent years, the application of artificial intelligence (AI) has become an integral part of a wide range of areas, including software engineering. By analyzing various data sources generated in software engineering, it can provide valuable insights into customer behavior, product performance, bugs and errors, and many more. In practice, however, AI for software analytics and business intelligence often gets stuck in a prototypical stage and the results are rarely used to make decisions based on data. To understand the underlying root causes of this phenomenon, we conduct both an explanatory case study and a survey on the challenges of realizing and utilizing artificial intelligence in the context of software-intensive businesses. As a result, we identify a vicious circle that prevents practitioners from moving from prototypical analytics to continuous and productively usable software analytics and business intelligence based on AI.

  • 45. Figalist, Iris
    et al.
    Elsner, Christoph
    Bosch, Jan
    Olsson, Helena Holmström
    Malmö University, Faculty of Technology and Society (TS), Department of Computer Science and Media Technology (DVMT).
    Business as Unusual: A Model for Continuous Real-time Business Insights Based on Low Level Metrics2019In: 2019 45th Euromicro Conference On Software Engineering And Advanced Applications (SEAA 2019) / [ed] Staron, M Capilla, R Skavhaug, A, IEEE, 2019, p. 66-73Conference paper (Refereed)
    Abstract [en]

    A wide variety of tools to monitor and track software systems, such as websites or smartphone applications, during runtime already exists. However, their aggregated results are often not sufficient to answer questions on a product management level since these questions address several levels of complexity and abstractions, and tend to be formulated on a rather high level, for instance concerning the efficiency of their website structure for their users. A straightforward mapping between low level metrics and high level insights is typically not possible. This causes a gap that makes it challenging to continuously provide quantitative high-level insights in real-time. In order to address this challenge, we conducted a study within three distinct platforms and products, and propose a model based on our results. After defining a case for each of the independent platforms and products, we implemented a process to measure high level insights using low level metrics for each of these cases. Next, we compared the procedures and steps that were taken in each of the cases and derived a model that describes a generic approach how to utilize and process data in order to gain higher level insights. Our model structures the steps from data to knowledge over different levels of complexity and abstraction, namely operational, tactical, and strategic. Thereby, the knowledge acquired in each phase serves as input in the next phase which increases the measurable level of complexity with each iteration. Since the steps in our model are specifically arranged as a pipeline, it enables practitioners to automate a continuous and quantitative measurement of high level insights in real-time.

  • 46.
    Figalist, Iris
    et al.
    Corporate Technology, Siemens AG, 81739, Munich, Germany.
    Elsner, Christoph
    Corporate Technology, Siemens AG, 81739, Munich, Germany.
    Bosch, Jan
    Department of Computer Science and Engineering, Chalmers University of Technology, Hörselgången 11, 412 96, Göteborg, Sweden.
    Olsson, Helena Holmström
    Malmö University, Faculty of Technology and Society (TS), Department of Computer Science and Media Technology (DVMT).
    Customer Churn Prediction in B2B Contexts2019In: Software Business: 10th International Conference, ICSOB 2019, Jyväskylä, Finland, November 18–20, 2019, Proceedings / [ed] Sami Hyrynsalmi, Mari Suoranta, Anh Nguyen-Duc, Pasi Tyrväinen, Pekka Abrahamsson, Springer, 2019, p. 378-386Conference paper (Refereed)
    Abstract [en]

    While business-to-customer (B2C) companies, in the telecom sector for instance, have been making use of customer churn prediction for many years, churn prediction in the business-to-business (B2B) domain receives much less attention in existing literature. Nevertheless, B2B-specific characteristics, such as a lower number of customers with much higher transactional values, indicate the importance of identifying potentially churning customers. To achieve this, we implemented a prediction model for customer churn within a B2B software product and derived a model based on the results. For one, we present an approach that enables the mapping of customer- and end-user-data based on “customer phases” which allows the prediction model to take all critical influencing factors into consideration. In addition to that, we introduce a B2B customer churn prediction process based on the proposed data mapping.

  • 47.
    Figalist, Iris
    et al.
    Siemens Corporate Technology, Germany.
    Elsner, Christoph
    Siemens Corporate Technology, Germany.
    Bosch, Jan
    Chalmers University of Technology.
    Olsson, Helena Holmström
    Malmö University, Faculty of Technology and Society (TS), Department of Computer Science and Media Technology (DVMT).
    Fast and curious: A model for building efficient monitoring- and decision-making frameworks based on quantitative data2021In: Information and Software Technology, ISSN 0950-5849, E-ISSN 1873-6025, Vol. 132, article id 106458Article in journal (Refereed)
    Abstract [en]

    Context: Nowadays, the hype around artificial intelligence is at its absolute peak. Large amounts of data are collected every second of the day and a variety of tools exists to enable easy analysis of data. In practice, however, making meaningful use of it is way more challenging. For instance, affected stakeholders often struggle to specify their information needs and to interpret the results of such analyses. Objective: In this study we investigate how to enable continuous monitoring of information needs, and the generation of knowledge and insights for various stakeholders involved in the lifecycle of software-intensive products. The overarching goal is to support their decision making by providing relevant insights related to their area of responsibility. Methods: We implement multiple monitoringand decision-making frameworks for six individual, real-world cases selected from three different platforms and covering four types of stakeholders. We compare the individual procedures to derive a generic process for instantiating such frameworks as well as a model to scale it up for multiple stakeholders. Results: For one, we discovered that information needs of stakeholders are often related to a limited subset of data sources and should be specified in stages. For another, stakeholders often benefit from sharing and reusing existing components among themselves in later phases. Specifically, we identify three types of reuse: (1) Data and knowledge, (2) tools and methods, and (3) concepts. As a result, key aspects of our model are iterative feedback and specification cycles as well as the reuse of appropriate components to speed up the instantiation process and maximize the efficiency of the model. Conclusion: Our results indicate that knowledge and insights can be generated much faster and stakeholders feel the benefits of the analysis very early on by iteratively specifying information needs and by systematically sharing and reusing knowledge, tools and concepts.

  • 48. Figalist, Iris
    et al.
    Elsner, Christoph
    Bosch, Jan
    Olsson Holmström, Helena
    Malmö University, Faculty of Technology and Society (TS), Department of Computer Science and Media Technology (DVMT).
    Scaling Agile Beyond Organizational Boundaries: Coordination Challenges in Software Ecosystems2019In: Agile Processes in Software Engineering and Extreme Programming: 20th International Conference, XP 2019, Montréal, QC, Canada, May 21–25, 2019, Proceedings, Springer, 2019, p. 189-206Conference paper (Refereed)
    Abstract [en]

    The shift from sequential to agile software development originates from relatively small and co-located teams but soon gained prominence in larger organizations. How to apply and scale agile practices to fit the needs of larger projects has been studied to quite an extent in previous research. However, scaling agile beyond organizational boundaries, for instance in a software ecosystem context, raises additional challenges that existing studies and approaches do not yet investigate or address in great detail. For that reason, we conducted a case study in two software ecosystems that comprise several agile actors from different organizations and, thereby, scale development across organizational boundaries, in order to elaborate and understand their coordination challenges. Our results indicate that most of the identified challenges are caused by long communication paths and a lack of established processes to facilitate these paths. As a result, the participants in our study, among others, experience insufficient responsivity, insufficient communication of prioritizations and deliverables, and alterations or loss of information. As a consequence, agile practices need to be extended to fit the identified needs.

    Download full text (pdf)
    FULLTEXT01
  • 49.
    Fredriksson, T.
    et al.
    Chalmers University of Technology.
    Mattos, D. I.
    Chalmers University of Technology.
    Bosch, J.
    Chalmers University of Technology.
    Olsson, Helena Holmström
    Malmö University, Faculty of Technology and Society (TS), Department of Computer Science and Media Technology (DVMT).
    Assessing the Suitability of Semi-Supervised Learning Datasets using Item Response Theory2021In: Proceedings - 2021 47th Euromicro Conference on Software Engineering and Advanced Applications, SEAA 2021, IEEE, 2021, p. 326-333Conference paper (Refereed)
    Abstract [en]

    In practice, supervised learning algorithms require fully labeled datasets to achieve the high accuracy demanded by current modern applications. However, in industrial settings supervised learning algorithms can perform poorly because of few labeled instances. Semi-supervised learning (SSL) is an automatic labeling approach that utilizes complete labels to infer missing labels in partially complete datasets. The high number of available SSL algorithms and the lack of systematic comparison between them leaves practitioners without guidelines to select the appropriate one for their application. Moreover, each SSL algorithm is often validated and evaluated in a small number of common datasets. However, there is no research that examines what datasets are suitable for comparing different SSL algorihtms. The purpose of this paper is to empirically evaluate the suitability of the datasets commonly used to evaluate and compare different SSL algorithms. We performed a simulation study using twelve datasets of three different datatypes (numerical, text, image) on thirteen different SSL algorithms. The contributions of this paper are two-fold. First, we propose the use of Bayesian congeneric item response theory model to assess the suitability of commonly used datasets. Second, we compare the different SSL algorithms using these datasets. The results show that with except of three datasets, the others have very low discrimination factors and are easily solved by the current algorithms. Additionally, the SSL algorithms have overlapping 90% credible intervals, indicating uncertainty in the difference between the accuracy of these SSL models. The paper concludes suggesting that researchers and practitioners should better consider the choice of datasets used for comparing SSL algorithms.

  • 50.
    Fredriksson, Teodor
    et al.
    Chalmers University of Technology, Gothenburg, Sweden.
    Bosch, Jan
    Chalmers University of Technology, Gothenburg, Sweden.
    Olsson, Helena Holmström
    Malmö University, Faculty of Technology and Society (TS), Department of Computer Science and Media Technology (DVMT).
    Classification of Complex-Valued Radar Data using Semi-Supervised Learning: a Case Study2023In: 2023 49th Euromicro Conference on Software Engineering and Advanced Applications (SEAA), Institute of Electrical and Electronics Engineers (IEEE), 2023Conference paper (Refereed)
    Abstract [en]

    In recent years, the interest in applying machine learning (ML) and deep learning (DL) has been increasing due to their ability to learn to predict and find structure in data. The most common approach of ML and DL is supervised learning. Supervised learning requires the input data to be labeled. However, as reported by many industries, such as the embedded systems domain, fully labeled datasets are difficult to obtain since data labeling is manually intensive. This paper uses a semi-supervised learning approach on real-world Pulse-Doppler data obtained from our industry collaborator Saab to address this challenge. We took inspiration from the FixMatch algorithm. To investigate whether unlabeled data can help improve classification accuracy, we compare FixMatch to a supervised baseline. We use five different settings for the number of available labels per class label to investigate how many labeled instances and how much manual effort is required for optimal accuracy. Bayesian Linear Regression is used to analyze the results. The results show that FixMatch can reach a higher accuracy than the supervised baseline. Furthermore, FixMatch requires more computation time but will help reduce manual effort. In addition, FixMatch will not underfit or overfit. Thanks to this study, practitioners know the benefits of utilizing FixMatch and when it is safe to use to improve a supervised baseline in the industry.

123 1 - 50 of 142
CiteExportLink to result list
Permanent link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf