Malmö University Publications
Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
The Evolution of Continuous Experimentation in Software Product Development: From Data to a Data-Driven Organization at Scale
Malmö högskola, Faculty of Technology and Society (TS).ORCID iD: 0000-0003-4908-2708
Analysis and Experimentation Microsoft, One Microsoft Way, Redmond, 98052, WA, United States.
Malmö högskola, Faculty of Technology and Society (TS).ORCID iD: 0000-0002-7700-1816
Chalmers University of Technology, Dep. of Computer Science and Eng, GOteborg, Sweden.
2017 (English)In: International Conference on Software Engineering. Proceedings, IEEE, 2017, p. 770-780Conference paper, Published paper (Refereed)
Abstract [en]

Software development companies are increasingly aiming to become data-driven by trying to continuously experiment with the products used by their customers. Although familiar with the competitive edge that the A/B testing technology delivers, they seldom succeed in evolving and adopting the methodology. In this paper, and based on an exhaustive and collaborative case study research in a large software-intense company with highly developed experimentation culture, we present the evolution process of moving from ad-hoc customer data analysis towards continuous controlled experimentation at scale. Our main contribution is the "Experimentation Evolution Model" in which we detail three phases of evolution: technical, organizational and business evolution. With our contribution, we aim to provide guidance to practitioners on how to develop and scale continuous experimentation in software organizations with the purpose of becoming data-driven at scale.

Place, publisher, year, edition, pages
IEEE, 2017. p. 770-780
Keywords [en]
A/B testing, continuous experimentation, data science, customer feedback, continuous product innovation, Experimentation Evolution Model, Experiment Owner
National Category
Engineering and Technology
Identifiers
URN: urn:nbn:se:mau:diva-12651DOI: 10.1109/ICSE.2017.76ISI: 000427091300068Scopus ID: 2-s2.0-85027682332Local ID: 24149OAI: oai:DiVA.org:mau-12651DiVA, id: diva2:1409698
Conference
International Conference on Software Engineering (ICSE), Buenos Aires, Argentina (20-28 May 2017)
Available from: 2020-02-29 Created: 2020-02-29 Last updated: 2024-06-18Bibliographically approved
In thesis
1. Data-Driven Software Development at Large Scale: from Ad-Hoc Data Collection to Trustworthy Experimentation
Open this publication in new window or tab >>Data-Driven Software Development at Large Scale: from Ad-Hoc Data Collection to Trustworthy Experimentation
2018 (English)Doctoral thesis, comprehensive summary (Other academic)
Abstract [en]

Accurately learning what customers value is critical for the success of every company. Despite the extensive research on identifying customer preferences, only a handful of software companies succeed in becoming truly data-driven at scale. Benefiting from novel approaches such as experimentation in addition to the traditional feedback collection is challenging, yet tremendously impactful when performed correctly. In this thesis, we explore how software companies evolve from data-collectors with ad-hoc benefits, to trustworthy data-driven decision makers at scale. We base our work on a 3.5-year longitudinal multiple-case study research with companies working in both embedded systems domain (e.g. engineering connected vehicles, surveillance systems, etc.) as well as in the online domain (e.g. developing search engines, mobile applications, etc.). The contribution of this thesis is three-fold. First, we present how software companies use data to learn from customers. Second, we show how to adopt and evolve controlled experimentation to become more accurate in learning what customers value. Finally, we provide detailed guidelines that can be used by companies to improve their experimentation capabilities. With our work, we aim to empower software companies to become truly data-driven at scale through trustworthy experimentation. Ultimately this should lead to better software products and services.

Place, publisher, year, edition, pages
Malmö university, Faculty of Technology and society, 2018. p. 357
Series
Studies in Computer Science ; 6
National Category
Engineering and Technology
Identifiers
urn:nbn:se:mau:diva-7768 (URN)10.24834/2043/24873 (DOI)24873 (Local ID)9789171049186 (ISBN)9789171049193 (ISBN)24873 (Archive number)24873 (OAI)
Public defence
2018-06-15, NI:B0E07, Nordenskiöldsgatan 1, 13:00 (English)
Opponent
Note

In reference to IEEE copyrighted material which is used with permission in this thesis, the IEEE does not endorse any of Malmö University's products or services. Internal or personal use of this material is permitted. If interested in reprinting/republishing IEEE copyrighted material for advertising or promotional purposes or for creating new collective works for resale or redistribution, please go to http://www.ieee.org/publications_standards/publications/rights/rights_link.html to learn how to obtain a License from RightsLink.

Available from: 2020-02-28 Created: 2020-02-28 Last updated: 2024-04-04Bibliographically approved

Open Access in DiVA

fulltext(3021 kB)1161 downloads
File information
File name FULLTEXT01.pdfFile size 3021 kBChecksum SHA-512
3fbd9b2c38c38f0469dc85262eccf78fa8949a16ce15527782a7652ee2ceecc9456e2aa5cc968fad3c2e45a412b16cd94c883765a2ca8f2486bec8aa6c54d3e0
Type fulltextMimetype application/pdf

Other links

Publisher's full textScopushttp://icse2017.gatech.edu/

Authority records

Fabijan, AleksanderOlsson Holmström, Helena

Search in DiVA

By author/editor
Fabijan, AleksanderOlsson Holmström, Helena
By organisation
Faculty of Technology and Society (TS)
Engineering and Technology

Search outside of DiVA

GoogleGoogle Scholar
Total: 1161 downloads
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

doi
urn-nbn

Altmetric score

doi
urn-nbn
Total: 97 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf