Malmö University Publications
Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Effective Online Controlled Experiment Analysis at Large Scale
Malmö University, Faculty of Technology and Society (TS), Department of Computer Science and Media Technology (DVMT).ORCID iD: 0000-0003-4908-2708
Microsoft, Analysis and Experimentation, Redmond, United States.
Malmö University, Faculty of Technology and Society (TS), Department of Computer Science and Media Technology (DVMT).ORCID iD: 0000-0002-7700-1816
Chalmers University of Tech., Dep. of Computer Science, Göteborg, Sweden.
2018 (English)In: Proceedings of the EUROMICRO Conference, IEEE, 2018, p. 64-67Conference paper, Published paper (Refereed)
Abstract [en]

Online Controlled Experiments (OCEs) are the norm in data-driven software companies because of the benefits they provide for building and deploying software. Product teams experiment to accurately learn whether the changes that they do to their products (e.g. adding new features) cause any impact (e.g. customers use them more frequently). Experiments also help reduce the risk from deploying software by minimizing the magnitude and duration of harm caused by software bugs, allowing software to be shipped more frequently. To make informed decisions in product development, experiment analysis needs to be granular with a large number of metrics over heterogeneous devices and audiences. Discovering experiment insights by hand, however, can be cumbersome. In this paper, and based on case study research at a large-scale software development company with a long tradition of experimentation, we (1) describe the standard process of experiment analysis, and (2) introduce an artifact to improve the effectiveness and comprehensiveness of this process.

Place, publisher, year, edition, pages
IEEE, 2018. p. 64-67
Series
Proceedings of the Euromicro Conference, ISSN 1089-6503
Keywords [en]
Online Controlled Experiments, A/B testing, Guided Experiment Analysis
National Category
Engineering and Technology
Identifiers
URN: urn:nbn:se:mau:diva-12777DOI: 10.1109/SEAA.2018.00020ISI: 000450238900011Scopus ID: 2-s2.0-85057181553Local ID: 27274OAI: oai:DiVA.org:mau-12777DiVA, id: diva2:1409824
Conference
44th Euromicro Conference on Software Engineering and Advanced Applications (SEAA), Prague, Czech Republic (29-31 Aug. 2018)
Available from: 2020-02-29 Created: 2020-02-29 Last updated: 2024-06-17Bibliographically approved

Open Access in DiVA

No full text in DiVA

Other links

Publisher's full textScopushttp://dsd-seaa2018.fit.cvut.cz/seaa/

Authority records

Fabijan, AleksanderOlsson, Helena Holmström

Search in DiVA

By author/editor
Fabijan, AleksanderOlsson, Helena Holmström
By organisation
Department of Computer Science and Media Technology (DVMT)
Engineering and Technology

Search outside of DiVA

GoogleGoogle Scholar

doi
urn-nbn

Altmetric score

doi
urn-nbn
Total: 31 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf