Malmö University Publications
Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
The Anatomy of a Large-Scale Experimentation Platform
Analysis and Experimentation Team, Microsoft Bellevue, WA, United States.
Analysis and Experimentation Team, Microsoft Bellevue, WA, United States.
Analysis and Experimentation Team, Microsoft Bellevue, WA, United States.
Analysis and Experimentation Team, Microsoft Bellevue, WA, United States.
Show others and affiliations
2018 (English)In: 2018 IEEE International Conference on Software Architecture (ICSA), IEEE, 2018Conference paper, Published paper (Refereed)
Abstract [en]

Online controlled experiments (e.g., A/B tests) are an integral part of successful data-driven companies. At Microsoft, supporting experimentation poses a unique challenge due to the wide variety of products being developed, along with the fact that experimentation capabilities had to be added to existing, mature products with codebases that go back decades. This paper describes the Microsoft ExP Platform (ExP for short) which enables trustworthy A/B experimentation at scale for products across Microsoft, from web properties (such as bing.com) to mobile apps to device drivers within the Windows operating system. The two core tenets of the platform are trustworthiness (an experiment is meaningful only if its results can be trusted) and scalability (we aspire to expose every single change in any product through an A/B experiment). Currently, over ten thousand experiments are run annually. In this paper, we describe the four core components of an A/B experimentation system: experimentation portal, experiment execution service, log processing service and analysis service, and explain the reasoning behind the design choices made. These four components work together to provide a system where ideas can turn into experiments within minutes and those experiments can provide initial trustworthy results within hours.

Place, publisher, year, edition, pages
IEEE, 2018.
National Category
Engineering and Technology
Identifiers
URN: urn:nbn:se:mau:diva-12386DOI: 10.1109/ICSA.2018.00009ISI: 000492762900001Scopus ID: 2-s2.0-85051109145Local ID: 28037OAI: oai:DiVA.org:mau-12386DiVA, id: diva2:1409433
Conference
IEEE International Conference on Software Architecture (ICSA), Seattle, WA, USA (2018)
Available from: 2020-02-29 Created: 2020-02-29 Last updated: 2024-06-17Bibliographically approved

Open Access in DiVA

No full text in DiVA

Other links

Publisher's full textScopushttps://ieeexplore.ieee.org/document/8417111

Authority records

Fabijan, Aleksander

Search in DiVA

By author/editor
Fabijan, Aleksander
By organisation
Department of Computer Science and Media Technology (DVMT)
Engineering and Technology

Search outside of DiVA

GoogleGoogle Scholar

doi
urn-nbn

Altmetric score

doi
urn-nbn
Total: 72 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf