Every time we conduct a Google Search, browse the Netflix catalog, check our Facebook feeds or call an Uber we are part of multiple experiments. Our interactions with these services shape the decisions on how these services evolve and thanks to large and ever growing user bases experimentation becomes easier for these companies over time, therefore accelerating the product development cycle.
In this talk we will look behind the scenes of data driven cultures, explore why Google does not (exclusively) rely on user click data for evaluating search ranking experiments and why Netflix optimizes for a single northstar metric. A|B testing is a simple but effective tool for establishing a causal relationship between product changes and the impact on business metrics, but there is a minefield of errors that can easily invalidate the outcomes.

Time: 10:15 - 10:45
Track: Track 3
Speaker: Jürgen Schwärzler