SVP, Data Science
Title: Estimating Causal Effect of Ads in a Real-Time Bidding Platform
Abstract: A real-time bidding platform responds to incoming ad-opportunities ("bid requests") by deciding whether or not to submit a bid and how much to bid. If the submitted bid wins, the user is shown an ad. Advertisers hope that ad-exposure leads to an increased likelihood of a desired action, such as a click or conversion (purchase, etc). So an important quantity that advertisers want to measure is the causal effect of advertising, namely, what is the response probability of an exposed user, compared with the counterfactual (un-observable) response-rate of the user if they were not exposed to the ad. In an ideal randomized test, the user is randomly assigned to test or control AFTER the submitted bid is won, and test users are served the ad in the normal way, while control users are not. While this is ideal from a statistical perspective, in practice this approach has the drawback that money spent by advertisers is wasted when a user is assigned to control. At Media Math we have developed a methodology for causal effect measurement where users are assigned to test or control BEFORE bid submission. One challenge here is that not all test-group users are exposed to an ad; only a winning bid results in ad exposure, and the winning population can have a significant bias. This talk will describe our approach to handle this and other challenges to ad impact measurement in this setting, and how we use MCMC Gibbs sampling to arrive at confidence intervals for ad-impact.