PAPIs 2018 has ended
Back To Schedule
Wednesday, October 17 • 2:30pm - 2:50pm
Would you have clicked on what we would have recommended?

Sign up or log in to save this to your schedule, view media, leave feedback and see who's attending!

Feedback form is now closed.
In this talk, we describe recent work on the offline estimation of recommender system A/B tests using counterfactual reasoning techniques. We can determine whether our customers would have clicked on what we would have recommended by adding stochasticity to our recommendations. This ensures non-zero probability of having shown our new recommendations at some point in the past, which can leverage using a technique known as Pareto-smoothed importance sampling. This allows us to create a low-bias, low-variance estimator of how our recommender systems would have performed had they been deployed.

avatar for Peter B. Golbus

Peter B. Golbus

Senior Data Scientist, Wayfair
Peter B. Golbus is a Senior Data Scientist at Wayfair. Peter joined Wayfair directly from his Ph.D. program at Northeastern University where he studied the offline evaluation of search engines with Javed A. Aslam. Four years later, he is still at Wayfair, and is now studying the offline... Read More →

Wednesday October 17, 2018 2:30pm - 2:50pm EDT
Horace Mann