A Comparison of Approaches to Advertising Measurement: Evidence from Big Field Experiments at Facebook

For your marketing

Possible founder application

Treat attribution models as hypotheses and validate them against holdouts where possible. When experiments are infeasible, document assumptions and do not present platform-attributed conversions as incremental impact.

Reported result

What the study found

Commonly used observational approaches based on industry-available data often failed to accurately measure the true advertising effect in these comparisons.

Research question

What the authors investigated

Can commonly used observational methods recover the causal effect of online ads as measured by randomized experiments?

Design

The authors compare observational measurement approaches with randomized Facebook ad experiments across 15 campaigns; the campaigns met minimum sample and conversion-tracking criteria.

Sample and setting

Fifteen Facebook campaigns, each with at least one million users and business-relevant conversion tracking, selected to compare measurement methods.

Scope and transfer

Limitations to keep in view

The analysis concerns a selected set of Facebook campaigns and methods available at the time. Results do not imply every observational model fails equally or that randomized tests are feasible for every startup campaign.

Research sources

INFORMS publisher abstract and author-hosted paper; the 15-campaign design and minimum sample criteria are specified in the manuscript.