Paid media and measurement · Large-scale randomized field experiment
Social Advertising Effectiveness Across Products: A Large-Scale Field Experiment
Huang, S., Aral, S., Hu, Y. J., & Brynjolfsson, E. (2020). Marketing Science, 39(6), 1142–1165.
Plain-English take
The experiment finds meaningful differences across customers in social advertising lift across categories and user characteristics. For your marketing: When testing paid social, segment hypotheses by product and audience rather than assuming platform-wide average effectiveness.
How much weight to give this
4.5/5 Strong design for its specific question
The 1–5 evidence-check score is a practical reading aid, not a formal peer review or a probability that a conclusion is true. It combines design strength (2 points), sample reporting (1), comparison/control clarity (1), and how much of the paper could be checked (1). Access affects how confidently this directory can assess a study, not the study's underlying quality. Blinding is reported separately because it is not relevant to every design.
Random assignment provides a strong comparison for effects in the tested setting.
A usable sample size is not stated in the profile's accessible source.
Randomized comparison; confirm the control condition and analysis in the paper.
An accessible full paper or manuscript is linked, making methods easier to check.
Double-blind? Blinding is not reported in the source reviewed; that does not establish whether blinding was or was not used.
Research question
What the authors investigated
How does social advertising effectiveness vary across product categories and consumer characteristics?
Design
The researchers partnered with a large online social network to run a randomized field experiment measuring social advertising effectiveness across 71 product categories.
Sample and setting
A large online social network and advertising across 71 product categories; the paper reports variation by product and user status.
Reported result
What the study found
The experiment finds meaningful heterogeneity in social advertising lift across categories and user characteristics. The authors report that social cues increase effectiveness more for some product types and users than for others.
This is a paraphrase of the source. Check the original paper for definitions, estimates, statistical uncertainty and analysis details.
Scope and transfer
Limitations to keep in view
The experiment was run on one social network with participating advertisers and products. Effects are platform-, category- and campaign-dependent and should not be generalized into one expected lift for a SaaS company.
Possible founder application
When testing paid social, segment hypotheses by product and audience rather than assuming platform-wide average effectiveness. Use randomization and business outcomes to estimate your own incremental lift.
Evidence access
Publisher record and author manuscript
INFORMS publisher record and accessible author manuscript; study scope and randomized design checked against the paper record.
If only an abstract or metadata was available, the profile is limited to claims supported there. A paper's paywall does not change its evidentiary status, but it limits what we can substantiate from accessible text.