Paid media and measurement · Large-scale randomized geographic field experiment
Consumer Heterogeneity and Paid Search Effectiveness: A Large-Scale Field Experiment
Blake, T., Nosko, C., & Tadelis, S. (2015). Econometrica, 83(1), 155–174.
Plain-English take
The study found little measurable short-run benefit from brand-keyword ads in this study. For your marketing: This shows why platform-attributed conversions can overstate incrementality and why brand and non-brand traffic, user familiarity and holdout design should be analyzed separately.
How much weight to give this
4.5/5 Strong design for its specific question
The 1–5 evidence-check score is a practical reading aid, not a formal peer review or a probability that a conclusion is true. It combines design strength (2 points), sample reporting (1), comparison/control clarity (1), and how much of the paper could be checked (1). Access affects how confidently this directory can assess a study, not the study's underlying quality. Blinding is reported separately because it is not relevant to every design.
Random assignment provides a strong comparison for effects in the tested setting.
A usable sample size is not stated in the profile's accessible source.
Randomized comparison; confirm the control condition and analysis in the paper.
An accessible full paper or manuscript is linked, making methods easier to check.
Double-blind? Blinding is not reported in the source reviewed; that does not establish whether blinding was or was not used.
Research question
What the authors investigated
What is the causal short-run effect of paid search advertising on eBay, and how does it vary across consumers?
Design
A large-scale field experiment varied paid-search advertising across geographic markets to estimate causal responses and compare users with different prior familiarity and purchase activity.
Sample and setting
eBay users and search activity in the markets included in the experiment. The findings concern one large, established online marketplace and the specific campaigns and time horizon studied.
Reported result
What the study found
The authors report little measurable short-run benefit from brand-keyword ads in this setting. Non-brand ads had positive responses among some newer and infrequent users, while many ad-attributed actions came from customers whose behavior was not changed by advertising; estimated average returns were negative under the studied conditions.
This is a paraphrase of the source. Check the original paper for definitions, estimates, statistical uncertainty and analysis details.
Scope and transfer
Limitations to keep in view
One large company with strong brand recognition, specific paid-search campaigns, and short-run outcomes. The results do not imply that all paid search is ineffective, particularly for less familiar products or different auction and conversion conditions.
Possible founder application
Shows why platform-attributed conversions can overstate incrementality and why brand and non-brand traffic, user familiarity and holdout design should be analyzed separately.
Evidence access
Publisher record and author-hosted paper
Peer-reviewed article metadata and author-hosted manuscript; findings summarized from the paper abstract and manuscript passages.
If only an abstract or metadata was available, the profile is limited to claims supported there. A paper's paywall does not change its evidentiary status, but it limits what we can substantiate from accessible text.