Consumer Heterogeneity and Paid Search Effectiveness: A Large-Scale Field Experiment

For your marketing

Possible founder application

Shows why platform-attributed conversions can overstate incrementality and why brand and non-brand traffic, user familiarity and holdout design should be analyzed separately.

Separate brand from non-brand campaigns before evaluating results. Where practical, compare exposed customers or markets against a randomized holdout, and distinguish new customers from people who already know the product. Use the difference in outcomes between groups to judge additional business generated, rather than treating every ad-attributed conversion as an additional customer.

Reported result

What the study found

The authors report little measurable short-run benefit from brand-keyword ads in this setting. Non-brand ads had positive responses among some newer and infrequent users, while many ad-attributed actions came from customers whose behavior was not changed by advertising; estimated average returns were negative under the studied conditions.

Research question

What the authors investigated

What is the causal short-run effect of paid search advertising on eBay, and how does it vary across consumers?

Design

A large-scale field experiment varied paid-search advertising across geographic markets to estimate causal responses and compare users with different prior familiarity and purchase activity.

Sample and setting

eBay users and search activity in the markets included in the experiment. The findings concern one large, established online marketplace and the specific campaigns and time horizon studied.

Scope and transfer

Limitations to keep in view

One large company with strong brand recognition, specific paid-search campaigns, and short-run outcomes. The results do not imply that all paid search is ineffective, particularly for less familiar products or different auction and conversion conditions.

Research sources

Peer-reviewed article metadata and author-hosted manuscript; findings summarized from the paper abstract and manuscript passages.