Acquisition and retention · Randomized field experiments
Personalization in Email Marketing: The Role of Noninformative Advertising Content
Sahni, N. S., Wheeler, S. C., & Chintagunta, P. (2018). Marketing Science, 37(2), 236–258.
Plain-English take
In the main experiment, adding the recipient’s name to the subject line raised the open probability from 9.05% to 10.80%, increased sales leads from 0.39% to 0.51%, and reduced unsubscribes from 1.2% to 1.0%. For your marketing: This is a strong example of testing a narrowly defined personalization change with downstream outcomes.
How much weight to give this
4.0/5 Useful evidence with meaningful caveats
The 1–5 evidence-check score is a practical reading aid, not a formal peer review or a probability that a conclusion is true. It combines design strength (2 points), sample reporting (1), comparison/control clarity (1), and how much of the paper could be checked (1). Access affects how confidently this directory can assess a study, not the study's underlying quality. Blinding is reported separately because it is not relevant to every design.
Random assignment provides a strong comparison for effects in the tested setting.
A usable sample size is not stated in the profile's accessible source.
Randomized comparison; confirm the control condition and analysis in the paper.
This profile could be checked against an abstract or metadata, not the full paper.
Double-blind? Blinding is not reported in the source reviewed; that does not establish whether blinding was or was not used.
Research question
What the authors investigated
Does adding consumer-specific but noninformative content to an email affect recipient response?
Design
Randomized field experiments with three companies selling different products; email messages were tailored with details such as recipients’ names.
Sample and setting
The experiments sent messages to millions of individuals. The abstract reports detailed results for a main experiment and replication across different advertiser objectives; it does not give a single pooled sample size.
Reported result
What the study found
In the main experiment, adding the recipient’s name to the subject line raised the open probability from 9.05% to 10.80%, increased sales leads from 0.39% to 0.51%, and reduced unsubscribes from 1.2% to 1.0%. Similar experiments at other firms found benefits across acquisition and retention objectives.
This is a paraphrase of the source. Check the original paper for definitions, estimates, statistical uncertainty and analysis details.
Scope and transfer
Limitations to keep in view
Effects are from specific companies, messages, audiences and campaign conditions. The main treatment is name insertion, not deep behavioral personalization. Email measurement and privacy practices have changed since these campaigns; the percentages should not be used as current benchmarks.
Possible founder application
A strong example of testing a narrowly defined personalization change with downstream outcomes. For a current campaign, randomize the treatment and measure outcomes that matter beyond opens.
Evidence access
Publisher and author abstract; full article access may vary
Publisher and Stanford GSB abstract; detailed full text was not available in the retrieved pages. The summary is restricted to the reported abstract results.
If only an abstract or metadata was available, the profile is limited to claims supported there. A paper's paywall does not change its evidentiary status, but it limits what we can substantiate from accessible text.