AdvertisingEmail marketing
Evidence check: 4.0/5Marketing Science · 2018
For your marketing
A strong example of testing a narrowly defined personalization change with downstream outcomes. For a current campaign, randomize the treatment and measure outcomes that matter beyond opens.
Plain-English take
In the main experiment, adding the recipient’s name to the subject line raised the open probability from 9.05% to 10.80%, increased sales leads from 0.39% to 0.51%, and reduced unsubscribes from 1.2% to 1.0%.
Learn more →
Advertising
Evidence check: 4.5/5Econometrica · 2015
For your marketing
Shows why platform-attributed conversions can overstate incrementality and why brand and non-brand traffic, user familiarity and holdout design should be analyzed separately.
Plain-English take
The study found little measurable short-run benefit from brand-keyword ads in this study.
Learn more →
AdvertisingPricing
Evidence check: 3.5/5Management Science · 2017
For your marketing
Measure discount-induced acquisition and spillovers separately from subsidized purchases. Include margin, timing shifts and post-promotion behavior when judging whether offers function as profitable advertising.
Plain-English take
During the offer period, offers increased average expenditure by $3.03 (37.2%).
Learn more →
Advertising
Evidence check: 2.5/5International Journal of Research in Marketing · 2020
For your marketing
Separate near-term sales objectives from longer-horizon brand or firm-value goals when comparing channels. Treat the study as context for measurement choices, not a universal channel ranking.
Plain-English take
Paid search had a more positive effect on sales than offline advertising, while display advertising had a relatively more positive effect on Tobin's q than offline advertising, consistent with different short- and long-run economic roles.
Learn more →
AdvertisingPricing
Evidence check: 3.0/5Journal of the Academy of Marketing Science · 2012
For your marketing
When promoting one tier or feature, examine whether demand shifts across the rest of the portfolio. For software, test the analogous upgrade, downgrade and add-on outcomes rather than importing retail estimates.
Plain-English take
About half of the promotions expanded own-category revenue, particularly deeper supported discounts.
Learn more →
AcquisitionAdvertising
Evidence check: 2.5/5Marketing Science · 2011
For your marketing
Track customer lifetime across channels when estimating paid-search value. Include repeat purchases and offline or sales-assisted spillovers instead of stopping at a platform-attributed conversion.
Plain-English take
Customers acquired through Google search advertising had higher transaction rates than customers from other channels in this study.
Learn more →
AcquisitionAdvertising
Evidence check: 2.5/5Journal of Marketing Research · 2006
For your marketing
Compare the long-term value of discounted and full-price cohorts, accounting for selection and acquisition cost. A lower initial conversion cost can bring customers with weaker repeat economics.
Plain-English take
In both datasets, deeper acquisition discounts were associated with lower repeat-buying rates and lower customer asset value.
Learn more →
AdvertisingContent marketing
Evidence check: 3.5/5Journal of Business Research · 2021
For your marketing
A concrete example of testing small post-format decisions against business outcomes. Re-run the comparison on your current platform and audience before adopting it.
Plain-English take
The experiment reports that placing the web link in the comments rather than the caption was more lucrative in the studied campaign.
Learn more →
AdvertisingFree trialsSales
Evidence check: 3.5/5Marketing Science · 2004
For your marketing
Separate new demand from timing shifts and substitution when evaluating free access. Define incremental outcomes and follow customers long enough to see whether a free experience creates durable adoption.
Plain-English take
Unlike coupons, the studied free samples produced measurable long-term sales effects observable up to 12 months later.
Learn more →
Advertising
Evidence check: 4.5/5Marketing Science · 2019
For your marketing
Treat attribution models as hypotheses and validate them against holdouts where possible. When experiments are infeasible, document assumptions and do not present platform-attributed conversions as incremental impact.
Plain-English take
Commonly used observational approaches based on industry-available data often failed to accurately measure the true advertising effect in these comparisons.
Learn more →
Advertising
Evidence check: 4.0/5Marketing Science · 2024
For your marketing
Test how creative changes demand across audience segments, not only average clicks. Identify which visitors the message attracts or repels and connect ad response to meaningful product outcomes.
Plain-English take
Ad content can polarize consumer responses.
Learn more →
Advertising
Evidence check: 4.0/5Marketing Science · 2017
For your marketing
Plan experiments around outcome variance and expected effect size. Large audiences alone do not guarantee useful estimates; choose an outcome and design that can detect a decision-relevant change.
Plain-English take
The paper's main point is that more granular data do not necessarily make advertising effects easier to estimate and examines tradeoffs between measurement precision and experimentation resources.
Learn more →
AdvertisingBuyer behavior
Evidence check: 4.0/5Journal of Marketing Research · 2015
For your marketing
Measure how campaigns affect more than the final conversion event, while preserving the experimental control that supports causal estimates. Check the paper's full results before mapping its model to a SaaS funnel.
Plain-English take
The paper uses randomized exposure data to estimate how display advertising affects multiple purchase-funnel outcomes rather than relying only on last-click attribution.
Learn more →
Advertising
Evidence check: 4.5/5Marketing Science · 2020
For your marketing
When testing paid social, segment hypotheses by product and audience rather than assuming platform-wide average effectiveness. Use randomization and business outcomes to estimate your own incremental lift.
Plain-English take
The experiment finds meaningful differences across customers in social advertising lift across categories and user characteristics.
Learn more →
AdvertisingReviews
Evidence check: 2.5/5Journal of Marketing Research · 2025
For your marketing
Social proof may matter more when brand familiarity is low, but the effect can depend on audience behavior. Test against downstream conversion and avoid optimizing for visible engagement alone.
Plain-English take
Displaying more likes increased click-through rates for lesser-known brands but not for well-known brands.
Learn more →
AdvertisingPricing
Evidence check: 2.5/5Marketing Science · 2001
For your marketing
When discounting, separate category growth from switching between vendors or purchase timing. For software, define the analogous incremental outcome and test it against a non-discounted control.
Plain-English take
The research estimates category-demand expansion effects rather than only brand switching and tests how marketing intensity and competition moderate promotion response.
Learn more →
Advertising
Evidence check: 2.5/5Marketing Science · 2013
For your marketing
Treat creative sequence and repetition as testable variables. Log exposure history and compare conversion outcomes when rotating messages, while accounting for audience and frequency differences.
Plain-English take
The model distinguishes exposure histories and creative content; simulations suggest advertisers can increase website visits and conversions by varying the creative shown to a person.
Learn more →
AdvertisingCloud computing
Evidence check: 2.5/5Marketing Science · 2017
For your marketing
A reminder that context can moderate campaign response. For location-sensitive products, test contextual variables with controlled allocation and include the cost and practical value of targeting.
Plain-English take
The study found that response was higher and faster in sunny than cloudy conditions, and lower and slower in rainy than cloudy conditions.
Learn more →
AdvertisingFree trialsProduct adoption
Evidence check: 2.5/5Marketing Science · 2016
For your marketing
Design trials around the user's chance to experience core value and the maturity of the product. Track activation and service quality during the trial alongside conversion, and test timing in your own setting.
Plain-English take
The abstract characterizes free trials as a double-edged promotion: trial timing and how intensively consumers use the service during the trial matter to effectiveness, particularly while quality is uncertain and changing.
Learn more →
AdvertisingPricing
Evidence check: 4.0/5Marketing Science · 2025
For your marketing
Shows why ad load can have delayed effects and why short tests may miss them. Treat the numbers as a measurement-design example, not a target for SaaS paywalls or sponsorship inventory.
Plain-English take
Consumption responses took more than a year to stabilize, while subscription responses stabilized in under six months.
Learn more →
Advertising
Evidence check: 4.0/5Journal of Marketing Research · 2016
For your marketing
When evaluating category advertising, account for the possibility that your spend increases category interest while also helping alternatives. Incrementality and competitor outcomes matter alongside attributed conversions.
Plain-English take
Ads increased sales likelihood for the advertiser and also for nonadvertised restaurants that served the same cuisine and had high ratings.
Learn more →
Advertising
Evidence check: 4.0/5Marketing Science · 2025
For your marketing
Useful for thinking about privacy-related measurement loss and incrementality, especially for small advertisers. The Meta campaign economics and targeting system are platform-specific and should not be treated as a universal CAC benchmark.
Plain-English take
The median cost per additional customer was estimated at $38.16 at baseline and $49.93 under the median loss in effectiveness when offsite data could not be used, a 31% increase.
Learn more →
Advertising
Evidence check: 2.5/5Journal of Marketing Research · 2017
For your marketing
A foundational reference for asking an ad platform or analytics team how holdouts can measure incremental impact at lower cost. Use current, independently validated implementation details for any live test.
Plain-English take
In the demonstration campaign, retargeting lifted website visits by 17.2% and purchases by 10.5%.
Learn more →
Advertising
Evidence check: 2.5/5Journal of Marketing Research · 2014
For your marketing
A useful framework for considering channel sequence, carryover, and retargeting harm when last-click reports over-credit later touches. Pair modeled attribution with holdouts where feasible.
Plain-English take
Attribution based on estimated channel effects differed significantly from common metrics.
Learn more →
AdvertisingEmail marketingSales
Evidence check: 2.5/5Information Systems Research · 2025
For your marketing
Encourages measuring email's full downstream value rather than clicks alone. For SaaS, connect email exposure to later product activation, opportunities, and revenue with a credible design.
Plain-English take
Email advertising was associated with increased sales across channels through higher purchase probability and a wider variety of products bought; effects varied across product types and customer segments.
Learn more →
Advertising
Evidence check: 4.5/5Management Science · 2023
For your marketing
Highlights the difference between predicting who will buy and identifying who changes behavior because of an ad. Ask for incrementality tests and examine whether optimization targets persuadable users.
Plain-English take
Ads increased purchase and return probabilities, but consumers targeted more aggressively were not more responsive to ads.
Learn more →
Advertising
Evidence check: 3.5/5Management Science · 2025
For your marketing
A useful test idea for combining audience intent with page context and matching the creative to a meaningful next action. Measure pipeline or product activation beyond soft conversions.
Plain-English take
Combining contextual targeting with retargeting improved website visits, engagement, and soft conversions.
Learn more →
Advertising
Evidence check: 3.5/5Quantitative Marketing and Economics · 2015
For your marketing
Useful when testing retargeting frequency and lifecycle messaging cadence. Randomize timing, not only total impressions, and track downstream purchases over a sufficient window.
Plain-English take
Ads increased purchase likelihood and effects carried over to later purchase occasions.
Learn more →
AdvertisingPlatforms
Evidence check: 3.5/5Marketing Science · 2025
For your marketing
A useful caution for campaign tests run in competitive auctions: your measured lift may depend on what other advertisers are doing simultaneously. Coordinate or repeat tests when market conditions shift.
Plain-English take
Ignoring concurrent competitive actions can misestimate advertising lift by a factor of two or more, potentially changing budget decisions.
Learn more →
AdvertisingAI
Evidence check: 3.5/5Marketing Science · 2023
For your marketing
A detailed example of measuring message components against both engagement and purchase outcomes. Test words, offer type, and audience interactions; do not optimize opens as a proxy for sales.
Plain-English take
Content and framing had heterogeneous effects along the funnel.
Learn more →
AdvertisingPlatforms
Evidence check: 4/5Marketing Science · 2010
For your marketing
Use as a bounded example of growing two-sided networks by advertising the user base: a field experiment; compare its context and design with your own market before applying its result.
Plain-English take
Advertising the size/composition of the user base changed seller participation; the effective message depended on which side of the market was being recruited.
Learn more →
Advertising
Evidence check: 4/5Science · 2012
For your marketing
Use as a bounded example of identifying influential and susceptible members of social networks; compare its context and design with your own market before applying its result.
Plain-English take
Influence and susceptibility varied systematically by age, gender, and marital status; network position and individual traits jointly predicted product adoption contagion.
Learn more →
Advertising
Evidence check: 4/5Journal of Marketing Research · 1986
For your marketing
Use as a bounded example of intervention analysis of a field experiment to assess the buildup effect of advertising; compare its context and design with your own market before applying its result.
Plain-English take
Advertising effects accumulated over exposure and could not be represented adequately by a simple contemporaneous response; intervention analysis detected buildup dynamics.
Learn more →
Advertising
Evidence check: 3/5Australasian Marketing Journal · 2018
For your marketing
Use as a bounded example of advertising effects? an elemental experiment; compare its context and design with your own market before applying its result.
Plain-English take
Specific advertising elements produced separable effects on attention, attitude, and response; aggregate creative labels obscured which elements drove performance.
Learn more →
Advertising
Evidence check: 4/5Management Science · 2025
For your marketing
Use this paper as evidence about long lags and large returns: experimental evidence from advertising to businesses in the stated context; compare its population, treatment and outcome with your own decision.
Plain-English take
Advertising to businesses generated unusually large returns relative to typical consumer-ad estimates; reported ROAS was 12.0 (95% CI 4.8–24.5), with long response lags.
Learn more →
AdvertisingFree trials
Evidence check: 2/5Management Science · 2021
For your marketing
Use this paper as evidence about experience effect in the impact of free trial promotions in the stated context; compare its population, treatment and outcome with your own decision.
Plain-English take
Initial usage predicted redemption and subsequent usage change: redemption rose from low- to high-percentile users, while usage change rose weakly monotonically through the 90th percentile and was generally insignificant above it.
Learn more →
Advertising
Evidence check: 4/5The Review of Economic Studies · 2020
For your marketing
Use this study as a bounded example related to does advertising serve as a signal? evidence from a field experiment in mobile search; evaluate whether its customers, intervention and outcome match your own context.
Plain-English take
The study provides experimental evidence on whether advertising conveys product/firm quality information, rather than merely persuading; exact effect sizes are not asserted here because the fetched DOI page did not expose the full abstract.
Learn more →
AdvertisingSales
Evidence check: 4/5Quantitative Marketing and Economics · 2014
For your marketing
Use this study as a bounded example related to online ads and offline sales: measuring the effect of retail advertising via a controlled experiment on yahoo!; evaluate whether its customers, intervention and outcome match your own context.
Plain-English take
The experiment finds a positive causal effect of online advertising on retail sales, including offline purchases, and reports that the campaign was profitable. It demonstrates that display advertising can generate measurable sales beyond the online channel.
Learn more →
Advertising
Evidence check: 4/5Marketing Science · 2011
For your marketing
Use this study as a bounded example related to online display advertising: targeting and obtrusiveness; evaluate whether its customers, intervention and outcome match your own context.
Plain-English take
Context matching and obtrusiveness each raised purchase intent separately, but their combination was ineffective and performed worse than using only one; the negative interaction was strongest among people refusing to report income and privacy-sensitive categories.
Learn more →
Advertising
Evidence check: 2/5Marketing Science · 2004
For your marketing
Use this study as a bounded example related to long-run effects of promotion depth on new versus established customers: three field studies; evaluate whether its customers, intervention and outcome match your own context.
Plain-English take
Promotion depth has different long-run effects for new and established customers, demonstrating that the immediate response to a discount is not sufficient to evaluate its longer-run customer-growth consequences.
Learn more →
Advertising
Evidence check: 4/5Journal of Marketing Research · 2013
For your marketing
Use this study as a bounded example related to when does retargeting work? information specificity in online advertising; evaluate whether its customers, intervention and outcome match your own context.
Plain-English take
Switching on retargeting caused 14.6% more users to return within four weeks; effects decayed with time since the initial visit, 33% of first-week effect occurred on day one, and week-two advertising worked better after week-one exposure.
Learn more →
Advertising
Evidence check: 4/5Journal of Marketing Research · 2013
For your marketing
Use this study as a bounded example related to when does retargeting work? information specificity in online advertising; evaluate whether its customers, intervention and outcome match your own context.
Plain-English take
Dynamic retargeting was, on average, less effective than generic brand advertising, but the disadvantage disappeared or reversed when browsing behavior indicated that consumers had developed specific preferences (for example, after visiting review sites). The implication is that retargeting content should match the consumer's decision stage and information specificity.
Learn more →
AdvertisingPlatforms
Evidence check: 4/5Management Science · 2022
For your marketing
Use this study as a bounded example of cold start to improve market thickness on online advertising platforms; assess whether its population, intervention and outcome match your own decision.
Plain-English take
The algorithm increased cold-start success rate by 61.62%, with a 0.717% short-term-revenue compromise. It increased overall advertising-market thickness by 3.13% and long-term advertising revenue by at least 5.35%.
Learn more →
AdvertisingPricing
Evidence check: 4/5Journal of Marketing Research · 2023
For your marketing
Use this study as a bounded example of from free to paid: monetizing a non-advertising-based app; assess whether its population, intervention and outcome match your own decision.
Plain-English take
Soft landing and exclusive secondary offerings each reduced existing users' willingness to subscribe, but their interaction was positive. A second field experiment produced consistent results, and the paper reports the firm's implementation of the best-performing strategy.
Learn more →
Advertising
Evidence check: 2.5/5Management Science · 2016
For your marketing
Use the study to frame a pricing or packaging hypothesis and identify outcomes to measure; estimate effects with your own customer mix and willingness-to-pay evidence.
Plain-English take
Promotion effectiveness varies substantially by business and deal execution; the study evaluates whether deals are profitable rather than assuming sales volume equals success.
Learn more →
Advertising
Evidence check: 3.5/5Marketing Science · 2015
For your marketing
Use to shape a locally controlled channel or message test; account for audience, timing and attribution limits before reallocating budget.
Plain-English take
Effectiveness depends jointly on what is personalized, when the ad appears, and where it appears; well-targeted personalization can increase responsiveness.
Learn more →
Advertising
Evidence check: 4/5Marketing Science · 2015
For your marketing
Use to shape a locally controlled channel or message test; account for audience, timing and attribution limits before reallocating budget.
Plain-English take
Television advertising changes online shopping activity, with effects varying by ad characteristics and shopping stage.
Learn more →
Advertising
Evidence check: 3.5/5Marketing Science · 2023
For your marketing
Use to shape a locally controlled channel or message test; account for audience, timing and attribution limits before reallocating budget.
Plain-English take
Digital advertising gains differ across restaurant types; the experiment identifies which businesses benefit more from advertising.
Learn more →
Advertising
Evidence check: 3.5/5Journal of Retailing · 2015
For your marketing
Use to shape a locally controlled channel or message test; account for audience, timing and attribution limits before reallocating budget.
Plain-English take
Greater personalization depth increased click-through only for a more trusted retailer and reduced it for a less trusted retailer; trust also shaped usefulness/reactance in the lab.
Learn more →
AdvertisingPlatforms
Evidence check: 3/5Information Systems Research · 2020
For your marketing
Use it to shape a local channel or message test; account for audience, timing and attribution limits before reallocating budget.
Plain-English take
Both tools increased traffic; only sponsored search significantly increased sales. Sponsored search worked better for low-reputation traffic and high-reputation sales; endorsement sales effects appeared only for high-reputation sellers.
Learn more →