Free trials · Randomized field experiment and policy evaluation
Design and Evaluation of Optimal Free Trials
Management Science · 2023
Summary
In this firm and experiment, a 7-day trial was the best uniform policy and increased subscriptions by 5.59% versus assigning everyone a 30-day trial.
For your marketing
Possible founder application
Useful when designing a local randomized trial-length experiment. Treat the paper as a design and evaluation example; estimate outcomes for your own activation path, sales cycle and user mix.
Choose trial lengths around the point where users can experience value, then compare them using randomized groups. Decide in advance how you will measure activation, paid conversion and longer-run customer value. Evaluate any personalized trial policy on users who were not used to build it; choosing the best-looking policy on the same data can overstate its value.
Reported result
What the study found
In this firm and experiment, a 7-day trial was the best uniform policy and increased subscriptions by 5.59% versus assigning everyone a 30-day trial. The best evaluated personalized policy, based on lasso, improved subscriptions by 6.81% versus that 30-day baseline; it also performed well on longer-run outcomes.
Research question
What the authors investigated
How do different free-trial lengths affect subscriptions, and can trial length be assigned more effectively using customer characteristics?
Design
A SaaS firm randomly assigned new users to 7-, 14- or 30-day trials. The authors compare uniform policies and evaluate personalized assignment rules using experimental data and inverse-propensity-score policy evaluation.
Sample and setting
337,724 new users across Australia, New Zealand, France, Germany, Japan, the United Kingdom and the United States; subscription and retention were followed for two years. The firm and product are anonymized.
Scope and transfer
Limitations to keep in view
One anonymized SaaS firm, product, trial design and historical period. Personalized-policy performance depends on the covariates, experiment and evaluation method. The result is not a general recommendation to shorten every SaaS trial.
Research sources
Full author preprint plus the peer-reviewed publisher record. Sample, random assignment, follow-up and headline results were checked in the paper text.