Customer behavior and adoption · Preregistered online randomized experiment with incentivized, occupation-specific writing tasks.
Experimental evidence on the productivity effects of generative artificial intelligence
Noy, S. & Zhang, W. (2023). Experimental evidence on the productivity effects of generative artificial intelligence. Science.
Plain-English take
In this study, ChatGPT reduced average time by 40% and raised output quality by 18%; inequality fell, and exposed workers were 2 times as likely to report real-job use two weeks later and 1.6 times as likely two months later. Founder use: Evaluate AI features with randomized task-level tests that include quality and time, then measure sustained adoption rather than assuming a novelty effect equals retention.
How much weight to give this
4.5/5 Strong experimental or quasi-experimental evidence for its stated setting
The 1–5 evidence-check score is a practical reading aid, not a formal peer review or a probability that a conclusion is true. It combines design strength (2 points), sample reporting (1), comparison/control clarity (1), and how much of the paper could be checked (1). Access affects how confidently this directory can assess a study, not the study's underlying quality. Blinding is reported separately because it is not relevant to every design.
Design description was checked against the directly fetched article source; conclusions remain specific to the studied setting.
Sample size or the source's lack of a reported count is stated explicitly.
The comparison or identification strategy is described from the article record.
Only the publisher page or abstract was confirmed.
Double-blind? Blinding is not stated in the source reviewed; that does not establish whether blinding was or was not used.
Research question
What the authors investigated
What is the causal effect of ChatGPT access on productivity, quality, inequality, and subsequent workplace adoption in professional writing tasks?
Design
Preregistered online randomized experiment with incentivized, occupation-specific writing tasks.
Sample and setting
453 college-educated professionals performing midlevel professional writing tasks; half were randomly exposed to ChatGPT.
Reported result
What the study found
ChatGPT reduced average time by 40% and raised output quality by 18%; inequality fell, and exposed workers were 2 times as likely to report real-job use two weeks later and 1.6 times as likely two months later.
This is a paraphrase of the source. Check the original paper for definitions, estimates, statistical uncertainty and analysis details.
Scope and transfer
Limitations to keep in view
Short online tasks, selected professionals, and one model/version; measured writing performance is not the same as long-run business productivity, customer value, or safe deployment.
Possible founder application
Evaluate AI features with randomized task-level tests that include quality and time, then measure sustained adoption rather than assuming a novelty effect equals retention.
Evidence access
Publisher abstract or article landing page captured; full text access not established
Publisher page directly retrieved with Firecrawl; scrape reference 01a1227f-03d4-711b-90ec-fe5cebeebe65. Claims are limited to the reported article record. Correction notices were not separately checked.
If only an abstract or metadata was available, the profile is limited to claims supported there. A paper's paywall does not change its evidentiary status, but it limits what we can substantiate from accessible text.