AI · Mixed-methods case study of an IBM AI code assistant under development: surveys of two user cohorts and unmoderated usability testing.
Examining the Use and Impact of an AI Code Assistant on Developer Productivity
Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems · 2025
Summary
The assistant generally produced net perceived productivity increases, but benefits were not experienced equally; the study also surfaced motivations, expectations about speed/quality, and questions about ownership and responsibility for generated code. For a SaaS founder: For developer-tool founders, measure perceived and behavioral productivity together, segment by user, and make provenance, ownership, responsibility, and output-quality workflows explicit. Avoid assuming that aggregate positive sentiment means every team or task benefits.
For your marketing
Possible founder application
For developer-tool founders, measure perceived and behavioral productivity together, segment by user, and make provenance, ownership, responsibility, and output-quality workflows explicit. Avoid assuming that aggregate positive sentiment means every team or task benefits.
Reported result
What the study found
The assistant generally produced net perceived productivity increases, but benefits were not experienced equally; the study also surfaced motivations, expectations about speed/quality, and questions about ownership and responsibility for generated code.
Research question
What the authors investigated
How does use of an AI code assistant affect developers' perceived productivity, usage choices, and concerns about generated-code responsibility?
Design
Mixed-methods case study of an IBM AI code assistant under development: surveys of two user cohorts and unmoderated usability testing.
Sample and setting
IBM developer users of a general-purpose assistant supporting common programming languages; publisher abstract reports survey cohorts totaling N=669 and unmoderated usability testing N=15.
Scope and transfer
Limitations to keep in view
This is an organizational case study using self-reported perceptions plus usability testing, not a randomized productivity trial; it concerns one assistant and IBM context, and the abstract cautions that gains vary across users.
Research sources
Direct publisher scrape verified title/record and available text; Firecrawl source ID 01a1228d-e249-7029-8a2f-e072d63d2379. Correction notices were not separately checked.