Developer experience · Asynchronous controlled experiment with random assignment to a tangled single pull request or logically decomposed pull requests, supplemented by questionnaires and qualitative screencast analysis.
The effects of change decomposition on code review—a controlled experiment
PeerJ Computer Science · 2019
Summary
Decomposition significantly reduced false positives and produced more structured context-seeking review behavior; it did not significantly change the number of defects found, suggested improvements, total review time, or measured rationale understanding. The authors recommend self-contained, untangled changes.
For your marketing
Possible founder application
Make small, self-contained changes and issue-aligned pull requests a first-class workflow; provide automated decomposition/reviewability guidance and track false positives, context switching, and review precision rather than only review speed.
Reported result
What the study found
Decomposition significantly reduced false positives and produced more structured context-seeking review behavior; it did not significantly change the number of defects found, suggested improvements, total review time, or measured rationale understanding. The authors recommend self-contained, untangled changes.
Research question
What the authors investigated
Does decomposing a tangled change into separate pull requests affect review effectiveness, false positives, suggested improvements, review time, defect lifetime, and understanding of change rationale?
Design
Asynchronous controlled experiment with random assignment to a tangled single pull request or logically decomposed pull requests, supplemented by questionnaires and qualitative screencast analysis.
Sample and setting
28 volunteer participants: experienced/professional developers from three companies, PhD students from three universities, and MSc students from Delft University of Technology; they reviewed refactoring and feature changes in the approximately 3,000-line open-source Java JPacman system through GitHub-style pull requests.
Scope and transfer
Limitations to keep in view
Small convenience sample, heterogeneous participants, one small teaching-oriented Java system, two pull requests, asynchronous execution, and non-core-developer reviewers limit generalizability; the authors explicitly note threats from sample size, uncontrolled surroundings, change size, and task complexity.
Research sources
Publisher DOI and article metadata: https://doi.org/10.7717/peerj-cs.193; full abstract, methods, sample, results and limitations: https://pmc.ncbi.nlm.nih.gov/articles/PMC7924728/ Firecrawl research job 01a14371-854f-71aa-ba3d-455ec397717d returned this citation and metadata.