Measurement and experiments · Skill file inspected; execution untested
Corey Haines: ab-testing
A source-inspected Agent Skills file for an experiment plan with comparison and analysis assumptions.
When this is useful
You can make a controlled change and need to design the measurement
Prepare the inputs
Hypothesis, randomization unit, baseline, primary outcome and practical constraints
This is an instruction file rather than a standalone application. The host needs to support the file format and any tools the task requires.
Expected artifact
An experiment plan with comparison and analysis assumptions
Our original evaluation criterion: Check sample and exposure assumptions; do not let the agent declare a winner from a small noisy result.
Install and evaluate
The source file is `skills/ab-testing/SKILL.md` in `coreyhaines31/marketingskills`. Follow that repository's current installation instructions. The README documents installing a selected skill with `npx skills add coreyhaines31/marketingskills --skill ab-testing`. Product-marketing context is shared across several skills.
Put it to work
- Read the linked skill source and its referenced files.
- Prepare the stated inputs and use a fictional or sanitized case first.
- Check sample and exposure assumptions; do not let the agent declare a winner from a small noisy result; record what needs verification before using the output.
Limits and tradeoffs
The source was read but this skill was not installed or run. Model quality, tool access and permissions vary by host. Review third-party instructions and provide real product evidence before execution.