Measurement plan

# SaaS measurement plan and event dictionary

Create one plan for one decision. Examples are fictional. Store sensitive identifiers and customer records in the appropriate restricted systems, not in a public worksheet.

## 1. Decision brief

- Decision owner:
- Decision to make:
- Options being considered:
- Questions the evidence should answer:
- Useful customer result:
- Observation window and review date:
- Evidence that would support each option:
- Qualitative evidence needed alongside counts:

## 2. Event dictionary

Copy one record for each event.

| Field | Definition |
|---|---|
| Event name and version | Fill in |
| Exact successful operation that triggers it | Fill in |
| Entity measured | Account, user, integration, action, or another defined entity |
| Authoritative source | Fill in |
| Required properties and allowed values | Fill in |
| Occurrence timestamp and timezone | Fill in |
| Unique event or operation identifier | Fill in |
| Retry and duplicate handling | Fill in |
| Repeated valid action handling | Fill in |
| Situations when it must not fire | Fill in |
| Implementation and definition owners | Fill in |
| Known gaps | Fill in |

Avoid recording free-text customer material or sensitive payloads when a categorical property answers the question.

## 3. Fictional completed event record

Event: `test_failure_investigated`, version 1. Trigger: an investigation outcome is successfully saved for a failure. Unit: investigation, associated with its account. Source: the application's saved investigation record. Required properties: account identifier, investigation identifier, outcome category, occurrence timestamp, and definition version. A retry of the same save operation must not create a second completed investigation. Opening the failure screen and a failed save do not trigger the event. Editing an already saved outcome does not count as another first completion. This event indicates a recorded investigation, not proof that its conclusion helped the customer.

## 4. Identity and eligibility

- Account and user relationship:
- Handling of users in several accounts:
- Anonymous-to-authenticated connection, if needed:
- Internal/test exclusions:
- Other eligibility rules:
- Merged and deleted entity handling:
- Access owner and retention decision:
- Identity gaps that affect interpretation:

## 5. Metric contract

| Field | Definition |
|---|---|
| Metric name | Fill in |
| Numerator | Exact qualifying behavior and counting unit |
| Denominator | Exact eligible population and counting unit |
| Cohort entry rule | Fill in |
| Outcome observation window | Fill in |
| Timezone | Fill in |
| Fully observed cohort cutoff | Fill in |
| Source and query/report owner | Fill in |
| Missing-data treatment | Fill in |
| Definition changes affecting comparison | Fill in |

Illustrative contract: eligible accounts created during a stated week that save an investigation outcome within seven days of creation, divided by eligible accounts created during that week. Only report the final seven-day rate after every included account has had that window. Seven days is a planning example, not a universal activation standard.

## 6. Quality checklist

- Successful operation creates the expected event.
- Failed operation does not create a completion event.
- Retry and duplicate delivery follow the documented rule.
- Repeated valid actions follow the documented rule.
- Events belong to the correct account or other entity.
- Internal test activity is identifiable and excluded as specified.
- A sample reconciles with authoritative application records.
- Known capture gaps and definition version are documented.

For each check, record tester, date, expected outcome, actual outcome, and unresolved issue.

## 7. Decision report

- Decision and owner:
- Period, cohort, window, and definition version:
- Numerator and denominator counts:
- Rate, if appropriate:
- Known gaps and uncertainty:
- Customer or support evidence:
- Proposed action and reasoning:
- Next review condition:

Illustrative reporting sentence: “12 of 30 eligible accounts reached the defined result within the completed observation window.” These numbers are fictional. Add limitations before using any real report to choose an action.

## Aggregation and operation checks

- Raw event counting unit:
- Stable operation/deduplication key:
- Metric counting unit:
- First qualifying event rule:
- Later valid events and effect on the metric:

Fictional first-result check: a successful first investigation qualifies its account once. A retry of the same save must not add a completion. Another investigation can add a valid event without adding another account to the first-result numerator. A failed save adds neither. Verify these cases independently.


---
Original resource by Jon Bishop. CC BY 4.0. Illustrative examples are not customer results.