KPI operational contract, guardrail thresholds, and review cadence

DEC-0066 - KPI operational contract, guardrail thresholds, and review cadence

Context

DEC-0054 established the minimum KPI pack, but operation-level contracts (denominator normalization, threshold alarms, and review rhythm) were still implicit, causing potential drift in interpretation across teams.

Decision

KPI operation rhythm:

• Daily D-1 KPI monitor for anomaly spotting.
• Weekly product KPI review for action decisions.
• Monthly KPI policy review for threshold/taxonomy adjustments.

KPI denominator and eligibility normalization:

• habit_retention: rolling 4-week signal, using valid completed-day events only.
• recommendation_to_attempt_rate: only count recommendation items that were rendered and visible.
• locked_feature_upgrade_conversion: measured from locked-entry exposure to checkout success in defined attribution window.

Default guardrail thresholds:

• habit_retention drop >5pp WoW -> trigger investigation.
• recommendation_to_attempt_rate drop >8pp WoW -> trigger recommendation-quality audit.
• locked_feature_upgrade_conversion drop >10pp WoW -> trigger paywall/offer audit.

Escalation protocol:

• 1 breach week: record diagnosis note.
• 2 consecutive breach weeks: open experiment ticket.
• 3 consecutive breach weeks: mandatory policy review and decision refresh candidate.

Segmentation baseline:

• KPI dashboards must include baseline cuts:
  • by program,
  • by tier,
  • by cold_start vs returning.

Decision Value

• Keeps KPI interpretation stable across product, design, and operation teams.
• Enables earlier issue detection with clear action thresholds.
• Improves long-term optimization discipline without adding metric sprawl.

Rationale

Small KPI sets only work when operation contracts are explicit. This decision turns KPI names into execution-ready governance.