How to Measure Behavior Change

Definition. Measuring behavior change means measuring a specific target behavior against a baseline with explicit denominators and time windows. In Behavioral Strategy, you treat behavior as the KPI (not clicks or attitudes).

From Behavioral Strategy, developed by Jason Hreha.

Axiom. If two people can’t observe it and agree whether it happened, it’s not measurable.

How this differs from common analytics

  • Not engagement-first: you measure the target behavior, not clicks, opens, or time in app.
  • Not attitude-first: self-report and NPS are supportive signals; they do not define behavior change.
  • Not windowless: every metric is reported with an explicit denominator and time window.

The measurement setup (5 steps)

  1. Define the target behavior operationally.
    • Template: population does action in context with frequency/window.
    • If two people can’t observe it and agree whether it happened, it’s not measurable.
  2. Choose the denominator (who counts).
    • Examples:
      • “All exposed users”
      • “All eligible users”
      • “All new accounts in cohort week X”
    • Avoid denominator drift (changing who counts midstream).
  3. Choose the time window (when it must happen).
    • Examples:
      • “Within first session”
      • “Within 24 hours”
      • “Within 7 days of signup”
    • You can’t interpret behavior without a window.
  4. Instrument behavior events (or observation rules).
    • Minimum event set (example naming aligned to a typical product flow):
      • exposure_shown
      • behavior_attempt_started
      • behavior_completed
      • behavior_abandoned
    • See Measurement Standards for reporting rules and event fields.
  5. Pre-register what “success” means.
    • Define thresholds and what decisions they trigger (proceed, redesign, reselect).
    • If you don’t pre-register, you’ll “discover” wins in noise.

Core metrics (what to report)

  • Δ‑B: change in target behavior completion rate vs baseline (percentage points), with denominator and window. See Δ‑B.
  • TTFB: time from first exposure/start to first valid completion of the behavior. Shorter TTFB is often a leading indicator of higher retention in products; validate per cohort. See TTFB.
  • Behavior retention: D30/D180 retention of the behavior (not app opens), with cohort definition.
  • bPMF: sustained behavior at scale in market conditions (behavioral product-market fit). See bPMF.

Common traps (what breaks measurement)

  • Proxy metrics: “engagement” or clicks that don’t represent the target behavior.
  • Undefined windows: measuring “eventually” guarantees false positives.
  • Survivorship bias: looking only at retained users instead of the eligible/exposed denominator.
  • Mixed behaviors: bundling multiple behaviors into one KPI hides failure points in the chain.
  • Changing instrumentation mid-test: makes baselines incomparable (log changes and reset baselines if needed).

Outputs (what you should have when you’re done)

  • A target behavior definition (operational, observable)
  • A denominator and time window (documented and stable)
  • An instrumentation spec (events/fields + success rules)
  • A baseline vs post report that includes Δ‑B, TTFB, and retention (with denominators/windows)

Templates (copy/paste)

Example measurement spec 

target_behavior:
  population: "New teams (B2B)"
  action: "Send 3 messages in #general"
  context: "Within 24 hours of workspace creation"
denominator: "All new workspaces created"
window: "First 24 hours"
success_criteria:
  completion_rate_pp: ">= 35"
  ttfb_median: "<= 15 minutes"
instrumentation:
  exposure_shown: { surface: "onboarding", variant: "v2" }
  behavior_completed: { behavior_id: "send_3_msgs_24h", channel: "#general" }
reporting:
  delta_b: { baseline_pp: 22, post_pp: 37, delta_pp: 15 }
  retention_d30_pp: { baseline_pp: 8, post_pp: 14 }

Frequently asked questions

What is Delta-B (Δ‑B)?

Δ‑B is the change in target behavior completion rate versus a baseline, reported in percentage points with a denominator and time window.

What is Time to First Behavior (TTFB)?

TTFB is the time from first exposure or start to the first valid completion of the target behavior.

What denominator should you use?

Use a stable denominator tied to who could realistically perform the behavior such as exposed users, eligible users, or a defined cohort.

What is behavior retention?

Behavior retention tracks repeat performance of the behavior over time for a defined cohort, such as D30 or D180 retention of the behavior.

What is bPMF?

bPMF is sustained behavior at scale in market conditions with viable economics rather than one-off activation wins.

See also