Change failure rate

How is change failure rate measured?

Change failure rate is a ratio, not a count: failed production deployments ÷ total production deployments over a rolling window (usually 30 or 90 days). The math is trivial; the discipline is in defining "failed" and "deployment" the same way every time, so this quarter's number is comparable to last quarter's.

Three definitions have to be pinned down before the number means anything:

1. What is a deployment? A new artifact version reaching production users: a container image promoted, a Helm release applied, a versioned artifact published behind a live route, or a feature flag flipped on. Pre-production promotions and preview environments do not count.
2. What is a failure? A deployment that required unplanned intervention because of that change: a rollback, an urgent hotfix, a re-deploy of the prior artifact, or a user-visible incident linked to that release. If the fix belonged to a later deployment, count it against that later one.
3. What is the window? Rolling 30-day and 90-day are the most common. Anything shorter is noisy for teams shipping less than daily; anything longer masks recent regressions.

The DORA State of DevOps reports group teams into performance bands:

Band	Change failure rate
Elite	0% – 5%
High	5% – 10%
Medium	10% – 15%
Low	30% – 60% (typical range)

Treat the bands as a direction signal, not a scoreboard. A payments platform holding at 8% with a mean time to restore under an hour is healthier than a marketing site sitting at 3% with three-hour recoveries. The number only becomes meaningful in the context of the other three DORA metrics. Read it alongside deployment frequency and MTTR, never on its own.

Why does change failure rate matter?

Change failure rate is the quality half of the DORA speed/stability pair. Deployment frequency tells you how often value moves; change failure rate tells you the cost of that motion. A team that doubles frequency while CFR stays flat is genuinely getting better. A team that doubles frequency while CFR climbs is exporting risk to users, and speed alone is hiding the regression.

Concretely, the metric points at three practical bottlenecks:

• A rising CFR at stable frequency usually means the test suite has drifted from what production actually does. Coverage looks fine but real-user paths are unguarded. Time to invest in smoke tests, contract tests, and better staging fidelity.
• A rising CFR at rising frequency is the "moving fast and breaking things" pattern. The fix is not to slow down deploys but to shrink the unit of each deploy: smaller merges, feature flags to decouple deploy from release, and gated rollouts like canary or blue-green so a bad change only ever hits a slice of traffic.
• A stubbornly flat CFR near zero often means the metric is being under-reported, not that the team is genuinely elite. Compare against your incident tracker and rollback log. If the tracker knows about failures the metric doesn't, the plumbing is lying to you.

The honest caveat is the one every DORA metric shares: the number is only as good as the pipeline events feeding it. If deployment events are logged by hand, or if "failure" is decided in a Slack channel, the chart is fiction. Automate the ingestion (deploy events straight from the pipeline, incident links straight from the tracker) before anyone reads a trend line from it.

How do popular CI/CD tools track change failure rate?

No CI/CD tool "computes" change failure rate on its own. The metric needs a deploy event and a failure signal (rollback, incident, hotfix) joined on the deployment. Each platform gives you different raw material for that join.

• GitHub Actions + GitHub: the deployments API records Deployment and DeploymentStatus objects with success/failure/inactive transitions, and community DORA actions read them directly. If you're already GitHub-native and your incidents live in GitHub Issues, the join is essentially free. Well-suited to teams that want to keep the whole delivery record inside one product.
• GitLab CI: Value Stream Analytics ships a built-in change failure rate chart on Premium/Ultimate, computed from environment transitions and linked incidents. If you're already paying for GitLab Premium, that is the cheapest credible CFR dashboard on the market. The data model is right there and the chart needs zero glue code. Hard to beat on price-per-chart if GitLab already owns your source and pipelines.
• Jenkins: no built-in DORA view, but the long-running build history is a solid event source. In shops where Jenkins already owns the audit trail, a small Groovy or shared-library step posting {status, artifact, commit, incident_id} to a metrics endpoint is the path of least resistance, and it keeps CFR reporting in the same tool auditors already trust.
• Argo CD / Argo Rollouts: the analysis templates in Argo Rollouts trigger automated rollbacks on failed metrics, and every one of those rollback events is a first-class Kubernetes resource. If you're all-in on Kubernetes with progressive delivery, Argo is the better fit here. The failure signal is already automated, structured and attached to the exact revision that caused it, no separate incident-tracker join needed.
• Specialised DORA platforms: Sleuth, LinearB and Jellyfish pull deploy events from CI/CD, incidents from PagerDuty/Opsgenie, and PRs from GitHub/GitLab, then present org-wide CFR alongside the other three keys. If you need a board-ready view across dozens of teams, a dedicated product still wins on polish and drilldowns.
• Buddy is one solid option when the friction is emitting clean deploy and failure events in the first place. ON_EVERY_PUSH pipeline triggers, an explicit rollback action, trigger_conditions on previous-action success/failure, and a built-in HTTP action mean a small team can wire "every production deploy, plus its outcome, plus any subsequent rollback" into whatever dashboard they already use in an afternoon. Buddy doesn't visualise CFR itself. The value is that every deploy leaves a structured, timestamped event with a success/failure verdict, so the ratio becomes a SELECT in whatever store you're already reading from.

There is no single winning tool here. Pick whichever already owns your deploy events and your incident data, and invest the time you save in shrinking the unit of change. A smaller merge with a gated rollout is worth more to CFR than any dashboard.

Example

A minimal Buddy pipeline that ships to production, records the deploy outcome, and (if a rollback runs later) emits a matching failure event against the same commit. The two events together are everything a CFR calculation needs: one denominator entry per deploy, one numerator entry per failure, joined on ${BUDDY_EXECUTION_REVISION}.

# .buddy/buddy.yml — every prod deploy leaves a structured outcome event;
# a triggered rollback marks that same commit as failed.
- pipeline: "deploy-production"
  events:
  - type: "PUSH"
    refs:
    - "refs/heads/main"
  variables:
  - key: "SERVICE"
    value: "checkout-api"
  - key: "METRICS_ENDPOINT"
    value: "https://metrics.example.com/deployments"
  actions:
  - action: "Build & deploy"
    type: "BUILD"
    docker_image_name: "node"
    docker_image_tag: "20"
    commands: |-
      npm ci
      npm run build
      ./scripts/deploy.sh

  - action: "Record deploy success"
    type: "HTTP"
    method: "POST"
    notification_url: "${METRICS_ENDPOINT}"
    headers:
    - name: "Content-Type"
      value: "application/json"
    content: |
      {
        "service":     "${SERVICE}",
        "commit":      "${BUDDY_EXECUTION_REVISION}",
        "environment": "production",
        "deployed_at": "${BUDDY_EXECUTION_FINISH_DATE}",
        "outcome":     "success"
      }

  - action: "Record deploy failure"
    type: "HTTP"
    method: "POST"
    trigger_time: "ON_FAILURE"
    run_only_on_first_failure: true
    notification_url: "${METRICS_ENDPOINT}"
    headers:
    - name: "Content-Type"
      value: "application/json"
    content: |
      {
        "service":     "${SERVICE}",
        "commit":      "${BUDDY_EXECUTION_REVISION}",
        "environment": "production",
        "deployed_at": "${BUDDY_EXECUTION_FINISH_DATE}",
        "outcome":     "failed"
      }

- pipeline: "rollback-production"
  variables:
  - key: "SERVICE"
    value: "checkout-api"
  - key: "METRICS_ENDPOINT"
    value: "https://metrics.example.com/deployments"
  actions:
  - action: "Roll back to previous artifact"
    type: "BUILD"
    docker_image_name: "node"
    docker_image_tag: "20"
    commands: |-
      ./scripts/rollback.sh

  - action: "Mark original deploy as failed"
    type: "HTTP"
    method: "POST"
    notification_url: "${METRICS_ENDPOINT}/failures"
    headers:
    - name: "Content-Type"
      value: "application/json"
    content: |
      {
        "service":       "${SERVICE}",
        "failed_commit": "${BUDDY_EXECUTION_REVISION}",
        "failure_type":  "rollback",
        "detected_at":   "${BUDDY_EXECUTION_FINISH_DATE}"
      }

Change failure rate then becomes one query against the events table: roughly count(failures) / count(deploys) WHERE environment = 'production' AND deployed_at > now() - 30 days. The metric is a by-product of how the pipeline already runs, not a separate reporting project competing for headcount. When the ratio starts drifting up, the conversation moves to the right place (merge size, test fidelity, and rollout gating) instead of to the dashboard.

content/terms/change-failure-rate.md