# Hotfixes & Break Glass

Emergency fixes skip review and later releases quietly drop them; Bumpkit gates hotfixes, catches missing back-ports and records every override, for on-call engineers and release managers.

A hotfix has to ship fast without becoming a back door. Bumpkit gates hotfixes with tighter rules, makes sure normal releases don't lose fixes that already shipped, and keeps a reviewable record of every emergency override.

## Cut a hotfix

`bumpkit hotfix version` prints the next free patch version for a hotfix:

```bash
bumpkit hotfix version
bumpkit hotfix version --from v2.3.0
```

It starts from the latest version tag reachable from `HEAD`, or from `--from`, and skips versions that already exist as tags locally or on `origin`. With `--json` it also returns the base and the skipped versions.

A typical flow:

```bash
git switch -c hotfix/checkout-timeout v2.3.0
git commit -am "fix: retry the payment provider on timeout"
NEXT=$(bumpkit hotfix version)
git push -u origin hotfix/checkout-timeout
```

Open a pull request from the branch, let the gate run, then tag `$NEXT` once it merges. `version` is currently the only `bumpkit hotfix` subcommand; see the [CLI Reference](/docs/cli).

## How a hotfix is detected

`bumpkit gate` treats a change as a hotfix when you pass `--hotfix`, or when the branch name starts with `hotfix/`, `hotfix-`, `hotfixes/`, `hotfixes-`, `release/`, `release-`, `releases/` or `releases-`. In CI the branch is the pull request's source branch (GitHub Actions, GitLab CI, Bitbucket Pipelines and Azure Pipelines); elsewhere it is the current branch.

```bash
bumpkit gate --hotfix --comment
```

A hotfix gate runs the faster check: deterministic checkers only, without the AI summary.

## Hotfix rules

Hotfixes follow the normal policy plus the [`hotfix`](/docs/bumpkit-yml#hotfix) section:

```yaml
gate:
  hotfix:
    block_on: [breaking, data_loss, migration, secret]
    max_files: 20
    review_within: 72h
    backports: approve
```

| Key | Default | Meaning |
|---|---|---|
| `block_on` | `[breaking, contract_breaking, data_loss, migration]` | Finding kinds a hotfix may not carry, at any severity, when they are deterministic and not suppressed. Allowed: `breaking`, `contract_breaking`, `data_loss`, `migration`, `dependency`, `vulnerable_dependency`, `config`, `secret`, `risk_area`, `untested` |
| `max_files` | no limit | A hotfix changing more files needs approval |
| `review_within` | `72h` | How soon an override needs its post-hoc review, as a duration up to 90 days, like `24h` or `72h` |
| `backports` | `approve` | What a normal release missing a shipped fix does: `block`, `approve` or `off` |

On top of `block_on` and `max_files`, a hotfix needs approval when:

- any commit is a feature (`feat:` or `feat(scope):`), because a hotfix should only fix
- its changes make it a minor or major release instead of a patch

## Back-port detection

When you gate a normal release (not a hotfix), Bumpkit looks for fixes that shipped in a recent release but never made it onto this branch. Releasing without them would bring the bug back for anyone who upgrades.

It looks at recent release tags (`v1.2.3` or `1.2.3`) that aren't merged into the branch and finds fixes whose change isn't on the branch, cherry-picks included. Each one is a high-severity `missing_backport` finding:

```text
v2.3.1's fix "fix: retry the payment provider on timeout" isn't on this branch
```

What the gate does depends on `hotfix.backports`:

| Value | Outcome |
|---|---|
| `approve` (default) | Needs approval; any reviewer can approve shipping without the fixes |
| `block` | Fails |
| `off` | Ignored |

In CI, `bumpkit gate` fetches tags from `origin` first. Check out with full history (`fetch-depth: 0` on GitHub Actions) so the comparison is complete.

### Open the back-port pull request

> **Beta, early access.** A workspace without access gets `403 FEATURE_EARLY_ACCESS` and a link to request it. Flag: `close_the_loop`. It also needs the Bumpkit GitHub App on the repository.

Bumpkit can do the cherry-pick for you and open it as a pull request:

```bash
bumpkit backport open 3f2a9c1 --onto main
bumpkit backport open 3f2a9c1 --onto release/2.2 --repository acme/api
```

Bumpkit fetches the repository through the GitHub App, cherry-picks the hotfix onto the tip of `--onto` (default: the default branch) and opens the pull request on `bumpkit/backport/<sha7>-to-<branch>`. The commit message keeps `(cherry picked from commit <sha>)`, so the missing back-port clears once it merges. Members and above can open one, there is one pull request per hotfix and branch, and Bumpkit never merges it. If the hotfix is already on the branch, nothing opens.

If the cherry-pick conflicts, Bumpkit opens nothing and answers `409`, naming the conflicting files and who shipped the hotfix:

```text
Error: Bumpkit API error (HTTP 409): Cherry-picking 5322103 (fix: charge 8% sales tax) onto main conflicts in services/pricing/tax.go, so Bumpkit opened nothing. Sam Rivera <sam@acme.test> shipped it and needs to back-port it by hand; the release stays gated until it lands.
```

The release page on the dashboard has an **Open back-port PR** button for every missing back-port the gate reported. `bumpkit loop --kind backport` lists each back-port with its status, pull request and proof. A back-port is marked verified when the release gate passes on its pull request.

## Find which releases contain a fix

`bumpkit contains` finds a commit by ref or by text in its message, and lists every release tag and remote branch that contains it, directly or as a cherry-pick:

```bash
bumpkit contains 3f2a9c1
bumpkit contains "null pointer in checkout"
```

```text
3f2a9c1 fix: null pointer in checkout
REF                 HOW
v2.3.1              contains
origin/release/2.2  cherry-picked
```

A text search matches up to 20 commits across all branches.

## Link the hotfix to an incident

If Bumpkit traced a production error to a release, record the hotfix that fixed it:

```bash
bumpkit incidents list
bumpkit incidents resolve 3f2c9a1e-8b7d-4c2a-9e1f-6a5b4c3d2e1f --version v2.3.1 --sha 4b7a1d2
```

See [Release Health & Forensics](/docs/release-health).

## Break glass

When a fix has to ship before the gate is satisfied, a workspace owner, admin or policy admin overrides the gate for that one commit, signed in to the dashboard: open the commit's check, enter the reason under **Override this commit** and choose **Override gate**.

- Only owners, admins and policy admins can override (see [Roles](/docs/sso#roles)), with a reason of 10 to 500 characters. It works with or without the Approvals page. An API key is refused with `403 HUMAN_APPROVAL_REQUIRED`, including from `bumpkit override` and the `override` MCP tool.
- The override applies to that exact commit; a new push needs a fresh decision. Each commit can be overridden once.
- The pull request's GitHub check re-runs right away and turns green, whether you override on the check page or the dashboard **Approvals** page.
- The gate decision keeps a `break_glass` reason: "Overridden by *email*: *reason*".

To forbid overrides in a repository:

```yaml
gate:
  break_glass: false
```

With `break_glass: false`, an override is still recorded, but the gate's outcome doesn't change. See [`break_glass`](/docs/bumpkit-yml#break-glass) and the [Release Gate](/docs/release-gate#overrides).

## Post-hoc review

Every override can be reviewed after the fact by a second person. The gate adds a reason with the review deadline (`hotfix.review_within`, default `72h`), and the review becomes overdue once it passes.

```bash
bumpkit override list
bumpkit override list --overdue
bumpkit override review 3f2c9a1e-0d4b-4c55-9e0a-7b1d2c3e4f50 --note "Diff matches the incident fix; tests added in #490"
```

- Owners, admins, policy admins and approvers can review, but not the person who overrode.
- The note is 10 to 1000 characters and is kept in the audit record.
- `override list` shows each override's status: `reviewed`, `overdue`, `awaiting review` or `no review required`.

The API equivalents are `GET /v1/gate/overrides` (with `?overdue=true`) and `POST /v1/gate/overrides/{id}/review`.

## Evidence of emergency changes

> **Beta, early access.** A workspace without access gets `403 FEATURE_EARLY_ACCESS` and a link to request it. Flag: `compliance_export`.

Every override is written to the workspace audit log as `gate.override`. Gate decisions are kept with the release and in the compliance export, which lists every override in a period under `emergency_changes`:

```bash
bumpkit evidence export --days 90 -o evidence.json
```

| Field | Meaning |
|---|---|
| `override_id` | The override |
| `repository`, `sha`, `pull_request` | What was let through |
| `reason` | Why, as given |
| `overridden_by`, `overridden_at` | Who and when |
| `review_due_at` | The deadline from `review_within` |
| `reviewed_at`, `reviewed_by`, `review_note` | The post-hoc review |
| `status` | Review status |
| `hash` | Tamper-evidence link |

The emergency change record is tamper-evident, so an auditor can verify none were removed. See [Compliance Evidence](/docs/compliance).