CI and git notes

tak stores measurements under refs/notes/tak as one JSON object per line. The notes merge with git's cat_sort_uniq strategy, allowing concurrent writers to deduplicate byte-identical records without a custom service.

Record a run

Build the subject first, then record the declared benchmarks:

tak run --record

Nothing leaves the machine until you push the notes:

tak push

Fetch measurements

tak history, tak log, and tak compare fetch remote measurements into the scratch ref refs/notes/tak-remote, then merge them into the local notes without discarding records that have not been pushed. Normally no manual fetch is needed.

For a read-only checkout that will never record measurements locally, teach its origin remote to fetch the notes ref with plain git fetch:

tak init
git fetch

The notes tree uses commit SHAs as path names rather than object references. A shallow fetch of the notes ref can therefore retrieve the full measurement history without fetching the annotated project commits.

Do not use that direct-fetch configuration in a checkout where you run tak run --record: plain git fetch has no way to merge unpushed local notes. Let tak history, tak log, tak compare, and tak push use the scratch ref instead.

Separate measurement from publication

A persistent or self-hosted runner should not receive a repository-write token while it builds or runs project code. Export the local measurement into an artifact instead:

tak artifact export --output tak-measurement.json

Upload that file, then download it in a separate trusted job. The publisher must supply the revision independently from its workflow context; the artifact is rejected if it names a different commit:

tak artifact publish tak-measurement.json --expect "$GITHUB_SHA"

Publication uses the same non-forced push and cat_sort_uniq retry path as tak push, so an old measurement and concurrent writers are preserved. The artifact format is pre-v1 and may change between Tak releases.

Compare revisions

Compare the current revision with the recorded baseline:

tak compare origin/main

An instruction-count increase beyond the configured gate fails the command. Wall-clock changes are displayed but never gate the result. A deliberate increase can be accepted for the benchmarks it affects.

So does finding nothing to compare: when no series was measured on both revisions, the report says **Nothing was compared and the command exits non-zero after printing it. That usually means the base was never recorded, its notes were not fetched, or the two were measured on different runner classes. Pass --allow-empty when that is expected: on the first pull request after adopting tak, or while a runner-class migration has left the base on the old class. Where it is routine, turn on the allow_empty setting instead. --no-gate also passes an empty comparison, since it never fails on the comparison. Errors, such as an invalid tak.toml, still fail.

Older tak releases print the same report and exit 0. A workflow that may run one should also fail when the report's first line starts with **Nothing was compared, as the pull-request example does. Check only the first line. tak writes that verdict before any benchmark name, and the rest of the report echoes text from the change under test.

When comparing nothing is routine

Some projects compare nothing often, for reasons that are not a broken setup:

• The runner class churns. A class that encodes the CI image or compiler version, as runner_class recommends when those change the numbers, starts a new series on every update. Until the main branch is measured on the new class, every pull request has nothing to compare against. tak's own class includes the GitHub image version, so each image update did this to every open pull request.
• The project is still adopting. Pull requests that branch from commits measured before tak was, and new benchmarks, have no base.

For those, turn on allow_empty so that comparing nothing warns instead of failing:

[gate]
allow_empty = true

With it on, tak compare, tak detect and tak run --baseline --gate exit 0 when nothing was compared. The report's first line is unchanged, so a script that reads it still sees **Nothing was compared. A line after it says the comparison passed with a warning because allow_empty is on, and tak prints one warning line on stderr:

warning: nothing was compared: no series was measured on both origin/main and HEAD; passing because `allow_empty` is on

Under GitHub Actions (GITHUB_ACTIONS set), that line is a ::warning:: workflow command, so it also appears as an annotation on the run. It goes to stderr, never to the report on stdout.

The cost: tak cannot tell a new runner class from a broken setup. A base that was never recorded, notes that were never fetched, or a recording step that stopped producing instruction counts also compare nothing, and with allow_empty on they warn instead of failing. Pair the setting with a check that the data is there, as tak's own main-branch workflow does before tak detect, and treat the warning as something to read.

Only the empty case is waived. A regression still fails, and so does anything else the gate fails on: an error, and, for tak run --baseline --gate, a failed check, a failed instruction count, or a comparison where only some benchmarks could be checked.

--allow-empty is the same setting, for one invocation, and TAK_ALLOW_EMPTY=0 turns it off for one. --no-gate takes precedence over both, and prints no warning. tak compare reads the setting from BASE's tak.toml, like the rest of its gate, so a pull request that turns it on still fails on an empty comparison, and the report says the gate policy changes once it is merged.

A workflow that classifies results should treat exit status 0 with a first line starting **Nothing was compared as its own outcome, neither a pass nor a failure. tak's own perf-pr.yml reports it as a neutral check with a warning. jdx/tak-action reads the first line whatever the exit status, so its fail-on-nothing-compared input still decides there: set it to false as well to make the action warn instead of failing.

Each benchmark can have its own gate in tak.toml: a different percentage, an absolute min_delta floor, or enabled = false to report a benchmark without failing on it. See per-benchmark gates. tak compare reads these, like the rest of its gate, from the base revision's tak.toml; see where the gate comes from.

When some benchmark's gate differs from [gate], the verdict says N benchmark(s) above their gate instead of N benchmark(s) above the 1% gate, and a report-only benchmark that rose is listed as N report-only benchmark(s) above their gate. Classify a result by its exit status first. tak compare exits non-zero only for a regression in a gated benchmark, for an empty comparison unless allow_empty is on, or for an error. The report echoes text from the change under test, such as benchmark names and trailer values, so use it only to tell those non-zero cases apart, and to tell an exit status of 0 that compared nothing from one that compared everything:

• Nothing compared: the report's first line starts with **Nothing was compared. tak writes that line before any name. With exit status 0, allow_empty or --no-gate let it pass; see when comparing nothing is routine.
• Regression: a line matching ^\*\*[0-9]+ benchmark\(s\) above (the .*% gate|their gate). Anchor the pattern to the start of a line, and match both forms.
• Anything else is an error.

tak rejects control characters in names and escapes them in reports, so echoed text cannot start a line. The exit status still keeps a check correct even if it did. tak's own perf-pr.yml classifies its check this way.

To see which functions a change came from, keep cachegrind's profiles on both sides; see Explain an instruction-count change.

Always keep measurements partitioned by runner class. Comparing numbers across runner classes turns an infrastructure change into an apparent code regression.

Where the gate comes from

tak compare BASE --rev REV takes its gate policy from the tak.toml in BASE's tree, read out of git, not from the working tree. The policy is [gate] (pct, min_delta, accept_trailers and allow_empty) and every per-benchmark gate table. In a pull-request job the working tree is the pull request, and a gate it can edit is a gate it can waive: raise pct, mark the benchmark it regressed enabled = false, turn on accept_trailers and add a trailer, or turn on allow_empty and compare nothing. The base has already been reviewed and merged.

• Flags and environment variables still override the file. --gate-pct, --gate-min-delta, --accept, --no-gate, --allow-empty, TAK_GATE_PCT, TAK_GATE_MIN_DELTA, TAK_ACCEPT_TRAILERS and TAK_ALLOW_EMPTY behave as before. The workflow sets them, not the pull request.
• The file is found the way tak run finds its own: searching upward from the current directory, but through BASE's tree, up to the repository root. Deleting, moving or adding a tak.toml in the pull request does not change which file gates it. A tak.toml committed as a symlink is followed within BASE's tree. A link that is absolute, leads outside the repository, or names a file BASE does not have is an error.
• No tak.toml at BASE means tak's defaults plus any flags and environment variables. The working tree's file is never used instead. The report says so in a line below the verdict.
• A BASE tak.toml that does not parse fails the comparison, naming the base. Fix it on the base branch. This includes a file an older tak accepted and the running tak does not.
• These failures are errors. An unparseable BASE tak.toml, a bad symlink, and missing BASE objects (below) fail before any report is printed. --no-gate never fails on the comparison, but errors still fail, and these are errors.
• Everything else still comes from the working tree. [report] credit only changes the footer, and nothing else in tak.toml affects a comparison: the benchmarks were already measured by tak run on each side.

When the working tree's tak.toml would gate some series differently from BASE's, under the same flags and environment, the report adds a line below the verdict saying which file applied, what changed, and that the change takes effect once it is merged. It never comes first, so the first-line check for **Nothing was compared still holds. Only effective values count: adding a benchmark with no gate table, which is held to [gate] either way, is not a change.

The gate comes from `tak.toml` at the base, `a1b2c3d4e5f6`. This revision changes the gate policy (`startup`), and the change takes effect once it is merged.

The list names [gate] when the global gate changed, accept_trailers or allow_empty when that did, and each benchmark or benchmark (subject) whose own gate changed.

This changes behaviour. tak 0.0.13 and earlier read [gate] from the working tree. Now a pull request that loosens a gate is compared under the old gate until it merges. To let a specific regression through on that pull request, pass --accept BENCH from the workflow, for example from a maintainer-applied label, rather than editing the gate. A pull request that tightens a gate or adds a benchmark's gate is also compared under the old one, and its gate applies to the pull requests after it.

BASE's commit and tree objects must be in the clone. The documented pull-request workflow fetches the base branch with full history before finding the merge base, so they are. When the base commit is missing, tak compare fails with cannot resolve BASE and asks for it to be fetched. When the commit is present but its tree is not, as in a --filter=tree:0 partial clone that cannot reach its remote, it fails with cannot read the gate policy from the base rather than falling back to the defaults.

This closes the route through tak.toml only. A pull-request job that installs tak from the pull request's own mise.toml, or runs pull-request code before comparing, is still running code the pull request controls.

Detect regressions that landed

tak compare only runs where a workflow runs it, usually on pull requests. A regression can still reach the main branch: the check was not required, someone merged over it, a commit was pushed directly, or several changes each stayed under the gate. tak detect looks at what landed. Run it in the main-branch workflow after recording and pushing:

tak run --record
tak push
tak detect

It walks the first-parent history of HEAD (or the revision given) back to the 20th commit with measurements (--window changes the count) and compares each series' consecutive recorded points. A series is a benchmark, tool, and runner class; different runner classes are never compared.

• A step onto the newest commit above the gate fails the command. No older step can: the run for the next commit passes, so a main workflow fails once, on the push that introduced the step, instead of on every push after it.
• A step across unrecorded commits is reported as a range, such as a1b2c3d4e5f6..0f9e8d7c6b5a (3 commits). tak cannot tell which commit in the range caused it. A push of several commits records only its tip, so this is common.
• Earlier steps above the gate within the window are listed without failing.
• Sustained drift is listed without failing: the series rose above the gate across the window, counted from its last above-gate step, while every individual step stayed under it.

Only instruction counts are examined. Wall-clock time is shown beside each step and never gates. Each series is held to the same gate tak compare would use: per-benchmark gates from the working tree's tak.toml, otherwise [gate]. That applies to steps and drift alike, min_delta floor included. A report-only benchmark (enabled = false) is still examined, and a step past its threshold is marked (not gated), but it never fails the command. When every benchmark is report-only, tak detect can fail only when nothing could be compared.

When the newest commit has no instruction counts, or no series on it has an earlier point in the window, the report says Nothing was compared and the command fails, because a check that examined nothing would otherwise look like a pass. allow_empty (--allow-empty for one run) waives this case only, with a warning, so a step onto the newest commit still fails. tak detect reads it from the working tree's tak.toml, which on a main-branch run is the merged commit. --no-gate makes the command report without failing on either, as it does for tak compare. Errors still fail.

On the first recording, or the first on a new runner class, there is nothing earlier to compare with. Pass --allow-empty for that run, or seed the history first by recording an earlier commit on the same runner class. If the runner class changes routinely, set [gate] allow_empty = true; see when comparing nothing is routine for what that costs. Other causes are configuration problems:

• The checkout has no history. The walk needs the commits between recorded points, and the default actions/checkout fetch of one commit leaves nothing to walk. The report notes when a shallow clone cut the walk short.
• The previous recording is more than 10,000 first-parent commits back. The walk stops at that limit, and the report says when the limit stopped it before the window filled.
• The recording step ran without Valgrind, so it stored timing but no instruction counts.
• The commit was never recorded. Run tak detect after tak run --record.

This is a simple step detector for near-deterministic counts, not statistical change-point detection. It does not model noise and does not look at timing metrics.

Accepted steps on main

tak detect understands acceptances the same way tak compare does. --accept BENCH is repeatable and takes exact names. Tak-Accept: trailers are honoured only when accept_trailers is on, and each names one benchmark, commas included; repeat the trailer to accept several. An accepted step onto the newest commit is marked (accepted), listed with where the acceptance came from, and does not fail the command. Acceptances that waived nothing are listed too.

Trailers are read from the step's own range: the first-parent commits after the series' previous recorded point, up to and including the newest commit. These are the commits that landed on the branch. A trailer on a pull-request branch commit behind a merge commit's second parent is not in that range. Under squash or rebase merging, the trailer has to survive into the commit that lands. Under merge commits, it has to be in the merge commit's message.

The two tools do not share their acceptances. An acceptance given on a pull request through a label is an --accept flag on that job, not part of git history. On main, tak detect does not see it and fails once, on the commit that introduced the step. There are two ways around that:

• A trailer. Put Tak-Accept: BENCH in the commit message that lands on main, and enable accept_trailers in the main-branch workflow, for example TAK_ACCEPT_TRAILERS=1 tak detect. On main, trailers are part of merged, reviewed history. That is why enabling them there is reasonable, unlike on pull requests, where the commits are the change being gated.
• A manual rerun. Re-run the main-branch check with tak detect --accept BENCH for that commit. The next push passes without it, because only the step onto the newest commit can fail.

Accept an intentional regression

Some changes make a benchmark more expensive on purpose. Rather than raising the gate for every benchmark or reaching for --no-gate, name the benchmarks the change is allowed to regress:

tak compare "$BASE_SHA" --accept startup

--accept is repeatable. Each value is one exact benchmark name. It is neither split on commas nor trimmed, so it can accept any name, and an empty value is an error. It accepts only the benchmarks it names. Every other benchmark still gates. An acceptance names a benchmark and covers each of its tools and runner classes. It has no size limit.

Each series is first judged against its own gate: the benchmark's gate if it has one, [gate] otherwise, including any min_delta floor. An acceptance only waives a change that crossed that gate and would otherwise fail.

An accepted regression still appears in the report. Its row is marked (accepted), and a separate line names its runner class, its gate when benchmarks have their own, and where the acceptance came from. It does not fail the command. The report also lists acceptances that accepted nothing, with the reason:

• the benchmark did not rise above its gate;
• the benchmark is report-only (gate = { enabled = false }), so it can never fail and needs no acceptance; or
• no benchmark by that name was compared.

A misspelt name does not fail the command by itself, but the regression it was meant to cover still does.

Accept from a pull-request label

The recommended CI path is a pull-request label that only maintainers can apply. The workflow maps the label onto --accept. The label names the benchmark, and a maintainer decides whether to accept the regression. For example, a GitHub Actions step can pass every tak-accept:NAME label on the pull request:

      - name: Compare and gate
        env:
          BASE_SHA: ${{ steps.base.outputs.sha }}
          LABELS: ${{ toJSON(github.event.pull_request.labels.*.name) }}
          TAK_ACCEPT_TRAILERS: "0"
        run: |
          accept=()
          while IFS= read -r label; do
            case "$label" in
              tak-accept:*) accept+=(--accept "${label#tak-accept:}") ;;
            esac
          done < <(jq -r '.[]' <<<"$LABELS")
          tak compare "$BASE_SHA" "${accept[@]}"

The labels are passed as JSON and read one per line, so a label for a benchmark whose name contains a space stays whole. TAK_ACCEPT_TRAILERS: "0" keeps trailers off on pull requests even if the base branch's tak.toml turns them on; see below.

To re-run the gate when a label changes, add labeled and unlabeled to the workflow's pull_request types. Only people with triage or write access to the repository can apply labels.

Accept from a commit trailer

A change can also carry its acceptance in a commit message. Put a Tak-Accept trailer in the final paragraph:

feat: load plugins at startup

Plugins now load eagerly so the first command does not pay for discovery.

Tak-Accept: startup

Trailers are ignored unless the project opts in. The commits that tak compare reads are the change being gated. On a pull request, they are the author's own commits. If tak honoured trailers by default, any change could waive its own gate by adding one line. Enable trailers only when every commit in the compared range is reviewed before the result matters, or when everyone who can push may waive the gate:

[gate]
accept_trailers = true

TAK_ACCEPT_TRAILERS=1 enables trailers for one invocation. tak compare reads accept_trailers from the base revision's tak.toml, like the rest of its gate, so a pull request cannot turn trailers on for itself. A project that sets accept_trailers = true in tak.toml for its main-branch tak detect turns them on for pull requests too, once that is merged. Set TAK_ACCEPT_TRAILERS=0 in the pull-request workflow to keep them off there. The environment takes precedence over the file.

With trailers off, the report still says when the compared range contains Tak-Accept trailers. That line names them and states that they were not honoured, so an author knows why their trailer had no effect.

With trailers on, tak compare BASE --rev REV reads trailers from every commit in BASE..REV, including commits reached through a merge commit's second parent. Each trailer names exactly one benchmark: its whole value is the name, commas included, because benchmark names are unrestricted and a,b is a valid one. To accept several benchmarks, repeat the trailer:

Tak-Accept: startup
Tak-Accept: resolve

git trims a trailer's value, so a benchmark whose name starts or ends with a space can only be accepted with --accept. Put the reason in the commit body; the trailer value contains only the name. The comparison needs every commit in BASE..REV. A shallow checkout that omits some commits also omits their trailers, and the regression then fails the gate.

Squash merges

A pull request gate reads the branch's own commits. After a squash merge, the only commit left in history is the squashed one. Keep the trailer in the final paragraph of that commit's message if later comparisons along the main branch should see the acceptance. git reads trailers only from the last paragraph. A squash message that concatenates each commit's message can leave Tak-Accept: in the middle of the body, where it is ordinary text. Check the merged commit with:

git log -1 --format='%(trailers:key=Tak-Accept)'

Browse history

tak history shows the records on one commit. tak log shows how each benchmark changed over time: it walks the first-parent history of a revision (HEAD by default) and prints one table per benchmark and runner class, newest commit first. Commits with nothing recorded are skipped. Several records for one series on one commit reduce to the minimum, the same rule tak compare uses.

tak log HEAD~10 -n 5 --bench startup

On a scratch repository with synthetic measurements recorded on most commits, with the footer lines and the count of commits walked omitted:

5 recorded commit(s) on the first-parent history of `HEAD~10`, 2026-08-09 to 2026-08-14. Older recorded commits are not shown; `-n` shows more.

### `startup` on `gha-linux-x64`

| commit    | date       | instructions |      Δ | wall (min) | subject |
|-----------|------------|-------------:|-------:|-----------:|---------|
| `cd025ff` | 2026-08-14 |   30,014,101 | +0.00% |    11.86ms | fix(run): keep stderr on failure (#114) |
| `8801dc0` | 2026-08-12 |   30,013,263 | +2.03% |    11.90ms | perf: cache parsed manifests (#112) |
| `09c874e` | 2026-08-11 |   29,415,344 | -0.01% |    13.27ms | refactor: split config loader (#111) |
| `e2ce659` | 2026-08-10 |   29,417,425 | +0.01% |    11.49ms | chore(deps): update serde (#110) |
| `1b45311` | 2026-08-09 |   29,414,506 |      — |    11.07ms | fix: skip empty lockfile (#109) |

Δ is the change in instruction count from the series' previous measurement, which may be several commits earlier. -n counts recorded commits, not commits walked. The walk stops at the first recorded commit past the limit, so a short report does not read the whole recorded history. With --bench, it stops there only once every named benchmark has also been seen, which can be much further back, or at the start of history if one never was. It says that older recorded commits exist, not how many. The output is Markdown, so it can be appended to $GITHUB_STEP_SUMMARY.

A series never crosses runner classes: moving to a new runner class starts a new table rather than a jump in an existing one. A clone made with --depth only has part of the history. When the walk runs out of commits in a shallow clone, tak log says that older measurements may exist. Check out with fetch-depth: 0 in CI.

tak log only displays measurements. It does not detect regressions, and it does not gate.

Publish a history report

tak log --html PATH writes the same history as one self-contained HTML page. Each series gets an instruction-count chart and a smaller wall-time chart beneath it. The page has no scripts and no external stylesheets or fonts. It follows the viewer's light or dark preference, and hovering a point shows its commit, subject, and values. When origin is a GitHub repository, points and table rows link to the commit.

The report's format is pre-v1 and may change between tak releases.

To publish it to GitHub Pages after each main-branch recording, add two jobs to the main-branch workflow from Adopt tak in a project, and set the repository's Pages source to GitHub Actions:

  report:
    needs: measure
    runs-on: ubuntu-24.04
    permissions:
      contents: read
    steps:
      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
        with:
          # The first-parent walk needs the commits, not only the notes.
          fetch-depth: 0
          persist-credentials: false

      - uses: jdx/mise-action@c2a87611a18de5b3828c5652fe268e992400cb5c # v4.3.0

      - name: Render the report
        run: |
          mkdir -p site
          tak log -n 200 --html site/index.html

      - uses: actions/upload-pages-artifact@fc324d3547104276b827a68afc52ff2a11cc49c9 # v5
        with:
          path: site

  deploy-report:
    needs: report
    runs-on: ubuntu-24.04
    permissions:
      pages: write
      id-token: write
    environment:
      name: github-pages
      url: ${{ steps.deployment.outputs.page_url }}
    steps:
      - id: deployment
        uses: actions/deploy-pages@368f82528645a54fb793d4d04e342629a3f51346 # v5.0.1

needs: measure orders rendering after the measurement job pushes its notes. tak log fetches refs/notes/tak from origin itself. That fetch is unauthenticated here because the checkout does not keep its credentials, which is enough for a public repository. For a private repository, fetch the notes before removing credentials, as the pull-request example in Adopt tak in a project does.

A repository has one Pages site. If Pages already serves the project's documentation, write the report into that site's build, for example as perf/index.html, instead of deploying it separately.