Documenting with AI/Chapter 28 of 35

The same-commit rule

4 min readEdit on GitHub

When you ship code, you ship the doc that describes it. Same commit. Always.

The rule

Code and its documentation move together. A PR that changes behavior without updating the corresponding doc is not ready to merge. A PR that updates documentation about behavior not yet shipped is not ready either.

No exceptions for "we'll do the docs next sprint." Next sprint never comes. Documentation debt compounds like code debt and is harder to repay because nobody is paged when it breaks.

What the rule applies to

  • Function/class signatures → docstrings
  • Public API surface → README and reference docs
  • Model behavior → model card
  • Performance characteristics → benchmark numbers
  • Data schema → data card and dataset README
  • New flags / config options → user-facing examples

Why this is hard with AI agents

The agent will, by default, edit code and skip the doc. Three reasons:

  1. The PR description summarizes the change, which feels like documentation.
  2. The doc lives in a different file/directory; the agent's "diff" mental model favors local edits.
  3. Updating the doc requires re-measuring numbers, which takes time.

Counter with explicit rules in CLAUDE.md:

## Documentation (hard rules)

- Every change to a public function updates its docstring in the same commit.
- Every change to model behavior updates the model card in the same commit.
- Every change to a measured number requires re-running the benchmark and
  updating the source script + result file in the same commit.
- "TBD" or empty cells are acceptable. Stale numbers are not.
- Doc-only commits are fine. Code-without-doc commits are not.

What "in sync" looks like

A change to src/my_pkg/model.py that adjusts the architecture:

  • src/my_pkg/model.py — the change itself
  • tests/unit/test_model.py — updated assertion if applicable
  • docs/models/<name>.md — updated architecture description, params count
  • BENCHMARK.md — updated number (if behavior changed)
  • benchmarks/results/<latest>.json — re-run output
  • CHANGELOG.md — line entry

One commit. Or one PR with one merge. Not "I'll PR the docs separately."

The propagation matrix

For a non-trivial repo, write down which doc gets updated for which kind of change. Example:

When you change…You must also update…
Public function signature in src/Its docstring + the API reference page
A metric or eval scriptBENCHMARK.md + the model card that cites the metric
Data preprocessingdocs/data/<dataset>.md + the manifest version
Model architectureModel card + BENCHMARK.md (re-run)
Hyperparameter defaultThe recipe / how-to that uses it
Dependency versionCHANGELOG.md if it affects behavior

Put this matrix in CLAUDE.md. The agent reads it; you don't have to repeat it.

Catching violations

A pre-merge check, runnable by a slash command:

#.claude/commands/doc-sync.md
Inspect the current diff. For each change, check whether the relevant doc
was updated in the same commit, per docs/06-documentation/same-commit-rule.md
and the project's propagation matrix.

Report:

- Files changed
- Required doc updates
- Whether each was done

Do not edit. Just report.

Wire to /preflight (see the guardrails checklist).

A specific anti-pattern

The "I'll write a big docs sweep" PR. Once a quarter, someone proposes a massive documentation update to "catch up." This:

  • Ships hundreds of changes at once with no clear review surface.
  • Almost always misses things because the author no longer remembers the commit-to-commit context.
  • Reinforces the bad habit ("docs can lag, we'll catch up later").

Better: small, continuous updates per PR. The doc surface stays current; no sweep is ever necessary.

When the doc must lead the code

A separate scenario: spec-driven work, where the doc describes a behavior that does not yet exist. This is fine — see the doc-driven development chapter. The "same-commit" rule applies in reverse: the spec ships in the same commit as the implementation, even if the spec was drafted earlier.

Doc that describes future behavior must be marked clearly:

> Status: planned for v0.4. Not implemented yet.

Otherwise readers assume the feature exists. Worse, the agent assumes the feature exists and writes code against it.

Cultural reinforcement

The team that practices this for six months stops noticing it. The team that doesn't accumulates a quarter of stale docs and starts losing reviewers.

The rule sounds heavy. It costs minutes per PR. It saves the alternative — fighting an unreviewable swamp later.