Skip to content

chore(changelog): one fragment file per entry in changelog.d/, collected at release (LAB-6153) - #80

Merged
27Bslash6 merged 4 commits into
mainfrom
lab-6153-changelog-fragments
Sep 28, 2026
Merged

27Bslash6 merged 4 commits into
mainfrom
lab-6153-changelog-fragments

Conversation

@27Bslash6

@27Bslash6 27Bslash6 commented Sep 28, 2026 •

Copy link
Copy Markdown
Contributor

Changelog entries now go in changelog.d/, one file per change, and CHANGELOG.md is rewritten only at release. Every PR used to add its entry right under the one ## [Unreleased] heading, so any two open PRs conflicted on the same hunk. Separate files cannot conflict.

The red changelog-fragments check on this PR is expected. This PR moves the existing Unreleased entries out of CHANGELOG.md, and the new guard fails any non-release/* PR that edits that file. This PR is the guard's first live failure. The guard never runs on push.

What changes

  • changelog.d/README.md is the contributor rule. Write the entry, exactly as it should appear, to changelog.d/<YYYYMMDD>_<ticket-id>.md.
  • changelog.d/20260328_unreleased-since-1.0.0.md holds every entry merged since 1.0.0, moved byte-for-byte out of CHANGELOG.md. The name sorts first, so the next release's section starts with it.
  • tools/changelog-collect.py VERSION runs on a release/VERSION branch. It writes ## [VERSION] - DATE at the <!-- changelog-insert-here --> marker, concatenates the fragments verbatim in filename order, and deletes them. It fails closed, writing nothing, on a missing or repeated marker, an existing version, a malformed version or date, no fragments, or an empty fragment.
  • tools/test_changelog_collect.py runs in verify.yml. It also dry-runs the next release against the real CHANGELOG.md and changelog.d/, so a fragment that would break a release fails CI first.
  • verify.yml also gets the changelog-fragments job. It runs on pull_request only, skips release/* heads, and diffs CHANGELOG.md between the test-merge commit and its first parent.

Why a script and not scriv, release-please or merge=union

  • scriv treats every ### heading as a category. On this changelog it merged the two ### SaaS API sections and reordered entries across the release. These entries are normative prose, so a verbatim concatenation is the only safe transform.
  • release-please and git-cliff only see squash titles, because this repo's squash message is blank. Generating on merge would also need a token that can push to main directly.
  • merge=union does not help. GitHub's mergeability check ignores custom merge drivers (community discussion).

Verification

  • python3 tools/test_changelog_collect.py passes 10/10. The cases cover:

    • a fixture whose filename order disagrees with lexical order, with a split duplicate heading;
    • a second release landing above the first;
    • the real-repo dry run;
    • six fail-closed refusals that each leave the files untouched.
  • Mutation testing: 6 of 7 mutants of the collect script fail the suite, across 6 runs:

    • reversed order;
    • fragments kept after collection;
    • an existing version allowed;
    • an empty fragment allowed;
    • regrouped entries;
    • altered content.

    The survivor, which drops the repeated-marker check, is equivalent: the split still raises on two markers.

  • The moved block is byte-identical to main's Unreleased section.

  • I took the five open PRs whose only conflict with main is CHANGELOG.md, moved each one's entry into a fragment, and merged all five into one branch: no conflicts.

  • The guard's step script, extracted with yq, fails on simulated test-merges that edit CHANGELOG.md and passes on fragment-only and unrelated changes. actionlint is clean.

Open PRs that still edit CHANGELOG.md need one last resolution after this merges. Keep main's CHANGELOG.md and move the PR's entry into a fragment. The guard's error message says the same.

Closes LAB-6153

This PR replaces scriv with an in-repo collector for release changelogs. Each changelog entry is now its own file in changelog.d/, so concurrent PRs no longer conflict on a shared ## [Unreleased] heading. CHANGELOG.md itself is only rewritten when a release is cut.

Why

scriv treated every ### heading as a category. That caused two problems for these entries:

  • It merged entries that shared a title (e.g. ### SaaS API).
  • It reordered entries across a release.

The entries are curated normative prose, so the new tool concatenates fragments verbatim instead.

Changes

  • New tool: tools/changelog-collect.py

    • CLI: python3 tools/changelog-collect.py VERSION [--date YYYY-MM-DD] [--root PATH]
    • Public function: collect(root: Path, version: str, date: str) -> list[Path]
    • Constant: MARKER = "<!-- changelog-insert-here -->"
    • Writes a ## [VERSION] - DATE section at the marker in CHANGELOG.md. The section holds every fragment (excluding README.md) verbatim, in filename order. The collected fragments are then deleted.
    • Nothing is parsed, regrouped or reordered.
    • It fails closed and writes nothing if any of these hold:
      • The marker is missing or appears more than once.
      • The version is not MAJOR.MINOR.PATCH.
      • The version section already exists.
      • There are no fragments.
      • Any fragment is empty.
  • CHANGELOG.md

    • The scriv markers (<!-- scriv-insert-here --> / <!-- scriv-end-here -->) are replaced by the single <!-- changelog-insert-here --> marker.
    • All unreleased entries accumulated since 1.0.0 move unchanged into changelog.d/20260328_unreleased-since-1.0.0.md. That file sorts first, so the next release contains it.
  • changelog.d/README.md

    • The release instructions now use tools/changelog-collect.py in place of uvx scriv@1.8.0 collect.
    • It clarifies that filename order reflects when an entry was written, not when it merged, and that no entry should depend on its position relative to another.
  • Tests: tools/test_changelog_collect.py (stdlib only) cover:

    • Verbatim output in filename order, with duplicate headings kept separate.
    • Deletion of fragments while the README is preserved.
    • Successive releases stacking correctly.
    • A dry run against the repository's real CHANGELOG.md and changelog.d/.
    • Each fail-closed refusal leaving all files untouched.
  • CI (.github/workflows/verify.yml)

    • Adds a "Changelog collect (stdlib only)" step running the test suite. A fragment or marker that would break the next release fails in CI first.

This PR makes the real-repo dry-run check in the changelog collection tests future-proof. The previous check stops being valid after the first fragment-based release.

Changes (tools/test_changelog_collect.py)

  • New helper dry_run_next_release(tmp): Replaces the inline check in main() that hardcoded version 1.1.0 and assumed ## [1.0.0] was the next heading. The helper:
    • Copies the repo's CHANGELOG.md and changelog.d/ into a temporary directory.
    • Skips with a message if there are no pending fragments (i.e., right after a release), rather than failing.
    • Finds the newest released version by parsing ## [X.Y.Z] headings with a regex, and targets one minor version above it (X.(Y+1).0).
    • Runs cc.collect on the copy with a fixed date (2026-10-01).
    • Checks that:
      • everything up to and including cc.MARKER is unchanged,
      • it is followed by the new version heading,
      • all existing content after the marker is kept intact,
      • the inserted body equals every pending fragment verbatim, joined by blank lines.
  • Adds an import re for version parsing.
  • Renames the check label from "first release holds every pending fragment verbatim" to "next release (<version>) holds every pending fragment verbatim."

Impact

Test-only change; no public or production APIs are modified. The only new callable is the test-module-level function dry_run_next_release. The existing refuses(...) negative cases are unchanged.

@coderabbitai

coderabbitai Bot commented Sep 28, 2026 •

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

Next included review available in 21 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Learn how review limits work.

Review configuration:

⚙️ Run configuration

Configuration used: Repository: cachekit-io/protocol/.coderabbit.yaml

Review profile: ASSERTIVE

Plan: Advanced

Run ID: f7db190e-45fc-4e5a-88fb-fb9be0b0ea16

📥 Commits

Reviewing files that changed from the base of the PR and between 965aeb0 and 9babfca.

📒 Files selected for processing (6)
  • .github/workflows/verify.yml
  • CHANGELOG.md
  • changelog.d/20260328_unreleased-since-1.0.0.md
  • changelog.d/README.md
  • tools/changelog-collect.py
  • tools/test_changelog_collect.py

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@kodus-27b

kodus-27b Bot commented Sep 28, 2026 •

Copy link
Copy Markdown

Code Review Completed! 🔥

The code review was successfully completed based on your current configurations.

Kody Guide: Usage and Configuration
Interacting with Kody
  • Request a Review: Ask Kody to review your PR manually by adding a comment with the `@kody start-review` command at the root of your PR.

  • Provide Feedback: Help Kody learn and improve by reacting to its comments with a 👍 for helpful suggestions or a 👎 if improvements are needed.

Providing Context (Files & MCPs)

Add these hints in your PR description (or a comment) to unlock deeper checks:

  • Ticket / Acceptance Criteria: `Refs: ABC-123` (Linear/Jira/Asana/ClickUp/Trello) or a direct ticket link.
  • Bugfix Validation: a Sentry/Datadog/Bugsnag event link (or paste the stack trace/error message).
  • Endpoint Risk: mention the route (e.g., `POST /api/payments`) or controller/action name.
  • Attach a repo file as context: use an explicit marker like `@file:docs/guide.mdx#L10-L50` (replace with your real path).
  • API Contract Docs: include `@file:openapi.yaml` or `@file:swagger.json` when changing routes/schemas.
  • Definition of Done / Standards: include `@file:DOD.md` or `@file:CONTRIBUTING.md` if your repo has them.
  • Design System Source of Truth: include `@file:ui/index.ts` (replace with your DS entrypoint path).
  • Feature Flags: include the flag key/name and `@file:flags.ts` / `@file:config.json` (and optionally the PostHog flag name).
  • Edge/CDN Rules: link the Cloudflare rule/zone or describe the intended redirect/header behavior.
  • Attach an MCP tool output: use `@mcp<provider|tool>` (replace with an installed MCP provider + tool, e.g., `@mcp<sentry|events.search>`).
Current Kody Configuration
Review Options

The following review options are enabled or disabled:

Options Enabled
Bug ✅
Performance ✅
Security ✅
Business Logic ✅

Access your configuration settings here.

Kody Code Review — 1 suggested fix.
Paste the prompt below to your agent and all review fixed at once!

🛠️ Open Agent Prompt
A code review identified the following issues in this pull request.
Each section describes what was found and includes a reference implementation where available.

Files involved:
- .github/workflows/verify.yml:88

---

### [1/1] .github/workflows/verify.yml:88
Issue identified during code review:
Authorization bypass in changelog-fragments job condition: the release exemption checks only `github.head_ref`, which the PR author controls, including on forks where the base repo's branch rules do not apply. When a contributor opens a PR from a fork branch named `release/fix` that edits `CHANGELOG.md` directly, the job is skipped, and because a skipped job reports as passing, the required check does not block the merge. Fix: also require `github.event.pull_request.head.repo.full_name == github.repository` so the exemption applies only to branches in this repository.
Reference implementation (from code review):

// .github/workflows/verify.yml:88
if: github.event_name == 'pull_request' && !(startsWith(github.head_ref, 'release/') && github.event.pull_request.head.repo.full_name == github.repository)

---

Review each issue in context, use the reference implementations as guidance, and apply fixes that are consistent with the surrounding codebase.

# changelog.d/ (one file each); only a release/* branch rewrites CHANGELOG.md.
changelog-fragments:
name: CHANGELOG.md is edited only by releases
if: github.event_name == 'pull_request' && !startsWith(github.head_ref, 'release/')

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

kody code-review Bug medium

Authorization bypass in changelog-fragments job condition: the release exemption checks only github.head_ref, which the PR author controls, including on forks where the base repo's branch rules do not apply. When a contributor opens a PR from a fork branch named release/fix that edits CHANGELOG.md directly, the job is skipped, and because a skipped job reports as passing, the required check does not block the merge. Fix: also require github.event.pull_request.head.repo.full_name == github.repository so the exemption applies only to branches in this repository.

if: github.event_name == 'pull_request' && !(startsWith(github.head_ref, 'release/') && github.event.pull_request.head.repo.full_name == github.repository)
Prompt for LLM

File .github/workflows/verify.yml:

Line 88:

Authorization bypass in changelog-fragments job condition: the release exemption checks only `github.head_ref`, which the PR author controls, including on forks where the base repo's branch rules do not apply. When a contributor opens a PR from a fork branch named `release/fix` that edits `CHANGELOG.md` directly, the job is skipped, and because a skipped job reports as passing, the required check does not block the merge. Fix: also require `github.event.pull_request.head.repo.full_name == github.repository` so the exemption applies only to branches in this repository.

Suggested Code:

if: github.event_name == 'pull_request' && !(startsWith(github.head_ref, 'release/') && github.event.pull_request.head.repo.full_name == github.repository)

Talk to Kody by mentioning @kody

Was this suggestion helpful? React with 👍 or 👎 to help Kody learn from this interaction.

​

​

Comment on lines +49 to +51
changelog.write_text(f"{head}{MARKER}\n\n{section}{rest.lstrip(chr(10))}", encoding="utf-8")
for p in fragments:
p.unlink()

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

kody code-review Bug medium

Partial-failure state in collect(): CHANGELOG.md is written before the fragments are unlinked, so an OSError on any unlink leaves the new version section in CHANGELOG.md while some fragments remain. When a fragment is read-only or its directory denies write permission, main() exits 1, a rerun is refused with 'already has a section', and unless someone removes the leftover fragments by hand, the next release collects them again and duplicates those entries. Fix: delete the fragments first and restore them from their saved contents if the CHANGELOG.md write fails, or verify every fragment is deletable (e.g. os.access on the parent directory) before writing.

new_text = f"{head}{MARKER}\n\n{section}{rest.lstrip(chr(10))}"
for p in fragments:
    p.unlink()
try:
    changelog.write_text(new_text, encoding="utf-8")
except OSError:
    for p, body in zip(fragments, originals):
        p.write_text(body, encoding="utf-8")
    raise
Prompt for LLM

File tools/changelog-collect.py:

Line 49 to 51:

Partial-failure state in collect(): CHANGELOG.md is written before the fragments are unlinked, so an OSError on any unlink leaves the new version section in CHANGELOG.md while some fragments remain. When a fragment is read-only or its directory denies write permission, main() exits 1, a rerun is refused with 'already has a <version> section', and unless someone removes the leftover fragments by hand, the next release collects them again and duplicates those entries. Fix: delete the fragments first and restore them from their saved contents if the CHANGELOG.md write fails, or verify every fragment is deletable (e.g. os.access on the parent directory) before writing.

Suggested Code:

    new_text = f"{head}{MARKER}\n\n{section}{rest.lstrip(chr(10))}"
    for p in fragments:
        p.unlink()
    try:
        changelog.write_text(new_text, encoding="utf-8")
    except OSError:
        for p, body in zip(fragments, originals):
            p.write_text(body, encoding="utf-8")
        raise

Talk to Kody by mentioning @kody

Was this suggestion helpful? React with 👍 or 👎 to help Kody learn from this interaction.

​

​

try:
collected = collect(args.root, args.version, args.date.isoformat())
except (ValueError, OSError) as e:
print(f"changelog-collect: {e}", file=sys.stderr)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

kody code-review Kody Rules low

Violates team rule 'Replace print statements with logging framework': Use the standard logging module (or your app's logger) instead of print() in committed code.

Also found in:

  • tools/changelog-collect.py:66-66
  • tools/changelog-collect.py:68-68
  • tools/test_changelog_collect.py:32-32
  • tools/test_changelog_collect.py:108-108
Prompt for LLM

File tools/changelog-collect.py:

Line 64:

Violates team rule 'Replace print statements with logging framework': Use the standard logging module (or your app's logger) instead of print() in committed code.

Talk to Kody by mentioning @kody

Was this suggestion helpful? React with 👍 or 👎 to help Kody learn from this interaction.

​

​

Comment thread tools/test_changelog_collect.py Outdated
_spec.loader.exec_module(cc)

BASE = f"# Changelog\n\n## [Unreleased]\n\nPointer.\n\n{cc.MARKER}\n\n## [1.0.0] - 2026-03-28\n\nInitial.\n"
FAILURES: list[str] = []

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

kody code-review Kody Rules high

WHAT: FAILURES is a module-level mutable list, and check() appends to it as a side effect.
WHY: Global mutable state makes results depend on import and call order. It also leaks state if the module is imported or reused, for example by a test runner.
HOW: Keep failures in a local list or a small results object created in main(). Pass it to check() (or have check return a bool that the caller collects), then compute the exit code from that local state.

Kody rule violation: Avoid Using Mutable Global Variables

Prompt for LLM

File tools/test_changelog_collect.py:

Line 28:

WHAT: `FAILURES` is a module-level mutable list, and `check()` appends to it as a side effect.
WHY: Global mutable state makes results depend on import and call order. It also leaks state if the module is imported or reused, for example by a test runner.
HOW: Keep failures in a local list or a small results object created in `main()`. Pass it to `check()` (or have `check` return a bool that the caller collects), then compute the exit code from that local state.

Talk to Kody by mentioning @kody

Was this suggestion helpful? React with 👍 or 👎 to help Kody learn from this interaction.

​

​

@kodus-27b

kodus-27b Bot commented Sep 28, 2026

Copy link
Copy Markdown

Code Review Completed! 🔥

The code review was successfully completed based on your current configurations.

Kody Guide: Usage and Configuration
Interacting with Kody
  • Request a Review: Ask Kody to review your PR manually by adding a comment with the `@kody start-review` command at the root of your PR.

  • Provide Feedback: Help Kody learn and improve by reacting to its comments with a 👍 for helpful suggestions or a 👎 if improvements are needed.

Providing Context (Files & MCPs)

Add these hints in your PR description (or a comment) to unlock deeper checks:

  • Ticket / Acceptance Criteria: `Refs: ABC-123` (Linear/Jira/Asana/ClickUp/Trello) or a direct ticket link.
  • Bugfix Validation: a Sentry/Datadog/Bugsnag event link (or paste the stack trace/error message).
  • Endpoint Risk: mention the route (e.g., `POST /api/payments`) or controller/action name.
  • Attach a repo file as context: use an explicit marker like `@file:docs/guide.mdx#L10-L50` (replace with your real path).
  • API Contract Docs: include `@file:openapi.yaml` or `@file:swagger.json` when changing routes/schemas.
  • Definition of Done / Standards: include `@file:DOD.md` or `@file:CONTRIBUTING.md` if your repo has them.
  • Design System Source of Truth: include `@file:ui/index.ts` (replace with your DS entrypoint path).
  • Feature Flags: include the flag key/name and `@file:flags.ts` / `@file:config.json` (and optionally the PostHog flag name).
  • Edge/CDN Rules: link the Cloudflare rule/zone or describe the intended redirect/header behavior.
  • Attach an MCP tool output: use `@mcp<provider|tool>` (replace with an installed MCP provider + tool, e.g., `@mcp<sentry|events.search>`).
Current Kody Configuration
Review Options

The following review options are enabled or disabled:

Options Enabled
Bug ✅
Performance ✅
Security ✅
Business Logic ✅

Access your configuration settings here.

shutil.copytree(REPO / "changelog.d", real / "changelog.d")
pending = [p.read_text().strip() for p in sorted((real / "changelog.d").glob("*.md")) if p.name != "README.md"]
if not pending:
print("skip repo dry run: no pending fragments (fresh after a release)")

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

kody code-review Kody Rules low

Team-rule violation in tools/test_changelog_collect.py: the skip message uses print() instead of the logging framework, which violates the team rule 'Replace print statements with logging framework'. When the repo dry run has no pending fragments after a release, the message goes straight to stdout and bypasses configured log levels and handlers. Fix: replace print() with a module-level logger call, such as logger.info(...).

Prompt for LLM

File tools/test_changelog_collect.py:

Line 67:

Team-rule violation in tools/test_changelog_collect.py: the skip message uses print() instead of the logging framework, which violates the team rule 'Replace print statements with logging framework'. When the repo dry run has no pending fragments after a release, the message goes straight to stdout and bypasses configured log levels and handlers. Fix: replace print() with a module-level logger call, such as logger.info(...).

Talk to Kody by mentioning @kody

Was this suggestion helpful? React with 👍 or 👎 to help Kody learn from this interaction.

​

​

@kodus-27b

kodus-27b Bot commented Sep 28, 2026

Copy link
Copy Markdown

Code Review Completed! 🔥

The code review was successfully completed based on your current configurations.

Kody Guide: Usage and Configuration
Interacting with Kody
  • Request a Review: Ask Kody to review your PR manually by adding a comment with the `@kody start-review` command at the root of your PR.

  • Provide Feedback: Help Kody learn and improve by reacting to its comments with a 👍 for helpful suggestions or a 👎 if improvements are needed.

Providing Context (Files & MCPs)

Add these hints in your PR description (or a comment) to unlock deeper checks:

  • Ticket / Acceptance Criteria: `Refs: ABC-123` (Linear/Jira/Asana/ClickUp/Trello) or a direct ticket link.
  • Bugfix Validation: a Sentry/Datadog/Bugsnag event link (or paste the stack trace/error message).
  • Endpoint Risk: mention the route (e.g., `POST /api/payments`) or controller/action name.
  • Attach a repo file as context: use an explicit marker like `@file:docs/guide.mdx#L10-L50` (replace with your real path).
  • API Contract Docs: include `@file:openapi.yaml` or `@file:swagger.json` when changing routes/schemas.
  • Definition of Done / Standards: include `@file:DOD.md` or `@file:CONTRIBUTING.md` if your repo has them.
  • Design System Source of Truth: include `@file:ui/index.ts` (replace with your DS entrypoint path).
  • Feature Flags: include the flag key/name and `@file:flags.ts` / `@file:config.json` (and optionally the PostHog flag name).
  • Edge/CDN Rules: link the Cloudflare rule/zone or describe the intended redirect/header behavior.
  • Attach an MCP tool output: use `@mcp<provider|tool>` (replace with an installed MCP provider + tool, e.g., `@mcp<sentry|events.search>`).
Current Kody Configuration
Review Options

The following review options are enabled or disabled:

Options Enabled
Bug ✅
Performance ✅
Security ✅
Business Logic ✅

Access your configuration settings here.

@27Bslash6
27Bslash6 merged commit fc89358 into main Sep 28, 2026
3 of 4 checks passed
@27Bslash6
27Bslash6 deleted the lab-6153-changelog-fragments branch September 28, 2026 21:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant