Skip to content

Make lint exit non-zero on invalid rules, and validate rules in CI - #241

Open
jpl1337 wants to merge 1 commit into
WithSecureOpenSource:masterfrom
jpl1337:feat/lint-fails-on-invalid-rules
Open

jpl1337 wants to merge 1 commit into
WithSecureOpenSource:masterfrom
jpl1337:feat/lint-fails-on-invalid-rules

Conversation

@jpl1337

@jpl1337 jpl1337 commented Sep 21, 2026

Copy link
Copy Markdown

chainsaw lint currently exits 0 even when rules fail to validate, so it cannot be used as a gate — in CI or anywhere else.

$ chainsaw lint --kind chainsaw rules/        # on master today
[!] /mft/netexec_mft.yml: did not find expected key at line 44 column 4, ...
[+] Validated 130 detection rules out of 131
$ echo $?
0

The Command::Lint arm already counts failed — it just never acts on it. This returns an error when that count is non-zero, then adds a workflow that runs the linter on pull requests.

Changes

1. lint fails when rules fail (src/main.rs, 3 lines)

 cs_eprintln!("[+] Validated {} detection rules out of {}", count, count + failed);
+if failed > 0 {
+    anyhow::bail!("{} detection rule(s) failed to validate", failed);
+}

Consistent with the existing bail! / return Err(anyhow!(...)) usage in other subcommands in the same function. Output on success is unchanged; on failure the existing per-file [!] lines are unchanged and a summary error is added.

2. .github/workflows/lint-rules.yml — runs on PRs touching rules/, src/ or the cargo manifests. It builds chainsaw from the PR's own source rather than downloading a release, so a rule that depends on a schema change in the same PR is validated against that change. Uses Swatinem/rust-cache, so only the first run pays the full build.

Verification

Built locally and tested both paths:

binary ruleset exit output
this branch rules/ with #240 applied 0 Validated 131 detection rules out of 131
this branch rules/ as on master 1 [x] 1 detection rule(s) failed to validate
released 2.13.1 rules/ as on master 0 Validated 130 … out of 131 — passes silently

The last row is the problem this fixes.

Note — CI here will be red until #240 merges

This PR's own check will fail, because master still contains the malformed rules/mft/netexec_mft.yml that #240 fixes. That failure is the check doing its job on its first run: the rule has been unloadable since it was added in #226 (2026-04-01) and nothing surfaced it.

Merge #240 first and this goes green. Happy to rebase, or to fold the two together if you would rather review one change.

Open questions

  • actions-rs/toolchain@v1 is used here to match the existing v1.yml / v2.yml, but that action is archived upstream. Happy to switch this workflow to dtolnay/rust-toolchain — I kept it consistent rather than introducing a second pattern unasked.
  • Unconditional failure vs a flag. I have made lint fail by default, on the reasoning that a linter which cannot fail is not useful for automation. If you would rather not change existing behaviour, this could be --fail-on-error / --strict instead, with CI passing the flag. Say which you prefer and I will adjust.
  • The workflow also runs on pushes to master. Easy to drop if you would rather keep it PR-only.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant