fix(bounty): never sign an unpaid epoch as scored by echobt · Pull Request #303 · CortexLM/cortex

echobt · 2026-09-16T13:48:29Z

Links Challenge Bounty rewards to the CortexLM/backend public bounty API, and closes the one
way that linkage could pay nobody while looking healthy.

The bug this fixes

The emitter treated a readable but unpayable feed as a successful score. With zero adjudicated
rows it signed NoScore(NotAttempted) for every participant in E and returned Scored.

That is wrong in the direction that matters. NotAttempted means "the challenge chose not to
invoke this miner" — false when the backend is up and simply has no crowned hotkey — and because
NotAttempted is not the burn cover, the epoch sealed as a legitimate-looking unpaid vector while
/v1/status reported success and can_score stayed true. The 2000 bps still burned to uid 0 and
nothing anywhere said so.

Readable-and-unpaid is now its own outcome, covering E with the same NoScore(ChallengeInternal)
a burn uses. D24 still holds, the share still burns, and the status says which half is missing.

What changed

Emitter (crates/bounty-challenge)

EmitOutcome::Unpaid — feed read, nothing payable. E covered, nobody paid, reported as unpaid
rather than scored. The three cases stay distinct: scored / unpaid (feed read) / burned
(feed unreadable), so an operator knows whether to wait on the backend or look at adjudication.
Held now also covers a feed that stays up and stops paying a hotkey it had already crowned, not
only an outage. A scored epoch is never taken back by a cover.
GET /v1/status publishes the emitter read side: emitter_wired, plus last_outcome,
last_feed_read, last_paid, last_participants, last_reason, scored_epoch.
last_feed_read is the tick's own record rather than inferred from the outcome. A live check
against a stand-in feed showed the inferred version reported false when the feed had been read
and the gateway failed — pointing an operator at the wrong service. ctx bounty status prints
the same fields with a plain-language note per outcome.

Tests

tests/rewards_linkage.rs — the whole chain in one test: pair → report → adjudicate → published
feed → signed leaves → sealed bundle. Asserts the hotkey that signed the pairing challenge is the
one holding the positive leaf (every link can pass while credit lands elsewhere), that a silent
hotkey is explicit rather than omitted, that the leaves verify under the trust-root key, and that a
paid bounty seals in one bundle beside a challenge that scored nothing.
tests/trust_root_linkage.rs — the committed prod and staging roots, verified under the owner
signature, keep bounty live at 2000 bps with shares summing to 10000 and mirroring each other.
tests/emit_fail_closed.rs — extended for the unpaid/burned/held split and the status fields.
Mutation-checked: reverting the guard fails 4 of these tests.

Config / ops

Bounty is already active at 2000 bps in both committed roots; the new tests now hold it there.
Fixed stale bounty 7000 + proof 3000 comments in local-e2e.sh and env-staging.yml — the
script really generated 2000/8000, so the comment was the only thing wrong.
local-e2e.sh asserts the emitter fields and flags unpaid + feed-read distinctly.
Docs: operator env table, adjudication as a hard dependency of the reward path, what validators do
and do not verify, and a /v1/status field reference (docs/BOUNTY.md, deploy/AGENTS.md,
docs/COMPLETENESS.md, docs/external-miner/bounty.md).

Unrelated but blocking: rustls → 0.23.45 for RUSTSEC-2026-0285. The advisory landed
2026-09-14, after the last green CI run, and fails cargo deny on every branch including
origin/main (verified against the unmodified lockfile).

Test plan

cargo test -p bounty-challenge                      # 35 tests: 8 + 18 + 5 + 6
cargo test -p bounty-challenge --test rewards_linkage
cargo test -p bounty-challenge --test trust_root_linkage
cargo fmt --all -- --check
cargo clippy --workspace --all-targets -- -D warnings
cargo test --workspace
cargo deny check
cargo run -p xtask -- loc-cap
cargo run -p xtask -- consensus-lint
cargo run -p xtask -- spec-check
cargo run -p xtask -- design-check
cargo run -p xtask -- external-docs-check
bash deploy/scripts/assert-compose-matrix.sh

All pass locally. One pre-existing failure is unrelated and environmental:
proof-challenge-bin::seed_pf_allocator_refuses_boot_when_a_topic_dir_cannot_be_read chmods a
directory to 0000 and expects the read to fail, which does not happen when the test runs as
uid 0. The file is untouched by this branch and the same test fails on origin/main here.

Runtime check (not just unit tests): the release binary was run against a local stand-in feed with
an unreachable gateway, and /v1/status reported last_outcome: error with
last_feed_read: true and an error naming the chain — the attribution the fix is about.

Not verified here: no live CortexLM/backend call (it answers 503 to this environment) and no
staging droplet run. The e2e path is exercised against a stand-in serving the two published routes.

Notes

Draft until CI is green; leaving the merge decision to Mathis.
No secrets, no baked hosts, no BOUNTY_FORCE_SIM.

The emitter turned a readable-but-unpayable feed into a "scored" tick. With zero adjudicated rows it emitted `NotAttempted` for every participant and reported success, so an epoch that paid nobody sealed as if the challenge had deliberately skipped the miners. That is the one outcome the reward path must not produce silently: the 2000 bps still burned to uid 0, and `can_score: true` plus a green tick said nothing was wrong. Readable-and-unpaid is now its own outcome. It covers `E` with the same `NoScore(ChallengeInternal)` a burn uses, so D24 still holds and the share still burns, and it is reported as `unpaid` rather than `scored`. The three cases stay distinct — `scored`, `unpaid` (feed read, nothing payable), `burned` (feed unreadable) — because an operator needs to know whether to wait on the backend or go look at adjudication. A hold now covers the same ground: a scored epoch is kept when the feed stops paying, not only when it stops answering. `GET /v1/status` publishes the emitter's read side (`emitter_wired` plus last outcome / feed-read / paid counts), and `ctx bounty status` prints it. `last_feed_read` is the tick's own record, not inferred from the outcome: a gateway failure after a successful read reports the feed as read, which a live check against a stand-in feed showed the inferred version got wrong. Tests walk the whole linkage rather than one link. `rewards_linkage.rs` drives pair → report → adjudicate → published feed → signed leaves, asserts the paired hotkey is the one paid, and seals a bundle where a paid bounty sits beside a challenge that scored nothing. `trust_root_linkage.rs` holds the committed prod and staging roots to bounty at 2000 bps with shares summing to 10000 under the owner signature, so the config side of the linkage cannot drift either. The emitter tests were mutation-checked against the old behaviour. Also bumps rustls to 0.23.45 for RUSTSEC-2026-0285, which landed after the last green CI run and now fails `cargo deny` on every branch. Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>

echobt · 2026-09-16T13:58:11Z

@greptileai review

greptile-apps · 2026-09-16T14:18:48Z

Greptile Summary

This update makes readable-but-unpayable bounty feeds explicit, emits fail-closed cover leaves instead of recording a successful score, preserves already-scored epochs, and exposes a coherent emitter snapshot through the status API.

It also records the pin block used for unpaid and burned covers, strengthens concurrent status coverage, and updates related linkage, deployment, CLI, and operator documentation.

Confidence Score: 5/5

Safe to merge.

All previously reported issues are fixed in the current code. Emitter status now publishes counters and last-tick fields under one lock, so readers receive one coherent snapshot. Unpaid and burned cover outcomes retain their actual pinned block. The concurrent status test is bounded, stops after the writer completes, and verifies the final deterministic snapshot.

_{Reviews (4): Last reviewed commit: "test(bounty): bound the status race test..." | Re-trigger Greptile}

greptile-apps · 2026-09-16T14:18:51Z

+        self.ticks.fetch_add(1, Ordering::Relaxed);
+        self.last_kind.store(tick.kind.code(), Ordering::Relaxed);
+        self.last_epoch.store(tick.epoch, Ordering::Relaxed);
+        self.last_pin_block.store(tick.pin_block, Ordering::Relaxed);
+        self.last_participants
+            .store(count_u64(tick.participants), Ordering::Relaxed);
+        self.last_paid
+            .store(count_u64(tick.paid), Ordering::Relaxed);
+        self.last_feed_read.store(tick.feed_read, Ordering::Relaxed);
+        self.scored_epoch
+            .fetch_max(tick.scored_epoch, Ordering::Relaxed);
+        tick.reason
+            .unwrap_or_default()
+            .clone_into(&mut lock(&self.last_reason));
+        tick.error
+            .unwrap_or_default()
+            .clone_into(&mut lock(&self.last_error));


Status Fields Can Mix

record updates one logical tick through separate relaxed atomics and two mutexes, while view reads those fields independently. A concurrent status request can therefore combine fields from different ticks—for example, last_outcome: scored with last_paid: 0, or an unpaid outcome with the previous reason. Since the CLI interprets these fields together, this can send operators or automation toward the wrong failure. Store and read the complete tick as one synchronized snapshot, or use a sequence protocol that retries reads overlapping an update.

greptile-apps · 2026-09-16T14:18:51Z

+            }) => EmitterTick {
+                kind: EmitterOutcomeKind::Unpaid,
+                epoch: *epoch,
+                pin_block: 0,


Cover Pins Are Discarded

Unpaid and burned ticks derive E at a known pin_block, but their status records hard-code last_pin_block to zero, which is documented as meaning “none yet.” After a successful cover, the API therefore publishes a real epoch and participant count while hiding the block used for that participant snapshot. Preserve the pin block in these outcomes and record it. The held case may need separate semantics because its standing score can come from an earlier tick.

Two review findings on the emitter read side, both about the status telling an operator something the tick did not. `record` published a tick through separate relaxed atomics and two mutexes, and `view` read them independently, so a concurrent `/v1/status` could pair one tick's outcome with another's counts — `scored` beside `paid: 0`, or an `unpaid` outcome with a previous tick's reason. Those are exactly the combinations an operator is told to read as a fault, so a reader could be sent after a failure that never happened. The per-tick fields now live in one mutex and are replaced as a single value; `ticks` is bumped last so the count never claims a tick whose fields have not landed. The counters stay separate: their value does not depend on which tick is last. Cover outcomes hard-coded `last_pin_block: 0`, which is documented as "no tick yet". A cover is still signed against a pinned participant snapshot, so the status published a real epoch and participant count while hiding the block a validator would have to reproduce. `Unpaid` and `Burned` now carry the pin they derived `E` at. `Held` keeps 0 on purpose: its leaves came from an earlier tick, and claiming a block this tick never used would be a fabrication. The concurrency test alternates two ticks that differ in every field, so a torn read cannot coincidentally look coherent. Mutation-checked: splitting the write again fails it with `scored` beside `participants: 0, paid: 0`. Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>

echobt · 2026-09-16T14:22:20Z

Both findings fixed in 851f788.

Status fields can mix — the per-tick fields now live in one mutex and are replaced as a single value, so a reader can no longer pair one tick's outcome with another's counts. ticks is bumped last, so the count never claims a tick whose fields have not landed; ticks/scored_epoch stay separate since their value does not depend on which tick is last. The concurrency test alternates two ticks that differ in every field, so a torn read cannot coincidentally look coherent. Mutation-checked: splitting the write again fails it with scored beside participants: 0, paid: 0 — your exact example.

Cover pins are discarded — Unpaid and Burned now carry the pin_block they derived E at, and the status publishes it. Held keeps 0 deliberately: its leaves came from an earlier tick, so claiming a block this tick never used would be a fabrication. Added tests for all three (a_cover_publishes_the_block_it_pinned_e, a_held_tick_does_not_claim_a_pin_it_did_not_use).

echobt · 2026-09-16T14:30:39Z

@greptileai review

greptile-apps · 2026-09-16T14:44:36Z

+            // The reader can legitimately race ahead of the writer's first
+            // tick; that is the documented initial state, not a torn read.
+            EmitterOutcomeKind::Never => {
+                assert_eq!(view.ticks, 0, "Never is only before the first tick");


Initial Status Race

view() copies the locked last-tick state before separately loading ticks. If the first record() completes between those operations, the returned view validly contains last_outcome: Never and ticks: 1. This assertion rejects that valid interleaving, so status_never_pairs_an_outcome_with_another_ticks_counts can fail intermittently under scheduling contention. Accept the independently read monotonic counter for Never, or publish both values as one snapshot if that invariant is required.

Artifacts

Evidence from the check

Evidence file captured while the check ran.

Command output from the check

The full command output behind this check.

Evidence from the check

Evidence file captured while the check ran.

Command output from the check

The full command output behind this check.

_{Ran code and verified through T-Rex}

`view()` copied the locked last-tick state and then loaded `ticks` separately, so a reader could legitimately observe `ticks: 1` beside the pre-tick `Never`. That is not a torn last-tick — the fields were already consistent — but it is still two states in one snapshot, and a reader should not have to know which pairings are legal. The counters and the last tick now live in one struct under one lock, so the whole view is a state some tick actually produced. The test asserts the invariant that makes true (`Never` only with `ticks: 0`) instead of tolerating the interleaving, and requires the reader to observe both alternating states so it cannot pass by never racing the writer. Mutation-checked: publishing the counters before the fields land fails it with exactly the reported `Never` beside `ticks: 1`. Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>

echobt · 2026-09-16T14:49:13Z

Fixed in 31a0380 — and by strengthening the invariant rather than loosening the assertion.

You are right that the interleaving was legal as written: view() copied the locked last-tick state and then loaded ticks separately, so ticks: 1 beside Never was reachable. Rather than accept the counter mismatch in the test, the counters now live in the same struct under the same lock, so the whole view is a state some tick actually produced and Never can only appear with ticks: 0. The test asserts that, and now also requires the reader to observe both alternating states so it cannot pass by never racing the writer.

Mutation-checked: publishing the counters before the fields land reproduces your exact case and fails the test with Never is only before the first tick.

echobt · 2026-09-16T14:57:41Z

@greptileai review

greptile-apps · 2026-09-16T15:06:26Z

+    let mut saw_unpaid = false;
+    // Keep reading while the writer is live *and* until both states have been
+    // observed, so the test cannot pass by never overlapping the writer.
+    while !(writer.is_finished() && saw_scored && saw_unpaid) {


Writer Completion Can Hang

If the writer completes before the reader observes a Scored state, its durable final snapshot is Unpaid. The loop still requires both states after completion, so saw_scored can never become true and this test spins forever rather than reporting a failure. That can hang local and CI test runs. Synchronize the overlap before allowing the writer to finish, or stop once it finishes and assert only states that could still be observed.

Artifacts

Evidence from the check

External temporary-Cargo reproduction source that joins the alternating writer before polling status, ending with a bounded proof of the persistent loop condition.

Command output from the check

Executed forced-schedule reproduction showing final Unpaid on attempts 1 and 1,000,000 while the supplied loop condition remains true, confirming nontermination.

Command output from the check

First real execution of the exact supplied test under a 12-second timeout; it passed in an ordinary overlapping schedule.

Command output from the check

Second real execution of the exact supplied test under the same 12-second timeout; it again passed in an ordinary overlapping schedule.

Command output from the check

Build record for the exact emit_fail_closed integration-test binary, which completed successfully before execution.

_{Ran code and verified through T-Rex}

The reader loop waited for the writer to finish *and* for both alternating states to be observed. If the writer completed first — leaving the durable final state `unpaid` — `saw_scored` could never become true, so the loop spun forever instead of failing. That is a hang in a test that guards a hang-free property, and it would have wedged local and CI runs. Termination no longer depends on which interleaving happens. The loop is bounded by a hard iteration cap and breaks when the writer finishes, and the deterministic assertions moved after the join: the last write is an odd index, so the final state is fixed and every field can be asserted outright rather than waited for. The concurrency check keeps its power — mutation-checked, a publish that lands the counters before the tick fields still fails it with `Never` beside a moved counter. Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>

echobt · 2026-09-16T15:10:44Z

Fixed in e7b1bc7 — that was a real hang, thank you.

You are right: if the writer completed first the durable final state was unpaid, so saw_scored could never become true and the loop spun forever instead of failing. A test guarding a hang-free property must not itself be able to hang.

Termination no longer depends on the interleaving. The loop is bounded by a hard iteration cap and breaks when the writer finishes; the deterministic assertions moved after the join, where the last write is an odd index so the final state is fixed (unpaid, ticks == WRITES, scored_epoch == 43) and every field can be asserted outright rather than waited for. Runtime is 0.17s.

The earlier comment about view() copying the last tick before loading ticks no longer applies: 31a0380 put the counters and the last tick under one lock, so Never is only observable with ticks: 0. Mutation-checked again after this change — publishing the counters before the tick fields still fails the test with Never is only before the first tick.

echobt · 2026-09-16T15:19:32Z

@greptileai review

greptile-apps Bot reviewed Sep 16, 2026

View reviewed changes

Navigation Menu

fix(bounty): never sign an unpaid epoch as scored - #303

fix(bounty): never sign an unpaid epoch as scored#303
echobt wants to merge 4 commits into
mainfrom
droid/bounty-rewards-cortex-api

echobt commented Sep 16, 2026

Uh oh!

echobt commented Sep 16, 2026

Uh oh!

greptile-apps Bot commented Sep 16, 2026 •

edited

Loading

Uh oh!

greptile-apps Bot Sep 16, 2026

Uh oh!

greptile-apps Bot Sep 16, 2026

Uh oh!

echobt commented Sep 16, 2026

Uh oh!

echobt commented Sep 16, 2026

Uh oh!

greptile-apps Bot Sep 16, 2026

Uh oh!

echobt commented Sep 16, 2026

Uh oh!

echobt commented Sep 16, 2026

Uh oh!

greptile-apps Bot Sep 16, 2026

Uh oh!

echobt commented Sep 16, 2026

Uh oh!

echobt commented Sep 16, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

echobt commented Sep 16, 2026

The bug this fixes

What changed

Test plan

Notes

Uh oh!

echobt commented Sep 16, 2026

Uh oh!

greptile-apps Bot commented Sep 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Greptile Summary

Confidence Score: 5/5

Uh oh!

greptile-apps Bot Sep 16, 2026

Choose a reason for hiding this comment

Uh oh!

greptile-apps Bot Sep 16, 2026

Choose a reason for hiding this comment

Uh oh!

echobt commented Sep 16, 2026

Uh oh!

echobt commented Sep 16, 2026

Uh oh!

greptile-apps Bot Sep 16, 2026

Choose a reason for hiding this comment

Uh oh!

echobt commented Sep 16, 2026

Uh oh!

echobt commented Sep 16, 2026

Uh oh!

greptile-apps Bot Sep 16, 2026

Choose a reason for hiding this comment

Uh oh!

echobt commented Sep 16, 2026

Uh oh!

echobt commented Sep 16, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

greptile-apps Bot commented Sep 16, 2026 •

edited

Loading