Add verify_loop.py: check/adjust convergence loop for scene placement - #217
Open
wilsonchenghy wants to merge 3 commits into
Open
Add verify_loop.py: check/adjust convergence loop for scene placement#217wilsonchenghy wants to merge 3 commits into
wilsonchenghy wants to merge 3 commits into
Conversation
Builds on set_pose actually surviving a reset now (#216) — before that fix, a correction loop had nothing that would actually stick between iterations. Deliberately NOT a new daemon command: the daemon stays atomic (one action/state-read per call), the loop lives entirely on the host as a small library composed out of existing commands (bbox/overlap/ query/set_pose). Same split as a Kubernetes controller's reconciliation loop against a plain CRUD API, or a WebDriver test framework's wait/retry helpers against a plain click/screenshot RPC — keep the execution layer dumb, keep convergence logic where it can be arbitrary Python instead of a wire-protocol DSL. Provides `until(check_fn, adjust_fn, ...)`, prebuilt checks (is_resting_on, no_overlap, is_within_bounds) on top of bbox/overlap, and a damped close_gap_along_axis adjustment (0.8x per step, not a full 1:1 correction, since a scaled prefab asset's visual bbox and actual physics-contact height can diverge — see SKILL.md). Verified against a live daemon, not just read for correctness: a first version's adjust_fn read the "current position" via query(), which oscillated instead of converging for a kinematic object (query() never reflects a kinematic object's set_pose USD-prim write — the exact gotcha SKILL.md already documents for a different command). Fixed by reading the current position from the bbox already computed in the check step instead. Retested: converges cleanly in 4 iterations (gap 0.125 -> 0.025 -> 0.005 -> 0.001) for both a kinematic and a dynamic test object. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01J2QJBPpoZGwXPvdN6f4f6X
Real dogfooding of verify_loop found a genuine gap: is_resting_on/ close_gap_along_axis (z-gap only) converged cleanly, but an object placed with only the z-gap corrected drifted and toppled under continued physics stepping — nothing was correcting x/y centering, so a few degrees of landing tilt compounded into sliding instead of settling. Adds is_stacked_on (checks z-gap AND x/y-centering together) and settle_stack (corrects both in one combined move per iteration) plus a `stack` CLI subcommand. Tried this on a 3-box YCB stack first (cracker/sugar/gelatin box) — it still weren't stable even with x/y correction, because the actual remaining cause was the SUPPORTING box's own residual tilt (~13 degrees), which gives gravity a persistent sideways component regardless of how precisely the object on top is centered. verify_loop only corrects the placed object's translation, not the support's rotation — a real physical limitation, not a loop bug. Kept as the reference dogfood example instead: tools/isaac_harness/scenes/bin_on_table_precise.sh — a KLT bin precisely seated on the packing table via `verify_loop stack`. Verified stable, not just convergent: positions were bit-identical across 300 additional physics steps after the loop reported success. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01J2QJBPpoZGwXPvdN6f4f6X
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01J2QJBPpoZGwXPvdN6f4f6X
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds
tools/isaac_harness/verify_loop.py— the self-loop primitive scoped in issue #209, item 2. Builds onset_posegenuinely surviving a reset now (#216); before that fix, a correction loop had nothing durable to converge toward.Deliberately not a new daemon command. The daemon (
isaac_session_daemon.py) stays atomic — one action or state-read per JSON command, unchanged. The actual "check, correct, recheck" loop lives entirely on the host, composed out of existing commands (bbox/overlap/query/set_pose). Same architectural split as a Kubernetes controller's reconciliation loop against a plain CRUD API, or a WebDriver test framework's wait/retry helpers against a plain click/screenshot RPC.Provides:
until(check_fn, adjust_fn, max_iters, timeout_s)— the generic loop, returns success + full per-iteration trace, not just a final verdict.is_resting_on,no_overlap,is_within_bounds.close_gap_along_axisadjustment (0.8x per step, not a full 1:1 correction) — a scaled prefab asset's visual bbox and actual physics-contact height can diverge (see SKILL.md), so a full-gap jump risks overshoot.verify_loop.py rest --name X --on Y) for quick manual checks.Test plan
query(), which oscillated instead of converging for a kinematic object (query()never reflects a kinematic object'sset_poseUSD-prim write — the same gotcha already documented forset_poseitself, just newly hit here too). Fixed by reading position from the bbox already computed in the check step.0.125 → 0.025 → 0.005 → 0.001) for both a kinematic and a dynamic test object.autopep8 --max-line-length 150clean, no diff.--helpwork without a running daemon.🤖 Generated with Claude Code
https://claude.ai/code/session_01J2QJBPpoZGwXPvdN6f4f6X