Fix a test that had been failing for two weeks, and run the suite in CI - #27
Merged
Merged
Conversation
Adding the docs job yesterday was the first time anything in this repo ran a test. Pointing it at the whole suite turned up four failures on a clean checkout, and they were not the same kind of thing. Three were tests/test_span_tree.py and tests/test_integrations.py importing agentx.integrations.google_adk at call time. That module raises ImportError by design when google-adk is absent, so the tests failed instead of skipping - while their crewai, litellm, llamaindex and autogen neighbours in the same files guard the same situation with pytest.importorskip. They now do too. With the extra installed all three run and pass, so this skips what is genuinely not installed rather than hiding anything. The fourth was real. test_trace_tool_call_emits_real_child_span asserted the root trace carries no flat tool_calls list when a tool call became a real child span. That was true when the test was written on 2026-08-07; four days later 40c6f6e deliberately added the dual-write, because the engine's built-in "Tool failure" check and the dashboard's Tool quality column read the root's flat list and a failed trace_tool_call() was invisible to both. The behaviour is right and documented in the code; the assertion was stale, and had been failing ever since - unnoticed, because no CI ran it. So the assertion is corrected rather than the code, and corrected upward: it now pins the dual-write positively and pins it at exactly one entry. Removing the dual-write fails it, and writing a third copy fails it too - neither of which the old assertion could catch, since it only checked for absence. docs.yml becomes tests.yml and runs the whole suite; the old name stopped being true once it covered more than the docs. Optional-extra tests skip themselves, so the base install is green: 73 passed, 28 skipped without extras, 76 passed, 25 skipped with google-adk. Installing the extras to actually run those is a fair next step and a slower job, not a different one. Verified on both paths with the exact CI command, and the corrected assertion verified to fail against a tree with the dual-write removed and against one that writes three times. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WYi6akpDa8qfiojRed2cAE
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up to #26. Adding that docs job was the first time anything in this repo ran a test —
release.ymlonly publishes. Pointing it at the whole suite turned up four failures on a clean checkout, and they weren't the same kind of thing.Three were missing optional extras
test_span_tree.pyandtest_integrations.pyimportagentx.integrations.google_adkat call time. That module raisesImportErrorby design when google-adk is absent, so the tests failed instead of skipping — while their crewai, litellm, llamaindex and autogen neighbours in the same files guard the identical situation withpytest.importorskip.They now do too. With the extra installed all three run and pass, so this skips what genuinely isn't installed rather than hiding anything.
The fourth was real — and stale by two weeks
test_trace_tool_call_emits_real_child_spanasserted the root trace carries no flattool_callslist when a tool call became a real child span:That was true when the test was written on 2026-08-07. Four days later,
40c6f6edeliberately added the dual-write — the code says why:The behaviour is right; the assertion was stale. It had been failing since 2026-08-11 — unnoticed, because no CI ran it. "Fixing" the code to satisfy it would have re-broken the Tool failure check and the Tool quality column.
So the assertion is corrected, and corrected upward:
It now pins the dual-write positively and pins it at exactly one entry. Verified against a modified tree: removing the dual-write fails it, and writing a third copy fails it too — neither of which the old assertion could catch, since it only checked for absence.
Widening the CI
docs.ymlbecomestests.ymland runs the whole suite. The old name stopped being true once it covered more than the docs..[langchain])Optional-extra tests skip themselves, so the base install is green — a skip is honest about not being covered, where a failure would have drowned out real ones. #28 takes the next step and actually runs them.
test_integration.pytalks to the hosted API and skips itself without a key; no secret is passed on purpose, so fork PRs behave the same.Verification
tests.ymlvalidated as YAMLTests and CI only; no library code changed, so no version bump is warranted by the diff.
🤖 Generated with Claude Code
https://claude.ai/code/session_01WYi6akpDa8qfiojRed2cAE