fix(k8s): guard np_trace_flush when tracing SDK is not loaded - #252
Merged
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Deployments were printing this at the exact moment they succeeded:
The deployment-ready branch in
wait_deployment_activeis gated oncommand -v np_scope_progress. That guard is always true:np_scope_progressis one of thenp_scope_*wrappers thatk8s/loggingdefines unconditionally, and each wrapper does its owncommand -v np_trace_* || return 0internally. So the wrapper existing tells you nothing about whether the tracing SDK (vendor/catalog-tracing-sh/nptrace.sh) was actually sourced — that only happens whenNP_API_KEYandNP_TRACEare both set and the vendored file is present.np_trace_flushcomes straight from the SDK, so with tracing off the block ran to completion and then hit an undefined command.wait_deployment_activehas noset -e, so it was non-fatal noise rather than a failed deployment — but it read as an error in the operator-facing log right where the deployment reported success.The fix is the guard already used at
k8s/apply_templates:100.Audit of the other callsites
Checked every
np_trace_*reference outside the SDK; no other unguarded one exists:k8s/logging:142,153(np_scope_step_end,np_scope_step_timeout) — both return early unless_NP_SCOPES_SUBSTEPis set, which onlynp_scope_step_beginsets, and that one guards onnp_trace_adopt.k8s/logging:167(np_scope_wait_heartbeat) — returns early unless_np_scopes_nodesucceeds, which guards onnp_trace_adopt.k8s/logging:335(_np_scopes_on_exit) — runs from anEXITtrap registered only inside the branch that sourced the SDK.scheduled_task/logging— same four, same structure.k8s/diagnose/utils/diagnose_utils—_np_trace_check_resultis a local function, unrelated.scheduled_taskinherits the k8s deployment steps, so it picks this up too.Test plan
Added a regression test that reproduces the production shape: the
np_scope_*wrappers are defined (as they are onceload loggingruns) whilenp_trace_flushis not.The pre-existing suite never caught this because it never defined
np_scope_progressat all, socommand -vwas false and the whole block was skipped — the tests went green without ever reaching the line.Confirmed the new test fails on the unfixed script and passes after:
Changelog
Cut as
1.16.3dated 2026-09-08.1.16.2was left untouched.