Skip to content

feat(gax): o11y tracer helper updates - #9269

Open
shivanee-p wants to merge 14 commits into
mainfrom
shivaneep-o11y-tracer-helper-updates
Open

feat(gax): o11y tracer helper updates#9269
shivanee-p wants to merge 14 commits into
mainfrom
shivaneep-o11y-tracer-helper-updates

Conversation

@shivanee-p

@shivanee-p shivanee-p commented Sep 8, 2026

Copy link
Copy Markdown
Contributor

Updates traceAttempt in TracerHelper.ts to manage span lifecycles across either Promises or streaming RPCs without closing spans prematurely.

  • Add overloads of traceAttempt() function call to account for Promises or Streams returned from GAX
  • Add helper functions to close spans if they are Promises or Streams
  • Make sure spans only close once

Note: confirmed that the google-gax test suite passes with npm run test-application

@shivanee-p
shivanee-p changed the base branch from main to shivaneep-o11y-method-name September 8, 2026 23:33

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request updates constructSettings to include internalMethodName in otherArgs when telemetry is enabled, and refactors traceAttempt in TracerHelper.ts to be synchronous. The refactored traceAttempt now dynamically manages OpenTelemetry span lifecycles based on whether the traced operation returns a Promise, an EventEmitter (stream), or a synchronous value. Comprehensive unit tests have been added to verify these behaviors. There are no review comments, so I have no feedback to provide.

@shivanee-p shivanee-p changed the title Shivaneep o11y tracer helper updates feat(gax): o11y tracer helper updates Sep 8, 2026
@shivanee-p
shivanee-p force-pushed the shivaneep-o11y-tracer-helper-updates branch from 5c87d1a to a32c4c4 Compare September 8, 2026 23:45
@shivanee-p
shivanee-p force-pushed the shivaneep-o11y-tracer-helper-updates branch 5 times, most recently from a4484e3 to f1ce37e Compare September 9, 2026 00:11
An error occurred while trying to automatically change base from shivaneep-o11y-method-name to main September 9, 2026 14:08
@shivanee-p
shivanee-p removed this pull request from stack #9270 September 9, 2026 15:29
@shivanee-p
shivanee-p changed the base branch from shivaneep-o11y-method-name to main September 9, 2026 15:29
@shivanee-p
shivanee-p added this pull request to stack #9275 September 9, 2026 15:29
@shivanee-p
shivanee-p force-pushed the shivaneep-o11y-tracer-helper-updates branch from 8c7ca7f to 739630a Compare September 9, 2026 15:34
@shivanee-p

Copy link
Copy Markdown
Contributor Author

/gemini review

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request refactors traceAttempt in TracerHelper.ts to support asynchronous span lifecycle management for both promise-based and stream-based operations, introducing helper functions handlePromise and handleStream along with comprehensive unit tests. The review feedback highlights two important improvements: first, to avoid deferring span completion to a microtask for synchronous operations, we should check if the result is a thenable and end the span synchronously if it is not; second, to prevent potential memory leaks, we should clean up the event listeners registered on the stream in handleStream once the span has ended.

Comment thread core/packages/gax/src/observability/TracerHelper.ts
Comment thread core/packages/gax/src/observability/TracerHelper.ts
@shivanee-p
shivanee-p force-pushed the shivaneep-o11y-tracer-helper-updates branch from 060ce94 to c717da1 Compare September 9, 2026 18:34
@shivanee-p
shivanee-p marked this pull request as ready for review September 9, 2026 18:34
@shivanee-p
shivanee-p requested a review from a team as a code owner September 9, 2026 18:35
@github-actions
github-actions Bot requested a review from bshaffer September 9, 2026 18:35
@shivanee-p

Copy link
Copy Markdown
Contributor Author

/gemini review

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces asynchronous span lifecycle management for both Promise-based and Stream-based operations in TracerHelper.ts, along with comprehensive unit tests. However, a critical issue was identified where spans are leaked (never ended) if the traced function returns a synchronous value or an unsupported type. To resolve this, an else block should be added to immediately end the span for synchronous or unsupported return types.

Comment thread core/packages/gax/src/observability/TracerHelper.ts
@shivanee-p
shivanee-p force-pushed the shivaneep-o11y-tracer-helper-updates branch 2 times, most recently from dabd501 to ae2d6f6 Compare September 10, 2026 01:46

@danieljbruce danieljbruce left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left a couple comments. Overall, I think I need to understand the problem and architecture more broadly to understand how traceAttempt and traceHelper.ts are used.

const cleanup = () => {
stream.removeListener('error', onError);
stream.removeListener('end', onEnd);
stream.removeListener('close', onClose);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know too much about how this code fits in with the broader architecture, but one thought I had is if we remove the listeners here which I see intends to remove the listeners applied on the stream.on('error', onError) lines of code then it is going to remove other listeners as well that we need?

I remember google-gax applies a lot of error listeners in its middleware to handle retries and other such tasks.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I double checked, and I confirmed taht removeListener() removes only the reference for that specific callback that's used. In contrast, removeAllListeners() cleans up all of the listeners and would affect the middleware listeners.

@@ -163,5 +172,572 @@ describe('TracerHelper', () => {
assert.strictEqual(spans.length, 1);
assert.strictEqual(spans[0].attributes['gcp.method.type'], 'http');
});

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could it be in scope to test retries here? I know they get complex with the streaming calls.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point, thank you! I've added tests for retries in streams

@shivanee-p
shivanee-p force-pushed the shivaneep-o11y-tracer-helper-updates branch from d66cee1 to f2218ab Compare September 10, 2026 18:37
@shivanee-p
shivanee-p removed this pull request from stack #9275 September 10, 2026 18:41
@shivanee-p
shivanee-p added this pull request to stack #9282 September 10, 2026 18:42

@danieljbruce danieljbruce left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok. The code LGTM. I see how the traceAttempt is used as a wrapper with invokeCall. Maybe consider inlining traceAttempt into createApiCall and then refactoring pieces of it out to eliminate the need to pass a fn: () => T parameter, but that shouldn't block. Maybe just consider it as a backlog item.

The broader thing that might just be good to check is to run the google-gax tests. After the monorepo migration, I don't think they run in the CI pipeline anymore. I'll send a link on chat for how to run the gax tests. Probably would be good to run the gax tests before merging.

@shivanee-p
shivanee-p force-pushed the shivaneep-o11y-tracer-helper-updates branch from f2218ab to 8b2e8a9 Compare September 11, 2026 18:08
@shivanee-p
shivanee-p requested review from quirogas and removed request for bshaffer September 11, 2026 20:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants