Skip to content

perf(spanner): bypass stream pipeline for single-chunk query results - #9279

Open
olavloite wants to merge 1 commit into
mainfrom
spanner-bypass-stream-for-single-chunk-queries
Open

perf(spanner): bypass stream pipeline for single-chunk query results#9279
olavloite wants to merge 1 commit into
mainfrom
spanner-bypass-stream-for-single-chunk-queries

Conversation

@olavloite

@olavloite olavloite commented Sep 10, 2026

Copy link
Copy Markdown
Contributor

Historically, database.run and transaction.run route all queries through a multi-stage Transform stream pipeline (Readable -> CheckpointStream -> PartialResultStream -> Transform). For small, single-chunk queries, stream state buffering, event emitter dispatch, and microtask scheduling introduce significant CPU and latency overhead.

This change introduces an internal direct path for single-chunk queries:

  • Direct execution: Database.run (with multiplexed sessions), Snapshot.run, and Transaction.run execute the gRPC call directly without creating an intermediate Transform stream pipeline upfront.
  • Single-chunk fast path: If the query completes in a single response chunk (chunk.last is true and chunkedValue is unset), rows are decoded directly in a tight synchronous loop (decodeRowsDirect) into pre-allocated row arrays or plain JSON objects via a unified formatRow helper.
  • Multi-chunk fallback: If the result spans multiple chunks, it seamlessly falls back to the full partialResultStream pipeline by replaying the first chunk through a pass-through stream with zero data loss or token mismatch.
  • Single-chunk stream optimization: PartialResultStream also utilizes decodeRowsDirect via _addSingleChunk when streaming queries arrive in a single chunk.
  • Compatibility & safety: Falls back to the legacy streaming pipeline (_runLegacy) when multiplexed sessions are disabled on Database.run or when runStream has been overridden.
  • Observability & Transactions: Preserves full OpenTelemetry trace span hierarchy, transaction retry semantics, and inline begin transaction handling for read-write transactions.

Benchmark Results: Steady Load Point-Select (Head-to-Head vs main)

Concurrently executed on Google Cloud Spanner benchmarks for 15 minutes under identical conditions (100 TPS, 100 threads, 4 vCPUs, sidecar enabled in europe-north1-a):

Metric main PR #9279 Delta
Mean Latency 4.944 ms 4.113 ms -16.80% (-0.831 ms)
P50 Latency 4.535 ms 3.778 ms -16.69% (-0.757 ms)
P90 Latency 5.917 ms 4.723 ms -20.18% (-1.194 ms)
P99 Latency 9.712 ms 6.652 ms -31.51% (-3.060 ms)
Total Ops 715,553 716,961 +0.20%

Results exclude the 1-minute warmup period (~716k operations sampled).

@product-auto-label product-auto-label Bot added the api: spanner Issues related to the Spanner API. label Sep 10, 2026

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a fast-path optimization (_runFast and decodeRowsDirect) for Spanner queries that return results in a single chunk, bypassing the streaming pipeline to improve performance. The review feedback identifies several critical issues: missing definitions and imports for originalDecode, originalConvertFieldsToJson, and GrpcService in partial-result-stream.ts that will cause runtime ReferenceErrors; an issue in Snapshot._runFast where the sequence number is incorrectly incremented during retries; and a misleading error message suggesting row.toJSON() when jsonMode is enabled.

Comment thread handwritten/spanner/src/partial-result-stream.ts Outdated
Comment thread handwritten/spanner/src/partial-result-stream.ts Outdated
Comment thread handwritten/spanner/src/transaction.ts Outdated
Comment thread handwritten/spanner/src/partial-result-stream.ts Outdated
@olavloite
olavloite force-pushed the spanner-bypass-stream-for-single-chunk-queries branch 4 times, most recently from d1b7acb to 661c9d6 Compare September 10, 2026 16:26
@olavloite

Copy link
Copy Markdown
Contributor Author

/gemini review

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces an optimized fast-path streaming model for executing queries in the Spanner client. When a query returns all results in a single chunk, it bypasses the standard streaming pipeline overhead and decodes rows directly. The changes also include fallback mechanisms to the standard multi-chunk stream when necessary, along with comprehensive unit tests. The review feedback correctly identifies two critical issues in transaction.ts: a missing import for PassThrough which would cause a runtime ReferenceError, and a TypeScript compilation error caused by an incorrect type annotation : traceConfig.

Comment thread handwritten/spanner/src/transaction.ts
Comment thread handwritten/spanner/src/transaction.ts Outdated
Historically, database.run and transaction.run route all queries
through a multi-stage Transform stream pipeline (Readable ->
CheckpointStream -> PartialResultStream -> Transform). For small,
single-chunk queries, stream state buffering, event emitter dispatch,
and microtask scheduling introduce significant CPU and latency overhead.

This change introduces an internal fast path for single-chunk queries:
- For Database.run (with multiplexed sessions) and Snapshot.run, executes
  the gRPC request directly without wrapping it in a Transform pipeline.
- If the query completes in a single response chunk (chunk.last is true
  and chunkedValue is unset), rows are decoded directly in a tight
  synchronous loop (decodeRowsDirect) into pre-allocated row arrays or
  plain JSON objects.
- If the result spans multiple chunks, it seamlessly falls back to the
  full partialResultStream pipeline by replaying the first chunk through
  a pass-through stream with zero data loss or token mismatch.
- PartialResultStream also utilizes decodeRowsDirect when streaming queries
  arrive in a single chunk.
- Preserves full OpenTelemetry trace span hierarchy and Cloud Spanner
  transaction retry semantics.
@olavloite
olavloite force-pushed the spanner-bypass-stream-for-single-chunk-queries branch from 661c9d6 to ce31c2f Compare September 10, 2026 16:41
@olavloite

Copy link
Copy Markdown
Contributor Author

/gemini review

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces an optimized fast-path streaming model for Spanner queries that return results in a single PartialResultSet, bypassing standard stream pipeline overhead by decoding rows directly. It updates Database, Snapshot, and PartialResultStream to implement this fast-path, with seamless fallback to standard multi-chunk streaming when necessary. Review feedback suggests critical improvements to the retry and error handling logic: adding a .catch() handler to the unawaited this.begin() call to prevent unhandled promise rejections, and introducing exponential backoff instead of immediate retries via setImmediate to avoid rapidly exhausting retry attempts during transient outages.

Comment thread handwritten/spanner/src/transaction.ts
Comment thread handwritten/spanner/src/transaction.ts
@olavloite
olavloite marked this pull request as ready for review September 10, 2026 16:55
@olavloite
olavloite requested a review from a team as a code owner September 10, 2026 16:55
@github-actions
github-actions Bot requested a review from shivanee-p September 10, 2026 16:56
@olavloite
olavloite removed the request for review from shivanee-p September 10, 2026 16:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

api: spanner Issues related to the Spanner API.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants