perf(spanner): bypass stream pipeline for single-chunk query results - #9279
perf(spanner): bypass stream pipeline for single-chunk query results#9279olavloite wants to merge 1 commit into
Conversation
There was a problem hiding this comment.
Code Review
This pull request introduces a fast-path optimization (_runFast and decodeRowsDirect) for Spanner queries that return results in a single chunk, bypassing the streaming pipeline to improve performance. The review feedback identifies several critical issues: missing definitions and imports for originalDecode, originalConvertFieldsToJson, and GrpcService in partial-result-stream.ts that will cause runtime ReferenceErrors; an issue in Snapshot._runFast where the sequence number is incorrectly incremented during retries; and a misleading error message suggesting row.toJSON() when jsonMode is enabled.
d1b7acb to
661c9d6
Compare
|
/gemini review |
There was a problem hiding this comment.
Code Review
This pull request introduces an optimized fast-path streaming model for executing queries in the Spanner client. When a query returns all results in a single chunk, it bypasses the standard streaming pipeline overhead and decodes rows directly. The changes also include fallback mechanisms to the standard multi-chunk stream when necessary, along with comprehensive unit tests. The review feedback correctly identifies two critical issues in transaction.ts: a missing import for PassThrough which would cause a runtime ReferenceError, and a TypeScript compilation error caused by an incorrect type annotation : traceConfig.
Historically, database.run and transaction.run route all queries through a multi-stage Transform stream pipeline (Readable -> CheckpointStream -> PartialResultStream -> Transform). For small, single-chunk queries, stream state buffering, event emitter dispatch, and microtask scheduling introduce significant CPU and latency overhead. This change introduces an internal fast path for single-chunk queries: - For Database.run (with multiplexed sessions) and Snapshot.run, executes the gRPC request directly without wrapping it in a Transform pipeline. - If the query completes in a single response chunk (chunk.last is true and chunkedValue is unset), rows are decoded directly in a tight synchronous loop (decodeRowsDirect) into pre-allocated row arrays or plain JSON objects. - If the result spans multiple chunks, it seamlessly falls back to the full partialResultStream pipeline by replaying the first chunk through a pass-through stream with zero data loss or token mismatch. - PartialResultStream also utilizes decodeRowsDirect when streaming queries arrive in a single chunk. - Preserves full OpenTelemetry trace span hierarchy and Cloud Spanner transaction retry semantics.
661c9d6 to
ce31c2f
Compare
|
/gemini review |
There was a problem hiding this comment.
Code Review
This pull request introduces an optimized fast-path streaming model for Spanner queries that return results in a single PartialResultSet, bypassing standard stream pipeline overhead by decoding rows directly. It updates Database, Snapshot, and PartialResultStream to implement this fast-path, with seamless fallback to standard multi-chunk streaming when necessary. Review feedback suggests critical improvements to the retry and error handling logic: adding a .catch() handler to the unawaited this.begin() call to prevent unhandled promise rejections, and introducing exponential backoff instead of immediate retries via setImmediate to avoid rapidly exhausting retry attempts during transient outages.
Historically,
database.runandtransaction.runroute all queries through a multi-stage Transform stream pipeline (Readable->CheckpointStream->PartialResultStream->Transform). For small, single-chunk queries, stream state buffering, event emitter dispatch, and microtask scheduling introduce significant CPU and latency overhead.This change introduces an internal direct path for single-chunk queries:
Database.run(with multiplexed sessions),Snapshot.run, andTransaction.runexecute the gRPC call directly without creating an intermediate Transform stream pipeline upfront.chunk.lastis true andchunkedValueis unset), rows are decoded directly in a tight synchronous loop (decodeRowsDirect) into pre-allocated row arrays or plain JSON objects via a unifiedformatRowhelper.partialResultStreampipeline by replaying the first chunk through a pass-through stream with zero data loss or token mismatch.PartialResultStreamalso utilizesdecodeRowsDirectvia_addSingleChunkwhen streaming queries arrive in a single chunk._runLegacy) when multiplexed sessions are disabled onDatabase.runor whenrunStreamhas been overridden.begintransaction handling for read-write transactions.Benchmark Results: Steady Load Point-Select (Head-to-Head vs
main)Concurrently executed on Google Cloud Spanner benchmarks for 15 minutes under identical conditions (100 TPS, 100 threads, 4 vCPUs, sidecar enabled in europe-north1-a):
mainResults exclude the 1-minute warmup period (~716k operations sampled).