Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
42 changes: 42 additions & 0 deletions core/packages/gax/client-libraries.md
Original file line number Diff line number Diff line change
Expand Up @@ -149,6 +149,48 @@ in the second parameter:
const [response] = await client.sampleMethod(request, options);
```

### Resumable uploads

Some APIs expose methods that upload large payloads through the resumable
upload protocol. For these methods, the client method no
longer returns the response directly; it returns a
[`ResumableUploadSession`](https://googleapis.dev/nodejs/google-gax/latest/classes/ResumableUploadSession.html)
helper. Call `start()` with a `ResumableSource` (see `getResumableSource()`
below) and await `finished()` for the final response:

```ts
const helper = await client.createResumableUpload(request);
await helper.start({
uploadSource: client.getResumableSource(filePath),
chunkSize: 8 * 1024 * 1024, // 8MB chunks
onProgress: status => {
console.log(`Committed ${status.bytesUploaded} bytes to ${status.uploadUrl}`);
},
});
Comment on lines +163 to +169

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Please do not manually edit auto-generated markdown files to make changes, as these edits will be overwritten during the next regeneration. Instead, apply the fix upstream in the generator or templates so that the example correctly uses uploadSource (of type ResumableSource) rather than uploadStream to avoid runtime errors.

References
  1. Do not manually edit auto-generated files to fix typos or make other changes, as these edits will be overwritten during the next regeneration. Instead, apply the fixes upstream in the generator or templates.

const response = await helper.finished();
```

The session URL is available as `helper.uploadUrl` once the upload has
started. Save it if you need to resume the upload later — for example after a
process crash or network drop. To resume, pass the saved URL to `start()` on a
new helper, along with a new source for the same payload:

```ts
const helper = await client.createResumableUpload();
await helper.start({
uploadSource: client.getResumableSource(filePath),
resumeUrl: savedUploadUrl,
});
Comment on lines +180 to +183

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Please do not manually edit auto-generated markdown files to make changes, as these edits will be overwritten during the next regeneration. Instead, apply the fix upstream in the generator or templates so that the example correctly uses uploadSource (of type ResumableSource) rather than uploadStream to avoid runtime errors.

References
  1. Do not manually edit auto-generated files to fix typos or make other changes, as these edits will be overwritten during the next regeneration. Instead, apply the fixes upstream in the generator or templates.

const response = await helper.finished();
```

The current implementation requires a seekable stream (for example, a file
stream). Errors fall into three categories: transient errors (retried with
exponential backoff), state mismatches (recovered by querying the server for
the committed byte offset), and fatal errors (propagated to the caller).
The whole session is bounded by a global deadline (10 minutes by default,
scaled up for large payloads and overridable via `globalDeadlineMs`).

### Long-running operations

Some methods are expected to run longer. They return an object of type
Expand Down
2 changes: 2 additions & 0 deletions core/packages/gax/src/clientInterface.ts
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@ import {
PageDescriptor,
StreamDescriptor,
} from './descriptor';
import {ResumableUploadDescriptor} from './resumableUpload';
import * as longrunning from './longRunningCalls/longrunning';
import * as operationProtos from '../protos/operations';

Expand All @@ -51,6 +52,7 @@ export interface Descriptors {
stream: {[name: string]: StreamDescriptor};
longrunning: {[name: string]: LongrunningDescriptor};
batching?: {[name: string]: BundleDescriptor};
resumableUpload?: {[name: string]: ResumableUploadDescriptor};
}

export interface Callback<
Expand Down
1 change: 1 addition & 0 deletions core/packages/gax/src/descriptor.ts
Original file line number Diff line number Diff line change
Expand Up @@ -31,3 +31,4 @@ export {LongRunningDescriptor as LongrunningDescriptor} from './longRunningCalls
export {PageDescriptor} from './paginationCalls/pageDescriptor';
export {StreamDescriptor} from './streamingCalls/streamDescriptor';
export {BundleDescriptor} from './bundlingCalls/bundleDescriptor';
export {ResumableUploadDescriptor} from './resumableUpload';
12 changes: 12 additions & 0 deletions core/packages/gax/src/fallback.ts
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,18 @@ export {
PageDescriptor,
StreamDescriptor,
} from './descriptor';
export {
ResumableUploadDescriptor,
ResumableUploadSession,
ResumableUploadState,
resumableUploadStub,
} from './resumableUpload';
export type {
ResumableUploadContext,
ResumableUploadProgress,
ResumableUploadStartParams,
ResumableSource,
} from './resumableUpload';

export {StreamType} from './streamingCalls/streaming';

Expand Down
14 changes: 14 additions & 0 deletions core/packages/gax/src/gax.ts
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@
*/

import type {Message} from 'protobufjs';
import type {ResumableUploadContext} from './resumableUpload';
import {warn} from './warnings';
import {GoogleError} from './googleError';
import {BundleOptions} from './bundlingCalls/bundleExecutor';
Expand Down Expand Up @@ -172,6 +173,11 @@ export interface CallOptions {
apiName?: string;
retryRequestOptions?: RetryRequestOptions;
enableTelemetryTracing?: boolean;
/**
* Internal context used by resumable upload methods. Populated by
* GAPIC-generated client libraries; do not set manually.
*/
resumableUpload?: ResumableUploadContext;
}

export class CallSettings {
Expand All @@ -189,6 +195,7 @@ export class CallSettings {
apiName?: string;
retryRequestOptions?: RetryRequestOptions;
enableTelemetryTracing?: boolean;
resumableUpload?: ResumableUploadContext;

/**
* @param {Object} settings - An object containing parameters of this settings.
Expand Down Expand Up @@ -223,6 +230,8 @@ export class CallSettings {
this.apiName = settings.apiName ?? undefined;
this.retryRequestOptions = settings.retryRequestOptions;
this.enableTelemetryTracing = settings.enableTelemetryTracing;
this.resumableUpload =
'resumableUpload' in settings ? settings.resumableUpload : undefined;
}

/**
Expand All @@ -247,6 +256,7 @@ export class CallSettings {
let apiName = this.apiName;
let retryRequestOptions = this.retryRequestOptions;
let enableTelemetryTracing = this.enableTelemetryTracing;
let resumableUpload = this.resumableUpload;

// If the user provides a timeout to the method, that timeout value will be used
// to override the backoff settings.
Expand Down Expand Up @@ -305,6 +315,9 @@ export class CallSettings {
if ('enableTelemetryTracing' in options) {
enableTelemetryTracing = options.enableTelemetryTracing;
}
if ('resumableUpload' in options) {
resumableUpload = options.resumableUpload;
}

return new CallSettings({
timeout,
Expand All @@ -318,6 +331,7 @@ export class CallSettings {
apiName,
retryRequestOptions,
enableTelemetryTracing,
resumableUpload,
});
}
}
Expand Down
13 changes: 13 additions & 0 deletions core/packages/gax/src/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,19 @@ export {
PageDescriptor,
StreamDescriptor,
} from './descriptor';
export {ResumableUploadDescriptor} from './resumableUpload';
export {
ResumableUploadSession,
resumableUploadStub,
ResumableUploadState,
} from './resumableUpload';
export type {
ResumableUploadContext,
ResumableUploadProgress,
ResumableUploadStartParams,
ResumableSource,
} from './resumableUpload';
export {resumableSourceFromFile} from './resumableSourceFromFile';
export {
CallOptions,
CallSettings,
Expand Down
42 changes: 42 additions & 0 deletions core/packages/gax/src/resumableSourceFromFile.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
/**
* Copyright 2026 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

import {createReadStream, statSync} from 'fs';

import {ResumableSource} from './resumableUpload';

/**
* Creates a {@link ResumableSource} backed by a local file.
*
* This factory lives outside the fallback transport entrypoint so browser
* builds do not pull in the Node.js `fs` module. Generated clients should
* delegate to this function rather than constructing file streams directly.
*/
export function resumableSourceFromFile(filePath: string): ResumableSource {
const stat = statSync(filePath);
return {
size: stat.size,
getStream: (offset?: number) => {
const start = offset ?? 0;
if (start < 0 || start > stat.size) {
throw new RangeError(
`Invalid start offset ${start} for file of size ${stat.size}.`,
);
}
return createReadStream(filePath, {start});
},
};
}
Loading
Loading