async inserts article refresh - #192
Conversation
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: b1759afef5
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| CREATE USER mv_sync_definer | ||
| IDENTIFIED WITH no_password | ||
| SETTINGS PROFILE mv_sync_insert; |
There was a problem hiding this comment.
When the ClickHouse HTTP or native interface is reachable by untrusted clients, IDENTIFIED WITH no_password creates a credential-free login that receives SELECT on the source and INSERT on the destination. The security warning does not mitigate that exposure; make this an internal-only identity with HOST NONE, or require strong authentication and an explicit host restriction.
AGENTS.md reference: AGENTS.md:L122-L128
Useful? React with 👍 / 👎.
| WHERE name = 'insert_deduplication_version'; | ||
| ``` | ||
|
|
||
| Do not move directly from `old_separate_hashes` to `new_unified_hash`. First run `compatible_double_hashes` on every replica. For Replicated* tables, keep it for at least the largest effective `replicated_deduplication_window_seconds` (one hour by default). For non-replicated tables, run enough inserts to cover the count-based `non_replicated_deduplication_window`. Then change to `new_unified_hash`. This period lets ClickHouse record unified IDs for data that a client might still retry. |
There was a problem hiding this comment.
For installations that have not overridden replicated_deduplication_window_seconds, the documented default is 604800 seconds (seven days), not one hour. Following this parenthetical can therefore move replicas to new_unified_hash almost a week too early and allow retries of older inserts to bypass deduplication; state seven days and still direct operators to verify every table's effective setting.
AGENTS.md reference: AGENTS.md:L53-L55
Useful? React with 👍 / 👎.
| Set `async_insert_use_adaptive_busy_timeout = 0` to use a fixed busy timeout. In this mode, ClickHouse uses `async_insert_busy_timeout_max_ms`. | ||
|
|
||
| `async_insert_busy_timeout_ms` is an alias for `async_insert_busy_timeout_max_ms`, not a separate setting. You can use either name, but use `async_insert_busy_timeout_max_ms` in new | ||
| configurations. |
There was a problem hiding this comment.
On pre-24.2 releases, async_insert_busy_timeout_max_ms does not exist and async_insert_busy_timeout_ms is the actual timeout setting rather than its alias, so the shown fixed-mode recipe fails despite this article explicitly discussing earlier releases. Mark this section as 24.2+ and retain async_insert_busy_timeout_ms as the fallback for older versions.
AGENTS.md reference: AGENTS.md:L108-L111
Useful? React with 👍 / 👎.
| Async INSERTs let ClickHouse® batch many small inserts in server memory before writing them to storage. Use them when clients cannot form sufficiently large batches themselves. Client-side batching remains preferable when you control the writers because it keeps buffering outside ClickHouse. | ||
|
|
||
| **Key Documentation:** [Official Async Inserts Documentation](https://clickhouse.com/docs/en/optimize/asynchronous-inserts) | ||
| Starting with the 26.2.4 release line, ClickHouse [enables async inserts by default](https://github.com/ClickHouse/ClickHouse/pull/97590). Earlier releases default to synchronous inserts. Set it explicitly in an ingestion user's profile when you need stable behavior across versions: |
There was a problem hiding this comment.
For ClickHouse Cloud deployments, async inserts were already enabled by default before 26.2.4, so the unconditional statement that all earlier releases default to synchronous inserts gives Cloud operators an incorrect historical boundary. Scope the transition to the upstream/self-managed server default and note the earlier Cloud override.
AGENTS.md reference: AGENTS.md:L100-L108
Useful? React with 👍 / 👎.
1 participant
Refresh Async INSERTs knowledge base article
Updates the Async INSERTs article for current ClickHouse behavior and operational guidance.
Changes
I have read the CLA Document and I hereby sign the CLA