-
Notifications
You must be signed in to change notification settings - Fork 809
Transactional storage access #2124
Copy link
Copy link
Open
Labels
enhancementNew feature or request.New feature or request.t-toolingIssues with this label are in the ownership of the tooling team.Issues with this label are in the ownership of the tooling team.
Milestone
Description
Activity
Metadata
Metadata
Assignees
Labels
enhancementNew feature or request.New feature or request.t-toolingIssues with this label are in the ownership of the tooling team.Issues with this label are in the ownership of the tooling team.
Port transactional storage access to Crawlee for Python: a request handler's storage writes should be buffered and committed atomically, so a handler that fails or gets retried leaves no partial writes behind. It should be enabled by default in every crawler and implemented at the storage level, so direct
Dataset/KeyValueStore/RequestQueuecalls are covered too, not just the crawling context helpers - which also absorbs the adaptive crawler's bespoke buffering.Follow the JS implementation:
✍️ Drafted by Claude Code