feat(dds_ingestion): tell the user when DDS cannot write to S3, in it… - #42
Merged
Conversation
…s words
Pairs with EEALakeHouse@46cc872, where the server learned to refuse a transfer
its storage self-test has already shown will fail: `503` with
`{"error": "storage_unavailable"}` and a message written for whoever is about to
read it — that DDS cannot write to the S3 behind the catalog, that this is a
dependency problem for an administrator rather than anything the caller did, and
that nothing has been uploaded.
Arriving here as a bare `IngestApiError`, that message came out wrapped as "DDS
ingest API error 503 on POST /api/v1/ingest/begin: …", which buries the
explanation behind plumbing and reads, in a notebook traceback, like the caller
broke something.
* `StorageUnavailableError` — a subclass, so existing `except IngestApiError`
handling still catches it — raised on DDS's own `storage_unavailable` slug and
nothing else. A 503 from a proxy or load balancer in front of the service has
no such body and stays a plain `IngestApiError`, rather than being mislabelled
as a storage fault.
* Its `str()` is the server's message verbatim; the class name carries what the
prefix used to. `status_code`, `message` and `where` stay available for code
that handles it programmatically.
* `_error_slug()` reads the machine-readable half of the `{error, message, path}`
body, which `_error_message()` was throwing away.
`FolderIngest.run()` therefore stops at `begin` with an error that explains
itself, having uploaded nothing — the point of the server-side pre-flight, and
what the new folder test asserts.
51 passed; mypy clean on the changed modules.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HFZRQnGnYw8mBCVsp9seds
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
…s words
Pairs with EEALakeHouse@46cc872, where the server learned to refuse a transfer its storage self-test has already shown will fail:
503with{"error": "storage_unavailable"}and a message written for whoever is about to read it — that DDS cannot write to the S3 behind the catalog, that this is a dependency problem for an administrator rather than anything the caller did, and that nothing has been uploaded.Arriving here as a bare
IngestApiError, that message came out wrapped as "DDS ingest API error 503 on POST /api/v1/ingest/begin: …", which buries the explanation behind plumbing and reads, in a notebook traceback, like the caller broke something.StorageUnavailableError— a subclass, so existingexcept IngestApiErrorhandling still catches it — raised on DDS's ownstorage_unavailableslug and nothing else. A 503 from a proxy or load balancer in front of the service has no such body and stays a plainIngestApiError, rather than being mislabelled as a storage fault.str()is the server's message verbatim; the class name carries what the prefix used to.status_code,messageandwherestay available for code that handles it programmatically._error_slug()reads the machine-readable half of the{error, message, path}body, which_error_message()was throwing away.FolderIngest.run()therefore stops atbeginwith an error that explains itself, having uploaded nothing — the point of the server-side pre-flight, and what the new folder test asserts.51 passed; mypy clean on the changed modules.
Claude-Session: https://claude.ai/code/session_01HFZRQnGnYw8mBCVsp9seds