fix: bound response read in integration catalog fetch - #3812
Open
Quratulain-bilal wants to merge 3 commits into
Open
fix: bound response read in integration catalog fetch#3812Quratulain-bilal wants to merge 3 commits into
Quratulain-bilal wants to merge 3 commits into
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
Bounds integration catalog HTTP responses to mitigate memory-based denial-of-service risks.
Changes:
- Uses the shared bounded-response reader.
- Applies the 1 MiB JSON metadata limit.
Show a summary per file
| File | Description |
|---|---|
src/specify_cli/integrations/catalog.py |
Limits catalog response reads before JSON parsing. |
Review details
Tip
Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
- Files reviewed: 1/1 changed files
- Comments generated: 2
- Review effort level: Medium
Quratulain-bilal
added a commit
to Quratulain-bilal/spec-kit
that referenced
this pull request
Jul 28, 2026
…egression test - Update FakeResponse.read() to accept size parameter for bounded reads - Add test_fetch_rejects_oversized_catalog_response regression test - Verifies _fetch_single_catalog uses MAX_JSON_METADATA_BYTES Fixes github#3812
…egression test - Update FakeResponse.read() to accept size parameter for bounded reads - Add test_fetch_rejects_oversized_catalog_response regression test - Verifies _fetch_single_catalog uses MAX_JSON_METADATA_BYTES Fixes github#3812
Quratulain-bilal
force-pushed
the
fix/unbounded-integrations-catalog-read
branch
from
July 29, 2026 16:05
4f47240 to
0b4066c
Compare
Contributor
There was a problem hiding this comment.
Review details
Suppressed comments (5)
src/specify_cli/integrations/catalog.py:208
- Use the catalog-size budget here, not the fixed-shape metadata budget.
_download_security.py:40-46reservesMAX_JSON_CATALOG_BYTES(8 MiB) for growing listings, and the extension, preset, workflow, step, and bundle catalog fetchers all use it; this 1 MiB cap would reject otherwise valid integration catalogs. ImportMAX_JSON_CATALOG_BYTESand pass it here.
max_bytes=MAX_JSON_METADATA_BYTES,
tests/integrations/test_integration_catalog.py:348
- This monkeypatch targets the metadata constant, so after the production fetch uses the repository's catalog-size constant the test will no longer lower the active limit and will fail with
Invalid JSONinstead ofexceeds maximum size. PatchMAX_JSON_CATALOG_BYTESinstead.
monkeypatch.setattr(catalog_module, "MAX_JSON_METADATA_BYTES", 32)
src/specify_cli/integrations/catalog.py:211
- The bounded
read_response_limited(...)call and its limit are unchanged from the base version; this line only adds UTF-8 decoding. Therefore the production diff does not perform the unbounded-to-bounded replacement described by the PR. Rebase against the current target and either retain the actual security change or close the now-redundant fix rather than presenting this decode/test churn as the mitigation.
).decode("utf-8")
src/specify_cli/integrations/catalog.py:26
- This repeats the identical import on line 24, which is a duplicate-definition lint error. Keep the catalog base import and remove this second
_download_securityimport.
This issue also appears in the following locations of the same file:
- line 208
- line 211
from .._download_security import MAX_JSON_METADATA_BYTES, read_response_limited
tests/integrations/test_integration_catalog.py:334
- The regression contract names the metadata limit, but integration catalogs are growing listings and should use
MAX_JSON_CATALOG_BYTESper_download_security.py:40-46. Update the docstring so it does not lock in the wrong size class.
This issue also appears on line 348 of the same file.
"""Regression: _fetch_single_catalog must use read_response_limited
with MAX_JSON_METADATA_BYTES, not unbounded resp.read()."""
- Files reviewed: 2/2 changed files
- Comments generated: 1
- Review effort level: Balanced
Comment on lines
+664
to
+665
| def read(self): | ||
| return self._data |
…eads - Remove duplicate imports of MAX_JSON_METADATA_BYTES and read_response_limited - Update FakeResponse.read() to accept size argument for read_response_limited - Add offset tracking for proper bounded read behavior Refs: github#3812
Contributor
Author
|
fixed |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Replace unbounded
esp.read() with
ead_response_limited() in integrations/catalog.py to prevent DoS via oversized catalog responses.