Skip to content

Skip describeTypes.json fetch for fully explicit --type queries - #41

Open
elhoim wants to merge 1 commit into
MISP:mainfrom
elhoim:fix/13-github-dependency-per-query
Open

Skip describeTypes.json fetch for fully explicit --type queries#41
elhoim wants to merge 1 commit into
MISP:mainfrom
elhoim:fix/13-github-dependency-per-query

Conversation

@elhoim

@elhoim elhoim commented Aug 31, 2026

Copy link
Copy Markdown
Member

Finding 13 (Medium) — bin/cli.py:850

Problem

describeTypes.json is fetched from raw.githubusercontent.com on EVERY query, including a fully explicit --type domain --value circl.lu, and failure returns 1 before any module is queried. README lists only the local misp-modules service as a requirement, so air-gapped use fails confusingly.

Fix

Finding 13 (github-dependency-per-query, medium): bin/cli.py unconditionally fetched MISP's describeTypes.json from raw.githubusercontent.com on every invocation, even for a fully explicit --type domain --value circl.lu query that names no type to guess or validate — a network failure there returned exit 1 before any module was queried, confusing in air-gapped setups since the README lists only the local misp-modules service as a requirement. Fix: skip that fetch (and the resulting valid-types filter) whenever --type is explicit and no type-listing/guessing output is requested; when the fetch is still needed (guessing, --list-supported-types, --verbose-types), route it through a new get_describe_types() helper that reuses the CLI's existing local cache file/TTL so repeated runs within the TTL don't hit GitHub again. Documented the dependency, its caching, and the --type skip behavior in README.md.

Verification

Reproduced against the unmodified code at 9b8c605, then re-checked after the change.

Before
Monkeypatched fetch_json to count calls to any URL containing "describeTypes"/"githubusercontent", then ran cli.main() with sys.argv = ["cli.py", "--type", "domain", "--value", "circl.lu", "--cache-file", "/tmp/repro13_cache.json"] (module POST also monkeypatched to fail so no real network is touched). Output: "describeTypes.json fetch calls for a fully-explicit --type query: 1" — confirms the unnecessary per-query GitHub fetch on the pre-fix code (checked out at 9b8c6052).
After
Same script re-run after the fix: "describeTypes.json fetch calls for a fully-explicit --type query: 0". A second script running the guessing path (no --type) 3 times in a row with a shared --cache-file shows "describeTypes.json fetch calls across 3 guessing runs (should be 1, cached): 1", confirming caching works when the fetch is still required.

python bin/cli.py --help exits 0 and the module still imports cleanly. Verification was performed offline against the pure functions — no running misp-modules instance is required.

Branched from 9b8c605. This PR addresses only this finding; the other findings from the same review are in separate PRs, so they will need rebasing against each other as they merge.

🤖 Generated with Claude Code

https://claude.ai/code/session_01DYX4TKA5inzByJ4qGWKjqh

Previously, every invocation fetched MISP's describeTypes.json from
raw.githubusercontent.com, even for a fully explicit query such as
'--type domain --value circl.lu' that names no type to guess or validate.
A network failure there returned exit 1 before any module was queried,
which is confusing in air-gapped environments since the README only
lists the local misp-modules service as a requirement.

Skip that fetch (and the subsequent valid-types filter) when --type is
given and no type listing/guessing output is requested. When the fetch
is still needed (guessing, --list-supported-types, --verbose-types),
cache the response in the existing local cache file/TTL so repeated
invocations don't hit GitHub every time. Documented the dependency,
its caching, and the --type skip in the README.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant