Skip to content

Prune expired cache entries on save - #33

Open
elhoim wants to merge 1 commit into
MISP:mainfrom
elhoim:fix/21-cache-never-pruned
Open

Prune expired cache entries on save#33
elhoim wants to merge 1 commit into
MISP:mainfrom
elhoim:fix/21-cache-never-pruned

Conversation

@elhoim

@elhoim elhoim commented Aug 31, 2026

Copy link
Copy Markdown
Member

Finding 21 (Medium) — bin/cli.py:661

Problem

Expired entries are never pruned - get_cached_response ignores them and set_cached_response only adds. cache.json grows without bound, and the only remedy (--purge-cache) deletes the entire file, so secret-bearing keys accumulate until a full wipe.

Fix

get_cached_response() correctly ignores TTL-expired entries when reading, but nothing ever removed them: set_cached_response() only adds, and cache.json (which holds full, potentially secret-bearing module responses) grows without bound. The only cleanup path was --purge-cache, which deletes the whole file. Added a prune_expired_entries(cache, now, ttl_seconds) helper and called it right before the cache is persisted in the query command, dropping any entry whose cached_at is older than the configured TTL (also dropping malformed entries). This bounds the cache to roughly one TTL window's worth of entries on every run that touches the cache, without requiring a full wipe.

Verification

Reproduced against the unmodified code at 9b8c605, then re-checked after the change.

Before
Loaded bin/cli.py via importlib on the original commit (9b8c605) and checked for a pruning function: hasattr(climod, "prune_expired_entries") -> False. Manually built a cache dict with one expired entry (cached_at 1000s in the past, ttl=60) and one fresh entry; get_cached_response correctly returned None for the expired key, but there was no code path anywhere (get_cached_response, set_cached_response, save_cache) that ever deleted the expired entry from cache["entries"] -- it would persist indefinitely across saves.
After
Same cache dict {"expired_key": ..., "fresh_key": ...} on the fixed branch: climod.prune_expired_entries(cache, now, ttl) returns 1 and list(cache["entries"].keys()) becomes ["fresh_key"] only -- the expired entry is dropped. This call is wired into the query command right before save_cache(), so cache.json shrinks to non-expired entries on every save instead of growing unbounded.

python bin/cli.py --help exits 0 and the module still imports cleanly. Verification was performed offline against the pure functions — no running misp-modules instance is required.

Branched from 9b8c605. This PR addresses only this finding; the other findings from the same review are in separate PRs, so they will need rebasing against each other as they merge.

🤖 Generated with Claude Code

https://claude.ai/code/session_01DYX4TKA5inzByJ4qGWKjqh

get_cached_response correctly ignores expired entries but neither it nor set_cached_response ever removed them, so cache.json (which stores full module responses, potentially secret-bearing) grew without bound. The only remedy was --purge-cache, which wipes the whole file.

Add prune_expired_entries() and call it before persisting the cache in the query command, dropping any entry whose cached_at predates the configured TTL. This bounds cache.json to roughly one TTL window of entries instead of accumulating indefinitely.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant