Add llm_memory_local sandbox and Conversation Memory Poisoning exploit (#22) - #60
Open
fasinet wants to merge 3 commits into
Open
Add llm_memory_local sandbox and Conversation Memory Poisoning exploit (#22)#60fasinet wants to merge 3 commits into
fasinet wants to merge 3 commits into
Conversation
Implements issue GenAI-Security-Project#22: a persistent-memory LLM sandbox and a working Conversation Memory Poisoning exploit. Sandbox (sandboxes/llm_memory_local): - SQLite-backed conversation memory (app/memory.py) with an intentionally unscoped fact store to demonstrate the vulnerability - FastAPI mock OpenAI endpoint wired to memory, plus /v1/memory/facts and /v1/memory/reset debug endpoints - session_id support across the API, Gradio UI, and test client Exploit (exploitation/memory_poisoning): - attack.py plants a fact as one session and confirms it leaks into an unrelated victim session - Makefile automation (setup/attack/stop/all) and README write-up Tested end-to-end containerized (Podman); passes black, isort, mypy.
Add explicit dark text color to styled diagram nodes so labels are readable on GitHub's dark theme, and shorten exploit diagram labels to avoid text cutoff. Also apply isort import ordering to gradio_app.py.
2 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Implements #22: a new
llm_memory_localsandbox with persistentconversation memory, plus a working Conversation Memory Poisoning exploit.
As discussed in #22 with @felipepenha, this is the contribution intended to
lead the memory-poisoning implementation.
What's included
Sandbox —
sandboxes/llm_memory_local/app/memory.py), with anintentionally unscoped long-term fact store that models the
vulnerability
/v1/memory/factsand
/v1/memory/resetdebug endpointssession_idsupport across the API, Gradio UI, and automated test clientExploit —
exploitation/memory_poisoning/attack.pyplants an instruction as one session, then confirms it leaksinto an unrelated victim session
setup/attack/stop/all) and a READMEwrite-up with attack diagram
How to test
cd exploitation/memory_poisoning make setup make attackExpected: the victim session's reply contains the attacker's planted
payload —
[!] VULNERABLE: poisoned instruction leaked...Notes
black,isort, andmypythreat_model/directory is inherited fromllm_localand does notyet cover the memory-specific attack surface (noted in the sandbox README)