ProtoForge

Protocol Inspection & Fuzzing Framework — for authorized lab environments and systems you own or have explicit permission to test.

ProtoForge helps you understand and test unknown, proprietary, binary or poorly documented TCP/UDP application protocols:

connect → record → inspect → infer → verify → replay → mutate → fuzz → minimize

It combines a traffic recorder, binary message inspector, framing/field inference engine, deterministic mutation engine, response classifier, response clustering and an inferred state-model view in one CLI tool.

Scope: ProtoForge implements analysis and fuzzing quality — no stealth, no evasion, no destructive payloads. A target-scope mechanism (below) is an operator safety net, not a security boundary.

Why ProtoForge

Understand before you break	Framing, field, checksum and state-model inference from recorded traffic — every guess confidence-scored, then verified against the live target
Deterministic by design	Same seed ⇒ same mutant sequence ⇒ reproducible runs; every finding exports as a replayable repro JSON
Stateful fuzzing	--prefix establishes protocol state (auth, HELLO) before every case, reaching gated messages single-shot mutations never touch
Evidence, not noise	Response fingerprinting, automatic clustering and cross-run memory keep repeat responses out of your face; interesting cases scored 0–10
Self-documenting results	Everything — sessions, messages, runs, clusters, findings, verdicts — lives in one SQLite file with portable JSON/raw/pcap/dot exports
Operator safety net	Target scope allowlist refuses public-internet addresses unless explicitly authorized

Contents

• Install · Quick start · Command map
• Configuration · Scope protection
• How the analysis works · Storage & portability
• Plugins · Testing · Limitations · Roadmap

---

Install

pip install -e .            # from the repository root
# optional:
pip install -e '.[pcap]'    # pcap import/export via scapy
pip install -e '.[dev]'     # pytest

Requires Python 3.11+. Runtime dependencies: typer, rich (both small).

Quick start

# 1. talk to a service; everything you send/receive is recorded
protoforge connect 10.10.10.20:9001

# inside the interactive shell:
#   /send cafe00020141      send raw hex
#   /sendtext HELLO         send text + newline
#   /recv                   wait for a response
#   /quit                   save session

# 2. overview + inference (framing, fields, checksum candidates)
protoforge sessions
protoforge inspect smoke
protoforge show smoke 5          # annotated hexdump + strings/ints/entropy
protoforge messages smoke

# 3. replay captured messages at the target
protoforge replay smoke --message 5

# 4. preview deterministic mutations (no traffic)
protoforge mutate smoke --message 5 --count 24 --seed 1337
protoforge mutate smoke --message 5 --field 2 --strategy boundary

# 4b. verify inferred hypotheses against the live target
protoforge verify smoke            # framing/checksum -> VERIFIED/SUPPORTED/REFUTED

# 5. fuzz the live service with field-aware mutations
#    use --prefix to establish protocol state first (e.g. authentication)
protoforge fuzz smoke --message 5 --cases 5000 --rate 50 --seed 1234 \
    --prefix-text HELLO
protoforge runs
protoforge show-case <run> <case>
protoforge replay-case <run> <case>              # reproduce a finding
protoforge replay-case --file repro.json        # from exported repro file
protoforge minimize-case <run> <case>           # shrink a trigger to its essence

# 6. differential analysis & state model
protoforge diff smoke:4 smoke:12
protoforge state smoke --export state.dot       # DOT | JSON

Command map

Command	Purpose
connect HOST:PORT	interactive recorded client session
record NAME HOST:PORT	same, with an explicit session name
listen PORT	recording fake server (--echo to bounce bytes)
sessions / messages S / show S N	browse recordings
inspect S	framing hypothesis, inferred fields, checksum candidate
infer S	run & persist inference explicitly
verify S	actively probe the target to confirm/refute hypotheses
replay S -m ID [--all]	re-send captured message(s)
mutate S -m ID	deterministic mutation preview
fuzz S -m ID	controlled fuzzing run against the live target
runs / show-case RUN CASE	inspect fuzz results
minimize-case RUN CASE	delta-debug a case to the smallest trigger (saved as finding + repro JSON)
replay-case RUN CASE	exact reproduction of one case
diff REF_A REF_B	byte-level differential (session:id, file, corpus, hex)
state S --export out.dot	inferred request/response state model
corpus list/add/show/minimize	message corpus management
export S --format json|jsonl|csv|raw|dot|pcap	export
import FILE	portable session JSON or .pcap capture
scope add/list/remove	target allowlist

Global options (place before the subcommand):

--verbose / --debug / --quiet      output control (--debug shows tracebacks)
--timeout SECONDS                  connect/recv timeout
--transport tcp|udp                default transport
--db PATH                          database file (default ./protoforge.db)
--config PATH                      config TOML

Configuration

protoforge.toml (project dir) or ~/.protoforge/config.toml:

[network]
timeout = 3
retries = 1

[fuzz]
rate = 50          # cases per second
seed = 1337        # determinism
cases = 1000
reconnect = "on_error"   # always | on_error | never
response_timeout = 2.0

[output]
timestamps = true
hex_width = 16

CLI flags override config values.

Scope protection (operator safety)

ProtoForge refuses obvious public-internet targets unless they are allowlisted or you pass --authorized-target. Loopback and RFC1918 targets always work.

protoforge scope add 10.10.10.20
protoforge scope add 10.10.20.0/24
protoforge scope list

Allowlist lives in ~/.protoforge/scope.json.

How the analysis works

• Framing inference compares many messages: fixed lengths, newline/NUL delimiters, length prefixes (u8/u16/u32 × BE/LE × total/remaining), constant headers/footers. The best rule can frame raw streams (try_frame).
• Field inference aligns messages per direction, splits variable regions into candidate fields, and tests hypotheses: length fields, counters, sequences, timestamps, opcodes/enums, strings, bitmasks, high-entropy IDs. Every result carries a confidence score — these are heuristic guesses, clearly labeled.
• Checksum detection tries CRC32, Adler-32, CRC16-CCITT, additive and XOR sums over plausible ranges. When confident, mutations get automatic checksum fixup so mutated frames stay well-formed.
• Fuzzing generates deterministic mutants (same seed ⇒ same sequence): field-aware mutations first (length mismatches, opcode neighbors, string stretching, counter resets), then generic strategies (bit flips, interesting integers, truncation/extension, duplication, dictionary). --prefix-text / --prefix-hex establish protocol state before every case (login, HELLO), enabling stateful fuzzing of otherwise gated messages.
• Classification & clustering fingerprints every response (shape, prefix, extracted strings, byte histogram) and labels outcomes: NORMAL, DIFFERENT_RESPONSE, NEW_PATTERN, PROTOCOL_ERROR, UNEXPECTED_LENGTH, DISCONNECT, RESET, TIMEOUT. Responses are clustered automatically; interesting cases are scored 0–10. Fingerprint memory is per-target across runs — a response seen in run #3 isn't news in run #20.
• Verification (verify) actively probes the target: strips the inferred delimiter and watches the server go quiet, sends frames with consistent vs absurd declared lengths, recomputes vs corrupts checksum candidates. Verdicts (VERIFIED / SUPPORTED / REFUTED / INCONCLUSIVE) are stored with the inference so guesses become evidence.
• Minimization (minimize-case) delta-debugs an interesting case against the live target until only the essential trigger bytes remain; the minimal trigger is saved as a finding and exportable as a repro JSON.
• Crash detection: after connection failures the engine probes only the configured service; if it stops accepting connections the case is saved as a finding and the run halts after repeated failures.
• State model projects observed request-class → response-class transitions onto bounded request histories. It is a behavioral projection — not a verified protocol state machine.

Storage & portability

Everything lives in SQLite (./protoforge.db by default): sessions, raw messages (exact bytes), events, persisted inference, fuzz runs/cases/clusters/ findings, corpus, state models. Schema versioning included.

Portable exports: --format json round-trips through import; raw writes numbered .bin files; pcap (via scapy) and Graphviz dot supported. PCAP import reconstructs bidirectional flows into sessions (classic pcap; scapy handles pcapng).

Plugins

Drop Python files into ~/.protoforge/plugins/. Implement any subset of:

from protoforge.plugins.base import ProtocolPlugin, DecodedMessage

class MyProto(ProtocolPlugin):
    name = "myproto"
    def identify(self, messages): ...   # confidence 0..1
    def frame(self, buffer): ...        # -> (message, rest) | None
    def decode(self, message): ...      # -> DecodedMessage | None
    def encode(self, fields): ...
    def mutation_hints(self): ...

Built-ins demonstrate the interface: newline text protocols and the binary format used by the bundled test server. The generic engine never requires a plugin.

Testing

python -m pytest tests/

The suite includes a stateful dummy protocol server (tests/dummy_server.py, text commands + length-prefixed binary frames + deliberate failure modes) used for transport, replay and fuzz integration tests — no external systems are touched. Run it standalone for manual experiments:

python tests/dummy_server.py --port 9001

Limitations (be skeptical of heuristics)

• Field/framing/checksum results are confidence-scored guesses; verify by replaying and observing server behavior.
• PCAP import assumes the first talker in a flow is the client; encrypted or fragmented traffic is not reassembled beyond single packets.
• Clustering uses structural similarity heuristics, not ML.
• UDP fuzzing cannot distinguish silent drops from remote crashes without targeted probing.
• State models are projections of observed conversations, not ground truth.

Next steps (roadmap)

1. Coverage feedback: track which response clusters new mutants reach and prioritize unexplored regions (AFL-style scheduling).
2. Grammar fuzzing: generate full multi-message interaction sequences from the state model, beyond single-message mutation with a fixed prefix.
3. TLS support for authorized lab targets.
4. Live stream framing: apply framing hypotheses while recording so noisy TCP streams are split into messages in real time.