.eml/.msg: quoted reply history is re-extracted in full for every message in a thread

[dvejsada]

Split out of the bilingual-contract table bloat work (branch claude/contract-table-bloat-v5wikg).

Problem

EmailLoader and OutlookMsgLoader (app/utils/document_loader.py) extract the whole message body, quoted history included. Mail clients quote the entire preceding thread in each reply, so uploading an 18-message negotiation thread as 18 .eml/.msg files extracts the first message 18 times, the second 17 times, and so on — quadratic growth in extracted text for linear growth in actual content.

For a legal office this is the common case, not an edge case: correspondence about a deal arrives as a thread, and each file is uploaded separately.

Secondary: HTML-only messages go through UnstructuredEmailLoader, which flattens the HTML with no control over signature blocks, disclaimers and tracking markup — a long firm footer repeated on every message in the thread compounds the same problem.

Suggested fix

• Detect and optionally trim quoted history (On <date>, <person> wrote:, -----Original Message-----, From:-block separators, >-prefixed lines, <blockquote> in the HTML part), behind a config flag defaulting to whichever behaviour we decide is safer.
• Keep it opt-outable: in a dispute the quoted text can be the evidence (what was actually sent back), so removing it silently is not obviously right. A flag that trims by default with EMAIL_KEEP_QUOTED_HISTORY=True to restore full bodies would match how the .docx knobs were done.
• Consider stripping known disclaimer/footer blocks separately from quoted history — they are noise in every case, where quoted history sometimes is not.

Notes

Deduplicating across files in one upload batch would be the real fix for the quadratic case, but that needs state the loader does not currently have. Per-file trimming is the cheap first step.