fix: refine contacts cumulative stats + refine timeout (SQL/backend)

[malek10xdev]

Summary

Split from #2881 (now closed): this PR keeps the SQL/backend half — the /contacts realtime INSERT gating moves to #2884, where it belongs with the frontend channel changes.

Fixes two bugs from the client demo against QA:

1. refine contacts SQL timed out — refine_persons scanned the whole (growing) pointsofcontact table with no (user_id, ...) index, blew Kong's ~60s upstream timeout, and only reflected the last mining cycle.
2. Redundant writes/events — re-mined folders duplicated POC rows and no-op person updates emitted realtime events for unchanged rows.

Key changes

refine_persons rewrite (SQL)

• messages becomes a permanent per-user ledger (refined_at marker) — never drained, never re-counted; pointsofcontact becomes transient staging (purged after refine) with idempotent inserts (ON CONFLICT ... DO NOTHING).
• refinedpersons now accumulates (counts += delta, recency/seniority GREATEST/LEAST, tags distinct-union; temperature computed inline matching the trigger). Stats are cumulative, no more double-counting.
• Partition-aware indexes on pointsofcontact (user_id, message_id), (user_id, person_id), and messages (user_id, message_id) — the POC join drops the nested-loop seq scan; one-time purge of ~2.5M orphaned POC rows.
• One-time orphan purge joins on the (id, user_id) PK, not ctid — ctid is only unique per partition on the HASH(user_id)-partitioned table, so cross-partition ctid collisions could have deleted valid rows.

Backend writes

• Idempotent POC bulk insert (ON CONFLICT (user_id, message_id, person_id) DO NOTHING) — also a prerequisite for feat(mining): incremental passive mining via per-source UID watermark #2882's at-least-once watermark semantics (cancel/uidvalidity re-fetch re-inserts POC rows).
• IS DISTINCT FROM change-guard on the single-row person upsert (the RETURNING + fallback re-select keeps person-id resolution correct), cutting redundant UPDATE writes/realtime events.
• 55s refine timeout with clear logging (refine stays awaited — the mining-complete email reads refined stats); Pipeline completion failure logging made structured.

Testing

• Cross-referenced NEW vs OLD stats on identical generated data (800 people, ~6.5k POC, tag edge cases): 0 diffs across all stat columns; temperature inline == trigger formula (0 mismatches).
• Load-tested at 60k messages / 10k people / 143k POC: backfill ~13s (under the 55s safety), incumbent re-run ~70ms, incremental +1000 msgs ~2.3s; accumulate exact (+50 → occ 7→57), no double-count, ledger fully marked, POC purged.
• Current CI baseline: backend tsc build clean, 605/605 unit tests green (after merging latest main, incl. fix(mining): surface real OAuth 401 cause in passive-mining; improve reauth classification + error messages #2880), prettier/eslint clean.

Migration notes for QA/prod

• Apply 20260904000001_refine_persons_ledger_accumulate.sql, then run one full SELECT private.refine_persons('<user_id>') so residual unrefined messages are accounted for.
• The migration runs the non-concurrent index builds and the batched orphan purge inside one transaction — expect write locks on pointsofcontact/messages during the deploy window.
• Concurrent refine_persons runs for the same user could double-count (the backend 55s timeout does not cancel server-side work) — pre-existing race class, not introduced here.
• The realtime tenant rate-limit raise (100 → 1000 events/s) is handled in leadminer.io (self-hosted Supabase config) — separate PR.

Resolves #2867.

[github-actions]

Coverage Report

✅ Passed

Commit: b3f01a2

Summary

	Name	Stmts	Branch	Funcs	Lines
🟡	Total	45.7%	43.2%	52%	46.1%

All files

	Name	Stmts	Branch	Funcs	Lines
🔴	backend/src/db/mail.ts	0%	0%	0%	0%
🔴	backend/src/db/pg/PgContacts.ts	33.7%	6%	40.7%	32.5%
🟢	backend/src/services/tasks-manager-v2/Pipeline.ts	83%	82.4%	82.4%	84.1%

No coverage changes

	Name	Stmts	Branch	Funcs	Lines
🔴	backend/src/db/mail.ts	0%	0%	0%	0%
🔴	backend/src/db/pg/PgContacts.ts	33.7%	6%	40.7%	32.5%
🟢	backend/src/services/tasks-manager-v2/Pipeline.ts	83%	82.4%	82.4%	84.1%

Generated by Test Coverage Reporter for commit b3f01a2