fix(collection): guard INT64_MIN count in take/drop

[belowzeroff]

Summary

• reject INT64_MIN scalar take counts before negating the count magnitude
• avoid negating INT64_MIN in drop-from-end
• add RFL regressions for take/drop edge cases

Tests

• make test TEST_CORES=2

[singaraiona]

Reviewed (with an adversarial verification pass). The guard itself is right, and the snapshot placement (before the per-branch negations) covers every scalar path in ray_take_fn/ray_drop_fn. Three confirmed issues around it:

1. The same -INT64_MIN UB survives on the select take: paths — src/ops/query.c:635 (no-sort path) and :870 (apply_sort_take) both do int64_t want = -atom_n; on the user-evaluated take count with no guard, and the DAG pushdown at :9914 passes -n_take to ray_tail. Your own test proves -9223372036854775808 parses to an INT64_MIN atom; select from t take: -9223372036854775808 negates it — the exact sibling of the pattern this PR fixes. Since these paths convert the atom to a range themselves, the new guard in ray_take_fn is never consulted for query take: clauses.

2. Pre-existing, same function, memory-unsafe: range-take end = start + amount overflow. In the string/vector/list range branches (src/ops/collection.c ~1665/1683/1757) amount is validated only non-negative and start clamped ≥ 0, so (take [1 2 3] [1 9223372036854775807]) overflows end, wraps negative, count goes hugely negative, and the vector branch calls memcpy with (size_t)(count * esz) — crash or memory corruption. The dict branch's if (end < start) end = start; is the fix pattern the other three branches lack. Flagging here because it's the same overflow class in the same function this PR hardens; happy to see it split into a follow-up instead.

3. The PR enshrines a take/drop asymmetry on the null sentinel. INT64_MIN is NULL_I64 (0N), so after this PR (take xs 0N) errors "range" while (drop xs 0N) silently returns empty — and the new rfl test pins the drop behavior. A null count from upstream data silently destroying a collection via drop while erroring via take is a footgun; note the codebase already has two more policies (window verbs reject null with a "type" error via ts_window_arg; neg/abs propagate null). Erroring in both take and drop seems like the conservative pick, but any single consistent policy beats three.

Minor: the guard's is_numeric && as_i64(n) == INT64_MIN also admits -RAY_F32, for which as_i64 falls back to a raw bit read — an f32 count of -0.0 (bit pattern 0x8000000000000000) trips the guard with a misleading "count magnitude out of range". Don't narrow the guard to -RAY_I64 (that would reintroduce the UB for that input); the clean fix is rejecting -RAY_F32 counts up front alongside the existing -RAY_F64 rejection. The leading ray_is_atom(n_obj) && is redundant (is_numeric only matches atom type codes).

[belowzeroff]

Thanks — reproduced all of this under UBSan against current dev; you're right on every count.

One meta note first: the guard in this PR already landed on dev via #409 (d59bb43e), so #419 is a duplicate of an already-merged fix. I'd suggest closing it as superseded and taking the gaps below into a focused follow-up rather than re-litigating the guard here.

1. select take: UB — confirmed, and a touch wider than flagged. select {take: -9223372036854775808 from: t} traps twice: the DAG pushdown -n_take at query.c:9941 (your :9914, line-drifted) and a cascade in exec.c:3022 (nrows - want → 3 - INT64_MIN). The no-sort (:635) and apply_sort_take (:870) -atom_n sites are the same class. As you note, these convert the atom to a range themselves, so the ray_take_fn guard never sees them — the follow-up will guard the count where it's negated, covering all of them.

2. Range-take end = start + amount overflow — confirmed, memory-unsafe. (take [1 2 3] [1 9223372036854775807]) overflows at collection.c:1683 then :1685, producing a bogus count; on this ASan/UBSan build it surfaces as oom, but on release it's exactly the memcpy((size_t)(count*esz)) corruption you describe. The dict branch's if (end < start) end = start; is the clamp the string/vec/list branches are missing. Agreed it's a distinct memory-safety class — I'll do it as its own commit in the follow-up, per your offer.

3. take/drop null-sentinel asymmetry — agreed; going with error-in-both. (take xs 0N) erroring while (drop xs 0N) silently empties is a footgun, and one consistent policy beats three. I'll take the conservative pick: reject an INT64_MIN/0N count in both verbs with a "type"-class error (matching ts_window_arg), and update the rfl test that currently pins drop→empty.

Minor — agreed. as_i64 on a -RAY_F32 atom hits the return x->i64 bit-read fallback (internal.h:119), so is_numeric && as_i64(n) == INT64_MIN mis-fires on an f32 whose bits alias INT64_MIN, with a misleading "range" message. Fix is to reject -RAY_F32 counts up front alongside the existing -RAY_F64 rejection (not to narrow the guard to -RAY_I64, which would reopen the UB). And yes — the leading ray_is_atom(n_obj) && is redundant since is_numeric only matches atom codes; will drop it.

Net: close #419 as superseded, and one follow-up PR covering (1) the select take:/DAG negation guard, (2) the range-end clamp, (3) the unified null policy, and (4) the F32 guard cleanup. Happy to open it whenever.

[belowzeroff]

Opened the follow-up: #424 — covers all four items (select take: negation guard across the no-sort / apply_sort_take / DAG-pushdown paths, the range-end overflow clamp, the unified take/drop null policy as a type error, and the f32 guard cleanup). Full suite green, UBSan-clean. Since this PR's guard already landed via #409, #419 can be closed as superseded.