fix: silent zero-match predicates — guid equality and like over list columns - #385
fix: silent zero-match predicates — guid equality and like over list columns#385protocolstardust wants to merge 2 commits into
Conversation
The fused expression program has no 16-byte loads, and the elementwise fallback routed GUID cells through the numeric loops — a guid equality predicate silently returned zero rows at any table size. Bail GUID columns out of the compiled program (EXPR_BAIL_GUID) and give the fallback a memcmp branch for ==/!= over guid operands; ordering operators raise a type error instead of comparing garbage.
|
Triage note on the red CI: the debug jobs fail on |
Splayed string columns load as a list of string atoms (col_load_str_list). ray_like_fn rejected that shape with a type error while str-find accepted it, and exec_like's fallback memset the result to all-false — a like predicate in select-where over such a column (or any unsupported type) silently matched nothing. Add the list branch to ray_like_fn (mirroring str-find), delegate exec_like's list case to it, and replace the silent memset with the type error the direct builtin raises.
a24744b to
d8c7707
Compare
|
Reviewed — including building this branch and reproducing the key findings empirically. The flat-column guid fix and the 1. Blocker — parted guid columns bypass 2. 3. 4. Test coverage gap — the select-where list test never exercises the new branch. 5. Pre-existing but in the restructured dispatch: Efficiency (non-blocking): the RAY_LIST branch sits after the glob compile + result alloc it then throws away — hoisting it above both removes the waste; the delegate's list loop is serial and selection-blind while exec_like's STR path pool-dispatches. Likewise the new guid memcmp loop is serial while the adjacent STR branch parallelizes at RAY_PARALLEL_THRESHOLD. For the record, things I checked that are fine: |
2 participants
Two select-where predicate shapes returned zero rows without any error, discovered while benchmarking filters over a 13.9M-row splayed table.
GUID equality predicates matched nothing
(select {from: t where: (== guid_col g)})returned an empty result at any table size:Fix: the expression compiler bails GUID columns (
EXPR_BAIL_GUID, mirroring the existingRAY_STRbail), andexec_elementwise_binarygets a memcmp branch for==/!=over guid operands. Ordering operators over guids raise a type error instead of comparing garbage.like over splayed string columns: type error direct, silent all-false in where
Splayed string columns load as a list of string atoms (
col_load_str_list).ray_like_fnrejected that shape (str-findaccepts it), andexec_like's fallbackmemsetthe result to all-false — so alikepredicate over such a column, or over any unsupported column type, silently matched nothing.Fix:
ray_like_fngets the list branch (mirroringstr-find's),exec_likedelegates the list case to it, and the silent memset is replaced with the type error the direct builtin raises.Tests
test/rfl/query/guid_like_predicates.rfl— guid predicates find their rows, compose with aggregation, ordering raises; like over a list column in where; like over a numeric column raises.test/rfl/strop/like.rfl— list-of-string/symbol inputs, mixed-type list raises.Benchmark (13.9M-row splayed table, 10 cores)
where (== LCID g)where (like reason_text "*Pending*")Both now within ~1.5x of kdb+ on the same data (6.5 / 50 ms).