FilterExec currently calls filter_record_batch and then pushes the result into LimitedBatchCoalescer, materializing an intermediate filtered RecordBatch on every batch.
Arrow's BatchCoalescer exposes push_batch_with_filter which can fuse the two steps and avoid that allocation when its internal conditions are met.
LimitedBatchCoalescer should expose the same API so FilterExec can use it directly.
FilterExeccurrently callsfilter_record_batchand then pushes the result intoLimitedBatchCoalescer, materializing an intermediate filteredRecordBatchon every batch.Arrow's
BatchCoalescerexposespush_batch_with_filterwhich can fuse the two steps and avoid that allocation when its internal conditions are met.LimitedBatchCoalescershould expose the same API soFilterExeccan use it directly.