Conversation
Enable AshSQLite to route supported aggregate operations through AshSQL's grouped planner. Implementation notes: - select `:grouped` in the SQL implementation; - advertise the supported aggregate kinds and relationship shapes; - delegate query return, filtering, calculations, and root aggregates to the shared AshSQL paths; - keep manual, parent-dependent, and no-attribute relationship shapes unsupported. The following commits add end-to-end coverage for each enabled path.
Provide the SQLite expression used by AshSQL's grouped list planner. Implementation notes: - build lists with windowed `json_group_array`; - filter nil values inside the SQL aggregate when `include_nil?` is false; - leave window planning and result typing in AshSQL; - keep SQLite syntax and JSON representation inside the adapter. The acceptance suite covers ordering, nil inclusion, defaults, constrained values, relationship bounds, and calculation fields.
Add a small adapter API for custom Ecto aggregate expressions. Implementation notes: - define callbacks for the aggregate expression and result type; - let custom modules use SQLite-specific functions; - leave relationships, filters, windows, and result loading in the shared planner. The acceptance suite exercises custom aggregates in root queries and related aggregate loads.
Add the main SQLite acceptance matrix for the grouped strategy. Coverage includes: - root and loaded scalar aggregates; - first, list, exists, and custom aggregates; - filtering, sorting, pagination, calculations, and join filters; - one-hop, unrelated, many-to-many, and multi-hop paths; - nil handling, defaults, distinct values, read actions, and stable unsupported errors. These tests verify the generated grouped and windowed SQL through the concrete SQLite adapter.
Add focused coverage for the parts of `first` that differ from a simple ordered `LIMIT 1` query. Coverage includes: - excluding or including nil values; - applying a fallback to an empty filtered result; - ascending and descending ordering; - all four Ash null-ordering variants. These tests protect behavior that was missing from the original grouped query implementation.
Exercise fieldless distinct counts against the primary-key shapes supported by Ash resources. Coverage includes: - a valid single primary key whose Ash name is not `:id`; - a composite primary key; - a resource without a primary key; - clear errors for the unsupported key shapes. These tests protect the grouped query path from assuming that every resource has an `:id` attribute.
Verify that grouped list and first expressions retain the aggregate item's type constraints. Implementation notes: - build the aggregate data-layer query directly; - inspect the generated Ecto type for the list value; - assert that `trim?` and `allow_empty?` survive query construction. This protects a constraint loss inherited from the earlier SQLite implementation rather than a defect introduced by the extraction.
Verify that an ordered root query with only an offset is bounded before aggregation. Implementation notes: - create three ordered records; - offset the first record without specifying a limit; - assert that the aggregate sees the remaining two records; - exercise SQLite's required `LIMIT -1` representation for a standalone offset. The missing offset condition came from existing AshSQL wrapping logic and was copied into the grouped path.
Add root-query regressions for bounded inputs and non-attribute aggregate fields. Coverage includes: - `first` and `exists` over a zero-limit query; - a sorted and limited query whose outer aggregate has its own sort; - an offset query with no remaining rows; - calculation fields and aggregate-on-aggregate fields. These tests protect grouped query behavior introduced by the aggregate extraction.
Add loaded-aggregate regressions for relationship bounds and rich related fields. Coverage includes: - per-parent relationship limits and offsets; - count and list results over each bounded relationship; - calculation fields on related records; - aggregate fields on related records; - sum, maximum, first, and list results. These tests protect grouped loading behavior introduced by the aggregate extraction.
Add a complete guide to the grouped aggregate behavior exposed by AshSQLite. Documentation includes: - supported root and loaded aggregate kinds; - calculations, filters, sorting, pagination, and relationship paths; - custom aggregates, limits, offsets, and unsupported shapes; - SQLite requirements for windows, `FILTER`, JSON, and null ordering; - navigation, tutorial, API grouping, and changelog updates. The guide reflects the behavior exercised by the preceding acceptance commits.
Remap internal aggregate aliases when returning normal query results. Add SQL regressions for nested bindings, separate relationship paths, string names, and reuse with identical or different definitions. Verify inline aggregate calculations now return their values with shared name normalization.
Add 24 public-API regression tests and controls for the shared AshSQL grouped aggregate fixes. Intermediate and join inputs could skip read preparations, lose actor/tenant or shared context, discard required action arguments, and count rows outside attribute-tenant scope. Cover configured and prepared query actions, arguments and context, plus multitenancy bypass across the relationship path without changing siblings. Verify read-action scope runs before relationship limits and aggregate filters afterward: first/list must not pull matching rows from outside a bounded relationship. Add constrained scalar-type regressions for loaded values and defaults, which previously returned raw stored integers instead of custom typed values. Document relationship preparation and filter ordering, and record the fixes in the changelog. Validation against the paired local AshSQL fix: 24 focused tests and all 274 SQLite tests pass. Failures were reproduced before their fixes, and an extra-high independent subagent approved the final implementation. Formatting, compilation, Credo, docs, dependency checks and corrected local REUSE pass; existing test-helper Dialyzer findings remain.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR adds aggregate support to AshSQLite, using the grouped aggregate strategy from ash-project/ash_sql#264.
I needed aggregates for a local app I’m building on SQLite, and spent a few days exploring the existing SQL adapters and getting this branch into shape with Codex helping lots along the way. My goal was to get most of the common usage of aggregates working with SQLite so that I could continue building my app!
Approach
This PR opts AshSQLite into the ash_sql grouped strategy and adds the sqlite specific implementation to meet the behaviour expectations. Mainly the sqlite specific JSON list aggregation and a custom aggregate behaviour.
I managed to get it all working using grouped or windowed subqueries, then joining the results back to the parent query.
Resource aggregates can be loaded, filtered, sorted, and used in expression calculations. The supported kinds are count, sum, avg, min, max, exists, first, list, and custom. Root query aggregates through
Ash.aggregate/2are also supported for count, sum, avg, min, max, first, and exists.Tests
The main ash_sql doesn't have a lot of tests, or at least we can't test specific integrations (sqlite & postgres) so a lot of the tests for the pull request 264 are in this PR. These should cover most of the use cases at least the ones I needed and should serve as a good starting point for future expansion.
With the paired ash_sql branch, all 274 SQLite tests pass. We also ran the PostgreSQL compatibility suite and Dialyzer in the development environment used by CI.
Contributor checklist
Leave anything that you believe does not apply unchecked.
notused in the creation of this PR.