[GP-02] Sequential mode: configurable carry-over-length, correct stitching for block-overlap != 1 by aoustry · Pull Request #274 · AntaresSimulatorTeam/GemsPy

aoustry · 2026-08-14T15:16:42Z

Process ID

Process: GP-02

Description

Closes #271.

In sequential-subproblems mode, the carry-over that stitches block N to block N+1 was hardcoded: it extracted block N's last timestep and pinned it to block N+1's first timestep. That pairing only refers to the same absolute timestep when block-overlap == 1; for block-overlap >= 2 it silently pinned the wrong pair of timesteps.

This PR makes the carry-over an explicit resolution setting and fixes the absolute-timestep alignment:

carry-over-length added to ResolutionConfig (optional; omitted → defaults to block-overlap, i.e. full pin of the overlap zone). Explicit 0 is legal and distinct from unset: blocks overlap for lag-constraint history but are not stitched at all.
Validation: 0 <= carry-over-length <= block-overlap (no special case at block-overlap == 0), plus the previously missing range check 0 <= block-overlap < block-length.
Runtime: _extract_carry_over now extracts effective_carry_over_length values starting at the earliest shared timestep (local index block_length - block_overlap) instead of always the last one; Phase 5 pins the next block's first k timesteps against them (clamped for truncated final blocks). Scalar initial_values keep the legacy single-timestep pin for direct build_problem callers.
Docs: docs/user-guide/optim-config.md documents the three parameters with an annotated timeline diagram (from the issue discussion), defaults, validation, and a breaking-change warning; docs/CHANGELOG.md gains a BREAKING entry with the migration note.

Breaking change: with block-overlap: 0 (the default) nothing is carried between blocks any more — the previous implicit single-timestep seeding is gone. Configs relying on state continuity (e.g. storage state-of-charge) must set block-overlap: 1 (and optionally carry-over-length: 1).

Impact Analysis

Affected modules:

optim_config/ — ResolutionConfig schema: new field + validators (src/gems_craft/optim_config/parsing.py).
gems_runner/session/ — _run_sequential / _extract_carry_over generalized to a multi-timestep window.
gems_runner/simulation/ — optimization.py Phase 5 carry-over constraints generalized; build_problem docstring updated.
docs/ — user guide + changelog.

Solver output values expected to change: yes, for sequential mode only:

block-overlap >= 2: outputs change because the previous stitching was incorrect (wrong absolute timestep pinned); the new behaviour pins each shared timestep to its matching absolute timestep.
block-overlap: 0: outputs change because blocks no longer inherit the previous block's final state (breaking change above).
block-overlap: 1: no change — the new default is identical to the old hardcoded single-timestep pin; the existing e2e tests (test_rolling_horizon_suboptimality.py, test_optim_modes.py) pass unmodified.
frontal / parallel-subproblems / benders-decomposition: unaffected.

Test coverage: 13 new unit tests for the validators and default resolution; new e2e module test_sequential_carry_over_length.py (block-length 6, block-overlap 3, carry-over 3/1/0 over an aperiodic 12-step demand) asserting every pinned shared timestep is identical in both blocks — all four cases fail against the previous runtime behaviour and pass with the fix.

Checklist

Unit tests pass (pytest) — 604 passed, 6 skipped, 2 xfailed
Type checking passes (mypy)
Formatting passes (black, isort)
pyproject.toml version bumped if applicable — not applicable (version bumps are handled via the feat(release): workflow)
AGENTS.md reviewed for impact and updated if needed — no update needed (resolution-mode internals are not described there)

Add 'carry-over-length' to ResolutionConfig (issue #271): optional, defaults to block-overlap via effective_carry_over_length, validated as 0 <= carry-over-length <= block-overlap with no special case at block-overlap == 0. Also add the previously missing range check 0 <= block-overlap < block-length. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018BsEZdPwCrguejkXxz33FS

…ial mode The sequential carry-over always extracted block N's last timestep and pinned it to block N+1's first timestep, which only refers to the same absolute timestep when block-overlap == 1; for overlap >= 2 it silently stitched the wrong pair of timesteps (issue #271). _extract_carry_over now extracts effective_carry_over_length values starting at local index block_length - block_overlap (the earliest shared timestep), re-indexed to time 0..k-1, and Phase 5 pins the next block's first k timesteps against them. Scalar initial_values (no time dim) keep the legacy single-timestep pin for direct build_problem callers. Behavioural consequence: with block-overlap: 0 nothing is carried between blocks any more (the previous implicit single-timestep seeding is gone) — state continuity now requires block-overlap >= 1. The new e2e test fails against the previous runtime behaviour for all four of its cases and passes with this fix; existing block-overlap: 1 e2e tests pass unmodified. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018BsEZdPwCrguejkXxz33FS

Document the new 'carry-over-length' resolution setting: parameter table entry, expanded sequential-subproblems section with an annotated timeline diagram (block-length / block-overlap / carry-over-length), default and explicit-zero semantics, validation rules, and a breaking-change warning for block-overlap: 0 configs. Add the corresponding CHANGELOG entries. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018BsEZdPwCrguejkXxz33FS

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018BsEZdPwCrguejkXxz33FS

With block-overlap: 0 no carry-over constraint is created: each block's initial state is free (block 1 serves its t=7 peak by pre-charging from a free SoC, which the old implicit single-timestep seeding made impossible), and the sequential result is identical to parallel-subproblems mode, which solves the same windows independently by construction. The test fails against the pre-#271 runtime behaviour. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018BsEZdPwCrguejkXxz33FS

Juliette-Gerbaux · 2026-08-18T07:47:46Z

+                else:
+                    # Scalar (no time dim): legacy form, pin the first timestep.
+                    self.linopy_model.add_constraints(
+                        linopy_var.isel(time=0) == init_val,  # type: ignore[arg-type]
+                        name=f"carry_over__{safe}",
+                    )


This scalar else branch looks unreachable: the only producer of initial_values, SimulationSession._extract_carry_over (session.py:263-272), always returns a DataArray with a "time" dim. Is there a live caller that still needs this legacy path, or can it be removed?

Good point; I have to investigate more how the time-independent variables would have to be handled by the carry-over mechanism.

Design choice: I propose not to fix any time-independant variables.

Implementation in 9032635

tbittar · 2026-08-19T09:09:30Z

+    """carry-over-length < block-overlap pins only the leading shared
+    timesteps; the rest of the overlap zone is re-optimized freely."""
+    raw = _run(tmp_path, "partial_pin", _sequential_config("carry-over-length: 1"))
+    _assert_pinned_window_consistent(raw, carry_over_length=1)


I think we should also assert that the 2nd and 3rd overlapping timesteps may differ between the blocks N and N+1 (I hope the test already leads to different solutions when pinning or not these timesteps)

Or even better, I think all tests in this test suite should rather focus on checking whether the bounds of the carried over variables are fixed / let free depending on the test case (or more precisely if an equality constraint exists for carried over variables). This is what we really want to test

Fixed in 7e5d0a8

tbittar · 2026-08-19T09:13:16Z

+    timesteps = set(raw["absolute_time_index"].dropna().astype(int))
+    assert timesteps == set(range(12))


I do not understand what is tested here, I have the feeling this test would pass whatever the carry over length (as all timesteps are anyway present in the simulation table). Like the previous test we may assert that at least the value of the first overlapping timestep differs accross successive block (or find at least one block where it is the case if the previous statement is too strong)

tbittar · 2026-08-19T09:15:59Z

+    Two complementary checks:
+
+    - Block 1 ([6..11], demand [2,4,0,4,4,0]) serves its t=7 peak by
+      pre-charging its *free* initial storage state.  Under the old implicit


Remove reference to historical design

tbittar · 2026-08-19T09:24:16Z

Functional question : Now for the use case with a storage on which we want continuity in the SoC, and let's say with weekly blocks, we should have block_length = 169 and block_overlap = carry_over_length = 1 ?

tbittar · 2026-08-19T09:27:14Z

    block_overlap: int = 0
+    carry_over_length: Optional[int] = None


Should we raise an error (or a warning) when block_overlap and carry_over_length are set in a different mode than sequential to avoid silent ignore of these parameters

Good catch - done in c97b584, as a hard error rather than a warning.

A new ResolutionConfig validator rejects block-overlap / carry-over-length in all three non-sequential modes (frontal, parallel-subproblems, benders-decomposition). parallel-subproblems was the motivating case: it does window the horizon, so an overlap there looks like it should do something and quietly does nothing.

tbittar · 2026-08-19T09:29:46Z

+                    local_start=block_length - block_overlap,
+                    length=carry_over_length,
                )
                t_start += block_length - block_overlap


block_length - block_overlap computed twice, could be extracted to improve readability

Co-authored-by: tbittar <thomas.bittar@rte-france.com>

Refactor carry_over extraction to use updated t_start.

Updated the test description to remove issue reference.

…quential mode `block-overlap` and `carry-over-length` steer the stitching of consecutive blocks, which only exists in `sequential-subproblems`. They were silently dropped in the three other modes — most confusingly in `parallel-subproblems`, which does window the horizon but ignores the overlap — while the user guide already documented them as sequential-only. A new `ResolutionConfig` validator now rejects either key when the mode is not sequential. The check is on the keys the user actually wrote (`model_fields_set`), so an explicit `block-overlap: 0` is caught too. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014b5oL6qL5BVN6MWwBLcJxc

…l_values contract (#282) * refactor(runner): drop the dead scalar carry-over branch, enforce the contract

…#283) * test(carry-over): assert the pinned constraint window, not the values Replace the four value-comparison tests of `carry-over-length` with one parametrized test that checks what the setting actually controls: which variables are fixed by a carry-over equality constraint and which are left free.

claude added 3 commits August 14, 2026 15:15

aoustry commented Aug 14, 2026

View reviewed changes

Comment thread docs/user-guide/optim-config.md Outdated

aoustry and others added 5 commits August 14, 2026 17:19

Apply suggestion from @aoustry

670af95

Update CHANGELOG.md

dca8015

docs(optim-config): mark block-overlap as sequential-mode only

51f1aa1

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018BsEZdPwCrguejkXxz33FS

Update test_sequential_carry_over_length.py

f843245

aoustry requested review from Juliette-Gerbaux and tbittar August 14, 2026 15:56

Juliette-Gerbaux requested changes Aug 18, 2026

View reviewed changes

tbittar reviewed Aug 19, 2026

View reviewed changes

aoustry and others added 6 commits August 19, 2026 11:54

Update tests/e2e/functional/test_sequential_carry_over_length.py

8c56495

Co-authored-by: tbittar <thomas.bittar@rte-france.com>

Update carry_over extraction logic in session.py

7e5a0a0

Refactor carry_over extraction to use updated t_start.

undo change on computation of local_start for carry_over

4431711

Modify E2E test description for clarity

11a5997

Updated the test description to remove issue reference.

Refactor block timing calculations in session.py

84479d3

aoustry mentioned this pull request Aug 19, 2026

[GP-02] Carry-over: remove the dead scalar branch, enforce the initial_values contract #282

Merged

5 tasks

aoustry added 2 commits August 19, 2026 15:50

[GP-02] Carry-over: remove the dead scalar branch, enforce the initia…

9032635

…l_values contract (#282) * refactor(runner): drop the dead scalar carry-over branch, enforce the contract

		timesteps = set(raw["absolute_time_index"].dropna().astype(int))
		assert timesteps == set(range(12))

		block_overlap: int = 0
		carry_over_length: Optional[int] = None

Uh oh!

Conversation

aoustry commented Aug 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Process ID

Description

Impact Analysis

Checklist

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

aoustry Aug 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

tbittar Aug 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

tbittar commented Aug 19, 2026

Uh oh!

tbittar Aug 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

aoustry Aug 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

aoustry commented Aug 14, 2026 •

edited

Loading

aoustry Aug 18, 2026 •

edited

Loading

tbittar Aug 19, 2026 •

edited

Loading

tbittar Aug 19, 2026 •

edited

Loading

aoustry Aug 19, 2026 •

edited

Loading