Skip to content

fix: ignore masked variable labels when matching constraints - #886

Open
YassineAbdelouadoud wants to merge 1 commit into
PyPSA:masterfrom
YassineAbdelouadoud:fix-remove-variables-masked
Open

fix: ignore masked variable labels when matching constraints#886
YassineAbdelouadoud wants to merge 1 commit into
PyPSA:masterfrom
YassineAbdelouadoud:fix-remove-variables-masked

Conversation

@YassineAbdelouadoud

@YassineAbdelouadoud YassineAbdelouadoud commented Aug 11, 2026

Copy link
Copy Markdown

I encountered this issue while updating to the latest linopy version and running the POMMES framework

Fixes #883

Note

The following content was generated by AI.

The bug

Model.remove_variables removes every constraint for which
constraint.has_variable(variable) is true. has_variable compared the
constraint's vars against the variable's labels without filtering the -1
sentinel, which means two different things on the two sides:

  • on the variable side, -1 marks a masked entry;
  • in a constraint's vars, -1 marks an empty (padded) term slot.

So any masked variable matched any constraint that has a padded term slot, even
when the two are entirely unrelated, and the constraint was silently deleted.
Models built with mask= can therefore lose constraints and solve to a wrong
optimum or unbounded.

The fix

Filter the -1 entries out of the variable labels before matching, through a
new common.assigned_labels helper, applied at the three places that match a
variable's labels against another object's labels:

  • Constraint.has_variable (dense path) — the one that was actually broken;
  • CSRConstraint.has_variable — a CSR matrix stores no empty slots, so this
    path was already correct; the guard is added for symmetry;
  • the objective cleanup at the end of remove_variables, which had the same
    latent mismatch (there it only dropped already-empty terms).

Test

test_remove_masked_variable_keeps_unrelated_constraints in test/test_model.py,
parametrized over freeze to cover both constraint representations. It builds a
constraint over a masked variable b (hence with padded term slots) plus an
unmasked c, and a second constraint that genuinely uses the masked a, then
removes a: the first constraint must survive, the second must still go.

On master the freeze=False case fails at assert not without_a.has_variable(a);
the freeze=True case passes before and after.

Verification
$ uv run pytest -q -p no:randomly --ignore=test/remote
2829 passed, 38 skipped, 192 warnings in 146.41s

$ uv run mypy .
Success: no issues found in 112 source files

$ uv run ruff check .
All checks passed!

test/remote was skipped locally because the oetc extra
(google-cloud-storage) is not installed in this environment; it is unrelated
to this change.

Reproduction on master, before the fix:

import pandas as pd
import linopy

i = pd.Index(range(3), name="i")
m = linopy.Model()
a = m.add_variables(coords=[i], name="a", mask=[True, False, True])
b = m.add_variables(coords=[i], name="b", mask=[True, False, True])
c = m.add_variables(coords=[i], name="c")
m.add_constraints(b.sum() + c >= 0, name="no_a")

m.remove_variables("a")          # UserWarning: ... also removes constraints ['no_a']
print(list(m.constraints))       # [] -- 'no_a' never referenced 'a'

@codspeed-hq

codspeed-hq Bot commented Aug 11, 2026

Copy link
Copy Markdown

Merging this PR will improve performance by 32.18%

⚠️ Different runtime environments detected

Some benchmarks with significant performance changes were compared across different runtime environments,
which may affect the accuracy of the results.

Open the report in CodSpeed to investigate

⚡ 3 improved benchmarks
✅ 172 untouched benchmarks
⏩ 175 skipped benchmarks1

Performance Changes

Mode Benchmark BASE HEAD Efficiency
Memory test_to_lp[storage-n=10] 2.9 MB 1.7 MB +70.76%
Memory test_to_lp[storage-n=250] 35.8 MB 29.7 MB +20.8%
Memory test_to_lp[expression_arithmetic-n=250] 45.8 MB 40.9 MB +11.94%

Tip

Curious why this is faster? Comment @codspeedbot explain why this is faster on this PR, or directly use the CodSpeed MCP with your agent.


Comparing YassineAbdelouadoud:fix-remove-variables-masked (f884458) with master (09e11ad)

Open in CodSpeed

Footnotes

  1. 175 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

`Model.remove_variables` deleted every constraint whose `has_variable`
matched the removed variable. For a masked variable that was every
constraint with a padded term slot: `-1` marks a masked entry on the
variable side and an empty term slot on the constraint side, and the
two were compared without filtering the sentinel.

Filter the -1 entries out of the variable labels before matching, via a
new `common.assigned_labels` helper, in both `has_variable`
implementations and in the objective cleanup of `remove_variables`.
Only the dense `Constraint` path was actually affected -- a CSR matrix
stores no empty slots -- but the same guard now holds for both.

Fixes PyPSA#883

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@YassineAbdelouadoud
YassineAbdelouadoud force-pushed the fix-remove-variables-masked branch from 7d2dc5e to f884458 Compare August 11, 2026 13:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

remove_variables deletes unrelated constraints: -1 means "masked" in a variable but "empty term slot" in a constraint

1 participant