fix: complete forecast, history rebuild and shutdown recovery - #1269
Conversation
Signed-off-by: Fredrik Ahlgren <fredrik@sourceful-labs.com>
Signed-off-by: Fredrik Ahlgren <fredrik@sourceful-labs.com>
Signed-off-by: Fredrik Ahlgren <fredrik@sourceful-labs.com>
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 4675122793
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| return nil | ||
| func (s *Store) runSeriesHourBackfill(ctx context.Context, retryDelay time.Duration) { | ||
| for ctx.Err() == nil { | ||
| attemptCtx, cancel := context.WithTimeout(ctx, 30*time.Second) |
There was a problem hiding this comment.
When summarizing one large Parquet day takes longer than 30 seconds, this deadline cancels summarizeParquetDay; that function persists only a completed file hash, so the next attempt restarts the same file from the beginning and can repeat forever. On slower hosts or unusually dense daily archives, the hourly rebuild therefore never writes its completion marker, leaving long-range charts on raw history and blocking retention paths that require ensureSeriesHours; retain a per-file progress checkpoint or give this non-resumable stage a sufficient deadline.
AGENTS.md reference: AGENTS.md:L24-L25
Useful? React with 👍 / 👎.
Signed-off-by: Fredrik Ahlgren <fredrik@sourceful-labs.com>
1 participant
A software update can reject unchanged forecast records, and an interrupted hourly-history rebuild never resumes. Shutdown can also abandon accepted ticks or skip the deferred database close on an API restart.
This change completes those recovery paths:
Validation:
make verifypasses locally. Regression tests cover encoder changes, damaged payloads, cursor/summary atomicity, reopen/resume, late samples, snapshot invalidation, sparse history, transient read failure, multi-batch shutdown and explicit drain failure. An opt-in test verifies a disposable large-history copy against raw counts, sums and extrema. A copied-history run completed and matched raw counts, sums and extrema. A target-box run confirms the bundled Energyplan worker plans, forecast records persist, and live commits continue while backfill advances. Physical EV charging is still unverified because no car is connected.Review follow-up: keep a two-hour attempt budget for Parquet files, which checkpoint at file completion. Per-operation SQLite deadlines remain five seconds; shutdown still cancels the work. The updated
make verifyrun passes.Closes #1265. Closes #1266. Closes #1268.
The open backup-pacing PR #1261 retains its separate scope and unresolved review findings; it is not included here. No UI or shared protocol registry changes.