Skip to content

Sweep job workspaces after 30 days - #27

Merged
brocla merged 1 commit into
mainfrom
sweep-old-jobs
Sep 10, 2026
Merged

Sweep job workspaces after 30 days#27
brocla merged 1 commit into
mainfrom
sweep-old-jobs

Conversation

@brocla

@brocla brocla commented Sep 10, 2026

Copy link
Copy Markdown
Owner

Summary

  • data/jobs grew without bound — nothing ever deleted a job's workspace once created. Add a daily age-based sweep mirroring the existing Drive-file retention sweep (internal/mailgun/retention.go): same shape, same crash-safety, applied to local job directories instead.
  • A completed job (jobs.completed_at IS NOT NULL) past 30 days has its data/jobs/<id> workspace removed via os.RemoveAll.
  • DB: added a nullable jobs.files_deleted_at column + jobs_sweep index, migrated in-place for existing databases (plain ALTER TABLE, no rebuild needed). The row is kept — files_deleted_at is set instead of deleting it, mirroring email_deliveries.deleted_at — so jobs get <id> keeps answering after the files are gone, and a swept row is never re-offered to the sweep.
  • Only completed_at gates the sweep query, so a queued/running job can never be swept regardless of age.
  • Wired into cmd/filemill/main.go unconditionally in continuous mode (!once), outside the mailgun-configured branch, since job accumulation happens whether or not email is set up.

Test plan

  • go build ./..., go vet ./..., go test ./... -count=1
  • New store-level tests: cutoff respected, swept rows never re-offered, queued/running jobs never candidates regardless of created_at age
  • New app-level tests: real workspace created via Submit gets deleted past the horizon while a fresh one is untouched; the row survives with status intact; a previously-removed workspace doesn't error the sweep (RemoveAll is idempotent)
  • Verified the migration against a copy of the real production database (135 job rows): row count preserved, every existing row correctly gets files_deleted_at=NULL, a second Open() is a no-op, and ExpiredJobs correctly identifies the 51 jobs that would be swept on first deploy

🤖 Generated with Claude Code

data/jobs grew without bound: nothing ever deleted a job's workspace once
created, regardless of outcome. Add a daily age-based sweep mirroring the
existing Drive-file retention sweep in internal/mailgun/retention.go — same
shape, same crash-safety properties, applied to local job directories
instead of published Sheets.

A completed job (jobs.completed_at IS NOT NULL) past 30 days has its
data/jobs/<id> workspace removed. The row itself is kept, with a new
files_deleted_at marker set instead of deleting it, matching how
email_deliveries.deleted_at already works: `jobs get <id>` keeps answering
after the files are gone, and a swept row is never offered to the sweep
again. Only completed_at gates the query, so a queued or running job can
never be swept no matter how old.

Verified the migration against a copy of the real production database
(135 job rows): row count preserved, every existing row correctly gets
files_deleted_at=NULL, a second Open() is a no-op, and ExpiredJobs
correctly identifies the 51 jobs that would be swept on first deploy.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@brocla
brocla merged commit 847f43b into main Sep 10, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant