storage/gcp: optionally set object retention on immutable log resources

[eperrine-ant]

What

Adds two optional fields to the GCP driver's Config:

• ObjectRetentionPeriod time.Duration — when non-zero, the log's immutable resources (full tiles and full entry bundles) are written with a GCS object retention configuration expiring that period after the write. The checkpoint and partial tiles/bundles are always written without retention so they can still be updated and garbage-collected.
• ObjectRetentionLocked bool — selects Locked (cannot later be reduced or removed) vs Unlocked mode for that retention.

Left unset, behaviour is unchanged. Both Appender and MigrationWriter honour it. New rejects a negative period. (Object retention can only be set through the JSON API, which the default client uses; the field doc notes this for callers supplying their own GCSClient.)

Internally, logResourceStore decides from the layout (partial vs full) whether a resource gets retention, and objStore.setObject gains a retention *storage.ObjectRetention argument which gcsStorage sets on the Writer.

Why

On a bucket with object retention enabled, this lets an operator make the log's write-once resources undeletable and unmodifiable (including by the log operator's own credentials) for a chosen period, as storage-level defence in depth for the log's append-only property. Doing it at write time avoids a second metadata-update round trip per object and the window in which the object exists unlocked.

A bucket-level retention policy cannot express this, because the checkpoint and .p/ partials must remain mutable/deletable — which is also why the driver, which owns the layout, is the right place to decide which objects are retained rather than the caller.

Testing

• TestImmutableResourceRetention drives logResourceStore over the in-memory objStore and checks the checkpoint and partial tile/bundle are never retained, full tile/bundle are retained with the configured mode and an expiry of now + period, and nothing is retained when no period is set.
• TestSetObjectRetention drives gcsStorage against an httptest fake of the GCS JSON API and checks a non-nil retention arrives as the uploaded object's retention metadata (mode, retainUntilTime) and a nil one sends none.
• TestObjectRetentionPeriodValidation covers New.

go test ./storage/gcp/..., gofmt, golangci-lint clean.

Alternative considered

A caller-supplied func(objName string) *storage.ObjectRetention hook evaluated per write. It is fewer lines in the driver and maximally flexible, but it makes the caller responsible for knowing which layout paths are safe to lock (get it wrong and checkpoint publication or GC breaks), so the driver-owned form above seemed the better default. Happy to go the other way if preferred.

[AlCutter]

Hi @eperrine-ant, thanks for the PR!

I'm trying to think a bit about the benefit which comes from these changes - I really like the intent to essentially "fool proof" the log/protect against insider risk, but I'm wondering how much of this we'll get.

In the worst-case, with these changes, a log can still lose its checkpoint and all partial tiles & bundles. Tiles can always be recomputed from the entry bundles, so arguably we can "not worry" too much about those.

The checkpoint ideally would have been witnessed before being published and subsequently lost, so at least in theory that still exists. So the irrecoverable loss is really just the partial entry bundle at the leading edge of the log.

So your clumsy operator/insider/attacker can still effectively kill the log and delete up-to the most recent 256 entries, but they cannot entirely wipe out the log's history.

Is that a fair understanding?