Skip to content

actionlog: compaction, and the attachment-* history lines nothing reads yet #162

Description

@willkg

Two pieces of internal/actionlog that #149 named and _plans/041_page-moved-detection.md deferred. Both are about the log as history rather than the log as a base, which is why they went together and why neither blocked #149: the base check reads only the last ok line for a file, and it works today.

Compaction

#149 promises a compaction setting rather than a truncation, because the history has uses beyond the base — #148's status view, and answering "why did this page change?" — so bounding growth must not mean throwing the history away.

The growth is real but slow: a line is ~180 bytes, so a 200-page tree published daily is ~13 MB of uncommitted local state a year. Nothing breaks at that size; Log.read is linear and runs once per root per run via actionlog.Cache. So this is eventually-real, not 1.0.0-real.

What it needs deciding:

  • What the setting is and where it lives. markfluence.yaml's settings are unknown-key-fatal by design (internal/project/config.go), which cuts both ways: adding a key later is cheap, but a project file written for a newer markfluence is refused by an older binary. That is the established trade and needs no new answer, only a name.
  • What compaction keeps. Keeping the last N lines per file is the obvious rule and is not the same as keeping the last N lines, since a batch writes one line per page. Keeping everything newer than a duration is the other shape.
  • Rewriting a file whose appends are unsynchronized. Every writer today is a single O_APPEND write with no lock, deliberately (the project.SetPageEntry/client.SetContentProperty posture — a lock file brings stale-lock handling to a verb a person invokes by hand). Compaction is a read-modify-write over the whole file, which is a different animal: a concurrent append during the rewrite is lost. Renaming within .markfluence/ is safe from git's point of view (the planted .gitignore holds *), so the temp-file-and-rename shape is available.
  • When it runs. Opportunistically on a write past a size threshold is the cheap answer; a markfluence verb for it is the explicit one.

One consequence already recorded in the plan: compaction adds another way to lose an entry, and that case is the same row as "no line naming this file" — unknown base, publish, silent. It needs no new semantics, which is what makes it safe to add after the fact.

attachment-* history lines

attachment-upload and attachment-download are #149's "history only" rows: they pair a local file with a page, so they could log, but nothing would read what they wrote.

They were deferred on a specific measurement rather than on effort: an attachment upload does not bump the page version, so an attachment write cannot invalidate a base and cannot participate in either check. A logged attachment-upload line would be a row no reader consults.

So this is machinery for a consumer to ask for, and the consumer is #148 — a status view that says "this page's diagram was replaced on Tuesday" wants exactly these rows. Filing it now so the reasoning survives; it should land with, or after, whatever reads it.

Deferred alongside these and not part of this issue: info reporting the base ("last published by markfluence at v44"), which is #148's business.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Fields

    Priority

    None yet

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions