Prepare v0.4: reliable memory, traceable corrections and executable evaluation

.github/workflows/ci.yml

@@ -2,68 +2,92 @@ name: ci
 
 on:
   push:
-    branches: [main]
+    branches: [main, 'feat/reliable-memory-*']
   pull_request:
+  workflow_dispatch:
+    inputs:
+      semantic:
+        description: Download and test the optional semantic model
+        type: boolean
+        default: false
+
+permissions:
+  contents: read
 
 jobs:
   test:
-    runs-on: ubuntu-latest
+    runs-on: ${{ matrix.os }}
     strategy:
       fail-fast: false
       matrix:
-        python-version: ["3.10", "3.12"]
+        include:
+          - {os: ubuntu-latest, python: '3.10'}
+          - {os: ubuntu-latest, python: '3.12'}
+          - {os: windows-latest, python: '3.12'}
+          - {os: macos-latest, python: '3.12'}
     env:
-      # Use the deterministic offline embedder so results are byte-stable.
       AGENT_MEMORY_EMBEDDER: hashing
+      PYTHONUTF8: '1'
     steps:
       - uses: actions/checkout@v4
       - uses: actions/setup-python@v5
         with:
-          python-version: ${{ matrix.python-version }}
-      - name: Install
-        # `dev` includes mcp and tiktoken: the MCP server is the headline
-        # feature and must be exercised, and the pinned tokenizer is what makes
-        # the published evaluation numbers reproducible.
-        run: pip install -e ".[dev]"
-      - name: Run tests
-        run: pytest -q
-      - name: Run evaluation
-        run: python eval/run_eval.py
-      - name: Fail if the published results are stale
-        # The README quotes these numbers. If a change moves them, the results
-        # files must be regenerated in the same commit.
+          python-version: ${{ matrix.python }}
+      - run: python -m pip install -e ".[dev]"
+      - run: python -m pytest -q -ra
+      - run: python eval/run_eval.py
+      - name: Keep published retrieval results reproducible
         run: git diff --exit-code -- eval/results.md eval/results.json
-      - name: Run the quickstart example
-        run: python examples/quickstart.py
+      - run: python eval/run_tasks.py --verify-fixtures
+      - run: python examples/quickstart.py
+      - run: python examples/handoff_demo.py
+      - run: ruff check --select E9,F63,F7,F82 src tests eval scripts examples
 
   mcp-versions:
-    # The MCP server broke once because `mcp` 2.0 renamed the server class and
-    # nothing in CI imported it. Both majors are tested from now on.
     runs-on: ubuntu-latest
     strategy:
       fail-fast: false
       matrix:
-        mcp-version: ["mcp>=1.0,<2", "mcp>=2.0"]
+        mcp-version: ['mcp>=1.0,<2', 'mcp>=2.0,<3']
     env:
       AGENT_MEMORY_EMBEDDER: hashing
     steps:
       - uses: actions/checkout@v4
       - uses: actions/setup-python@v5
         with:
-          python-version: "3.12"
-      - name: Install with ${{ matrix.mcp-version }}
-        run: |
-          pip install -e ".[dev]"
-          pip install "${{ matrix.mcp-version }}"
-      - name: Show resolved version
-        run: pip show mcp | head -2
-      - name: Test the MCP server
-        run: pytest -q tests/test_mcp_server.py
-      - name: Start the server entry point
-        # Smoke-test the console script an agent config actually launches:
-        # it must reach the stdio loop rather than exit on an import error.
-        run: |
-          timeout 10s agent-memory-mcp < /dev/null; status=$?
-          if [ $status -ne 0 ] && [ $status -ne 124 ]; then
-            echo "agent-memory-mcp failed to start (exit $status)"; exit 1
-          fi
+          python-version: '3.12'
+      - run: python -m pip install -e ".[dev]" "${{ matrix.mcp-version }}"
+      - run: python -m pytest -q tests/test_mcp_server.py
+      - run: python examples/handoff_demo.py
+
+  package:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-python@v5
+        with:
+          python-version: '3.12'
+      - run: python -m pip install build
+      - run: python -m build
+      - name: Install the wheel outside the source tree
+        run: python scripts/check_wheel.py
+      - uses: actions/upload-artifact@v4
+        with:
+          name: distributions
+          path: dist/*
+
+  semantic:
+    if: github.event_name == 'workflow_dispatch' && inputs.semantic
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-python@v5
+        with:
+          python-version: '3.12'
+      - run: python -m pip install -e ".[dev,real]"
+      - run: python -m pytest -q -ra tests/test_sentence_transformers.py
+      - run: python eval/run_eval.py --embedder sentence-transformers --out-dir /tmp
+      - uses: actions/upload-artifact@v4
+        with:
+          name: semantic-evaluation
+          path: /tmp/results_sentence_transformers.*

.gitignore

@@ -12,3 +12,5 @@ venv/
 .DS_Store
 # local memory store
 .agent_memory/
+# Agent evaluation runs may contain local diagnostics; publish reviewed results explicitly.
+eval/task-results/

CHANGELOG.md

@@ -0,0 +1,21 @@
+# Changelog
+
+## 0.4.0rc1 — release candidate
+
+- Generate UUID4 memory IDs and reject duplicate explicit IDs; retain legacy and caller-supplied identities.
+- Add revisions, bounded history, source metadata and explicit supersession. Require inspected revisions for MCP/CLI edits; report actionable conflicts across MCP SDK versions.
+- Replace age-based lock stealing with OS locks; reject stale snapshot saves; validate loaded data; roll back failed writes and embeddings.
+- Persist complete embedding configuration and retain an existing store's backend by default.
+- Use conservative exact deduplication so negation and changed numbers cannot disappear as similar text.
+- Apply startup freshness rules and budget complete rendered context. Fix Claude prompt payload handling and Git worklog filename/baseline handling.
+- Add inspect, doctor, recall explanations, explicit snapshot export and bounded Markdown import chunks.
+- Add a two-process MCP handoff demo, 30 executable coding fixtures, and a provider-neutral evaluation adapter contract with honest missing-telemetry reporting.
+- Rewrite onboarding, document compatibility/trust boundaries, and add package/OS/SDK checks.
+- Handle quoted/Windows hook executables without duplicating hooks or deleting unrelated commands; retry transient Windows file-sharing conflicts during atomic replacement.
+- Add an optional Codex CLI evaluation adapter with setup checks, completion-event usage reporting and process cleanup; extend CI to macOS.
+
+**Compatibility:** format 3 writes, required MCP/CLI `expected_revision`, changed deduplication and rendered-budget behavior. Upgrade all shared writers together. See [migration](docs/migration-v0.4.md).
+
+**Verified:** the full hashing suite passes on Linux, Windows and macOS; both MCP SDK majors and fresh wheel installation pass in [CI](https://github.com/Ninadnj/agent-memory-engine/actions/runs/34687492192). See the [verification record](docs/verification-codex-adapter.md) for counts and scope.
+
+**Deferred:** the live-agent coding comparison is outside this candidate's release scope. Actual coding-client MCP sessions and the optional semantic model remain unverified. This candidate makes no coding-performance or whole-session token-savings claim. See the [release notes](docs/release.md).

CONTRIBUTING.md

@@ -0,0 +1,24 @@
+# Contributing
+
+Start with a reproducible problem and the smallest change that solves it. This project favors a local store and explicit behavior over infrastructure added in anticipation of scale.
+
+```bash
+python -m venv .venv
+source .venv/bin/activate
+python -m pip install -e ".[dev]"
+AGENT_MEMORY_EMBEDDER=hashing python -m pytest -q -ra
+python eval/run_eval.py
+python eval/run_tasks.py --verify-fixtures
+python examples/handoff_demo.py
+python -m build
+```
+
+On PowerShell, activate `.venv\Scripts\Activate.ps1` and set `$env:AGENT_MEMORY_EMBEDDER = "hashing"` before running tests.
+
+For a bug, first demonstrate a failing regression and record the existing baseline. Preserve current work in a dedicated branch/worktree. Do not loosen behavioral assertions to accommodate a change. Include focused and full test results, migration effects and any unrun checks in the pull request.
+
+Keep generated retrieval results current if their underlying behavior changes. Keep calibration and test tasks separate; do not tune retrieval on the coding test split. Never publish fixture/reference results as model performance. Include actual model/settings identifiers with agent runs.
+
+Core dependencies should stay small. MCP and model integrations remain optional. For persistence changes, cover multiple instances, malformed data, failed writes, legacy IDs and restart behavior. For tool changes, test the real MCP schema and stdio boundary.
+
+Feature requests should include a concrete workflow and what currently fails. A benchmark showing where the current store stops working is more useful than a new backend in search of a workload.