FTWDB — archived energy-storage test lab

Archived on 8 September 2026. This repository preserves the experimental engine, fault tests, sanitized workloads and database comparison results. It has no active release or support programme.

FTW now embeds DuckDB in Core for history and keeps SQLite for configuration. FTWDB is no longer part of the FTW runtime. New storage work belongs in FTW Core.

Start with the archive record and lessons, the engine comparison, and the archived benchmark runners and raw measurements. The final local work is preserved on archive/final-lab-snapshot-2026-09-08. That snapshot is separate from the newer fixes already on main.

Historical engine and lab commands

The material below describes the experiment at archive time. Commands are for isolated research on copies of data, not a supported installation path.

Forecasts, Telemetry & Watts.

FTWDB is an experimental embedded database for energy systems. The goal is a small Rust engine that is fast on time-window aggregates, survives abrupt power loss, and minimizes write amplification on SD cards and other constrained edge storage.

The last published release is v0.1.0-alpha.1. It is an evaluation build for Unix systems, not a production release. Download release archives from GitHub Releases, or install the CLI from its immutable tag:

cargo install --git https://github.com/srcfl/ftwdb --tag v0.1.0-alpha.1 --locked --bin ftw

This repository currently contains the first executable storage slice:

• append-only, checksummed atomic batches;
• configurable durability (Always, byte-grouped sync, or explicit sync);
• recovery that removes a torn final batch but reports earlier corruption;
• three-dimensional time (valid, knowledge, and change) plus run_id;
• atomic mixed transactions for assets, topology, series, runs, plans, and points;
• persistent catalog recovery and exact-time plan-versus-actual queries;
• latest-revision and point-in-time queries;
• mergeable gauge, energy-integral, and reset-aware counter aggregates;
• immutable compressed raw and rollup segments with checksummed manifests;
• persistent fixed and IANA-calendar rollups, including DST-correct energy;
• automatic late-data invalidation, rebuild, cached queries, and retention gates;
• exact per-source ingress replay receipts that survive reopen;
• a draft, bounded Go-portable shadow protocol and local Unix sidecar;
• tests, property tests, Criterion benchmarks, and a competitor benchmark plan.

The current source prepares 0.1.0-alpha.2 for bounded, opt-in shadow collection alongside the FTW beta. SQLite/Parquet remain authoritative. See collection limits and rollback.

It is not production-ready. The active log still rebuilds an in-memory read index, and real SD-card power-cut evidence has not yet been collected. Writable stores can seal live raw into immutable segments, reclaim the active log, and run explicit background maintenance for rollups.

Platform support

FTWDB v0.1 supports Unix targets only. CI tests Ubuntu Linux and macOS. A build for a non-Unix target stops with a clear compile error; FTWDB does not replace directory syncs with no-ops because that would weaken the stated durability contract. The NBD block-device smoke test needs Linux and runs separately from the portable Linux/macOS CI matrix.

Quick start

cargo test --all-targets
cargo clippy --all-targets --all-features -- -D warnings
cargo bench --bench storage
cargo bench --bench energy_compare -- --quick

Generate one deterministic portable workload and run FTWDB against it:

cargo run --release -- generate ./bench-results/workload --sites 1 --days 7 --cadence-seconds 60 --seed 42
cargo run --release -- bench-ftwdb ./bench-results/workload ./bench-results/ftwdb-manual --durability manual

Inspect a database file:

cargo run --release -- inspect ./data.ftwdb

Verify, snapshot, restore, or salvage a directory store:

cargo run --release -- check-store ./energy.ftwdb
cargo run --release -- backup ./energy.ftwdb ./backups/energy-2026-07-21.ftwdb
cargo run --release -- restore ./backups/energy-2026-07-21.ftwdb ./restored-energy.ftwdb
cargo run --release -- salvage ./damaged-energy.ftwdb ./salvaged-energy.ftwdb

Run the local shadow sidecar with sync-on-every-batch durability:

cargo run --release --bin ftwdb-shadow -- ./shadow.ftwdb ./run/ftwdb-shadow.sock

The sidecar protocol is still a draft. Do not make FTW control or production reads depend on it. See the shadow guide and its beta gates below.

Design documents

• Architecture and invariants
• Energy model, plans, and rollups
• Storage format v1
• Immutable segment format
• Persistent rollups and retention
• Integrity checks, backup, restore, and salvage
• FTW shadow sidecar and beta gates
• OSS database research
• Benchmark protocol
• Deterministic energy workload
• Bootstrap benchmark result
• TSBS and robustness result
• Release policy and process
• Roadmap

License

Apache-2.0.