Redaction, crypto-shredding and encryption at rest#

Status: proposed (2026-11-05), binding after approval. Authoritative English design for erasure ("Schwärzen statt Löschen"), per-scope backup keys, encryption at rest and groundnote doctor. It aligns with the German contract (hq datenmodell.md, sections "Schwärzen statt Löschen" and "Verschlüsselung und Krypto-Shredding"; api-vertrag.md, B "Schwärzen" and G; hq commit 3b17f78d) and lists every departure in Deviations. It supersedes the erasure proposal in security/data-at-rest.md ("Erasure despite append-only": an ops command groundnote redact with a free-text reason) and keeps that file's decision on volume encryption after re-evaluating SQLCipher (section 10). Migration numbers are provisional; they take the next free numbers when a slice lands.

0. Decisions in brief#

1. Terms#

• Target: what a person redacts, one ref without version: rule/<id> (all versions), source/<id>, entity/<id> (all versions), link/<id>.
• Tombstone: the redaction row of one target. Seal: the redaction_seal row that closes it; a tombstone is unsealed only while its own INSERT statement runs.
• Redacted value: the fixed value a content column takes (section 3.2).
• Purge: removal of derived copies (FTS, lexical tables, search log telemetry, WAL, free pages).
• Journal: the content-free copy of every tombstone in ops.sqlite (redaction_log), used to replay tombstones into a restored scope file.

2. What is removed and what stays#

Dependents. The preview offers further targets; the person chooses, each becomes its own tombstone in the same request: every link with the target as from_ref or to_ref (preselected, its reason and evidence usually describe the target), and for a source every rule citing it (offered, not preselected: the statement may be fine without the quote).

Derived data purged in the same transaction (section 4): rule_fts, rejected_fts, lex_word, lex_stem, lex_tri, lex_doc rows of a redacted rule (the rule leaves search entirely; a rule that only lost its quote is re-indexed without the quote); the entity_fts row of a redacted entity; in search_log the columns terms, variants, stages become '[]' in every row whose refs names a target, and for source and entity targets also in every row whose terms or variants contain a normalised token (≥ 3 characters) of a removed name, alias, title, sender or speaker. search_log.input_sha256, refs, feedback stay (refs and hashes only). Stage b keeps its vectors in the scope file (table vec_rule, D101, built; design in search/stages-b-c.md §11). When a rule is rejected, its vectors are deleted (and the indexer skips rejected rules, so they are not re-embedded). The purge transaction calls deleteRuleVectors(db, ruleId) (rows of every version of a redacted rule) and invalidateVectors(db) (in-memory matrix); the scope file's VACUUM covers the freed pages. deleteVersionVectors(db, ruleId, version) removes the rows of one version (for D-R4 later). Source, entity and link targets leave vec_rule untouched, because quote, speaker and source link are never embedded; the background indexer never re-embeds a redacted rule. This replaces the earlier VectorIndex.remove(scope, seqs) of a separate vector file.

Known residue (not content, stays by design): ids and refs. A mail id is mail:<RFC 5322 message-id> and the thread is the root message-id; both are keys referenced everywhere and cannot be removed. Message-ids are usually random, but a sender's domain or local part can occur (open question 6).

3. Schema#

3.1 Tombstone tables (scope migration 0005-redaction.ts)#

CREATE TABLE redaction (
  seq INTEGER PRIMARY KEY,
  id TEXT NOT NULL UNIQUE,                                -- <scope>-t<nnnn>, never reused
  target TEXT NOT NULL CHECK (target GLOB 'rule/*' OR target GLOB 'source/*'
                              OR target GLOB 'entity/*' OR target GLOB 'link/*'),
  category TEXT NOT NULL CHECK (category IN ('erasure-request','confidential','capture-error','other')),
  decided_by TEXT NOT NULL CHECK (decided_by GLOB 'person:*'),   -- grant id
  decided_by_identity TEXT NOT NULL,                      -- email from the verified credential
  redacted_at TEXT NOT NULL CHECK (redacted_at GLOB '????-??-??T??:??:??*Z'),
  CHECK (target NOT GLOB '*@*'));                         -- whole targets only (D-R4)
CREATE UNIQUE INDEX redaction_once ON redaction(target);
CREATE TRIGGER redaction_in_scope BEFORE INSERT ON redaction
WHEN NEW.id IS NOT (SELECT scope FROM meta) || '-t' || printf('%04d', NEW.seq)
BEGIN SELECT RAISE(ABORT, 'redaction-id-not-in-scope'); END;

CREATE TABLE redaction_seal (redaction_seq INTEGER PRIMARY KEY REFERENCES redaction(seq));
CREATE VIEW redaction_open AS SELECT r.target FROM redaction r
  WHERE NOT EXISTS (SELECT 1 FROM redaction_seal s WHERE s.redaction_seq = r.seq);

CREATE TRIGGER redaction_no_update BEFORE UPDATE ON redaction BEGIN SELECT RAISE(ABORT, 'append-only'); END;
CREATE TRIGGER redaction_no_delete BEFORE DELETE ON redaction BEGIN SELECT RAISE(ABORT, 'append-only'); END;
CREATE TRIGGER redaction_seal_no_update BEFORE UPDATE ON redaction_seal BEGIN SELECT RAISE(ABORT, 'append-only'); END;
CREATE TRIGGER redaction_seal_no_delete BEFORE DELETE ON redaction_seal BEGIN SELECT RAISE(ABORT, 'append-only'); END;

A tombstone may name a target that does not exist in this file (replay into an older restored state, section 7): it then blocks the target from being created again, for example a feeder re-delivering the redacted mail. Whether a new redaction names an existing target is checked in code (404 not-found).

The same migration rebuilds search_log (as 0002 did) to add the operations redact and preview_redaction to its CHECK, and replaces link_end_once (below).

3.2 Redacted values#

The CHECKs of migration 0001/0002 forbid NULL and '' in several content columns (statement 10–500 characters, quote ≥ 20, segment text non-empty, entity name 1–200, reason non-blank). Rebuilding six append-only tables to relax them would need foreign_keys = OFF outside a transaction and dropping their guards; instead each column has one fixed redacted value that satisfies the existing CHECKs:

REDACTED_TEXT = '[redacted, see tombstone]' (25 characters, ASCII). It is never indexed: ftsRow maps it to '', and a redacted rule has no index rows at all. Because a redacted source's excerpt now contains the marker, a new proposal citing it is stopped by the insert guard (redacted), not by the quote check.

The per-column rules live in one frozen TypeScript table REDACTION_COLUMNS in src/db/migrations/scope/0005-redaction.ts; the trigger SQL in that migration is generated from it once and stored as a literal. test/db/redaction-schema.test.ts asserts that the literal equals the generator's output and that every TEXT column of every append-only table is classified as either frame (unchanged) or content (one rule above), so a future column cannot slip through unclassified.

3.3 The single exemption#

Apply trigger (the one transition; owner: schema):

CREATE TRIGGER redaction_apply AFTER INSERT ON redaction BEGIN
  UPDATE source SET title = CASE WHEN title IS NULL THEN NULL ELSE :M END, sender = NULL, link = NULL,
         excerpt = CASE WHEN excerpt IS NULL THEN NULL ELSE :M END,
         fulltext_sha256 = CASE WHEN fulltext_sha256 IS NULL THEN NULL ELSE '' END
   WHERE NEW.target = 'source/' || id;
  UPDATE source_segment SET text = :M, speaker = NULL WHERE NEW.target = 'source/' || source_id;
  UPDATE source_attachment SET filename = :M, external = NULL,
         text = CASE WHEN text IS NULL THEN NULL ELSE :M END WHERE NEW.target = 'source/' || source_id;
  UPDATE rule_version SET quote = :M, speaker = NULL, source_link = NULL WHERE NEW.target = 'source/' || source_id;
  UPDATE rule_version SET statement = :M, quote = :M, speaker = NULL, relations = '[]', keywords = '[]',
         reason = :M, source_link = NULL WHERE NEW.target = 'rule/' || rule_id;
  UPDATE decision SET reason = :M WHERE NEW.target = 'rule/' || rule_id AND reason IS NOT NULL;
  UPDATE entity_version SET name = :M, aliases = '[]', evidence = NULL, reason = :M WHERE NEW.target = 'entity/' || id;
  UPDATE link SET reason = CASE WHEN reason IS NULL THEN NULL ELSE :M END, evidence = NULL WHERE NEW.target = 'link/' || id;
  UPDATE review SET reason = :M WHERE reason IS NOT NULL AND (target = NEW.target      -- link/<id>
         OR substr(target, 1, length(NEW.target) + 1) = NEW.target || '@');         -- entity/<id>@<n>
  INSERT INTO redaction_seal (redaction_seq) VALUES (NEW.seq);
END;

(:M is the REDACTED_TEXT literal; ids are [a-z0-9-], so prefix comparisons are exact.)

Guard triggers. Every <table>_no_update trigger of source, source_segment, source_attachment, rule_version, decision, entity_version, review is replaced by <table>_redaction_only, and link_end_once by link_update_guard. Each aborts with append-only unless all of the following hold for the row:

1. every frame column is unchanged (NEW.c IS OLD.c);
2. an unsealed tombstone (redaction_open) covers the row with a target kind that is allowed to change it (table in section 2);
3. every content column of that target kind equals its redacted value of OLD, and every content column of other target kinds is unchanged.

Example, rule_version:

CREATE TRIGGER rule_version_redaction_only BEFORE UPDATE ON rule_version WHEN NOT (
  NEW.rule_id IS OLD.rule_id AND NEW.version IS OLD.version AND NEW.kind IS OLD.kind
  AND NEW.source_id IS OLD.source_id AND NEW.timestamp_s IS OLD.timestamp_s AND NEW.said_at IS OLD.said_at
  AND NEW.valid_from IS OLD.valid_from AND NEW.valid_until IS OLD.valid_until
  AND NEW.supersedes IS OLD.supersedes AND NEW.superseded_by IS OLD.superseded_by
  AND NEW.status IS OLD.status AND NEW.changed_by IS OLD.changed_by AND NEW.changed_at IS OLD.changed_at
  AND (
    (EXISTS (SELECT 1 FROM redaction_open WHERE target = 'rule/' || OLD.rule_id)
      AND NEW.statement IS :M AND NEW.quote IS :M AND NEW.speaker IS NULL AND NEW.relations IS '[]'
      AND NEW.keywords IS '[]' AND NEW.reason IS :M AND NEW.source_link IS NULL)
    OR
    (EXISTS (SELECT 1 FROM redaction_open WHERE target = 'source/' || OLD.source_id)
      AND NEW.statement IS OLD.statement AND NEW.relations IS OLD.relations
      AND NEW.keywords IS OLD.keywords AND NEW.reason IS OLD.reason
      AND NEW.quote IS :M AND NEW.speaker IS NULL AND NEW.source_link IS NULL)))
BEGIN SELECT RAISE(ABORT, 'append-only'); END;

link_update_guard allows the existing end-once transition (only valid_to from NULL to a value) or the redaction of reason/evidence under an open link/<id> tombstone, nothing else. All *_no_delete triggers stay untouched; DELETE is allowed only on derived tables.

Why "same transaction" holds. A trigger cannot see transaction boundaries, so the binding is structural: the tombstone is unsealed only during its own INSERT statement, because the apply trigger seals it as its last step. No later statement, in this or any other transaction, ever sees an open tombstone, so after the statement every update aborts again. Sealing early (a stray INSERT INTO redaction_seal) only closes the window; a seal for a missing tombstone fails the foreign key. The TypeScript caller adds the purge and the search-log scrub inside the same db.transaction; if anything fails, the tombstone and all content changes roll back together.

3.4 Writes to redacted targets#

BEFORE INSERT guards raise redacted (HTTP 409 redacted) for: a rule_version of a redacted rule or citing a redacted source; a decision on a redacted rule; an entity_version of a redacted entity; a review of a redacted entity or link; a link with a redacted endpoint; a source, source_segment or source_attachment of a redacted source. On POST /v1/sources the guard surfaces as the generic 409 source-conflict (D66: a conflict reveals nothing). Outside the approval channel a redacted rejected rule stays 404 (invariant 8 wins).

3.5 Property tests (test/db/redaction-triggers.test.ts, fast-check 4.10.2)#

• P1 For any committed state and any UPDATE on any append-only table, with arbitrary column subsets and values (including the redacted values themselves), the statement aborts with append-only.
• P2 After INSERT INTO redaction for a random existing target, every covered row equals redact(OLD) column by column, every uncovered row of every table is byte-identical to the snapshot before, and every table has the same row count.
• P3 DELETE, INSERT OR REPLACE and UPSERT on append-only tables abort, also while a redaction is applied (with recursive_triggers = ON).
• P4 A second tombstone for the same target fails (redaction_once); a seal without tombstone fails; redaction_open is empty after every committed statement.
• P5 Every insert listed in 3.4 aborts with redacted once its target has a tombstone.
• P6 The classification test of 3.2 (no unclassified TEXT column).

4. Purge, secure delete and VACUUM#

Always on (slice R1). src/db/connection.ts sets secure_delete = ON (freed cells and pages are overwritten with zeros) and temp_store = MEMORY on every connection, scope and ops, service, CLI, verify and restore handles. temp_store matters because VACUUM and large sorts otherwise write a temporary copy of the database to the OS temp directory, which may lie outside the encrypted volume. Both pragmas are per connection, not stored in the file.

FTS5. A plain FTS5 delete leaves the tokens in older index segments until a merge. Migration 0004-secure-delete.ts sets INSERT INTO <t>(<t>, rank) VALUES('secure-delete', 1) on rule_fts, rejected_fts, lex_word, lex_stem, lex_tri and entity_fts (persistent in <t>_config); LexicalIndex.open sets it again whenever it recreates the lex_* tables. After the migration commits, the runner executes once wal_checkpoint(TRUNCATE), VACUUM, wal_checkpoint(TRUNCATE) (outside the migration transaction; a new afterCommit: "vacuum" flag of the migration), so free pages written before the pragma was on are gone.

Per redaction (src/redaction/purge.ts), inside the transaction: delete the derived rows (section 2), re-index rules that only lost their quote (reindexRule), then INSERT INTO <t>(<t>) VALUES('optimize') on every touched FTS5 table, then the search-log scrub, then drop the connection's cached LexicalIndex. After commit: wal_checkpoint(TRUNCATE); if it reports busy (a verify snapshot still reads), retry five times with 200 ms back-off, then record purge_state = 'pending' and retry every minute in the background. Then VACUUM and wal_checkpoint(TRUNCATE) again, and purge_state = 'purged'.

Full VACUUM rather than incremental: incremental vacuum needs auto_vacuum = INCREMENTAL from file creation and only returns trailing free pages; it adds nothing to confidentiality on top of secure_delete. Full VACUUM rewrites the file from live content only, which removes residue from any earlier pragma state, at O(file size) on files of a few tens of MB.

Deviation from section 4 of the architecture ("no manual checkpoints, no VACUUM"): both run only after migration 0004 and after a redaction.

Litestream. Litestream holds a read transaction to control checkpoints, so our TRUNCATE stays busy while it runs, and it keeps copies of the pages in its local metadata and in the replica. With backup.kind = litestream the redaction completes logically (tombstone, content, indexes) and answers purge: "litestream"; the physical purge of the live file happens at Litestream's next checkpoint, and the replica keeps the old pages until its retention ends. The operating guide gets a "reseed" procedure (stop Litestream, delete the scope's prefix and local metadata, start again: a new snapshot without history). This is one more reason why copy is the default for scopes with personal data.

5. Approval channel flow#

Links to the preview page sit on the rule page (rule, its source), on the entity and link views of "Structure changes". A service identity on these routes is 403 channel-not-allowed (channel table); a missing decide right is 403 channel-not-allowed; a foreign scope 403 scope-not-allowed.

Preview token (stateless, like the CSRF token): base64url(payload) + "." + base64url(HMAC-SHA256(GROUNDNOTE_SESSION_SECRET, "redaction-preview\0" + payload)), payload {v: 1, scope, primary, offered: [ref], stamp: {<ref>: {rows, max_version, max_decision}}, identity, exp} with exp = now + 15 minutes. The stamp is counts and version numbers only; append-only means any change shows up there. Compared in constant time.

// src/redaction/schemas.ts (owner: src/redaction/**)
export const RedactionCategory = z.enum(["erasure-request", "confidential", "capture-error", "other"]);
export const RedactionTarget = z.union([RuleRef, SourceRef, EntityRef, LinkRef]); // refs without @version
export const RedactionRequest = z.object({
  preview: z.string().min(1).max(4096),
  targets: z.array(RedactionTarget).min(1).max(50),   // ⊆ {primary} ∪ offered, must contain primary
  category: RedactionCategory,
  confirm: RedactionTarget,                           // must equal primary
  rotate_key: z.boolean().default(false),
}).strict();
export const Redacted = z.object({
  ref: Ref,
  redacted: z.object({ at: Instant, category: RedactionCategory }).strict(),
}).strict();                                          // every read path outside approval
export const RedactionResult = z.object({
  tombstones: z.array(z.object({ id: z.string(), target: RedactionTarget }).strict()),
  removed: z.record(z.string(), z.number().int().nonnegative()), // same keys as the preview counts
  purge: z.enum(["purged", "pending", "litestream"]),
  old_backups: z.object({ oldest_state: Day.nullable(), readable_until: Day.nullable(), key_id: z.string().nullable() }).strict(),
  rotation: z.enum(["requested", "not-requested"]),
}).strict();

Check order of POST /v1/redactions: same origin and csrf (403 csrf-failed) → category present and in the enum (400 category-unknown) → schema (400 validation-failed) → token signature, identity equals caller (409 preview-invalid), expiry (409 preview-expired) → confirm equals primary (400 confirmation-mismatch) → targets ⊆ primary ∪ offered (400 validation-failed) → scope check of every target (pipeline 5.2) → stamp unchanged (409 preview-outdated) → no tombstone yet (409 already-redacted) → journal in ops.sqlite (section 7) → scope transaction (section 3.3 and 4) → post-commit purge → 200 RedactionResult. search_log gets one row with operation redact (refs: the tombstone ids and targets).

old_backups: oldest_state = today − backup.retention_days, readable_until = the day the oldest state containing the content expires (the tombstone day + retention), key_id = the scope's current key. rotate_key: true records a rotation request in the journal; rotation itself is an operator step (section 9), which the result page and doctor name.

Read paths. Outside the approval channel a redacted rule, source or entity reads as Redacted everywhere (read, list, provenance, resolve by id, links); it is never a search hit (it has no index rows). A redacted link keeps type and ends; reason and evidence are omitted and redacted is added. Bundles count redacted entries without title or excerpt. Only the approval channel shows who redacted.

6. Module placement and architecture tests#

Architecture tests (test/arch/architecture.test.ts):

• AT1 (extended) the import closure of src/mcp/** contains no file of src/redaction/**.
• AT2c files of src/redaction/** other than replay.ts and schemas.ts are imported only by src/http/routes/redactions.ts and src/approval/**; replay.ts only by src/main.ts and src/backup/restore.ts.
• AT6 the static import closure of src/ops/** contains no file of src/redaction/** except replay.ts and schemas.ts (the dynamic import() of main.ts for serve is excluded, as for AT1).
• AT3 (extended) INSERT INTO redaction occurs only in src/redaction/redact.ts, src/redaction/replay.ts and src/db/migrations/**; redaction_seal only in migrations; rejected_fts writers additionally src/redaction/purge.ts; AT5 (entity FTS writers) additionally src/redaction/purge.ts.
• AT4 (extended) no MCP tool name matches /confirm|reject|revoke|review|decide|approve|redact|erase|purge|tombstone/.
• AT7 the ops app registers no route matching /redact/; the CLI has no command redact.

7. Journal and restore replay#

Ops migration 0003-redaction-keys.ts (owner of every ops table: src/db/ops-store.ts):

CREATE TABLE redaction_log (                       -- append-only by trigger, no content
  id TEXT PRIMARY KEY, scope TEXT NOT NULL, seq INTEGER NOT NULL, target TEXT NOT NULL,
  category TEXT NOT NULL, decided_by TEXT NOT NULL, decided_by_identity TEXT NOT NULL,
  redacted_at TEXT NOT NULL, rotate_requested INTEGER NOT NULL CHECK (rotate_requested IN (0,1)),
  UNIQUE (scope, seq));
CREATE TABLE redaction_purge (                     -- mutable state
  id TEXT PRIMARY KEY REFERENCES redaction_log(id),
  state TEXT NOT NULL CHECK (state IN ('journaled','purged','pending','litestream')), updated_at TEXT NOT NULL);
CREATE TABLE scope_key (                           -- ids and public recipients, never identities
  scope TEXT NOT NULL, key_id TEXT NOT NULL, recipient TEXT NOT NULL CHECK (recipient GLOB 'age1*'),
  created_at TEXT NOT NULL, rotated_at TEXT, discarded_at TEXT,
  PRIMARY KEY (scope, key_id));
CREATE UNIQUE INDEX scope_key_current ON scope_key(scope) WHERE rotated_at IS NULL;
-- backup_state gains last_copy_key_id, last_copy_sha256 (last_backup_at exists since ops 0003, architecture 4.2)

Scope and ops are separate WAL files, so one transaction cannot span both. Order: journal first (state = 'journaled'), then the scope transaction, then purged/pending. A crash after the journal leaves a decided but unapplied redaction; replay applies it at the next open. The journal is therefore the intent record, and the scope file converges to it.

Replay (replay.ts): on every scope open, before the scope answers (it is 503 scope-locked until then), every journal entry of the scope that is missing in the file is inserted with its original id, seq, person, category and time; the apply trigger does the rest, then the purge. groundnote restore runs the same replay on the temporary file before it hard-links the target, so a restored file never lands without its tombstones. If ops.sqlite itself comes from an older state, the journal is incomplete; only a discarded key protects then (section 9), which is why rotation is recommended after an erasure request.

8. Export and backup behaviour#

• Export (src/export/**): redacted columns appear with their redacted values in history.jsonl; redaction and redaction_seal rows are exported (tombstones present); INDEX.md lists a redacted rule as redacted (<date>, <category>) without statement; <id>.md carries front matter id, version, status, redacted: {at, category} and no body. FTS tables are not exported (unchanged). lex_doc/lex_meta are derived and move to NOT_EXPORTED.
• Backup verify: export-hashes names a rule redacted after the backup's newest change as exception redacted-since-backup; new check tombstones-subset: every tombstone in the backup is in the live journal (else fail). The count checks are unaffected (no row is deleted).
• Encrypted files (section 9): copies are <backup.directory>/scopes/<scope>.sqlite.age and <backup.directory>/ops.sqlite.age; exports are <scope>-<YYYY-MM-DD>.tar.gz.age. Both are readable without Groundnote: age -d -i <key> f.tar.gz.age | tar xz.

9. Crypto-shredding with per-scope age keys#

Library: age-encryption (typage, Filippo Valsorda; 0.3.1, BSD-3-Clause, published 2026-08-28; pure TypeScript on @noble/* and @scure/base, ESM, Node 20+). Reasons: the file format is the age standard, so an operator decrypts copies and exports with the age or rage CLI without Groundnote; X25519 recipients let every encrypting path use the public recipient only; no native build. libsodium (libsodium-wrappers 0.8.4, ISC; sodium-native 5.2.0, MIT, native) offers the primitives but no file format: Groundnote would define its own container that no standard tool reads. The documented typage API works on whole Uint8Arrays; files above backup.max_plain_bytes (default 1 GiB) fail with backup-too-large instead of being buffered. Streaming is used if the pinned version offers it (checked in R5).

Key source. GROUNDNOTE_KEY_DIR (environment, an absolute path; required as soon as one scope exists). Layout: instance/<key-id>.key for ops.sqlite copies, scopes/<scope>/<key-id>.key per scope; each file one line AGE-SECRET-KEY-1…, mode 0600 (0400 accepted), owner the service user; directories 0700 (0500 accepted). The directory must lie outside the data directory, backup.directory and every export target, and none of them inside it. key_id = k + first 12 hex characters of SHA-256 of the recipient string. Identities are never logged, never in an error, never in /v1/status; key_id may be. Files rather than environment variables because rotation needs several identities per scope, and the environment is inherited by the Litestream and restic children the runner spawns and is visible in docker inspect. The operator keeps a second copy of every identity in a vault, or a lost disk makes all backups unreadable.

The service only reads the key directory (it may be a read-only mount). Keys are created by the CLI, which writes the file (O_EXCL, 0600) and then registers the public recipient over the ops socket; the service checks that it can read the matching identity and records it in scope_key, so it stays the only writer of ops.sqlite.

Encrypting paths. groundnote backup run [--scope <s>] (ops socket; scheduling stays with the operator's timer): VACUUM INTO a temporary file in a .backup-* directory inside the data directory, encrypt to the scope's current recipient (exactly one recipient, so discarding it shreds), write next to the target and rename (mode 0600), remove the temporary file, update backup_state. GET /v1/export/:scope answers the archive encrypted to the scope recipient, or to caller-given X25519 recipients (recipient=age1…, at most 3; open question 2); the CLI writes .tar.gz.age. fetchBackup decrypts with every non-discarded identity of the scope into the private verify directory; a plaintext copy (SQLite format 3\0 header) is refused with copy-not-encrypted.

Rotation and "discard old states" (CLI, ops socket):

1. groundnote key rotate --scope <s>: new identity, registered as current; the previous key gets rotated_at. New copies and exports use only the new key.
2. groundnote backup run --scope <s> and groundnote backup verify --scope <s>: a copy encrypted to the new key passes the restore probe.
3. groundnote key discard --scope <s> --key <old-id>: shows what becomes unreadable (every copy and export of the scope encrypted before rotated_at, that is every backup state older than the rotation, also content unrelated to the erasure) and needs --confirm <old-id>. The service refuses (409 discard-needs-verified-backup) unless the key is not current and a verify newer than rotated_at passed on a copy encrypted to the current key. It records discarded_at; the CLI deletes the file. The operator destroys the vault copy (procedure; not verifiable).
4. Without an erasure request a rotated key becomes discardable when rotated_at + backup.retention_days has passed; doctor then warns.

After an erasure-request redaction the recommendation is rotate and discard; the alternative is waiting until readable_until. Exports a reader encrypted to their own recipient are outside Groundnote's reach. Litestream replicas are not encrypted with the scope key; shredding there means deleting the prefix and trusting the provider.

10. Encryption at rest: volume encryption mandatory, SQLCipher not adopted#

Decision: the operator must place data/ on an encrypted volume (LUKS/dm-crypt, FileVault, BitLocker, ZFS native encryption, or an encrypted cloud block volume). at_rest.kind must be configured (volume); doctor checks it (section 11). sqlcipher stays a reserved value that the config schema rejects in Phase 1.

SQLCipher evaluation (facts retrieved 2026-11-05 via pnpm view and the GitHub API):

Reasoning: the binding check passes, but with the key on the same host SQLCipher protects against nothing the volume does not already cover in this threat model, it excludes Litestream, and it puts a single-maintainer native fork and a second secret into every path. Backups and exports, the paths that leave the host, are encrypted by age (section 9). Revisit when Litestream is dropped or supports it, when a deployment cannot control its volume, or when the binding gains a second maintainer. Migration path then: swap the package (same API), PRAGMA cipher = 'sqlcipher', legacy = 4, key per connection, offline re-encryption of existing files.

11. groundnote doctor#

groundnote doctor [--json] runs in the CLI, reads files read-only and asks the running service over the ops socket (new GET /ops/status) for live connection state. Each check reports ok, warn or fail with a remedy. Exit code 0 without fail, 3 with any fail, 1 if it cannot run (config unreadable). /v1/status shows the same results without paths.

12. Configuration and environment contract#

// src/config/file.ts (owner); groundnote.json is written by the operator
at_rest: z.discriminatedUnion("kind", [
  z.object({ kind: z.literal("volume"), attested_at: Day.optional() }).strict(),
]),                                                   // required: startup refuses without it (open question 5)
keys: z.object({ rotation_days: z.number().int().min(30).default(365) }).strict(),
backup: /* existing */ .extend({
  retention_days: z.number().int().min(1),            // the operator's restic/Litestream retention = erasure deadline
  max_plain_bytes: z.number().int().min(1).default(1 << 30),
  bucket_encryption_attested_at: Day.optional(),      // litestream only
}),
doctor: z.object({
  max_backup_age_hours: z.number().int().min(1).default(26),  // open question 1
  max_probe_age_hours: z.number().int().min(1).default(26),
}).strict(),
// src/config/env.ts: GROUNDNOTE_KEY_DIR (absolute path, never inside GROUNDNOTE_DATA_DIR)

13. Slice plan#

Order: R1 → R2 → R3 → R4; R5 after R1 (independent of R2–R4); R6 after R3 and R5; R7 last. Every slice ships with its tests; pnpm check green.

The canary used below: a token such as qzxv-<uuid without dashes> written into every content column of section 2 (statement, quote, excerpt, segment, attachment text and filename, title, sender, speaker, entity name and alias, link reason and evidence, decision and review reason), searched through the API so it also lands in search_log terms, and chosen so its trigrams occur nowhere else in the fixture.

14. Deviations#

Provisional numbers D-R1…; they get the next free D numbers when merged into abweichungen.md.

15. Glossary additions#

Merged into glossary.md, section "Redaction and encryption".

16. Open questions#

1. doctor backup age: 24 h as requested, or 26 h as in the hq data model (no flapping with a daily job)?
2. Export recipients: may a reader request an export encrypted to their own age recipient (readable by them, outside crypto-shredding), or only to the scope key?
3. Right: reuse decide, or a separate grant right redact (person only) so not every decider can erase?
4. Breaking: once R5 lands, plaintext copy backups are refused and operator .backup scripts must switch to groundnote backup run. Accepted?
5. Breaking: at_rest becomes required in groundnote.json (startup refusal). Accepted, or a warning first?
6. Message-ids and thread ids stay as identifiers even when they contain a sender's name or domain. Accepted?
7. Litestream with personal data: accept the reseed procedure (drops replica history), or refuse redaction when backup.kind = litestream?

17. Sources (retrieved 2026-11-05)#

• npm registry via pnpm view: better-sqlite3-multiple-ciphers 13.0.3, better-sqlite3 13.0.3, @journeyapps/sqlcipher 6.0.0, @signalapp/better-sqlite3 9.0.13, age-encryption 0.3.1, libsodium-wrappers 0.8.4, sodium-native 5.2.0 (versions, licenses, publish times, engines).
• GitHub release v13.0.3 of m4heshd/better-sqlite3-multiple-ciphers (dependency versions, prebuilds, Node-API) and its docs/compilation.md (compile options incl. SQLITE_ENABLE_FTS5).
• FiloSottile/typage README (API: Encrypter, Decrypter, generateIdentity, identityToRecipient; Node 20+).
• Litestream issue #777 "Add SQLCipher (Encrypted SQLite) Support to Litestream", closed 2025-10-11; Litestream configuration reference as cited in security/data-at-rest.md.