Search stages b (local embeddings) and c (JEV reranker): module cut#

Status: built (cut drafted 2026-11-06, implemented on feat/search-stages). Sections 1 to 14 keep the design text; where the build differs, section 15 names the deviation and wins. Measurements are in section 16. Open points are in section 14. Contract sections: architecture.md §6.5, §7, §15.4; providers and data location: providers.md; purge: redaction.md §2, §4; deviation: abweichungen.md D101.

1. Summary#

Stage a (lexical, synchronous) stays the recall floor. Stage b embeds the query with a local ONNX model (@huggingface/transformers, optional dependency, model files only from disk) and scores it brute force against the vec_rule table in the scope file; a and b are fused by RRF (k = 60). Stage c sends the query and the statements of the top 50 heads of a scope in one call to JEV (jev-1.14.0, TypeSafe AI, USA), keeps heads with a score ≥ 0.5 and orders by that score. Every response names state and reason per stage; b or c not running is never an error. search() becomes async.

2. Starting state (evidence, before the build)#

Callers of search(): src/http/routes/search.ts:16 (handler already async), src/mcp/tools/search-rules.ts:29 (inside the synchronous runTool, src/mcp/result.ts:71) and the test helper test/search/support.ts:4 (used by test/search/*.test.ts, test/fuzz/search.test.ts and others). No other module imports the function; src/approval/queries.ts, src/decision/rejected-similar.ts, src/domain/proposals.ts, src/domain/provenance.ts import only types.

3. Module cut#

Architecture tests (same pattern as the existing import tests): @huggingface/transformers only in transformers-embedder.ts; the string vec_rule only in vector-store.ts and the migration; api.typesafe.ai only in jev-client.ts; ai_usage only in budget.ts and the migration.

4. Interfaces#

// src/search/stage.ts
export interface Candidate { scope: string; ruleId: string; seq: number; version: number; score: number }

export interface StageQuery {
  text: string | undefined;              // request.text; stages b and c need it
  terms: readonly string[];              // as today
  variants: Readonly<Record<string, readonly string[]>>;
  includeProposals: boolean;
}

export type StageOutcome =
  | { state: "ran"; reason: string | null; candidates: Candidate[] }   // b: neighbours; c: scored heads
  | { state: "failed" | "budget"; reason: string };

/** Per request; b embeds the query once in `start`, `run` is per scope. */
export interface StageRequest {
  run(input: { scope: string; db: Db; candidates: readonly Candidate[] }): Promise<StageOutcome>;
}

export interface SearchStage {
  name: "b" | "c";
  /** Non-null = the stage does not run for this scope (off, not-allowed, failed before start). */
  availability(scope: string): StageState | null;
  start(input: { query: StageQuery; deadline: number; signal: AbortSignal }): StageRequest;
}

// src/search/fusion.ts
export const RRF_K = 60;
/** Σ 1 / (k + rank) over the lists that contain the key; rank is 1-based. */
export function rrf<K>(lists: readonly (readonly K[])[], k = RRF_K): Map<K, number>;

// src/search/semantic/embedder.ts
export type EmbedKind = "query" | "document";
export interface Embedder {
  readonly model: ModelSpec;
  /** L2-normalized, length model.dims each. Queries jump the queue ahead of indexing batches. */
  embed(texts: readonly string[], kind: EmbedKind, signal?: AbortSignal): Promise<Float32Array[]>;
}
export type EmbedderLoad =
  | { ok: true; embedder: Embedder }
  | { ok: false; reason: "package-missing" | "model-missing" | "checksum-mismatch" | ModelDirReason | "load-error"; detail: string };
// ModelDirReason: "model-dir-symlink" | "model-dir-owner" | "model-dir-writable" | "model-dir-parent" | "model-dir-unreadable"
export type EmbedderLoader = (spec: ModelSpec, modelDir: string) => Promise<EmbedderLoad>;
/** Pooling done here from token-level output and attention mask, independent of pipeline options. */
export function pool(tokens: Float32Array, dims: number, mask: readonly number[], how: Pooling): Float32Array;

// src/search/semantic/vector-store.ts (single writer of vec_rule)
export interface VectorRow { seq: number; version: number; model: string; dims: number; textSha256: string; vector: Float32Array }
export interface IndexStats { total: number; embedded: number; otherModel: number }
export function writeVectors(db: Db, rows: readonly VectorRow[], now: Date): { written: number; discarded: number };
export function missingVectors(db: Db, spec: ModelSpec, opts?: { force: boolean }): { seq: number; version: number; text: string }[];
export function indexStats(db: Db, spec: ModelSpec): IndexStats;
export function nearest(db: Db, spec: ModelSpec, query: Float32Array, k: number, includeProposals: boolean): { seq: number; version: number; ruleId: string; cosine: number }[];
export function deleteVersionVectors(db: Db, ruleId: string, version: number): number;   // rows removed
export function deleteRuleVectors(db: Db, ruleId: string): number;                      // all versions
export function invalidateVectors(db: Db): void;                                         // drop the matrix cache

// src/search/semantic/indexer.ts
export interface VectorIndexer {
  notify(scope: string): void;                     // after a committed write (debounced 500 ms)
  rebuild(scope: string): { queued: number };       // force: re-embed every current version
  status(scope: string): IndexStats & { running: boolean; lastError: string | null };
  stop(): Promise<void>;
}

// src/search/rerank/jev-client.ts
export const JEV_MODELS = { "jev-1.14.0": { nanoUsdPerInputToken: 42 } } as const;   // 0.042 USD / 1M
export type JevResult =
  | { ok: true; scores: number[]; inputTokens: number; outputTokens: number; attempts: number }
  | { ok: false; reason: "timeout" | "network" | "http"; status: number | null; inputTokens: number | null };
export interface JevClient {
  score(input: { query: string; statements: readonly string[]; deadline: number; signal: AbortSignal }): Promise<JevResult>;
}
export function jevClient(deps: { apiKey: string; model: keyof typeof JEV_MODELS; fetch: typeof fetch; now: () => number }): JevClient;

// src/search/rerank/budget.ts (single writer of ai_usage)
export function monthIn(zone: string, now: Date): string;                               // "YYYY-MM"
export function spentThisMonthUsd(ops: OpsStore, now: Date, zone: string): number;       // also read by `doctor`
export function recordUsage(ops: OpsStore, entry: { now: Date; zone: string; provider: "typesafe";
  model: string; scope: string; inputTokens: number | null; outputTokens: number | null }): void;

HeadHit (src/search/bundle.ts:40) gains rankB, rankRrf, rankReranker (each number | null); buildBundle writes them into rank instead of the three nulls.

5. Stage b#

5.1 Model table (semantic/models.ts)#

Both candidates are covered; the measurement decides which one the docs recommend.

export type Pooling = "cls" | "last_token";
export interface ModelFile { path: string; sha256: string; bytes: number }
export interface ModelSpec {
  id: "bge-m3" | "qwen3-embedding-0.6b";     // config value and vec_rule.model
  repo: string; revision: string;            // Hugging Face repo and pinned commit (40 hex)
  dtype: "q8" | "fp32" | "fp16";
  files: readonly ModelFile[];               // every file transformers.js reads, relative to <repo>/
  pooling: Pooling; dims: number; maxTokens: number;
  queryPrefix: string; documentPrefix: string;
}

revision, files[].sha256 and files[].bytes are taken once from the pinned commit when the model is added (download, hash, record); a unit test checks that every entry has a 40-hex revision and 64-hex hashes. The Qwen instruction text is the exact text of the measurement (src/search/semantic/models.ts); changing it needs a new measurement. maxTokens 512: a rule is short (statement ≤ 500 chars); a long query is truncated by the tokenizer.

5.2 Loading without network#

transformers-embedder.ts does await import("@huggingface/transformers"); ERR_MODULE_NOT_FOUND → package-missing. Before the import it checks the files with verifyModelFiles (size, then SHA-256 streamed, once per start, in the background). Then: env.allowRemoteModels = false, env.allowLocalModels = true, env.localModelPath = model_dir, env.useFSCache = false (names checked against the pinned version); AutoTokenizer and AutoModel.from_pretrained(repo, { dtype, local_files_only: true }). Pooling and normalisation in pool() from the token-level output and the attention mask (CLS = position 0, last token = last position with mask 1), so the result does not depend on which pooling options the pinned version of the library knows. Package version exact in optionalDependencies (no ^); onnxruntime-node comes with it (check pnpm build-script allowance, section 14).

Queue: one inference at a time; queries go before indexing batches; at most 4 waiting queries, more → failed "embedder busy". An inference cannot be cancelled: at the deadline the stage stops waiting, the result is discarded.

5.3 What is embedded and when#

embeddingText = statement + \n + keywords joined by , + \n + relations as kind name. No quote, speaker or source link: the quote is source text (often personal data), JEV was measured on rule texts, and a source redaction then never touches vectors.

Indexing cannot run in the write transaction (async). Proposal: background catch-up.

1. ScopeStore gets an optional onWrite(scope) listener; recordWrite calls it after the commit (the five existing call sites need no change). main.ts wires it to indexer.notify.
2. reindexRule (src/db/fts.ts:84) also calls invalidateVectors(db) (drops the matrix cache; no SQL on vec_rule).
3. The indexer, per scope with b available: missingVectors (current, not rejected, not redacted versions without a row of the configured model, dims and text hash) → batches of 8 → embed(…, "document") → writeVectors. Runs at start for every scope (catch-up after crash, restore, model change) and 500 ms after the last notify.
4. writeVectors writes in one short transaction and discards a row when, at write time, its version is no longer the current version, the rule is rejected or redacted, or the text hash differs from the current embeddingText. So an in-flight embedding can never bring back a redacted text. In the same transaction it deletes older-version rows of the same seq.
5. Search reports gaps (section 9): rows of another model or dimension count as missing.

5.4 Scoring#

nearest() keeps one matrix per scope connection (WeakMap<Db, …> like lexicalIndexes, src/db/fts.ts:64): Float32Array of n × dims plus seq, version, rule_id, status, built from vec_rule joined with current_version (v.version = c.version, v.model = spec.id, v.dims = spec.dims). Vectors are normalised, cosine = dot product. Proposals only with include_proposals. Top SEMANTIC_POOL = 50 by cosine, ties by seq. Memory ≈ 4 KB per rule (10 000 rules ≈ 40 MB); 10 000 × 1024 multiply-adds ≈ a few ms.

6. Storage#

6.1 Scope migration 0005-vectors.ts (section 14.4)#

CREATE TABLE vec_rule (                  -- derived; rebuilt from rule_version at any time
  seq INTEGER NOT NULL REFERENCES rule(seq),
  version INTEGER NOT NULL CHECK (version >= 1),
  model TEXT NOT NULL CHECK (length(model) BETWEEN 1 AND 64),     -- ModelSpec.id
  dims INTEGER NOT NULL CHECK (dims BETWEEN 1 AND 8192),
  text_sha256 TEXT NOT NULL CHECK (length(text_sha256) = 64),     -- of embeddingText()
  vector BLOB NOT NULL CHECK (length(vector) = dims * 4),         -- float32 little-endian (explicit), L2-normalized
  embedded_at TEXT NOT NULL CHECK (embedded_at GLOB '????-??-??T??:??:??*Z'),
  PRIMARY KEY (seq, version));
CREATE INDEX vec_rule_model ON vec_rule (model, dims);

No WITHOUT ROWID (rows of 4 KB). Not append-only (no trigger), so backup verify (not-ahead-of-live, export-hashes) ignores it; not part of the export. The BLOB is written and read as explicit little-endian; the vector store checks os.endianness(), and on a big-endian host stage b is failed (the service still starts, stage a runs).

6.2 Ops migration 0004-ai-usage.ts#

CREATE TABLE ai_usage (
  month TEXT NOT NULL CHECK (month GLOB '[0-9][0-9][0-9][0-9]-[0-9][0-9]'),   -- in config.zone
  provider TEXT NOT NULL CHECK (provider IN ('typesafe')),
  model TEXT NOT NULL,
  scope TEXT NOT NULL,
  calls INTEGER NOT NULL DEFAULT 0 CHECK (calls >= 0),
  unmetered_calls INTEGER NOT NULL DEFAULT 0 CHECK (unmetered_calls >= 0),   -- timeout/network: tokens unknown
  input_tokens INTEGER NOT NULL DEFAULT 0 CHECK (input_tokens >= 0),
  output_tokens INTEGER NOT NULL DEFAULT 0 CHECK (output_tokens >= 0),
  cost_nusd INTEGER NOT NULL DEFAULT 0 CHECK (cost_nusd >= 0),               -- nano-USD, exact integer
  updated_at TEXT NOT NULL CHECK (updated_at GLOB '????-??-??T??:??:??*Z'),
  PRIMARY KEY (month, provider, model, scope));
CREATE TRIGGER ai_usage_grows BEFORE UPDATE ON ai_usage
WHEN NEW.calls < OLD.calls OR NEW.unmetered_calls < OLD.unmetered_calls OR NEW.input_tokens < OLD.input_tokens
  OR NEW.output_tokens < OLD.output_tokens OR NEW.cost_nusd < OLD.cost_nusd
BEGIN SELECT RAISE(ABORT, 'append-only'); END;
CREATE TRIGGER ai_usage_no_delete BEFORE DELETE ON ai_usage BEGIN SELECT RAISE(ABORT, 'append-only'); END;

recordUsage upserts (ON CONFLICT … DO UPDATE SET calls = calls + excluded.calls, …); cost = input_tokens × 42 nano-USD (JEV_MODELS). spentThisMonthUsd = SUM(cost_nusd) / 1e9 for monthIn(zone, now). The month is the calendar month in config.zone (2026-12-31T23:30Z is 2027-01 in Europe/Berlin). Concurrent searches may overshoot the cap by the calls already in flight (cents at most); an unmetered call (timeout) is booked with an estimate of its input tokens like every call without usage (section 15.11). A usage write that fails (SQLITE_BUSY) is kept in memory per ai_usage row, counts against the cap and is written before the next budget check and with the next usage.

7. Flow of search() and time budget#

async search(deps, caller, request)
  t0 = monotonic()
  b = stages.b.start({query, deadline: t0 + SEMANTIC_DEADLINE_MS, signal})   // starts the query embedding (ORT threads)
  per scope: a-hits = runStageA(...)                                        // synchronous, runs while b embeds
  per scope (parallel): b-outcome = availability(scope) ?? await b.run({scope, db, candidates: a-hits})
  per scope: pool = a-hits ∪ b-hits                                         // every a-hit stays (recall floor)
             order = b ran ? rrf([a order, b order]) : a order
             b-only hits: coverage, matched_terms, found_via via LexicalResult.explain(seqs); found_via += "semantic"
             heads = resolve → relation → entity → dedupe (as scopeHeads today)
  c = stages.c.start({query, deadline: now + reranker.timeout_ms, signal})
  per scope (parallel): c-outcome = availability(scope) ?? await c.run({scope, db, candidates: heads[0..50)})
             c ran: keep heads with score ≥ threshold, order by score desc (ties rrf, then seq); heads past 50 drop
             c not ran: min_coverage as today
  top `limit` per scope → interleave → bundles → search_log per scope → abstention

• SEMANTIC_DEADLINE_MS = 1000 (code constant, see section 14), c: reranker.timeout_ms. Worst case ≈ a + 1000 ms + 1000 ms; usual (measured JEV p50 171 ms) a + embedding + ~200 ms.
• rank.b = cosine, 4 decimals, null if b did not find the head; rank.rrf = RRF sum, 6 decimals, null if b did not run; rank.reranker = JEV noul, 3 decimals, null if not sent or c did not run.
• c sends per scope one request: state.request = request.text (or the terms joined by spaces), questions keyed "0"…"49" (positions, no rule ids or scope names), instructions.rule = the head's statement only, with the criteria of the measurement (fixed text in jev-client.ts). Retry once on 429/529 if Retry-After (or 200 ms) still fits before the deadline; no retry on timeout, network or other status. Any missing or out-of-range answer → failed.
• Before the call: spentThisMonthUsd(…) ≥ monthly_cap_usd → budget, no call. After it: recordUsage (also for a failed call: unmetered_calls + 1 when tokens are unknown).
• Cross-scope aggregation (response contract unchanged): stages.x.state = ran if the stage ran in at least one scope, else the state of the first scope; duration_ms = the longest; reason = null when all scopes agree on a null reason, else "<scope>: <reason>; …" in request order. search_log.stages of each scope holds that scope's states.
• Reasons carry counts and limits only, never query or rule text (they land in search_log).

Async callers: src/http/routes/search.ts:16 → await search(…); src/mcp/tools/search-rules.ts:29 → runToolAsync(…, async () => …) (new in src/mcp/result.ts, the error mapping of runTool moves into a shared function); test/search/support.ts:4 and every test using it → await. SearchDeps gains stages: { b: SearchStage | null; c: SearchStage | null }, ops: OpsStore, monotonic?: () => number.

8. Configuration#

const SemanticModel = z.enum(["bge-m3", "qwen3-embedding-0.6b"]);
const Semantic = z.discriminatedUnion("enabled", [
  z.object({ enabled: z.literal(false) }).strict(),
  z.object({
    enabled: z.literal(true),
    model: SemanticModel.default("bge-m3"),                // stored in every vec_rule row
    model_dir: z.string().min(1).optional(),               // absolute or relative to GROUNDNOTE_DATA_DIR; default <data_dir>/models
  }).strict(),
]);
const Reranker = z.discriminatedUnion("enabled", [
  z.object({ enabled: z.literal(false) }).strict(),
  z.object({
    enabled: z.literal(true),
    provider: z.literal("typesafe").default("typesafe"),
    model: z.enum(["jev-1.14.0"]).default("jev-1.14.0"),   // pinned; a new model is a code change with a re-measure
    monthly_cap_usd: z.number().min(0).max(1000).default(5),
    timeout_ms: z.number().int().min(100).max(5000).default(1000),
    threshold: z.number().min(0).max(1).default(0.5),
  }).strict(),
]);
// search: { …, semantic: Semantic.prefault({ enabled: false }), reranker: Reranker.prefault({ enabled: false }) }
// scopes[]: { name, providers_allowed, reranker: z.boolean().default(false) }

Docs: providers.md "Configuration" gets this shape; next to scopes[].reranker it repeats that query and candidate statements are processed by TypeSafe AI in the USA.

9. States and reasons#

10. Commands#

• groundnote model fetch [--model <id>] [--json] (local, no ops socket; model and model_dir from groundnote.json): downloads https://huggingface.co/<repo>/resolve/<revision>/<path> (https only, also after redirects) into <model_dir>/<repo>/<path>.part, hashes while streaming, renames only on a SHA-256 match; directories 0700, files 0600; a present file with the right hash is skipped. Mismatch → .part deleted, exit 3. groundnote model verify [--model <id>] → exit 3 on any mismatch. The service never downloads.
• Rebuild (route name from architecture §12 kept): POST /ops/vector-index/:scope/rebuild → 202 {scope, model, queued}; unknown scope 400 scope-unknown; b unavailable 409 semantic-unavailable. Re-embeds every current version in place (search keeps serving the old rows until each is replaced). GET /ops/vector-index → [{scope, model, total, embedded, other_model, running, last_error}]. CLI: groundnote vector-index rebuild --scope <s>, groundnote vector-index status [--json].
• groundnote doctor (15.3) shows b readiness per scope and spentThisMonthUsd against the cap.

11. Redaction#

src/redaction/purge.ts calls, inside the purge transaction, deleteRuleVectors(db, ruleId) for every redacted rule (all versions) and invalidateVectors(db); the following VACUUM of the scope file (redaction.md §4) covers the freed pages, secure_delete overwrites them already. Source, entity and link targets leave vec_rule untouched (no quote, speaker or source link is embedded). missingVectors and writeVectors skip redacted rules, so the indexer never re-embeds them. deleteVersionVectors(db, ruleId, version) exists for a single-version redaction (D-R4: not in Phase 1) and for tests.

12. Test plan (no network in tests)#

A vitest setup file (new, in vitest.config.ts) replaces globalThis.fetch with a function that fails the test for any non-loopback URL. The embedder is a FakeEmbedder (deterministic vectors from a word-hash, controllable delay); JEV is a fetch fake returning scripted responses.

A manual eval pnpm eval:semantic with the real model runs only when package and model are present (skipped otherwise, never in CI).

13. Follow-ups for the documentation (done 2026-11-06)#

architecture.md §3, §6.5, §7.8, §11, §12, §15.4; abweichungen.md D2, D3, D101; providers.md; glossary.md (<scope>.vector.sqlite is gone, timeout_ms); threat-model.md (external judge, supply chain); redaction.md §4.

14. Decisions on the open questions (2026-11-06)#

1. Key name: timeout_ms, as confirmed by the coordinator (the CLI doctor reads it); glossary.md line 130 follows. candidates (50) stays a code constant.
2. Empty result after c: abstained.reason gains below-reranker-threshold (additive).
3. min_coverage with b/c: where c ran in a scope, its threshold replaces min_coverage there; without c, min_coverage applies to all heads as today.
4. Migration numbers: scope 0005-vectors.ts (feat/approval-modes holds 0004-approval.ts and merges first), ops 0004-ai-usage.ts. The redaction worker counts from 0006.
5. SEMANTIC_DEADLINE_MS: code constant, 1000 ms.
6. Docker image: stays lexical-only; stage b needs the optional package and a fetched model, documented in providers.md.
7. Channels: c runs on every channel of a scope where it is switched on; providers.md recommends it for proactive mail matching.
8. Default model: bge-m3 (model defaults to it when enabled: true). Measured on the synthetic set (300 rules, 180 queries, stage b alone, q8): bge-m3 Recall@5 0.97, MRR 0.89, query p50 12 ms, +1.3 GB RSS, 587 MB on disk, MIT; qwen3-embedding-0.6b (with query instruction) Recall@5 0.92, MRR 0.79, p50 39 ms, +1.5 GB RSS, 625 MB, Apache-2.0.

15. Deviations of the build from the cut#

1. ops belongs to the stage, not to SearchDeps. RerankStage receives the ops store in its constructor; SearchDeps gains only stages: { b, c } and monotonic?.
2. RerankSearchStage carries timeoutMs (search.reranker.timeout_ms), so search() sets the deadline of stage c from the stage, not from a constant.
3. JevResult has a fourth failure reason invalid (answer parsed but unusable: missing or out-of-range score, wrong shape); it maps to failed with invalid answer.
4. Stage c is always built (not null when off). It answers availability() with off or not-allowed itself, so the aggregation in search() has one path.
5. Equal score, pool order. Where two heads have the same c score (or b cosine), the order of the pool decides, not seq.
6. Package import before verifyModelFiles. transformers-embedder.ts first imports the package (package-missing is cheap to detect), then hashes the model files.
7. One reason without a scope: prefix when all scopes agree. The prefix form of section 7 applies only if the scopes differ in state or reason.

Added during the build (built; the code is authoritative):

8. Minimum cosine 0.45 for b neighbours. A neighbour below 0.45 does not enter the pool. The threshold comes from the cosine distribution in section 16 (at 0.45, 88 % of no-answer queries still have a neighbour, 96 % of answerable ones keep the relevant rule).
9. The request to c is cut to 4000 characters of query text.
10. Response body of JEV at most 1 MiB; larger → failed.
11. Calls without usage are booked with an estimate (characters / 4 as input tokens), and every HTTP attempt counts, also a retry, so the cap cannot be undercut by a provider that omits the field.
12. Proposals never go to c. Heads with status proposed are appended after the c-ranked heads in a+b order; their rank.reranker is null. Foreign text from a proposal thus never reaches the external judge.
13. A stage that throws is failed with internal error. The search answers with the result of a (or a+b); no exception reaches the caller.

16. Measurement (pnpm eval:semantic, eval/semantic/results/latest.md)#

Synthetic set: 300 rules, 180 queries, bge-m3 (q8), limit 10, through the real search(); the variants are a, ab (a + b, RRF) and abc (+ JEV).

The largest gains from b are in paraphrase (R@5 0.30 → 0.54) and cross_lingual (0.48 → 0.85); c lifts paraphrase to 0.84 and long_mail to 0.89. JEV: 180 calls, 1 441 694 input tokens, 0.0606 USD for the run. Peak memory with the model loaded was 6.6 GB RSS in the eval process.

Real data (74 rules, 52 queries): a Hit@5 0.82, Recall@5 0.75, MRR 0.66, false alarms 11/12; ab Hit@5 0.88, Recall@5 0.82, MRR 0.77, false alarms 12/12.

False alarms (a no-answer query that still returns a head) come from stage a already when min_coverage is 0; b adds no filter, and only stage c lowers them (synthetic: 16/16 → 5/16). For b alone the minimum cosine of section 15.8 is the lever.

Scale of nearest() (random unit vectors, 1024 dims, top 50, 40 queries):

Linear extrapolation: nearest() passes 100 ms at about 98 000 rules per scope. From there, consider an index extension (sqlite-vec) instead of the JS brute force.

Known limit: matrix rebuild after every write. Each write to a rule or a vector of a scope drops its matrix cache, and the next search rebuilds it synchronously on the event loop: 409 ms at 20 000 and about 1 s at 50 000 rules (table above). In that time the process answers no other request. An incremental update of the cached matrix is not built.

Model comparison bge-m3 against qwen3-embedding-0.6b: see section 14.8 (bge-m3 is the default).