Roadmap proposals#

Status: proposals, 2026-11-05. Each item is backed by a measurement on a private test set built from one operator's real meetings, mails, chats and documents (74 rules, 52 queries, 7 scenarios; the data never left that machine and is not in this repository). Small sample: one case moves a scenario score by up to 0.33, so read the numbers as direction, not as benchmarks.

Baseline on that set, Phase 1 stage a as built: recall@5 0.63, MRR 0.52, nDCG@5 0.51; every query without a matching rule still returned hits (12 of 12).

1. Vocabulary-aware term selection for stage a#

Problem. Section 7.1 keeps the first 12 terms of a text. In real mails and transcripts the first terms are greetings, names and small talk; the subject words come later. Stage a misses rules whose words appear only after the twelfth token (recall 0.40–0.50 in the "same rule, different wording", "mail thread" and "German/English" scenarios).

Proposal. Pick the 12 terms of the text that are rarest in the scope's own index (document frequency over rule_fts), limited to terms that prefix-match an indexed token. Deterministic, no model, nothing leaves the server, and the scope file already has the statistics.

Measured. Reference implementation of section 7 (recall 0.60, MRR 0.50; the built stage a scores 0.63 / 0.52), only term selection changed: recall@5 0.60 → 0.85, MRR 0.50 → 0.68. That matches what adding local embeddings with RRF gave on the same set (a+b: 0.85 / 0.72), at zero cost.

Contract impact. Changes section 7.1; needs a contract change first, then search/terms.ts. Risk: terms that are rare because they are misspelt. Mitigation: keep given terms first, as now.

2. A relevance gate with a "why this hit" line before anything proactive#

Problem. Without a threshold every stage answers every query: 12 of 12 queries without a matching rule returned at least one bundle (stage a, a+b alike). For search on demand that is acceptable; for proactive notices (phase 4) it is noise, and noise teaches people to ignore the notice that matters.

Proposal. (a) A gate per scope, set on the scope's own test set (contract already says the threshold is tuned there, not guessed): stage c score when stage c is on, otherwise a similarity threshold for stage b; stage a alone never triggers a proactive notice. (b) Every head carries a why built in code: the matched terms (found_via exists) plus the shortest quote span that contains them, so a person sees in one line why the rule came up.

Measured (first, smaller set of 35 queries). Embeddings plus the JEV reranker at threshold 0.5 cut false alarms on queries without a matching rule from 10 to 2, at recall@5 0.84 (MRR 0.82). Embeddings alone without a threshold: recall 1.00 but 10 false alarms. The trade-off is real and must be tuned per scope.

3. Contradiction check at proposal time#

Problem. Superseding only works if the proposer names the rule to supersede. If a new rule contradicts a confirmed one and nobody links them, both stay valid and search shows both.

Proposal. When a proposal is stored, run a search with its statement and keywords in its scope and show the top confirmed rules on the approval page as "may supersede" next to the existing "possible duplicates". The person decides; nothing is linked automatically.

Measured. When the link exists, Groundnote never showed a superseded or expired rule (0 forbidden hits in the "newer replaces older" and "outdated rule" scenarios, in every stage). Finding the candidate is the hard part: for the replacing rule's query, MRR was 0.83 with stage a and 1.00 with stage a+b.

Not proposed#

Comparison with a flat full-text search over documents and mails (the operator's existing assistant): it could not answer 24 of 52 queries (meetings and chats not indexed); on the 6 queries whose rule came from a document it also holds, it ranked that document in the top five once (MRR 0.05), against 4 of 6 for stage a and 6 of 6 for stage a+b, because it has no notion of a rule. That supports the core design (rules as first-class records with quote and validity), not a new item.