Optional AI providers and data location#

Status. Search stages b (local embeddings) and c (reranker, TypeSafe AI JEV) are built and off by default. Stage b runs only on your server; stage c sends the query and candidate rule statements to TypeSafe AI in the USA. The module cut is in search/stages-b-c.md; the configuration is at the end of this page.

What runs where#

If you do not want any text to leave your server, leave stages b and c off (or use a local model for b). Stage a works fully on its own, and the response always reports stage states (ran, off, failed, budget, not-allowed).

Per-scope gate: providers_allowed#

Each scope in groundnote.json lists the providers it may use:

{ "scopes": [ { "name": "sales", "providers_allowed": ["local"] } ] }

Allowed values: local, voyage, openai, jina, typesafe. A stage whose provider is not in the scope's list is not run for that scope and reports not-allowed. Put scopes with sensitive content on ["local"] or an empty list.

Provider locations#

Each API provider needs your own API key, supplied through the environment, never the config file. Groundnote does not ship a shared key.

Semantic stage b (local embeddings)#

Stage b embeds the query with a local model and compares it with the stored rule vectors of each scope; its hits are fused with stage a (RRF). Every stage-a hit stays in the result.

• Model: bge-m3 is the default (measured on a synthetic set of 300 rules and 180 queries: Recall@5 0.97, about 12 ms per query, about 1.3 GB more memory, 587 MB on disk). The alternative qwen3-embedding-0.6b measured lower (Recall@5 0.92).
• Install: the optional package @huggingface/transformers and the model files. The service never downloads; fetch the files once with groundnote model fetch (checked by SHA-256, groundnote model verify checks again). The Docker image stays lexical-only.
• Without the package or the model only stage a runs; the start logs semantic-unavailable and every response reports stages.b as failed with the reason (for example model bge-m3 not downloaded (groundnote model fetch)).
• Rule vectors are built in the background after each write and at start; groundnote vector-index status shows progress, groundnote vector-index rebuild --scope <s> re-embeds a scope.
• Only statement, keywords and relations are embedded, never the quote, speaker or source link.
• Without the downloaded model, stage b does not run: stage a answers alone and the response says so in stages.b (state: "failed" with the reason), never as an error.

Reranker stage c with TypeSafe AI (JEV): what you should know#

Stage c is off by default (global, optional) but recommended for proactive mail matching, where it cuts false alarms (see Measurement below). Facts as of 2026-11-05, from the provider's terms (Master Customer Agreement of 2026-09-23, Data Processing Addendum of 2026-04-24, Privacy Policy of 2025-11-19, https://docs.typesafe.ai/legal.md). Verify them yourself before production use; terms change.

• Search query and candidate texts are transmitted to TypeSafe AI in the USA and processed there.
• For EU data the provider relies on Standard Contractual Clauses (per the DPA).
• The terms allow the provider to derive and use telemetry (logs, hashes, statistics, classifications) from your data, and the usage right is broadly worded and not time limited. Customer data is not used for training without consent.
• Retention is "as long as necessary", with no fixed period. Zero Data Retention is offered only on the Enterprise plan.
• Proposals never go to stage c. Unconfirmed rule text stays on your server; proposals are appended after the reranked heads in the a+b order, with rank.reranker: null.
• Whether the DPA applies automatically to self-service accounts, and the list of sub-processors, were not verified.
• The service is proprietary and in early access; there is no price guarantee.
• Do not put personal data into rules that are sent for reranking. If you are subject to the GDPR, conclude the DPA yourself.

If you do not accept this, leave stage c off.

Measurement#

Real data, 52 queries (no content published), model jev-1.14.0, threshold 0.5, fan-out of 50 candidates in one request:

Stage c removes most false alarms at a small cost in recall.

Failure, cost and cap#

If the provider fails, times out (1000 ms) or the monthly cap is reached, the search returns the result of a + b and stages.c reports failed or budget; nothing else changes.

Measured cost with 50 short candidates (one or two sentences): about 0.00016 USD per search, computed from the API's reported token usage, not from an invoice. Longer texts cost proportionally more.

The monthly cap (default 5 USD, enforced in code, about 31,000 searches with 50 short candidates) ends stage c for the rest of the month when reached. Measured latency was about 170 ms median and 230 ms p95 from Germany.

Configuration#

Both stages are off unless you switch them on in groundnote.json. Shown with every default:

{
  "scopes": [
    // b needs "local"; c needs "typesafe" and "reranker": true for the scope
    // "reranker": true sends query and rule statements to TypeSafe AI in the USA
    { "name": "support", "providers_allowed": ["local", "typesafe"], "reranker": true },
    { "name": "hr", "providers_allowed": ["local"] }   // "reranker" defaults to false
  ],
  "search": {
    "semantic": {
      "enabled": true,
      "model": "bge-m3",            // or "qwen3-embedding-0.6b"
      "model_dir": "models"         // optional; absolute or relative to GROUNDNOTE_DATA_DIR
    },
    "reranker": {
      "enabled": true,
      "provider": "typesafe",
      "model": "jev-1.14.0",        // pinned; a new model is a code change with a new measurement
      "monthly_cap_usd": 5,         // 0 to 1000
      "timeout_ms": 1000,           // 100 to 5000
      "threshold": 0.5              // heads below it are dropped where c ran
    }
  }
}

Stage c: the search query and the statements of up to 50 candidate rules are processed by TypeSafe AI in the USA. No rule ids or scope names are sent. Switch scopes[].reranker on only for scopes whose rule texts may leave your server (see the terms above).

Recommendation: switch stage c on for the scope that serves proactive mail matching (the proactive channel), where false alarms cost the most; it runs on every channel of a scope where it is on. A scope with sensitive content stays on ["local"].