Bring Your Own Model Key for Memory and Search

Decide which memory operations need inference

Before entering a provider key, split the memory service into three operations: finding recorded text, turning conversations into durable facts, and retrieving facts through embeddings. PostgreSQL’s full-text search documentation describes parsing and normalizing words into searchable lexemes. That gives you a useful baseline without sending a search request to a model provider.

For example, a search for a deployment service name may work well with text matching. A question phrased as “who approves production changes?” may benefit from semantic retrieval when the memory says “release sign-off belongs to the platform lead.” Mistral’s embedding documentation explains that text is converted into numerical vectors for semantic search. Whether that improves your own recall needs measurement.

Extraction is a different decision: a model reads selected turns and proposes memories worth retaining. Keep manual recording available while evaluating that behavior. A workspace can need better retrieval without wanting automatic conversation capture; see extracting memories from agent transcripts for the capture decision.

Test extraction and embeddings independently

A successful chat response does not verify an embedding integration. Mistral documents separate /v1/chat/completions and /v1/embeddings endpoints with different request bodies. These synthetic requests exercise both formats. Set MISTRAL_API_KEY through your test environment’s secret mechanism; use no real conversation data in this first check.

curl https://api.mistral.ai/v1/chat/completions \
  -H "Authorization: Bearer $MISTRAL_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "mistral-large-latest",
    "messages": [{
      "role": "user",
      "content": "Extract only the explicit durable fact: The platform team owns release approval. Return JSON with a facts array."
    }],
    "response_format": {"type": "json_object"},
    "max_tokens": 150
  }'

curl https://api.mistral.ai/v1/embeddings \
  -H "Authorization: Bearer $MISTRAL_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "mistral-embed",
    "input": "The platform team owns release approval."
  }'

The first request tests structured extraction output; inspect whether its fact matches the input. JSON mode verifies syntax, not truth. The second should return an embedding in data and usage information. Record the returned model identifiers and vector length. These are direct provider checks, not SPYN3 configuration instructions or proof that a particular memory adapter supports the pair.

Approve the text leaving the memory service

Draw the extraction request with its actual inputs: selected conversation turns, extraction instructions and, if merging requires them, existing memories. Draw embedding requests separately: stored memory text during indexing, and the user’s search text during lookup. Check application logs and retry queues for copies. The provider key identifies an account; it does not describe this data flow.

Mistral’s zero-data-retention documentation illustrates why account settings matter. It distinguishes training opt-out from retention, requires a request for ZDR approval, and covers supported stateless endpoints including chat completions and embeddings. Stateful products such as files and conversations are excluded, and Labs models are excepted. Verify the setting for the actual organization and endpoint rather than inferring it from possession of a paid API key.

For your review, record processing location, retention settings and access to request logs for both operations. An EU memory database alone does not establish where a separate model request is processed. Give extraction inputs and extracted memories distinct deletion rules; memory retention and deletion policies covers those lifetimes.

Treat compatible APIs as a contract to test

A base URL, model identifier and key are connection inputs. Compatibility still needs verification against the operations your memory implementation sends. Ollama explicitly describes its OpenAI compatibility as a subset and documents both chat completions and embeddings. Its local examples also explain that the client’s required key value is ignored by the local server.

That distinction matters when evaluating self-hosted inference: supplying a key field is not evidence that a server enforces authentication. Check the actual endpoint’s authentication and network access before sending memory text. A hosted memory worker also cannot reach your laptop through its own localhost; establish a reachable, authenticated service address.

For each candidate, test extraction output, embedding response shape, input limits and timeout behavior from the machine that will make production calls. Ask which generation and embedding models the adapter selects. Do not assume that a single model identifier supports both tasks.

Budget for prompts, indexing and repeated extraction

Estimate the monthly inference workload before enabling capture across a fleet. Use a worksheet with four quantities: transcript tokens submitted for extraction, generated memory tokens, memory tokens embedded after creation or edits, and query tokens embedded during recall. Multiply each by the chosen provider’s applicable rate. Add retries, reprocessing and any separate infrastructure charges.

For an illustrative pilot, 20 engineers producing eight captured sessions per working day across 20 days generate 3,200 extraction jobs. At an assumed 6,000 input tokens each, that is 19.2 million extraction input tokens before retries or existing-memory context. These are planning assumptions, not measured usage or a provider quote.

Watch repeated merging: resending an expanding memory file can make extraction input grow even when session length stays constant. Set worker concurrency and retry limits, and verify whether provider budget settings actually reject requests or only alert. Compare cost per useful retained fact and cost per successful recall, alongside the invoice total.

Separate key rotation from model migration

Recommend a dedicated provider credential for the memory workload, held by the service making inference calls. Agent laptops should receive the credential needed to call the memory service. Keep the provider credential out of memory files, shared skills and copied MCP configurations; that avoids distributing the extraction account’s authority to every reader.

To rotate a key, install its replacement, test extraction and embedding calls, then revoke the old credential. Confirm failures do not produce endless retries or unbounded transcript retention. During a provider outage, your own design should preserve text lookup and manual recording where possible. Choose a finite deadline for queued raw turns.

Changing embedding models requires a separate migration plan. Treat vectors from different models as incompatible unless compatibility is established, even when their lengths match. Retain source text, build the replacement index, and compare retrieval against a fixed query set before switching. Label vectors with the model and preprocessing version in your implementation so a credential change cannot silently become an index change.

Choosing model-backed memory in SPYN3

SPYN3’s Memory & skills setting is opt-in and off by default. Search uses Postgres full-text search without a model. A workspace can supply its own OpenAI-compatible base URL, model and key for semantic search and memory extraction; SPYN3 supplies no model. Automatic capture is a separate opt-in setting and requires that key. Memory writes require write:memory. Raw turns are deleted after extraction. Start with text search, then add the workspace’s chosen provider when you are ready to enable model-backed memory.

Make the provider choice with a bounded pilot

Use synthetic conversations containing a release owner, a corrected preference, an expired workaround and a sentence explicitly marked as temporary. Check which facts extraction retains, then ask both exact-term and paraphrased queries. Include a deliberately irrelevant query to see whether semantic recall returns misleading context.

Before rollout, demonstrate four outcomes: the intended provider receives only the approved inputs; a rejected credential produces a bounded failure; the extraction result preserves corrections; and semantic retrieval improves the queries that text search missed. Inspect request metadata without retaining conversation bodies as debugging evidence.

Choose acceptance thresholds before comparing providers. If extraction adds unreliable facts or semantic search does not improve recall enough to justify its cost, keep the simpler path. Use evaluating agent memory recall to turn the provider decision into a repeatable check when a model, endpoint or extraction prompt changes.