File-System Memory vs Vector Memory for Agents

Separate the memory record from the retrieval method

A directory of Markdown files and an embedding index answer different questions. The directory gives each memory a location and readable contents; the index proposes memories related to a query. A database can store both text and vectors, and a path can identify a database record without representing a file on disk. The important distinction is whether the original memory remains independently readable and editable.

Consider a hypothetical payments team recording why it rejected a larger database connection pool. A record at /memories/payments/connection-pool.md can hold the decision, incident reference and review date. An embedding can help find it when someone asks about checkout stalls. My recommendation is to retain that record even if semantic retrieval becomes the main discovery route. An opaque vector ID alone gives the reviewer too little to investigate.

Paths make corrections explicit, but navigation takes work

With a known path, an agent can request the pool decision directly. An engineer can read the whole note, distinguish the current conclusion from an old experiment and correct the sentence that caused a bad recommendation. Design the memory interface to expose the author, revision and supporting reference alongside the body; a filename alone does not establish whether the decision still applies.

Anthropic's context-engineering guidance describes using lightweight references, including file paths, to load information on demand. It also identifies the trade-off: runtime exploration can be slower and requires suitable tools and guidance. Applied here, a short payments-memory index may help the agent find the pool note without reading every incident.

Directory structure also creates maintenance work. If the same pool decision appears under both payments and database operations, establish which record owns it and make the other entry a reference. Otherwise a direct read can faithfully return a stale duplicate. Browsability helps diagnose that mistake; it does not prevent it.

Establish a lexical baseline you can inspect

Before introducing embeddings, test whether ordinary text retrieval finds the pool note. PostgreSQL documents to_tsvector, query parsing and ranking functions for this purpose. For an illustrative table memory_records with text columns path and body, this query retrieves notes containing both normalized search terms:

SELECT path,
       ts_rank(to_tsvector('english', body),
               plainto_tsquery('english', 'pool timeout')) AS rank
FROM memory_records
WHERE to_tsvector('english', body)
      @@ plainto_tsquery('english', 'pool timeout')
ORDER BY rank DESC, path
LIMIT 5;

PostgreSQL's English configuration normalizes words and removes stop words; full-text matching is therefore different from byte-for-byte matching. Keep exact identifiers in dedicated fields when equality matters. The rank is a lexical score, not a probability that the remembered decision is correct.

PostgreSQL also documents GIN expression indexes for accelerating these searches. Match the query's text-search configuration to the indexed expression. This example assumes the calling service has already restricted access to the appropriate records; the SQL shown is a retrieval baseline, not an authorization implementation.

Vectors help with paraphrases, with two distinct recall limits

A prompt about checkout requests waiting for database connections may describe the pool incident without using its stored terminology. Microsoft documents vector search as useful for conceptual similarity without keyword overlap, while keyword search can work better for identifiers and specialized terms. Whether an embedding finds this particular incident remains something to measure.

Separate representation quality from index approximation. The pgvector documentation says its default nearest-neighbor search is exact; HNSW and IVFFlat indexes trade some recall for speed. Exact nearest neighbors still need not be the memories that resolve the task. They are exact under the chosen vector distance, not under an engineer's judgment of relevance.

For a wrong pool recommendation, record the candidate paths, distances, embedding version and retrieved text revisions. Compare the approximate results with exact search before blaming the embedding model. Preserve readable text and metadata with the vectors: pgvector explicitly supports storing vectors alongside other PostgreSQL data, so semantic retrieval need not make memory opaque.

Budget for navigation, embeddings and stale copies

Compare complete retrieval paths. For browsing, count directory listings, searches and reads before the agent sees the pool decision. For semantic retrieval, count document and query embedding work, index storage, search latency and returned context. Include model inference whether embeddings run locally or through a paid API. Avoid assuming that either approach has lower total cost without measuring the workload.

Use the same context allowance for both. Returning five long incident notes can cost more context than opening one precise decision file. Conversely, several failed path guesses can consume time and tokens before a semantic search would have found a useful candidate. Track retrieval latency separately from the agent's final response time.

Editing the pool note should also trigger an explicit index-maintenance rule. In a design with separately stored chunks, attach the source revision to each chunk and reject outdated results before inserting them into context. Test deletion through both direct reads and search. Removing the readable record while leaving a retrievable copy defeats the purpose of having an authoritative record.

Test a hybrid route against the actual memory questions

Build a small comparison set around the payments example: request the known decision path, ask about pool timeouts, paraphrase the symptom, ask about another service's pool and ask a question the notes cannot answer. Label which memory revisions are useful and which must be excluded. Run browsing, lexical retrieval and vector retrieval against the same corpus and context allowance.

Then test a combined route. Azure AI Search documents parallel full-text and vector retrieval merged with reciprocal rank fusion. That is a concrete implementation of hybrid search, not evidence that every combined system will improve your agents. A simpler application can first honor an explicit path, then search when discovery is needed. Compare those routes before adding more ranking machinery.

Score useful-memory recall, irrelevant context and the final recommendation. A system that retrieves the pool note but repeats its superseded experiment still fails. Include negative cases where returning no memory is preferable. See evaluating agent memory recall for the broader test design, and hybrid versus vector retrieval for the ranking comparison.

How SPYN3 combines paths with optional semantic search

SPYN3's opt-in context layer stores path-addressed memories in its EU database, with context_list, context_read and context_search available to any MCP client. Search uses PostgreSQL full-text search by default, without a model. A workspace can supply its own OpenAI-compatible model key to enable semantic search and memory extraction. Reading needs call:tools; writing needs the separate write:memory permission. Start by enabling Memory & skills in workspace Settings, then assess whether semantic search improves your memory questions.

Choose according to what your agents keep missing

• Start with paths and lexical search when memories are curated decisions, service names are stable and operators need to inspect the exact record an agent used.
• Add vector retrieval when evaluation shows useful memories are repeatedly missed by wording differences, and the improvement justifies embedding and index maintenance.
• Combine them when agents need direct access to named decisions and discovery across differently worded incidents. Keep returned candidates tied to readable records and revisions.

Before choosing, replay one wrong recommendation about the connection pool. Can you identify the record, explain how it entered context, correct it and verify the next retrieval? That repair loop is a more useful acceptance criterion than the presence of either folders or embeddings.