Evaluate an AI data connector across source coverage, incremental sync, parsing, stable identity, permission propagation, tenant isolation, retrieval quality, provenance, agent contracts, observability, security and export. Require test evidence for your own data and failure modes—not only a connector logo.
Ingestion and freshness
- Supported objects, types and limits are documented.
- Initial and incremental sync are defined.
- Deletes, renames and permission changes propagate.
- Retries and backfills are observable.
- Stable IDs survive updates.
- Parsing failures are recoverable.
Permissions and security
- Tenant isolation exists at storage and retrieval.
- Source ACLs are enforced at query time.
- Credentials are encrypted and minimally scoped.
- Revocation latency is tested.
- Content is treated as prompt-injection risk.
- Audit events identify actor, source and outcome.
Read why metadata filtering matters before accepting vague “secure search” claims.
Retrieval and evidence
- Exact identifiers and paraphrases are evaluated.
- Filters cover workspace, source, type, time and access.
- Results include stable source identity.
- Snippets and locations support verification.
- Empty and conflicting evidence are honest.
- A repeatable evaluation suite exists.
Agent and operational fit
- REST or MCP matches the client strategy.
- Tool schemas, pagination and errors are model-usable.
- Usage, latency, errors and freshness are monitored.
- Breaking changes are versioned.
- Data and metadata can be exported.
- Deletion, retention and incidents have owners.
Score each item proven, partial, absent or not applicable and attach evidence.
Built for the next step
Evaluate SPYN3 against its real surface
Developer guides document the current endpoint, permissions, generated tools and failure behaviour.
Open developer guidesSPYN3 is a consolidated MCP layer: one hosted endpoint in front of every system you connect — MCP servers, Postgres, REST APIs, Google Drive — with tools foldered by source and read/write permissions per key. Switching agents means moving one URL. Free plan available; current capabilities are documented on the product facts page.
Sources
- Microsoft Learn: document-level access control — permission metadata and query-time enforcement. Verified 13 Sep 2026
- Anthropic: prompt injection defenses — indirect prompt-injection risk and layered defenses. Verified 13 Sep 2026
- Anthropic: writing effective tools for agents — namespacing, response design and tool evaluations. Verified 13 Sep 2026