Local and Self-Hosted Models With Hosted MCP Tools

Keep inference, orchestration and tool execution separate

Consider an engineer asking a laptop model to explain an order’s status using a company API. Three components participate: the inference server generates the next assistant message; the agent host manages the conversation and executes tool requests; the hosted MCP server accesses the order system. Running inference locally changes the first component. It does not require moving the order connector onto the laptop.

The MCP tools specification defines discovery through tools/list and invocation through tools/call. Those operations belong in the host’s MCP client. A model that can generate function arguments still needs that client; pointing its inference API at an MCP URL will not connect the two protocols.

Choose a host that supports your actual combination of inference backend, tool calling, Streamable HTTP and endpoint authentication. If you are writing the host, implement that adapter explicitly. The same arrangement works when inference runs on your own GPU server: account for the additional host-to-inference network boundary.

Prove local tool calling before adding company access

Ollama’s tool-calling guide demonstrates qwen3 with function definitions and returned tool messages. With that model installed locally and Ollama running, use a synthetic order lookup to check the inference side. This probe declares a function; it does not execute it or contact an order service.

curl http://localhost:11434/api/chat \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "qwen3",
    "stream": false,
    "messages": [{"role": "user", "content": "Look up order demo-42."}],
    "tools": [{
      "type": "function",
      "function": {
        "name": "lookup_order",
        "description": "Look up a demonstration order by its ID",
        "parameters": {
          "type": "object",
          "properties": {"id": {"type": "string"}},
          "required": ["id"]
        }
      }
    }]
  }'

The chat API accepts a tools array; stream: false makes this diagnostic response easier to inspect. Check for a structured tool request with the expected function name and order ID. A sentence saying it will perform the lookup is insufficient. Repeat with a nonexistent ID and an unrelated question to test whether the model invents arguments or calls unnecessarily.

For an explicitly local deployment, Ollama documents disable_ollama_cloud: true in ~/.ollama/server.json, or OLLAMA_NO_CLOUD=1 in the server environment. Restart Ollama after changing it. Its default listener is 127.0.0.1:11434; connecting hosted tools does not require exposing that listener to the internet.

Probe the hosted endpoint independently

Next, test MCP without involving the model. Use the endpoint’s HTTPS URL and a bearer token issued for that endpoint, supplied through your runtime’s secret mechanism. The following discovery probe follows the MCP Python SDK v2 documentation. It prints the actual names and schemas, including additional pages, so you can choose a narrowly scoped lookup before introducing autonomous execution.

import asyncio
import os
import httpx2
from mcp import Client
from mcp.client.streamable_http import streamable_http_client

async def main():
    async with httpx2.AsyncClient(
        headers={"Authorization": "Bearer " + os.environ["MCP_TOKEN"]},
        timeout=httpx2.Timeout(30.0, read=300.0),
    ) as http:
        transport = streamable_http_client(
            os.environ["MCP_URL"], http_client=http
        )
        async with Client(transport) as client:
            cursor = None
            while True:
                page = await client.list_tools(cursor=cursor)
                for tool in page.tools:
                    print(tool.name, tool.input_schema)
                if page.next_cursor is None:
                    break
                cursor = page.next_cursor

asyncio.run(main())

Use a matching v2 SDK installation: these imports and schema attributes differ from older examples. The transport documentation puts custom headers and timeouts on httpx2.AsyncClient. Streamable HTTP can return JSON or SSE; let the SDK handle those responses and protocol compatibility.

This probe assumes an existing bearer credential. OAuth sign-in requires a client authorization flow, including discovery and token handling; an arbitrary API key is not an OAuth implementation. Keep endpoint credentials out of model messages. For the storage boundary, see where agent OAuth tokens should live.

Connect the two sides with an explicit tool loop

Once both probes work, implement the following sequence in the host. Use the discovered schema rather than reusing the synthetic order definition: the deployed tool might require a different identifier or additional parameters.

1. Select an allowed MCP tool. Build a function definition with its description and map its input_schema to Ollama’s function.parameters. Maintain an explicit mapping from the model-facing alias to the actual MCP name.
2. Send the user conversation and selected function definitions to local inference. Preserve the returned assistant message, including its tool calls, in conversation history.
3. For each proposed call, reject unknown aliases, validate arguments against the selected schema, and apply the host’s execution policy. Invoke the mapped tool with the SDK’s client.call_tool(name, arguments).
4. Check result.is_error and convert supported result content into an Ollama message with role: tool, the corresponding tool_name and textual content. Send the updated conversation back to inference.

Handle every returned call, or reject unsupported multi-call responses explicitly. Set a maximum number of calls and inference rounds. Define handling for structured results and non-text content rather than silently dropping them. Preserve errors as failures; an unavailable order API must not become a successful lookup with an empty status.

Trace what leaves the machine, including memory

A local prompt can produce a remote lookup containing an order ID, email address or fragment copied from a file. The hosted endpoint receives those arguments and returns company data to the host, which places it in the model’s context. Review those payloads separately from whether the model provider receives prompts. Also inspect host tracing and any configured fallback inference backend.

Remote memory is another outbound path. A host that writes a summary to a memory tool has exported that summary even if every inference token was generated locally. Start with explicit memory reads and user-approved writes. Send only the selected record or summary, and decide separately whether extraction or embeddings use a local model or an external API. For that decision, see bringing your own model key for memory and search.

Keep company credentials behind the connector, but treat the host’s endpoint token as valuable: it still authorizes access. Enforce allowed sources and operations at the service. The MCP specification treats tool annotations as untrusted unless supplied by trusted servers; a read-only description alone cannot enforce permissions.

Using SPYN3 with local inference

SPYN3 provides one workspace MCP endpoint at https://api.spyn3.com/agent/mcp; it does not proxy model inference. Your host authenticates with a SPYN3 API key or MCP sign-in while upstream credentials stay envelope-encrypted in the workspace. Scope a key to chosen sources and call:tools; writes require separate grants. Opt-in memory supports full-text search without a model. Semantic search and extraction require the workspace’s own model key. Connect the host to this endpoint while keeping its inference connection local.

Test the complete path before expanding the catalog

Run a small acceptance set against the exact local model, host and hosted endpoint you intend to deploy. Keep the order lookup narrow enough that you can inspect every proposed argument and returned record.

• Grounded answer: ask for an order’s status, verify the tool request, and compare the answer with the returned status. Change the fixture and confirm the answer changes.
• Denied access: request an ungranted source or write operation. Confirm the service denies it even when the model proposes a plausible call.
• Outage: make the endpoint unavailable. The host should report that it cannot verify the status and stop within its configured deadline.
• Context pressure: test a longer conversation and larger lookup response. Record wrong-tool choices, invalid arguments, total calls and answer accuracy.
• Revocation: revoke the endpoint credential and verify that another call fails. Check that no upstream credential was needed locally.

Measure local inference time separately from remote tool time. Expand the catalog only after these cases behave acceptably; a functioning HTTP connection proves transport compatibility, while these checks test the agent’s actual use of company data.