Untrusted Skills: Reviewing Agent Skills Before Sharing Them

Freeze the bundle before an agent discovers it

Start with an inspection copy outside every directory your agents search for skills. Record its upstream repository, exact commit, publisher and intended task before installing anything. Use a plain text viewer for the first pass. Asking a privileged agent to assess a skill after making it discoverable puts the candidate instructions inside the environment you are trying to protect.

The Agent Skills specification defines a directory with SKILL.md, with optional scripts, references and assets. Its description helps agents identify relevant tasks, and referenced material can load later. Review the description as well as the body: an unrelated skill that advertises itself for every coding task deserves scrutiny.

Inventory every bundled file, including hidden files and symbolic links. Resolve links without executing their targets, and identify anything outside the captured directory. Your review record should distinguish files you inspected from destinations you merely recognized. A familiar repository name does not establish what this particular revision contains.

Trace instructions from the task to the destination

Anthropic's article on equipping agents with skills explicitly recommends auditing less-trusted bundles, including dependencies, scripts and external connections. It warns that malicious skills can direct data exfiltration or unintended actions. Turn that warning into a concrete question for each instruction: what input does this require, which action follows, and where does the result go?

For a release-note skill, reading a selected diff and producing Markdown has a clear purpose. Reading shell profiles, collecting environment variables or uploading the repository to a diagnostic endpoint needs a separate justification. Check examples and templates too: a sample upload command can become the agent's chosen implementation.

• Authority: flag instructions that claim to override the user, suppress confirmation or conceal actions.
• Data: list requested files, environment values, tool responses and generated artifacts.
• Destinations: identify every hostname, mailbox, repository and shared context path receiving data.
• Persistence: inspect requests to change agent rules, install hooks or save instructions into memory.

Reject unexplained transfers rather than accepting a reassuring label such as telemetry. Carry persistence findings into the separate review of how poisoned memory influences later agent sessions.

Follow executable paths and dependencies

Read each script from its entry point through subprocess calls, imports and network requests. Record the effective working directory, files it can overwrite and credentials it expects. A helper named validate.py deserves the same inspection as an installer if both execute with the developer's access.

For this review, require an explanation for dependency installation, downloaded executables and package lifecycle scripts. Prefer an inspectable, fixed dependency set for the trial. A reviewed wrapper that fetches its real implementation from a moving branch leaves that implementation outside the approval.

Write a small data-flow note: selected diff to local parser to Markdown output. Add every actual detour, including error reporting and temporary files. If you cannot complete that note because code is generated, obfuscated or fetched at runtime, hold the skill for clarification. Text searches for suspicious commands can help locate work, but they do not replace following the execution path.

Check what the client grants when the skill loads

Review the candidate against the client that will consume it. Claude Code's skills documentation describes allowed-tools as pre-approval, not a restriction on available tools. Do not read allowed-tools: Read Grep as a sandbox. It also documents shell-command interpolation that runs before skill content reaches the model; inspect those expressions before invoking a candidate.

For a reviewed release-note procedure intended for explicit invocation, this is a minimal Claude Code file at .claude/skills/review-release-notes/SKILL.md:

---
name: review-release-notes
description: Draft release notes from a user-selected diff.
disable-model-invocation: true
---
Read only the diff the user selected.
Draft Markdown release notes in the conversation.
Ask before publishing or changing repository files.

The documented disable-model-invocation field prevents automatic invocation in Claude Code. The body expresses the procedure; it does not enforce filesystem or network boundaries. Keep enforcement outside these instructions. The Agent Skills specification marks allowed-tools experimental, with support varying between implementations, so repeat the client review when sharing the bundle across harnesses.

Exercise the candidate with disposable data

After static review, run the captured revision in a disposable environment with synthetic release diffs, no mounted home directory, no production credentials and no connected company tools. Restrict outbound access to the destinations the trial actually needs. Explicitly invoke the candidate only after those boundaries are in place.

Claude Code's permissions documentation distinguishes file-tool rules from OS enforcement: arbitrary subprocesses can read files indirectly, beyond the coverage of Read rules. It directs users to sandboxing for process-level restrictions. Verify your test environment's actual mounts and network controls rather than assuming a prompt or permission entry isolates the process.

1. Give the skill an ordinary diff and check the proposed notes against it.
2. Include a synthetic instruction in a changed file asking for an unrelated upload. Check whether the agent treats it as input data.
3. Make a required reference unavailable and check whether the skill reports the gap or fetches an unapproved replacement.
4. Inspect tool calls, network attempts, created files and persistent instruction changes.

A failed case blocks promotion until explained. A passed trial supports that revision under those conditions; it does not prove every future run will behave identically.

Approve an exact revision with an owner

Make the approval useful to the next engineer. Record the captured commit or content digest, complete file inventory, reviewer, permitted clients, expected inputs and outputs, external destinations, required permissions and observed test results. Put unresolved questions beside the approval decision. An approval that only says safe loses the assumptions the next operator needs.

Require another review when the description broadens, a reference changes, a dependency moves, a destination appears or a tool grant grows. Compare the entire bundle, including assets and installation files. For externally maintained skills, have a maintainer propose updates rather than allowing every agent to fetch the newest revision independently.

Publish only the reviewed bundle to your team catalog and retain the previous approved version for rollback. Keep catalog write access narrow enough that an ordinary skill consumer cannot silently replace instructions for everyone. The companion guide to maintaining a shared skills repository covers the distribution workflow.

Where SPYN3 fits in distributing reviewed skills

With Memory & skills enabled by an admin, SPYN3 stores shared skills at spyn3://workspace/skills/<name>/SKILL.md. Reading needs call:tools; writing needs the separate write:memory permission. Put only reviewed instructions into the shared catalog and restrict which keys can change them. SPYN3 labels recalled memory as recorded data, scopes access and audit-logs memory writes; it does not detect malicious skills or prompt injection. Keep the bundle review and client execution controls in your release process.

Withdraw a suspect revision and verify the replacement

If a shared skill changes unexpectedly, stop distributing that revision and preserve its files and execution records. Identify which agents loaded it, which tools they could reach and which destinations they contacted. Remove persistent changes it requested. Revoke affected credentials based on actual exposure, and restore the last reviewed bundle.

Restart affected sessions for the replacement trial instead of assuming that changing a catalog entry clears instructions already loaded into a conversation. Re-run the relevant synthetic cases, then document the cause and the review check that will catch a recurrence. The release decision should identify the exact bundle you are sharing and the evidence for allowing its specific actions.