Agent Skills Assess

agentsplatformsecurityagent-skillsskill-packsplatform-engineeringdevopsthreat-modelingcontext-engineeringsupply-chain-securityskill-evaluation

Overview

Agent Skills are reusable, filesystem-based capability packages that give AI agents task-specific instructions, workflow context, reference documents, scripts and templates. Each skill carries metadata plus Markdown instructions in a SKILL.md file, optionally accompanied by scripts and resources, and is loaded through a progressive disclosure pattern: advertise skill names, load instructions, read resources, run scripts (Microsoft Agent Framework). Anthropic introduced the idea in October 2025 and published SKILL.md as an open standard on 18 December 2025; OpenAI formally adopted the format in Codex roughly two months later, and by March 2026 around thirty tools were reading the same directory layout (Reactify, Luc Segbedzi).

What changed this quarter is the shape of the ecosystem rather than the format. The skill supply has broadened out of coding assistance into operations and security: cross-agent DevOps/DevSecOps, Kubernetes, CI/CD, GitOps, policy and SRE skill packs (ops-engineering-skills), a platform engineering handbook packaged as a skill with a plugin layer (platform-skills v1.0.0), and dedicated threat-modelling skills (alpha-omega-security/threat-model). That makes the practical distinction a governance one: curated internal skills — owned, reviewed, versioned and scoped to your estate — versus reusable community skill packs, which are best read as reference implementations and starting points, not dependencies you install unexamined.

Agent Skills stay in assess because the packaging layer is settled while the trust and quality layers are not. Registry-level studies find that most indexed "skills" are not installable reusable skills at all, and that almost none carry outcome evidence (State of Agent Skills 2026), while security research has repeatedly measured a quarter to a third of published skills containing exploitable patterns (arXiv 2601.10338, Cloud Security Alliance). Build a small, owned skill library and instrument it; do not treat community packs as a supply chain yet.

Adoption Signals

  • The format is stable in a second major vendor stack: Agent Skills for .NET and Python in Microsoft Agent Framework have both left experimental preview and ship as production APIs with governance controls (.NET, Python).
  • Client coverage is broad and cross-vendor: Claude Code, OpenAI Codex, ChatGPT, GitHub Copilot, Cursor, VS Code, Gemini CLI, JetBrains Junie, Kiro, Goose, Snowflake, Databricks and others read the same skill directories (Reactify, Luc Segbedzi).
  • Ops and platform skill packs are now first-class: cross-agent skills for DevOps, DevSecOps, cloud, Kubernetes/platform engineering, CI/CD, GitOps/Argo, policy, security scanning, SRE, compliance, MLOps and AgentOps, built once for Claude Code, Copilot, Codex, Cursor and Gemini CLI (ops-engineering-skills).
  • Platform engineering content is being released with real release engineering around it: Platform Skills v1.0.0 covers eight domains (Kubernetes, OpenShift, Argo CD, Flux CD, AWS, Azure, Terraform, GitHub Actions) and ships CI workflows for structure validation, markdown/YAML/Terraform linting, secret scanning, action SHA pinning and automated releases (platform-skills v1.0.0).
  • Security workflows are being packaged as skills too, including an agent skill for producing threat models of open-source projects (alpha-omega-security/threat-model).
  • Curated engineering skill libraries have become mainstream reference material — one production-grade engineering skill collection has ~49k stars, 30 contributors, an MIT licence and tagged releases mapping skills to spec/plan/build/test/review commands (addyosmani/agent-skills).
  • Enterprise and vendor publishing continues: organization-wide provisioning and default-enabled skills for Team and Enterprise plans plus stock finance, legal and HR plug-ins in February 2026 (Agentman), and domain vendors shipping their own packs — Atlassian, Databricks, GitHub, Google, Microsoft, OpenAI and Stripe, with Socotra as the first insurance core (Socotra).

Risks

  • Measured supply-chain risk, not theoretical. A study of 42,447 collected skills analysed 31,132 and found 26.1% with at least one vulnerability across 14 patterns (prompt injection, data exfiltration, privilege escalation, supply chain), with data exfiltration at 13.3%, privilege escalation at 11.8% and 5.2% showing high-severity patterns strongly suggesting malicious intent (arXiv 2601.10338); a separate audit put the flaw rate at 36.82% (Cloud Security Alliance).
  • Bundled scripts materially raise exposure. Skills that ship executable scripts are 2.12x more likely to contain vulnerabilities than instruction-only skills (OR=2.12, p<0.001) (arXiv 2601.10338) — a direct problem for ops and platform packs, which are script-heavy by nature.
  • The skills layer is its own attack surface. Threat taxonomies for the format and a top-ten threat list for practitioners both stress that best practices are immature, trust boundaries are unclear, and early adopters carry elevated risk (arXiv 2604.02837, Akamai).
  • Registry signal quality is weak. In a quality-ranked sample of 1,000 indexed repositories, only 9.4% showed strict agent-skill evidence, 29.4% were really domain workflow tools or ecosystem dependencies, 2.2% were lists rather than installable skills, and only 120 had any reported agent outcome (State of Agent Skills 2026).
  • Evaluation is still anecdotal. As skill repositories grow, quality assessment is typically "ask the agent to try the skill and watch a demo", which yields no reproducible signal on change (AEVAL) — meaning skill drift in encoded APIs, build steps or platform conventions can go undetected.
  • Runtime behaviour needs policing, not just review. Emerging defences monitor execution traces against behavioural policies declared in skill specifications and operator constraints, catching violations that span multiple tool calls (awesome-agent-skills-security); static review of SKILL.md alone does not cover value flows across a workflow.
  • Licence and provenance gaps. Roughly 18% of an already quality-filtered sample had no declared licence (State of Agent Skills 2026), which is a blocker for redistributing community packs inside a regulated codebase.

Pros & Cons

Advantages

  • Skills are a cheap, version-controllable context-engineering primitive: a folder with a SKILL.md, optional reference documents and scripts, loaded progressively so agents gain domain procedure without permanently inflating every prompt.
  • The format is genuinely cross-vendor, with the open standard published in December 2025, Codex adopting it soon after, and 25-32 client tools reading the same directory layout by mid-2026, plus stable production APIs in Microsoft Agent Framework for .NET and Python.
  • The skill supply now reaches well beyond coding prompts into platform engineering, DevOps/DevSecOps, GitOps, threat modelling and vendor-published domain skills, so teams can study working reference material instead of authoring every workflow from scratch.

Disadvantages

  • Large-scale security analysis of 42,447 published skills found 26.1% carrying at least one vulnerability and 5.2% showing high-severity patterns suggesting malicious intent, with script-bundling skills 2.12x more likely to be vulnerable than instruction-only ones.
  • Registry quality is poor: in a 1,000-repository analysed sample only 9.4% had strict reusable-skill evidence and just 120 skills had any reported agent outcome, so stars and topics are not readiness signals.
  • Evaluation practice is still immature — testing a skill today is often a developer watching one demo — so drift in encoded APIs, build steps or platform conventions can silently degrade agent behaviour until deterministic skill testing is wired into CI.

Recommendation

Assess Agent Skills by building a small curated internal library rather than by installing community packs. Pick two or three high-repetition workflows where procedure, not capability, is the bottleneck — release checks, repository onboarding, migration runbooks, Terraform or Kubernetes change patterns, threat-model drafting — and author narrow skills with precise activation descriptions, short instructions, explicit references and named owners in your own version control. Prefer instruction-only skills where possible and treat any bundled script as code that needs review, pinning and secret scanning, given the measured 2.12x vulnerability multiplier on script-bundling skills (arXiv 2601.10338). The release tooling in platform-oriented packs is a good template here: structure validation, markdown/YAML/Terraform linting, secret and action-pinning scans, and automated versioned releases (platform-skills v1.0.0).

Treat reusable community skill packs as reference material with a quarantine step. Ops, DevSecOps, platform and threat-model packs (ops-engineering-skills, alpha-omega-security/threat-model) and large curated engineering libraries (addyosmani/agent-skills) are worth mining for structure and content, but fork what you use, strip what you do not, check the licence, and re-review on every upstream bump. Assume a meaningful fraction of anything pulled from a marketplace is unsafe or mislabelled (Cloud Security Alliance, State of Agent Skills 2026).

The exit criteria from assess are organisational, not technological. Before promoting Agent Skills to trial, put in place a skill inventory with owners, peer review on every skill change, deterministic evaluation in CI instead of demo-based judgement (AEVAL), sandboxing plus runtime policy monitoring of skill-driven tool calls (awesome-agent-skills-security), a documented threat model for the skills layer itself (Akamai), and a retirement path for stale workflows. Keep using typed tools and MCP servers to expose capabilities and skills only to describe how to use them; avoid broad, vague skills that behave like hidden system prompts. Where you need a stable API surface for skills inside application code, the now-GA .NET and Python skill APIs are a reasonable place to prototype (Microsoft Agent Framework).

Sources

Overview

Agent Skills are reusable, filesystem-based capability packages that give AI agents task-specific instructions, workflow context, examples, scripts, templates, and other resources. Anthropic introduced them as organized folders that agents can discover and load dynamically, with each skill centered on a SKILL.md file containing YAML frontmatter and Markdown instructions (Anthropic Engineering). The format has since been published as an open standard: a skill directory must contain SKILL.md with at least name and description metadata, and may include scripts/, references/, assets/, or other supporting files (Agent Skills specification).

The main design pattern is progressive disclosure. Agents initially load only each skill's name and description, activate the full SKILL.md when the task matches the description, and then read referenced files or execute bundled scripts only when needed (Agent Skills overview). This makes skills a practical context-engineering primitive: teams can capture migration playbooks, review checklists, data-quality procedures, brand rules, and tool-specific workflows without permanently inflating every prompt.

Skills complement, rather than replace, tool protocols such as MCP. MCP exposes resources and actions, while skills describe how an agent should use tools, files, and procedures in a repeatable sequence. Anthropic explicitly frames skills as a way to teach agents more complex workflows involving external tools and software, making them useful as the procedural layer above typed tools, MCP servers, and agent orchestration (Anthropic Engineering).

Adoption Signals

  • Anthropic supports Agent Skills across Claude.ai, Claude Code, the Claude Agent SDK, and the Claude Developer Platform, with pre-built skills for document workflows such as PowerPoint, Excel, Word, and PDF (Claude API Docs).
  • The public anthropics/skills repository provides example skills, document skills, templates, and the Agent Skills specification, with guidance for installing skill bundles into Claude Code as plugins (Anthropic skills repository).
  • The open standard client showcase lists adoption across developer tools and coding agents including Gemini CLI, OpenCode, OpenHands, Cursor, Goose, GitHub Copilot, VS Code, Claude Code, Claude, OpenAI Codex, Databricks Genie Code, Snowflake Cortex Code, Kiro, Roo Code, Tabnine, and others (Agent Skills client showcase).
  • Ecosystem coverage reported partner-built skills from Atlassian, Figma, Canva, Stripe, Notion, and Zapier, plus enterprise management features for centralized provisioning and workflow control in Team and Enterprise environments (VentureBeat).

Risks

  • Skill supply-chain risk is the primary concern. Skills should be treated like third-party software because they can contain instructions, scripts, assets, dependencies, and external references that may not match their stated purpose (Claude API Docs).
  • Prompt injection and hidden-instruction risk increases when a skill reads external URLs, bundled documents, images, or generated files. Anthropic recommends installing skills only from trusted sources and auditing less-trusted skills before use, especially for network calls, file access patterns, scripts, and bundled resources (Anthropic Engineering).
  • Runtime portability is incomplete despite the open format. Claude's documentation notes that custom skills do not automatically sync across Claude surfaces, and runtime constraints differ across claude.ai, Claude API, Claude Code, AWS, and Microsoft Foundry, including network access and package installation behavior (Claude API Docs).
  • Governance must be explicit. The specification includes validation guidance such as skills-ref validate ./my-skill, recommends keeping SKILL.md under 500 lines, and defines optional fields such as compatibility and experimental allowed-tools, but it does not by itself enforce ownership, review, evaluation, sandboxing, signing, or lifecycle management (Agent Skills specification).
  • Retention and deployment settings matter. Claude's documentation states that Agent Skills are not eligible for Zero Data Retention and that skill definitions and execution data are retained under Anthropic's standard data retention policy, so regulated teams should review retention and deployment surface before using hosted skills with sensitive workflows (Claude API Docs).
  • Skill drift can silently degrade agent behavior. If skills encode old APIs, deprecated build steps, stale architecture decisions, or obsolete business rules, agents may execute outdated workflows with high confidence unless skills are versioned, tested, and retired like shared code.

Pros & Cons

Advantages

  • Package repeatable workflows, institutional knowledge, examples, scripts, and templates into reusable capabilities that agents can load on demand.
  • Progressive disclosure keeps default context small while allowing detailed procedures and executable helpers to be pulled in only when relevant.
  • The open Agent Skills format improves portability across Claude, Codex, Cursor, VS Code, GitHub Copilot, Gemini CLI, Goose, OpenHands, and other agent clients.

Disadvantages

  • Skills expand the agent supply chain: malicious or stale instructions, bundled scripts, dependencies, and external URLs can steer agents toward unsafe actions or data exfiltration.
  • Support differs by client and runtime, so skills that work in one agent may fail or behave differently in another because of filesystem, network, package, and tool-approval constraints.
  • Poorly owned skill libraries can silently encode outdated processes, broken commands, insecure defaults, or organization-specific assumptions that are hard for users to notice.

Recommendation

Trial Agent Skills for high-repetition, high-context workflows where agents need consistent procedural guidance: repository onboarding, migration runbooks, release checks, code review policy, design-system usage, data-quality validation, document generation, and operational playbooks. Start with a small, owned skill library in version control, require peer review for skill changes, validate frontmatter against the open specification, test skills against representative tasks, and scan bundled scripts and resources for secrets, unsafe commands, external network calls, and prompt-injection patterns.

Adopt a layered model: use typed tools or MCP servers to expose capabilities, and use Agent Skills to teach agents when and how to use those capabilities. Avoid broad, vague skills that behave like hidden system prompts. Prefer narrow skills with precise activation descriptions, short SKILL.md files, explicit references, deterministic helper scripts, clear owners, compatibility notes, and a removal path for obsolete workflows. Move from trial to adopt only after the organization has skill inventory, ownership, review, evaluation, sandboxing, and audit logging in place.

Sources