Validate SKILL.md frontmatter in CI - #440
Open
Mosquito1123 wants to merge 2 commits into
Open
Mosquito1123 wants to merge 2 commits into
Mosquito1123 wants to merge 2 commits into
Conversation
scripts/validate-plugins.mjs only validated marketplace.json and each plugin.json, and the workflow's paths filter excluded **/SKILL.md, so skill frontmatter was never checked and could drift silently. - Add schemas/skill.schema.json describing SKILL.md frontmatter (required `name` and `description`; name in kebab-case; unknown keys allowed so plugins can keep harness-specific options). - Parse and validate every SKILL.md frontmatter with Ajv, and require the frontmatter `name` to match the skill's folder name. - Add **/SKILL.md and scripts/** to the workflow paths filter and install the `yaml` parser. Fix the violations this exposes: - agent-compatibility/skills/check-agent-compatibility: quote a description containing ": " so the frontmatter parses (closes cursor#381). - third_party/x/skills/x-api-mcp-guide: name "X MCP guide" -> "x-api-mcp-guide" (closes cursor#269). - pstack/skills/poteto-mode: name "Poteto Mode" -> "poteto-mode" (closes cursor#237). - pstack/skills/make-bot-ui: name "Make Bot UI" -> "make-bot-ui". - third_party/x/skills/x-chat: name "X Chat" -> "x-chat". - third_party/x-money/skills/x-money-guide: name "X Money guide" -> "x-money-guide". The validator is green on the updated tree and reports each of these when reintroduced.
The validator only iterated the marketplace entries, so a plugin directory added without a marketplace entry would pass CI and never be listed. Walk the repository for `.cursor-plugin/plugin.json` files and fail when one is missing from `.cursor-plugin/marketplace.json`.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
scripts/validate-plugins.mjsonly validatedmarketplace.jsonand eachplugin.json, and the workflow'spathsfilter excluded**/SKILL.md, soskill frontmatter was never checked and could drift silently.
Changes
schemas/skill.schema.jsondescribingSKILL.mdfrontmatter(required
nameanddescription;namein kebab-case; unknown keysallowed so plugins keep harness-specific options).
SKILL.mdfrontmatter with Ajv, and require thefrontmatter
nameto match the skill's folder name.**/SKILL.mdandscripts/**to the workflowpathsfilter andinstall the
yamlparser.Fixes for the violations this exposes
agent-compatibility/skills/check-agent-compatibility: quote a descriptioncontaining
": "so the frontmatter parses (closes check-agent-compatibility: SKILL.md description has an unquoted colon, YAML parse fails #381)third_party/x/skills/x-api-mcp-guide: name"X MCP guide"→"x-api-mcp-guide"(closes X plugin: SKILL.md name "X MCP guide" does not match folder x-api-mcp-guide #269)pstack/skills/poteto-mode: name"Poteto Mode"→"poteto-mode"(closes pstack: poteto-modenamebreaks the documented identifier constraint and fails to load on other harnesses #237)pstack/skills/make-bot-ui: name"Make Bot UI"→"make-bot-ui"third_party/x/skills/x-chat: name"X Chat"→"x-chat"third_party/x-money/skills/x-money-guide: name"X Money guide"→"x-money-guide"The validator is green on the updated tree and reports each of these cases
when reintroduced.
Note for reviewers
Enforcing
name == folderis the one deliberate policy choice here: 91 of the96 skills already match, and #237 / #269 are filed about the mismatches. If you
would rather allow a separate display name, I'm happy to relax that check to
the kebab-case pattern only.
Note
Low Risk
Changes are limited to CI validation scripts and skill metadata fixes; no runtime product or auth/data paths are modified.
Overview
Adds CI validation for
SKILL.mdYAML frontmatter so skill metadata cannot drift without failing the validate-plugins workflow.A new
schemas/skill.schema.jsonrequiresname(kebab-case) anddescription, with extra frontmatter keys still allowed.scripts/validate-plugins.mjsnow walks everySKILL.md, parses frontmatter with theyamlpackage, validates against that schema, and requiresnameto match the parent folder name. The same script also fails if any repo directory with.cursor-plugin/plugin.jsonis missing frommarketplace.json.The GitHub workflow runs on changes to
**/SKILL.mdandscripts/**and installsyamlalongside Ajv. Several existing skills are updated to pass the new rules (kebab-case names aligned with folders, quoted description where": "broke YAML parsing).Reviewed by Cursor Bugbot for commit 3ea2c18. Bugbot is set up for automated code reviews on this repo. Configure here.