# VALIDATION-REPORT.md — Test Prompts, Outputs, and Checklist Results **Author:** Simon-Pierre Boucher **Contact:** contact@spboucher.ai **Date:** 2026-08-05 Validation of the two Phase-2 example skills per the CLAUDE.md protocol, plus mechanical verification of the whole repository with `tools/validate_skills.py` (the deterministic linter that enforces the Sharp Skill Checklist). --- ## Skill 1 — `profiling-csv-data` (deterministic / verifiable) ### Test prompts | # | Prompt | Expected | Result | |---|---|---|---| | 1 | "Profile customers.csv and tell me about its data quality" | TRIGGER | ✅ matches "profile", "data quality", "csv" in description | | 2 | "What columns and missing values are in validation/fixtures/customers.csv?" | TRIGGER | ✅ matches "columns", "missing/null values", "CSV" | | 3 | "Convert customers.csv to JSON" | NO TRIGGER | ✅ description explicitly excludes "converting, transforming" — falls through to processing-csv / direct answer | ### Execution evidence (real runs, not simulated) - `python3 skill-1-profiling-csv-data/scripts/profile_csv.py validation/fixtures/customers.csv` → valid JSON: 8 rows, 6 columns, types inferred correctly (integer/string/date/float), 1 null in `age`, 1 null in `city`, 1 duplicate row detected. ✅ - Rendered report per `references/report-format.md`: see [validation/fixtures/customers-profile.md](validation/fixtures/customers-profile.md) — verdict rule 2 applied (🟡, 3 issues), naming rule respected (`customers.csv` → `customers-profile.md`). ✅ - Edge cases: - `ragged.csv` → `ragged_rows: 2` detected ✅ - `headeronly.csv` → `rows: 0`, exit 0 (→ ⚠️ verdict path) ✅ - `missing.csv` → clear error on stderr, exit code 2 ✅ ### Sharp Skill Checklist | Item | Result | Note | |---|---|---| | A1 what + when in description | ✅ | | | A2 literal user phrases | ✅ | "profile", "what's in this CSV", "duplicate rows" | | A3 third person, directive, ≤1024 chars | ✅ | 441 chars | | A4 no false-positive neighbors | ✅ | conversion/editing/plotting explicitly excluded | | A5 name gerund, lowercase-hyphens | ✅ | `profiling-csv-data` | | B1 body <500 lines | ✅ | 44 lines | | B2–B3 no known-content, one default | ✅ | script is the single source of truth | | B4 checklist workflow + validation loop | ✅ | Step 5 validates rendering | | B5 output fully specified | ✅ | template + `-profile.md` naming + destination | | B6 examples where format matters | ✅ | exact template in reference file | | B7 consistent terms, no time-sensitive info | ✅ | | | B8 failure modes addressed | ✅ | missing/empty/ragged/encoding | | C1 references one level deep | ✅ | one reference file | | C2 TOC for >100-line references | ✅ (N/A) | reference is 62 lines | | C3 execute-vs-read explicit | ✅ | "execute, do not read" | | C4 scripts solve-don't-defer, constants justified | ✅ | all 4 constants commented; errors handled | | C5 dependencies stated | ✅ | stdlib-only | | D1–D4 tests incl. negative | ✅ | above | | E1 author headers | ✅ | linter-verified | **Initial failures found and fixed during validation:** 1. Linter flagged `name != folder` because CLAUDE.md mandates `skill-1-` folder prefixes — resolved by encoding the project convention into the linter (frontmatter name must equal folder minus `skill-N-` prefix). Re-checked: pass. --- ## Skill 2 — `writing-release-notes` (stylistic / subjective) ### Test prompts | # | Prompt | Expected | Result | |---|---|---|---| | 1 | "Write release notes for v2.8 from these commits: [list]" | TRIGGER | ✅ matches "write release notes" | | 2 | "Draft the what's new section for our app update" | TRIGGER | ✅ matches "'what's new' section", "version announcement" | | 3 | "Write a commit message for this diff" | NO TRIGGER | ✅ description explicitly excludes git commit messages | ### Simulated run (prompt 1, commit list from references/examples.md) Input: 6 commits (SAML SSO, query-planner perf, avatar NPE fix, CI bump, billing refactor, breaking `/v1/auth` removal). Output produced by following the skill: ```markdown ## Acme 2.8.0 — 2026-08-05 > **Breaking:** The legacy `/v1/auth` endpoint is removed. Switch integrations to `/v2/auth` before upgrading. Faster searches and single sign-on headline this release. ### Added - Sign in with your company account: SAML single sign-on is now available on all Team plans. ### Improved - Search is dramatically faster — most queries now return in about 0.2 seconds instead of nearly a second. ### Fixed - Fixed a crash when opening a profile that has no avatar. ``` Verified against the five house rules: benefit-first ✅, plain language ✅ (no "leverage/robust/seamless", no ticket IDs), exact three-heading grouping ✅, breaking change at top as blockquote ✅, noise cut ✅ (CI bump and refactor correctly absent). ### Sharp Skill Checklist | Item | Result | Note | |---|---|---| | A1–A3 description what+when, literal phrases, 3rd person, ≤1024 | ✅ | 418 chars | | A4 no false positives | ✅ | commit messages / PR descriptions / API docs excluded | | A5 name | ✅ | `writing-release-notes` (gerund) | | B1 body <500 lines | ✅ | 68 lines | | B3 one default | ✅ | single format, single grouping scheme | | B4 workflow + self-review loop | ✅ | step 5 re-checks the five rules | | B5 output fully specified | ✅ | template + delivery rule (chat vs CHANGELOG.md) | | B6 positive AND negative examples | ✅ | in body + references/examples.md | | B8 edge cases | ✅ | no user-visible changes / ambiguity / unknown version | | C1–C2 references one level deep, TOC | ✅ | examples.md has TOC | | C3–C4 scripts | ✅ (N/A) | prose-only skill by design (subjective domain) | | D1–D4 tests incl. negative | ✅ | above | | E1 headers | ✅ | linter-verified | **Boundary-condition check (mandated for the stylistic skill):** the skill defines both an exclusion list AND a tie-breaker procedure (ask "end users or engineers?") for ambiguous audience — pass. --- ## Repository-wide mechanical validation `python3 tools/validate_skills.py .` enforces: frontmatter parses; name rules (≤64 chars, charset, reserved words, folder match); description rules (≤1024, "Use when" trigger clause, "Do not use" boundary, third person, no XML tags); author header in every .md/.py/.sh file; body ≤500 lines; no broken reference links; no backslash paths. ### Final run (after all six collections were built) ``` $ python3 tools/validate_skills.py . checked 72 skill(s) all checks passed ``` Coverage: 2 example skills (Phase 2), 10 doc-skills, 10 frontend-skills, 10 db-skills, 20 backend-skills, 10 writing-skills, 10 finance-skills — 72 skills, 157 markdown/Python files, every one carrying the author header and every description carrying both a "Use when …" trigger clause and a "Do not use …" boundary. **Failures caught by the linter during the build (all fixed):** 1. `name` vs mandated `skill-N-` folder prefix conflict → linter updated to encode the project convention (see Skill 1 section). 2. Several transient broken-reference failures while build agents were mid-write → re-checked after completion; all resolved. **Trigger de-confliction:** each collection README documents a pairwise boundary table; every skill description names its nearest neighboring intent and explicitly excludes it, so no request should plausibly fire two skills.