spb/ultra-sharp-agent-skills Public

1<!--2Author: Simon-Pierre Boucher3Contact: contact@spboucher.ai4-->56# VALIDATION-REPORT.md — Test Prompts, Outputs, and Checklist Results78**Author:** Simon-Pierre Boucher9**Contact:** contact@spboucher.ai10**Date:** 2026-08-051112Validation of the two Phase-2 example skills per the CLAUDE.md protocol, plus13mechanical verification of the whole repository with `tools/validate_skills.py`14(the deterministic linter that enforces the Sharp Skill Checklist).1516---1718## Skill 1 — `profiling-csv-data` (deterministic / verifiable)1920### Test prompts2122| # | Prompt | Expected | Result |23|---|---|---|---|24| 1 | "Profile customers.csv and tell me about its data quality" | TRIGGER | ✅ matches "profile", "data quality", "csv" in description |25| 2 | "What columns and missing values are in validation/fixtures/customers.csv?" | TRIGGER | ✅ matches "columns", "missing/null values", "CSV" |26| 3 | "Convert customers.csv to JSON" | NO TRIGGER | ✅ description explicitly excludes "converting, transforming" — falls through to processing-csv / direct answer |2728### Execution evidence (real runs, not simulated)2930- `python3 skill-1-profiling-csv-data/scripts/profile_csv.py validation/fixtures/customers.csv`31  → valid JSON: 8 rows, 6 columns, types inferred correctly (integer/string/date/float),32  1 null in `age`, 1 null in `city`, 1 duplicate row detected. ✅33- Rendered report per `references/report-format.md`: see34  [validation/fixtures/customers-profile.md](validation/fixtures/customers-profile.md)35  — verdict rule 2 applied (🟡, 3 issues), naming rule respected36  (`customers.csv` → `customers-profile.md`). ✅37- Edge cases:38  - `ragged.csv` → `ragged_rows: 2` detected ✅39  - `headeronly.csv` → `rows: 0`, exit 0 (→ ⚠️ verdict path) ✅40  - `missing.csv` → clear error on stderr, exit code 2 ✅4142### Sharp Skill Checklist4344| Item | Result | Note |45|---|---|---|46| A1 what + when in description | ✅ | |47| A2 literal user phrases | ✅ | "profile", "what's in this CSV", "duplicate rows" |48| A3 third person, directive, ≤1024 chars | ✅ | 441 chars |49| A4 no false-positive neighbors | ✅ | conversion/editing/plotting explicitly excluded |50| A5 name gerund, lowercase-hyphens | ✅ | `profiling-csv-data` |51| B1 body <500 lines | ✅ | 44 lines |52| B2–B3 no known-content, one default | ✅ | script is the single source of truth |53| B4 checklist workflow + validation loop | ✅ | Step 5 validates rendering |54| B5 output fully specified | ✅ | template + `<stem>-profile.md` naming + destination |55| B6 examples where format matters | ✅ | exact template in reference file |56| B7 consistent terms, no time-sensitive info | ✅ | |57| B8 failure modes addressed | ✅ | missing/empty/ragged/encoding |58| C1 references one level deep | ✅ | one reference file |59| C2 TOC for >100-line references | ✅ (N/A) | reference is 62 lines |60| C3 execute-vs-read explicit | ✅ | "execute, do not read" |61| C4 scripts solve-don't-defer, constants justified | ✅ | all 4 constants commented; errors handled |62| C5 dependencies stated | ✅ | stdlib-only |63| D1–D4 tests incl. negative | ✅ | above |64| E1 author headers | ✅ | linter-verified |6566**Initial failures found and fixed during validation:**671. Linter flagged `name != folder` because CLAUDE.md mandates `skill-1-` folder68   prefixes — resolved by encoding the project convention into the linter69   (frontmatter name must equal folder minus `skill-N-` prefix). Re-checked: pass.7071---7273## Skill 2 — `writing-release-notes` (stylistic / subjective)7475### Test prompts7677| # | Prompt | Expected | Result |78|---|---|---|---|79| 1 | "Write release notes for v2.8 from these commits: [list]" | TRIGGER | ✅ matches "write release notes" |80| 2 | "Draft the what's new section for our app update" | TRIGGER | ✅ matches "'what's new' section", "version announcement" |81| 3 | "Write a commit message for this diff" | NO TRIGGER | ✅ description explicitly excludes git commit messages |8283### Simulated run (prompt 1, commit list from references/examples.md)8485Input: 6 commits (SAML SSO, query-planner perf, avatar NPE fix, CI bump,86billing refactor, breaking `/v1/auth` removal).87Output produced by following the skill:8889```markdown90## Acme 2.8.0 — 2026-08-059192> **Breaking:** The legacy `/v1/auth` endpoint is removed. Switch integrations to `/v2/auth` before upgrading.9394Faster searches and single sign-on headline this release.9596### Added97- Sign in with your company account: SAML single sign-on is now available on all Team plans.9899### Improved100- Search is dramatically faster — most queries now return in about 0.2 seconds instead of nearly a second.101102### Fixed103- Fixed a crash when opening a profile that has no avatar.104```105106Verified against the five house rules: benefit-first ✅, plain language ✅107(no "leverage/robust/seamless", no ticket IDs), exact three-heading grouping ✅,108breaking change at top as blockquote ✅, noise cut ✅ (CI bump and refactor109correctly absent).110111### Sharp Skill Checklist112113| Item | Result | Note |114|---|---|---|115| A1–A3 description what+when, literal phrases, 3rd person, ≤1024 | ✅ | 418 chars |116| A4 no false positives | ✅ | commit messages / PR descriptions / API docs excluded |117| A5 name | ✅ | `writing-release-notes` (gerund) |118| B1 body <500 lines | ✅ | 68 lines |119| B3 one default | ✅ | single format, single grouping scheme |120| B4 workflow + self-review loop | ✅ | step 5 re-checks the five rules |121| B5 output fully specified | ✅ | template + delivery rule (chat vs CHANGELOG.md) |122| B6 positive AND negative examples | ✅ | in body + references/examples.md |123| B8 edge cases | ✅ | no user-visible changes / ambiguity / unknown version |124| C1–C2 references one level deep, TOC | ✅ | examples.md has TOC |125| C3–C4 scripts | ✅ (N/A) | prose-only skill by design (subjective domain) |126| D1–D4 tests incl. negative | ✅ | above |127| E1 headers | ✅ | linter-verified |128129**Boundary-condition check (mandated for the stylistic skill):** the skill130defines both an exclusion list AND a tie-breaker procedure (ask "end users or131engineers?") for ambiguous audience — pass.132133---134135## Repository-wide mechanical validation136137`python3 tools/validate_skills.py .` enforces: frontmatter parses; name rules138(≤64 chars, charset, reserved words, folder match); description rules (≤1024,139"Use when" trigger clause, "Do not use" boundary, third person, no XML tags);140author header in every .md/.py/.sh file; body ≤500 lines; no broken reference141links; no backslash paths.142143### Final run (after all six collections were built)144145```146$ python3 tools/validate_skills.py .147checked 72 skill(s)148all checks passed149```150151Coverage: 2 example skills (Phase 2), 10 doc-skills, 10 frontend-skills,15210 db-skills, 20 backend-skills, 10 writing-skills, 10 finance-skills —15372 skills, 157154markdown/Python files, every one carrying the author header and every155description carrying both a "Use when …" trigger clause and a156"Do not use …" boundary.157158**Failures caught by the linter during the build (all fixed):**1591. `name` vs mandated `skill-N-` folder prefix conflict → linter updated to160   encode the project convention (see Skill 1 section).1612. Several transient broken-reference failures while build agents were162   mid-write → re-checked after completion; all resolved.163164**Trigger de-confliction:** each collection README documents a pairwise165boundary table; every skill description names its nearest neighboring intent166and explicitly excludes it, so no request should plausibly fire two skills.167