Build a trigger test set for a Claude skill in twenty minutes
Twenty prompts, half that should use the skill and half that should not, tell you whether a description works. How to write them so the test means something.
Whether a skill triggers is a yes or no per prompt, which makes it easy to test and easy to fool yourself about. The test set is where honesty lives.
The two halves
| Half | What goes in it |
|---|---|
| Should fire (10) | Real requests for the job, phrased several ways, including lazy ones |
| Should not fire (10) | Near-misses in the same subject that this skill must leave alone |
For a design-system skill, should-fire prompts include "make this settings page look less generic" and "build a dark dashboard hero". Should-not prompts include "fix this React state bug" and "write tests for the header": same codebase, different job.
Rules that keep it honest
- Do not reuse the description's wording. If the prompts copy it, every test passes and proves nothing.
- Include at least three sloppy prompts: lowercase, no punctuation, half a sentence.
- Write the set before you change the description, and keep it in the repo next to the skill.
- Record the date and the result each time you run it.
Reading the result
You want most of the first half and almost none of the second. A skill that fires on near-misses is worse than one that misses a few real requests, because it intercepts work it cannot do. What to change when the second half fails is in a skill that fires too often; when the first half fails, see how to write a skill description.
Questions
How many prompts does a trigger test need?
Twenty is a good start: ten that should use the skill and ten that should not. Add more when a real request surprises you.
Why include prompts that should not trigger the skill?
Because a description broad enough to catch everything will also fire on work it cannot do. The should-not half is what catches that.
Who should write the prompts?
Ideally someone who did not write the skill, in the way they would really type. The author tends to reuse the description's own words, which makes every test pass.