Writing prompts that should not fire your skill
The half of a trigger test people skip: near-miss prompts that must leave the skill alone. How to write them so they actually catch over-triggering.
A trigger test full of unrelated prompts always passes. "Write a haiku" will not fire a React standards skill. The prompts worth writing are the ones that sit right next to the skill's job.
Recipes for good near-misses
- Same codebase, different job: "fix the failing test in the header" for a design skill.
- Same word, different meaning: "design the database schema" for a UI design skill.
- Adjacent skill's job: "review this PR for security" for a style review skill.
- Too small to need it: "change this button label".
Each one should be something a real person might type in the same session. Record which ones fire; those are the words to remove from the description, as in a skill that fires too often. The full test is in a trigger test set.
Questions
What is a near-miss prompt?
A request in the same area as the skill's job, using some of the same words, that the skill should nevertheless leave alone.
Why not use obviously unrelated prompts?
Because a skill almost never fires on those. They pass trivially and tell you nothing. Near-misses are where over-triggering shows.
How many should-not prompts do I need?
As many as should-fire prompts, so both numbers carry the same weight. Ten each is a good start.