How to measure a skill's trigger accuracy, before and after
Trigger accuracy is two numbers from one prompt set: how often it fires when it should, and how rarely when it should not. How to measure and report both.
"It triggers better now" is a feeling. A measurement is a prompt set, a date, and two numbers you could hand to someone else to reproduce.
The two numbers
| Measure | Question |
|---|---|
| Fires when it should | Of the should-fire prompts, how many used the skill? |
| Quiet when it should be | Of the should-not prompts, how many left it alone? |
Keep the comparison fair
- Use the identical prompt set before and after.
- Record the Claude Code version, and which other skills were installed.
- Change only the skill between runs.
- Run more than once if results vary, and report the spread.
Report it like a measurement
Date, prompt count, both numbers, and a link to the prompts. We hold ourselves to that: a skill in our catalogue shows "not measured yet" until a run exists, rather than a number we did not take. Building the prompts is covered in a trigger test set; rerunning after Claude updates is in regression testing.
Questions
Is trigger accuracy one number?
Report two: the share of should-fire prompts that fired, and the share of should-not prompts that did not. One blended number hides a skill that fires on everything.
How do I compare before and after fairly?
Same prompt set, same Claude Code version, same other skills installed, run on the same day if you can. Change only the skill.
Should I publish the numbers?
Only numbers you measured, with the date and the prompt set, so someone else could rerun them. A number with no date or method is marketing, not a measurement.