Why the same skill gives different results, and what to do
Run the same prompt twice and a skill may fire once and not the next time, or follow a rule differently. Where the variation comes from and how to reduce it.
Some variation is normal. Model output is not deterministic, and whether a skill is picked can depend on context. What you can control is how often a prompt lands in the uncertain zone.
Where variation comes from
- A description that only partly matches the request.
- Earlier conversation steering what seems relevant.
- Other installed skills competing, as in two skills wanting the same prompt.
- Rules written loosely enough to be read two ways.
Reduce it
- Test in fresh sessions so history is not a variable.
- Rewrite borderline descriptions with the words from the prompts that wavered.
- Turn vague rules into checkable ones: "one accent per screen" rather than "use the accent sparingly".
- Run each prompt several times and record the rate, not a single result.
If results changed across the board after an update, that is a different problem: regression testing after a Claude update.
Questions
Why does my skill trigger only some of the time?
Usually because the description is a borderline match for the prompt. Small differences in wording or context tip it either way. A clearer description moves those prompts out of the borderline.
Does the rest of the conversation affect skill choice?
It can. Earlier messages and other installed skills change what looks relevant. Test in a fresh session to compare like with like.
How do I report results that vary?
Run each prompt a few times and report how many runs fired, rather than a single yes or no.