You ask Copilot to fix a bug. It finds the right function, makes a sensible change and adds a test. Then you notice it ran the wrong test command. Again.
For me it was bare pytest from the root of a PySpark project. That also collects the integration tests, which need a live Databricks connection, so they failed, and a couple of times the agent's idea of a fix was to edit the tests. The unit suite was one command away: pytest tests/unit. I said so, it reran, everything passed. A week later, in a fresh chat, I typed the same correction again.
Once is fine, but by the twelfth time I was really annoyed. Which command runs the tests, where the existing patterns live, what "done" means in this repo: those are facts about the project, and I kept typing them into whichever chat happened to be open. After a few months, they're spread across a hundred conversations that nobody else will ever read.
The usual fix is a longer prompt. A detail goes in here, a reminder there, and eventually every task starts with a page of instructions that was copied from a colleague and edited by three other people. It works until somebody deletes a paragraph because it looked redundant.
Skills are a standard approach, and I use them to write down the procedures I keep re-explaining. The model didn't get smarter. The knowledge just lives in the repo, where a colleague can read it, instead of in my chat history.
Why a long prompt isn't enough
A good prompt gets you a good result once. Hand it to a colleague for a different task and it tends to fall apart.
Say two engineers ask an agent to add validation to an endpoint. One types:
Add validation for this field and write tests.
The other types:
Find a similar endpoint first and follow its validation pattern. Write behavior-focused tests using the existing fixtures. Run the relevant tests and the project's required checks, and tell me anything you couldn't verify.
The second engineer gets the better patch, and you can guess why. They've watched the agent cut corners and worded the request to close those gaps, probably after binning a few bad patches. Everyone else on the team is still sending the first version.
Once people rely on a prompt like that and nobody has written it down, it's infrastructure, and nobody can tell you which version is current. A prompt is meant to describe a task. I don't think it should also be where a team keeps its working method.
What I mean by a skill
A skill, to me, is a reusable bundle of instructions for one kind of work. Depending on your tooling it may also carry scripts, examples or reference docs. What matters is the split between the task and the method. The task changes with every request: "reject booking requests where the departure date is before the arrival date." The skill describes how that kind of work gets done in your codebase. Read the existing validation, use the established error format, test the boundaries, run the relevant checks, say what happened.
When I look at a skill I ask four things. When does it apply? What does the agent actually do? What shows it worked? What happens when something can't be verified? A skill that can't answer those is just a long prompt in a folder.
It also guarantees nothing. The agent has to find the skill and then follow it, so you still need checks that look at the result.
A made-up bug: reversed dates
Picture a booking app where every request has an arrival date and a departure date. Someone reports that the API happily accepts a departure earlier than the arrival.
The code change is maybe two lines. The time goes on everything around it. Is a same-day stay allowed? Which error format do clients expect? The validation might already sit in a shared model, or another endpoint might enforce the same rule. And which test would actually prove this bug? If all you say is "fix date validation," the agent has to guess at every one of those.
A bug-fixing skill could lay out the order of work like this:
## Fixing a reported bug
### Before editing
- Read the reported behavior and the acceptance criteria.
- Find the request model, the existing validation and the related tests.
- List any open behavior questions. Do not answer them yourself; ask.
### Reproduce
- Write or find a focused test that shows the failure.
- Confirm it fails for the reason given in the report.
### Implement
- Make the smallest complete change.
- Reuse the existing validation and error-handling patterns.
### Verify
- Run the reproduction test, the related tests and the required project checks.
- Report each command and its outcome, including anything that did not run.
Same-day bookings don't appear anywhere in that list, on purpose. Whether they're valid is a requirements question, and I don't want the agent answering it by accident while it chooses between < and <=. If it runs into the question, the skill tells it to write it down and ask.
Where to start
Scroll back through your recent review comments. "Use the existing fixture." "That command doesn't run the integration tests." "Don't we already have a helper for this?" "You changed the assertion, but the original behavior is still broken."
Any one of those is ordinary feedback. Once you've left the same one on five pull requests, that piece of process has no home, and you've found your first skill.
Mine was the test command from the top of this post. I put pytest tests/unit into a skill with a note that integration tests need a cluster and get skipped locally, and I stopped typing it.
Don't try to plan a whole library up front. Pick one recurring problem where you'd actually notice the improvement. Running focused tests is a good candidate: choosing the right suite, knowing what environment it needs, telling a collection error from a failed assertion. So is hunting for prior art, which just means reading the neighboring code and shared helpers before writing anything new. Keep the first one small. A short procedure that stops one repeated mistake gets used. A thorough document nobody follows doesn't.
How skills go wrong
The most common problem is vague wishes instead of clear instructions. “Be thorough” and “follow best practices” sound nice, but the agent cannot act on them. “Look at an existing endpoint with similar validation and reuse its error format” is different. The agent can do it, and you can check afterwards whether it did.
The second problem is a skill that forces the agent to do everything. A small docs change does not need the full migration workflow, and a one-line fix does not need the whole test suite. If the skill always asks for the heaviest process, people start to avoid it. Or the agent spends more effort following the skill than solving the actual problem.
Skills can also get old. Commands are renamed, test folders move, a dependency is removed, and the skill still describes the old setup. This can do more damage than a forgetful colleague, because the agent follows the skill exactly. A skill depends on your environment, so it is closer to a script than to documentation. Someone has to take care of it.
The last problem is changing the skill too quickly. When the agent makes a mistake, it is tempting to add a new rule right away. I would wait. If you add a rule after every small problem, the skill slowly turns into a long list of warnings. Some of them contradict each other, and the agent does not know which ones matter.
Wait until the same problem happens again, or until one mistake is really expensive. Then check whether the skill already covers it, and change only the part that is related to the problem. For example, if the agent keeps changing a failing test instead of finding out why it fails, make the testing steps clearer. If it keeps running the wrong tests because two commands have similar names, add a clearer rule for choosing between them, or write a small script.
"Done" needs evidence
If I could change one thing about how people use coding agents, I'd replace "finish the task" with a specific definition of finished. For a bug fix, that's a reproduction test that failed before the change and passes after it. For a refactor, existing behavior that is still covered. For documentation, examples and links that somebody actually checked. There's no single checklist, because the right evidence depends on the work.
The final report matters as much as the change. "Tests passed" tells you almost nothing. Compare it with: "The focused validation tests passed. The integration suite couldn't start because its database service was unavailable." That's less comforting, and it's the only version that tells you what to go and check yourself. A command that ran hasn't necessarily passed a check, and a test that was never collected hasn't passed anything. Put that in the skill, and "I couldn't verify this" becomes a normal thing for the agent to say instead of something it skates past.
My pytest problem was a small version of the same thing. Integration tests failing because there's no cluster to connect to is an environment problem, and an agent that starts editing the tests has treated it as a code problem. One line in a skill saying those tests need a cluster and get skipped locally closes that gap.
Test-writing needs extra care, since an agent can produce impressive numbers while proving very little. It can execute every line of a function and still miss the rule that matters or write assertions that faithfully preserve an existing bug. In the booking example, a test could confirm that the current code accepts reversed dates. It would describe the implementation perfectly and say nothing about the requirement. So the skill should say where expected behavior comes from: the acceptance criteria or an established contract, never a reading of the implementation. Coverage reports are still handy for spotting branches nobody tested. Check those gaps against the intended rules and don't read the percentage as a verdict.
Instructions, skills and agents
People mix these up, and I understand why. The way I split them: repository instructions cover the broad expectations, like naming, tooling, error handling, architectural boundaries and security rules. A skill covers the procedure for one activity, say reproducing a bug or investigating a dependency upgrade. An agent is a role with boundaries that can lean on skills. A testing agent might use a test-running skill, while an implementation agent uses one for finding existing patterns.
I wouldn't paste the general conventions into every skill. Copies drift, and sooner or later two of them disagree. Keep the conventions in one place, keep the activity-specific steps in the skill and link to the shared guidance.
What's left for the prompt
You still write prompts, they just get shorter. A prompt describes the problem: the intended behavior, the constraints, the acceptance criteria and the decisions that need a human. The recurring procedure lives somewhere else.
For me that means fewer minutes spent telling the agent which command to run, and more review time spent on whether the change makes sense.
If your team is past the pilot stage with coding agents and the same review comments keep coming back, this is the kind of engineering workflow we set up with clients: skills, repository instructions and checks that show what was verified. Talk to our team about what it would look like in your codebase.
Sources
