How do you find skills, keep them organized, and make sure they actually work? Do you keep improving them over time?
I believe skills will eventually be eating by model capabilities, but until then I'm just looking for a better way to manage things.
Skills is just a tech bro word for a simple markdown file with instructions.
No need to over complicate it. Write down things you feel like re-using. Like how to specifically implement something in your system ("when adding a new API endpoint we need to do x y and z", or "when making a github PR we tag Æ and Å") so you don't have to repeat it. And I mostly add it in cases where it didn't infer it itself. So very reactive, not proactive.
Most public skills are useless and over complicated. Lots of people are spending too much time on their harness, than actually making stuff.
Edit: but do get inspired by public ones. For instance a "grill me" skill can ve be useful, but I find the public one very mumbo-jumbo. But the idea of forcing the agent to ask clarifying questions is good.
I use an agent plugin spec repo. Codex is already compatible with it and it supports skills + MCP definitions.
I have my skills in my dotfiles repo, then symlink them to my home directory and/or projects where I want to use them. Project specific ones go into the project.
Don't find them. Ask the AI to do something. When it does it correctly, ask it to make a skill for it. Clear the session, try to use the skill, fix any problems found, modify your repo and harness if necessary. Repeat until skill works 0-shot. Improve with the same process.
This largely works with a specific model, specific harness, specific prompt, specific context. You may need to modify your agent harness to manage skills depending on runtime parameters. Pi is a great general purpose agent for the these modifications.
If you do find other skills and want to use them, put them through the loop above. But keep in mind that since they were created in their own circumstances, they may not work in yours.
Also separate rules from skills. Rules tell AI when to do things, skills tell AI how to do things. Tool call/MCP limitations, agent configurations, and harness extensions, can help it stay on track.
I have a separate repo which has to be pulled locally and the skills and agents are sym linked to projects.
Openspec has a subcommand (init) to manage them: clever because they provide also an update path.
Any skills, I just add into the tool itself. I then have the py tools in their PWD, don’t bother with mcp.
Not really related but I wonder how people benchmark the effectiveness of skills/agents?
I'm seeing the agent working quite fine with just direct prompting and the agent doing things by itself rather than using skills. Is it better for certain task size?
With a custom skills workspace in mininote
https://mininote.ink/docs/mcp-docs
Agent can use mcp to update its own skills, or I can copy template skills into local dorectories via the api. Very useful, like notion on steroids but is completely free.
i have a docs/
it has all the skills/docs my particular application needs
i treat it as ADRs as it helps the AI understand the parts of the system it is working on
I wrote a small command-line tool that installs skill packs into agent-specific project folders. It works pretty much like `brew` (or any package manager, really). The skills are compiled into the binary so that I don't have to worry about where they're located and can quickly move the skills between machines by copying the tool.
Making sure they actually work? Trial and error, mostly. I know some folks have tried auto-researcher approaches, but I haven't found that to be the best use of time in my work.
how did stupid markdown file become a thing, this industry really went to shit.
[dead]
[dead]
[flagged]
[dead]
[flagged]
[flagged]
[dead]
[flagged]
[flagged]
[dead]
[dead]
[flagged]
[dead]
[dead]
[dead]
One of the engineers I know is building this product called SkillEd for just this. Lemme know if you need an invite
WTF is a "skill"? I really think people are getting ahead of themselves here.
You wrote some bullet points so your agent harness doesn't keep making builds in the wrong environment? You have a very specific debugging setup? Your agent doesn't understand when to rebase?
README is where you should be writing anything specific to your project, and if you're worried about context size then your README is too long, it should be just enough information for any competent dev or agent to get the gist of how you do things around here and where to look for deeper answers.
If your particular harness / orchestrator is just not pushing back enough or can't seem to solve certain problems then thats a tool issue, either edit the tool system prompts or move to better tools or models.
Calling this 'skills' is disingenuous, this word was chosen by marketers and implies some kind of deeper learning. I'm not saying there's no value in tuning prompts, but your 'skills' should be managed in only 2 ways: 1. It's specific to your project, it's a README, or 2. It's specific to your tooling, it's part of config, system prompts etc.
Why do you think like that? "I believe skills will eventually be eating by model capabilities, but until then I'm just looking for a better way to manage things."