--- title: "skills are a sham" description: "how to stop treating a help center abstraction like a capability, and why the real compounding happens in code" published: "2026-05-30T15:11:00+00:00" canonical: https://jaskaran.me/blog/skills-are-a-sham markdown: https://jaskaran.me/blog/skills-are-a-sham.md --- # skills are a sham skills are a sham. not useless, just miscategorized. people are using a help-center abstraction to do a capability job, and that’s why it doesn’t compound. a skill tells an agent what to do; a tool lets it actually do the thing. only one of those scales. if you’re building agents, you’ve probably heard the advice: write a better skill. your agent’s hitting a wall, so you write longer instructions, more context, better examples. and it works, sort of, for a while. then it breaks differently and you’re writing longer instructions again. you’re not building leverage, you’re renting an outcome. and it keeps getting more expensive. we ended up in a place where our Sidebar system prompts and skills were getting longer and longer (upto 10-15k tokens in some cases). this is why you keep hearing things on linkedin like “i’ve run out of tokens so quickly”. it’s because we’re stuffing the skills so much to make it work, there’s hardly enough headroom left for actual work. **otoh, here’s what actually compounds:** code. composable, testable, reusable code. tools that work the same way every time because agents are not reasoning their way through every execution. the agent that’s been running for six months should be structurally better than a fresh instance, not because its prompts got better but because it’s built a library of capabilities it can reach for. that’s the bet. ----- ## 1. the help center analogy i’ve spoken about this in my [AX article](https://www.jaskaran.me/blog/the-new-user-persona): a good skill is equivalent to a good help center article. that’s the ceiling. think about how help center articles work. someone writes down how to do a thing. a human reads it and gets it done. the human still has hands, a browser, judgment, the ability to click around and recover from uncertainty. the skill does the same job, except for an agent. “here’s how to do this thing” except the agent still has to conjure the execution out of nothing. **a help center article is documentation.** it says “here’s how a human should think about this.” a skill is documentation too–it just assumes the reader doesn’t have hands. and so we end up in a situation where the skill is actually documentation of capability you’re *hoping* exists and works, not capability that actually does. you can load a skill with intent all you want, but without actual mechanisms underneath, you’re just giving the model a more detailed description of something it still can’t execute well, or consistently ----- ## 2. nineteen dollars a task and a slot machine we’ve been trying to make some things automated within the SpotDraft CLM and one of the things that we were doing was giving our internal agent neo the ability to on-board entire teams automatically. initially, we created a skill that could log on to SpotDraft and create and on-board the entire team but every run was costing us $19 and 30 minutes of time. another example is when a team-mate of mine wanted to fetch logs periodically from Langsmith. the first thing that we did was create a skill and attach the Langsmith MCP so that the agent could do that but obviously that didn't give us a predictable output within manageable costs. the why is clear once you see it: the agent is figuring everything out from scratch every single time. no memory of a working path, no reusable mechanism, just reasoning its way through the same maze on every run. and it’s not just expensive, it’s probabilistic. hit and trial. you don’t know if it’ll work this time. that’s the real tax: you can’t build on something that works 70% of the time and you can’t predict which 70%. another example: a team member wanted to turn “what pull requests do i need to review” into a skill, when most of that is deterministic. fetch the github api, filter by assignee, sort by time, format it. that’s not a reasoning problem. that’s a tool call. that’s the perfect case where something should never have been a reasoning problem in the first place. when I took over the Langsmith log creation task, I just asked cursor to write me a snippet of code, which I then broke down into three separate nodes that I started running on n8n. no managed agents, just a simple API call that formats and sends me back what I need on slack every day at 5pm from Langsmith basically for free. the variance is the villain here, not just the height of the cost. you can live with $0.50 a task if it’s $0.50 every time and promises consistent outcomes. you can’t build on a system that costs $3 on tuesday and $40 on thursday. ![image](https://dzcjqzzfpmulffvcphbd.supabase.co/storage/v1/object/public/blog-post-images/blog-uploads/36f832f4-287a-43a9-b771-c404de417a4d.png) _here's how the average cost has been fluctuating for a specific agent/use-case on neo_ ----- ## 3. unstructured ad hoc is becoming the default (and that’s dangerous) throwing a model at a problem feels like progress, so it’s become the reflex for technical folks, EMs, PMs, everyone. the path of least resistance is “give it a model and a skill.” and on the surface that looks reasonable because models are good at reasoning through ambiguous things. so we’ve made every problem “ambiguous and complex” by default, even the ones that aren’t. here’s the thing: most recurring tasks aren’t actually ad hoc. they’re structured and premeditated. we’re just treating them as ad hoc because that’s the easy default. write a skill, throw it at the model, see what happens. > the danger isn’t that it fails. the danger is that it doesn’t compound. every run starts at zero. no memory, no abstraction, no asset being built. i built a [framework](https://www.jaskaran.me/blog/the-predictability-quadrant) here that might be worth leaning on: structured vs unstructured, premeditated vs ad hoc. most of what people are skill-ifying lives in the quadrant that should be code, not vibes. the structured, premeditated stuff: that’s a workflow. the unstructured, ad hoc stuff: that’s where you need the model in the loop, interpreting intent, handling ambiguity, making judgment calls. ![image](https://dzcjqzzfpmulffvcphbd.supabase.co/storage/v1/object/public/blog-post-images/blog-uploads/6091b1a8-e660-4e94-9cac-c3ad3cd39d9c.png) ----- ## 4. collapse ad hoc into deterministic where you can this is the core operational move i'd recommend. turn unstructured ad hoc into structured predictable, as far down as you can push it. in a lot of cases, the planning is where the decisionmaking, open-endedness should be: “the agent should figure out _what_ to do.” but the failure is when you leave the agent to figure out _how_ to do it too, running the execution in vibes. btw, not everything needs decision-making. and if it doesn’t need decision-making, it probably doesn’t need an LLM. here’s the reframe that people resist: conditional logic is good. people are scared of writing if/else trees because it feels like “not AI enough,” like you’re admitting defeat. but a well-designed tool that encapsulates a decision tree beats asking the model to re-reason the same decision tree every time. more reliable, cheaper, auditable. and the kicker: you’re not even writing the if/else anymore. you just ask the agent to write it. the agent authors its own rails. **where the model still belongs: interpretation.** understanding what the user actually wants, handling malformed input, graceful recovery, deciding which tool chain to invoke. that’s where prompt quality moves the needle. that’s where the reasoning tax is worth paying. but the moment you’re past “understand intent,” you want deterministic paths. you want rails. > use the model to get to the tool call; use the tool to do the actual thing. what changes in practice: instead of writing a skill that says “call the github api and filter PRs assigned to the user,” you write a tool that does exactly that, with typed params. the skill becomes a routing instruction: “when the user asks about their PRs, call the github-my-prs tool.” ----- ## 5. an agent that builds its own toolkit everyone’s excited about llm-wikis right now because they work. synthesized knowledge written down once doesn’t need to be re-derived. i maintain and use my own [jaswiki](http://jaskaran.me/blog/building-a-brain). we've built a self-updating wiki for SpotDraft that neo writes into (and it's been doing great). the idea of storing synthesis across many tasks is well-established by now. and it’s genuinely good because a fact in a wiki stays put; it doesn’t get re-explained. now imagine the same thing but for capability. an ever-growing toolkit where the agent writes code for itself once. the loop looks like this: (1) a task runs, (2) outcome succeeds, (3) the agent runs a reflection pass where it asks: what did i do from scratch here? what was general vs what was one-off? can i abstract this into a callable tool with the right params? if yes, it ships that tool for itself. the key distinction: this is **not** writing into a skill “here’s the code to rewrite next time.” that’s documentation of a thing the agent should do. this is literally saving the code as a callable, parameterized tool. the difference between a recipe and a kitchen appliance. the recipe is nice documentation; the appliance actually gets the outcome done. bootstrap it. the agent that’s been running for six months should be structurally better than a fresh instance, not because its prompts got better but because it has primitives that didn’t exist on day one. it built them over time. ![image](https://dzcjqzzfpmulffvcphbd.supabase.co/storage/v1/object/public/blog-post-images/blog-uploads/a5e5516d-cc96-4103-ba13-d8038ac8bb5d.png) ----- ## 6. why capability compounds faster than knowledge **knowledge in a wiki is fuzzy.** a fact can be stale, partially true, context-dependent. you sometimes have to re-litigate whether a thing is actually true or just seemed true last time. a wiki compounds but with noise. **a tool is binary.** it runs and returns the right thing or it doesn’t. the feedback loop is tighter, so the compounding is cleaner and faster. a tool that worked last month will work this month, assuming the underlying system didn’t change. you can build on it. moreover, functions and use-cases written in code can be paired with tests. more on this in section 8 below. **think about how a senior engineer operates.** they don't rewrite the same utility function every project. they have a personal library. trusted abstractions. “oh this is just a variant of X i’ve solved before.” the advantage of experience isn’t just that they're smarter, it’s that they've built a library they can reach for. your agent system should be that, except the library grows every task, not every year. a senior eng gets leverage through a library. why wouldn’t an agent? ![image](https://dzcjqzzfpmulffvcphbd.supabase.co/storage/v1/object/public/blog-post-images/blog-uploads/e6fe5b0c-00e0-4207-be64-23380990d939.jpg) ----- ## 7. the hard parts i'm not a snake oil salesman. **bad reflection equals duplicate junk.** the reflection step sounds simple: what was reusable? but it’s actually a code review on yourself. distinguishing a genuinely reusable abstraction from a one-off that just looks generalizable is non-trivial judgment. get the reflection wrong and you end up with a library of near-duplicate tools, almost-the-same utilities that should have been one thing, and the cost goes up instead of down. **warm bloat, the cold-start inverse.** the early days are clean because everything you add is obviously novel. something you’ve never solved before. but at 200 tools you have a retrieval problem: which tool? compose two existing tools? write new? the agent now has to navigate its own library. it's the tradeoff between “retrieve the right tool from my library” vs “bake the tool into the agent’s base knowledge.” retrieval isn’t free. **compositionality is not automatic.** it would be nice if the agent naturally discovered that two tools should be composed into one higher-level abstraction. it doesn’t, at least not reliably. you have to actively refactor. you have to treat the tool library like code. because it is code. these aren’t solved problems. they’re the interesting open problems, which is partly why the experiment is worth doing. if it were easy, everyone would already be doing it. ----- ## 8. why code wins: composable, testable, publishable ![image](https://dzcjqzzfpmulffvcphbd.supabase.co/storage/v1/object/public/blog-post-images/blog-uploads/dd493657-ea3f-4808-921b-eb98babb6e11.jpg) **tests are a contract, not just a quality gate.** a skill tells you _roughly_ what should happen. a test tells you what _actually does_ happen, given these inputs, this output, verified cases. tests double as documentation of how to use a tool. the test suite literally tells the agent what the tool is for, what inputs it expects, what output to expect back. this also doubles up as a few-shot example library for the agent. this helps reasoning, and also retrieval. when you compose two tools and write a test against the composition, you’ve created something the agent can treat as a black box it trusts, not a reasoning problem it has to solve. **composition and tiering fixes warm bloat.** you notice two tools share an underlying pattern, you extract it into a primitive, the higher-level tool calls down. small primitives at the bottom, composed utilities in the middle, use-case-level tools on top. the agent only needs to see the top of the stack for most tasks. it never has to touch the underlying plumbing. a good composition becomes its own improved test case. **the open source trajectory makes sense.** this is basically how unix evolved: small, composable, well-tested utilities shared because the abstractions were general enough to escape their original context. grep, sed, awk. people use them in ways the original authors didn’t anticipate. npm followed the same pattern. agent utils will too. small tools with clear contracts, test suites that act as readme and CI at once, shared across projects and eventually across teams. ----- ## 9. the architecture, all together **skills: intent and routing.** prose, fuzzy, interpretable. this is how the agent understands what someone wants. “when the user asks about their code review backlog, route to the github-my-prs-to-review intent.” **llm-wiki: synthesized knowledge.** known facts, inferred connections, not re-derived. a fact about your contract taxonomy. a definition your team agreed on. a pattern you’ve seen in documents. **tool library: capability layer.** code, composable, testable, versioned, publishable. a tool that does a thing and does it the same way every time. primitives that compose into utilities that compose into use-case solutions. a request flows top-down: user input → skill interprets intent → routes to tool chain → deterministic execution → outcome. the model’s job ends at routing. the tools handle execution. the metric that tells you it’s working isn’t accuracy. it’s token cost per successful outcome trending down over time. if cost isn’t falling, the agent is still reasoning through things it should already have solved. ![image](https://dzcjqzzfpmulffvcphbd.supabase.co/storage/v1/object/public/blog-post-images/blog-uploads/1c981e3d-7bde-4868-83ee-d792dd1608e0.png) ----- ## 10. what i'm actually doing about it this isn't just theory. i've been doing the structured-code version of this for a while now, somewhat instinctively, and only recently put the framing around it. i pull langsmith logs via api with a script i wrote, not through an mcp call every time. i have scripts that monitor stock and gold prices on a schedule, rather than asking an agent "what's the price today" every morning and paying for that reasoning each time. both of these started as things people told me to "just write a skill for." i did, but realized i can do this better when i wrote code. it runs, it returns the right thing, it costs nothing per call. same pattern in n8n: i have workflows for structuring user research notes, tracking grocery bills, a handful of other things that look like "agent tasks" on the surface. the reflex everyone has when they see these is to reach for a skill. my reflex now is to throw an agent at the problem once, understand what it's doing, and then collapse it into a deterministic workflow. let the model figure out the structure, then remove the model from the execution path. we've been doing this with sidebar too, which is the more interesting version. a user gives sidebar a task, the agent figures it out, and then we use that as signal to build specific tooling for that class of task. the agent is doing discovery, not production execution. coding agents already do a version of this by default: they write code to solve a problem, and the good ones accumulate that code as reusable context rather than rewriting from scratch. our explicit goal at sidebar is to keep giving the agent specific tools so it doesn't have to write code every time it needs to do something it's done before. a good example is redlining in sidebar. we could have claude write code to edit docx files on every redlining request. instead, we built specific tooling for docx editing. sidebar calls the tools, it doesn't reinvent the wheel. the toolkit is deterministic, testable, reliable. the model's job is to decide what edits to make, not to figure out how to manipulate a docx for the nth time. the pattern across all of this: throw an agent at it, understand what it figured out, then turn the repeatable parts into code. you're using the model's reasoning as a one-time cost to design the tool, not as a recurring cost to run it. the next experiment is taking this further and more deliberately, building a personal mcp toolkit as the explicit testbed for this whole framing. the self-growing, self-refactoring, composable util library, for real, on my own workflows first. testing whether the reflection loop actually works in practice. seeing if warm bloat is a real problem or whether a well-designed tool naming scheme is enough to stay navigable. building up abstractions and seeing if the composition layer works the way the theory suggests. the bet is that the agent six months in is structurally better, not just better-prompted. it has more tools, better tools, tools it built for itself. and the cost per successful outcome is lower because it's not reasoning through things it already solved. if it works, i'll have a library and anecdote worth sharing. if it doesn't, i'll have learned something concrete about what actually compounds and what doesn't. either way, that's the move. ## Images - https://dzcjqzzfpmulffvcphbd.supabase.co/storage/v1/object/public/blog-post-images/blog-uploads/36f832f4-287a-43a9-b771-c404de417a4d.png - https://dzcjqzzfpmulffvcphbd.supabase.co/storage/v1/object/public/blog-post-images/blog-uploads/6091b1a8-e660-4e94-9cac-c3ad3cd39d9c.png - https://dzcjqzzfpmulffvcphbd.supabase.co/storage/v1/object/public/blog-post-images/blog-uploads/a5e5516d-cc96-4103-ba13-d8038ac8bb5d.png - https://dzcjqzzfpmulffvcphbd.supabase.co/storage/v1/object/public/blog-post-images/blog-uploads/e6fe5b0c-00e0-4207-be64-23380990d939.jpg - https://dzcjqzzfpmulffvcphbd.supabase.co/storage/v1/object/public/blog-post-images/blog-uploads/dd493657-ea3f-4808-921b-eb98babb6e11.jpg - https://dzcjqzzfpmulffvcphbd.supabase.co/storage/v1/object/public/blog-post-images/blog-uploads/1c981e3d-7bde-4868-83ee-d792dd1608e0.png