SkillWire aims to measure whether a coding-agent 'skill' actually works, not just how many stars it has

By Hacker News (SkillWire) · October 30, 2026.

SkillWire is a searchable catalog of "skills" for coding agents, presented as a "Show HN" on Hacker News. A skill is a folder of instructions and supporting files (with a SKILL.md) that an agent such as Codex or Claude Code can load for a specific task. At the time of publication, the Hacker News post had 2 points and no comments: it is a very early-stage project and should be treated as such.

According to the site itself, the catalog lets users filter by category, execution environment (harness) and evaluation evidence. Install counts come from skills.sh and stars are those of the source repository. Users can assemble a "stack" of up to ten public folders and download it as a ZIP with the original bytes, file hashes and any lines that triggered warnings. According to the site, SkillWire does not execute the downloaded files. It also publishes "Research picks" (an October 28, 2026 selection) which, it says, are not ranked by stars or payment.

The most distinctive feature is the paired-test workflow. The user picks a skill, freezes the task, the evaluator, the settings and the order of the arms, and runs the same throwaway project twice locally: with the skill and without it. They then judge the outputs blind (A/B, with a tie or "can't judge" as valid outcomes) before revealing which arm had the skill. Everything runs on the user's subscription; according to the site, it uses no API keys or hosted evaluation service.

The site also documents its own results, and this is where the substance lies. According to SkillWire, in a first family of ten pairs with a pinned TDD (test-driven development) skill, the task was passed 8 out of 10 times with the skill and 7 out of 10 without it. The body of the skill appeared in only one tool trace; in nine reads it could not be observed, so internal usage is unknown. In addition, the frozen reference accepted an ASCII DEL character where the task required rejecting it, so the site itself rates that family as disputed. It also cites an earlier benchmark (SkillsBench) in which the skill was opened in only 3 of 8 tasks and no improvement distinguishable from noise was detected. These are the site's own figures, not independently verified, and the site itself warns that they should not be combined or read as general proof.

**Our take.** The thesis: the skills ecosystem has a measurement problem before a supply problem. Today, skills are chosen by stars and installs, which measure popularity, not usefulness. A catalog openly admitting that its 8 out of 10 versus 7 out of 10 does not amount to a claim of benefit is rare, and it is exactly the attitude we call for when evaluating models: measure rather than believe the marketing.

There is also a security angle. A skill can include scripts that run with the agent's permissions; that is why it is sensible for SkillWire to recommend reviewing license and warnings and testing in a throwaway project without credentials. It connects with an idea we champion: as agents gain the ability to act, trust (provenance, permissions, auditing) becomes as important as the model.

Clear limits: a single task per test does not generalize, the project is new with no visible traction, and the catalog showed "0 entries" in the snapshot we analyzed, probably due to dynamic loading, so we cannot assess its real coverage. More broadly, if these local paired tests catch on, skills could go from repository folklore to evidence-backed components. It is a small piece of the transition toward reliable agents: unglamorous, but the kind that ensures the software abundance we promise in the long term arrives with guarantees.