Agent Ecosystem Design & Audit
- ·The audit runs first: which agents get dispatched, which are never called, and which overlap enough that one of them is dead weight.
- ·What follows is a smaller roster and a written admission standard, so the next addition has to argue for itself.
- ·The measuring tool is published and openable, and the finding it returned on this ecosystem is on the record with it.
| Built with | TypeScript · Next.js · Node.js · Vercel |
| Delivered by | chief-of-staff · prompt-engineer · knowledge-librarian |
| Skills used | agent-forge · ecosystem-groomer |
Receipts
What this involves
Most organisations that have been building agents for a year do not need more of them. They need to know which ones are being used, which are never called, and which two overlap enough that one is paying rent for nothing. That is an audit, and it is a different engagement from a design.
The measuring instrument is published and openable. polis reads an agent ecosystem off disk, renders it as a city you can walk around, and audits it while reading: 59 agents, 86 skills, 8 divisions, 6 guilds, 219 declared edges (data/ecosystem.json, generated 2026-09-05; npm run snapshot). Pointed at this ecosystem it found 8 unreachable members and 33 never-dispatched agents, out of 59, and the never-dispatched figure came from the dispatch log rather than from an opinion about which agents felt useful.
Neither finding was comfortable and both are on the record, because an audit that flatters the thing it audits was not an audit. The eight unreachable members were a violation of the ecosystem's own written reciprocity rule that its own weekly groomer had not caught, which is the ordinary way governance rules fail: quietly, in a document everyone agrees with.
What follows an audit is a smaller roster and a written admission standard, so the next addition has to argue for itself against the members that already exist. The standard matters more than the cut: a roster trimmed once with no rule attached grows straight back.
Pointed at its own author's ecosystem, polis found 8 unreachable members and 33 never-dispatched agents, out of 59. The second figure came from the dispatch log, not from an opinion.
How the work runs
1. Discovery
We agree what the project has to achieve, what it must not break, and how we will both know it worked, before any code is written.
2. Build
The work is built with an AI agent as the working partner and a human reviewing every change before it lands. You get commits you can read rather than a status update.
3. Review
You review running software at each milestone, not a screenshot of it, and the next milestone absorbs what you send back.
4. Launch
We deploy to production with checks in the pipeline and alerting that reaches a human, and you hold the repository and the infrastructure accounts.
Questions people actually ask
Why start with an audit rather than a design?
Because the common failure is not too few agents, it is too many that nobody dispatches. An audit is cheap, it is checkable, and it tells you whether a design engagement is even the right purchase. If the finding is that a third of the roster has never been called, the useful next step is a deletion, not a deck.
What does the audit actually read?
The charter files on disk and the dispatch log, which are two different sources answering two different questions. The charters say what the ecosystem claims about itself, including which members are supposed to hand work to which. The dispatch log says what actually ran. Governance findings come from the first; usage findings come only from the second.
Is this tied to one vendor's agent framework?
The reading is done over files and logs, so the method carries across frameworks. What does not carry is the specific parser: a new layout needs one written for it, and that is scoped work rather than a configuration setting. The version in this repository reads the layout it was built for and says so.