Five tasks from the documentation alone

Can an agent that has never seen hadley work on a hadley site with nothing but its documentation? This is how I measured it, in November 2026, and what came out.

The setup

• The site: new, made with hadley init --language en,de and the theme hems, nothing changed.
• The program: hadley 0.3.0 for Linux, from the release, checked against its SHA256SUMS.
• The agent: Codex with the model gpt-6.1-sol, in a virtual machine without web access. Only the model’s provider was reachable.
• The documentation: an offline copy of hadley’s user documentation and of the pages on Markanto and Lima, about 26,000 tokens, counted as bytes divided by four.

The tasks

1. A new article, “A week in Porto”, in English and German, with a date, a description, two tags and a picture as its cover.
2. A short note that points to hadley’s repository.
3. An author for the article, Rosa Kessler, with a short bio in both languages.
4. The site always dark, without the button that switches between light and dark.
5. Comments turned on, for a service at a given address.

And at the end, the site had to build for publishing without a problem.

The result

All five were done, and the site built, in 3 minutes 34 seconds. The agent read 9 of the 15 documentation files. I built its result again myself and looked: the article in both languages with its cover and its author, the note, the author’s German bio, the site dark without the button, the comment form pointing to the service. It also added a test file that no task asked for.

Without the documentation

The same tasks, the same agent, the same site, but no documentation: all five done as well, in 4 minutes 3 seconds. It found its way through the sample content of the new site, the README that comes with the program and --help. It changed 12 files instead of 9, the home page’s template among them, which no task asked for.

So the documentation made the agent faster and kept it to the task. And a new hadley site explains itself well enough that an agent finds its way even without it.

What both runs found

Both ran into the same two gaps in hadley’s own themes, and wrote the same fixes for them: hems and punch had no template for notes, and a reader’s earlier choice of light could override a site set to dark. Both are fixed for hadley 0.4.0.

What this does not show

One agent, one model, one run each, on one machine. No browser looked at the pages, and no comments service ran. It shows that the documentation is enough for everyday work, not how every agent will do.