How to Build a Documentation Site with Thally

I built and maintain Agent Lookout, an open-source tool for developers who run more than one AI coding agent at a time. It is a command-line tool, a local dashboard and a Mac app. It shows you the Claude Code, Codex and other coding-agent sessions running on your machine, and which of them are waiting on you. It reads eight agents with no setup, and support is deepest for Claude Code.

The supported agents table on the docs site says exactly what each one gets.

Until this week it had a README and a guide file in the repository, but no documentation site.

In this tutorial, you will learn how I built a small documentation site for it with Thally, a docs-as-code engine that publishes from a GitHub repository. You will see the setup, the template, the writing, the configuration and the publishing step. You will also see every place where I got stuck, so you can avoid the same traps.

By the end, you will be able to reproduce a site like this one in an afternoon.

The finished site is at agent-lookout.thally.app.

The source is at github.com/Olanetsoft/agent-lookout-docs.

Prerequisites

To follow along, you need:

A GitHub account.
A project you know well enough to document. I used Agent Lookout. Any tool with a command you can run will do.
Node.js 22.12 or newer and npm. Agent Lookout needs that version to run, and you need it to build the site on your machine before you push.

How to Create the Site

Step 1: Site details

Enter a name, a one-line summary and a subdomain.

I used "Agent Lookout" as the name. For the summary, I wrote what the tool is and who the docs are for. For the subdomain I used agent-lookout, which became agent-lookout.thally.app.

Step 2: Docs source

Thally gives you three options. You can create a new repository from Thally's template, import an existing docs site from its URL, or connect a repository that already holds a Thally project.

I took the recommended option and let Thally create the repository.

Two things to know here.

First, the repository name must not already exist under your GitHub account. I had created an empty repository with the obvious name before I started, so the wizard refused it. I could only use that name after I deleted the empty repository.

Second, the "Private repository" toggle is on by default. If anyone else needs to read your source, switch it off before you continue.

Step 3: Product sources

This is the interesting step i find very useful. Thally asks which repositories change your product. Its Track feature reads merged pull requests in those repositories and works out what each one means for your documentation.

I selected only the Agent Lookout repository.

Step 4: Knowledge surfaces

This step is for other repositories where Thally may open draft pull requests, such as a marketing site or a help centre. I skipped it.

Step 5: Review and publish

Thally created the repository from its template and ran the first deployment.

Within a few minutes the site was live. It showed the template's own placeholder content: an introduction, a quickstart, a "Design your docs" group, an API Reference tab with a sample OpenAPI file, and a changelog.

How the Template Is Organised

Clone the new repository. These are the parts that matter:

src/content/ holds the pages as MDX files with front matter.
docs.json defines the navigation: tabs, groups and pages.
src/data/site.ts holds the site identity and the header links.
AGENTS.md is a file for coding agents. It has a product context placeholder for you to fill in.
npm run check runs Thally's checker. npm run build builds the site.

The README tells you to edit pages in src/content/ and to organise navigation in docs.json.

It does not describe the shape of docs.json. It does not list the front matter keys a page can use. It does not explain how the homepage works.

I learned all of that by reading the template's own pages. The homepage is the page whose front matter contains mode: home, and Thally serves it at /.

Here is the front matter of my introduction page, which is also the homepage:

---
title: Introduction
description: Agent Lookout shows the coding agent sessions on your machine on one page, with the ones waiting on you first.
mode: home
---

A page can set navTitle to show a shorter name in the sidebar. The full list of page modes lives only in src/data/docs.ts.

Navigation is a list of sections, each with groups and pages. This is the whole block for my site:

  "navigation": {
    "display": "dropdown"
  },
  "tabs": [
    {
      "tab": "Documentation",
      "groups": [
        {
          "group": "Get started",
          "pages": ["introduction", "quickstart"]
        },
        {
          "group": "Guides",
          "pages": ["guides/jump-to-window"]
        }
      ]
    }
  ]

I started with two top tabs, "Get started" and "Guides". On a three-page site, a tab that holds one page looks thin, so I moved to one section with "display": "dropdown". That removes the tab row and leaves one sidebar with two groups. One gotcha comes with it with a single section, Thally prints the section's name above the sidebar, and I hid it with a style rule in src/mdx/custom-components.tsx.

The <Hero> and <CardGroup> components on the homepage are not documented in the README either. The icon names come from a map in src/components/mdx/content-icon.tsx.

None of this is hard to find. All of it should be written down.

How to Write the Content

A small tool needs three pages:

An introduction. What the tool is, who it is for, and what it does not do.
A quickstart. Prerequisites, one command, one useful result, and how to stop.
Pick the feature with the most real behaviour behind it. For Agent Lookout that is jumping from the dashboard to the right VS Code window or tmux pane for a Claude Code session, which is the one agent Jump supports.

How to Configure the Site

Replacing the template content meant five jobs:

Rewrite the homepage hero and cards for Agent Lookout.
Replace the introduction and quickstart text.
Add the guide under the Guides group.
Remove the "Design your docs" group, the API Reference tab, the sample openapi.yaml and the changelog.
Point the header links at the Agent Lookout repository.

Five things were not obvious.

The Changelog link lives outside docs.json

Removing the changelog tab left a dead link in the header. That link is defined in src/data/site.ts, not in docs.json. The framework still builds /changelog/rss.xml after you remove the page.

Nothing tells you how to remove the API Reference

The README says to replace openapi.yaml. It does not say that you can remove the tab. After I removed both, llms.txt still advertised an OpenAPI spec, and that route now serves a spec for the docs site's own read-only API.

CI runs only on pull requests

The README implies that npm run check and the CI workflow run the same check. In fact the workflow only runs on pull_request, so a push to main runs nothing.

I ran thally check --ci, the typecheck and the build on my machine instead. I ran the checker twice, once with the latest CLI and once with the version CI pins, because the two can disagree. This is what the check prints when the site is clean:

> npx --yes --package=@thallylabs/cli@latest thally check --ci

thally check: 0 error(s), 0 warning(s)

Long inline code breaks tables

My quickstart had a table of start-up errors. On the live site, the long error messages pushed the "What to do" column out of view. I turned the table into a list. The guide's tables, with shorter cells, were fine.

A custom wrapper works locally and not on the live site

A wrapper component in src/mdx/custom-components.tsx works in npm run dev and in a local build. The published site renders pages through an interpreter that ignores it. Named components work in both places, so each page now starts with a <PageLayout /> component instead. Related: every page started 40px lower than its padding says, because a hidden <script> is the first child of a spaced container. One style rule removes the gap.

How to Publish

Page edits appeared on the live site within a minute or two. The change to src/data/site.ts took several minutes, because a config change triggers a full rebuild. The only place that explained the difference was a guide in the template content I had already deleted.

The identity surprise

I changed the site description in src/data/site.ts. My local build's llms.txt showed the new text. The live llms.txt did not.

Here is why. On a Thally Cloud site, the name, description and repository URL saved in the Cloud dashboard override site.ts. The file is only a fallback. The fix lives in the dashboard settings, not in the repository. Saving the new description there was not enough on its own either. It went live about three minutes after I pressed Publish in the dashboard.

That Publish did more than I expected. It committed "chore: publish saved site settings" to main and rewrote the header links in site.ts: Support and GitHub now pointed at the docs repository instead of the product, and a Changelog link to a page that does not exist came back. I restored them by hand.

Also, llms.txt is cached for an hour, so an identity change can take that long to show up.

Build-time variables

If you host the site yourself, set THALLY_SITE_URL when you build, not when you run. Pages are prerendered. If you set the variable only at runtime, every canonical URL points at localhost:3040.

Two things that happened without me

A Thally bot pushed a framework upgrade straight to main. The commit was called "chore: upgrade starter" and changed 1,230 lines across 22 files, with no pull request. The wizard had said future changes arrive as pull requests, and the template's own AGENTS.md says upgrades come as a reviewed pull request that is never merged automatically. I only noticed because my next push needed a pull first.

I also tested Track. I merged a pull request in the Agent Lookout repository that contained the same three pages. Track picked up the merge within a minute, compared it with the docs repository, and reported "no changes needed" after about four minutes. That was the correct call. It still opened a pull request on the docs repository, whose only change was its own output/run-report.md. The run cost about 490 of the 50,000 included credits.

A docs-only merge that changes nothing on the site is the case I would want it to skip, with no pull request at all.

Every Obstacle, in One List

The repository name collided with one I had created in advance.
"Private repository" is on by default.
The docs.json schema, the front matter keys and the homepage mode are not in the README.
The hero and card component props are not documented.
The header Changelog link is defined outside docs.json.
There is no documented way to remove the API Reference, and llms.txt keeps advertising a spec.
CI runs only on pull requests, and the local and CI checker versions can differ.
The Cloud dashboard silently overrides site.ts.
THALLY_SITE_URL must be set at build time.
Every page logs a 404 for the header logo until you set one.
Config changes deploy more slowly than page changes, and only the deleted starter content said so.
A bot pushed a 1,230-line upgrade directly to main with no pull request.
A Track run on a docs-only merge that needed no changes still cost about 490 credits.
Tables with long inline code overflow on the live site.
The install is 757 npm packages for a three-page site, and Thally's own quickstart opens with a 17-step prompt for a coding agent before the steps for a person.
A custom wrapper component works locally and is ignored on the published site.
Every page starts 40px lower than its padding says, because of a hidden script in the layout.
Publishing from the dashboard commits to main and rewrites the header links in site.ts.
Track opened a pull request containing only its own run report after concluding nothing needed to change.
With one navigation section, the section name prints above the sidebar and has to be hidden by hand.

What I Would Change First

Two additions to the README would remove most of that list.

The first is a short reference for docs.json, the front matter keys and the homepage components, kept somewhere that survives deleting the sample pages.

The second is a plain statement that on Thally Cloud the dashboard owns the site identity, with site.ts as the fallback, or a warning when the two disagree.

After those, ship framework upgrades as the reviewed pull requests the template promises, make the dashboard's Publish open a pull request instead of committing to main, add one command that removes all the sample content consistently, make tables wrap long code, and have Track stay quiet when it has nothing to propose.

Conclusion

You have seen how to take a tool with only a README and give it a documentation site on Thally, create the site in the wizard, learn the template, write three pages from the code, replace the sample content, and publish with a push.

You have also seen where the sharp edges are. The wizard's repository name check, the private toggle, the undocumented docs.json schema, the dashboard override, the build-time variable, and the upgrade that landed on main without a pull request.

The site took an afternoon. Most of that afternoon went into the writing, which is where it should go. Thally handled the rest, and where it did not, you now know what to look for.

If you run more than one coding agent at a time, you can try Agent Lookout with one command: npx agent-lookout. The project site is agentlookout.app and the docs are at agent-lookout.thally.app.