
I built Agent Lookout and I maintain it. It is an open-source tool for developers who run more than one AI coding agent at the same time. It has three parts: a command-line tool, a local dashboard and a Mac app. It shows the Claude Code, Codex and other coding-agent sessions on your machine. It also shows which sessions wait for you. It reads eight agents with no setup. Support is deepest for Claude Code.
The supported agents table on the docs site tells you what each agent gets.
Until this week, the project had a README and a guide file in the repository. It did not have a documentation site.
In this tutorial, you will learn how I built a small documentation site for it with Thally. Thally is a docs-as-code engine that publishes from a GitHub repository. The tutorial shows the setup, the template, the content, the configuration and the publish step. It also shows each problem I found, so that you can prevent the same problems.
When you finish, you will be able to build a site like this one in an afternoon.
The finished site is at agent-lookout.thally.app.
The source is at github.com/Olanetsoft/agent-lookout-docs.
Prerequisites
You need these items:
- A GitHub account.
- A project that you know well enough to document. I used Agent Lookout. Any tool with a command that you can run is sufficient.
- Node.js 22.12 or newer, and npm. Agent Lookout needs that version to run. You also need it to build the site on your machine before you push.
How to Create the Site
Sign up at thally.app. Then open the Create site wizard. The wizard has five steps.
Step 1: Site details
Enter a name, a one-line summary and a subdomain.
I used "Agent Lookout" as the name. In the summary, I said what the tool is and who the docs are for. I used agent-lookout as the subdomain. The site address became agent-lookout.thally.app.
Step 2: Docs source
Thally gives you three options:
- Create a new repository from the Thally template.
- Import an existing docs site from its URL.
- Connect a repository that already contains a Thally project.
I used the recommended option. Thally created the repository.
You must know two things.
First, the repository name must not exist in your GitHub account. I created an empty repository with the same name before I started. The wizard refused the name. I deleted the empty repository, and then the wizard accepted the name.
Second, the "Private repository" toggle is on by default. If other people must read your source, set the toggle to off before you continue.
Step 3: Product sources
This step was the most useful to me. Thally asks which repositories change your product. The Track feature reads the merged pull requests in those repositories. Then it finds what each pull request changes in your documentation.
I selected only the Agent Lookout repository.
Step 4: Knowledge surfaces
This step is for other repositories where Thally can open draft pull requests. Examples are a marketing site or a help centre. I skipped this step.
Step 5: Review and publish
Thally created the repository from its template. Then it ran the first deployment.
The site was live in a few minutes. It showed the placeholder content of the template: an introduction, a quickstart, a "Design your docs" group, an API Reference tab with a sample OpenAPI file, and a changelog.
How the Template Is Organised
Clone the new repository. These parts are important:
src/content/contains the pages. The pages are MDX files with front matter.docs.jsondefines the navigation: tabs, groups and pages.src/data/site.tscontains the site identity and the header links.AGENTS.mdis a file for coding agents. It has a placeholder for the product context. You fill in the placeholder.npm run checkruns the Thally checker.npm run buildbuilds the site.
The README tells you to edit the pages in src/content/. It also tells you to organise the navigation in docs.json.
The README does not give this information:
- The structure of
docs.json. - The front matter keys that a page can use.
- How the homepage works.
I found this information in the pages of the template. The homepage is the page that has mode: home in its front matter. Thally serves that page at /.
This is the front matter of my introduction page. This page is also the homepage:
---
title: Introduction
description: Agent Lookout shows the coding agent sessions on your machine on one page, with the ones waiting on you first.
mode: home
---
A page can set navTitle to show a shorter name in the sidebar. The full list of page modes is only in src/data/docs.ts.
The navigation is a list of sections. Each section has groups and pages. This is the full block for my site:
"navigation": {
"display": "dropdown"
},
"tabs": [
{
"tab": "Documentation",
"groups": [
{
"group": "Get started",
"pages": ["introduction", "quickstart"]
},
{
"group": "Guides",
"pages": ["guides/jump-to-window"]
}
]
}
]
At first, I had two top tabs, "Get started" and "Guides". The site has three pages, so one tab contained only one page. That looked empty. I changed to one section with "display": "dropdown". This setting removes the tab row. One sidebar with two groups remains.
There is one side effect. With a single section, Thally shows the section name above the sidebar. I hid the name with a style rule in src/mdx/custom-components.tsx.
The README also does not document the <Hero> and <CardGroup> components on the homepage. The icon names come from a map in src/components/mdx/content-icon.tsx.
None of this is difficult to find. Thally should write all of it down.
How to Write the Content
A small tool needs three pages:
- An introduction. It says what the tool is, who it is for, and what it does not do.
- A quickstart. It gives the prerequisites, one command, one useful result, and how to stop the tool.
- A guide. Select the feature that has the most real behaviour. For Agent Lookout, that feature is Jump. Jump goes from the dashboard to the correct VS Code window or tmux pane for a Claude Code session. Claude Code is the only agent that Jump supports.
How to Configure the Site
To replace the template content, do these five tasks:
- Rewrite the homepage hero and cards for your tool. I wrote them for Agent Lookout.
- Replace the introduction and quickstart text.
- Add the guide to the Guides group.
- Remove the "Design your docs" group, the API Reference tab, the sample
openapi.yamland the changelog. - Point the header links at your product repository.
Five things were not clear.
The Changelog link is not in docs.json
After I removed the changelog tab, the header had a dead link. src/data/site.ts defines that link. docs.json does not. The framework continues to build /changelog/rss.xml after you remove the page.
The README does not tell you how to remove the API Reference
The README tells you to replace openapi.yaml. It does not tell you that you can remove the tab. I removed the tab and the file. After that, llms.txt continued to show an OpenAPI spec. That route now serves a spec for the read-only API of the docs site.
CI runs only on pull requests
The README suggests that npm run check and the CI workflow run the same check. But the workflow runs only on pull_request. Thus a push to main runs no checks.
I ran thally check --ci, the typecheck and the build on my machine. I ran the checker two times: one time with the latest CLI, and one time with the version that CI pins. The two versions can give different results. When the site has no problems, the check shows this:
> npx --yes --package=@thallylabs/cli@latest thally check --ci
thally check: 0 error(s), 0 warning(s)
Long inline code breaks tables
My quickstart had a table of start-up errors. On the live site, the long error messages pushed the "What to do" column out of view. I changed the table to a list. The tables in the guide have shorter cells, and they were correct.
A custom wrapper works locally but not on the live site
A wrapper component in src/mdx/custom-components.tsx works in npm run dev and in a local build. The published site renders pages through an interpreter. The interpreter ignores the wrapper. Named components work in the two environments. Thus each page now starts with a <PageLayout /> component.
There is a related problem. Each page started 40px lower than its padding specifies. The cause is a hidden <script> that is the first child of a spaced container. One style rule removes the gap.
How to Publish
To publish, push to main. Page edits showed on the live site in one or two minutes. The change to src/data/site.ts took several minutes, because a config change starts a full rebuild. Only one guide in the template content explained the difference. I had already deleted that guide.
The identity surprise
I changed the site description in src/data/site.ts. The llms.txt of my local build showed the new text. The live llms.txt did not.
This is the cause. On a Thally Cloud site, the Cloud dashboard saves the name, description and repository URL. These values override site.ts. The file is only a fallback. You must make the correction in the dashboard settings, not in the repository.
It was also not sufficient to save the new description there. The description went live approximately three minutes after I pressed Publish in the dashboard.
That Publish did more than I expected. It committed "chore: publish saved site settings" to main. It also changed the header links in site.ts. The Support and GitHub links now pointed at the docs repository, not at the product. A Changelog link to a page that does not exist came back. I restored the links manually.
Also, llms.txt has a cache of one hour. Thus an identity change can take one hour to show.
Build-time variables
If you host the site yourself, set THALLY_SITE_URL when you build, not when you run. Thally prerenders the pages. If you set the variable only at runtime, each canonical URL points at localhost:3040.
Two things that occurred without me
A Thally bot pushed a framework upgrade directly to main. The commit name was "chore: upgrade starter". It changed 1,230 lines in 22 files, and it had no pull request.
The wizard said that future changes come as pull requests. The AGENTS.md of the template says that an upgrade comes as a reviewed pull request and that nothing merges it automatically. I saw the commit only because my next push needed a pull first.
I also tested Track. I merged a pull request in the Agent Lookout repository. The pull request contained the same three pages. Track found the merge in less than one minute. It compared the merge with the docs repository. After approximately four minutes, it reported "no changes needed". That result was correct.
But Track also opened a pull request on the docs repository. The only change in that pull request was its own output/run-report.md. The run used approximately 490 of the 50,000 included credits.
A docs-only merge that changes nothing on the site is the case that I want Track to skip. It should not open a pull request.
Problems I Found
None of these problems stopped me. But each one used time. For each one, this list tells you what occurred and what to do.
- The wizard refuses a repository name that you already own. I created an empty
agent-lookout-docsrepository before I started. The wizard did not use it. Let Thally create the repository, or delete the empty repository first. - New repositories are private by default. The Private toggle is on in the Docs source step. If other people must read the source, set the toggle to off before you continue.
- The README does not describe the files that you must edit. It does not document the
docs.jsonschema, the front matter keys or themode: homehomepage mechanism. It also does not document the props of<Hero>and<CardGroup>. Read the pages of the template andsrc/data/docs.tsbefore you delete them. They are the only reference. - The header links are in
src/data/site.ts, not indocs.json. When I removed the Changelog tab, a dead Changelog link stayed in the header. The framework also continued to build/changelog/rss.xml. Editsite.tseach time you remove a page. - There is no documented procedure to remove the API Reference. After I deleted the tab and
openapi.yaml,llms.txtcontinued to show a spec at/openapi.yaml. Examinellms.txtafter you remove sample content. - CI runs only on pull requests. The README suggests that CI runs the same check as
npm run check. But a push tomainruns no checks. Also, the CLI version that CI pins can be different from the version that you installed. Before you push, runthally check --cilocally with the two versions. - The Cloud dashboard overrides
site.tsand does not tell you. I changed the site description insite.ts. The local build showed the change. The livellms.txtdid not. On a Cloud site, the name, description and repository URL in the dashboard have priority. Change the identity in the dashboard.llms.txtcan take one hour to update because of its cache. - A publish from the dashboard commits to
main. When I pressed Publish, Thally added a "chore: publish saved site settings" commit. The commit changed the header links insite.ts. It pointed Support and GitHub at the docs repository, and it brought back the dead Changelog link. Pull before your next push. Examinesite.tsafter each dashboard publish. - A bot pushed a 1,230-line upgrade directly to
main. The wizard and theAGENTS.mdof the template say that upgrades come as reviewed pull requests. My upgrade came as a direct commit that changed 22 files. I saw it only because Git rejected my next push. Always pull before you push. - Set
THALLY_SITE_URLat build time. Thally prerenders the pages. If you set the variable only at runtime, the canonical URLs point atlocalhost:3040. This is important only if you host the site yourself. - Config changes deploy more slowly than page changes. A page edit was live in two minutes. A change to
site.tstook several minutes because it starts a full rebuild. Only one starter page explained this, and I had already deleted that page. - A custom
wrappercomponent works locally, but the published site ignores it. The published site renders MDX through an interpreter that does not apply the wrapper. Use named components. They work in the two environments. - Tables with long inline code overflow on the live site. The error messages in my quickstart table pushed the next column out of view. Use a list when a cell contains long code.
- Track does work when there is nothing to change. I merged a docs-only pull request in the product repository. Track correctly found that the site needed no changes. But it used approximately 490 credits to find that. It also opened a pull request that contained only its own
output/run-report.md. Close those pull requests. Plan for the credits that merges with documentation changes use. - Each page has small layout problems. Each page logs a 404 for the header logo until you set a logo. Each page starts 40px lower than its padding specifies, because a hidden script is the first child of a spaced container. With a single navigation section, the section name shows above the sidebar. You need a style rule to hide it.
- The install is large for the site that it builds. A three-page site needs 757 npm packages. Also, the Thally quickstart starts with a 17-step prompt for a coding agent. The steps for a person come after that.
What I Would Change First
Two additions to the README would remove most of that list.
The first addition is a short reference for docs.json, the front matter keys and the homepage components. Keep it in a location that stays when you delete the sample pages.
The second addition is a clear statement about Thally Cloud. The statement must say that the dashboard owns the site identity and that site.ts is the fallback. An alternative is a warning when the two values are different.
After those additions, I would make these changes:
- Ship framework upgrades as reviewed pull requests, as the template promises.
- Make the Publish button in the dashboard open a pull request. It must not commit to
main. - Add one command that removes all the sample content consistently.
- Make tables wrap long code.
- Make Track do nothing when it has no change to propose.
Conclusion
You saw how to give a documentation site on Thally to a tool that had only a README. You create the site in the wizard. You learn the template. You write three pages from the code. You replace the sample content. Then you publish with a push.
You also saw where the problems are:
- The repository name check in the wizard.
- The private toggle.
- The undocumented
docs.jsonschema. - The dashboard override.
- The build-time variable.
- The upgrade that went to
mainwithout a pull request.
The site took an afternoon. I used most of that time to write the content, which is correct. Thally did the other work. Where it did not, you now know what to look for.
If you run more than one coding agent at the same time, try Agent Lookout with one command: npx agent-lookout. The project site is agentlookout.app and the docs are at agent-lookout.thally.app.
About the author
Software Engineer | Developer Advocate | Technical Writer | Content Creator
Tutorials
In this series, I'll be sharing incredible content on How- to's, Building and Deploying Applications using Modern Web Technologies.



