packages.debian.org with an AI agent
The short answer: packages.debian.org is Debian's package directory, searched by name and release. We gave Claude Code 2 real tasks there, each in a brand-new session. With Playwright MCP a task used 254K tokens on average. With Dejo it used 150K the first time and 83K on every run after, 67% fewer, because Dejo replays a learned task without the model reading the pages again.
The tasks
- “search the Debian package directory for the package named moon-buggy in the oldstable distribution and tell me its version number”
- “look up the Debian package pacman4console in the stable release (trixie) and tell me its installed size on the armhf architecture”
What each tool used
| task | Dejo, first run | Dejo, every run after | Playwright MCP | Playwright CLI |
|---|---|---|---|---|
| 1 | 175K tokens, 18 s | 82K tokens, 1 call, 10 s, right | 215K tokens, 3 calls, 22 s, right | 451K tokens, 7 calls, right |
| 2 | 126K tokens, 15 s | 83K tokens, 1 call, 11 s, right | 293K tokens, 5 calls, 24 s, right | 717K tokens, 11 calls, right |
Calls are browser tool calls the agent made. Right or wrong is checked against the answer read off the live site before any tool ran.
Learned and replayed with Dejo on 13 November 2026
We ran the task again with Dejo on 13 November 2026, so you can see what it saves today. These are the steps Dejo kept, exactly as it reported them. Each one carries a check that it landed, and a replay walks them in plain code.
1. go https://www.debian.org/distrib/packages
2. fill ref "e83" (by name)
3. select combobox "Distribution:" "oldstable" (by name)
4. click button "Search" (by name)
5. click link "bookworm (oldstable)" (by name)
6. answer (by position)Replay: 6 steps in 0.7 s, with no page reading. Answer: “Package: moon-buggy (1:1.0.51-13)” (right).
A new task, once Dejo knows the site
In our site-memory test, the task “look up the Debian package pacman4console in the stable release (trixie) and tell me its installed size on the armhf architecture” ran twice in fresh sessions: once with an empty memory, and once after Dejo had learned a different task on packages.debian.org. It used 224K tokens the first way (5 turns) and 178K the second (4 turns), right both times.
How we measured
Claude Sonnet 5 at medium effort, a brand-new Claude Code session per task and tool, @playwright/mcp 0.0.80, October 2026. The full method, and the sites that blocked every tool, are on the benchmark page. Raw data: benchmark-real-2026-10-21.json, benchmark-real-2026-10-23.json, benchmark-real-2026-10-23-fixes.json, benchmark-site-memory-2026-11-10.json.
Other websites we measured
- scrapethissite.com: a sandbox site for practising web scraping, with tables, forms and paging
- rfc-editor.org: the official home of the RFC series of internet standards
- catalog.ccv.edu: the course catalog of the Community College of Vermont
- ctan.org: the Comprehensive TeX Archive Network, where LaTeX packages live
- practicesoftwaretesting.com: a demo tool shop built for practising test automation
- demowebshop.tricentis.com: Tricentis's demo web shop for test automation
- collection.internationalfolkart.org: the online collection of the Museum of International Folk Art in Santa Fe
- gutenberg.org: Project Gutenberg, a library of free ebooks
- pub.dev: the package site for Dart and Flutter
- hackage.haskell.org: the Haskell community's package archive
FAQ
How many tokens does an AI agent use on packages.debian.org?
In our test, Claude Code with Playwright MCP used 254K tokens per task on packages.debian.org on average. Dejo used 150K the first time and 83K on every run after, 67% fewer than Playwright MCP once the task was learned.
Can Claude Code automate packages.debian.org?
Yes. In fresh Claude Code sessions on 2 tasks there, Dejo answered 2 of 2 right, Playwright MCP 2 and Playwright CLI 2. With Dejo, a task done once is replayed later without the model reading the pages again.
Does a new task on packages.debian.org get cheaper once Dejo knows the site?
In our site-memory test, a new task there used 224K tokens with an empty memory and 178K after Dejo had learned a different task on the same site (5 turns against 4).