Playwright MCP token usage, measured over 90 sessions

Where the savings come from

Look at how many tool calls each session made, run by run:

Playwright MCP and Chrome DevTools MCP start from nothing in every session. The model opens the page, reads it, decides, clicks, reads again. It does the same work on run 10 as on run 1, and sometimes more: one Playwright MCP session took 18 calls.

Dejo works differently. On run 1 the agent explores, and Dejo saves the route with a check on every step. From then on, one dejo_run call replays the whole route in plain code, with no page reading and no model calls inside Dejo. The two calls you see are the agent finding the tool and calling it.

Averaged over the three tasks:

Where Dejo is not cheaper

Two honest limits, both visible in the data:

• Run 1 costs the most. Dejo's first session used about 415K tokens, more than either other tool, because the agent does the job and saves the route. You get that back from the second run on.
• One-page lookups. If the task is "open one page, read one line", every tool needs two or three calls and there is little for memory to save. In our one-page tests on six sites, ten sessions each, Dejo used fewer tokens than Playwright MCP on pkg.go.dev and more on the other five.

Also worth knowing: most of the 104K tokens in a warm Dejo session are not Dejo at all. They are the agent's own instructions and tool list, which every session loads, and which the model provider caches. That fixed cost is the same for every tool, which is why the total saving is 47% and not 90%.

How we tested

• Agent: Claude Code in headless mode (claude -p), model Claude Sonnet 5 at medium effort.
• One tool per session. Each session could use only the browser tool under test. Built-in web tools such as WebFetch were turned off.
• A brand-new session every run, so nothing carried over in the chat. Dejo's memory on disk is the only thing that persists, and that is the point being measured.
• The tasks, each asked the same way: "Open <url> in the browser and <task>. Reply with only that text and nothing else."
  1. github.com/microsoft/playwright: open the Issues tab and say how many open issues there are.
  2. quotes.toscrape.com: open the About page of the first quote's author and say when they were born.
  3. books.toscrape.com: open the Travel category, open the first book, and say its price.
• Versions pinned: @playwright/mcp@0.0.80 and chrome-devtools-mcp@1.8.0, so a rerun measures the same thing.
• Tokens are everything the session processed: input, output, cache reads and cache writes, as reported by Claude Code. Tool calls include the agent's tool-search calls.
• Answered means the session finished without an error and replied with a value. The script does not check the value itself, so we checked every reply by hand:
  • Quotes and books: all 60 replies from every tool were right (March 14, 1879 and £45.17).
  • GitHub: the page showed two different open-issue counts at the time, 155 and 157. Dejo's saved route read 157 on runs 2 to 10, Playwright MCP said 155, Chrome DevTools MCP said 155 nine times and 157 once. One Playwright MCP reply said "1,155", which was wrong.

What we did not test: other models, other agents, long tasks with many pages, and sites that need a sign-in. Playwright MCP has had newer releases since this run.

Run it yourself

The script and the raw results are in the open-source repository:

git clone https://github.com/dejohq/dejo-mcp
cd dejo-mcp
python benchmarks/benchmark_agents.py --journeys --runs 10

It needs the claude command signed in, and Node for the two npx servers it compares against. Every number on this page comes from benchmarks/benchmark-journeys10.json. The file was recorded while Dejo was still called by its old name, so it says cairn where it means Dejo.

FAQ