For AI agents: the index of this site is https://dejo.run/llms.txt, and every page is also served as markdown (add .md to its address, or send Accept: text/markdown).

dejo

Playwright MCP vs Playwright CLI vs memory (measured)

BlogBy Rohit Kashyap

I build Dejo, one of the three options here, so read this with that in mind. The numbers come from a benchmark you can rerun and from runs I did by hand on 4 November 2026. Where Dejo loses, I say so.

Short answer: for a task you'll do once, Playwright MCP and Playwright CLI cost about the same, and the MCP was cheaper in our Claude Code test (409K vs 559K tokens on average over 19 real-site tasks). The CLI got one more answer right, and it got through 2 of the 6 blocked tasks (both on one site) where the others got through none. For a task you'll do again, both start from nothing each time; a browser that remembers the task (Dejo) used 251K tokens the first time and 89K every time after.

if you…use
explore a site once, or write Playwright testsPlaywright MCP or Playwright CLI
work in an agent that's good with a shell and skillsPlaywright CLI
need Firefox or WebKitPlaywright MCP or CLI (Dejo drives Chrome or Chromium only)
hit sites that block automated browsersPlaywright CLI got through 2 of the 6 blocked tasks in our test (both on one site); the others got through none
repeat the same website task in new sessionsa tool with memory, such as Dejo

The three options in one paragraph each

Playwright MCP is Microsoft's MCP server. The agent calls tools like browser_navigate and browser_click and reads back a snapshot of the page, an outline of every element with a reference like e21.

Playwright CLI is the same set of actions as shell commands (playwright-cli open, click e21, snapshot) plus a skill file that teaches the agent how to use them. Its README says "If you are using coding agents, that is the best fit", because "CLI invocations are more token-efficient: they avoid loading large tool schemas and verbose accessibility trees into the model context." It keeps MCP for "long-running autonomous workflows where maintaining continuous browser context outweighs token cost concerns." Like Playwright MCP, it's free and open source under Apache 2.0 (its LICENSE file).

Memory is the third option, and it's what Dejo does. The first time, the agent works out the task step by step like it would with the other two. Dejo saves the steps with a check on each one. The next session calls dejo_run once and Dejo walks the saved steps in plain code, with no page reading and no model calls inside Dejo.

What we measured in Claude Code

19 tasks on 11 lesser-known websites (a college course catalog, the Debian package pages, the RFC Editor, a museum collection, a Dart package registry and others), each in a brand-new Claude Code session, Claude Sonnet 5 at medium effort, @playwright/mcp@0.0.80. The right answer to every task was written down before any tool ran.

per task, averagedright answerstokens (average)tokens (median)timecost
Playwright MCP17 of 19409K257K40 s$0.23
Playwright CLI18 of 19559K496K38 s$0.28
Dejo, first run19 of 19251K216K31 s$0.17
Dejo, every run after19 of 1989K82K16 s$0.02

Cost is what Claude Code reported at list price. The Playwright rows and Dejo's rows were recorded one day apart, and Dejo's are from the release code after fixing what this benchmark found. The results files are benchmark-real-2026-10-21.json (all three tools) and -10-23.json with -10-23-fixes.json (Dejo).

The CLI result surprised me, because its README says it saves tokens. Per task, it used more than the MCP on 16 of the 19 tasks. The three where it used fewer were the longest ones. They are the first three rows below. The last two rows are the opposite case, two tasks where the CLI used far more:

taskPlaywright MCPPlaywright CLIDejo first runDejo after
college catalog, filter by subject, read credits (CLI used fewer)1,355K1,167K324K83K
college catalog, second task (CLI used fewer)1,005K973K480K82K
museum collection, two filters (CLI used fewer)983K671K398K166K
Debian package lookup (CLI used more)293K717K126K83K
Haskell package page (CLI used more)331K958K331K79K

I don't know the whole reason. Two things in the data: the CLI sessions took more turns (9.4 on average against 8.4), and every turn sends the conversation so far back to the model, so turns add up. And the CLI sessions started bigger: the median first turn was 48.6K tokens of context against 39.7K for the MCP. Across all 25 CLI sessions, Claude made 201 Bash calls, loaded the skill 24 times and read a snapshot file 19 times.

There's one place the CLI clearly won. Six more tasks, on three sites, were blocked for automated browsers (Cloudflare checks and plain 403s). Playwright MCP and Dejo were stopped on all six. The CLI got through 2 of the 6 blocked tasks (both on one site), at 1.2M and 1.7M tokens. Those six are in the results file and not in the table.

They now return the same page, the same way

On 4 November 2026 I installed the current versions, @playwright/mcp@0.0.84 and @playwright/cli@0.1.23, and drove both by hand (no model) through one small task on the practice site books.toscrape.com: open the home page, click Travel, open the first book, read its price.

Both wrote snapshot files of the same size: 31,876 bytes for the home page, 21,318 for the Travel list and 10,174 for the book. I compared the sizes, not the contents. The first line of playwright-cli --help is "playwright-cli - run playwright mcp commands from terminal", and both packages pin the same playwright-core build, so this makes sense: it's one engine with two front doors.

What changed since our benchmark is where the snapshot goes. In 0.0.84, a click or navigate reply was 384 to 511 bytes and pointed to the snapshot file; only an explicit browser_snapshot call put the full outline (32,037 bytes for the home page) in the reply. The CLI works the same way. So the old "the MCP pushes a snapshot after every step" complaint is less true than it was. I haven't rerun the Claude benchmark on 0.0.84, so I can't say how much that saves in a real session.

Others have seen the same thing. The post most people quote for "about 114K tokens with MCP, 26K to 27K with the CLI" added an update on 24 July 2026: rerun with @playwright/mcp@0.0.78 and @playwright/cli@0.1.17, the two returned about 3,580 and 3,620 characters on the same seven-step login flow, and the author wrote "The ~4× token gap has essentially closed." GitHub's agentic workflows team moved their built-in Playwright tool to the CLI in September for other reasons: tool schemas loaded on every turn, a smaller thing to audit, less configuration to drift. They didn't publish measurements.

The other fixed cost is the tool list, which an MCP client loads into every session:

what the agent has to readPlaywright MCP 0.0.84Playwright CLI 0.1.23Dejo 0.4.0
tool definitions25 tools, 22,121 bytesnone (it's a shell command)5 tools, 4,260 bytes
instructionsin the tool definitionsskill file, 15,702 bytes when loadedin the tool definitions
the Books task, page text returned63,368 bytes of snapshots63,368 bytes of snapshots2,228 bytes (first run), 427 bytes (every run after)

Bytes aren't tokens, and an agent doesn't always read a whole snapshot file. Treat this as the size of what's on offer, not a token count.

What memory changes

Here is the Books task through Dejo. The first call showed a short outline of the page (1,670 bytes). The second sent the steps:

click link "Travel"
click css "article.product_pod h3 a" first
read css "div.product_main p.price_color" as price

and got back price = £45.17, saved as four steps. In a fresh server process, the same request was one call:

done: "price of the first Travel book" on books.toscrape.com (profile default), 4 steps replayed in 427 ms, no page reading.
price = £45.17

That's the part MCP and CLI don't do. Each new session with them starts from the home page with no idea it has done this before. In the 19-task benchmark, 17 of Dejo's 19 repeats were a single tool call, and the other two took three.

Where memory doesn't help

Which one I'd pick

If every task is new, pick whichever of Playwright MCP or CLI your agent handles better, and try both on one of your own tasks. In our Claude Code run the MCP was cheaper on most tasks and the CLI was cheaper on the longest ones. If you run the same tasks again (a report every Monday, a status page every morning), add a tool that remembers, because neither of them will. You can keep Playwright MCP installed next to Dejo; they're separate servers. There's more on that comparison in the Playwright MCP alternative page, and on why agents keep re-reading pages in browser agent memory.

How we tested

The Claude Code benchmark: benchmark_learning.py --bench real in the Dejo repository, tasks in real_tasks.py, one fresh session per run, recorded 21 to 23 October 2026. The results files call Dejo by its old name in places. The CLI version wasn't recorded in the file. Tokens include cache reads and writes as Claude Code reports them.

The hand-driven runs: 4 November 2026, a Linux cloud machine with Chromium, a small Python MCP client sending fixed calls and a script calling playwright-cli, no model. One task on one practice site, two runs of Playwright MCP, one of the CLI, and one learn plus two replays of Dejo 0.4.0.

What we didn't test: other models or agents (Cursor, Codex), the CLI's own --persistent profile, the newer MCP and CLI versions inside Claude Code, and Firefox or WebKit.

Related reading: why Playwright MCP uses so many tokens, browser agent token statistics and Dejo vs agent-browser.

FAQ

Which is better, Playwright MCP or Playwright CLI?

For one-off tasks they are close. In our test of 19 real-site tasks in fresh Claude sessions, Playwright MCP averaged 409K tokens and got 17 right; Playwright CLI averaged 559K and got 18 right. Microsoft recommends the CLI for coding agents. Neither one remembers a task, so a repeated task costs the same every time.

Is Playwright CLI more token-efficient than Playwright MCP?

Its README says so, because it does not load tool schemas or push page snapshots into the model. In our Claude Code test it used more tokens than MCP on 16 of 19 tasks, and fewer on the 3 longest ones. Your result depends on the task and the agent.

Is Playwright MCP free?

Yes, Playwright MCP and Playwright CLI are both free and open source under Apache 2.0. What you pay for is the model reading what they return.

What does memory add that MCP and CLI don't have?

A tool with memory saves the steps of a task the first time and replays them later without the model. In our test Dejo's first run averaged 251K tokens and every run after 89K, with 17 of 19 repeats done in one tool call.