Test WebMCP tools like an API (with Playwright)

The Chrome team's best practices and use cases guides end on evaluation tests.

Evals cover the agent's judgement. Did it pick the right tool, did it fill the form the way the user meant, did it stop when it should have asked. You can't hard-code those outcomes, which is why the guides send you to evaluation-driven development.

Underneath that sits a layer that is deterministic and dull. Once your page calls registerTool, it advertises callable functions with typed schemas to whatever agent is driving. That's an API. Get an enum wrong and an agent walks a path your buttons never allow, and the eval that catches it will report that the model got confused. Your UI tests will never see it.

I measured what Chrome 152 does when an agent calls these methods.