Loading patterns…
OSWorld
Executable desktop environments where an agent is scored on the state it leaves behind after doing real work across applications, files and the operating system.
In 30 seconds
- What
- Runs an agent on a real desktop and scores the state of the machine afterwards, across applications, files and the operating system underneath them.
- When to use
- Your agent is expected to operate software through the same interface a person gets, rather than through an API built for it.
- Watch out
- Failures cluster in knowing where to click rather than what to do, and an end state reached by a route no operator would sanction still scores as success.
Ask the AI expert about this pattern
Opens the assistant with your question prefilled. You review it before sending.
OSWorld: Overview
Executable desktop environments where an agent is scored on the state it leaves behind after doing real work across applications, files and the operating system.
- Real operating systems, not simulations of them
- Execution-based scoring against the final machine state
- Tasks spanning several applications and the filesystem
- Open-ended workflows with no single correct action sequence
- Long-horizon variant (OSWorld 2.0) alongside the original short tasks
- Human baseline collected on the same environments
Get the Agent Evals field guide
All 25 agent evaluation methods condensed into one guide: which benchmark measures what, when a public score misleads you, and how to build evals out of your own failures. The link arrives with your confirmation, alongside the weekly Agent Architect.
Weekly email, one-click unsubscribe. We only use your address to send the briefing.
Also known as: Computer use benchmark, Desktop agent benchmark, GUI agent evaluation
References
The papers, specifications, and repositories this pattern is based on.
From the engineer behind this catalog
Find out what your evals miss
Measuring an agent is harder than shipping one, and most suites stay green while production drifts. Have your evaluation setup reviewed end to end: what you measure today, what you cannot see yet, and the regressions your current suite would let through.
€750 instead of €1,500, one week, written report and walkthrough call, until 30 September