Loading patterns…
OSWorld
Executable desktop environments where an agent is scored on the state it leaves behind after doing real work across applications, files and the operating system.
In 30 seconds
- What
- Runs an agent on a real desktop and scores the state of the machine afterwards, across applications, files and the operating system underneath them.
- When to use
- Your agent is expected to operate software through the same interface a person gets, rather than through an API built for it.
- Watch out
- Failures cluster in knowing where to click rather than what to do, and an end state reached by a route no operator would sanction still scores as success.
Ask the AI expert about this pattern
Opens the assistant with your question prefilled. You review it before sending.
OSWorld: Overview
Executable desktop environments where an agent is scored on the state it leaves behind after doing real work across applications, files and the operating system.
- Real operating systems, not simulations of them
- Execution-based scoring against the final machine state
- Tasks spanning several applications and the filesystem
- Open-ended workflows with no single correct action sequence
- Long-horizon variant (OSWorld 2.0) alongside the original short tasks
- Human baseline collected on the same environments
The Agent Architect
One pattern, one tradeoff, one production failure story. A short weekly briefing for people building agentic systems.
Weekly email, one-click unsubscribe. We only use your address to send the briefing.
Also known as: Computer use benchmark, Desktop agent benchmark, GUI agent evaluation
References
The papers, specifications, and repositories this pattern is based on.