パターンを読み込んでいます…
🖥️
OSWorld
Executable desktop environments where an agent is scored on the state it leaves behind after doing real work across applications, files and the operating system.
複雑さ: high評価とモニタリング
30秒でわかる概要
- 概要
- Runs an agent on a real desktop and scores the state of the machine afterwards, across applications, files and the operating system underneath them.
- 使いどころ
- Your agent is expected to operate software through the same interface a person gets, rather than through an API built for it.
- 注意点
- Failures cluster in knowing where to click rather than what to do, and an end state reached by a route no operator would sanction still scores as success.
このパターンについてAIエキスパートに質問
質問が入力済みの状態でアシスタントが開きます。送信前に内容を確認できます。
OSWorld: 概要
Executable desktop environments where an agent is scored on the state it leaves behind after doing real work across applications, files and the operating system.
- Real operating systems, not simulations of them
- Execution-based scoring against the final machine state
- Tasks spanning several applications and the filesystem
- Open-ended workflows with no single correct action sequence
- Long-horizon variant (OSWorld 2.0) alongside the original short tasks
- Human baseline collected on the same environments
The Agent Architect
1つのパターン、1つのトレードオフ、1つの本番障害事例。エージェントシステムを構築する人のための短い週刊ブリーフィング。
週1回のメール、ワンクリックで購読解除できます。アドレスはブリーフィングの送信のみに使用します。
別名： Computer use benchmark, Desktop agent benchmark, GUI agent evaluation
参考文献
このパターンの根拠となる論文、仕様、リポジトリです。