Why another benchmark
Computer-use agents must decide whether a screenshot after a click or drag represents the actual result. Remote input, app rendering, and capture are asynchronous, so stale, transient, or unrelated screens can be mistaken for progress.
The July 28 preprint Desktop-Delta Bench measures causal transition understanding between actions rather than only final task success or single-frame grounding.
Data and tasks
It contains 2,013 human-verified instances from roughly 15 Linux applications and 50 task domains. There are 463 three-frame ordering cases and 1,550 before-and-after cases covering five action families and their payloads. Some ordering cases include a decoy frame from another trajectory.
Results and meaning
Across eight open and closed model families, best exact-match ordering reached 65.1% without decoys and 65.7% with decoys. Task context improved decoy identification by 6.9 percentage points but reduced ordinary ordering by 2.2 points. Models also systematically copied the presented A-B-C order.
Limits and practice
This is a non-peer-reviewed offline benchmark and does not measure recovery in live systems. It nevertheless supports verifying expected change, observed change, and source application before advancing. Delay-aware retries, state hashes, and independent transition checks deserve testing.