Desktop-Delta Bench Tests Whether GUI Agents Understand What Changed

The benchmark uses 2,013 human-verified cases across about 15 apps and 50 task domains. Best temporal-order accuracy remained near 65%, even with current model families.

Why another benchmark

Computer-use agents must decide whether a screenshot after a click or drag represents the actual result. Remote input, app rendering, and capture are asynchronous, so stale, transient, or unrelated screens can be mistaken for progress.

The July 28 preprint Desktop-Delta Bench measures causal transition understanding between actions rather than only final task success or single-frame grounding.

Data and tasks

It contains 2,013 human-verified instances from roughly 15 Linux applications and 50 task domains. There are 463 three-frame ordering cases and 1,550 before-and-after cases covering five action families and their payloads. Some ordering cases include a decoy frame from another trajectory.

Results and meaning

Across eight open and closed model families, best exact-match ordering reached 65.1% without decoys and 65.7% with decoys. Task context improved decoy identification by 6.9 percentage points but reduced ordinary ordering by 2.2 points. Models also systematically copied the presented A-B-C order.

Limits and practice

This is a non-peer-reviewed offline benchmark and does not measure recovery in live systems. It nevertheless supports verifying expected change, observed change, and source application before advancing. Delay-aware retries, state hashes, and independent transition checks deserve testing.

Primary source