The missing class in binary detection
A July 29 study argues that human-versus-bot labels cannot represent browser-automation AI agents. It builds a controlled benchmark with separate human, conventional-bot, and AI-agent classes.
Reported detection results
A binary MLP mislabeled 39.1% of real AI agents as human, and a binary SAINT model mislabeled 34.5%. Adding an agent class produced agent F1 of 1.000 across 30 runs covering three model families and ten seeds.
The signal was not evidence of reasoning. It was a Playwright automation artifact: missing raw pointer-move and wheel-delta streams from physical devices. Two features—mouse event rate and teleport click ratio—produced 100% observed recall and 0.994 precision across the authors' evasion ladder.
Deployment cautions
Operators should distinguish disclosed crawlers, user-delegated agents, and hostile automation rather than treating every agent as malicious. Detection is better used for rate limits, step-up verification, and audit signals than automatic blocking.
The near-perfect result comes from a controlled stack and may not generalize to agents relaying physical input or using other browser systems. It is not a real-internet false-positive estimate, and the paper is not yet peer reviewed.