Intellectual hub of the topic
Evaluating the effectiveness of autonomous systems requires tools that go far beyond classic performance tests. In this collection, we focus on benchmarking methodologies that measure an agent's capacity for deduction, planning, and the accurate execution of multi-step instructions within dynamic environments. Here, you will find analytical breakdowns of existing frameworks, critical reviews of metrics, and test results for software entities operating under conditions of uncertainty.
Hcompany has introduced Holo3, an agent model that set a record on a key computer operation benchmark and is designed for autonomous work in corporate environments.
Company H has announced the release of Holo3, a model that has set a new record on the leading benchmark for AI agents that operate computers.
AI: Events
Products
Mixedbread has released Search v3 – a retrieval model that significantly narrows the gap between what an agent actually finds and what is theoretically discoverable within the data.
The Allen Institute has introduced MolmoWeb, an open-source web agent. It navigates browsers visually, much like a human, and outperforms many proprietary competitors.
Databricks has developed its own approach to creating AI agents – the coSTAR system, which allows the team to work quickly without losing control over quality.
We explore why assessing AI agents' skills isn't just a formality, but a crucial step toward building systems you can trust with real-world tasks.
LightOn has introduced the NOVA evaluation system. We explore how it works and why a «gut feeling» isn't enough to verify AI agents.
AI: Events
Regulation
In partnership with a national laboratory, OpenAI has developed a tool to evaluate AI agents for speeding up federal approvals and is already seeing the first measurable results.
AI: Events
Products
Liquid AI has introduced the LFM2-24B-A2B model, capable of running AI agents with tool-calling capabilities directly on consumer hardware – without the cloud or latency.
Don’t miss a single experiment!
Subscribe to our Telegram channel —
we regularly post announcements of new books, articles, and interviews.