Intellectual hub of the topic
A large-scale test of 16 AI models on real-world documents revealed surprising results: expensive solutions don't always outperform their more affordable counterparts.
AI: Events
Research
Google DeepMind has introduced a cognitive framework for assessing progress toward artificial general intelligence (AGI) and launched a Kaggle hackathon to develop relevant benchmarks.
AMD explains why comparing AI accelerators using a single performance metric is misleading and advocates for a multi-dimensional evaluation approach.
AI: Events
Research
Researchers have introduced M4-RAG, a large-scale benchmark for evaluating systems that answer questions about images by drawing on external knowledge and operating in multiple languages.
Sber researchers have launched an open-source platform for the objective assessment of how accurately AI models can predict chains of events over long-term horizons.
Lab
Electrical Engineering & System Sciences
Researchers tested whether AI systems can comprehend real-world medical conversations – and the results delivered a harsh verdict for the entire industry.
Stanford researchers tested leading AI models on their ability to navigate space and found surprisingly poor results.
LightOn has released EDiTh, an open-source benchmark that allows testing corporate search on realistic documents without the risk of leaking confidential data.
AI: Events
Research
The OpenHands team explains how their benchmark for evaluating AI agents works and why conventional metrics don't always reflect the true picture.
Want to dive deeper into the world
of neuro-creativity?
Be the first to learn about new books, articles, and AI experiments
on our Telegram channel!