Intellectual hub of the topic
Understanding how modern intelligent systems are built is impossible without analyzing their «blueprints».In this collection, we gather materials dedicated to the internal logic of neural networks – from the fundamental basics of transformers to experimental topologies. We view architecture not merely as a technical parameter, but as the defining factor of a specific model's capabilities and limitations.
AI: Events
Technical context • Development
Students leveraged language models to optimize a cryptographic core on an FPGA, achieving a twofold performance increase in just two weeks.
We explain how the Mixture of Experts architecture works – an approach that makes models smarter without making them 'think' harder.
AI: Events
Infrastructure
Why modern AI can't be truly smart without structured data, and how metadata, reference data, and knowledge graphs shape its 'brain'.
NeuroBlog
Artificial intelligence • Technologies
The longer the conversation with AI lasts, the more it loses the thread – like a conversational partner getting too tired to keep everything said earlier in their head.
AI: Events
Development
Allen AI has introduced Olmix, an open-source framework for data mixing in the language model training process, including pre-training, instruction tuning, and alignment.
AI: Events
Technical context • Development
Unsloth's new kernels and mathematical optimizations slash memory requirements by 35%, boost training speeds by 12x, and enable context windows six times longer than the original.
AI: Events
Technical context • Infrastructure
The Perplexity AI team has demonstrated how direct server-to-server data transfer technology helps language models run faster and more efficiently by eliminating bottlenecks in network infrastructure.
AI: Events
Technical context • Infrastructure
Zyphra's new OVQ-attention layer aims to reduce memory and computational overhead when working with long contexts while maintaining high sequence processing quality.
AI: Events
Research
Yao Shunyu's team from Tencent demonstrated why the ability to work with context may become a key factor for applying models in real-world tasks.
Don’t miss a single experiment!
Subscribe to our Telegram channel —
we regularly post announcements of new books, articles, and interviews.