When it comes to training large language models, people usually picture server racks filled with NVIDIA GPUs. But the industry is gradually expanding, and AMD continues to develop its AI ecosystem. A recent update to the verl framework brings support for AMD ROCm 7.0.0 and the ability to scale across hundreds of GPUs simultaneously.
Что такое verl и для чего он нужен
What is verl and why is it needed?
verl is a framework for training large language models using reinforcement learning from human feedback (RLHF). This technology helps make models more useful and safer by training them on examples of what responses humans consider good versus bad.
Simply put: first, the model generates several response options to the same prompt. Then, another model (or humans) evaluates these options. Based on these evaluations, the original model gradually learns to provide higher-quality answers. This is exactly how models like ChatGPT were trained.
verl is developed by the Volcano Engine team, the cloud division of ByteDance. The framework was initially created for the company's internal needs but was later made open-source.
Что нового в последней версии verl
What's new in the latest version?
The main change is full support for the AMD ROCm 7.0.0 platform. This is significant because, until recently, almost all tools for working with large models were tailored to NVIDIA hardware. AMD is actively working to change this situation, and verl is one example of this movement.
Additionally, the new version can scale to clusters of hundreds of GPUs. This is crucial for training truly large models, where one or even ten cards are clearly not enough. The framework supports distributing computations across nodes and is optimized for such environments.
verl also integrates support for vLLM, a popular engine for fast inference of large models. This helps speed up the response generation process during the training phase, ultimately reducing the time and cost of the entire procedure.
Почему это важно для индустрии ИИ
Why is this important for the industry?
Reinforcement learning is not just a buzzword. It's one of the key methods that transforms a raw language model into a useful tool. Without this stage, a model might generate text that is technically correct but useless or even harmful.
But there's a catch: this type of training requires immense computational resources. The model must repeatedly generate responses, receive evaluations, update its parameters, and repeat the cycle over and over. All of this happens with large volumes of data and involves models containing billions of parameters.
That's why having working tools on alternative platforms is a major step forward. It gives teams more hardware options, reduces dependency on a single manufacturer, and potentially makes training more accessible.
Дальнейшие перспективы развития AMD AI
What's next?
AMD continues to develop its AI ecosystem. ROCm 7.0.0 is the seventh major version of their software platform, and tool support grows with each iteration. verl is just one example of how third-party developers are adapting their solutions for this platform.
For those working with large models, this means more options are becoming available. You can use servers with AMD GPUs if they are more cost-effective or have better availability. You can experiment with different configurations and compare performance.
Of course, NVIDIA's ecosystem is still more mature and widely supported. But the situation is gradually changing, and verl with its ROCm 7.0.0 support is another step in this direction.