Higress Joins CNCF: Impact for AI Application Developers

Some infrastructure news may, at first glance, appear purely “internal” – relevant only to engineers who understand cloud system architecture. However, sometimes such news holds significant implications for the entire AI development ecosystem. Higress's entry into the CNCF (Cloud Native Computing Foundation) is one such instance.

AI Application Development Context

First, a Little Context

When developers create AI applications, they need to do more than just call various language models. They also have to solve a whole range of practical problems: how to route requests to different models, how to limit the load so that one active user doesn't “take down” the service for everyone else, how to monitor what's happening “inside,” and how to manage it all without endlessly writing code for every scenario.

This is where the concept of an AI Gateway comes in. Simply put, it's an intermediate layer between your application and the models – it receives requests, routes them, applies security rules, collects statistics, and much more. To draw a familiar comparison, it's like a smart traffic controller at the entrance who not only lets traffic through but also maintains order.

Higress is exactly such a gateway, developed at Alibaba Cloud. It was designed from the ground up with AI workloads in mind, rather than being adapted for them after the fact. And now, it has become part of the CNCF as a Sandbox project – the first official stage of acceptance into the foundation.

What is the CNCF and Its Significance?

What is the CNCF and Why Does It Matter?

The CNCF is a non-profit foundation under whose wing many key open-source cloud infrastructure projects reside. Suffice it to say, it is home to Kubernetes, Prometheus, and Envoy – tools that underpin a significant portion of modern cloud development.

Getting into the CNCF isn't just a formality. It means the project has been vetted for maturity, openness, and compliance with community standards. The Sandbox status is preliminary, but it confirms that the project is recognized as promising and gains access to the foundation's resources: infrastructure, legal support, and, importantly, visibility among developers worldwide.

For Higress, this is a signal: the project is no longer just “a tool from Alibaba” and is becoming part of a neutral, open ecosystem.

Key Capabilities of Higress for AI Development

What Higress Can Do – and Why You Need It

Putting the corporate jargon aside, Higress solves several specific problems that teams face when building AI products in a production environment (that is, in real, working systems, not just educational projects).

First, a single point of access to different models. Today, a typical AI application might access several model providers at once – OpenAI, Anthropic, its own self-hosted solutions, and so on. Higress handles the routing between them: you can configure rules to send some requests to one model and others to another, while the application itself “doesn't know” about these details.

Second, protection against overloads. Language models are an expensive pleasure, and an uncontrolled flow of requests can quickly drain a budget or crash a service. Higress supports rate limiting, automatic failover, and other mechanisms that are collectively known as “high availability.”

Third, observability. Simply put, understanding what's going on. How many tokens are being consumed, which models are responding slowly, where errors are occurring. This isn't just a convenience; it's a necessity for any serious service.

Finally, security. Higress supports granular access control – you can define different permissions for different applications and teams, rather than just “giving everyone the same key.”

Migrating from Nginx Ingress to Higress for AI

The Path Forward for Nginx Ingress Users

Migration from Nginx Ingress is a separate story. It's a popular tool for managing incoming traffic in Kubernetes clusters that many teams have been using for years. The problem is that it was created long before AI workloads were even on the horizon, and it lacks native support for what AI applications require today.

Higress offers a compatible migration path: it understands the same Ingress configurations as Nginx, so teams can switch without a complete infrastructure rewrite. It's not a matter of “just switch tools” – it's a deliberately designed route for those who want to gain AI capabilities without starting from scratch.

If you've ever moved from one apartment to another and know how frustrating it is when all the electrical outlets are in the wrong places, that's roughly what changing an infrastructure tool without compatibility feels like. Higress tries to ensure the “outlets” are in the same spots.

The Principle of Openness in Higress

Openness as a Principle

An important detail worth emphasizing: Higress is an open-source project. This means its code is open, can be studied, modified, and used without being tied to a specific cloud provider. This is precisely what makes joining the CNCF a logical step – the foundation, by definition, deals with open, vendor-neutral projects.

For developers, this means they aren't “locked in” to a single provider's ecosystem. You can deploy Higress on your own infrastructure, in any cloud, and not depend on what decisions Alibaba Cloud makes next year.

The Growing Need for AI Gateways

Why Now?

A year or two ago, the conversation about “AI gateways” was mostly theoretical. Today, it's a very real engineering challenge: companies are actively moving AI tools from experimental mode to production, and with that comes the need for proper infrastructure.

The need for tools that manage not just traffic, but AI traffic – with its tokens, context windows, multiple providers, and specific failure modes – has taken shape relatively recently. Higress emerged as a direct response to this need, and its acceptance into the CNCF can be seen as recognition that this challenge has become important enough for the entire industry.

As the Alibaba Cloud community notes, before the native plugin was available, developers had to use workarounds – for example, connecting via an OpenAI-compatible protocol, which didn't cover nearly all scenarios. Now, the integration is fully-fledged, and Dify has officially added the Higress plugin to its catalog, simplifying connection for those building AI applications on its platform.

Higress's Future in the CNCF Ecosystem

What's Next?

Sandbox status in the CNCF is a beginning, not an end. The next stages – Incubating and Graduated – signify greater maturity and wider adoption by the community. Kubernetes, for example, completed this journey long ago and has become the de facto standard.

For Higress, this opens the door to attracting contributors from around the world, not just from Alibaba, and gradually building an independent community around the project. How quickly this will happen, only time will tell. But the very fact of its acceptance shows that the problem Higress solves is now seen as a shared one, not a niche one.

In short: AI infrastructure is maturing. And the emergence of tools like this in the open-source space is one of the signs.