RSS

TechTalk #29: Our Ingress Odyssey, From Ingress-NGINX to Gateway API

TechTalk #29: Our Ingress Odyssey, From Ingress-NGINX to Gateway API

We picked Ingress-NGINX for our Nine Kubernetes Engine (NKE) years ago because it was the most widely used, safe choice. Ingress-NGINX’s deprecation, driven by a lack of time and maintainers, was then announced in November 2025. The most widely used Ingress controller was facing its end, and for us began a journey through Traefik, HAProxy and Gateway API. Sebastian Nickel, known internally as Nick, Senior Engineer on our Platform team, told that story, titled «Our Ingress Odyssey: Evaluating K8s Traffic Solutions After the Elephant Left the Room», as the first of three talks at TechTalk #29, opened as always with a short introduction from Thomas Hug, our CEO and founder.

Where We Started

When we built our Nine Kubernetes Engine (NKE), we chose Ingress-NGINX because it was, at the time, the most widely used solution for Ingress. Support for the Ingress-NGINX annotations used for configuration was already built into many Helm charts. Since everything on NKE is self-serviceable, customers create an Ingress-NGINX object on our own API, our reconciler renders the official Helm chart and installs it with three replicas, and the public IP is realised through our own BGP/MetalLB setup. We even monitor reachability from the outside, not just internally, and since it’s self-service, all of that has to be fully automated.

The Deprecation

That stability held for years, until Nick and the team heard a maintainer admit at KubeCon that Ingress-NGINX was maintained by only one or two people working on it in their free time, overwhelmed with issues. In November 2025, the maintainers officially announced the project’s retirement: security fixes until March 2026, then nothing. The Kubernetes project’s own Steering and Security Response Committees later confirmed just how widely used it still was, citing Datadog telemetry showing around 50% of cloud native environments relying on it. Customers started asking us directly what our plan was. We didn’t have a clean answer yet.

Gateway API is the official successor, and Traefik looked like a strong candidate thanks to its own Ingress-NGINX compatibility layer. In practice, that layer only covered a fraction of what our customers actually used: of our roughly 1,600 Ingresses at the time, about 80% relied on annotations it simply didn’t support. Supporting the rest properly would have meant adopting Traefik’s own CRDs, real added complexity for what’s a single resource today. Its TLS/ACME handling also only worked reliably with a single replica, anything else needed a shared ReadWriteMany volume (in practice NFS), which we wanted to avoid, and an independent Gateway API benchmark showed it performing poorly besides.

But Gateway API itself had a structural issue too: it separates infrastructure provider, cluster operator and application developer roles, and puts TLS configuration with the cluster operator rather than inside the resource a developer controls. For us, that directly contradicted our self-service model: any change to hostnames or certificates would have meant customers filing a support ticket with us every time. So the team made a deliberate choice: wait, keep Ingress-NGINX patched until March 2026, and watch the ecosystem mature, in particular a proposal called ListenerSet, which lets customers extend the Gateway definition, which otherwise stays under our control, in a controlled way, so they can adjust the hostnames and certificates used for TLS communication themselves.

HAProxy

The push toward a decision came from a fairly mundane place: a teammate spotted on Hacker News that the HAProxy Ingress community project had published a tool that automatically translated existing Ingress-NGINX definitions into HAProxy equivalents. That’s how we found out HAProxy already supported most of the settings we relied on with Ingress-NGINX, so we tested it further, specifically: using multiple configurations, we checked whether a bug we’d fought with Ingress-NGINX before also showed up in HAProxy. The bug: a single broken Ingress (say, a syntax error in its annotations) would fail config validation and silently keep serving the last known good state, but the failed attempt still wrote a new temporary file every time, until the node ran into disk pressure, got its pods evicted, and lost that last known good state entirely. We couldn’t reproduce it in HAProxy: it held up, and we went with it.

Because HAProxy’s annotation-based model mapped so closely to what we already had, we decided to offer the HAProxy Ingress Controller as a resource in our own API. That let customers install it as an additional controller in their clusters, which in turn made it possible to migrate individual Ingresses over via DNS, one at a time, instead of simply swapping the entire Ingress-NGINX Controller for HAProxy in one go.

WAF, and Gateway API Again

At some point, customers started asking for a Web Application Firewall. HAProxy’s own WAF support was still experimental, and a commercial product suggested from the customer side didn’t hold up either: its Java-based controller dumped complete stack traces straight into the status of the CRD resources used for configuration. On top of that, the reconciliation loop didn’t always seem to work, so some changes triggered no update at all.

Another product we tested was Airlock Micro Gateway from Ergon. It relies exclusively on Gateway API and covers a broad range of security requirements beyond the OWASP Top 10. That meant the exact question we’d shelved was back on the table: dealing with Gateway API instead of classic Ingress. By then, luckily, enough had changed: ListenerSet reached Gateway API’s production readiness track, cert-manager added stable support for it, and we got early beta access to Ergon’s corresponding release. That also solved the ownership problem: TLS configuration can now live where the application developer expects it.

Where This Leaves Us

HAProxy Ingress is now our new default for classic Ingress. If you’re still running the Ingress-NGINX Controller today, we’d recommend prioritising the move: as we wrote separately, Ingress-NGINX no longer receives security patches since March 2026. Airlock Micro Gateway on Gateway API is currently in testing with a limited set of customers for WAF use cases. Get in touch if that’s relevant to you. Implementing Airlock Micro Gateway in our own API also gives us a reference architecture for further Gateway API controllers (with ListenerSet support) down the line.

Two things stood out in hindsight: sometimes the right strategy really is to wait for an ecosystem to mature, and any replacement has to avoid adding meaningfully more complexity than what customers already have today.

We wrote about the retirement of the Ingress-NGINX project itself back in December 2025, this talk is the update on where that search led us. Find out more about NKE on our product page. Questions about anything from the talk? Get in touch. We announce the next TechTalk on our channels, including our Meetup group «TechTalk @ Nine».

Comments & Questions

A GitHub account is required to comment.

Want to stay up to date?

Subscribe to our YouTube channel and visit the Blog on our website.