What is a Kubernetes Ingress Controller?

A Kubernetes Ingress Controller is a specialized load balancer that manages external HTTP/HTTPS access to services within a Kubernetes cluster. It implements the Ingress resource specification, providing routing rules, TLS termination, virtual hosting, and URL path-based routing from a single entry point.

Why am I getting 502 Bad Gateway errors from my Ingress Controller?

A 502 Bad Gateway from an Ingress Controller usually means the backend service is unreachable. Common causes include: the service selector not matching any pods, pods not being in Ready state, the service port not matching the container port, or network policies blocking traffic between the ingress controller and backend pods.

How do I debug Ingress routing issues in Kubernetes?

Start by running kubectl describe ingress to check rules and events, then verify the backend service with kubectl get endpoints to confirm pods are registered. Check the Ingress Controller logs with kubectl logs and test connectivity directly from the controller pod to the backend service.

Kubernetes Ingress Controller Troubleshooting

Understanding Kubernetes Ingress

In Kubernetes, Services expose pods within the cluster, but they don’t provide external HTTP routing, TLS termination, or virtual hosting. The Ingress resource fills this gap by defining rules for routing external HTTP/HTTPS traffic to internal services based on hostnames and URL paths.

An Ingress Controller is the component that implements these rules. Popular controllers include:

NGINX Ingress Controller (most widely used)
Traefik (auto-discovery and Let’s Encrypt integration)
HAProxy Ingress
AWS ALB Ingress Controller (for EKS)
Istio Gateway (service mesh integration)

This guide focuses on troubleshooting the NGINX Ingress Controller, but the diagnostic methodology applies to all controllers.

Prerequisites

A running Kubernetes cluster (1.24+).
An Ingress Controller installed (e.g., ingress-nginx).
kubectl configured with cluster access.
At least one Service and Deployment running in the cluster.

Step-by-Step Troubleshooting

1. Verify the Ingress Resource

kubectl get ingress -A
kubectl describe ingress <ingress-name> -n <namespace>

Check for:

Rules: Correct host and path definitions.
Backend: The service name and port must match an existing service.
Events: Look for error messages from the controller.
Address: If the Address field is empty, the controller hasn’t processed the Ingress yet.

Example of a correctly configured Ingress:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: myapp-ingress
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /
spec:
  ingressClassName: nginx
  tls:
    - hosts:
        - myapp.example.com
      secretName: myapp-tls
  rules:
    - host: myapp.example.com
      http:
        paths:
          - path: /api
            pathType: Prefix
            backend:
              service:
                name: api-service
                port:
                  number: 8080
          - path: /
            pathType: Prefix
            backend:
              service:
                name: frontend-service
                port:
                  number: 80

2. Check Backend Service and Endpoints

# Verify the service exists and has the correct port
kubectl get svc <service-name> -n <namespace>

# Check if endpoints (pod IPs) are registered
kubectl get endpoints <service-name> -n <namespace>

If endpoints are empty, the service selector doesn’t match any running, ready pods. Check labels:
kubectl get pods -n <namespace> --show-labels
kubectl describe svc <service-name> -n <namespace>  # Check selector

3. Inspect Ingress Controller Logs

# Find the controller pod
kubectl get pods -n ingress-nginx

# View logs
kubectl logs <ingress-controller-pod> -n ingress-nginx --tail=100

# Follow logs in real-time
kubectl logs -f <ingress-controller-pod> -n ingress-nginx

Common log messages and their meanings:

Log Message	Meaning
`upstream sent no valid HTTP/1.0 header`	Backend is not responding with valid HTTP
`connect() failed (111: Connection refused)`	Backend pod is not listening on the expected port
`no resolver defined to resolve`	DNS resolution issue within the cluster
`SSL certificate verify failed`	TLS certificate chain issue on backend

4. Troubleshoot 502 Bad Gateway

A 502 means the Ingress Controller reached the backend but received an invalid response:

# Test connectivity from inside the cluster
kubectl run debug --image=busybox --rm -it -- wget -qO- http://<service-name>.<namespace>.svc.cluster.local:<port>

# Or from the ingress controller pod itself
kubectl exec -it <ingress-controller-pod> -n ingress-nginx -- curl -v http://<service-name>.<namespace>.svc.cluster.local:<port>

Common causes:

Pod is crash-looping — check kubectl get pods and kubectl logs.
Container is listening on a different port than the service targets.
Readiness probe is failing, so the pod is removed from endpoints.
Backend requires HTTPS but Ingress sends HTTP — add annotation: nginx.ingress.kubernetes.io/backend-protocol: "HTTPS".

5. Fix TLS Certificate Issues

# Verify the TLS secret exists
kubectl get secret <secret-name> -n <namespace>

# Check the certificate details
kubectl get secret <secret-name> -n <namespace> -o jsonpath='{.data.tls\.crt}' | base64 -d | openssl x509 -text -noout

Common TLS issues:

Secret not found: The secretName in the Ingress doesn’t match any secret in the same namespace.
Expired certificate: Check the Not After date in the certificate output.
Wrong hostname: The certificate’s Common Name or SAN must match the host in the Ingress rule.
cert-manager not issuing: Check kubectl describe certificate and kubectl describe certificaterequest.

6. Debug Path-Based Routing

Path-based routing issues are common with the pathType field:

Exact: Matches only the exact path (e.g., /api won’t match /api/users).
Prefix: Matches the path prefix (e.g., /api matches /api, /api/users, /api/v2/items).
ImplementationSpecific: Depends on the IngressClass.

# Test specific paths
curl -H "Host: myapp.example.com" http://<ingress-controller-ip>/api/health
curl -H "Host: myapp.example.com" http://<ingress-controller-ip>/

Gotchas and Edge Cases

ingressClassName is required in Kubernetes 1.22+. Omitting it may result in no controller picking up the Ingress.
Annotation differences: Each controller has its own annotation namespace (e.g., nginx.ingress.kubernetes.io/* vs traefik.ingress.kubernetes.io/*).
Rate limiting applies to config reloads: Rapid Ingress changes can cause the NGINX controller to queue config reloads, introducing temporary inconsistencies.
Default backend: If no rule matches the request, the controller returns its default backend (usually a 404 page). Configure a custom default with --default-backend-service.
Websocket support: Add nginx.ingress.kubernetes.io/proxy-read-timeout: "3600" and nginx.ingress.kubernetes.io/proxy-send-timeout: "3600" annotations for websocket connections.

Summary

The Ingress Controller implements routing rules for external HTTP/HTTPS traffic into the cluster.
Troubleshoot 502 errors by verifying backend service endpoints, pod readiness, and port mappings.
Check TLS secrets exist in the correct namespace with valid, non-expired certificates.
Use kubectl describe ingress and controller logs as the primary diagnostic tools.
Always specify ingressClassName in Kubernetes 1.22+.