Ingress Exposure

TinySystems components can expose HTTP endpoints to the internet via Kubernetes Ingress. This guide covers ingress configuration and common patterns.

Overview

+-----------------------------------------------------------------------------+
|                         INGRESS EXPOSURE                                     |
+-----------------------------------------------------------------------------+

  Internet                 Ingress Controller              Module Pods
      |                          |                             |
      |  https://api.example.com |                             |
      |  ----------------------> |                             |
      |                          |                             |
      |                          |  Route to Service           |
      |                          |  ----------------------->   |
      |                          |                             |
      |                          |           Load Balance      |
      |                          |  +------------------------->| Pod 1
      |                          |  |                          |
      |                          |  +------------------------->| Pod 2
      |                          |  |                          |
      |                          |  +------------------------->| Pod 3
      |                          |                             |

Basic Ingress Setup

Exposing a Port

Port exposure is provided by the SDK's resource manager, delivered to components through the module.ClientAware capability. The signature is:

ExposePort(ctx context.Context, autoHostName string, hostnames []string, port int) ([]string, error)
DisclosePort(ctx context.Context, port int) error

type Server struct {
    client module.Client // ExposePort / DisclosePort
}

// OnClient is called once by the framework during node update.
func (s *Server) OnClient(c module.K8sClient) {
    if pc, ok := c.(module.Client); ok {
        s.client = pc
    }
}

func (s *Server) exposeHTTP(ctx context.Context) error {
    hostnames, err := s.client.ExposePort(ctx, "my-api", []string{"api.example.com"}, s.currentPort)
    if err != nil {
        return err
    }
    // hostnames contains the final list, including any auto-generated hostname
    return nil
}

ExposePort does not create new Kubernetes objects. It locates the module release's existing Service and Ingress (by the Helm release labels) and mutates them:

• adds a port<N> entry to the Service
• appends one Ingress rule per hostname (path /, pathType: Prefix) pointing at the Service
• appends one TLS entry per new hostname with secret name <hostname>-tls

Resulting Changes

Service (module release Service, port appended):

spec:
  ports:
    # existing ports ...
    - name: port8080
      port: 8080
      targetPort: 8080

Ingress (module release Ingress, rule and TLS entry appended):

spec:
  tls:
    - hosts:
        - api.example.com
      secretName: api.example.com-tls
  rules:
    - host: api.example.com
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: <module-release-service>
                port:
                  number: 8080

Automatic Hostnames

When the release Ingress carries the tinysystems.io/ingress-hostname-suffix annotation, passing a non-empty autoHostName appends an extra hostname <autoHostName>-<suffix>:

// With annotation tinysystems.io/ingress-hostname-suffix: "playground.example.com"
hostnames, _ := s.client.ExposePort(ctx, "my-api", nil, 8080)
// hostnames == []string{"my-api-playground.example.com"}

At least one of autoHostName / hostnames must be provided, otherwise ExposePort returns an error.

TLS Configuration

TLS is not optional: every new hostname gets a TLS entry with secret name <hostname>-tls. If cert-manager is configured for the release Ingress (via its annotations), certificates are provisioned automatically. There is no parameter for a custom certificate secret — to use an existing certificate, manage the Ingress manually via the raw Kubernetes client (module.K8sClient).

Multiple Hostnames

Expose the same port on multiple domains:

hostnames, err := s.client.ExposePort(ctx, "", []string{
    "api.example.com",
    "api.example.org",
    "api.example.net",
}, 8080)

Each hostname gets its own rule and its own TLS entry/secret:

spec:
  tls:
    - hosts: [api.example.com]
      secretName: api.example.com-tls
    - hosts: [api.example.org]
      secretName: api.example.org-tls
    - hosts: [api.example.net]
      secretName: api.example.net-tls
  rules:
    - host: api.example.com
      http:
        paths:
          - path: /
            pathType: Prefix
            backend: { service: { name: <module-release-service>, port: { number: 8080 } } }
    - host: api.example.org
      http:
        paths:
          - path: /
            pathType: Prefix
            backend: { service: { name: <module-release-service>, port: { number: 8080 } } }
    - host: api.example.net
      http:
        paths:
          - path: /
            pathType: Prefix
            backend: { service: { name: <module-release-service>, port: { number: 8080 } } }

Removing Exposure

DisclosePort reverses ExposePort: it removes the port from the Service and drops every Ingress rule (and associated TLS entry) that points at it:

if err := s.client.DisclosePort(ctx, 8080); err != nil {
    return err
}

Path-Based Routing

Route different paths to different components:

# Manual ingress for advanced routing
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: multi-path-ingress
spec:
  rules:
    - host: api.example.com
      http:
        paths:
          - path: /users
            pathType: Prefix
            backend:
              service:
                name: user-service
                port:
                  number: 8080
          - path: /orders
            pathType: Prefix
            backend:
              service:
                name: order-service
                port:
                  number: 8080

Ingress Annotations

NGINX Ingress Controller

metadata:
  annotations:
    # Timeouts
    nginx.ingress.kubernetes.io/proxy-connect-timeout: "60"
    nginx.ingress.kubernetes.io/proxy-send-timeout: "60"
    nginx.ingress.kubernetes.io/proxy-read-timeout: "60"

    # Body size
    nginx.ingress.kubernetes.io/proxy-body-size: "10m"

    # WebSocket support
    nginx.ingress.kubernetes.io/websocket-services: "<module-release-service>"

    # Rate limiting
    nginx.ingress.kubernetes.io/limit-rps: "100"

    # CORS
    nginx.ingress.kubernetes.io/enable-cors: "true"
    nginx.ingress.kubernetes.io/cors-allow-origin: "*"

Where Annotations Live

ExposePort takes no annotations parameter. Annotations belong to the module release's Ingress (set via the Helm chart values) and apply to every hostname exposed through it. A component that needs per-endpoint annotations must manage its own Ingress object using the raw Kubernetes client delivered via module.ClientAware (GetK8sClient()).

WebSocket Support

Enable WebSocket upgrade:

metadata:
  annotations:
    nginx.ingress.kubernetes.io/proxy-http-version: "1.1"
    nginx.ingress.kubernetes.io/upstream-hash-by: "$request_uri"
    nginx.ingress.kubernetes.io/configuration-snippet: |
      proxy_set_header Upgrade $http_upgrade;
      proxy_set_header Connection "upgrade";

Health Checks

Readiness Probe

Ensure pods are ready before receiving traffic:

# In module Helm chart
spec:
  containers:
    - name: module
      readinessProbe:
        httpGet:
          path: /healthz
          port: 8080
        initialDelaySeconds: 5
        periodSeconds: 10

Backend Health

NGINX ingress checks backend health:

metadata:
  annotations:
    nginx.ingress.kubernetes.io/upstream-vhost: "api.example.com"
    nginx.ingress.kubernetes.io/health-check-path: "/healthz"

Load Balancing

Default (Round Robin)

Kubernetes Service distributes traffic evenly.

Session Affinity

Sticky sessions for stateful applications:

metadata:
  annotations:
    nginx.ingress.kubernetes.io/affinity: "cookie"
    nginx.ingress.kubernetes.io/session-cookie-name: "route"
    nginx.ingress.kubernetes.io/session-cookie-expires: "172800"

Monitoring Ingress

Check Ingress Status

kubectl get ingress -n tinysystems
kubectl describe ingress -n tinysystems -l app.kubernetes.io/name=tinysystems-operator

View Logs

# NGINX Ingress Controller logs
kubectl logs -n ingress-nginx -l app.kubernetes.io/name=ingress-nginx

Metrics

# Prometheus metrics from ingress controller
curl http://ingress-controller:10254/metrics

Troubleshooting

Certificate Not Ready

# Check cert-manager (secret/certificate is named <hostname>-tls)
kubectl get certificates -n tinysystems
kubectl describe certificate api.example.com-tls -n tinysystems

# Check challenges
kubectl get challenges -n tinysystems

502 Bad Gateway

• Check pod readiness
• Verify service selector matches pods
• Check pod logs for errors

kubectl get pods -n tinysystems -l app=http-module
kubectl logs -n tinysystems -l app=http-module

DNS Not Resolving

• Verify DNS record exists
• Check ingress host configuration
• Ensure ingress controller has external IP

kubectl get svc -n ingress-nginx
nslookup api.example.com

Best Practices

1. Expose from the Leader Only

ExposePort mutates shared Service/Ingress objects. In multi-replica modules, gate it with utils.IsLeader(ctx) so only one pod performs the mutation.

2. Configure Timeouts

Match timeouts to your processing time:

nginx.ingress.kubernetes.io/proxy-read-timeout: "300"  # 5 minutes for long operations

3. Enable Rate Limiting

Protect against abuse:

nginx.ingress.kubernetes.io/limit-rps: "50"
nginx.ingress.kubernetes.io/limit-connections: "10"

4. Monitor Certificate Expiry

Set up alerts for certificate expiration:

# Prometheus rule
- alert: CertificateExpiringSoon
  expr: certmanager_certificate_expiration_timestamp_seconds - time() < 604800
  for: 1h

Next Steps

• Resource Manager - Resource management
• Multi-Replica Coordination - Server patterns
• Observability - Monitoring