Traffic Management — VirtualService, DestinationRule, Circuit Breaking

Traffic Management Layer trong Istio

Istio traffic management hoạt động ở L7 (application layer), không chỉ L4 như NetworkPolicy. Điều này có nghĩa Envoy có thể:

• Route traffic dựa trên HTTP headers, paths, cookies
• Apply retry logic cho cụ thể HTTP status codes
• Circuit break khi upstream service unhealthy
• Inject faults để test resilience
• Mirror traffic sang secondary service

Tất cả được define qua các Istio CRDs, không cần thay đổi application code.

VirtualService: Routing Rules

Khái niệm cơ bản

VirtualService define làm thế nào requests đến một service được routed:

apiVersion: networking.istio.io/v1
kind: VirtualService
metadata:
  name: payment-service-vs
  namespace: production
spec:
  # Hosts mà VS này apply
  hosts:
  - payment-service          # Short name, resolved trong namespace
  - payment.example.com      # External hostname

  # Gateways (cho external traffic qua Gateway resource)
  # Nếu không có gateways field → chỉ apply cho mesh traffic
  gateways:
  - mesh                     # Cho internal mesh traffic
  - production-gateway       # Cho external traffic

  http:
  - match:
    - uri:
        prefix: /api/v2
    route:
    - destination:
        host: payment-service
        subset: v2
      weight: 100

  - match:
    - uri:
        prefix: /api/v1
    route:
    - destination:
        host: payment-service
        subset: v1
      weight: 90
    - destination:
        host: payment-service
        subset: v2
      weight: 10   # 10% traffic sang v2 (canary)

Route matching: các loại match

http:
- match:
  # URI matching
  - uri:
      exact: /api/v1/payments      # Exact match
  - uri:
      prefix: /api/                 # Prefix match
  - uri:
      regex: '/api/v[0-9]+/.*'     # Regex match

  # Header matching
  - headers:
      x-user-type:
        exact: premium
      cookie:
        regex: 'user=.*beta.*'

  # Query parameter matching
  - queryParams:
      version:
        exact: "2"

  # Source labels (chỉ route nếu request từ specific app)
  - sourceLabels:
      app: order-service
    sourceNamespace: production

  # Port matching
  - port: 8080

Canary Deployment với VirtualService

# Stage 1: 100% v1
apiVersion: networking.istio.io/v1
kind: VirtualService
metadata:
  name: my-service-vs
spec:
  hosts:
  - my-service
  http:
  - route:
    - destination:
        host: my-service
        subset: v1
      weight: 100

---
# Stage 2: 5% → v2
    - destination:
        host: my-service
        subset: v1
      weight: 95
    - destination:
        host: my-service
        subset: v2
      weight: 5

---
# Stage 3: 50/50 split
# Stage 4: 100% v2

---
# Header-based routing (dev/QA testing)
http:
- match:
  - headers:
      x-canary:
        exact: "true"
  route:
  - destination:
      host: my-service
      subset: v2

- route:  # Default: không match header → v1
  - destination:
      host: my-service
      subset: v1

Traffic Mirroring (Shadow)

http:
- route:
  - destination:
      host: my-service
      subset: v1
    weight: 100
  mirror:
    host: my-service
    subset: v2
  mirrorPercentage:
    value: 10.0  # Mirror 10% traffic sang v2
                 # Response từ v2 bị discard
                 # Chỉ dùng để test v2 với real traffic

Retry Policy

http:
- route:
  - destination:
      host: payment-service
  retries:
    attempts: 3
    perTryTimeout: 5s
    retryOn: >-
      gateway-error,
      connect-failure,
      retriable-4xx,
      refused-stream,
      reset,
      retriable-status-codes
    retryRemoteLocalities: true  # Retry trên different locality
  
  # Retry conditions:
  # gateway-error: 502, 503, 504
  # connect-failure: upstream connection failure
  # retriable-4xx: 409 Conflict
  # refused-stream: upstream từ chối stream
  # reset: connection reset
  # retriable-status-codes: custom codes

Anti-pattern: Retry quá aggressive

# SAI: Retry POST requests → có thể create duplicate transactions
retries:
  attempts: 5
  retryOn: "5xx"  # Không specify retriable-4xx; POST không idempotent

# ĐÚNG: Chỉ retry cho safe operations
retries:
  attempts: 3
  retryOn: "gateway-error,connect-failure,reset"
  # Không bao gồm retriable-4xx cho non-idempotent endpoints

Timeout Policy

http:
- route:
  - destination:
      host: slow-service
  timeout: 10s  # Total request timeout

# Per-try timeout + overall timeout
- route:
  - destination:
      host: payment-service
  retries:
    attempts: 3
    perTryTimeout: 3s   # Mỗi retry tối đa 3s
  timeout: 10s           # Total: phải < attempts × perTryTimeout
                         # Nếu không: timeout sẽ cut retries ngắn

Fault Injection (Testing Resilience)

# Inject delay để test timeout handling
http:
- match:
  - headers:
      x-test-delay:
        exact: "true"
  fault:
    delay:
      percentage:
        value: 100
      fixedDelay: 5s
  route:
  - destination:
      host: my-service

---
# Inject 10% errors
http:
- fault:
    abort:
      percentage:
        value: 10
      httpStatus: 503
  route:
  - destination:
      host: my-service

DestinationRule: Load Balancing và Circuit Breaking

DestinationRule basics

DestinationRule define policies cho traffic sau khi routing (được define bởi VirtualService):

apiVersion: networking.istio.io/v1
kind: DestinationRule
metadata:
  name: payment-service-dr
  namespace: production
spec:
  host: payment-service
  
  # Traffic policy mặc định cho tất cả subsets
  trafficPolicy:
    connectionPool:
      tcp:
        maxConnections: 100
        connectTimeout: 3s
        tcpKeepalive:
          time: 7200s
          interval: 75s
      http:
        h2UpgradePolicy: UPGRADE   # Prefer HTTP/2
        http1MaxPendingRequests: 1000
        http2MaxRequests: 10000
        maxRequestsPerConnection: 0  # 0 = unlimited
        maxRetries: 3
        useClientProtocol: false
    
    loadBalancer:
      simple: ROUND_ROBIN  # Default

    outlierDetection:  # Circuit breaking
      consecutiveGatewayErrors: 5
      consecutive5xxErrors: 5
      interval: 30s
      baseEjectionTime: 30s
      maxEjectionPercent: 50
  
  # Subsets cho versioned routing
  subsets:
  - name: v1
    labels:
      version: v1
    trafficPolicy:
      loadBalancer:
        simple: LEAST_CONN  # Override cho v1
  - name: v2
    labels:
      version: v2

Load Balancing Algorithms

trafficPolicy:
  loadBalancer:
    # ROUND_ROBIN: Default, distribute đều
    simple: ROUND_ROBIN
    
    # LEAST_CONN: Route đến endpoint với ít connections nhất
    # Tốt cho long-lived connections (gRPC, WebSocket)
    simple: LEAST_CONN
    
    # RANDOM: Random selection
    # Hiệu quả nhất khi tất cả endpoints đều healthy
    simple: RANDOM
    
    # PASSTHROUGH: Không load balance, dùng client-side selection
    simple: PASSTHROUGH

    # Consistent Hash (sticky sessions)
    consistentHash:
      # Hash dựa trên HTTP header
      httpHeaderName: x-user-id
      
      # Hoặc cookie
      httpCookie:
        name: session-id
        ttl: 0s
      
      # Hoặc source IP
      useSourceIp: true
      
      # Minimum number of virtual nodes
      minimumRingSize: 1024

Locality-based Load Balancing

trafficPolicy:
  loadBalancer:
    # Ưu tiên endpoints gần hơn
    localityLbSetting:
      enabled: true
      distribute:
      - from: "us-central1/*"
        to:
          "us-central1/*": 70   # 70% local
          "us-east1/*": 20      # 20% sang region khác
          "us-west1/*": 10      # 10% sang west
      failover:
      - from: us-central1
        to: us-east1            # Failover sang us-east1 nếu local unhealthy

Circuit Breaking: Envoy Outlier Detection

Vấn đề circuit breaking giải quyết

Không có circuit breaking:

Service A → Service B → Database
           ↑
           Database slow → Service B threads blocked
           → Service B queues fill up
           → Service A requests timeout/rejected
           → Cascade failure

Với circuit breaking:

Service A → Circuit Breaker → Service B
                ↓
           Detect 5 consecutive errors
           → Eject Service B endpoint
           → Return 503 immediately (fail fast)
           → After 30s, let Service B try again
           → If success, restore traffic

Outlier Detection Configuration

trafficPolicy:
  outlierDetection:
    # Eject sau N consecutive errors (HTTP 5xx)
    consecutive5xxErrors: 5
    
    # Eject sau N consecutive gateway errors (502, 503, 504)
    consecutiveGatewayErrors: 3
    
    # Eject sau N consecutive local origin errors
    # (connection refused, connection timeout, etc.)
    consecutiveLocalOriginFailures: 5
    
    # Interval để kiểm tra điều kiện eject
    interval: 10s
    
    # Thời gian giữ endpoint trong ejected state
    baseEjectionTime: 30s
    
    # Tối đa % endpoints bị eject cùng lúc
    # Tránh eject toàn bộ cluster!
    maxEjectionPercent: 50
    
    # Min request count trước khi apply outlier detection
    minHealthPercent: 50
    
    # Tính cả local origin errors không chỉ upstream errors
    splitExternalLocalOriginErrors: true

Ejection Timeline:

t=0:   Endpoint bắt đầu có errors
t=30s: 5 consecutive errors → Endpoint bị eject cho 30s (baseEjectionTime × 1)
t=60s: Endpoint được allow 1 probe request
       - Nếu thành công → restored
       - Nếu fail → eject thêm 60s (baseEjectionTime × 2)
t=120s: Probe lại
       - Fail → eject thêm 120s (exponential backoff, max 3600s)

Connection Pool Limiting

trafficPolicy:
  connectionPool:
    tcp:
      maxConnections: 100        # Max TCP connections đến upstream
      connectTimeout: 3s
    http:
      http1MaxPendingRequests: 100   # Max pending requests (queue size)
      http2MaxRequests: 1000         # Max parallel HTTP/2 requests
      maxRequestsPerConnection: 10   # Force close connection sau N requests
      # (helps distribute load khi endpoints được replaced)

Connection pool overflow → 503 overflow:

# Monitor circuit breaker tripped
kubectl exec -n istio-system deployment/prometheus -- \
  curl -s 'localhost:9090/api/v1/query?query=envoy_cluster_upstream_cx_overflow' | \
  jq '.data.result[]'

Gateway: Ingress Traffic Management

Gateway resource

Gateway configure L4-L6 cho ingress/egress traffic. Kết hợp với VirtualService để route traffic vào mesh:

apiVersion: networking.istio.io/v1
kind: Gateway
metadata:
  name: production-gateway
  namespace: production
spec:
  selector:
    istio: ingressgateway   # Targets Istio ingress gateway pods
  servers:
  - port:
      number: 443
      name: https
      protocol: HTTPS
    tls:
      mode: SIMPLE          # TLS termination
      credentialName: tls-cert-secret   # K8s TLS secret
    hosts:
    - "api.example.com"
    - "*.example.com"
  
  - port:
      number: 80
      name: http
      protocol: HTTP
    tls:
      httpsRedirect: true   # Redirect HTTP → HTTPS
    hosts:
    - "api.example.com"

# VirtualService link đến Gateway
apiVersion: networking.istio.io/v1
kind: VirtualService
metadata:
  name: api-vs
spec:
  hosts:
  - "api.example.com"
  gateways:
  - production/production-gateway   # Gateway namespace/name
  - mesh                            # Cũng apply cho mesh traffic
  http:
  - match:
    - uri:
        prefix: /api/payments
    route:
    - destination:
        host: payment-service
        port:
          number: 8080

mTLS ở Gateway (TLS Passthrough)

# Cho downstream mTLS (end-to-end encryption)
servers:
- port:
    number: 443
    protocol: TLS
  tls:
    mode: PASSTHROUGH   # Không terminate TLS, forward nguyên
  hosts:
  - "payment.internal.example.com"

Egress Gateway

# Control egress traffic ra ngoài mesh
apiVersion: networking.istio.io/v1
kind: Gateway
metadata:
  name: egress-gateway
  namespace: istio-system
spec:
  selector:
    istio: egressgateway
  servers:
  - port:
      number: 443
      name: tls
      protocol: TLS
    tls:
      mode: PASSTHROUGH
    hosts:
    - external-api.vendor.com

---
# Route external traffic qua egress gateway
apiVersion: networking.istio.io/v1
kind: VirtualService
metadata:
  name: external-api-egress
spec:
  hosts:
  - external-api.vendor.com
  gateways:
  - mesh
  - istio-system/egress-gateway
  http:
  - match:
    - gateways:
      - mesh
      port: 443
    route:
    - destination:
        host: istio-egressgateway.istio-system.svc.cluster.local
        port:
          number: 443
  - match:
    - gateways:
      - istio-system/egress-gateway
      port: 443
    route:
    - destination:
        host: external-api.vendor.com
        port:
          number: 443

ServiceEntry: External Services

# Đăng ký external service vào mesh
apiVersion: networking.istio.io/v1
kind: ServiceEntry
metadata:
  name: external-database
spec:
  hosts:
  - postgres.cloud.example.com
  ports:
  - number: 5432
    name: tcp-postgres
    protocol: TCP
  location: MESH_EXTERNAL  # External to mesh
  resolution: DNS           # Resolve via DNS

---
# Apply DestinationRule cho external service
apiVersion: networking.istio.io/v1
kind: DestinationRule
metadata:
  name: external-db-dr
spec:
  host: postgres.cloud.example.com
  trafficPolicy:
    connectionPool:
      tcp:
        maxConnections: 50
        connectTimeout: 5s
    tls:
      mode: SIMPLE   # TLS đến external service
      sni: postgres.cloud.example.com

Debug Traffic Management

istioctl proxy-config

# Xem routing config của Envoy
istioctl proxy-config routes my-pod -n production --name 8080

# Output (simplified):
# NAME     DOMAINS              MATCH                     VIRTUAL SERVICE
# 8080     payment-service      /api/v1/* -> v1 (90%)    payment-vs.production
#                               /api/v1/* -> v2 (10%)    payment-vs.production
#                               /api/v2/* -> v2 (100%)   payment-vs.production

# Xem clusters
istioctl proxy-config clusters my-pod -n production | grep payment

# Xem endpoints
istioctl proxy-config endpoints my-pod -n production | grep payment

# Xem listeners
istioctl proxy-config listeners my-pod -n production --port 15001

Envoy Admin Interface

# Access Envoy admin
kubectl port-forward my-pod 15000:15000

# Xem cluster stats (circuit breaker state)
curl localhost:15000/clusters | grep payment-service | grep -E "cx_active|cx_overflow|upstream_rq_pending_overflow"

# Output:
# payment-service.production|8080||v1::cx_active: 5
# payment-service.production|8080||v1::cx_overflow: 0     # 0 = circuit not tripped
# payment-service.production|8080||v2::upstream_rq_pending_overflow: 3  # Queue overflow!

Kiểm tra VirtualService apply đúng chưa

# Check nếu VS có conflicts
istioctl analyze -n production

# Example output:
# Warning [IST0101] (VirtualService payment-vs) 
#   VirtualService "payment-vs" has no matching service "payment-service"

# Verify destination subset tồn tại trong DestinationRule
kubectl get destinationrule payment-service-dr -n production -o yaml | grep -A5 subsets

Kết luận

Traffic management trong Istio là powerful nhưng cần hiểu rõ:

1. VirtualService = define routing rules (WHERE traffic goes)
2. DestinationRule = define policies (HOW traffic is handled)
3. Gateway = external traffic entry/exit point
4. ServiceEntry = register external services vào mesh

Key production patterns:

• Canary: Weighted routing trong VirtualService
• Circuit breaking: Outlier detection trong DestinationRule
• Retry safety: Chỉ retry idempotent operations
• Timeout layering: perTryTimeout < overall timeout / attempts

References

• Istio Traffic Management
• VirtualService reference
• DestinationRule reference
• Circuit Breaking
• Canary Deployments
• Gateway reference