🕸️Istio
🕸️ What is Istio?
›Why service mesh?
| Without Istio | With Istio |
|---|---|
| Each service implements retries, timeouts | Envoy sidecar handles all traffic policies |
| Plain HTTP between pods (insecure) | Automatic mTLS between all services |
| Manual canary deployment code | Weight-based routing in VirtualService |
| Add tracing SDK to each service | Automatic trace headers propagation |
| No circuit breaking | Outlier detection in DestinationRule |
Install Istio + enable sidecar injection
🔀 Traffic Management
›VirtualService canary + retries + fault injection + circuit breaker
🔒 mTLS & Authorization
›PeerAuthentication mTLS + AuthorizationPolicy + JWT
🏗️ Istio Architecture — Control Plane and Data Plane
›Two planes — one manages, one handles traffic
| Component | Plane | What it does |
|---|---|---|
| istiod | Control Plane | The brain: manages Envoy config, issues mTLS certs, handles service discovery. One istiod manages the whole mesh. |
| Envoy sidecar | Data Plane | A proxy injected into every pod. Intercepts all inbound/outbound traffic. Enforces policies, collects telemetry. |
| Pilot | Control Plane (inside istiod) | Converts Istio config (VirtualService, DestinationRule) into Envoy xDS config and pushes to sidecars. |
| Citadel | Control Plane (inside istiod) | Certificate Authority — issues and rotates mTLS certificates for every service. |
| Galley | Control Plane (inside istiod) | Validates Istio config before applying. |
Sidecar injection — automatic vs manual
# Enable automatic injection for a namespace kubectl label namespace production istio-injection=enabled # After labeling: every new pod gets an Envoy sidecar injected automatically # Verify: kubectl describe pod mypod -n production # You will see two containers: your-app AND istio-proxy # Manual injection (without namespace label) istioctl kube-inject -f deployment.yaml | kubectl apply -f -
🔀 Traffic Management — VirtualService and DestinationRule
›The two core Istio objects for traffic control
VirtualService = routing rules. Where does traffic go? Split percentages, match on headers, retry logic.
DestinationRule = what happens after routing. Connection pools, outlier detection, load balancing, mTLS mode.
# VirtualService — route 90% to v1, 10% to v2 (canary)
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: payment-api
spec:
hosts:
- payment-api # matches the K8s Service name
http:
- match:
- headers:
x-canary-user:
exact: "true" # beta users always get v2
route:
- destination:
host: payment-api
subset: v2
- route: # everyone else: 90/10 split
- destination:
host: payment-api
subset: v1
weight: 90
- destination:
host: payment-api
subset: v2
weight: 10
retries:
attempts: 3
perTryTimeout: 5s
retryOn: 5xx,reset,connect-failure
timeout: 30s
---
# DestinationRule — define subsets and circuit breaker
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
name: payment-api
spec:
host: payment-api
trafficPolicy:
connectionPool:
http:
http2MaxRequests: 1000
http1MaxPendingRequests: 100
outlierDetection: # circuit breaker
consecutive5xxErrors: 5 # eject after 5 consecutive 5xx
interval: 30s
baseEjectionTime: 30s
maxEjectionPercent: 50 # eject max 50% of endpoints
subsets:
- name: v1
labels:
version: v1
- name: v2
labels:
version: v2
Circuit breaker — what outlier detection does
When a pod returns 5 consecutive 5xx errors in 30 seconds, Istio ejects it from the load balancing pool for 30 seconds. Traffic stops going to that pod. It can recover and be re-admitted. This prevents a single failing pod from degrading the whole service.
🔐 mTLS and Observability
›mTLS — automatic mutual TLS between all services
With Istio, every service-to-service call is automatically encrypted and mutually authenticated — no code changes required. Envoy sidecars handle the TLS handshake. Each service has a SPIFFE-format identity certificate issued by Citadel.
# Enforce strict mTLS across entire mesh
apiVersion: security.istio.io/v1beta1
kind: PeerAuthentication
metadata:
name: default
namespace: istio-system # applies mesh-wide
spec:
mtls:
mode: STRICT # reject all non-mTLS traffic
# Allow permissive (both mTLS and plaintext) during migration
# mode: PERMISSIVE
Built-in observability — the three signals
| Signal | What Istio provides | Where to view |
|---|---|---|
| Metrics | Request rate, error rate, latency (P50/P95/P99) per service pair — automatically from every Envoy sidecar | Prometheus + Grafana (Istio dashboards built-in) |
| Traces | Distributed traces across microservices — Envoy propagates trace headers | Jaeger or Zipkin |
| Logs | Access logs for every request — source, destination, status code, duration | ELK or Loki |
Zero instrumentation required. Install Istio, label namespaces — every service automatically gets request metrics, traces, and logs. This is Istio's biggest operational value: instant observability across all microservices without changing any application code.
🎯 Interview Questions
›ISTIO · ARCHITECT
What is a service mesh and when do you actually need Istio?
A service mesh is a dedicated infrastructure layer for managing service-to-service communication. It handles traffic management (retries, timeouts, circuit breaking, canary deployments), security (mTLS between all services, authorization policies), and observability (distributed tracing, metrics per service pair) without requiring changes to application code. The sidecar proxy (Envoy in Istio) is injected into every pod and intercepts all traffic. When do you NEED Istio: more than 5 microservices with cross-cutting concerns (you do not want to implement retries and circuit breaking in every service), need mTLS between all services for compliance (PCI-DSS, SOC2), need canary deployments at the network level without application changes, or need distributed tracing across all services without code changes. When you do NOT need Istio: small number of services, team new to Kubernetes (Istio adds significant complexity), or when you can solve the problem with simpler tools (Kubernetes NetworkPolicy for isolation, application-level retry logic). The honest answer: Istio solves real problems but adds operational complexity. Run it in production only if the benefits clearly outweigh the learning curve. At HPE scale with 15+ telecom microservices: Istio makes sense for mTLS compliance and traffic management.
ISTIO · ENGINEER
What is a service mesh and what problems does Istio solve?
A service mesh is an infrastructure layer that handles service-to-service communication. Without a mesh: every microservice must implement its own retry logic, timeouts, circuit breakers, mutual TLS, and observability instrumentation. With Istio: all of this is handled at the infrastructure level by Envoy sidecar proxies — application code stays simple. Problems Istio solves: Traffic management — canary deployments, A/B testing, traffic mirroring, fault injection for testing resilience, all without code changes. Security — automatic mTLS between all services with zero code changes. Certificates issued and rotated automatically by Citadel. Observability — automatic metrics (request rate, error rate, latency), distributed traces, and access logs for every service pair. Authorization — fine-grained policies: "payment service can call database service on port 5432, but order service cannot." Load balancing — round-robin, least-connection, consistent-hash (sticky sessions) per service. The cost: Envoy sidecar adds ~50MB memory per pod and ~1ms latency per call. For complex microservice architectures this is worth it. For simple 3-service applications, it may be overkill.
ISTIO · ENGINEER
Explain VirtualService and DestinationRule — what each one does.
VirtualService defines routing rules — what happens when traffic matches specific conditions. It answers: where does this traffic go? You can route based on URI, headers, source namespace, or weight percentages. Example: route requests with header x-beta-user: true to subset v2, route 90% to v1 and 10% to v2 for canary, retry on 5xx up to 3 times, enforce a 30-second timeout. VirtualService does NOT create services — it routes to existing Kubernetes Services. DestinationRule defines policies applied to traffic AFTER routing decisions. It answers: how do we connect to the destination? You define subsets (v1 pod = label version:v1, v2 pod = label version:v2), connection pool settings (max concurrent requests, max pending requests), outlier detection (circuit breaker — eject pods returning errors), and TLS mode (whether to use mTLS or plaintext). Together: VirtualService says "send 10% to subset v2". DestinationRule says "subset v2 means pods with label version=v2, and circuit break if they return 5 errors in a row." Neither works without the other when using subsets.
ISTIO · PRODUCTION
Pods are not communicating after Istio is installed. How do you troubleshoot?
Step 1: check if sidecar is injected. kubectl describe pod mypod shows two containers: your app and istio-proxy. If only one container: namespace is not labeled or pod was created before labeling. kubectl label namespace mynamespace istio-injection=enabled then restart pods. Step 2: check mTLS policy. If PeerAuthentication is set to STRICT, any service not in the mesh (no sidecar) cannot send traffic. kubectl get peerauthentication --all-namespaces shows current policies. Temporarily set to PERMISSIVE to test. Step 3: check if request succeeds from inside the pod. kubectl exec -it pod -- curl http://other-service:port/health. If this works: the issue is at the Istio level (policy blocking). If this fails: the underlying K8s networking or service is broken. Step 4: check Envoy proxy status. kubectl exec mypod -c istio-proxy -- pilot-agent request GET /clusters shows all upstream clusters Envoy knows about. Step 5: check proxy sync. istioctl proxy-status shows if all proxies are in sync with istiod. STALE means the proxy has outdated config. Step 6: AuthorizationPolicy. kubectl get authorizationpolicy --all-namespaces — a DENY policy might be blocking traffic. istioctl analyze -n mynamespace checks for config issues and explains them in plain English.
ISTIO · ARCHITECT
How does Istio mTLS work and why is it better than application-level TLS?
Istio mTLS (mutual TLS) is implemented at the sidecar layer — the Envoy proxies, not the applications. When service A calls service B: service A's Envoy initiates a TLS connection to service B's Envoy. Both present certificates and verify each other's identity. The application code in both pods uses plain HTTP to its local Envoy — the Envoy handles TLS transparently. Certificates are SPIFFE-format X.509 certs issued by Istio's Citadel CA: spiffe://cluster.local/ns/production/sa/payment-service-sa. This identity is cryptographically verified on every call. Why better than application-level TLS: zero code changes — you don't modify any application. Automatic rotation — Citadel rotates all certs every 24 hours by default, no manual certificate management. Uniform policy — enforce mTLS for all services in the mesh with one PeerAuthentication resource. Fine-grained AuthorizationPolicy — "only payment-service identity can call database-service on port 5432" enforced at the sidecar level, not in application code. With application-level TLS: each team manages their own certs, rotation, and mutual auth code. One team forgetting to implement it creates a security gap. Istio closes all gaps uniformly.
Continue Learning